Reflections on ‘the incredible machine – what next for TEF’, a WonkHE conference
By Martin Pearson, Research Manager, Chartered ABS
I recently attended a conference hosted by WonkHE on the subject of the Teaching Excellence Framework entitled ‘the incredible machine – what next for TEF?’. As a research and data person the session that I found most interesting was on the Longitudinal Education Outcomes (LEO) data presented by WonkHE Deputy Editor David Morris.
(LEO) data enables us to know how much UK graduates of different courses at different universities are earning now and one, three, five and ten years since graduating. It does this by linking up tax, benefits, and student loans data. It also draws on a background of material from DLHE the UK Labour Force Survey, and an IFS/Nuffield study entitled Estimating Human Capital of Graduates. When reporting it can be broken down by a range of variables such as gender, ethnicity, region, prior attainment and subdivided by HESA subject codes.
When describing LEO does it can be very helpful to clarify what it does not do. LEO is not a performance indicator. It does not help identify the universities with the best or most effective teaching, nor is it a measure of the ‘value added’ by a university degree. As it is historical data it isn’t even a predictor of how much students at any university or on any course will earn in the future. Leo is certainly not perfect as an information source. For example, it does not control for regional pay differences meaning institutions based in London and South-East England may be seen as having some advantage as their graduates will be more likely to live in high wage areas. However, it is potentially commercial and political dynamite and the implications for the future size, shape and funding of higher education could be huge. For example, it is possible that LEO may feature in a future iteration of the TEF or in the major review of tertiary education promised by the Government.
During his presentation David gave examples of interesting early findings from the experimental data released last year. He showed that it is often not how much any particular university appears to matter for many graduates’ earnings prospects, but how little. This is because prior attainment appears to be a significant factor in earnings attainment and outside most of the specialist sectors such as medicine, veterinary, architecture, and education, high ‘prior attainers’ will always outperform ‘lower attainers’ regardless of the subject studied. One other striking feature from the experimental data was that the gender pay gap appears as soon as students leave university, and proceeds to grow from there. The gap varies between graduates of different subjects, but nonetheless exists in all of them.
Following David’s presentation there was a panel discussion on LEO and related subjects including questions from the audience. This featured Joy Elliott-Bowman (Policy and Public Affairs Manager, Independent Higher Education), Matt Hiely-Rayner (Director of Intelligent Metrix and Head of Planning, Kingston University) and Jackie Njoroge (Director of Strategy, University of Salford).
Concern was expressed from the audience that LEO implies that the value of a job is solely related to how well it is remunerated and the panel largely agreed with this viewpoint. Jackie Njoroge noted that the influence of this type of data will now be difficult to move away from as legislation has been passed (via a little-noticed addition to the Digital Economy Act) to make possible the data sharing that underpins LEO possible. Matt Heily-Rayner was a little more positive, reflecting that some, if not all students will find the information valuable. Joy Elliot-Bowman described LEO as having the potential to be ‘an exercise in benchmarks gone mad’, with students likely to see unrealistic or misleading data on possible future salaries. Institutions therefore need to be proactive in explaining and caveating LEO to them. This was a thought provoking end to a thought provoking session.
