What league tables leave out
Every autumn, a handful of organisations publish lists that claim to rank the world's universities, and every autumn the results are greeted with a mixture of celebration and dismay. University leaders issue press releases when their institution climbs a few places; politicians cite the number of 'world-class' universities in their country as evidence of national success. For prospective students, especially those planning to study abroad, the tables offer something that is otherwise hard to find: a simple way of comparing institutions they may never visit before they apply. Some applicants, and many of their parents, treat a place in the top hundred as a guarantee of quality.
It would be unfair to dismiss the rankings entirely. Before they existed, information about universities was scattered, inconsistent and often controlled by the institutions themselves, and the tables have undoubtedly made some of it more public. Some universities have also responded to a poor position by investing in things that genuinely matter to students, such as libraries and support services. Defenders of the rankings go further, arguing that competition between universities raises standards across the whole sector. That may be so, but it is a claim that is very hard to test, since standards are precisely what the rankings measure so imperfectly.
The central problem is that the most influential tables reward what is easy to count. Research output, citations and the proportion of international staff can all be measured with reasonable precision; the quality of teaching cannot, at least not cheaply. As a result, the indicators that carry the most weight tend to favour large, wealthy, research-intensive institutions. A small college that teaches brilliantly but publishes little may not appear at all. Nor do the tables say much about what happens to students once they arrive: how often they see their tutors, how much feedback they receive, or whether they complete their degree. Several tables also rely partly on reputation surveys, in which academics are asked to name the best institutions in their field. It is hardly surprising that they tend to name the ones that already top the tables.
This would matter less if the rankings were treated as one rough guide among many. In practice, they shape behaviour. A fall of a few places can make headlines, even when it reflects nothing more than a change in the way the scores are calculated. Some universities have been accused of hiring highly cited researchers mainly to improve their scores, or of moving money from teaching into the activities that the tables reward. Some governments even use ranking positions when deciding which foreign graduates to welcome or which students to fund abroad. An often-quoted principle holds that when a measure becomes a target, it ceases to be a good measure. University rankings offer an unusually clear illustration of it.
None of this means that comparison is impossible, or that applicants should ignore the tables altogether. It means that they should read them as they would read any league table: as a record of who scores well on a particular set of measures, not as a verdict on which institution is best. A student who wants small classes, close contact with tutors or good employment prospects in a particular field will usually learn more from subject-level information and from current students than from a university's overall position. The tables answer a question; the difficulty is that it is rarely the question applicants most need answered.