Issue #14: Winter/Spring 2006

Why Maclean’s Rankings Rankle

By William Bruneau

Every autumn, Maclean’s magazine publishes a “Universities Issue” and a separate, more costly Guide to Canadian Universities, with various rankings of universities across the country.

The editors gather information in various ways, but mostly through the institutions themselves. Maclean’s says the Guide outsells the usual Maclean’s run by a factor of ten. It’s a money-maker for Rogers Corp., the latest owners of the magazine.

When the rankings first appeared in 1992, they caused joy in some quarters (the University of Toronto), and misery in others ( Trent University). But in that year, premiers Ralph Klein ( Alberta) and Bob Rae ( Ontario) announced their intentions to connect university funding to “market measures,” (Klein) and “performance indicators” (Rae). The worry – or the source of joy, depending where you were in the rankings – was that systems like Maclean’s would be used to decide at least some university funding. In the end, Maclean’s rankings were not used this way, but provincial performance indicators were. Any joy was replaced almost at once with hand-wringing.

Fourteen years later, the shine has begun to wear off the rankings. They attracted less attention in the media this year than at any time since their creation. In all the provinces the use of performance indicators continues but has not replaced reasoned and less arbitrary ways of deciding how to support universities. But even in those universities and colleges that refuse to participate in them, the rankings attract worried attention. At the University of Saskatchewan, President Ivany remarked in 2003 that he dared not refuse Maclean’s informational inquiries, as the political price would be too high.

The fact remains: rankings have become a part of the vocabulary of students, pundits, governments, and profs in Canada. They demand and deserve close and sceptical study.

To give the question even more point, over the past twenty years new rankings have joined the old: the Globe and Mail has one, and the granting councils (Sciences, Canada Council, Social Sciences and Humanities, Medical Research) have theirs. Then there’s the Shanghai Jiao Tong University ranking of the top 500 universities in the world (UBC is 37, whatever that means). And we have rankings by citation counts (number of times other researchers quote the work of professors at any one university) in every field.

It all matters, as a high rank means that a university has to be taken seriously. And we don’t want to spend too much time thinking what a low ranking would mean.

Maclean’s says it judges the “comparative strengths” of public universities in Canada by counting things. The key questions are: Where are the brightest first-year students? Where are the smallest classes? Where are the most tenured faculty? Where are the richest library resources? Which university has made the largest commitment to student services, scholarships and bursaries? And which university has the best reputation for quality and innovation?

Most of these questions are answered with simple arithmetic. A library with three million copies of Whittaker’s Almanac would win over UBC’s with two million carefully selected volumes. Incoming student averages are calculated without considering the historic, social, cultural, or any other resources that make students interesting and strong.

Faced with the rankings’ simplistic arithmetic, faced with their popularity, and faced with the dangers they pose, Canadian universities have adopted two strategies. Universities either ignore them, or pick out rankings that suit them. In 2003-4, UBC trumpeted its first place in the reputation ranking, even though it was only fourth in general ranking.

Five years ago, UBC’s rankings fell. It worried the administration. Steps were taken. UBC reviewed the way it reported student-teacher ratios, and the way it counted how many students were being taught by tenured professors rather than by part-timers or sessionals.

In 2003, 2004, and 2005, UBC did move—from fifth to fourth place. President Piper still plans to push UBC past Toronto, McGill and UWO. But that is no easy matter: UBC students will have to out-do the rest of Canada in entering averages. And UBC will have to find a way to get more of its students out and graduated “on time,” reduce class sizes, increase the number of students taught by tenured professors and do something about that pesky library. Somehow or other, UBC would have to persuade alumni, and academic and business leaders that UBC deserves high levels of funding and support, just because the rankings say so.

University rankings are, then, a kind of craziness. After all, Canada’s great public universities have legislated mandates. They are supposed to prepare people of all ages and all backgrounds to function well and happily as citizens, to work as professionals in a bewildering range of fields, to act as creators and thinkers, shaping generations to come. This is a matter for very long-term thinking. The curriculum and the research-and-development done in these places are best put in the hands of practised researcher-teachers, working together with the public. The work of higher education depends on effective and open university senates and boards, broad accessibility to all who could benefit.

Alas, a university that passively accepts rankings and indicators will soon forget this mandate and accept a long series of shotgun marriages with passing economic fantasy. Remember the idea that every university student should consider a career in the dot-com industries? In this case, the rankings-driven university isn’t much different from the automobile manufacturer who really does have to worry about markets and public fancies.

There is another, highly practical aspect to the craziness: U of T, McGill, UBC, Laurentian and many more have found a dreadful paradox in rankings. Government may note an improved ranking or indicator. But that may justify less public funding, not more. The reason is, if you’re good, why not give your poorer, struggling neighbour university the cash it needs, so it can be good too? At the same time, declines in the rankings invite punishing cuts.

Universities cannot win the rankings game.

So why are rankings still with us?

1. Maclean’s is Not Alone

From the earliest years of the Thatcherite and Reaganite neo-conservative experiments, new ways to rank and categorize sprang up annually across Europe and North America. The intention was to justify bureaucratic control – cost-benefit analyses of courses in English and Art, studies of the cost per square metre per student of BSc degrees in Chemistry, Degree Quality Assessment Boards, and so on – and to provide new techniques of financial management, so sustained cuts in public expenditure could be more easily justified.

Since the late 1970s in the countries of the oecd (Organisation for Economic Cooperation and Development), rankings and statistics were all the rage. The British brought in a highly bureaucratic and expensive system of reports and inspection. Neo-conservatives said they wanted less government, but acted to ensure there would be much, much more government involvement in higher education.

In the United States, state and federal governments acquired a new fascination for “performance indicators,” which were used not to improve quality or accessibility, but to justify immediate cuts and heightened bureaucratic control in higher education. There, as in Canada, the new system had a partner in the annual rankings produced after 1983 by US News and World Reports, the parents of Maclean’s.

At UBC, as elsewhere in Canada, a further result of rankings has been an increase in the bureaucracy of “reporting.” Universities must make detailed statistical reports on nearly everything they do. From the innocent days in the late ’ 60s when UBC professor Robert Clark helped create offices for straightforward budget and program planning, UBC has been frog-marched into a new world. It is a world of endless detail and management and close supervision and control. We should have known.

Another reason for the rankings’ persistence is that they are part of a movement. The movement favours efficiency (but produces costly bureaucracy and hasty decisions), and it claims to be “accountable” (but not to the public, only to the market).

2. Universities in the Passive Mood

In plain language, the frenzied rush to be #1 and stay that way makes universities less autonomous. Yes, universities may be more immediately responsive to the pressures of government and industry. But to whose tunes are they truly dancing?

Canadian experience suggests the profound unwisdom of a university system whose task is to respond, to do as it is told, and to consider its identity is revealed in Maclean’s. But remember Mr. Rae and Mr. Klein. Their views have weight and momentum. No wonder the rankings are still around.

3: A Horse Race

Everyone knows universities compete with one another for reputation, students and cash. In that sense, Maclean’s has been a good thing, making the annual university horse race a bit more amusing than it would otherwise be. The horse race analogy points to a human fascination: people like to look at races, to be excited by them, and even to bet on them.

But this kind of competition often brings out the worst in those competing. Where does this notion fit in the definition of “university”? Is there a place for driving ambition, manipulation and the fastest possible adaptation to the ever-changing economic environment? It is impossible to deny the existence in universities of people who do those things: drive, manipulate, adapt to economic fantasy. But they are not the university.

What, then, is the university?

For the purposes of a piece about rankings, the university is, or should be, a community. It relies on reason and debate to do its central work of teaching and research. Its governance is open and considered and sometimes slow. Access to the university’s riches is broad and fair. The university does its work equitably.

It is a collection of professors whose effectiveness would end in an instant if their academic freedom and tenure were seriously threatened.

It is a body of students whose energy, curiosity and social power will make courses worth taking and teaching, and whose future is to reconstruct culture.

It is a body of talented staff members whose commitment is to the whole society, not just to the next ranking or the next pay cheque.

It is a collection of administrators who help keep the place safe from mindless market intervention, to co-ordinate the work of all the others in the community, and to keep the purpose of the university constantly before the public.

The great goal of these superposed communities is the remaking of minds, the refashioning of skill, and increased moral consciousness of everyone in community.

A university worth the name depends on sustained, society-wide support. It has significant public funding. It can rely on long-term public and private commitment to the university’s overriding purpose. For that reason, the academic community must constantly talk to its supporting communities.

If it could be shown that ranking contributed directly to all, some or even one of these things, I would back off the criticisms I have made.

I accept that some university practices invite statistical summary and description. But it is hard to see how or why universities benefit from it, or how ranking could help or “inform” Canadian society and Canadian politicians.

Ann Dowsett Johnson of Maclean’s says rankings help to “inform” students and their parents. She has a point, but one suspects Ms Johnson means the rankings inform us about the magazine, just as much as they do about universities. Come to think of it, Maclean’s comes first. ¤

William (Bill) Bruneau taught in the department of Educational Studies from 1971 to 2003. He writes mostly on the history and politics of Canadian and European universities. His most recent book (co-authored with David Gordon Duke) is a biography of BC composer Jean Coulthard, but he has also co-authored (with Donald Savage) a surprisingly long book on performance indicators and rankings, Counting Out the Scholars (2002).