In the realm of college admissions, today is a day to rejoice—or rant. It all depends on your opinion of college rankings (or, perhaps, your college’s place on U.S. News & World Report’s annual list).
Today the magazine unveiled the 2011 edition of Best Colleges. As you may have heard, some university in Massachusetts topped the list of national universities, and a small college in the same state took the top spot on the list of liberal-arts colleges.
Although some things never change, the ranking methodology does. This year, for the first time, U.S. News included the views of high-school counselors in its measure of “academic reputation,” perhaps the most controversial aspect of the rankings. Previously, the magazine used only an annual “peer assessment” survey of college presidents, provosts, and admissions deans to calculate this measure, which accounted for 25 percent of each college’s overall ranking.
This year, U.S. News lowered the weight for academic reputation to 22.5 percent (for national universities and liberal-arts colleges only). Ratings by nearly 1,800 high-school counselors surveyed accounted for a third of that measure, and ratings by college administrators accounted for two-thirds. In other words, the opinions of college officials carry less weight than they did last year.
Complaints about the peer-assessment survey were among the reasons U.S. News brought counselors into the fold, says Robert J. Morse, the magazine’s director of data research. Over time, participation in the annual survey has declined steadily (this year, 48 percent of college officials who received questionnaires responded, the same as last year). For years, Mr. Morse has said that U.S. News would invite other experts to participate in the rankings, if necessary.
“We went out and searched for people who had a stake in admissions, who had a certain expertise,” Mr. Morse says. “High-school counselors play a big part in college admissions, so we counted their votes.”
The significance of this change may be more symbolic than substantial. Sure, the power of the peer-assessment survey, long loathed by some college officials and high-school counselors, has been diluted. Nevertheless, reputation—that slippery and subjective thing—still matters a lot in the U.S. News formula. The mix of reputational experts has just become more diverse.
“The concerns people have about rankings will not be assuaged by giving high-school counselors a voice in them,” says James W. Jump, director of guidance at St. Christopher’s School, in Richmond, Va., and the departing president of the National Association for College Admission Counseling. “The idea that No. 9 is better than No. 20 concerns me. Ranking simplifies what should be a complex process.”
In other changes this year, U.S. News raised the weight of the “predicted graduation rate” to 7.5 percent, from 5 percent, of a college’s overall ranking. The magazine also expanded the number of institutions ranked in each category, and it changed the names of two categories (“Universities-Master’s” and “Baccalaureate Colleges”) that had puzzled readers.
U.S. News considered at least one change that it did not make. In June, Mr. Morse wrote on his blog that he and his colleagues might add “yield” (the percentage of admitted applicants who enroll) back into the rankings. In 2003, the magazine’s editors removed the measure from its formula amid criticisms that the rankings had driven colleges to become obsessed with yield. Never mind that colleges have long had plenty of other reasons to fret about yield, or that yield accounted for only 1.5 percent of a college’s ranking by U.S. News.
“In the end,” Mr. Morse says, “we didn’t want the discussion of yield to take away from the other changes we were making.”
13 Responses to The Rankings, Rejiggered
davi2665 - August 17, 2010 at 11:34 am
USNWR is becoming more and more like a vanity press. They alter their criteria extensively from year to year, without telling anyone how or why. Their rankings of hospitals are a total joke- the number #1 hospital for pulmonary medicine is a hospital that lists less than 10 patient discharges per year, as opposed to many thousands of discharges for the major academic medical centers ranked lower. A hospital that was ranked in the upper teens last year is now ranked below 100, without anything changing from year to year. And worst of all, USNWR is now charging substantial fees for the right to use their name in association with promoting or publically referring to the ranking of their hospital or specialties. It’s like some of the Who’s Who volumes that invite your participation if you buy a copy for $1000 plus. What started as a reasonable idea- ranking of institutions based on excellence- has turned into a game. And the sad fact is that university administrators and hospital administrators stand on their heads and manuever every which way they can to “improve” their rankings, even to the point of including an improvement in their USNWR ranking as part of their academic strategic plan. The entire trend with USNWR appears to be heading for “pay to play.” How pathetic.
greeneyeshade - August 17, 2010 at 12:05 pm
This re-jiggering shows how utterly arbitrary these ratings are.And then there’s the phenomenon that’s similar to teachers who teach to the test: universities make adjustments that they know will enhance their scores.All is vanity, sayeth Qoheleth.
marka - August 17, 2010 at 2:21 pm
Hmm … Actually, USNWR publishes criteria, and notes changes, as this article demonstrates. They do a much better job that many other ‘rankings,’ such as those collected & published by students in student guides. One can always criticize any kind of measurement – it is altogether another thing to create a better one.So, here is the challenge. If you think USNWR isn’t doing a good enough job, come up with your own measurement, and go out & use it.This reminds me of all the criticisms of other measurements: SAT, ACT, IQ tests, grades, etc. Well, admittedly, all these are imperfect. So, come up with something better. Let’s not be hypocritical — ALL colleges use some kind of ‘measurement’ when it comes to admissions, whether it is SAT, ACT, grades, high school graduation rank, or ‘recommendations’ from teachers & others.
tbone56 - August 17, 2010 at 5:22 pm
Regrettably, hardly any of the measurements typically used to evaluate colleges and universities deal with what most people would like to know: how well do they teach, and how much learning takes place in them? USNWR’s “Best Colleges,” when you disaggregate their results, is essentially a measure of the SAT/ACT scores of entering students, a measure that correlates very strongly with socio-economic status (far more strongly than it correlates with college success). I strongly suspect (but have only anecdotal evidence) that if we actually used the measure of how much learning takes place, i.e., how much intellectual improvement takes place in students, the leaders would be regional comprehensive “second-tier” universities–places like Appalachian State or Western Washington or Illinois State. The Harvards and Wellesleys don’t have much work to do; they just gild the lily.
11262324 - August 17, 2010 at 5:46 pm
After all is said and done, how many prospective students/parents believe the USNWR is the bible? I would guess that if you surveyed students in 85% of the institutions in the country, they actually used something much more substantive than this report to make a decision on where to apply. In reality, USNWR is much to do about nothing…..
tee_bee - August 17, 2010 at 8:43 pm
High school guidance counselors? Really? I guess things have changed. When I graduated high school, many moons ago, our counselors couldn’t have found any college with two hands and a map. The very idea of them rating colleges is risible, unless you’re talking about guidance counselors at top-performing high schools, in which case this just introduces a different kind of error to the process.
jesor - August 18, 2010 at 5:02 pm
I still have a problem when a ranking system purports to determine which college is best, and a general opinion survey counts for three times as much as whether or not a student is likely to graduate. The only reason for doing that is to take advantage of potential ad revenue from more prestigious (i.e. more wealthy) institutions.
aparentsviewpoint - August 19, 2010 at 1:44 pm
Sure USNWR rankings are fun to look at, but as a parent they don’t mean didily-squat. My college bound son only wants to know which schools are within a five hour radious of our house. I live in the Washington D.C. metropolitan area so there are quite a few. Because we are middle class we won’t qualify for any aid–not one penny. When it comes time to choose, it won’t be which one’s best, it’s how much it costs and whether or not we can afford it. I question the sticker price at some of these institutions. [value > < or = $$] It reminds me of buying a car. As far as high school counselors weighing in–I'd re-consider the source.
rickinchina09 - August 20, 2010 at 2:06 am
marka-Evidently, you haven’t heard of The Gourman Report, which has been around longer than the USNWR an isn’t trying to hawk for customers.UNNWR editors are now filled with self-importance. Perhaps it comes from having to work for an otherwise failing news publication.
goxewu - August 20, 2010 at 11:44 am
Looked up The Gourman Report. Doesn’t seem all that good. One-horse operation, and Dr. Gourman seems to be one of those big-time-at-mid-level educrats with lots of degrees (he cites “five earned degrees”). He doesn’t disclose his criteria and some people say he’s got a jones for big state schools. Maybe he racks up a lot of frequent flyer miles and subscribes to ESPNU. Would appreciate somebody elaborating on The Gourman Report’s virtues other than it “isn’t trying to hawk for customers.”
nuclear_engineer - August 23, 2010 at 12:51 pm
The weight of the purely subjective “academic reputation” metric should be at most 10%. Even lower (e.g., 5%) would be even better.If there is one metric that I would like to see incorporated in scoring, it would a “How much did they actually learn” metric. But that will never happen. Too objective. Too expensive. No one in the “academic industrial complex” really wants to go there.
jca360 - August 23, 2010 at 6:21 pm
High school counselors are quite vulnerable to the blandishments of college admissions offices. My child went to an elite private school that devoted lots of resources to college admissions, yet they pushed a decidedly second or third tier university over several top-15 colleges and universities, just because of the snow job. So this change will introduce even more randomness and, worse, manipulability into the rankings.
raincross - August 23, 2010 at 8:27 pm
Re: #5, I actually think that most high school seniors use much *less* “substantive” (if same refers to “quality”) criteria (viz #8) like proximity, cost, where their friends are going, how they felt when they strolled the campus.In truth, the measure that most students say they care about isn’t anywhere to be found here. I don’t hear many young folks talking about the quality of their learning, but I hear lots of focus on how employable they’ll be upon graduation. If you want to rely on reputational perspectives, ask the heads of HR from the Fortune 500. Proportionally speaking, very few of our 17 million college students will end up working in the academy. The vast majority want good jobs that will allow them to use the abilities they honed in college. That may sound distastefully pragmatic, but especially in this economy, it’s just the painful truth.And while I’m sure there are very competent, dedicated school counselors out there, I’m not sure what makes them qualified judges of “quality” any more than administrators at peer institutions. My high school counselor was a sweet soul, but hardly an expert on the subtleties of pedagogical nuance.