Wednesday, February 16, 2000

11 Major Universities Opt Out of Yahoo's Most-Wired Campus Competition By KELLY McCOLLUM At least 11 leading universities have declined to participate this year in Yahoo! Internet Life magazine's popular survey of campus technology, citing flaws in the survey and fundamental objections to its ranking system. The annual survey, which provides a ranking of "America's 100 most wired colleges," has been met every year by complaints about its methodology and the apparent subjectiveness of its rankings. The magazine has maintained that the feature provides an overview of campuses' technology for prospective students and their parents and has tweaked the survey each year to improve its usefulness. According to Betty Leydon, vice provost for information technology at Duke University, the survey tries to empirically compare criteria that are valued very differently from institution to institution. "It's really not clear what they're trying to measure or how they decide to rank the schools once they get the information," she says. "We really felt that a better thing to do would be to demonstrate that in some way by not responding to the survey." After discussing this year's survey, Ms. Leydon and several of her colleagues at other institutions decided not to participate. She says the institutional solidarity "does make it easier, but I think that everybody is making their own decision and they have their own process on their campuses for talking about it." Robert Bernstein, the magazine's senior editor who oversees the survey, says the non-participating universities won't make much difference. "We're not that concerned about it, but we realize we need to address it." He says he plans to send a letter to those institutions to respond to their concerns. But, he says, most of those not participating probably wouldn't have ranked in the top 20 anyway. "I think our list again this year will accurately reflect the most-wired schools in the country," he says. Because the magazine relies on the detailed information institutions provide to determine its rankings, universities not participating will not be ranked. Mr. Bernstein says the magazine will most likely note those non-participants that would probably have ranked high in the survey. Of the boycott, he adds, "we think it's unfortunate, we think it's a disservice to our readers and to people trying to find out how schools rank." Ira Fuchs, Princeton University's vice president for computing and information technology, says he has been dissatisfied with the survey in years past and did not respond this year. Now, he says, "they're asking more questions, but they still don't do a very good job of reflecting whether an institution's goals are being well served by its information technology." For example, Mr. Fuchs says, questions about a university's online course offerings don't necessarily indicate anything about a given university. "Princeton is not necessarily interested in having its students be able to take all their courses without ever going to class. That's not what Princeton is about," he says. Institutions that do emphasize distance education would, therefore, have an advantage in the rankings over those that don't. Administrators seem to be most concerned about the survey's ranking system. "Certainly parents and students should be able to get to information about information technology on the campuses they're considering," says Mr. Fuchs. "But the idea that they can put a value on each of the aspects and compare them for vastly different institutions is where the thing doesn't work." Ms. Leydon of Duke says, "If a student is going to college and has some idea of what they want to study and what kind of environment they want, then having information on institutions will help a student make a decision, but I don't think it's appropriate really to rank institutions in that way. It doesn't really make any sense." Some of those worries center around confusion about how the magazine determines its list. Gregory A. Jackson, the chief information officer for the University of Chicago, says "they are absolutely steadfast in refusing to tell how they do their ratings. Many of us believe that is just dead wrong." He says that because the magazine won't reveal its formula, "there continues to be the suspicion that things get ranked not entirely because of information that's on the survey." Mr. Bernstein says the magazine is only protecting its journalistic standards by keeping its criteria secret. "We feel like we want to protect our formula, we don't want to quibble with each of 3,000 schools in the country as to how something should be weighed," he says. With each institution placing different emphasis on each aspect of technology, no standard ranking scheme is going to please everyone, he adds. "We're trying to create a single standard and then measure schools against that," he says. "When people look at this ranking, they're trusting the brand name Yahoo! Internet Life," he says. "It's our ranking, it's our criteria, and that's just the way it is." And ranking, says Mr. Bernstein, is a vital part of the survey. "A lot of the value of this feature is for students to be able to look at a list of schools and determine which schools are offering the kinds of services they want and which schools aren't offering those kinds of services, and a lot of that really comes out of a ranking." Mr. Bernstein adds that not all information gathered by the survey is weighed heavily in determining the rankings. Criteria like distance-education offerings may not be easy to measure consistently from one campus to another. In such cases, he says, the magazine would give those criteria less weight. Mr. Bernstein says some of the recurring complaints about the survey may simply be sour grapes. "When a school does well, they applaud the methodology. When a school does poorly, it's no longer a reason to cheer," he says. Of the institutions that have informed him that they will not respond this year, none were in last year's top 10. According to Mr. Fuchs and Mr. Jackson, the list of institutions not participating this year includes Brown, Cornell, Duke, Harvard, Princeton, Stanford, and Yale Universities; and the Universities of Chicago, California at Berkeley, Michigan, and Washington. Universities may also be worried about increased competition this year, says Mr. Bernstein. The magazine, which works with Peterson's, a publisher of guidebooks about colleges, to conduct the survey, received responses from about 600 institutions last year. This year, responding to complaints from institutions that were left out of previous surveys, the magazine put the survey on the Web and invited more than 3,000 colleges and universities to participate. Some 1,200 have responded. "Unfortunately, it seems, they would rather lose by omission than lose facing the competition," he says of the universities that have decided to opt out. Chicago's Mr. Jackson acknowledges that the increased participation makes it mathematically likely that a given institution's ranking will slip, and that was one factor he considered. "Either I'm going to have a conversation about why we didn't participate or I'm going to have a conversation about why we dropped back to number 63," he says. "And frankly I think the better conversation to have on this campus is why we didn't participate rather than splitting hairs about how we did this and they did that." At Duke, too, not participating in the survey may offer a chance to focus on what the university rates as important. "Being ranked by Yahoo isn't quite as important as talking about the things that matter to you in terms of technology," says Ms. Leydon. "And by not being there, maybe that'll give me the opportunity to talk about them." Background stories from The Chronicle: Universities Question Fairness of Magazine's 'Most Wired' Survey (5/21/1999 ) Case Western Reserve U. Tops 1999 List of 'Most Wired' Colleges and Universities (4/23/1999 ) Yahoo!'s Magazine Selects Dartmouth as the 'Most Wired' College (4/24/1998 ) Easy-to-print version E-mail this article

Lawmakers tell N.I.H. to justify the president's call for a budget increase Iowa State U. weighs sale of its nonprofit press U. of Pennsylvania sit-in ends as president agrees to protesters' demand Canadian agency that is accused of mismanaging funds faults accounting by 12 colleges House votes $6.9-billion for information-technology research 11 major universities opt out of Yahoo's most-wired campus competition British government announces plans for distance-education venture