David Glenn's June 13 article on the "long awaited" NRC evaluation of doctoral programs in the United States reminded me of how easily I had forgotten that the long-delayed report had not yet appeared.

Years ago, when I was the president of the American Council of Learned Societies, it was my responsibility to appoint representatives to the committee that undertook the evaluation on behalf of its several sponsoring institutions. I had been vaguely aware of the existence of this allegedly decennial assessment exercise as a faculty member, but I had always taught in institutions that were sufficiently high on the academic food chain that the ranking of my own department was of no particular interest to me.

So when I had to focus on the NRC assessment process in the early 1990s, I was surprised to learn that the survey was largely reputational—departmental chairs (and perhaps deans) were asked to rank other doctoral departments according to their personal impressions of academic quality. I was also disappointed to find that the survey managers at NRC were reluctant to introduce changes into the process. For instance, when I complained that many of the newer fields in the humanities (my special charge at ACLS, after all) were not included in the survey, I was told that such fields (Women's Studies, for instance) could not be added since there would be no longitudinal information to link them to. Natch. For these and other reasons, it seemed to me that the survey was not very useful in the fields I most cared about, and in any case it certainly took more than a decade to produce the "decennial" survey that my organization was partially responsible for. I confess that I pretty much lost interest in the NRC report by the time that it appeared in 1995 (the previous report had appeared, I think, in 1982), and I certainly have not been holding my breath to study the next report, which was due several years ago. It is apparently still not clear when the latest report, hardlly "decennial" any more, will see the light of day.

Which is not to say that I am not grateful that the NRC has been trying to improve the quality and thus utility of its rankings of doctoral programs. A year or so ago The Chronicle ran a short piece commenting on the ongoing attempt to move beyond reputational surveys, and to develop "objective" markers of quality. I wrote to one of my Princeton colleagues, the astrophysicist Jerry Ostriker, whom I knew was a member of the NRC committee, and asked what objective indicators were being used in the humanities, since it seemed to me that these were a good deal harder to specify in the humanities than in the hard sciences. Jerry took me to lunch and convinced me that a good-faith effort was being made to evaluate such markers as book prizes, and that such data might help to compare the quality of humanities-department faculty. I expressed skepticism, but told Jerry that I would be eager to review the results this year when they were published.

But I gather from Glenn's article that the attempt to improve the quality of evaluation data has led to problems in cleaning the data (to be sure that all institutions are using the mutually agreed upon definitions of what the data represent), and the data problems apparently persist. The result is that the report has been further delayed, and meanwhile the data on which the evaluations are based is rapidly aging. Some, or perhaps many, of the faculty upon whom the departments' reputations were based are no longer members of those departments. Jerry Ostriker is quoted in the article as urging us not to worry, since "the new report's data will be no more stale than the data used in the NRC's previous reports." This seems like a weak defense of the current process and product, but we can better judge the report when and if it ever appears. We are also told that the report will offer the data in two ways: "One will be based on 20 objective elements, such as faculty publications and students' average time-to-degree. The other will be based on extrapolations from survey respondents' subjective assessments of particular programs." Thus some of the new and some of the old. We shall see.

Many Chronicle readers will understand how important the NRC rankings have been to university provosts and deans in conducting their never-ending Race to the Top. How many poor department chairs have been warned that their success will be measured by whether their department can pull itself up into the Top 10 (or the Top 20)? The NRC always warns us of the limitations of its survey, and lack of timeliness is only the most obvious of those limitations. Up to now we have not been able to rely on much more than reputation, a fragile indicator at best, to establish the rankings. We'll see how "objective" the new rankings can be.

But it seems to me that serious educators ought to have the confidence to develop their own internal objective criteria of success, and ought to stop looking desperately ahead to see where there are open spaces in the back of the pack. Just possibly even the NRC can't do much better than U.S. News and World Report, and we ought to stop worrying about the Race to the Top.