From the issue dated March 1, 2002
|
|
The Random Insanity of Letters of RecommendationBy DEIRDRE McCLOSKEYEvery autumn brings The Letters. Thousands of deans, mostly sane, call for The Letters to be written, receive them in bulk, pass them from hand to hand, interpret them, and pronounce them Good or Bad. "She's got Good Letters," says a member of the dean's promotion and tenure committee, meaning: The department chair has begged N professors at other colleges to make N inscriptions claiming the candidate is superbly, wonderfully, perfectly, incredibly qualified for her first job or tenure or a full professorship. "But have you seen the Bad one from Smith?" "Oh, that Smith's just a curmudgeon: Look at the other Letters." I say the practice is insane. Dr. Johnson remarked that "in lapidary inscriptions a man is not upon oath." Nor are the inscribers of unpaid letters of recommendation -- not in a world in which the chair solicits 10 or 20 for each promotion, instead of one serious letter written with the understanding that it is to have real weight. Scientifically speaking, the letters are meaningless and should be disregarded. When the letters are passed around at meetings of my departments I resist a natural impulse to participate in this meaningless gossip. So should you. Colleges should make their own decisions about tenure and promotion, doing the homework required and using gingerly the outside world's information -- actual information, not "information" valid only in a world of saints. The only correct procedure for assessing scholarship in hiring or promotion is for the responsible body to read the candidate's work and discuss its intellectual quality with immediate colleagues in a context of believably disinterested assessments from the outside. Examples of such assessments would include referee reports for publishers, book reviews in journals, or offers to move to another college. They could of course include ad hoc recommendations by an outsider paid a considerable sum to set aside her interests and passions and alternative paid work to provide a genuinely disinterested letter. A serious letter deserves serious payment. Your local economist can explain why pricing scarce resources correctly will result in their rational use. A letter that cost $1,000 would not be solicited casually, would not be written without research, would not be handled without care. The Nobel committees pay these amounts for reports on candidates. Remember that the discounted present value (on this, check again with your local economist) of a permanent appointment is well over the present cash value of a Nobel. If you are serious about standards, you will pay. The soliciting and writing and reading of unpaid letters of recommendation is a scandalous failure of common sense. It is corrupt, dishonest, unscientific. The deans who depend on it should be ashamed of themselves. Instead of doing their job, which is to make sure that the departments are doing their job (see the italicized sentence above), they are handing the job over to a phony criterion that would make any sensible child (Harry Potter, for example) laugh out loud. You call that magic? "Oh, that McCloskey -- she's just a curmudgeon. Look at how many colleges do it." Ah, well, that's certainly a powerful argument for academics to deploy. Yes, of course: If many people do it, more every year, then of course it must be correct, like the eating of saturated fat and the buying of sports-utility vehicles. You bet. A dozen years ago I tried to draw attention to the scandal in a little piece in Change. There was no reaction from the thousands of deans who read the magazine. I really do accept that it's always the writer's fault when her point does not carry the day, so I was not indignant that my articulation of the truth wasn't persuasive. I then tried to get at least the dean at my (then) college of liberal arts to respond to the fully 18 crushing arguments I had made against the practice. No dice. He wouldn't reply. Stonewall. He was determined to go on imposing the practice, of course, his sole argument being (I am making an inference from his pattern of silence) that outside letters raise standards. This would be funny if it were not so damaging to standards. No one is willing to make arguments in favor of what has become so obviously irrational a practice, because there aren't any arguments. Yet on and on and on it goes, getting worse every autumn, imposed by provosts on deans and by deans on chairs and by chairs on faculty members. It gives everyone the impression of doing something. Therefore no one is willing to impose real standards -- making the future colleagues of the candidate read his or her work, deliberate seriously, and then report on their deliberations. Bad procedure has driven out good. Come to think of it, the substitution of phony standards for real ones is pretty common in academic life -- witness statistical significance in the social sciences and in medicine; or pretentious jargon in literary studies; or grant-getting fashion-mongering in the bench sciences -- so I guess I shouldn't be surprised. In 1922, A.E. Housman, the poet and professor of Latin, wrote an essay called "The Application of Thought to Textual Criticism," in which he recommended replacing the little formulas of his trade ("The more sincere text is the better one") with -- get this -- actual thinking. His was a shocking proposal -- one that would, if adopted, revolutionize most fields of the intellect, and certainly academic administration. My dream is that if standards for hiring and promotion were to become serious the other lacks of seriousness would themselves dissolve. But you know what a cockeyed optimist I am. "Come now," I hear a dean retorting. "The Letters are merely one element in our beautifully crafted steps. True, none of the steps makes sure that the colleagues have actually read the materials. We don't have time for that. I certainly don't: I have numerous meetings with other deans to attend. The quick scanning of letters by people I do not know about people I am not acquainted with in fields I am ignorant of will have to do. I am, you know, brilliant at interpreting such letters. Surely, my good woman, you would admit that the letters are worth something, that they are some sort of evidence?" No, my dear dean, they are not worth anything. They are not any sort of evidence. That, you see, is the problem. The letters have a hopeless selection bias. No responsible person, for example, writes a letter if she dislikes the candidate. So the letters are a sample with shockingly bad statistical properties. The implied reader (Literary Jargon Alert) is not specified -- is it the provost, the dean, the committee, the colleagues? Cui bono? Whose interest is being served here? So the writer writes strategically, if not confusedly -- anyway, not truthfully. The letters are routinely mishandled -- they are in some states, for example, read by the candidate. And the chair can deep-six letters he does not like (after all, they come to him). So the writer and some of the readers truncate the sample. The writer, therefore, has no incentive to be truthful. If she disapproves of the candidate, she will probably (in the University of California system, certainly) be sued. The rare truthful letter is therefore a disaster. The field for subtle malice is open. That truth-telling should be made so mischievous is an undermining of the moral basis of academic life. The fiction is that other colleges' professors are gentlemen [sic] of leisure who have scads of spare time each autumn to craft their single man-to-man letter this year. The rising N in recent years makes the fiction insulting. In some fields (economics, for example, and now imposed on all fields in a decanal passion for uniformity), it has been commonplace for 20 years to seek 10 letters for each promotion. More recently, it's gotten worse. Figure out the math, and you will see why prominent scholars get an uneasy feeling round about October. Yet the workperson is never deemed worthy of her hire. Aside from the "sincere gratitude" of the chair, no one is offered a cash stipend, or a book or two from the local university press, or the coin of serious consideration (everyone knows that nine other letters are being solicited), or even the courtesy (sometimes promised but rarely acted on) of telling the writer after the dust has cleared what happened and why. The gentlemanly fiction is that the job will be done out of a sense of duty. Under the circumstances it is surprising that so many high-minded people in fact waste their time trying to do a serious job on a letter that is misused or not used at all. High-minded people should not be exploited by a system already corrupted by low-minded, or merely thoughtless, people unwilling to defend their practice. The rhetoric of letters varies radically from field to field. In historical economics, for example, the dual standard of excellence in economics and in history, with two very different scholarly standards, makes even a favorable letter interpretable as bad: "Williams is an excellent archival scholar" can be read as saying that Williams is a bit of a dolt in using economics. In history itself only long letters count, a waste of scholarly time. True, in mathematics the letter can take the form of an ordinal number: "Schwartz is the 16th-best number theorist at a state university in California." Mathematicians think this way. But in many fields (and in math itself, I'll bet) such sports talk is pretty silly. It's not soft-minded to think that intellectual qualities are a vector (or, indeed, a matrix in interaction with outside conditions). It's true. What is worse, the rhetoric varies radically from person to person. In economics it is well known that a certain Nobel laureate claims in every recommendation he has written that the present recommendee is "the best student I have ever had," in a remarkable monotone increasing series since around 1953. The story is a good one, and true. I should make clear, though, that I am not proposing that recommendations for students be abandoned -- the students have no recourse, since they have not yet become active scientists and scholars whose work is available for public inspection (yet it is obvious, alas, that many of the warnings about the invalidity of letters do still apply). Some people -- I can name an eminent historical economist -- are known (well, the one I'm thinking of is known to some other people, in this case, me) to be incompetent at such breathless rhetoric. Pity the candidate who chooses him as her letter writer. But the P&T committee staffed with professors of geology and French does not know this. And in truth, though the N's are large every year, they are not large enough to make the overstatements and understatements knowable even to most people in a particular field. In such circumstances, no one can interpret any but the most damaging letter ("Smith is an idiot. For Lord's sake, fire him"), least of all the collection of a chemist, a biologist, an English professor, and a professor of economics that makes up the typical P&T committee. Forget about the provost being actually able to read the letters as truth, though she will think she can, with great if misled intensity, paying due attention to her mistaken notions of who is best qualified to write, and will imagine she is doing her job. That's nice. We are all glad to make provosts and deans and P&T committees feel good. But some day they should grow up and face the facts. That earlier generations of scholars were the "beneficiaries" of letters does not mean that they should be required now to continue such an idiotic system. That would be a boot-camp or medical-education argument: I suffered through cleaning the latrine with a toothbrush or treating patients after being awake for 36 hours straight, so you should, too. No one but a drill sergeant or the dean of a medical school would think such an argument made sense. In short, the letters are insane. If tables of random numbers became fashionable for deciding on hires, tenurings, promotions, I suppose you, as a serious scholar, would object. And at least a table of random numbers is what it says: random. Why haven't you objected to the system of letters, which has notably less integrity than a table of random numbers? Deirdre McCloskey is a professor of economics, history, and English at the University of Illinois at Chicago and a visiting professor of economics, philosophy, art, and cultural studies at Erasmus University of Rotterdam. She is the author of How to Be Human*: *Though an Economist (University of Michigan Press, 2000). This essay originally appeared in the winter 2002 issue of the Eastern Economic Review. http://chronicle.com Section: The Chronicle Review Page: B14
|
Copyright © 2002 by The Chronicle of Higher Education