First Person

External Reviewers

Article tools

Printer friendly	E-mail article	Subscribe	Order reprints
	Discuss any Chronicle article in our forums
Latest Headlines First Person The Rejection Letter I Wish I Could Send If we had to make up a story for why you might be interested in our position, then interviewing you was too risky. Peer Review Hirings and Firings The new law school at the University of California at Irvine gets some high-profile hires ... and other appointment news. Ms. Mentor Does This Make Me Look Old? Advice on how best to dress, and act, when you look as young as your students. Career News Gone, and Being Forgotten Why are some of the greatest thinkers being expelled from their disciplines? Resource Salaries: Faculty | Administrative Presidential pay: Private | Public Financial resources: Salary and cost-of-living calculators Career resources: Academic | Nonacademic Library: Previous articles by topic | by date | by column Career Talk, Ms. Mentor, and more... Landing your first job On the tenure track Mid-career and on Administrative careers Nonacademic careers for Ph.D.'s Talk about your career Blogs

By Dennis Baron

I have been illustrating this series on the tenure process from the department's point of view with references to the recent tenuring at my university of Alison Porchnik (not her real name). Now I will turn to what some of Porchnik's external reviewers said in their letters about her scholarship and how tenure committees responded to those letters. And I will compare the Porchnik promotion with other recent cases to illustrate some of the other things that can happen when external reviewers weigh in.

Promotion committees rely on external reviews. It is true that the department, the college, and the institution ultimately decide the fate of their faculty members -- to put it bluntly, who can stay and who must leave. But we need the guidance and objectivity that outside reviewers bring to the tenure process to reach an informed decision.

Review letters for candidates in English run long, perhaps three pages, single-spaced. Letters that are much longer (I've seen them up to six pages) or much shorter can raise suspicions. (In contrast, mathematicians seem happy to average a page or less.) Here is my summary of three of Porchnik's outside letters, and the issues that they raised:

Reviewer 1

This reviewer traced Porchnik's intellectual history from her earliest publications down to her in-press book, stressing the significance of the intellectual problems that Porchnik tackled in her work, her growth as a scholar, and the tantalizing promise of her new project.

But the letter wasn't all good news. The reviewer expressed concern about Porchnik's propensity for overstatement, one or two unconvincing arguments that she had made, and an overlooked line of investigation that the reviewer thought Porchnik really should have pursued. Still, the report ended, on balance, with a recommendation to promote.

This is a favorable letter with two issues worth elaborating. First, the reviewer did have some negative things to say about the book. Any negatives have to be acknowledged, both by the department's tenured professors and in my summary of the case for the dean's committee. If I don't explain why the department considered the reader's objections minor, I will probably get called before that committee to explain not just that, but other things as well, perhaps a low student evaluation in a seminar, or a work in progress that seems not to be progressing.

In Porchnik's case I argued successfully that if she had followed some of the reviewer's suggestions, she would have written the reviewer's book and not her own. I also mentioned that three graduate students came out of the seminar in question with dissertation topics, not something that happens every semester.

The other issue is this: I never asked the reviewer to tell me if Porchnik should be promoted. The tenured members of the department make that evaluation themselves. Nonetheless, some letter writers do recommend promotion, and some committees worry when a reviewer doesn't answer the unasked question. The letter may say, for example, "Professor X would certainly be promoted at my institution." Sometimes -- and this is nice for the candidate -- reviewers voice surprise that the person hadn't already been promoted.

In a promotion case for another faculty member -- let's call him Daniel Rose -- an outside reviewer wrote, "Here at Ivory Tower University, we are very fussy about these things," meaning "We certainly wouldn't promote anyone with Rose's record but that doesn't stop Lesser State U. from doing so." We voted to promote Rose, and I had to defend my department against the implication that our standards were too low.

Reviewer 2

This letter started out with some general praise: Porchnik identified a "significant" research problem and produced a "competent" book. There were detailed examples of Porchnik's strengths as a scholar, and even a comment that a conference paper Porchnik gave at the MLA suggested that she was a dedicated and effective teacher. This is indirect evidence, but it's strong.

I got a very different kind of letter from another one of Daniel Rose's referees: The reviewer mentioned Rose's work briefly, then went on to talk about himself for three pages. The shape of that reviewer's career might be fascinating in some other context -- say in his retirement speech, delivered to a crowd who came to make sure he really was retiring. But the reviewer was supposed to be talking about Dan Rose in this letter, not about himself.

The first time I saw a letter like this I thought it was an anomaly, that the referee was simply too full of himself. But I've seen at least three such letters in the past six years, and that's enough to constitute a subgenre, the letter of self-reference. Each time the reviewer mentioned Rose it was to show how he had failed to follow up on some strand of the letter writer's groundbreaking scholarship.

I wanted to write in my report to the dean that this particular reviewer was an idiot. But in promotion cases you don't assassinate referees whom you've chosen yourself. In the end I wrote, "This reviewer doesn't focus on Daniel Rose's work as much as I would have liked, but he does conclude that the work has serious merit."

There was one problem with Porchnik's second letter. The reviewer called her book "competent." I've heard more than one colleague insist that "competent" means the work is truly awful. I disagree. To me "competent" means not stellar, but not bad, either. Sometimes a cigar is just a cigar. Not every letter needs to be wildly enthusiastic.

Certainly candidates' cases are strengthened when letters are full of phrases like "field changing," "redirects the conversation," or "best book on the subject in the last 20 years." But not every book can be the best, not every tenurable candidate signals paradigm shift. Sometimes three reviewers push the superlatives, while the other two balance positive comments with cautionary statements to the effect that the candidate has yet to solve a major problem or crack the top journals. Am I to believe the majority, for whom the candidate can do no wrong? Or the two who take a more balanced approach to the case, concluding that the candidate is promotable, even if she isn't God?

I hate to see reference letters go the way of movie reviews, where only extreme praise counts as positive. Unfortunately it may not be long before letter writers find that even "a bold and imaginative" assessment packs as little of a wallop as "competent," and they start penning claims like, "Alison Porchnik -- the subaltern speaks!" Or worse yet, "Porchnik: Socko Boffo!"

Reviewer 3

This reviewer used all the right phrases, gave lots of particular examples, compared Porchnik favorably to scholars recently promoted at Yale and NYU, and even to one new full professor at the University of Virginia. The letter was music to my ears -- there was absolutely nothing wrong with it, and even though the reviewer was selected from Porchnik's list, she was someone who had never met Porchnik in person and had never had any professional contact with her.

Let me relate the far different letter written by a reviewer I'll call Phil Moscowitz for a faculty member, Harry Block. Moscowitz used his review to deconstruct Block's scholarship so meticulously that everyone who read it suspected there had to be a story behind this hatchet job. It wasn't just a story, it was a revenge tragedy.

A few years earlier Moscowitz had given a paper, and Block was in the audience. Block was convinced that the intellectual life lived at its fullest required him to point out a senior scholar's mistake, so during the question session he stood up and offered a corrective. Three years later Block, feeling he had engaged that senior scholar in a significant exchange, placed Moscowitz's name on his list of potential outside reviewers.

On the surface, Moscowitz's review of Block appeared rational: Each objection raised had merit, and Moscowitz insisted that he had gone out of his way to like the work, but just couldn't. In their totality the objections could damage the promotion beyond repair, but although the letter had no impact within the department, once the story became known, I couldn't ask the college to discount a letter that I had commissioned.

Fortunately for Block, a member of the dean's promotion committee from another department knew the story of L'affaire Block-Moscowitz, and reported it during the discussion of the case (I am not permitted to be present during the discussion of a case from my own department). Block's promotion met no further problems, and I now advise tenure candidates to be very careful whom they put on their list of reviewers.

Another of Alison Porchnik's reviewers gave a general, positive testimonial about her work. There were no details, no examples of strong arguments or ways in which the work was having an impact. Although positive, and strongly recommending promotion, the letter came across as totally without energy. This does no direct harm, but neither does it bolster the case.

If one purpose of external letters is to allow the department's tenured faculty members to judge the reception of the candidate's scholarship, then the general testimonial, like the letter of self-reference, is pretty much a waste. I assume both letter writers meant to support their candidate enthusiastically. In fact, they are wasting the department's time and jeopardizing the candidate's tenure.

The tenured professors in my department know the ins and outs of external letters, but they worry that the college and campus committees won't read these letters with the same understanding. When I drew up my first promotion dossier some old hands argued, "Place the most negative letter in the middle, and put a really strong letter last." That way the reader comes away thinking positive thoughts. I took that advice, only to realize that readers don't always read linearly, even when they're reading tightly plotted novels. Readers of promotion dossiers may jump back and forth, comparing a line in the vita with the candidate's personal statement, comparing a class evaluation with another candidate, generally looking for consistency and for contradiction.

Smart people read these dossiers, and even if they lack disciplinary expertise, they notice a negative that has been camouflaged by positives. The dean's committee can generally tell whether that negative is significant or minor. And if they're not sure, they're not too shy to ask. So the second time I drew up a promotion dossier, I arranged the outside letters alphabetically.

Alison Porchnik had no trouble getting through the English department. The tenured faculty members supported her promotion unanimously. The merits of her case seemed apparent to everyone, and discussion focused not on whether to promote her, but on how to get her through the next level of review most effectively.

When the collegewide committee looked Porchnik's case over fairly closely, its vote on her tenure, though positive, was split. In my next column I'll look at some of the different ways that departmental and college committees talk about, and evaluate, tenure cases.