College administrators are primary targets of an unbelievable amount of old-fashioned junk mail, so I have a longstanding habit of opening my mail by size. I start with the smallest letters (usually thank-you notes from students, faculty members, and local residents), then move to the standard letter-sized ones (usually campus mail with forms that need signing), and finally to the oversized packets. I tend to flip through the large ones absent-mindedly because they are typically advertisements.
Except sometimes they aren't, as I discovered when I opened one large packet recently to find I had been summoned for academe's version of jury duty: the annual college rankings survey by U.S. News & World Report.
I've served on jury duty in three states now. In Louisiana (New Orleans, no less) I felt like I had walked into some sort of satire that was straight out of Walker Percy or John Kennedy Toole. In Mississippi I was trapped in a jury that was a bizarre incarnation of 12 Angry Men, complete with crying and near-shouting in the jury room. Not long ago I was called up for duty in my current city in Tennessee, an experience that was mildly inconvenient but also a sobering reminder of the importance of the peer-juror to the legal process.
In each of those experiences, I have been struck by just how much grousing took place in the court antechamber prior to the judges' arrivals. People bandied about every excuse in the book in an attempt to escape. When the judge appeared, though, especially after the obvious shirkers were humbled, the room always became much more serious. On the occasions when I have been empaneled on an actual jury for a specific trial, there was rarely laughter in the jury room. Lots of chit-chat, to be sure, but little levity. Too serious a task was under way. We faced the reality that if we found the accused guilty, we would have to go back into that courtroom and look him or her in the eyes with the full knowledge that we had made a reservation at a prison for a very real person. It's a somber experience.
For years I've wondered what goes through the minds of a different kind of juror, the evaluators who contribute to the survey. As academics, we love to complain about the rankings even as we trumpet their results on our Web sites and recruiting brochures. I know that I've certainly done my share of eye-rolling, even as I have bragged on my current institution's consistent results and my undergraduate alma mater's perch atop its category. Do the evaluators ever smirk at the names of rival institutions and adjust their scoring downward? Do they rank their alma maters with a bit of grace? Do they take into account a bruising letter of rejection they received many years ago from a particular college?
This is my first year as the chief academic officer of my university, and I've had a number of moments when I've felt the gravity of the position. When they scanned my signature for insertion into our diplomas. When I read the names of the graduates at my first December graduation. When I worked on my first budget cycle as a member of the leadership team.
For some reason, the arrival of the U.S. News survey form struck me the same way, as a kind of milestone.
Once I realized what the packet was, I actually parked it on my worktable for a couple of days. The survey deadline was a few weeks away, so I knew I had some time. During my next meeting with our executive vice president, who had preceded me as chief academic officer, I asked for advice about filling out the survey.
"Just be honest," she said. "And don't be afraid to do some research if you aren't sure about something. You know how important this is." She should have been wearing a black robe and carrying a gavel when she said that. It had the same sense as a judge's instructions to the jury.
Later in the week, I had my assistant stake out a couple of hours so I could work on the survey uninterrupted. I read the instructions as though they were legal documents. They were clear and helpful. I unfolded the survey itself, noting that it was grouped by state and then alphabetical order. When I did not see the University of Alabama at the top of the list, I realized that somehow I had misunderstood that the academic-perceptions rankings—the part in which I was participating—were limited to true peers. I was evaluating only the institutions that shared our classification, not the entire region's roster of institutions. It was regional universities only for my review.
My next surprise was how many institutions I had never heard of on the list. I blog for The Chronicle and keep up with academe pretty closely, so that was a significant discovery for me. The instructions indicated that if I did not know about an institution, I should mark "Don't Know." I decided that if I literally had no idea about an institution, I would mark such an answer, even if I had a suspicion that some of those institutions lacked true quality. Ignorance, I determined, holds a kind of declarative value in these cases.
As I began to place my black ink "X" in the appropriate boxes, I stopped again. For some reason, an image popped into my mind of the countless course evaluations I've completed in my lifetime. I knew that tenure and promotions rested on those results. I also knew that students tended to inflate the scores unless they were having terrible experiences in a course.
The top rating on the U.S. News form is "distinguished," and I determined in advance that I would reserve that level for a truly small handful of institutions. The image of a bell-shaped curve suddenly was pretty comforting to my conscience.
Working down the listings, my mind wandered in different ways about my perceptions of the institutions. In some cases, I thought about their football programs (amazing how pervasively sports shapes our perceptions of institutions, even in the realm of academics). In others, I thought of particular academic programs (interesting how schools of pharmacy or law pop into mind as markers of overall excellence). In still others, I smiled as I thought of friends who taught at the institutions (though I likewise found that I sensed little correlation between my perceptions of the institutions themselves and my fondness for some of the faculty members).
This year the survey included a number of additional categories for separate review. I filled those out fairly quickly and decided to let the completed form sit for a bit before mailing it, which was a good thing since I found an error I'd made. For some reason I decided to make a photocopy of the survey for my files to compare against next year's form, after I've had more experience and contact with other institutions in my new administrative role.
So the results of this peer-juror's deliberations went to the great faraway judge who will compile the various verdicts from across the region, delivering the ultimate decision as the fall semester opens. I was glad to be rid of it because my to-do list is so long at this time of year. Whew.
The last time I served on a local jury, I left the courthouse feeling like, had I been the accused, I would have been glad to know just how seriously the jury took the task. I feel that way about participating in the rankings survey. I am glad that it is, like much of higher education, a peer process and not something left up to remote bureaucrats and talking heads. Indeed, the affirmative scores that my own institutions have received in the past seem somehow more humbling now that I've seen how the proverbial sausage is made.
