When a colleague told me last year about a free Web site, RateMyProfessors.com, that allowed students to post ratings of their professors from colleges across the country, I was happy to hear it. I have believed for several years now that student evaluations of teaching should be made public to the students and other faculty members within their own college or university.
Such public airing of the students' evaluations would replace the informal, random, and probably wildly inaccurate snatches of information students get from one another about their professors with a more complete and consistent picture of each one of us.
Then I did what every academic who has ever heard about this site has done: I went online and looked up my own ratings.
Be careful what you wish for.
While I was happy to see mostly positive reviews of my teaching on the site, some of the ratings were positive about a part of me that has little relation on my teaching -- two of them, in fact, praised the fetching character of what was referred to as my "buns."
I needed a few minutes to adjust to seeing my rear end discussed in a public forum. But eventually I soldiered on, and looked more closely at the Web site's system for rating professors. Students were able to provide ratings for their professors in three categories: helpfulness, clarity, and easiness. They had the additional option of bestowing a "hot tamale" on a professor, which meant that they rated him or her as sexy.
In addition to the numerical rankings, students were able to provide a few written comments, and it was in these comments that I found the explicit references to the aforementioned part of my anatomy.
Of course everyone likes a compliment, even when you've done nothing to deserve it -- the only exercise my buns receive is chasing my 1-year-old around the house -- so I found the comments flattering. But, as you might expect, I also found them a little disconcerting.
A few months after I discovered the site, an old college roommate sent me an e-mail that perhaps best encapsulated the reason for my apprehension: "I happened to stumble my way onto RateMyProfessors.com, and noticed that you have a hot tamale by your name, as well as several rather explicit references to your butt. I'm not sure what you've been up to, but it sounds more like Larry Flynt than Chaucer." (Literary aside: Anyone who has read Chaucer's "The Miller's Tale" knows that Larry Flynt and Chaucer aren't all that far apart, but that's another story.)
Comments like my former roommate's, and the inappropriate comments of the students, no doubt confirm for many faculty members what they have suspected or believed for years: that student ratings of teaching are a sham, and should not be used in the evaluation of teaching for tenure or promotion.
A recent article on the influence that a professor's attractiveness has on his or her ratings -- an article for which I was interviewed, and in which my buns again made a public appearance, to the apparent delight of academic friends and former colleagues around the country, many of whom forwarded me the article with appropriately ribald comments -- would seem to provide further evidence that student evaluation forms are valueless.
I have two responses to that: yes and no.
Yes, some student rating or evaluation forms are meaningless. And the ratings on RateMyProfessors.com, I can assure you, fall into that category.
The principle to follow when considering the value of a student rating form or service is simple: Ask stupid questions, get stupid (and meaningless) answers. So if you are asking students to rate their professors on sexiness and easiness, you are not going to get answers that will prove helpful in making thoughtful and careful evaluations of the professors' teaching. Even the questions about clarity and helpfulness, which are potentially interesting, are obviously too vague and general to mean much.
But the silliness of such sites, and the meaningless data they provide, should not be used as a club to bash away at all student ratings. The principle applies the other way as well: Ask good questions, get good (and useful) answers.
In fact, a considerable body of research suggests that as a measure of student learning, responses to some questions on teaching-evaluation forms correlate very closely with data from more-direct measurements, such as exams. The trick is to identify those questions, and find a way to get them onto your evaluation forms.
Professors and administrators interested in learning more about finding those questions should consult "Using Student Ratings," a white paper by Ken Bain, the director of the New York University Center for Teaching Excellence, on teaching-evaluation forms. Bain makes an argument in that essay for five broad questions that have received strong support from the research -- an argument too full and convincing in its original form for me to summarize here. But I'm sure you're interested in the five questions. The first three ask students to provide an overall rating for the instructor and for the course, and to estimate how much they learned in the course. The remaining two questions ask students to rate how effectively the instructor stimulated their interest in the subject and how effectively the course challenged them intellectually.
Many teaching-evaluation forms, unfortunately, are filled with silly and irrelevant questions that have nothing to do with excellence in teaching, and have no support from the research. At my college, we have a question on our forms that asks whether the professor starts and ends class on time. Whenever I see that question, I feel like I'm just a couple of bad classes away from being handed a time card.
Worse still are the questions these forms don't ask. Out of the dozens of questions on our form, not one -- not one -- asks students whether they have learned anything in the course. Silly me, for imagining that the form might ask a question about the most important part of my job: helping students to learn.
So sometimes our form feels to me about as useful as the questions on RateMyProfessors.com, which is a shame. Silly questions like those only fuel faculty skepticism about the relevance of student evaluations of teaching, and allow many faculty members to dismiss an opportunity to receive feedback on their teaching that might, done well, prove useful to their development as educators.
All of which is not to say that I have not tried to make use of the ratings of my physical appearance on RateMyProfessors.com. In fact, when my wife and I are having one of our occasional marital spats, I like to remind her of her good fortune in even having me, and my buns, around the house.
"Listen," I say to her. "Count yourself lucky. Some people would love to spend the rest of their lives with these buns."
"Have another cupcake, size 36 and growing," she says, not nearly as concerned as I would like her to be.
Her skepticism I can tolerate; I've been living with it for 10 years now. I'll really be worried when the provost calls me into his office to deliver a stern warning:
"I don't want to sound too many alarms, Jim, but we're concerned that your student ratings have fallen. ... Have you considered the Stairmaster?"
