Proving that he is not only forward thinking about making his research open, Mark Sample yesterday wrote about making teaching evaluations public. Why–he reasons–allow a self-selected group of student to provide public feedback on a site like RateMyProfessor when he has the full spectrum of qualitative and quantitative data from a much wider sample (pardon the pun) of his students?
Sample is using Scribd to host PDFs of his evaluations and has even released them under a Creative Commons license, anticipating that someone might come up with a way to remix them. This is something that I plan to do myself in the near future. But since I know I’m prone to moving toward openness like this, I wonder reasons one might have for keeping evaluations private. Thoughts?
Follow Profhacker through your favorite RSS reader: 
17 Responses to Short-circuiting RateMyProfessor
Boone Gorges - August 6, 2009 at 12:54 pm
When I read about Mark’s proposal I wondered about how the students doing the evaluating would respond to his openness. Mark says that he’s releasing their evaluations under a CC license – but does he have the right to do so? Who owns student evals?
From the faculty point of view, though, I’m totally game for this kind of sharing. Seems to me that one of the most commons arguments against faculty openness (when it comes to course materials) has to do with protection of one’s intellectual property. But surely that argument isn’t applicable in the case of evaluations. What’s left to recommend privacy? Fear that everyone else will find out how much your students hate you? Or maybe how much they love you?
Brian Croxall - August 6, 2009 at 2:06 pm
It’s an interesting point you make, Boone, about the copyright to the students’ evaluations. Based on my experience working on author correspondence, the copyright automatically goes to the author, even if the author is anonymous. That said, I can’t see anyone wanting the rights to what they have written since is expression that is supposed to be reflective of another (the person being evaluated) rather than creative expression (which even letters can be since they are reflections of the self).
Perhaps if more people move to making evaluations open, schools would place disclaimers on the evaluations that give professors the right to make such things public.
G. Michael Guy - August 6, 2009 at 2:19 pm
I’ve been publishing all my student evaluations on my website for several years. I tell all my students that they will be there, like all the others they can read, before they write them. I’ve never had anyone object to it yet. They are, after all, anonymous. Some are very harsh (from long ago) and some are much more flattering. But they go up, regardless. I painstakingly type (or more recently use Voice to Text) every handwritten comment–when legible. I keep the originals around in case someone doesn’t believe I’ve done it accurately. I’m hoping my current university goes to electronic evaluations soon so that I can save myself this step and further insure their accuracy. But I’m not holding out waiting for that!! I’ll admit that I never thought about who “owns” them.
Additionally, I provide my students with an anonymous feedback form on my website so they can give me feedback in time to make a difference in their class. I can only be responsive to feedback after I receive it. This method gives students an anonymous voice before its too late for them. It also gives the students less need to complain to others since they can bring it right to the source. That’s a win-win!
Boone Gorges - August 6, 2009 at 2:23 pm
Brian, your comment on prior notification is echoed by a response I got on Twitter a few minutes ago by G Michael Guy, a CUNY math prof. If institutions don’t print disclaimers on the evaluations, individual instructors can put them in the syllabus.
On Guy’s website (go to the Teaching section) he posts PDFs of student responses going back a few years. Very cool to see that people are doing this. I think it would be really great if there were a standard format for releasing this data, as I bet you could do some cool analysis on large amounts of it.
Brian Croxall - August 6, 2009 at 11:41 pm
Michael, Glad to hear that students haven’t protested about your doing this. Do you keep stats of how many page hits your evals get? Do you think incoming students actually read them? Or do outgoing students come back and see what others in the class say?
Brian - August 7, 2009 at 7:32 am
Nels, you bring up a very good point that I (as a white guy) hadn’t thought about. I will confer tomorrow with some of my female colleagues to get a sense of the quality of their evaluation comments and whether they would be comfortable in sharing them. If comments regularly and blatantly misrepresent an instructor rather than discuss his or her teaching, then they suddenly become much less useful for all.
Do your instructors ever vary the questions to try to cut down on queries that seem to prompt particular and uncalled for responses? I’ve found that one of the best things I’ve done in my recent teaching is to write my own evaluation forms. While students are certainly free to write whatever they want, asking the right questions tends to get me more valuable feedback.
Nels - August 6, 2009 at 9:02 pm
I have to read all evaluations for everyone who teaches in our writing program, and I think it would be absolutely horrible to have some of the sexist, racist, and homophobic things that some students write on evaluations be given an even bigger audience and sense of legitimacy. I don’t like reading about what students think of a particular instructor’s breasts, but I have to do it. And I think it would be wrong of me to require faculty in my program to post such evaluations online. Should students know how fat a professor is before walking into her or his class because half the students mention it on evals? Does an instructor have to know she is walking into a class for the first time where everyone in it has read the evals that call her a racist b****? Some of those students will have judged her before meeting her and not even give her a chance. Should a woman have to have eval after eval posted where, under “What can the instructor do to improve the course?”, students have repeatedly written, “Give us blowjobs?”
Sure, I’ll post mine. As I found out going up for tenure this past year, I have some of the highest scores and best comments anyone around here has ever seen. It’s easy for some of us white guys to put ourselves out there like that. But having read hundreds of evaluations over the years, I would hate for some of the things students call faculty to be made public.
Benjamin Miller - August 7, 2009 at 9:50 am
“While students are certainly free to write whatever they want, asking the right questions tends to get me more valuable feedback.”
Which is one of the reasons that RateMyProfessor’s chili peppers are especially problematic: it invites the kinds of sexist and sexual comments that Nels describes. In light of his comments (and I’ve heard similar things from some of my grad school classmates), I certainly don’t think any instructor should be required to post their evals – especially going back through the entire archive.
At the same time, I wonder whether telling students in advance that their evaluations will be public, as Michael has been doing, would help to curb some of the nastier impulses. Describing inkshedding – in brief, jotting down ideas, passing the papers, writing in response, and repeating the cycle – Russell Hunt (among others) has written about “the way audience pressure (which the writer may or may not be conscious of) affects the shape of the text.” If making your writing public to your peers can improve “things like clarity of handwriting and increased explicitness of reference,” could it also usefully increase a sense of shame at inappropriate sexual remarks?
I hope so, and I want to believe so. Still, as I write this, I realize that part of what causes that productive self-consciousness is the immediacy of readership, the fact that authors themselves immediately become readers of similar texts, and (often) the lack of anonymity: you pass your text to someone next to you. Since anonymity is the rule of teacher evaluations, rather than the exception, and since there is almost always a delay between the writing and the reading of them (while teachers submit grades), there’s a big risk that the payoff won’t actually appear.
So now I’m thinking that what really enables Michael’s successes is his – sorry, that’s some mouthful! – his real-time, term-time feedback form. If students see that their words are actually being taken seriously by someone with authority, maybe they’ll take their own words more seriously thereafter.
Nels - August 7, 2009 at 11:17 am
We can add questions to our evaluations, but we cannot change what is already there. And adjunct faculty especially would be criticized for altering forms. Our questions are basic and open-ended. What did the instructor do well? What could the instructor do to improve? I like the questions that the university requires. And I don’t think students will change their answers if they know they will be made public as long as they remain anonymous. I think, knowing they will be public, that some students will leave even wilder and crazier answers so they can go online and point to the one they wrote. Anonymity plus public legitimacy can be dangerous.
I used to explain to my students how my evaluations were used, who read them and when. They would start to get crazy in their comments. “Give him more money!” “Hey, he said the dean will see this, and I want him to know X, Y, and Z,” which have nothing to do with me or my class. “Nels is cool, but my prof in X dept who did not want us doing evals is horrible. Fire him.” I’ve stopped telling them who reads the evals so they’ll just write about me and my class.
And I did not mean to imply that these comments only happen to women. I’ve gotten homophobic comments before about how I seemed nice enough but they’d never want to be alone in the office of a fag. And I’ve read comments about how hot a particular man’s ass looks in a certain pair of pants he would wear. And there are those about how fat particular men and women are.
I think the idea of making our evaluations public sounds great when some of us just look at our own evals and those of our immediate colleagues. But adjunct faculty and graduate students might not feel so open considering that they do their jobs in a different environment than tenured and tenure-track faculty.
Jason B. Jones - August 7, 2009 at 11:55 am
@Nels: Of course it’s true that adjunct faculty and graduate students might not feel so open; that said, they are already open to at least some extent–that is, the extent to which ratemyprofessor.com comments exist.
Also, some universities post only the numeric scores for faculty, which is disappointing.
So while I don’t think making every single evaluation available online would be particularly helpful, I do think that even marginalized faculty might benefit from putting some evaluations online, and then commenting directly on them. Not the hateful ones, of course–but a helpful reframing of even middling comments might be useful.
(I remember my first semester teaching at UWM as a grad student–one of my peers got as an “evaluation” an incredibly detailed critique of her fashion choices. It was a revelatory experience.)
I have filed this for too long under the “things I’d like to do, but can’t quite bring myself to bother with” category . . . maybe soon.
Nels - August 7, 2009 at 2:22 pm
I might support the idea of putting some evaluations online. So far, it seemed like the conversation was about putting them all online. And I get called “fag” enough in my life without having to look at it on those evaluations every time I go to whatever place they are posted; I hated having to handle them for my tenure dossier even if they were the distinct minority. And I suspect that my colleagues who get called words I don’t feel comfortable typing on here might feel the same way.
Do students still use RateMyProfessor? No one has posted a thing about me on there (or on the one on MySapce) since 2005. And I just checked for several faculty in my department and found practically nothing for anyone. And no one said anything half as cruel or harsh as they do on the “private” evals.
Mark Sample - August 7, 2009 at 3:04 pm
I’m happy to have generated such thoughtful comments about putting my teaching evaluations online. I agree that there are some questions to consider. Boone asks who owns the evaluations, and in my case I think that answer is easy: it’s the Commonwealth of Virginia, who has already made the quantitative part of the evaluations public (but very hard to actually access). As I mentioned in my original post, many other aspects of my job (including my salary) are already public information, so it makes since that my evals, upon which my salary is somewhat based, are too. Also, because the anonymous written comments are given back to me with no further instructions, I consider it a kind of “fair use” to make them public.
The possibility of degrading comments gaining a wider audience is something I hadn’t considered. I’ve been lucky that most of the comments I receive actually do pertain to my teaching (I’ve received the more personal comments about looks or clothing on RateMyProfessor). Still, an occasional personal attack is something I can live with and I don’t believe the public airing of it would give any legitimacy to the offensive remark. On the contrary, I’d see it as something to address in the reflective scaffolding I aim to build up around the teaching evaluations.
Nels - August 7, 2009 at 3:55 pm
I really don’t mean to keep dragging this conversation on, but I keep feeling like I’m being misunderstood. And that’s largely my fault because I dumped a lot of individual points into quick comments. But I’ll leave it after this because I’m starting to sound like a whiner. The last point I think I should make, though, is that calling someone a fag or using the N-word or other racial epithets or a cunt or something like that is not a simple personal attack. It is part of larger racist, sexist, homophobic ideologies. And, to be fair, every student who has ever called me a fag on evaluations has given me a perfect score in almost every quantitative category. They comments have been things like, “Dr. Nels is cool, but I’d never go to his office because I wouldn’t be alone with a fag” or “He’s the best prof I’ve had here, but I won’t be taking anymore of his classes because it creeps me out to be that close to a fag.”
Comments about looks and clothes perhaps belong in a separate category of personal attacks, though I think size-ism and ageism is often at play in those comments. It’s great if you have the strength of character to let such comments role off your back as mere occasional, personal attacks, but for those of us who live with those attacks on our evaluations as well as so many other aspects of our professional and personal lives, it’s not that simple. Or maybe I’m just weak.
I was told after last year’s tenure case that certain members of the various committees were stunned at the high marks and intense praise given to me on my evaluations. I was earning perfects scores in my first semester here, which meant that I actually failed the part of my evaluation that asks if there has been consistent improvement over time. But even the people who gave me such perfect scores sometimes said things that I don’t want to be made public. Yes, if I were to make them public, then we could have a conversation about those words and those ideologies and all that, but some of us who live with these kinds of attacks on a regular basis get tired of having our lives used in such ways and put on display in such ways. We get that people are often trying to have our best interests at heart, but sometimes it just hurts.
Jason B. Jones - August 7, 2009 at 10:24 pm
Nels, I’m sorry that you’ve felt misunderstood here: I do understand that the phenomena you’re describing are related to structures of privilege.
Derek - August 8, 2009 at 10:33 am
I just wanted to make a quick comment here about student perceptions of course evaluations. We have online evaluations at my institution and response rates can vary wildly–sometimes as low as 30%, other times well over 80%–depending on the instructor and the class. That, along with other anecdotal evidence (like Nels’ anecdote above about students writing off-topic messages to their deans), says to me that our students don’t really understand the purposes of course evaluations.
We’ve had some talk at our institution about working to improve students’ understanding of and involvement in the course evaluation process. I wonder if making evaluations more public might help with those efforts.
Harvard University takes an interesting approach to making evaluations public. A team of students read all course evaluations and summarize the responses to the open-ended questions for each course and instructor. These summaries, along with the quantitative data, are available online in the Q Guide. I know that students frequently consult this guide when selecting courses. Producing the Q Guide is quite an effort, but it’s one way of making evaluations public while taking away the power of individual, offensive comments.
The History Enthusiast - September 17, 2009 at 11:26 am
As a woman who has received inappropriate evals that don’t actually address the course content (I’ve been called a b, for example), I would not be comfortable putting all my evaluations online. Students are not necessarily discerning enough to give instructors and professors a fair shake after reading those sorts of comments; remember, first impressions are really important. Imagine this scenario, wherein I have posted everything online: early in the semester I put my foot down in class about an assignment’s due date, and students immediately start thinking, “man, she really IS a b like that other person said online.” Then the negative comments will just be perpetuated again and again whenever I assert myself as the instructor and maintain class discipline, because student experiences will be drawn from their interpretations and reactions to those past evals, and not solely from their personal experiences in my class. I could certainly see an eval or two that says, “I totally agree with her evals from last year where people said she was b****y.”
I’m not sure if I’m explaining this clearly, but hopefully you get the drift. It sounds like it could turn into a nasty cycle of “is she, or isn’t she?”
[EDITED by ProfHacker to fix formatting]
The History Enthusiast - September 17, 2009 at 11:27 am
Not sure why the formatting went haywire, but by “b” I mean “bit–.”