On Twitter and in comments, Jessica has asked about “interpreting student evaluations that are heavy on stats and light on narrative.” Since my department doesn’t use our university’s standard evaluation form, and doesn’t have any bubble-sheet-type questions, I’m probably a curious choice to respond, but I thought ProfHacker readers could chime in.
First, it’s always helpful to keep in mind that institutional evaluation forms are probably not designed to glean useful information about your teaching practice or about your course. They’re (obviously) designed to cover a whole range of courses and teaching styles, and so are only going to partially reflect the realities of your class. Moreover, the real purpose of an institution-level evaluation form is to compare your teaching with others at your institution, usually for the purposes of promotion, tenure, or other review. The numbers may not immediately tell you whether to incorporate more collaborative assignments, but they probably will speak to the question of whether your teaching is broadly comparable in quality to that found in other classes.
A corollary of this probably rises to the level of ProfHacker dogma: If what you want is information about your pedagogy and your classes, then the best thing to do is design your own evaluation form. You design an evaluation that assesses the goals and effectiveness of your pedagogy, and offer that as a supplement. (For further points on this, see: Billie’s post on “Mid-Term Evaluations,” Brian’s on “Getting the Most Out of Your Evaluations,” plus Billie on “Reflexive Pedagogy.”
Having said *all* of that, however, it seems to me that the most important use of the numbers is historical: Rather than relying on your memory of how one iteration of a course does rather than another (which can easily lead to the overreacting I copped to yesterday), you can see how the students responded. Also, in some departments you could probably open a discussion about best practices by looking at anonymous statistical summaries of everyone teaching a particular course (such as a survey, or Biology I, or something).
ProfHacker readers: Aside from producing pretty, pretty graphs, how do you mine numerically-based student evaluations?
Image by Flickr user D. Sharon Pruitt / Creative Commons licensed
Follow Profhacker through your favorite RSS reader: 
6 Responses to Working with Evaluations
William Patrick Wend - February 19, 2010 at 5:15 pm
We get an institutional mean score for each component, which I found very helpful. I saw not only what my students thought of me, but how I stood compared to others both in my department, and across the entire campus.
Jason B. Jones - February 21, 2010 at 10:21 am
I was probably being a bit cynical: Certainly, the evaluations properly signal to students that their views matter.
Nels - February 19, 2010 at 1:22 pm
Tria, I know of schools that say students cannot get their grades until they complete all evaluations for all their courses, but that’s something beyond your control. I’m surprised a school would go online but not build in some kind of assurance they will be completed.
Tria - February 19, 2010 at 1:08 pm
My institution has students go online to fill out an electronic evaluation on their own time. This means that I’ll have classes of 24 students where only about 3 to 8 of them actually complete the evaluation, although I remind, ask, and beg my classes to do so. I do conduct my own mini-evaluations in the form of “Start, Stop, Continue” from time to time, which are useful, but I’d like to have more “official” data to work with.
I’d be interested in suggestions for how to get more students to actually complete the evaluations.
Nels - February 18, 2010 at 7:10 pm
At my university, you not only get your numbers but the average of the department, and that has proven useful for a lot of people. If you get all 3.8 on a 5.0 scale, that doesn’t say a lot. But if you compare that 3.8 to a department average of 3.9 or 4.4, then you have a greater chance of learning something.
I write this as an administrator who has often has meetings with adjunct faculty who get their evals back, see that they are below the department average, and want to know why they are lower than expected. The answer is usually not clear, but the conversations are almost always productive. When I look at how our department averages compare to other departments, I often see major differences. The subsequent conversations about why can be really useful.
So, in a nutshell, it’s not the numbers that are instructive in and off themselves but the subsequent conversations that can be very useful.
JoVE - February 19, 2010 at 11:23 am
If there is a freeform comments piece to the institutional form, you can also guide students in what to put there. Ask them for the information you want.
I used to ask them to tell me what was the best session of the term (or up to best 3) and why and “If I was going to spend time seriously revising one session of this course before I teach it again, which one needs the most serious attention”. That got really good information.
My interpretation of the institutional purpose is different than yours (though not incompatible). The purpose of having institutional teaching evaluation forms is to demonstrate to students (and others) that the institution cares about teaching. Whether they actually care or whether the results of student evaluations are actually used in a meaningful way is neither here nor there.
When collecting your own information, it is also worth thinking about timing. The last day of class is the point where students are most likely to feel anxious about their performance and thus think they didn’t learn anything. I’ve had students come up to me after an exam asking if they can change their evaluation because they were better prepared for teh exam than they thought they would be :-)