next »

[anon1787]

I've noticed previous discussions on here about student evaluations and some helpful tips on how to improve them (though one can never have enough of them), but I would appreciate any advice on explaining "bad" evaluations in a required written review. Here is my situation:

I lecture at a large public university where it appears that the administration seems to be obsessed with the numbers on student evaluations (which I have painfully learned is common). Moreover, the average numbers seem to be inflated (like student grades). On a 5-point scale, our department mean on overall teaching effectiveness is about 4.4. In my case, my average was 0.20 below the department average in most of my courses (the rest being average). This might not seem like much, but in a previous evaluation the chair felt the need to note that my average was 0.07 below the department mean, which seems to have been the main reason why my performance was rated as merely "satisfactory" on a similar scale (i.e. a "C"). I can point out that this was the first time teaching these courses (as in the prior case of sub-performing by .07) and do the "what have I learned from the criticisms by snowflakes demanding more youtube videos" routine (and yes, I actually received that criticism), but I don't know whether that's enough to prevent a dreaded "needs improvement" evaluation, which would seriously damage my employment prospects.  Note also that we now have a new department chair (who seems a bit less well-disposed towards me). Any good spin doctors around? Thanks.

[circularity]

So, you have to be in the top half of the department to avoid a rebuke?

Assuming you get your evaluation forms back, you might look through them to see if there are any outliers (like a student who gives you all 1s and rants about something unrelated to the class in his comments) you can argue should be thrown out before calculating your "real" average. One or two outliers can really hurt you because students who like you can only help you by ranking you slightly above the average, whereas students who dislike you can hurt you by ranking you way below the average. If you're on a 5-point scale and the average score is 4.4, a single 1 is 3.4 points below the average, which takes 6 5's to cancel out and bring you back above the average score! (One 1 and five 5's only average out to 4.33.)

[caravaggiojr82]

When I was in an administrative role (department-level), evals mattered to me only in the sense that I wanted to see how the instructor and/or job applicant characterized them, what their attitude was. When we had applicants who had some no-so-nice batches of evals, I was looking for an attitude that didn't dismiss the evals, but rather an attitude that said, "Yes, I have certain pedagogical weaknesses--and I'm working on them in the following ways..." What turned me off the most--the absolute most--was the applicant that proceeded to lecture the search committee, at some length, about the inherent flaws and absurdities of "student evals" (yep, he used "air-quotes"). When he finished his lecture, I found that I'd agreed with most everything he said about why evals are often problematic, but I didn't want to hire him. For me, it was all about attitude. Tell me what you learned from the evals--whether the evals were good or bad--and how that learning is informing your growth as a teacher.

[spyzowin]

So, you have to be in the top half of the department to avoid a rebuke?

Assuming you get your evaluation forms back, you might look through them to see if there are any outliers (like a student who gives you all 1s and rants about something unrelated to the class in his comments) you can argue should be thrown out before calculating your "real" average. One or two outliers can really hurt you because students who like you can only help you by ranking you slightly above the average, whereas students who dislike you can hurt you by ranking you way below the average. If you're on a 5-point scale and the average score is 4.4, a single 1 is 3.4 points below the average, which takes 6 5's to cancel out and bring you back above the average score! (One 1 and five 5's only average out to 4.33.)

The middle is pretty much useless. The comments usually come from the 5's and the 1's.

The best way to handle the evaluation issue is to wrangle your way onto the committee responsible and start an effort to rewrite the questions to remove anything that is merely popularity. Make everything quantitative. That'll level the playing field. How often was the professor late to class? how often did the professor cancel class? how long did it take for the professor to return graded assignments? what % of the textbook did the professor assign? and so on...

[mountainguy]

Caravaggiojr82's advice is spot on here. Be ready to explain what you've learned from the evals, and how you plan to improve your teaching. You can ignore the nasty or pointless student comments, but I find that most eval sets contain at least one or two suggestions that I can implement for the next semester. Try to focus on those. Another possibility is to offer your chair evidence that you're improving your teaching through other means; ask colleagues to come observe you if you don't have peer observations already and ask them to write up a report for you. Then tell your chair how you're implementing those suggestions.

Oh, and administrators who act punitively toward instructors for falling slightly below the department averages are just plain bone-headed. It's called an "average" for a reason. There's a real difference between getting consistently rated in the 3-4 range (with 5 being the best) vs. being consistently rated in the 1 range (and I have seen some instructors get ratings that low).

[avaya]

For the future, have you considered doing a mid-semester review by which your students can give feedback on what is going well and not so well in the class?  These tend to improve your end-of-semester evaluations for at least two reasons:

1.  You can make some changes mid-semester that actually improve student learning.

2.  Students feel that you have listened to them and thus will rate you higher.

I know it sounds like a fluff exercise, but in a few cases, I had no idea that some issues were really hindering student learning.  I made some changes that did not affect the integrity of the course, but that helped students a great deal.  In one class, almost 50% of the students independently named one thing that was really bothering them.  Fixing it was easy.  My course evals for the end of the semester for that class were particularly high.

(BTW, I have students me what is going well in addition to what can be improved.  I tell them that this way, if 5 students don't like something, but 25 do, that gives me better information than if I had not asked what students felt was going well.)

[mountainguy]

Like Avaya, I too have found success with giving midterm evaluations. The students feel like they're being heard. I'm curious though to hear what others who give midterm evaluations do when students complain about things that you're not open to changing. Do you typically ignore those suggestions, or do you you reiterate why those policies/assignments/whatever are in place???

Anyway, I don't mean to hijack the thread. It sounds like the OP is facing a seriously stressful situation, and I hope things improve.

[philrels108]

One factor that can provide some context for the interpretation of the data that students surveys provide is the distribution of grades in the class or at least the average grade earned.  If the average grade earned in your courses were lower than the department average -- or anything below a B, if the department averages aren't available to you -- you could play that up. Of course, if the average is lower than the department average, this won't work to your advantage.

Tangent/rant (= tangerant?): I understand why administrators yield to the temptation to treat student surveys as evaluations of teaching, but I wish they'd stop abdicating their responsibilities here.  Everything I've ever read about student surveys makes it clear that they're not themselves evaluations, but provide data that should be taken into account in evaluating teaching.  It's appalling that anyone would take your being "0.07 below the department mean" seriously.  I'm lucky enough to be at a school where admins don't over-emphasize the importance of the numbers the surveys provide.

Good luck!

[msparticularity]

The other thing I have to wonder is whether the evals for all of the courses are being averaged into your departmental mean. If so, do you have any way to gain access to the averages for courses that are comparable to your own? Often new faculty teach more lower-division courses, and evals in those just tend to be lower because the students enrolled include many who shouldn't be there. Faculty who teach only upper-division courses by and large do not have to deal with the students who really don't belong in the course--either due to skills/knowledge or due to interest in the subject.

That said, it's also probably worthwhile to look at the individual scores. My evals last year (my first semester here) were good overall, except on the ones that asked students to rate whether assignments were a clear and whether I grade fairly, where they were pretty mediocre. I teach the one-and-only writing intensive course that every grad student in my department has to take, regardless of their area of specialization. Thus, I get lots of people in there who are accustomed to solving problems and bubbling in exam answers in Scantrons--which they believe is "clear," "fair" and "objective." Ergo, writing assignments are subjective and inherently less fair. I discussed this disconnect between expectations and my course with my faculty mentor and my chair, as well as ways I could try to cope with it. I've gotten much more transparent about how weird/different/difficult this is for many of my students, and they've become more patient with me about sucking it up and just doing the work. I also do an early assignment as a formative assessment, and that grade gets dropped if they improve their skills later. I'm still doing pretty much the same assignments, but it does seem to have made a difference. 

[kedves]

If your grade distribution is much more rigorous than your colleagues', that might also help explain the difference in a way that a chair would hear as pleasing or acceptable.  Anticipated grade is the best single predictor of rating.  (You also need to relax your distribution if that is the case, though.)  Most other explanations you make--and there are many things that could be happening, all of them real reasons beyond your control--are going to be heard as excuses. 

The main thing to do is to develop a plan for improvement to show that you take these things seriously, including your own mid-term evaluations.  There are many things you can try to improve ratings, but there is also a definite limit to how much you can change them.  It is hard to know your chair's perspective, if this is a genuinely felt albeit statistically pointless concern, or if the teaching-improvement directive is coming from higher up and this is how it is performed symbolically, or something else.

Good luck.  It sounds stressful.

[anon1787]

Thanks for all of the helpful suggestions. Just a couple of responses:

1) Given the administration's obsession with the numbers and the customer satisfaction orientation that it implies, statistical arguments about things like outliers seems a bit futile except perhaps in a small (<25) class (and I scored above the department average in the one small class that I taught).

2) I definitely will bite my tongue and resist the strong temptation to criticize students evaluations as a tool, and tell them about the adjustments I'll make in the future in response to a couple of student criticisms.

3) Some students did grumble about my courses being too demanding, but while the chair will likely acknowledge this, I don't think that it offsets it enough in the chair's mind. My plan is to mention it, say that I may have overestimated what students are capable of, and slightly lower the pitch of the course (and the tests in particular). I tried this before--in exchange for making my grading curve less generous--and it seemed to improve matters (tough exams seem to cause students to become far more angry and frustrated even if the curve is generous).

4) The midterm evaluation is an interesting idea that I'm considering. I've read several things about what kinds of questions to ask. I'd like to make it a 2-way street, though, and ask questions that deal with whether the student has been putting in enough time, attending class, etc. to emphasis the responsibilities of the student in order to maximize student achievement/customer satisfaction. Is this advisable?  And like Mountainguy, I'd like to know how people handle student complaints when student suggestions aren't implemented. Do you say at the beginning that some things aren't open to being changed to avoid most of those types of suggestions?

[lousia]

4) The midterm evaluation is an interesting idea that I'm considering. I've read several things about what kinds of questions to ask. I'd like to make it a 2-way street, though, and ask questions that deal with whether the student has been putting in enough time, attending class, etc. to emphasis the responsibilities of the student in order to maximize student achievement/customer satisfaction. Is this advisable?  And like Mountainguy, I'd like to know how people handle student complaints when student suggestions aren't implemented. Do you say at the beginning that some things aren't open to being changed to avoid most of those types of suggestions?

Last semester for a midterm evaluation I asked:

1) What can I as an instructor do to help you learn more in this class?

2) What can you as a student do to help you learn more in this class?

3) What is the most helpful thing currently about this class?

After I read the comments, some of them were very useful and mostly minor for me to implement (e.g., more vocabulary at the beginning of the unit, more explanation before activities).  Many of the students admitted that they had expected me to do the heavy lifting of understanding for them and then beaming the information into their brains so that the biggest help for them would be to take responsibility and employ the study skills they know they should do.

For the suggestions that I certainly wasn't going to implement, even though they might be reasonable in other classes, I pulled the syllabus back out, pointed to the class goals, and supported my methods with references from research on effective educational methods.  Yes, the hands-on, minds-on work is hard for many students and many of them would prefer to memorize a lecture, but the science education literature is very clear that those activities are the best ones to use to promote learning for introductory classes.  Most of the students accepted that.

For the suggestions that I just didn't want to implement, I picked the two that had the most student support and explained that I was not going to do those things because I am in charge of the class and when they are in charge of the class, then they will see why those things like allowing assignment to be turned up to two weeks late just because and having all of the grades available through Blackboard by the morning after the assignments are handed in are murder on the instructor and defeat some of the pedagogical purposes.  When asked directly by some students why I didn't include their comment, I said with a clear conscience that I'm focusing on the big picture painted by the majority of the students.

Now, I did get dinged on the end of the semester evaluations by a few students for not implementing some of the suggestions that were pedagogically unsound for my purposes or just too much trouble for me to do considering my 18 hours a week in the classroom.  However, most of the students who commented on that aspect commended me for my responsiveness to their concerns and making the mid-course modifications to the class to better accommodate the students I had who were very different from the students that I had expected to have when I was prepping the course.

[conjugate]

What angers me about these standards of evaluations is the poorly-defined terms in the rankings.  I mean, on a one-to-five scale, one school I taught at asked students to rate instructors as 1 (far below average), 2 (below average), 3 (average), 4 (above average), or 5 (excellent) as compared to other instructors at the same institution.  We too were expected to get 4.4 or so. 

This school wasn't even Lake Woebegone U.

Student evaluations are ideally only one piece of the analysis of instructional quality, and relying on them wholly is (in my view) an invitation to pandering.  Sure, you can get great evals by reassuring students that if they show up every day or provide an excuse, you'll guarantee them at least a C in the course (I was told this by a colleague at my former school who used this technique, and sadly followed through with it).  But what job is it that you are supposed to be doing?  Surely not this.

OP, you might also look at the "Jedi Mind Tricks" thread for other approaches to this problem.  I wish you the best of luck.  I haven't seen my evaluations from Fall at my new school yet, myself.

[boethius]

One thing that annoys me about evaluations like this is that two sections of the same class can have have wildly different averages.  Same material.  Same assignments.  Same professor.  One friend of mine had a large lecture that multiple discussion sections attended.  His numbers were very different for the different sections, even though everyone was attending the same lectures. 

Another problem: the number of respondents (or class size).  If the numbers are small enough, then one malcontent throws off the averages.  You don't need a PhD in statistics to understand that, but some administrators think the class mean is whoever is nastiest to the prof.

[msparticularity]

Another problem: the number of respondents (or class size).  If the numbers are small enough, then one malcontent throws off the averages.  You don't need a PhD in statistics to understand that, but some administrators think the class mean is whoever is nastiest to the prof.

Yup--this is what gets me every time. I teach a master's-level core course that usually has around 15 students per semester. As I've mentioned before here, it's the only writing-intensive class in the program, too. I've just looked at last semester's evals, and once again I'm hovering just under a 4 on several items because 13 of 15 liked or loved me and everything I did (4 or 5), one found me and the course fairly mediocre, and another was cranky about everything. My scores make perfect sense, viewed in terms of whether the students think this course ought to be required for them, but of course that's not always taken into consideration by the Power That Be.

And, sadly, this is a course where I can't use YouTube (The Neosporin of classroom relationships, to paraphrase John Lithgow's character on "Third Rock") since I can't get a technology-equipped classroom.