next »

[abitatd]

We are bring in candidates next month and the candiates have been asked to teach a lecture in my class. The department chair wants the students' opinions, but he won't have the ability to stay and ask them for feedback (and 90% of them have a class immediately after). It was thought that putting an evaluation of some type up on WebCT/Blackboard might solve our evaluation issues, but the problem is - what do we (I) put up? The any other comments or observations question is easy, but the chair would like something a bit more quantitative/something the students might actually respond to? Any thoughts on questions that might be good to include - keep in mind this is a one shot deal for the candidates and students, so it's not like they can evaluate beyond what is presented in the class.

This is the first time I've had this happen in my class, so any thoughts or assistance is appreciated.

[msparticularity]

Speaking from the other side of the equation--last year when I was on the market and guest-taught classes at a couple of schools, the students filled out written evaluations. I have no idea what was on the forms, of course, but I saw them writing periodically, and also saw them turn in the forms on the way out the door.

I would think this would be the most effective tactic, since unless there is course credit they might or might not respond promptly on Blackboard.

[unstuck]

How about these (with some 1-5 or so system of agree strongly to disagree strongly):

    The instructor was well-prepared
    The instructor used visual aids well
    The instructor showed enthusiasm for the course
    The explanations and examples were clear and helped my understanding
     I would like to take a course from this instructor

   etc.

[kedves]

Speaking from the other side of the equation--last year when I was on the market and guest-taught classes at a couple of schools, the students filled out written evaluations. I have no idea what was on the forms, of course, but I saw them writing periodically, and also saw them turn in the forms on the way out the door.

I would think this would be the most effective tactic, since unless there is course credit they might or might not respond promptly on Blackboard.

I strongly agree.  Even the best-intentioned students will forget to complete the survey on Blackboard.  If it's quick and they can complete it on the spot, you will get responses.

[prof_tournesol]

Can I ask for the question about "used visual aids well" to be removed, unless it's a job in art history?

I have never, ever, ever used visual aids in a teaching demo because the odds of your PPT not working with the host school is so great (computers and video projectors can smell stress), that it's not worth the risk.

Otherwise I would do what unstuck suggests, but stick with things like enthusiasm, clarity etc

[melaw]

When I did job talks and teaching demos in front of students, the students usually had a written evaluation sheet that was passed out before I began and turned in as they left the classroom.  The evaluations that I saw (& yes, I did take a peek at them before beginning!) only had open ended questions, but there is no reason why you couldn't include both types.  

The questions were very similar to this poster's suggestions with the addition of asking about organization.

How about these (with some 1-5 or so system of agree strongly to disagree strongly):

    The instructor was well-prepared
    The instructor used visual aids well
    The instructor showed enthusiasm for the course
    The explanations and examples were clear and helped my understanding
     I would like to take a course from this instructor

While I see no problem of including quantitative questions, I think you also need to include some qualitative questions.  Students attending might not really have any basis of comparison--after all, how many job talks have they heard before? If all the job candidates were speaking in front of the same class the numerical evaluations might be more valid, but in my experience this rarely happens.

Good luck with your search!

[melaw]

Can I ask for the question about "used visual aids well" to be removed, unless it's a job in art history?

I have never, ever, ever used visual aids in a teaching demo because the odds of your PPT not working with the host school is so great (computers and video projectors can smell stress), that it's not worth the risk.

Otherwise I would do what unstuck suggests, but stick with things like enthusiasm, clarity etc

I think visual aids are important to consider (self-disclosure--I tend to use Powerpoint).  In all my talks I passed out a handout that included an outline of what I would be talking about and important concepts that undergraduates may not be familiar with.  I also used a markerboard at one place.  While I agree that candidates should not have to use high-tech equipment (you're right about them smelling fear!), this does not mean that other sorts of visual aids aren't appropriate.

Maybe say, "Use of visual aids, if applicable."

[present_mirth]

Speaking from the other side of the equation--last year when I was on the market and guest-taught classes at a couple of schools, the students filled out written evaluations. I have no idea what was on the forms, of course, but I saw them writing periodically, and also saw them turn in the forms on the way out the door.

I've interviewed at a couple of places that had students fill out pen-and-paper evaluations and turn them in immediately, and I have to admit that it felt very awkward.  From the candidate's perspective, you don't want to seem like you're trying to sneak a peek -- or even like you're paying any attention to the fact that students are evaluating you at all -- but when you're in the middle of this flurry of people filling out papers and passing them to the search committee, it's almost impossible to ignore or look elsewhere.  At one school, the search chair then had to escort me to a meeting with an administrator who turned out to be late, so for around ten minutes, I had to stand around in the corridor making small talk with him while he was holding a stack of evaluations with some of the numbers showing.  Very awkward!

I like the Blackboard idea; that sounds much more discreet and more comfortable for the candidate.

On the other hand, I did get a job offer from both schools, and the one where the search chair stood around chatting with an armful of evals is the one where I currently work, so I can't complain too much!

[thisisme]

While it certainly is important to make it easy for students to visualize what is explained, even in areas where visual ads are common, some care need to be taken in how to word that question.

A good and lively explanation using the board often is welcomed by students more than a power point presentation. I have seen multiple evidence for that. "The instructor used visual aids well" will only sufficiently distinguish between the good and the bad ppt talk. But an (assumed) great instructor who was the best but did not use ppt, may just get a "no". Despite, not every student may be certain what is included in "visual ads" and what not.

So, I'd rather ask whether it was easy/difficult to follow, whether it was interesting/boring, as whether or not this happens based on visual aids or not really does not matter.

[thisisme]

Another thought: Is there/are there (an) additional student(s) in the SC that will attend the classes and can comment on them later?

There is also the (possibly additional) possibility to take few students who attend every candidate's talk and ask them for their impression afterwards. That way, issues in question can be clarified and the origin of their judgement becomes more evident. I have seen it from both sides (SC member and applicant) and thought it is effective.

For me, if there is no other reasons against rather than "no time", this really does not seem like a good  reason. They could be allowed to skip the class after, asked to return later (they will, as they like to have influence) or likewise.

[imawakenow]

Just a caution about having students go to WebCT later to evaluat candidates--if your participation rates are anything like online course evaluations, you will likely have a very limited sample.

My own anecdote: I teach adjunct at one school that went to online course evaluations last year and the participation rate for my evaluations has dropped to around 20%. I guess you could make the argument that students would be motivated to evaluate the candidates out of self-interest, but my guess is that unless you provided a strong incentive that few would do it.

I've been to two campus visits and both places had me end the class early so students could complete a short survey. I agree it is a little uncomfortable, but not any more uncomfortable than any other aspect of an interview.

Anyway, if I were designing it, I would use unstuck's questions and then include space for other comments.

[grendels_mother]

I was on a search committee and the primary instrutor of a class that two visiting candidates guest-taught.   We had students write down their feedback during the first few minutes of the next class (after both candidates had left-- they taught consecutive classes).  The evaluation was completely open-ended, and we got some very good feedback.

[daniel_von_flanagan]

Give every student a ticket, and put two boxes on the wall or floor outside the room.  One is for "yes", one for "no".  There is one question:

"Based on today's sample, would you sign up for a class with this instructor?  If you have no opinion, there no need to vote."

On a linear numerical scale, this is really the only thing that matters, and pretty much the only thing that the students are absolutely qualified to judge. You can also invite them to email further remarks at a later time if they are so inclined. - DvF

[abitatd]

Thanks for the ideas - I think Unstuck came up with the perfect questions to ask. I will be adding two open-ended questions, one for further comment, and one to quiz the students on the presenter as I will need to put the evaluation up on Blackboard/WebCT because of the class conflicts.

Additionally, the candidates will be teaching course content (it's a science course and this is material that they should be able to teach without an issue), that the students will need to know, so I want them to be able to take notes and then reflect about the material and not trying to focus on the evaluation instead of the class or the other way around.

As far as response rates, the students are given an online quiz every week and I am replacing the online quizzes they take the weeks the candidates are coming in for the evaluations. I'm also giving 2.5 points per eval in bonus points towards their quiz grade to encourage feedback. Hence, the quiz question on the presenter/presentation to double-check their attendance.

I appreciate all the ideas - they have been so helpful.

[dogvomit]

Did you like the candidate. 1-5
Would you take a course from this candidate. 1-5

Thats really all you want to know from them anyway.