Some interesting findings and unanswered questions about clicker implementations

January 4, 2012, 8:00 am

I have been using clickers in my classes for three years now, and for me, there’s no going back. The “agile teaching” model that clickers enable suits my teaching style very well and helps my students learn. But I have to say that until reading this Educause article on the flight out to Boston on Sunday, I hadn’t given much thought to how the clicker implementation model chosen by the institution might affect how my students learn.

Different institutions implement clickers differently, of course. The article studies three different implementation models: the students-pay-without-incentive (SPWOI) approach, where students buy the clickers for class but the class has no graded component for clicker use; the the students-pay-with-incentive (SPWI) approach, where students purchase clickers and there’s some grade incentive in class for using them (usually participation credit, but this can vary too); and the institution-pays-clicker-kit (IPCK) approach, where the institution purchases a box of clickers (a “clicker kit”) for an instructor, and the instructor brings them to class.

For me, the most interesting finding in the study was that there appears to be a threshhold for the perceived usefulness of clickers among students. The study found that in the SPWOI approach, 72% of student respondents said they would buy a clicker if it was used in at least three courses they were taking per semester. But drop that number to “at least two courses” and the percentage drops to 24%! So once the saturation level of clicker use reaches something like 50–75% of a student’s course load, they start seeing the devices as worth the money, even with no grade attached to its use. (Only a depressing 13% of students said they would pay $50 for a clicker based solely on its value as a learning tool. We have some P.R. to do, it seems.)

In the SPWI approach, 65% of respondents said they would buy a clicker if the contribution of clicker use toward their course grades was between 3% and 5%. (This is sort of mystifying. What do the other 35% do? Steal one? Just forfeit that portion of their grade?) The study doesn’t say explicitly, but it implies that if the grade contribution is less than 3%, the percentage would drop — how precipitously, we don’t know.

The study goes on to give a decision tree to help institutions figure out which implementation model to choose. Interestingly, if it gets down to choosing between the SPWI and SPWOI models, the deciding factor is whether the institution can manage cheating with the clickers. If so, then go with SPWI. Otherwise, go SPWOI — that is, if you can’t control cheating, don’t offer incentives.

Here at GVSU, I use the SPWI approach. Students have to pay for the clickers, but they get 5% of their course grade for participation. I take attendance at each class using the Attendance app for the iPhone. Then, once or twice a week, I’ll cross-check the attendance records with the clicker records for the day. If a student is present but doesn’t respond to all the clicker questions, they lose participation credit for the day. This method also mitigates cheating; if a student is absent for the day but has records of clicker response, then I hold the student guilty of cheating, because someone else is entering data for them. (Putting the burden on the absent student makes it less likely they’ll give their clicker to someone else to cheat for them.)

Absent from this study is the actual instructional method used with the clickers. The authors point out, correctly, that such things are difficult to control for in a study, but still — clicker technology is basically neutral, and its only function is to enable active pedagogies. The choice of pedagogy matters. If you only use them to take attendance, students are going to have a lower perception of their usefulness than if you do something interesting like peer instruction with them.

What about you? Are you using classroom response systems in any kind of way, and if so, what method of implementation are you using, and how is that going?

Image: http://www.flickr.com/photos/alumroot/

This entry was posted in Clickers, Educational technology, Peer instruction, Teaching, Technology and tagged Agile teaching, Classroom response systems, Clickers, ed tech, Educational technology, Educause. Bookmark the permalink.

E-mail

ddrake_

Why is anyone buying clickers, when 100-minus-epsilon percent of your students are already carrying a cell phone? There are systems like Poll Everywhere and the University of Minnesota’s ChimeIn software (https://chimein.cla.umn.edu/), which also works in a browser. You can bet your students *always* have their cell phones, so just use the technology they already have.

Oh, and your students already send tons of texts anyway, so the marginal cost to send a few more a week is effectively zero.

Okay, so I’ve never used clickers, so maybe I’m missing something. The smartphone in my pocket seems like it’s millions of times more powerful than a clicker. Am I wrong?

http://chronicle.com/blognetwork/castingoutnines Robert Talbert

I think the model is moving toward software rather than hardware. As to why it’s not *already* the model, that’s an institution-by-institution question. Some have been using clickers for several years and us the IPCK implementation model, and it would be more expensive to move away from clickers than to stick with them. Some of the software is pricey. There could be other reasons. But just because lots of students have cell phones in itself isn’t a sufficient reason for a switch, apparently.
http://twitter.com/jaxbartram Jacqueline Bartram

This is just a cost issue, poll anywhere and responseware are currently too expensive for us to consider for large groups.
electronicmuse

Yes, you are missing something. You’re overlooking the fact, that once students’ use of cellphones in class is countenanced, they will be doing all kinds of things with them OTHER than paying attention in class.

There is still a reason to use clickers–immediate feedback, without the attendant problems that other eMedia bring into the classroom.
kgodwin

Not all students have smart phones. Yeah, most of them have cells – but what do you do for the students who don’t have smart phones or even cells? Tell them they can’t take your class, even if you’re the only one teaching it this year? That seems unfair…
dvakil

There is a significant issue requiring students to use their cellphones as clickers. If using a wi-fi transmission, the classroom may not support the network usage demands required by the number of clickers/phones accessing the network simultaneously. (Simultaneous access is very important.)

If using a cellphone network for information transit, you have to be sure all classrooms get good reception for all majors carriers throughout the duration of the class. Schools have very little, if any, control over this.

Most schools can’t guarantee the hardware and network access is sufficient. These problems may disappear in 5 years, but they’re present today.

http://derekbruff.com Derek Bruff

I was going to point out these issues, but @chronicle-ec345a52da71eed7ff2623f72b7498bc:disqus beat me to it. For more thoughts on roadblocks in the transition from clicker-based to phone-based classroom response systems, check out my recent interview on the Chronicle’s Tech Therapy podcast: http://chronicle.com/blogs/techtherapy/2011/12/07/episode-90-growing-pains-for-%E2%80%98clickers%E2%80%99/.

The worry expressed by @chronicle-246f947646b15a69ca268cac574c1dc7:disqus above about students using their mobile devices for off-topic purposes is a very, very common one, and it’s probably the biggest roadblock to the adoption of phone-based systems. However, from talking with faculty who go that route, they usually find that students use their mobile devices for self-distraction *less*. That is, if you give students something engaging to do with their devices that’s on-topic, they’re not as inclined to use those devices to go off-topic.

http://twitter.com/polarisdotca Peter Newbury

ddrake_: Personally, I’m all in favour of finding ways to take advantage of the smartphone almost every student has in his or her hand. For the vast majority of instructors I know, though, cell phones are a Pandora’s box. Many have convinced themselves that smartphones and laptops mean students will do nothing but text and facebook during class. Doug Duncan (U Colorado) is about to publish a study correlating in-class cellphone use with a drop in grades although these are not classes where students are explicitly asked to use their smartphones for learning.

Also, here in Canada, we can’t use iclicker’s webclicker or smartphone clicker because it requires transmitting personal information across the border to US servers, a violation of our protection of privacy laws. I’m happy to say, though, that iclicker is working on setting up servers in Canada so we’ll be able to look into using smartphones and webclickers.

The upshot of all this: I think using smartphones as clickers, as search tools, as calculators, for simulations, for peer interaction, for twitter backchannel,… is a fantastic use of technology — when we figure out the right way to do it. Until then (which I don’t think will be too long from now), the instructors I know will continue to require students to buy clickers.

ddrake_

So your short answer is: because clickers can’t be used to surf Facebook. That’s an excellent answer!

I do hope the U of M open-sources their ChimeIn software, so that you can install it locally, or have a Canadian company install it so that you can legally use the service.
http://twitter.com/TRFletcher Prof Rick Fletcher

I’m so happy to hear there is a study underway that looks at cells in class. Last semester I actively encouraged cell use – backchannel questions, clicker type use and unlimited texting to me for help. In the first half of the course student performance dropped so drastically and complaints about distraction were up so high and felt I had to discontinue the practice. Performance came back up – and my averages are based on thousands of students over many years – no question this performance was different – two std devns away from my mean. I look forward to Duncan’s work.

fizmath

Do you have a plan B for days when the projector or computers are not working?

How much time is spent preparing a Power Point presentation? I knew one teacher who would spend 5 hours to prepare just one lecture. Are you using most pre-packaged clicker questions provided by the textbook publisher?

http://chronicle.com/blognetwork/castingoutnines Robert Talbert

Before I used clickers, I gave out four 3×5 notecards to each student and had them put “A”, “B”, “C”, and “D” on them. Then the students just hold up the appropriate notecard for the question. I’ve seen other teachers use colored cards or make little 5×7″ whiteboards for their students. Any classroom voting scheme can be done without any kind of electronic technology at all. The electronics just make the data collection and storage easier.

I usually make up my own questions, and I don’t find it takes any more time than a straight lecture. Usually 30-60 minutes of prep for a 50-minute class for me. When I get stumped on finding a question, I usually head over to MathQuest: http://mathquest.carroll.edu/ (Disclaimer: I am semi-involved with contributing questions and data to the MathQuest project right now.)

KMHahn

I use a technology called “raising your hand.” I find it an elegant solution for my needs.

http://chronicle.com/blognetwork/castingoutnines Robert Talbert

Unfortunately the instructor’s needs are not the point of the class, and in any class there’s a large number of students who, for whatever reason, don’t volunteer answers if it singles them out. Rather than have the most vocal students dominate classroom discussions, I think it’s worthwhile to think about any technology or technique that gets all students involved.

KMHahn

Robert, faculty needs are not the point, have you been to a faculty senate meeting? I say this is jest. I agree that the students do come first and I hope you can extend to me the assumption that “my needs” are consistent with what helps the students learn. For me this means encouraging students to have a public voice in the classroom. Education is a social creation and part of the power is created in dialogue and defense of ideas. I fear that technology is used as an answer that undermines the point. The point is not to create the most educated society with no voice. The point is to have an educated populace with a voice and the ability to act. I am a recovering shy person, I know how difficult it can be to speak up, but I also know how great it is to get beyond this fear and have a voice.

kphagen

Another reason raising your hand is not as good a choice as using clickers is that less engaged students can just wait to see what the majority of the class answers and go along with them instead of (with the clickers) having to make an intellectual stand of their own.
http://derekbruff.com Derek Bruff

I wholeheartedly agree with @rtalbert:disqus’s response to this comment about hand-raising, and I’ll add there are a lot of reasons students don’t speak up in class. Some are worried about looking dumb in front of their peers, others wait to see what their peers say before thinking too hard about a question themselves, and some very bright students don’t want to “hog” the conversation space in the classroom. It’s important to find ways to find out what *all* one’s students are thinking during class.

velvis

I use the IPCK and bring my tote bag to class twice a week. I use these not just for attendance and random quizzing/participation but also for voting on debates, taking exams, rating writing along rubrics etc. The up side the version of clicker we have is that they can be used with any program as the “box” just takes a screen shot.

My students like them, incorporate the use in their own presentations and many have wanted to purchase their own but since I’m one of two professors on campus that uses them on a regular (daily) basis. There is no point to adding to the amount of money they have to spend each semester. I have chosen NOT to use their phones as I have control over this there’s no additional information on my clicker other than their assigned number, there’s none of the “the app isn’t working,” “I left my phone at home,” “I down loaded the wrong app.” And shockingly enough not every student of mine can afford a smart phone, I didn’t have one until 3 months ago. The questions don’t have to be more than what you want them to be. But honestly since I have a smart board and ONLY a smart board in my classroom if it goes out — looks like they just have to figure out where I was going once they get home.
http://twitter.com/jaxbartram Jacqueline Bartram

We use IPCK but there is a lot of panic about ensuring we get them all back each time and this can delay students leaving and cause aggravation. I know other institutions loan them from the library like a book putting the onus on the student and the same penalties for non-return that books have.

http://chronicle.com/blognetwork/castingoutnines Robert Talbert

My university has that model in place, but it’s only for first-time clicker users. I’d prefer to go with this model EVERY time and may start lobbying to do so.

drjennycrisp

I have a classroom set (IPCK) even though I have no particular objection to students’ own technology. But the idea that all students, or nearly all students, own smart phones or laptops may hold true for some schools, but certainly not all. Our demographic does not guarantee that at all; in any given class, I can still be pretty sure that several do not even have a computer at home. So, I use an institutional set of clickers, and, times being what they are, I write small grants to purchase others for the department.

The worry over getting the clickers back can be simplified by numbering them and assigning each student a number. I make them responsible for picking up their own clicker when they enter and returning it when they leave.

Editing to add that I really like the idea of the library model!
wassall

So why can’t a clicker-like device be embedded into every desk/table surface in the classroom?

http://chronicle.com/blognetwork/castingoutnines Robert Talbert

That’s another model that the article brings up, but I didn’t highlight it in my post. It has some advantages. The biggest disadvantage, which seems like a deal-killer to me, is that when the clicker technology changes, you would have to go in and physically remove the devices from the furniture. Expensive unless you can pull off the physical installation right.

http://derekbruff.com Derek Bruff

Desk installations also mean that only instructors who teach in those classrooms can use the response devices. That may be fine if you’re mostly interested in supporting response systems in large classes (since there are only so many large classrooms on any campus to equip), but response systems work just as well in small classes (any class with more than 12 students or so). I’d hate to see small-class instructors left out.

When I taught at Harvard back in 2004, the clickers were portable, but the receiver units were not. The big science center classrooms had receiver units hardwired in, but for those of us teaching smaller classes (30-student calculus courses, in my case), we had to borrow something we called “the Beast.” It was a desktop computer on a computer cart with a six-foot-long 2×4 nailed to the side. The receiver unit was attached to the top of the 2×4. We were using an infrared system then, so the students needed line-of-sight to the receiver unit, thus its height.

Watching a math prof wheel the Beast through the halls and elevators between classes was comical.

proftucker

When I’ve used clickers, it has usually been in a “SPWI” mode with a low value for clicker marks (~5%). I don’t use them for attendance only, but offer part marks for answering a question and additional part marks for getting it right. I have also used them in a “SPWOI” setting where the students already had the clickers for another “core” class, I just leveraged the devices again in my class. Even though it wasn’t worth any marks, I think the students appreciated that they could get more use out of their device.

This past term I used Tophat Monocle’s “MonocleCAT” system. Questions are pushed out to student’s devices (cell phones, iPods, iPads, laptops…) and they can respond via SMS text message or through the web, iPhone app, etc. The upside as ddrake_ noted is that (nearly) all the students have a device and it is permanently affixed to their hand/hip pocket. They could also use multiple devices—phone if they’re traveling light, laptop if they’ve got it with them. The downside was, because MonocleCAT “pushes” the questions out, students could be sitting in their dorm room or other non-classroom location and still “participate”.
scharlson

For those using MOODLE as their LMS, it has a response system function called Choice. With one click the instructor can add a question and display the results to students in real-time either as Anonymous (graphed results) or the module will actually display the users and how they responded to the question. All the student needs is a web enabled device. Cost= $0 and it’s already an integrated part of the LMS.
http://twitter.com/TRFletcher Prof Rick Fletcher

I’ve used smartphones as clickers for two semesters. First – a summer class, I used polleverywhere. The class size was small, 21 students and while it was a bit klunky to get the answers in a displayable format, it worked. They told me it won’t work for large classes. In a regular semester my class size is closer to 250 and I tried using a different free service – I won’t say their name but i used them because they promised me things would work with this class size – I was in pretty close contact with them – and it failed miserably. I can’t tell you how silly it feels to ask the class to answer questions the 5th, 6th, 10th day in class, promising that the problems have been fixed, to have it fail every time. I won’t say their name because I believe they are trying and perhaps in the last few months, they have solved the problems.

But the experiences have left me suspicious. It’s risky to use free services when there is more than yourself on the line. Teaching my science class is already high risk enough for some of my students, I don’t need to give them reasons to fail more. I am using a paid clicker service this semester and I think things will go much better.
jazrea

I took some clicker training and think they are definitely
worth the effort. The problem at my university is that there are very few sets
of clickers available for classroom use; they must be requested and checked
out. The bigger problem is that the clicker sets have become the monopoly
property of the professors who began using them first, so they basically are
not available to most instructors. Requiring students to make a one-time
purchase would solve the problem.
brianborchers

My institution has an interesting variation on the SP approaches- the institution bought clickers but rents them out to students for a couple of bucks per semester. This is less expensive for the students then buying the clickers and the rental fees are sufficient to pay for replacements when the clicks are lost, stolen, or broken.
Edtechnerd

I am the Director of Educational Technology at a largr PK-12 private school in Fort Lauderdale. We are currently piloting a new app that will truly change your perception of how clickers work. I wanted to share the app information with all of you because I feel like it may be an answer to what many of you are looking for, along with unbelievable features. The website is http://www.nearpod.com. Feel free to message me on twitter if you have any questions or need any other information. I can’t wait for this to come onto the market for others to experience, but you can sign up for the beta testing process and the developers will come right to you to help set things up. It’s been an amazing experience getting to be piloting a groundbreaking app, such as this.