An E-Mail Experiment Helps a Duke Economist Ponder His Students’ Cheating Hearts

August 10, 2011, 6:08 pm

As the fall semester approaches, a word of advice to students: If you’re absolutely determined to cheat, do it in a course taught by a professor who’s obsessed with cheating. Your behavior will depress him, but he’ll at least have the consolation of a deeper understanding of the problem.

Last spring, Dan Ariely, a professor of behavioral economics at Duke University, and his teaching assistants found evidence of cheating in the weekly quizzes in his 500-student lecture course.

“People who were sitting together had similar answers,” Mr. Ariely said in an e-mail message to The Chronicle. “The class was very large, and it was hard to space people more, which I would have done if I had space.”

What happened next was described in a post on Mr. Ariely’s blog on Wednesday. After the class discussed cheating and honor codes—topics with which Mr. Ariely has been concerned for many years—two students in the course decided to run a little experiment. (Their project was an outgrowth of one of the class assignments.)

They sent all 498 of their classmates e-mail messages from a fictitious student explaining how to download Mr. Ariely’s final-exam questions and answers from the previous year. Half the messages included this postscript: “P.S. I don’t know if this is cheating or not, but here’s a section of the university’s honor code that might be pertinent. Use your own judgment: ‘Obtaining documents that grant an unfair advantage to an individual is not allowed.’”

Among students who were reminded of the honor code, 41 percent clicked on the link (which did not actually contain any test material). But among students who were not reminded of the honor code, a much higher proportion—69 percent—clicked on the link.

The day after the actual final exam, Mr. Ariely conducted an anonymous survey, asking students whether they had cheated, and also for their estimates of how many of their fellow students had done so.

Very few students admitted to cheating—and Mr. Ariely believes their self-reports are basically accurate. If there had been much cheating on the final, he writes, the grades would have been better. (The average grade was 70.)

But Mr. Ariely does not take much comfort from that, because the students estimated that 30 to 45 percent of their fellow students had cheated. Those estimates are probably much higher than the reality, but Mr. Ariely argues that “such an overestimation of the real amount of cheating can become an incredibly damaging social norm. … If the perception of cheating is that it runs rampant, what are the chances that next year’s students will not adopt even more lenient moral standards and live up to the perception of cheating among their peers?”

Mr. Ariely talked about cheating two weeks ago on public radio, and his work was also profiled in The Chronicle back in June.

This entry was posted in Research, Student Life. Bookmark the permalink.

E-mail

http://twitter.com/curtyowell Curt Yowell

I’m not sure if clicking on a link constitutes cheating, might it simply be curiosity on the part of some of the students. I would be interested to see a similar study carried out where the link went to actual test material.
http://twitter.com/curtyowell Curt Yowell

or should I say, not sure if clicking on the link constitutes intent to cheat
arrive2__net

The professor’s blog shows the actual email messages, it does seem clear enough that using the download would be cheating. I wonder if anybody mentioned or reported the email to the professor. I wonder if the students knew the email was sent to all students in the class and what percent of students actually opened and read the email.

Including a quote from the honor code could have reduced the percentage of downloads by prompting, or the student could have interpreted the quote from the honor code as an added indication that downloading would be cheating. The students might have thought it was some kind of cheating sting set-up, or even that it was an experiment.

I think Curt Yowell’s comment is right that the student’s would naturally be curious…they would have to wonder if the download offer was for real, and what the other students in the class were being offered.

The perception of the prevalence of cheating could have been influenced by the email. If the email was considered an invitation to cheating, and every student received it, that would be likely to influence the perception of cheating in the class. Did students contact each other with the news that this email had arrived, and did that influence the prevalence of opening the email and clicking on the link?

Bart Schuster
Arrive2.net
Twitter.com/arrive2_net
sfoderick

I would be interested to know if this went by the school’s IRB? Using students (without their consent) for research would seem as unethical as anything that the students themselves were involved in.
https://plus.google.com/115213066046539868050?hl=en anonymous

Give me a break. Clearly a lot was learned from this experiment and there was no harm to students.
mbelvadi

It is the sad reality in today’s social science research climate that sfoderick has a point – most IRBs would probably consider that such a study should have been approved by them first. Maybe it was? Given what Dan Ariely teaches, and the mention of a course assignment, maybe he did make sure all student projects did have their IRB’s approval.
williamalpert

Let’s begin with the code statement, ” ‘Obtaining documents that grant an unfair advantage to an individual is not allowed.’” Aside from the fact that it’s already open to interpretation that one document (singular) is permitted; it’s rather absurd to prohibit a student from obtaining academic articles that might be footnoted or referenced in the text that allows them to provide a superior answer to other students who did not take such inititive thus giving the curious student “an unfair advantage” over a less curious one. I am certain that is not the intent of the Code, but that is, in fact, a reasonable interpretation.

Second, so far as I can tell the experiment demonstrates almost nothing about either cheating or behavior. In fact, perhaps there was a great deal of cheating (real and/or experimental) and the grades without the cheating might well have averaged much less than 70. The Academy clearly has cheating problems (as does the rest of society), but this article sheds little useful light on the topic. I wish it did and perhaps the “Chronicle” do us all a service by printing a series of useful articles soon. That series might begin with an examination of honor codes and examine their utility and claity of meaning.
gellci

Well, what is cheating anyways? If CONTENT remains the main goal in many courses, versus HOW TO, why are we worried on how one learns the content? By the way, does 500 students in a class sound…not cheating?
archaeopteryx

As a student who has actually found previous final exams lying around a student lounge (which had been left there by the professor for students to pick up at the end of finals), I agree with Curt: a lot of students probably clicked the link out of curiosity, but would not have studied the previous exams. It would be interesting to put actual tests (or what looks like test material, anyway) and then measure how long the students who clicked the link looked at the webpage, and/or whether they downloaded the tests.
panacea

Here’s a question that doesn’t seem to have been asked.

Did any of these students contact the professor and alert him to a potential cheating situation?
bghansel

In a class of 500 students, does any individual student believe his or her email will be read?
austinbarry

If something is sent to all students, does making use of it constitute an unfair advantage? The one flaw in this experiment is that students talk to one another, and since it was sent to all students the probability of a a trusted friend having received the email is 1 in 1.
translog

Ii not just a link that gives the user an unfair advantage but also the emotional needs of either perties to academic community. Look at this experiment conducted by one of my associates in authenic leadership program

It was about screening medical student applicants for EQ competence as a part of their ticket to admission. They want medical students to be able to understand how someone else thinks and feels and then to express oneself with the others’ perspective in mind. Can you imagine the impact this will have on patients?
davi2665

I had a similar cheating problem in one course with 180 students. The solution was simple- I permuted the questions into several exam forms, and told no one. We had some scores that were actually lower than random probability. Those students received an F on the exam and an F in the course. This stopped the cheating very quickly. Some standardized exams have gone to random selection of questions from a huge bank of possible questions, presented in random order, so that no two students have the same examination, but all have questions of similar difficulty. It works.
not4nothin

Oh! Give! Me! A! Break! Everyone knows the role of the IRB is to determine if research using human subjects sufficiently protects the subjects from harm. It is not up to the researchers to determine. And it’s not about how much is learned.
dominiqueh

Downloading previous exam questions and answers is not automatically cheating. There are schools which actually sell previous years’ exams back to students and professors will sometimes provide them as a useful study guide to what types of questions to expect. If access is readily available and not against the professor’s own stated policies, studying previous exams constitutes good study habits — not cheating.
This article from Inside Higher Ed highlights some of the issues. http://www.insidehighered.com/news/2008/06/10/postyourtest
http://www.facebook.com/profile.php?id=1487003047 Darcie Callahan

I totally agree about the curiosity factor. I’m a very curious person and when a student, can imagine I might have clicked the link just to see what was there, wondering if it would be the exam or some kind of joke. I also would have been suspicious that my “helpful” classmate might be sending out a bogus exam in order to ruin the grades of the rest of the class — and being a good student, I would have been able to tell if that was the case.

The experiment could be repeated with a link to a previous exam that includes a ”red herring” question. Students who then get that question “correct” would be known to be cheaters.
http://www.facebook.com/people/Duane-Fitzhugh/1325605833 Duane Fitzhugh

At the risk of sounding like a goody-two-shoes, I would never have clicked on such a link (partially because anytime I do anything unethical, illegal, or immoral I get caught). That said, I can understand the curiosity factor. I am VERY bad at math, and would have been sorely tempted to see if anything there could help me. Nevertheless, my conscious would most likely have prohibited me from actually doing it.
11122741

research one does to evaluate and improve one’s classes and instruction do not have to go to IRB for approval and do not come under their purview and the same is true of discussions of such work in the educational channels such as this article, a conference paper and similar professional discussions. These things are exempt for several obvious reasons including they would swamp and shut down IRB except for commenters such as sdoderick and her/his 8 supporters who probably are doing little research to improve their instruction other than tryiong to one up someone who is. Grow up please.
katisumas

I know, I know, it’s not feasable in this day and age, but remember that this sort of cheating would not be possible if students were assigned essay exams…..

Isn’t it some variety of “cheating” for students who have a photographic memory to memorize a whole bunch of material in twitter form of phrases and then immediately forget it after the test? Have you ever wondered how your top graded students would do if tested again a year (or more) after taking the course without any further preparation?

And better, what if you had control groups of students who had taken objective tests and students who had taken essay tests and then retested them again at some future point?

Or even better what if you compared students who tested A on the objective test with students who were given a take home essay test a year later? Of course some time would be requird to devise essay questions that are unique to your individual course so the responses would be unlikely to be found online, actually, come to think of it, faculty used to have to devise objective questions based on courses that they did not repeat verbatim from one term to another…

Your “bank of possible questions” assigned randomly by a computer should worry you because it will sooner or later make your job obsolete Even objective test questions are supposed to be specific to the way you teach a course, otherwise why not put the course itself on line and join the unemployed lining up at that food bank?

(I don’t have to worry about my job because I’m retired but I sure worry about the ongoing dumbing down of the US as fewer and fewer people seem able to write coherent sentences and actually absorb knowledge, not to mention critically deal with the information we are bombarded with. I’m digressing, but it seems that we might be getting as ignorant (or more!) of the world around as thirteenth century European folks were. We swallow any wild tale just as easily (or more) than they did… the only difference is in the quantity of wild tales being spun….)
katisumas

You’re so right! And it would require profs to rethink exam questions and perhaps rethink at least part of the same course they might be teaching over and over again. This would keep their course fresh and interesting.

The selling part makes me uneasy though…
katisumas

I would have clicked on the link even though I’ve never cheated when a student. As Dominiqueh noted, in many courses, past exams questions will be provided to students to help them study and absorb the material. I used to provide past exam questions to my students myself. I would expect any intelligent student to look up the link for the same purpose.
katisumas

I would summarize the comments that I made in response to others: It’s better to be obsessed with teaching than with cheating.
activelylearningtolearn

Each university’s IRB is different (the board after all is just a conglomerate of people with different, often conflicting opinions), but many IRB’s do not require approval for studies in which participating students do not experience a difference from non-participating students (for example, a study of pass rates) and if the data is reported only in the aggregate or internally (within a department). In other words, certain IRB’s do leave a certain level of discretion up to the faculty, and I would assume that while most IRB’s would want to see this submitted for review, many wouldn’t lose their heads if Ariely didn’t submit it.

But what I find most interesting is, whenever Chronicle posts an article about a singular professor’s study of students, commenters immediately question whether the professor submitted the study to his institution’s IRB. Without evidence to the contrary, why not just assume that the professor followed institutional protocol?
sfoderick

Thanks for your comments 11122741 – I am sorry that you rush to the assumption that asking for student privacy is tying up those who would otherwise engage in brilliant research. The truth is that these students have now been identified as cheaters in an educational environment, with a concommitent loss of respect no doubt from those who identified them (including the teacher that will be giving them grades). I have no idea if the school’s IRB would have okayed the research (we don’t have all the information), but do know that there are rules in place not to be obstructive but because we are often our own worst evaluation committee. Sorry you feel the need to ad hom argue and not sure what growing up has to do with the situation.
gellci

I strongly agree.
antiutopia

How the heck do you grade those?
https://plus.google.com/115213066046539868050?hl=en anonymous

Its difficult to consent students into this type of experiment without ruining the scientific rigor of the experiment. If you consent students into the experiment telling them that you are doing a study on cheating, then you would probably find different results than if people did not know they were in an experiment. Not knowing that they are in the experiment is ideal. Its better not to consent them.
panacea

Not the point. There are lots of ways to report something other than email. A phone call. A visit to office hours. Catch the guy after class. Catch his TA after class. Call the Dean. Something.

A cynical assumption that there’s no point in bothering to report wrong doing is as scary a proposition as the bad ethics of the students who were willing to click on a link purporting to be a test.
sciencegrad

I’ve had several professors test on material not covered in the course, simply because they barely revise old exams. I would be tempted to use this material based on my past experiences.
drjeff

I once had an issue with a particular class, so I made up 6 different versions of the test: same questions, but in different order. Yes, I had to make up 6 answer keys, too, but it worked: no student was in sight of another with the same version of the test, and the scores were what I would have expected without cheating. Halfway into the test, one student actually asked me how many versions of the test there were! (I told him.) That may be a new example of chutzpah.
http://www.facebook.com/char.mentor Char Psi Tutor Mentor

Great concept, well done to the students who designed the experiment and their ‘supervisor’ for supporting their application of new knowledge
mbelvadi

This is not universally true. At my institution, any study involving students must be approved by the IRB if the results are to be used outside of the department, including presentations at conferences. Our IRB does have a fast-track procedure for asking for waivers from the full review in such cases, but it still requires writing up a 10+ page report to them to get permission and waiting for their reply before proceeding with the research, which means planning a semester ahead.
megginson

That is an excellent point. In some departments (mine, for example), common midterm and final exams in introductory first-year courses are automatically posted online for student use. Students who have seen some departments and instructors do this may not perceive that they have violated the honor code as stated. Also, frankly, the wording of the alternative e-mail reminding students of the honor code sounds odd for a message purporting to offer unethical help to a stranger from a stranger, and my own impression, had this come to me as a student, would have been that it had “sting” written all over it.
prof291

I knew without even reading the comments that someone would bring in “IRB” to bludgeon the professor. First, since the knowledge generated isn’t generalizable, it isn’t research, and therefore it’s outside the IRB purview. Not every human interaction is “research”. Second, internal institutional studies are rarely in that purveiw anyway. Moreover, looking for cheaters, whatever one’s views on the subject, is a disciplinary action that professors are empowered to take. And finally, since IRB’s have so grotesquely jumped the rails over the last few years, it pains me to see academics further demand that they be called in for every transaction.
harveyking

My favorite is the student who called me over and asked if she had the wrong test, since the person beside her had different questions (because they were in a different order).
http://twitter.com/CuffeL Laurence Cuffe

Copies of previous papers, with answers, are routinely available in many institutions, so many of these students may not have felt they were cheating at all.
http://shevralay.wordpress.com Emily Chapman

I can’t speak for the OP, but I know scantrons have “test form” options, so you can have multiple keys for the exam corresponding to different question sets. I’m assuming that’s how.
http://twitter.com/M_Arias5 Meghan Arias

Many universities basically give professors blanket approval for students conducting research in approved courses. The instructor is ultimately responsible and has to monitor the students to ensure they don’t cross any lines. Otherwise, you’d have the 30 students in every Intro to Research class bombarding the IRB each semester!
rod2312

I’m not a proponent of cheating by any means, but I have a big problem with this. Indeed part of the problem stems from deception/involuntary research. One of the problems with involuntary research is that one doesn’t volunteer for something and then has the additional burden of someone interpreting those results. Was curiosity a factor? Is clicking on a link a form of cheating? If it were real, wouldn’t the person who supposedly posted the material the one who would have the real problem?

In addition, there is the “internet” factor. The recent generation of students has a different attitude toward the internet than we older folk do. I noted in previous years that students have an almost implicit attitude that things on their internet are free to pilfer. Some depend on the internet for information and think that the information doesn’t have to be attributed to its author when using it for something other than discussions. Therefore, the click factor might also have to do with the underlying internet factor more than it does with cheating in the sense that many of us consider cheating. It is for this reason that I try to refrain from posting unpublished material related to my research on the internet unless it’s something I’m not concerned with or that only specialists would read.

Another issue has to do with the curiosity issue but it’s more complicated. Clicking on links is a habit, regardless of what it leads to. One would hope a researcher would not set up intentional links to be clicked on. I’ve seen that before. Would the students just as easily clicked on a link that took them to some other kind of site? I bet internet advertisers count on that.

Finally there is the issue of volition. Having the answer link sent to you is not the same as going out to look for the answers to cheat. That doesn’t make it okay to cheat – cheating hurts the other students just as much as it hurts the professor and the student who cheated. So I return to the issue of who would be in the most trouble if it was real – the answer should be – the person who posted the answers and sent out the email.

By the way, II do not consent.