ProfHacker icon

Previous

Writer's Bootcamp: Breaking Writer's Block

Next

Back to School!

March 25, 2010, 06:00 PM ET

Getting Students to Do the Reading: Pre-Class Quizzes on Wordpress

[In this guest post, Derek Bruff returns to combine two great tastes that taste great together: WordPress and LMS-based reading quizzes. --JBJ]

Harvard physics professor Eric Mazur likes to talk about learning as a two-step process: First there’s the transfer of information (from a source of knowledge, like an instructor, to the student), then there’s the assimilation of that information by the student.  Since the assimilation step is the more difficult of the two, he argues that it should be tackled during class time, when the instructor and other students are present to help out.  Doing so, however, requires shifting the transfer step to occur before class, which means that the usual method of handling transfer—the instructor’s lecture—doesn’t work anymore.  Instead, students need to have their first exposure to the course material happen some other way—like reading their textbook.

I have adopted this approach in the mathematics courses I teach.  I expect that my students read their textbooks before class, then spend class time participating in a variety of activities (like clicker-facilitated peer instruction) designed to help them make sense of the course material.  Okay, okay, I know all the humanities folks who read this blog are rolling their eyes here.  “Have the students do the reading before class and then discuss it during class?  That’s the bread and butter of our teaching!”  That’s true, but in the sciences and mathematics, this turns the traditional approach on its head.  And in all fields, there’s still the challenge of motivating students to actually do the pre-class readings! Research indicates that if you simply ask students nicely to do the readings, only around 30% of them will do so.  That means that 70% of students are likely to come to class unprepared to engage in the kinds of discussions and activities we use in class.  As they say on the Interwebs, #fail.

How to hold students accountable for their pre-class reading assignments?  I have found short, online reading quizzes consisting of open-ended questions that are due several hours before class starts to do the job.  Most of the quiz questions are meant to help students focus on and make some sense of key concepts from the textbook section.  The final question on each quiz is some variety of “What’s one question you have about the reading?”  (I’ll have to credit Eric Mazur again on that question.)  Students submit answers to these questions online before class, and I grade their quizzes on effort.  If it looks like an answer wasn’t typed by a monkey, it usually gets full credit.  I don’t think it’s fair to hold students accountable for understanding the course material on a first pass through it, thus the grading on effort.  I have found this provides sufficient motivation for about 80-85% of my students to do the reading—and make at least a little sense of it—before each class session.

Moreover, these pre-class reading quizzes allow me to practice what is often called “just-in-time teaching.”  I’ll often scan through the student responses—particularly the questions they have on the reading—prior to class, and then adjust my lesson plan to better respond to the students’ particular difficulties with the material.  Sure, I know where students in general are likely to struggle with particular content, but each class of students is unique, so it helps to know more precisely where I might take class that day.  (The first time I teach a course, this just-in-time feedback on student learning is even more valuable, of course!)

Plus, I’ll occasionally use one of the pre-class reading quiz questions as a clicker question during class, copying a few select student quiz responses in as answer choices to the multiple-choice clicker question.  This works particularly well in helping students learn to communicate mathematically, since they often have to select the clearest answer from among several answers that all have merit.

Logistically, how do I implement these quizzes?  I used to post them in my local course management system, but I found the system to cumbersome to use for all the usual reasons.  Inspired by a talk by Gardner Campbell, I started using a course blog as a “home base” for each of my courses about a year ago.  I find it much easier to post course documents to a WordPress blog, and I like that it makes my course more open to those not enrolled in it.  Plus, thanks to some advice I received on this very site, I create a Facebook fan page for each of my courses that pulls in the course blog content via RSS.  This allows my students to become “fans” of the course (on Facebook if not in real life!) and keep up with course news from their favorite social networking site without having to “friend” me on Facebook. So I now post my pre-class reading quizzes on my course blogs, tagged with a “PCRQ” for easy locating.  I put the reading assignment, three or four questions, and a due date in the body of the post, and I ask my students to answer the questions in the comments.  Students need not use their real names in their comments (beware FERPA!), but if they use pseudonyms, they have to let me know which ones they’re using.

Last summer, I taught a course on the history and mathematics of cryptography, and most of my pre-class reading questions were very open-ended “what’s your opinion on X?” or “do you agree with the author’s statement Y?” kinds of questions.  These questions permitted multiple defensible answers, so I used the default comments feature on WordPress to have students reply to them.  This meant that students could read each other’s answers, which, for these questions, only enhanced the learning experience.  In fact, students would frequently reply not just to the questions, but to their peers’ comments as well. Here’s an example. However, for my more traditional math courses, I’m often asking pre-class reading questions that have one correct response each. Having students read their peers’ responses to the questions would rob the students of the experience of grappling with the content on their own.  So I looked around for a way to have students comment on posts semi-privately—where I could see their comments but they couldn’t see each other’s comments.  I found a WordPress plug-in called, appropriately, Semi-Private Comments!  (Plug-ins—yet another reason I prefer WordPress to a course management system.)  After installing this plug-in, I could select on a per-post basis to make the comments for a post “semi-private.”  Doing so required my students to take one extra step in the commenting process: They were required to create a login account on my course blog.  Once they did, however, they could post semi-private comments in just the fashion I wanted.

WordPress’s comments feature allows me to implement pre-class reading quizzes very easily.  The main limitation is that it doesn’t help me grade those quizzes.  I must pull up each pre-class reading quiz post and scan through the comments, denoting on a printed course roster which students answered the questions that day.  I then transfer that data into the Excel spread sheet I use to manage course grades.  With 20-30 students, this is a quick process, but it doesn’t scale well to larger courses, unfortunately.  It’s worth noting, however, that WordPress lets you search your blog’s comments, which has helped greatly when I needed to double-check how many quizzes a particular student completed.

Do you have your students engage in pre-class assignments?  How do you hold them accountable for doing so?  Do you use a tool that scales up well to larger classes?

Image by Flickr user moriza / Creative Commons licensed

Comments

1. Rebecca - March 25, 2010 at 07:43 pm

I JUST started doing pre-class "homework" this semester with my Psych 101 classes. The non-reading portion of that group was probably around 95%, and it was making in-class examples pretty inefficient, so now they have a multiple-choice homework assignment due before each class. It's worth about a 3rd of their course grade, so they can't pass without doing them.

I use multiple choice questions because my course management system can grade them, although I find I spend about 30 minutes per week dealing with the ones who have technical difficulties. I use the testing function of blackboard, but don't treat it like a test; these are open book, open notes (heck, open internet) activities that they can do up to 3 times. They get to keep the highest score. The only "trick" I use (their word, not mine) is that each time it randomizes the order of the questions and the answers, so they have to at least scan them the second and third time to get it right. I even let them print it out, work on it with the book, then enter their answers later.

On the one hand, I know that MOST are now doing the reading. (Not all, and this still puzzles me. The person inside me who has spent too many years in school thinks this sounds like a cakewalk and can't imagine why some don't do it. On the other, they still struggle with some of the exercises when I try to move them beyond just memorizing facts (the infamous "organ recital") and into understanding how the different elements of psychology interact. (Conditioning to overcome phobias...whodathunkit....) I really feel as though no one has ever expected that type of higher-order thinking from them, and that kind of scares me.

2. Robert Talbert - March 25, 2010 at 08:04 pm

Great post, Derek. You're a dependable source of good ideas.

A response to your ending question, and then a few questions of my own for you and others:

I'm teaching an introductory MATLAB course right now using an "inverted classroom" approach similar to what Derek's describing. MATLAB is a professional scientific computing platform and this is a once-weekly, one-credit lab course for second-semester freshmen on using it. We don't have a textbook for the course; instead I assemble collections of video tutorials, usually about half an hour of viewing per week, focused on a particular skill or set of commands for the week. Students watch these and practice along with them during the week. Along with those videos I have a set of specific learning objectives and a collection of homework questions. The HW usually consists of some very basic practice with the stuff from the viewing, along with samples of MATLAB code where they are basically asked "What does all this stuff do?" Then when they come in for the lab, they take a 10-minute quiz consisting of short-answer questions that have the form: "What would you enter in to MATLAB in order to ____ ?" They get to have MATLAB up and running during the quiz, so if they are unsure of an answer, they can just pop over to the system and try it out. This teaches the important skill of quickly checking your code before you execute it.

Since I do the quiz on our course management system (Moodle), it's auto-graded. Although I usually have to go in and fix some of the computer's grading, it still scales up well to larger classes.

Now some questions for Derek and others: (1) Does the university have any qualms about class blogs? I suppose the answer is "no" since they are hosting the sites, but are there any sort of issues with students posting their work to the web where anybody can see it?

(2) Any problems stemming from the fact that this isn't proctored? Do you think there is any evidence of cheating, such as students getting other students to write their answers for them?

(3) What exactly is the FERPA issue with students using real names in the comments?

Thanks again.

3. Rana - March 25, 2010 at 08:31 pm

I do use pre-class reading quizzes, both as a way to encourage/reward students who do the reading ahead of time and as a tool for gauging class understanding of the materials. I also ask them to do a post-class forum comment in which they can ask questions or react to the materials or class discussion.

One thing I've found is that it's better to not call the pre-reading exercises "quizzes" - when I did, I got a lot of student complaints about having to learn the information without help. Students thought, perhaps correctly, that part of the point of class time was to help them make sense of what they'd read. I found that complaints went down - and compliance went up - when I renamed the things something like "reading preparedness assessments" (yeah, I know - but there's no short one-word equivalent to "quiz" or "test" that gets at the idea, as far as I know).

Students also appreciate getting partial credit for wrong answers (as opposed to not answering at all, which again encourages participation) and being able to drop the two or three lowest scores.

I use the course management software our campus uses (it's a Blackboard equivalent) which allows me to set up timed quizzes and exams that open and close at times I specify (I set them to close an hour before class).

4. Mitch Keller - March 25, 2010 at 10:23 pm

Robert,

Let me throw in my $0.02 on a couple of your questions. I'll stay away from (1) since I do my reading questions on our LMS.

(2) In my case, the students are authenticated on our LMS, so it should be them. Even on a blog-based system, however, I don't think cheating would be a problem. I very clearly communicate to the students that these questions are to help make classes more useful to them. They're only worth 5% of their course grade and graded like Derek does. (I like stating the standard as not looking like it was typed by a monkey.) There's really no incentive to cheat here, because they're really only hurting themselves then. Also, if you ask them to give some form of explanation rather than just a number, function, etc., it's too much effort to make up answers for friends, too.

(3) I'd say the potential FERPA issue with real names and public comments is that an ambitious student could go through and tabulate how many reading quizzes a classmate has done. If this is a grade component, then the student has information about a classmate's grade (even if it's only a tiny portion), which is a FERPA issue.

5. Mitch Keller - March 25, 2010 at 10:28 pm

I agree about not calling them quizzes. I've just been calling them "Reading Questions" and had reasonable success with student attitudes. I also make it really explicit in my course policy document that this is not busywork. I make it clear to them that it's to allow us to make better use of our time together in class. I also make sure to incorporate their responses into my class (as Derek suggests) and then tell them that I'm doing it. I've found that when you let them know how you've adjusted the plan for the day based on their responses, they put in better effort on the questions in advance.

6. richard - March 26, 2010 at 12:34 am

Monkeys first: in my second-year comp class today for Education students planning to teach English, while we were talking about a makeup Moodle quiz to cover their abysmal initial in-class quiz, one student asked if there'd be a negative mark consequence if they did worse on the Moodle quiz. When I said there wouldn't be, he said, "Awesome -- I'm pretty busy, but I'll monkey-click my way through and see what happens." Sigh.

Great post, Derek. In my university's freshman comp class that I coordinate (46 sections, 33 students each), which is almost exclusively taken as a requirement to enter or complete non-English programs, 30% would be a good estimate of the pre-reading rate. These flash exercises would be a good way to encourage pre-reading, but I've been reluctant to add even more grading time, since our class size is at 165% of the standard recommended by the National Council of Teachers of English, and many of our instructors (excluding me) are sessional instructors and graduate students.

How can I hit as many student tasks of value as possible, without overtaxing the instructor? Hmm -- maybe the answer's as simple as embracing the monkey-click, or the not-quite-monkey typist.

Make 'em read, make 'em reply, make 'em turn up. Is it really that simple?

7. Kevin Brady - March 26, 2010 at 11:18 am

You might find this link useful -- Learning and Transfer http://www.nap.edu/openbook.php?record_id=6160&page=39

8. Rana - March 26, 2010 at 01:38 pm

I should add, I also use these things as a learning tool. I try, in the multiple-choice questions I pose, to teach as well as test - that is, the questions and answers serve as pointers to the information I hoped the students would make note of as they read their chapters. Even if they get the questions wrong, they still get information (I hope) about what the key points of the chapter are, and how to think about them.

9. Heather Whitney - March 26, 2010 at 03:16 pm

I do something similar in my survey course, after taking the ideas from a journal article entitled, "Reading Questions: Encouraging Students to Read the Text before Coming to Class" (Journal of College Science Teaching, v35 n7 p46-50 Jul-Aug 2006).

8:00am before each class period, they are to submit several "reading questions" on the material that will be covered that day. I ask them to form questions that they would like answered as a result of participating in the day's class, ideally ones that show that they're coming to class with interest in the material and having done a basic read-through of the text (and the worked examples in the text). I do grade them on level of thought, because I want to be able to get fundamentals, such as knowing basic equations like V=IR, out of the way before class starts. I also, like Derek, usually select a few to sprinkle throughout the class. Sometimes my memory is good and I can actually mention who asked the question, which seems to impress students and lets them know I'm really paying attention.

I've been administering them through Blackboard with good success so far on the logistics. Often, I use the comment space to answer a student's question directly when appropriate. However, I've got a bit of refinement to do on helping the students understand the motivation behind the system. On my mid-term evaluations there was a large show of discontent with the element, with many of the students saying they felt like it was busy work. I feel that part of the problem is misconception about how much work should be done for a class outside of the classroom.

10. Rana - March 28, 2010 at 05:45 pm

I feel that part of the problem is misconception about how much work should be done for a class outside of the classroom.

Oh, yes. This is especially true if you're teaching new students. Last semester, when I hadn't yet figured out how to frame the outside-the-classroom work, I had a couple of students complain that my courses were "teach yourself" courses, simply because I expected them to read and take notes on the materials outside of class. It was a useful reminder that many students these days are used to the idea of homework building off of work done in class, but not so much work that prepares them for class in the first place.

11. Rebecca - March 28, 2010 at 07:32 pm

Rana, that is SO true. There is a discussion going on currently on the Teach Psych mailing list about students who complain that things are on the test that the instructor didn't cover in class. These things are amply covered in the book, but there is this worrying trend to expect us to hand the students the answers.

I try hard to explain to my students that I am 1 teacher, but the textbook author is another. They don't seem to get it. Frankly that is what drove me to rest so much of their grade on the pre-class assignments. They can't do the work, but you can bet they can do the math to figure out what grade they would get if they never did one of these assignments. My points are chosen to make it crystal clear that they CANNOT pass with even a C unless they do at least some of these assignments.

12. Scott - March 28, 2010 at 10:49 pm

I also create video lectures (with Camtasia) that students watch prior to coming to class. To ensure that they watch the videos, I have tried several things: (1) An in-class video lecture quiz where they get to use their notes they were to have taken while watching the video lecture and I ask them for the solution to a particular example; (2) Links embedded in the video that took them to a 1 question quiz (I used Google Docs for this and it was a headache to record grades); (3) An embedded password that they write down that gives them access to a password enabled quiz in our LMS.

The first method was effective but it did take up some valuable class time. The 2nd method was a pain to grade and record the grades in a gradebook. I am currently using method 3.

So in theory they have watch the video lecture and now we use class time to do what I used to give as homework. We spend about 2/3 of our class doing this in groups (I have to rearrange the desks) and the other 1/3 of classtime is spent with them doing graded boardwork.

I have done this with intro college level (college alg, gen ed. math, statsistics) and developmental mathematics (elementary alg.). So far the results have been positive. About 85% of the students are actively engaged with the material and the other 15% either don't show up or come without having watched the video. I read them the riot act the first few times this happens.

13. Derek Bruff - March 30, 2010 at 12:55 am

Great questions, Robert. My response to your second question is basically the same as Mitch's response--these pre-class reading quizzes don't count for much in the overall course grade (maybe 5%) so the effort of cheating isn't quite worth the reward. Given the benefits to me and to the students, I'm okay if a handful of students cheat on these quizzes. The quizzes don't count for enough to give cheating students any real edge in the course. If they don't know their stuff, they'll still fail my tests.

As for the FERPA concern, it's not about revealing students' grades online (although that's a definite no-no). My understanding of FERPA is that students can't be required to share online the work they do for a course (be that a paper, a homework set, or a set of responses to a pre-class reading quiz) in a personally identifiable manner. The choice to share their work in a course with anyone outside the course should be the student's choice, not mine. So when I'm not using the Semi-Private Comments plug-in to keep student quiz responses private, I give the students the option of using a pseudonym on the blog. Doing so allows them to complete the reading quizzes while also abiding by the spirit of FERPA.

That gets at your first question, as well. Clearly, the university is okay with the idea of its faculty and staff setting up blogs, otherwise it wouldn't be running a blog server. I haven't received any pushback from the university for my course blogs, but I would imagine that if the university had any concern, it would be the FERPA concern I've just mentioned.

In most instances, my university is very supportive of getting the good work of our faculty, staff, and students out there for all to see. Our public affairs office has an active Web presence that does just that. They seem fairly confident that making more visible the teaching and learning that goes on at my university will only add to the reputation of the university.

14. Derek Bruff - March 30, 2010 at 12:58 am

That's my approach, too, Mitch--to be very clear with the students why I'm having them complete these pre-class assignments. This is spelled out in my syllabus, but I find that students pretty much forget everything in the syllabus after the first week of classes, so I hit this point several times throughout the semester.

Unlike Rana, I haven't gotten any student pushback on the use of the term "quiz." However, I alternate between calling them "quizzes" and "assignments," mostly because I want the students to recognize that part of what I'm asking them to do is to read their textbook. There's an assignment here--read the section and answer these questions--not just a quiz.

15. Derek Bruff - March 30, 2010 at 01:05 am

Thanks for sharing the "monkey-click" story, Richard. I'll have to remember that term!

Wow, 46 sections with 33 students each? Yeah, you'll need strategies that scale up very well!

I guess I didn't say much in my post about the choice between open-ended pre-class questions and multiple-choice questions. As I indicated in my first ProfHacker post, I'm a big fan of multiple-choice questions. I think they're usually underestimated by instructors. However, I think one of the reasons I use open-ended pre-class questions is that I don't want my students to "monkey-click" their way through these questions. Open-ended questions don't take too much more time to grade than multiple-choice questions (given how I grade on effort), but they raise the bar for the students to spend a little more time on the reading.

I suspect that many of my students just scan the reading for answers to the particular questions I pose, instead of reading the whole section first and then tackling the questions. (Several students have indicated in course evaluations that they take this approach.) This indicates to me that some students will do whatever it takes to get through the pre-class reading quiz as efficiently as possible. With multiple-choice questions, that would be monkey-clicking. With open-ended questions, at least I motivate them to read enough of the section to answer a few particular questions about it! The ball is then in my court to ask useful questions!

16. Derek Bruff - March 30, 2010 at 01:16 am

The Charles Henderson article! Yes, I've read that one. Henderson and I corresponded briefly about pre-class reading questions when that article came out. He makes a strong case in the article for this kind of activity.

I've experimented with grading my pre-class reading quizzes on a finer scale than the 0-1 scale I use now. (0 = Looks like a monkey typed it. 1 = Everything else.) In particular, I've experimented with using a 0-1-2 scale for the "What question do you have about the reading?" question, much like Henderson describes in his article. It took me a little while to develop a clear set of criteria for deciding whether a response was worth 1 point or 2 points, and it was hard to communicate these to students. I think the effort is potentially worth it, however, since learning to ask good questions is an important skill.

On the other hand, taking the time to read responses carefully enough to decide between 1 and 2 points for each response was fairly onerous. Also time-consuming was providing sufficient feedback to students on their 1-point questions, feedback that would be useful in helping them develop 2-point questions instead. In the end, I didn't feel that the value of this finer grading scale was worth my time.

As for the complaint by students that the pre-class assignments are busywork, I got that a lot, too. My response--which completely eliminated those complaints when I tried it last fall--was to give students some choice in how the pre-class assignments contributed to their overall course grades.

Students received a clicker grade for the semester out of 100 based on the percentage of clicker questions they answered (right or wrong). Students also received a pre-class reading grade (again, out of 100), based on the percentage of the online reading quizzes they completed.

I then took these two grades, added them up, and divided by 150, capping the final participation grade at 100. That way, a student could come answer all of the clicker questions and only half of the pre-class reading quizzes and still get a 100 for class participation. Or a student could do all the quizzes and skip class half the time and also make 100. More typically, however, students made 80s and 90s on both the clicker and quiz grades, again giving them 100s for participation grades.

Giving students the choice–do the pre-class reading or come to class or both–seemed to eliminate some of the complaints I’ve received in the past about both kinds of participation.

17. Derek Bruff - March 30, 2010 at 01:20 am

Thanks for sharing this, Scott. I like the idea of creating a pre-class "first encounter" for the students. The textbook can be that first encounter, but an instructor-created video can work very well, too.

One day, I'm going to ditch my textbook-based pre-class assignments and have students watch videos from the Khan Academy instead, just to see how this video approach works differently. I think many of my students would get a lot out of a short video explanation of a problem or two as a "first encounter" with the material.

18. Derek Bruff - March 30, 2010 at 01:22 am

I surveyed my students one semester about how they use their textbooks in their other math, science, and engineering courses. I think one student out of 56 indicated that they read their textbooks before class in other courses. All the others said that they only used textbooks after class (mainly to find examples helpful in completing problem sets), if at all.

So I'm very much going against the grain here when it comes to student expectations. As I mentioned in a comment above, I really have to sell the students on this!

19. Mitch Keller - March 30, 2010 at 07:39 am

Interesting remark regarding FERPA. I'd not thought about that aspect. (Fortunately, for the YouTube video assignment I gave this term, I told them they could make the video private as long as they gave my account access to view it, so we're clear on that front.) I think one of the biggest problems with FERPA is that individual institutions have so much latitude in how to interpret it. It wouldn't surprise me if I talked to our Registrar about this and she said it was fine so long as grades couldn't be determined from the public data. As a prime example of institutional interpretation, see the way Cornell has recently revised their interpretation in light of growing suicide concerns.

20. Mitch Keller - March 30, 2010 at 07:44 am

One problem in this regard is that most math books (I can't speak for the other sciences, since it's been too long since I've interacted with textbooks outside of math) are not designed to be read. (OK, books for math majors are, but that's a different audience for the most part.) I was adopting a new precalculus text last fall and was dismayed at the review copies the major publishers sent me. Most were just a collection of examples with a little bit of "connective tissue" inserted. The students would only consult the book to find an example that matched the homework problem they were working on. I found one book that met my needs and swiftly adopted it. The book was actually written with the idea that students would read it, rather than consult it while doing homework. For the upper-division (but still service course) text I'm co-authoring right now, we're taking a similar approach. I wish more authors would step up and write these sorts of books!

Add Your Comment

Commenting is closed.