next »

[octoprof]

I have an online quiz for every module (roughly chapter) in my full online course. These are 10 questions and largely objective (mixed types). Students may take and retake the quiz as many times as they like until the due date. They receive the highest score earned. Their average on these quizzes is worth 10% of the course grade.

A colleague (in another discipline) uses quizzes similarly in a web enhanced course, but they count 30% of the course grade. Both of us use a question base at least 4 times bigger than the size of the quiz, with random or semi-random (by groups) assignment of questions to each quiz.

After the semester is over, we are going to pool our data and study the relationship between quiz-taking (re-taking), quiz scores, student attributes and course grades.

How do you use online quizzes?  If you use them somewhat similarly, I'd love to hear about it, for sure. If you use them very differently, that might also be interesting.

[wanna_writemore]

In my hybrid survey, I have 3 online quizzes worth 33-35 points each.  The total is worth 10-15% of the course grade (depending on the semester).  I make it clear that the quizzes are open-book and open-note, but that students can only finish if they have already done the reading before starting.  Quizzes are timed (30 minutes if all multiple-choice and 45 minutes if they include writing a short paragraph).  They can be taken only once during a 3-day window.

There are no makeup quizzes without documented excuses, but students can occasionally earn extra points that are applied to the quiz grade, which can go over 100 points.  These opportunities include completing an (anonymous) mid-semester survey or watching an online video and writing a few paragraphs on a topic related to class topics.

[changinggears]

Please let us know your findings, octoprof!  I've read different takes on online quizzes from the research people.  Some research has shown online quizzes to not be effective, for various reasons.  In a study on online learning that I read recently, they collated student feedback on the quizzes, which were very enlightening.  The students pointed out the high amount of cheating that went on and also the lack of challenge with repeatable quizzes (the "just keep guessing until you get it right" technique).  Also, many students viewed the quizzes as busy work and some were upset that students who didn't complete the readings were just as likely to pass the quizzes as those who did the reading, so why bother with doing the reading? But I can see where you could overcome these issues by designing the quizzes effectively.  I like the idea of allowing multiple attempts but have often wondered about just the points brought up by the students in this research.  A large question pool would lessen these issues but I often struggle with coming up with effective questions to begin with, so creating a large pool would be time-consuming.  A timed quiz would also eliminate the treasure hunt scenario, but if the whole point of online, multiple attempt quizzes is to allow students to learn at their own speed (somewhat), learn from their incorrect answers (in which case, they actually learn more than they do from getting an answer correct, or so I've read somewhere), and have multiple encounters with the course content, then adding a time limit seems incompatible with those objectives.

Just thinking out loud here.

Anywho, I usually make the quizzes 10% of the course grade and cumulatively equal to an exam grade.  I allow three untimed attempts (the questions are MC with 4-6 choices), but from a relatively small question pool (for reasons stated up-post).  This is the first semester I've used the online quizzes and I'm asking for student feedback but promised not to read it until after finals, so I can't comment on the effectiveness of this system from the students' POV.

[caesura]

One solution is to have practice quizzes that can be done multiple times, then a real quiz that can be done only once.  Give minimal points for completing the practice (regardless of times taken), then grade the quiz that "counts." 

[proftowanda]

I also would appreciate any further findings on this, also having read and heard a range of studies and colleagues' feedback on quizzing often, open book, repeatedly until passing, etc.  I have not included these in my current online courses, as they are designed to correlate in a sequence of courses without such quizzes.

But I'm soon to embark on putting together another online course, one not in a sequence and all my own.  So I am considering the task of putting together a lot of quizzes, too.

A question, though:  My online courses may be 16 weeks in a regular semester or only three to four weeks long in accelerated summer and winter terms.  I think that more testing -- more than a couple of midterms and a final -- could do good to keep students going and growing through four months on their own online.  But I think that three tests already come along quite quickly in an accelerated of only three to four weeks.  If you quiz a lot in such accelerated terms, say a quiz per chapter, that would mean a quiz every day.  Do any of you do so?  And then not allow several days for each quiz (many of my students are so muddled already in following a fast course schedule) but only a day, so that students move along to the next chapter, material, etc.?

[blackadder]

For my current class I'm using the quiz function as midterm and final exam. However, I was "strongly encouraged" to have the exams proctored, which was pretty complex given that it's an online course. Thankfully all the students attend this one school.

I've spent quite a bit of time creating a good Question Library and always randomize the exams. It took about a year to get there though.

Not sure what I'm going to do for two online 6-week summer courses. I want to do some weekly reading "assessments" and think that I can have short and long answers since the classes are small (10 or less each). Still random questions though.

Hope you publish that work on the quiz/class comparison....or at least share here! Hm...I sense a project in my future as well.

[octoprof]

proftowanda, I post the quizzes at the beginning of the term (basically I post everything at the beginning of the term). So, students have access to each module/chapter quiz until its due date. In a regular semester, they are due (roughly) weekly. If I taught a summer term, they'd be due about every other day. That's killer. But, taking accounting in a summer term is nuts anyhow!

Since everything has a due date, most students tend to wait until that date, unfortunately. However, they are allowed to work ahead and a very few do.

[neutralname]

Reviving this thread to ask some related questions on online quizzes.

So far the discussion has been about untimed tests with deadlines.  Any experience of timed tests? 

If I did give a timed test for an online course, then how do I deal with students with disabilities who need special testing conditions or longer test times?

Wouldn't giving a timed test mean that students have the learn the material more thoroughly?

[torshi]

Reviving this thread to ask some related questions on online quizzes.

So far the discussion has been about untimed tests with deadlines.  Any experience of timed tests? 

If I did give a timed test for an online course, then how do I deal with students with disabilities who need special testing conditions or longer test times?

Wouldn't giving a timed test mean that students have the learn the material more thoroughly?

I've given 200-300 online tests over the past few years, mostly timed, from 10 to 50 questions.  But I don't teach online courses, so I don't know if this is the perspective you want.

Students with disability accommodation get the recommended accommodation.  Usually it is for 1.5 X the time, but sometimes for 2 X the time.  They exceed the permitted time, the learning management system records that fact and their time, and I check their time when OK-ing the score.  If the student needs isolated or quiet test conditions, how do you provide that now?  That hasn't been presented as a concern with online testing in my classes.  Students who need distractionless test conditions have needed those only for an alternative to an in-the-room test. 

Do students have to learn the material more thoroughly?  Compared to an untimed test, they have to know more content because they don't know what they'll be asked (unless the test is not well designed and they are in a cheating ring).  They can't not learn it, then look it up during the test, or practice with the test to learn the material.  (It's possible to design tests as a way of teaching the material, but I don't do that.)  They don't have to learn it more thoroughly than any paper-pencil quiz or exam in a classroom or testing center. 

For timed tests, the most effective method for learning has been short weekly quizzes, top 10 of 14 or similar, with mostly multiple choice questions and occasionally some short-answer application questions.  I've had problems with longer written-answer questions.  I follow multiple-choice and online test-making best practices I've found in various places, some of which have been mentioned here.   The unproctored online environment is different from the in-class environment in terms of cheating.  But it is possible to have low-stakes cheating-possible quizzes combined with high-stakes cheating-difficult exams, as long as the students know what's coming.

[fishprof]

I am short for time, but I want to get in on this discussion.

I solved the guess until you get it right problem by providing ONLY the answer given, and whether it was right or wrong.  Providing them with the correct answer just drove a pathetic kind of cheating.  A student took a quiz 27 times in about 40 seconds (total, not each) and then two days later, got a perfect score on the quiz.  How?  He took the quiz, submitted no answers, and then assembled all the feedback into a file with the questions and the correct answers.

Clever, and yet so stupid....

So now, they can only know if their answer was right or wrong, and then they can try again.  With a large pool of questions (like torshi 4-5x the quiz size), it becomes easier to just learn the damn material.

[octoprof]

I am short for time, but I want to get in on this discussion.

I solved the guess until you get it right problem by providing ONLY the answer given, and whether it was right or wrong.  Providing them with the correct answer just drove a pathetic kind of cheating.  A student took a quiz 27 times in about 40 seconds (total, not each) and then two days later, got a perfect score on the quiz.  How?  He took the quiz, submitted no answers, and then assembled all the feedback into a file with the questions and the correct answers.

Clever, and yet so stupid....

So now, they can only know if their answer was right or wrong, and then they can try again.  With a large pool of questions (like torshi 4-5x the quiz size), it becomes easier to just learn the damn material.

I also use a pool of 4x or so the quiz size. However, I do want them to know why the answer was wrong or right, particularly the calculation questions.

I want them to learn how to do it right, not just keep trying (given there are many ways of getting it wrong) endlessly.  Of course, these quizzes are only a very small fraction of the course grade, since, in theory, they should be able to get 100% on all of them (which they do not do).

[fishprof]

I also use a pool of 4x or so the quiz size. However, I do want them to know why the answer was wrong or right, particularly the calculation questions.

I want them to learn how to do it right, not just keep trying (given there are many ways of getting it wrong) endlessly.  Of course, these quizzes are only a very small fraction of the course grade, since, in theory, they should be able to get 100% on all of them (which they do not do).

That makes sense for calculations.  My goal is to get them to read, attempt to answer the questions, and then go back and read more closely the parts they didn't get.

What I left out was that the chapter quizzes have a due date of before I lecture on the material.  I was tired of having blank faces stare at me when I introduced terms in class.  I'd rather use the class time to explain, clarify and expand on the text, rather than repeat it.

[hennypenny]

I've experimented with on-line quizzes for introductory social science courses.

Multiple choice, timed, within a (I think) 24 or 48 hour window, as many attempts as desired, 20 questions out of a pool of 60 or so, questions could vary from attempt to attempt, order of answers shuffled each time, score = average of all attempts.

These last parameters cut down on answer-fishing.

There was a quiz for each module in a 5-module course.  The lowest quiz of the 5 was dropped.

I didn't notice any obvious difference in class average scores compared with similar quizzes given in-class.

I did this all with Blackboard and am interested in people's comments about implementing on-line evaluation in Sakai.

hp

[rod_torfelson_armada]

I teach a grad course online in the summer, that has a very broad mix of students.

I use quizzes more as a "stick" motivator, to (try to) make sure students are doing the reading & keeping up in a fully online course. I have 3 quizzes, each short (10-15 questions) from a larger pool of items. I keep them timed to minimize cheating, so everyone has 1 minute per question essentially. Everyone gets a randomized set of questions & I only give students one attempt - they can go back and look at their answers only after the quiz is completed for everyone.

I make other arrangements for students with disabilities that need accommodations.

The 3 quizzes are worth about 20% of their total grade, so they are "medium stakes." There is no way to know for sure, but this seems to serve the function I want without a lot of cheating.

This is interesting to read & I'm curious about how others are using quizzes.

[prufrock]

I've taken to online quizzes in recent years, probably ten to twelve over the course of a semester, usually 10 questions each (mostly MC/TF but some short answer and occasionally short writing), with a 20-minute-ish time limit and only one attempt allowed.  The sum of their quiz scores makes up 10% or so of the overall grade.  I view them as evaluative but primarily as pedagogical tools, as a way to encourage students to review their notes etc. It gives me a sense of how they're doing, which helps me see what I need to re-cover and drive home (yet again).  I also include a question or two on an upcoming reading, as a way to encourage their doing that before class.

I regularly remind students that quizzes are a way to encourage them to engage in a crucial practice that many of them rarely bother with otherwise: regularly reviewing their notes.  I assume (with no evidence) that making quizzes relatively low-stakes reduces the temptation to cheat; I try to convince them that they're better off finding out that they don't really understand x so that can try to figure it out come exam time than they are cheating to get x right on a low-stakes evaluation, which will come back to bite them come exam time.  I find regularly reminding them of why we're doing quizzes seems to help students regarding them as helpful rather than as nuisances.  If my course surveys are to be believed, students seem to buy into this, and most seem to like way quizzes incentivize staying on top of the material.

Plus, I don't have to take up class time giving quizzes, and I don't have to hand-grade them, for the most part.

Online quizzes: me likey.