Textbooks and Tests That Talk Back New software allows professors to provide instant feedback to students By LISA GUERNSEY Raleigh, N.C. Up here on the second floor of Cox Hall at North Carolina State University, the students in Physics 208 are facing their first quiz of the semester. ALSO SEE: How a Quiz Provides Instant Feedback Links to on-line demonstrations of computerized homework assignments and tests: A demonstration of "physlets" -- or miniature Web-based computer programs -- that simulate physics laboratory experiments: http://webphysics.davidson.edu/ Applets/Applets.html A demonstration of WebAssign: http://www.webassign.net/info/ The quiz isn't a typical problem scribbled on a chalkboard or printed on paper. Nor are its questions of the "plug and chug" variety, in which students are expected to recall formulas they've memorized and plug in the correct numbers. Instead, the quiz exists on a World-Wide Web page, displayed on 18 laptop computers. The 48 students, working in groups of two and three, have been asked to measure the strength of two electric fields simulated on the computer screens. To answer the two questions, they must play with the simulation, clicking and dragging a green dot that represents a test charge. As they move the dot around, they can see how strongly the test charge is attracted to, or repelled by, the electric fields. "The whole setup," one student will say after the class is over, "is pretty slick." But right now -- with 15 minutes left to finish the quiz -- coming up with an answer is paramount. The students type their answers into boxes on the Web page and hit the "submit" button. Red X's appear if the answers are wrong. As the room fills with the chatter of students debating and making suggestions, the X's are popping up everywhere. The students shake their heads and try the problem again and again, jotting calculations in their notebooks. The red X's may elicit frustration from those in the throes of a quiz, but to many educators such automatic feedback is a critical feature of computer-based teaching. The programs are designed to let students check their answers instantly on quizzes or homework, without waiting for professors or teaching assistants to grade the work. Often, say professors who use the programs, immediate feedback encourages students to keep working on problems that are giving them trouble. This Friday morning, for example, the instructors in Physics 208 -- John S. Risley, a veteran physics professor, and Scott Bonham, a postdoctoral researcher -- aren't worried about the students' needing repeated attempts to answer the question. The instructors' objective is to teach the students how to think about physics problems, not how to plug and chug. The two physicists have asked the members of each team to record, on paper, how they came up with an answer. It is those explanations, rather than the red X's, that will count toward the grades. "I'm not so concerned that they don't know the answer right away," Mr. Risley says later. "I want the quiz to enable them to learn the material." The quiz is part of WebAssign, a program designed largely by N.C. State professors for use in introductory math and science courses. In addition to automatic feedback, the software records every submission, providing professors with detailed accounts of students' performance (http://www.webassign.net/info/). WebAssign is now the primary vehicle for quizzes and homework problems in about 60 physics, math, computer-science, and statistics classes at N.C. State. More than 45 high schools and universities around the United States are also giving it a try, helping N.C. State test the feasibility of selling the software commercially. If the product takes off, professors here say, WebAssign would be part of a revolution in teaching introductory science courses. The software, they say, can alert professors and students alike to gaps in the students' comprehension, helping to avert the low test scores that tend to turn people away from science. "WebAssign can make physics more attractive," Mr. Risley says. "The vast majority of people need to be encouraged to learn this stuff." For students, WebAssign is like having a textbook or quiz sheet that talks back. The software offers many of the same homework problems that appear in the textbooks, and it checks the answers at any time, day or night. In most classes here, multiple tries don't count against the students, as long as they find the right answer by the time the homework is due. Tim Goetz, a sophomore in Physics 208, says the software's instant feedback has become a critical part of how he learns. "There's nothing better," he says. "It gets to where you expect it. You can resubmit and resubmit until you get it right." For professors, the program functions like a gradebook that fills itself in, analyzing how individual students and entire classes are handling assignments. Instructors use passwords to tap into WebAssign's logs and call up charts showing which students are having trouble and which material needs to be better explained in class. Such computerized grading and feedback systems are becoming an integral part of on-line teaching, whether it occurs as part of traditional courses or in those taught at a distance. Automatic assessment programs now come with courseware like Blackboard, TopClass, or WebCT, and with distance-education systems like those offered by Real Education Inc. Computer-savvy professors and instructional designers on campuses around the country are developing similar software that is customized for specific courses. Software for essay grading has appeared on the horizon as well -- attended by some controversy. A researcher at the University of Colorado at Boulder is experimenting with a computerized system that evaluates essays by comparing them to ones it has read before (The Chronicle, September 4, 1998). And the Educational Testing Service has announced that it will start using computers, in addition to humans, to help grade essays on the Graduate Management Admission Test (The Chronicle, January 29). Some people have reacted to the developments with trepidation, and have wondered whether such subjective evaluations should be trusted to computers. Even the thought that computers might grade multiple-choice or fill-in-the-blank quizzes leaves some educators rattled. Are computer programs, they ask, replacing professors? Professors at N.C. State answer that question with a hearty No. If used correctly, they contend, computerized grading programs let professors spend less time grading and more time teaching. WebAssign, they add, is designed to be used for homework and quizzes, not for exams that carry significant weight in determining students' grades. Mr. Risley, the professor administering today's quiz, has used WebAssign for nearly four semesters. He views the software as a tool that entices students to learn. "Students see immediately how they are doing in the class, so that helps them stay motivated," he says. In large math and science courses where quizzes are graded by hand, students may not find out how well they've done until long after a concept has been taught. "If you just get quizzes back three to four weeks later, you lose interest," Mr. Risley says. The problem presented by quizzes and homework assignments in large science courses has hounded physics professors here, as well as elsewhere, for years. How, they ask, can a professor effectively grade hundreds of assignments every week, even with a team of teaching assistants? Some instructors, Mr. Risley says, have simply decided not to collect homework, leaving students with little incentive to complete it. By contrast, professors at N.C. State who use WebAssign are asking students to take quizzes and do homework more often than before, because the software takes care of the grading for them. Professors can also encourage students to talk to each other about the problems without worrying that one student will simply give other students the final answer. That's because WebAssign displays questions differently to each student: Randomized numbers are inserted in math problems, so that no two students, or teams of students, are working on exactly the same one. Professors also say students using WebAssign are more inclined to seek help from professors and teaching assistants throughout the semester, instead of waiting until crunch times. Larry Martin, a visiting professor of physics at N.C. State and WebAssign's lead developer, says that after professors here started using the software, students showed up at the university's tutorial center in a steady stream throughout the semester. "It used to be that two weeks before the exam, they'd have 200 people jammed in there screaming, 'I need some help!'" Mr. Martin designed the WebAssign prototype four years ago, when he was teaching at North Park University, in Chicago. A year later, at a meeting for physics teachers, he met an N.C. State graduate student in physics named Aaron Titus who was creating a similar program. In 1997, Mr. Martin came to N.C. State to work with Mr. Titus in developing the software further. Mr. Titus, who is now an assistant professor of physics at North Carolina A&T State University, had already begun creating a program that used a data base of Webbased physics questions. He chose simple multiple-choice questions culled from textbooks whose publishers allowed him to experiment with their material. He and Mr. Martin sought questions using video and animation, too, and they soon incorporated a series of such items that were being designed by Wolfgang Christian, a physics professor at Davidson College. Mr. Christian had been experimenting with Java, the computer language that enables mini-programs called applets to be run over the Web. He had created animated physics-instruction programs that he called "physlets." So far, he has designed 25 physlets, which have earned him both a National Science Foundation grant and, just last month, an award from the journal Computers in Physics. One of his physlets is the electric-fields problem that Mr. Risley asked his Physics 208 students to solve. The combination of Mr. Martin's automatic-grading prototype, Mr. Titus's data base, and Mr. Christian's physlets resulted in WebAssign, which was used first in N.C. State's introductory physics courses in the fall of 1997. Since then, more than half a dozen textbook publishers -- including Addison Wesley Longman and Prentice Hall -- have jumped on board. They say they are more than willing to allow their textbook questions to become part of WebAssign's data base of homework and quiz problems, as long as students in courses using the data base are asked to buy the books, too. North Carolina State has designed the data base so that, for each course, WebAssign displays questions only from the books that professors have asked their students to use. Now North Carolina State is starting to market the software, using advertising and demonstrations at physics-and-technology meetings to get the word out. Professors who want to use a basic version of WebAssign are charged a $250 setup fee and are asked to pay N.C. State $5 per student per class per semester. The university owns the copyright to the program, and Mr. Martin and Mr. Titus receive royalties from its sales. Physics professors at about half a dozen universities have signed up so far. Evaluations from students at several of the institutions have been favorable. But WebAssign isn't perfect, say a few students here. Willy Huang, a sophomore in Physics 208, says that even though he likes WebAssign, he sometimes gets nostalgic for old-fashioned homework, like black-and-white questions at the end of a textbook chapter. "When I have to open a book, I feel like I'm learning a lot more," he says. "Sometimes, when I'm using WebAssign, I feel like I'm using it just to get to the right answer, not to really learn it." In addition, those who aren't accustomed to computers or the Web sometimes have trouble with the software. "I think the majority of students are intimidated to a degree," says Mr. Huang's classmate, Mr. Goetz. "It causes stress -- not that we don't overcome it." Students at N.C. State, he adds, know that technology is going to be a big part of their classroom experience. "When you arrive here, it's 'Welcome to the electronic age' in a hurry." Technical glitches, too, can create problems with the electronic teaching tools. In fact, when professors arrived at work on this Friday morning, they found notices taped to their doors: The power would be off for the weekend to accommodate new construction. For the past few hours, Mr. Risley's colleagues have scrambled to find a way to keep WebAssign's servers running despite the scheduled outage. Meanwhile, he and Mr. Bonham are collecting quiz papers and leaning over laptops, answering questions from a few still-perplexed students. Several other test-takers have managed to solve the quiz problems: A chart at the top of their screens shows their progress, and the red Xs have disappeared. Those students are now flipping shut their laptops, slinging backpacks over their shoulders, and heading out the door. When Mr. Risley returns to his office, he hears good news: A generator has been found and the power crisis averted. WebAssign will be available all weekend, as usual, sending instant feedback to the students whenever they decide to take a shot at their homework, at any hour of the day or night. http://chronicle.com Section: Information Technology Page: A21