Thursday, June 1, 2000

Merlot Project Brings Peer Review to Web Materials for Teaching By JEFFREY R. YOUNG If your peers reviewed your course Web site, how would it rate on a scale of 1 to 5? You might soon find out, thanks to a new guide to online teaching materials that is systematically rating academic Web sites -- and allowing users to add their own comments. The guide, called Merlot, already lists nearly 3,000 items, but the professors involved in the project are only now beginning to evaluate and rate the sites. In addition to deciding which sites get five stars and which get less praise, reviewers also give detailed reports about each site. Authors of the materials are notified when their sites are up for review, and they are given a chance to improve the sites based on reviewers' comments. The Merlot project was conceived last year as a collaboration among the California State University system, the University System of Georgia, the University of North Carolina system, the Oklahoma State Regents, and the State Higher Education Executive Officers, a national association. The universities have provided release time or stipends to the professors who serve as content reviewers. The project's founders have set up a not-for-profit corporation to administer Merlot, and they are recruiting other institutions to participate. Sixteen more institutions are considering committing resources to the project, says Gerard L. Hanley, acting senior director for academic technology at the C.S.U. system. "I believe Merlot is the only project that is attempting to evaluate the quality of learning objects," says Rhonda Epper, a Merlot leader who is a project director for the State Higher Education Executive Officers. "It's hard for professors to find good material out there in any kind of context that makes sense to use in their courses." The plan is to form 12-member review teams for each discipline. So far, however, only four teams have been chosen, in biology, physics, teacher education, and business. The review process is modeled on the way peer review is done for academic journals. Three reviewers working in the site's discipline review each teaching element. The reviewers consider three major factors: quality of the content, potential effectiveness as a teaching-and-learning tool, and ease of use. The guide focuses on pieces of course Web sites, or "modules," rather than sites as a whole. The idea is that professors might want to incorporate those modules into their own course Web sites. Materials reviewed include visual simulations, animations, tutorials, and exercises. Materials that get high ratings come up first when visitors use Merlot's search engine. And Merlot's leaders plan to recognize professors who have developed excellent sites by mailing notices to their department heads. "If you spend a lot of time working on a module, then you should get some credit for it," says Cathy Owens Swift, a professor of marketing at Georgia Southern University who is one of Merlot's reviewers. "People spend a lot of time developing modules, but nobody else ever sees them except their students." So far, only a handful of sites have been peer-reviewed. One example is a physics Web module, created at the University of Oregon, that demonstrates the concept of thermodynamic equilibrium. The module is a Java applet -- a small computer program that can run inside a Web browser -- that shows how gas particles move at different temperatures. Reviewers gave it four stars. "The graphical nature of this material is excellent [and] useful for developing conceptual understanding," reviewers wrote. The reviewers also list "points of concern" for every module. For the physics module, they noted other features that might be added to improve the teaching tool. Merlot is not the only guide to academic Web sites. Among other large efforts to catalog course Web sites are the University of Texas World Lecture Hall and WebCT's E-Learning Hub. And many scholarly associations and other groups have developed guides to course materials in specific subjects. Those sites generally don't assign ratings to resources, however. Jessica A. Somers, who is director of academic innovation at the University System of Georgia and who is working on Merlot, says that several professors were hesitant about the idea of a rating system. Some of the reviewers are considering withholding reviews that earn only one or two stars, as a way of not criticizing professors who have taken the time to develop course materials. But Ms. Swift says that the ratings will make the site more useful than it would be without them. "It may not sound very academic," she says, "But at least it gives people an opportunity to say, 'This looks like a good one,' or, 'This may not be very good.'" Easy-to-print version E-mail this article

Asian-American scholars call for U.S. lab boycott to protest treatment of Los Alamos scientist Blue-ribbon panel on college-sports issues will reconvene this fall U. of Cambridge may end Shakespeare requirement for students of English Spring commencement speakers are announced by 13 colleges Lawsuit pits 2 instructors against anonymous "teacher review" postings Merlot project brings peer review to Web materials for teaching