Monday, July 10, 2000

Computer-Savvy Students Perform Poorly on Handwritten Composition Tests By SCOTT CARLSON Students who have been taught to write using a computer don't perform as well on composition exams that use paper and pencil, researchers at Boston College have found. Although the study was conducted on groups of elementary-, middle-, and high-school students, Michael K. Russell, one of the lead researchers in the project, says it might apply to the test performance of college students as well. However, Mr. Russell says the Massachusetts Department of Education, which provided a grant for the study, didn't approve of its results and asked to have its name removed from the report. The study tested more than 500 students from grades 4, 8, and 10 from five schools in the Wellesley, Mass., school district. The students, who were randomly assigned to write their essays on computer or with paper and pencil, answered a composition question from the state's assessment test. Before the essays were graded, all were transcribed to computer text. Out of a total of 20 points, students scored about 2 points higher when using computers. The report says that if students were allowed to use computers on all of the essay and open-ended questions on the state test, the number of students performing in the advanced category would double. About 19 percent of the fourth graders tested would move from the "needs improvement" category to the "proficient" category. Before taking the test, the students completed a survey that assessed their past computer use. The students had used computers for more than three years on average, most said they used a computer when composing final drafts, and most said they preferred to write on computer rather than on paper. In an interview, Mr. Russell says the results are "indirectly relevant" to higher-education concerns. An increasing number of states are using assessment tests to make decisions about diplomas. "If we're mismeasuring kids and making bad decisions about which kids get diplomas, it raises questions about which kids go on to having higher-ed eligibility," Mr. Russell says. "We want to make sure that we're providing students the best opportunity to show what they can do." If the results apply to older students, Mr. Russell says, the questions raised by the study apply to college tests as well. "Within colleges themselves, you have final exams and doctoral examinations, and a lot of those are written. If you want to get accurate measures of what students can do, you have to ask, should professors start thinking about if students should have the option of writing on computers." Wayne J. Camara, vice-president for research and development at the College Board, says the board has studied the issue over the past few years. He says the results of the Boston College study were most relevant to the Advanced Placement exams, which include a 90-minute essay portion, and the SAT II writing tests, which are required by all the Ivy League colleges and many large university systems. Students use paper and pencil on both tests. Mr. Camara thinks that administering the tests by computer will be the "wave of the future." But for the time being, he says, a shortage of resources make that impossible. Although a high school might have several computers, he says, they are generally scattered throughout the building; A.P. and SAT II tests are offered in one large space, where only a few teachers are needed to monitor several students. In a short, prepared statement, David P. Driscoll, commissioner of the Massachusetts Department of Education, also said that the department couldn't put the results to use for some time. "We find the results of the limited study to be very interesting," he said in the statement. "It will take quite a while before it can be implemented. We'd have to make the availability and the keyboarding experience equal throughout the Commonwealth for all children, and that's going to take a while." "Frankly, I think that's their way of justifying not providing an option because it complicates procedures," Mr. Russell says, adding that the department wasn't satisfied with the study's conclusions, and "when they found the results, they essentially tried to discount them." "For a long time they resisted us releasing the study," he says, "and then when it was released they asked us to take their names off the press release." Gregory G. Nadeau, chief technology officer at the department, said that he didn't remember how talks of the report's release were negotiated. But he said that the report might overstate the underachievement of computer-savvy students. The state assessment test's grading scale, he noted, works on the assumption that children are writing with pencil and paper, and that the scale might be tougher if they someday start taking the tests on computers. Indeed, an addendum to Mr. Russell's study showed that graders might treat handwritten essays differently. In a follow-up study, he gave the same essays to graders in their original handwritten and typewritten formats. He found that graders were more lenient on the handwritten essays. But he wasn't sure if that was because it was more difficult to find errors in the handwritten forms, because graders associate typed text with final drafts and are more critical of errors, or because graders felt connected to writers through their handwriting and "gave them the benefit of the doubt." However, Mr. Nadeau added: "It's a good study, it's well done, and it points to the fact that we should be doing more work on this next year." The study follows two similar but more limited ones conducted by Mr. Russell, which reported that middle-school students performed much better on essay tests when they used computers. Mr. Russell plans to conduct a broader version of the study, and he's seeking financing in part from computer companies. Easy-to-print version E-mail this article

College's fraternity rule does not violate antitrust laws, federal judge rules Human-rights groups warn of dangers to Afghan academics in exile in Pakistan New degrees, majors, or programs announced by 10 colleges Computer-savvy students perform poorly on handwritten composition tests Army plans to offer laptops and distance education to all soldiers India uses distance education to meet huge demand for degrees