The three tests used by institutions participating in the Voluntary System of Accountability are compatible in determining students' skills in critical thinking and writing, according to the results of a study released today.
The accountability system is used by more than 300 public colleges -- all members of either the American Association of State Colleges and Universities or the Association of Public and Land-Grant Universities -- to provide the public with information about life and learning on their campuses.
As part of that system, each participating institution chooses one of three tests to administer to students in order to measure their learning from freshman to senior year. The three tests are the Collegiate Learning Assessment, sponsored by the Council for Aid to Education; the Collegiate Assessment of Academic Proficiency, from ACT Inc.; and the Measure of Academic Proficiency and Progress, offered by the Educational Testing Service.
"This study enables ... participating institutions to confidently choose one of the three tests to measure the core learning outcomes for critical thinking and written communications while ensuring comparable results," said David Shulenberger, vice president for academic affairs at the association of land-grant universities.
Mr. Shulenberger said the study was important to validate the accountability system's allowing three different tests -- a policy that was chosen to balance the need for comparing results with institutional autonomy.
The results may also help persuade other institutions to sign up for the accountability system, he said.
The study was conducted at 13 universities that participate in the Voluntary System of Accountability. More than 1,100 students were given 13 different tests in critical thinking, reading, mathematics, writing, and science.
The research was financed with a grant from the U.S. Department of Education's Fund for the Improvement of Postsecondary Education.
Comments
1. 12052592 - November 03, 2009 at 04:51 pm
Great. Another test to show how far behind African American and Latino students are from Asians and Whites.
2. browng8 - November 03, 2009 at 05:40 pm
It is no surprise that they are consistent, as in consistently useless for engaging community to develop strategies for improvement.
3. kassoy - November 03, 2009 at 05:41 pm
What is it that we academics want students to learn? How should they be able to demonstrate it? For 45 years I thought that class exams and term papers were about demonstrating knowledge learned. The interest in assessment is driven by other constituencies, public and political, that have different learning agendas. I'd like to know if a student can solve partial differential equations, and interpret the results. I doubt that the interested citizen would be turned on or off by that objective. Rather the taxpayer citizen wants to know if the investment in higher ed generates capable employees, and reliable citizens with traditional values. People seem to worry that the educational experience will foster heretical perspectives that threaten the society.
Back in the 60's as a graduate student I had u.g. friends at Michigan who were excited about left-wing politics (e.g. SDS). Their parents articulated great fear that they had been lost to Communism. I assured them that their students, once graduated, employed, married and parenting would exhibit the value systems that the parents had laid on them. With one exception every friend did just that. Its the pragmatics of life that drives tradition. One needs income, friends, a support system and eventually communal approval.
A student's success at coping with life's challenges (employment, marriage, parenthood) is a reflection of what was gained from the higher ed. experience. This is about learning to accept responsibility for what comes at you, about communicating in written and oral form, about working with others to accomplish objectives, and about life-long learning needed to be a functioning human being. Can assessment exam outcomes predict any of this? I'd be curious to know.
4. rachel312 - November 04, 2009 at 06:16 am
Perhaps these tests simply demonstrate that 18 year olds still have some cognitive maturity to attain, and that if we let them be in a relatively safe and relatively structured environment with their peers for a few years, with regular stimuli drawn from the sciences and humanities, that their minds mature and they can solve problems better by the age of 22?
5. grifflee - November 04, 2009 at 09:17 am
A study shows that all three tests are consistent with each other, and voila - we have validity! But if all three tests are measuring something other than what is taught, the results of this study confirm only that they are equally invalid.
Compared to the intellectual tasks that most first-year composition courses emphasize, tests measure only a small portion of the lowest-level range of skills. Examples of skills in typical undergraduate education: sorting through glut of available information, evaluating sources, reconciling contradictory information, seeking multiple perspectives on issues, creating complex theses, synthesizing material from multiple sources, revising not only to correct surface errors but to re-envision ideas, addessing rhetorical issues in real and complex contexts, predictive and reflective thinking, and writing-as-learning.
The intellectual tasks demanded of students on tests are severely constrained by two factors: 1) the need to maintain the integrity of the test and avoid cheating precludes students from engaging with any part of the real world where the real action is; and 2) the capacity of the human bladder, which requires that the kids get a break after a couple hours. As hard a testmakers try, these two conditions make it impossible for tests to reflect the kind of cognitive work students must do in the real worlds of education and work.
Assessments are not necessarily tests. Many new assessment methods are capable of measuring the most complex and rigorous cognitive tests and need only a little more time and attention to reach fruition. That should be our target.
Merilee Griffin
6. comphx - November 04, 2009 at 10:27 am
How about institutions of higher education create an instrument to measure the effectiveness and accountability of our elected political heros...or hold them accountable for a blantant disregard to education by voting them out of office, replacing them with educators and citizens proud of the public good (not private good) of a quality education, determined not by the score of an exam, but rather the acquisition and application of knowledge.
7. ianative - November 04, 2009 at 10:36 am
I find these comments quite fascinating. All are negatively phrased, but none of the "commenters" appear to be familiar with the tests themselves. One actually goes so far as to say that tests (any test, apparently) can only measure low-level thinking, a broad-brush statement easily negated by scores of existing tests that do, in fact, measure higher-order thinking.
Sure looks like some self-appointed experts could use a little critical thinking instruction themselves... especially that part on gathering reliable evidence prior to making judgments.
8. grifflee - November 04, 2009 at 11:07 am
To ianative:
I am, in fact, fairly familiar with the three tests in question. I see nothing in your response that refutes my claim that tests limited to two hours in which students can work only with whatever knowledge and opinion they bring to the testing station do not measure the highest level of intellectural work, which necessitates engaging with a cognitively complex, messy, and very real world.
Our disagreement may lie in differing definitions of "higher-order thinking." I think students need far more advanced skills than those measured during test periods.
Merilee Griffin