Online Course Provider, StraighterLine, to Offer Critical-Thinking Tests to Students

January 19, 2012, 12:29 pm

As alternatives to the college diploma have been bandied about recently, one question always seems to emerge: How do you validate badges or individual classes as a credential in the absence of a degree?

One company that has been hailed by some as revolutionizing introductory courses might have an answer.

The company, StraighterLine, announced on Thursday that beginning this fall it will offer students access to three leading critical-thinking tests, allowing them to take their results to employers or colleges to demonstrate their proficiency in certain academic areas.

The tests—the Collegiate Learning Assessment, sponsored by the Council for Aid to Education, and the Proficiency Profile, from the Educational Testing Service—each measure critical thinking and writing, among other academic areas. The iSkills test, also from ETS, measures the ability of a student to navigate and critically evaluate information from digital technology.

Until now, the tests were largely used by colleges to measure student learning, but students did not receive their scores. That’s one reason that critics of the tests have questioned their effectiveness since students have little incentive to do well.

Burck Smith, the founder and chief executive of StraighterLine, which offers online, self-paced introductory courses, said on Thursday that students would not need to take classes with StraighterLine in order to sit for the tests. But he hopes that, for students who do take both classes and tests, the scores on the test will help validate StraighterLine courses.

StraighterLine doesn’t grant degrees and so can’t be accredited. It depends on accredited institutions to accept its credits, which has not always been an easy task for the company.

“For students looking to get a leg up in the job market or getting into college,” Mr. Smith said, “this will give them a way to show they’re proficient in key academic areas.”

Mr. Smith said many details of the program, dubbed My Line, still need to be worked out, including price. He expects the cost of a test will probably be under $100.

This entry was posted in Uncategorized. Bookmark the permalink.

E-mail

11134078

I am delighted to learn there is a test to measure whether or not I can “think critically.” I wonder how Walt Whitman would have scored. Or for that matter Einstein.
archman

It depends largely on whether or not Walt Whitman or Einstein would have access to the test bank questions beforehand. Standardized testing is an amazing strategy for enabling cheating behavior.
3224243

Critical thinking can be learned and having access to a test bank could certainly help a student approach problem-solving in a different way. I’ve been buying puzzle books for years that have helped me become much better at tackling all problems with a more critical mind.
josephofoley

Mr. Smith continues his individualistic efforts to realize the potential of online instruction. His accomplishments almost obscured by a chorus of naysayers, he has developed useful online courses demonstrating that basic education can be delivered cheaply and at a profit. I wish him success as he tries to make meaningful evaluation more widely available. At the very least, he is one bright star in the sorry sky of proprietary education.
betterschool

Anyone interested in these kinds of assessments is advised to examine closely the conceptual and scientific validity of the current generation of instruments purporting to assess “critical thinking” in an unselected college population.

Under even a weak notion of “critical thinking,” these instruments lack validity. Publishing houses vitiate the potential for scandal and even sanctions through disclaimers and fine print in which “critical thinking” devolves to “thinking logically,” the latter being a necessary but not sufficient condition to many conceptions of critical thinking. (If “thinking logically” is what we want to assess, there are much better tools available.)

Here is what science tells us (the rest is hucksterism): (a) the construct “critical thinking” is family resemblance in nature (i.e., there are no non-trivial criteria that run through all cases and contexts in which the label “critical thinking” is appropriate, (b) (from ‘a’) the definition of “critical thinking” is context-dependent, (c) when focused on the construct, no instrument at play today possesses validity statistics adequate to support high-stakes decisions, (d) even allowing for substandard validity statistics, no instrument on the market today is useful across an unselected college population, and (e) there are some contexts, narrowly and well defined, for which adequate assessments of critical thinking have been developed; generally, these assessments focus on authentic activities, not M/C tests.

Considering all of the above, it is generally unethical to purport that one can assess critical thinking validly and especially to take money for having done so. Imagine the public outcry if it were revealed that costly cholesterol tests bore no valid relationship to an individual’s cholesterol values.
lizgibbons

I agree. We have a faculty member in our philosophy department who claims that only faculty in the philosophy department can teach critical thinking. I sincerely doubt that he would be able to address critical thinking in dance (or music, for that matter), in which “thinking in movement” (see Sheets-Johnstone”) is primarily non-verbal in nature.
betterschool

I would advise your colleague in the philosophy department to review Wittgenstein’s concept of “family resemblance” and see how well it fits the various things there are to mean by “critical thinking.” I understand music well enough to see the differences between CT there and how it applies to my analytic world. I cannot begin to comprehend dance, except to believe that it must be different as well.
bscmath78

“. . . the study provides only very weak evidence of correlation of test scores for instruments measuring similar constructs. In particular the CLA shows only weak evidence of convergent or divergent validity between the CLA and the MAPP and the CAAP.”

“In short, it is troubling that there is no systematic relationship between the scores generated for the same students performing tasks designed to showcase similar skills (writing and critical thinking) within a similar time frame. At worst, it calls into question the validity of the instruments, and at best, it highlights the differences between the constructs and intentions of the instruments.”

Both quotations are from “Cohort V Final Report – University of Cincinnati” http://ncepr.org/finalreports/cohort5/UC%20Final%20Report.pdf

The first quote references the “Test Validity Study (TVS) Report” which seems to show unimpressive individual student-level correlation. It found only 33.64% of the CLA PT score and 22.09% of the CLA CA was predicted by the individual student CAAP score. See Table 2a on page 24 of http://www.voluntarysystem.org/docs/reports/LearningOutcomes/2009/FIPSETVSReport.pdf and square the corresponding correlation coefficients.

What correlation was observed may just be related to the correlation previously observed between the ability to guess the correct answer to a reading comprehension passage question WITHOUT even see the passage (!) and the SAT-V score. Which suggested that SAT-V was measuring the ability to effectively guess strategically.

To summarize, testing the same student around the same time for the same type of things shows that the tests do NOT produce very consistent results, which suggest that they are NOT very useful.

This is yet another reminder of Inconvenient Truth #15: Many Vocal Professors Study and Learn Little About Student Studying and Learning.

“Garbage In, Garbage Out (GIGO)” seems to be a principle that is repeatedly ignored, which implies a fundamental lack of “critical thinking.”

“Academically Adrift” is based on Collegiate Learning Assessment (CLA) results. Some other studies are based on the Collegiate Assessment of Academic Proficiency (CAAP). Yet they seem unreliable for measuring individual “student achievement in the areas of critical thinking, effective writing, and problem solving.” They seem to correlate poorly with each other and do not seem to provide consistent results for the same student. It seems remarkable that so many are so gullible.
bscmath78

CLA scores are easily misused, even if you believe that at test like CLA provides reliable, consistent scores for an individual that actually accurately measure something useful, which you shouldn’t.

Dubious use of CLA is illustrated by “Academically Adrift” and “Improving Undergraduate Learning: Findings and Policy Recommendations from the SSRC-CLA Longitudinal Project” by Richard Arum, Josipa Roksa and Esther Cho:

http://highered.ssrc.org/files/SSRC_Report.pdf

which it says “extends findings” of “Academically Adrift” by Arum and Roksa.

It shows that “Predicted CLA scores, by institutional selectivity” differ by less than 6% between “High Selectivity” and “Low Selectivity” colleges. Please note the old trick of having the graph’s Y-axis start the CLA score at 1080, instead of zero, which greatly exaggerates the difference between High Selectivity and Low Selectivity scores.

So it appears that paying 400+% more may result in a benefit of 6%! Which ignores the likelihood that 6% is less than the variation that you would expect in scores from the same individual running the test on 3 days. Which would mean that there is no real difference.

Even if you believe the 6% is real, it seems to imply that “High Selectivity” colleges fail abysmally at selecting better students and then hardly improve them much, which would seem to be quite the scandal. Yet strangely enough, in the CHE and elsewhere I don’t see anyone other than myself pointing this out or commenting on it.

For further discussion of the dubious use of CLA scores:

http://chronicle.com/blogs/innovations/12-inconvenient-truths-about-american-higher-education/31282#comment-414583460

Other posts in the same thread illustrate the dubious use of SAT Verbal and Mathematics scores.
bscmath78

betterschool, I generally agree with your criticisms of these tests. I would add that depending on your understanding of “thinking logically” the ability to create/understand/apply mathematical proofs or code bug-free computer code or understand symbolic logic are true tests. Would you agree?

But these are capabilities that are not directly related to gullibility, skepticism, innovation, creativity, debunking rhetoric, sophistry, cons or avoiding “BS baffles brains.” Copernicus, Kepler, Galileo, Newton, Einstein, Gödel and Turing illustrate some very valuable forms of “critical thinking” that are quite different from those promoted in English departments, which seem to result in many unhappy adjuncts.

Newton illustrates how an individual can be excellent in some areas, bizarre in others and stupid still others. It is said that the only thing that Newton as MP for Cambridge University said in Parliament was, “The window needs closing.” There is also the story of his installing separate cat-doors for his cat and her kittens.

A test of algebra, calculus, geometry, symbolic logic and computer programming would work quite well to score an “unselected college population” in useful applications of logic. If they didn’t take these things in high school there are remedial courses in college. What would be your concerns with such a test with the definition that “critical thinking” is restricted to that which is productive in a white collar, computer and data driven, modern Fortune 500 bureaucracy?

Outside of academia, excellence in “critical thinking” in Analytic Philosophy typically isn’t prized. So your Wittgenstein example from a 2011 thread really doesn’t apply outside the academic environment. Excellence in “critical thinking” in many academic areas is likely to be undesirable in the regular business working world, while recognizing that a few with Analytic Philosophy can do well in other environments, though most of those will have been at Harvard, Yale, Princeton, Stanford or a few other places that would imply they had very different initial characteristics compared to most students.

In another post in this thread, I have provided a source that questions the validity of CLA and other instruments at an individual level. I was wonder if you might post the sources that were the basis for the comments you made.