Are Graduation and Retention Rates the Right Measures?

Trudging 10 Miles Through Snow to School ...

August 23, 2009, 09:53 AM ET

Is It Time to Get on Board With Online Education?

In several recent interviews and blog posts, I've expressed my hesitation about the move toward online learning in higher education. My concerns are fairly common ones and go like this: How do we know that students are engaged, or even awake, when participating online? How do we know that online learning is as effective as classroom learning? How do we know that any negative consequences outweigh the cost savings? And what exactly are those cost savings? (After all, technology isn't cheap.) And finally, despite claims to the contrary, the digital divide still exists -- so how do we know that low-income and rural populations will get the access to online learning they need?

Admittedly, I'll always be forced to note that for most of these big questions there's little evidence to the contrary -- e.g. we don't know much about the effectiveness of classroom learning in higher education either, we don't know its relative cost-effectiveness, and we don't know how many are left out of higher education because they can't make it to a classroom setting.

But, in this case I've tended toward the traditional and in some sense the sociological -- prioritizing the value of in-person face-to-face social interactions over online ones, and assuming that more mentoring occurs in an in-person relationship, adding value to the instruction. So, I tend to say things like "the move to online education is premature" and "we need more evidence."

OK, so this summer the U.S. Department of Education came out with a decent response in the report "Evaluation of Evidence-based Practices in Online Learning: A Meta-Analysis and Review of Online Studies." It came out in May -- yes, I'm late to the game here (but honestly, the thing is 93 pages and I read it cover to cover before writing this post). In typical What Works Clearinghouse fashion, the authors pay detailed attention to the methods used in each study they reviewed, and I'm very comfortable with the standards of evidence employed (though I have to note, not every study was peer reviewed -- many were dissertations). They also took care to distinguish between the populations considered in each study (e.g., k-12 versus higher education), and the type and quality of online education examined.

This report taught me the following: (1) There's been much more assessment of the effectiveness of online learning in higher education, compared to K-12. (2) Student outcomes of online learning seem to be somewhat better than those of classroom learning -- but it's not exactly clear that apples and apples comparisons are being made, mainly because the amount of actual instructional time in online courses is greater than that in classroom settings. As the authors write, "Despite what appears to be strong support for online learning applications, the studies in this meta-analysis do not demonstrate that online learning is superior as a medium. In many of the studies showing an advantage for online learning, the online and classroom conditions differed in terms of time spent, curriculum and pedagogy. It was the combination of elements in the treatment conditions (which was likely to have included additional learning time and materials as well as additional opportunities for collaboration) that produced the observed learning advantages."

In some sense this is the kind of evidence I wanted to see in order to be a bit more comfortable with the accelerated pace towards online education. Yet at the same time, I'm not convinced. Apart from the caveat regarding the actual medium, mentioned by the authors above, another reason is that while the authors are right that most of the studies of online learning aren't in K-12, it's also clear from reading the bibliography that they're not in typical undergraduate education either. The meta-analysis is dominated by studies of students in undergraduate education, yes, but of students who have declared their major in undergrad and are taking a specific kind of course (e.g. nursing). (I count only five to seven studies in this meta-analysis that involve more entry-level courses, or those for struggling learners.) I'd argue this is a very specific, more highly motivated group of adult learners than the folks that a scale-up of online instruction in undergrad education is bound to reach.

Reading between the lines a bit, it seems clear that the U.S. Department of Education won't be motivated to fund more evaluations of online learning outside of K-12 in the near future. I think that would be a mistake -- we need to know more about which kinds of online learning work for which undergraduates and under what conditions. We also need to know more specifics about both costs and impacts, allowing for judgements to be made in a cost-effectiveness framework. In the meantime, however, I'm a bit more convinced that online ed is a reasonable way to move forward in solving crowding problems in specific majors, particularly with more advanced students.

E-mail

Comments

1. 11167997 - August 27, 2009 at 07:58 am

Niether my friend, Sara, nor the U.S.E.D. study took up the role of online learning on access and degree completion, along with its position in attendance patterns, all of which we need to know a great deal more about independent of both content and effectiveness. Educated hunch: it's become damned important in reaching folks who could not enter the higher education system any other way; it's got enormous potential for those who had started college years ago, left without earning a degree of any kind, and who could be brought back to finish; it's a critical part of the landscape of part-time attendance, without which our access rates would be miserable. Think of the 300-350 thousand active duty military taking higher education courses through the Voluntary Education Program: with all their redeployments, do you imagine they are sitting in brick-and-mortar classrooms? How else do you work on your accounting degree from a submarine in the middle of the Pacific? For adult populations, a degree of autodidacticism and more than a dollop of persistence is necessary to get through such an experience, but aren't those characteristics what we desire to engender in all our students?
Lots of questions here, and the research community doesn't have to wait for the Feds to sponsor investigation.

2. mheffley - August 27, 2009 at 07:58 am

I've taught primarily the "the entry-level courses, or those for struggling learners" mentioned above (basic writing and Humanities survey courses) for the U. of Phoenix's version of online Community College, Axia College, for 3 years. Before that I taught similar courses to similarly open-admission students at Rutgers, in on-ground classrooms. In terms of effective teaching and learning of those particular subjects, I think the online modality is better than on-ground.

The two student bodies differed mainly in age, Axia including older adults (maybe roughly half) as well as the more typically college-aged. Other demographics seem similar: international, and a skill-level spread of about 20-40-20 below/average/above.

Once I got used to teaching all online, and developed the tricks, techniques, and voice appropriate to it, it was clear that its strongest potential is the same as that of a language immersion course: we all have to do everything in print, not in person (talk, body language, etc.). What better, more holistic way to grasp and shore up grammatical and artful literacy (which is the main and most crucial thing I end up teaching in the Humanities as well as the Writing courses)? I model as well as expound on it, constantly. My students get to work on it in the comfort of their own homes and schedules. Their weaknesses in both the skills and general discipline come to the fore immediately, to be faced and addressed before they can proceed; their strengths can grow just as immediately, without being dragged down or delayed by a group slog.

I suppose many subjects would be better taught in on-ground or mixed ("flexnet") modalities, but the most basic platform of literacies and expository skills I've taught the most, to the most in need of them, is the one I've come to feel best constructed all online.

3. 11167997 - August 27, 2009 at 07:59 am

Forgot to add an ID to the comment above: Cliff Adelman, Institute for Higher Education Policy.

4. abmcg - August 27, 2009 at 06:56 pm

Just for information, the link to the actual report above is broken; the report is at http://www.ed.gov/rschstat/eval/tech/evidence-based-practices/finalreport.pdf

5. mselliem - August 27, 2009 at 09:05 pm

Part 1 of several. I had a lot to say on this topic.

If one browses the catalogs of any major publisher of university-level texts, one will find quite a few titles related to teaching online, ranging from creating learning environments, to helpful materials and methods, to types of online platforms, to the redefinition of an instructor's role. Clearly, there is a trend that is moving us toward online teaching-learning environments. Some of us want to jump on this bandwagon right away, while others need more information, or need better research, or otherwise find excuses not to learn about a very different way of interacting with students.

6. mselliem - August 27, 2009 at 10:58 pm

Part 2 of several, but actually the last and a link.

I don't know if it's the size of my response, or if there are other issues with the web site, but I've decided to post my response on my blog site.

Instead of giving this piece-meal, feel free to view my response on my blog at http://blog.emillereducation.com . Thanks!

7. 11122741 - August 28, 2009 at 12:03 pm

In a variety of data I have reviewed from studies of on-line learning there are strong indications that there are many student-charatceristics-treatment-types interactions going on that none of these research (or this review) formally examine explicitly and closely. The strong suggestion is that on-line works for some and really bombs for others (recall Cronach and Snow's classic work on ATI's). The long and the short I believe is that one cannot generalize in any significant way about on-line instruction (yet) because of the design flaws in the studies and the essentially behaviorist view of learning and instruction inherent in the studies and reviews being done. The interaction question is the key and whether the interactions are prior learning based or more hard-wired or both ...a flaw very similar to studies in medicine until about a decade ago; we are not all the same although I know it may appear that way to many looking at a computer screen rather than actual live and reacting eyeballs

8. mselliem - August 28, 2009 at 02:41 pm

Comment to 11122741--

Proprietary online colleges have this data. The software platforms used for their online programs keep track of interactions as well as key strokes and time spent on task. Granted, the quality and substance of the interactions may not be measured, nor can the platforms measure the time students spend printing material and reading it off-line, but there really is a lot of data that can be analyzed.

The question is, why aren't more for-profit colleges sharing this data?

9. eelalien - August 29, 2009 at 03:15 pm

Online learning has been defined as a "disruptive innovation", and it is dependent on the very technologies that some people fight as being "questionable" in the learning environment. Well, welcome to the future - online teaching and learning simply bypassed all those STILL fretting and gnashing their teeth at the idea of children being exposed to computers and the Internet. The needs are there, the solutions are available, and social research, as opposed to scientific or medical research, has never prevented needed change from occurring. And, as our system of education is in desperate need of change, it truly is a case of either you are part of the solution, or you are a part of the problem.