Jumping to Conclusions

In a recent post on Education Week’s blog, Debra Viadero offers a caution about President Obama’s support for community colleges. Pointing to her recent article on community college research that indicated how much more we need to know about how best to improve completion rates in that sector, she questions whether the president would be wiser to place his bets on career colleges. She says that a recent study by the Educational Policy Institute (EPI) and an ongoing program of research by James Rosenbaum and colleagues support her contention that community colleges ought to take cues from career colleges.

In my opinion, this talented reporter is jumping to conclusions.

Yes, the graduation rates at two-year for-profit colleges exceed those at two-year public colleges. No one disputes that. That does not necessarily mean, however, that career colleges are outperforming community colleges, or that community colleges should take steps to become more like career colleges. The plausible alternative explanations for the differences in results are numerous.

For example, the students attending the two types of colleges may differ in important yet unmeasured ways, ways that are associated with chances of graduation (what researchers refer to as selection bias). Is one group more economically or educationally advantaged? More motivated? More apt to have a family, nighttime work, or receive tuition support from an employer? It’s also possible that the differences in graduation rates stem from constraints that community colleges face but career colleges do not — for example, inadequate resources or a lack of control over mission or governance. It’s one thing to point to differences in practices between the community colleges and for-profit colleges, and another thing to attribute those differences to variation in the “will” or intentions of practitioners, rather than attribute them to underfunding and all that comes with it.

Establishing that community colleges have poorer graduation rates than career colleges for reasons they can and should do something about requires evidence that the two differ on one or more key aspects that is causally linked to college completion. Say we knew that smaller classes caused better student retention and community colleges have larger classes than career colleges. We’d then be able to say, there’s something community colleges ought to fix. But we don’t have evidence that that’s the case.

Instead, research simply establishes that (a) career and community colleges have different graduation rates and (b) career and community colleges (sometimes) employ different institutional practices. Rosenbaum and his colleagues have done a nice job, as Viadero notes, of documenting the latter — using qualitative methods, mostly at colleges in the Chicago area. But they have not shown that those practices cause observable differences in graduation rates.

Moreover, while they’ve produced one paper indicating that differences in the student populations at career and community colleges do not appear to account for disparities in outcomes, that analysis is based solely on a limited set of observable characteristics, and therefore don’t rule out the possibility that different levels of student motivation, for example, are really the culprit. Just think about how students get to college — many at career colleges are actively recruited (sometimes off their living room couches) while many at community colleges effectively wander in the door. Why would we think, then, that career and community colleges are serving the same kinds of people and producing different results?

There’s another consideration Viadero neglects, and that’s college costs. Students at career colleges leave with far more debt than students at community colleges. Data from the 2007-2008 National Postsecondary Student Aid Study reveal that 61 percent of community-college students graduate with less than $10,000 in debt, compared to only 22 percent of students graduating from two-year for-profits. In contrast, 19 percent of graduates from two-year for-profits have $30,000 or more in student loans, compared to only 5 percent of community-college graduates.

Nearly all students (98 percent) finishing at two-year for-profit colleges have taken on a loan, compared to just 38 percent of community-college graduates. Is that a problem? Is it offset by higher rates of graduation? The answers are far from clear. Absent better ones we shouldn’t be relying on evidence like EPI’s — a study of career colleges’ high graduation rates that was supported by the Imagine America Foundation, formerly the Career College Foundation, established in 1982 as the research, scholarship, and training provider for the nation’s career colleges. Full text of that study wasn’t even placed online for researchers to fully vet!

Community colleges have a long, rich history of serving this nation. Sure, there’s room for improvement, but without more solid evidence of which changes are needed let’s not jump to conclusions and tout the for-profits as a model to which they ought to aspire. We might end up in a bigger mess than we’re already in.

POSTSCRIPT:

I have now obtained a copy of the full EPI report.  My suspicions were correct: the authors use nothing more than simple descriptive comparisons of students’ characteristics and degree completion rates (calculated using NCES’s DAS system, likely without propering weighting) to support their causal claims about the “benefits” of attending community college. For example, they write “The report suggests that career colleges work harder to provide appropriate student services and support” but present no data on institutional services or effort expended, particularly any tied to student outcomes.  Their final conclusion — “statistically, not only do students attending career colleges perform as well as or better than many other students attending comparative public institutions, but they persist in and complete their education while typically being more economically, educationally and socially challenged than other students”– is based on nothing more than comparisons of sample means (no regression, no nothing).  C’mon folks, this ain’t the kind of research any consumer ought to be taking seriously.  Glad to see Kevin Carey agrees.