In the ongoing conversation about institutional quality, I continuously see references to retention and graduation rates as “the” — or at least “a” — key indicator of institutional quality. But what if we have it all wrong? What if measuring graduation and retention rates as evidence of institutional quality is like taking a person’s temperature to evaluate the efficacy or effectiveness (these are experimentally different measures) of his blood pressure medication?
While it may be true that, at the margins, some institutions are more successful at retaining and graduating disadvantaged students than others, and we should certainly study those institutions to find out what they do “right,” in general, retention and graduation rates correlate more closely to student socioeconomic status and family situation than they do to institutional type, sector, or quality. Even at the highest-ranked institutions where drop-out rates are low in general, disadvantaged students are far more likely to leave without a degree than are their more advantaged peers.
Schools that serve a large number of wealthy students can win the numbers game when graduation and retention rates are reported as averages among the entire student body. Conversely, schools that serve large numbers of disadvantaged students have nowhere to hide.
No matter how rigorous the curriculum, dedicated the faculty, or plentiful the academic support services, a student who is working many hours or juggling demanding family obligations while attending school may simply lack the time to take advantage of the services offered or even to complete the work assigned. Learning is not a spectator sport, and no college administrator — no matter how insightful, dedicated or motivated — can create more time in the day of an overcommitted student. If the student does not have time to participate in learning, he or she is unlikely to be successful.
Perhaps the real reason that retention and graduation rates are so low among disadvantaged students is that the Federal financial-aid system is based on the principle of giving a little bit of money to a lot of students rather than giving enough money to any students. There are real political advantages associated with this strategy.
In pharmacology, scientific studies carefully determine the effective dose of a drug — one that produces a therapeutic response in 50 percent of the people taking it. Perhaps it is time to coduct a scientific study to determine just how much aid is required to produce a 50-percent cohort graduation rate among each incoming class of Pell recipients. It may be hard — even impossible — to provide funding at a level that makes a difference, but at least we could be honest about the real cause for the low graduation and retention rates achieved by the neediest students. We might also, then, be able to have an honest dialogue about institutional quality and assessment — one that doesn’t make underdogs out of the very institutions working hardest to provide a glimmer of hope to the neediest of students.
(Brainstorm illustration incorporating photos by Flickr users ausiegall and amer3000)
12 Responses to Are Graduation and Retention Rates the Right Measures?
comosifuera - August 22, 2009 at 3:02 pm
Rating institutional quality by graduation and retention rates often benefits colleges whose curricula are not very demanding. Moreover, it certainly disadvantages colleges with very rigorous academics. I graduated from a school that placed a high priority on providing a world-class education, at the expense of admitted students who were less equipped for such a workload. I applaud my alma mater for doing this, because it gives students without elite admissions stats the opportunity to have an elite education. Given this disconnect between its admissions profile and its academic quality, a high graduation rate could only be achieved by watering down the course loads. I know that French universities have an open admissions policy for their medical schools. Practically anyone can go, and the difficulty of the exams weeds people out effectively. By global standards, French health care is superlative. There’s an old saying, “nobody fails out of Harvard.” Because of the inordinate focus on graduation rates, it is in the interest of top-tier schools to do what they can to keep their students– even if that means being overly lenient.
goxewu - August 23, 2009 at 9:01 am
“But what if we have it all wrong?”Now she asks?
epacchetti - August 24, 2009 at 8:55 am
Great article! I like the idea of a study to figure out what it would take to reach a 50% graduation rate for Pell Grant recipients. Likewise, instead of judging schools based on their graduation rate, where schools who admit the “best” students will always excel, how about estimating what a college or university’s graduation rate should be based on the number of low-income students they enroll, and then comparing that to their actual graduation rate. A measure like this would not penalize students that enroll a large number of low-income students because their “expected graduation rate” would be lower in the first place.
tridaddy - August 24, 2009 at 9:29 am
This is certainly a timely topic as many states are now politicizing retention and graduation rates in their funding formulas for universities. Unlike industrialized production that can apply Six Sigma or some other metric to efficiency and loss, human beings don’t function like inanimate materials. Take wood and cotton into a factor and out comes furniture and cloth. Politicians and the general public seem to think education works the same way. Do we need to evaluate retention and graduation rates and look for ways to increase them? Yes. However, we need to be cautious regarding the issues the inputs (students) drag with them to our institutions.
pattpeterson - August 24, 2009 at 9:42 am
Retention and graduation rates at the university or college level may not yield much, but the same statistics at the program level are valuable, especially when trended over time – There is a growing phenomena of online for-profit colleges running a program with 900 students when the norm is well under a 100 students. As you would imagine, the retention and graduation rates in these programs are extremely low. Without the threat of accreditation focused review there will be no incentive to improve a set up that is all about generating revenue, with little concern for student learning.
yorklibrary - August 24, 2009 at 9:54 am
I think we already know how to improve graduation rates among disadvantaged students. HEOP and SEEK programs routinely graduate students at or above the average rates for their institutions at large.
dank48 - August 24, 2009 at 10:20 am
It seems to me that “measuring graduation and retention rates as evidence of institutional quality” of a school is like measuring survival and recovery rates as evidence of the quality of medical treatment. How do you compare the results of the treatments provided by, say, a Park Avenue specialist in “dis-eases” of the wealthy, pampered, and underexercised to the results of the treatments provided by a battlefield medic? Obviously, you don’t. I fail to see (a) how any meaningful comparison can be made and (b) to whom it would be meaningful and for what legitimate purpose.
sisgett - August 24, 2009 at 10:44 am
As long as graduation and retention rates have to be reported to the DOE, they will be used as a convenient and misleading comparison tool between unlike institutions. Institutions that admit based on high selectivity will always have higher rates. Never mind that selectivity is a function of social opportunity. Students from the middle and upper socio-economic groups will have the leg up they don’t need. Ranking systems, such as U.S. News, will continue to perpetuate the idea that they really identify the “best” colleges. Everybody’s ticket gets punched, and nothing really changes. These statements are not an argument against reporting graduation and retention rates, just that those rates mean much less than most folk assume, as the article states.
lynnosh - August 24, 2009 at 2:25 pm
Graduation rates — good and bad — at just about any institution can be misleading. When I’m looking at grad rates, I like to check out the Education Trust’s CollegeResults.org. It provides four,five and six-year rates, but just as importantly it breaks down those rates by ethnic groups — African-American, Hispanic, Native American Indian, Asian and white. It also provides male and female grad rates, which are often quite different.Lynn O’ShaughnessyTheCollegeSolutionBlog.com
rburns - August 24, 2009 at 3:05 pm
Degree completion and graduation rates are only part of the story. Growing numbers of students attend to gain specific course sequences, certificates or just skill sets they need for careers, employment, promotion, enrichment, etc. So, Student A enrolls (goes through the formal degree application process just to gain admission), takes the 24 credits she needs for a promotion at the civil engineering firm, and when she completes her goal, we list her as a drop out. She came for a specific reason, got what she was looking for, and went away happy. She may return when she recognizes yet another need. Why would we count that a failure for the institution or program. It is more difficult to track such students, but it needs to be done.
diehl - September 6, 2009 at 6:46 pm
Diane, could you please cite your data source for the following? “in general, retention and graduation rates correlate more closely to student socioeconomic status and family situation than they do to institutional type, sector, or quality. Even at the highest-ranked institutions where drop-out rates are low in general, disadvantaged students are far more likely to leave without a degree than are their more advantaged peers.”Thanks.
diehl - September 6, 2009 at 6:46 pm
Diane, could you please cite your data source for the following? “in general, retention and graduation rates correlate more closely to student socioeconomic status and family situation than they do to institutional type, sector, or quality. Even at the highest-ranked institutions where drop-out rates are low in general, disadvantaged students are far more likely to leave without a degree than are their more advantaged peers.”Thanks.