next »

[ghillbilly]

I'm reading back issues of The Chronicle and the article on how universities are influencing their rankings on U.S. News and World Report was interesting.  The article was by Morphew and Taylor in October.  This came as a surprise to me...do universities not publish learning outcomes??  How is reputation and quality determined then? 

[svenc]

Someone can correct me if I'm mistaken, but I don't know of any rankings that are based substantially on LEARNING outcomes.

Reputation is indeed a big part of many of these rankings.  Others get at more objective measures of RESEARCH (impact factors, grant dollars, pages per faculty, etc.) or at alumni JOB MARKET outcomes (% employed in desired field, starting salaries, average salaries, etc.).  But neither of these are even close to good proxies for learning outcomes.

Moreover, I can't imagine that most university or department learning assessment plans would lend themselves to good measures, and even for those that do, comparability across institutions would be impossible.  I certainly don't see how this can change any time in the future, since most institutions seem to deal with such things by telling faculty (who are rarely trained in such matters - I know I am not) to come up with and implement assessments of learning with no additional resources to do so.

So no, the rankings don't reflect learning outcomes, and they are very unlikely to do so at any point in the foreseeable future.

[kedves]

I don't know of any rankings system based on learning outcomes, but this proposed new federal office might be something like that.  I think it would be hard to standardize "learning outcomes" in a way that was useful for administrators, parents, or students.

The US News rankings are a business of that publication, not of the colleges and universities in it.  Here is information about the US News system, including classifications, variables, and weighting.

At the undergraduate and graduate level, much of a university's reputation is based on the students it attracts and graduates, who apply to the place based on its reputation.  Quality, measured by test scores of students or selectivity of institution, is self-perpetuating to some extent unless something changes.  Higher education tends to reflect and maintain existing social differences (including the range of quality of K-12 education) rather than changing those. 

Universities collect and report many measures of goal achievement, including but not limited to the education of students.  Remember that research--the discovery and communication of knowledge--is a mission of many universities.  Service to the state or to the community may be important as well.   Outcomes are defined and measured within institutions in terms of mission, or within institutional types, rather than being comparable across all institutions.

[svenc]

I don't know of any rankings system based on learning outcomes, but this proposed new federal office might be something like that.  I think it would be hard to standardize "learning outcomes" in a way that was useful for administrators, parents, or students.

Maybe we can have rankings based on the fewest number of parent complaints to Assistant Deans of Student Coddling!

[larryc]

Ivy league schools are based on the amount of ivy on the walls. Teams of researchers fan out across campus with calipers and transits and such and rate the ivy for stalk size, height, lushness and imponderable "ivyness factor" of the plant.

All other schools are rated by their football rankings.

[mad_doctor]

Think about it - if you're an education bureaucrat or administrator, there is no incentive to measure actual learning outcomes, because it can only make you look bad.  Specifically, colleges and universities are expected to provide a good quality education, so a report of good learning outcomes gets a response of "So what? You're supposed to be doing that.  Congratulations for doing your job", while bad outcomes make you look bad and get you in trouble.  This is why they're interested in things like retention, graduation rate, and all those other meaningless measures of "quality".

Reputation tends to be more about the size of the endowment and the volume and quality of faculty research.  "Quality" is whatever the administrators at that school choose to call quality.  If they want to define quality as high retention and graduation rates, "high-touch", lots of student services and activites, beatiful modern dorms, and good food, then they're going to go around bragging about the "quality" of that experience and recruit students who agree with them that's what quality is all about.

[kedves]

This is why they're interested in things like retention, graduation rate, and all those other meaningless measures of "quality".

If part of the mission of the institution is to give students (and other payers of the costs of education including state taxpayers) a good return on investment, then retention and graduation rates are very meaningful.  There is a modest difference in unemployment and earnings comparing "high school graduate" and "some college," and a big difference between "some college" and "college graduate" (figures here).

That issue is separate from the issue of how much colleges and universities can affect retention and graduate rates. 

[fizmath]

I think there are different systems based on who is doing the ranking.  Chinese students who want to study in the USA have their own system.  You can search around and find them.  It is interesting to see how they rank a school.  One factor is the number of Nobel laureates and Fields medal winners working at the school and also those who are school alumni.

I once saw a ranking of business schools compiled by HR professionals.  It had little resemblance to the normal top 20 list.

[voxprincipalis]

I once saw a ranking of business schools compiled by HR professionals.  It had little resemblance to the normal top 20 list.

Many of the things that reflect the point of view of HR professionals have little resemblance to the way things actually happen in academia.

VP

[aandsdean]

I once saw a ranking of business schools compiled by HR professionals.  It had little resemblance to the normal top 20 list.

Many of the things that reflect the point of view of HR professionals have little resemblance to the way things actually happen in academia.

VP

I'd REALLY like to see this list.  It would have to be fascinating.

[prof_smartypants]

How would one rank colleges and universities on learning outcomes? Standardized tests?

They already include things like graduation rates; graduate programs usually highlight placements; but I wouldn't call those learning outcomes.

I'm not saying that the current ranking system isn't a load of BS, but I'm not sure how you would go about ranking schools based on learning outcomes with existing data. 

[the_honey_badger]

I once saw a ranking of business schools compiled by HR professionals.  It had little resemblance to the normal top 20 list.

Many of the things that reflect the point of view of HR professionals have little resemblance to the way things actually happen in academia.

VP

Me too! I'd pay good American *money* to see a copy of that.


I'd REALLY like to see this list.  It would have to be fascinating.

[marigolds]

How would one rank colleges and universities on learning outcomes? Standardized tests?

They already include things like graduation rates; graduate programs usually highlight placements; but I wouldn't call those learning outcomes.

I'm not saying that the current ranking system isn't a load of BS, but I'm not sure how you would go about ranking schools based on learning outcomes with existing data. 

I'm thinking that it sounds a f*** of a lot like No Child Left Behind's standards (= learning objectives) and end-of-grade testing (=learning outcomes.)  I can't imagine that university administrators would make less of a hash of that than the NCLB designers did.  (Ed.D's, every one, I'm sure.) 

[sciencephd]

How are HR professionals asssssssessed ?