next »

[tortugaphd]

What do you think of the article on outcomes assessment?

Somehow, I'm right on the ball with the author.

[anthroid]

Yeah, me too.  There are interesting points here about deconstruction, post-structuralism, and all that (as we called it in grad school) litcrits*** leading to outcomes assessment.  I'm in the "this is a passing phase" camp myself.

[juliorules]

What a wonderful article! And my God the woman can write:

"For all the highfalutin pronouncements accompanying the current May Day parade of outcomes assessment, in the end they boil down to a wholesale abandonment of the very idea of higher education. Whatever their purpose, outcomes-assessment practices force-march professors to a Maoist countryside where they are made to dig onions until they are exhausted, and then compelled to spend the rest of their waking hours confessing how much they've learned by digging onions. The mentor-protégé model of a college education is gone. We now confront the robot model, in which knowledge is reduced to what Nietzsche called "knowledge stones" — bits of information that administrators can count and students can digest without thinking."

[trabb]

What a wonderful article! And my God the woman can write:

"For all the highfalutin pronouncements accompanying the current May Day parade of outcomes assessment, in the end they boil down to a wholesale abandonment of the very idea of higher education. Whatever their purpose, outcomes-assessment practices force-march professors to a Maoist countryside where they are made to dig onions until they are exhausted, and then compelled to spend the rest of their waking hours confessing how much they've learned by digging onions. The mentor-protégé model of a college education is gone. We now confront the robot model, in which knowledge is reduced to what Nietzsche called "knowledge stones" — bits of information that administrators can count and students can digest without thinking."

While I would agree that the piece is well-written in the sense that the author has command of an excellent vocabulary and is able to use that vocabulary to craft witty, engaging prose, I found the rhetoric of the piece ultimately to be unsatisfying.  As an argument about the problems created by outcomes-assessment, the piece fails because of the extremity of its language.  I have real trouble characterizing any trend in higher education as having nothing whatsoever to offer.  The whole article, in fact, seems to occupy some limbo-like space between a rational critique and a satire.  I have just as much trouble taking seriously someone who completely dismisses the value of outcomes-assessment as I do someone who completely dismisses the value of good old-fashioned teacher-centered learning.   

While the article does raise some legitimate points about problems related to outcomes-assessment in higher ed, the vitriol with which the author approaches the subject smacks of someone who's pissed off because the dean didn't like the way the author and her committee completed the task of writing a report.

[zharkov]

I myself don't buy the author's argument. It seems like a cop out.

To summarize, my understanding of the argument is: Art teaching and learning have a certain je nais se quoi, so don't expect us to actually tell you our program's learning outcomes and how they might be measured.

[outlier]

I myself don't buy the author's argument. It seems like a cop out.

While the article does raise some legitimate points about problems related to outcomes-assessment in higher ed, the vitriol with which the author approaches the subject smacks of someone who's pissed off because the dean didn't like the way the author and her committee completed the task of writing a report.

I'm with Trabb and Zharkov on this. The cop-out, it seems to me, was in not standing up for the decent work she was originally doing, and finding useful, and instead capitulating to the reductionist approach. Anything that has to do with accreditation is going to be the object of power grabs by bureaucrats and factions, but instead of defending the validity of what she was doing, she gave in, then wrote a snarky article about the crappy outcomes assessment that others are making her do. How does that help?

Outcomes assessment can work. Faculty can write outcomes to reflect complex concepts and goals, and they can use assessments that deal with that complexity, and doing that is a world better than the original state of affairs she described:

"With guaranteed salaries and retirement plans unavailable to working artists who struggle to make a living off their art, we've spent decades repeating the self-serving mantra that 'art can't really be taught' and telling our students to go home, grab a No. 2 pencil, and, using only line, bring in a couple of new pieces of ... well, whatever, next class. Then we stroke our chins and say, mysteriously, 'Yeah, this one works,' or 'Sorry, this one just doesn't work for me.'

It's not an either/or choice between the world she described as the original state of affairs and an ETS-like standardized, bean-counter environment. She was doing something that worked; she just seems not to have cared enough or believed in it enough to argue about it. This article comes across as a passive-aggressive snit to me.

[chronanon]

How do you take seriously someone who starts off with a statement like, "No matter how much they purport to be about "standards" or student "needs," they are in fact scams run by bloodless bureaucrats who ... do not understand the holistic nature of a good college education."  and then lambasts faculty who believe there may be something valuable in outcomes assessment as "second-rate teachers, who never really got art in the first place."  If you're able to get towards the end of the essay (which I found a bit too vitriolic and rambling to categorize as very good writing), you see that she had a bad experience that poisoned her view of the entire process in all forms in all places.

Where she really lost credibility for me was when she explained that she was once a proponent of outcomes assessment, and now rails against it in its entirety because she ran first into someone asking her to shift perspective: "...'teacher centered' teaching instead of ... 'student centered' learning," which unfortunately - admittedly - appears to have then evolved into at best overly burdensom bureacracy or at worst administrative micromanagement of the process. 

The irony is she captures the core principle quite nicely: "It didn't seem like such a bad idea in principle. It seemed to mean simply that we could no longer base our teaching on the assumption that because we are active professionals in the art world, our students would automatically learn, by some sort of osmosis, to become artists themselves. Outcomes assessment meant that we would have to figure out if our students were actually learning what we assumed they were learning, or, indeed, if they were learning anything at all. And if they weren't, we'd have to fix the problem."  She even goes on to give a highly encouraging example of how she seems to have led efforts to those ends. 

In the end, the essay comes off to me as someone angry that she couldn't do things her way, irritated by being asked to seriously consider the students' perspective, aggravated by bureacracy, and consequently having a tantrum.  Now instead of trying to make the best of the imperfect system she's in, she's just going to pack up her toys and go home.

The question I'm left with is, other than the therapeutic value of a good rant, what was the point of writing (or publishing) this piece?  There's no reasonable criticism of outcomes assessment per se, just of her institution's handling of it, and there's certainly no suggestion of an alternative approach or even how to address her institution's missteps.

[busyslinky]

She could have said it much better and more clearly in many fewer words.  I was getting very tired reading this article and wanted a stronger and more effective punch line.  Too much meandering and lack of focus. I blame the editor.

[prof_viola]

Sure, there's a lot of b.s. in outcomes-assessment-speak.  But it's here to stay, at least as long as state legislatures believe in it. 

When I directed a writing program, I had to adapt or die, and I have to say that I did find assessment worthwhile.  That may be somewhat discipline bound:  I think that faculty should be able to get together, as my colleagues and I did in an ad hoc committee, and decide (a) what constitutes good student writing and (b) quantifiable ways to measure it.  We devised a rubric--a good one on which I still draw--and we did a week of "norming sessions" in which we agreed on our standards. 

We (to quote our students) "defiantly" did learn.  We discovered strong and weak points in our program, and we improved it.  Best of all, we were able to correlate student progress (from a sample of hundreds of students) with all sorts of incoming indicators that gave us the necessary proof--and won us the funds--to start a basic writing program, something the SLAC desperately needed.  It enabled the SLAC to retain quite a few students, as we were able to prove several years later. 

Assessment can be a powerful tool for figuring out what's working, or not, in our classes, and for demonstrating where resources will make a difference.  So, yeah, I'm a believer. . . .

But I'll leave you with this (written in my head on the freeway while singing along with Louis and Ella this a.m., so still working out the kinks).  You can sing it to the tune of "Our Love Is Here To Stay":

It's very clear
Assessment's here to stay
Not for a year
Forever and a day
Oh, E-Portfolios and Blackboard
And the standards that we know
May just be passing fancies
And in time may go
But oh my dear
Assessment's here to stay
Our outcomes are
Going a long, long way
In time, the rubrics may crumble,
The platforms may tumble,
The networks may decay
But--Assessment's here to stay.
;)

[sibyl]

I'll see your Ella and raise you a Harry James:  It seems to me that I have heard this song before.  Faculty frequently complain about assessment, and frequently encounter pointy-headed administrators who discard well-crafted plans for silly reasons.  Yes, the essay has a certain style, but it's ultimately unhelpful because it doesn't provide any suggestions for what to do next.  (Except the "sleeping bag" comment, which essentially boils down to "make the most of it."  Not a particularly inspirational rallying cry.)

Assessment persists because it has an internal logic to it and because it appears to meet a perceived need for a better explanation of what happens at college.  It may be true that assessment is inadequate to that need, but that doesn't change that various publics -- including state and national legislatures and employers, and yes, some parents too -- possess this need.  It seems to me that if we want to get assessment off our backs we need to find a way to acknowledge the reality of this need, and articulate "the holistic nature of a good college education" in a way that meets this need. 

Opponents of assessment are glad to say what a college education is NOT, i.e. a set of outcomes, but they do not say what a college education IS.  I'd be a lot readier to fight "outcomes" if I had something else I wanted to put in its place. 

[pyshnov]

I told you so. Just a couple of days ago I wrote about cultural revolution Chinese style.
It's amazing that some still are able to wake up.
It's a long article that pays attention to particular hun-wey-bin language of today's academia. And, of course, cripple language betrays cripple intention.

The question is: Why the business of abolishing our civilization became so popular?
My answer (that I think goes one step deeper into the cause) is this:
The culture is acquired in two ways: by brain and by senses. Yet, many were never taught to use their senses. Good taste is rare.

A person who has good taste would never ever accept "outcomes assessment", "deconstruction" and all similar crap. But one who is desperately trying to become an educated intellectual turns to that crap as a method to learn culture. Well, you can learn it this way but you will never feel it...

Yet, mass education has no choice, it gives you a description of culture, not the culture. That's why so many new words, mathematical descriptions, ridiculous attempts to give definitions to things that can be learned only by senses.

And, this is also why it is so frustrating to argue against "deconstruction" by pure logic. You either feel it, or you don't.

The lady who wrote the article is in Art; she tried to argue..., but she should have just said: "Damn you!"

[jonesey]

To summarize, my understanding of the argument is: Art teaching and learning have a certain je nais se quoi, so don't expect us to actually tell you our program's learning outcomes and how they might be measured.

I'm going to go out on a limb here and guess that you don't teach an artistic discipline... (theatre, creative writing, art, etc).  If you did, or if you came from that background, you'd be familiar with her argument, which is not uncommon and, I feel, correct. 

The Arts are not the sciences, where results are accurately measured and calibrated, grades assigned based on correct answers on tests, projects, etc.  The Arts are much more, squishy, for lack of a better term.  This is why most MFA programs don't have letter grades, just Pass or Fail; how do you rate a persons art as good or bad; it's a purely subjective analysis. 

Putting outcome assessments on these kind of classes is challenging, to say the least.  It's even more challenging because many professors in the arts are, by nature of their personality and background/education, diametrically opposed to this sort of cold, calibrated analysis of their classes. 

A person who has good taste would never ever accept "outcomes assessment", "deconstruction" and all similar crap. But one who is desperately trying to become an educated intellectual turns to that crap as a method to learn culture. Well, you can learn it this way but you will never feel it...

Yet, mass education has no choice, it gives you a description of culture, not the culture...this is also why it is so frustrating to argue against "deconstruction" by pure logic. You either feel it, or you don't.

Pyshnov actually makes very good points here, and captures the essence of the challenge with these assessments that, as others have said, are here to stay, unfortunately.

[neutralname]

I've heard other faculty protest against outcomes assessment saying that it cannot capture the transcendant and ineffable qualities of education in arts and humanities, and while such claims might be true, they don't go down well with assessors or administrators.  They prefer to stick within the language of social science/educationese.

I think it might be more effective to argue against assessment on its own terms: i.e., what assessment has been done of assessment, and what evidence is there that it improves education in the arts and humanities?  I've never seen any.

[outlier]

The Arts are not the sciences, where results are accurately measured and calibrated, grades assigned based on correct answers on tests, projects, etc.  The Arts are much more, squishy, for lack of a better term.  This is why most MFA programs don't have letter grades, just Pass or Fail; how do you rate a persons art as good or bad; it's a purely subjective analysis. 

Putting outcome assessments on these kind of classes is challenging, to say the least.  It's even more challenging because many professors in the arts are, by nature of their personality and background/education, diametrically opposed to this sort of cold, calibrated analysis of their classes. 

The arts are not the sciences, but the sciences are not necessarily as easily measured or calibrated as you suggest either; nor are the social sciences. Everyone can concoct arguments about why specifying what you want from students, teaching to that, and assessing whether and how well they've learned it shouldn't apply to their discipline.

It's all challenging. The challenge is not to superimpose a cold, calibrated analysis, but rather to define what you want from students and how you'll know when you see it in a way that works in your discipline. Another challenge is to stay focused on that core principle that the author summarizes:

"It didn't seem like such a bad idea in principle. It seemed to mean simply that we could no longer base our teaching on the assumption that because we are active professionals in the art world, our students would automatically learn, by some sort of osmosis, to become artists themselves. Outcomes assessment meant that we would have to figure out if our students were actually learning what we assumed they were learning, or, indeed, if they were learning anything at all. And if they weren't, we'd have to fix the problem."

And the final challenge is to stand for that in the face of attempts to reduce and distort those efforts. It seems to me that that is where faculty have the most difficulty--they crumple at the first use of the words "mandate" and "requirement" instead of challenging and resisting those terms. And they could resist, because the way that many colleges say and perhaps even believe they are required to do things is a misinterpretation by small-minded paper pushers.

[mountain_ivy]

I think that outcomes assessment has some potentially extremely serious issues for higher education and the perhaps dated notion of academic freedom.

At my institution, OA has been used as s bludgeon by the administration to subvert the process of curriculum development, usually the purview of faculty.

To make a long story short:  OAs have to be approved by the Academic VP;  if not approved by that official, they are revised until they are acceptable to VP.  Once the VP signs off, that OA becomes "law" even if the process by which it was reached is spurious.

This past year, admin used an OA to force revision of general education; the committee charged with gen ed defended their recommendations with "It's required by OA."

OA in and of itself is not evil; however, I can certainly understand the author's anger that the concept of OA has been subverted at her institution--and at mine.  In fact, all studio arts were recommended for removal from gen ed at my school, even though state regs would allow them as part of gen ed.

OA needs to be watched carefully, lest it be misused.  Outliers' final paragraph is well put.