• Sunday, February 19, 2012
February 19, 2012, 01:00:20 PM *
Welcome, Guest. Please login or register.

Login with your Chronicle username and password
News: For all you tweeters, follow The Chronicle on Twitter.
 
Pages: 1 2 [3]
  Print  
Author Topic: Chronicle article on outcomes assessment  (Read 9971 times)
pyshnov
had touched the tip of the iceberg
Senior member
****
Posts: 626


WWW
« Reply #30 on: June 13, 2007, 09:51:49 AM »

Outlier:
Quote
Not to contradict you, but I would suggest considering how the view of teaching as "information delivery" and presentation. Maybe you use these terms as shorthand for the teaching process, but really, where and how does learning happen? Is it simply transfer of information?
I was using shorthand, and this is bad because we start with words and end with words. I stopped being discriminating and used shorthand that I hate myself.

I think this thread is superior to many others. May be finally everybody realised that he or she is an individual and that this is the first thing needed for a teacher, and that this freedom is not to be taken away by making universal instructions for the instructor (using "instructor" here is again a bit of a shorthand). When somebody is trying to give interesting meaning to the "outcomes assessment" is  laudable, because this "assessment" is killed when it is transformed into something original.

I ridicule the words "workplace", "workforce", "human resources" etc. "Teaching process" is just teaching. Unfortunately, people learned that using deceptive words that take your mind in the direction of "change", pays "in terms of" money and promotions.
Logged
outlier
Senior member
****
Posts: 732


« Reply #31 on: June 14, 2007, 01:44:02 PM »

"Teaching process" is just teaching.

Yes. I fell into the jargon pit. Thanks for the rope.

When somebody is trying to give interesting meaning to the "outcomes assessment" is  laudable, because this "assessment" is killed when it is transformed into something original.

Some instructors don't do assessment. They just lecture, and they never look at the students to see who understands and who looks lost. But if they do only that--looking at the students to see where the light is on or off--then they are doing assessment. Assessing is simply gathering and interpreting information. We all do that, all the time. When they give a test, they are doing assessment. When they give feedback on a paper or homework assignment they are doing assessment; when they meet with a student and say, "I've noticed this, and here's what you can do," they are doing assessment. All that can and should count as assessment. Instructors have expertise, and they bring it to bear on the observations they make--that is assessment. When they do that assessment with the goal in mind--the outcome they want-- then they are doing outcomes assessment. And I think they always have the goal or outcome in mind; they know what they want from students, but sometimes students don't know that.

I don't care if the B.A.s in the Official Office of Outcomes, Assessment, Accreditation and All Forms of Burdensome Paperwork say they're not; they still are [and I have a Ph.D., so I know ; ) ]

Each faculty member has a choice, as the original author put it, though I think she left out the most viable, productive option: They can join 'em, becoming the mediocrities and sycophants that Larryc hates (and I am not one! I'm not! I'm not!); they can submit, go through the motions, and wait for the whole trend to pass; they can refuse and just do nothing; or they can do outcomes assessment in a way that makes sense and works for them, which means fighting over terms, definitions and implementation.

I think outcomes should reflect the complexities of what we want from students: knowledge of facts (or where to find them), sure, but also "tacit knowledge," metacognitive skills, an accurate mental model of the discipline--the ability to put all the knowledge and skills together and use them proficiently (or at least at a basic level of proficiency) to do something worthwhile.
Logged
pyshnov
had touched the tip of the iceberg
Senior member
****
Posts: 626


WWW
« Reply #32 on: June 15, 2007, 01:44:12 PM »

outlier,
I never doubted that assessment is taking place all the time, but can you uncover the real reason for the emergence of the new "outcomes assessment" policy beside the reason that it creates jobs for the bureaucracy? Usually, when you see such things, they are created with the goal that the new policy or new study will proclaim results that differ from the facts known before; usually they are more supportive of "progress", and, sometimes, even designed to dispel the well known old facts, declare them a "myth". What it is this time?
Logged
csguy
Distinguished Senior Member
*****
Posts: 1,221

Computer Science faculty


« Reply #33 on: June 16, 2007, 01:39:25 PM »

outlier,
I never doubted that assessment is taking place all the time, but can you uncover the real reason for the emergence of the new "outcomes assessment" policy beside the reason that it creates jobs for the bureaucracy? Usually, when you see such things, they are created with the goal that the new policy or new study will proclaim results that differ from the facts known before; usually they are more supportive of "progress", and, sometimes, even designed to dispel the well known old facts, declare them a "myth". What it is this time?
Let me find my Marxist hat:

Greater control over the faculty. Reduce the faculty to the status of assembly line workers. Reduce the power of academic departments.
Logged
larryc
Hu hatin'
Distinguished Senior Member
*****
Posts: 17,568

Eschew the hu.


WWW
« Reply #34 on: June 16, 2007, 03:05:45 PM »

Outlier, my apologies to you, for both my words and my tone.

There may be value to assessment of which I am unaware. There is absolutely no value to it the way my institution is doing it, and it is being pushed and controlled by the very worst elements on my campus. The author of the article seems to be at a similar place. But I should not ave take that sample of two and drawn such sweeping generalizations.
Logged

outlier
Senior member
****
Posts: 732


« Reply #35 on: June 17, 2007, 01:55:15 AM »

Larryc, thanks.

There may be value to assessment of which I am unaware. There is absolutely no value to it the way my institution is doing it, and it is being pushed and controlled by the very worst elements on my campus.

I'm familiar enough with the trend that I know what you mean; I have been frustrated to attend workshops and hear about others that were full of misinformation presented as requirements, and some people just love to tell others what is and isn't assessment, as if they actually knew. It's a power thing, isn't it?

If you don't have to do it, great. Apparently some colleges are able to meet accreditation requirements without actually having professors do any of it in the classroom, or they manage to put it off for now. But if you do have to do it, you might as well do it in a way that works for you. I'm sorry to keep repeating myself, but the whole thread seems stuck in victim mode (they're doing this to us and we don't have any power over it) and I don't think faculty are natural victims.

I'm not saying that outcomes assessment is necessarily a good or bad thing. The way it's being implemented seems overwhelmingly negative. But the accreditation standards themselves generally just say something like "The institution uses established procedures to design, identify student learning outcomes for, approve, administer, deliver and evaluate courses and programs." I took that from the regional accreditation organization I'm most familiar with, WASC/ACCJC's Standard Two, Part A (http://www.wascweb.org), which goes on to talk about the role of faculty in instructional quality and about ongoing assessment and evaluation. Now, that's bureaucratese, but it doesn't require a college to implement outcomes assessment in any particularly onerous or byzantine way; they do that all on their own, and since it's done locally, it can be fought locally.

But personally, I've found outcomes assessment useful, perhaps because the people telling me how to do it haven't had any power over me so I've done it my way. Here's what I do, and in describing it I'm not saying anyone else should have to do it as well, or that there aren't ten other equally effective methods. I'm just telling you what I mean, specifically, when I talk about outcomes assessment in classes I teach.

I write what I want the students to be able to do by the end of the course as my outcomes, and I include things that I/all the faculty want them to be working on to be able to do by the end of their program. These are not discrete "competencies" (have you heard that word? Perhaps not if you're in the humanities, you might be lucky in that at least) but rather more complex. Things like:

Construct a well-reasoned argument using relevant evidence from course readings and other credible documents.
and 
Produce competent academic writing with attention to scholarly tone, vocabulary, formatting and citations.
Demonstrate behavior that contributes to a positive and productive learning environment.

Those are off the top of my head so they'd probably need revision, but most instructors want students to be able to do something like that, and they know what it looks like when the students do it. I let the students know what it looks like using the dreaded R-word (yes, rubrics).

That last outcome, the one about behavior, seems to be unique to me but I think it's incredibly important. Take a look at all the topics about student behavior, and the employer complaints about "indigo children" as new employees--some of these students don't know how to behave and they need to learn it somewhere. It works for me to make it part of the formal course outcomes, acknowledge that they need to learn it, and include it in the grade rather than making it a separate category called classroom management problems.

As for assessment, I tell the students that it is their job to show me that they have mastered the course outcomes with multiple pieces of evidence (another of the outcomes I use is about self-assessment and learning to recognize what they know and what they still don't understand). I give assignments and a take-home midterm (and I see their writing enough to recognize when it's not theirs), but they can also print out and include their online discussion contributions as evidence of their mastery of an outcome, as well as including homework, papers, and projects. I give project options and have them choose and come up with a plan that will be interesting to them, give them ample opportunities to learn what they need to know, and produce the evidence they need. And I grade them on what they turn in. If they turn in 4 pieces of evidence that they can "construct a well-reasoned argument..." then they get the points for that toward their grade; if they do it, but not so well, the grade's a little lower. I don't care if they got a D on a quiz the first week of the course; if they knew it all coming in they wouldn't need to be there. I care that they can consistently do what I've described in the outcomes, at least at an adequate level of proficiency, by the end of the course. If they can, they get the grade consistent with that level of achievement.

That works for me because students know what they're supposed to learn, and it puts a lot of the responsibility for learning it, and documenting what they can do, on them. It doesn't require more work on my part than the usual approach, though I don't/didn't have big classes so I can't speak to that. I think that for many faculty, it's an entirely different way of looking at instruction, but you should see how involved the students get, how much they learn, how much responsibility they take, and how they remember and keep learning. I've had students come up to me months later and tell me they're still thinking about what they learned (I know, so have a lot of you using whatever methods you use). I've also had faculty tell me they can see the difference in skills in the next class those students take. So that's what I'm talking about when I say outcomes assessment.

Pyshnov, I don't know the history of it, and it doesn't matter to me. In terms of the big picture, I imagine the forces for standardized testing are gearing up their lobbying in hopes of taking over when outcomes assessment dies out. As long as people are taking out tens or hundreds of thousands of dollars in student loans to pay for higher education, they're going to want some kind of semblance of public accountability, and if they can't get it this way, they'll probably want the standardized testing.

Sorry that was so long. Time for bed.
Logged
pyshnov
had touched the tip of the iceberg
Senior member
****
Posts: 626


WWW
« Reply #36 on: June 17, 2007, 04:17:26 PM »

I would still argue that outcomes assessment does only harm.
It introduces another (in addition to different treatment of "minorities") possibility for arbitrary judgements. I do not believe for a second that emphasis on intellectual growth (what outlier uses) is meant.

There will be renewed demands for standardized testing. In this situation, standardized testing is the last resort. Sadly so, because the strength of university education and its whole value for the country is in the wide differences between professors, their views and character, their specialization, approaches to science, etc., etc. To make it all by necessity directed toward meeting the standard testing is to kill everything that was understood as academia (great universities produced different schools and individuals). The university graduates do not have to have the same knowledge and training.
Logged
pyshnov
had touched the tip of the iceberg
Senior member
****
Posts: 626


WWW
« Reply #37 on: June 21, 2007, 09:25:23 AM »

Today, there is an article and discussion about assessments, here:
http://www.insidehighered.com/news/2007/06/21/assessments
In the discussion, there is a call for "nuanced understandings" - including the "sex, age, race and ethnicity,...." Had to be expected!
Logged
aandsdean
I feel affirmed that I'm truly a 6,000+ post
Distinguished Senior Member
*****
Posts: 6,411

Positively impactful on stakeholder synergies


« Reply #38 on: June 21, 2007, 09:59:08 AM »

Here's an irony:

Assessment, on the whole, has been pushed at the governance/legislative level by conservatives--those who find the things that have been taught at colleges and universities, particularly in the arts, humanities, and social sciences--to be at minimum troubling, as implied by the author of the original article.  The legitimacy (or not) of their responses isn't really an issue, but I am sure that a big source of the assessment push comes from discomfort with the intangibles and apparent subversiveness of theoretical relativism and such.

The irony is two-fold:  First, these are the same people who generally insist that the "market" is the solution to all woes.  Of course, higher education has a very effective market system, certainly somewhat distorted by issues such as financial accessibility but nevertheless highly effective at allocating resources (both financial and in terms of distributing student demand), and assessment is plainly an attempt to intervene in the market (I have no patience with the idea that assessment as it's currently conceived will make the market more transparent for "consumers"--i.e. students and parents--because we know very well that it will not do so). 

The other irony is that these advocates are 1) complainers constantly about public schools, which have at least in part been brutalized by mandatory assessments, both in the NCLB context and earlier; and 2) nostalgic for the "good old days" of higher education (when most of us here were educated, and earlier), when assessment was not in the cards at all except for in certain professional programs (e.g., nursing). 

I agree with Outlier that, done right, outcomes assessment can have some--even a great deal of--positive value.  There is nothing whatever the matter with reflecting on one's practices as a teacher, nor with considering whether the curricula of one's program or school are working as well as they might.  However, the problem with assessment as it's currently conceived and institutionalized is that it doesn't much help with either of these goals.  It is, as LarryC remarks, driven by the worst elements of many institutions (I would say that a lot of it is at my current school) and it has become a colossal waste of time and a diversion of energy that could be better spent on actually improving teaching, or for that matter doing scholarship and/or creative activities of other kinds. 

As for the idea of a standardized test at the end for all college students...good Lord, stuff like this has ruined secondary education, so why in the world would we do it to colleges and universities?   
« Last Edit: June 21, 2007, 09:59:50 AM by aandsdean » Logged

Wearing a black armband for Lucy
dogvomit
Senior member
****
Posts: 947


« Reply #39 on: June 30, 2007, 09:31:02 AM »

The purpose of setting assessment goals is to improve things.

How does one ensure that assessment goals are met?
Set assessment goals that are so low and non-ambitious that, even if you fail, you attained them!
This is what we did when we were told we had to meet them..
This proliferates status quo!
It also sets a stopping point.

Wouldn't it make more sense to simply work 100% of the time on improving whatever you can improve?  Then, eventually you might improve everything or even something? 

I see no improvements at my institution that are in any way related to assessment or assessment goals. 

In fact, I see three kinds of institutions out there:
1. those that are good
2. those that are getting good
3. those that pretend to be good

Unfortunately there are increasingly more of #3, only a few of #1, and #2 is decreasing fast. 
Thats why so many schools are marketing their student center and "fun" instead of their academic prestige and employment opportunities!
Logged
csguy
Distinguished Senior Member
*****
Posts: 1,221

Computer Science faculty


« Reply #40 on: June 30, 2007, 11:48:00 AM »

The purpose of setting assessment goals is to improve things.

How does one ensure that assessment goals are met?
Set assessment goals that are so low and non-ambitious that, even if you fail, you attained them!
This is what we did when we were told we had to meet them..
This proliferates status quo!
It also sets a stopping point.

Wouldn't it make more sense to simply work 100% of the time on improving whatever you can improve?  Then, eventually you might improve everything or even something? 

Precisely. If assessment is seen as threatening then programs will go for the easy win. The fact that administration and accrediting agencies are mandating them and there's pressure to make them public makes them threatening.

This is similar to the role of student evaluations. If faculty use feedback from student evaluations (assessment) to improve their courses this is a good thing. It is when the administration uses the evals to "measure" job performance that faculty are motivated to improve their evaluation scores by whatever means available (gaming the system).
Logged
dogvomit
Senior member
****
Posts: 947


« Reply #41 on: June 30, 2007, 04:32:23 PM »

CSguy, your brilliant!
Logged
Pages: 1 2 [3]
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.9 | SMF © 2006-2008, Simple Machines LLC Valid XHTML 1.0! Valid CSS!