next »

[pyshnov]

When a lot of new measures, regualtions, policies, etc., that at first glance seem legitimate, are introduced, I know with absolute certainty that they are deliberately made for misuse. Such is, simply, the machinery of subverting anything that is targeted for subvertion.

When you see something very wrong happening, you know that those responsible will say: "This was done for a different reason", "This had a different purpose", "This was an unintended consequence" and "This was a misuse".

And, it became a rule for the media in every case when the new law or policy, etc. is introduced, to make a note about "potential misuse". Only an idiot does not see now the "potential misuse" as the primary goal of it.

It relates to the things small and big. In the past the law was always saying: "The man desires the consequences of his actions." Now, there is a huge practice of the so-called law of unintended consequences. It's time to return to the old dictum.

[tortugaphd]

I think it might be more effective to argue against assessment on its own terms: i.e., what assessment has been done of assessment, and what evidence is there that it improves education in the arts and humanities?  I've never seen any.

neutralname brings up a good point, here, and I think that this is where the crux of the issue lies.  Perhaps a process of meta-assessment is in order?

[mountain_ivy]

Pyshnov--I'm confused as to whether you are calling me an idiot. Otherwise, I agree with your perspective.

[pyshnov]

ab1997by:

Otherwise, I agree with your perspective

.
How can I call you anything but a deep-thinking person?

[neutralname]

I think it might be more effective to argue against assessment on its own terms: i.e., what assessment has been done of assessment, and what evidence is there that it improves education in the arts and humanities?  I've never seen any.

neutralname brings up a good point, here, and I think that this is where the crux of the issue lies.  Perhaps a process of meta-assessment is in order?

Maybe a comparison of program development done with outcomes assessment and program development done via the standard random and intuitive approaches. 


Here's a related question: has there been assessment of evidence-based medicine versus other forms of practice?  Does evidence-based medicine actually provide better results?  I would hope so, but if not, then the chance of these positivistic approaches working in education seem even more hopeless. 

[outlier]

I think it might be more effective to argue against assessment on its own terms: i.e., what assessment has been done of assessment, and what evidence is there that it improves education in the arts and humanities?  I've never seen any.

neutralname brings up a good point, here, and I think that this is where the crux of the issue lies.  Perhaps a process of meta-assessment is in order?

Maybe a comparison of program development done with outcomes assessment and program development done via the standard random and intuitive approaches. 

Here's a related question: has there been assessment of evidence-based medicine versus other forms of practice?  Does evidence-based medicine actually provide better results?  I would hope so, but if not, then the chance of these positivistic approaches working in education seem even more hopeless. 

A problem with such a comparison is that there's no common meaning for "outcomes assessment." I've said that outcomes assessment can be useful. I use it and get good results. But when I go to trainings on outcomes assessment, when I hear others report on what they've been told at workshops they've gone to, and when I read about outcomes assessment on CHE forum threads (going back to one of the first I read, last fall, a discussion of grading students based on outcomes), so much of what is called outcomes assessment is just so much bullsh*t.

I'm lucky; the college where I worked ten years ago was an early adopter and I got to learn about it before all this mandatory nonsense started (even so, it took three years before I got a definition I could understand and use). Since then I've become an "expert" and I can tell you that most of what you hear about what you "have to do" is fabrication. It's not in the accreditation standards. Those standards usually just say that you must define and assess the outcomes of student learning, not how you have to do that. The minutia on implementation usually comes not from outside requirements but from some committee in your own college. The people who volunteer to be on those committees are (in my totally biased opinion) most likely to be the ones who like to make policies and rules for everyone else to follow (not always, but too often).

Since the b.s. is locally produced, faculty can usually do something about it. The problem is, they believe the b.s. is in the outside mandate, not in their college's interpretation, so they don't bother finding out what's really required and arguing with the committee for the latitude to do what works.

Another problem, of course, is that faculty are experts in their disciplines but not in cognitive science and research on learning, so they are vulnerable to people who spout dictums about best practices in higher education (which also, often, have little to do with cognitive science and research on learning). Again, in my highly biased opinion, faculty who are expert in their discipline and have become expert teachers through experience are more likely to know what works than committees that have researched "best practices" but don't understand how students learn.

Faculty have more power than you think, but exercising that power involves doing a little work to find out what is really required for accreditation (the driver behind outcomes assessment) and taking on the big bad committees.

Of course, you can just fold your arms and wait, because you will outlast outcomes assessment; it'll just take a while and maybe require you to do some piddling useless b.s. for ten or fifteen years before it fades. At worst, the feds will point to how outcomes assessment and colleges self-regulating isn't working and every student will have to take a standardized test to graduate from college. The ETS will get richer, and if you've invested in it, your retirement will be secure ; )

[pyshnov]

outlier:

...there's no common meaning for "outcomes assessment."

Here is an idea about the outcomes. First, I can say that examinations tell you little about outcomes. Those who have developed a short-time memorizing ability get the good grades. There are other people who are trying to understand the logic of the particular branch of science as exemplified on its basic facts. They don't get best grades, but later become scientists.

Let's say in the first two years students are learning from five courses. And let's give them examinations in the next (third) year at totally unexpected dates. Student comes on Monday after swimming at 2 pm and gets the questions from the course he was studying two years ago, and a room. Wouldn't we get the outcome assessment?

[outlier]

outlier:

...there's no common meaning for "outcomes assessment."

Here is an idea about the outcomes. First, I can say that examinations tell you little about outcomes. Those who have developed a short-time memorizing ability get the good grades. There are other people who are trying to understand the logic of the particular branch of science as exemplified on its basic facts. They don't get best grades, but later become scientists.

Let's say in the first two years students are learning from five courses. And let's give them examinations in the next (third) year at totally unexpected dates. Student comes on Monday after swimming at 2 pm and gets the questions from the course he was studying two years ago, and a room. Wouldn't we get the outcome assessment?

You'll find out who (still) knows what's on the exam. If you test on the results of rote memorization, you'll find out students have probably forgotten. If you test on understanding of concepts, organization of facts into a workable mental model of the domain, you'll find out who learned at a deeper level. Pyshnov, when I say I use outcomes assessment, I mean that what I do is better suited to the second type of student you described, the one who is trying to understand the the logic of the particular branch of science (or other domain) as exemplified by its basic facts. The way that I organize instruction, and assess learning and outcomes, that student would get a better grade.

I think that you assumed I was defending the reduction of instruction and assessment to what can be memorized and tested with multiple choice, and judging by much of what I've heard and read here, that is a common way to do it. But it doesn't have to be that way. That's my point. You can define what you want from students: "understand the logic of the particular branch of science as exemplified on its basic facts." You could probably describe what it looks like when you see a student do it, and devise a task that lets you see whether and how well the student can do it. That's outcomes assessment; that's what we could be doing. It's not either totally intuitive and mysterious or totally simplistic and meaningless, unless everyone who understands the complexities just sits back and lets the simplistic-ones control it.

[pyshnov]

outlier:

I think that you assumed I was defending the reduction of instruction and assessment to what can be memorized and tested with multiple choice...

No, I did not assume this; I am not saying that a much better way than a conventional examination can not be found by asking sensible questions. But, I suppose it will take an uncommon thinking and probably some courage to design such testing.
Are there also handbooks that teach in that manner? The handbooks determine a lot and the trend in the handbooks is just in the opposite direction.
I also was thinking that science must be taught from the point of view of the concrete scientists who actually discovered this or that law of nature. I.e. - to give the students: 1) the state of understanding before the discovery was made, 2) the logic of that discoverer, 3) all the rest.
This is only occasionally being delivered, but everyone studying must be made an actor, so to speak, in the field.

[outlier]

I also was thinking that science must be taught from the point of view of the concrete scientists who actually discovered this or that law of nature. I.e. - to give the students: 1) the state of understanding before the discovery was made, 2) the logic of that discoverer, 3) all the rest.
This is only occasionally being delivered, but everyone studying must be made an actor, so to speak, in the field.

Funny we should come around to this. I used the example of my SO teaching himself math in answer to a question on the "In the Classroom" thread on students asking for "more practice problems":
"My SO is a software engineer. He was an engineering major in college, many years ago, and he did a master's in engineering at Stanford, also many years ago. So he took a lot of math. Now he's reviewing and teaching himself linear algebra and multidimensional geometry; he's finding that as he learns this time around, he is understanding concepts that he never got during his undergrad or graduate level math classes. There, he learned procedures but not what they meant or why they were done. Now he uses several books for each topic, cross-references the information, finds the explanation that makes sense to him, uses it to make sense of something else, draws lots of pictures, reviews it by explaining it to me--though I don't get it at all ; ) --he spends enormous amounts of time and energy to make sure he understands because now he needs to know, and he knows what he wants to accomplish with this knowledge, how he wants to apply what he's learned. Do you know if your students get the underlying concept, or if they just go through the problem-solving steps? And do they have any idea how they will ever put this information to use? Knowing how something is relevant can help motivate someone to make the energy and time investment in learning it.

"Finally, as I mentioned above, he uses lots of references and finds the one that presents the information in a way he can understand, then uses that as a key to unlock more. Do you present the information in different ways so students can get it? I'm not just talking learning styles here, but also applying concepts to different fields and presenting them in different ways. He was talking about how no one uses analogy to teach mathematical concepts because it's not rigorous enough--the analogy doesn't hold up in every single instance so instructors don't use it. But for him, analogy is a way in, then once he understands a concept he can see where the analogy fails. But without the analogy it's harder to get the concept to begin with."

Pyshnov, my SO and I have been talking about math instruction as he teaches himself: how mathematical discoveries happened over centuries, with mathematicians working away, discussing, getting sidetracked, making errors, correcting each other, arguing, and working some more, and all that process gets covered in 10 minutes of lecture by presenting the conclusions. I think the same thing happens in other disciplines. When a subject is taught as if the goal is delivery of specific, uncontroversial facts for memorization, so much is lost. And I agree that outcomes assessment as it is commonly practiced, and as administrators tend to implement it, takes instruction in the direction of reduction of learning to discrete bits and competencies. I find that incredibly frustrating.

However, what I really like about outcomes assessment is that if you write the outcomes and specify meaningful assessments to represent more complexity, you can use it to resist and oppose such factory-model education, and you can get great results. For example, using your objectives--
"to give the students: 1) the state of understanding before the discovery was made, 2) the logic of that discoverer" -- you can write a course outcome something like, "Students will demonstrate their understanding of a (historical) process of scientific discovery and the logic of the discoverers" and the assessment can be the documentation that comes out of a learning project. So, they research conditions at the time, the state of understanding, what the scientists were trying to accomplish, steps they took, etc. Some of this is from books and articles, some from recreating experiments where that's possible. You might do some of this in class, or give the outline and have students research different sources and report back on their findings. Then you can have each student, or groups, do research projects and assess the progress reports they turn in, what they tell you, and their final reports, presentations, and other documentation.

There's your outcome and assessment, and it all involves the students pursuing understanding and creating knowledge; you don't have to give a test, and in this case a test would be beside the point.

And, you asked if there's a handbook? There are some, like Angelo and Cross' Classroom Assessment Techniques, and there are some briefs, but I don't know if they do what you're asking. A friend has been telling me we need to write that handbook, because so much of what people are being told is frustrating and counterproductive, but so far we've just done some presentations.

[pyshnov]

What you are saying is very pleasant to hear. I would think that historically meaningful delivery would be most difficult in mathematics, just because I don't know mathematics. But of course I am wrong. I always liked stories about discoveries, how they were made. Only now just when writing this, I more and more become convinced that this is the correct way to deliver knowledge. And actually science is the several things that were chosen by researchers; there could be different people who could have done different things. So, lectures follow these things, and it is a travesty to present science as something independent, impersonal.
Science is no more than a history of it. Unfortunately, it is mainly popular books that take this task. In fact it is much more serious. When you already know the area, you don't need the names and the stories. But in learning, this is the way to understand the area logically, and - with interest. Apparently, such handbooks delivering serious material do not exist, and all that is needed is the first example being written and printed (the area is not so important). The important thing is to try to give, for every item, the question that the person was asking before he did something new, his line of thought. We know why inventions were made, but we don't have the same for discoveries, not in the handbooks.
What you are saying about using analogy makes sense because analogy translates the problem into the closest mechanistic example; making analogy means simplifying; no one makes analogy with more complicated thing. And, really, we are not capable to understand anything unless we have a mechanical analogy, well, at least I need it. I think it is connected with the working of the brain and I believe that every thought, no matter how abstract, gives in the brain an impulse to the sensory area there.
I've said enough to be contradicted.

[outlier]

I always liked stories about discoveries, how they were made. Only now just when writing this, I more and more become convinced that this is the correct way to deliver knowledge. And actually science is the several things that were chosen by researchers; there could be different people who could have done different things. So, lectures follow these things, and it is a travesty to present science as something independent, impersonal.
...
What you are saying about using analogy makes sense because analogy translates the problem into the closest mechanistic example; making analogy means simplifying; no one makes analogy with more complicated thing. And, really, we are not capable to understand anything unless we have a mechanical analogy, well, at least I need it. I think it is connected with the working of the brain and I believe that every thought, no matter how abstract, gives in the brain an impulse to the sensory area there.
I've said enough to be contradicted.

Not to contradict you, but I would suggest considering how the view of teaching as "information delivery" and presentation. Maybe you use these terms as shorthand for the teaching process, but really, where and how does learning happen? Is it simply transfer of information?

I would say that learning is the process of building a workable, flexible, extensible mental model of the domain or discipline and filling in that model with information and skills in order to perform the tasks or activities of a practitioner. This looks different in different disciplines, and of course, students start out  at a very basic level, with lots of gaps and mistakes and need for guidance. Learning happens in part through information transfer, presentations and lectures, but also through practice, research, projects--experience. I believe that thinking of it this way can be useful to faculty because it helps avoid the common difficulty of reducing teaching and learning to simplistic outcomes that can be tested with multiple choice items and then forgotten.

[larryc]

I agree with every word and applaud the combative style of the writing. Assessment is a steaming pile of sh*t put forward by mediocrities and sycophants. It is the tool by which the EdDs triumph.

[neutralname]

I might mention that I have colleagues in our education school who also think that assessment is a steaming pile of sh*t.  In fact, they have suffered more from it than arts & sciences, because of the quickness with which education accrediting agencies have seen they can keep themselves in business by insisting on assessment.  Nobody who has gone through the assessment process comes out enthusiastic. 

[outlier]

I agree with every word and applaud the combative style of the writing. Assessment is a steaming pile of sh*t put forward by mediocrities and sycophants. It is the tool by which the EdDs triumph.

Well, of course this insightfully reasoned and brilliantly explicated argument absolutely devastates any opposing views.

I think that the way outcomes assessment is implemented is frequently bizarre, wrong-headed and pettily control freakish--frequently, not always. I have found ways to make it work for me. I don't care much about "outcomes assessment" or whatever the trend of the decade is. I do care about students learning something meaningful, valuable, and useful, and I care about faculty teaching well and finding it satisfying. So if outcomes assessment is the way it's going to be for now, then I will do my best to find a way to make it work. And I have found ways to make it work.

Larryc, this is to you especially. Neither I nor anyone else gets a guarantee that when they log in and read something, it will be civil or agreeable. I have noticed that faculty are no better than students at maintaining a reasoned discussion when something raises their ire, and this trend raises a lot of people's ire. Still, I am not a mediocrity and/or a sycophant for finding a way to work with something you abhor, or even for putting forward the view that if faculty believe they have to do it, they do it in a way that works for them rather than simply complaining about it. I'm not looking for an argument with you. I have usually found your posts to be both wise and generally kind, and I thank you for your advice on another thread. This time, I felt attacked by implication because you used such a sweeping generality that whether you meant me personally or not, I was included in that generalization. I have no right to any particular treatment, so I am not trying to tell you how you should behave; I'm just telling you how I perceived this particular message.