• November 23, 2014

Education-School Deans Challenge Methodology of Planned Ranking of Teacher-Preparation Programs

A group of education-college leaders is raising concerns about an ambitious effort, announced last month by the National Council on Teacher Quality and U.S. News & World Report, to rank the nation's nearly 1,400 teacher-preparation programs.

In a February 3 letter to Brian Kelly, the magazine's editor, deans and other leaders of colleges of education at more than a dozen member institutions of the Association of American Universities expressed "concerns about the reliability and accuracy of the data collected" by the council as part of the project.

Areas the deans cited as particularly troublesome included a lack of transparency in the methodology and standards the council is using, along with a provision to "fail" programs in measurement areas that they opted out of or did not provide information for. That "implied coercion," the letter said, "will cast doubt on the results of the entire evaluation."

The education-school leaders urged Mr. Kelly to look at existing measures—specifically standards created as part of the Interstate Teacher Assessment and Support Consortium—to determine the criteria for how colleges of education should be ranked.

The council, an advocacy group, has rankled education-college leaders before in projects that assessed teacher-preparation programs in Illinois and Texas. The deans, in their letter to Mr. Kelly, brought forward some of the criticisms of those projects, including a report by the education research and consulting firm Eduventures on the council's work in Illinois.

In a letter of response to the deans issued on Monday, the council's president, Kate Walsh, defended her group's overall methodology but said the council was working to put "a more transparent process" in place.

She also stated that the evaluation would not fail schools in areas for which they did not provide information or opted out of and, instead, would use publicly available information about those areas. And she agreed to release the "standards and relative indicators" that are being used as part of the study.

She questioned, however, the assertion that there were other, perhaps more effective, methods to rank programs, including the standards created by the Interstate Teacher Assessment and Support Consortium. "If these standards, first developed in 1992, were going to have a major impact on the field of teacher preparation," she wrote, "they would have already done so by now."

Ms. Walsh also played down Eduventures' criticism, saying its report was commissioned by Illinois education schools and was released before the council issued its final report.

Representatives of U.S. News & World Report could not be reached for comment late Monday.

Comments

1. disembedded - February 08, 2011 at 01:51 am

Challenged by people from twelve programs, out of 1,400 programs? I wouldn't call that much of a challenge.

2. impossible_exchange - February 08, 2011 at 08:41 am

The teachers of teachers don't want to be held to the same sort of standards that the teachers are held to.

Education programs are such a joke.

There are a small handful that have game but the vast majority are just teacher accreditation mills.

That said I find such rankings intellectually lame, just I find standardized tests to be pedagogically lame.

3. johnnugent - February 08, 2011 at 09:12 am

The NCTQ and US News have scheduled a webinar for Wednesday, Feb. 9, in what seems to me an attempt to do damage control and keep institutions from refusing to participate at all. Generally speaking, I hope these kinds of efforts do not become more common -- private entities installing themselves as quasi-accreditation bodies who use various threats to get institutions to participate in their ranking schemes.

4. bowl_haircut - February 08, 2011 at 09:40 am

But the more pressing and relevant problem, impossible_exchange, is that many, many administrators, politicians, and the general public do *not* find these rankings lame. Indeed, they cite them in all manner of contexts. That's why we need to educate (speaking of education) these so-called "stake-holders" as to the utter lame-ness of program rankings and standardized testing.

5. clementj - February 08, 2011 at 11:55 am

At presnt there is not any good method for ranking teachers. Student evaluations are seriously biased, and computed gain on high stakes testing is only reliable at the 14% level. So how do you rank schools of education? They should be ranked on the basis of how well their students ultimately succeed in the classroom, but that information can not be obtained.

There are programs which can train teachers to be more successful based on research, but only a very few schools implement them. In the sciences such programs such as Modeling at ASU or Teaching by Inquiry at Washington State have been very successful. But they are deprecatd by "conservatives", so most rankings are made on the basis of the raters bias. The two programs I mentioned have been shown by research to improve student understanding by large factors. Indeed the research based science teaching achieves 30 to 70% normalized gain as compared to 0 to 25% for conventional lecture classes.

Most education schools teach teachers about education rather than immersing them into a program which helps them succeed in the classroom. There is now research which shows that you can teach students how to poperly interact, and other research which shows that teachers can be taught specifically what to do to achieve good class control. If this is combined with good interactive pedagogy the teachers will all be excellent.

About the only measue which might currenty have meaning is the success rate of teachers schools in placing students into good jobs, because there is not valid way of rating the colleges.

6. quicksilver - February 08, 2011 at 12:53 pm

One of the biggest obstacles in teaching teachers is finding faculty who have useful and extensive non-college teaching experience. I remember when I was in college taking ed classes, my classroom management teacher had had 2 years of "real" experience, and that had been 15 years ago. The irony was not lost on my class that here was the blind leading the blind. Yes, it is very possible to teach teaching, but it takes someone with years of in-classroom experience upon which to draw to be truly effective. And employers can forget rankings, as they, like all accreditation reports that can be "crafted", are essentially meaningless.

7. archman - February 08, 2011 at 04:03 pm

Please reform college education departments... PLEASE. I am getting bored consistently correlating education majors as one of my poorest performing academic groups. It's not particularly inspiring for the poor secondary school kids to hear, either.

8. snitter - February 08, 2011 at 04:42 pm

It is amazing that academics would support a reviewing agency like NCTQ that has produced prior work which has not been replicable, has not produced evidence for conclusions, and has relied on methods that would not make it through and IRB review for undergraduate research.

Reading carefully the letter of the deans, they have encouraged evaluation and accountability by reputable research organizations.

In many ways, supporting the NCTQ study parallels the "evaluations" of biology departments by creation scientists. NCTQ's president is a former board member of ABCTE, a group which recieved major federal money to devise a "test" that would certify teachers without any classroom field experiences or coursework. That initiative has largely disappeared after the massive publicity and investment of tax dollars. NCTQ has devised its "own standards", purports to measure the quality of programs by the course descriptions and syllabi. Is this really what you would want when university programs are evaluated? And, if it goes here, it will soon be coming to you in your disciplines.

I can't imagine the concern of psychologist when they are ranked ineffective because they have programs with too much or too little emphasis on cognition or perceptual processes. I can't imagine the reaction of English programs when they are praised for a heavy emphasis on Shakespeare and the European classics and are failed for a more global and diverse literary curriculum. I wonder at the impact on minorities and students in poverty when schools are slammed for admitting anyone not in the top 20% of their class for fear of bad ratings.

I would hope, fractured as we are, that all of us would draw the line at poorly designed research studies which cannot provide accurate information for the public or to improve programs. I encourage you to look at the methodology of NCTQ and see if it would pass IRB review at your institution.



9. eelalien - February 08, 2011 at 11:10 pm

The NCTQ's flawed "research" methodologies have been well-exposed by no;, the other incredible supposedly "expert" entity in this endeavor is... U.S. News & World Report??!! I mean, c'mon - the magazine actually folded in on itself years ago (but I do remember my dad reading it decades ago), and a quick review of its website will reveal that it is simply an empty vessel for corporate shilling in this day and age. USN&WR credibility = NULL.

10. dnewton137 - February 09, 2011 at 04:15 pm

Among the many misunderstandings and misrepresentations of the NCTQ review project reflected above, I would mention just two. First, in the Chronicle title, the project will NOT "rank" teacher-prep programs. Second, it is NOT a research project. It is a review.

Add Your Comment

Commenting is closed.

subscribe today

Get the insight you need for success in academe.