next »

[sueenglish]

Evening All,

I wondered how your universities are aiming to deal with the uncertainties of the REF? We have just had a restructure, which may account for the contradictory messages about our approach to it - at present these seem to be:

1. Aim for four outputs - but make sure two are great.
2. We are understaffed,  but have no money: research has to be squeezed in - we'll pray it works.
3. Student experience comes first - then ditto number 2.
4. Research leave is available, but don't bank on it once the dept. budget has been top-sliced - ditto number 2.
5. Aim for journal articles, not monographs.
6. Monographs are better than articles.

Sorry the list is haphazard, but we seem to have no direction on the REF (or about it!) We're told in press by 2011 - but this is still a vague date. We've also got major work intensification going on - two folks on maternity leave, one long-term sick and barely any cover. We're all up to the 462 brink of contact hours.

[ramani]

Ha! This is so familiar in many ways, except that while we are understaffed and have no money and are all teaching crazy loads and covering other people's subjects, and the students are getting a really bum deal, the institutional rhetoric is all about research. Apparently the institution 'won't be supporting' 1* and 2* research, which leaves us all wondering how they are going to tell until it is published, circulated, assimilated, responded to, etc etc etc. And don't even get me started on impact...

[sueenglish]

Ha! This is so familiar in many ways, except that while we are understaffed and have no money and are all teaching crazy loads and covering other people's subjects, and the students are getting a really bum deal, the institutional rhetoric is all about research. Apparently the institution 'won't be supporting' 1* and 2* research, which leaves us all wondering how they are going to tell until it is published, circulated, assimilated, responded to, etc etc etc. And don't even get me started on impact...

Oh Gawd. Same old story then? Yep: we have no idea what rating stuff will get! Impact...oh sigh oh sigh. I would't mind if it was meaningful. There can be impact in all sorts of ways. My worry is it will be a badly handled idea and measured in weird ways.

[science_expat]

Lots of good stuff here - http://www.kcl.ac.uk/iss/research/ref/ref3.html

At present:

REF is scheduled for 2013 so outputs through 2012 will be included. These will be associated with researchers, not institutions. Model will remain selected outputs and the likely number is 4 although 3 is possible. There may be a reduction for "significant outputs", e.g. monographs.

Impact must be "demonstrable outside the discipline" so it's not about citations etc... but take up. Impact will be described by narratives and case studies and may relate to work that occurred pre REF (though probably time limited) if it is currently being "exploited" within the UofA. Impact is expected to remain with the department (given the above conditions) even if the person who produced the work has left.

Bibliometrics will inform peer review but not replace it, even in hard sciences. Not clear but likely to be a profile of a UofA's numbers not per paper.

Outputs at least 50-60% of total score, though environment and impact will be "substantial" as "there's no point in assessing something that only counts for 5%". My personal guess is 60/20/20 across all panels.

Bottom line

Do good research, publish in top quality locations, and keep track of take-up - past and present.

And pity the poor saps - likely including me - who will have to write the submissions!

[observer3]

Ugh. It is so stressful to work under ever-changing rules. Especially when one's HoD is frantically trying to make his career via impressing the center by constantly adding on more things for us to do, to the point of absurdity. The latest is an official listing of only half the hours we actually spend on teaching and grad students, to make official way of course for who knows what. Apparantly that will pave the way to a deanship or something. It's a crap incentive structure.

[science_expat]

Anyone had a look at the consultation?

I'm trying to write a draft response and am not happy - "impact" permeates the entire document!

[qrypt]

Anyone had a look at the consultation?

I'm trying to write a draft response and am not happy - "impact" permeates the entire document!

What, you can't show how your research will cure cancer before the next deadline??

[scotia]

We have been told three outputs rather than four, and monographs may count as two outputs.

There is talk of a "critical mass" of researchers, which will penalise small diverse departments (like mine) who need a range of specialisms in order to cover the teaching (I suppose we could recruit all basket weavers, but that would leave us with basket weavers struggling to cover the pottery classes). So even if every individual is producing excellent research we are going to be at a disadvantage compared to other places where there are 15 or so people in my sub-field, rather than just me and a couple of people in contiguous sub-fields who help out with teaching.

There is a lot of talk of impact, but none of it seems to make sense when you dig deeper. I was particularly puzzled by this paragraph:

"Limitations of metrics. There are limitations in the extent to which the impacts of research can be ‘measured’ through quantifiable indicators. Rather than seek to measure the impacts in a quantifiable way, impact will be assessed in the REF. Expert panels will review narrative evidence supported by appropriate indicators, and produce graded impact sub-profiles for each submission; they will not seek to quantify the impacts."

In my world a grade is a measurement, but it is given a letter rather than a number (it can be converted to a number if you so wish), so this seems nonsense. I get the idea that the people putting together the REF have no idea how they are going to assess impact so they are stringing together a few ultimately meaningless phrases in the hope that at some point someone will magically find the light switch. Certainly, vague discussions about case studies are not helpful, and nor is the suggestion that we look at assessing what was done up to 10 - 15 years ago: I don't think anyone currently in post was here 10-15 years ago and we are small enough that one case study would need to cover people in very diverse sub-fields.

All-in-all the current proposals seem to signal a desire to concentrate research in larger centres. I suspect a lot of smaller research departments will struggle to survive after 2013.

The best quote to date on impact was addressed to one of the Divinity professors: "You guys are probably the only ones that have much chance - pray hard that we do well in this exercise and we will report your fantastic impact next time around!" (the professor in question is an atheist, but he agreed the sentiment).

[wegie]

The best quote to date on impact was addressed to one of the Divinity professors: "You guys are probably the only ones that have much chance - pray hard that we do well in this exercise and we will report your fantastic impact next time around!" (the professor in question is an atheist, but he agreed the sentiment).

Oooh! Nice ;-)

Although I'm thankfully semi-detached from this round (only semi-detached as I'm currently a student in an OU department (classics) that is a prime candidate for the chop), I'm getting the distinct impression that a lot of what REF is about is turning back the clock to the days pre-RAE when research excellence was de facto assumed to not exist outside the big research groups. If there has been one thing the RAE has been good at, it has been at showing that big is not necessarily better, and that excellence in research can exist almost anywhere. However, in terms of doling out the money, it's rather obvious that the Research Councils don't like to spread funding thinly.