next »

[science_expat]

It's 3 years until REF submission time but no doubt preparations are already underway at many institutions - and will intensify when the report on the impact pilot is published in November. Here's a thread to b!tch and moan about the run-up to the REF - from any perspective.

To start, it's Saturday afternoon and I have just gone through the publication lists of all my colleagues to choose their "best" 4. Now I have to go through again and put star grades (2* etc) against each one. As above, the REF is 3 years away but the Pro Vice Chancellor requires this information next week.

<SE, drinking wine, grousing, and procrastinating on the forum>

[cranefly]

How are the publications evaluated--I assume by journal rankings? Or is it by impact factor?
And where do the journal rankings come from that are used? (the European rankings?)

[science_expat]

How are the publications evaluated--I assume by journal rankings? Or is it by impact factor?
And where do the journal rankings come from that are used? (the European rankings?)

It's a black art.

At the moment I'm going on journal impact factors (from ISI - I'm in STEM), titles, and abstracts. When it gets real then we'll need to be serious about selection but it's way too early for that much effort. (In my opinion at least - not so sure about the PVC's.)

[babbinacara]

Our HoD is dragging us in and having us imagine up ourselves how many stars our submissions should be worth.
Black art indeed.

[qrypt]

Our HoD is dragging us in and having us imagine up ourselves how many stars our submissions should be worth.
Black art indeed.

We've had to do that as well.  The overall picture has then been "calibrated": sent to external readers, to determine whether our judgment matches theirs. 

I don't mind all that much, actually.  It's very much in my interest that my department do well.  I'm all set, I have a sufficient number of publications and am working on some that might improve the set -- but some of my colleagues are in real trouble, and it's better to have that identified now.

So I'm not moaning.  But then, I don't have to do the kind of work SE is describing -- I'm not senior enough, and I intend to keep it that way for a while, given how things are going here. 

[scotia]

We are part way through a similar exercise to the one that S_E describes, and in a couple of weeks a colleague and I will start meeting with staff to agree the rankings of their outputs. Frankly at this stage the time (mine and that of the people we are ranking) would be better spent doing research, particularly as we went through a similar exercise, though with fewer forms, only a few months ago. I have a strong suspicion it has something to do with various administrators looking over their shoulders with some alarm at the current uncertainty and trying to justify why they are indispensable.

[science_expat]

We are part way through a similar exercise to the one that S_E describes, and in a couple of weeks a colleague and I will start meeting with staff to agree the rankings of their outputs. Frankly at this stage the time (mine and that of the people we are ranking) would be better spent doing research, particularly as we went through a similar exercise, though with fewer forms, only a few months ago.

This is why I've taken control of the exercise - no reading groups, no consultation, just my best guess as to quality. There's no point in wasting colleagues' time if I can possibly avoid it.

I'm also striving for inclusivity at this point in order to give folks a chance to develop. Some justification for this approach comes from a colleague in a different research unit who has strong data from 2008 showing that quality and quantity increased significantly just before the deadline.

[scotia]

We are part way through a similar exercise to the one that S_E describes, and in a couple of weeks a colleague and I will start meeting with staff to agree the rankings of their outputs. Frankly at this stage the time (mine and that of the people we are ranking) would be better spent doing research, particularly as we went through a similar exercise, though with fewer forms, only a few months ago.

This is why I've taken control of the exercise - no reading groups, no consultation, just my best guess as to quality. There's no point in wasting colleagues' time if I can possibly avoid it.

Sadly I do not  have that power......

I'm also striving for inclusivity at this point in order to give folks a chance to develop. Some justification for this approach comes from a colleague in a different research unit who has strong data from 2008 showing that quality and quantity increased significantly just before the deadline.

That is interesting. I don't suppose the data is published anywhere?

[science_expat]

We are part way through a similar exercise to the one that S_E describes, and in a couple of weeks a colleague and I will start meeting with staff to agree the rankings of their outputs. Frankly at this stage the time (mine and that of the people we are ranking) would be better spent doing research, particularly as we went through a similar exercise, though with fewer forms, only a few months ago.

This is why I've taken control of the exercise - no reading groups, no consultation, just my best guess as to quality. There's no point in wasting colleagues' time if I can possibly avoid it.

Sadly I do not  have that power......

Technically, I probably don't either but all I'm being asked for is an assessment - not a description of the methodology by which it was achieved.

I'm also striving for inclusivity at this point in order to give folks a chance to develop. Some justification for this approach comes from a colleague in a different research unit who has strong data from 2008 showing that quality and quantity increased significantly just before the deadline.

That is interesting. I don't suppose the data is published anywhere?

Sorry, no. It came from an internal discussion on the merits of this type of exercise.
__

To be fair, I do think there is merit to reviewing publications at this stage. It forces the REF coordinator to look at things in detail which, in turn, will no doubt lead to interesting discussions with some members of staff. But I think it should be as light touch as possible.

[lilac53]

Unrelated, and sorry for the thread hijack. Science_expat, I sent you a PM earlier - let me know if you didn't receive it, I've been having problems sending them lately.

[thaddy]

Two colleagues have suggested this week that the REF might be completely canned in the wake of the CSR.  Anyone else hearing rumors like this?  Wishful thinking on their part? 

[the_walrus]

I have heard precisely the opposite from someone I believe is in a position to know about such things.  They say it'll go ahead, deadline will be end of Dec, 2013 and that impact is likely to count for 20%.

[science_expat]

Of the two versions, I think TW is much more likely to be correct.

HEFCE are convinced that the REF will go ahead as they believe it delivers exactly what the Treasury wants - a robust mechanism for measuring excellence. They also think that the Treasury wishes they could assess teaching quality as robustly.

In terms of impact, I'm certain that it will be included and 20% is a good guess. The results of the pilot will be published on the 11th but - as disseminated at many meetings - the basic story is that all 5 panels reported that they could confidently assess impact quality.

[fourhats]

(Bookmarking)

[science_expat]

We are part way through a similar exercise to the one that S_E describes, and in a couple of weeks a colleague and I will start meeting with staff to agree the rankings of their outputs. Frankly at this stage the time (mine and that of the people we are ranking) would be better spent doing research, particularly as we went through a similar exercise, though with fewer forms, only a few months ago.

This is why I've taken control of the exercise - no reading groups, no consultation, just my best guess as to quality. There's no point in wasting colleagues' time if I can possibly avoid it.

Sadly I do not  have that power......

Technically, I probably don't either but all I'm being asked for is an assessment - not a description of the methodology by which it was achieved.

To follow-up, in a meeting today I informed the PVC that I have assigned star ratings based on journal quality and abstract as I felt it was much too early to spend a lot of time assessing articles that might not end up going forward for possible inclusion. He seemed to agree - at least I haven't been sacked yet.