next »

[xeno_cratus]

Colleagues,

I'd very much appreciate your advice as to best practices for voting on complex questions, such as faculty hires, in department meetings.  I'm not talking about simple up or down votes on, say, a motion.  Example: We've interviewed four candidates for a job; the whole department (15+) sits around the table; we discuss the pros and cons of the various candidates, but need to come up with a ranked list.  The time comes for a vote.  Often, in our department, we would start by seeing if there is a motion to eliminate a given candidate.  If there is overwhelming support for axing one candidate, that is done. Then each of us simply writes the names of his or her top two or three on a ballot, and then rank the candidates according to the outcome. 

This works sometimes, but often several candidates get the same number, or the differences are so small as to seem negligible, and perhaps indecisive, leaving one with the feeling the decision was as much based on chance (who spoke for what candidate last?) as wisdom.

I wonder if this is the best way.  I've heard of preferential voting, and other methods.  I don't know if anyone has figured out how to apply these to faculty meetings.

Any suggestions?

[daniel_von_flanagan]

Rational choice theory is a long and interesting field, crossing several disciplines (notably mathematics, economics, political science, and philosophy).  Just coming up with an ad-hoc procedure is almost a guarantee of doing it badly (though by Arrow's Theorem this is pretty much unavoidable).

Many such procedures work very well for departmental decisions.  The wikipedia articles on voting are very good. For the more complicated ones (eg "votefair" aka Kemeny-Young, STV, and so on) there are websites and software around that will help you tabulate. - DvF

[stickball]

I'm not sure this answers the question, but why bother voting at all?  Why not use a matrix to evaluate and rank the candidates and then simply go by the numbers?  It's much less personal and eliminates "feelings" at this level.  Of course, those constructs can also be manipulated to reflect "feelings" by the those entering data, but at least there's something quantifiable that can be defended.

[englitprof]

I'm not sure this answers the question, but why bother voting at all?  Why not use a matrix to evaluate and rank the candidates and then simply go by the numbers?  It's much less personal and eliminates "feelings" at this level.  Of course, those constructs can also be manipulated to reflect "feelings" by the those entering data, but at least there's something quantifiable that can be defended.

What would such a matrix look like?

[secretweapon]

You each anonymously score candidates on various factors (i.e. teaching, research, whatever), and then these scores are tabulated together.  The results might suprise you!  Then you bicker about it.

[englitprof]

You each anonymously score candidates on various factors (i.e. teaching, research, whatever), and then these scores are tabulated together.  The results might suprise you!  Then you bicker about it.

Thanks, secretweapon.  This essentially sounds like the rubric I use in grading essays, where students are scored on different elements separately.  Based on my experience with that rubric, this seems like a very useful tool to me for voting in such situations.  I've found that I am much more objective in my grading, at least.

[dr_crankypants]

We would do a ranking of all the candidates, giving top candidate one point and then going through the whole range.  If it gets down to a tie, those can be voted on separately. 

I cannot imagine my department EVER agreeing to that matrix, much less agreeing on how to tabulate the various elements.  Especially because, in hiring decisions, some of it IS personal (which of these candidates do we most want around?) 

[stickball]

Yes, secretweapon nailed it right on the head.  Identify the criteria upon which you'll be judging the candidates based upon the Minimum Qualifications and Desired Qualifications as listed in the posted job announcement/description. You know... a one-through-five scale.  Then each committee member evaluates the candidates and assigns a number to each point of the list and then it's simply a math problem.

[stickball]

dr_crankypants makes a valid point.  The matrix is usually used to get to the shortlist and top three candidates.  I agree... there needs to be some way to make this more than just an objective math exercise.  I think that's what the discussions are for...

[secretweapon]

Yes, hence my final step... bickering. 

You might end up with a top candidate emerging from this matrix, and jaws dropping around the table:  "HU?  But hu was weird!  And smelly!  How could we have picked hu?" 

But this could be good... it's a bit like when you flip a coin to make a decision, and your disappointment at what the coin determines reveals to you what you really want to do ("Oh.  I was kind of hoping it would have been heads.").

- SW, who is tired and becoming increasingly ungrammatical

[dr_crankypants]

My problem with the matrix is that it lends an aura of objectivity to a process that is necessarily subjective.  And, in my department, I can easily see one faction using the "objective" results to argue for the candidate that they just like better.  This is all the more problematic because candidates are often apples and oranges.  Does candidate A get a higher research ranking, simply because of the number of publications?  Or does candidate B get a higher research ranking because the research field is more interesting?  Or candidate C because, even though she has only minimal publications, she's probably writing a better book than the seemingly more productive candidate A?  What about other factors, such as how the candidates contribute to the methodological, racial, or gender diversity of the department?  Or whether or not the candidates somehow overlap with current research? 

I can see using a matrix for my own personal use, but I would be horrified at the thought of reducing a complex and holistic evaluation to a matrix.  The point of matrices is that they're supposed to take some of the personal out of the equation, and to make it seem possible to compare apples and oranges.  But I don't see how that's helpful in making an important decision that can and ought to go beyond a 5-point matrix.

Oh, and I can't see my department agreeing on the weighting of the matrix, either. 

[xeno_cratus]

We would do a ranking of all the candidates, giving top candidate one point and then going through the whole range.  If it gets down to a tie, those can be voted on separately.

May I clarify? Colleagues give their top candidate 1, their second choice 2, their third 3, and so on.  Then all these numbers are tallied.  Then the person with the lowest score is No. 1, second lowest, No. 2, etc.  Is this right?

I cannot imagine my department EVER agreeing to that matrix, much less agreeing on how to tabulate the various elements. Especially because, in hiring decisions, some of it IS personal (which of these candidates do we most want around?)

Precisely the same would be the case in my department.

[xeno_cratus]

For the more complicated ones (eg "votefair" aka Kemeny-Young, STV, and so on) there are websites and software around that will help you tabulate. - DvF

Hmmm.  I looked at the VoteFair.org web site and find it intriguing.  I wonder if any departments actually use such a voting scheme, and how it is practically implemented.  To use it in my department I would have to (1) convince everyone it was worth considering, (2) get them to study it to the point that they understood it, (3) figure out how to implement it in a meeting situation in such a way that it was not too complicated or time-consuming, (4) have them all vote (!) on it.

For starters, I would be interested in (3), examples of how such a scheme can be actually implemented in a meeting context.  Most of what I see on the web is for on-line surveys.

[science_expat]

We use single transferable vote at institution level to elect members to Senate and certain interview panels and committees. It seems to work well but in Europe we're more acquainted with the system.

Here's a good link http://www.electoral-reform.org.uk/votingsystems/systems3.htm

[daniel_von_flanagan]

In case there's any doubt, the matrix/rubric method discussed above is a form of voting, and like any other can give unexpected results.

The main problem with both votefair and single transferable vote is that for neither is the tabulation method clear and transparent.  The latter is quite handy for proportional representation, and easy to do on a computer, which is why it has renewed popularity (it is 150 years old at least).  It has some zealous supporters, and some zealous opponents.

One nice thing about votefair for candidate selection is that you need not rank all the candidates; you just need to rank pairs of candidates, and consistency be damned.  In other words, given candidates A,B, and C, in most preferential voting procedures each voter needs to produce a list of these candidates in order of preference, and it excludes the possibility that you prefer A to B, B to C, but C to A.  Votefair will happily accept the latter as a valid vote.  In my experience on search committees, textbook selection committees, and so on, this is not uncommon. - DvF