If you’re a lawyer, predicting the outcome of a case is an important part of your job. It helps you advise your client, decide what strategy to employ, know when to settle, etc. That expertise is part of why lawyers can charge more per hour than the rest of us make in a day.
But—guess what?—lawyers aren’t so good at evaluating those odds, according to a new paper published in Psychology, Public Policy, and Law. That’s because they’re biased in favor of their own chances; they think they’re going to win, and often they’re wrong. The study surveyed nearly 500 lawyers and had them predict the outcome of an active case and then compared those predictions with what actually happened. From the paper:
Lawyers frequently made substantial judgmental errors, showing a proclivity to overoptimism. The most biased estimates were expressed with very high initial confidence: In these instances, lawyers were extremely overconfident.
But, surely, more experienced lawyers have a better sense of who’s going to win, right? Nope. In fact, “the data provided no support for the hypothesis that lawyers with more practical experience are better calibrated than lawyers with less experience.” So much for the Matlock Effect.
Okay, but certainly it would help if, when making predictions, lawyers were asked to give reasons why they might not win before being asked to rate how confident they were. The researchers thought this “debiasing technique” would encourage lawyers to give more realistic assessments.
It didn’t. They were still overconfident.
So what’s wrong with a little wishful thinking? Well, as the authors point out, imagining a positive outcome significantly increases one’s disappointment when things don’t work out. But it’s more pernicious than that. Here’s what an earlier paper on the topic of confidence has to say:
It can be argued that people’s willingness to engage in military, legal, and other costly battles would be reduced if they had a more realistic assessment of their chances of success. We doubt that the benefits of overconfidence outweigh its costs.
Worth noting is that women tended to be less susceptible to the overconfidence bias than men and better at determining whether they had a moderate or a high probability of success. Something to consider next time you want to sue.
(A PDF of the paper, “Insightful or Wishful: Lawyers’ Ability to Predict Case Outcomes,” can be downloaded here. The authors are Jane Goodman-Delahunty, Pär Anders Granhag, Maria Hartwig, and Elizabeth F. Loftus. The paper quoted at the end is “The weighing of evidence and the determinants of confidence” by Griffin, D., Tversky, A. (1992). Don’t mess with Andy Griffith.)
12 Responses to Your Lawyer Is Wrong
xtrcrnchy4 - June 4, 2010 at 3:48 pm
There are many interesting comments to be made about this study. Is it a necessary conclusion that lawyers really overestimate their chances? Could be, but could it also be an unwillingness to tell a third party researcher an answer that isn’t positive? Recall that the lawyer’s answer to a survey question about the ongoing case could be discoverable by an opponent. While not very relevant to the case at hand, it is also not in any way privileged from discovery. This is the same reason attorneys for corporations going through an annual audit don’t want to report to the auditors that an ongoing case is likely to be lost, even if they suspect so. That audit report and the attorney’s comment are discoverable by third parties. This is all to say that, as is often the case, answers to survey questions are not always reliable bits of data. Also, with regard to whether women are better at estimating their chances, there are so many other factors at play here. Women might, in fact, be better at estimating, but the data only show that the women were more likely to predict a result closer to the real outcome, but that doesn’t say whether the outcomes, on average, were higher or lower. Does this comment also on the relative level of confidence of women attorneys? Who knows, but this could be another factor in play, rather than simply “Women are better and more realistic.” Finally, just for fun, this is only one more reason that anyone who thinks they know something about the practice of law because they watch TV, see movies, or read John Grisham novels, really knows not very much.
11299051 - June 4, 2010 at 4:32 pm
The says is “pride goeth before a fall” but perhaps it should be, “overconfidence breeds pride that goeth before a fall.”
11299051 - June 4, 2010 at 4:33 pm
Oops, sorry, should be The saying is…
counselorpa - June 4, 2010 at 4:41 pm
I agree with the first poster. I do not think that many attorneys would report accurate bases for an unsuccessful resolution nor concede a losing case in a survey regardless of assurances of privilege. Also, I am not sure that this article really shows overconfidence after reading the article. The resolution of a case is a social process and even 6-12 months prior to a trial, an attorney may not have all of the relevant information to offer a completely informed final goal. Considering the fact that such a large percentage of these cases involve civil plaintiffs, who appear to be suing businesses (workplace, contracts, products), the plaintiff’s attorney may not be privy to goal changing information until closer to trial. If the resolution of a case in that instance does not equal the prediction of the minimum goal, but still puts the client in a better position than prior to the start of the case or the client is happy with the services, that would appear to be “objective” success. I have experience in the criminal arena and there are an infinite amount of human factors at play that are unpredictable. I imagine that you cannot capture them even if you are self-hating and think you will lose every case. For instance, in many jurisdictions, a defense attorney could never fully apprise the “court context” of a case that far in advance. The meteorologist analogy would leave me agree with the 1st poster’s final assessment as well.
davidbinder - June 4, 2010 at 6:32 pm
To add to posters 1 and 4′s comments, I’ve had this discussion with a friend who is considered to be a very competent criminal defense attorney. Her report is that 95%+ of her clients are, to use her words, “dead-bang gulity.” “Success” in her world translates to getting charges reduced to the lowest possible level of severity and to get the least onerous sentences for clients, two things at which she is quite successful. Her clients may imagine some magical world in which they are not guilty, but the last data I’ve seen is that over 90% of those who plead not-guilty and go to trial are found guilty. Even those folks who sincerely believe themselves to be innocent often take an “Alford Plea” as part of a plea bargain to avoid the risk of a trial resulting in a finding of guilty and a more severe sentence. I’ve some experience with civil cases from the perspective of a senior business executive. #4 is quite correct that we do consider the trade-off of cost of litigation to range of possible settlements and often do settle cases that we could defend (and possibly win) but at a cost greater than that of settling. It is a pure financial decision … which outcome is less costly. I’ve also been involved in negotiating consent agreements to settle issues. As these usually involve no admission of guilt or culpability, I’ve used these as an inexpensive way to settle what otherwise might be expensive litigation with an uncertain outcome. A belief that the allegations were unfounded played little role in deciding whether to defend or settle. The decision is based on the least bad alternative. None the less, I suspect that the basic premise of the article is correct. It certainly fits the research on how people mentally estimate positive and negative outcomes (see, for example, Kahneman, D. and Tversky, A. (1973) On the Psychology of Prediction. Psychological Review: 80, 237-251).
gavinmoodie - June 4, 2010 at 11:33 pm
Only about 5% of disputes go to trial. Most are settled well before they proceed very far and several more are settled ‘at the doors of the court’. This is because lawyers for each side are able to agree on a likely outcome acceptable to their clients. So the study was only of the 5% of the most difficult cases. It is like judging the success of the whole health care system on the survival rate of patients who undergo major surgery.
rpm13 - June 5, 2010 at 12:39 am
Posters make some good points, but many of the criticisms of the study are not sustainable against the actual methods described in the original publication. There are studies of many professions that show overconfidence and other judgment biases. Predictably, practitioners often fail to recognize these biases in their own work.
generally_academic - June 5, 2010 at 2:09 am
I trust my wife. Never wrong. Saved my assets a few times, too.
bdbailey - June 5, 2010 at 6:58 am
This is also consistent with attribution theory. Attribution theory says that when things go well for us, we tend to attribute it to our own abilities. When things don’t go well for us, we tend to attribute it to something external.
mbelvadi - June 7, 2010 at 12:05 pm
#9, to pick up on a point from the article, doesn’t attribution theory research also show that women are more ‘realistic’ than men in their attributions of credit/blame? [Vague recollection from undergrad social psych course...]
dmaller - June 14, 2010 at 1:36 pm
In my experience working at a major law firm, lawyers were always supremely confident in their chances of winning, not because they were not good at predicting outcomes, but rather because their clients demanded it. If a lawyer didn’t project full, unequivocal confidence in the client’s position, the client would in many cases loose confidence and consider taking their business elsewhere. Most attorneys aren’t willing to take that risk, especially with large institutional clients are a good source of repeat litigation business. If you asked litigators to predict the outcomes of cases to which they were not involved in (and for which their livelihoods did not hang in the balance), I have no doubt they’d be a lot better at predicting outcomes.
violetpetran - July 8, 2010 at 3:14 pm
I mentioned your article in my blog about this study: http://lawblog.legalmatch.com/2010/07/07/lawyers-dont-make-very-good-fortune-tellers/I think that people put a lot of pressure on lawyers to always win cases without looking at the realities of their case. And then the lawyer feeds into the pressure and further hypes up his abilities and skills and this makes for a bad situation that can lead to potential attorney malpractice issues.