Función de Demandas · A conversation with José Astigarraga
The people who decide an arbitration are human beings, and as human beings they are subject to biases. In this episode, Jorge Fantuzzi talks with José Astigarraga, an international arbitration expert, partner at Reed Smith, with a significant practice as an arbitrator and mediator, about what behavioral economics can teach us about how judges and arbitrators decide. They discuss confirmation bias, extremeness aversion, anchoring, authority bias and hindsight bias, and what can be done to mitigate them, including the use of artificial intelligence.
«We make these mistakes subconsciously, and that is where the danger lies.»
Listen to the full episode here:
You can check the transcript for episode S04 E08 of Función de Demandas here:
From a lunch to this episode
José, thank you very much for joining us. I want to tell how we came to this topic. We met many years ago; I don’t know if you remember, at an IPA lunch we were seated next to each other and you started talking about biases. I had just come back from Chicago, where we had taken courses in behavioral economics, so it was a very enjoyable conversation. We met again at the beginning of this year, and I happened to be reading “Noise,” the book by Kahneman, Sibony and Sunstein, which looks at how behavioral economics applies, among other things, to decision-making. And the favorite example is precisely how judges make decisions. Hence my wish to talk with you. I have my doubts, by the way, about whether this needs the label of “behavioral economics,” but we can discuss that.
Jorge, on the contrary, thank you. I’m honored that you included me. I’ve been following the podcast closely and I appreciate the opportunity to spend this time with you and your listeners, to share some ideas about biases and how we humans reason.
Biases operate without our noticing
There is something basic: we have to assume that, for example in arbitration, which is what brings us together, the decision-makers are people, and as people they are subject to biases. What is the starting point, bearing in mind that we are talking about the design of a dispute resolution mechanism that we hope will be as fair as possible?
That’s right, Jorge. This topic has been studied extensively and, as we will discuss, it was even the subject of a Nobel Prize in Economics, the one awarded to Professor Kahneman. The reality is that human reasoning is a marvel, but it also suffers from flaws and shortcomings that lead us to wrong conclusions. And the key is that we do it without realizing it. We make these mistakes subconsciously, and that is where the danger, the harm, lies.
If, as a society, we want a fair justice system, forgive the redundancy, to resolve our disputes, it is extremely important that each party feels that, whether it won or lost, at least the process was fair. And how can we define fairness? As an objective application of the law to what happened, that is, to the facts. What happens when biases affect us? One or the other can go off the rails: our perception of the facts can differ from the reality of the facts. That is the flaw and the danger that arises when biases affect the decision-making process in our society.
And we tend to think of it in very binary terms, black or white, but there is a lot in between. The same book cited a study, I don’t remember exactly where it was conducted, that analyzed how harsh judges’ decisions were depending on whether they were closer to or further from lunchtime, that is, whether they were hungry. One might say the decision won’t change from black to white, but within the blacks it may be a bit darker. Biases can affect even that kind of decision.
Absolutely, and it is an emblematic case. To tell our listeners: it was a study conducted in the Israeli justice system, in which researchers analyzed thousands of parole applications by prisoners, that is, when a prisoner serving a sentence asks, on grounds of good behavior, to be released before completing it. They looked at whether the judge granted or denied the request and discovered a pattern: judges tended to grant applications at the beginning of the session, that is, at the start of the day; then the approval rate spiked again in mid-morning and rose once more at the beginning of the afternoon.
The researchers studied this and what they found, and I’ll quote what they said, in an informal translation, was that it had to do with the judge having eaten: after breakfast, after the mid-morning snack and after lunch. They say: “Our findings support the view that the law is indeterminate by showing that legally irrelevant situational determinants,” that is, conditions in the environment that have nothing to do with the case, in this instance simply taking a food break, “may lead a judge to rule differently in cases with similar legal characteristics.” Apples to apples, and the outcomes differ because the judge happened to have had breakfast.
Think of us as a society, and in particular of our arbitral community, which has an interest in ensuring that arbitration is reliable and inspires confidence in users. Imagine the injustice. There would be a battle to be first in line. That’s my point: as a community, we have to work to root out these flaws from the process, because we all agree that it is deeply unfair for one person to be granted freedom and another denied it because of the time of day the case was decided.
Confirmation bias
There are biases we could call non-systematic. For example, a judge having an aversion to a certain type of person, say, to redheads. That is impossible to foresee, to control, and probably even to recognize. But there are other, more systematic biases, where you can understand why they occur, and if you understand why they occur, you can look for ways to address them. Are there any of these more systematic, more predictable biases that you see in the world of arbitration, in the decision-making of arbitrators?
Absolutely, Jorge, and you raise a very important point. When we talk about biases we are not talking about prejudices. We have to acknowledge that, as humans, prejudices exist, racial ones, for example, but this has nothing to do with that. These are mental flaws that occur subconsciously. And it is not simply an academic matter, “how interesting this criminal law stuff is”: I have seen how they affect arbitral justice, and I can give you examples.
For me, one of the most relevant biases is confirmation bias. I can summarize it like this: once a person forms a preliminary opinion about something, they tend to subconsciously filter information so that they embrace what supports their thesis and do not even see, or reject, what goes against it. That is extremely dangerous, because you see only, through your filter, what you believe supports your thesis and ignore important information pointing the other way. Leaving arbitration aside, in my view confirmation bias is the most important and harmful bias in the world today, especially in the polarized world we live in.
On social media: I only listen to people who think like me.
Exactly. And it’s not just that I only listen to those who think like me. Let me cite an experiment showing that it is not only that I shut myself in what Americans call the “echo chamber,” but that I literally do not hear the information that goes against me.
It is a study from Emory University. During the three months before the 2004 U.S. presidential election, researchers placed Democratic and Republican voters in MRI scanners to analyze what was happening in their brains. Each was presented with information that contradicted their beliefs: a Republican was told, with facts, that his candidate had lied about something, and the same was done to Democrats about theirs. What did they find? In an informal translation: “We did not see any increased activation of the parts of the brain normally engaged during reasoning. None of the circuits involved in conscious reasoning were particularly engaged.” Notably, there was no increase in activation of the dorsolateral prefrontal cortex, the part of the brain most associated with reasoning.
Let’s pause and think about what they are saying. A person was so biased about their candidate that they did not even make a rational decision along the lines of “that doesn’t sound right, because I was there and I know it wasn’t like that.” No: the information did not get into their brain, it did not get into their computer. So it is impossible for that person to make an objective decision, to weigh it up and think, “maybe I have a doubt, let me think about it.” And we are living that today, at least in the United States, where polarization is so fundamental that you can see people are not listening to each other, not even hearing each other. That is very serious for society, never mind arbitration and criminal cases.
We, as economic experts, in the arbitral community, see this confirmation bias a lot with respect to our disciplines. If an arbitrator receives an expert report on a chemical compound used in construction, he probably knows nothing about it and will accept what the experts say. But on economic matters, if someone says the business’s growth rate was going to be 8%, and the arbitrator knows something about business, he may say, “No, 8% seems too high to me, because I don’t think this business was that good.” And that is where the bias kicks in: what my prior was, my first idea on the subject, and I only pay attention to what keeps confirming it.
I have seen it in arbitrations, and I can tell you about it. There was an arbitration I wasn’t involved in. I had been that client’s lawyer for many years, but another firm was handling that arbitration. The client’s in-house counsel called me and said, “José, I don’t like how the arbitration is going; please, I’d like you to join.” I joined, and what I discovered was that the presiding arbitrator was so biased, not out of hatred or anything of the sort, but simply because his mind was so closed, that there was no way our side could put forward anything he would look at objectively. The case was already lost, given how closed his mind had become.
And one last comment on confirmation bias: honestly, if we humans found a way to overcome it, I think we would live in a different world. Let me give you an example showing that it has global consequences. We all know that the United States invaded Iraq years ago looking for weapons of mass destruction, and that it was genuinely convinced they existed. It turned out they didn’t, and a commission was then appointed to look into what had happened. Part of what contributed was confirmation bias. The report to the president says: “In some cases, analysts’ presumptions were so firm that they simply disregarded evidence that did not support their hypothesis.” And this is not bad faith: these were people who genuinely wanted to do their job, but the bias they suffered from blinded them and prevented them from seeing information that ran counter to it. Momentous global consequences that occur as a result of these biases.
Splitting the baby: extremeness aversion
There is another bias I see very often, even in how cases are framed and strategy is set, behind closed doors. I would call it the Solomonic bias, though it may have another name: the idea that, if one party asks me for zero and the other for 100, something between 40 and 60 will leave both a little satisfied, as if both had won. The saying that everyone thinks they won the arbitration is not uncommon. That is also an institutional problem one can notice.
A very insightful observation. A couple of comments. You know, because I overwhelmed you with this years ago when we first met, that this is a topic that has interested me enormously, and I have spoken about it quite a lot around the world. Something I have encountered is skepticism, especially from arbitrators: “Please, those studies are with people who don’t have the same training.” But many of the studies I cite were conducted with judges, American and others; they are trained people. That is my first observation. The second is that we must acknowledge that, in some respects, arbitrators, because they have training and preparation, and because we have been talking about this community, may be a bit more alert to guarding against these biases.
But the idea you mention is what in English is called “baby splitting”: the Solomonic solution, splitting the baby in two. The reality is that humans suffer from another bias, extremeness aversion. Let me cite an experiment that is amusing. Researchers offered a group of people a choice between two cameras: one for 169 dollars and another for 469. The group split in two, I’m summarizing very quickly: half wanted the 169 one and the other half the 469 one. Then they repeated the experiment with three options: a camera for 169, one for 239 and one for 469 dollars. Logic would suggest that someone choosing between 169 and 469 wouldn’t change their criterion. But what happened? Purchases of the 239 camera shot up, and the choice of both the 169 and the 469 cameras fell once the middle option was offered. Humans tend to avoid extremes.
And I think that, to some extent, arbitrators, not all of them and not in every instance, but often, even during deliberations within the tribunal, look for something everyone can live with. That human tendency to avoid extremes comes into play, and it is real in arbitration.
Anchoring
No doubt. And it starts with strategy: amounts are put forward with the expectation that the arbitrator will decide in Solomonic fashion, and the arbitrator, in turn, thinks, “They must have framed it that way, so I’ll go for something in between.” It becomes a game, I would say.
Absolutely. When you talk about strategy, that ties in with another bias: anchoring. I imagine the audience is familiar with it, but let me give you an example from an experiment Kahneman used. Anchoring has to do with the fact that, once a number, or even a concept, is presented to a person, we tend to frame our decisions, our world, in relation to that point.
The experiment went like this: a group of subjects was asked to estimate, without using a phone or a computer, the product of 1 × 2 × 3 × 4 × 5 × 6 × 7 × 8.
That’s called eight factorial.
Eight factorial, exactly, here’s the economist, thank you. The group’s average answer was 512. Another comparable group was asked to do the same exercise, but in reverse: 8 × 7 × 6 × 5 × 4 × 3 × 2 × 1. We all know that the order in which you multiply doesn’t matter; the result is exactly the same, and that is what our minds should tell us. What was the average in the second exercise? 2,250. Starting with 1, the result was 512; starting with 8, it was 2,250, because of anchoring: we start with a larger number and that frames and limits our reasoning. And note that the product is the same.
What you have pointed out is the number we start with when we try to set the frame of reference for the arbitral decision. There is some science to this, but it is also an art, because you can’t start with 1 billion dollars either: there comes a point where you lose credibility. But it is definitely something lawyers, as advocates whose job is persuasion, have to take into account. And the arbitrator, as you rightly said, has to be very alert and ask, “Wait, am I falling victim to my own anchoring bias? Am I letting the number I was given at the outset artificially affect my reasoning about what the number should be?” I think what you’ve pointed out is extremely important.
Something similar happens with economic reports, not so much in international arbitration but in domestic arbitration, where reports are submitted during the evidentiary stage and do not necessarily match the amount claimed. For example, the claimant hires me, I review the evidence and arrive at a figure 20% lower or 20% higher. That also prompts a reaction from the arbitrator, who understands that the figure is honest, since it differs from what the party that hired me claimed, and, on the other hand, understands that the original figure was not a bluff, but was based on expectations. It’s not the same thing, but it’s interesting.
Authority bias
José, do you have any other bias in mind?
There are several. First, this conversation could be fascinating quite apart from arbitration, because there are countless biases, but the ones I mention are those I have experienced in arbitrations, in real cases. One is authority bias: a psychological phenomenon by which humans tend to give more weight to the opinions of a person vested with authority. Years ago, when the doctor in his white coat said something, people said “all right.” The world has changed, but I’m not referring to formal authority, like a police officer telling you “do as I say because I can take you to jail.” I mean the psychological authority of a professor, for example.
I wasn’t going to mention it, but there is a very funny experiment. They conducted a study with university students: they told them the professor couldn’t be there that day and that a substitute would come. To one group they introduced him as a postdoctoral fellow; to another group, the same person, as an assistant professor; and to another, as a full professor. Each time they asked how tall he was, and they found that the same person was about two inches taller when he was a professor than when he was a mere postdoc. Look at the psychological effect of vesting someone with a title.
And let me tell you how I have seen it in arbitration, something that honestly bothers me, that causes me friction: I have seen how tribunals treat a company’s CEO or chairman very differently from other witnesses, or from anyone else who comes with authority. It’s not a question of respect, because I believe every human being deserves respect, regardless of their title. Let me give you an example. On one occasion I was cross-examining the chairman of a large company, a very skillful man, who was dodging and not answering. I repeatedly asked the presiding arbitrator to instruct the witness to answer my questions, which were very clear and left no room for doubt. Eventually, the presiding arbitrator told me, “This gentleman is answering more than many other chairmen I have seen.” Note: he didn’t say “your question is improper, Mr. Astigarraga,” or “the gentleman has answered your question,” or “the gentleman is answering like any other witness.” He told me he was answering better than other chairmen. That cannot be the standard. The testimony of this gentleman was being given additional weight it did not deserve, merely because he was a chairman. It must be based on the objective reliability of his testimony. It is an example of how, subconsciously, we suffer from biases; in this case, authority bias.
There is something similar with expert reports, which is what I see more often: a renowned expert is assessed differently from a young expert. And in the end what matters is not who has the best argument, but who has more history on their shoulders to defend it.
Mitigating biases: mindfulness, time and artificial intelligence
José, there is a question I want to ask you before we run out of time. There may be many biases, those we have discussed and many more, but the important thing, once you realize they exist, is to look for ways to design mechanisms to reduce them, to mitigate them. Do you have any ideas on how to deal with this so that, at least in the world of arbitration, they affect arbitral decisions less?
Extremely important, because so far we have been talking about the problem, and you’re focusing on the solution. Excellent question, Jorge. The awareness we now have of how biases affect our profession and our role in society is something other professions have also been studying. Medicine, in particular, is very advanced in this: doctors are very aware of how biases operate and are working very consciously to overcome them. The U.S. judiciary is also very aware of how biases affect its decision-making. The Federal Judicial Center has been studying the subject, and one of its recommendations is what is called mindfulness.
Let me quote a Federal Judicial Center report: “Mindfulness is an extraordinarily simple and universal concept. Its value has drawn growing attention in business, education, medicine, the legal context and mediation. In essence, it involves slowing down one’s mental processes enough to notice as much as possible about a given moment or situation, and then acting carefully based on what one has noticed. It is sometimes described as thinking while one thinks.” What’s the point? What they are saying is: “Slow down, wait, we are processing too fast. Audit what you are doing. Do you really have grounds for that conclusion?”
And we come back to Kahneman, where we started. Kahneman won the Nobel Prize, so I don’t pretend to summarize his work here, but if I have to sum it up, he says human beings operate on two levels, System 1 and System 2. System 1 is what we use in everyday life: driving, thinking, talking to the kids. System 2 is when we are truly focused, trying to use our best reasoning to make the right decision. Both systems suffer from biases and flaws, but System 1 is much more vulnerable. What the federal judiciary is doing is saying, “Wait, let’s recognize that and slow down.” That helps reduce biases.
I have two more comments on how to counter biases. The next one is not controversial: time. Again, Kahneman says that if we move too fast we tend to make these mistakes. To overcome, for example, anchoring or confirmation bias, you need more time to make sure you are not making mistakes. In the case of the presiding arbitrator I told you about, that wasn’t possible, because the arbitration was already well advanced. But perhaps, with more time, I could have gotten that arbitrator, who was not a malicious person, to say, “Gosh, am I being fair? Let me open my mind a little.” For that you need time, and we didn’t have it. So that is the second factor. And the third, and here we’re going to be a little provocative, is artificial intelligence.
In fact, in medicine it has been a very effective way of reducing biases in some cases.
Of course. In fact, I recently gave the keynote speech at New York Arbitration Week on precisely this topic: how artificial intelligence can help us overcome our biases. I started playing around, that’s the best way to put it, with ChatGPT and some other models to see how they help us, and I can give you two very simple examples. I mentioned Kahneman’s experiment of multiplying from 1 to 8, which yields 512, and that if you do it the other way round the mind suffers from anchoring and comes up with a much larger number. I ran exactly that test, and ChatGPT gave me the correct answer instantly, without suffering from anchoring.
If we want to apply it to arbitration, I don’t claim to solve it right now, but I can see that artificial intelligence could be very useful in protecting us against the biases we suffer from. For example, and I may say something outrageous, let’s see if we can provoke a reaction: an arbitrator could take a draft award and ask artificial intelligence whether everything is well reasoned, in line with the evidence.
Whether it’s logical.
Exactly. And I know some people will be alarmed, but I don’t in the least suggest that artificial intelligence should replace the arbitrator’s judgment. It is simply a check, a verification. And I can easily see written submissions being analyzed and someone saying, “There is a bias at work here, relating to the numbers,” as you said.
Another example is hindsight bias, which we hadn’t discussed. It says that humans tend to overestimate the probability that something was going to happen once we know it has happened. As the Spanish saying goes: after the bull has passed, we’re all bullfighters. Studies have shown that judges suffer from hindsight bias; think of the injustice of that. I have seen it in my arbitrations, for example in a mergers and acquisitions arbitration, an M&A arbitration, in which there was a warranty that was triggered or not depending on whether the party knew that something had happened or was going to happen. After the event occurs, the warranty is triggered, and the arbitrators have to assess whether the person who gave the warranty should have known about it. Of course, after the fact, everyone knows the answer to every question. I have seen hindsight bias at work in that situation: it is very easy to say “you should have foreseen this,” when in fact nobody foresaw it ex ante.
How does artificial intelligence help us with that? One way to mitigate hindsight bias is to present the alternatives: the scenarios, the different ways in which the world could have unfolded. You can ask artificial intelligence, and I did the exercise: “Tell me all the ways in which these events could have unfolded, what could have happened.” It gave me a very long list of scenarios. And once you present that to the arbitrator, hopefully he can say, “Gosh, I’m being unfair, because all these possibilities could indeed have occurred.”
It’s called counterfactual thinking. It’s my favorite topic, José: damages estimates are always the search for a good counterfactual. But we can talk about that some other day. I want to thank you for such an enjoyable conversation. I’m sure everyone listening will enjoy this episode very much, because these are things that must sound familiar to them, even, I would say, part of their everyday practice in the world of arbitration.
On the contrary, thank you and FK for having me. Delighted to take part in this excellent program.
