Katy Milkman: Imagine this scene: Two men, let’s call them Mike and Dave, separately go out for a few beers at their local bar. Both men are 35 years old. They both weigh about 190 pounds. They play some darts with their buddies. They each have a plate of nachos, and they each watch some game highlights on TV screens hanging over the bar. At the end of the night, each man has had five drinks. Both Mike and Dave are now clearly intoxicated. Mike fumbles for his keys and gets into his car. Dave fumbles for his keys and gets into his car. They both have a five-mile drive to their respective homes, and both men have clean driving records.
Mike weaves in and out of traffic, blows through a stop sign, and nearly misses a telephone pole, but ultimately, he makes it home in one piece. Dave weaves in and out of traffic, blows through a stop sign, and smacks right into a telephone pole. His car is wrecked. He’s broken one leg, several ribs and has multiple lacerations. He was lucky there were no other cars involved. Dave is placed under arrest at the hospital for driving under the influence. Mike, on the other hand, woke up with a bit of a hangover the next morning, went to work, came home, walked the dog and then went out to the bar for a few drinks.
Mike and Dave made the same choice on that fateful night. Mike feels like his choice paid off. He made it home safely and saved some money on cab fare. Dave feels he made a terrible choice, and he’s got the stitches, cast and criminal record to prove it. The thing is, Mike and Dave made exactly the same decision under exactly the same conditions. Now, I hope you would never get behind the wheel after a few drinks. It’s illegal, unethical and can have horrible consequences, not only for drunk drivers but for innocent victims. What’s peculiar about the story about Dave and Mike is that even though they made the same bad decision, the very different outcomes they experienced strongly shaped the way they thought about their choices. Today, we’re going to look at the way we tend to judge the quality of our decisions based on their results rather than what we knew back when we made them.
I’m Katy Milkman, and this is Choiceology, an original podcast from Charles Schwab. It’s a show about decisions and the impact those decisions have on our lives. It’s also a show about subtle human biases that push us in one direction or another, often without us even realizing it. We try to give you some tools to fight back against those psychological forces and to help you avoid costly mistakes.
Before we get started, I should warn you that there are some descriptions of medical procedures in this episode that some listeners may find uncomfortable or upsetting.
Bud Shaw: I arrived in Pittsburgh in the summer of 1981 and was taken around the hospital in a tour by the nurse coordinator, and she took me to the intensive care unit where the adult …
Katy Milkman: This is Dr. Bud Shaw. He just arrived at the University of Pittsburgh Hospital to begin a surgery fellowship.
Bud Shaw: A young medical student in a white coat with a stethoscope draped over her shoulder came running up, and she had in her arms a petition, and she asked me to sign. I said, “What’s this for?” She said, “We’ve got a petition circulating against liver transplantation. We want to stop them because they’re unethical.” I explained to her that I was a new fellow and had just joined the transplant team. She said, “Well, then you should definitely sign it.”
Katy Milkman: Liver transplants, at the time, were brutal affairs and often unsuccessful.
Bud Shaw: When I got interested in transplantation in general, liver transplantation was really not on the forefront of my mind at all. In fact, that was in 1978 or ’79. The survival rate among liver transplant recipients then was about 35, 38%. Somebody once made a comment at an international transplant meeting that liver transplantation was a way to torture somebody before they died. That was, I think, the general attitude. I certainly signed up with Dr. Starzl with the attitude that I would get exposure to lots of other kinds of transplants, but that I probably wouldn’t be that interested in livers just because I thought they were kind of hopeless.
Katy Milkman: Dr. Shaw’s mentor, Dr. Thomas Starzl, had performed the world’s first successful liver transplant in 1967. The patient survived for 400 days before ultimately succumbing to a recurrence of liver cancer. Over the next decade, some advances were made, but progress was slow. Starzl would go on to start a liver transplant program at the University of Pittsburgh about six months before Bud Shaw arrived.
Bud Shaw: The thinking among the staff at that time was that this was hopeless. I think that it was mostly related to the fact that I think they had done maybe six liver transplants in adults by the time I got there. All six of those patients eventually died.
Katy Milkman: These were not encouraging results, but Dr. Starzl figured there must be solutions to the problems they were facing.
Bud Shaw: It was early days at Pittsburgh, and there were still lots of kinks to work out in setting up the program, but I think Dr. Starzl felt that all of the problems they were having and all of the reasons that the patients were dying were all things that they could solve if they just kept working harder.
Katy Milkman: Starzl’s optimism convinced Dr. Shaw to join the liver transplant team, and he began to observe and assist during the procedures.
Bud Shaw: I’d say that in the earliest part of it, the biggest risk was the surgery. It was a matter of trying to figure out how do you take out a liver from somebody, keep them alive while there’s no liver in there, all of the blood that goes to the lower part of your body has to go through the big vein behind the liver, which is called the vena cava, and back to the heart. When you go to take the liver out, you’ve got to cut through all those huge veins. There’s a matter of tremendous bleeding risk. I think that’s one of the most dramatic parts of some of the early days of liver transplantation was the massive amount of blood that would be lost and have to be replaced rapidly during the surgery. That was an astounding thing, and I think that was one of the biggest hurdles.
Katy Milkman: These surgeries were intense. The procedures took many hours to complete and required extreme focus and concentration and skill.
Bud Shaw: In the late winter, early spring of 1982, I got a phone call that they had a donor liver for one of the patients who had been waiting on our list for probably six months or so. It was a fairly healthy but older woman that I had actually seen in the clinic and evaluated before. I was pretty excited about it, but then I quickly learn when I started making phone calls to try to find a senior surgeon that none of them were in town. I had to call up the chairman of the department of surgery because I was a fellow. I did not have privileges to operate alone.
I told him that we had this donor liver and that this lady has been waiting a long time and was getting sicker and sicker and needed a transplant and that I felt confident that I could do the transplant. It was left up to me to go down to this area outside the operating rooms, in the recovery room area, and interview the patient, interview her husband, do a complete physical exam, and get her to sign a consent form for the liver transplant and make sure that they both had a good understanding of the risk of complications, the risk of death and that sort of thing.
The woman was actually much sicker looking than I had remembered. You could see hollows in her cheek—the areas around her temples where the muscles usually fill out were quite emaciated and hollow looking. They kept saying, “Where’s Dr. Starzl?” I kept saying, “Well, he’s out of town.” They said, “Well, who’s going to do my surgery?” I said, “Well, I’m going to be doing it.” At one point, the husband even looked at me and asked me how old I was. That’s when I told him I was 32 years old.
Katy Milkman: He was 32 years old, but Dr. Shaw had completed a number of liver transplants at this stage of his career. He felt reasonably confident that he could handle this particular operation on his own. Besides, he had no choice. He was the only liver transplant surgeon available.
Bud Shaw: The operating room in Pittsburgh that we did most of the liver transplants in was pretty large, and it also had a gallery over top, so there’d be times when you’re doing an operation, where if you looked up, you’d see this window that circled all the way around the ceiling of the operating room, and you’d see people up there, looking down. There would often be lots of visiting surgeons from China and Japan and Italy. I actually felt quite a bit of confidence and didn’t think much about the people in the gallery looking down on me. The operation actually went pretty well. We got scrubbed. The patient came in.
Katy Milkman: The surgery began. The delicate procedure went without a hitch for several hours. The new liver had been successfully transplanted and everything was going smoothly, but the patient was still losing blood.
Bud Shaw: We were sitting there, working our way of getting the blood to stop, and I was starting to think maybe it was coming around, when all of a sudden, her heart stopped, just for no reason. She didn’t have a drop in blood pressure. Her heart just stopped. We got it going pretty quickly, but then her blood pressure started dropping and stopped again.
Katy Milkman: What Dr. Shaw and his colleagues didn’t know at the time, because it hadn’t been discovered yet, was that the donated blood used during the operation tended to leech calcium out of the body. This could result in poor blood clotting and cardiac arrest.
Bud Shaw: I had a sense of impending doom about the second or third time we had to pump on her chest, and she began taking longer and longer to come around. I think I went on and on and on with the pumping on her chest. Now, if I would’ve just opened my eyes a little bit more, I would’ve realized that everybody thought this was hopeless already. I didn’t. I kept going. I think when I finally called it and kind of looked up and I said, “Well, I’m afraid we’re going to have to quit,” I realized that nobody was looking at the patient. Everybody was looking at me. I think that I was overwhelmed by the sense that they all knew what had just happened. I had just lost a patient on a table. I was not prepared for that. That was pretty devastating.
I think it’s more a feeling of being completely numb and a sense of utter despair and failure and this. … Eventually, I started thinking about what we could’ve done differently. At that moment, it’s like being unable to feel anything, but then I realized that I was going to have to go talk to the husband. I realized that that man who had been so incredibly nervous and who wanted Dr. Starzl to do the operation, I was going to have to talk to him, and I had told him, “Don’t worry. Everything will be fine. Your wife is going to be fine. I’ve done lots of these.”
I found the patient’s husband in the surgery waiting room. There’s nobody else there. There was no other surgery going on. He was sitting over in the corner. He had a magazine in his lap, but he looked like he had been dozing. He stood up, and he looked expectant. He didn’t look worried that much. I walked up and I said, “I’m really sorry that we lost her.” He was absolutely shocked at that moment. He couldn’t believe it. He said, “What do you mean you lost her?” I told him that her heart has stopped, that we had tried to keep it going.
I noticed that he still had that magazine in his hand, and he had begun rolling it up, tapping it on the side of his leg, kind of nervously and pacing. I didn’t know what else to say at that point. I told him that somebody would come talk to him. One of the things we need to know is if there’s any family that we should contact. He told me that I had just killed his only family, that they didn’t have any kids, that they didn’t have any other relatives, that his wife was his only family.
One of the surgeons that had been with me that night was a surgeon from Scotland. I walked in the room, and he was sitting down on the floor beside the door. He came over and helped me. What we’d have to do is pack the body up in what’s called a death kit. We’d wrap them up and then take them in a cart to the morgue, not exactly what you want to do after you’ve just operated on them and they died on your hand, in your hands. I asked him to go get a gurney, and he came back. He said he couldn’t find one. I suddenly got incredibly lightheaded and just started sort of weeping and kind of started to collapse. I had that kind of sense of the black curtain coming down over me. I think he tried to grab me, and we both sort of collapsed to the floor. I don’t remember much after that.
Katy Milkman: This is a moment that, unfortunately, most surgeons face if they do complicated procedures.
Bud Shaw: I think there’s a temptation whenever anything goes bad like this is to think that, well, I shouldn’t do this again. The outcome is bad. I did everything I could. The outcome is bad. I don’t want to do that again.
Katy Milkman: When it comes to medical procedures, even when everything is done right, there are many factors that can lead to bad outcomes. Surgery is just inherently risky. The body is delicate and complex. Our instinct when something terrible happens is to quit, to avoid potentially experiencing that horrible outcome ever again in the future. Thankfully for the medical community and the many patients who needed him, rather than quit, Dr. Shaw took inspiration from his mentor.
Bud Shaw: One of the reasons that Tom Starzl was called a pioneer was because he was one of those people that could keep going even in the face of failure. For him, although I’m sure the outcomes were important to him, the fact that patients would die or live was not the main inspiration that he had to keep going. He had confidence that it was much more complicated than that, that the outcome would improve. Just because the outcome today was a failure didn’t mean that the outcome tomorrow wouldn’t be better.
Katy Milkman: While this particular surgery failed, Dr. Shaw was eventually able to move on and achieve a great deal of good and save many more lives in his career. He was able to recognize that even though this outcome had been bad, his training and performance in the operating room weren’t the problem. He wasn’t a flawed surgeon. Thank goodness he didn’t quit.
Bud Shaw: In many ways, it was devastating and horrible, and the memory of it has stayed with me. In another way, it was a turning point in terms of recognizing that you can learn something from these experiences that will prevent it from happening later, and that’s why you should keep going. That’s why you should do another one. It felt to me like it was an obligation to take that knowledge and what I’d learned from that case forward and make things safer for others.
Katy Milkman: Dr. Bud Shaw is the chairman of the University of Nebraska Medical Center Surgery Department. He’s also the author of Last Night in the OR: A Transplant Surgeon’s Odyssey. I’ve got links in the show notes and at schwab.com/podcast.
Dr. Shaw failed in his attempt to save this particular patient’s life. It was a devastating blow, but remember, the procedure at the time was incredibly risky. It’s likely that had the surgery been performed by his mentor, Dr. Thomas Starzl, the result would’ve been the same. Regardless, Dr. Shaw was crushed. We’re all lucky that he picked himself up and continued to perform liver transplants. Each operation yielded valuable information and contributed to improved outcomes for future transplant patients. It’s in no small part thanks to his work and that of his colleagues and his mentor, Dr. Starzl, that, according to a recent study, people who’ve had a liver transplant in the U.S. now have a 75% chance of living at least five years. That’s an improvement of around 40 percentage points from the time when Dr. Shaw began performing these procedures.
The aspect of this story that I find most interesting is the way Dr. Shaw viewed his performance of the transplant procedure. Going into the operation, Dr. Shaw was confident he could get a successful outcome, and then things outside of his control went wrong, and he lost his patient on the operating room table. He was devastated, and he felt that he was a failure. Was the decision to operate a good one or a bad one? The patient desperately needed a transplant to have any hope of survival. A donor organ was available, and Dr. Shaw was the only surgeon in the vicinity who was qualified to perform the procedure. Dr. Shaw couldn’t have known at the time that donated blood was problematic, so it’s not fair to include that variable in our evaluation of his decision, but the patient did die. Was it a mistake to operate?
There are many situations where the results of a decision affect the way we perceive the quality of the decision. I want to switch from the high-stakes operating room to a much lower-stakes setting. We sent one of our producers to a basketball practice to try out an experiment. We’re running a hypothetical scenario using real players. Here’s the situation we presented. Their team is two points down. The clock has nearly run out, and the coach has called her players over to discuss a play to shoot from the three-point line to win the game.
Coach: We’re down by two here. We have a few seconds left. You pass to him. You’re going to take the three-point shot for the win. The game rests on your shoulders.
Player 1: “Team” on three.
Group: One, two, three, team.
Player 2: Go.
Player 1: All right. Clear out, clear out, clear out.
Player 2: Go. Go.
Coach: Let’s go.
Group: Aww.
Producer: OK. What just happened there? Did you make the shot, or did you miss the shot?
Player 1: Missed.
Producer: Do you think the coach made the right call going for the win with the three-point shot?
Player 1: The coach made the wrong call. We should’ve gone for the two.
Katy Milkman: OK, makes sense to question the strategy. The team could’ve averted a loss in that scenario. Here’s another group of players in a similar scenario, but with a different result.
Group: Yeah.
Producer: What just happened there?
Player 3: I scored a three-pointer, and it’s a game-winning shot.
Producer: Do you think the coach made the right call by getting you to go for the three points?
Player 3: Yeah, good coach, good strategy, and we won.
Katy Milkman: It’s funny to hear those two scenarios back to back, isn’t it? The coach made the same call in both situations, but the result was different. When they made the shot, the players felt the strategy was the right one. When they missed the shot, the players doubted the strategy. Of course, the sample size for this test is tiny, but similar, larger experiments demonstrate this type of reasoning is common. Importantly, there’s a bias at play here. It’s called outcome bias, and it’s exactly what it sounds like. It was first documented in the 1980s by two of my emeritus colleagues at the University of Pennsylvania, Jack Hershey and Jon Baron.
Outcome bias describes people’s tendency to think if the outcome was good, the decision was good. If the outcome was bad, the decision must’ve been bad, but this is not strictly rational. That’s because a good decision cannot guarantee a good outcome, given that all real decisions are made in the face of some uncertainty. To paraphrase the great decision theorist Ward Edwards, a decision is a bet, and evaluating it as good or not depends on the stakes and the odds, not the outcome. Francesca Gino is a behavioral scientist and professor of business administration at the Harvard Business School. She joins me to help explain outcome bias.
Fran, thank you so much for joining me on the show. I really appreciate it.
Francesca Gino: Katy, it’s such a pleasure to be here with you today.
Katy Milkman: Let me start by asking you a question that’s sort of key to this episode, which is, could you tell us a little bit about outcome bias—what is it, and where does it come from?
Francesca Gino: The very simple insight behind the outcome bias is the fact that we often judge the quality of the decision not based on the decision itself or the process that led to it, but we are influenced by the outcome, and so whether the outcome was a good one or a bad one directly influences the way we think about the decision itself. It’s quite problematic because if you think about it, process matters. There is helpful information in both the intention and the process that should really influence how we think about the quality of the decision, and yet, it is trumped by the outcome.
Katy Milkman: Fran, what does research point to as the reason for this bias because, of course, outcomes are important, but why is it that we weigh them so much, relative to process?
Francesca Gino: Part of it is that we believe that the outcome is the most important thing for us to look at. Somehow, it tends to color all other important factors that should go into the decision, like the process that led us there. It’s interesting because you can look at very different context where, often, the outcome is not a positive one. It’s a negative one. For instance, with some of my colleagues, Max Bazerman here at HBS, but also Don Moore at Berkeley, for a while, we would look at situations where the process is a shady one. Think about people engaging in misconduct or cutting corners, but the outcome is a good one. Usually, nobody investigates or is curious about how we got where we got. For us, that was problematic because, again, it might make us miss important processes that are shady and should, in fact, be questioned.
Katy Milkman: Fran, since you mentioned those studies you’ve done with Don Moore and Max Bazerman, could you tell us a little bit more about the specifics of these studies that you did on outcome bias and ethical decision making?
Francesca Gino: Yes. We were interested in situations where processes or actions that people engage in are shady. If you just look at the actions per se, we would say, “Yup, they are ethically questionable, or they’re not actions we were going to engage in,” but the way we presented people with different scenarios were situations where we actually told them about the outcome. It could be situations where a pharmaceutical company puts out product where initially the tests were worrisome because they showed that there might be some side effects that are bad for people. Then, when they actually put the product on the market, it seems as if the patients that are using the products, or the customers that are using the products, are not suffering any of the side effects. Or situations where you’re trying to sell a product to a client, and you’ve run some tests, and in the presentation you are misleading because you only focus on the data that suggest that the tests were good.
Again, you can imagine in those situations where the actions are shady that the outcome is either a good one—your client buys the product or the service and they’re happy with it—or is bad—the client buys the product or the service, and then they end up with a product that is not a good one for them or a service that is problematic. What we were interested in understanding is how the knowledge about the outcome, so good or bad, would influence how people judge the ethicality of the actions. What we found across many different studies that all involve this type of vignette is that once we have knowledge of the outcome, as long as the outcome was good, we think of the action as less ethically problematic.
Katy Milkman: That’s so interesting. Of course, really important as we think about who gets punished in society and who is a real risk, because punishment is also meant to alleviate future risk, right? Is that the core of the concern that this raised for you?
Francesca Gino: That’s exactly right. In fact, in some of the studies as an additional measure that we looked at is whether people would punish the actor behind the actions that we described in the vignette. What we found is that people generally punish others when they think that their actions are ethically problematic, and since a good outcome lowers the perceptions that an action is ethically problematic, people would be more lenient towards others whose unethical actions, basically, led to good outcomes.
Katy Milkman: Have you read about or done any research into solutions for outcome bias? So if we wanted to help people be less outcome biased and more focused on the process and the inputs, what might we want to do?
Francesca Gino: It’s a really important to ask because both you and I and many other behavioral scholars are really interested in fixing problems rather than just pointing to the problems themselves in the first place. We tried to focus people’s attention on the intentions behind certain actions. We weren’t super successful, I would say, in fixing the problem. It’s interesting that what this might suggest is that the bias is quite pernicious, if you will, and it takes quite a bit of deliberation to make people aware of the error that they are making. What seems to be helpful is giving people the counterfactual—meaning, if you walk them through the possibility that the good outcome was potentially a lucky one, but that they could’ve been in a situation where the outcome was bad, and so people can sort of evaluate both the situations simultaneously, then they seem to be able to understand that the action, per se, was unethical. That seems an interesting potential intervention, but if you think about how we go through life, counterfactual thinking—thinking about how the same action could’ve lead to different outcomes—is something that we don’t do naturally.
Katy Milkman: Thank you so much, Francesca. It was wonderful to talk to you about this.
Francesca Gino: Thank you so much, Katy, for having me.
Katy Milkman: Francesca Gino is a behavioral scientist and professor of negotiations, organizations and markets at the Harvard Business School. She’s also the author of the book, Rebel Talent: Why It Pays to Break the Rules at Work and in Life. I’ve got a link in the show notes and at schwab.com/podcast.
Outcome bias can affect your ability to stay the course with an investment portfolio aligned with your goals. Just imagine buying a stock that quickly went up in value but actually exposed you to more risk than you’re comfortable with. The short-term outcome might seem to justify the move, but then …? That’s why Schwab also produces the podcast Financial Decoder. It’s designed for people who want to make better decisions with their money. Mark Riepe, head of the Schwab Center for Financial Research, hosts the show. Mark and his guests dissect the financial choices you might be facing, and offer tips to mitigate the impact of biases on your financial life. You can find it at schwab.com/financialdecoder or wherever you listen to podcasts.
What can you do about outcome bias? Well, one important strategy is to force yourself to focus on the decision-making process itself and not just the result it produced. Sometimes, a good process produces a bad outcome, and sometimes, a bad process produces a good outcome. Your goal should be to suss out if good and bad outcomes you experienced are due to luck or the choices and processes that preceded them. There are a number of questions you can ask to get at this. In short, you can do a process audit. If you’ve achieved a positive outcome, it’s still useful to ask yourself if there was a better or more efficient or more ethical way to get to that outcome. If the result was negative, it’s worth it to ask yourself if you made the same decision and got a positive outcome, would you feel the same way about your choice?
Maybe this particular negative result was just a product of chance. Maybe it was simply bad luck. Is there really a flaw in the process that needs correcting? It might also be useful to question the timing of the decision. Would the decision have changed if it was made later, for example, after more information had come to light? Outcome bias can lead you to repeat poor decisions, like the drunk drivers at the beginning of the show. In that scenario, an informed decision not to drive under the influence would’ve been a much better choice. The odds of a safe trip home were unquestionably higher, even if they’re never 100%. Accidents still happen when drivers are sober, and drunk drivers can get home safely, but those outcomes obviously don’t mean that driving drunk is a good decision.
In the case of Dr. Bud Shaw, outcome bias could have led him and his colleagues to abandon liver transplant surgery altogether because of some of the poor patient outcomes in the early days of the procedure, but he was able to take a longer view. Sometimes, good surgeons do everything right and lose patients on the operating table, particularly with risky new procedures. Sometimes, surgeons do things wrong, and their patients turn out to be just fine. The key is to identify what aspects of the process are contributing to a good outcome, and which aspects of the process are contributing to a bad outcome, and then make improvements, and eventually achieve better outcomes on average. By evaluating the process as well as the outcome, you can make better decisions in the future.
You’ve been listening to Choiceology, an original podcast from Charles Schwab. If you’ve enjoyed the show, leave us a review on Apple Podcasts, and while you’re there, you can subscribe for free. Same goes for other podcasting apps. Subscribe and you won’t miss an episode. Next time on the show, we’ll bring you a fascinating story about one of the most pivotal moments in World War II. From there, we examine a tendency people have to see certain events in history as inevitable. I’m Katy Milkman, talk to you next time.
Speaker 10: For important disclosures, see the show notes or visit schwab.com/podcast.
