Judge the Judges: With Guests Daniel Kahneman, James Hutchinson & G.M. Pucilowski

March 16, 2020
The chance variability of human judgment is a widespread, costly problem—and one that can be measured even when the supposed correct answer isn't known.

Listen on Apple Podcasts, Google Podcasts, Spotify or copy to your RSS reader.

Transcript Open new window

Many episodes of this podcast deal with cognitive biases that can hinder our decision-making abilities. In this episode of Choiceology with Katy Milkman, we look at a different kind of error: how completely irrelevant information can negatively influence our judgments, making them varied and unpredictable.

This variability of human judgment—or noise—is the topic of an upcoming book by Nobel Prize-winning economist Daniel Kahneman, with Cass Sunstein and Olivier Sibony. You'll hear an interview with Kahneman later in the episode where he explains his preoccupation with the substantial and expensive effects of noise. He proposes ways to reduce the problem of noise for industries, businesses, and individuals who need to make more objective judgements.

But first, we'll dive into the world of wine judging. G.M. "Pooch" Pucilowski will take you on a guided tour of the California State Fair Commercial Wine Competition. You'll hear about the criteria for judging different varietals—and the accompanying challenges that wine judges face as they swirl, sniff, and sip through dozens and sometimes hundreds of different wines.

After years of coordinating and observing the judges, Pooch noticed a large amount of variability in the results. This may not be surprising, since taste is subjective. But after some tweaks to the process, he even began to notice that judges were inconsistent with themselves!

Enter vintner and retired oceanographer Robert Hodgson. Pooch teamed up with Hodgson to devise a way to study and improve the consistency of wine judging and push for a more objective competition. The results were promising, but not without controversy.

You can read Robert Hodgson's research paper on wine judging here.

G.M. "Pooch" Pucilowski is a speaker, writer, wine judge, and educator. He runs wine appreciation classes through his University of Wine.

You'll also hear about the potential role of chemical analysis and artificial intelligence in improving the results of wine judging from James Hutchinson, formerly of the Royal Society of Chemistry and currently CEO of KiwiNet.

And finally, Katy explores the potential of leveraging noise to produce better decisions by employing the wisdom of crowds—and even "the crowd within."

Choiceology is an original podcast from Charles Schwab.

If you enjoy the show, please leave a rating or review on Apple Podcasts.

Learn more about behavioral finance. 

Mary Anne's Story: How Can You Leave a Legacy and Honor a Loved One?

How did one woman honor her twin sister through a charitable gift?

Should You Start Giving Money to Your Heirs Now or Leave a Bequest?

Giving away assets in one's lifetime might make sense tax-wise, but the issue is deeply emotional and personal for most people.

Mesmerized: With Guests Mara Rockliff & John List

Why is it so difficult to separate correlation from causation?

Related topics

All expressions of opinion are subject to change without notice in reaction to shifting market conditions.

The comments, views, and opinions expressed in the presentation are those of the speakers and do not necessarily represent the views of Charles Schwab.

Data contained herein from third-party providers is obtained from what are considered reliable sources. However, its accuracy, completeness or reliability cannot be guaranteed.

Investing involves risk, including loss of principal.

Apple Podcasts and the Apple logo are trademarks of Apple Inc., registered in the U.S. and other countries.

Google Podcasts and the Google Podcasts logo are trademarks of Google LLC.

Spotify and the Spotify logo are registered trademarks of Spotify AB.

0320-0FLK