Speaker 1: M. This is Mesters in Business with Very Results on
Speaker 1: Bloomberg Radio this weekend on the podcast What Can I Say?
Speaker 1: Another extra special guest Danny Kaneman, no Bell Prize winner,
Speaker 1: author of Thinking Fast and Slow. His new book is Noise,
Speaker 1: a Fawn Human Judgment and Danny is just so knowledgeable.
Speaker 1: Please call me Danny. I I feel like I have
Speaker 1: to call him Professor Khneman, and he he insists. Uh,
Speaker 1: He's eighty seven years old and incredibly sharp and insightful
Speaker 1: and just so much wisdom and knowledge. If you liked
Speaker 1: Thinking Fast and Slow, which is about judgment error in
Speaker 1: humans in individuals, well, Noise is about how flaws and
Speaker 1: in judgment within broader institutions come about. And it's a
Speaker 1: totally different area and it's absolutely fascinating. I'm a big
Speaker 1: fan of behavioral finance in general, plus all of uh
Speaker 1: Danny's work historically. If you are remotely interested in this,
Speaker 1: then strap yourself. And this is another doozy with no
Speaker 1: further ado. My conversation with Danny Koneman. This is mesters
Speaker 1: in Business with Very Results on Bloomberg Radio. My extra
Speaker 1: special guest this week is Danny Khneman. He was awarded
Speaker 1: the two thousand and two Nobel Memorial Prize in Economic Sciences,
Speaker 1: which he shared with Vernon Smith for his empirical findings
Speaker 1: the work he did with Amos Tversky. And what's so
Speaker 1: fascinating about that Nobel Prize is that Danny is a psychologist.
Speaker 1: The work they did challenge the prevailing thoughts in economic
Speaker 1: theory by establishing a basis for common human eras his
Speaker 1: previous book, Thinking Fast and Slow, was the best seller
Speaker 1: of two thousand and eleven and one a variety of
Speaker 1: different awards, including the National Academy's Communication Award for Best
Speaker 1: Creative Work. His latest book is Just Out Noise, A
Speaker 1: Flaw in Human Judgment, which Danny Koneman wrote with Olive
Speaker 1: Simony and Cass Sunstein. Danny Kneman, welcome back to Bloomberg.
Speaker 1: I'm delighted to be here. You always say call me Danny,
Speaker 1: and I always feel awkward and I feel like I
Speaker 1: should call you professor. But let me just get that call,
Speaker 1: all right, Danny. So let's start very basically. What is noise?
Speaker 1: How does it happen? And where does it come from? Okay, well,
Speaker 1: noise isn't accepted term in statistics. We talk about statistical noise,
Speaker 1: which is variability, and that's where it comes from. We
Speaker 1: talk about noise a measurement, which is unreliability in uh
Speaker 1: in measurement, where measurements that should be identical turn out
Speaker 1: to vary. So that's the background in the use of
Speaker 1: the term as we use it specifically, we intend we
Speaker 1: speak about judgment noise, and this is the situation in
Speaker 1: which judgments should be identical people or the same individual
Speaker 1: judging the same object at different times, or different people
Speaker 1: judging the same object. If they don't agree and I
Speaker 1: expected to agree, we speak about judgment noise, and in general,
Speaker 1: people are expected to agree when they're trying to be accurate.
Speaker 1: So when you have a group of people trying to
Speaker 1: make their best guess about the quantity, it could be
Speaker 1: the symptoms that somebody should should get for a crime.
Speaker 1: It could be the value of the company. It could
Speaker 1: be the premium that somebody should be charged. Oh, it
Speaker 1: could be a diagnosis, a medical diagnosis. In all these cases,
Speaker 1: you might have several people looking at the same information
Speaker 1: making judgments. If they don't agree, there is noise, and
Speaker 1: noise is the topic of the book we wrote. So
Speaker 1: it's fascinating how we start to see noisy decision making
Speaker 1: come up over and over again in the same fields.
Speaker 1: And you just mentioned a few medicine, criminal justice, finance.
Speaker 1: Are there certain fields that are more susceptible two problems
Speaker 1: in expert judgments than others? Or is it just that
Speaker 1: the results of those sort of noisy decisions are so
Speaker 1: much more significant than other fields. Well, we use the
Speaker 1: word judgment when there is room for reasonable disagreement, that is,
Speaker 1: you know, we don't use the word judgment for computation,
Speaker 1: and when compute station is appropriate, we wouldn't be talking
Speaker 1: of noise. We would be talking of people making mistakes.
Speaker 1: And we talked about noise when when it's a matter
Speaker 1: of judgment. And and so the existence of noise by
Speaker 1: itself is not a surprise. Whatever the surprise is the
Speaker 1: amount of noise just a lot more then would be expected.
Speaker 1: And here, I think the best way to explain this
Speaker 1: is too to tell you the story of how I
Speaker 1: started to work on noise. Then where the whole thing began.
Speaker 1: So I was consulting in an insurance company, said than
Speaker 1: eight years ago, and I had the idea of running
Speaker 1: But today we would call a noise audit, that is underwriters.
Speaker 1: To take one example, we had several underwriters, so some
Speaker 1: realistic cases, the same cases. They were constructed by executives
Speaker 1: of experts and underwriting, so they were completely realistic, and
Speaker 1: you might have fifty underwriters looking at the same premium.
Speaker 1: Now nobody would expect the numbers to be exactly the same.
Speaker 1: But I asked executives, if you take a pair of
Speaker 1: underwriters at random, by how much would you expect them
Speaker 1: to differ in percentages? That is, you take the average
Speaker 1: or the pairer, you take the difference, you divide the
Speaker 1: difference by the average. What percentage looks reasonable to you?
Speaker 1: And they're answer typically with ten percent. And we have,
Speaker 1: by the way, we have surveyed hundreds of executives since then,
Speaker 1: and ten percent seems to be what we expect the
Speaker 1: reasonable difference to be, which is tolerable when two people
Speaker 1: make judgments of a quantity. Now, the correct answer among
Speaker 1: underwriters in that company with sixty five percent more than
Speaker 1: five times as much as expected. That's the phenomenon. So
Speaker 1: we expect this agreement where judgment is involved, We just
Speaker 1: don't expect that much disagreement. And this basically was the
Speaker 1: observation that started us on that part of writing a book,
Speaker 1: because it turns out that you find astonishing amount of
Speaker 1: disagreement when you look for it, and you find it
Speaker 1: wherever judgment is involved. So engineers who make estimates on
Speaker 1: the basis of objected data, they don't have a problem
Speaker 1: of judgment, but to the extent they do have a
Speaker 1: problem of judgment, you will expect a lot of noise.
Speaker 1: So that's the basic finding. And you know, wherever precision
Speaker 1: is important, wherever it is important to get to the
Speaker 1: right number, noise is a source of there. So where
Speaker 1: some people over estimating and others underestimating now making errors. Huh.
Speaker 1: So let me roll back to that insurance company, which
Speaker 1: you discuss in the book, and there were two particular
Speaker 1: areas where we're there were these broad disagreements. The first
Speaker 1: was when people were trying to estimate the risk involved
Speaker 1: with some insurance and so how you price that very
Speaker 1: much determines. If you're too expensive, meaning you think it's
Speaker 1: a high risk, you're not gonna win the business. And
Speaker 1: if it's too cheap relative to the risk, well you'll
Speaker 1: win the business, but it won't be profitable. The costs
Speaker 1: will be higher. And then on the other end, in
Speaker 1: the appraisal of hey, what are the damages here? Figuring
Speaker 1: out how much something should be covered by insurance, what
Speaker 1: the dollar amount is, and the same situation. You can't
Speaker 1: be too stingy or you lose customers, but you can't
Speaker 1: be too generous when you give the house away. How
Speaker 1: significant a financial issue was this for the insurance company. Well,
Speaker 1: you know, it's not easy to estimate that exactly, but
Speaker 1: I can tell you the question that I asked some executives.
Speaker 1: I said, suppose there is a correct number, say for
Speaker 1: the underwriters, and and you have somebody who overestimates the
Speaker 1: underwriting cost by fifteen, how much would you expect that
Speaker 1: to cost the company? And the same question for underestimating
Speaker 1: by fifteen. In fact, fifteen percent on either side is
Speaker 1: much less than noise than we had discovered. But people
Speaker 1: estimated on that basis that this could be in the
Speaker 1: hundreds of millions or billions of dollars. This is a
Speaker 1: very large company, so uh underwriters have a lot of
Speaker 1: important decisions to make claims, adjustice, make important decisions which
Speaker 1: are really consequential for the company. And errors of the
Speaker 1: magnitude that we observe are costly. The main reason that
Speaker 1: they may be less costly is that if error is
Speaker 1: present in all insurance companies, if all insurance companies are noisy,
Speaker 1: then some of the damage to each individual company will
Speaker 1: be reduced. But that's the best that we can say. Well,
Speaker 1: one would imagine the insurance company that could reduce noise
Speaker 1: would find itself at a competitive advantage. Absolutely, there was
Speaker 1: something you had written that really stood out to me.
Speaker 1: There's an assumption when you have noisy systems and everything
Speaker 1: from criminal justice to medicine to insurance, that these errors
Speaker 1: tend to cancel out. But you found out that noisy
Speaker 1: systems have errors. Not only do they not cancel out,
Speaker 1: they tend to add up. Explain, well, if you have
Speaker 1: two separate underwriters estimating the same risk and you average
Speaker 1: their ratings, then the average will be usually more precise
Speaker 1: than the individual judgments because errors in measuring the same
Speaker 1: object do cancel out, but errors when you're responding to
Speaker 1: different objects do not cancel out. So if you overprice
Speaker 1: one policy and you underprice the another policy that doesn't
Speaker 1: cancel out, you've made two mistakes, and you know it's
Speaker 1: the same thing with two with two judges. If one
Speaker 1: defendant is studish too much and another defendant is spunished
Speaker 1: too little. On average, you know, punishment was right, but
Speaker 1: two cares about the average two mistakes were made. So
Speaker 1: there is some confusion because people think about canceling out.
Speaker 1: But that happens when people evaluate, or judge, or measure
Speaker 1: the same thing there and errors do cancel out. I
Speaker 1: recall a book a couple of years ago called The
Speaker 1: End of Average that looked at that exact issue and said,
Speaker 1: you know, we we tend to look at these averages
Speaker 1: as if anyone is experiencing an average. But what you're
Speaker 1: really saying is, hey, if it averages out to be
Speaker 1: the right answer, it means you have a lot of
Speaker 1: wrong answers. That's right. Averaging out to the right answer
Speaker 1: is not a guarantee. And that is a nice example
Speaker 1: of the phenomenon we're discussing in the book, the neglect
Speaker 1: of knowledge. People really tend to focus on bias, which
Speaker 1: is the average era. But you can have a zero
Speaker 1: bias and the very poor performance if you have a
Speaker 1: lot of over estimates, and they love about the estimate.
Speaker 1: Quite interesting. So one of the things in the book
Speaker 1: that I was so taken by had to do with
Speaker 1: the admissions committee for a university, and they used to
Speaker 1: have all the admission officers do a blind review and
Speaker 1: get together and try and hash out who they thought
Speaker 1: would be a good fit for the school and who
Speaker 1: wouldn't um. But it led to a problem, and they
Speaker 1: started having the first person who who reviewed the application
Speaker 1: put their review number on the corner like they would
Speaker 1: actually put their rating on the page, and then hand
Speaker 1: it off to the second person. And you described this
Speaker 1: as the illusion of agreements in organizations. Tell us about that, Well, uh,
Speaker 1: you know, this is an experience as any teacher has
Speaker 1: has had. For example, when you're looking at the test booklet,
Speaker 1: the student has written several essay. If you score a
Speaker 1: test booklet, you score the first question, then the second,
Speaker 1: then the third, then in general you'll find that your
Speaker 1: grades do not vary very much. On the other hand,
Speaker 1: if you read the same test across all students and
Speaker 1: write the score at the back of the of the
Speaker 1: booklet so that you don't know when you read the
Speaker 1: second question where the first question was, you will often
Speaker 1: be shocked by the discrepancy between the first and the second.
Speaker 1: There is a mechanism by which people, if you gave
Speaker 1: a good grade the first time, you're going to be
Speaker 1: inclined to give the benefit of the doubt to the
Speaker 1: student if there is any ambiguos ambiguous answered. Exactly the
Speaker 1: same thing happens in deliberations. And in the example that
Speaker 1: we gave, the admissions committee used to operate in what
Speaker 1: we consider the correct manner. That is, everybody would individually
Speaker 1: make their judgments and then they would reveal all judgments
Speaker 1: together and average them. But they changed the system so
Speaker 1: that now people spoke in sequence, and the question was asked,
Speaker 1: why do you do this, This is not optimal, and
Speaker 1: they say, well, we used to do it the other way.
Speaker 1: We used to have people prepare their judgments individually, but
Speaker 1: there was so much disagreement that we stopped. And that's
Speaker 1: an example where people managed to avoid finding out how
Speaker 1: much noise there really is because when they when people
Speaker 1: are allowed to influence each other or influence themselves in
Speaker 1: the case of the teacher reading multiple booklets, when when
Speaker 1: judgments are not independent, they are less effective statistically, you
Speaker 1: just have less information. Think of the example in which
Speaker 1: the first person to talk is the CEO, and then
Speaker 1: everybody agrees, then the agreement of other people is not informative.
Speaker 1: In fact, you had one person making the judgment. That's
Speaker 1: the extreme of abolishing, of eliminating the appearance of noise
Speaker 1: without eliminating the reality of not So it sounds like
Speaker 1: groups and corporations, institutions, schools, they seem to amplify noise.
Speaker 1: Is that just the nature of bigger numbers of people
Speaker 1: working together that they're going to create additional noise? No
Speaker 1: not necessarily what happens in a group if they made
Speaker 1: their judgments individually, is not that noise is amplified. The
Speaker 1: true noise is revealed. So suppose you had underwriters. Suppose
Speaker 1: you had multiple underwriters judging routinely every every risk, then
Speaker 1: the optimal procedure would be to have them making independent
Speaker 1: drug ugments and only then then revealing the two judgments
Speaker 1: and averaging them. That's clearly the optimal procedure, and and
Speaker 1: the optimal procedure reveals noise and then reduces it by averaging.
Speaker 1: But when a sail individual makes a judgment, that judgment
Speaker 1: will be noisy. And when individuals are allowed to influence
Speaker 1: each other, then it's more like a single judgment than
Speaker 1: it is, like having multiple judgments or the same opta.
Speaker 1: So you use the phrase naive realism, What what does
Speaker 1: that mean relative to noise in groups? Well, what made
Speaker 1: realism means is is a statement which most of us
Speaker 1: are most of the time, that we think we're right,
Speaker 1: We think we have the right view of situation, we
Speaker 1: think we understand strengths correctly. In short, we see the
Speaker 1: world as the world is. That's native realism. And if
Speaker 1: I see the world as it is, and you know,
Speaker 1: they are friends and colleagues looking at the same world,
Speaker 1: and I like and respect them, then naturally I assume
Speaker 1: that they see the world as I do because I
Speaker 1: see it right, and if I respect them, they see
Speaker 1: it right as well. So that's naive realism. And naive
Speaker 1: realism prevents us from becoming aware of the amount of
Speaker 1: noise that there is. We're just assume noise away. We
Speaker 1: saw that very nicely among underwriters. You know, when you
Speaker 1: interview an underwriter, what happens to them? But how does
Speaker 1: an underwriter become expert in the absence of any feedback
Speaker 1: Because they don't they don't get any feedback from reality
Speaker 1: about their underwriting and what happens is that they become
Speaker 1: increasingly confident, and largely because they agree with themselves. So
Speaker 1: when you agree with yourself a lot, and you think
Speaker 1: you're right, and you make judgments with increasing speed and confidence,
Speaker 1: so that makes you think that you're even rter. That's
Speaker 1: naive realism, allowing massive noise to occur with everybody convinced
Speaker 1: that they're doing the right thing, but in fact they
Speaker 1: may not be doing the right thing because as they
Speaker 1: were looking at the same problem, there would be the
Speaker 1: food quite fascinating. So we become familiar with a particular area.
Speaker 1: That familiarity leads us to think that we're developing an expertise.
Speaker 1: We tend to make more snap judgments and without any
Speaker 1: sort of feedback loop, how can we possibly know that
Speaker 1: we're right? And yet that absence of feedback seems to
Speaker 1: strengthen people's self confidence. Do I have that right? And
Speaker 1: think of the number of situations in which exactly this
Speaker 1: whole there's a judge doesn't have feedback as to whether
Speaker 1: judgment was correct or on bail judge. Sometimes there is feedback,
Speaker 1: but it's a symmetry. So bail judge may get feedback
Speaker 1: on somebody who was released and committed the crime, but
Speaker 1: the bail judge will never know if somebody was incarcerated
Speaker 1: would have committed the crime. So feedback is a massive problem.
Speaker 1: And many professionals at the minimum feedback, and yet they
Speaker 1: become confident and they feel their expots. But in those
Speaker 1: cases there is a high risk of noise and a
Speaker 1: lot of that feedback seems to be only at the extreme.
Speaker 1: A bridge collapses, there are a plane crashes, somebody dies,
Speaker 1: there's someone out on bail commits a crime. What about
Speaker 1: all of the lack of a better word, near missus
Speaker 1: where there is a bad judgment, something happens. It's not
Speaker 1: quite as terrible as a as an airplane plane crash,
Speaker 1: and it it's resolved before there's damage, but it's pretty
Speaker 1: clear the basic judgment was wrong. How does that affect
Speaker 1: a person's future judgment. Well, in situations where there are
Speaker 1: near missus, there is an opportunity to learn. And in
Speaker 1: world run you know, well run airlines and and and
Speaker 1: air traffic systems keep track very closely of near missus
Speaker 1: because those are their opportunities to learn without without tragedies.
Speaker 1: But in many situations you get no feedback at all.
Speaker 1: In the idea of having senses in bridges that gives
Speaker 1: you a sensitive measurement of how much stress there is
Speaker 1: that necessarily recent there used to be very poor feedback
Speaker 1: on whether a bride would collapse or not, and in
Speaker 1: many situations that professionals make jugment on, there's no feedback
Speaker 1: at all. Quite interesting. So let's talk about this book,
Speaker 1: which was a collaboration. What was it like working with
Speaker 1: those two gentlemen versus thinking Fast and Slow, which I
Speaker 1: kind of get the sense was you sitting down and
Speaker 1: putting a lot of your previous work into a context
Speaker 1: for public consumption. Well, writing Fast and Slow was mostly
Speaker 1: a very lonely experience, and writing with collaborators was really
Speaker 1: a pleasure. So it was it was a relief to
Speaker 1: be able to count on people to find mistakes to
Speaker 1: correct them, and and a lot of the text UH
Speaker 1: was actually written by Olivier and by Kass. I had
Speaker 1: a lot to do with outlining and with critiquing and
Speaker 1: with rejecting drafts. But I was spared much of the
Speaker 1: things that I'm mostly traid of in writing. So it
Speaker 1: was a very good collaboration. And by the way, we
Speaker 1: benefited a lot from from COVID because that forced stuff
Speaker 1: into quite an efficient way of collaborating. We used to visit.
Speaker 1: Olivier would come to New York from Paris, and I
Speaker 1: would visit Paris for a few days every month, and
Speaker 1: we had a very good time, but it wasn't productive.
Speaker 1: Zooming one or two hours a day turned out to
Speaker 1: be a much better way of writing the book. And
Speaker 1: this is what happened. Uh, it sounds like it was
Speaker 1: just a good excuse to get together in New York
Speaker 1: in Paris and have a little bit of fun. Well,
Speaker 1: I mean, you know, we didn't think of it as
Speaker 1: a good excuse, but it turned out that would waste
Speaker 1: a lot of time and the fair amount of money.
Speaker 1: So you you mentioned you reviewed a lot of manuscript
Speaker 1: from Olivier and Cass and rejected stuff. You and Amos
Speaker 1: very famously would agonize over every sentence in all of
Speaker 1: your publications. You seem to have spent a lot of
Speaker 1: time writing meticulously and very thoughtfully. How has that evolved
Speaker 1: over time? Is this a little easier to sort of
Speaker 1: be the orchestrator and the editor as opposed to, you know,
Speaker 1: just agonizingly putting down every single word. No, it isn't.
Speaker 1: I mean, my this is part of sort of my
Speaker 1: intellectual personality of character that I think most clearly when
Speaker 1: I find flaws in existing text, and I'm not good
Speaker 1: at anticipating the flaws. So I see a flow and
Speaker 1: I correct it, and then there is new text, and
Speaker 1: then I discover a new flow, and and I tend
Speaker 1: to work that way, which is infuriating to make collaborators
Speaker 1: and wish for a lot of time and efforts, but
Speaker 1: that's the way I. On the other hand, I do
Speaker 1: tend to be very critical, and most of the flaws
Speaker 1: that I find do exist, so it tends to lead
Speaker 1: to a good project in a very inefficient way. So
Speaker 1: despite that perfectionism, you know, we all evolve over or time.
Speaker 1: As you were preparing Noise, did you find any of
Speaker 1: your previous writings or research that you either disagree with
Speaker 1: or see from a different perspective or light when you're
Speaker 1: putting this book together? No, not really. I mean, in
Speaker 1: the book, we actually relied on ideas from Thinking Fast
Speaker 1: and Slow, But the book is really fundamentally different. Thinking
Speaker 1: Fast and Slow is a book about individuals and about
Speaker 1: how and it was a book about the average or
Speaker 1: a typical individual and how the average or typical mind works.
Speaker 1: Noise is about individual differences. It's about the way that
Speaker 1: the different people think differently, and so this is a
Speaker 1: really different cut about thinking. It's a different way of
Speaker 1: looking and thinking. So we did use some of the material,
Speaker 1: but the Noise is not a revision of Thinking Fast show.
Speaker 1: It is about the truly different topics that we didn't
Speaker 1: even touch and thinking it clearly, it goes in a
Speaker 1: very different direction, and it looks at some systems and
Speaker 1: some organizations that I don't believe you touched on in
Speaker 1: Thinking Fast. It's kind of interesting because we've already talked
Speaker 1: about medicine and criminal justice and finance. There was one
Speaker 1: section I was fascinated by where you discussed hiring and
Speaker 1: promotions and how I don't want to use the word random,
Speaker 1: but how much noise is in that system and how
Speaker 1: unreliable many organizations hiring processes are. Tell us a little
Speaker 1: bit about that. Well, it terms that people like hiring
Speaker 1: by interviewing people and for me, a general image of
Speaker 1: the individual that they're thinking of hiring. And it turns
Speaker 1: out this is not a good way of doing it.
Speaker 1: A much better way of doing it is what it's
Speaker 1: called the structured interview, the structured process where you accumulate
Speaker 1: information systematically about different characteristics of the person. That is
Speaker 1: less pleasant, it's it's less enjoyable, but much better. And
Speaker 1: better yet is having several in several people do the hiring,
Speaker 1: each of them forming an independence impression, and then they discuss,
Speaker 1: then they average, and then they discuss the average. And
Speaker 1: this is the procedure for example, and and it's about
Speaker 1: state of view. But many places are way short of
Speaker 1: a state of the art. I should add that state
Speaker 1: of the art. Hiring doesn't mean that you're guaranteed the
Speaker 1: perfect sit There's so much there's so much luck in
Speaker 1: the world. There's so much uncertainty that the person to
Speaker 1: how it may be very good but may run into
Speaker 1: difficulties with the boss doesn't like her or something like that.
Speaker 1: And by chance alone you can get a lot of variety. Chance,
Speaker 1: by the way, is not noise. Chance is something that
Speaker 1: happens in the real world. Noise is differences among judgments.
Speaker 1: So hiring is buy and love really very poorly done.
Speaker 1: And it's very poorly done because it doesn't control noise.
Speaker 1: Quite fascinating. So the book goes over how noise affects
Speaker 1: judgment and how it introduces a variety of errors into
Speaker 1: our institutional decision making process. What can we do to
Speaker 1: improve that process? Well, in the book, we we introduce
Speaker 1: a concept that we call deci isn't hygiene And you
Speaker 1: know the word is that particularly appealing. It's intended to
Speaker 1: drink to mind the image of washing your hands. And
Speaker 1: there is a contrast between the biasing and the certain hygiene.
Speaker 1: The bias thing is like medication or like vaccination. It's
Speaker 1: specific to a particular disease. When you wash your hands,
Speaker 1: you don't know what germs you're killing, and if you're successful,
Speaker 1: you'll never know. So the certain hygiene is oriented to
Speaker 1: improving decision making an avoiding errors, specifically avoiding noise, but
Speaker 1: incidentally also avoiding bias without knowing precisely what biases you're
Speaker 1: trying to control. And we have a variety of procedures
Speaker 1: that we think of as the certain HyG Give us
Speaker 1: a few examples. What what are some of the procedures. Well,
Speaker 1: I'll give you an example that has to do with
Speaker 1: the certain making. So suppose you are making a decision
Speaker 1: and so step you one will tell you is you
Speaker 1: have to consider your options and have the best possible
Speaker 1: set of options. But now you come to evaluate options,
Speaker 1: how do you do that? And here actually our advice,
Speaker 1: we have a slogan we say options are like candidates.
Speaker 1: You should think of options in the same way that
Speaker 1: organizations are in our advised to operate when they hire candidates.
Speaker 1: And we were talking about that earlier structure, the thinking,
Speaker 1: break up the each option, look at the various aspects
Speaker 1: of it, make assess these aspects in the fact based way,
Speaker 1: to the equivalent of interviewing somebody about different aspects with
Speaker 1: her character or her experience, and then create a profile
Speaker 1: of all the information you have about that option, and
Speaker 1: only then invoke intuition. That there is a key principle
Speaker 1: of decision hygiene is not to avoid intuition altogether, but
Speaker 1: to delay it, because intuition is way more effective if
Speaker 1: it is preceded by a period in which you accumulate
Speaker 1: information systematically. So that's an example. I have many others,
Speaker 1: but this is when and there were quite a few
Speaker 1: in the book. There were some things that really surprised
Speaker 1: me about that decision making process. How people's moods affect
Speaker 1: their decisions, even the weather affects decision making. We are
Speaker 1: essentially different people at different times. Oh, yes, that is
Speaker 1: there are different sources of noise that we talk about.
Speaker 1: So there are three of them. Are truly that one
Speaker 1: of them is what we call within person noise, and
Speaker 1: that is that the individual is indeed making different judgments
Speaker 1: depending on circumstances that irrelevant. So it's true. There is
Speaker 1: evidence that mood really affects the way that people think. Uh,
Speaker 1: people tend to be more creative when they're in a
Speaker 1: good mood, but they tend to be also more gullible
Speaker 1: and they are more critical when they're in a bad mood.
Speaker 1: So mood affects the way we think, and it also
Speaker 1: affects we're more prone to see good things when we're
Speaker 1: in a good mood. Mood is important. There is evidence
Speaker 1: that judges who pass sentences on criminals are more severe
Speaker 1: on hot days, and they are more severe if their
Speaker 1: football team lost the game last Sunday. So there are
Speaker 1: a lot of irrelevant events or circumstances that influence our judgment.
Speaker 1: This is one of the three major sources of judgment.
Speaker 1: Let's get to the other two. What are the other
Speaker 1: two sources well, and one other which is easy to
Speaker 1: think about. It's very easy to think about it. In
Speaker 1: terms of judges. Some judges are more severe than others,
Speaker 1: so their sentences on average are more severe than the
Speaker 1: sentences of other judges. That's one aspect, and the same
Speaker 1: as to by the web underwriters. Some underwriters write large
Speaker 1: premiums on average, and other underwriters write small premiums on average,
Speaker 1: So there are differences of that kind. But it turns
Speaker 1: out that the biggest source of noise, and that came
Speaker 1: as a surprise to us. The biggest source of noise
Speaker 1: is that people really don't see the world in the
Speaker 1: same way, so that different judges have different tastes in
Speaker 1: crimes and some tastes in criminals, so they somebody may
Speaker 1: be particularly severe about repeat offenders that somebody else might
Speaker 1: be with extremely lenient, say about white color crime, but
Speaker 1: really upset by violence. And it turns out that there
Speaker 1: is we call that a pattern noise. That is, each judge,
Speaker 1: each individual has a pattern of judgments which are this
Speaker 1: is different from the pattern of judgments of other people,
Speaker 1: and that is the major source of noise. And people
Speaker 1: who are consistent in that way. So for example, suppose
Speaker 1: you're a judge and somebody reminds you of your daughter,
Speaker 1: whether that makes you more lenient or more severe, probably
Speaker 1: more lenient. Now on another day, that same person would
Speaker 1: also remind you of your daughter. So this is not
Speaker 1: noisy within the individual. This is a characteristic of the individual,
Speaker 1: but no other judge shares it. And it turns out
Speaker 1: that this highly case specific distances in attitudes that are
Speaker 1: difficult to pin down. They are noise. Judges have personalities
Speaker 1: and judgments differ as much as personalities too. And then
Speaker 1: what is the third source of noise that you identified
Speaker 1: in the post? But those are the three are differences
Speaker 1: in average level for judge's severity, differences in taste what
Speaker 1: we call pattern noise, and within subjects, within person variability,
Speaker 1: we call that occasion noise because on different occasions you
Speaker 1: make different judgements and it's to some of these three
Speaker 1: sources of noise that that creates. That's the noise that
Speaker 1: we observe in the system. All three operate on any
Speaker 1: particular judgment. So I'm gonna ask the question I was
Speaker 1: thinking about a little differently based on what you just said,
Speaker 1: what fields seem to manage reducing noise better than others.
Speaker 1: And are there any fields that are especially susceptible the noise.
Speaker 1: That's a very good question to which I do not
Speaker 1: have a very good answer, because in our work we
Speaker 1: we found noise wherever we looked for it. Indeed, our
Speaker 1: summary conclusion is wherever there is judgment, there is noise,
Speaker 1: and more of it than you think. You know this
Speaker 1: is this has been our conclusion. So we haven't found
Speaker 1: places that control noise very efficiently. The only way, by
Speaker 1: the way to get rid of noise, and that's really
Speaker 1: quite important is average judgments. Take multiple judgments of the
Speaker 1: case and average them, and this mechanically eliminates noise if
Speaker 1: you have enough judgments the average. It may be biased
Speaker 1: because averaging there's nothing to reduce bias, but it eliminates noise.
Speaker 1: So that's a pure far way of eliminating noise. Is
Speaker 1: averaging multiple case very interesting. Let's let me throw a
Speaker 1: curveball at you. If you were designing a system to
Speaker 1: introduce noise to short circuit human judgment, what would you
Speaker 1: create to make judgment less effective noisier. I don't think
Speaker 1: I would do things very differently from the way that
Speaker 1: they have done in many institutions. Now I would I
Speaker 1: would let people make individual judgments without feedback. That's that's
Speaker 1: all that's needed, make their individual decisions without feedback, which
Speaker 1: is a situation that's very common, and that will create
Speaker 1: a lot of noise eventually. And noise is reduced by feedback.
Speaker 1: Sometimes it's the feedback of other people. So case conferences
Speaker 1: can be arranged to some extent control noise. But you know,
Speaker 1: you you don't have to try very hard to create
Speaker 1: a lot of noise. I think the existing organizations do
Speaker 1: very little to control noise. So let's talk a little
Speaker 1: bit about ways to control noise. And you describe a
Speaker 1: difference between rules and standards. Tell us about that. Well,
Speaker 1: Standards is a way of when you say, for example,
Speaker 1: that your obscenity is something that you recognize, so there
Speaker 1: is a standard to avoid obscenity that you do not
Speaker 1: define it. That's a standard. A rule is more precise
Speaker 1: than that, and it does you specifically what you have
Speaker 1: to do, and rules, if followed, they're like computations. The
Speaker 1: computation is a is a rule, and rules tend to
Speaker 1: eliminate noise. Standards sometimes reduce noise, but standards do not
Speaker 1: eliminates because they so the seven words you can say
Speaker 1: on television is a rule, but pornography, I know when
Speaker 1: I see it is a standard. Is that the difference
Speaker 1: nicely pre firstly quite quite interesting. So so, given all
Speaker 1: of the work you've done over the years, all of
Speaker 1: your research, you seem to have continually identified flaws incognition,
Speaker 1: flaws in human judgment. Has this affected the way you
Speaker 1: view other people? Do you? Do you turn around and say, wow,
Speaker 1: these this species is a terrible decision making apparatus or
Speaker 1: is it something less comprehensive than that? No, I've actually
Speaker 1: for my career, I've been interested in intuition and intuitive thinking,
Speaker 1: and I've been interested in that's A lecturer used to
Speaker 1: give many years intuitions marvels and flaws, because intuitions is marvelous.
Speaker 1: Intuition is marvelous, but it's also flawed. And it's true
Speaker 1: that I have found it more interesting to study the
Speaker 1: flaws of intuition than it's marvels. And there a lot
Speaker 1: to do to correct the flaws of intuition. But to
Speaker 1: say that this has turned me into a pessimist, or
Speaker 1: that they dislike people because their minds are flawed I
Speaker 1: think the minds are pretty marvelous, but they are certainly
Speaker 1: far from perfect, right, So, so you're focusing on the
Speaker 1: small bits that we get wrong. But overall we managed
Speaker 1: to navigate through life pretty effectively. Well, we certainly managed
Speaker 1: to navigate through life. And you know it's it would
Speaker 1: be absurd to focus on the floors of the human
Speaker 1: beings when you can see what they're capable of. On
Speaker 1: the other hand, if you want to do things better,
Speaker 1: then you'd better focus on the floors rather than on
Speaker 1: what is going well. You know, one of the things
Speaker 1: you said when we spoke last about Thinking Fast and Slow,
Speaker 1: I asked you about your own investing process, and you said,
Speaker 1: despite knowing everything that you know about you human decision making,
Speaker 1: you still catch yourself making the same sort of mistakes
Speaker 1: that everybody makes. Is that still the case? Do you
Speaker 1: still feel that way? Oh? Yes, I mean, you know,
Speaker 1: I've been at it for more than sixty years, and
Speaker 1: I'm really not better than I was. In general, my
Speaker 1: thinking has been And it was true when I wrote
Speaker 1: Thinking Fast and Slow to just focused on the individuals,
Speaker 1: that the hope of improving thinking is in organizations, because
Speaker 1: organizations think slowly and they have procedures, and it's by
Speaker 1: imposing procedures, by adopting procedures, that you can improve things.
Speaker 1: And in the case of noise, we have a procedure
Speaker 1: that we recommend to get started, and that's measured knowledge.
Speaker 1: If you're in an organization where you have multiple people
Speaker 1: making the same judgment and no very good feedback, conduct
Speaker 1: what we call the noise audit, give them the same
Speaker 1: problem and look at their solution. We predict that you'll
Speaker 1: find more noise then than you think you will. That's
Speaker 1: that's our prediction, and that's some that's a recommendation to organizations.
Speaker 1: It's not something that you can recommend to an individual.
Speaker 1: Quite interesting, I have to ask you before we get
Speaker 1: to our favorite questions, what's the next project, what's the
Speaker 1: next book look like? What is tickling your curiosity these days? Well,
Speaker 1: that's actually back to a topic that I was working
Speaker 1: on but years ago, and I have almost by accident
Speaker 1: and back studying well being, and I'm involved in several
Speaker 1: research projects. None of them is as big or ambitious
Speaker 1: as Noise was, or thinking fast and slow, but all
Speaker 1: of them are quite interesting. So I'm not bored. I
Speaker 1: can't picture you board because you always seem to have
Speaker 1: a lot of different things going on. Let me ask
Speaker 1: my favorite questions that I ask all of our guests,
Speaker 1: and let's start with, what are you doing to stay
Speaker 1: entertained during this pandemic lockdown? In addition to working on
Speaker 1: the book? What are you streaming? What are you watching
Speaker 1: on Netflix of anything? Oh, I've been watching several series,
Speaker 1: several very good series. Let's see the last ones. There
Speaker 1: is a political series on Netflix, Le Bon Wi, which
Speaker 1: is a French political thriller that is very good. There
Speaker 1: is a Danish political series Borgan, that is very good.
Speaker 1: I am now watching Babylon Berlin about Berlin in the
Speaker 1: nineteen twenties, which is excellent. And so I do mind
Speaker 1: watching me will I exercise? And I exercise a fair amount.
Speaker 1: But so I've seen a lot of series since, well,
Speaker 1: from for the last few years. So baum Noir was
Speaker 1: the French one. What was the Danish one? Borgan? Borgan
Speaker 1: is Bridge Actually the Danish one. Bogan is a thriller.
Speaker 1: It's a Scandinavian swiller. There is a Danish one about
Speaker 1: a woman prime minister, and it's not Borgan, and I
Speaker 1: not block on its name, but it would be easy
Speaker 1: to find, and I really recommended it, is sup all right,
Speaker 1: I will I will check that out. So let's talk
Speaker 1: about your early mentors who helped to shape your career.
Speaker 1: And I guess we have to include collaborators in that
Speaker 1: as well. Well, I mean there were There's been one
Speaker 1: major influence on my career, and it was in Sisk.
Speaker 1: The collaboration with him completely changed my life. And uh,
Speaker 1: and it changed the way I do things, but and
Speaker 1: it gave me Yeah, it changed my life and it
Speaker 1: was the best period of my life too. Professionally. The
Speaker 1: thing I recall from Michael Lewis is undoing project is
Speaker 1: that people said, you guys would lock yourself into an
Speaker 1: office or a classroom and all they would hear all
Speaker 1: day long is peals of laughter coming from within. Is
Speaker 1: that true? Is that an exaggeration or did you guys
Speaker 1: know that's really not an exaggeration. Amos and I worked
Speaker 1: very closely together for about twelve years, and we spent
Speaker 1: many hours a day together. And he was very funny.
Speaker 1: He had an excellent sense of humor and he loved laughing,
Speaker 1: and in his presence I also became funny. So we
Speaker 1: were amusing each other and the field that we studied, uh,
Speaker 1: was was one that was ducive to luster because we
Speaker 1: were looking for mistakes in our own thinking and to
Speaker 1: trap ourselves or to see that you are attempted to
Speaker 1: make a stupid error. That is quite funny. And that's
Speaker 1: the game that we engaged in in studying judgment and
Speaker 1: in studying decision making, was looking for errors in our
Speaker 1: own thinking. And that was very amusing, I can imagine.
Speaker 1: So let's talk about books. What are some of your
Speaker 1: all time favorites and what are you reading right now? Well,
Speaker 1: I would say my all time favorites of recent years
Speaker 1: with Sapiens. I think it's many people's favorite book by Valli.
Speaker 1: I read it twice, which is something that I really do.
Speaker 1: And right now while I'm reading the new edition of Nudge,
Speaker 1: which is coming out I think in August, and it's
Speaker 1: called Nudge. The final edition by Dick Taylor and Catherine
Speaker 1: Steam was in and it's quite different from the original
Speaker 1: note which appeared I think indo peplem an eight uh
Speaker 1: and it's what but it it had the same characteristic
Speaker 1: that not said. It's wise and it's funny, right, Dick said,
Speaker 1: it's about two thirds new and I think that's August
Speaker 1: four that comes out the other's right. I happened to
Speaker 1: be reading that right now, August three. I'm looking at
Speaker 1: a message from him. He's um. He's a very amusing
Speaker 1: person to begin with. And if if you're telling me
Speaker 1: the book is funny, then I am really looking forward
Speaker 1: the book is. You know, he just he can't help himself.
Speaker 1: He's funny all the time. He's my best friend, my
Speaker 1: best living friend. Let me ask you this question. If
Speaker 1: a recent college graduate asked you for some advice, if
Speaker 1: he was interested in a career in either psychology or
Speaker 1: behavioral finance, what sort of advice might you give him? Well,
Speaker 1: you know, I tend to refrain from advice because I
Speaker 1: don't believe I have a crystal ball into the future.
Speaker 1: I can tell you what I would have been doing
Speaker 1: if I was starting today. The fields that are very
Speaker 1: exciting from my perspective are neuroscience, including neuroeconomics, which is
Speaker 1: the neuroscience of decision making, and artificial intelligence. I mean,
Speaker 1: in those two areas right now, there are extraordinary developments,
Speaker 1: very exciting. And so when you see that and they're
Speaker 1: attracting massive talent both areas so you know that for
Speaker 1: the next decade or so they'll be cooking A lot
Speaker 1: is going to happen. And what happens after that, I
Speaker 1: have no idea. And in our final question, what do
Speaker 1: you know about the world of psychology, g behavioral finance
Speaker 1: economics today that you wish you knew fifty or so
Speaker 1: years ago when you were first getting started? Oh? Well,
Speaker 1: so much has been learned. I you know, if I
Speaker 1: I can't say that I wish I had known earlier.
Speaker 1: Has been so much fun to find out over the years,
Speaker 1: both in my work and in the work of others.
Speaker 1: So I can't think of thinking that would have made
Speaker 1: me act de simply. But all I can say you
Speaker 1: to you is, oh, yes, things have really changed and
Speaker 1: so have been in that field. Huh quite fascinating. Thank you,
Speaker 1: Danny for being so generous with your time. We have
Speaker 1: been speaking with Danny Kahneman, whose new book Noise, A
Speaker 1: Flawing Human Judgment, was co authored with Olivier Simone and
Speaker 1: Cass Sunstein. If you enjoy this conversation, check out any
Speaker 1: of our previous four hundred such interviews. You can find
Speaker 1: those at iTunes, Spotify, Google, Bloomberg dot Com, wherever you
Speaker 1: get your podcast each week. We love your comments, feedback
Speaker 1: and suggestions. Write to us at m IB podcast at
Speaker 1: Bloomberg dot net. You can sign up for my Daily
Speaker 1: Reads at Ridholts dot com. Check out my weekly column
Speaker 1: on Bloomberg dot com slash Opinion. Follow me on Twitter
Speaker 1: at Ridholts. I would be remiss if I did not
Speaker 1: think the crack staff that helps put together this conversation
Speaker 1: each week. Tim Harrow is my audio engineer. Alatico val
Speaker 1: Bron is my project manager. Michael Boyle is my producer.
Speaker 1: Michael Batnick is my head of research. I'm Barry Riholts.
Speaker 1: You've been listening to Master's Business on Bloomberg Radio.