WEBVTT - Daniel Kahneman on Behavioral Economics (Podcast) 0:00:00.080 --> 0:00:06.080 M. This is Mesters in Business with Very Results on 0:00:06.240 --> 0:00:10.920 Bloomberg Radio this weekend on the podcast What Can I Say? 0:00:11.240 --> 0:00:16.400 Another extra special guest Danny Kaneman, no Bell Prize winner, 0:00:16.760 --> 0:00:20.439 author of Thinking Fast and Slow. His new book is Noise, 0:00:20.560 --> 0:00:26.360 a Fawn Human Judgment and Danny is just so knowledgeable. 0:00:26.520 --> 0:00:28.720 Please call me Danny. I I feel like I have 0:00:28.760 --> 0:00:32.680 to call him Professor Khneman, and he he insists. Uh, 0:00:32.680 --> 0:00:37.120 He's eighty seven years old and incredibly sharp and insightful 0:00:37.440 --> 0:00:41.960 and just so much wisdom and knowledge. If you liked 0:00:42.520 --> 0:00:45.879 Thinking Fast and Slow, which is about judgment error in 0:00:46.040 --> 0:00:51.680 humans in individuals, well, Noise is about how flaws and 0:00:52.159 --> 0:00:56.480 in judgment within broader institutions come about. And it's a 0:00:56.520 --> 0:01:01.320 totally different area and it's absolutely fascinating. I'm a big 0:01:01.360 --> 0:01:05.360 fan of behavioral finance in general, plus all of uh 0:01:05.480 --> 0:01:09.600 Danny's work historically. If you are remotely interested in this, 0:01:09.800 --> 0:01:14.080 then strap yourself. And this is another doozy with no 0:01:14.160 --> 0:01:21.240 further ado. My conversation with Danny Koneman. This is mesters 0:01:21.240 --> 0:01:26.920 in Business with Very Results on Bloomberg Radio. My extra 0:01:27.000 --> 0:01:30.880 special guest this week is Danny Khneman. He was awarded 0:01:31.280 --> 0:01:35.280 the two thousand and two Nobel Memorial Prize in Economic Sciences, 0:01:35.720 --> 0:01:40.000 which he shared with Vernon Smith for his empirical findings 0:01:40.480 --> 0:01:43.560 the work he did with Amos Tversky. And what's so 0:01:43.640 --> 0:01:48.240 fascinating about that Nobel Prize is that Danny is a psychologist. 0:01:48.760 --> 0:01:53.080 The work they did challenge the prevailing thoughts in economic 0:01:53.160 --> 0:01:58.920 theory by establishing a basis for common human eras his 0:01:59.440 --> 0:02:02.960 previous book, Thinking Fast and Slow, was the best seller 0:02:03.520 --> 0:02:05.960 of two thousand and eleven and one a variety of 0:02:05.960 --> 0:02:10.520 different awards, including the National Academy's Communication Award for Best 0:02:10.520 --> 0:02:15.320 Creative Work. His latest book is Just Out Noise, A 0:02:15.480 --> 0:02:20.400 Flaw in Human Judgment, which Danny Koneman wrote with Olive 0:02:20.680 --> 0:02:26.040 Simony and Cass Sunstein. Danny Kneman, welcome back to Bloomberg. 0:02:26.400 --> 0:02:29.360 I'm delighted to be here. You always say call me Danny, 0:02:29.400 --> 0:02:31.600 and I always feel awkward and I feel like I 0:02:31.600 --> 0:02:36.919 should call you professor. But let me just get that call, 0:02:37.800 --> 0:02:42.600 all right, Danny. So let's start very basically. What is noise? 0:02:42.680 --> 0:02:46.680 How does it happen? And where does it come from? Okay, well, 0:02:47.840 --> 0:02:53.480 noise isn't accepted term in statistics. We talk about statistical noise, 0:02:53.520 --> 0:02:57.840 which is variability, and that's where it comes from. We 0:02:57.919 --> 0:03:03.360 talk about noise a measurement, which is unreliability in uh 0:03:03.360 --> 0:03:07.320 in measurement, where measurements that should be identical turn out 0:03:07.320 --> 0:03:10.600 to vary. So that's the background in the use of 0:03:10.600 --> 0:03:15.720 the term as we use it specifically, we intend we 0:03:15.760 --> 0:03:20.600 speak about judgment noise, and this is the situation in 0:03:20.639 --> 0:03:25.399 which judgments should be identical people or the same individual 0:03:26.080 --> 0:03:30.000 judging the same object at different times, or different people 0:03:30.680 --> 0:03:34.800 judging the same object. If they don't agree and I 0:03:34.920 --> 0:03:39.960 expected to agree, we speak about judgment noise, and in general, 0:03:40.000 --> 0:03:42.880 people are expected to agree when they're trying to be accurate. 0:03:42.960 --> 0:03:45.440 So when you have a group of people trying to 0:03:45.480 --> 0:03:48.480 make their best guess about the quantity, it could be 0:03:48.760 --> 0:03:53.320 the symptoms that somebody should should get for a crime. 0:03:53.840 --> 0:03:56.200 It could be the value of the company. It could 0:03:56.280 --> 0:04:00.680 be the premium that somebody should be charged. Oh, it 0:04:00.760 --> 0:04:05.400 could be a diagnosis, a medical diagnosis. In all these cases, 0:04:05.760 --> 0:04:09.040 you might have several people looking at the same information 0:04:09.400 --> 0:04:13.960 making judgments. If they don't agree, there is noise, and 0:04:14.160 --> 0:04:17.880 noise is the topic of the book we wrote. So 0:04:18.680 --> 0:04:22.880 it's fascinating how we start to see noisy decision making 0:04:22.920 --> 0:04:25.720 come up over and over again in the same fields. 0:04:25.760 --> 0:04:30.720 And you just mentioned a few medicine, criminal justice, finance. 0:04:31.200 --> 0:04:36.520 Are there certain fields that are more susceptible two problems 0:04:36.640 --> 0:04:40.320 in expert judgments than others? Or is it just that 0:04:41.000 --> 0:04:44.920 the results of those sort of noisy decisions are so 0:04:45.000 --> 0:04:48.880 much more significant than other fields. Well, we use the 0:04:48.920 --> 0:04:55.200 word judgment when there is room for reasonable disagreement, that is, 0:04:55.360 --> 0:04:58.040 you know, we don't use the word judgment for computation, 0:04:58.320 --> 0:05:02.040 and when compute station is appropriate, we wouldn't be talking 0:05:02.040 --> 0:05:05.960 of noise. We would be talking of people making mistakes. 0:05:06.000 --> 0:05:09.799 And we talked about noise when when it's a matter 0:05:09.800 --> 0:05:14.960 of judgment. And and so the existence of noise by 0:05:15.000 --> 0:05:18.719 itself is not a surprise. Whatever the surprise is the 0:05:18.800 --> 0:05:24.760 amount of noise just a lot more then would be expected. 0:05:25.400 --> 0:05:28.120 And here, I think the best way to explain this 0:05:28.279 --> 0:05:32.600 is too to tell you the story of how I 0:05:32.800 --> 0:05:35.760 started to work on noise. Then where the whole thing began. 0:05:37.520 --> 0:05:41.800 So I was consulting in an insurance company, said than 0:05:41.920 --> 0:05:49.080 eight years ago, and I had the idea of running 0:05:49.120 --> 0:05:53.320 But today we would call a noise audit, that is underwriters. 0:05:53.440 --> 0:05:58.440 To take one example, we had several underwriters, so some 0:05:58.560 --> 0:06:04.159 realistic cases, the same cases. They were constructed by executives 0:06:04.240 --> 0:06:08.920 of experts and underwriting, so they were completely realistic, and 0:06:08.960 --> 0:06:12.479 you might have fifty underwriters looking at the same premium. 0:06:12.520 --> 0:06:16.560 Now nobody would expect the numbers to be exactly the same. 0:06:17.200 --> 0:06:21.520 But I asked executives, if you take a pair of 0:06:21.720 --> 0:06:25.840 underwriters at random, by how much would you expect them 0:06:25.880 --> 0:06:29.600 to differ in percentages? That is, you take the average 0:06:29.680 --> 0:06:32.280 or the pairer, you take the difference, you divide the 0:06:32.360 --> 0:06:36.320 difference by the average. What percentage looks reasonable to you? 0:06:37.400 --> 0:06:41.000 And they're answer typically with ten percent. And we have, 0:06:41.320 --> 0:06:47.200 by the way, we have surveyed hundreds of executives since then, 0:06:47.400 --> 0:06:50.440 and ten percent seems to be what we expect the 0:06:50.520 --> 0:06:55.200 reasonable difference to be, which is tolerable when two people 0:06:55.279 --> 0:06:59.000 make judgments of a quantity. Now, the correct answer among 0:06:59.120 --> 0:07:04.320 underwriters in that company with sixty five percent more than 0:07:04.440 --> 0:07:09.400 five times as much as expected. That's the phenomenon. So 0:07:09.560 --> 0:07:12.560 we expect this agreement where judgment is involved, We just 0:07:12.640 --> 0:07:18.320 don't expect that much disagreement. And this basically was the 0:07:18.360 --> 0:07:23.200 observation that started us on that part of writing a book, 0:07:23.280 --> 0:07:27.480 because it turns out that you find astonishing amount of 0:07:27.520 --> 0:07:31.080 disagreement when you look for it, and you find it 0:07:31.160 --> 0:07:36.560 wherever judgment is involved. So engineers who make estimates on 0:07:36.600 --> 0:07:39.960 the basis of objected data, they don't have a problem 0:07:40.040 --> 0:07:42.520 of judgment, but to the extent they do have a 0:07:42.560 --> 0:07:45.559 problem of judgment, you will expect a lot of noise. 0:07:46.120 --> 0:07:51.120 So that's the basic finding. And you know, wherever precision 0:07:51.240 --> 0:07:53.880 is important, wherever it is important to get to the 0:07:54.000 --> 0:07:57.680 right number, noise is a source of there. So where 0:07:57.840 --> 0:08:02.880 some people over estimating and others underestimating now making errors. Huh. 0:08:03.200 --> 0:08:06.440 So let me roll back to that insurance company, which 0:08:06.480 --> 0:08:10.720 you discuss in the book, and there were two particular 0:08:10.840 --> 0:08:15.640 areas where we're there were these broad disagreements. The first 0:08:15.920 --> 0:08:21.320 was when people were trying to estimate the risk involved 0:08:21.840 --> 0:08:26.520 with some insurance and so how you price that very 0:08:26.600 --> 0:08:30.280 much determines. If you're too expensive, meaning you think it's 0:08:30.320 --> 0:08:33.600 a high risk, you're not gonna win the business. And 0:08:33.640 --> 0:08:37.480 if it's too cheap relative to the risk, well you'll 0:08:37.480 --> 0:08:41.559 win the business, but it won't be profitable. The costs 0:08:41.600 --> 0:08:44.240 will be higher. And then on the other end, in 0:08:44.280 --> 0:08:48.600 the appraisal of hey, what are the damages here? Figuring 0:08:48.600 --> 0:08:52.800 out how much something should be covered by insurance, what 0:08:52.920 --> 0:08:56.040 the dollar amount is, and the same situation. You can't 0:08:56.080 --> 0:08:58.920 be too stingy or you lose customers, but you can't 0:08:58.960 --> 0:09:01.840 be too generous when you give the house away. How 0:09:02.000 --> 0:09:08.120 significant a financial issue was this for the insurance company. Well, 0:09:08.360 --> 0:09:12.160 you know, it's not easy to estimate that exactly, but 0:09:12.240 --> 0:09:16.640 I can tell you the question that I asked some executives. 0:09:16.920 --> 0:09:20.840 I said, suppose there is a correct number, say for 0:09:20.880 --> 0:09:26.880 the underwriters, and and you have somebody who overestimates the 0:09:27.320 --> 0:09:31.800 underwriting cost by fifteen, how much would you expect that 0:09:31.880 --> 0:09:36.760 to cost the company? And the same question for underestimating 0:09:37.080 --> 0:09:42.199 by fifteen. In fact, fifteen percent on either side is 0:09:42.320 --> 0:09:46.760 much less than noise than we had discovered. But people 0:09:47.000 --> 0:09:50.520 estimated on that basis that this could be in the 0:09:50.600 --> 0:09:53.880 hundreds of millions or billions of dollars. This is a 0:09:53.960 --> 0:09:59.520 very large company, so uh underwriters have a lot of 0:09:59.559 --> 0:10:04.000 important decisions to make claims, adjustice, make important decisions which 0:10:04.000 --> 0:10:08.280 are really consequential for the company. And errors of the 0:10:08.320 --> 0:10:13.320 magnitude that we observe are costly. The main reason that 0:10:13.400 --> 0:10:17.280 they may be less costly is that if error is 0:10:17.360 --> 0:10:21.920 present in all insurance companies, if all insurance companies are noisy, 0:10:22.280 --> 0:10:25.320 then some of the damage to each individual company will 0:10:25.360 --> 0:10:29.400 be reduced. But that's the best that we can say. Well, 0:10:29.480 --> 0:10:33.000 one would imagine the insurance company that could reduce noise 0:10:33.600 --> 0:10:38.000 would find itself at a competitive advantage. Absolutely, there was 0:10:38.040 --> 0:10:40.680 something you had written that really stood out to me. 0:10:41.360 --> 0:10:44.720 There's an assumption when you have noisy systems and everything 0:10:44.800 --> 0:10:49.319 from criminal justice to medicine to insurance, that these errors 0:10:49.360 --> 0:10:53.480 tend to cancel out. But you found out that noisy 0:10:53.559 --> 0:10:57.360 systems have errors. Not only do they not cancel out, 0:10:57.559 --> 0:11:01.800 they tend to add up. Explain, well, if you have 0:11:02.040 --> 0:11:08.200 two separate underwriters estimating the same risk and you average 0:11:08.520 --> 0:11:13.880 their ratings, then the average will be usually more precise 0:11:14.480 --> 0:11:19.920 than the individual judgments because errors in measuring the same 0:11:20.200 --> 0:11:25.560 object do cancel out, but errors when you're responding to 0:11:25.800 --> 0:11:30.120 different objects do not cancel out. So if you overprice 0:11:30.240 --> 0:11:35.200 one policy and you underprice the another policy that doesn't 0:11:35.240 --> 0:11:39.160 cancel out, you've made two mistakes, and you know it's 0:11:39.160 --> 0:11:43.600 the same thing with two with two judges. If one 0:11:44.000 --> 0:11:48.440 defendant is studish too much and another defendant is spunished 0:11:48.480 --> 0:11:52.680 too little. On average, you know, punishment was right, but 0:11:52.760 --> 0:11:56.040 two cares about the average two mistakes were made. So 0:11:57.280 --> 0:12:00.920 there is some confusion because people think about canceling out. 0:12:01.000 --> 0:12:04.800 But that happens when people evaluate, or judge, or measure 0:12:05.080 --> 0:12:09.920 the same thing there and errors do cancel out. I 0:12:09.960 --> 0:12:12.560 recall a book a couple of years ago called The 0:12:12.720 --> 0:12:17.240 End of Average that looked at that exact issue and said, 0:12:17.920 --> 0:12:20.520 you know, we we tend to look at these averages 0:12:20.559 --> 0:12:24.240 as if anyone is experiencing an average. But what you're 0:12:24.280 --> 0:12:27.160 really saying is, hey, if it averages out to be 0:12:27.240 --> 0:12:29.000 the right answer, it means you have a lot of 0:12:29.040 --> 0:12:34.600 wrong answers. That's right. Averaging out to the right answer 0:12:34.760 --> 0:12:38.280 is not a guarantee. And that is a nice example 0:12:38.320 --> 0:12:42.520 of the phenomenon we're discussing in the book, the neglect 0:12:42.600 --> 0:12:46.560 of knowledge. People really tend to focus on bias, which 0:12:46.600 --> 0:12:50.000 is the average era. But you can have a zero 0:12:50.160 --> 0:12:54.680 bias and the very poor performance if you have a 0:12:54.679 --> 0:12:57.360 lot of over estimates, and they love about the estimate. 0:12:58.240 --> 0:13:01.240 Quite interesting. So one of the things in the book 0:13:01.320 --> 0:13:04.560 that I was so taken by had to do with 0:13:04.679 --> 0:13:09.720 the admissions committee for a university, and they used to 0:13:09.800 --> 0:13:15.000 have all the admission officers do a blind review and 0:13:15.160 --> 0:13:18.360 get together and try and hash out who they thought 0:13:18.360 --> 0:13:20.240 would be a good fit for the school and who 0:13:20.280 --> 0:13:24.520 wouldn't um. But it led to a problem, and they 0:13:24.559 --> 0:13:29.960 started having the first person who who reviewed the application 0:13:30.200 --> 0:13:33.280 put their review number on the corner like they would 0:13:33.280 --> 0:13:36.120 actually put their rating on the page, and then hand 0:13:36.160 --> 0:13:39.760 it off to the second person. And you described this 0:13:39.880 --> 0:13:47.960 as the illusion of agreements in organizations. Tell us about that, Well, uh, 0:13:48.120 --> 0:13:51.080 you know, this is an experience as any teacher has 0:13:51.360 --> 0:13:57.400 has had. For example, when you're looking at the test booklet, 0:13:57.880 --> 0:14:02.160 the student has written several essay. If you score a 0:14:02.320 --> 0:14:05.960 test booklet, you score the first question, then the second, 0:14:06.040 --> 0:14:10.160 then the third, then in general you'll find that your 0:14:10.200 --> 0:14:13.839 grades do not vary very much. On the other hand, 0:14:14.240 --> 0:14:19.080 if you read the same test across all students and 0:14:19.200 --> 0:14:22.120 write the score at the back of the of the 0:14:22.240 --> 0:14:25.400 booklet so that you don't know when you read the 0:14:25.440 --> 0:14:28.760 second question where the first question was, you will often 0:14:28.840 --> 0:14:32.560 be shocked by the discrepancy between the first and the second. 0:14:32.840 --> 0:14:36.920 There is a mechanism by which people, if you gave 0:14:36.960 --> 0:14:39.720 a good grade the first time, you're going to be 0:14:39.760 --> 0:14:42.280 inclined to give the benefit of the doubt to the 0:14:42.320 --> 0:14:47.160 student if there is any ambiguos ambiguous answered. Exactly the 0:14:47.240 --> 0:14:51.160 same thing happens in deliberations. And in the example that 0:14:51.200 --> 0:14:56.520 we gave, the admissions committee used to operate in what 0:14:56.760 --> 0:15:02.479 we consider the correct manner. That is, everybody would individually 0:15:02.560 --> 0:15:05.640 make their judgments and then they would reveal all judgments 0:15:05.640 --> 0:15:09.320 together and average them. But they changed the system so 0:15:09.520 --> 0:15:13.600 that now people spoke in sequence, and the question was asked, 0:15:13.640 --> 0:15:16.440 why do you do this, This is not optimal, and 0:15:16.480 --> 0:15:19.240 they say, well, we used to do it the other way. 0:15:19.600 --> 0:15:23.000 We used to have people prepare their judgments individually, but 0:15:23.080 --> 0:15:28.120 there was so much disagreement that we stopped. And that's 0:15:28.160 --> 0:15:34.320 an example where people managed to avoid finding out how 0:15:34.400 --> 0:15:38.240 much noise there really is because when they when people 0:15:38.280 --> 0:15:42.080 are allowed to influence each other or influence themselves in 0:15:42.120 --> 0:15:45.560 the case of the teacher reading multiple booklets, when when 0:15:45.640 --> 0:15:50.880 judgments are not independent, they are less effective statistically, you 0:15:51.080 --> 0:15:54.400 just have less information. Think of the example in which 0:15:54.680 --> 0:15:57.960 the first person to talk is the CEO, and then 0:15:58.080 --> 0:16:03.200 everybody agrees, then the agreement of other people is not informative. 0:16:03.800 --> 0:16:07.000 In fact, you had one person making the judgment. That's 0:16:07.040 --> 0:16:12.520 the extreme of abolishing, of eliminating the appearance of noise 0:16:12.840 --> 0:16:17.200 without eliminating the reality of not So it sounds like 0:16:17.360 --> 0:16:24.680 groups and corporations, institutions, schools, they seem to amplify noise. 0:16:25.560 --> 0:16:29.240 Is that just the nature of bigger numbers of people 0:16:29.320 --> 0:16:33.240 working together that they're going to create additional noise? No 0:16:33.720 --> 0:16:37.840 not necessarily what happens in a group if they made 0:16:38.000 --> 0:16:42.520 their judgments individually, is not that noise is amplified. The 0:16:42.640 --> 0:16:48.240 true noise is revealed. So suppose you had underwriters. Suppose 0:16:48.320 --> 0:16:55.200 you had multiple underwriters judging routinely every every risk, then 0:16:55.280 --> 0:16:59.680 the optimal procedure would be to have them making independent 0:16:59.720 --> 0:17:04.359 drug ugments and only then then revealing the two judgments 0:17:04.359 --> 0:17:09.960 and averaging them. That's clearly the optimal procedure, and and 0:17:10.080 --> 0:17:15.359 the optimal procedure reveals noise and then reduces it by averaging. 0:17:15.880 --> 0:17:19.880 But when a sail individual makes a judgment, that judgment 0:17:19.920 --> 0:17:25.200 will be noisy. And when individuals are allowed to influence 0:17:25.240 --> 0:17:29.200 each other, then it's more like a single judgment than 0:17:29.280 --> 0:17:32.399 it is, like having multiple judgments or the same opta. 0:17:33.560 --> 0:17:37.560 So you use the phrase naive realism, What what does 0:17:37.600 --> 0:17:42.600 that mean relative to noise in groups? Well, what made 0:17:42.640 --> 0:17:46.719 realism means is is a statement which most of us 0:17:46.760 --> 0:17:50.040 are most of the time, that we think we're right, 0:17:50.840 --> 0:17:53.520 We think we have the right view of situation, we 0:17:53.600 --> 0:17:57.560 think we understand strengths correctly. In short, we see the 0:17:57.600 --> 0:18:02.080 world as the world is. That's native realism. And if 0:18:02.119 --> 0:18:05.639 I see the world as it is, and you know, 0:18:05.760 --> 0:18:08.359 they are friends and colleagues looking at the same world, 0:18:08.760 --> 0:18:12.280 and I like and respect them, then naturally I assume 0:18:12.359 --> 0:18:14.399 that they see the world as I do because I 0:18:14.480 --> 0:18:17.280 see it right, and if I respect them, they see 0:18:17.280 --> 0:18:21.240 it right as well. So that's naive realism. And naive 0:18:21.280 --> 0:18:26.040 realism prevents us from becoming aware of the amount of 0:18:26.080 --> 0:18:29.919 noise that there is. We're just assume noise away. We 0:18:30.000 --> 0:18:33.159 saw that very nicely among underwriters. You know, when you 0:18:33.280 --> 0:18:38.320 interview an underwriter, what happens to them? But how does 0:18:38.320 --> 0:18:42.359 an underwriter become expert in the absence of any feedback 0:18:42.800 --> 0:18:46.320 Because they don't they don't get any feedback from reality 0:18:46.400 --> 0:18:50.679 about their underwriting and what happens is that they become 0:18:50.720 --> 0:18:55.240 increasingly confident, and largely because they agree with themselves. So 0:18:55.320 --> 0:18:58.360 when you agree with yourself a lot, and you think 0:18:58.400 --> 0:19:03.440 you're right, and you make judgments with increasing speed and confidence, 0:19:03.800 --> 0:19:06.919 so that makes you think that you're even rter. That's 0:19:07.200 --> 0:19:12.960 naive realism, allowing massive noise to occur with everybody convinced 0:19:12.960 --> 0:19:16.320 that they're doing the right thing, but in fact they 0:19:16.359 --> 0:19:18.359 may not be doing the right thing because as they 0:19:18.359 --> 0:19:20.520 were looking at the same problem, there would be the 0:19:20.640 --> 0:19:26.000 food quite fascinating. So we become familiar with a particular area. 0:19:26.680 --> 0:19:31.480 That familiarity leads us to think that we're developing an expertise. 0:19:31.960 --> 0:19:35.080 We tend to make more snap judgments and without any 0:19:35.080 --> 0:19:38.840 sort of feedback loop, how can we possibly know that 0:19:38.880 --> 0:19:43.359 we're right? And yet that absence of feedback seems to 0:19:43.480 --> 0:19:48.800 strengthen people's self confidence. Do I have that right? And 0:19:49.080 --> 0:19:53.960 think of the number of situations in which exactly this 0:19:54.200 --> 0:19:58.480 whole there's a judge doesn't have feedback as to whether 0:19:58.640 --> 0:20:03.200 judgment was correct or on bail judge. Sometimes there is feedback, 0:20:03.240 --> 0:20:06.760 but it's a symmetry. So bail judge may get feedback 0:20:06.800 --> 0:20:09.800 on somebody who was released and committed the crime, but 0:20:09.920 --> 0:20:13.720 the bail judge will never know if somebody was incarcerated 0:20:13.960 --> 0:20:18.160 would have committed the crime. So feedback is a massive problem. 0:20:18.560 --> 0:20:22.639 And many professionals at the minimum feedback, and yet they 0:20:22.720 --> 0:20:26.679 become confident and they feel their expots. But in those 0:20:26.720 --> 0:20:29.800 cases there is a high risk of noise and a 0:20:29.800 --> 0:20:32.879 lot of that feedback seems to be only at the extreme. 0:20:33.520 --> 0:20:37.879 A bridge collapses, there are a plane crashes, somebody dies, 0:20:38.040 --> 0:20:41.560 there's someone out on bail commits a crime. What about 0:20:41.600 --> 0:20:44.720 all of the lack of a better word, near missus 0:20:45.440 --> 0:20:49.520 where there is a bad judgment, something happens. It's not 0:20:49.640 --> 0:20:53.040 quite as terrible as a as an airplane plane crash, 0:20:53.640 --> 0:20:56.639 and it it's resolved before there's damage, but it's pretty 0:20:56.680 --> 0:21:02.280 clear the basic judgment was wrong. How does that affect 0:21:02.280 --> 0:21:07.240 a person's future judgment. Well, in situations where there are 0:21:07.359 --> 0:21:11.200 near missus, there is an opportunity to learn. And in 0:21:11.359 --> 0:21:16.399 world run you know, well run airlines and and and 0:21:16.520 --> 0:21:21.119 air traffic systems keep track very closely of near missus 0:21:21.280 --> 0:21:25.240 because those are their opportunities to learn without without tragedies. 0:21:25.720 --> 0:21:29.920 But in many situations you get no feedback at all. 0:21:30.320 --> 0:21:33.960 In the idea of having senses in bridges that gives 0:21:34.000 --> 0:21:38.080 you a sensitive measurement of how much stress there is 0:21:38.480 --> 0:21:42.480 that necessarily recent there used to be very poor feedback 0:21:42.680 --> 0:21:46.080 on whether a bride would collapse or not, and in 0:21:46.160 --> 0:21:50.320 many situations that professionals make jugment on, there's no feedback 0:21:50.359 --> 0:21:55.359 at all. Quite interesting. So let's talk about this book, 0:21:56.240 --> 0:22:01.120 which was a collaboration. What was it like working with 0:22:01.160 --> 0:22:04.879 those two gentlemen versus thinking Fast and Slow, which I 0:22:05.000 --> 0:22:08.800 kind of get the sense was you sitting down and 0:22:09.520 --> 0:22:13.840 putting a lot of your previous work into a context 0:22:14.520 --> 0:22:21.000 for public consumption. Well, writing Fast and Slow was mostly 0:22:21.040 --> 0:22:25.960 a very lonely experience, and writing with collaborators was really 0:22:25.960 --> 0:22:30.520 a pleasure. So it was it was a relief to 0:22:30.600 --> 0:22:33.480 be able to count on people to find mistakes to 0:22:33.560 --> 0:22:37.560 correct them, and and a lot of the text UH 0:22:38.080 --> 0:22:42.040 was actually written by Olivier and by Kass. I had 0:22:42.080 --> 0:22:45.080 a lot to do with outlining and with critiquing and 0:22:45.160 --> 0:22:49.879 with rejecting drafts. But I was spared much of the 0:22:49.960 --> 0:22:53.120 things that I'm mostly traid of in writing. So it 0:22:53.200 --> 0:22:56.440 was a very good collaboration. And by the way, we 0:22:56.880 --> 0:23:02.399 benefited a lot from from COVID because that forced stuff 0:23:02.400 --> 0:23:06.080 into quite an efficient way of collaborating. We used to visit. 0:23:06.720 --> 0:23:09.320 Olivier would come to New York from Paris, and I 0:23:09.359 --> 0:23:12.480 would visit Paris for a few days every month, and 0:23:12.520 --> 0:23:14.960 we had a very good time, but it wasn't productive. 0:23:16.240 --> 0:23:19.040 Zooming one or two hours a day turned out to 0:23:19.080 --> 0:23:22.200 be a much better way of writing the book. And 0:23:22.359 --> 0:23:24.879 this is what happened. Uh, it sounds like it was 0:23:24.920 --> 0:23:27.200 just a good excuse to get together in New York 0:23:27.200 --> 0:23:29.720 in Paris and have a little bit of fun. Well, 0:23:29.760 --> 0:23:31.719 I mean, you know, we didn't think of it as 0:23:31.760 --> 0:23:34.280 a good excuse, but it turned out that would waste 0:23:34.359 --> 0:23:36.359 a lot of time and the fair amount of money. 0:23:36.800 --> 0:23:40.400 So you you mentioned you reviewed a lot of manuscript 0:23:40.560 --> 0:23:45.760 from Olivier and Cass and rejected stuff. You and Amos 0:23:45.920 --> 0:23:51.480 very famously would agonize over every sentence in all of 0:23:51.520 --> 0:23:55.240 your publications. You seem to have spent a lot of 0:23:55.280 --> 0:24:02.159 time writing meticulously and very thoughtfully. How has that evolved 0:24:02.200 --> 0:24:04.479 over time? Is this a little easier to sort of 0:24:04.520 --> 0:24:09.320 be the orchestrator and the editor as opposed to, you know, 0:24:09.840 --> 0:24:14.879 just agonizingly putting down every single word. No, it isn't. 0:24:14.960 --> 0:24:17.800 I mean, my this is part of sort of my 0:24:17.920 --> 0:24:23.320 intellectual personality of character that I think most clearly when 0:24:23.320 --> 0:24:26.760 I find flaws in existing text, and I'm not good 0:24:26.800 --> 0:24:30.359 at anticipating the flaws. So I see a flow and 0:24:30.440 --> 0:24:32.719 I correct it, and then there is new text, and 0:24:32.760 --> 0:24:36.080 then I discover a new flow, and and I tend 0:24:36.160 --> 0:24:39.800 to work that way, which is infuriating to make collaborators 0:24:40.280 --> 0:24:42.960 and wish for a lot of time and efforts, but 0:24:43.480 --> 0:24:45.960 that's the way I. On the other hand, I do 0:24:46.119 --> 0:24:48.560 tend to be very critical, and most of the flaws 0:24:48.600 --> 0:24:52.440 that I find do exist, so it tends to lead 0:24:52.480 --> 0:24:55.560 to a good project in a very inefficient way. So 0:24:55.800 --> 0:25:00.520 despite that perfectionism, you know, we all evolve over or time. 0:25:01.080 --> 0:25:05.159 As you were preparing Noise, did you find any of 0:25:05.200 --> 0:25:10.120 your previous writings or research that you either disagree with 0:25:10.280 --> 0:25:13.720 or see from a different perspective or light when you're 0:25:13.760 --> 0:25:16.760 putting this book together? No, not really. I mean, in 0:25:16.880 --> 0:25:22.520 the book, we actually relied on ideas from Thinking Fast 0:25:22.560 --> 0:25:26.719 and Slow, But the book is really fundamentally different. Thinking 0:25:26.760 --> 0:25:30.320 Fast and Slow is a book about individuals and about 0:25:30.359 --> 0:25:33.280 how and it was a book about the average or 0:25:33.320 --> 0:25:37.760 a typical individual and how the average or typical mind works. 0:25:38.400 --> 0:25:42.080 Noise is about individual differences. It's about the way that 0:25:42.560 --> 0:25:47.000 the different people think differently, and so this is a 0:25:47.160 --> 0:25:51.320 really different cut about thinking. It's a different way of 0:25:51.320 --> 0:25:55.200 looking and thinking. So we did use some of the material, 0:25:55.680 --> 0:26:00.440 but the Noise is not a revision of Thinking Fast show. 0:26:00.840 --> 0:26:03.639 It is about the truly different topics that we didn't 0:26:03.640 --> 0:26:07.159 even touch and thinking it clearly, it goes in a 0:26:07.280 --> 0:26:10.240 very different direction, and it looks at some systems and 0:26:10.320 --> 0:26:13.960 some organizations that I don't believe you touched on in 0:26:14.480 --> 0:26:18.080 Thinking Fast. It's kind of interesting because we've already talked 0:26:18.080 --> 0:26:22.040 about medicine and criminal justice and finance. There was one 0:26:22.040 --> 0:26:27.080 section I was fascinated by where you discussed hiring and 0:26:27.160 --> 0:26:31.240 promotions and how I don't want to use the word random, 0:26:31.400 --> 0:26:35.720 but how much noise is in that system and how 0:26:35.800 --> 0:26:42.040 unreliable many organizations hiring processes are. Tell us a little 0:26:42.040 --> 0:26:47.760 bit about that. Well, it terms that people like hiring 0:26:48.200 --> 0:26:52.480 by interviewing people and for me, a general image of 0:26:52.560 --> 0:26:57.280 the individual that they're thinking of hiring. And it turns 0:26:57.280 --> 0:26:59.760 out this is not a good way of doing it. 0:27:00.520 --> 0:27:02.879 A much better way of doing it is what it's 0:27:02.920 --> 0:27:07.760 called the structured interview, the structured process where you accumulate 0:27:07.840 --> 0:27:13.560 information systematically about different characteristics of the person. That is 0:27:13.760 --> 0:27:20.080 less pleasant, it's it's less enjoyable, but much better. And 0:27:20.200 --> 0:27:26.160 better yet is having several in several people do the hiring, 0:27:26.680 --> 0:27:31.920 each of them forming an independence impression, and then they discuss, 0:27:32.160 --> 0:27:35.720 then they average, and then they discuss the average. And 0:27:35.800 --> 0:27:39.639 this is the procedure for example, and and it's about 0:27:39.720 --> 0:27:45.000 state of view. But many places are way short of 0:27:45.359 --> 0:27:49.159 a state of the art. I should add that state 0:27:49.200 --> 0:27:52.760 of the art. Hiring doesn't mean that you're guaranteed the 0:27:52.920 --> 0:27:56.679 perfect sit There's so much there's so much luck in 0:27:56.720 --> 0:28:00.159 the world. There's so much uncertainty that the person to 0:28:00.280 --> 0:28:03.000 how it may be very good but may run into 0:28:03.040 --> 0:28:08.560 difficulties with the boss doesn't like her or something like that. 0:28:09.240 --> 0:28:13.879 And by chance alone you can get a lot of variety. Chance, 0:28:13.960 --> 0:28:16.720 by the way, is not noise. Chance is something that 0:28:16.800 --> 0:28:22.080 happens in the real world. Noise is differences among judgments. 0:28:22.119 --> 0:28:26.800 So hiring is buy and love really very poorly done. 0:28:27.240 --> 0:28:30.200 And it's very poorly done because it doesn't control noise. 0:28:30.680 --> 0:28:36.280