1 00:00:00,080 --> 00:00:06,080 Speaker 1: M. This is Mesters in Business with Very Results on 2 00:00:06,240 --> 00:00:10,920 Speaker 1: Bloomberg Radio this weekend on the podcast What Can I Say? 3 00:00:11,240 --> 00:00:16,400 Speaker 1: Another extra special guest Danny Kaneman, no Bell Prize winner, 4 00:00:16,760 --> 00:00:20,439 Speaker 1: author of Thinking Fast and Slow. His new book is Noise, 5 00:00:20,560 --> 00:00:26,360 Speaker 1: a Fawn Human Judgment and Danny is just so knowledgeable. 6 00:00:26,520 --> 00:00:28,720 Speaker 1: Please call me Danny. I I feel like I have 7 00:00:28,760 --> 00:00:32,680 Speaker 1: to call him Professor Khneman, and he he insists. Uh, 8 00:00:32,680 --> 00:00:37,120 Speaker 1: He's eighty seven years old and incredibly sharp and insightful 9 00:00:37,440 --> 00:00:41,960 Speaker 1: and just so much wisdom and knowledge. If you liked 10 00:00:42,520 --> 00:00:45,879 Speaker 1: Thinking Fast and Slow, which is about judgment error in 11 00:00:46,040 --> 00:00:51,680 Speaker 1: humans in individuals, well, Noise is about how flaws and 12 00:00:52,159 --> 00:00:56,480 Speaker 1: in judgment within broader institutions come about. And it's a 13 00:00:56,520 --> 00:01:01,320 Speaker 1: totally different area and it's absolutely fascinating. I'm a big 14 00:01:01,360 --> 00:01:05,360 Speaker 1: fan of behavioral finance in general, plus all of uh 15 00:01:05,480 --> 00:01:09,600 Speaker 1: Danny's work historically. If you are remotely interested in this, 16 00:01:09,800 --> 00:01:14,080 Speaker 1: then strap yourself. And this is another doozy with no 17 00:01:14,160 --> 00:01:21,240 Speaker 1: further ado. My conversation with Danny Koneman. This is mesters 18 00:01:21,240 --> 00:01:26,920 Speaker 1: in Business with Very Results on Bloomberg Radio. My extra 19 00:01:27,000 --> 00:01:30,880 Speaker 1: special guest this week is Danny Khneman. He was awarded 20 00:01:31,280 --> 00:01:35,280 Speaker 1: the two thousand and two Nobel Memorial Prize in Economic Sciences, 21 00:01:35,720 --> 00:01:40,000 Speaker 1: which he shared with Vernon Smith for his empirical findings 22 00:01:40,480 --> 00:01:43,560 Speaker 1: the work he did with Amos Tversky. And what's so 23 00:01:43,640 --> 00:01:48,240 Speaker 1: fascinating about that Nobel Prize is that Danny is a psychologist. 24 00:01:48,760 --> 00:01:53,080 Speaker 1: The work they did challenge the prevailing thoughts in economic 25 00:01:53,160 --> 00:01:58,920 Speaker 1: theory by establishing a basis for common human eras his 26 00:01:59,440 --> 00:02:02,960 Speaker 1: previous book, Thinking Fast and Slow, was the best seller 27 00:02:03,520 --> 00:02:05,960 Speaker 1: of two thousand and eleven and one a variety of 28 00:02:05,960 --> 00:02:10,520 Speaker 1: different awards, including the National Academy's Communication Award for Best 29 00:02:10,520 --> 00:02:15,320 Speaker 1: Creative Work. His latest book is Just Out Noise, A 30 00:02:15,480 --> 00:02:20,400 Speaker 1: Flaw in Human Judgment, which Danny Koneman wrote with Olive 31 00:02:20,680 --> 00:02:26,040 Speaker 1: Simony and Cass Sunstein. Danny Kneman, welcome back to Bloomberg. 32 00:02:26,400 --> 00:02:29,360 Speaker 1: I'm delighted to be here. You always say call me Danny, 33 00:02:29,400 --> 00:02:31,600 Speaker 1: and I always feel awkward and I feel like I 34 00:02:31,600 --> 00:02:36,919 Speaker 1: should call you professor. But let me just get that call, 35 00:02:37,800 --> 00:02:42,600 Speaker 1: all right, Danny. So let's start very basically. What is noise? 36 00:02:42,680 --> 00:02:46,680 Speaker 1: How does it happen? And where does it come from? Okay, well, 37 00:02:47,840 --> 00:02:53,480 Speaker 1: noise isn't accepted term in statistics. We talk about statistical noise, 38 00:02:53,520 --> 00:02:57,840 Speaker 1: which is variability, and that's where it comes from. We 39 00:02:57,919 --> 00:03:03,360 Speaker 1: talk about noise a measurement, which is unreliability in uh 40 00:03:03,360 --> 00:03:07,320 Speaker 1: in measurement, where measurements that should be identical turn out 41 00:03:07,320 --> 00:03:10,600 Speaker 1: to vary. So that's the background in the use of 42 00:03:10,600 --> 00:03:15,720 Speaker 1: the term as we use it specifically, we intend we 43 00:03:15,760 --> 00:03:20,600 Speaker 1: speak about judgment noise, and this is the situation in 44 00:03:20,639 --> 00:03:25,399 Speaker 1: which judgments should be identical people or the same individual 45 00:03:26,080 --> 00:03:30,000 Speaker 1: judging the same object at different times, or different people 46 00:03:30,680 --> 00:03:34,800 Speaker 1: judging the same object. If they don't agree and I 47 00:03:34,920 --> 00:03:39,960 Speaker 1: expected to agree, we speak about judgment noise, and in general, 48 00:03:40,000 --> 00:03:42,880 Speaker 1: people are expected to agree when they're trying to be accurate. 49 00:03:42,960 --> 00:03:45,440 Speaker 1: So when you have a group of people trying to 50 00:03:45,480 --> 00:03:48,480 Speaker 1: make their best guess about the quantity, it could be 51 00:03:48,760 --> 00:03:53,320 Speaker 1: the symptoms that somebody should should get for a crime. 52 00:03:53,840 --> 00:03:56,200 Speaker 1: It could be the value of the company. It could 53 00:03:56,280 --> 00:04:00,680 Speaker 1: be the premium that somebody should be charged. Oh, it 54 00:04:00,760 --> 00:04:05,400 Speaker 1: could be a diagnosis, a medical diagnosis. In all these cases, 55 00:04:05,760 --> 00:04:09,040 Speaker 1: you might have several people looking at the same information 56 00:04:09,400 --> 00:04:13,960 Speaker 1: making judgments. If they don't agree, there is noise, and 57 00:04:14,160 --> 00:04:17,880 Speaker 1: noise is the topic of the book we wrote. So 58 00:04:18,680 --> 00:04:22,880 Speaker 1: it's fascinating how we start to see noisy decision making 59 00:04:22,920 --> 00:04:25,720 Speaker 1: come up over and over again in the same fields. 60 00:04:25,760 --> 00:04:30,720 Speaker 1: And you just mentioned a few medicine, criminal justice, finance. 61 00:04:31,200 --> 00:04:36,520 Speaker 1: Are there certain fields that are more susceptible two problems 62 00:04:36,640 --> 00:04:40,320 Speaker 1: in expert judgments than others? Or is it just that 63 00:04:41,000 --> 00:04:44,920 Speaker 1: the results of those sort of noisy decisions are so 64 00:04:45,000 --> 00:04:48,880 Speaker 1: much more significant than other fields. Well, we use the 65 00:04:48,920 --> 00:04:55,200 Speaker 1: word judgment when there is room for reasonable disagreement, that is, 66 00:04:55,360 --> 00:04:58,040 Speaker 1: you know, we don't use the word judgment for computation, 67 00:04:58,320 --> 00:05:02,040 Speaker 1: and when compute station is appropriate, we wouldn't be talking 68 00:05:02,040 --> 00:05:05,960 Speaker 1: of noise. We would be talking of people making mistakes. 69 00:05:06,000 --> 00:05:09,799 Speaker 1: And we talked about noise when when it's a matter 70 00:05:09,800 --> 00:05:14,960 Speaker 1: of judgment. And and so the existence of noise by 71 00:05:15,000 --> 00:05:18,719 Speaker 1: itself is not a surprise. Whatever the surprise is the 72 00:05:18,800 --> 00:05:24,760 Speaker 1: amount of noise just a lot more then would be expected. 73 00:05:25,400 --> 00:05:28,120 Speaker 1: And here, I think the best way to explain this 74 00:05:28,279 --> 00:05:32,600 Speaker 1: is too to tell you the story of how I 75 00:05:32,800 --> 00:05:35,760 Speaker 1: started to work on noise. Then where the whole thing began. 76 00:05:37,520 --> 00:05:41,800 Speaker 1: So I was consulting in an insurance company, said than 77 00:05:41,920 --> 00:05:49,080 Speaker 1: eight years ago, and I had the idea of running 78 00:05:49,120 --> 00:05:53,320 Speaker 1: But today we would call a noise audit, that is underwriters. 79 00:05:53,440 --> 00:05:58,440 Speaker 1: To take one example, we had several underwriters, so some 80 00:05:58,560 --> 00:06:04,159 Speaker 1: realistic cases, the same cases. They were constructed by executives 81 00:06:04,240 --> 00:06:08,920 Speaker 1: of experts and underwriting, so they were completely realistic, and 82 00:06:08,960 --> 00:06:12,479 Speaker 1: you might have fifty underwriters looking at the same premium. 83 00:06:12,520 --> 00:06:16,560 Speaker 1: Now nobody would expect the numbers to be exactly the same. 84 00:06:17,200 --> 00:06:21,520 Speaker 1: But I asked executives, if you take a pair of 85 00:06:21,720 --> 00:06:25,840 Speaker 1: underwriters at random, by how much would you expect them 86 00:06:25,880 --> 00:06:29,600 Speaker 1: to differ in percentages? That is, you take the average 87 00:06:29,680 --> 00:06:32,280 Speaker 1: or the pairer, you take the difference, you divide the 88 00:06:32,360 --> 00:06:36,320 Speaker 1: difference by the average. What percentage looks reasonable to you? 89 00:06:37,400 --> 00:06:41,000 Speaker 1: And they're answer typically with ten percent. And we have, 90 00:06:41,320 --> 00:06:47,200 Speaker 1: by the way, we have surveyed hundreds of executives since then, 91 00:06:47,400 --> 00:06:50,440 Speaker 1: and ten percent seems to be what we expect the 92 00:06:50,520 --> 00:06:55,200 Speaker 1: reasonable difference to be, which is tolerable when two people 93 00:06:55,279 --> 00:06:59,000 Speaker 1: make judgments of a quantity. Now, the correct answer among 94 00:06:59,120 --> 00:07:04,320 Speaker 1: underwriters in that company with sixty five percent more than 95 00:07:04,440 --> 00:07:09,400 Speaker 1: five times as much as expected. That's the phenomenon. So 96 00:07:09,560 --> 00:07:12,560 Speaker 1: we expect this agreement where judgment is involved, We just 97 00:07:12,640 --> 00:07:18,320 Speaker 1: don't expect that much disagreement. And this basically was the 98 00:07:18,360 --> 00:07:23,200 Speaker 1: observation that started us on that part of writing a book, 99 00:07:23,280 --> 00:07:27,480 Speaker 1: because it turns out that you find astonishing amount of 100 00:07:27,520 --> 00:07:31,080 Speaker 1: disagreement when you look for it, and you find it 101 00:07:31,160 --> 00:07:36,560 Speaker 1: wherever judgment is involved. So engineers who make estimates on 102 00:07:36,600 --> 00:07:39,960 Speaker 1: the basis of objected data, they don't have a problem 103 00:07:40,040 --> 00:07:42,520 Speaker 1: of judgment, but to the extent they do have a 104 00:07:42,560 --> 00:07:45,559 Speaker 1: problem of judgment, you will expect a lot of noise. 105 00:07:46,120 --> 00:07:51,120 Speaker 1: So that's the basic finding. And you know, wherever precision 106 00:07:51,240 --> 00:07:53,880 Speaker 1: is important, wherever it is important to get to the 107 00:07:54,000 --> 00:07:57,680 Speaker 1: right number, noise is a source of there. So where 108 00:07:57,840 --> 00:08:02,880 Speaker 1: some people over estimating and others underestimating now making errors. Huh. 109 00:08:03,200 --> 00:08:06,440 Speaker 1: So let me roll back to that insurance company, which 110 00:08:06,480 --> 00:08:10,720 Speaker 1: you discuss in the book, and there were two particular 111 00:08:10,840 --> 00:08:15,640 Speaker 1: areas where we're there were these broad disagreements. The first 112 00:08:15,920 --> 00:08:21,320 Speaker 1: was when people were trying to estimate the risk involved 113 00:08:21,840 --> 00:08:26,520 Speaker 1: with some insurance and so how you price that very 114 00:08:26,600 --> 00:08:30,280 Speaker 1: much determines. If you're too expensive, meaning you think it's 115 00:08:30,320 --> 00:08:33,600 Speaker 1: a high risk, you're not gonna win the business. And 116 00:08:33,640 --> 00:08:37,480 Speaker 1: if it's too cheap relative to the risk, well you'll 117 00:08:37,480 --> 00:08:41,559 Speaker 1: win the business, but it won't be profitable. The costs 118 00:08:41,600 --> 00:08:44,240 Speaker 1: will be higher. And then on the other end, in 119 00:08:44,280 --> 00:08:48,600 Speaker 1: the appraisal of hey, what are the damages here? Figuring 120 00:08:48,600 --> 00:08:52,800 Speaker 1: out how much something should be covered by insurance, what 121 00:08:52,920 --> 00:08:56,040 Speaker 1: the dollar amount is, and the same situation. You can't 122 00:08:56,080 --> 00:08:58,920 Speaker 1: be too stingy or you lose customers, but you can't 123 00:08:58,960 --> 00:09:01,840 Speaker 1: be too generous when you give the house away. How 124 00:09:02,000 --> 00:09:08,120 Speaker 1: significant a financial issue was this for the insurance company. Well, 125 00:09:08,360 --> 00:09:12,160 Speaker 1: you know, it's not easy to estimate that exactly, but 126 00:09:12,240 --> 00:09:16,640 Speaker 1: I can tell you the question that I asked some executives. 127 00:09:16,920 --> 00:09:20,840 Speaker 1: I said, suppose there is a correct number, say for 128 00:09:20,880 --> 00:09:26,880 Speaker 1: the underwriters, and and you have somebody who overestimates the 129 00:09:27,320 --> 00:09:31,800 Speaker 1: underwriting cost by fifteen, how much would you expect that 130 00:09:31,880 --> 00:09:36,760 Speaker 1: to cost the company? And the same question for underestimating 131 00:09:37,080 --> 00:09:42,199 Speaker 1: by fifteen. In fact, fifteen percent on either side is 132 00:09:42,320 --> 00:09:46,760 Speaker 1: much less than noise than we had discovered. But people 133 00:09:47,000 --> 00:09:50,520 Speaker 1: estimated on that basis that this could be in the 134 00:09:50,600 --> 00:09:53,880 Speaker 1: hundreds of millions or billions of dollars. This is a 135 00:09:53,960 --> 00:09:59,520 Speaker 1: very large company, so uh underwriters have a lot of 136 00:09:59,559 --> 00:10:04,000 Speaker 1: important decisions to make claims, adjustice, make important decisions which 137 00:10:04,000 --> 00:10:08,280 Speaker 1: are really consequential for the company. And errors of the 138 00:10:08,320 --> 00:10:13,320 Speaker 1: magnitude that we observe are costly. The main reason that 139 00:10:13,400 --> 00:10:17,280 Speaker 1: they may be less costly is that if error is 140 00:10:17,360 --> 00:10:21,920 Speaker 1: present in all insurance companies, if all insurance companies are noisy, 141 00:10:22,280 --> 00:10:25,320 Speaker 1: then some of the damage to each individual company will 142 00:10:25,360 --> 00:10:29,400 Speaker 1: be reduced. But that's the best that we can say. Well, 143 00:10:29,480 --> 00:10:33,000 Speaker 1: one would imagine the insurance company that could reduce noise 144 00:10:33,600 --> 00:10:38,000 Speaker 1: would find itself at a competitive advantage. Absolutely, there was 145 00:10:38,040 --> 00:10:40,680 Speaker 1: something you had written that really stood out to me. 146 00:10:41,360 --> 00:10:44,720 Speaker 1: There's an assumption when you have noisy systems and everything 147 00:10:44,800 --> 00:10:49,319 Speaker 1: from criminal justice to medicine to insurance, that these errors 148 00:10:49,360 --> 00:10:53,480 Speaker 1: tend to cancel out. But you found out that noisy 149 00:10:53,559 --> 00:10:57,360 Speaker 1: systems have errors. Not only do they not cancel out, 150 00:10:57,559 --> 00:11:01,800 Speaker 1: they tend to add up. Explain, well, if you have 151 00:11:02,040 --> 00:11:08,200 Speaker 1: two separate underwriters estimating the same risk and you average 152 00:11:08,520 --> 00:11:13,880 Speaker 1: their ratings, then the average will be usually more precise 153 00:11:14,480 --> 00:11:19,920 Speaker 1: than the individual judgments because errors in measuring the same 154 00:11:20,200 --> 00:11:25,560 Speaker 1: object do cancel out, but errors when you're responding to 155 00:11:25,800 --> 00:11:30,120 Speaker 1: different objects do not cancel out. So if you overprice 156 00:11:30,240 --> 00:11:35,200 Speaker 1: one policy and you underprice the another policy that doesn't 157 00:11:35,240 --> 00:11:39,160 Speaker 1: cancel out, you've made two mistakes, and you know it's 158 00:11:39,160 --> 00:11:43,600 Speaker 1: the same thing with two with two judges. If one 159 00:11:44,000 --> 00:11:48,440 Speaker 1: defendant is studish too much and another defendant is spunished 160 00:11:48,480 --> 00:11:52,680 Speaker 1: too little. On average, you know, punishment was right, but 161 00:11:52,760 --> 00:11:56,040 Speaker 1: two cares about the average two mistakes were made. So 162 00:11:57,280 --> 00:12:00,920 Speaker 1: there is some confusion because people think about canceling out. 163 00:12:01,000 --> 00:12:04,800 Speaker 1: But that happens when people evaluate, or judge, or measure 164 00:12:05,080 --> 00:12:09,920 Speaker 1: the same thing there and errors do cancel out. I 165 00:12:09,960 --> 00:12:12,560 Speaker 1: recall a book a couple of years ago called The 166 00:12:12,720 --> 00:12:17,240 Speaker 1: End of Average that looked at that exact issue and said, 167 00:12:17,920 --> 00:12:20,520 Speaker 1: you know, we we tend to look at these averages 168 00:12:20,559 --> 00:12:24,240 Speaker 1: as if anyone is experiencing an average. But what you're 169 00:12:24,280 --> 00:12:27,160 Speaker 1: really saying is, hey, if it averages out to be 170 00:12:27,240 --> 00:12:29,000 Speaker 1: the right answer, it means you have a lot of 171 00:12:29,040 --> 00:12:34,600 Speaker 1: wrong answers. That's right. Averaging out to the right answer 172 00:12:34,760 --> 00:12:38,280 Speaker 1: is not a guarantee. And that is a nice example 173 00:12:38,320 --> 00:12:42,520 Speaker 1: of the phenomenon we're discussing in the book, the neglect 174 00:12:42,600 --> 00:12:46,560 Speaker 1: of knowledge. People really tend to focus on bias, which 175 00:12:46,600 --> 00:12:50,000 Speaker 1: is the average era. But you can have a zero 176 00:12:50,160 --> 00:12:54,680 Speaker 1: bias and the very poor performance if you have a 177 00:12:54,679 --> 00:12:57,360 Speaker 1: lot of over estimates, and they love about the estimate. 178 00:12:58,240 --> 00:13:01,240 Speaker 1: Quite interesting. So one of the things in the book 179 00:13:01,320 --> 00:13:04,560 Speaker 1: that I was so taken by had to do with 180 00:13:04,679 --> 00:13:09,720 Speaker 1: the admissions committee for a university, and they used to 181 00:13:09,800 --> 00:13:15,000 Speaker 1: have all the admission officers do a blind review and 182 00:13:15,160 --> 00:13:18,360 Speaker 1: get together and try and hash out who they thought 183 00:13:18,360 --> 00:13:20,240 Speaker 1: would be a good fit for the school and who 184 00:13:20,280 --> 00:13:24,520 Speaker 1: wouldn't um. But it led to a problem, and they 185 00:13:24,559 --> 00:13:29,960 Speaker 1: started having the first person who who reviewed the application 186 00:13:30,200 --> 00:13:33,280 Speaker 1: put their review number on the corner like they would 187 00:13:33,280 --> 00:13:36,120 Speaker 1: actually put their rating on the page, and then hand 188 00:13:36,160 --> 00:13:39,760 Speaker 1: it off to the second person. And you described this 189 00:13:39,880 --> 00:13:47,960 Speaker 1: as the illusion of agreements in organizations. Tell us about that, Well, uh, 190 00:13:48,120 --> 00:13:51,080 Speaker 1: you know, this is an experience as any teacher has 191 00:13:51,360 --> 00:13:57,400 Speaker 1: has had. For example, when you're looking at the test booklet, 192 00:13:57,880 --> 00:14:02,160 Speaker 1: the student has written several essay. If you score a 193 00:14:02,320 --> 00:14:05,960 Speaker 1: test booklet, you score the first question, then the second, 194 00:14:06,040 --> 00:14:10,160 Speaker 1: then the third, then in general you'll find that your 195 00:14:10,200 --> 00:14:13,839 Speaker 1: grades do not vary very much. On the other hand, 196 00:14:14,240 --> 00:14:19,080 Speaker 1: if you read the same test across all students and 197 00:14:19,200 --> 00:14:22,120 Speaker 1: write the score at the back of the of the 198 00:14:22,240 --> 00:14:25,400 Speaker 1: booklet so that you don't know when you read the 199 00:14:25,440 --> 00:14:28,760 Speaker 1: second question where the first question was, you will often 200 00:14:28,840 --> 00:14:32,560 Speaker 1: be shocked by the discrepancy between the first and the second. 201 00:14:32,840 --> 00:14:36,920 Speaker 1: There is a mechanism by which people, if you gave 202 00:14:36,960 --> 00:14:39,720 Speaker 1: a good grade the first time, you're going to be 203 00:14:39,760 --> 00:14:42,280 Speaker 1: inclined to give the benefit of the doubt to the 204 00:14:42,320 --> 00:14:47,160 Speaker 1: student if there is any ambiguos ambiguous answered. Exactly the 205 00:14:47,240 --> 00:14:51,160 Speaker 1: same thing happens in deliberations. And in the example that 206 00:14:51,200 --> 00:14:56,520 Speaker 1: we gave, the admissions committee used to operate in what 207 00:14:56,760 --> 00:15:02,479 Speaker 1: we consider the correct manner. That is, everybody would individually 208 00:15:02,560 --> 00:15:05,640 Speaker 1: make their judgments and then they would reveal all judgments 209 00:15:05,640 --> 00:15:09,320 Speaker 1: together and average them. But they changed the system so 210 00:15:09,520 --> 00:15:13,600 Speaker 1: that now people spoke in sequence, and the question was asked, 211 00:15:13,640 --> 00:15:16,440 Speaker 1: why do you do this, This is not optimal, and 212 00:15:16,480 --> 00:15:19,240 Speaker 1: they say, well, we used to do it the other way. 213 00:15:19,600 --> 00:15:23,000 Speaker 1: We used to have people prepare their judgments individually, but 214 00:15:23,080 --> 00:15:28,120 Speaker 1: there was so much disagreement that we stopped. And that's 215 00:15:28,160 --> 00:15:34,320 Speaker 1: an example where people managed to avoid finding out how 216 00:15:34,400 --> 00:15:38,240 Speaker 1: much noise there really is because when they when people 217 00:15:38,280 --> 00:15:42,080 Speaker 1: are allowed to influence each other or influence themselves in 218 00:15:42,120 --> 00:15:45,560 Speaker 1: the case of the teacher reading multiple booklets, when when 219 00:15:45,640 --> 00:15:50,880 Speaker 1: judgments are not independent, they are less effective statistically, you 220 00:15:51,080 --> 00:15:54,400 Speaker 1: just have less information. Think of the example in which 221 00:15:54,680 --> 00:15:57,960 Speaker 1: the first person to talk is the CEO, and then 222 00:15:58,080 --> 00:16:03,200 Speaker 1: everybody agrees, then the agreement of other people is not informative. 223 00:16:03,800 --> 00:16:07,000 Speaker 1: In fact, you had one person making the judgment. That's 224 00:16:07,040 --> 00:16:12,520 Speaker 1: the extreme of abolishing, of eliminating the appearance of noise 225 00:16:12,840 --> 00:16:17,200 Speaker 1: without eliminating the reality of not So it sounds like 226 00:16:17,360 --> 00:16:24,680 Speaker 1: groups and corporations, institutions, schools, they seem to amplify noise. 227 00:16:25,560 --> 00:16:29,240 Speaker 1: Is that just the nature of bigger numbers of people 228 00:16:29,320 --> 00:16:33,240 Speaker 1: working together that they're going to create additional noise? No 229 00:16:33,720 --> 00:16:37,840 Speaker 1: not necessarily what happens in a group if they made 230 00:16:38,000 --> 00:16:42,520 Speaker 1: their judgments individually, is not that noise is amplified. The 231 00:16:42,640 --> 00:16:48,240 Speaker 1: true noise is revealed. So suppose you had underwriters. Suppose 232 00:16:48,320 --> 00:16:55,200 Speaker 1: you had multiple underwriters judging routinely every every risk, then 233 00:16:55,280 --> 00:16:59,680 Speaker 1: the optimal procedure would be to have them making independent 234 00:16:59,720 --> 00:17:04,359 Speaker 1: drug ugments and only then then revealing the two judgments 235 00:17:04,359 --> 00:17:09,960 Speaker 1: and averaging them. That's clearly the optimal procedure, and and 236 00:17:10,080 --> 00:17:15,359 Speaker 1: the optimal procedure reveals noise and then reduces it by averaging. 237 00:17:15,880 --> 00:17:19,880 Speaker 1: But when a sail individual makes a judgment, that judgment 238 00:17:19,920 --> 00:17:25,200 Speaker 1: will be noisy. And when individuals are allowed to influence 239 00:17:25,240 --> 00:17:29,200 Speaker 1: each other, then it's more like a single judgment than 240 00:17:29,280 --> 00:17:32,399 Speaker 1: it is, like having multiple judgments or the same opta. 241 00:17:33,560 --> 00:17:37,560 Speaker 1: So you use the phrase naive realism, What what does 242 00:17:37,600 --> 00:17:42,600 Speaker 1: that mean relative to noise in groups? Well, what made 243 00:17:42,640 --> 00:17:46,719 Speaker 1: realism means is is a statement which most of us 244 00:17:46,760 --> 00:17:50,040 Speaker 1: are most of the time, that we think we're right, 245 00:17:50,840 --> 00:17:53,520 Speaker 1: We think we have the right view of situation, we 246 00:17:53,600 --> 00:17:57,560 Speaker 1: think we understand strengths correctly. In short, we see the 247 00:17:57,600 --> 00:18:02,080 Speaker 1: world as the world is. That's native realism. And if 248 00:18:02,119 --> 00:18:05,639 Speaker 1: I see the world as it is, and you know, 249 00:18:05,760 --> 00:18:08,359 Speaker 1: they are friends and colleagues looking at the same world, 250 00:18:08,760 --> 00:18:12,280 Speaker 1: and I like and respect them, then naturally I assume 251 00:18:12,359 --> 00:18:14,399 Speaker 1: that they see the world as I do because I 252 00:18:14,480 --> 00:18:17,280 Speaker 1: see it right, and if I respect them, they see 253 00:18:17,280 --> 00:18:21,240 Speaker 1: it right as well. So that's naive realism. And naive 254 00:18:21,280 --> 00:18:26,040 Speaker 1: realism prevents us from becoming aware of the amount of 255 00:18:26,080 --> 00:18:29,919 Speaker 1: noise that there is. We're just assume noise away. We 256 00:18:30,000 --> 00:18:33,159 Speaker 1: saw that very nicely among underwriters. You know, when you 257 00:18:33,280 --> 00:18:38,320 Speaker 1: interview an underwriter, what happens to them? But how does 258 00:18:38,320 --> 00:18:42,359 Speaker 1: an underwriter become expert in the absence of any feedback 259 00:18:42,800 --> 00:18:46,320 Speaker 1: Because they don't they don't get any feedback from reality 260 00:18:46,400 --> 00:18:50,679 Speaker 1: about their underwriting and what happens is that they become 261 00:18:50,720 --> 00:18:55,240 Speaker 1: increasingly confident, and largely because they agree with themselves. So 262 00:18:55,320 --> 00:18:58,360 Speaker 1: when you agree with yourself a lot, and you think 263 00:18:58,400 --> 00:19:03,440 Speaker 1: you're right, and you make judgments with increasing speed and confidence, 264 00:19:03,800 --> 00:19:06,919 Speaker 1: so that makes you think that you're even rter. That's 265 00:19:07,200 --> 00:19:12,960 Speaker 1: naive realism, allowing massive noise to occur with everybody convinced 266 00:19:12,960 --> 00:19:16,320 Speaker 1: that they're doing the right thing, but in fact they 267 00:19:16,359 --> 00:19:18,359 Speaker 1: may not be doing the right thing because as they 268 00:19:18,359 --> 00:19:20,520 Speaker 1: were looking at the same problem, there would be the 269 00:19:20,640 --> 00:19:26,000 Speaker 1: food quite fascinating. So we become familiar with a particular area. 270 00:19:26,680 --> 00:19:31,480 Speaker 1: That familiarity leads us to think that we're developing an expertise. 271 00:19:31,960 --> 00:19:35,080 Speaker 1: We tend to make more snap judgments and without any 272 00:19:35,080 --> 00:19:38,840 Speaker 1: sort of feedback loop, how can we possibly know that 273 00:19:38,880 --> 00:19:43,359 Speaker 1: we're right? And yet that absence of feedback seems to 274 00:19:43,480 --> 00:19:48,800 Speaker 1: strengthen people's self confidence. Do I have that right? And 275 00:19:49,080 --> 00:19:53,960 Speaker 1: think of the number of situations in which exactly this 276 00:19:54,200 --> 00:19:58,480 Speaker 1: whole there's a judge doesn't have feedback as to whether 277 00:19:58,640 --> 00:20:03,200 Speaker 1: judgment was correct or on bail judge. Sometimes there is feedback, 278 00:20:03,240 --> 00:20:06,760 Speaker 1: but it's a symmetry. So bail judge may get feedback 279 00:20:06,800 --> 00:20:09,800 Speaker 1: on somebody who was released and committed the crime, but 280 00:20:09,920 --> 00:20:13,720 Speaker 1: the bail judge will never know if somebody was incarcerated 281 00:20:13,960 --> 00:20:18,160 Speaker 1: would have committed the crime. So feedback is a massive problem. 282 00:20:18,560 --> 00:20:22,639 Speaker 1: And many professionals at the minimum feedback, and yet they 283 00:20:22,720 --> 00:20:26,679 Speaker 1: become confident and they feel their expots. But in those 284 00:20:26,720 --> 00:20:29,800 Speaker 1: cases there is a high risk of noise and a 285 00:20:29,800 --> 00:20:32,879 Speaker 1: lot of that feedback seems to be only at the extreme. 286 00:20:33,520 --> 00:20:37,879 Speaker 1: A bridge collapses, there are a plane crashes, somebody dies, 287 00:20:38,040 --> 00:20:41,560 Speaker 1: there's someone out on bail commits a crime. What about 288 00:20:41,600 --> 00:20:44,720 Speaker 1: all of the lack of a better word, near missus 289 00:20:45,440 --> 00:20:49,520 Speaker 1: where there is a bad judgment, something happens. It's not 290 00:20:49,640 --> 00:20:53,040 Speaker 1: quite as terrible as a as an airplane plane crash, 291 00:20:53,640 --> 00:20:56,639 Speaker 1: and it it's resolved before there's damage, but it's pretty 292 00:20:56,680 --> 00:21:02,280 Speaker 1: clear the basic judgment was wrong. How does that affect 293 00:21:02,280 --> 00:21:07,240 Speaker 1: a person's future judgment. Well, in situations where there are 294 00:21:07,359 --> 00:21:11,200 Speaker 1: near missus, there is an opportunity to learn. And in 295 00:21:11,359 --> 00:21:16,399 Speaker 1: world run you know, well run airlines and and and 296 00:21:16,520 --> 00:21:21,119 Speaker 1: air traffic systems keep track very closely of near missus 297 00:21:21,280 --> 00:21:25,240 Speaker 1: because those are their opportunities to learn without without tragedies. 298 00:21:25,720 --> 00:21:29,920 Speaker 1: But in many situations you get no feedback at all. 299 00:21:30,320 --> 00:21:33,960 Speaker 1: In the idea of having senses in bridges that gives 300 00:21:34,000 --> 00:21:38,080 Speaker 1: you a sensitive measurement of how much stress there is 301 00:21:38,480 --> 00:21:42,480 Speaker 1: that necessarily recent there used to be very poor feedback 302 00:21:42,680 --> 00:21:46,080 Speaker 1: on whether a bride would collapse or not, and in 303 00:21:46,160 --> 00:21:50,320 Speaker 1: many situations that professionals make jugment on, there's no feedback 304 00:21:50,359 --> 00:21:55,359 Speaker 1: at all. Quite interesting. So let's talk about this book, 305 00:21:56,240 --> 00:22:01,120 Speaker 1: which was a collaboration. What was it like working with 306 00:22:01,160 --> 00:22:04,879 Speaker 1: those two gentlemen versus thinking Fast and Slow, which I 307 00:22:05,000 --> 00:22:08,800 Speaker 1: kind of get the sense was you sitting down and 308 00:22:09,520 --> 00:22:13,840 Speaker 1: putting a lot of your previous work into a context 309 00:22:14,520 --> 00:22:21,000 Speaker 1: for public consumption. Well, writing Fast and Slow was mostly 310 00:22:21,040 --> 00:22:25,960 Speaker 1: a very lonely experience, and writing with collaborators was really 311 00:22:25,960 --> 00:22:30,520 Speaker 1: a pleasure. So it was it was a relief to 312 00:22:30,600 --> 00:22:33,480 Speaker 1: be able to count on people to find mistakes to 313 00:22:33,560 --> 00:22:37,560 Speaker 1: correct them, and and a lot of the text UH 314 00:22:38,080 --> 00:22:42,040 Speaker 1: was actually written by Olivier and by Kass. I had 315 00:22:42,080 --> 00:22:45,080 Speaker 1: a lot to do with outlining and with critiquing and 316 00:22:45,160 --> 00:22:49,879 Speaker 1: with rejecting drafts. But I was spared much of the 317 00:22:49,960 --> 00:22:53,120 Speaker 1: things that I'm mostly traid of in writing. So it 318 00:22:53,200 --> 00:22:56,440 Speaker 1: was a very good collaboration. And by the way, we 319 00:22:56,880 --> 00:23:02,399 Speaker 1: benefited a lot from from COVID because that forced stuff 320 00:23:02,400 --> 00:23:06,080 Speaker 1: into quite an efficient way of collaborating. We used to visit. 321 00:23:06,720 --> 00:23:09,320 Speaker 1: Olivier would come to New York from Paris, and I 322 00:23:09,359 --> 00:23:12,480 Speaker 1: would visit Paris for a few days every month, and 323 00:23:12,520 --> 00:23:14,960 Speaker 1: we had a very good time, but it wasn't productive. 324 00:23:16,240 --> 00:23:19,040 Speaker 1: Zooming one or two hours a day turned out to 325 00:23:19,080 --> 00:23:22,200 Speaker 1: be a much better way of writing the book. And 326 00:23:22,359 --> 00:23:24,879 Speaker 1: this is what happened. Uh, it sounds like it was 327 00:23:24,920 --> 00:23:27,200 Speaker 1: just a good excuse to get together in New York 328 00:23:27,200 --> 00:23:29,720 Speaker 1: in Paris and have a little bit of fun. Well, 329 00:23:29,760 --> 00:23:31,719 Speaker 1: I mean, you know, we didn't think of it as 330 00:23:31,760 --> 00:23:34,280 Speaker 1: a good excuse, but it turned out that would waste 331 00:23:34,359 --> 00:23:36,359 Speaker 1: a lot of time and the fair amount of money. 332 00:23:36,800 --> 00:23:40,400 Speaker 1: So you you mentioned you reviewed a lot of manuscript 333 00:23:40,560 --> 00:23:45,760 Speaker 1: from Olivier and Cass and rejected stuff. You and Amos 334 00:23:45,920 --> 00:23:51,480 Speaker 1: very famously would agonize over every sentence in all of 335 00:23:51,520 --> 00:23:55,240 Speaker 1: your publications. You seem to have spent a lot of 336 00:23:55,280 --> 00:24:02,159 Speaker 1: time writing meticulously and very thoughtfully. How has that evolved 337 00:24:02,200 --> 00:24:04,479 Speaker 1: over time? Is this a little easier to sort of 338 00:24:04,520 --> 00:24:09,320 Speaker 1: be the orchestrator and the editor as opposed to, you know, 339 00:24:09,840 --> 00:24:14,879 Speaker 1: just agonizingly putting down every single word. No, it isn't. 340 00:24:14,960 --> 00:24:17,800 Speaker 1: I mean, my this is part of sort of my 341 00:24:17,920 --> 00:24:23,320 Speaker 1: intellectual personality of character that I think most clearly when 342 00:24:23,320 --> 00:24:26,760 Speaker 1: I find flaws in existing text, and I'm not good 343 00:24:26,800 --> 00:24:30,359 Speaker 1: at anticipating the flaws. So I see a flow and 344 00:24:30,440 --> 00:24:32,719 Speaker 1: I correct it, and then there is new text, and 345 00:24:32,760 --> 00:24:36,080 Speaker 1: then I discover a new flow, and and I tend 346 00:24:36,160 --> 00:24:39,800 Speaker 1: to work that way, which is infuriating to make collaborators 347 00:24:40,280 --> 00:24:42,960 Speaker 1: and wish for a lot of time and efforts, but 348 00:24:43,480 --> 00:24:45,960 Speaker 1: that's the way I. On the other hand, I do 349 00:24:46,119 --> 00:24:48,560 Speaker 1: tend to be very critical, and most of the flaws 350 00:24:48,600 --> 00:24:52,440 Speaker 1: that I find do exist, so it tends to lead 351 00:24:52,480 --> 00:24:55,560 Speaker 1: to a good project in a very inefficient way. So 352 00:24:55,800 --> 00:25:00,520 Speaker 1: despite that perfectionism, you know, we all evolve over or time. 353 00:25:01,080 --> 00:25:05,159 Speaker 1: As you were preparing Noise, did you find any of 354 00:25:05,200 --> 00:25:10,120 Speaker 1: your previous writings or research that you either disagree with 355 00:25:10,280 --> 00:25:13,720 Speaker 1: or see from a different perspective or light when you're 356 00:25:13,760 --> 00:25:16,760 Speaker 1: putting this book together? No, not really. I mean, in 357 00:25:16,880 --> 00:25:22,520 Speaker 1: the book, we actually relied on ideas from Thinking Fast 358 00:25:22,560 --> 00:25:26,719 Speaker 1: and Slow, But the book is really fundamentally different. Thinking 359 00:25:26,760 --> 00:25:30,320 Speaker 1: Fast and Slow is a book about individuals and about 360 00:25:30,359 --> 00:25:33,280 Speaker 1: how and it was a book about the average or 361 00:25:33,320 --> 00:25:37,760 Speaker 1: a typical individual and how the average or typical mind works. 362 00:25:38,400 --> 00:25:42,080 Speaker 1: Noise is about individual differences. It's about the way that 363 00:25:42,560 --> 00:25:47,000 Speaker 1: the different people think differently, and so this is a 364 00:25:47,160 --> 00:25:51,320 Speaker 1: really different cut about thinking. It's a different way of 365 00:25:51,320 --> 00:25:55,200 Speaker 1: looking and thinking. So we did use some of the material, 366 00:25:55,680 --> 00:26:00,440 Speaker 1: but the Noise is not a revision of Thinking Fast show. 367 00:26:00,840 --> 00:26:03,639 Speaker 1: It is about the truly different topics that we didn't 368 00:26:03,640 --> 00:26:07,159 Speaker 1: even touch and thinking it clearly, it goes in a 369 00:26:07,280 --> 00:26:10,240 Speaker 1: very different direction, and it looks at some systems and 370 00:26:10,320 --> 00:26:13,960 Speaker 1: some organizations that I don't believe you touched on in 371 00:26:14,480 --> 00:26:18,080 Speaker 1: Thinking Fast. It's kind of interesting because we've already talked 372 00:26:18,080 --> 00:26:22,040 Speaker 1: about medicine and criminal justice and finance. There was one 373 00:26:22,040 --> 00:26:27,080 Speaker 1: section I was fascinated by where you discussed hiring and 374 00:26:27,160 --> 00:26:31,240 Speaker 1: promotions and how I don't want to use the word random, 375 00:26:31,400 --> 00:26:35,720 Speaker 1: but how much noise is in that system and how 376 00:26:35,800 --> 00:26:42,040 Speaker 1: unreliable many organizations hiring processes are. Tell us a little 377 00:26:42,040 --> 00:26:47,760 Speaker 1: bit about that. Well, it terms that people like hiring 378 00:26:48,200 --> 00:26:52,480 Speaker 1: by interviewing people and for me, a general image of 379 00:26:52,560 --> 00:26:57,280 Speaker 1: the individual that they're thinking of hiring. And it turns 380 00:26:57,280 --> 00:26:59,760 Speaker 1: out this is not a good way of doing it. 381 00:27:00,520 --> 00:27:02,879 Speaker 1: A much better way of doing it is what it's 382 00:27:02,920 --> 00:27:07,760 Speaker 1: called the structured interview, the structured process where you accumulate 383 00:27:07,840 --> 00:27:13,560 Speaker 1: information systematically about different characteristics of the person. That is 384 00:27:13,760 --> 00:27:20,080 Speaker 1: less pleasant, it's it's less enjoyable, but much better. And 385 00:27:20,200 --> 00:27:26,160 Speaker 1: better yet is having several in several people do the hiring, 386 00:27:26,680 --> 00:27:31,920 Speaker 1: each of them forming an independence impression, and then they discuss, 387 00:27:32,160 --> 00:27:35,720 Speaker 1: then they average, and then they discuss the average. And 388 00:27:35,800 --> 00:27:39,639 Speaker 1: this is the procedure for example, and and it's about 389 00:27:39,720 --> 00:27:45,000 Speaker 1: state of view. But many places are way short of 390 00:27:45,359 --> 00:27:49,159 Speaker 1: a state of the art. I should add that state 391 00:27:49,200 --> 00:27:52,760 Speaker 1: of the art. Hiring doesn't mean that you're guaranteed the 392 00:27:52,920 --> 00:27:56,679 Speaker 1: perfect sit There's so much there's so much luck in 393 00:27:56,720 --> 00:28:00,159 Speaker 1: the world. There's so much uncertainty that the person to 394 00:28:00,280 --> 00:28:03,000 Speaker 1: how it may be very good but may run into 395 00:28:03,040 --> 00:28:08,560 Speaker 1: difficulties with the boss doesn't like her or something like that. 396 00:28:09,240 --> 00:28:13,879 Speaker 1: And by chance alone you can get a lot of variety. Chance, 397 00:28:13,960 --> 00:28:16,720 Speaker 1: by the way, is not noise. Chance is something that 398 00:28:16,800 --> 00:28:22,080 Speaker 1: happens in the real world. Noise is differences among judgments. 399 00:28:22,119 --> 00:28:26,800 Speaker 1: So hiring is buy and love really very poorly done. 400 00:28:27,240 --> 00:28:30,200 Speaker 1: And it's very poorly done because it doesn't control noise. 401 00:28:30,680 --> 00:28:36,280 Speaker 1: Quite fascinating. So the book goes over how noise affects 402 00:28:36,359 --> 00:28:41,240 Speaker 1: judgment and how it introduces a variety of errors into 403 00:28:41,320 --> 00:28:46,240 Speaker 1: our institutional decision making process. What can we do to 404 00:28:46,400 --> 00:28:51,520 Speaker 1: improve that process? Well, in the book, we we introduce 405 00:28:51,600 --> 00:28:57,640 Speaker 1: a concept that we call deci isn't hygiene And you 406 00:28:57,680 --> 00:29:01,040 Speaker 1: know the word is that particularly appealing. It's intended to 407 00:29:01,160 --> 00:29:04,479 Speaker 1: drink to mind the image of washing your hands. And 408 00:29:04,560 --> 00:29:10,640 Speaker 1: there is a contrast between the biasing and the certain hygiene. 409 00:29:11,040 --> 00:29:14,880 Speaker 1: The bias thing is like medication or like vaccination. It's 410 00:29:14,920 --> 00:29:19,080 Speaker 1: specific to a particular disease. When you wash your hands, 411 00:29:19,160 --> 00:29:22,719 Speaker 1: you don't know what germs you're killing, and if you're successful, 412 00:29:22,960 --> 00:29:28,320 Speaker 1: you'll never know. So the certain hygiene is oriented to 413 00:29:28,600 --> 00:29:34,200 Speaker 1: improving decision making an avoiding errors, specifically avoiding noise, but 414 00:29:34,320 --> 00:29:41,200 Speaker 1: incidentally also avoiding bias without knowing precisely what biases you're 415 00:29:41,200 --> 00:29:44,720 Speaker 1: trying to control. And we have a variety of procedures 416 00:29:44,760 --> 00:29:47,719 Speaker 1: that we think of as the certain HyG Give us 417 00:29:47,720 --> 00:29:50,560 Speaker 1: a few examples. What what are some of the procedures. Well, 418 00:29:50,600 --> 00:29:53,440 Speaker 1: I'll give you an example that has to do with 419 00:29:53,520 --> 00:29:57,560 Speaker 1: the certain making. So suppose you are making a decision 420 00:29:58,080 --> 00:30:01,960 Speaker 1: and so step you one will tell you is you 421 00:30:02,040 --> 00:30:05,520 Speaker 1: have to consider your options and have the best possible 422 00:30:05,560 --> 00:30:09,640 Speaker 1: set of options. But now you come to evaluate options, 423 00:30:09,680 --> 00:30:13,560 Speaker 1: how do you do that? And here actually our advice, 424 00:30:14,520 --> 00:30:18,120 Speaker 1: we have a slogan we say options are like candidates. 425 00:30:18,960 --> 00:30:22,600 Speaker 1: You should think of options in the same way that 426 00:30:22,800 --> 00:30:29,440 Speaker 1: organizations are in our advised to operate when they hire candidates. 427 00:30:29,560 --> 00:30:33,600 Speaker 1: And we were talking about that earlier structure, the thinking, 428 00:30:34,120 --> 00:30:39,160 Speaker 1: break up the each option, look at the various aspects 429 00:30:39,160 --> 00:30:43,080 Speaker 1: of it, make assess these aspects in the fact based way, 430 00:30:43,560 --> 00:30:47,600 Speaker 1: to the equivalent of interviewing somebody about different aspects with 431 00:30:48,680 --> 00:30:54,719 Speaker 1: her character or her experience, and then create a profile 432 00:30:55,440 --> 00:30:58,720 Speaker 1: of all the information you have about that option, and 433 00:30:58,960 --> 00:31:04,320 Speaker 1: only then invoke intuition. That there is a key principle 434 00:31:04,440 --> 00:31:09,880 Speaker 1: of decision hygiene is not to avoid intuition altogether, but 435 00:31:09,960 --> 00:31:14,600 Speaker 1: to delay it, because intuition is way more effective if 436 00:31:14,640 --> 00:31:18,320 Speaker 1: it is preceded by a period in which you accumulate 437 00:31:18,400 --> 00:31:22,640 Speaker 1: information systematically. So that's an example. I have many others, 438 00:31:22,640 --> 00:31:24,680 Speaker 1: but this is when and there were quite a few 439 00:31:24,720 --> 00:31:27,560 Speaker 1: in the book. There were some things that really surprised 440 00:31:27,560 --> 00:31:32,640 Speaker 1: me about that decision making process. How people's moods affect 441 00:31:32,640 --> 00:31:36,760 Speaker 1: their decisions, even the weather affects decision making. We are 442 00:31:36,840 --> 00:31:40,920 Speaker 1: essentially different people at different times. Oh, yes, that is 443 00:31:41,320 --> 00:31:44,880 Speaker 1: there are different sources of noise that we talk about. 444 00:31:45,320 --> 00:31:48,280 Speaker 1: So there are three of them. Are truly that one 445 00:31:48,360 --> 00:31:51,920 Speaker 1: of them is what we call within person noise, and 446 00:31:52,000 --> 00:31:57,160 Speaker 1: that is that the individual is indeed making different judgments 447 00:31:57,240 --> 00:32:01,960 Speaker 1: depending on circumstances that irrelevant. So it's true. There is 448 00:32:02,040 --> 00:32:08,000 Speaker 1: evidence that mood really affects the way that people think. Uh, 449 00:32:08,760 --> 00:32:11,040 Speaker 1: people tend to be more creative when they're in a 450 00:32:11,080 --> 00:32:14,560 Speaker 1: good mood, but they tend to be also more gullible 451 00:32:15,080 --> 00:32:19,000 Speaker 1: and they are more critical when they're in a bad mood. 452 00:32:19,400 --> 00:32:23,000 Speaker 1: So mood affects the way we think, and it also 453 00:32:23,040 --> 00:32:26,280 Speaker 1: affects we're more prone to see good things when we're 454 00:32:26,280 --> 00:32:29,680 Speaker 1: in a good mood. Mood is important. There is evidence 455 00:32:29,800 --> 00:32:34,440 Speaker 1: that judges who pass sentences on criminals are more severe 456 00:32:34,520 --> 00:32:37,840 Speaker 1: on hot days, and they are more severe if their 457 00:32:37,880 --> 00:32:43,080 Speaker 1: football team lost the game last Sunday. So there are 458 00:32:43,120 --> 00:32:48,600 Speaker 1: a lot of irrelevant events or circumstances that influence our judgment. 459 00:32:48,720 --> 00:32:52,040 Speaker 1: This is one of the three major sources of judgment. 460 00:32:52,320 --> 00:32:54,040 Speaker 1: Let's get to the other two. What are the other 461 00:32:54,080 --> 00:32:58,400 Speaker 1: two sources well, and one other which is easy to 462 00:32:58,440 --> 00:33:01,760 Speaker 1: think about. It's very easy to think about it. In 463 00:33:01,880 --> 00:33:05,920 Speaker 1: terms of judges. Some judges are more severe than others, 464 00:33:05,960 --> 00:33:10,040 Speaker 1: so their sentences on average are more severe than the 465 00:33:10,160 --> 00:33:15,800 Speaker 1: sentences of other judges. That's one aspect, and the same 466 00:33:15,880 --> 00:33:21,600 Speaker 1: as to by the web underwriters. Some underwriters write large 467 00:33:21,680 --> 00:33:27,200 Speaker 1: premiums on average, and other underwriters write small premiums on average, 468 00:33:27,480 --> 00:33:30,959 Speaker 1: So there are differences of that kind. But it turns 469 00:33:30,960 --> 00:33:34,200 Speaker 1: out that the biggest source of noise, and that came 470 00:33:34,240 --> 00:33:37,480 Speaker 1: as a surprise to us. The biggest source of noise 471 00:33:37,520 --> 00:33:40,120 Speaker 1: is that people really don't see the world in the 472 00:33:40,200 --> 00:33:45,000 Speaker 1: same way, so that different judges have different tastes in 473 00:33:45,160 --> 00:33:51,480 Speaker 1: crimes and some tastes in criminals, so they somebody may 474 00:33:51,520 --> 00:33:57,320 Speaker 1: be particularly severe about repeat offenders that somebody else might 475 00:33:57,400 --> 00:34:03,320 Speaker 1: be with extremely lenient, say about white color crime, but 476 00:34:03,760 --> 00:34:07,560 Speaker 1: really upset by violence. And it turns out that there 477 00:34:07,720 --> 00:34:13,160 Speaker 1: is we call that a pattern noise. That is, each judge, 478 00:34:13,440 --> 00:34:17,160 Speaker 1: each individual has a pattern of judgments which are this 479 00:34:17,560 --> 00:34:21,200 Speaker 1: is different from the pattern of judgments of other people, 480 00:34:22,120 --> 00:34:26,040 Speaker 1: and that is the major source of noise. And people 481 00:34:26,040 --> 00:34:28,960 Speaker 1: who are consistent in that way. So for example, suppose 482 00:34:29,000 --> 00:34:32,440 Speaker 1: you're a judge and somebody reminds you of your daughter, 483 00:34:32,840 --> 00:34:35,680 Speaker 1: whether that makes you more lenient or more severe, probably 484 00:34:35,719 --> 00:34:39,319 Speaker 1: more lenient. Now on another day, that same person would 485 00:34:39,320 --> 00:34:42,360 Speaker 1: also remind you of your daughter. So this is not 486 00:34:42,680 --> 00:34:46,839 Speaker 1: noisy within the individual. This is a characteristic of the individual, 487 00:34:47,160 --> 00:34:50,560 Speaker 1: but no other judge shares it. And it turns out 488 00:34:51,000 --> 00:34:56,839 Speaker 1: that this highly case specific distances in attitudes that are 489 00:34:56,960 --> 00:35:02,239 Speaker 1: difficult to pin down. They are noise. Judges have personalities 490 00:35:02,440 --> 00:35:07,000 Speaker 1: and judgments differ as much as personalities too. And then 491 00:35:07,080 --> 00:35:09,759 Speaker 1: what is the third source of noise that you identified 492 00:35:09,760 --> 00:35:14,919 Speaker 1: in the post? But those are the three are differences 493 00:35:14,960 --> 00:35:19,640 Speaker 1: in average level for judge's severity, differences in taste what 494 00:35:19,840 --> 00:35:25,200 Speaker 1: we call pattern noise, and within subjects, within person variability, 495 00:35:25,920 --> 00:35:29,840 Speaker 1: we call that occasion noise because on different occasions you 496 00:35:29,960 --> 00:35:33,400 Speaker 1: make different judgements and it's to some of these three 497 00:35:33,600 --> 00:35:37,480 Speaker 1: sources of noise that that creates. That's the noise that 498 00:35:37,560 --> 00:35:41,960 Speaker 1: we observe in the system. All three operate on any 499 00:35:41,960 --> 00:35:46,400 Speaker 1: particular judgment. So I'm gonna ask the question I was 500 00:35:46,440 --> 00:35:49,480 Speaker 1: thinking about a little differently based on what you just said, 501 00:35:50,320 --> 00:35:55,720 Speaker 1: what fields seem to manage reducing noise better than others. 502 00:35:56,560 --> 00:36:00,440 Speaker 1: And are there any fields that are especially susceptible the noise. 503 00:36:01,560 --> 00:36:04,000 Speaker 1: That's a very good question to which I do not 504 00:36:04,160 --> 00:36:09,800 Speaker 1: have a very good answer, because in our work we 505 00:36:09,800 --> 00:36:14,919 Speaker 1: we found noise wherever we looked for it. Indeed, our 506 00:36:15,360 --> 00:36:19,440 Speaker 1: summary conclusion is wherever there is judgment, there is noise, 507 00:36:19,600 --> 00:36:22,239 Speaker 1: and more of it than you think. You know this 508 00:36:22,440 --> 00:36:25,320 Speaker 1: is this has been our conclusion. So we haven't found 509 00:36:25,680 --> 00:36:30,759 Speaker 1: places that control noise very efficiently. The only way, by 510 00:36:30,800 --> 00:36:33,759 Speaker 1: the way to get rid of noise, and that's really 511 00:36:33,840 --> 00:36:38,480 Speaker 1: quite important is average judgments. Take multiple judgments of the 512 00:36:38,600 --> 00:36:43,879 Speaker 1: case and average them, and this mechanically eliminates noise if 513 00:36:43,920 --> 00:36:48,520 Speaker 1: you have enough judgments the average. It may be biased 514 00:36:48,880 --> 00:36:53,840 Speaker 1: because averaging there's nothing to reduce bias, but it eliminates noise. 515 00:36:54,200 --> 00:36:58,440 Speaker 1: So that's a pure far way of eliminating noise. Is 516 00:36:58,560 --> 00:37:02,719 Speaker 1: averaging multiple case very interesting. Let's let me throw a 517 00:37:02,800 --> 00:37:07,480 Speaker 1: curveball at you. If you were designing a system to 518 00:37:07,800 --> 00:37:12,200 Speaker 1: introduce noise to short circuit human judgment, what would you 519 00:37:12,360 --> 00:37:19,160 Speaker 1: create to make judgment less effective noisier. I don't think 520 00:37:19,160 --> 00:37:21,600 Speaker 1: I would do things very differently from the way that 521 00:37:21,640 --> 00:37:25,239 Speaker 1: they have done in many institutions. Now I would I 522 00:37:25,280 --> 00:37:31,000 Speaker 1: would let people make individual judgments without feedback. That's that's 523 00:37:31,040 --> 00:37:35,719 Speaker 1: all that's needed, make their individual decisions without feedback, which 524 00:37:35,760 --> 00:37:39,400 Speaker 1: is a situation that's very common, and that will create 525 00:37:39,440 --> 00:37:44,560 Speaker 1: a lot of noise eventually. And noise is reduced by feedback. 526 00:37:44,719 --> 00:37:49,560 Speaker 1: Sometimes it's the feedback of other people. So case conferences 527 00:37:49,640 --> 00:37:54,719 Speaker 1: can be arranged to some extent control noise. But you know, 528 00:37:54,840 --> 00:37:57,760 Speaker 1: you you don't have to try very hard to create 529 00:37:57,800 --> 00:38:01,840 Speaker 1: a lot of noise. I think the existing organizations do 530 00:38:02,040 --> 00:38:04,959 Speaker 1: very little to control noise. So let's talk a little 531 00:38:04,960 --> 00:38:08,880 Speaker 1: bit about ways to control noise. And you describe a 532 00:38:09,000 --> 00:38:13,640 Speaker 1: difference between rules and standards. Tell us about that. Well, 533 00:38:14,040 --> 00:38:19,200 Speaker 1: Standards is a way of when you say, for example, 534 00:38:19,360 --> 00:38:23,080 Speaker 1: that your obscenity is something that you recognize, so there 535 00:38:23,160 --> 00:38:25,960 Speaker 1: is a standard to avoid obscenity that you do not 536 00:38:26,120 --> 00:38:30,560 Speaker 1: define it. That's a standard. A rule is more precise 537 00:38:30,640 --> 00:38:34,080 Speaker 1: than that, and it does you specifically what you have 538 00:38:34,239 --> 00:38:39,560 Speaker 1: to do, and rules, if followed, they're like computations. The 539 00:38:39,640 --> 00:38:43,120 Speaker 1: computation is a is a rule, and rules tend to 540 00:38:43,160 --> 00:38:48,680 Speaker 1: eliminate noise. Standards sometimes reduce noise, but standards do not 541 00:38:48,800 --> 00:38:53,319 Speaker 1: eliminates because they so the seven words you can say 542 00:38:53,360 --> 00:38:57,279 Speaker 1: on television is a rule, but pornography, I know when 543 00:38:57,320 --> 00:38:59,760 Speaker 1: I see it is a standard. Is that the difference 544 00:39:00,040 --> 00:39:04,399 Speaker 1: nicely pre firstly quite quite interesting. So so, given all 545 00:39:04,440 --> 00:39:08,239 Speaker 1: of the work you've done over the years, all of 546 00:39:08,280 --> 00:39:15,200 Speaker 1: your research, you seem to have continually identified flaws incognition, 547 00:39:15,400 --> 00:39:19,719 Speaker 1: flaws in human judgment. Has this affected the way you 548 00:39:19,920 --> 00:39:23,760 Speaker 1: view other people? Do you? Do you turn around and say, wow, 549 00:39:23,840 --> 00:39:27,960 Speaker 1: these this species is a terrible decision making apparatus or 550 00:39:28,080 --> 00:39:33,440 Speaker 1: is it something less comprehensive than that? No, I've actually 551 00:39:33,520 --> 00:39:38,720 Speaker 1: for my career, I've been interested in intuition and intuitive thinking, 552 00:39:39,200 --> 00:39:42,000 Speaker 1: and I've been interested in that's A lecturer used to 553 00:39:42,040 --> 00:39:49,040 Speaker 1: give many years intuitions marvels and flaws, because intuitions is marvelous. 554 00:39:49,040 --> 00:39:53,280 Speaker 1: Intuition is marvelous, but it's also flawed. And it's true 555 00:39:53,800 --> 00:39:56,319 Speaker 1: that I have found it more interesting to study the 556 00:39:56,400 --> 00:40:01,120 Speaker 1: flaws of intuition than it's marvels. And there a lot 557 00:40:01,200 --> 00:40:05,080 Speaker 1: to do to correct the flaws of intuition. But to 558 00:40:05,200 --> 00:40:07,560 Speaker 1: say that this has turned me into a pessimist, or 559 00:40:07,680 --> 00:40:11,000 Speaker 1: that they dislike people because their minds are flawed I 560 00:40:11,040 --> 00:40:14,760 Speaker 1: think the minds are pretty marvelous, but they are certainly 561 00:40:14,800 --> 00:40:18,000 Speaker 1: far from perfect, right, So, so you're focusing on the 562 00:40:18,080 --> 00:40:21,720 Speaker 1: small bits that we get wrong. But overall we managed 563 00:40:21,760 --> 00:40:25,760 Speaker 1: to navigate through life pretty effectively. Well, we certainly managed 564 00:40:25,800 --> 00:40:29,839 Speaker 1: to navigate through life. And you know it's it would 565 00:40:29,840 --> 00:40:34,520 Speaker 1: be absurd to focus on the floors of the human 566 00:40:34,560 --> 00:40:38,759 Speaker 1: beings when you can see what they're capable of. On 567 00:40:38,800 --> 00:40:41,160 Speaker 1: the other hand, if you want to do things better, 568 00:40:41,640 --> 00:40:44,400 Speaker 1: then you'd better focus on the floors rather than on 569 00:40:44,520 --> 00:40:47,160 Speaker 1: what is going well. You know, one of the things 570 00:40:47,280 --> 00:40:51,920 Speaker 1: you said when we spoke last about Thinking Fast and Slow, 571 00:40:52,160 --> 00:40:57,280 Speaker 1: I asked you about your own investing process, and you said, 572 00:40:57,360 --> 00:41:02,160 Speaker 1: despite knowing everything that you know about you human decision making, 573 00:41:03,239 --> 00:41:07,719 Speaker 1: you still catch yourself making the same sort of mistakes 574 00:41:07,760 --> 00:41:11,360 Speaker 1: that everybody makes. Is that still the case? Do you 575 00:41:11,400 --> 00:41:14,400 Speaker 1: still feel that way? Oh? Yes, I mean, you know, 576 00:41:14,520 --> 00:41:18,400 Speaker 1: I've been at it for more than sixty years, and 577 00:41:19,840 --> 00:41:23,960 Speaker 1: I'm really not better than I was. In general, my 578 00:41:24,120 --> 00:41:27,240 Speaker 1: thinking has been And it was true when I wrote 579 00:41:27,280 --> 00:41:30,600 Speaker 1: Thinking Fast and Slow to just focused on the individuals, 580 00:41:31,280 --> 00:41:36,040 Speaker 1: that the hope of improving thinking is in organizations, because 581 00:41:36,200 --> 00:41:42,040 Speaker 1: organizations think slowly and they have procedures, and it's by 582 00:41:42,080 --> 00:41:47,680 Speaker 1: imposing procedures, by adopting procedures, that you can improve things. 583 00:41:47,800 --> 00:41:51,719 Speaker 1: And in the case of noise, we have a procedure 584 00:41:51,800 --> 00:41:56,960 Speaker 1: that we recommend to get started, and that's measured knowledge. 585 00:41:57,520 --> 00:42:00,840 Speaker 1: If you're in an organization where you have multiple people 586 00:42:01,280 --> 00:42:05,560 Speaker 1: making the same judgment and no very good feedback, conduct 587 00:42:05,640 --> 00:42:08,520 Speaker 1: what we call the noise audit, give them the same 588 00:42:08,520 --> 00:42:12,120 Speaker 1: problem and look at their solution. We predict that you'll 589 00:42:12,160 --> 00:42:17,080 Speaker 1: find more noise then than you think you will. That's 590 00:42:17,080 --> 00:42:22,960 Speaker 1: that's our prediction, and that's some that's a recommendation to organizations. 591 00:42:23,239 --> 00:42:26,000 Speaker 1: It's not something that you can recommend to an individual. 592 00:42:26,880 --> 00:42:30,080 Speaker 1: Quite interesting, I have to ask you before we get 593 00:42:30,080 --> 00:42:34,000 Speaker 1: to our favorite questions, what's the next project, what's the 594 00:42:34,040 --> 00:42:39,040 Speaker 1: next book look like? What is tickling your curiosity these days? Well, 595 00:42:39,800 --> 00:42:44,040 Speaker 1: that's actually back to a topic that I was working 596 00:42:44,080 --> 00:42:49,360 Speaker 1: on but years ago, and I have almost by accident 597 00:42:49,480 --> 00:42:53,640 Speaker 1: and back studying well being, and I'm involved in several 598 00:42:53,960 --> 00:42:57,440 Speaker 1: research projects. None of them is as big or ambitious 599 00:42:57,480 --> 00:43:00,839 Speaker 1: as Noise was, or thinking fast and slow, but all 600 00:43:00,880 --> 00:43:05,319 Speaker 1: of them are quite interesting. So I'm not bored. I 601 00:43:05,360 --> 00:43:07,560 Speaker 1: can't picture you board because you always seem to have 602 00:43:07,600 --> 00:43:11,800 Speaker 1: a lot of different things going on. Let me ask 603 00:43:11,880 --> 00:43:15,200 Speaker 1: my favorite questions that I ask all of our guests, 604 00:43:15,840 --> 00:43:18,600 Speaker 1: and let's start with, what are you doing to stay 605 00:43:18,760 --> 00:43:23,239 Speaker 1: entertained during this pandemic lockdown? In addition to working on 606 00:43:23,280 --> 00:43:25,880 Speaker 1: the book? What are you streaming? What are you watching 607 00:43:25,920 --> 00:43:30,040 Speaker 1: on Netflix of anything? Oh, I've been watching several series, 608 00:43:30,400 --> 00:43:35,640 Speaker 1: several very good series. Let's see the last ones. There 609 00:43:35,719 --> 00:43:39,479 Speaker 1: is a political series on Netflix, Le Bon Wi, which 610 00:43:39,560 --> 00:43:43,319 Speaker 1: is a French political thriller that is very good. There 611 00:43:43,400 --> 00:43:46,840 Speaker 1: is a Danish political series Borgan, that is very good. 612 00:43:47,320 --> 00:43:52,000 Speaker 1: I am now watching Babylon Berlin about Berlin in the 613 00:43:52,080 --> 00:43:58,799 Speaker 1: nineteen twenties, which is excellent. And so I do mind 614 00:43:58,840 --> 00:44:02,760 Speaker 1: watching me will I exercise? And I exercise a fair amount. 615 00:44:02,840 --> 00:44:06,640 Speaker 1: But so I've seen a lot of series since, well, 616 00:44:06,760 --> 00:44:10,520 Speaker 1: from for the last few years. So baum Noir was 617 00:44:10,600 --> 00:44:14,560 Speaker 1: the French one. What was the Danish one? Borgan? Borgan 618 00:44:14,680 --> 00:44:18,800 Speaker 1: is Bridge Actually the Danish one. Bogan is a thriller. 619 00:44:18,840 --> 00:44:23,000 Speaker 1: It's a Scandinavian swiller. There is a Danish one about 620 00:44:23,040 --> 00:44:27,040 Speaker 1: a woman prime minister, and it's not Borgan, and I 621 00:44:27,360 --> 00:44:29,360 Speaker 1: not block on its name, but it would be easy 622 00:44:29,400 --> 00:44:33,239 Speaker 1: to find, and I really recommended it, is sup all right, 623 00:44:33,280 --> 00:44:36,080 Speaker 1: I will I will check that out. So let's talk 624 00:44:36,120 --> 00:44:40,800 Speaker 1: about your early mentors who helped to shape your career. 625 00:44:41,440 --> 00:44:43,840 Speaker 1: And I guess we have to include collaborators in that 626 00:44:43,960 --> 00:44:47,359 Speaker 1: as well. Well, I mean there were There's been one 627 00:44:47,600 --> 00:44:50,640 Speaker 1: major influence on my career, and it was in Sisk. 628 00:44:52,080 --> 00:44:58,080 Speaker 1: The collaboration with him completely changed my life. And uh, 629 00:44:58,080 --> 00:45:02,000 Speaker 1: and it changed the way I do things, but and 630 00:45:02,160 --> 00:45:05,920 Speaker 1: it gave me Yeah, it changed my life and it 631 00:45:06,040 --> 00:45:09,560 Speaker 1: was the best period of my life too. Professionally. The 632 00:45:09,640 --> 00:45:13,200 Speaker 1: thing I recall from Michael Lewis is undoing project is 633 00:45:13,239 --> 00:45:16,879 Speaker 1: that people said, you guys would lock yourself into an 634 00:45:16,880 --> 00:45:19,839 Speaker 1: office or a classroom and all they would hear all 635 00:45:19,960 --> 00:45:24,319 Speaker 1: day long is peals of laughter coming from within. Is 636 00:45:24,360 --> 00:45:27,040 Speaker 1: that true? Is that an exaggeration or did you guys 637 00:45:27,280 --> 00:45:34,959 Speaker 1: know that's really not an exaggeration. Amos and I worked 638 00:45:35,320 --> 00:45:39,440 Speaker 1: very closely together for about twelve years, and we spent 639 00:45:40,719 --> 00:45:45,200 Speaker 1: many hours a day together. And he was very funny. 640 00:45:45,320 --> 00:45:48,680 Speaker 1: He had an excellent sense of humor and he loved laughing, 641 00:45:49,280 --> 00:45:52,520 Speaker 1: and in his presence I also became funny. So we 642 00:45:52,520 --> 00:45:57,400 Speaker 1: were amusing each other and the field that we studied, uh, 643 00:45:57,719 --> 00:46:01,680 Speaker 1: was was one that was ducive to luster because we 644 00:46:01,680 --> 00:46:05,360 Speaker 1: were looking for mistakes in our own thinking and to 645 00:46:05,600 --> 00:46:09,640 Speaker 1: trap ourselves or to see that you are attempted to 646 00:46:09,800 --> 00:46:13,760 Speaker 1: make a stupid error. That is quite funny. And that's 647 00:46:13,880 --> 00:46:16,840 Speaker 1: the game that we engaged in in studying judgment and 648 00:46:16,920 --> 00:46:20,719 Speaker 1: in studying decision making, was looking for errors in our 649 00:46:20,840 --> 00:46:25,560 Speaker 1: own thinking. And that was very amusing, I can imagine. 650 00:46:25,840 --> 00:46:28,080 Speaker 1: So let's talk about books. What are some of your 651 00:46:28,080 --> 00:46:31,600 Speaker 1: all time favorites and what are you reading right now? Well, 652 00:46:31,880 --> 00:46:35,600 Speaker 1: I would say my all time favorites of recent years 653 00:46:35,640 --> 00:46:42,040 Speaker 1: with Sapiens. I think it's many people's favorite book by Valli. 654 00:46:43,600 --> 00:46:46,520 Speaker 1: I read it twice, which is something that I really do. 655 00:46:47,160 --> 00:46:51,200 Speaker 1: And right now while I'm reading the new edition of Nudge, 656 00:46:52,040 --> 00:46:55,120 Speaker 1: which is coming out I think in August, and it's 657 00:46:55,160 --> 00:46:59,200 Speaker 1: called Nudge. The final edition by Dick Taylor and Catherine 658 00:46:59,280 --> 00:47:04,759 Speaker 1: Steam was in and it's quite different from the original 659 00:47:04,880 --> 00:47:08,319 Speaker 1: note which appeared I think indo peplem an eight uh 660 00:47:08,760 --> 00:47:12,120 Speaker 1: and it's what but it it had the same characteristic 661 00:47:12,160 --> 00:47:15,640 Speaker 1: that not said. It's wise and it's funny, right, Dick said, 662 00:47:15,680 --> 00:47:18,840 Speaker 1: it's about two thirds new and I think that's August 663 00:47:18,920 --> 00:47:23,680 Speaker 1: four that comes out the other's right. I happened to 664 00:47:23,680 --> 00:47:27,000 Speaker 1: be reading that right now, August three. I'm looking at 665 00:47:27,040 --> 00:47:30,920 Speaker 1: a message from him. He's um. He's a very amusing 666 00:47:31,000 --> 00:47:34,680 Speaker 1: person to begin with. And if if you're telling me 667 00:47:34,800 --> 00:47:37,680 Speaker 1: the book is funny, then I am really looking forward 668 00:47:39,080 --> 00:47:42,720 Speaker 1: the book is. You know, he just he can't help himself. 669 00:47:42,760 --> 00:47:45,879 Speaker 1: He's funny all the time. He's my best friend, my 670 00:47:45,880 --> 00:47:49,319 Speaker 1: best living friend. Let me ask you this question. If 671 00:47:49,400 --> 00:47:54,160 Speaker 1: a recent college graduate asked you for some advice, if 672 00:47:54,160 --> 00:47:58,600 Speaker 1: he was interested in a career in either psychology or 673 00:47:58,719 --> 00:48:04,240 Speaker 1: behavioral finance, what sort of advice might you give him? Well, 674 00:48:04,360 --> 00:48:07,640 Speaker 1: you know, I tend to refrain from advice because I 675 00:48:07,680 --> 00:48:10,360 Speaker 1: don't believe I have a crystal ball into the future. 676 00:48:11,400 --> 00:48:13,839 Speaker 1: I can tell you what I would have been doing 677 00:48:13,880 --> 00:48:17,879 Speaker 1: if I was starting today. The fields that are very 678 00:48:17,960 --> 00:48:24,880 Speaker 1: exciting from my perspective are neuroscience, including neuroeconomics, which is 679 00:48:25,080 --> 00:48:31,239 Speaker 1: the neuroscience of decision making, and artificial intelligence. I mean, 680 00:48:31,400 --> 00:48:37,760 Speaker 1: in those two areas right now, there are extraordinary developments, 681 00:48:37,920 --> 00:48:41,080 Speaker 1: very exciting. And so when you see that and they're 682 00:48:41,080 --> 00:48:45,080 Speaker 1: attracting massive talent both areas so you know that for 683 00:48:45,160 --> 00:48:48,479 Speaker 1: the next decade or so they'll be cooking A lot 684 00:48:48,640 --> 00:48:51,440 Speaker 1: is going to happen. And what happens after that, I 685 00:48:51,480 --> 00:48:56,719 Speaker 1: have no idea. And in our final question, what do 686 00:48:56,800 --> 00:49:01,640 Speaker 1: you know about the world of psychology, g behavioral finance 687 00:49:01,719 --> 00:49:05,960 Speaker 1: economics today that you wish you knew fifty or so 688 00:49:06,080 --> 00:49:09,040 Speaker 1: years ago when you were first getting started? Oh? Well, 689 00:49:09,440 --> 00:49:13,400 Speaker 1: so much has been learned. I you know, if I 690 00:49:14,239 --> 00:49:17,040 Speaker 1: I can't say that I wish I had known earlier. 691 00:49:18,040 --> 00:49:21,560 Speaker 1: Has been so much fun to find out over the years, 692 00:49:21,680 --> 00:49:24,400 Speaker 1: both in my work and in the work of others. 693 00:49:24,440 --> 00:49:27,120 Speaker 1: So I can't think of thinking that would have made 694 00:49:27,160 --> 00:49:30,840 Speaker 1: me act de simply. But all I can say you 695 00:49:31,200 --> 00:49:35,560 Speaker 1: to you is, oh, yes, things have really changed and 696 00:49:35,760 --> 00:49:40,799 Speaker 1: so have been in that field. Huh quite fascinating. Thank you, 697 00:49:40,880 --> 00:49:43,319 Speaker 1: Danny for being so generous with your time. We have 698 00:49:43,480 --> 00:49:47,520 Speaker 1: been speaking with Danny Kahneman, whose new book Noise, A 699 00:49:47,600 --> 00:49:52,120 Speaker 1: Flawing Human Judgment, was co authored with Olivier Simone and 700 00:49:52,239 --> 00:49:56,200 Speaker 1: Cass Sunstein. If you enjoy this conversation, check out any 701 00:49:56,239 --> 00:49:59,439 Speaker 1: of our previous four hundred such interviews. You can find 702 00:49:59,520 --> 00:50:03,960 Speaker 1: those at iTunes, Spotify, Google, Bloomberg dot Com, wherever you 703 00:50:04,120 --> 00:50:08,360 Speaker 1: get your podcast each week. We love your comments, feedback 704 00:50:08,400 --> 00:50:11,920 Speaker 1: and suggestions. Write to us at m IB podcast at 705 00:50:11,960 --> 00:50:15,160 Speaker 1: Bloomberg dot net. You can sign up for my Daily 706 00:50:15,239 --> 00:50:18,840 Speaker 1: Reads at Ridholts dot com. Check out my weekly column 707 00:50:18,840 --> 00:50:22,240 Speaker 1: on Bloomberg dot com slash Opinion. Follow me on Twitter 708 00:50:22,320 --> 00:50:25,160 Speaker 1: at Ridholts. I would be remiss if I did not 709 00:50:25,280 --> 00:50:28,719 Speaker 1: think the crack staff that helps put together this conversation 710 00:50:28,840 --> 00:50:33,080 Speaker 1: each week. Tim Harrow is my audio engineer. Alatico val 711 00:50:33,120 --> 00:50:37,839 Speaker 1: Bron is my project manager. Michael Boyle is my producer. 712 00:50:38,400 --> 00:50:41,960 Speaker 1: Michael Batnick is my head of research. I'm Barry Riholts. 713 00:50:42,239 --> 00:50:45,960 Speaker 1: You've been listening to Master's Business on Bloomberg Radio.