1
00:00:02,240 --> 00:00:06,840
Speaker 1: This is Masters in Business with Barry Ridholts on Bloomberg Radio.

2
00:00:10,520 --> 00:00:13,960
Speaker 1: This week on the podcast, I have a fascinating guest,

3
00:00:14,120 --> 00:00:20,479
Speaker 1: And if you're at all interested in statistical analysis of sports, UH,

4
00:00:20,760 --> 00:00:27,760
Speaker 1: behavioral finance data analysis, understanding um streakiness, understanding the money

5
00:00:27,840 --> 00:00:32,480
Speaker 1: hole problem, and then extrapolating that towards things like, UH,

6
00:00:32,520 --> 00:00:35,320
Speaker 1: the hot hand in basketball, you're gonna find this to

7
00:00:35,360 --> 00:00:40,600
Speaker 1: be absolutely fascinating. UH. Joshua Benjamin Miller comes from California,

8
00:00:40,680 --> 00:00:43,320
Speaker 1: where he basically racked up all the degrees he could

9
00:00:44,240 --> 00:00:46,879
Speaker 1: at some of the u C schools before getting his

10
00:00:47,200 --> 00:00:51,559
Speaker 1: PhD at in economics UH in Minnesota. Josh and his

11
00:00:51,600 --> 00:00:54,880
Speaker 1: co author have taken apart some of the more interesting

12
00:00:55,600 --> 00:01:01,240
Speaker 1: statistical assumptions made UH in the original hand study with

13
00:01:01,360 --> 00:01:05,280
Speaker 1: Tom Gilovich and Amos Tversky UH, and they found something

14
00:01:05,840 --> 00:01:10,400
Speaker 1: really unusual by looking at the data from a slightly

15
00:01:10,440 --> 00:01:14,479
Speaker 1: different perspective. And I approached their paper with tremendous amount

16
00:01:14,480 --> 00:01:18,920
Speaker 1: of skepticism. I thought the randomness of the hot hands

17
00:01:19,800 --> 00:01:24,480
Speaker 1: was a fairly well proven study that Tversky and Gilovich did,

18
00:01:25,000 --> 00:01:26,840
Speaker 1: But when you look at the data and you look

19
00:01:26,840 --> 00:01:30,080
Speaker 1: at how they analyzed it, it's hard not to reach

20
00:01:30,120 --> 00:01:33,280
Speaker 1: the conclusion that there is some sort of a hot hand.

21
00:01:33,360 --> 00:01:38,960
Speaker 1: It's quite sophisticated mathematics, but Josh does a very nice

22
00:01:39,080 --> 00:01:45,520
Speaker 1: job reducing it to some very easily understandable UM. Probability,

23
00:01:45,560 --> 00:01:49,320
Speaker 1: we don't we don't know math is required. Um, you

24
00:01:49,440 --> 00:01:51,160
Speaker 1: just have to know the difference between a head or

25
00:01:51,160 --> 00:01:53,440
Speaker 1: a tail when you're flipping a coin. If you're at

26
00:01:53,480 --> 00:01:58,760
Speaker 1: all interested in anything probability, sports related statistical, you're gonna

27
00:01:58,760 --> 00:02:02,360
Speaker 1: find this to be a fascinating, lee wonky and tremendously

28
00:02:02,400 --> 00:02:07,280
Speaker 1: interesting conversation. So, with no further ado, my conversation with

29
00:02:07,360 --> 00:02:17,160
Speaker 1: the economists and statistician Josh Miller. My special guest today

30
00:02:17,360 --> 00:02:21,760
Speaker 1: is Joshua Benjamin Miller UH. He is the co author,

31
00:02:21,880 --> 00:02:27,040
Speaker 1: along with Adam Sanjorro, of a fascinating paper that puts

32
00:02:27,200 --> 00:02:31,799
Speaker 1: challenge to the myth of the myth of the hot hands. Uh.

33
00:02:31,840 --> 00:02:34,360
Speaker 1: He comes to us with a BA in economics and

34
00:02:34,400 --> 00:02:38,720
Speaker 1: an m a UH in mathewmatical statistics from UC Santa Barbara.

35
00:02:38,760 --> 00:02:42,600
Speaker 1: He has his PhD from University of Minnesota, and he

36
00:02:42,720 --> 00:02:46,200
Speaker 1: is currently a professor in the economics department at the

37
00:02:46,320 --> 00:02:51,320
Speaker 1: University of Alacante. In Spain, where he focuses his research

38
00:02:51,440 --> 00:02:57,120
Speaker 1: on behavioral economics, judgment and decision making, game theory, and

39
00:02:57,320 --> 00:03:02,840
Speaker 1: statistical and experimental methods. Josh Miller, Welcome to Bloomberg. Thanks

40
00:03:02,880 --> 00:03:06,040
Speaker 1: for having me, Barry, So a little background. We kind

41
00:03:06,040 --> 00:03:10,800
Speaker 1: of met after I interviewed Thomas Gilovich, who I was

42
00:03:10,880 --> 00:03:13,600
Speaker 1: mostly interested in due to all of his work on

43
00:03:13,639 --> 00:03:17,920
Speaker 1: behavioral finance, but he also co authored a fascinating paper,

44
00:03:18,639 --> 00:03:22,440
Speaker 1: uh that basically pointed out the hot hands. You co

45
00:03:22,520 --> 00:03:24,920
Speaker 1: authored that with Amos Tversky, by the way, that the

46
00:03:24,960 --> 00:03:26,919
Speaker 1: hot hand was really a myth and it was just

47
00:03:27,080 --> 00:03:30,000
Speaker 1: we were all being fooled by randomness. How did that

48
00:03:30,040 --> 00:03:34,040
Speaker 1: paper come to your attention? And what what fascinated you

49
00:03:34,080 --> 00:03:37,520
Speaker 1: buy it? So that paper came to the attention of

50
00:03:37,520 --> 00:03:39,880
Speaker 1: my co author Adam who's also at University of Vallecante.

51
00:03:40,320 --> 00:03:43,560
Speaker 1: Pretty Much everyone who takes a behavioral economics class and

52
00:03:43,600 --> 00:03:46,200
Speaker 1: even earlier gets exposed to that paper. It's like it's

53
00:03:46,240 --> 00:03:48,640
Speaker 1: one of the you know, the prime examples of a

54
00:03:48,680 --> 00:03:51,480
Speaker 1: bias because it's such apparently power. It's part of the

55
00:03:51,560 --> 00:03:55,000
Speaker 1: canon of oh, look how easily we're all fooled exactly,

56
00:03:55,280 --> 00:03:57,640
Speaker 1: and in the in the beginning of any kind of

57
00:03:57,680 --> 00:04:00,240
Speaker 1: behavior economics class, you have to show the real world

58
00:04:00,240 --> 00:04:02,840
Speaker 1: implications first to kind of motivate students. And here is

59
00:04:02,880 --> 00:04:07,440
Speaker 1: this um one that professionals are faul victim too, and

60
00:04:07,480 --> 00:04:10,040
Speaker 1: they're so resistant to it. I mean they were shown

61
00:04:10,080 --> 00:04:11,520
Speaker 1: that this hot I mean, we haven't to find hot

62
00:04:11,560 --> 00:04:13,040
Speaker 1: hand yet, but you know, the hot hand is this

63
00:04:13,080 --> 00:04:15,880
Speaker 1: idea that you're in the zone. You know that success

64
00:04:15,920 --> 00:04:19,040
Speaker 1: breeds success. And if you look at basketball players, UM

65
00:04:19,160 --> 00:04:21,840
Speaker 1: and coaches, they all believe in this thing. And so

66
00:04:21,880 --> 00:04:25,880
Speaker 1: when they discovered that there was no pattern there and

67
00:04:25,920 --> 00:04:29,520
Speaker 1: they came and revealed revealed the output of their research,

68
00:04:29,760 --> 00:04:32,680
Speaker 1: the professionals were it was difficult to convince them. Oh,

69
00:04:32,680 --> 00:04:36,200
Speaker 1: there was tremendous pushback. There's a famous quote you that's

70
00:04:36,240 --> 00:04:40,640
Speaker 1: been referenced was red or back up at the Boston Celtics.

71
00:04:40,960 --> 00:04:43,240
Speaker 1: I don't I don't care what this professor says. So

72
00:04:43,279 --> 00:04:45,279
Speaker 1: they do a study. Who cares? Right? Yeah? So, I

73
00:04:45,279 --> 00:04:48,400
Speaker 1: mean the stubbornness that that came out of the practitioners

74
00:04:48,480 --> 00:04:51,200
Speaker 1: was was really dramatic because typically you can convince someone

75
00:04:51,279 --> 00:04:53,640
Speaker 1: that is motivated to get things right if you can

76
00:04:53,839 --> 00:04:56,920
Speaker 1: demonstrate that they'll benefit from it, Um, and they just

77
00:04:57,080 --> 00:04:59,960
Speaker 1: discounted it. And so there's this famous quote from Amis Diversky,

78
00:05:00,440 --> 00:05:03,160
Speaker 1: after all the stubbornness that they encountered repeatedly, of people

79
00:05:03,200 --> 00:05:05,720
Speaker 1: just not even looking at the evidence they were showing them,

80
00:05:05,839 --> 00:05:08,520
Speaker 1: Um that he said, I've been in thousand arguments, one

81
00:05:08,600 --> 00:05:13,440
Speaker 1: them all, but convinced no one. And he was He

82
00:05:13,560 --> 00:05:17,000
Speaker 1: was very famous for being um, not only quite brilliant,

83
00:05:17,120 --> 00:05:19,880
Speaker 1: but a little hardheaded and a little aggressive when it

84
00:05:19,920 --> 00:05:24,160
Speaker 1: came to debating people. According at least according right, at

85
00:05:24,200 --> 00:05:28,200
Speaker 1: least according to Michael Lewis's book The Undoing Project. Between

86
00:05:28,320 --> 00:05:31,560
Speaker 1: Konomen and Seversky, they were two very distinct personality types.

87
00:05:32,000 --> 00:05:34,080
Speaker 1: So so let's get back to you before we were

88
00:05:34,080 --> 00:05:36,760
Speaker 1: gonna spend a lot of time on the hot hands. Um,

89
00:05:36,800 --> 00:05:41,000
Speaker 1: you're not what I would think of as a traditional economist.

90
00:05:41,120 --> 00:05:43,360
Speaker 1: What what sort of work do you focus on? Both

91
00:05:43,440 --> 00:05:46,360
Speaker 1: my ca author and I focused on individual decision making, right,

92
00:05:46,440 --> 00:05:49,400
Speaker 1: so we're looking at is it individual decision making within

93
00:05:49,400 --> 00:05:53,200
Speaker 1: a group, within an institution, or just as a lone wolf.

94
00:05:53,600 --> 00:05:56,839
Speaker 1: So so there you know, there are the psychological factors

95
00:05:56,880 --> 00:06:00,760
Speaker 1: like UM, my co author works on search and attention

96
00:06:00,839 --> 00:06:03,640
Speaker 1: and things like this UM, but there are also UM

97
00:06:03,880 --> 00:06:06,640
Speaker 1: factors of the institution, the design, like how information is

98
00:06:06,680 --> 00:06:09,800
Speaker 1: presented to you UM, and these things, while it may

99
00:06:09,800 --> 00:06:12,360
Speaker 1: be important to an individual, they also bubble up in

100
00:06:12,440 --> 00:06:15,120
Speaker 1: terms of how it affects, you know, decision making in groups,

101
00:06:15,160 --> 00:06:17,200
Speaker 1: and how it affects financial markets, and so in the

102
00:06:17,279 --> 00:06:22,359
Speaker 1: end it does impact UM policy and economic outcomes. It

103
00:06:22,400 --> 00:06:25,240
Speaker 1: has real world effects. In other words, these aren't just

104
00:06:25,400 --> 00:06:30,480
Speaker 1: ivory tower abstract discussions. There's real world application for how

105
00:06:30,560 --> 00:06:34,200
Speaker 1: decisions are making and how information is presented, so that

106
00:06:34,320 --> 00:06:37,839
Speaker 1: that's really quite interesting. UM. When you and you mentioned

107
00:06:37,839 --> 00:06:41,039
Speaker 1: one of your research areas is behavioral finance. Has all

108
00:06:41,080 --> 00:06:44,560
Speaker 1: the low hanging fruit in this space been picked or

109
00:06:44,680 --> 00:06:47,200
Speaker 1: is there still lots and lots of things to be discovered.

110
00:06:47,800 --> 00:06:51,080
Speaker 1: There's still lots of fruit, whether it's low hanging. I

111
00:06:51,240 --> 00:06:53,600
Speaker 1: think you always have to work for it, right right, UM,

112
00:06:53,680 --> 00:06:57,359
Speaker 1: So I think the way you get the fruit, you

113
00:06:57,400 --> 00:06:59,400
Speaker 1: have to think a lot about how to measure things,

114
00:06:59,760 --> 00:07:02,880
Speaker 1: how a theoretical grounding and what you're trying to get at,

115
00:07:03,360 --> 00:07:06,600
Speaker 1: And you can't just rely on existing data and existing

116
00:07:06,640 --> 00:07:08,240
Speaker 1: things that have been counted. You have to go out

117
00:07:08,240 --> 00:07:11,040
Speaker 1: and measure things yourself. A bit um and do some

118
00:07:11,080 --> 00:07:15,000
Speaker 1: work to collect that data. UM and so you know what,

119
00:07:15,520 --> 00:07:17,280
Speaker 1: Like so a lot of the modern work you'll see

120
00:07:17,400 --> 00:07:19,440
Speaker 1: is going beyond just the choices that people make, like

121
00:07:19,440 --> 00:07:21,320
Speaker 1: when you're paying them to make decisions and looking at

122
00:07:21,320 --> 00:07:25,360
Speaker 1: their choices, and you can get a lot more about

123
00:07:26,040 --> 00:07:27,960
Speaker 1: You learn a lot more about what people want and

124
00:07:27,960 --> 00:07:30,960
Speaker 1: what they believe by looking at other things like reaction times,

125
00:07:31,280 --> 00:07:33,160
Speaker 1: how they search for things, how you know what they're

126
00:07:33,160 --> 00:07:36,880
Speaker 1: paying attention to, um And in other words, you're not

127
00:07:37,000 --> 00:07:39,560
Speaker 1: just bringing in a bunch of undergrads, sticking them in

128
00:07:39,560 --> 00:07:41,920
Speaker 1: a room, giving them twenty bucks for the night, and

129
00:07:41,960 --> 00:07:43,840
Speaker 1: saying we're gonna put you through a series of things.

130
00:07:44,200 --> 00:07:46,400
Speaker 1: You're looking at very You're looking at a very different

131
00:07:46,840 --> 00:07:50,040
Speaker 1: data set that's measuring very different things. Yeah, I mean

132
00:07:50,080 --> 00:07:52,200
Speaker 1: you can improve even with the undergrads. But I think

133
00:07:52,960 --> 00:07:55,600
Speaker 1: a lot of the innovative work goes and collects unique

134
00:07:55,760 --> 00:07:58,320
Speaker 1: data from unique subjects. Like I have a friend alex Emis,

135
00:07:58,360 --> 00:08:01,320
Speaker 1: I just present this very interesting it's on in the

136
00:08:01,360 --> 00:08:03,920
Speaker 1: top topic of finance. We went and looked at institutional

137
00:08:03,920 --> 00:08:08,880
Speaker 1: investors tons of data. These are people with big positions um,

138
00:08:08,920 --> 00:08:12,880
Speaker 1: and found that there they're actually quite skilled at buying um,

139
00:08:13,280 --> 00:08:15,520
Speaker 1: buying stocks, but they aren't so skilled at selling them

140
00:08:15,920 --> 00:08:19,360
Speaker 1: UM and it seems to be distinct skills. It's very

141
00:08:19,360 --> 00:08:22,560
Speaker 1: distinct skills because it's easy to buy, that's the easy part.

142
00:08:22,880 --> 00:08:26,480
Speaker 1: Sellings where the money gets made, those are not um

143
00:08:26,800 --> 00:08:30,920
Speaker 1: equal level of difficulty things. I'm absolutely not surprised to

144
00:08:30,960 --> 00:08:35,240
Speaker 1: hear the selling demonstrates less skill than buying. Is that

145
00:08:35,360 --> 00:08:38,040
Speaker 1: basically what you know? So the finding is that they

146
00:08:38,040 --> 00:08:44,040
Speaker 1: were unloaded extreme winners um too quickly, um before they

147
00:08:44,559 --> 00:08:47,440
Speaker 1: really exploited the you know, information advantage that they had.

148
00:08:47,440 --> 00:08:49,280
Speaker 1: So they made a good job choosing it, but they

149
00:08:49,360 --> 00:08:52,719
Speaker 1: sold it too soon, a classic mistake. Let's talk a

150
00:08:52,800 --> 00:08:59,320
Speaker 1: little bit about the original hot hand paper, which, as

151
00:08:59,360 --> 00:09:05,559
Speaker 1: we discussed earlier, became cannon in the world of behavioral finance. Uh,

152
00:09:05,960 --> 00:09:08,600
Speaker 1: when did you first start to get an inkling that

153
00:09:09,160 --> 00:09:14,079
Speaker 1: the original thesis might not have been all it seemed

154
00:09:14,120 --> 00:09:18,520
Speaker 1: to be. So the original inkling was that people sometimes overreact.

155
00:09:18,559 --> 00:09:21,280
Speaker 1: It's that it's a myth, the thing doesn't exist. Why

156
00:09:21,320 --> 00:09:25,440
Speaker 1: does that generate such a strong intuitive pushback from people?

157
00:09:25,880 --> 00:09:28,200
Speaker 1: I mean, I have my thesis, I'm curious as to yours.

158
00:09:28,520 --> 00:09:32,400
Speaker 1: I think everyone has some experience in their own athletic

159
00:09:32,440 --> 00:09:35,960
Speaker 1: performance where they have moments where they're particularly locked in,

160
00:09:36,800 --> 00:09:40,120
Speaker 1: and then they realized that outside of athletic performance, you

161
00:09:40,160 --> 00:09:42,720
Speaker 1: just have these moments where you're in the you're in

162
00:09:42,720 --> 00:09:45,520
Speaker 1: the zone, that's the best word for it. Um. You're

163
00:09:45,559 --> 00:09:48,880
Speaker 1: just you're firing on all cylinders, and you would expect

164
00:09:48,960 --> 00:09:51,560
Speaker 1: that you would see that um somehow in basketball data

165
00:09:51,600 --> 00:09:54,880
Speaker 1: as well. So so my personal experience. I used to

166
00:09:54,880 --> 00:09:57,480
Speaker 1: play hoops as a kid, but as I've gotten older,

167
00:09:57,480 --> 00:10:00,320
Speaker 1: I've become a tennis player, and I know from personal

168
00:10:00,320 --> 00:10:03,560
Speaker 1: experience it takes a good twenty minutes for me to

169
00:10:03,720 --> 00:10:08,800
Speaker 1: calibrate my forehand so that I am consistently hitting the

170
00:10:08,840 --> 00:10:12,079
Speaker 1: bowl more or less towards where I want it, more

171
00:10:12,160 --> 00:10:14,439
Speaker 1: or less with the right amount of spin, more or

172
00:10:14,520 --> 00:10:17,320
Speaker 1: less with the right height. But it's not something that

173
00:10:17,320 --> 00:10:19,360
Speaker 1: I could just grab a racket and swing and oh

174
00:10:19,400 --> 00:10:22,160
Speaker 1: there it is. It takes a while, too too fast,

175
00:10:22,240 --> 00:10:25,600
Speaker 1: too much whip, keep it loosening your wrist, bringing around,

176
00:10:25,720 --> 00:10:28,120
Speaker 1: make sure you're dropping the head like I'm running through

177
00:10:28,120 --> 00:10:30,800
Speaker 1: a series of steps in my head. Hey you're too close.

178
00:10:30,840 --> 00:10:34,000
Speaker 1: Watch your footwork well, one after another and I am

179
00:10:34,040 --> 00:10:36,240
Speaker 1: now good enough to know I suck. I'm in that

180
00:10:36,679 --> 00:10:40,280
Speaker 1: Dunning Krueger drop where Oh you know, I used to

181
00:10:40,280 --> 00:10:43,040
Speaker 1: think I was good. Now I realize I'm really I'm

182
00:10:43,040 --> 00:10:46,520
Speaker 1: good enough to know how good I actually am not. Um,

183
00:10:46,559 --> 00:10:49,640
Speaker 1: But it takes a while to calibrate that. I imagine

184
00:10:49,679 --> 00:10:52,760
Speaker 1: a basketball player in the midst of a game has

185
00:10:52,800 --> 00:10:55,840
Speaker 1: to go through some sort of fine tuning of their shooting.

186
00:10:56,080 --> 00:10:59,280
Speaker 1: You can warm up all you want, um, when you're

187
00:10:59,320 --> 00:11:02,240
Speaker 1: just shooting by yourself before the game, but when people

188
00:11:02,240 --> 00:11:03,760
Speaker 1: are on you and you're running, it has to be

189
00:11:03,760 --> 00:11:07,080
Speaker 1: a very very different set of circumstances. Or am I

190
00:11:07,160 --> 00:11:11,319
Speaker 1: overstating this? Well, I mean that's that's the strongest intuition

191
00:11:11,800 --> 00:11:14,280
Speaker 1: is based on this calibration thing. I mean there's probably

192
00:11:14,280 --> 00:11:16,600
Speaker 1: other elements. Will not get to that later. Um, but

193
00:11:16,720 --> 00:11:18,839
Speaker 1: if you look at um, yeah, I mean, if you're

194
00:11:18,840 --> 00:11:20,640
Speaker 1: sitting on the bench for ten minutes and then you

195
00:11:20,960 --> 00:11:23,640
Speaker 1: come off, that's very different. I mean in the NFL

196
00:11:23,720 --> 00:11:26,079
Speaker 1: you see field goal kickers warming up go on. You

197
00:11:26,080 --> 00:11:27,560
Speaker 1: don't see that so much in the NBA. They don't

198
00:11:27,600 --> 00:11:29,920
Speaker 1: have like an extra hoop on this, that's right, So

199
00:11:30,000 --> 00:11:32,680
Speaker 1: I'd imagine, yeah, that's an important element there. What else

200
00:11:32,800 --> 00:11:37,560
Speaker 1: is so intuitively attractive about the idea of the hot hand?

201
00:11:37,760 --> 00:11:41,400
Speaker 1: Is it simply just the zone? Is it the adrenaline

202
00:11:41,400 --> 00:11:44,960
Speaker 1: and the endorphins. Why do we think that, hey, suddenly

203
00:11:45,000 --> 00:11:46,920
Speaker 1: I'm on I'm on a streak. Why do we believe

204
00:11:46,960 --> 00:11:49,320
Speaker 1: that streak is going to continue? And I'm not talking

205
00:11:49,320 --> 00:11:52,920
Speaker 1: about blackjack or roulette or games of chance. These are

206
00:11:53,360 --> 00:11:57,440
Speaker 1: really games of skill played at the highest level. So

207
00:11:57,480 --> 00:12:00,640
Speaker 1: why we believe it? Um? I would imagine sometimes when

208
00:12:00,640 --> 00:12:02,880
Speaker 1: we believe it, it's not really there. And so there

209
00:12:02,920 --> 00:12:05,120
Speaker 1: is this feeling, you know. You know, part part of

210
00:12:05,120 --> 00:12:07,680
Speaker 1: the feeling is the feedback, right you You see that

211
00:12:07,720 --> 00:12:10,160
Speaker 1: your successful gives you some confidence, you know. So it's

212
00:12:10,200 --> 00:12:14,840
Speaker 1: not always simply this zone that emerges. Um. Sometimes you

213
00:12:14,880 --> 00:12:16,720
Speaker 1: get a few successes in a row and it gives

214
00:12:16,720 --> 00:12:19,520
Speaker 1: you more confidence in your training. You don't overthink it,

215
00:12:20,080 --> 00:12:22,960
Speaker 1: and you return and trust your training. So you're essentially

216
00:12:23,040 --> 00:12:25,600
Speaker 1: unconscious and you bring that whereas if you maybe miss

217
00:12:25,600 --> 00:12:28,319
Speaker 1: a funeral, you lose your confidence. You start making adjustments,

218
00:12:29,080 --> 00:12:31,320
Speaker 1: and if you're making adjustments, you're not going to have

219
00:12:31,440 --> 00:12:35,480
Speaker 1: much consistency. So let's go back to the original research.

220
00:12:35,840 --> 00:12:38,800
Speaker 1: Tom Gilovich one of the co authors said about the

221
00:12:38,840 --> 00:12:43,320
Speaker 1: work that you and Adam did, this is unlike a

222
00:12:43,320 --> 00:12:47,080
Speaker 1: lot of stuff that's come down the pike, since this

223
00:12:47,200 --> 00:12:51,800
Speaker 1: is truly interesting. How encouraging it was that from one

224
00:12:51,840 --> 00:12:55,880
Speaker 1: of the original authors who ostensibly disproved the hot hand.

225
00:12:56,559 --> 00:12:59,480
Speaker 1: I mean, it's always nice when somebody appreciates your work,

226
00:12:59,559 --> 00:13:02,520
Speaker 1: especially in one of Tom Gilovich's stature at the time

227
00:13:02,559 --> 00:13:04,480
Speaker 1: that he said that our our paper, while it had

228
00:13:04,559 --> 00:13:07,560
Speaker 1: gone through the public peri review process, it hadn't gone

229
00:13:07,559 --> 00:13:09,960
Speaker 1: through the formal one. And so just last week our

230
00:13:10,000 --> 00:13:12,160
Speaker 1: paper was finally published its online, not in the print

231
00:13:12,240 --> 00:13:15,280
Speaker 1: edition yet in Econometrica, which is you know, top journal

232
00:13:15,320 --> 00:13:19,160
Speaker 1: and economics. There's the top five. They're kind of all equal, um,

233
00:13:19,200 --> 00:13:21,600
Speaker 1: And you know, so now it's been kind of formally

234
00:13:22,080 --> 00:13:23,840
Speaker 1: taken in, so I think Tom Gilvitch might have a

235
00:13:23,840 --> 00:13:26,560
Speaker 1: different opinion, um, now that it's gone through this process.

236
00:13:26,559 --> 00:13:29,160
Speaker 1: So so the paper is ready to be published or

237
00:13:29,200 --> 00:13:33,080
Speaker 1: was just published. Yeah, so the November issue of Econometrica, Um,

238
00:13:33,120 --> 00:13:35,880
Speaker 1: it came out and that's got to be very exciting. Oh,

239
00:13:36,000 --> 00:13:39,280
Speaker 1: very exciting. Yeah, it's something. So what's the takeaway from

240
00:13:39,320 --> 00:13:43,600
Speaker 1: the original research, What was it that was wrong in

241
00:13:43,640 --> 00:13:48,360
Speaker 1: the structure of the original myth of the hot hands paper. Right,

242
00:13:48,440 --> 00:13:52,880
Speaker 1: So um, the original hot hand paper. They're interested in

243
00:13:53,280 --> 00:13:56,720
Speaker 1: seeing if people do better after recent success then after

244
00:13:56,760 --> 00:14:00,440
Speaker 1: recent failure. Um, that was the most important measure, Like,

245
00:14:00,559 --> 00:14:02,679
Speaker 1: is your probability of success increase when you've hit in

246
00:14:02,679 --> 00:14:06,040
Speaker 1: a few funeral versus if you've missed a funeral? And well,

247
00:14:06,040 --> 00:14:08,960
Speaker 1: we don't know what someone's probability is. It seems like

248
00:14:08,960 --> 00:14:10,719
Speaker 1: our best guests would be would just look at the

249
00:14:10,760 --> 00:14:13,320
Speaker 1: percentage of time they make it, right, And so they

250
00:14:13,360 --> 00:14:15,800
Speaker 1: just look at all the events when you've had a

251
00:14:15,840 --> 00:14:18,560
Speaker 1: streak of recent successes and all the events when you

252
00:14:18,559 --> 00:14:20,720
Speaker 1: have a streaking recent failures, and just see what the

253
00:14:20,800 --> 00:14:24,200
Speaker 1: change in your shooting percentages between those two conditions. And

254
00:14:24,760 --> 00:14:27,360
Speaker 1: you know, that's very natural and very intuitive to expect

255
00:14:27,680 --> 00:14:29,640
Speaker 1: that would be your best guess. And they do that

256
00:14:29,680 --> 00:14:33,720
Speaker 1: and they don't find any difference. Um. And so that's

257
00:14:33,720 --> 00:14:36,600
Speaker 1: how the problem was set up. So before we get

258
00:14:36,640 --> 00:14:40,720
Speaker 1: to your solution, the the immediate pushback is, Hey, after

259
00:14:40,760 --> 00:14:43,640
Speaker 1: a shooter gets on a bit of a streak, the

260
00:14:43,680 --> 00:14:47,520
Speaker 1: defense collapses on them. They become they're forced to either

261
00:14:47,760 --> 00:14:50,520
Speaker 1: pass the ball and more or take more difficult shots.

262
00:14:51,600 --> 00:14:55,880
Speaker 1: At the time in there was no way to account

263
00:14:55,960 --> 00:15:00,480
Speaker 1: for that difference. However, in the intervening years, every shot

264
00:15:00,880 --> 00:15:03,360
Speaker 1: gets marked. You you describe this in one of your

265
00:15:04,000 --> 00:15:08,840
Speaker 1: um publications recently, explain the degree of difficulty that is

266
00:15:08,880 --> 00:15:13,240
Speaker 1: now tracked on every single basketball shot that's taken. Right, So,

267
00:15:14,000 --> 00:15:16,480
Speaker 1: there there's a it's a new company now I don't

268
00:15:16,480 --> 00:15:18,920
Speaker 1: remember the company, but sport View was the first company

269
00:15:18,960 --> 00:15:24,360
Speaker 1: that did this, where they have optical tracking of the

270
00:15:25,000 --> 00:15:27,520
Speaker 1: I mean, the precision isn't super high, but it's it

271
00:15:27,560 --> 00:15:30,320
Speaker 1: gets in the general area, and so you can control

272
00:15:30,360 --> 00:15:33,240
Speaker 1: for a lot more factors than you could say, or

273
00:15:33,240 --> 00:15:34,840
Speaker 1: they had the seventies sixers and they're just looking at

274
00:15:34,880 --> 00:15:37,920
Speaker 1: the play by playout and so you know, even in

275
00:15:37,960 --> 00:15:39,720
Speaker 1: that data, what they would find is, yes, they'd have

276
00:15:39,800 --> 00:15:42,440
Speaker 1: this evidence of the defense adjusting to what they believed

277
00:15:42,480 --> 00:15:44,880
Speaker 1: to be a hoth hand, making it more difficult for

278
00:15:44,920 --> 00:15:47,200
Speaker 1: the player. But the player stats to shoot from time

279
00:15:47,200 --> 00:15:49,800
Speaker 1: to time to keep the defense honest. And so the

280
00:15:49,840 --> 00:15:51,840
Speaker 1: important thing isn't so much as the player doing better

281
00:15:51,840 --> 00:15:53,400
Speaker 1: in the context of the game, but as it helped

282
00:15:53,400 --> 00:15:55,400
Speaker 1: the teammates if they're hot, because then it opens upthing

283
00:15:55,440 --> 00:15:58,400
Speaker 1: for their teammates. Makes sense. Yeah, So the the innovations

284
00:15:58,720 --> 00:16:00,720
Speaker 1: that have happened, I think the first in vation actually

285
00:16:00,760 --> 00:16:04,160
Speaker 1: was Justin Rao, who's u the head economists at Home Away.

286
00:16:04,200 --> 00:16:06,920
Speaker 1: He was the first one to actually come out and

287
00:16:06,920 --> 00:16:09,000
Speaker 1: and measure how many defenders are around the player and

288
00:16:09,000 --> 00:16:11,000
Speaker 1: try to control from these things in a different way

289
00:16:11,000 --> 00:16:13,800
Speaker 1: by using the videos and show that yes, there's a

290
00:16:13,840 --> 00:16:16,400
Speaker 1: lot of evidence that there's this defense factor, and if

291
00:16:16,400 --> 00:16:18,240
Speaker 1: you just control for a few of these things, the

292
00:16:18,240 --> 00:16:21,280
Speaker 1: effects that they had found in the previous study went away.

293
00:16:21,360 --> 00:16:23,720
Speaker 1: So in other words, what looks like it's random is

294
00:16:24,200 --> 00:16:27,600
Speaker 1: you're shooting the same percentage but with a whole lot

295
00:16:27,920 --> 00:16:32,000
Speaker 1: more defensive activity on you. Therefore, it's a continuation of

296
00:16:32,040 --> 00:16:34,760
Speaker 1: the of the streak. Yes, so he didn't necessarily find

297
00:16:34,800 --> 00:16:37,160
Speaker 1: evidence of the streak there because he controlled for a

298
00:16:37,160 --> 00:16:39,920
Speaker 1: subset of factors. But as you add more more controls,

299
00:16:40,160 --> 00:16:42,480
Speaker 1: it looks like there might be some evidence there. But

300
00:16:42,560 --> 00:16:44,600
Speaker 1: these are very difficult things to measure in the context

301
00:16:44,600 --> 00:16:47,400
Speaker 1: of the game. The original study had this critical test

302
00:16:47,640 --> 00:16:50,680
Speaker 1: and it's been repeated with other teams. Um where they

303
00:16:50,680 --> 00:16:52,560
Speaker 1: take them and they pay them to shoot the basketball.

304
00:16:52,960 --> 00:16:55,040
Speaker 1: So in other words, you're you're not playing during a

305
00:16:55,080 --> 00:16:57,840
Speaker 1: live game, you're just doing foul shooting with three point

306
00:16:57,880 --> 00:17:00,640
Speaker 1: shooting or whatever exactly. Or you'll to the NBA three

307
00:17:00,640 --> 00:17:03,680
Speaker 1: point shooting contest and and and in those studies you

308
00:17:03,720 --> 00:17:05,399
Speaker 1: can get rid of the defense and get a little

309
00:17:05,400 --> 00:17:08,320
Speaker 1: more zero in on your question. A bit more. Let's

310
00:17:08,320 --> 00:17:12,480
Speaker 1: talk a little bit about the surprising math of coin flips.

311
00:17:13,080 --> 00:17:17,560
Speaker 1: My best guests and my understanding of statistics has always been,

312
00:17:17,600 --> 00:17:20,600
Speaker 1: if you take a true coin and flip it, the

313
00:17:20,640 --> 00:17:23,159
Speaker 1: odds of a head of a tail is fifty fifty.

314
00:17:23,600 --> 00:17:26,199
Speaker 1: This is regardless of what came before it. Coins have

315
00:17:26,320 --> 00:17:31,600
Speaker 1: no memory. But you found something surprising in the data set.

316
00:17:32,440 --> 00:17:35,960
Speaker 1: After you flip a coin a hundred times, and you

317
00:17:36,240 --> 00:17:40,960
Speaker 1: were to pick a specific series, the odds are somewhat different.

318
00:17:41,280 --> 00:17:44,960
Speaker 1: Explain that. Yeah, So, my author Adamson, and I, after

319
00:17:45,040 --> 00:17:47,239
Speaker 1: having watched the NBA three point shooting contests, we had

320
00:17:47,240 --> 00:17:50,720
Speaker 1: a particular player, Craig Hodges, who was obviously hot, and

321
00:17:50,720 --> 00:17:53,720
Speaker 1: we went and used the original analysis on his data

322
00:17:53,720 --> 00:17:55,960
Speaker 1: and it said that he wasn't and that was puzzling,

323
00:17:56,600 --> 00:17:58,760
Speaker 1: and so we had to go and see, well, we

324
00:17:58,800 --> 00:18:03,120
Speaker 1: don't really know how Craig Hodges generated his shots. It's

325
00:18:03,160 --> 00:18:06,120
Speaker 1: kind of a black box. But let's create an environment

326
00:18:06,119 --> 00:18:08,920
Speaker 1: where we have the ground truth when we know what's happening,

327
00:18:08,920 --> 00:18:11,080
Speaker 1: and so coin flips is a world like this. So

328
00:18:11,119 --> 00:18:13,760
Speaker 1: you can actually go and flip a coin many many times,

329
00:18:13,840 --> 00:18:16,080
Speaker 1: or do it on a computer and see what do

330
00:18:16,160 --> 00:18:19,560
Speaker 1: you get if you analyze we're interested, is the probability

331
00:18:19,560 --> 00:18:21,720
Speaker 1: of heads after a few heads different than the probablity

332
00:18:21,720 --> 00:18:24,240
Speaker 1: of heads after a few tails? We know that's the same.

333
00:18:24,520 --> 00:18:26,760
Speaker 1: That's we have the ground truth. But now let's go

334
00:18:26,840 --> 00:18:30,160
Speaker 1: out and generate that data and make our best guests

335
00:18:30,160 --> 00:18:33,680
Speaker 1: from that data. What's our best guests is the percentage

336
00:18:33,720 --> 00:18:35,560
Speaker 1: of heads that you get after a few heads in

337
00:18:35,560 --> 00:18:37,439
Speaker 1: a row the same as the percentage of heads you

338
00:18:37,480 --> 00:18:39,960
Speaker 1: get after a few tails in a row. And analyzing

339
00:18:40,000 --> 00:18:42,040
Speaker 1: it in the way they analyzed it, we found that no,

340
00:18:42,200 --> 00:18:45,200
Speaker 1: it's different. The percentage of tails after a few heads

341
00:18:45,200 --> 00:18:49,919
Speaker 1: in a row is higher, which which is so counterintuitive

342
00:18:50,080 --> 00:18:55,520
Speaker 1: because perspectively, so understand before people lose their mind and

343
00:18:55,560 --> 00:18:59,119
Speaker 1: start sending emails, what we're not talking about is looking

344
00:18:59,160 --> 00:19:03,280
Speaker 1: forward in a live situation, no matter what the previous

345
00:19:03,520 --> 00:19:06,440
Speaker 1: with with a true coin. You could have a thousand

346
00:19:06,440 --> 00:19:10,720
Speaker 1: heads in a row. Highly improbable, but not mathematically impossible.

347
00:19:11,040 --> 00:19:13,639
Speaker 1: The odds on that next flip are still gonna be

348
00:19:15,280 --> 00:19:17,680
Speaker 1: That's not what we're saying. We're saying, flip a coin

349
00:19:17,720 --> 00:19:20,400
Speaker 1: a hundred times, look at the data set, and then

350
00:19:20,440 --> 00:19:24,160
Speaker 1: go back and randomly pick any head in that order

351
00:19:24,240 --> 00:19:26,800
Speaker 1: or any tail in the order. What are the odds

352
00:19:26,880 --> 00:19:30,359
Speaker 1: that the next flip is ahead of a tail? And

353
00:19:30,440 --> 00:19:35,480
Speaker 1: it turns out that's not so? Explain that because it's

354
00:19:35,600 --> 00:19:39,679
Speaker 1: it's a complete It blows people's minds because you've been

355
00:19:39,680 --> 00:19:42,640
Speaker 1: told over and over again, hey, coins have no memory.

356
00:19:42,640 --> 00:19:44,960
Speaker 1: But that's not what this is. This is an existing

357
00:19:45,040 --> 00:19:48,880
Speaker 1: data set. When we randomly pull any of those flips,

358
00:19:49,280 --> 00:19:52,000
Speaker 1: what are the probabilities as to the outcome in the

359
00:19:52,040 --> 00:19:55,640
Speaker 1: next flip after it's already been done. So, so how

360
00:19:55,640 --> 00:19:59,560
Speaker 1: do you end up with instead of so the complete

361
00:19:59,560 --> 00:20:02,240
Speaker 1: exper name would take some time, but we can kind

362
00:20:02,240 --> 00:20:05,000
Speaker 1: of get it an intuition. Um, if you flip a

363
00:20:05,000 --> 00:20:06,920
Speaker 1: coin a hundred times, there's gonna be a certain number

364
00:20:06,960 --> 00:20:12,360
Speaker 1: of heads and tails there when when you're done about guarantee,

365
00:20:12,359 --> 00:20:15,040
Speaker 1: but it's gonna be some number. Now, if you just

366
00:20:15,160 --> 00:20:18,680
Speaker 1: choose any flip your best guess, just choose a flip

367
00:20:18,720 --> 00:20:21,760
Speaker 1: your best guess. Is if I choose flip forty two,

368
00:20:21,920 --> 00:20:27,560
Speaker 1: for example, my best guess or heads, that's my you know,

369
00:20:28,640 --> 00:20:32,680
Speaker 1: And so that's different though then if I choose flip

370
00:20:32,680 --> 00:20:36,399
Speaker 1: forty two because flip forty one is the heads, So

371
00:20:36,440 --> 00:20:38,600
Speaker 1: if I choose one of the flips where the previous

372
00:20:38,600 --> 00:20:42,000
Speaker 1: flip is the heads, or just choose a flip that's

373
00:20:42,000 --> 00:20:43,919
Speaker 1: ahead and see what the next flip is in the

374
00:20:43,960 --> 00:20:47,640
Speaker 1: same way of looking at it. Now there's something else

375
00:20:47,680 --> 00:20:50,760
Speaker 1: because the flip you chose because the previous flip is ahead,

376
00:20:51,240 --> 00:20:56,679
Speaker 1: is using information about the outcomes of adjacent flips, and

377
00:20:56,680 --> 00:20:59,240
Speaker 1: that information kind of gets contained within your flip, and

378
00:20:59,280 --> 00:21:02,200
Speaker 1: that's this. This gets a little complicated, but one way

379
00:21:02,200 --> 00:21:04,280
Speaker 1: to think about it is you've taken a head's away

380
00:21:04,359 --> 00:21:06,679
Speaker 1: from the finite number of heads that you have and

381
00:21:06,720 --> 00:21:08,800
Speaker 1: you can't see it again. You've reduced that data set,

382
00:21:08,800 --> 00:21:13,040
Speaker 1: and the remaining tails now should be slightly than it shifts.

383
00:21:13,040 --> 00:21:15,800
Speaker 1: And there's another element that uses the kind of space

384
00:21:15,840 --> 00:21:18,600
Speaker 1: away there arrange that makes it a bit stronger. But

385
00:21:18,640 --> 00:21:23,280
Speaker 1: that that that's this smells to me slightly like the

386
00:21:23,320 --> 00:21:26,760
Speaker 1: Monty Hall problem. Is there any element in the Monty

387
00:21:26,800 --> 00:21:30,520
Speaker 1: Hall problem? You go from choosing one in three to

388
00:21:30,840 --> 00:21:33,200
Speaker 1: one and two. So suddenly what was the thirty three

389
00:21:33,200 --> 00:21:36,320
Speaker 1: percent chance becomes a fifty fifty Why not make the

390
00:21:36,359 --> 00:21:40,359
Speaker 1: switch that That is a little counterintuitive, but once you

391
00:21:40,400 --> 00:21:43,680
Speaker 1: see the statistics, it you can't unsee it. It always

392
00:21:43,720 --> 00:21:47,040
Speaker 1: you always should make the change. There is a is

393
00:21:47,040 --> 00:21:50,120
Speaker 1: there a tiny element of this and that more than tiny.

394
00:21:50,280 --> 00:21:52,760
Speaker 1: So my co author and Adamson Hot and I also

395
00:21:52,800 --> 00:21:56,080
Speaker 1: wrote another paper connecting this to the Monty Hall problem

396
00:21:56,320 --> 00:21:59,760
Speaker 1: and explaining it via this principle from Bridge, which is

397
00:21:59,760 --> 00:22:03,480
Speaker 1: the stable restricted choice, which is essentially the intuition of

398
00:22:03,800 --> 00:22:06,119
Speaker 1: Bay's rule, and so the way to think about it

399
00:22:06,119 --> 00:22:08,080
Speaker 1: in the Monty Hall problem, you of these three doors, right,

400
00:22:08,080 --> 00:22:10,320
Speaker 1: so you're on this game show. There's three doors. Well,

401
00:22:10,400 --> 00:22:13,600
Speaker 1: let's make this our problem exactly the same as money,

402
00:22:13,680 --> 00:22:16,879
Speaker 1: except it's a hundred doors, not necessarily three doors, so

403
00:22:16,920 --> 00:22:19,359
Speaker 1: it becomes much harder. We can make it three doors.

404
00:22:19,520 --> 00:22:23,040
Speaker 1: So your games show, you're on a game show, and

405
00:22:23,119 --> 00:22:24,879
Speaker 1: usually that you have this car and two goats, and

406
00:22:24,880 --> 00:22:26,760
Speaker 1: you gotta find the car, and there's a car behind

407
00:22:26,760 --> 00:22:28,760
Speaker 1: one of the doors. You gotta guess. Well, let's get

408
00:22:28,840 --> 00:22:30,480
Speaker 1: rid of the goats and cars. Now, let's just flip

409
00:22:30,480 --> 00:22:33,040
Speaker 1: a coin behind each door. So eat behind the shorts

410
00:22:33,080 --> 00:22:35,639
Speaker 1: fifty fifty. But you're the contestant. The host knows what

411
00:22:35,680 --> 00:22:37,920
Speaker 1: the outcome of the flips are. You don't you want

412
00:22:37,920 --> 00:22:41,120
Speaker 1: to guess, Hey, where's it? Where's the heads? Let's say

413
00:22:41,160 --> 00:22:43,800
Speaker 1: you want to find the heads um, so you guess

414
00:22:44,359 --> 00:22:47,800
Speaker 1: you know door three. Now, if you guess door three,

415
00:22:47,880 --> 00:22:49,879
Speaker 1: let's say the host looks behind the door. You didn't

416
00:22:49,880 --> 00:22:52,480
Speaker 1: guess door one and two, and he's going to reveal

417
00:22:52,520 --> 00:22:54,840
Speaker 1: the heads if you can't. So let's say the host

418
00:22:55,040 --> 00:22:57,600
Speaker 1: opens door one and shows you ahead. Do you want

419
00:22:57,600 --> 00:23:00,840
Speaker 1: to switch or do you want to stay? If you're

420
00:23:00,840 --> 00:23:04,320
Speaker 1: looking for the heads, you want to stay. If you're

421
00:23:04,320 --> 00:23:07,719
Speaker 1: looking for the tails, you want to switch. Now, the

422
00:23:07,800 --> 00:23:10,960
Speaker 1: intuition is not going to be clear immediately, but if

423
00:23:11,000 --> 00:23:14,479
Speaker 1: you think about now, the host looking at door one

424
00:23:14,520 --> 00:23:18,880
Speaker 1: and two used information about both doors to determine which

425
00:23:18,920 --> 00:23:21,359
Speaker 1: door to open up to you. Now, if both doors

426
00:23:21,359 --> 00:23:25,080
Speaker 1: were heads, the host could have opened door too. But

427
00:23:25,119 --> 00:23:28,040
Speaker 1: if it was heads tails, the door host had to

428
00:23:28,080 --> 00:23:29,679
Speaker 1: open door one. Because the host is gonna show your

429
00:23:29,680 --> 00:23:31,560
Speaker 1: heads if you can't, right, he doesn't want to show

430
00:23:31,560 --> 00:23:33,840
Speaker 1: you a tail because that's what you're looking to avoid.

431
00:23:34,280 --> 00:23:36,720
Speaker 1: That's the goat. Yea. So the world we don't know

432
00:23:36,760 --> 00:23:38,840
Speaker 1: which world we're in where the first is heads and

433
00:23:38,880 --> 00:23:40,440
Speaker 1: the second is tails. Of the first his heads and

434
00:23:40,480 --> 00:23:44,680
Speaker 1: the you know second, his heads. But the world where

435
00:23:44,680 --> 00:23:47,720
Speaker 1: it's heads tails is the world where the host is

436
00:23:47,760 --> 00:23:50,959
Speaker 1: more restricted. The host has to open door one, and

437
00:23:51,000 --> 00:23:53,160
Speaker 1: so the world that so that you should avoid door

438
00:23:53,160 --> 00:23:56,159
Speaker 1: two in those circumstances because it's a higher probability of

439
00:23:56,440 --> 00:23:58,399
Speaker 1: if you're if you're hunting for the heads you should have,

440
00:23:58,800 --> 00:24:02,399
Speaker 1: you should avoid door too because tails is more likely

441
00:24:02,480 --> 00:24:05,200
Speaker 1: because in that world of heads tails, the host had

442
00:24:05,200 --> 00:24:08,520
Speaker 1: to open door one and it's yeah, so it's a

443
00:24:08,600 --> 00:24:12,119
Speaker 1: higher probability. So that door three, even though it's a

444
00:24:12,119 --> 00:24:15,200
Speaker 1: coin that's flipped independent of the other two, when you're

445
00:24:15,200 --> 00:24:17,960
Speaker 1: dealing with that data set, you're better off with three

446
00:24:18,040 --> 00:24:21,560
Speaker 1: because of the circumstances that lead the host to pick

447
00:24:21,640 --> 00:24:26,200
Speaker 1: one and not pick two. That makes some rational degree

448
00:24:26,200 --> 00:24:29,080
Speaker 1: of sense. Once you get the monty whole aspect of this,

449
00:24:29,520 --> 00:24:31,520
Speaker 1: it makes a whole lot more sense. It's just it

450
00:24:31,600 --> 00:24:36,200
Speaker 1: just seems, uh, it's quite fascinating. We were discussing um

451
00:24:36,240 --> 00:24:40,080
Speaker 1: the coin flip issue and the hot hand scenario. Let's

452
00:24:40,119 --> 00:24:43,760
Speaker 1: circle back to that hot hand and the original research.

453
00:24:43,800 --> 00:24:47,399
Speaker 1: The original research said that if there's a streak of

454
00:24:47,480 --> 00:24:52,120
Speaker 1: three hits in basketball or three misses in basketball, the

455
00:24:52,160 --> 00:24:55,800
Speaker 1: odds of the next shot going in or not is

456
00:24:56,240 --> 00:25:02,320
Speaker 1: whatever the shooters historical shooting percentage hges, which sort of

457
00:25:02,359 --> 00:25:06,800
Speaker 1: seems that there's no hot hand. But that presumes after

458
00:25:06,960 --> 00:25:11,320
Speaker 1: streak that their next shot should be dead center in

459
00:25:11,359 --> 00:25:15,280
Speaker 1: their percentage. You found out it should be worse than that.

460
00:25:15,920 --> 00:25:19,040
Speaker 1: Explain exactly. So that's the counterintuitive thing. If you go

461
00:25:19,080 --> 00:25:21,399
Speaker 1: out and you watch a player shoot a basketball and

462
00:25:21,440 --> 00:25:23,879
Speaker 1: you look at their shooting percentage after a streak of

463
00:25:23,960 --> 00:25:26,480
Speaker 1: hits and compare it to their shooting percentage after a

464
00:25:26,480 --> 00:25:29,720
Speaker 1: streak of misses, and you find that it's the same

465
00:25:30,800 --> 00:25:32,600
Speaker 1: and not. The intuitive thing is to say, oh, they're

466
00:25:32,640 --> 00:25:35,400
Speaker 1: just they have the same rate, but actually you would

467
00:25:35,400 --> 00:25:38,480
Speaker 1: expect them to do worse. Explain that, because that's the

468
00:25:38,480 --> 00:25:41,199
Speaker 1: most fascinating part of it. Someone is on a shooting streak,

469
00:25:41,840 --> 00:25:44,440
Speaker 1: we take a data set of a whole run of shots.

470
00:25:45,320 --> 00:25:48,720
Speaker 1: What do you find after the streak, and why is that?

471
00:25:48,880 --> 00:25:52,480
Speaker 1: So you said, you find their percentage actually goes down

472
00:25:52,560 --> 00:25:55,640
Speaker 1: after a streak. In the world where there's no hot hand,

473
00:25:55,640 --> 00:25:58,600
Speaker 1: where they're a consistent shooter, their percentage will go down

474
00:25:58,640 --> 00:26:01,280
Speaker 1: after a streak in the data. Not in reality, their

475
00:26:01,280 --> 00:26:04,000
Speaker 1: probability is always the same. But we don't observe the probability.

476
00:26:04,000 --> 00:26:06,639
Speaker 1: We calculate the percentage, and that's where the biases come in.

477
00:26:07,960 --> 00:26:10,520
Speaker 1: And so the original authors found that the shooting percentage

478
00:26:10,560 --> 00:26:12,320
Speaker 1: was around the same, and that's correct. We go and

479
00:26:12,359 --> 00:26:14,320
Speaker 1: we check and they were right. They report they did

480
00:26:14,359 --> 00:26:17,800
Speaker 1: all the analysis in that sense, the calculations correctly. But

481
00:26:18,000 --> 00:26:21,760
Speaker 1: the mistake is understanding the benchmark. You have to go

482
00:26:21,800 --> 00:26:24,280
Speaker 1: out and say, okay, now let's look at the world

483
00:26:24,320 --> 00:26:27,280
Speaker 1: where we know, um, we can control it. So on

484
00:26:27,320 --> 00:26:29,359
Speaker 1: a computer you can say, we can generate coin flips,

485
00:26:29,359 --> 00:26:31,120
Speaker 1: so we can make a player that has no hot hand,

486
00:26:31,119 --> 00:26:32,919
Speaker 1: and then look at how that player does when we

487
00:26:33,200 --> 00:26:35,399
Speaker 1: analyze the data and we realize, oh, they should do

488
00:26:35,440 --> 00:26:36,919
Speaker 1: worse after a few in a row. So once you

489
00:26:36,960 --> 00:26:41,920
Speaker 1: adjust for that bias, you find out that actually, if

490
00:26:41,920 --> 00:26:45,000
Speaker 1: they're doing the same, that's indicative that they're doing about

491
00:26:45,080 --> 00:26:48,359
Speaker 1: ten percentage points are more better after hitting a funer

492
00:26:48,440 --> 00:26:51,040
Speaker 1: row than missing in a few funeroe and that's huge.

493
00:26:51,040 --> 00:26:53,119
Speaker 1: That's like the difference between the median and the best

494
00:26:53,240 --> 00:26:57,359
Speaker 1: NBA three points, thereby confirming the hot hands. So I

495
00:26:57,480 --> 00:27:02,080
Speaker 1: have to challenge the data set because it again, everything

496
00:27:02,119 --> 00:27:05,440
Speaker 1: about this, each step along the way, is so counterintuitive.

497
00:27:05,880 --> 00:27:09,919
Speaker 1: So why would we expect a shooter who's on a streak,

498
00:27:09,960 --> 00:27:13,040
Speaker 1: who's in the zone, who has the hot hand, whatever

499
00:27:13,080 --> 00:27:15,720
Speaker 1: we want to call it, Why would we expect his

500
00:27:15,840 --> 00:27:21,800
Speaker 1: shooting percentage to be lower after they hit several shots

501
00:27:21,800 --> 00:27:24,280
Speaker 1: in a row. Why would we expect it to be

502
00:27:24,320 --> 00:27:27,760
Speaker 1: lower for a real human or for for anybody, for

503
00:27:27,800 --> 00:27:29,719
Speaker 1: a professional, for a real human. When you look at

504
00:27:29,720 --> 00:27:33,119
Speaker 1: a data set of here's here's all the NBA streak

505
00:27:33,160 --> 00:27:36,399
Speaker 1: shooters or all the NBA shooters, what does the data

506
00:27:36,520 --> 00:27:41,800
Speaker 1: show after a streak they're shooting percentage actually becomes So

507
00:27:41,840 --> 00:27:45,040
Speaker 1: if you're talking about live action games, we have those

508
00:27:45,080 --> 00:27:48,040
Speaker 1: issues that we spoke about. The defense will adjust and

509
00:27:48,119 --> 00:27:50,119
Speaker 1: so that becomes a little bit more complished. So so

510
00:27:50,200 --> 00:27:54,920
Speaker 1: let's talk about three point contests. So if the hot

511
00:27:54,960 --> 00:27:59,119
Speaker 1: hand didn't exist in a world like that, we would

512
00:27:59,119 --> 00:28:02,359
Speaker 1: expect players to shoot worse after making a few in

513
00:28:02,400 --> 00:28:05,320
Speaker 1: a row in the data, so simply just mean reversion,

514
00:28:05,400 --> 00:28:08,120
Speaker 1: is that all. It's not mean resursion. It's the same

515
00:28:08,160 --> 00:28:10,719
Speaker 1: thing we talked about with the coin flips. Right. And

516
00:28:10,800 --> 00:28:13,760
Speaker 1: so as a researcher, you're taking the data after it's

517
00:28:13,800 --> 00:28:16,040
Speaker 1: already been generated, and you're picking through it, and you're

518
00:28:16,040 --> 00:28:17,840
Speaker 1: looking only at the events that you're interested in. Right,

519
00:28:17,840 --> 00:28:20,359
Speaker 1: You're looking at their probability of success given recent success.

520
00:28:20,440 --> 00:28:23,760
Speaker 1: You're just picking out those events when they had recent success.

521
00:28:23,840 --> 00:28:25,640
Speaker 1: Let's say where they just made three in a row.

522
00:28:25,680 --> 00:28:29,000
Speaker 1: So you're changing the data set so now there's three less.

523
00:28:29,040 --> 00:28:31,480
Speaker 1: So if you someone shoots three in a row, when

524
00:28:31,480 --> 00:28:33,960
Speaker 1: we're looking at the data set, let's say they've shot

525
00:28:34,000 --> 00:28:36,560
Speaker 1: twenty shots, and after three in a row, how they

526
00:28:36,560 --> 00:28:40,840
Speaker 1: do well, guess what, You've pulled three hits out of

527
00:28:40,840 --> 00:28:43,920
Speaker 1: the set, meaning there's a disproportionate number of mrs left.

528
00:28:44,080 --> 00:28:46,840
Speaker 1: That's part of the bias. And there's this other element

529
00:28:46,880 --> 00:28:49,600
Speaker 1: that didn't quite get into is that you can you

530
00:28:49,680 --> 00:28:52,480
Speaker 1: have this essentially a stopping rule, so as you collect

531
00:28:52,520 --> 00:28:55,000
Speaker 1: the data, the moment they miss it, you're not interested

532
00:28:55,040 --> 00:28:57,160
Speaker 1: anymore in looking because you're gonna wait for a streak

533
00:28:57,200 --> 00:29:00,360
Speaker 1: of hits again, so you've kind of you're biased towards

534
00:29:00,360 --> 00:29:02,440
Speaker 1: stopping at a miss, so you might get a miss

535
00:29:02,520 --> 00:29:06,960
Speaker 1: right away right then everything you collected in those events

536
00:29:07,040 --> 00:29:09,200
Speaker 1: of their shots or misses because you just collected one

537
00:29:09,200 --> 00:29:11,640
Speaker 1: shot and they're all misses, and so you're you can

538
00:29:11,760 --> 00:29:15,600
Speaker 1: biasing towards collecting misses in a side that that that's

539
00:29:15,680 --> 00:29:20,160
Speaker 1: quite that's quite fascinating. So what other areas like this

540
00:29:20,320 --> 00:29:24,360
Speaker 1: are you studying, because it's really it's really quite quite

541
00:29:24,400 --> 00:29:28,760
Speaker 1: fascinating stuff. Are there other sport myths that you're looking

542
00:29:28,800 --> 00:29:33,840
Speaker 1: at that have a probabilistic element that's very counterintuitive or

543
00:29:33,920 --> 00:29:37,600
Speaker 1: is this pretty much the biggest one out there? Um,

544
00:29:37,640 --> 00:29:39,720
Speaker 1: this is the biggest one that we're studying. A lot

545
00:29:39,720 --> 00:29:44,120
Speaker 1: of what you're doing is statistical and probability work at

546
00:29:44,120 --> 00:29:46,840
Speaker 1: a level that the average sports fan is really not

547
00:29:46,960 --> 00:29:50,760
Speaker 1: familiar with. Forget the live game. When you explain relative

548
00:29:50,800 --> 00:29:55,600
Speaker 1: to three point shooting contest, it's really not so much

549
00:29:55,640 --> 00:29:59,240
Speaker 1: about the streakiness of the shooter, but the mathematics of

550
00:29:59,280 --> 00:30:03,240
Speaker 1: the data set it. And I think that is really counterintuitive,

551
00:30:03,760 --> 00:30:06,960
Speaker 1: but it doesn't seem anyone's been able to disprove what

552
00:30:07,040 --> 00:30:09,280
Speaker 1: you and your co author found. So there have been

553
00:30:09,320 --> 00:30:12,360
Speaker 1: a lot of challenges to that original study, a lot

554
00:30:12,400 --> 00:30:15,120
Speaker 1: of challenge and legitimate challenges. You know, there are issues

555
00:30:15,200 --> 00:30:18,880
Speaker 1: with UM what they call statistical power. Right, so we

556
00:30:18,960 --> 00:30:21,600
Speaker 1: have we have a friend and colleague, Daniel Stone, who

557
00:30:21,640 --> 00:30:23,680
Speaker 1: made this nice point that you have this thing called measurement.

558
00:30:23,680 --> 00:30:25,800
Speaker 1: Are we want to know how do you do after

559
00:30:25,880 --> 00:30:28,040
Speaker 1: hitting a few in a row? Um, that's what we

560
00:30:28,080 --> 00:30:31,000
Speaker 1: actually look at. But what we're really interested in is

561
00:30:31,720 --> 00:30:34,360
Speaker 1: how will you do when you're hot? So hitting a

562
00:30:34,400 --> 00:30:37,080
Speaker 1: few in a row, you're not always hot, and so

563
00:30:37,120 --> 00:30:40,920
Speaker 1: you can underestimate how hot someone is if you use

564
00:30:42,120 --> 00:30:44,240
Speaker 1: only the data that you can observe, which is zeros

565
00:30:44,280 --> 00:30:47,160
Speaker 1: and one. So the you know, the econometrician, the statistician

566
00:30:47,240 --> 00:30:50,520
Speaker 1: has kind of a weak measure of that. So you know,

567
00:30:50,920 --> 00:30:54,680
Speaker 1: these this kind of evidence is um just the mathematical evidence.

568
00:30:54,960 --> 00:30:56,960
Speaker 1: Do you ever do interviews of players? Do you ever

569
00:30:57,440 --> 00:31:00,080
Speaker 1: say to them, Hey, were you in the zone? How

570
00:31:00,080 --> 00:31:03,719
Speaker 1: did you feel? How? How how do you find that

571
00:31:03,800 --> 00:31:08,280
Speaker 1: data set? So that data UM, the original study looked

572
00:31:08,280 --> 00:31:11,280
Speaker 1: at data like that. They spoke to the seventies sixers

573
00:31:11,320 --> 00:31:14,400
Speaker 1: and they asked them kind of qualitative questions. Do you

574
00:31:14,440 --> 00:31:18,120
Speaker 1: get in the zone and you feel hot? And they

575
00:31:18,120 --> 00:31:20,880
Speaker 1: all do right, Um, but it's it's hard to work

576
00:31:20,920 --> 00:31:23,120
Speaker 1: with that. That's just looking at whether they believe in

577
00:31:23,160 --> 00:31:26,239
Speaker 1: it or not, but then getting a sense of do

578
00:31:26,280 --> 00:31:28,720
Speaker 1: they believe in it too much or not? That get

579
00:31:28,760 --> 00:31:30,640
Speaker 1: that gets a bit harder because you have to be

580
00:31:30,680 --> 00:31:33,160
Speaker 1: able to somehow measure you know, they have to decide

581
00:31:33,160 --> 00:31:35,040
Speaker 1: when are they hot? You know, so you really need

582
00:31:35,040 --> 00:31:37,080
Speaker 1: a lot more cooperation from say like a coach or

583
00:31:37,120 --> 00:31:38,680
Speaker 1: player to kind of sit there and maybe watch the

584
00:31:38,680 --> 00:31:41,120
Speaker 1: games with you or something like that. That would be

585
00:31:41,200 --> 00:31:44,840
Speaker 1: maybe a a better way of testing their you know,

586
00:31:44,920 --> 00:31:50,240
Speaker 1: their beliefs. So so when um Traverseking Gilvitch's original study

587
00:31:50,360 --> 00:31:55,760
Speaker 1: came out, I'm forgetting the third person in g VT.

588
00:31:56,440 --> 00:31:59,160
Speaker 1: When when that study came out, there was a tremendous

589
00:31:59,160 --> 00:32:02,400
Speaker 1: amount of push back from coaches amount around the league.

590
00:32:02,400 --> 00:32:06,479
Speaker 1: We mentioned ridd are back. Your study comes out and

591
00:32:06,520 --> 00:32:10,520
Speaker 1: you basically say, no, you professional coaches, You were right.

592
00:32:11,160 --> 00:32:13,280
Speaker 1: There is a hot hand, there is a streak. What

593
00:32:13,400 --> 00:32:16,560
Speaker 1: sort of feedback have you gotten from players and coaches

594
00:32:16,600 --> 00:32:22,040
Speaker 1: about your research? Well, we're not entirely sure whether players

595
00:32:22,040 --> 00:32:26,360
Speaker 1: and coaches were ever frowsled by the original study, so,

596
00:32:26,800 --> 00:32:30,440
Speaker 1: you know, validating their beliefs for them. It's so, yeah,

597
00:32:30,440 --> 00:32:33,000
Speaker 1: we kind of never believe that result to begin with.

598
00:32:33,080 --> 00:32:35,840
Speaker 1: So so we haven't gone and sought the opinion of

599
00:32:36,160 --> 00:32:39,360
Speaker 1: you know, players and coaches because it's it's not so

600
00:32:39,440 --> 00:32:43,520
Speaker 1: clear how far that original conclusion reached into that world. Um,

601
00:32:43,720 --> 00:32:47,200
Speaker 1: while it did, especially you can see announcers mentioning it. Um.

602
00:32:47,240 --> 00:32:50,680
Speaker 1: But yeah, yeah, so so when you what about some

603
00:32:50,720 --> 00:32:52,760
Speaker 1: of the outlawer players. If you look at a Michael

604
00:32:52,840 --> 00:32:59,640
Speaker 1: Jordan's um or a Steph Carry, guys who literally become

605
00:32:59,720 --> 00:33:03,080
Speaker 1: just conscious and what Reggie Miller is another one, and

606
00:33:03,320 --> 00:33:08,040
Speaker 1: the most improbable shots on a consistent basis start to

607
00:33:08,160 --> 00:33:11,480
Speaker 1: drop when when you look at players like that, do

608
00:33:11,680 --> 00:33:15,600
Speaker 1: different players seem to have a different set of streakiness,

609
00:33:15,640 --> 00:33:19,680
Speaker 1: a different hot hand? Can you can you calibrate how

610
00:33:19,760 --> 00:33:23,400
Speaker 1: much of a hot hand different players have? So using

611
00:33:23,440 --> 00:33:25,480
Speaker 1: game data that's a bit more of a challenge. So

612
00:33:26,200 --> 00:33:29,360
Speaker 1: my couth or Adam and I looked at Spanish semipro players.

613
00:33:29,400 --> 00:33:30,800
Speaker 1: We could collect a lot more data and we had

614
00:33:30,840 --> 00:33:33,280
Speaker 1: more of their cooperation, and there seemed to be a

615
00:33:33,280 --> 00:33:35,320
Speaker 1: clear difference with players. I mean, there's the obvious one

616
00:33:35,360 --> 00:33:37,560
Speaker 1: is that you know, centers and forwards, people that don't

617
00:33:37,560 --> 00:33:40,040
Speaker 1: shoot that often. It's hard for them to get on

618
00:33:40,080 --> 00:33:42,360
Speaker 1: a roll because you have to be consistent and they're

619
00:33:42,400 --> 00:33:44,880
Speaker 1: kind of not that consistent when they don't touch the ball.

620
00:33:44,920 --> 00:33:48,360
Speaker 1: All that and all that, right, and so you know,

621
00:33:48,680 --> 00:33:50,920
Speaker 1: those are the people you'd expect maybe you know, they

622
00:33:50,920 --> 00:33:53,240
Speaker 1: can't really sustain a streak. And that's what we find.

623
00:33:53,280 --> 00:33:54,920
Speaker 1: You know, so there's some players that can and some

624
00:33:54,960 --> 00:33:57,000
Speaker 1: players that seem like they can't. Um. If we go

625
00:33:57,080 --> 00:33:59,600
Speaker 1: to real NBA players, you know, that's a bit of

626
00:33:59,600 --> 00:34:01,000
Speaker 1: a challe And so we have looked at the three

627
00:34:01,000 --> 00:34:04,560
Speaker 1: point shooting contest and we have a paper on that. Um.

628
00:34:04,720 --> 00:34:06,720
Speaker 1: The issue with the three point shooting contest is a

629
00:34:06,760 --> 00:34:08,520
Speaker 1: lot of the players don't have much more than say

630
00:34:08,560 --> 00:34:11,160
Speaker 1: a hundred shots total in the contest. You know, may

631
00:34:11,280 --> 00:34:13,080
Speaker 1: some have a few more. You have a Craig Hodges

632
00:34:13,080 --> 00:34:14,759
Speaker 1: who has over five hundred in our data, and we

633
00:34:14,800 --> 00:34:18,040
Speaker 1: find evidence there. Um. But what we can say is

634
00:34:18,080 --> 00:34:22,640
Speaker 1: that among all the three point shooting contest contestants, there

635
00:34:22,640 --> 00:34:25,560
Speaker 1: were way more that did better after a few in

636
00:34:25,600 --> 00:34:27,920
Speaker 1: a row than making a few row the missing a

637
00:34:28,320 --> 00:34:31,200
Speaker 1: funeral than you'd expect, but you don't really know which

638
00:34:31,239 --> 00:34:33,160
Speaker 1: of them are really hot. You just know there's more

639
00:34:33,200 --> 00:34:35,479
Speaker 1: of them than you expect, but you need more data

640
00:34:35,520 --> 00:34:37,759
Speaker 1: to be really confident when you pick out an individual.

641
00:34:38,120 --> 00:34:41,000
Speaker 1: So at this point in the state of research on

642
00:34:41,040 --> 00:34:43,799
Speaker 1: the hot hand, do you have any doubt that the

643
00:34:43,840 --> 00:34:47,160
Speaker 1: hot hand exists? I don't have any doubt that the

644
00:34:47,160 --> 00:34:50,040
Speaker 1: hot hand exists. What you mean by the hot hand

645
00:34:50,680 --> 00:34:53,280
Speaker 1: is where the doubts come in, because there's many different

646
00:34:53,280 --> 00:34:56,799
Speaker 1: mechanisms that can lead to evidence in the data that

647
00:34:56,880 --> 00:34:59,680
Speaker 1: your success after recent success is higher than you know

648
00:34:59,719 --> 00:35:03,960
Speaker 1: that it's higher than after recent So so the confidence factor,

649
00:35:04,000 --> 00:35:08,919
Speaker 1: the endorphin factor, the further pressure that the other team

650
00:35:08,960 --> 00:35:12,160
Speaker 1: is placing. All those things they add up. You ask

651
00:35:12,239 --> 00:35:14,360
Speaker 1: a player, they're gonna say, yeah, of course you get hot.

652
00:35:14,800 --> 00:35:18,680
Speaker 1: But when you ask the stat statistician, the data supports

653
00:35:18,719 --> 00:35:22,880
Speaker 1: it as well. Right, quite fascinating. We have been speaking

654
00:35:22,920 --> 00:35:26,560
Speaker 1: to Joshua Miller. He is an economics professor and researcher

655
00:35:26,920 --> 00:35:31,160
Speaker 1: at the University of Alacante in Spain. If you enjoy

656
00:35:31,239 --> 00:35:34,319
Speaker 1: this conversation, well be sure and come back and check

657
00:35:34,320 --> 00:35:36,760
Speaker 1: out our podcast extras where we keep the tape rolling

658
00:35:37,040 --> 00:35:42,239
Speaker 1: and we continue discussing all things statistical, sports and behavior.

659
00:35:43,040 --> 00:35:47,520
Speaker 1: You can find that at iTunes, overcast, at your Bloomberg

660
00:35:47,640 --> 00:35:52,080
Speaker 1: dot com, wherever your finer podcasts are sold. We love

661
00:35:52,120 --> 00:35:56,680
Speaker 1: your comments, feedback and suggestions right to us at m

662
00:35:56,680 --> 00:35:59,880
Speaker 1: IB podcast at Bloomberg dot net. You can follow me

663
00:36:00,000 --> 00:36:02,320
Speaker 1: on Twitter at rid Halts, or check out my daily

664
00:36:02,360 --> 00:36:07,240
Speaker 1: column at Bloomberg dot com slash Opinion. I'm Barry Ridhults.

665
00:36:07,239 --> 00:36:25,440
Speaker 1: You're listening to Masters in Business on Bloomberg Road. Welcome

666
00:36:25,480 --> 00:36:27,879
Speaker 1: to the podcast. So, Josh, I have to tell you

667
00:36:28,400 --> 00:36:32,480
Speaker 1: I was very much a skeptic. Um a little background.

668
00:36:33,120 --> 00:36:36,680
Speaker 1: So first, I'm a fan of Gilovich for a long

669
00:36:36,719 --> 00:36:38,920
Speaker 1: time when I you could take noise. We don't know

670
00:36:39,360 --> 00:36:41,839
Speaker 1: when I started in this business a hundred years ago

671
00:36:41,880 --> 00:36:44,760
Speaker 1: as a trader. It was before the bad old days

672
00:36:44,840 --> 00:36:48,160
Speaker 1: of behavioral economics had made its way to Wall Street,

673
00:36:48,719 --> 00:36:51,680
Speaker 1: and I found a book by Gilovich, How We Know

674
00:36:51,840 --> 00:36:57,160
Speaker 1: What Isn't? So it was the first mass book, more

675
00:36:57,200 --> 00:37:00,880
Speaker 1: more popular book, not that it was all that popular,

676
00:37:00,880 --> 00:37:03,360
Speaker 1: but it was the first book for a popular audience

677
00:37:04,200 --> 00:37:07,400
Speaker 1: that had an enormous behavioral finance component. To it, so

678
00:37:07,560 --> 00:37:11,120
Speaker 1: I found him absolutely intriguing. He led me down the

679
00:37:11,239 --> 00:37:15,960
Speaker 1: rabbit hole of behavioral finance, and it's been an enormous

680
00:37:16,040 --> 00:37:21,759
Speaker 1: influence on my um professional career because very often, when

681
00:37:21,800 --> 00:37:23,560
Speaker 1: I couldn't figure out what the hell is going on

682
00:37:23,840 --> 00:37:28,120
Speaker 1: according to what the head trader was saying, behavioral finance

683
00:37:28,360 --> 00:37:30,640
Speaker 1: gave a much better answer. And the same is true

684
00:37:30,640 --> 00:37:32,640
Speaker 1: when you're looking at markets, or the economy or what

685
00:37:33,160 --> 00:37:38,120
Speaker 1: people get wrong. So my bias was to say, Traversky Gilvitch,

686
00:37:38,200 --> 00:37:41,239
Speaker 1: these are two legends. Of course they're right. But I

687
00:37:41,320 --> 00:37:44,160
Speaker 1: have to tell you this, having gotten through as much

688
00:37:44,160 --> 00:37:47,200
Speaker 1: of your paper as I could until the formulas started

689
00:37:47,239 --> 00:37:51,279
Speaker 1: to show up. It's a compelling argument that when we

690
00:37:51,320 --> 00:37:55,879
Speaker 1: look at the data set, players on a streak from

691
00:37:56,040 --> 00:38:00,759
Speaker 1: within that data set should have a lower shooting percentage

692
00:38:01,280 --> 00:38:05,880
Speaker 1: following three in a row. Then you would intuitively inspect, expect,

693
00:38:05,920 --> 00:38:09,960
Speaker 1: and when they don't shoot worse, It in and of

694
00:38:09,960 --> 00:38:13,680
Speaker 1: itself is evidence of the hot hand. It's such an

695
00:38:13,800 --> 00:38:18,279
Speaker 1: eloquent and unexpected way to do the analysis of the

696
00:38:18,320 --> 00:38:21,200
Speaker 1: hot hand. I have to ask, how did you guys

697
00:38:21,680 --> 00:38:24,920
Speaker 1: come upon that? I mean, I would never I'm not

698
00:38:24,960 --> 00:38:27,680
Speaker 1: a statistician but I would never have thought, because so

699
00:38:27,760 --> 00:38:31,080
Speaker 1: much of it is so intuitive, I would not have thought, hey,

700
00:38:31,160 --> 00:38:35,640
Speaker 1: let's look at what the expected shot is, because with

701
00:38:35,719 --> 00:38:38,400
Speaker 1: coins it should be fifty fifty. Why would you expect

702
00:38:38,480 --> 00:38:42,600
Speaker 1: it to be anything less following three in a row?

703
00:38:42,400 --> 00:38:45,800
Speaker 1: How did you sort of work your way towards that research?

704
00:38:46,200 --> 00:38:49,319
Speaker 1: So you know, both my cauthor, Adamson ho Ho and

705
00:38:49,360 --> 00:38:54,200
Speaker 1: I we didn't see any problem in that respect um

706
00:38:54,239 --> 00:38:56,560
Speaker 1: with the original papers. So we didn't say, oh, they're

707
00:38:56,560 --> 00:38:59,120
Speaker 1: clearly making a mistake here. No one did, you know,

708
00:38:59,280 --> 00:39:01,200
Speaker 1: since we discuss for this thing, we've gone and we've

709
00:39:01,200 --> 00:39:03,440
Speaker 1: asked statisticians, people that are very good. They look at

710
00:39:03,440 --> 00:39:05,479
Speaker 1: the that test and they say, oh, you know, maybe

711
00:39:05,480 --> 00:39:07,960
Speaker 1: it's underpowered, or they might have some little quibbles, but

712
00:39:08,000 --> 00:39:11,239
Speaker 1: they don't have any expectation that you would shoot worse

713
00:39:11,320 --> 00:39:14,040
Speaker 1: after funeral. In order to do that, you actually have

714
00:39:14,080 --> 00:39:15,960
Speaker 1: to go out and simulate or go sit down and

715
00:39:16,000 --> 00:39:19,000
Speaker 1: really calculate, and so it doesn't strike you in any way.

716
00:39:19,040 --> 00:39:22,600
Speaker 1: So we discovered it. You know, it was a bit

717
00:39:22,640 --> 00:39:25,120
Speaker 1: of a stroke of luck. Um. We were looking at

718
00:39:25,120 --> 00:39:27,600
Speaker 1: the NBA three point contest data. We had to analyze

719
00:39:27,640 --> 00:39:30,359
Speaker 1: it very quickly. Using a method different than the way

720
00:39:30,400 --> 00:39:32,600
Speaker 1: we used it. So we just used the method of

721
00:39:32,600 --> 00:39:35,200
Speaker 1: the original study, which was much quicker to run. So

722
00:39:35,239 --> 00:39:36,880
Speaker 1: we ran that and we found this player who we

723
00:39:36,920 --> 00:39:39,200
Speaker 1: knew was hot and have mentioned that earlier, Craig Hodges,

724
00:39:39,600 --> 00:39:42,440
Speaker 1: and he shot no better after making a few in

725
00:39:42,440 --> 00:39:44,200
Speaker 1: a row, and that just didn't make sense. Was it

726
00:39:44,280 --> 00:39:48,400
Speaker 1: was that a brute force quit down and dirty? So

727
00:39:48,400 --> 00:39:53,040
Speaker 1: so you moved to something a little more um sophisticated,

728
00:39:53,080 --> 00:39:55,680
Speaker 1: what's the better word for this. So so the sophistication

729
00:39:55,719 --> 00:39:57,799
Speaker 1: came later. So we we you know, we just took

730
00:39:57,800 --> 00:39:59,719
Speaker 1: the test that used in the original study and that

731
00:40:00,000 --> 00:40:02,440
Speaker 1: measure and it wasn't showing anything, and we that didn't

732
00:40:02,440 --> 00:40:04,520
Speaker 1: agree with our perception of what we saw in those

733
00:40:04,600 --> 00:40:08,000
Speaker 1: videos and and and and some of the elementary things

734
00:40:08,000 --> 00:40:09,640
Speaker 1: he did, like he hit nineteen in a row at

735
00:40:09,680 --> 00:40:11,640
Speaker 1: one point, never missed more than five, and he was

736
00:40:11,640 --> 00:40:14,120
Speaker 1: around fifty percent shooter, which would be you'd never expect

737
00:40:14,160 --> 00:40:17,160
Speaker 1: from a point and so, okay, nineteen a row is

738
00:40:17,160 --> 00:40:20,120
Speaker 1: astonished as this instance. Yeah, it's incredible. So then we

739
00:40:20,160 --> 00:40:23,200
Speaker 1: went and we said, well, what if he were a coin?

740
00:40:23,239 --> 00:40:25,160
Speaker 1: What if Craig Hodges was a coin? So let's just

741
00:40:25,280 --> 00:40:27,040
Speaker 1: generate his shots as if he was a coin where

742
00:40:27,040 --> 00:40:29,839
Speaker 1: he's really and we repeat this, like, imagine we did

743
00:40:29,840 --> 00:40:32,040
Speaker 1: this many many times, and look what we'd expect from all.

744
00:40:32,200 --> 00:40:34,359
Speaker 1: You know, if we run this many times and we see, oh,

745
00:40:34,400 --> 00:40:37,480
Speaker 1: you'd actually shoot worse after making a funeral, and that

746
00:40:37,520 --> 00:40:41,560
Speaker 1: seems very count We were struck. This doesn't seem like

747
00:40:41,680 --> 00:40:44,600
Speaker 1: it's right. But this is what the analysis is giving us.

748
00:40:44,719 --> 00:40:46,719
Speaker 1: We have to understand this. This is what the data

749
00:40:46,760 --> 00:40:52,120
Speaker 1: is saying. Two things we've discussed. One is after you

750
00:40:52,160 --> 00:40:54,520
Speaker 1: have a streak of six in a row and you

751
00:40:54,560 --> 00:40:57,040
Speaker 1: have a finite number of shots, well now there are

752
00:40:57,080 --> 00:41:01,719
Speaker 1: six less heads in the in groups, so therefore there's

753
00:41:01,719 --> 00:41:05,799
Speaker 1: a higher probability of tails after that. That makes perfect sense, right,

754
00:41:05,840 --> 00:41:08,640
Speaker 1: because you're just changing the remaining data set by what

755
00:41:08,680 --> 00:41:11,160
Speaker 1: you're looking at given a fixed data, given a fixed

756
00:41:11,200 --> 00:41:14,439
Speaker 1: number of coins, fixed number of shots um. And then

757
00:41:14,480 --> 00:41:18,080
Speaker 1: of course mean reversion assumes after a long streak of

758
00:41:18,120 --> 00:41:21,000
Speaker 1: heads you should start to see a streak of tails,

759
00:41:21,360 --> 00:41:23,759
Speaker 1: which like Gambler's fallacy a little bit. So let's let's

760
00:41:23,760 --> 00:41:25,920
Speaker 1: go into that. Explain that. Yes, so I mean the

761
00:41:25,920 --> 00:41:29,160
Speaker 1: gambler's fallacy is this idea that comes out of the casino,

762
00:41:29,239 --> 00:41:32,319
Speaker 1: and it's been known for hundreds of years that if

763
00:41:32,360 --> 00:41:35,320
Speaker 1: you see say five six blacks in a road or

764
00:41:35,440 --> 00:41:38,439
Speaker 1: that table, it feels like that that the red must

765
00:41:38,480 --> 00:41:42,680
Speaker 1: be more likely, right, um, And so people get drawn

766
00:41:42,719 --> 00:41:46,600
Speaker 1: into this and they start betting more. Maybe, but it's

767
00:41:46,640 --> 00:41:51,040
Speaker 1: still fort and the green. Yeah, but it's almost fifty

768
00:41:51,040 --> 00:41:54,760
Speaker 1: fifty regardless, right, So in reality the probabilities haven't changed.

769
00:41:55,440 --> 00:41:57,400
Speaker 1: But but when you but when you look at a

770
00:41:58,040 --> 00:42:00,080
Speaker 1: when you look at a fixed data set that you

771
00:42:00,120 --> 00:42:04,239
Speaker 1: expect to be fifty fifty, not not perspectively at the

772
00:42:04,320 --> 00:42:07,520
Speaker 1: roulette table in real time. But we know that, hey,

773
00:42:07,520 --> 00:42:10,960
Speaker 1: there's a hundred coin flips, we're gonna assume half of

774
00:42:11,000 --> 00:42:16,120
Speaker 1: them are tails and half our heads. After you've had

775
00:42:16,160 --> 00:42:20,440
Speaker 1: a wrong long streak of heads, the assumption is that

776
00:42:20,520 --> 00:42:22,680
Speaker 1: out of that full data set, there should be more

777
00:42:22,680 --> 00:42:25,880
Speaker 1: tails coming up. I'm I'm in real time, it's truly

778
00:42:25,920 --> 00:42:29,520
Speaker 1: the gambler's fallacy. But when you're looking retrospectively with the

779
00:42:29,640 --> 00:42:33,319
Speaker 1: data set, it's basically just a variation of, Hey, you've

780
00:42:33,320 --> 00:42:35,560
Speaker 1: already exhausted a lot of heads, therefore there are more

781
00:42:35,600 --> 00:42:39,400
Speaker 1: tails out there. Yes, exactly and as we mentioned before,

782
00:42:39,440 --> 00:42:41,799
Speaker 1: there's a little bit there's an extra wrinkle on top.

783
00:42:42,120 --> 00:42:44,480
Speaker 1: You know that it determines on how you know, how

784
00:42:44,520 --> 00:42:46,840
Speaker 1: are these streaks ordered. So like when you when you

785
00:42:46,880 --> 00:42:50,160
Speaker 1: pick a pick up a shot, because the previous three

786
00:42:50,480 --> 00:42:53,919
Speaker 1: were heads, the shot you pick up, these are either

787
00:42:54,000 --> 00:42:55,680
Speaker 1: heads or tails, but it's much more likely to be

788
00:42:55,719 --> 00:42:58,680
Speaker 1: tails one because of the heads that were removed too,

789
00:42:58,800 --> 00:43:01,880
Speaker 1: because if it were a tail, you've interrupted the streak

790
00:43:01,960 --> 00:43:04,480
Speaker 1: and you you can't begin until you have to wait

791
00:43:04,560 --> 00:43:07,000
Speaker 1: until you begin. So so there's you're pulling a big

792
00:43:07,120 --> 00:43:10,960
Speaker 1: chunk of the possible selections out, so all the streaks

793
00:43:11,000 --> 00:43:14,640
Speaker 1: come out they're all heads, so you're not picking that one.

794
00:43:14,880 --> 00:43:17,719
Speaker 1: And and plus the total number of heads that you've used,

795
00:43:17,960 --> 00:43:22,320
Speaker 1: so what's left becomes just from a data set group,

796
00:43:22,400 --> 00:43:29,160
Speaker 1: what's less have become a not probability, which is which

797
00:43:29,200 --> 00:43:32,239
Speaker 1: is fast. So you guys are doing this research, at

798
00:43:32,280 --> 00:43:35,839
Speaker 1: what point do you say, holy cow, this is really

799
00:43:35,880 --> 00:43:40,080
Speaker 1: a fascinating discovery, Like it's it's not just a tiny chance.

800
00:43:40,680 --> 00:43:44,400
Speaker 1: Ten percent is a huge number in this sort of

801
00:43:44,520 --> 00:43:46,879
Speaker 1: data series. When did you guys look at each other

802
00:43:46,880 --> 00:43:50,080
Speaker 1: and say, hey, this is something really important. We knew

803
00:43:50,120 --> 00:43:51,799
Speaker 1: it was a big deal of the moment we saw it.

804
00:43:51,960 --> 00:43:53,960
Speaker 1: Really we were on the phone where you didn't set

805
00:43:54,000 --> 00:43:56,600
Speaker 1: yourself this has to be wrong. Ten P. How did

806
00:43:56,640 --> 00:43:59,920
Speaker 1: nobody pick this up? In thirty years? Nobody has seen this?

807
00:44:00,480 --> 00:44:03,239
Speaker 1: So we we had this is two years after we

808
00:44:03,320 --> 00:44:05,160
Speaker 1: had begun the project. Well maybe not that long, but

809
00:44:05,200 --> 00:44:08,640
Speaker 1: almost two years, and we had read every paper in

810
00:44:08,680 --> 00:44:10,880
Speaker 1: the literature, so we knew no one had had had

811
00:44:10,920 --> 00:44:12,480
Speaker 1: nobody had seen this, no, no one had said so

812
00:44:12,480 --> 00:44:14,080
Speaker 1: we knew it was a big deal for that literature.

813
00:44:14,120 --> 00:44:18,480
Speaker 1: So the only question we had is how new. Well,

814
00:44:18,480 --> 00:44:20,400
Speaker 1: I mean, we'd run you know, we knew the you

815
00:44:20,400 --> 00:44:22,640
Speaker 1: know we can trust the computer, right, And of course

816
00:44:22,640 --> 00:44:24,040
Speaker 1: you have to make sure you didn't make an error

817
00:44:24,040 --> 00:44:25,840
Speaker 1: in your code. You have to sit down and do

818
00:44:25,880 --> 00:44:27,400
Speaker 1: the simple example to make sure you didn't do a

819
00:44:27,400 --> 00:44:29,360
Speaker 1: calculation there. So once we did that with okay, this

820
00:44:29,440 --> 00:44:32,239
Speaker 1: is clearly a true thing. Now the only question is

821
00:44:32,640 --> 00:44:35,080
Speaker 1: did anyone know this about coin flips before? Is this

822
00:44:35,120 --> 00:44:39,120
Speaker 1: a new discovery about coin flips? And yes, there's some

823
00:44:39,160 --> 00:44:41,200
Speaker 1: mathematical things that are somewhat related, but no, it was

824
00:44:41,239 --> 00:44:42,839
Speaker 1: even new in that dimension, So we knew we had

825
00:44:42,880 --> 00:44:45,160
Speaker 1: something really big and that was exciting because you have

826
00:44:45,239 --> 00:44:48,000
Speaker 1: this moment where you're the only person in the world

827
00:44:48,000 --> 00:44:50,399
Speaker 1: that knows something. It's kind of it's an exciting moment.

828
00:44:50,440 --> 00:44:52,160
Speaker 1: I feel that way every day I wake up and

829
00:44:52,200 --> 00:44:55,640
Speaker 1: I have that sensation, so I can appreciate you probably

830
00:44:55,680 --> 00:44:58,680
Speaker 1: not as solidly based as as yours, at least that's

831
00:44:58,680 --> 00:45:02,040
Speaker 1: what my wife wee. So that's amazing. You guys come

832
00:45:02,120 --> 00:45:07,000
Speaker 1: up with this incredible breakthrough. Nobody has has found this.

833
00:45:07,920 --> 00:45:12,440
Speaker 1: It's been decades and it's it's been widely accepted. It's

834
00:45:12,480 --> 00:45:17,000
Speaker 1: become part of the cannon. But it's classic confirmation bias,

835
00:45:17,040 --> 00:45:21,239
Speaker 1: which is so um reflexive and meta. There is a

836
00:45:21,360 --> 00:45:24,879
Speaker 1: study that says people are fooled by randomness and think

837
00:45:24,920 --> 00:45:28,960
Speaker 1: there are streaks, which turns out perhaps to be confirmation

838
00:45:29,040 --> 00:45:33,400
Speaker 1: biased by behaviorists who are warning people against being fooled

839
00:45:33,400 --> 00:45:36,600
Speaker 1: by randomness and seeing what they want to see. It's

840
00:45:36,640 --> 00:45:40,359
Speaker 1: got a little bit of Mandel brought reflectiveness built into it.

841
00:45:40,360 --> 00:45:43,520
Speaker 1: It's it's quite amazing. Yes, you know, in a sense

842
00:45:43,600 --> 00:45:47,560
Speaker 1: that mistake proves kind of the spirit of the general

843
00:45:47,600 --> 00:45:50,920
Speaker 1: point about misinterpreting randomness. Even the best of us, the

844
00:45:50,960 --> 00:45:55,160
Speaker 1: best researchers there are out there still make these mistakes

845
00:45:55,760 --> 00:45:58,720
Speaker 1: due to randomness, and while saying others are making the mistake,

846
00:45:59,400 --> 00:46:04,000
Speaker 1: you're making the mistake even within. So they accidentally proved

847
00:46:04,080 --> 00:46:07,960
Speaker 1: their point, which is it's very easy to be fooled

848
00:46:08,000 --> 00:46:11,480
Speaker 1: by a random data set into thinking there's a broader

849
00:46:11,480 --> 00:46:18,120
Speaker 1: conclusion there, until subsequent research discovers that, hey, this isn't

850
00:46:18,160 --> 00:46:20,759
Speaker 1: quite as random as you think it is. There's a

851
00:46:20,800 --> 00:46:25,439
Speaker 1: ten gap between true randomness and the remaining data set.

852
00:46:25,880 --> 00:46:29,160
Speaker 1: That that's quite that's quite fascinating. So you guys look

853
00:46:29,200 --> 00:46:31,959
Speaker 1: at each other and say, hey, we're onto something real.

854
00:46:32,480 --> 00:46:34,759
Speaker 1: How did it progress from there? What year was this?

855
00:46:34,760 --> 00:46:40,320
Speaker 1: This was? This was February. Found this and we knew

856
00:46:40,560 --> 00:46:43,800
Speaker 1: so we knew it was important. So, um, we presented

857
00:46:43,800 --> 00:46:46,000
Speaker 1: our work and when you see the eyes light up,

858
00:46:46,040 --> 00:46:48,440
Speaker 1: you realize it's even bigger than you thought it was.

859
00:46:48,800 --> 00:46:50,799
Speaker 1: And then you realize, hey, wait a minute, we don't

860
00:46:50,840 --> 00:46:54,840
Speaker 1: have the paper yet, and now other people know about it.

861
00:46:54,840 --> 00:46:58,279
Speaker 1: Who did you present it to originally? Um? So at

862
00:46:58,280 --> 00:47:01,719
Speaker 1: Oxford University. That was the first review, right, and you

863
00:47:01,920 --> 00:47:04,640
Speaker 1: know you see the eyes light up in the room. Um,

864
00:47:04,680 --> 00:47:07,480
Speaker 1: are you genuinely concerned at that moment? Oh? Someone's gonna

865
00:47:07,520 --> 00:47:10,279
Speaker 1: try and beat us to publication, and so we put

866
00:47:10,320 --> 00:47:13,800
Speaker 1: everything aside and we just we we went to the grind.

867
00:47:13,840 --> 00:47:16,000
Speaker 1: We within two months we had the paper and no

868
00:47:16,040 --> 00:47:17,680
Speaker 1: one was going to catch you. At that point you

869
00:47:17,840 --> 00:47:19,719
Speaker 1: would enough of the head start and you were the

870
00:47:19,760 --> 00:47:23,120
Speaker 1: original people who found this. So two months later the

871
00:47:23,120 --> 00:47:26,360
Speaker 1: preliminary papers come out. We put online. You put you

872
00:47:26,400 --> 00:47:29,680
Speaker 1: posted online n b R and everywhere else or wherever

873
00:47:29,760 --> 00:47:32,280
Speaker 1: you know, just get that time stamp right where wherever

874
00:47:32,440 --> 00:47:36,760
Speaker 1: finer white papers are are sold. Um. And so that's

875
00:47:37,680 --> 00:47:43,600
Speaker 1: what April of the paper went online June. What's the

876
00:47:43,640 --> 00:47:51,120
Speaker 1: response to that? Um? The response was big. So statistician

877
00:47:51,160 --> 00:47:54,160
Speaker 1: at Columbia University, Andrew Gelman, who has this blog and

878
00:47:54,200 --> 00:47:56,680
Speaker 1: everybody's heard of Andrew Galman. Well, or let me rephrase that,

879
00:47:56,960 --> 00:48:00,560
Speaker 1: anybody who's interested in statistics knows who Gellman It Columbia

880
00:48:00,640 --> 00:48:04,280
Speaker 1: is fair, fair statement. He's at the crossroads of pretty

881
00:48:04,360 --> 00:48:07,000
Speaker 1: much all the social sciences, sciences when it comes to

882
00:48:07,080 --> 00:48:11,280
Speaker 1: data and statistics, right, and so getting attention from Andrew

883
00:48:11,320 --> 00:48:14,920
Speaker 1: Gelman huge, It's huge, and that that was you know,

884
00:48:15,120 --> 00:48:18,520
Speaker 1: high fives all around. Yeah, but it's also scary when

885
00:48:18,520 --> 00:48:20,839
Speaker 1: you get attention from Andrew Gelman, because if you made

886
00:48:20,840 --> 00:48:24,000
Speaker 1: a mistake, it's open peer review season. They're getting in

887
00:48:24,040 --> 00:48:26,000
Speaker 1: there in the comments. He'll get you, you know, like

888
00:48:26,120 --> 00:48:28,960
Speaker 1: they're just having fun. They love talking about data and

889
00:48:29,400 --> 00:48:31,239
Speaker 1: and they're not gonna worry about how you feel about

890
00:48:31,280 --> 00:48:34,160
Speaker 1: it because they're just interested in the main points, like

891
00:48:34,200 --> 00:48:36,640
Speaker 1: what did the statistics say? And you're you know, you're

892
00:48:36,680 --> 00:48:38,480
Speaker 1: sitting there sweat and ball. It's hoping you got you

893
00:48:38,520 --> 00:48:42,359
Speaker 1: didn't make a mistake somewhere. It's at that level. This

894
00:48:42,400 --> 00:48:45,120
Speaker 1: isn't you know, Twitter fights and ad homin Hum attacks.

895
00:48:45,280 --> 00:48:48,239
Speaker 1: It's hey, let's get into the math. Let's see if

896
00:48:48,280 --> 00:48:51,239
Speaker 1: they did they're crunching their numbers correctly. Let's see if

897
00:48:51,280 --> 00:48:53,520
Speaker 1: we can find an error in their modeling. What what

898
00:48:53,600 --> 00:48:58,480
Speaker 1: did Gelman discover? So Gelman went and did the work himself,

899
00:48:59,120 --> 00:49:02,160
Speaker 1: and he found what he found agreed with what we found,

900
00:49:02,360 --> 00:49:04,560
Speaker 1: and so he said, hey, guess what. There is a

901
00:49:04,600 --> 00:49:07,879
Speaker 1: hot hand that was his post and then it's kind

902
00:49:07,880 --> 00:49:09,799
Speaker 1: of snowballed from there. That's it. So then there's a

903
00:49:09,800 --> 00:49:12,640
Speaker 1: Wall Street Journal piece on it, and then there was

904
00:49:12,680 --> 00:49:15,440
Speaker 1: an ESPN or a sports illustrator was one of the

905
00:49:15,440 --> 00:49:19,600
Speaker 1: sports are there, yeah, there there was ridiculous, like we're like, okay,

906
00:49:19,600 --> 00:49:22,880
Speaker 1: when's the fifteen minutes gonna end? But I guess that,

907
00:49:22,960 --> 00:49:25,320
Speaker 1: you know, the news world is so kind of balkanized

908
00:49:25,360 --> 00:49:28,800
Speaker 1: by this point that like its not from subject to subject,

909
00:49:29,040 --> 00:49:31,719
Speaker 1: it just kept rotating. And I saw something, and you

910
00:49:31,760 --> 00:49:36,160
Speaker 1: guys published another a number of fair I have to say,

911
00:49:36,640 --> 00:49:42,839
Speaker 1: you're published popular stuff. I think you undersold the math

912
00:49:42,960 --> 00:49:45,680
Speaker 1: on this because it's not that you dumbed it down,

913
00:49:46,320 --> 00:49:52,640
Speaker 1: it's that you were so circumspect. And so maybe modest

914
00:49:52,800 --> 00:49:56,800
Speaker 1: is the right word. Like if I'm a different person

915
00:49:56,840 --> 00:49:59,919
Speaker 1: than you, I would have written written something that said, dude,

916
00:50:00,040 --> 00:50:03,480
Speaker 1: just listen up the whole no hot hand things. Let

917
00:50:03,520 --> 00:50:06,319
Speaker 1: us show you why that's not true. Here's the math.

918
00:50:06,840 --> 00:50:11,520
Speaker 1: It's ten. It's a giant impact, and here's why. Like

919
00:50:11,600 --> 00:50:14,920
Speaker 1: I thought, you guys were very circumspect in your what

920
00:50:15,080 --> 00:50:18,600
Speaker 1: was it the conversation or the yeah, the Australian conversation. Yes,

921
00:50:18,719 --> 00:50:22,920
Speaker 1: that was like a fairly modest discussion. You know. I

922
00:50:22,960 --> 00:50:25,880
Speaker 1: would have been like, hey, pay attention to this. We're

923
00:50:25,960 --> 00:50:29,600
Speaker 1: changing an understanding of sports streaking nous, this is a

924
00:50:29,600 --> 00:50:34,600
Speaker 1: big deal. What other applications are there? Of of the

925
00:50:34,680 --> 00:50:39,520
Speaker 1: finding of both the flips of coins and the streak

926
00:50:39,600 --> 00:50:43,520
Speaker 1: nous of shooters. Where else can this be applied? Are

927
00:50:43,560 --> 00:50:47,239
Speaker 1: there other uses of this mathematical? Why should call its

928
00:50:47,239 --> 00:50:52,040
Speaker 1: statistical observation? Yes? So the bias that that we found

929
00:50:52,120 --> 00:50:55,759
Speaker 1: has uh and it can it can manifest itself in

930
00:50:55,920 --> 00:50:58,200
Speaker 1: many areas. So it's not just about time, right, So

931
00:50:58,440 --> 00:51:00,600
Speaker 1: we're looking at like how how you did recently? Does

932
00:51:00,600 --> 00:51:02,399
Speaker 1: that affect how you do in the future or how

933
00:51:02,400 --> 00:51:04,560
Speaker 1: you do next? Right? If we found some biases there,

934
00:51:04,560 --> 00:51:07,360
Speaker 1: But it's not it's not about time. It's essentially about

935
00:51:07,360 --> 00:51:10,480
Speaker 1: space because you're looking at data and we represent time

936
00:51:10,480 --> 00:51:13,680
Speaker 1: with space because we have period one, period two, period three,

937
00:51:13,680 --> 00:51:15,520
Speaker 1: they're all next to each other, and so you have

938
00:51:15,560 --> 00:51:19,680
Speaker 1: this kind of one dimensional spatial thing continuing the line.

939
00:51:19,719 --> 00:51:21,680
Speaker 1: But it can go in either directions. So it's not

940
00:51:21,760 --> 00:51:24,400
Speaker 1: you know, time's arrow that's determining it. Right. If I

941
00:51:24,480 --> 00:51:26,680
Speaker 1: hit three in a row, the chance that the previous

942
00:51:26,719 --> 00:51:28,960
Speaker 1: one that just preceded that streak as it heads is

943
00:51:29,000 --> 00:51:33,439
Speaker 1: actually lower two um for the exact same reason, which

944
00:51:33,480 --> 00:51:36,480
Speaker 1: means that the actual streaking nous of the player isn't

945
00:51:36,520 --> 00:51:40,080
Speaker 1: relevant to the prior one, even though we would expect

946
00:51:40,120 --> 00:51:43,359
Speaker 1: it to be relevant to the subsequent one. It's all

947
00:51:43,400 --> 00:51:48,560
Speaker 1: the same statistical data set prior less, less heads in

948
00:51:48,600 --> 00:51:51,840
Speaker 1: the remaining pool, etcetera. So you can extend this beyond

949
00:51:51,880 --> 00:51:54,040
Speaker 1: time and talk about space. Right, So if you're interested

950
00:51:54,040 --> 00:51:57,840
Speaker 1: in you know, if I'm surrounded by you know, red people,

951
00:51:58,800 --> 00:52:01,319
Speaker 1: am I more likely to be blue? You might go

952
00:52:01,400 --> 00:52:03,040
Speaker 1: and hey, let's look at the data set. And this

953
00:52:03,200 --> 00:52:07,200
Speaker 1: is the ping pun bulls in the vase statistical problem. Yeah,

954
00:52:07,239 --> 00:52:10,520
Speaker 1: so you know, the people study segregation and clustering, you know,

955
00:52:10,719 --> 00:52:12,920
Speaker 1: and where people live and things like this, and and see,

956
00:52:12,920 --> 00:52:14,960
Speaker 1: you might go into data set and use this intuitive

957
00:52:14,960 --> 00:52:16,719
Speaker 1: measure like let's see if I'm more likely to be

958
00:52:16,719 --> 00:52:19,640
Speaker 1: blue if I'm surrounded by reds. Um, you have the

959
00:52:19,719 --> 00:52:22,560
Speaker 1: same issue here now, you know, if I were a blue,

960
00:52:22,760 --> 00:52:26,760
Speaker 1: I've kind of excluded other possibilities of being surrounded by reds.

961
00:52:26,760 --> 00:52:29,200
Speaker 1: Wherever that blue is what it actually makes blue more

962
00:52:29,280 --> 00:52:31,960
Speaker 1: likely for some of the same reasons why we have

963
00:52:32,040 --> 00:52:34,279
Speaker 1: this bias, you know, when we're talking about time, and

964
00:52:34,360 --> 00:52:38,160
Speaker 1: so there are potentially many other areas where bias is

965
00:52:38,200 --> 00:52:40,920
Speaker 1: similar to this could could manifest themselves. It might be

966
00:52:40,960 --> 00:52:43,480
Speaker 1: stumbling it so I remember a couple of years ago

967
00:52:43,640 --> 00:52:48,600
Speaker 1: the cancer clusters around power lines, um, which a lot

968
00:52:48,600 --> 00:52:52,040
Speaker 1: of statisticians came out and said, well, no, this is

969
00:52:52,160 --> 00:52:55,880
Speaker 1: just you know, the heads and tails problem. Again, you

970
00:52:55,960 --> 00:52:58,800
Speaker 1: have a lot all these non clusters around other power lines.

971
00:52:58,880 --> 00:53:01,400
Speaker 1: So if it's a cause element, why is it causing

972
00:53:01,440 --> 00:53:03,640
Speaker 1: it here but not a half mile down the same

973
00:53:03,680 --> 00:53:07,279
Speaker 1: power line. It's just a random aggregation of data, and

974
00:53:07,320 --> 00:53:10,560
Speaker 1: you're seeing something that it's of course you're gonna get

975
00:53:10,560 --> 00:53:12,319
Speaker 1: ten heads in a row if you flip a coin

976
00:53:12,360 --> 00:53:16,279
Speaker 1: a million times. That's all you're seeing. Do you have

977
00:53:16,320 --> 00:53:20,480
Speaker 1: an application to those sort of of cognitive issues, um,

978
00:53:20,640 --> 00:53:23,520
Speaker 1: So we haven't found the specific application, to be honest,

979
00:53:23,520 --> 00:53:26,319
Speaker 1: we haven't scoured that literature. UM. You know, we have

980
00:53:26,400 --> 00:53:29,799
Speaker 1: found papers that have measures of clustering, like how likely

981
00:53:29,840 --> 00:53:31,840
Speaker 1: am I going to live next to someone of who's

982
00:53:31,920 --> 00:53:34,440
Speaker 1: like me, you know versus not like me depending on

983
00:53:34,480 --> 00:53:37,080
Speaker 1: who's around? Um. And there are some measures that are

984
00:53:37,120 --> 00:53:39,440
Speaker 1: biased for a similar reason. Um that we have this

985
00:53:39,520 --> 00:53:42,239
Speaker 1: bias now in the councilor cluster one. UM. You know

986
00:53:42,320 --> 00:53:44,600
Speaker 1: that's that's a little bit different. UM. And it's and

987
00:53:44,640 --> 00:53:46,359
Speaker 1: it's because of you know what you say is it's

988
00:53:46,400 --> 00:53:48,719
Speaker 1: kind of blade of grass fallacy that you know. You know,

989
00:53:48,960 --> 00:53:51,440
Speaker 1: there's lots of blade of grass. You know, you shoot

990
00:53:51,440 --> 00:53:53,760
Speaker 1: a speck of water and it hits the blade of grass,

991
00:53:53,760 --> 00:53:55,799
Speaker 1: the blade of grass. Oh look, I'm so lucky it

992
00:53:55,920 --> 00:53:57,960
Speaker 1: was coming for me. What had to hit some blade

993
00:53:58,000 --> 00:54:00,320
Speaker 1: of grace? Right now, someone's gonna win the lot. Someone's

994
00:54:00,360 --> 00:54:01,840
Speaker 1: got to win the lot everyone. You know, the chances

995
00:54:01,880 --> 00:54:07,719
Speaker 1: that somebody wins the lottery is super high. Not so much. Yeah,

996
00:54:07,920 --> 00:54:12,080
Speaker 1: so that's interesting. So before I get to my favorite questions,

997
00:54:12,120 --> 00:54:14,200
Speaker 1: I asked all my guests, I have to ask you

998
00:54:14,320 --> 00:54:19,000
Speaker 1: what else do you guys um working on? What are

999
00:54:19,000 --> 00:54:22,400
Speaker 1: the research is coming from the minds that brought you

1000
00:54:23,120 --> 00:54:26,040
Speaker 1: um proof that the hot hands exists? Well, you know,

1001
00:54:26,080 --> 00:54:29,480
Speaker 1: so in our world, it's very tempting to move on

1002
00:54:29,560 --> 00:54:33,359
Speaker 1: to the next one before you know, finishing what you started, right,

1003
00:54:33,400 --> 00:54:36,680
Speaker 1: so you have not exhausted everything out of this one

1004
00:54:37,760 --> 00:54:39,319
Speaker 1: one piece of day, right, So we have we have

1005
00:54:39,400 --> 00:54:42,040
Speaker 1: a lot of kind of uh you know, I used

1006
00:54:42,080 --> 00:54:43,759
Speaker 1: to dot, he used to cross. But you know a

1007
00:54:43,760 --> 00:54:45,120
Speaker 1: little bit more than that. You know, you want to

1008
00:54:45,160 --> 00:54:47,239
Speaker 1: you want to finish and get the message out, but

1009
00:54:47,360 --> 00:54:49,960
Speaker 1: also share, you know, the other insights that we have,

1010
00:54:50,040 --> 00:54:52,480
Speaker 1: because you know, they come out of the same you know,

1011
00:54:52,640 --> 00:54:54,680
Speaker 1: so this is our you know, say, the main insight,

1012
00:54:54,760 --> 00:54:57,000
Speaker 1: but there are other very subtle and interesting insights that

1013
00:54:57,040 --> 00:54:59,839
Speaker 1: we have because you know, when you master something, yeah,

1014
00:54:59,840 --> 00:55:02,000
Speaker 1: and you come back after working on something for a while,

1015
00:55:02,000 --> 00:55:04,160
Speaker 1: there's a lot to share. So tell us some what

1016
00:55:04,320 --> 00:55:08,200
Speaker 1: what other insights can be derived from from the hot

1017
00:55:08,280 --> 00:55:11,680
Speaker 1: hand papers. Yeah, so there's another result in in that study,

1018
00:55:11,960 --> 00:55:15,640
Speaker 1: um um Gilvich introversity study that Gilvich mentioned in the

1019
00:55:15,640 --> 00:55:18,839
Speaker 1: book that you talked about earlier, which is okay, so

1020
00:55:19,400 --> 00:55:21,640
Speaker 1: they kind of got that people, uh you know, that

1021
00:55:21,680 --> 00:55:24,759
Speaker 1: they measured hot hand in a certain way, and they realize, well,

1022
00:55:24,800 --> 00:55:27,600
Speaker 1: maybe we're not capturing everything that means about hot hand,

1023
00:55:27,600 --> 00:55:30,400
Speaker 1: and maybe some players are are seeing something that we,

1024
00:55:30,560 --> 00:55:34,160
Speaker 1: the statisticians, the econometricians you know, aren't measuring. And they

1025
00:55:34,480 --> 00:55:36,799
Speaker 1: went and had people predict and bet on outcomes and

1026
00:55:36,800 --> 00:55:40,120
Speaker 1: they found that they their bets don't really correlate with

1027
00:55:40,120 --> 00:55:43,759
Speaker 1: the outcomes, and so that's kind of evidence. Well, okay, well,

1028
00:55:43,800 --> 00:55:45,880
Speaker 1: even if we're not measuring everything, look if the players

1029
00:55:45,920 --> 00:55:48,480
Speaker 1: are seeing something you think they would bet successfully, and

1030
00:55:48,480 --> 00:55:50,839
Speaker 1: that you could also take that as evidence that now

1031
00:55:50,920 --> 00:55:53,239
Speaker 1: that there is a hot hand there, well, it's at

1032
00:55:53,320 --> 00:55:55,840
Speaker 1: least it's evidence that there's somehow not using exploiting it

1033
00:55:55,880 --> 00:55:58,200
Speaker 1: in a profitable way. But there was actually a mistake

1034
00:55:59,040 --> 00:56:01,840
Speaker 1: as well in in that in that analysis, which is,

1035
00:56:02,360 --> 00:56:05,000
Speaker 1: even if someone were perfect at detecting the hot hand,

1036
00:56:05,040 --> 00:56:07,080
Speaker 1: they knew you know, you can imagine Ann and Bob.

1037
00:56:07,080 --> 00:56:10,319
Speaker 1: Bob's a shooter and his you know, predictor she's observing Bob.

1038
00:56:10,400 --> 00:56:12,799
Speaker 1: She knows when Bob's hot, and whenever Bob's hot, she's

1039
00:56:12,800 --> 00:56:15,360
Speaker 1: going to predict that Bob's going to make the shot. Now,

1040
00:56:15,600 --> 00:56:18,279
Speaker 1: you would expect if she's good, then that good, then

1041
00:56:18,360 --> 00:56:21,080
Speaker 1: her bets, her predictions are going to correlate really well,

1042
00:56:21,400 --> 00:56:24,640
Speaker 1: um with Bob's outcome. But actually you wouldn't expect that.

1043
00:56:24,680 --> 00:56:27,680
Speaker 1: And that's that's another counterintuitive thing is that while she's

1044
00:56:27,760 --> 00:56:30,399
Speaker 1: perfect at detecting his state, the outcome of the state

1045
00:56:30,480 --> 00:56:32,759
Speaker 1: is noisy. You're just getting one draw from Bob's earns.

1046
00:56:32,760 --> 00:56:35,200
Speaker 1: So even if Bob's moving from a seventy to eight

1047
00:56:36,160 --> 00:56:39,640
Speaker 1: probability shooter, if you only take one draw from that urn,

1048
00:56:40,200 --> 00:56:42,560
Speaker 1: you're not getting a very good signal on Bob's state

1049
00:56:43,640 --> 00:56:45,600
Speaker 1: you need a lot. Yeah, and so even if you're

1050
00:56:45,600 --> 00:56:47,799
Speaker 1: getting many predictions from Ann and Bob, you're still only

1051
00:56:47,840 --> 00:56:51,040
Speaker 1: getting you know, one draw on each one. And so

1052
00:56:52,120 --> 00:56:54,760
Speaker 1: the evidence that they have there was actually enough evidence

1053
00:56:54,800 --> 00:56:58,040
Speaker 1: that's consistent with and being very good at detecting it

1054
00:56:58,320 --> 00:57:01,120
Speaker 1: and actually with you, Rhianna. Is the data you find

1055
00:57:01,160 --> 00:57:05,799
Speaker 1: that Bob shoots seven around seven percentage points better when

1056
00:57:05,840 --> 00:57:08,960
Speaker 1: and predicts that he's gonna make it, you miss it.

1057
00:57:09,080 --> 00:57:11,200
Speaker 1: So in their data they have real people that are

1058
00:57:11,239 --> 00:57:14,200
Speaker 1: paid basketball players are betting on each other's shots. And

1059
00:57:14,800 --> 00:57:18,439
Speaker 1: that's the evidence that we find that's quite interesting. That

1060
00:57:18,600 --> 00:57:23,200
Speaker 1: sounds that betting on the outcome of a shot sounds

1061
00:57:23,320 --> 00:57:27,560
Speaker 1: very much like fund managers selecting stocks for a portfolio.

1062
00:57:28,120 --> 00:57:33,439
Speaker 1: Have you applied any of the hot hands? Two? How

1063
00:57:33,440 --> 00:57:36,840
Speaker 1: do fund managers do when they're on a hot streak

1064
00:57:36,960 --> 00:57:39,120
Speaker 1: or a cold streak? And there's a ton of mean

1065
00:57:39,200 --> 00:57:42,760
Speaker 1: reversion in that data series, right, So we haven't gone

1066
00:57:42,800 --> 00:57:45,240
Speaker 1: and analyzed and the mechanisms for being hot in the

1067
00:57:45,280 --> 00:57:47,200
Speaker 1: financial world are going to be quite different than in

1068
00:57:47,200 --> 00:57:49,880
Speaker 1: the basketball world. Right. So, you know, one way of thinking,

1069
00:57:50,000 --> 00:57:52,600
Speaker 1: you look for sec indictments, you look for no I'm

1070
00:57:52,600 --> 00:57:57,600
Speaker 1: just kidding, um, for sure. It becomes so affected by

1071
00:57:57,640 --> 00:58:01,720
Speaker 1: such large macro things it's hard to give credit or

1072
00:58:01,840 --> 00:58:04,000
Speaker 1: or not or your You know, your model of the

1073
00:58:04,040 --> 00:58:07,280
Speaker 1: world happens to be uniquely fit the current situation, and

1074
00:58:07,320 --> 00:58:09,600
Speaker 1: you recognize that, but you know that may or may

1075
00:58:09,640 --> 00:58:11,320
Speaker 1: not be temperary, and that's going to know that that

1076
00:58:11,360 --> 00:58:14,040
Speaker 1: would probably expire, but that's a very different mechanism, and

1077
00:58:14,160 --> 00:58:16,320
Speaker 1: say how it would emergency a basketball game? So I

1078
00:58:16,360 --> 00:58:18,480
Speaker 1: interrupted you, what what else did you do you see

1079
00:58:18,480 --> 00:58:23,400
Speaker 1: as an application of this elsewhere, um, an application of our,

1080
00:58:23,800 --> 00:58:27,120
Speaker 1: of your of what you've discovered to to the world

1081
00:58:27,120 --> 00:58:30,480
Speaker 1: of finance, to the world of finance. So the immediate

1082
00:58:30,520 --> 00:58:34,960
Speaker 1: applications maybe not so much. But if you think about

1083
00:58:35,520 --> 00:58:38,360
Speaker 1: people picking stocks, let's say not so much investing, but

1084
00:58:38,400 --> 00:58:41,200
Speaker 1: someone wants to prove that they're good, um at predicting

1085
00:58:41,200 --> 00:58:42,720
Speaker 1: when a stock is going to go up or down.

1086
00:58:43,080 --> 00:58:47,000
Speaker 1: You have to pay attention not too how often they're right,

1087
00:58:47,520 --> 00:58:50,440
Speaker 1: but how much money they make when they're right and

1088
00:58:50,480 --> 00:58:53,720
Speaker 1: when they're wrong. Because it's very easy to gain these things.

1089
00:58:53,720 --> 00:58:56,640
Speaker 1: So if I were to say, let's say it's fifty fifty,

1090
00:58:56,680 --> 00:58:58,920
Speaker 1: I want to prove that I'm good at predicting coin flips.

1091
00:58:59,320 --> 00:59:01,680
Speaker 1: And so if every month you know a coin is flipped,

1092
00:59:01,680 --> 00:59:04,240
Speaker 1: each day stock goes up or down. But I only

1093
00:59:04,320 --> 00:59:07,520
Speaker 1: bet when there's three heads in a row. When the

1094
00:59:07,520 --> 00:59:09,040
Speaker 1: stock goes up three times in a row, and I

1095
00:59:09,240 --> 00:59:12,240
Speaker 1: bet it's gonna go down. Right in any given month,

1096
00:59:13,120 --> 00:59:16,840
Speaker 1: I'm gonna be right more often than I'm wrong, And

1097
00:59:16,920 --> 00:59:19,720
Speaker 1: so I can game you know if you don't, if you,

1098
00:59:19,760 --> 00:59:21,640
Speaker 1: if you, if you brack it to the month level,

1099
00:59:22,160 --> 00:59:24,440
Speaker 1: I'm gonna be more often right in certain months, and

1100
00:59:24,480 --> 00:59:26,000
Speaker 1: it's gonna look like I'm doing well. But the thing

1101
00:59:26,080 --> 00:59:29,720
Speaker 1: you haven't paid attention to is how often I was

1102
00:59:29,840 --> 00:59:31,520
Speaker 1: right and half when I was wrong in the months

1103
00:59:31,520 --> 00:59:34,920
Speaker 1: that I did poorly. And so if I'm always betting

1104
00:59:34,960 --> 00:59:36,880
Speaker 1: that it's gonna go down when there's a few ups

1105
00:59:36,880 --> 00:59:38,600
Speaker 1: in a row, there's gonna be those months when it

1106
00:59:38,680 --> 00:59:41,840
Speaker 1: keeps going down, right, But you know there's gonna be

1107
00:59:41,880 --> 00:59:43,280
Speaker 1: a few of those months, and there's gonna be many

1108
00:59:43,320 --> 00:59:45,440
Speaker 1: months when I did well, but I didn't predict very

1109
00:59:45,480 --> 00:59:47,560
Speaker 1: many times, and so you're not controlling for how often

1110
00:59:47,600 --> 00:59:49,840
Speaker 1: I predicted, and so it looks like like I do

1111
00:59:49,920 --> 00:59:52,120
Speaker 1: really well. But if I were to bet I wouldn't

1112
00:59:52,120 --> 00:59:54,000
Speaker 1: be making any money because I'd be losing a lot

1113
00:59:54,000 --> 00:59:55,920
Speaker 1: of money in the months where I didn't predict that well,

1114
00:59:56,200 --> 00:59:58,880
Speaker 1: only winning a little bit of money in the months

1115
00:59:58,880 --> 01:00:01,240
Speaker 1: that I predicted well. The interesting thing is if you

1116
01:00:01,320 --> 01:00:06,000
Speaker 1: talk to active traders UM who have been successful, they're

1117
01:00:06,000 --> 01:00:09,600
Speaker 1: not aiming for fifty fifty. They're aiming for those opportunities

1118
01:00:09,920 --> 01:00:12,600
Speaker 1: where a trade becomes a winner and they don't sell

1119
01:00:12,640 --> 01:00:16,600
Speaker 1: too early. So it's not your banning average, it's how

1120
01:00:16,680 --> 01:00:19,040
Speaker 1: far the ball goes when you actually hit it. That's right,

1121
01:00:19,080 --> 01:00:23,240
Speaker 1: Meaning you could have a winning trading record, but in

1122
01:00:23,360 --> 01:00:26,160
Speaker 1: terms of percentage of winning trades, but in terms of

1123
01:00:26,200 --> 01:00:29,000
Speaker 1: dollars one and lost, those twenty more than make up

1124
01:00:29,040 --> 01:00:31,520
Speaker 1: for the remaining eighty. And I always find a lot

1125
01:00:31,560 --> 01:00:34,840
Speaker 1: of new traders don't understand that. They think they're hitting

1126
01:00:34,840 --> 01:00:37,800
Speaker 1: for percentage, but they're not. They're heading for distance. To

1127
01:00:37,880 --> 01:00:41,320
Speaker 1: bring in a different sports metaphor anything else you want

1128
01:00:41,320 --> 01:00:44,240
Speaker 1: to share about UM, the research, or what you guys

1129
01:00:44,320 --> 01:00:46,640
Speaker 1: might have coming out in the near future, you and

1130
01:00:46,680 --> 01:00:50,040
Speaker 1: your co author UM. Off the top of my head, no,

1131
01:00:50,160 --> 01:00:52,600
Speaker 1: I think we The one thing that we will say

1132
01:00:52,720 --> 01:00:56,200
Speaker 1: is actually there's one thing, So it's not simply that

1133
01:00:56,240 --> 01:00:59,600
Speaker 1: we found that the original analysis was vowed in the

1134
01:00:59,600 --> 01:01:02,360
Speaker 1: original conclusions were invalid. You know, if you go back

1135
01:01:02,400 --> 01:01:06,400
Speaker 1: and you reanalyze that data, you find that players shoot

1136
01:01:06,400 --> 01:01:08,720
Speaker 1: a lot better. But it's not simply in that data set.

1137
01:01:08,880 --> 01:01:11,120
Speaker 1: So we've gone and collected many other data sets that

1138
01:01:11,200 --> 01:01:14,280
Speaker 1: replicated their original study got the same conclusion because they

1139
01:01:14,360 --> 01:01:16,920
Speaker 1: had the same bias that the original study had. And

1140
01:01:16,960 --> 01:01:18,720
Speaker 1: so when you go back and you fix that, you

1141
01:01:18,760 --> 01:01:22,720
Speaker 1: find evidence everywhere, and and and that's we have a

1142
01:01:22,760 --> 01:01:26,600
Speaker 1: paper that we're finishing that's showing how robust our conclusions were.

1143
01:01:27,560 --> 01:01:30,760
Speaker 1: So I know that there are all sorts of interesting

1144
01:01:31,560 --> 01:01:35,600
Speaker 1: UM awards from mathematical and statistical research. Are you looking

1145
01:01:35,640 --> 01:01:38,080
Speaker 1: at applying for any of these? How how does that work?

1146
01:01:38,280 --> 01:01:41,480
Speaker 1: Are you can you self nominate? Does the institution have

1147
01:01:41,520 --> 01:01:44,560
Speaker 1: to nominate you? How do you say? How how does

1148
01:01:44,600 --> 01:01:46,680
Speaker 1: that process work? Have you guys thought about this? It's

1149
01:01:46,720 --> 01:01:49,040
Speaker 1: not something we thought about. Oh well, let me plant

1150
01:01:49,080 --> 01:01:52,160
Speaker 1: that seed. And if this is significant enough, you should

1151
01:01:52,240 --> 01:01:56,840
Speaker 1: uh apply for either a grant or a mathematical award,

1152
01:01:56,880 --> 01:01:59,040
Speaker 1: although most of these you have to be nominated by

1153
01:01:59,080 --> 01:02:02,640
Speaker 1: other of people. But how hard is that to have

1154
01:02:02,720 --> 01:02:05,360
Speaker 1: your department share and nominate you. That's that's easy enough.

1155
01:02:05,600 --> 01:02:07,800
Speaker 1: I have to ask you. I didn't ask this earlier.

1156
01:02:08,200 --> 01:02:10,520
Speaker 1: You grew up in California, you went to Santa Barbara.

1157
01:02:10,680 --> 01:02:15,120
Speaker 1: How did you end up in Spain? Oh? So I

1158
01:02:15,120 --> 01:02:17,480
Speaker 1: think you remember two thousand and eight, two thousand nine

1159
01:02:17,520 --> 01:02:21,480
Speaker 1: a little bit. So that academic jar market was an

1160
01:02:21,520 --> 01:02:23,680
Speaker 1: interesting one. I was going on the market in late

1161
01:02:23,680 --> 01:02:27,160
Speaker 1: two thousand and eight, two thousand nine, and so a

1162
01:02:27,240 --> 01:02:32,080
Speaker 1: lot of academic appointments, non appointments, advertisers for positions were

1163
01:02:32,120 --> 01:02:35,080
Speaker 1: disappearing because of you know, the crisis. You would think

1164
01:02:35,160 --> 01:02:39,920
Speaker 1: academia is with large endowments and what have you, someone

1165
01:02:39,400 --> 01:02:44,439
Speaker 1: insulated from the vagaries of the stock market and even

1166
01:02:44,480 --> 01:02:48,680
Speaker 1: the broader economy. But apparently not yea. And so my

1167
01:02:48,800 --> 01:02:52,600
Speaker 1: advisor came to me and said, look, this year, everyone's

1168
01:02:52,600 --> 01:02:54,440
Speaker 1: applying everywhere, so you need to apply to Europe, even

1169
01:02:54,440 --> 01:02:56,200
Speaker 1: if you weren't even think, you weren't thinking about it.

1170
01:02:56,600 --> 01:02:58,600
Speaker 1: So at that time I applied everywhere and it was

1171
01:02:58,640 --> 01:03:00,640
Speaker 1: great because it opened my mind mine to the great

1172
01:03:00,680 --> 01:03:03,240
Speaker 1: opportunities that are there. So right, so I moved to

1173
01:03:03,280 --> 01:03:05,440
Speaker 1: Italy in two thousand nine. That was my first stop

1174
01:03:05,680 --> 01:03:08,919
Speaker 1: Where were you in Italy? Bacona University in Milan? That

1175
01:03:09,280 --> 01:03:11,400
Speaker 1: there are worse places in the world to ride out

1176
01:03:11,480 --> 01:03:13,480
Speaker 1: a recession. Yeah, no, it was a good It was

1177
01:03:13,480 --> 01:03:15,920
Speaker 1: a good time, I can imagine. And then from Milan,

1178
01:03:15,960 --> 01:03:18,400
Speaker 1: how do you end up in in Spain? So I

1179
01:03:18,400 --> 01:03:20,520
Speaker 1: wanted to join my co author and finish our work.

1180
01:03:20,680 --> 01:03:22,960
Speaker 1: Is that where he was located? Yeah, and he still is.

1181
01:03:23,000 --> 01:03:25,040
Speaker 1: So we're both the University of the Content on the

1182
01:03:25,200 --> 01:03:30,480
Speaker 1: lovely Mediterranean ceo of there there again, parts of that

1183
01:03:30,600 --> 01:03:35,680
Speaker 1: whole whole Millranian coast is just spectacular, isn't it. So

1184
01:03:36,280 --> 01:03:39,160
Speaker 1: you don't miss California too much? I get back a

1185
01:03:39,200 --> 01:03:41,600
Speaker 1: couple of times a year. Quite quite interesting. All right,

1186
01:03:41,800 --> 01:03:44,600
Speaker 1: let's jump to our favorite questions. I can't believe you

1187
01:03:44,640 --> 01:03:46,760
Speaker 1: guys never thought of saying, hey, maybe we should apply

1188
01:03:46,880 --> 01:03:48,680
Speaker 1: just for some of these grants and some of these

1189
01:03:49,000 --> 01:03:51,040
Speaker 1: award rates. We think of applying to grants, But the

1190
01:03:51,280 --> 01:03:56,240
Speaker 1: award thing, I don't know. Yeah, all right. I always

1191
01:03:56,240 --> 01:03:59,400
Speaker 1: thought academics had to do stuff like that in order

1192
01:03:59,440 --> 01:04:06,600
Speaker 1: to maintain their academic standing. Grants and I apply for money,

1193
01:04:06,760 --> 01:04:11,160
Speaker 1: that's for sure, But the the the recognition, um, it's

1194
01:04:11,160 --> 01:04:13,400
Speaker 1: a good idea. Yeah, not a bad idea. Let me

1195
01:04:13,520 --> 01:04:17,160
Speaker 1: let me when you give your acceptance speech. Yeah, definitely,

1196
01:04:17,960 --> 01:04:21,880
Speaker 1: all right. So, um, let's jump into a favorite questions,

1197
01:04:21,880 --> 01:04:27,040
Speaker 1: which I'll modify slightly because, um, most people, I think

1198
01:04:27,320 --> 01:04:30,040
Speaker 1: I don't know you personally. So let me ask the question,

1199
01:04:30,160 --> 01:04:33,440
Speaker 1: what's the most important thing, um, that your friends and

1200
01:04:33,480 --> 01:04:39,800
Speaker 1: family don't know about you? Friends and families? So you

1201
01:04:39,800 --> 01:04:41,760
Speaker 1: had mentioned this question earlier, and I was going to say,

1202
01:04:41,800 --> 01:04:45,320
Speaker 1: the most important thing we've actually already revealed, which is

1203
01:04:45,440 --> 01:04:49,240
Speaker 1: which is it's what most people didn't know we haven't

1204
01:04:49,240 --> 01:04:52,919
Speaker 1: shared this much? Is that this might be in this research, um,

1205
01:04:52,960 --> 01:04:57,200
Speaker 1: the first joint eureka moment, right, Usually someone discovers something.

1206
01:04:57,240 --> 01:05:01,000
Speaker 1: You when they say simultaneous discovery, someone discover something at

1207
01:05:01,040 --> 01:05:02,880
Speaker 1: one point in time and somebody at another, but no

1208
01:05:02,880 --> 01:05:05,360
Speaker 1: one knew about it. Did The electric light bulb as

1209
01:05:05,400 --> 01:05:09,040
Speaker 1: a classical radio is another classic example. So, but my

1210
01:05:09,080 --> 01:05:10,840
Speaker 1: cauthor and I were in this, you know, we were

1211
01:05:10,920 --> 01:05:12,400
Speaker 1: in the same room, we were on the phone at

1212
01:05:12,440 --> 01:05:14,360
Speaker 1: the same time, and we both had the you know,

1213
01:05:14,440 --> 01:05:19,560
Speaker 1: the gold is there and then how often does that happen? So?

1214
01:05:19,640 --> 01:05:23,360
Speaker 1: Who were some of your mentors in your early career?

1215
01:05:23,920 --> 01:05:27,120
Speaker 1: So in my early career, um, I would say, and

1216
01:05:27,160 --> 01:05:30,080
Speaker 1: my author would say the same. You know, my my advisor,

1217
01:05:30,200 --> 01:05:33,280
Speaker 1: Alto ruster KINI is a professor at University of Minnesota,

1218
01:05:33,320 --> 01:05:35,880
Speaker 1: and he's just you know, he's a neuroscientist, he's a mathematician,

1219
01:05:35,920 --> 01:05:38,600
Speaker 1: he's an economist. He's all about the science, right, And

1220
01:05:38,600 --> 01:05:41,440
Speaker 1: I'm thinking a renaissance person. Oh, he's a renaissance person.

1221
01:05:41,560 --> 01:05:44,000
Speaker 1: And you know when you see that, um, and you

1222
01:05:44,080 --> 01:05:46,200
Speaker 1: see someone that's just you know, zeroed in on that

1223
01:05:46,240 --> 01:05:47,800
Speaker 1: and you see how they work, you kind of you

1224
01:05:47,800 --> 01:05:50,600
Speaker 1: get you kind of absorbed what they do through osmosis.

1225
01:05:50,600 --> 01:05:51,960
Speaker 1: And my CA author with the same to say the

1226
01:05:51,960 --> 01:05:55,400
Speaker 1: same thing about his advisor, Vince Crawford at Oxford University

1227
01:05:55,440 --> 01:05:57,600
Speaker 1: and a very deep guy, very brilliant, you know, both

1228
01:05:57,680 --> 01:06:02,040
Speaker 1: very brilliant. Um. And and those are formative years when

1229
01:06:02,040 --> 01:06:05,520
Speaker 1: you're in grad school for for sure. For sure. UM,

1230
01:06:05,560 --> 01:06:12,680
Speaker 1: So what other behaviorists and statisticians influenced your approach um

1231
01:06:12,680 --> 01:06:18,440
Speaker 1: to thinking about the mathiness of things like shooting streaks?

1232
01:06:18,480 --> 01:06:21,680
Speaker 1: So that I would say the statisticians that have influenced

1233
01:06:21,680 --> 01:06:24,000
Speaker 1: me out there is one, right, there's Andrew Galman reading

1234
01:06:24,040 --> 01:06:26,760
Speaker 1: his blog has been eye opening for so many people.

1235
01:06:27,480 --> 01:06:31,920
Speaker 1: I mean, he just introduced how to think about data

1236
01:06:32,320 --> 01:06:35,800
Speaker 1: in a way that most people don't get in their

1237
01:06:35,800 --> 01:06:38,920
Speaker 1: formal training because he's dealing with real practical examples all

1238
01:06:39,000 --> 01:06:41,760
Speaker 1: the time. Um, and so I would say he's been

1239
01:06:41,880 --> 01:06:46,200
Speaker 1: one of the biggest influences interesting about the what about

1240
01:06:46,240 --> 01:06:50,040
Speaker 1: on the behavioral side? On the behavioral side, there's just

1241
01:06:50,080 --> 01:06:52,440
Speaker 1: so many great you know. I mean, there there was

1242
01:06:52,440 --> 01:06:54,920
Speaker 1: this vanguard of the people that the folks that came

1243
01:06:54,960 --> 01:06:57,080
Speaker 1: in in the eighties that really had to fight through,

1244
01:06:57,560 --> 01:07:00,760
Speaker 1: you know, the review process of all the skepticism towards

1245
01:07:00,800 --> 01:07:03,680
Speaker 1: you know, why a psychology you know, relevant to economics,

1246
01:07:03,680 --> 01:07:06,240
Speaker 1: Why are these other social science disciplines? What do they

1247
01:07:06,240 --> 01:07:08,560
Speaker 1: have to say about? People really had to fight a

1248
01:07:08,600 --> 01:07:11,280
Speaker 1: lot of skepticism. So give us some names. I'm putting

1249
01:07:11,320 --> 01:07:13,240
Speaker 1: it on the spot, Okay. So the people that had

1250
01:07:13,280 --> 01:07:14,720
Speaker 1: to fight through that, I mean that's like, you know,

1251
01:07:14,920 --> 01:07:18,120
Speaker 1: Amos Tversky and Daniel Kaneman were very influential, but they

1252
01:07:18,480 --> 01:07:21,240
Speaker 1: know they were within psych psychologist they were fine. So

1253
01:07:21,280 --> 01:07:23,280
Speaker 1: the people that had to deal, you know, with this

1254
01:07:23,360 --> 01:07:25,000
Speaker 1: kind of pushback. I mean you say, like you know,

1255
01:07:25,120 --> 01:07:28,600
Speaker 1: Richard Taylor, you know, as much as you know he

1256
01:07:28,600 --> 01:07:31,000
Speaker 1: he's been a bit skeptical of our work, but you

1257
01:07:31,040 --> 01:07:34,200
Speaker 1: have to respect, you know, and and you know, both

1258
01:07:34,320 --> 01:07:38,760
Speaker 1: his the insights he has into human behavior UM, and

1259
01:07:38,800 --> 01:07:41,760
Speaker 1: also just what he had to fight through get get

1260
01:07:41,800 --> 01:07:44,080
Speaker 1: listened to UM. So he was a guest, and my

1261
01:07:44,160 --> 01:07:48,480
Speaker 1: favorite quote from him was UM. Early on he decided

1262
01:07:48,520 --> 01:07:51,880
Speaker 1: he would never convince his peers. So he thought, I'm

1263
01:07:51,880 --> 01:07:53,920
Speaker 1: going to bypass them and just try and convince the

1264
01:07:53,960 --> 01:07:57,440
Speaker 1: grad students, and we'll just wait it out. After enough funerals,

1265
01:07:57,480 --> 01:08:00,680
Speaker 1: we will have one. And and it's really turned out

1266
01:08:00,760 --> 01:08:04,800
Speaker 1: to be quite true. If you if you are influencing

1267
01:08:04,840 --> 01:08:08,160
Speaker 1: the next generation, that's more and more impactful than what

1268
01:08:08,240 --> 01:08:13,080
Speaker 1: Tversky said, winning all these arguments and convincing nobody. Uh,

1269
01:08:13,160 --> 01:08:15,720
Speaker 1: it turned out to be very clever. Anybody else you

1270
01:08:15,720 --> 01:08:18,160
Speaker 1: want to mention from that group or oh no, I mean,

1271
01:08:19,439 --> 01:08:21,800
Speaker 1: I mean I wouldn't want to single out any person,

1272
01:08:22,600 --> 01:08:24,400
Speaker 1: you know, you just you look at the people that

1273
01:08:24,439 --> 01:08:26,000
Speaker 1: really kind of did a lot of the fighting that

1274
01:08:26,040 --> 01:08:28,360
Speaker 1: you're kind of push push the ideas through. But you know,

1275
01:08:28,400 --> 01:08:31,360
Speaker 1: in terms of the ideal dal level, there's so many right,

1276
01:08:31,520 --> 01:08:33,519
Speaker 1: you know, so even in the you know, when we

1277
01:08:33,520 --> 01:08:35,360
Speaker 1: were presenting our work. You know, you have somebody to say,

1278
01:08:35,360 --> 01:08:39,040
Speaker 1: like Colin camer cal Tech. You know, he actually just

1279
01:08:39,160 --> 01:08:42,000
Speaker 1: met him at a conference and he's a he's really

1280
01:08:42,040 --> 01:08:44,280
Speaker 1: a fascinating dude. Oh yes, And you know there's a

1281
01:08:44,320 --> 01:08:46,479
Speaker 1: lot of similarities you know that I recognize in him.

1282
01:08:46,479 --> 01:08:49,080
Speaker 1: That's kind of similar to the advisor that Altarosta Kidia

1283
01:08:49,200 --> 01:08:51,960
Speaker 1: University of Minnesota. You know, just this, You know that

1284
01:08:52,040 --> 01:08:54,320
Speaker 1: he's all about the science and really, I mean he

1285
01:08:54,360 --> 01:08:57,280
Speaker 1: came to one of our talks and he brought up

1286
01:08:57,280 --> 01:09:01,439
Speaker 1: Footnotes seventy two, which was the weakest point, the point

1287
01:09:01,439 --> 01:09:03,439
Speaker 1: he really wanted, and he found it. He found it.

1288
01:09:03,720 --> 01:09:05,600
Speaker 1: You know, we had a nice discussion about it. He

1289
01:09:06,960 --> 01:09:09,519
Speaker 1: saw our perspective after we had to talk. It's like, Wow,

1290
01:09:09,600 --> 01:09:12,439
Speaker 1: he's really taking this seriously, and it's just it's it's

1291
01:09:12,520 --> 01:09:14,680
Speaker 1: nice to see that that. That's got to be so delightful.

1292
01:09:15,840 --> 01:09:18,080
Speaker 1: I believe we have him tied up for the spring

1293
01:09:18,320 --> 01:09:21,680
Speaker 1: as a guest. Um. Yeah, he's really I love the

1294
01:09:21,720 --> 01:09:26,720
Speaker 1: work he does with virtual reality and showing people in

1295
01:09:26,720 --> 01:09:29,280
Speaker 1: incredible detail what they're gonna look like when they're older,

1296
01:09:29,800 --> 01:09:33,639
Speaker 1: and it affects their decision making. Dramatically in terms of planning,

1297
01:09:33,680 --> 01:09:37,320
Speaker 1: not just like a computer generated picture that's been aged,

1298
01:09:37,600 --> 01:09:41,599
Speaker 1: but when you have this immersive VR experience of here's

1299
01:09:41,600 --> 01:09:44,880
Speaker 1: your life when you're eighty, it leads to all sorts

1300
01:09:44,920 --> 01:09:48,520
Speaker 1: of amazing changes when you're forty. It it's quite astonishing.

1301
01:09:48,880 --> 01:09:50,920
Speaker 1: I'm gonna to do that. So I'm glad, glad you

1302
01:09:50,960 --> 01:09:53,240
Speaker 1: brought that up. Um, so I'm gonna put down Gellman

1303
01:09:53,360 --> 01:09:55,800
Speaker 1: as one of those people who influenced your approach to

1304
01:09:56,000 --> 01:09:58,920
Speaker 1: UH statistics. Let's talk about books. What are some of

1305
01:09:58,960 --> 01:10:02,559
Speaker 1: your favorite books? So I'll pick out a book. Um,

1306
01:10:02,600 --> 01:10:04,320
Speaker 1: you know, we could pick out a lot of nonfiction

1307
01:10:04,360 --> 01:10:07,200
Speaker 1: books and a lot of books like that. It hits

1308
01:10:07,240 --> 01:10:09,120
Speaker 1: you at the right time. And then if I tell

1309
01:10:09,160 --> 01:10:11,559
Speaker 1: you that book, I might you know, if I were

1310
01:10:11,560 --> 01:10:13,759
Speaker 1: to look at it now, it might feel trivial obvious

1311
01:10:13,800 --> 01:10:15,360
Speaker 1: things like this. You never know if the book is

1312
01:10:15,760 --> 01:10:17,479
Speaker 1: targeted for the right person. So I'll bring up a

1313
01:10:17,520 --> 01:10:19,960
Speaker 1: book that both my cauthor and I were a lot

1314
01:10:19,960 --> 01:10:22,640
Speaker 1: of were very much influenced by. But it's literature. So

1315
01:10:22,680 --> 01:10:25,439
Speaker 1: there's this book called The Alexandria Quartet by Lawrence Durrow.

1316
01:10:25,520 --> 01:10:28,040
Speaker 1: We both read it in our university days, and it's

1317
01:10:28,120 --> 01:10:30,479
Speaker 1: kind of like The Blind Men and the Elephant, but

1318
01:10:30,560 --> 01:10:33,320
Speaker 1: for human relationships, and it has this really novel. Idea's

1319
01:10:33,400 --> 01:10:36,320
Speaker 1: four books. The first three books are about three different

1320
01:10:36,320 --> 01:10:40,880
Speaker 1: perspectives on a serious relationships and events that happened in

1321
01:10:40,920 --> 01:10:44,720
Speaker 1: a particular time in Egypt before World War two, um,

1322
01:10:44,760 --> 01:10:48,559
Speaker 1: and from different people's perspectives. And so that's the space

1323
01:10:48,640 --> 01:10:51,200
Speaker 1: kind of. So it was inspired a bit by Einstein's relativity.

1324
01:10:51,240 --> 01:10:53,880
Speaker 1: So there's like three different perspectives on space and then

1325
01:10:53,880 --> 01:10:56,880
Speaker 1: they go forward in time and do the reflection back

1326
01:10:57,160 --> 01:10:59,920
Speaker 1: on on those relationships and it gives you, you know,

1327
01:11:00,040 --> 01:11:02,960
Speaker 1: this kind of you know, humility, kind of see like

1328
01:11:03,080 --> 01:11:06,600
Speaker 1: how how small your perspective is, how much missing information

1329
01:11:06,680 --> 01:11:09,120
Speaker 1: you have about what's happening, And it's it's it's it's

1330
01:11:09,160 --> 01:11:10,760
Speaker 1: a it's a nice read, at least it was. And

1331
01:11:10,760 --> 01:11:13,000
Speaker 1: when I was in my twenties, that sounds quite fascinating

1332
01:11:13,120 --> 01:11:18,400
Speaker 1: that this question is what people ask me more about

1333
01:11:18,640 --> 01:11:22,360
Speaker 1: than any other question because they want to get a

1334
01:11:22,400 --> 01:11:26,800
Speaker 1: book recommendation from somebody who's accomplished something done, something interesting,

1335
01:11:27,320 --> 01:11:30,080
Speaker 1: has some experience. And when someone says, oh, and by

1336
01:11:30,080 --> 01:11:33,400
Speaker 1: the way, this book is worth reading, it's the greatest

1337
01:11:33,479 --> 01:11:36,479
Speaker 1: endorsement anybody can ever get. So I'm going to press

1338
01:11:36,520 --> 01:11:38,679
Speaker 1: you and say, give us one or two more books,

1339
01:11:39,160 --> 01:11:41,720
Speaker 1: even if you think they may have been very time

1340
01:11:41,800 --> 01:11:45,519
Speaker 1: specific to you. So Duncan Watts has this book called

1341
01:11:45,640 --> 01:11:50,479
Speaker 1: Everything Is Obvious. Beautiful book. It's so interesting. It's all

1342
01:11:50,520 --> 01:11:54,080
Speaker 1: about client side bias and how you see things after

1343
01:11:54,160 --> 01:11:57,040
Speaker 1: the fact. It's it's you're the first person who's brought

1344
01:11:57,080 --> 01:11:59,280
Speaker 1: that book up and I find it. I love the

1345
01:11:59,320 --> 01:12:03,080
Speaker 1: cover with the wheel. I think that's a triangle instead

1346
01:12:03,080 --> 01:12:06,400
Speaker 1: of a circle. It's um, it's really a very fascinating book.

1347
01:12:06,600 --> 01:12:08,639
Speaker 1: It's you know, it's as you have this curse of knowledge,

1348
01:12:08,680 --> 01:12:10,600
Speaker 1: right once you know something, it's obvious to you and

1349
01:12:10,680 --> 01:12:13,040
Speaker 1: you can't imagine how not obvious it would be to

1350
01:12:13,080 --> 01:12:17,720
Speaker 1: someone else. And in the references in that book, I

1351
01:12:17,760 --> 01:12:21,320
Speaker 1: mean he you know, he's an academic, so he's really

1352
01:12:21,360 --> 01:12:23,439
Speaker 1: given you, you know, the road map, and like if

1353
01:12:23,439 --> 01:12:25,800
Speaker 1: you want to go beyond that book, all the references

1354
01:12:25,800 --> 01:12:28,240
Speaker 1: are there in that book. It's great. Um. And then

1355
01:12:28,960 --> 01:12:31,200
Speaker 1: I would say another book that's like Along the Lines,

1356
01:12:31,240 --> 01:12:34,120
Speaker 1: only because in the last year I read it. Um.

1357
01:12:34,160 --> 01:12:39,320
Speaker 1: You know. Super Forecasting by Philip another prior guest, delightful. Yeah,

1358
01:12:39,360 --> 01:12:41,880
Speaker 1: I mean it gives you, you know, humility to kind

1359
01:12:41,880 --> 01:12:44,200
Speaker 1: of realize how how you know, if you want to

1360
01:12:44,200 --> 01:12:48,080
Speaker 1: start projecting three years, five years and you're wasting you're

1361
01:12:48,080 --> 01:12:49,680
Speaker 1: wasting your time. But you know there's a lot more

1362
01:12:49,760 --> 01:12:52,000
Speaker 1: than that, right, just a disciplined approach. Right, So it's

1363
01:12:52,000 --> 01:12:54,840
Speaker 1: not simply you know and one one mistake you can make,

1364
01:12:54,840 --> 01:12:57,639
Speaker 1: because you know mistake I made, say around the financial

1365
01:12:57,680 --> 01:13:00,559
Speaker 1: crisis time, I really was convinced City Bank would be

1366
01:13:00,560 --> 01:13:03,320
Speaker 1: bailed out, they wouldn't let City Bank completely crash. Well,

1367
01:13:03,360 --> 01:13:06,160
Speaker 1: you were, you were not wrong. They did bail it. Eventually,

1368
01:13:06,520 --> 01:13:09,080
Speaker 1: it was two hours at the time, but still they

1369
01:13:09,120 --> 01:13:12,719
Speaker 1: were bailed out. Had you said the same about leaven Brothers,

1370
01:13:12,760 --> 01:13:17,120
Speaker 1: that would have been a different situation. What's most fascinating

1371
01:13:17,160 --> 01:13:25,120
Speaker 1: about Tetlock talk about recognizing your own issue. Tetlock's original book,

1372
01:13:25,640 --> 01:13:29,240
Speaker 1: If You Go Back, was on expert political judgment and

1373
01:13:29,320 --> 01:13:33,920
Speaker 1: that nobody is good at forecast now, and he then

1374
01:13:34,840 --> 01:13:39,000
Speaker 1: over time led to what what led from him going

1375
01:13:39,040 --> 01:13:42,519
Speaker 1: from oh, we're really bad as a species at forecasting too,

1376
01:13:43,080 --> 01:13:45,320
Speaker 1: But a handful of people have do a number of

1377
01:13:45,320 --> 01:13:48,400
Speaker 1: things that make them better at it, and therefore that's

1378
01:13:48,400 --> 01:13:51,520
Speaker 1: how you end up with super forecasters. That's a fascinating

1379
01:13:51,720 --> 01:13:55,160
Speaker 1: arc over I don't know, third twenty years separating the

1380
01:13:55,200 --> 01:13:58,400
Speaker 1: two books. Yes, and and the thing I really got

1381
01:13:58,439 --> 01:14:00,320
Speaker 1: from that book is not getting fixed at it on

1382
01:14:00,360 --> 01:14:03,599
Speaker 1: your your one insight and putting all your cards on that.

1383
01:14:03,640 --> 01:14:07,400
Speaker 1: You know. So, so you know, these these super forecasters

1384
01:14:07,720 --> 01:14:09,880
Speaker 1: are right in the long run. You know they're using

1385
01:14:09,880 --> 01:14:11,880
Speaker 1: the law of large numbers. It's not that they're saying, oh,

1386
01:14:11,920 --> 01:14:14,000
Speaker 1: I have this one idea, I'm gonna fixate city Bank

1387
01:14:14,040 --> 01:14:16,200
Speaker 1: has to be bailed out. No, no, take all your

1388
01:14:16,240 --> 01:14:19,360
Speaker 1: ideas and spread your bets across all your ideas, just

1389
01:14:19,400 --> 01:14:22,160
Speaker 1: like the forecasters, and you'll do well eventually. But don't

1390
01:14:22,280 --> 01:14:24,080
Speaker 1: get fixated on that one. And I think that's a

1391
01:14:24,200 --> 01:14:27,960
Speaker 1: nice feature, but quite fascinating any of the books before

1392
01:14:28,000 --> 01:14:30,240
Speaker 1: we move on, I think that's good. Those three, those

1393
01:14:30,240 --> 01:14:33,000
Speaker 1: are three three good ones. Um, so what are you

1394
01:14:33,040 --> 01:14:35,840
Speaker 1: excited about right now? What what do you jazzed about

1395
01:14:35,880 --> 01:14:38,360
Speaker 1: in the world of academic research. Well, and I think

1396
01:14:38,400 --> 01:14:41,160
Speaker 1: mastery is addictive, right, So there's a story. There's a

1397
01:14:41,200 --> 01:14:44,640
Speaker 1: lot of drive by research out there, and you know,

1398
01:14:44,640 --> 01:14:47,479
Speaker 1: we're all a little guilty of it because you know

1399
01:14:47,520 --> 01:14:51,360
Speaker 1: there's this pressure to publish, and when you've gone and

1400
01:14:51,360 --> 01:14:54,160
Speaker 1: you've really dug into something, you've really mastered something, and

1401
01:14:54,160 --> 01:14:56,200
Speaker 1: just mind all as much of the gold as you have,

1402
01:14:56,280 --> 01:14:58,160
Speaker 1: but you also have this feeling of mastery. It just

1403
01:14:58,200 --> 01:15:02,519
Speaker 1: feels so great and wanting to do that again. Right, So,

1404
01:15:02,520 --> 01:15:06,360
Speaker 1: so the exciting thing is to take that understanding of

1405
01:15:06,400 --> 01:15:09,280
Speaker 1: how good it feels, how fun it is to master something,

1406
01:15:09,320 --> 01:15:11,840
Speaker 1: and take it to the next subject. Well, of course,

1407
01:15:11,880 --> 01:15:14,840
Speaker 1: still finishing what you started. So that's that's the exciting things.

1408
01:15:15,000 --> 01:15:19,040
Speaker 1: What's next really really interesting? There have been all sorts

1409
01:15:19,200 --> 01:15:25,200
Speaker 1: of criticisms of the lack of reproducibility in academic research.

1410
01:15:25,560 --> 01:15:28,720
Speaker 1: What changes are you looking forward to into. Do you

1411
01:15:28,800 --> 01:15:34,200
Speaker 1: think that increased big data and AI is ever going

1412
01:15:34,280 --> 01:15:38,840
Speaker 1: to help us with this reproducibility problem we're running into

1413
01:15:39,280 --> 01:15:43,080
Speaker 1: in academic research and in other research. Why aren't we

1414
01:15:43,120 --> 01:15:47,519
Speaker 1: seeing academic research being replicated and and even corporate research

1415
01:15:47,960 --> 01:15:50,840
Speaker 1: being replicated. I think there's a lot of cherry picking

1416
01:15:50,840 --> 01:15:53,400
Speaker 1: that happens. Um. So when you go out and you

1417
01:15:54,000 --> 01:15:56,600
Speaker 1: analyze something, and you measure ten different things, and you

1418
01:15:56,680 --> 01:15:59,800
Speaker 1: just pick out the things that worked, Um, you're you're

1419
01:16:00,120 --> 01:16:02,600
Speaker 1: not acknowledging what didn't work, and so you're gonna you

1420
01:16:02,640 --> 01:16:04,840
Speaker 1: have this kind of you know, winner's curse in a

1421
01:16:04,960 --> 01:16:10,280
Speaker 1: sense where you know I won the that's sailors of

1422
01:16:10,320 --> 01:16:12,559
Speaker 1: his early books. Yeah, we don't have time to explain

1423
01:16:12,600 --> 01:16:15,080
Speaker 1: that I realized. But but um so, let me re

1424
01:16:15,200 --> 01:16:17,800
Speaker 1: ask that question. So what are you looking forward to?

1425
01:16:18,000 --> 01:16:22,200
Speaker 1: What changes do you think you're gonna affect um your

1426
01:16:22,280 --> 01:16:25,880
Speaker 1: your world, the changes that are gonna affect the academic world.

1427
01:16:26,000 --> 01:16:28,760
Speaker 1: Um so, I think that the important change that's going

1428
01:16:28,800 --> 01:16:31,559
Speaker 1: to make this better. They're gonna gonna fix this, fix

1429
01:16:31,720 --> 01:16:35,000
Speaker 1: maybe not, but make it better. Is the idea of preregistration,

1430
01:16:35,400 --> 01:16:38,880
Speaker 1: meaning what you pre register what you're going to analyze.

1431
01:16:38,960 --> 01:16:43,559
Speaker 1: You register your predictions, and so you know your hands

1432
01:16:43,560 --> 01:16:45,640
Speaker 1: are tied. So therefore you're going to go out and

1433
01:16:45,720 --> 01:16:48,880
Speaker 1: actually research what you're claiming as opposed to, oh, look

1434
01:16:48,880 --> 01:16:51,360
Speaker 1: at this anomaly, let's talk about that, even though it

1435
01:16:51,400 --> 01:16:53,600
Speaker 1: could be random or cherry pick or what have you,

1436
01:16:54,240 --> 01:16:58,479
Speaker 1: and still valuing even whatever your conclusion is, don't take

1437
01:16:58,600 --> 01:17:01,080
Speaker 1: that as truth when we have to make sure it

1438
01:17:01,120 --> 01:17:04,479
Speaker 1: also replicates, because even then you may have what if

1439
01:17:04,520 --> 01:17:06,559
Speaker 1: you don't find it you decide not to write it up, Right,

1440
01:17:06,800 --> 01:17:09,680
Speaker 1: isn't that a big issue that people don't publish on

1441
01:17:10,640 --> 01:17:13,479
Speaker 1: um negative find things because there is value to say, Hey,

1442
01:17:13,520 --> 01:17:15,880
Speaker 1: we analyze this, couldn't find anything. It's a huge issue

1443
01:17:15,880 --> 01:17:18,200
Speaker 1: because then you get this kind of implicit cherry picking

1444
01:17:18,240 --> 01:17:19,880
Speaker 1: and that like, I don't want to spend my time

1445
01:17:19,880 --> 01:17:22,559
Speaker 1: writing up this paper because it's not a big finding. Well,

1446
01:17:22,600 --> 01:17:24,479
Speaker 1: then no one's seeing that. So then the papers we

1447
01:17:24,520 --> 01:17:28,400
Speaker 1: see are the ones that are kind of implicitly selected,

1448
01:17:28,439 --> 01:17:30,120
Speaker 1: and so you have the same kind of degrees of

1449
01:17:30,160 --> 01:17:32,559
Speaker 1: freedom that's happening. But it's like socially in what do

1450
01:17:32,560 --> 01:17:35,760
Speaker 1: you call survivorship bias about things that don't work out?

1451
01:17:36,560 --> 01:17:39,640
Speaker 1: So I guess it is just straight up survivorship bias, right,

1452
01:17:39,720 --> 01:17:42,920
Speaker 1: In other words, what's public's negative shelter? You know, the

1453
01:17:44,720 --> 01:17:47,679
Speaker 1: research ideas that don't make it to the publication stage

1454
01:17:47,760 --> 01:17:50,559
Speaker 1: have died, and so the publication ones are kind of

1455
01:17:50,560 --> 01:17:52,800
Speaker 1: the ones that are kind of randomly better but not

1456
01:17:52,880 --> 01:17:56,560
Speaker 1: necessarily truly better. Interesting tell us about the time you

1457
01:17:56,640 --> 01:18:00,519
Speaker 1: failed and what you learned from the experience. So one failure,

1458
01:18:01,000 --> 01:18:03,400
Speaker 1: it was a success in a failure. So a friend

1459
01:18:03,439 --> 01:18:05,800
Speaker 1: of mine, Patrick Flannagan from graduate school. We said, up

1460
01:18:05,800 --> 01:18:07,680
Speaker 1: this garage band hedge fund we called it. We were

1461
01:18:07,680 --> 01:18:09,800
Speaker 1: loaning money on the internet, and we thought we had

1462
01:18:09,800 --> 01:18:11,479
Speaker 1: this great idea and it was I mean, it weathered

1463
01:18:11,520 --> 01:18:13,960
Speaker 1: the crisis. We didn't lose money. We got like five percent.

1464
01:18:14,120 --> 01:18:15,960
Speaker 1: We weren't. It wasn't big money. It's maybe a hundred

1465
01:18:16,000 --> 01:18:18,720
Speaker 1: thousand or something. We're students, but this is peer to

1466
01:18:18,760 --> 01:18:23,120
Speaker 1: peer lending. That But um, the thing we didn't anticipate was,

1467
01:18:23,360 --> 01:18:25,799
Speaker 1: you know, the legal uncertainty of the of the enterprise.

1468
01:18:25,840 --> 01:18:27,840
Speaker 1: And we weren't lawyers, and we just oh we we

1469
01:18:27,840 --> 01:18:30,240
Speaker 1: we modeled. We we had a real confidence in our model.

1470
01:18:30,280 --> 01:18:32,360
Speaker 1: We had our automated bidding algorithm. It was like it

1471
01:18:32,400 --> 01:18:35,120
Speaker 1: was great, we're doing well. But the thing we didn't

1472
01:18:35,160 --> 01:18:39,280
Speaker 1: get is that the SEC could potentially crack down on

1473
01:18:39,280 --> 01:18:41,559
Speaker 1: on this and ruin our business model because then they

1474
01:18:41,680 --> 01:18:44,240
Speaker 1: changed the rules completely that made it impractical. So we

1475
01:18:44,320 --> 01:18:47,439
Speaker 1: just we just left. Because instead of having a direct

1476
01:18:48,360 --> 01:18:50,400
Speaker 1: a connection to the person that you're loaning, and it

1477
01:18:50,439 --> 01:18:53,200
Speaker 1: was now mediated through the company, and now you had

1478
01:18:53,200 --> 01:18:56,200
Speaker 1: to somehow pricing the risk of the company itself rather

1479
01:18:56,280 --> 01:19:00,320
Speaker 1: than a loan. So um and and so interest. Yeah,

1480
01:19:00,479 --> 01:19:03,040
Speaker 1: quite interesting. What do you do for fun when you're

1481
01:19:03,040 --> 01:19:05,320
Speaker 1: not crunching numbers? Well, I mean this is pretty fun

1482
01:19:05,479 --> 01:19:08,479
Speaker 1: right here, I am in New York. You know, you

1483
01:19:08,479 --> 01:19:10,680
Speaker 1: get to travel around, you get to meet with your

1484
01:19:10,680 --> 01:19:14,160
Speaker 1: co authors, finish your papers, and nice locations, and you know,

1485
01:19:14,240 --> 01:19:18,360
Speaker 1: meet interesting people. Um, I would say just seeing family,

1486
01:19:18,479 --> 01:19:20,040
Speaker 1: because I get to travel so much. I get to

1487
01:19:20,080 --> 01:19:22,120
Speaker 1: see my family and friends in different cities, and that's

1488
01:19:22,120 --> 01:19:24,559
Speaker 1: a great thing. What sort of advice would you give

1489
01:19:24,600 --> 01:19:27,440
Speaker 1: to a millennial or recent college grad who was interested

1490
01:19:27,520 --> 01:19:31,760
Speaker 1: in a career either in behavioral finance or stat statistics

1491
01:19:31,920 --> 01:19:34,639
Speaker 1: or um, any of the sort of work that you

1492
01:19:34,640 --> 01:19:38,680
Speaker 1: you do, economics, whatever. Don't be in too impatient to

1493
01:19:38,840 --> 01:19:42,439
Speaker 1: have life figured out. Um, it's not too late. I've

1494
01:19:42,479 --> 01:19:47,240
Speaker 1: seen people in their twenties and their thirties go back change,

1495
01:19:47,439 --> 01:19:49,760
Speaker 1: go back to school. You know, maybe they have to

1496
01:19:49,800 --> 01:19:52,080
Speaker 1: start a little lower rank school than they wanted to

1497
01:19:52,120 --> 01:19:54,479
Speaker 1: begin with. But you can get funding at those schools,

1498
01:19:54,520 --> 01:19:56,240
Speaker 1: and if you work hard, you can transfer, you can

1499
01:19:56,280 --> 01:19:59,320
Speaker 1: apply to another school, you can move up. And a

1500
01:19:59,320 --> 01:20:01,320
Speaker 1: lot of people get this kind of false notion that, oh,

1501
01:20:01,360 --> 01:20:03,600
Speaker 1: if I didn't if I wasn't a serious student, in

1502
01:20:03,680 --> 01:20:07,920
Speaker 1: high schoore in college that I'm too far behind. It's like, no,

1503
01:20:08,040 --> 01:20:10,360
Speaker 1: if you're really motivated and you're you know, you're you're capable,

1504
01:20:10,360 --> 01:20:12,519
Speaker 1: and there's plenty of people that are. You can catch up.

1505
01:20:12,560 --> 01:20:14,760
Speaker 1: You just have to be patient, take a few years off.

1506
01:20:14,960 --> 01:20:17,120
Speaker 1: It's never too late to get serious. Yeah, never too

1507
01:20:17,200 --> 01:20:19,680
Speaker 1: late to get serious. And our final question, what do

1508
01:20:19,720 --> 01:20:23,360
Speaker 1: you know about the world of statistics and data analytics today?

1509
01:20:23,800 --> 01:20:27,200
Speaker 1: You wish you knew a decade or so ago. Fake

1510
01:20:27,439 --> 01:20:32,040
Speaker 1: data simulation basically creating you want to know. You can't

1511
01:20:32,040 --> 01:20:36,280
Speaker 1: just go out and analyze data and show that however

1512
01:20:36,320 --> 01:20:38,280
Speaker 1: I analyze it, I still get the same result. No,

1513
01:20:38,439 --> 01:20:42,400
Speaker 1: you have to sit down and generate fake data. So

1514
01:20:42,560 --> 01:20:45,160
Speaker 1: what if the world looked like this, how would my

1515
01:20:45,200 --> 01:20:47,720
Speaker 1: analysis behave? What if the world look like that? How

1516
01:20:47,880 --> 01:20:50,240
Speaker 1: the analysis behaves? You have to do this hard work

1517
01:20:50,280 --> 01:20:53,000
Speaker 1: of like building models of the world and then seeing

1518
01:20:53,920 --> 01:20:57,679
Speaker 1: what does your analytical approach tell you under those different

1519
01:20:57,720 --> 01:20:59,640
Speaker 1: kind of assumptions about the model of the world. And

1520
01:20:59,680 --> 01:21:02,280
Speaker 1: to do that you need fake data. And that when

1521
01:21:02,280 --> 01:21:04,320
Speaker 1: you say fake data, I think of that as a

1522
01:21:04,360 --> 01:21:07,559
Speaker 1: counterfactual or how do you contextual? I mean, I guess

1523
01:21:07,560 --> 01:21:10,840
Speaker 1: everything is a counter factual because all models are wrong, right, Um,

1524
01:21:10,920 --> 01:21:14,080
Speaker 1: But but but some are useful, but some are useful,

1525
01:21:14,160 --> 01:21:17,400
Speaker 1: And you want to know how how you know under

1526
01:21:18,400 --> 01:21:22,600
Speaker 1: under different assumptions. If the world you know looked differently

1527
01:21:22,640 --> 01:21:26,040
Speaker 1: than you think it looks, is your analysis still gonna

1528
01:21:26,160 --> 01:21:28,640
Speaker 1: say something meaningful or not? Um And you need to

1529
01:21:28,640 --> 01:21:30,280
Speaker 1: actually go out and check that, And a lot of

1530
01:21:30,280 --> 01:21:32,760
Speaker 1: people don't do that. And so that's what happened in

1531
01:21:32,800 --> 01:21:35,400
Speaker 1: this hot hand example, right, I mean, what would this

1532
01:21:35,439 --> 01:21:38,840
Speaker 1: analysis give you if there were no hot hand? Quite

1533
01:21:38,920 --> 01:21:42,160
Speaker 1: quite interesting. We have been speaking to Josh Miller of

1534
01:21:42,200 --> 01:21:46,240
Speaker 1: the University of Alcante. If you enjoy this conversation, we'll

1535
01:21:46,240 --> 01:21:47,760
Speaker 1: be sure and look up an inch or down an

1536
01:21:47,800 --> 01:21:51,559
Speaker 1: inch on Apple iTunes, where you can see the past

1537
01:21:51,960 --> 01:21:56,479
Speaker 1: two and fifty or so previous conversations we've had. We

1538
01:21:56,600 --> 01:21:59,920
Speaker 1: love your comments, feedback and suggestions right to us at

1539
01:22:00,640 --> 01:22:04,040
Speaker 1: m IB podcast at Bloomberg dot net. I would be

1540
01:22:04,080 --> 01:22:06,439
Speaker 1: remiss if I did not thank the crack staff that

1541
01:22:06,479 --> 01:22:10,519
Speaker 1: helps put together these conversations each week. Medina Parwana is

1542
01:22:10,520 --> 01:22:13,639
Speaker 1: our producer, Michael bat Nick is my head of research.

1543
01:22:14,280 --> 01:22:18,120
Speaker 1: Taylor Riggs is our booker. Slash producer. Attica val Bron

1544
01:22:18,240 --> 01:22:21,799
Speaker 1: is our project manager. Tim Harrow is our audio engineer.

1545
01:22:22,120 --> 01:22:25,439
Speaker 1: I'm Barry Ritolts. You've been listening to Masters in Business

1546
01:22:25,760 --> 01:22:26,880
Speaker 1: on Bloomberg Radio.