Speaker 1: This is Masters in Business with Barry Ridholts on Bloomberg Radio.
Speaker 1: This week on the podcast, I have a fascinating guest,
Speaker 1: And if you're at all interested in statistical analysis of sports, UH,
Speaker 1: behavioral finance data analysis, understanding um streakiness, understanding the money
Speaker 1: hole problem, and then extrapolating that towards things like, UH,
Speaker 1: the hot hand in basketball, you're gonna find this to
Speaker 1: be absolutely fascinating. UH. Joshua Benjamin Miller comes from California,
Speaker 1: where he basically racked up all the degrees he could
Speaker 1: at some of the u C schools before getting his
Speaker 1: PhD at in economics UH in Minnesota. Josh and his
Speaker 1: co author have taken apart some of the more interesting
Speaker 1: statistical assumptions made UH in the original hand study with
Speaker 1: Tom Gilovich and Amos Tversky UH, and they found something
Speaker 1: really unusual by looking at the data from a slightly
Speaker 1: different perspective. And I approached their paper with tremendous amount
Speaker 1: of skepticism. I thought the randomness of the hot hands
Speaker 1: was a fairly well proven study that Tversky and Gilovich did,
Speaker 1: But when you look at the data and you look
Speaker 1: at how they analyzed it, it's hard not to reach
Speaker 1: the conclusion that there is some sort of a hot hand.
Speaker 1: It's quite sophisticated mathematics, but Josh does a very nice
Speaker 1: job reducing it to some very easily understandable UM. Probability,
Speaker 1: we don't we don't know math is required. Um, you
Speaker 1: just have to know the difference between a head or
Speaker 1: a tail when you're flipping a coin. If you're at
Speaker 1: all interested in anything probability, sports related statistical, you're gonna
Speaker 1: find this to be a fascinating, lee wonky and tremendously
Speaker 1: interesting conversation. So, with no further ado, my conversation with
Speaker 1: the economists and statistician Josh Miller. My special guest today
Speaker 1: is Joshua Benjamin Miller UH. He is the co author,
Speaker 1: along with Adam Sanjorro, of a fascinating paper that puts
Speaker 1: challenge to the myth of the myth of the hot hands. Uh.
Speaker 1: He comes to us with a BA in economics and
Speaker 1: an m a UH in mathewmatical statistics from UC Santa Barbara.
Speaker 1: He has his PhD from University of Minnesota, and he
Speaker 1: is currently a professor in the economics department at the
Speaker 1: University of Alacante. In Spain, where he focuses his research
Speaker 1: on behavioral economics, judgment and decision making, game theory, and
Speaker 1: statistical and experimental methods. Josh Miller, Welcome to Bloomberg. Thanks
Speaker 1: for having me, Barry, So a little background. We kind
Speaker 1: of met after I interviewed Thomas Gilovich, who I was
Speaker 1: mostly interested in due to all of his work on
Speaker 1: behavioral finance, but he also co authored a fascinating paper,
Speaker 1: uh that basically pointed out the hot hands. You co
Speaker 1: authored that with Amos Tversky, by the way, that the
Speaker 1: hot hand was really a myth and it was just
Speaker 1: we were all being fooled by randomness. How did that
Speaker 1: paper come to your attention? And what what fascinated you
Speaker 1: buy it? So that paper came to the attention of
Speaker 1: my co author Adam who's also at University of Vallecante.
Speaker 1: Pretty Much everyone who takes a behavioral economics class and
Speaker 1: even earlier gets exposed to that paper. It's like it's
Speaker 1: one of the you know, the prime examples of a
Speaker 1: bias because it's such apparently power. It's part of the
Speaker 1: canon of oh, look how easily we're all fooled exactly,
Speaker 1: and in the in the beginning of any kind of
Speaker 1: behavior economics class, you have to show the real world
Speaker 1: implications first to kind of motivate students. And here is
Speaker 1: this um one that professionals are faul victim too, and
Speaker 1: they're so resistant to it. I mean they were shown
Speaker 1: that this hot I mean, we haven't to find hot
Speaker 1: hand yet, but you know, the hot hand is this
Speaker 1: idea that you're in the zone. You know that success
Speaker 1: breeds success. And if you look at basketball players, UM
Speaker 1: and coaches, they all believe in this thing. And so
Speaker 1: when they discovered that there was no pattern there and
Speaker 1: they came and revealed revealed the output of their research,
Speaker 1: the professionals were it was difficult to convince them. Oh,
Speaker 1: there was tremendous pushback. There's a famous quote you that's
Speaker 1: been referenced was red or back up at the Boston Celtics.
Speaker 1: I don't I don't care what this professor says. So
Speaker 1: they do a study. Who cares? Right? Yeah? So, I
Speaker 1: mean the stubbornness that that came out of the practitioners
Speaker 1: was was really dramatic because typically you can convince someone
Speaker 1: that is motivated to get things right if you can
Speaker 1: demonstrate that they'll benefit from it, Um, and they just
Speaker 1: discounted it. And so there's this famous quote from Amis Diversky,
Speaker 1: after all the stubbornness that they encountered repeatedly, of people
Speaker 1: just not even looking at the evidence they were showing them,
Speaker 1: Um that he said, I've been in thousand arguments, one
Speaker 1: them all, but convinced no one. And he was He
Speaker 1: was very famous for being um, not only quite brilliant,
Speaker 1: but a little hardheaded and a little aggressive when it
Speaker 1: came to debating people. According at least according right, at
Speaker 1: least according to Michael Lewis's book The Undoing Project. Between
Speaker 1: Konomen and Seversky, they were two very distinct personality types.
Speaker 1: So so let's get back to you before we were
Speaker 1: gonna spend a lot of time on the hot hands. Um,
Speaker 1: you're not what I would think of as a traditional economist.
Speaker 1: What what sort of work do you focus on? Both
Speaker 1: my ca author and I focused on individual decision making, right,
Speaker 1: so we're looking at is it individual decision making within
Speaker 1: a group, within an institution, or just as a lone wolf.
Speaker 1: So so there you know, there are the psychological factors
Speaker 1: like UM, my co author works on search and attention
Speaker 1: and things like this UM, but there are also UM
Speaker 1: factors of the institution, the design, like how information is
Speaker 1: presented to you UM, and these things, while it may
Speaker 1: be important to an individual, they also bubble up in
Speaker 1: terms of how it affects, you know, decision making in groups,
Speaker 1: and how it affects financial markets, and so in the
Speaker 1: end it does impact UM policy and economic outcomes. It
Speaker 1: has real world effects. In other words, these aren't just
Speaker 1: ivory tower abstract discussions. There's real world application for how
Speaker 1: decisions are making and how information is presented, so that
Speaker 1: that's really quite interesting. UM. When you and you mentioned
Speaker 1: one of your research areas is behavioral finance. Has all
Speaker 1: the low hanging fruit in this space been picked or
Speaker 1: is there still lots and lots of things to be discovered.
Speaker 1: There's still lots of fruit, whether it's low hanging. I
Speaker 1: think you always have to work for it, right right, UM,
Speaker 1: So I think the way you get the fruit, you
Speaker 1: have to think a lot about how to measure things,
Speaker 1: how a theoretical grounding and what you're trying to get at,
Speaker 1: And you can't just rely on existing data and existing
Speaker 1: things that have been counted. You have to go out
Speaker 1: and measure things yourself. A bit um and do some
Speaker 1: work to collect that data. UM and so you know what,
Speaker 1: Like so a lot of the modern work you'll see
Speaker 1: is going beyond just the choices that people make, like
Speaker 1: when you're paying them to make decisions and looking at
Speaker 1: their choices, and you can get a lot more about
Speaker 1: You learn a lot more about what people want and
Speaker 1: what they believe by looking at other things like reaction times,
Speaker 1: how they search for things, how you know what they're
Speaker 1: paying attention to, um And in other words, you're not
Speaker 1: just bringing in a bunch of undergrads, sticking them in
Speaker 1: a room, giving them twenty bucks for the night, and
Speaker 1: saying we're gonna put you through a series of things.
Speaker 1: You're looking at very You're looking at a very different
Speaker 1: data set that's measuring very different things. Yeah, I mean
Speaker 1: you can improve even with the undergrads. But I think
Speaker 1: a lot of the innovative work goes and collects unique
Speaker 1: data from unique subjects. Like I have a friend alex Emis,
Speaker 1: I just present this very interesting it's on in the
Speaker 1: top topic of finance. We went and looked at institutional
Speaker 1: investors tons of data. These are people with big positions um,
Speaker 1: and found that there they're actually quite skilled at buying um,
Speaker 1: buying stocks, but they aren't so skilled at selling them
Speaker 1: UM and it seems to be distinct skills. It's very
Speaker 1: distinct skills because it's easy to buy, that's the easy part.
Speaker 1: Sellings where the money gets made, those are not um
Speaker 1: equal level of difficulty things. I'm absolutely not surprised to
Speaker 1: hear the selling demonstrates less skill than buying. Is that
Speaker 1: basically what you know? So the finding is that they
Speaker 1: were unloaded extreme winners um too quickly, um before they
Speaker 1: really exploited the you know, information advantage that they had.
Speaker 1: So they made a good job choosing it, but they
Speaker 1: sold it too soon, a classic mistake. Let's talk a
Speaker 1: little bit about the original hot hand paper, which, as
Speaker 1: we discussed earlier, became cannon in the world of behavioral finance. Uh,
Speaker 1: when did you first start to get an inkling that
Speaker 1: the original thesis might not have been all it seemed
Speaker 1: to be. So the original inkling was that people sometimes overreact.
Speaker 1: It's that it's a myth, the thing doesn't exist. Why
Speaker 1: does that generate such a strong intuitive pushback from people?
Speaker 1: I mean, I have my thesis, I'm curious as to yours.
Speaker 1: I think everyone has some experience in their own athletic
Speaker 1: performance where they have moments where they're particularly locked in,
Speaker 1: and then they realized that outside of athletic performance, you
Speaker 1: just have these moments where you're in the you're in
Speaker 1: the zone, that's the best word for it. Um. You're
Speaker 1: just you're firing on all cylinders, and you would expect
Speaker 1: that you would see that um somehow in basketball data
Speaker 1: as well. So so my personal experience. I used to
Speaker 1: play hoops as a kid, but as I've gotten older,
Speaker 1: I've become a tennis player, and I know from personal
Speaker 1: experience it takes a good twenty minutes for me to
Speaker 1: calibrate my forehand so that I am consistently hitting the
Speaker 1: bowl more or less towards where I want it, more
Speaker 1: or less with the right amount of spin, more or
Speaker 1: less with the right height. But it's not something that
Speaker 1: I could just grab a racket and swing and oh
Speaker 1: there it is. It takes a while, too too fast,
Speaker 1: too much whip, keep it loosening your wrist, bringing around,
Speaker 1: make sure you're dropping the head like I'm running through
Speaker 1: a series of steps in my head. Hey you're too close.
Speaker 1: Watch your footwork well, one after another and I am
Speaker 1: now good enough to know I suck. I'm in that
Speaker 1: Dunning Krueger drop where Oh you know, I used to
Speaker 1: think I was good. Now I realize I'm really I'm
Speaker 1: good enough to know how good I actually am not. Um,
Speaker 1: But it takes a while to calibrate that. I imagine
Speaker 1: a basketball player in the midst of a game has
Speaker 1: to go through some sort of fine tuning of their shooting.
Speaker 1: You can warm up all you want, um, when you're
Speaker 1: just shooting by yourself before the game, but when people
Speaker 1: are on you and you're running, it has to be
Speaker 1: a very very different set of circumstances. Or am I
Speaker 1: overstating this? Well, I mean that's that's the strongest intuition
Speaker 1: is based on this calibration thing. I mean there's probably
Speaker 1: other elements. Will not get to that later. Um, but
Speaker 1: if you look at um, yeah, I mean, if you're
Speaker 1: sitting on the bench for ten minutes and then you
Speaker 1: come off, that's very different. I mean in the NFL
Speaker 1: you see field goal kickers warming up go on. You
Speaker 1: don't see that so much in the NBA. They don't
Speaker 1: have like an extra hoop on this, that's right, So
Speaker 1: I'd imagine, yeah, that's an important element there. What else
Speaker 1: is so intuitively attractive about the idea of the hot hand?
Speaker 1: Is it simply just the zone? Is it the adrenaline
Speaker 1: and the endorphins. Why do we think that, hey, suddenly
Speaker 1: I'm on I'm on a streak. Why do we believe
Speaker 1: that streak is going to continue? And I'm not talking
Speaker 1: about blackjack or roulette or games of chance. These are
Speaker 1: really games of skill played at the highest level. So
Speaker 1: why we believe it? Um? I would imagine sometimes when
Speaker 1: we believe it, it's not really there. And so there
Speaker 1: is this feeling, you know. You know, part part of
Speaker 1: the feeling is the feedback, right you You see that
Speaker 1: your successful gives you some confidence, you know. So it's
Speaker 1: not always simply this zone that emerges. Um. Sometimes you
Speaker 1: get a few successes in a row and it gives
Speaker 1: you more confidence in your training. You don't overthink it,
Speaker 1: and you return and trust your training. So you're essentially
Speaker 1: unconscious and you bring that whereas if you maybe miss
Speaker 1: a funeral, you lose your confidence. You start making adjustments,
Speaker 1: and if you're making adjustments, you're not going to have
Speaker 1: much consistency. So let's go back to the original research.
Speaker 1: Tom Gilovich one of the co authors said about the
Speaker 1: work that you and Adam did, this is unlike a
Speaker 1: lot of stuff that's come down the pike, since this
Speaker 1: is truly interesting. How encouraging it was that from one
Speaker 1: of the original authors who ostensibly disproved the hot hand.
Speaker 1: I mean, it's always nice when somebody appreciates your work,
Speaker 1: especially in one of Tom Gilovich's stature at the time
Speaker 1: that he said that our our paper, while it had
Speaker 1: gone through the public peri review process, it hadn't gone
Speaker 1: through the formal one. And so just last week our
Speaker 1: paper was finally published its online, not in the print
Speaker 1: edition yet in Econometrica, which is you know, top journal
Speaker 1: and economics. There's the top five. They're kind of all equal, um,
Speaker 1: And you know, so now it's been kind of formally
Speaker 1: taken in, so I think Tom Gilvitch might have a
Speaker 1: different opinion, um, now that it's gone through this process.
Speaker 1: So so the paper is ready to be published or
Speaker 1: was just published. Yeah, so the November issue of Econometrica, Um,
Speaker 1: it came out and that's got to be very exciting. Oh,
Speaker 1: very exciting. Yeah, it's something. So what's the takeaway from
Speaker 1: the original research, What was it that was wrong in
Speaker 1: the structure of the original myth of the hot hands paper. Right,
Speaker 1: So um, the original hot hand paper. They're interested in
Speaker 1: seeing if people do better after recent success then after
Speaker 1: recent failure. Um, that was the most important measure, Like,
Speaker 1: is your probability of success increase when you've hit in
Speaker 1: a few funeral versus if you've missed a funeral? And well,
Speaker 1: we don't know what someone's probability is. It seems like
Speaker 1: our best guests would be would just look at the
Speaker 1: percentage of time they make it, right, And so they
Speaker 1: just look at all the events when you've had a
Speaker 1: streak of recent successes and all the events when you
Speaker 1: have a streaking recent failures, and just see what the
Speaker 1: change in your shooting percentages between those two conditions. And
Speaker 1: you know, that's very natural and very intuitive to expect
Speaker 1: that would be your best guess. And they do that
Speaker 1: and they don't find any difference. Um. And so that's
Speaker 1: how the problem was set up. So before we get
Speaker 1: to your solution, the the immediate pushback is, Hey, after
Speaker 1: a shooter gets on a bit of a streak, the
Speaker 1: defense collapses on them. They become they're forced to either
Speaker 1: pass the ball and more or take more difficult shots.
Speaker 1: At the time in there was no way to account
Speaker 1: for that difference. However, in the intervening years, every shot
Speaker 1: gets marked. You you describe this in one of your
Speaker 1: um publications recently, explain the degree of difficulty that is
Speaker 1: now tracked on every single basketball shot that's taken. Right, So,
Speaker 1: there there's a it's a new company now I don't
Speaker 1: remember the company, but sport View was the first company
Speaker 1: that did this, where they have optical tracking of the
Speaker 1: I mean, the precision isn't super high, but it's it
Speaker 1: gets in the general area, and so you can control
Speaker 1: for a lot more factors than you could say, or
Speaker 1: they had the seventies sixers and they're just looking at
Speaker 1: the play by playout and so you know, even in
Speaker 1: that data, what they would find is, yes, they'd have
Speaker 1: this evidence of the defense adjusting to what they believed
Speaker 1: to be a hoth hand, making it more difficult for
Speaker 1: the player. But the player stats to shoot from time
Speaker 1: to time to keep the defense honest. And so the
Speaker 1: important thing isn't so much as the player doing better
Speaker 1: in the context of the game, but as it helped
Speaker 1: the teammates if they're hot, because then it opens upthing
Speaker 1: for their teammates. Makes sense. Yeah, So the the innovations
Speaker 1: that have happened, I think the first in vation actually
Speaker 1: was Justin Rao, who's u the head economists at Home Away.
Speaker 1: He was the first one to actually come out and
Speaker 1: and measure how many defenders are around the player and
Speaker 1: try to control from these things in a different way
Speaker 1: by using the videos and show that yes, there's a
Speaker 1: lot of evidence that there's this defense factor, and if
Speaker 1: you just control for a few of these things, the
Speaker 1: effects that they had found in the previous study went away.
Speaker 1: So in other words, what looks like it's random is
Speaker 1: you're shooting the same percentage but with a whole lot
Speaker 1: more defensive activity on you. Therefore, it's a continuation of
Speaker 1: the of the streak. Yes, so he didn't necessarily find
Speaker 1: evidence of the streak there because he controlled for a
Speaker 1: subset of factors. But as you add more more controls,
Speaker 1: it looks like there might be some evidence there. But
Speaker 1: these are very difficult things to measure in the context
Speaker 1: of the game. The original study had this critical test
Speaker 1: and it's been repeated with other teams. Um where they
Speaker 1: take them and they pay them to shoot the basketball.
Speaker 1: So in other words, you're you're not playing during a
Speaker 1: live game, you're just doing foul shooting with three point
Speaker 1: shooting or whatever exactly. Or you'll to the NBA three
Speaker 1: point shooting contest and and and in those studies you
Speaker 1: can get rid of the defense and get a little
Speaker 1: more zero in on your question. A bit more. Let's
Speaker 1: talk a little bit about the surprising math of coin flips.
Speaker 1: My best guests and my understanding of statistics has always been,
Speaker 1: if you take a true coin and flip it, the
Speaker 1: odds of a head of a tail is fifty fifty.
Speaker 1: This is regardless of what came before it. Coins have
Speaker 1: no memory. But you found something surprising in the data set.
Speaker 1: After you flip a coin a hundred times, and you
Speaker 1: were to pick a specific series, the odds are somewhat different.
Speaker 1: Explain that. Yeah, So, my author Adamson, and I, after
Speaker 1: having watched the NBA three point shooting contests, we had
Speaker 1: a particular player, Craig Hodges, who was obviously hot, and
Speaker 1: we went and used the original analysis on his data
Speaker 1: and it said that he wasn't and that was puzzling,
Speaker 1: and so we had to go and see, well, we
Speaker 1: don't really know how Craig Hodges generated his shots. It's
Speaker 1: kind of a black box. But let's create an environment
Speaker 1: where we have the ground truth when we know what's happening,
Speaker 1: and so coin flips is a world like this. So
Speaker 1: you can actually go and flip a coin many many times,
Speaker 1: or do it on a computer and see what do
Speaker 1: you get if you analyze we're interested, is the probability
Speaker 1: of heads after a few heads different than the probablity
Speaker 1: of heads after a few tails? We know that's the same.
Speaker 1: That's we have the ground truth. But now let's go
Speaker 1: out and generate that data and make our best guests
Speaker 1: from that data. What's our best guests is the percentage
Speaker 1: of heads that you get after a few heads in
Speaker 1: a row the same as the percentage of heads you
Speaker 1: get after a few tails in a row. And analyzing
Speaker 1: it in the way they analyzed it, we found that no,
Speaker 1: it's different. The percentage of tails after a few heads
Speaker 1: in a row is higher, which which is so counterintuitive
Speaker 1: because perspectively, so understand before people lose their mind and
Speaker 1: start sending emails, what we're not talking about is looking
Speaker 1: forward in a live situation, no matter what the previous
Speaker 1: with with a true coin. You could have a thousand
Speaker 1: heads in a row. Highly improbable, but not mathematically impossible.
Speaker 1: The odds on that next flip are still gonna be
Speaker 1: That's not what we're saying. We're saying, flip a coin
Speaker 1: a hundred times, look at the data set, and then
Speaker 1: go back and randomly pick any head in that order
Speaker 1: or any tail in the order. What are the odds
Speaker 1: that the next flip is ahead of a tail? And
Speaker 1: it turns out that's not so? Explain that because it's
Speaker 1: it's a complete It blows people's minds because you've been
Speaker 1: told over and over again, hey, coins have no memory.
Speaker 1: But that's not what this is. This is an existing
Speaker 1: data set. When we randomly pull any of those flips,
Speaker 1: what are the probabilities as to the outcome in the
Speaker 1: next flip after it's already been done. So, so how
Speaker 1: do you end up with instead of so the complete
Speaker 1: exper name would take some time, but we can kind
Speaker 1: of get it an intuition. Um, if you flip a
Speaker 1: coin a hundred times, there's gonna be a certain number
Speaker 1: of heads and tails there when when you're done about guarantee,
Speaker 1: but it's gonna be some number. Now, if you just
Speaker 1: choose any flip your best guess, just choose a flip
Speaker 1: your best guess. Is if I choose flip forty two,
Speaker 1: for example, my best guess or heads, that's my you know,
Speaker 1: And so that's different though then if I choose flip
Speaker 1: forty two because flip forty one is the heads, So
Speaker 1: if I choose one of the flips where the previous
Speaker 1: flip is the heads, or just choose a flip that's
Speaker 1: ahead and see what the next flip is in the
Speaker 1: same way of looking at it. Now there's something else
Speaker 1: because the flip you chose because the previous flip is ahead,
Speaker 1: is using information about the outcomes of adjacent flips, and
Speaker 1: that information kind of gets contained within your flip, and
Speaker 1: that's this. This gets a little complicated, but one way
Speaker 1: to think about it is you've taken a head's away
Speaker 1: from the finite number of heads that you have and
Speaker 1: you can't see it again. You've reduced that data set,
Speaker 1: and the remaining tails now should be slightly than it shifts.
Speaker 1: And there's another element that uses the kind of space
Speaker 1: away there arrange that makes it a bit stronger. But
Speaker 1: that that that's this smells to me slightly like the
Speaker 1: Monty Hall problem. Is there any element in the Monty
Speaker 1: Hall problem? You go from choosing one in three to
Speaker 1: one and two. So suddenly what was the thirty three
Speaker 1: percent chance becomes a fifty fifty Why not make the
Speaker 1: switch that That is a little counterintuitive, but once you
Speaker 1: see the statistics, it you can't unsee it. It always
Speaker 1: you always should make the change. There is a is
Speaker 1: there a tiny element of this and that more than tiny.
Speaker 1: So my co author and Adamson Hot and I also
Speaker 1: wrote another paper connecting this to the Monty Hall problem
Speaker 1: and explaining it via this principle from Bridge, which is
Speaker 1: the stable restricted choice, which is essentially the intuition of
Speaker 1: Bay's rule, and so the way to think about it
Speaker 1: in the Monty Hall problem, you of these three doors, right,
Speaker 1: so you're on this game show. There's three doors. Well,
Speaker 1: let's make this our problem exactly the same as money,
Speaker 1: except it's a hundred doors, not necessarily three doors, so
Speaker 1: it becomes much harder. We can make it three doors.
Speaker 1: So your games show, you're on a game show, and
Speaker 1: usually that you have this car and two goats, and
Speaker 1: you gotta find the car, and there's a car behind
Speaker 1: one of the doors. You gotta guess. Well, let's get
Speaker 1: rid of the goats and cars. Now, let's just flip
Speaker 1: a coin behind each door. So eat behind the shorts
Speaker 1: fifty fifty. But you're the contestant. The host knows what
Speaker 1: the outcome of the flips are. You don't you want
Speaker 1: to guess, Hey, where's it? Where's the heads? Let's say
Speaker 1: you want to find the heads um, so you guess
Speaker 1: you know door three. Now, if you guess door three,
Speaker 1: let's say the host looks behind the door. You didn't
Speaker 1: guess door one and two, and he's going to reveal
Speaker 1: the heads if you can't. So let's say the host
Speaker 1: opens door one and shows you ahead. Do you want
Speaker 1: to switch or do you want to stay? If you're
Speaker 1: looking for the heads, you want to stay. If you're
Speaker 1: looking for the tails, you want to switch. Now, the
Speaker 1: intuition is not going to be clear immediately, but if
Speaker 1: you think about now, the host looking at door one
Speaker 1: and two used information about both doors to determine which
Speaker 1: door to open up to you. Now, if both doors
Speaker 1: were heads, the host could have opened door too. But
Speaker 1: if it was heads tails, the door host had to
Speaker 1: open door one. Because the host is gonna show your
Speaker 1: heads if you can't, right, he doesn't want to show
Speaker 1: you a tail because that's what you're looking to avoid.
Speaker 1: That's the goat. Yea. So the world we don't know
Speaker 1: which world we're in where the first is heads and
Speaker 1: the second is tails. Of the first his heads and
Speaker 1: the you know second, his heads. But the world where
Speaker 1: it's heads tails is the world where the host is
Speaker 1: more restricted. The host has to open door one, and
Speaker 1: so the world that so that you should avoid door
Speaker 1: two in those circumstances because it's a higher probability of
Speaker 1: if you're if you're hunting for the heads you should have,
Speaker 1: you should avoid door too because tails is more likely
Speaker 1: because in that world of heads tails, the host had
Speaker 1: to open door one and it's yeah, so it's a
Speaker 1: higher probability. So that door three, even though it's a
Speaker 1: coin that's flipped independent of the other two, when you're
Speaker 1: dealing with that data set, you're better off with three
Speaker 1: because of the circumstances that lead the host to pick
Speaker 1: one and not pick two. That makes some rational degree
Speaker 1: of sense. Once you get the monty whole aspect of this,
Speaker 1: it makes a whole lot more sense. It's just it
Speaker 1: just seems, uh, it's quite fascinating. We were discussing um
Speaker 1: the coin flip issue and the hot hand scenario. Let's
Speaker 1: circle back to that hot hand and the original research.
Speaker 1: The original research said that if there's a streak of
Speaker 1: three hits in basketball or three misses in basketball, the
Speaker 1: odds of the next shot going in or not is
Speaker 1: whatever the shooters historical shooting percentage hges, which sort of
Speaker 1: seems that there's no hot hand. But that presumes after
Speaker 1: streak that their next shot should be dead center in
Speaker 1: their percentage. You found out it should be worse than that.
Speaker 1: Explain exactly. So that's the counterintuitive thing. If you go
Speaker 1: out and you watch a player shoot a basketball and
Speaker 1: you look at their shooting percentage after a streak of
Speaker 1: hits and compare it to their shooting percentage after a
Speaker 1: streak of misses, and you find that it's the same
Speaker 1: and not. The intuitive thing is to say, oh, they're
Speaker 1: just they have the same rate, but actually you would
Speaker 1: expect them to do worse. Explain that, because that's the
Speaker 1: most fascinating part of it. Someone is on a shooting streak,
Speaker 1: we take a data set of a whole run of shots.
Speaker 1: What do you find after the streak, and why is that?
Speaker 1: So you said, you find their percentage actually goes down
Speaker 1: after a streak. In the world where there's no hot hand,
Speaker 1: where they're a consistent shooter, their percentage will go down
Speaker 1: after a streak in the data. Not in reality, their
Speaker 1: probability is always the same. But we don't observe the probability.
Speaker 1: We calculate the percentage, and that's where the biases come in.
Speaker 1: And so the original authors found that the shooting percentage
Speaker 1: was around the same, and that's correct. We go and
Speaker 1: we check and they were right. They report they did
Speaker 1: all the analysis in that sense, the calculations correctly. But
Speaker 1: the mistake is understanding the benchmark. You have to go
Speaker 1: out and say, okay, now let's look at the world
Speaker 1: where we know, um, we can control it. So on
Speaker 1: a computer you can say, we can generate coin flips,
Speaker 1: so we can make a player that has no hot hand,
Speaker 1: and then look at how that player does when we
Speaker 1: analyze the data and we realize, oh, they should do
Speaker 1: worse after a few in a row. So once you
Speaker 1: adjust for that bias, you find out that actually, if
Speaker 1: they're doing the same, that's indicative that they're doing about
Speaker 1: ten percentage points are more better after hitting a funer
Speaker 1: row than missing in a few funeroe and that's huge.
Speaker 1: That's like the difference between the median and the best
Speaker 1: NBA three points, thereby confirming the hot hands. So I
Speaker 1: have to challenge the data set because it again, everything
Speaker 1: about this, each step along the way, is so counterintuitive.
Speaker 1: So why would we expect a shooter who's on a streak,
Speaker 1: who's in the zone, who has the hot hand, whatever
Speaker 1: we want to call it, Why would we expect his
Speaker 1: shooting percentage to be lower after they hit several shots
Speaker 1: in a row. Why would we expect it to be
Speaker 1: lower for a real human or for for anybody, for
Speaker 1: a professional, for a real human. When you look at
Speaker 1: a data set of here's here's all the NBA streak
Speaker 1: shooters or all the NBA shooters, what does the data
Speaker 1: show after a streak they're shooting percentage actually becomes So
Speaker 1: if you're talking about live action games, we have those
Speaker 1: issues that we spoke about. The defense will adjust and
Speaker 1: so that becomes a little bit more complished. So so
Speaker 1: let's talk about three point contests. So if the hot
Speaker 1: hand didn't exist in a world like that, we would
Speaker 1: expect players to shoot worse after making a few in
Speaker 1: a row in the data, so simply just mean reversion,
Speaker 1: is that all. It's not mean resursion. It's the same
Speaker 1: thing we talked about with the coin flips. Right. And
Speaker 1: so as a researcher, you're taking the data after it's
Speaker 1: already been generated, and you're picking through it, and you're
Speaker 1: looking only at the events that you're interested in. Right,
Speaker 1: You're looking at their probability of success given recent success.
Speaker 1: You're just picking out those events when they had recent success.
Speaker 1: Let's say where they just made three in a row.
Speaker 1: So you're changing the data set so now there's three less.
Speaker 1: So if you someone shoots three in a row, when
Speaker 1: we're looking at the data set, let's say they've shot
Speaker 1: twenty shots, and after three in a row, how they
Speaker 1: do well, guess what, You've pulled three hits out of
Speaker 1: the set, meaning there's a disproportionate number of mrs left.
Speaker 1: That's part of the bias. And there's this other element
Speaker 1: that didn't quite get into is that you can you
Speaker 1: have this essentially a stopping rule, so as you collect
Speaker 1: the data, the moment they miss it, you're not interested
Speaker 1: anymore in looking because you're gonna wait for a streak
Speaker 1: of hits again, so you've kind of you're biased towards
Speaker 1: stopping at a miss, so you might get a miss
Speaker 1: right away right then everything you collected in those events
Speaker 1: of their shots or misses because you just collected one
Speaker 1: shot and they're all misses, and so you're you can
Speaker 1: biasing towards collecting misses in a side that that that's
Speaker 1: quite that's quite fascinating. So what other areas like this
Speaker 1: are you studying, because it's really it's really quite quite
Speaker 1: fascinating stuff. Are there other sport myths that you're looking
Speaker 1: at that have a probabilistic element that's very counterintuitive or
Speaker 1: is this pretty much the biggest one out there? Um,
Speaker 1: this is the biggest one that we're studying. A lot
Speaker 1: of what you're doing is statistical and probability work at
Speaker 1: a level that the average sports fan is really not
Speaker 1: familiar with. Forget the live game. When you explain relative
Speaker 1: to three point shooting contest, it's really not so much
Speaker 1: about the streakiness of the shooter, but the mathematics of
Speaker 1: the data set it. And I think that is really counterintuitive,
Speaker 1: but it doesn't seem anyone's been able to disprove what
Speaker 1: you and your co author found. So there have been
Speaker 1: a lot of challenges to that original study, a lot
Speaker 1: of challenge and legitimate challenges. You know, there are issues
Speaker 1: with UM what they call statistical power. Right, so we
Speaker 1: have we have a friend and colleague, Daniel Stone, who
Speaker 1: made this nice point that you have this thing called measurement.
Speaker 1: Are we want to know how do you do after
Speaker 1: hitting a few in a row? Um, that's what we
Speaker 1: actually look at. But what we're really interested in is
Speaker 1: how will you do when you're hot? So hitting a
Speaker 1: few in a row, you're not always hot, and so
Speaker 1: you can underestimate how hot someone is if you use
Speaker 1: only the data that you can observe, which is zeros
Speaker 1: and one. So the you know, the econometrician, the statistician
Speaker 1: has kind of a weak measure of that. So you know,
Speaker 1: these this kind of evidence is um just the mathematical evidence.
Speaker 1: Do you ever do interviews of players? Do you ever
Speaker 1: say to them, Hey, were you in the zone? How
Speaker 1: did you feel? How? How how do you find that
Speaker 1: data set? So that data UM, the original study looked
Speaker 1: at data like that. They spoke to the seventies sixers
Speaker 1: and they asked them kind of qualitative questions. Do you
Speaker 1: get in the zone and you feel hot? And they
Speaker 1: all do right, Um, but it's it's hard to work
Speaker 1: with that. That's just looking at whether they believe in
Speaker 1: it or not, but then getting a sense of do
Speaker 1: they believe in it too much or not? That get
Speaker 1: that gets a bit harder because you have to be
Speaker 1: able to somehow measure you know, they have to decide
Speaker 1: when are they hot? You know, so you really need
Speaker 1: a lot more cooperation from say like a coach or
Speaker 1: player to kind of sit there and maybe watch the
Speaker 1: games with you or something like that. That would be
Speaker 1: maybe a a better way of testing their you know,
Speaker 1: their beliefs. So so when um Traverseking Gilvitch's original study
Speaker 1: came out, I'm forgetting the third person in g VT.
Speaker 1: When when that study came out, there was a tremendous
Speaker 1: amount of push back from coaches amount around the league.
Speaker 1: We mentioned ridd are back. Your study comes out and
Speaker 1: you basically say, no, you professional coaches, You were right.
Speaker 1: There is a hot hand, there is a streak. What
Speaker 1: sort of feedback have you gotten from players and coaches
Speaker 1: about your research? Well, we're not entirely sure whether players
Speaker 1: and coaches were ever frowsled by the original study, so,
Speaker 1: you know, validating their beliefs for them. It's so, yeah,
Speaker 1: we kind of never believe that result to begin with.
Speaker 1: So so we haven't gone and sought the opinion of
Speaker 1: you know, players and coaches because it's it's not so
Speaker 1: clear how far that original conclusion reached into that world. Um,
Speaker 1: while it did, especially you can see announcers mentioning it. Um.
Speaker 1: But yeah, yeah, so so when you what about some
Speaker 1: of the outlawer players. If you look at a Michael
Speaker 1: Jordan's um or a Steph Carry, guys who literally become
Speaker 1: just conscious and what Reggie Miller is another one, and
Speaker 1: the most improbable shots on a consistent basis start to
Speaker 1: drop when when you look at players like that, do
Speaker 1: different players seem to have a different set of streakiness,
Speaker 1: a different hot hand? Can you can you calibrate how
Speaker 1: much of a hot hand different players have? So using
Speaker 1: game data that's a bit more of a challenge. So
Speaker 1: my couth or Adam and I looked at Spanish semipro players.
Speaker 1: We could collect a lot more data and we had
Speaker 1: more of their cooperation, and there seemed to be a
Speaker 1: clear difference with players. I mean, there's the obvious one
Speaker 1: is that you know, centers and forwards, people that don't
Speaker 1: shoot that often. It's hard for them to get on
Speaker 1: a roll because you have to be consistent and they're
Speaker 1: kind of not that consistent when they don't touch the ball.
Speaker 1: All that and all that, right, and so you know,
Speaker 1: those are the people you'd expect maybe you know, they
Speaker 1: can't really sustain a streak. And that's what we find.
Speaker 1: You know, so there's some players that can and some
Speaker 1: players that seem like they can't. Um. If we go
Speaker 1: to real NBA players, you know, that's a bit of
Speaker 1: a challe And so we have looked at the three
Speaker 1: point shooting contest and we have a paper on that. Um.
Speaker 1: The issue with the three point shooting contest is a
Speaker 1: lot of the players don't have much more than say
Speaker 1: a hundred shots total in the contest. You know, may
Speaker 1: some have a few more. You have a Craig Hodges
Speaker 1: who has over five hundred in our data, and we
Speaker 1: find evidence there. Um. But what we can say is
Speaker 1: that among all the three point shooting contest contestants, there
Speaker 1: were way more that did better after a few in
Speaker 1: a row than making a few row the missing a
Speaker 1: funeral than you'd expect, but you don't really know which
Speaker 1: of them are really hot. You just know there's more
Speaker 1: of them than you expect, but you need more data
Speaker 1: to be really confident when you pick out an individual.
Speaker 1: So at this point in the state of research on
Speaker 1: the hot hand, do you have any doubt that the
Speaker 1: hot hand exists? I don't have any doubt that the
Speaker 1: hot hand exists. What you mean by the hot hand
Speaker 1: is where the doubts come in, because there's many different
Speaker 1: mechanisms that can lead to evidence in the data that
Speaker 1: your success after recent success is higher than you know
Speaker 1: that it's higher than after recent So so the confidence factor,
Speaker 1: the endorphin factor, the further pressure that the other team
Speaker 1: is placing. All those things they add up. You ask
Speaker 1: a player, they're gonna say, yeah, of course you get hot.
Speaker 1: But when you ask the stat statistician, the data supports
Speaker 1: it as well. Right, quite fascinating. We have been speaking
Speaker 1: to Joshua Miller. He is an economics professor and researcher
Speaker 1: at the University of Alacante in Spain. If you enjoy
Speaker 1: this conversation, well be sure and come back and check
Speaker 1: out our podcast extras where we keep the tape rolling
Speaker 1: and we continue discussing all things statistical, sports and behavior.
Speaker 1: You can find that at iTunes, overcast, at your Bloomberg
Speaker 1: dot com, wherever your finer podcasts are sold. We love
Speaker 1: your comments, feedback and suggestions right to us at m
Speaker 1: IB podcast at Bloomberg dot net. You can follow me
Speaker 1: on Twitter at rid Halts, or check out my daily
Speaker 1: column at Bloomberg dot com slash Opinion. I'm Barry Ridhults.
Speaker 1: You're listening to Masters in Business on Bloomberg Road. Welcome
Speaker 1: to the podcast. So, Josh, I have to tell you
Speaker 1: I was very much a skeptic. Um a little background.
Speaker 1: So first, I'm a fan of Gilovich for a long
Speaker 1: time when I you could take noise. We don't know
Speaker 1: when I started in this business a hundred years ago
Speaker 1: as a trader. It was before the bad old days
Speaker 1: of behavioral economics had made its way to Wall Street,
Speaker 1: and I found a book by Gilovich, How We Know
Speaker 1: What Isn't? So it was the first mass book, more
Speaker 1: more popular book, not that it was all that popular,
Speaker 1: but it was the first book for a popular audience
Speaker 1: that had an enormous behavioral finance component. To it, so
Speaker 1: I found him absolutely intriguing. He led me down the
Speaker 1: rabbit hole of behavioral finance, and it's been an enormous
Speaker 1: influence on my um professional career because very often, when
Speaker 1: I couldn't figure out what the hell is going on
Speaker 1: according to what the head trader was saying, behavioral finance
Speaker 1: gave a much better answer. And the same is true
Speaker 1: when you're looking at markets, or the economy or what
Speaker 1: people get wrong. So my bias was to say, Traversky Gilvitch,
Speaker 1: these are two legends. Of course they're right. But I
Speaker 1: have to tell you this, having gotten through as much
Speaker 1: of your paper as I could until the formulas started
Speaker 1: to show up. It's a compelling argument that when we
Speaker 1: look at the data set, players on a streak from
Speaker 1: within that data set should have a lower shooting percentage
Speaker 1: following three in a row. Then you would intuitively inspect, expect,
Speaker 1: and when they don't shoot worse, It in and of
Speaker 1: itself is evidence of the hot hand. It's such an
Speaker 1: eloquent and unexpected way to do the analysis of the
Speaker 1: hot hand. I have to ask, how did you guys
Speaker 1: come upon that? I mean, I would never I'm not
Speaker 1: a statistician but I would never have thought, because so
Speaker 1: much of it is so intuitive, I would not have thought, hey,
Speaker 1: let's look at what the expected shot is, because with
Speaker 1: coins it should be fifty fifty. Why would you expect
Speaker 1: it to be anything less following three in a row?
Speaker 1: How did you sort of work your way towards that research?
Speaker 1: So you know, both my cauthor, Adamson ho Ho and
Speaker 1: I we didn't see any problem in that respect um
Speaker 1: with the original papers. So we didn't say, oh, they're
Speaker 1: clearly making a mistake here. No one did, you know,
Speaker 1: since we discuss for this thing, we've gone and we've
Speaker 1: asked statisticians, people that are very good. They look at
Speaker 1: the that test and they say, oh, you know, maybe
Speaker 1: it's underpowered, or they might have some little quibbles, but
Speaker 1: they don't have any expectation that you would shoot worse
Speaker 1: after funeral. In order to do that, you actually have
Speaker 1: to go out and simulate or go sit down and
Speaker 1: really calculate, and so it doesn't strike you in any way.
Speaker 1: So we discovered it. You know, it was a bit
Speaker 1: of a stroke of luck. Um. We were looking at
Speaker 1: the NBA three point contest data. We had to analyze
Speaker 1: it very quickly. Using a method different than the way
Speaker 1: we used it. So we just used the method of
Speaker 1: the original study, which was much quicker to run. So
Speaker 1: we ran that and we found this player who we
Speaker 1: knew was hot and have mentioned that earlier, Craig Hodges,
Speaker 1: and he shot no better after making a few in
Speaker 1: a row, and that just didn't make sense. Was it
Speaker 1: was that a brute force quit down and dirty? So
Speaker 1: so you moved to something a little more um sophisticated,
Speaker 1: what's the better word for this. So so the sophistication
Speaker 1: came later. So we we you know, we just took
Speaker 1: the test that used in the original study and that
Speaker 1: measure and it wasn't showing anything, and we that didn't
Speaker 1: agree with our perception of what we saw in those
Speaker 1: videos and and and and some of the elementary things
Speaker 1: he did, like he hit nineteen in a row at
Speaker 1: one point, never missed more than five, and he was
Speaker 1: around fifty percent shooter, which would be you'd never expect
Speaker 1: from a point and so, okay, nineteen a row is
Speaker 1: astonished as this instance. Yeah, it's incredible. So then we
Speaker 1: went and we said, well, what if he were a coin?
Speaker 1: What if Craig Hodges was a coin? So let's just
Speaker 1: generate his shots as if he was a coin where
Speaker 1: he's really and we repeat this, like, imagine we did
Speaker 1: this many many times, and look what we'd expect from all.
Speaker 1: You know, if we run this many times and we see, oh,
Speaker 1: you'd actually shoot worse after making a funeral, and that
Speaker 1: seems very count We were struck. This doesn't seem like
Speaker 1: it's right. But this is what the analysis is giving us.
Speaker 1: We have to understand this. This is what the data
Speaker 1: is saying. Two things we've discussed. One is after you
Speaker 1: have a streak of six in a row and you
Speaker 1: have a finite number of shots, well now there are
Speaker 1: six less heads in the in groups, so therefore there's
Speaker 1: a higher probability of tails after that. That makes perfect sense, right,
Speaker 1: because you're just changing the remaining data set by what
Speaker 1: you're looking at given a fixed data, given a fixed
Speaker 1: number of coins, fixed number of shots um. And then
Speaker 1: of course mean reversion assumes after a long streak of
Speaker 1: heads you should start to see a streak of tails,
Speaker 1: which like Gambler's fallacy a little bit. So let's let's
Speaker 1: go into that. Explain that. Yes, so I mean the
Speaker 1: gambler's fallacy is this idea that comes out of the casino,
Speaker 1: and it's been known for hundreds of years that if
Speaker 1: you see say five six blacks in a road or
Speaker 1: that table, it feels like that that the red must
Speaker 1: be more likely, right, um, And so people get drawn
Speaker 1: into this and they start betting more. Maybe, but it's
Speaker 1: still fort and the green. Yeah, but it's almost fifty
Speaker 1: fifty regardless, right, So in reality the probabilities haven't changed.
Speaker 1: But but when you but when you look at a
Speaker 1: when you look at a fixed data set that you
Speaker 1: expect to be fifty fifty, not not perspectively at the
Speaker 1: roulette table in real time. But we know that, hey,
Speaker 1: there's a hundred coin flips, we're gonna assume half of
Speaker 1: them are tails and half our heads. After you've had
Speaker 1: a wrong long streak of heads, the assumption is that
Speaker 1: out of that full data set, there should be more
Speaker 1: tails coming up. I'm I'm in real time, it's truly
Speaker 1: the gambler's fallacy. But when you're looking retrospectively with the
Speaker 1: data set, it's basically just a variation of, Hey, you've
Speaker 1: already exhausted a lot of heads, therefore there are more
Speaker 1: tails out there. Yes, exactly and as we mentioned before,
Speaker 1: there's a little bit there's an extra wrinkle on top.
Speaker 1: You know that it determines on how you know, how
Speaker 1: are these streaks ordered. So like when you when you
Speaker 1: pick a pick up a shot, because the previous three
Speaker 1: were heads, the shot you pick up, these are either
Speaker 1: heads or tails, but it's much more likely to be
Speaker 1: tails one because of the heads that were removed too,
Speaker 1: because if it were a tail, you've interrupted the streak
Speaker 1: and you you can't begin until you have to wait
Speaker 1: until you begin. So so there's you're pulling a big
Speaker 1: chunk of the possible selections out, so all the streaks
Speaker 1: come out they're all heads, so you're not picking that one.
Speaker 1: And and plus the total number of heads that you've used,
Speaker 1: so what's left becomes just from a data set group,
Speaker 1: what's less have become a not probability, which is which
Speaker 1: is fast. So you guys are doing this research, at
Speaker 1: what point do you say, holy cow, this is really
Speaker 1: a fascinating discovery, Like it's it's not just a tiny chance.
Speaker 1: Ten percent is a huge number in this sort of
Speaker 1: data series. When did you guys look at each other
Speaker 1: and say, hey, this is something really important. We knew
Speaker 1: it was a big deal of the moment we saw it.
Speaker 1: Really we were on the phone where you didn't set
Speaker 1: yourself this has to be wrong. Ten P. How did
Speaker 1: nobody pick this up? In thirty years? Nobody has seen this?
Speaker 1: So we we had this is two years after we
Speaker 1: had begun the project. Well maybe not that long, but
Speaker 1: almost two years, and we had read every paper in
Speaker 1: the literature, so we knew no one had had had
Speaker 1: nobody had seen this, no, no one had said so
Speaker 1: we knew it was a big deal for that literature.
Speaker 1: So the only question we had is how new. Well,
Speaker 1: I mean, we'd run you know, we knew the you
Speaker 1: know we can trust the computer, right, And of course
Speaker 1: you have to make sure you didn't make an error
Speaker 1: in your code. You have to sit down and do
Speaker 1: the simple example to make sure you didn't do a
Speaker 1: calculation there. So once we did that with okay, this
Speaker 1: is clearly a true thing. Now the only question is
Speaker 1: did anyone know this about coin flips before? Is this
Speaker 1: a new discovery about coin flips? And yes, there's some
Speaker 1: mathematical things that are somewhat related, but no, it was
Speaker 1: even new in that dimension, So we knew we had
Speaker 1: something really big and that was exciting because you have
Speaker 1: this moment where you're the only person in the world
Speaker 1: that knows something. It's kind of it's an exciting moment.
Speaker 1: I feel that way every day I wake up and
Speaker 1: I have that sensation, so I can appreciate you probably
Speaker 1: not as solidly based as as yours, at least that's
Speaker 1: what my wife wee. So that's amazing. You guys come
Speaker 1: up with this incredible breakthrough. Nobody has has found this.
Speaker 1: It's been decades and it's it's been widely accepted. It's
Speaker 1: become part of the cannon. But it's classic confirmation bias,
Speaker 1: which is so um reflexive and meta. There is a
Speaker 1: study that says people are fooled by randomness and think
Speaker 1: there are streaks, which turns out perhaps to be confirmation
Speaker 1: biased by behaviorists who are warning people against being fooled
Speaker 1: by randomness and seeing what they want to see. It's
Speaker 1: got a little bit of Mandel brought reflectiveness built into it.
Speaker 1: It's it's quite amazing. Yes, you know, in a sense
Speaker 1: that mistake proves kind of the spirit of the general
Speaker 1: point about misinterpreting randomness. Even the best of us, the
Speaker 1: best researchers there are out there still make these mistakes
Speaker 1: due to randomness, and while saying others are making the mistake,
Speaker 1: you're making the mistake even within. So they accidentally proved
Speaker 1: their point, which is it's very easy to be fooled
Speaker 1: by a random data set into thinking there's a broader
Speaker 1: conclusion there, until subsequent research discovers that, hey, this isn't
Speaker 1: quite as random as you think it is. There's a
Speaker 1: ten gap between true randomness and the remaining data set.
Speaker 1: That that's quite that's quite fascinating. So you guys look
Speaker 1: at each other and say, hey, we're onto something real.
Speaker 1: How did it progress from there? What year was this?
Speaker 1: This was? This was February. Found this and we knew
Speaker 1: so we knew it was important. So, um, we presented
Speaker 1: our work and when you see the eyes light up,
Speaker 1: you realize it's even bigger than you thought it was.
Speaker 1: And then you realize, hey, wait a minute, we don't
Speaker 1: have the paper yet, and now other people know about it.
Speaker 1: Who did you present it to originally? Um? So at
Speaker 1: Oxford University. That was the first review, right, and you
Speaker 1: know you see the eyes light up in the room. Um,
Speaker 1: are you genuinely concerned at that moment? Oh? Someone's gonna
Speaker 1: try and beat us to publication, and so we put
Speaker 1: everything aside and we just we we went to the grind.
Speaker 1: We within two months we had the paper and no
Speaker 1: one was going to catch you. At that point you
Speaker 1: would enough of the head start and you were the
Speaker 1: original people who found this. So two months later the
Speaker 1: preliminary papers come out. We put online. You put you
Speaker 1: posted online n b R and everywhere else or wherever
Speaker 1: you know, just get that time stamp right where wherever
Speaker 1: finer white papers are are sold. Um. And so that's
Speaker 1: what April of the paper went online June. What's the
Speaker 1: response to that? Um? The response was big. So statistician
Speaker 1: at Columbia University, Andrew Gelman, who has this blog and
Speaker 1: everybody's heard of Andrew Galman. Well, or let me rephrase that,
Speaker 1: anybody who's interested in statistics knows who Gellman It Columbia
Speaker 1: is fair, fair statement. He's at the crossroads of pretty
Speaker 1: much all the social sciences, sciences when it comes to
Speaker 1: data and statistics, right, and so getting attention from Andrew
Speaker 1: Gelman huge, It's huge, and that that was you know,
Speaker 1: high fives all around. Yeah, but it's also scary when
Speaker 1: you get attention from Andrew Gelman, because if you made
Speaker 1: a mistake, it's open peer review season. They're getting in
Speaker 1: there in the comments. He'll get you, you know, like
Speaker 1: they're just having fun. They love talking about data and
Speaker 1: and they're not gonna worry about how you feel about
Speaker 1: it because they're just interested in the main points, like
Speaker 1: what did the statistics say? And you're you know, you're
Speaker 1: sitting there sweat and ball. It's hoping you got you
Speaker 1: didn't make a mistake somewhere. It's at that level. This
Speaker 1: isn't you know, Twitter fights and ad homin Hum attacks.
Speaker 1: It's hey, let's get into the math. Let's see if
Speaker 1: they did they're crunching their numbers correctly. Let's see if
Speaker 1: we can find an error in their modeling. What what
Speaker 1: did Gelman discover? So Gelman went and did the work himself,
Speaker 1: and he found what he found agreed with what we found,
Speaker 1: and so he said, hey, guess what. There is a
Speaker 1: hot hand that was his post and then it's kind
Speaker 1: of snowballed from there. That's it. So then there's a
Speaker 1: Wall Street Journal piece on it, and then there was
Speaker 1: an ESPN or a sports illustrator was one of the
Speaker 1: sports are there, yeah, there there was ridiculous, like we're like, okay,
Speaker 1: when's the fifteen minutes gonna end? But I guess that,
Speaker 1: you know, the news world is so kind of balkanized
Speaker 1: by this point that like its not from subject to subject,
Speaker 1: it just kept rotating. And I saw something, and you
Speaker 1: guys published another a number of fair I have to say,
Speaker 1: you're published popular stuff. I think you undersold the math
Speaker 1: on this because it's not that you dumbed it down,
Speaker 1: it's that you were so circumspect. And so maybe modest
Speaker 1: is the right word. Like if I'm a different person
Speaker 1: than you, I would have written written something that said, dude,
Speaker 1: just listen up the whole no hot hand things. Let
Speaker 1: us show you why that's not true. Here's the math.
Speaker 1: It's ten. It's a giant impact, and here's why. Like
Speaker 1: I thought, you guys were very circumspect in your what
Speaker 1: was it the conversation or the yeah, the Australian conversation. Yes,
Speaker 1: that was like a fairly modest discussion. You know. I
Speaker 1: would have been like, hey, pay attention to this. We're
Speaker 1: changing an understanding of sports streaking nous, this is a
Speaker 1: big deal. What other applications are there? Of of the
Speaker 1: finding of both the flips of coins and the streak
Speaker 1: nous of shooters. Where else can this be applied? Are
Speaker 1: there other uses of this mathematical? Why should call its
Speaker 1: statistical observation? Yes? So the bias that that we found
Speaker 1: has uh and it can it can manifest itself in
Speaker 1: many areas. So it's not just about time, right, So
Speaker 1: we're looking at like how how you did recently? Does
Speaker 1: that affect how you do in the future or how
Speaker 1: you do next? Right? If we found some biases there,
Speaker 1: But it's not it's not about time. It's essentially about
Speaker 1: space because you're looking at data and we represent time
Speaker 1: with space because we have period one, period two, period three,
Speaker 1: they're all next to each other, and so you have
Speaker 1: this kind of one dimensional spatial thing continuing the line.
Speaker 1: But it can go in either directions. So it's not
Speaker 1: you know, time's arrow that's determining it. Right. If I
Speaker 1: hit three in a row, the chance that the previous
Speaker 1: one that just preceded that streak as it heads is
Speaker 1: actually lower two um for the exact same reason, which
Speaker 1: means that the actual streaking nous of the player isn't
Speaker 1: relevant to the prior one, even though we would expect
Speaker 1: it to be relevant to the subsequent one. It's all
Speaker 1: the same statistical data set prior less, less heads in
Speaker 1: the remaining pool, etcetera. So you can extend this beyond
Speaker 1: time and talk about space. Right, So if you're interested
Speaker 1: in you know, if I'm surrounded by you know, red people,
Speaker 1: am I more likely to be blue? You might go
Speaker 1: and hey, let's look at the data set. And this
Speaker 1: is the ping pun bulls in the vase statistical problem. Yeah,
Speaker 1: so you know, the people study segregation and clustering, you know,
Speaker 1: and where people live and things like this, and and see,
Speaker 1: you might go into data set and use this intuitive
Speaker 1: measure like let's see if I'm more likely to be
Speaker 1: blue if I'm surrounded by reds. Um, you have the
Speaker 1: same issue here now, you know, if I were a blue,
Speaker 1: I've kind of excluded other possibilities of being surrounded by reds.
Speaker 1: Wherever that blue is what it actually makes blue more
Speaker 1: likely for some of the same reasons why we have
Speaker 1: this bias, you know, when we're talking about time, and
Speaker 1: so there are potentially many other areas where bias is
Speaker 1: similar to this could could manifest themselves. It might be
Speaker 1: stumbling it so I remember a couple of years ago
Speaker 1: the cancer clusters around power lines, um, which a lot
Speaker 1: of statisticians came out and said, well, no, this is
Speaker 1: just you know, the heads and tails problem. Again, you
Speaker 1: have a lot all these non clusters around other power lines.
Speaker 1: So if it's a cause element, why is it causing
Speaker 1: it here but not a half mile down the same
Speaker 1: power line. It's just a random aggregation of data, and
Speaker 1: you're seeing something that it's of course you're gonna get
Speaker 1: ten heads in a row if you flip a coin
Speaker 1: a million times. That's all you're seeing. Do you have
Speaker 1: an application to those sort of of cognitive issues, um,
Speaker 1: So we haven't found the specific application, to be honest,
Speaker 1: we haven't scoured that literature. UM. You know, we have
Speaker 1: found papers that have measures of clustering, like how likely
Speaker 1: am I going to live next to someone of who's
Speaker 1: like me, you know versus not like me depending on
Speaker 1: who's around? Um. And there are some measures that are
Speaker 1: biased for a similar reason. Um that we have this
Speaker 1: bias now in the councilor cluster one. UM. You know
Speaker 1: that's that's a little bit different. UM. And it's and
Speaker 1: it's because of you know what you say is it's
Speaker 1: kind of blade of grass fallacy that you know. You know,
Speaker 1: there's lots of blade of grass. You know, you shoot
Speaker 1: a speck of water and it hits the blade of grass,
Speaker 1: the blade of grass. Oh look, I'm so lucky it
Speaker 1: was coming for me. What had to hit some blade
Speaker 1: of grace? Right now, someone's gonna win the lot. Someone's
Speaker 1: got to win the lot everyone. You know, the chances
Speaker 1: that somebody wins the lottery is super high. Not so much. Yeah,
Speaker 1: so that's interesting. So before I get to my favorite questions,
Speaker 1: I asked all my guests, I have to ask you
Speaker 1: what else do you guys um working on? What are
Speaker 1: the research is coming from the minds that brought you
Speaker 1: um proof that the hot hands exists? Well, you know,
Speaker 1: so in our world, it's very tempting to move on
Speaker 1: to the next one before you know, finishing what you started, right,
Speaker 1: so you have not exhausted everything out of this one
Speaker 1: one piece of day, right, So we have we have
Speaker 1: a lot of kind of uh you know, I used
Speaker 1: to dot, he used to cross. But you know a
Speaker 1: little bit more than that. You know, you want to
Speaker 1: you want to finish and get the message out, but
Speaker 1: also share, you know, the other insights that we have,
Speaker 1: because you know, they come out of the same you know,
Speaker 1: so this is our you know, say, the main insight,
Speaker 1: but there are other very subtle and interesting insights that
Speaker 1: we have because you know, when you master something, yeah,
Speaker 1: and you come back after working on something for a while,
Speaker 1: there's a lot to share. So tell us some what
Speaker 1: what other insights can be derived from from the hot
Speaker 1: hand papers. Yeah, so there's another result in in that study,
Speaker 1: um um Gilvich introversity study that Gilvich mentioned in the
Speaker 1: book that you talked about earlier, which is okay, so
Speaker 1: they kind of got that people, uh you know, that
Speaker 1: they measured hot hand in a certain way, and they realize, well,
Speaker 1: maybe we're not capturing everything that means about hot hand,
Speaker 1: and maybe some players are are seeing something that we,
Speaker 1: the statisticians, the econometricians you know, aren't measuring. And they
Speaker 1: went and had people predict and bet on outcomes and
Speaker 1: they found that they their bets don't really correlate with
Speaker 1: the outcomes, and so that's kind of evidence. Well, okay, well,
Speaker 1: even if we're not measuring everything, look if the players
Speaker 1: are seeing something you think they would bet successfully, and
Speaker 1: that you could also take that as evidence that now
Speaker 1: that there is a hot hand there, well, it's at
Speaker 1: least it's evidence that there's somehow not using exploiting it
Speaker 1: in a profitable way. But there was actually a mistake
Speaker 1: as well in in that in that analysis, which is,
Speaker 1: even if someone were perfect at detecting the hot hand,
Speaker 1: they knew you know, you can imagine Ann and Bob.
Speaker 1: Bob's a shooter and his you know, predictor she's observing Bob.
Speaker 1: She knows when Bob's hot, and whenever Bob's hot, she's
Speaker 1: going to predict that Bob's going to make the shot. Now,
Speaker 1: you would expect if she's good, then that good, then
Speaker 1: her bets, her predictions are going to correlate really well,
Speaker 1: um with Bob's outcome. But actually you wouldn't expect that.
Speaker 1: And that's that's another counterintuitive thing is that while she's
Speaker 1: perfect at detecting his state, the outcome of the state
Speaker 1: is noisy. You're just getting one draw from Bob's earns.
Speaker 1: So even if Bob's moving from a seventy to eight
Speaker 1: probability shooter, if you only take one draw from that urn,
Speaker 1: you're not getting a very good signal on Bob's state
Speaker 1: you need a lot. Yeah, and so even if you're
Speaker 1: getting many predictions from Ann and Bob, you're still only
Speaker 1: getting you know, one draw on each one. And so
Speaker 1: the evidence that they have there was actually enough evidence
Speaker 1: that's consistent with and being very good at detecting it
Speaker 1: and actually with you, Rhianna. Is the data you find
Speaker 1: that Bob shoots seven around seven percentage points better when
Speaker 1: and predicts that he's gonna make it, you miss it.
Speaker 1: So in their data they have real people that are
Speaker 1: paid basketball players are betting on each other's shots. And
Speaker 1: that's the evidence that we find that's quite interesting. That
Speaker 1: sounds that betting on the outcome of a shot sounds
Speaker 1: very much like fund managers selecting stocks for a portfolio.
Speaker 1: Have you applied any of the hot hands? Two? How
Speaker 1: do fund managers do when they're on a hot streak
Speaker 1: or a cold streak? And there's a ton of mean
Speaker 1: reversion in that data series, right, So we haven't gone
Speaker 1: and analyzed and the mechanisms for being hot in the
Speaker 1: financial world are going to be quite different than in
Speaker 1: the basketball world. Right. So, you know, one way of thinking,
Speaker 1: you look for sec indictments, you look for no I'm
Speaker 1: just kidding, um, for sure. It becomes so affected by
Speaker 1: such large macro things it's hard to give credit or
Speaker 1: or not or your You know, your model of the
Speaker 1: world happens to be uniquely fit the current situation, and
Speaker 1: you recognize that, but you know that may or may
Speaker 1: not be temperary, and that's going to know that that
Speaker 1: would probably expire, but that's a very different mechanism, and
Speaker 1: say how it would emergency a basketball game? So I
Speaker 1: interrupted you, what what else did you do you see
Speaker 1: as an application of this elsewhere, um, an application of our,
Speaker 1: of your of what you've discovered to to the world
Speaker 1: of finance, to the world of finance. So the immediate
Speaker 1: applications maybe not so much. But if you think about
Speaker 1: people picking stocks, let's say not so much investing, but
Speaker 1: someone wants to prove that they're good, um at predicting
Speaker 1: when a stock is going to go up or down.
Speaker 1: You have to pay attention not too how often they're right,
Speaker 1: but how much money they make when they're right and
Speaker 1: when they're wrong. Because it's very easy to gain these things.
Speaker 1: So if I were to say, let's say it's fifty fifty,
Speaker 1: I want to prove that I'm good at predicting coin flips.
Speaker 1: And so if every month you know a coin is flipped,
Speaker 1: each day stock goes up or down. But I only
Speaker 1: bet when there's three heads in a row. When the
Speaker 1: stock goes up three times in a row, and I
Speaker 1: bet it's gonna go down. Right in any given month,
Speaker 1: I'm gonna be right more often than I'm wrong, And
Speaker 1: so I can game you know if you don't, if you,
Speaker 1: if you, if you brack it to the month level,
Speaker 1: I'm gonna be more often right in certain months, and
Speaker 1: it's gonna look like I'm doing well. But the thing
Speaker 1: you haven't paid attention to is how often I was
Speaker 1: right and half when I was wrong in the months
Speaker 1: that I did poorly. And so if I'm always betting
Speaker 1: that it's gonna go down when there's a few ups
Speaker 1: in a row, there's gonna be those months when it
Speaker 1: keeps going down, right, But you know there's gonna be
Speaker 1: a few of those months, and there's gonna be many
Speaker 1: months when I did well, but I didn't predict very
Speaker 1: many times, and so you're not controlling for how often
Speaker 1: I predicted, and so it looks like like I do
Speaker 1: really well. But if I were to bet I wouldn't
Speaker 1: be making any money because I'd be losing a lot
Speaker 1: of money in the months where I didn't predict that well,
Speaker 1: only winning a little bit of money in the months
Speaker 1: that I predicted well. The interesting thing is if you
Speaker 1: talk to active traders UM who have been successful, they're
Speaker 1: not aiming for fifty fifty. They're aiming for those opportunities
Speaker 1: where a trade becomes a winner and they don't sell
Speaker 1: too early. So it's not your banning average, it's how
Speaker 1: far the ball goes when you actually hit it. That's right,
Speaker 1: Meaning you could have a winning trading record, but in
Speaker 1: terms of percentage of winning trades, but in terms of
Speaker 1: dollars one and lost, those twenty more than make up
Speaker 1: for the remaining eighty. And I always find a lot
Speaker 1: of new traders don't understand that. They think they're hitting
Speaker 1: for percentage, but they're not. They're heading for distance. To
Speaker 1: bring in a different sports metaphor anything else you want
Speaker 1: to share about UM, the research, or what you guys
Speaker 1: might have coming out in the near future, you and
Speaker 1: your co author UM. Off the top of my head, no,
Speaker 1: I think we The one thing that we will say
Speaker 1: is actually there's one thing, So it's not simply that
Speaker 1: we found that the original analysis was vowed in the
Speaker 1: original conclusions were invalid. You know, if you go back
Speaker 1: and you reanalyze that data, you find that players shoot
Speaker 1: a lot better. But it's not simply in that data set.
Speaker 1: So we've gone and collected many other data sets that
Speaker 1: replicated their original study got the same conclusion because they
Speaker 1: had the same bias that the original study had. And
Speaker 1: so when you go back and you fix that, you
Speaker 1: find evidence everywhere, and and and that's we have a
Speaker 1: paper that we're finishing that's showing how robust our conclusions were.
Speaker 1: So I know that there are all sorts of interesting
Speaker 1: UM awards from mathematical and statistical research. Are you looking
Speaker 1: at applying for any of these? How how does that work?
Speaker 1: Are you can you self nominate? Does the institution have
Speaker 1: to nominate you? How do you say? How how does
Speaker 1: that process work? Have you guys thought about this? It's
Speaker 1: not something we thought about. Oh well, let me plant
Speaker 1: that seed. And if this is significant enough, you should
Speaker 1: uh apply for either a grant or a mathematical award,
Speaker 1: although most of these you have to be nominated by
Speaker 1: other of people. But how hard is that to have
Speaker 1: your department share and nominate you. That's that's easy enough.
Speaker 1: I have to ask you. I didn't ask this earlier.
Speaker 1: You grew up in California, you went to Santa Barbara.
Speaker 1: How did you end up in Spain? Oh? So I
Speaker 1: think you remember two thousand and eight, two thousand nine
Speaker 1: a little bit. So that academic jar market was an
Speaker 1: interesting one. I was going on the market in late
Speaker 1: two thousand and eight, two thousand nine, and so a
Speaker 1: lot of academic appointments, non appointments, advertisers for positions were
Speaker 1: disappearing because of you know, the crisis. You would think
Speaker 1: academia is with large endowments and what have you, someone
Speaker 1: insulated from the vagaries of the stock market and even
Speaker 1: the broader economy. But apparently not yea. And so my
Speaker 1: advisor came to me and said, look, this year, everyone's
Speaker 1: applying everywhere, so you need to apply to Europe, even
Speaker 1: if you weren't even think, you weren't thinking about it.
Speaker 1: So at that time I applied everywhere and it was
Speaker 1: great because it opened my mind mine to the great
Speaker 1: opportunities that are there. So right, so I moved to
Speaker 1: Italy in two thousand nine. That was my first stop
Speaker 1: Where were you in Italy? Bacona University in Milan? That
Speaker 1: there are worse places in the world to ride out
Speaker 1: a recession. Yeah, no, it was a good It was
Speaker 1: a good time, I can imagine. And then from Milan,
Speaker 1: how do you end up in in Spain? So I
Speaker 1: wanted to join my co author and finish our work.
Speaker 1: Is that where he was located? Yeah, and he still is.
Speaker 1: So we're both the University of the Content on the
Speaker 1: lovely Mediterranean ceo of there there again, parts of that
Speaker 1: whole whole Millranian coast is just spectacular, isn't it. So
Speaker 1: you don't miss California too much? I get back a
Speaker 1: couple of times a year. Quite quite interesting. All right,
Speaker 1: let's jump to our favorite questions. I can't believe you
Speaker 1: guys never thought of saying, hey, maybe we should apply
Speaker 1: just for some of these grants and some of these
Speaker 1: award rates. We think of applying to grants, But the
Speaker 1: award thing, I don't know. Yeah, all right. I always
Speaker 1: thought academics had to do stuff like that in order
Speaker 1: to maintain their academic standing. Grants and I apply for money,
Speaker 1: that's for sure, But the the the recognition, um, it's
Speaker 1: a good idea. Yeah, not a bad idea. Let me
Speaker 1: let me when you give your acceptance speech. Yeah, definitely,
Speaker 1: all right. So, um, let's jump into a favorite questions,
Speaker 1: which I'll modify slightly because, um, most people, I think
Speaker 1: I don't know you personally. So let me ask the question,
Speaker 1: what's the most important thing, um, that your friends and
Speaker 1: family don't know about you? Friends and families? So you
Speaker 1: had mentioned this question earlier, and I was going to say,
Speaker 1: the most important thing we've actually already revealed, which is
Speaker 1: which is it's what most people didn't know we haven't
Speaker 1: shared this much? Is that this might be in this research, um,
Speaker 1: the first joint eureka moment, right, Usually someone discovers something.
Speaker 1: You when they say simultaneous discovery, someone discover something at
Speaker 1: one point in time and somebody at another, but no
Speaker 1: one knew about it. Did The electric light bulb as
Speaker 1: a classical radio is another classic example. So, but my
Speaker 1: cauthor and I were in this, you know, we were
Speaker 1: in the same room, we were on the phone at
Speaker 1: the same time, and we both had the you know,
Speaker 1: the gold is there and then how often does that happen? So?
Speaker 1: Who were some of your mentors in your early career?
Speaker 1: So in my early career, um, I would say, and
Speaker 1: my author would say the same. You know, my my advisor,
Speaker 1: Alto ruster KINI is a professor at University of Minnesota,
Speaker 1: and he's just you know, he's a neuroscientist, he's a mathematician,
Speaker 1: he's an economist. He's all about the science, right, And
Speaker 1: I'm thinking a renaissance person. Oh, he's a renaissance person.
Speaker 1: And you know when you see that, um, and you
Speaker 1: see someone that's just you know, zeroed in on that
Speaker 1: and you see how they work, you kind of you
Speaker 1: get you kind of absorbed what they do through osmosis.
Speaker 1: And my CA author with the same to say the
Speaker 1: same thing about his advisor, Vince Crawford at Oxford University
Speaker 1: and a very deep guy, very brilliant, you know, both
Speaker 1: very brilliant. Um. And and those are formative years when
Speaker 1: you're in grad school for for sure. For sure. UM,
Speaker 1: So what other behaviorists and statisticians influenced your approach um
Speaker 1: to thinking about the mathiness of things like shooting streaks?
Speaker 1: So that I would say the statisticians that have influenced
Speaker 1: me out there is one, right, there's Andrew Galman reading
Speaker 1: his blog has been eye opening for so many people.
Speaker 1: I mean, he just introduced how to think about data
Speaker 1: in a way that most people don't get in their
Speaker 1: formal training because he's dealing with real practical examples all
Speaker 1: the time. Um, and so I would say he's been
Speaker 1: one of the biggest influences interesting about the what about
Speaker 1: on the behavioral side? On the behavioral side, there's just
Speaker 1: so many great you know. I mean, there there was
Speaker 1: this vanguard of the people that the folks that came
Speaker 1: in in the eighties that really had to fight through,
Speaker 1: you know, the review process of all the skepticism towards
Speaker 1: you know, why a psychology you know, relevant to economics,
Speaker 1: Why are these other social science disciplines? What do they
Speaker 1: have to say about? People really had to fight a
Speaker 1: lot of skepticism. So give us some names. I'm putting
Speaker 1: it on the spot, Okay. So the people that had
Speaker 1: to fight through that, I mean that's like, you know,
Speaker 1: Amos Tversky and Daniel Kaneman were very influential, but they
Speaker 1: know they were within psych psychologist they were fine. So
Speaker 1: the people that had to deal, you know, with this
Speaker 1: kind of pushback. I mean you say, like you know,
Speaker 1: Richard Taylor, you know, as much as you know he
Speaker 1: he's been a bit skeptical of our work, but you
Speaker 1: have to respect, you know, and and you know, both
Speaker 1: his the insights he has into human behavior UM, and
Speaker 1: also just what he had to fight through get get
Speaker 1: listened to UM. So he was a guest, and my
Speaker 1: favorite quote from him was UM. Early on he decided
Speaker 1: he would never convince his peers. So he thought, I'm
Speaker 1: going to bypass them and just try and convince the
Speaker 1: grad students, and we'll just wait it out. After enough funerals,
Speaker 1: we will have one. And and it's really turned out
Speaker 1: to be quite true. If you if you are influencing
Speaker 1: the next generation, that's more and more impactful than what
Speaker 1: Tversky said, winning all these arguments and convincing nobody. Uh,
Speaker 1: it turned out to be very clever. Anybody else you
Speaker 1: want to mention from that group or oh no, I mean,
Speaker 1: I mean I wouldn't want to single out any person,
Speaker 1: you know, you just you look at the people that
Speaker 1: really kind of did a lot of the fighting that
Speaker 1: you're kind of push push the ideas through. But you know,
Speaker 1: in terms of the ideal dal level, there's so many right,
Speaker 1: you know, so even in the you know, when we
Speaker 1: were presenting our work. You know, you have somebody to say,
Speaker 1: like Colin camer cal Tech. You know, he actually just
Speaker 1: met him at a conference and he's a he's really
Speaker 1: a fascinating dude. Oh yes, And you know there's a
Speaker 1: lot of similarities you know that I recognize in him.
Speaker 1: That's kind of similar to the advisor that Altarosta Kidia
Speaker 1: University of Minnesota. You know, just this, You know that
Speaker 1: he's all about the science and really, I mean he
Speaker 1: came to one of our talks and he brought up
Speaker 1: Footnotes seventy two, which was the weakest point, the point
Speaker 1: he really wanted, and he found it. He found it.
Speaker 1: You know, we had a nice discussion about it. He
Speaker 1: saw our perspective after we had to talk. It's like, Wow,
Speaker 1: he's really taking this seriously, and it's just it's it's
Speaker 1: nice to see that that. That's got to be so delightful.
Speaker 1: I believe we have him tied up for the spring
Speaker 1: as a guest. Um. Yeah, he's really I love the
Speaker 1: work he does with virtual reality and showing people in
Speaker 1: incredible detail what they're gonna look like when they're older,
Speaker 1: and it affects their decision making. Dramatically in terms of planning,
Speaker 1: not just like a computer generated picture that's been aged,
Speaker 1: but when you have this immersive VR experience of here's
Speaker 1: your life when you're eighty, it leads to all sorts
Speaker 1: of amazing changes when you're forty. It it's quite astonishing.
Speaker 1: I'm gonna to do that. So I'm glad, glad you
Speaker 1: brought that up. Um, so I'm gonna put down Gellman
Speaker 1: as one of those people who influenced your approach to
Speaker 1: UH statistics. Let's talk about books. What are some of
Speaker 1: your favorite books? So I'll pick out a book. Um,
Speaker 1: you know, we could pick out a lot of nonfiction
Speaker 1: books and a lot of books like that. It hits
Speaker 1: you at the right time. And then if I tell
Speaker 1: you that book, I might you know, if I were
Speaker 1: to look at it now, it might feel trivial obvious
Speaker 1: things like this. You never know if the book is
Speaker 1: targeted for the right person. So I'll bring up a
Speaker 1: book that both my cauthor and I were a lot
Speaker 1: of were very much influenced by. But it's literature. So
Speaker 1: there's this book called The Alexandria Quartet by Lawrence Durrow.
Speaker 1: We both read it in our university days, and it's
Speaker 1: kind of like The Blind Men and the Elephant, but
Speaker 1: for human relationships, and it has this really novel. Idea's
Speaker 1: four books. The first three books are about three different
Speaker 1: perspectives on a serious relationships and events that happened in
Speaker 1: a particular time in Egypt before World War two, um,
Speaker 1: and from different people's perspectives. And so that's the space
Speaker 1: kind of. So it was inspired a bit by Einstein's relativity.
Speaker 1: So there's like three different perspectives on space and then
Speaker 1: they go forward in time and do the reflection back
Speaker 1: on on those relationships and it gives you, you know,
Speaker 1: this kind of you know, humility, kind of see like
Speaker 1: how how small your perspective is, how much missing information
Speaker 1: you have about what's happening, And it's it's it's it's
Speaker 1: a it's a nice read, at least it was. And
Speaker 1: when I was in my twenties, that sounds quite fascinating
Speaker 1: that this question is what people ask me more about
Speaker 1: than any other question because they want to get a
Speaker 1: book recommendation from somebody who's accomplished something done, something interesting,
Speaker 1: has some experience. And when someone says, oh, and by
Speaker 1: the way, this book is worth reading, it's the greatest
Speaker 1: endorsement anybody can ever get. So I'm going to press
Speaker 1: you and say, give us one or two more books,
Speaker 1: even if you think they may have been very time
Speaker 1: specific to you. So Duncan Watts has this book called
Speaker 1: Everything Is Obvious. Beautiful book. It's so interesting. It's all
Speaker 1: about client side bias and how you see things after
Speaker 1: the fact. It's it's you're the first person who's brought
Speaker 1: that book up and I find it. I love the
Speaker 1: cover with the wheel. I think that's a triangle instead
Speaker 1: of a circle. It's um, it's really a very fascinating book.
Speaker 1: It's you know, it's as you have this curse of knowledge,
Speaker 1: right once you know something, it's obvious to you and
Speaker 1: you can't imagine how not obvious it would be to
Speaker 1: someone else. And in the references in that book, I
Speaker 1: mean he you know, he's an academic, so he's really
Speaker 1: given you, you know, the road map, and like if
Speaker 1: you want to go beyond that book, all the references
Speaker 1: are there in that book. It's great. Um. And then
Speaker 1: I would say another book that's like Along the Lines,
Speaker 1: only because in the last year I read it. Um.
Speaker 1: You know. Super Forecasting by Philip another prior guest, delightful. Yeah,
Speaker 1: I mean it gives you, you know, humility to kind
Speaker 1: of realize how how you know, if you want to
Speaker 1: start projecting three years, five years and you're wasting you're
Speaker 1: wasting your time. But you know there's a lot more
Speaker 1: than that, right, just a disciplined approach. Right, So it's
Speaker 1: not simply you know and one one mistake you can make,
Speaker 1: because you know mistake I made, say around the financial
Speaker 1: crisis time, I really was convinced City Bank would be
Speaker 1: bailed out, they wouldn't let City Bank completely crash. Well,
Speaker 1: you were, you were not wrong. They did bail it. Eventually,
Speaker 1: it was two hours at the time, but still they
Speaker 1: were bailed out. Had you said the same about leaven Brothers,
Speaker 1: that would have been a different situation. What's most fascinating
Speaker 1: about Tetlock talk about recognizing your own issue. Tetlock's original book,
Speaker 1: If You Go Back, was on expert political judgment and
Speaker 1: that nobody is good at forecast now, and he then
Speaker 1: over time led to what what led from him going
Speaker 1: from oh, we're really bad as a species at forecasting too,
Speaker 1: But a handful of people have do a number of
Speaker 1: things that make them better at it, and therefore that's
Speaker 1: how you end up with super forecasters. That's a fascinating
Speaker 1: arc over I don't know, third twenty years separating the
Speaker 1: two books. Yes, and and the thing I really got
Speaker 1: from that book is not getting fixed at it on
Speaker 1: your your one insight and putting all your cards on that.
Speaker 1: You know. So, so you know, these these super forecasters
Speaker 1: are right in the long run. You know they're using
Speaker 1: the law of large numbers. It's not that they're saying, oh,
Speaker 1: I have this one idea, I'm gonna fixate city Bank
Speaker 1: has to be bailed out. No, no, take all your
Speaker 1: ideas and spread your bets across all your ideas, just
Speaker 1: like the forecasters, and you'll do well eventually. But don't
Speaker 1: get fixated on that one. And I think that's a
Speaker 1: nice feature, but quite fascinating any of the books before
Speaker 1: we move on, I think that's good. Those three, those
Speaker 1: are three three good ones. Um, so what are you
Speaker 1: excited about right now? What what do you jazzed about
Speaker 1: in the world of academic research. Well, and I think
Speaker 1: mastery is addictive, right, So there's a story. There's a
Speaker 1: lot of drive by research out there, and you know,
Speaker 1: we're all a little guilty of it because you know
Speaker 1: there's this pressure to publish, and when you've gone and
Speaker 1: you've really dug into something, you've really mastered something, and
Speaker 1: just mind all as much of the gold as you have,
Speaker 1: but you also have this feeling of mastery. It just
Speaker 1: feels so great and wanting to do that again. Right, So,
Speaker 1: so the exciting thing is to take that understanding of
Speaker 1: how good it feels, how fun it is to master something,
Speaker 1: and take it to the next subject. Well, of course,
Speaker 1: still finishing what you started. So that's that's the exciting things.
Speaker 1: What's next really really interesting? There have been all sorts
Speaker 1: of criticisms of the lack of reproducibility in academic research.
Speaker 1: What changes are you looking forward to into. Do you
Speaker 1: think that increased big data and AI is ever going
Speaker 1: to help us with this reproducibility problem we're running into
Speaker 1: in academic research and in other research. Why aren't we
Speaker 1: seeing academic research being replicated and and even corporate research
Speaker 1: being replicated. I think there's a lot of cherry picking
Speaker 1: that happens. Um. So when you go out and you
Speaker 1: analyze something, and you measure ten different things, and you
Speaker 1: just pick out the things that worked, Um, you're you're
Speaker 1: not acknowledging what didn't work, and so you're gonna you
Speaker 1: have this kind of you know, winner's curse in a
Speaker 1: sense where you know I won the that's sailors of
Speaker 1: his early books. Yeah, we don't have time to explain
Speaker 1: that I realized. But but um so, let me re
Speaker 1: ask that question. So what are you looking forward to?
Speaker 1: What changes do you think you're gonna affect um your
Speaker 1: your world, the changes that are gonna affect the academic world.
Speaker 1: Um so, I think that the important change that's going
Speaker 1: to make this better. They're gonna gonna fix this, fix
Speaker 1: maybe not, but make it better. Is the idea of preregistration,
Speaker 1: meaning what you pre register what you're going to analyze.
Speaker 1: You register your predictions, and so you know your hands
Speaker 1: are tied. So therefore you're going to go out and
Speaker 1: actually research what you're claiming as opposed to, oh, look
Speaker 1: at this anomaly, let's talk about that, even though it
Speaker 1: could be random or cherry pick or what have you,
Speaker 1: and still valuing even whatever your conclusion is, don't take
Speaker 1: that as truth when we have to make sure it
Speaker 1: also replicates, because even then you may have what if
Speaker 1: you don't find it you decide not to write it up, Right,
Speaker 1: isn't that a big issue that people don't publish on
Speaker 1: um negative find things because there is value to say, Hey,
Speaker 1: we analyze this, couldn't find anything. It's a huge issue
Speaker 1: because then you get this kind of implicit cherry picking
Speaker 1: and that like, I don't want to spend my time
Speaker 1: writing up this paper because it's not a big finding. Well,
Speaker 1: then no one's seeing that. So then the papers we
Speaker 1: see are the ones that are kind of implicitly selected,
Speaker 1: and so you have the same kind of degrees of
Speaker 1: freedom that's happening. But it's like socially in what do
Speaker 1: you call survivorship bias about things that don't work out?
Speaker 1: So I guess it is just straight up survivorship bias, right,
Speaker 1: In other words, what's public's negative shelter? You know, the
Speaker 1: research ideas that don't make it to the publication stage
Speaker 1: have died, and so the publication ones are kind of
Speaker 1: the ones that are kind of randomly better but not
Speaker 1: necessarily truly better. Interesting tell us about the time you
Speaker 1: failed and what you learned from the experience. So one failure,
Speaker 1: it was a success in a failure. So a friend
Speaker 1: of mine, Patrick Flannagan from graduate school. We said, up
Speaker 1: this garage band hedge fund we called it. We were
Speaker 1: loaning money on the internet, and we thought we had
Speaker 1: this great idea and it was I mean, it weathered
Speaker 1: the crisis. We didn't lose money. We got like five percent.
Speaker 1: We weren't. It wasn't big money. It's maybe a hundred
Speaker 1: thousand or something. We're students, but this is peer to
Speaker 1: peer lending. That But um, the thing we didn't anticipate was,
Speaker 1: you know, the legal uncertainty of the of the enterprise.
Speaker 1: And we weren't lawyers, and we just oh we we
Speaker 1: we modeled. We we had a real confidence in our model.
Speaker 1: We had our automated bidding algorithm. It was like it
Speaker 1: was great, we're doing well. But the thing we didn't
Speaker 1: get is that the SEC could potentially crack down on
Speaker 1: on this and ruin our business model because then they
Speaker 1: changed the rules completely that made it impractical. So we
Speaker 1: just we just left. Because instead of having a direct
Speaker 1: a connection to the person that you're loaning, and it
Speaker 1: was now mediated through the company, and now you had
Speaker 1: to somehow pricing the risk of the company itself rather
Speaker 1: than a loan. So um and and so interest. Yeah,
Speaker 1: quite interesting. What do you do for fun when you're
Speaker 1: not crunching numbers? Well, I mean this is pretty fun
Speaker 1: right here, I am in New York. You know, you
Speaker 1: get to travel around, you get to meet with your
Speaker 1: co authors, finish your papers, and nice locations, and you know,
Speaker 1: meet interesting people. Um, I would say just seeing family,
Speaker 1: because I get to travel so much. I get to
Speaker 1: see my family and friends in different cities, and that's
Speaker 1: a great thing. What sort of advice would you give
Speaker 1: to a millennial or recent college grad who was interested
Speaker 1: in a career either in behavioral finance or stat statistics
Speaker 1: or um, any of the sort of work that you
Speaker 1: you do, economics, whatever. Don't be in too impatient to
Speaker 1: have life figured out. Um, it's not too late. I've
Speaker 1: seen people in their twenties and their thirties go back change,
Speaker 1: go back to school. You know, maybe they have to
Speaker 1: start a little lower rank school than they wanted to
Speaker 1: begin with. But you can get funding at those schools,
Speaker 1: and if you work hard, you can transfer, you can
Speaker 1: apply to another school, you can move up. And a
Speaker 1: lot of people get this kind of false notion that, oh,
Speaker 1: if I didn't if I wasn't a serious student, in
Speaker 1: high schoore in college that I'm too far behind. It's like, no,
Speaker 1: if you're really motivated and you're you know, you're you're capable,
Speaker 1: and there's plenty of people that are. You can catch up.
Speaker 1: You just have to be patient, take a few years off.
Speaker 1: It's never too late to get serious. Yeah, never too
Speaker 1: late to get serious. And our final question, what do
Speaker 1: you know about the world of statistics and data analytics today?
Speaker 1: You wish you knew a decade or so ago. Fake
Speaker 1: data simulation basically creating you want to know. You can't
Speaker 1: just go out and analyze data and show that however
Speaker 1: I analyze it, I still get the same result. No,
Speaker 1: you have to sit down and generate fake data. So
Speaker 1: what if the world looked like this, how would my
Speaker 1: analysis behave? What if the world look like that? How
Speaker 1: the analysis behaves? You have to do this hard work
Speaker 1: of like building models of the world and then seeing
Speaker 1: what does your analytical approach tell you under those different
Speaker 1: kind of assumptions about the model of the world. And
Speaker 1: to do that you need fake data. And that when
Speaker 1: you say fake data, I think of that as a
Speaker 1: counterfactual or how do you contextual? I mean, I guess
Speaker 1: everything is a counter factual because all models are wrong, right, Um,
Speaker 1: But but but some are useful, but some are useful,
Speaker 1: And you want to know how how you know under
Speaker 1: under different assumptions. If the world you know looked differently
Speaker 1: than you think it looks, is your analysis still gonna
Speaker 1: say something meaningful or not? Um And you need to
Speaker 1: actually go out and check that, And a lot of
Speaker 1: people don't do that. And so that's what happened in
Speaker 1: this hot hand example, right, I mean, what would this
Speaker 1: analysis give you if there were no hot hand? Quite
Speaker 1: quite interesting. We have been speaking to Josh Miller of
Speaker 1: the University of Alcante. If you enjoy this conversation, we'll
Speaker 1: be sure and look up an inch or down an
Speaker 1: inch on Apple iTunes, where you can see the past
Speaker 1: two and fifty or so previous conversations we've had. We
Speaker 1: love your comments, feedback and suggestions right to us at
Speaker 1: m IB podcast at Bloomberg dot net. I would be
Speaker 1: remiss if I did not thank the crack staff that
Speaker 1: helps put together these conversations each week. Medina Parwana is
Speaker 1: our producer, Michael bat Nick is my head of research.
Speaker 1: Taylor Riggs is our booker. Slash producer. Attica val Bron
Speaker 1: is our project manager. Tim Harrow is our audio engineer.
Speaker 1: I'm Barry Ritolts. You've been listening to Masters in Business
Speaker 1: on Bloomberg Radio.