Speaker 1: Hey, welcome to Stuff to Blow Your Mind. My name
Speaker 1: is Robert Lamb and I'm Joe McCormick. And it's Saturday.
Speaker 1: Time to go into the vault. This episode originally published
Speaker 1: on August three, and it's called Regression to the Mean,
Speaker 1: which is about regression to the mean. Uh, it's an
Speaker 1: episode about a statistical phenomenon. But don't don't don't run away.
Speaker 1: It's actually, I think super interesting and having this idea
Speaker 1: in your toolkit really helps you understand the information that
Speaker 1: you encounter in the world, uh much better. Absolutely, So
Speaker 1: let's dive right in. Welcome to stot to Blow Your
Speaker 1: Mind production of My Heart Radio. Hey, welcome to Stuff
Speaker 1: to Blow Your Mind. My name is Robert Lamb and
Speaker 1: I'm Joe McCormick. And in today this episode, we are
Speaker 1: going to be focusing on a topic that is already
Speaker 1: something that's very well known to people who are familiar
Speaker 1: with quantitative research and statistics, but less known to the
Speaker 1: general public. And uh, and I think that's a tragedy
Speaker 1: because it's an idea that should really be part of
Speaker 1: everybody's basic critical thinking tool kit, no matter what your
Speaker 1: job is. And so in order to introduce this concept.
Speaker 1: I thought it would be best to start with a
Speaker 1: with a direct illustration from the real world of people
Speaker 1: reaching incorrect conclusions by not understanding the subject of today's episode.
Speaker 1: And so the illustration I want to start with is
Speaker 1: an interesting story told by the psychologist Daniel Kaneman that's
Speaker 1: about the illusory power of screaming at pilots. Uh So,
Speaker 1: the context of the story is that Knemon says he
Speaker 1: was giving a lecture about positive reinforcement to a group
Speaker 1: of flight instructures. I think this was in the nineteen sixties,
Speaker 1: and Kneman was trying to inform them about what he
Speaker 1: believed at the time was the best consensus of scientific
Speaker 1: research on learning and reinforcement, which was at the time
Speaker 1: that if these flight instructors wanted their students to have
Speaker 1: the best possible outcomes, they should focus more on praising
Speaker 1: the students when they did well, then on chewing them
Speaker 1: out when they did something wrong. And Knomon says that
Speaker 1: when he finished his talk, one of the flight instructors
Speaker 1: that he had been giving this lecture two got up
Speaker 1: and tried to dispute him. He said, no, you're wrong.
Speaker 1: And so the direct quote economy and gives from the
Speaker 1: instructor here is, on many occasions I have praised flight
Speaker 1: cadets for clean execution of some aerobatic maneuver, and in general,
Speaker 1: when they try it again, they do worse. On the
Speaker 1: other hand, I've often screamed at cadets for bad execution,
Speaker 1: and in general they do better the next time. So
Speaker 1: please don't tell us that reinforcement works and punishment does not,
Speaker 1: because the opposite is the case. So you might think
Speaker 1: he has a good point here. If you accept that
Speaker 1: this flight instructor has had a lot of direct experience
Speaker 1: working with students, and you trust him to remember the
Speaker 1: relative frequency of these events pretty well, you might assume
Speaker 1: that he has a meaningful rebuke to konom In. Here again,
Speaker 1: he says that most of the time, after a cadet
Speaker 1: does something bad and he screams at them, they do
Speaker 1: better the next time, And after a cadet does something
Speaker 1: good and he praises them, they actually do worse the
Speaker 1: next time. So if he's remembering these experiences correctly, and
Speaker 1: he's had a lot of them, it would really seem
Speaker 1: like evidence that praise has a negative effect on learning,
Speaker 1: maybe by making the student pilots soft and overconfident. Or something,
Speaker 1: and getting chewed out is good for skill development. I
Speaker 1: think it's quite easy to see the allure of this,
Speaker 1: this false conclusion right right, And it's and you can
Speaker 1: also easily imagine how you kind of build upon this
Speaker 1: with certain loosely backed up you know, folk ideas about
Speaker 1: how you encourage people and how people learn, and you
Speaker 1: got to stay on and if they if you tell
Speaker 1: them they're doing a good job, they'll get lazy, right,
Speaker 1: folk wisdom, tough guy mentality. But Koneman saw something different
Speaker 1: in this response, and he says that he immediately set
Speaker 1: up an experiment on the spot to demonstrate the flaw
Speaker 1: in the flight instructors thinking here, so I want to
Speaker 1: read from Knomen's description, He says, I immediately arranged a
Speaker 1: demonstration in which each participant tossed two coins at a
Speaker 1: target behind his back without any feedback. We measured the
Speaker 1: distances from the target and could see that those who
Speaker 1: had done best the first time had mostly deteriorated on
Speaker 1: their second try, and vice versa. But I knew that
Speaker 1: this demonstration would not undo the effects of lifelong exposure
Speaker 1: to a perverse contingency. So to explain this, this experiment
Speaker 1: a little bit better. Right. He has people stand with
Speaker 1: their backs to a target so they couldn't see it,
Speaker 1: and they would take two a attempts to throw a
Speaker 1: coin and hit the target without any feedback of any kind.
Speaker 1: So they're not getting praised, they're not getting chewed out, nothing. Uh.
Speaker 1: And after staging a number of these, he found again
Speaker 1: what he suspected, that the people who were the closest
Speaker 1: on the first throw did worse on their second throw,
Speaker 1: and the people who were farthest away on their first
Speaker 1: throw tended to do better on the second throw. So
Speaker 1: what condiment is actually demonstrating here is something that doesn't
Speaker 1: really have anything to do with learning or reinforcement, or
Speaker 1: really skills or even human psychology. Instead, this demonstration is
Speaker 1: showing the effects of chance, luck, and statistics. What he
Speaker 1: was showing is the subject we're talking about today, regression
Speaker 1: to the mean. Uh. You'll you'll see that phrase a
Speaker 1: lot in in scientific literature and in statistics. But if
Speaker 1: it helps to put it in more everyday terms, anytime
Speaker 1: you see regression to the mean, you can translate it
Speaker 1: in your head as trending toward the average, trending toward
Speaker 1: the average. So to make the coin tossing illustration even clearer.
Speaker 1: Imagine you throw the coin not twice, but that you
Speaker 1: throw the coin a hundred times. So you stand there
Speaker 1: throwing the coin a hundred times. And then let's say
Speaker 1: afterwards you average together the distance from the target across
Speaker 1: all a hundred throws, and you'll come up with some
Speaker 1: kind of average distance from target. Uh, just to make
Speaker 1: up a number for the sake of argument. Common doesn't
Speaker 1: give this. But let's say the average distance from the
Speaker 1: target across all your throws is nine centimeters. And remember
Speaker 1: that you're getting no feedback at all here, so it's
Speaker 1: unlikely that you will be getting much better as you
Speaker 1: go on. So, given that the average distance from the
Speaker 1: target is nine cimeters, if you throw a coin once
Speaker 1: and it happens to be two centimeters from the target
Speaker 1: so really close, is your next throw likely to be
Speaker 1: about the same as that one, better or worse. Obviously,
Speaker 1: it is overwhelmingly likely that your next throw will be worse,
Speaker 1: just due to chance, probably closer to the average of
Speaker 1: nine cimeters away. And the same goes for throws that
Speaker 1: are really far off. You throw something three hundred centimeters
Speaker 1: off your next random toss just by chance is likely
Speaker 1: to be much better, much closer. So simply put, most
Speaker 1: of the time, if you're sampling something in a series
Speaker 1: over time, if one sample produces an extreme value, the
Speaker 1: next one in the series is more likely to be
Speaker 1: closer to the average instead of extreme in the same way.
Speaker 1: In my experience. Uh, this is this is why it
Speaker 1: can sometimes be liberating to start off a game of
Speaker 1: bowling with just a disastrous gutter ball, because because I
Speaker 1: know that I'm good enough that that's probably not going
Speaker 1: to happen twice in a row, but it's definitely going
Speaker 1: to happen at some point in the game because I'm
Speaker 1: not that good, you know. I put like playing you know,
Speaker 1: once a year or even with less frequency these days.
Speaker 1: Oh yeah. And also like why I think a lot
Speaker 1: of us have intuitions that when you try something for
Speaker 1: the first time and you do really good on the
Speaker 1: first attempt, that makes you kind of nervous because you
Speaker 1: just know you're probably not going to live up to
Speaker 1: that repeatedly. Yeah, like if you get if you get
Speaker 1: a strike that first time, then that that first um
Speaker 1: what is it round? I can't even remember. This is
Speaker 1: how one frequently I bowl, Um, the first role. So
Speaker 1: the first role first, the first column. You know, So,
Speaker 1: the tendency of regression to the mean or or trending
Speaker 1: towards the average is pretty obvious when you're dealing with
Speaker 1: something like lots of random coin tosses with no feedback,
Speaker 1: But it becomes much more obscure when you're dealing with, say,
Speaker 1: a more more limited numbers of outcomes. In the series,
Speaker 1: you're looking at and introducing possibly influential variables like pilot
Speaker 1: skill and instructor feedback. After all, we would expect that
Speaker 1: some variables having to do with instructor feedback should have
Speaker 1: an effect on pilots ill, right, That's the point of
Speaker 1: teaching is to have an effect over time, and after all,
Speaker 1: in this one scenario, the conomen describes the the instructor
Speaker 1: believed that his verbal abuse of the students was so
Speaker 1: motivating that it made them instantly better on the stick.
Speaker 1: And you can't necessarily rule that out, but it's unlikely.
Speaker 1: I think I'm convinced that regression to the mean could
Speaker 1: more easily explain this flight instructor's belief that screaming at
Speaker 1: pilots for screw ups made them better at planes, because, again,
Speaker 1: on average, even in the absence of any feedback at all.
Speaker 1: If a pilot in training executes a maneuver perfectly, the
Speaker 1: random fluctuation from one execution to the next will tend
Speaker 1: to mean that their next attempt probably won't be as
Speaker 1: good as that really good when the last time. And likewise,
Speaker 1: if they make a major error totally botch a maneuver,
Speaker 1: they're more likely to do better the next time just
Speaker 1: by chance. Both of these tendencies are regression towards the mean.
Speaker 1: But then Conomon actually draw is a really interesting observation
Speaker 1: about about about our psychology and about culture from this fact,
Speaker 1: so to quote him directly, this was a joyous moment
Speaker 1: in which I understood an important truth about the world.
Speaker 1: Because we tend to reward others when they do well
Speaker 1: and punish them when they do badly, and because there
Speaker 1: is regression to the mean, it is part of the
Speaker 1: human condition that we are statistically punished for rewarding others
Speaker 1: and rewarded for punishing them. And that was one of
Speaker 1: those things that when I read it, I was just like,
Speaker 1: oh my god, that's so true. Um, yeah, yeah, And
Speaker 1: in this specific instance, it makes me think about the
Speaker 1: special effect of reversion to the mean, fallacies on motivating
Speaker 1: belief in the effectiveness of of not just screaming at
Speaker 1: pilots in this one case, but all kinds of punishment behaviors,
Speaker 1: for example, corporal punishment. Thankfully you hear this less often
Speaker 1: these days, but I remember when I was younger, I
Speaker 1: used to hear people who would defice end the parental
Speaker 1: practice of spanking children by saying, you know, I don't
Speaker 1: I don't care what the site scientists say. I don't
Speaker 1: care what the research says. I know from experience that
Speaker 1: it works. To the extent that comments like this were
Speaker 1: based on any real experience and observation and not just
Speaker 1: sort of a free form, self justifying statement that had
Speaker 1: nothing to do with experience. I bet a lot of
Speaker 1: it was fallacious inference of causation actually based on regression
Speaker 1: to the mean, just like in this condiment example. But anyway,
Speaker 1: I thought it would be interesting to talk a bit
Speaker 1: more about regression to the mean today, because it's one
Speaker 1: of those things that, again, once you see it, it's
Speaker 1: it's pretty simple, it's actually actually pretty clear, but understanding
Speaker 1: it can help you have a better sense of how
Speaker 1: good science works and help keep you from drawing hasty
Speaker 1: inferences in everyday life. Yeah, because it is it is
Speaker 1: interesting how kind of an insidious the results can be
Speaker 1: the idea that that again, praise is ultimately punished because
Speaker 1: is there's going to be a regression to the mean,
Speaker 1: to to to to the mean, and then likewise there
Speaker 1: can be this illusion, uh that uh that's screaming at
Speaker 1: pilots and so forth is going to be the successful
Speaker 1: way to go about things. Um. So yeah, this is
Speaker 1: I think this is an important episode to cover because
Speaker 1: it's the kind of thing that it's the kind of
Speaker 1: tool you kind of need tucked in your back pocket,
Speaker 1: even if you're just doing something like like scanning science
Speaker 1: headlines on a you know, a news server or social
Speaker 1: media message board. Yeah, because of course, understanding regression to
Speaker 1: the mean is extremely important in what scientists do when
Speaker 1: they design good experiments. If you don't take into account
Speaker 1: regression to the mean, you can incorrectly believe you have
Speaker 1: discovered some kind of tiger repellent or something. Uh. This
Speaker 1: concern plays a huge role in the history of medicine.
Speaker 1: It's part of the design of good medical research, or
Speaker 1: really any field that seeks to find remedies for problems.
Speaker 1: So consider a very basic hypothetical, uh path medicines, say
Speaker 1: from a hundred years ago. So you know, you have
Speaker 1: you have a foot pain that you've never really had before. Uh,
Speaker 1: you know, you want it to go away. So you
Speaker 1: go to the store and you buy a bottle of
Speaker 1: doctor Field Grades No Fail Pantasy for tumors, ulcers, cramps,
Speaker 1: and rooms, and you you pull the cork out, you
Speaker 1: chug it, and then the next day your foot feels better.
Speaker 1: Now you can conclude from this that the doctor Field
Speaker 1: Grades cured you. But how do you know actually that
Speaker 1: the feelings in your foot didn't just regress to the
Speaker 1: mean because the average is a low amount or no
Speaker 1: amount of foot pain. And if you don't have a
Speaker 1: medication that's tested with control groups and and randomized allocation
Speaker 1: into the groups, then how do you know that that
Speaker 1: the medicine actually did anything at all? Yeah? Yeah, So
Speaker 1: many of the examples you see for this and the applications,
Speaker 1: you're dealing with some sort of situation in the world,
Speaker 1: whether there is fluctuation and or change happening, often separately
Speaker 1: from whatever is being tested. So in this case, yeah,
Speaker 1: the doctor Field grads could have just been like just water.
Speaker 1: It just just you know, but there is the illusion
Speaker 1: that it worked because things got better. But if you
Speaker 1: don't have a control group and to you know, to
Speaker 1: drive home what that is. That would be like if
Speaker 1: you had a had like three different groups and a
Speaker 1: study of doctor Field Greats elixir. Here, one group was
Speaker 1: taking doctor Field graades elixer, another group was taking I
Speaker 1: don't know, let's say a half dose of Feel Grade
Speaker 1: or maybe a competitor's tonic. And in one group the
Speaker 1: control group was taking nothing was or was taking you know,
Speaker 1: just water or something to that effect, something completely innate. Uh.
Speaker 1: And that would be that would be a group that
Speaker 1: you would judge the results of the other categories by, right,
Speaker 1: and you would need to randomly sort the people into
Speaker 1: those groups. So it wasn't just that, you know, the
Speaker 1: only the people with real severe foot pain we're taking
Speaker 1: the doctor Field grades because the more extreme their pain
Speaker 1: to begin with, probably the more likely they are are
Speaker 1: to have that pain be lessened or go away over time,
Speaker 1: just naturally. Right. And uh. And I'm going to have
Speaker 1: a more specific example of this a little later in
Speaker 1: the podcast. So if you if you still don't get it,
Speaker 1: just hang on we'll we'll have another example in a
Speaker 1: bit thank. I was looking at an article in the
Speaker 1: British Medical Journal from nineteen that was just a collection
Speaker 1: of different examples of regression to the mean in real
Speaker 1: life medical research. This was by J. Martin Bland and
Speaker 1: Douglas J. Altman called statistics notes some examples of regression
Speaker 1: towards the mean, and they point out a very common
Speaker 1: type of example. So this will be similar to what
Speaker 1: we just talked about. The author's right. In clinical practice,
Speaker 1: there are many measurements such as weight, serum, cholesterol concentration,
Speaker 1: or blood pressure, for which particularly high or low values
Speaker 1: are signs of underlying disease or risk factors for disease.
Speaker 1: People with extreme values of the measurements such as high
Speaker 1: blood pressure may be treated to bring their values closer
Speaker 1: to the mean. If they are measured again, we will
Speaker 1: observe that the mean of the extreme group is now
Speaker 1: closer to the mean of the whole population. That is,
Speaker 1: it is reduced. This should not be interpreted as showing
Speaker 1: the effect of the treatment. Even if subjects are not treated,
Speaker 1: the mean blood pressure will go down owing to regression
Speaker 1: towards the means. So again something starts with an extreme
Speaker 1: value in certain types of cases, you would just expect
Speaker 1: it to have a less extreme value the next time
Speaker 1: due to random fluctuation. Uh So again, you know this
Speaker 1: could fill you with despair because you might wonder, well,
Speaker 1: then how could you ever know if a treatment was
Speaker 1: effective or not. But again, this is where the standard
Speaker 1: practices of science based medicine come to play. Instead of
Speaker 1: just taking people with some extreme measurement and giving them
Speaker 1: a treatment, you randomize them into test groups and control groups,
Speaker 1: like we were just talking about. So if you have
Speaker 1: a large enough sample, you really randomize the groups. People
Speaker 1: with the extreme starting conditions will somewhat regress towards the mean,
Speaker 1: but they will all regress toward the mean on average
Speaker 1: the same rate, whether they're receiving a real potential treatment
Speaker 1: or they're in the placebo group. But if the treatment
Speaker 1: actually does something helpful, this effect will manifest as the
Speaker 1: difference between the two groups. So good scientific research, good
Speaker 1: medical research has methods for excluding the effects of reversion
Speaker 1: to the mean on their findings. We have the tools,
Speaker 1: but we can still fall into the trap of regression
Speaker 1: to the mean fallacies, especially in our day to day lives.
Speaker 1: Drawing inferences the way that that the pilot and Inconomens
Speaker 1: story did, or or even in science if we're not
Speaker 1: careful and deliberate about designing experiments. And in addition to
Speaker 1: just a methodology design that has you know, a randomized
Speaker 1: groups and control groups, there are also ways of trying
Speaker 1: to counteract regression to the mean, just through statistical methods
Speaker 1: that are maybe less reliable, but there are statistical methods
Speaker 1: people can used to try to apply sort of modifiers
Speaker 1: to data in order to estimate regression to the mean
Speaker 1: and uh and counteract its effects. So again, we have
Speaker 1: tools within scientific research to to figure this out, and
Speaker 1: it's a lot of what science does is trying to
Speaker 1: sort out the difference between regression to the mean and
Speaker 1: actual effects of interventions. But in our day to day lives,
Speaker 1: we still fall for regression to the mean fallacies all
Speaker 1: the time. Yeah, and it's important to realize too that
Speaker 1: it's not just a situation where regression towards the mean
Speaker 1: could create an illusion of something working when it doesn't. Uh.
Speaker 1: You know, sometimes it can just potentially overstate um the
Speaker 1: effects of something. For an example of that that I
Speaker 1: was looking at was that regression towards the mean, or
Speaker 1: the failure to account for it can also overstate the
Speaker 1: effectiveness of something like traffic light cameras. Is it making
Speaker 1: a difference and cutting down on accidents? Perhaps, but any
Speaker 1: actual effectiveness could potentially be overstated by failure to account
Speaker 1: for just regression towards the mean. Oh yeah, so where
Speaker 1: do you tend to install things like that? High acts
Speaker 1: like problem areas? Right, So, if there's like a stretch
Speaker 1: of road that has a lot of problems on people
Speaker 1: really speeding a lot there or crashing a lot there,
Speaker 1: that might be where you stage the intervention. It's possible
Speaker 1: some things like that fluctuate naturally over time in different locations,
Speaker 1: and you put the cameras in place, and it could
Speaker 1: have an effect, but maybe not as much of an
Speaker 1: effect as it looks like it is taking place. Again,
Speaker 1: if you don't factor regression towards the mean into the study.
Speaker 1: Right now, While our TM is a very important phenomenon
Speaker 1: to understand and take into account, it certainly doesn't apply
Speaker 1: to every sequence of values you could repeatedly sample, so
Speaker 1: you also have to be careful not to apply it
Speaker 1: in situations where it isn't warranted. I was you know,
Speaker 1: there are a million examples. You could cite one that
Speaker 1: came to my mind as the orbital decay of a satellite.
Speaker 1: Let's say you've got a communication satellite in lower orbit
Speaker 1: and you get a reading on its altitude and the
Speaker 1: reading is lower than the satellites average altitude. Uh. Now
Speaker 1: you might say, hey, I think this means we need
Speaker 1: to program a reboost to insert it back into the
Speaker 1: orbit where it's supposed to be. And somebody could erroneously
Speaker 1: apply regression to the mean here and say, nah, we
Speaker 1: don't need to do that. The satellite might just return
Speaker 1: to its average altitude. It doesn't apply in this scenario,
Speaker 1: even though you are taking repeated measurements of a value
Speaker 1: over time, because we know things about the physical characteristics
Speaker 1: determining the orbit of satellites and in lower th orbit uh,
Speaker 1: and that due to factors like atmospheric drag, their altitude
Speaker 1: tends to trend steadily downward over time in a consistent
Speaker 1: direction down down, down, So eventually you will need a
Speaker 1: reboost in order to put it back up to the
Speaker 1: correct distance. So regression to the mean apply is to
Speaker 1: certain kinds of data that are repeatedly sampled data where
Speaker 1: there is natural random fluctuation back and forth, not a
Speaker 1: steady trend in the data in one direction on the
Speaker 1: relevant time scale. The other thing that's important to understand
Speaker 1: is that systems where you expect to find regression to
Speaker 1: the mean are systems in which the repeated data values
Speaker 1: you're sampling are to some degree determined by luck or chance.
Speaker 1: If a series of values is influenced almost entirely by
Speaker 1: deterministic influence, like in the satellite example, by like the
Speaker 1: laws of physics, or by some extremely reliable skill with
Speaker 1: little room for variation, values don't really regress towards the
Speaker 1: mean in the same way because there's just less random
Speaker 1: fluctuation back and forth to begin with. The more chance
Speaker 1: and random variation plays a role in the outcome, the
Speaker 1: more you will tend to observe regression towards the mean
Speaker 1: after an extreme sample in in whatever it is you're
Speaker 1: looking at, I've I've read that the progression towards the
Speaker 1: mean is is not to be confused with the law
Speaker 1: of large numbers. For example, uh. This is the the
Speaker 1: law that that states, as a sample size becomes larger,
Speaker 1: the sample mean gets closer to the expected value. So
Speaker 1: a coin flipping example is key here. Flip a coin
Speaker 1: and the random results are going to ultimately average out
Speaker 1: to a point five proportion. But if you only flip
Speaker 1: the coin ten times, you might not see this breakdown. Um.
Speaker 1: And this also applies to say, even odds on the
Speaker 1: rolling of a of a D six of a six
Speaker 1: sided die. Uh So for example, two regular people, that's
Speaker 1: just to die that nerves like us, it's a D six. Yeah.
Speaker 1: D six is what I could get my hands on.
Speaker 1: Because I was like, well, I'm gonna do an example.
Speaker 1: I'm gonna try it myself. So while I was putting
Speaker 1: together notes for this, I went ahead and rolled ten times,
Speaker 1: and I got even even odd even odd even even
Speaker 1: even even odd. So that's that's seven to three in
Speaker 1: favor of even. So it might make you wonder, well,
Speaker 1: is this die broken? Does this D six need to
Speaker 1: go away? Because it can't be trusted to roll? Uh?
Speaker 1: You know a balanced array of odd and even numbers. Well, no,
Speaker 1: that's not the case. Uh. And if I were to
Speaker 1: roll this, say another hundred times, another thousand times, I
Speaker 1: would see things even out even more to where we
Speaker 1: would see this, uh, this point five proportion of odd
Speaker 1: versus even right. So these are not exactly the same thing,
Speaker 1: regression to the mean and the law of large numbers,
Speaker 1: but they are closely related. Both observations require you to
Speaker 1: think about statistical tendencies over time, over a time period
Speaker 1: of repeated sampling, and both are premised on the knowledge
Speaker 1: that repeated samples will tend towards the average. But regression
Speaker 1: to the mean has to do with the idea that
Speaker 1: if you start with an extreme observation and there is
Speaker 1: some role of chance or luck in determining the value
Speaker 1: of this observation, the next time you sample it, it's
Speaker 1: more likely to be closer to the average. The law
Speaker 1: of large numbers is that if in the real world,
Speaker 1: the more times you run something, the closer your outcomes
Speaker 1: in the real world will will be to the sort
Speaker 1: of perfect mathematical average that you would estimate just given
Speaker 1: the chances to begin with. Now, I want to come
Speaker 1: back to regression towards the mean in um in medical
Speaker 1: studies because I found a really interesting one that came
Speaker 1: out earlier this year. Uh So, a lot of a
Speaker 1: lot of the examples you find involving regression to the
Speaker 1: mean involved sports or economics, and I found. This one
Speaker 1: discussed in a New York Times article again from earlier
Speaker 1: this year titled Intense strength training does not ease knee pain,
Speaker 1: study finds by Gina Colada. Uh, this is referring to
Speaker 1: a study published in Jama that entailed an eighteen month
Speaker 1: clinical trial involving three d and seventy seven participants. Okay, okay,
Speaker 1: So the basic situation, the setup for this paper is
Speaker 1: that a lot of people have knee osteoarthritis, and one
Speaker 1: of the go to treatment recommendations has long been and
Speaker 1: strength training. So in this study they decided to look
Speaker 1: into it with three basic groups, one that received intense
Speaker 1: strength training, another that received moderate strength training, and another
Speaker 1: that received counseling on healthy living. So that third group,
Speaker 1: that's the control group, they did not have any amount
Speaker 1: of strength training, just uh, you know, some positive counseling
Speaker 1: about healthy living. Sure, so the researchers here apparently actually
Speaker 1: expected to see the intense strength training take the lead
Speaker 1: that they were looking to identify what has been just
Speaker 1: sort of accepted wisdom, um and and again this this
Speaker 1: has been the predominant treatment idea. But instead they found
Speaker 1: that the results were the same for all three groups quote,
Speaker 1: everyone reported slightly less pain, including those who had received
Speaker 1: only counseling. Now why is that? Well, as Colotta points out,
Speaker 1: there's there's always room for other effects, especially say the
Speaker 1: placebo effect. Uh but regression to the is also a
Speaker 1: heavy consideration here and certainly could work in congress with
Speaker 1: the placebo effect. Right, So, you don't necessarily have to
Speaker 1: assume that the counseling actually helped to heal people's knees,
Speaker 1: though it may have in in in some it may
Speaker 1: have had some kind of mechanistic effect in some way,
Speaker 1: a mind body kind of thing. But you would also
Speaker 1: just expect over time, people who have an extreme starting position,
Speaker 1: who were starting with a lot of knee pain, to
Speaker 1: get gradually better over time. Yeah, so a Colatta rights
Speaker 1: quote are the right As symptoms tend to surge and subside,
Speaker 1: and people tend to seek out treatments when the pain
Speaker 1: is at its peak, when it declines, as it would
Speaker 1: have anyway, they ascribed the improvement to the treatment. Uh.
Speaker 1: So you know this would this would roughly equate to
Speaker 1: yelling at your knee when it's in pain, and it
Speaker 1: really make it certainly relates to many other health scenarios
Speaker 1: as well various medications and even things like prayer and
Speaker 1: you know, supernatural um treatments and attempts to to deal
Speaker 1: with pain, et cetera. Yeah, I mean it could apply
Speaker 1: to any intervention that is aimed at influencing something that
Speaker 1: is naturally variable on its own, right. Yeah, and you
Speaker 1: know something that's again any kind of system in which
Speaker 1: change occurs when fluctuation occurs. Uh, you know, you can
Speaker 1: you can see this applying to not only physical pain,
Speaker 1: but also uh, emotional distress, things of that nature, you know.
Speaker 1: So again, I think this is an important tool to
Speaker 1: have in our our logic tool kit. Now there are
Speaker 1: even cases where I'm tempted to think about the application
Speaker 1: of regression to the mean, but but where it's probably
Speaker 1: a lot harder to quantify exactly what the effects are.
Speaker 1: It's cases where it can be difficult to separate out,
Speaker 1: say the effects of some kind of deterministic influence like
Speaker 1: skill versus how how strong the effect of chance or
Speaker 1: luck is. But I think about things even in the
Speaker 1: world of the like I think about, you know, the
Speaker 1: sophomore album by by a band that has like a
Speaker 1: really stellar debut album. Uh, you know, often that is
Speaker 1: perceived is disappointing, and you have to wonder, like, Okay,
Speaker 1: is it is that often true? Because I don't know
Speaker 1: if people get famous and it goes to their heads
Speaker 1: and then they you know, they get full of themselves
Speaker 1: and make something dumb, or is it because when somebody
Speaker 1: has a debut album that's really well received to some extent,
Speaker 1: it's so good partially because of luck or chance, and
Speaker 1: that's an outlier that you're as you're starting sample yeah, yeah,
Speaker 1: And certainly this is an area that's there's a lot
Speaker 1: more subjectivity here and and so it's not the kind
Speaker 1: of thing you can necessarily have a control group for anything.
Speaker 1: But but I think it is quite interesting. And I
Speaker 1: did find as I was looking around for some jazzy
Speaker 1: or examples or possible examples of aggression to the mean, um,
Speaker 1: I found one that that actually gets into a little
Speaker 1: bit into the idea of you know, the first and
Speaker 1: second album. But also uh, the idea of follow up
Speaker 1: films and Hollywood sequels has pointed out both good Yeah.
Speaker 1: Has pointed out by Joanna Deong in two thou eighteen
Speaker 1: on the blogs scientifically sound movie sequels are potentially a
Speaker 1: great example of aggression to the mean. Quote, Hollywood sequels
Speaker 1: are only made if the original film is a quote
Speaker 1: unquote high quality success. But the average quality of sequels
Speaker 1: will be closer to the mean than average quality of
Speaker 1: originals of sequels because of regression to the means, So
Speaker 1: sequels tend to be of lower quality than the original.
Speaker 1: Now I might somewhat dispute the premise here that Hollywood
Speaker 1: sequels are only made to films that are high quality
Speaker 1: to begin with. Um, But but I still think this
Speaker 1: is onto something because there is a movie that gets
Speaker 1: a sequel tends to have something about it, something that
Speaker 1: people are responding to, whether it's a movie that I
Speaker 1: would like or not. Right, I mean, so sometimes obviously
Speaker 1: the situation is the film just made a lot of mine.
Speaker 1: I mean, I guess that's the key thing. It didn't
Speaker 1: make a lot of money. If so, producers are going
Speaker 1: to be more inclined to say, let's do that again,
Speaker 1: Let's have that experience again of all that money coming in.
Speaker 1: And sometimes this this certainly matches up with a quality film.
Speaker 1: You have something that really captures people's imagination and it
Speaker 1: is of high quality and uh and you know, so
Speaker 1: it's really firing on all cylinders. But you know, and yes,
Speaker 1: certainly in some cases it's just the right film at
Speaker 1: the right time. Or or maybe it has nothing to
Speaker 1: do with the film itself. Maybe it's who's in it,
Speaker 1: or I don't know what's going on in the zeitgeist
Speaker 1: during that particular era. Well, the way I would think
Speaker 1: about this is, and I think again, this is onto something.
Speaker 1: It highlights that when we experience confusion where we say, like, wow,
Speaker 1: you know, the Exorcist is such a great horror movie
Speaker 1: and The Exorcist Too is so bad? How could that
Speaker 1: be the case? You know, why is it? Why is
Speaker 1: such a bad sequel to such a great movie? It's
Speaker 1: because of the compare a son of the original to
Speaker 1: the sequel that we're experiencing this confusion. Another way you
Speaker 1: could just look at it is most horror movies are
Speaker 1: direc most movies are bad, and it is only by
Speaker 1: comparing the The Exorcist Too to The Exorcist that you
Speaker 1: notice this steep drop off. Where another way of looking
Speaker 1: at it is that The Exorcist Too is bad like
Speaker 1: most horror movies are, and the first one was an outlier.
Speaker 1: At the beginning, it was a first film in a
Speaker 1: series that happened to be really good A cut above. Yeah, absolutely, like, yeah,
Speaker 1: I think this is the correct way to look at it,
Speaker 1: and also keeping in mind that just how amazing it
Speaker 1: is that any film gets completed, like even a bad film,
Speaker 1: Like a lot of people probably worked pretty hard to
Speaker 1: make that happen, even if the end results don't really
Speaker 1: please anyone at all. But but yeah, I think this
Speaker 1: is also an interesting inversion of the opening example of
Speaker 1: yelling at pilots as well, because most of the time,
Speaker 1: if a flawed movie comes out, people are not clamoring
Speaker 1: for the sequel. Um Sequels are rarely guaranteed, so you're
Speaker 1: not often going to hear things like, oh, well, that
Speaker 1: wasn't great. I hope the next one is an improvement.
Speaker 1: I mean some people say that, some people I've said
Speaker 1: things like that before, where it will be like, oh,
Speaker 1: really flawed film, but maybe there's like a cool idea
Speaker 1: I kind of wish it would they would remake it,
Speaker 1: even though there's no like logical reason that there would
Speaker 1: be like a there would be money behind that idea. Well,
Speaker 1: I guess it's kind of different when you're talking about
Speaker 1: a one off creative project versus something. I mean, we
Speaker 1: live in a kind of different era now because we
Speaker 1: were at the height of this you know, cinematic universe
Speaker 1: thing with a huge number of the big budget movies
Speaker 1: that come out. The big event movies are not one
Speaker 1: off creative products, but they are a product that exists
Speaker 1: within some kind of franchise or universe or something. So
Speaker 1: you just know automatically that there's gonna be another one,
Speaker 1: whether this one is good or not. Yeah, like either
Speaker 1: it's an established film universe where like you know, they
Speaker 1: put out another Marvel movie and it's just terrible, Well,
Speaker 1: obviously there's enough momentum. They're not going to stop. They're
Speaker 1: not gonna be like, oh, well, less and learned, Well
Speaker 1: we'll stop then. No, No, there's gonna be another. Another
Speaker 1: example of this might be a successful franchise in another medium,
Speaker 1: say a book series, so like the Harry Potter books
Speaker 1: for example, or I don't know, Lord of the Rings,
Speaker 1: where you know that once they make the Fellowship of
Speaker 1: the Rings, there's going to be a follow up. They're
Speaker 1: gonna do another one. So in these ways, unless it's
Speaker 1: the seventies and it's uh, Lord of the Rings movie
Speaker 1: that that ends with Helm's Deep. Well, but they picked
Speaker 1: that up eventually. But kay, but but yeah, probably the
Speaker 1: Harry Potter films are a better example. And there may
Speaker 1: be spe specific you know, things about how that wasn't
Speaker 1: guaranteed either. Uh, you know, the economic reality can always
Speaker 1: come into play. But for the most part, like those
Speaker 1: were when when that started, you knew they were gonna
Speaker 1: keep making these at least they were going to make
Speaker 1: a follow up, so you could have comments like, well
Speaker 1: there were that was just on of flawed in some
Speaker 1: of the some of its execution. I hope that they
Speaker 1: fix that in the next film. For the most part, yeah,
Speaker 1: with one offs, this is not the case. It's like,
Speaker 1: if if this film fizzles, then only you know a
Speaker 1: few like rare people are going to be clamoring for
Speaker 1: a sequel or dreaming about what the sequel would be. Yeah,
Speaker 1: I think this observation, but regression to the mean and
Speaker 1: movie sequels is actually very on point, but more so
Speaker 1: for the films of yester Year, where the more the
Speaker 1: more common thing was you'd have a an independent sort
Speaker 1: of creative product that it's its own thing, and then
Speaker 1: if it resonated with somebody, if it did well, there
Speaker 1: would be sequels. I think it's a little it applies
Speaker 1: a little bit less today when there's just you know,
Speaker 1: we're in the world of franchises and extended universes and
Speaker 1: there's just sort of like a guaranteed, ongoing uh conveyor
Speaker 1: belt of of new stuff within the Marvel world or
Speaker 1: the Star Wars world or whatever. Yeah, but I think
Speaker 1: it it is a worthwhile way to think about creative
Speaker 1: the creative process, and you know, as opposed to some
Speaker 1: of these alternate sort of folk wisdomy ways of thinking
Speaker 1: about it. For example, on Weird House Cinema, we recently
Speaker 1: talked about Toby Hooper. Toby Hooper is one of those
Speaker 1: directors who's often you'll often you'll see descriptions. I think
Speaker 1: we've even read part of a review where they they
Speaker 1: really they talk about, oh, well, you know he put
Speaker 1: out Texas Chainsaw Masacre directed that film and this was great.
Speaker 1: It was, you know, just a real lightning bolt um
Speaker 1: to the cinematic world into horror itself as a genre.
Speaker 1: And then the idea that well he was never able
Speaker 1: to capture that magic again. You know that his his
Speaker 1: career was just like one long slide after that, which
Speaker 1: I don't think is a fair assessment, especially if you
Speaker 1: employ regression to the mean, you know, the idea being that, yeah,
Speaker 1: he did kind of get lightning in a bottle with that,
Speaker 1: with that first big film, that that he was able
Speaker 1: to to really bring something together that is an outlier, um,
Speaker 1: but that that that's just going to happen. That's just
Speaker 1: the way these things work, right, So most movies aren't
Speaker 1: that good, So you of the random chance of like
Speaker 1: how how good his ideas and execution are from one
Speaker 1: year to the next is going to set in and
Speaker 1: you might have a different idea about his career if
Speaker 1: you were to say, like randomly chronologically reorder all his movies, right,
Speaker 1: you know, like if you were to put the worst
Speaker 1: ones earlier on or something, people might feel differently about it. Yeah,
Speaker 1: well then they would talk about, well, okay, TCM was
Speaker 1: peak Toby Hooper, like this was his peak output. Because
Speaker 1: this is the kind of the kind of view of
Speaker 1: an artist's you know, creative trajectory that we tend to
Speaker 1: want to um to follow along, you know, because it's
Speaker 1: more story shaped, the idea of assent and then eventually
Speaker 1: decent that there's gonna be uh, there's gonna be a
Speaker 1: period of high noon in their creative out output, and
Speaker 1: sometimes that does match up with the reality. But I
Speaker 1: don't know. Even then, we I think we tend to
Speaker 1: overlook the dogs in the filmographies of people we love,
Speaker 1: you know. Oh yeah, uh, But then again, I mean,
Speaker 1: this is interesting because in talking about regression to the
Speaker 1: mean applying to creative products like movies, we are acknowledging
Speaker 1: that the creative process is not purely a product of
Speaker 1: talent and skill, that there is a significant amount of
Speaker 1: chance and luck involved in something like how good a
Speaker 1: movie turns out to be? Um, And it's hard to
Speaker 1: know exactly how to like how to picture that influence
Speaker 1: of chance and luck, you know, like, what what is
Speaker 1: that in the creative process? It's obviously true because there
Speaker 1: are people who can be incredibly skilled in one instance
Speaker 1: and then I don't know, things just don't go right
Speaker 1: the next time, and to make something that nobody really likes.
Speaker 1: But uh, but that's that's just not often how people
Speaker 1: like to think about creative talents, and people like to
Speaker 1: think about the creative process like it is much more
Speaker 1: strictly deterministic. Yeah yeah, or or you look at things
Speaker 1: like the Star Wars films, and you kind of like
Speaker 1: fall into this idea of thinking this is stuff that
Speaker 1: is mind out of the mythic earth, and you know,
Speaker 1: it just makes sense that things would accumulate and get better.
Speaker 1: So um, but really looking back on it, especially if
Speaker 1: you actually like watch documentaries, and there's some great ones
Speaker 1: about the production of those films, like it's it's amazing
Speaker 1: that Star Wars, the first one in New Hope was
Speaker 1: as good as it was, and then it's nothing short
Speaker 1: of I mean, it's it's just a pure miracle that
Speaker 1: the second one was so much better and like really
Speaker 1: nailed it. Like if if the second film had had floundered,
Speaker 1: I mean, just imagine how different the cinematical landscape would
Speaker 1: have been for decades to come. Yeah, So it's it's
Speaker 1: amazing if the first film in a series is good,
Speaker 1: and it's super amazing if the second one is good.
Speaker 1: And and this is why I think we often find
Speaker 1: too that if if part one in part two of
Speaker 1: something are of high quality, then you've got to look
Speaker 1: out for that part three because that part three, that
Speaker 1: part three may be coming to get you. But likewise,
Speaker 1: if a part two is rubbish, um, you know, subjectively,
Speaker 1: then then part three might pick it up and uh
Speaker 1: and get things back on track. So you certainly see
Speaker 1: that that kind of fluctuation as well. I have a
Speaker 1: question I actually don't know the answer to, but this
Speaker 1: would be interesting in terms of I don't know the
Speaker 1: high performing output, whether that is in whether that is
Speaker 1: a creative endeavor like you know, writing books or creating movies,
Speaker 1: or whether that's something even like athletics, like athletic performance,
Speaker 1: do you expect to see more random fluctuation in the
Speaker 1: performance of collaborative output versus individual output? So say, um,
Speaker 1: do you expect more influence of random chance and fluctuation
Speaker 1: in the quality of uh books written by a single
Speaker 1: author versus you know, movies that have the input of
Speaker 1: hundreds of thousands of people? Uh? Or in in the
Speaker 1: realm of say sports, like do you expect more random
Speaker 1: variation in the output of an individual athletes like you know,
Speaker 1: an individual gymnast or something, or in team sports? Yeah?
Speaker 1: I could see it going both ways, because yeah, if
Speaker 1: you think too hard to about even just like the
Speaker 1: film and aology, you can easily get into discussions of
Speaker 1: like okay, well is it the same cast and crew
Speaker 1: that are producing the sequel. Uh, you know, what happens
Speaker 1: when the budget is different, what happens when there are
Speaker 1: other constraints, what happens when suddenly there are a whole
Speaker 1: bunch of producers that have their ideas about what things
Speaker 1: should be. I mean, there's so many different factors to
Speaker 1: take into place. Uh. You know, with this example that
Speaker 1: you know, perhaps doesn't bear too close of scrutiny, but
Speaker 1: but but it's but it's still I think serves as
Speaker 1: a nice um illustration of the overall trend that we're
Speaker 1: talking about here. Well, it does bring up the fact
Speaker 1: that since I mentioned athletes that you know, I don't
Speaker 1: know a lot about sports. I'm not a big sports fan.
Speaker 1: But but clearly, but regression to the mean is something
Speaker 1: that has widely been applied to the world of sports. Uh.
Speaker 1: For example, in the observation that often after having a
Speaker 1: really stellar season, either an individual athlete or a sports
Speaker 1: team will be perceived to underperform the next season. And again,
Speaker 1: that very well could have something to do with regression
Speaker 1: to the mean. Like, you know, the fact that they're
Speaker 1: observed having in a using season is actually an outlier.
Speaker 1: You're starting your expectations then and saying like, Okay, now
Speaker 1: they're going to be the best forever. Just by random
Speaker 1: fluctuation over time, you would expect their next season to
Speaker 1: probably be not as good as the first. I wonder
Speaker 1: to what an extent this can be applied to, say,
Speaker 1: the world of the culinary arts, or even just like
Speaker 1: various food crops, like say the selecting a cantalope at
Speaker 1: the grocery store, that sort of thing. I mean, I
Speaker 1: guess it would apply to pretty much anything where you're
Speaker 1: sampling in a series over time, there's plenty of random
Speaker 1: fluctuation in what you're sampling, and the first thing you
Speaker 1: sample is an outlier in some way really good or
Speaker 1: really bad. If those things hold true, then you can
Speaker 1: probably expect you're going to see some regression one way
Speaker 1: or the other. Yeah. Yeah. By the way, I was
Speaker 1: looking around for like really stellar examples of a sequel
Speaker 1: film that is widely believed to be uh rubbish, and
Speaker 1: I think The Exorcist Too is the primary example. Like
Speaker 1: you get into some of the other examples that pop up,
Speaker 1: I feel like there's room for disagreement. Um. For instance,
Speaker 1: Texas Chainsaw Masacre two is one which I saw popping
Speaker 1: up on some of these lists for disappointing sequels. But
Speaker 1: I think that's entirely based on who you ask. I
Speaker 1: think if you ask us, we will agree that that
Speaker 1: that t c M Two is is actually a great film.
Speaker 1: It's different from the first one, perhaps if you go
Speaker 1: into if you go into part two with the expectations
Speaker 1: you had for part one, you may see it as
Speaker 1: a dip in quality. But depending on what else you're
Speaker 1: bringing to the table, you might see it as an
Speaker 1: increase in in quality, or at least or something that
Speaker 1: maybe is different but on par with the original. I mean,
Speaker 1: it's certainly not for everybody. I mean, it is a
Speaker 1: It is a gross, disgusting film in in a way
Speaker 1: like the first one, probably even grosser, but also a
Speaker 1: sort of satirical masterpiece. Um but I just had another
Speaker 1: thought when you said that The Exorcist Too is regarded
Speaker 1: as like one of the best examples of a sequel.
Speaker 1: That's really rubbish. I mean, it makes me also wonder
Speaker 1: about the pretty high estimation critics generally have of the
Speaker 1: exer Is three. Makes me wonder if the effect of
Speaker 1: The Exorcist to being so bad actually makes people sort
Speaker 1: of over. You know, they're like they're ready to be
Speaker 1: impressed by the Exorcist three. Yeah. Yeah, I wonder if
Speaker 1: that's the case too with it with especially when you
Speaker 1: have a situation with the part three coming back and
Speaker 1: restoring uh some I don't know, some level of quality
Speaker 1: to a franchise. I mean there's also like the Star
Speaker 1: Trek example, right, I mean that was long the Long
Speaker 1: held up as an example of like, okay, you have
Speaker 1: you even Star Treks and your odd Star Treks, right, uh.
Speaker 1: And I think you've made a similar case for the
Speaker 1: Faster and Furious movies, right, I mean, once you get
Speaker 1: to a certain point in the series, I think it's
Speaker 1: pretty much all uh, you know, a nitrous boosted brain.
Speaker 1: It's it gets you know, it's all like we're driving
Speaker 1: cars in space now and flying and all that. But um,
Speaker 1: but for the earlier ones, yeah, I'd say the odd
Speaker 1: ones are better. Like, uh, three is the first one
Speaker 1: where it really starts getting ludicrously weird. Four is kind
Speaker 1: of a uh and then five starts. Five is when
Speaker 1: the rock shows up, and then but by seven year
Speaker 1: golden all right, well we're gonna go ahead and close
Speaker 1: this one out here. But we'd obviously love to hear
Speaker 1: from everyone about this about regression towards the mean, just
Speaker 1: in our daily lives, in various scientific studies. Perhaps you
Speaker 1: have thoughts about how this applies to something we've discussed
Speaker 1: on the show in the past, because I know we've
Speaker 1: we've mentioned regression to the mean in passing before, but
Speaker 1: certainly we've never taken the opportunity to really dive into
Speaker 1: it and explain it like we did today. Yeah, I
Speaker 1: know it's come up in passing, just in us making
Speaker 1: comments here and there about like the importance of of
Speaker 1: randomized trials and control groups and all that. In the meantime,
Speaker 1: if you would like to listen to other episodes of
Speaker 1: Stuff to Blow Your Mind, you will find them wherever
Speaker 1: you get your podcast. Just look for the Stuff to
Speaker 1: Blow your Mind podcast feed. We have our core episodes
Speaker 1: on Tuesdays and Thursdays, Artifact episodes on Wednesday, listener mail
Speaker 1: on Monday's. On Fridays, we do a little bit of
Speaker 1: a weird house cinema. That's our times. We just talk
Speaker 1: about some sort of a strange film. Uh, and you know,
Speaker 1: tease apart what makes it strange? Uh, let's see what
Speaker 1: else so yeah, have you go to stuff to Blow
Speaker 1: your Mind dot com that will send you to the
Speaker 1: I Heart listing for our show, and there's a button
Speaker 1: there for a store if you want to click on
Speaker 1: that wealth then you can buy some merchandise that has
Speaker 1: Stuff to Blow your Mind logos and whatnot on it,
Speaker 1: or perhaps weird House cinema logos and whatnot on it,
Speaker 1: and you can get mugs, t shirts, all that kind
Speaker 1: of stuff. Uh, just for fun, you know, or but
Speaker 1: if you want to be all business about it, you
Speaker 1: can make it work to get our merch Now, now
Speaker 1: this this is just for fun, Like, don't use these
Speaker 1: these clothing items to clothe your nakedness. These are just
Speaker 1: for fun. This is just purely extra clothing. Okay, this
Speaker 1: should not be your core clothing. Huge thanks as always
Speaker 1: to our excellent audio producer Seth Nicholas Johnson. If you
Speaker 1: would like to get in touch with us with feedback
Speaker 1: on this episode or any other to suggest a topic
Speaker 1: for the future, just to say hello, you can email
Speaker 1: us at contact at stuff to Blow your Mind dot com.
Speaker 1: Stuff to Blow Your Mind is production of I Heart Radio.
Speaker 1: For more podcasts for My Heart Radio, visit the I
Speaker 1: Heart Radio app, Apple podcasts, or wherever you listening to
Speaker 1: your favorite shows.