WEBVTT - From the Vault: Regression to the Mean 0:00:05.680 --> 0:00:08.360 Hey, welcome to Stuff to Blow Your Mind. My name 0:00:08.400 --> 0:00:11.440 is Robert Lamb and I'm Joe McCormick. And it's Saturday. 0:00:11.520 --> 0:00:15.520 Time to go into the vault. This episode originally published 0:00:15.560 --> 0:00:19.919 on August three, and it's called Regression to the Mean, 0:00:20.120 --> 0:00:23.440 which is about regression to the mean. Uh, it's an 0:00:23.480 --> 0:00:27.400 episode about a statistical phenomenon. But don't don't don't run away. 0:00:27.600 --> 0:00:31.440 It's actually, I think super interesting and having this idea 0:00:31.560 --> 0:00:35.800 in your toolkit really helps you understand the information that 0:00:35.880 --> 0:00:39.839 you encounter in the world, uh much better. Absolutely, So 0:00:39.960 --> 0:00:45.599 let's dive right in. Welcome to stot to Blow Your 0:00:45.600 --> 0:00:55.320 Mind production of My Heart Radio. Hey, welcome to Stuff 0:00:55.360 --> 0:00:57.960 to Blow Your Mind. My name is Robert Lamb and 0:00:58.040 --> 0:01:01.319 I'm Joe McCormick. And in today this episode, we are 0:01:01.360 --> 0:01:04.240 going to be focusing on a topic that is already 0:01:04.280 --> 0:01:07.600 something that's very well known to people who are familiar 0:01:07.640 --> 0:01:11.600 with quantitative research and statistics, but less known to the 0:01:11.640 --> 0:01:14.640 general public. And uh, and I think that's a tragedy 0:01:14.680 --> 0:01:17.360 because it's an idea that should really be part of 0:01:17.520 --> 0:01:21.160 everybody's basic critical thinking tool kit, no matter what your 0:01:21.240 --> 0:01:25.080 job is. And so in order to introduce this concept. 0:01:25.120 --> 0:01:27.039 I thought it would be best to start with a 0:01:27.160 --> 0:01:30.720 with a direct illustration from the real world of people 0:01:30.800 --> 0:01:35.959 reaching incorrect conclusions by not understanding the subject of today's episode. 0:01:36.280 --> 0:01:38.080 And so the illustration I want to start with is 0:01:38.319 --> 0:01:43.080 an interesting story told by the psychologist Daniel Kaneman that's 0:01:43.120 --> 0:01:47.960 about the illusory power of screaming at pilots. Uh So, 0:01:48.000 --> 0:01:50.800 the context of the story is that Knemon says he 0:01:50.880 --> 0:01:54.800 was giving a lecture about positive reinforcement to a group 0:01:54.880 --> 0:01:58.360 of flight instructures. I think this was in the nineteen sixties, 0:01:59.120 --> 0:02:03.320 and Kneman was trying to inform them about what he 0:02:03.320 --> 0:02:06.160 believed at the time was the best consensus of scientific 0:02:06.200 --> 0:02:10.240 research on learning and reinforcement, which was at the time 0:02:10.320 --> 0:02:13.160 that if these flight instructors wanted their students to have 0:02:13.200 --> 0:02:17.120 the best possible outcomes, they should focus more on praising 0:02:17.200 --> 0:02:20.040 the students when they did well, then on chewing them 0:02:20.080 --> 0:02:23.520 out when they did something wrong. And Knomon says that 0:02:23.680 --> 0:02:27.000 when he finished his talk, one of the flight instructors 0:02:27.000 --> 0:02:28.880 that he had been giving this lecture two got up 0:02:29.200 --> 0:02:31.720 and tried to dispute him. He said, no, you're wrong. 0:02:32.080 --> 0:02:34.480 And so the direct quote economy and gives from the 0:02:34.520 --> 0:02:38.400 instructor here is, on many occasions I have praised flight 0:02:38.440 --> 0:02:43.160 cadets for clean execution of some aerobatic maneuver, and in general, 0:02:43.200 --> 0:02:46.000 when they try it again, they do worse. On the 0:02:46.000 --> 0:02:50.040 other hand, I've often screamed at cadets for bad execution, 0:02:50.360 --> 0:02:53.520 and in general they do better the next time. So 0:02:53.600 --> 0:02:57.240 please don't tell us that reinforcement works and punishment does not, 0:02:57.560 --> 0:03:01.120 because the opposite is the case. So you might think 0:03:01.160 --> 0:03:03.160 he has a good point here. If you accept that 0:03:03.240 --> 0:03:06.520 this flight instructor has had a lot of direct experience 0:03:06.639 --> 0:03:10.360 working with students, and you trust him to remember the 0:03:10.400 --> 0:03:14.320 relative frequency of these events pretty well, you might assume 0:03:14.520 --> 0:03:18.160 that he has a meaningful rebuke to konom In. Here again, 0:03:18.240 --> 0:03:20.560 he says that most of the time, after a cadet 0:03:20.600 --> 0:03:23.440 does something bad and he screams at them, they do 0:03:23.520 --> 0:03:26.359 better the next time, And after a cadet does something 0:03:26.400 --> 0:03:29.240 good and he praises them, they actually do worse the 0:03:29.320 --> 0:03:33.240 next time. So if he's remembering these experiences correctly, and 0:03:33.280 --> 0:03:35.680 he's had a lot of them, it would really seem 0:03:35.760 --> 0:03:39.480 like evidence that praise has a negative effect on learning, 0:03:40.000 --> 0:03:43.880 maybe by making the student pilots soft and overconfident. Or something, 0:03:44.200 --> 0:03:47.800 and getting chewed out is good for skill development. I 0:03:47.800 --> 0:03:50.120 think it's quite easy to see the allure of this, 0:03:50.120 --> 0:03:53.680 this false conclusion right right, And it's and you can 0:03:53.720 --> 0:03:56.120 also easily imagine how you kind of build upon this 0:03:56.280 --> 0:04:01.320 with certain loosely backed up you know, folk ideas about 0:04:01.360 --> 0:04:03.880 how you encourage people and how people learn, and you 0:04:03.960 --> 0:04:05.520 got to stay on and if they if you tell 0:04:05.560 --> 0:04:07.760 them they're doing a good job, they'll get lazy, right, 0:04:07.840 --> 0:04:11.840 folk wisdom, tough guy mentality. But Koneman saw something different 0:04:11.880 --> 0:04:15.040 in this response, and he says that he immediately set 0:04:15.120 --> 0:04:19.080 up an experiment on the spot to demonstrate the flaw 0:04:19.279 --> 0:04:22.120 in the flight instructors thinking here, so I want to 0:04:22.160 --> 0:04:25.960 read from Knomen's description, He says, I immediately arranged a 0:04:26.040 --> 0:04:30.039 demonstration in which each participant tossed two coins at a 0:04:30.120 --> 0:04:34.520 target behind his back without any feedback. We measured the 0:04:34.560 --> 0:04:37.440 distances from the target and could see that those who 0:04:37.440 --> 0:04:41.359 had done best the first time had mostly deteriorated on 0:04:41.400 --> 0:04:44.719 their second try, and vice versa. But I knew that 0:04:44.800 --> 0:04:48.560 this demonstration would not undo the effects of lifelong exposure 0:04:48.800 --> 0:04:53.760 to a perverse contingency. So to explain this, this experiment 0:04:53.800 --> 0:04:56.000 a little bit better. Right. He has people stand with 0:04:56.080 --> 0:04:58.560 their backs to a target so they couldn't see it, 0:04:58.960 --> 0:05:01.040 and they would take two a attempts to throw a 0:05:01.080 --> 0:05:04.400 coin and hit the target without any feedback of any kind. 0:05:04.400 --> 0:05:08.240 So they're not getting praised, they're not getting chewed out, nothing. Uh. 0:05:08.240 --> 0:05:10.880 And after staging a number of these, he found again 0:05:10.920 --> 0:05:13.960 what he suspected, that the people who were the closest 0:05:14.040 --> 0:05:17.040 on the first throw did worse on their second throw, 0:05:17.440 --> 0:05:20.120 and the people who were farthest away on their first 0:05:20.120 --> 0:05:23.680 throw tended to do better on the second throw. So 0:05:24.040 --> 0:05:27.520 what condiment is actually demonstrating here is something that doesn't 0:05:27.600 --> 0:05:31.560 really have anything to do with learning or reinforcement, or 0:05:31.600 --> 0:05:36.320 really skills or even human psychology. Instead, this demonstration is 0:05:36.400 --> 0:05:41.320 showing the effects of chance, luck, and statistics. What he 0:05:41.360 --> 0:05:44.880 was showing is the subject we're talking about today, regression 0:05:45.000 --> 0:05:48.000 to the mean. Uh. You'll you'll see that phrase a 0:05:48.000 --> 0:05:50.800 lot in in scientific literature and in statistics. But if 0:05:50.800 --> 0:05:53.960 it helps to put it in more everyday terms, anytime 0:05:53.960 --> 0:05:57.000 you see regression to the mean, you can translate it 0:05:57.000 --> 0:06:01.039 in your head as trending toward the average, trending toward 0:06:01.160 --> 0:06:05.919 the average. So to make the coin tossing illustration even clearer. 0:06:06.520 --> 0:06:09.400 Imagine you throw the coin not twice, but that you 0:06:09.440 --> 0:06:12.720 throw the coin a hundred times. So you stand there 0:06:12.720 --> 0:06:14.800 throwing the coin a hundred times. And then let's say 0:06:14.839 --> 0:06:19.960 afterwards you average together the distance from the target across 0:06:20.040 --> 0:06:22.600 all a hundred throws, and you'll come up with some 0:06:22.720 --> 0:06:26.160 kind of average distance from target. Uh, just to make 0:06:26.240 --> 0:06:28.440 up a number for the sake of argument. Common doesn't 0:06:28.440 --> 0:06:30.760 give this. But let's say the average distance from the 0:06:30.760 --> 0:06:34.880 target across all your throws is nine centimeters. And remember 0:06:34.920 --> 0:06:37.320 that you're getting no feedback at all here, so it's 0:06:37.400 --> 0:06:39.599 unlikely that you will be getting much better as you 0:06:39.640 --> 0:06:43.040 go on. So, given that the average distance from the 0:06:43.080 --> 0:06:46.279 target is nine cimeters, if you throw a coin once 0:06:46.279 --> 0:06:49.120 and it happens to be two centimeters from the target 0:06:49.200 --> 0:06:52.640 so really close, is your next throw likely to be 0:06:52.720 --> 0:06:56.680 about the same as that one, better or worse. Obviously, 0:06:56.720 --> 0:07:00.480 it is overwhelmingly likely that your next throw will be worse, 0:07:00.680 --> 0:07:04.400 just due to chance, probably closer to the average of 0:07:04.520 --> 0:07:08.159 nine cimeters away. And the same goes for throws that 0:07:08.160 --> 0:07:11.280 are really far off. You throw something three hundred centimeters 0:07:11.280 --> 0:07:14.800 off your next random toss just by chance is likely 0:07:14.840 --> 0:07:18.320 to be much better, much closer. So simply put, most 0:07:18.360 --> 0:07:22.200 of the time, if you're sampling something in a series 0:07:22.440 --> 0:07:27.400 over time, if one sample produces an extreme value, the 0:07:27.480 --> 0:07:30.600 next one in the series is more likely to be 0:07:30.840 --> 0:07:34.320 closer to the average instead of extreme in the same way. 0:07:34.880 --> 0:07:37.440 In my experience. Uh, this is this is why it 0:07:37.480 --> 0:07:40.760 can sometimes be liberating to start off a game of 0:07:40.800 --> 0:07:44.760 bowling with just a disastrous gutter ball, because because I 0:07:44.840 --> 0:07:48.080 know that I'm good enough that that's probably not going 0:07:48.120 --> 0:07:50.280 to happen twice in a row, but it's definitely going 0:07:50.320 --> 0:07:52.400 to happen at some point in the game because I'm 0:07:52.440 --> 0:07:56.120 not that good, you know. I put like playing you know, 0:07:56.280 --> 0:07:58.880 once a year or even with less frequency these days. 0:07:59.320 --> 0:08:01.720 Oh yeah. And also like why I think a lot 0:08:01.760 --> 0:08:04.160 of us have intuitions that when you try something for 0:08:04.200 --> 0:08:06.360 the first time and you do really good on the 0:08:06.440 --> 0:08:09.440 first attempt, that makes you kind of nervous because you 0:08:09.520 --> 0:08:11.320 just know you're probably not going to live up to 0:08:11.400 --> 0:08:13.680 that repeatedly. Yeah, like if you get if you get 0:08:13.680 --> 0:08:16.840 a strike that first time, then that that first um 0:08:17.120 --> 0:08:18.760 what is it round? I can't even remember. This is 0:08:18.760 --> 0:08:22.240 how one frequently I bowl, Um, the first role. So 0:08:22.400 --> 0:08:26.200 the first role first, the first column. You know, So, 0:08:26.280 --> 0:08:29.800 the tendency of regression to the mean or or trending 0:08:29.840 --> 0:08:32.760 towards the average is pretty obvious when you're dealing with 0:08:32.840 --> 0:08:36.719 something like lots of random coin tosses with no feedback, 0:08:37.400 --> 0:08:40.800 But it becomes much more obscure when you're dealing with, say, 0:08:40.960 --> 0:08:44.800 a more more limited numbers of outcomes. In the series, 0:08:44.840 --> 0:08:50.319 you're looking at and introducing possibly influential variables like pilot 0:08:50.480 --> 0:08:54.280 skill and instructor feedback. After all, we would expect that 0:08:54.760 --> 0:08:58.400 some variables having to do with instructor feedback should have 0:08:58.520 --> 0:09:01.280 an effect on pilots ill, right, That's the point of 0:09:01.280 --> 0:09:04.440 teaching is to have an effect over time, and after all, 0:09:04.480 --> 0:09:07.880 in this one scenario, the conomen describes the the instructor 0:09:08.000 --> 0:09:11.920 believed that his verbal abuse of the students was so 0:09:12.080 --> 0:09:15.000 motivating that it made them instantly better on the stick. 0:09:15.480 --> 0:09:19.640 And you can't necessarily rule that out, but it's unlikely. 0:09:19.840 --> 0:09:22.880 I think I'm convinced that regression to the mean could 0:09:22.920 --> 0:09:27.200 more easily explain this flight instructor's belief that screaming at 0:09:27.240 --> 0:09:30.839 pilots for screw ups made them better at planes, because, again, 0:09:30.920 --> 0:09:34.280 on average, even in the absence of any feedback at all. 0:09:34.840 --> 0:09:38.800 If a pilot in training executes a maneuver perfectly, the 0:09:39.000 --> 0:09:42.800 random fluctuation from one execution to the next will tend 0:09:42.840 --> 0:09:45.760 to mean that their next attempt probably won't be as 0:09:45.800 --> 0:09:48.880 good as that really good when the last time. And likewise, 0:09:48.920 --> 0:09:51.720 if they make a major error totally botch a maneuver, 0:09:51.960 --> 0:09:54.320 they're more likely to do better the next time just 0:09:54.400 --> 0:09:57.920 by chance. Both of these tendencies are regression towards the mean. 0:09:58.679 --> 0:10:01.839 But then Conomon actually draw is a really interesting observation 0:10:01.920 --> 0:10:05.920 about about about our psychology and about culture from this fact, 0:10:06.520 --> 0:10:10.040 so to quote him directly, this was a joyous moment 0:10:10.080 --> 0:10:13.000 in which I understood an important truth about the world. 0:10:13.559 --> 0:10:16.600 Because we tend to reward others when they do well 0:10:17.080 --> 0:10:20.160 and punish them when they do badly, and because there 0:10:20.320 --> 0:10:23.080 is regression to the mean, it is part of the 0:10:23.200 --> 0:10:27.760 human condition that we are statistically punished for rewarding others 0:10:28.000 --> 0:10:31.280 and rewarded for punishing them. And that was one of 0:10:31.280 --> 0:10:33.160 those things that when I read it, I was just like, 0:10:33.200 --> 0:10:37.720 oh my god, that's so true. Um, yeah, yeah, And 0:10:37.760 --> 0:10:40.640 in this specific instance, it makes me think about the 0:10:40.720 --> 0:10:44.640 special effect of reversion to the mean, fallacies on motivating 0:10:44.679 --> 0:10:47.720 belief in the effectiveness of of not just screaming at 0:10:47.760 --> 0:10:51.320 pilots in this one case, but all kinds of punishment behaviors, 0:10:51.920 --> 0:10:55.680 for example, corporal punishment. Thankfully you hear this less often 0:10:55.760 --> 0:10:58.400 these days, but I remember when I was younger, I 0:10:58.520 --> 0:11:01.120 used to hear people who would defice end the parental 0:11:01.160 --> 0:11:04.280 practice of spanking children by saying, you know, I don't 0:11:04.320 --> 0:11:06.439 I don't care what the site scientists say. I don't 0:11:06.480 --> 0:11:09.200 care what the research says. I know from experience that 0:11:09.320 --> 0:11:12.760 it works. To the extent that comments like this were 0:11:12.880 --> 0:11:15.880 based on any real experience and observation and not just 0:11:15.920 --> 0:11:18.760 sort of a free form, self justifying statement that had 0:11:18.800 --> 0:11:21.480 nothing to do with experience. I bet a lot of 0:11:21.480 --> 0:11:26.240 it was fallacious inference of causation actually based on regression 0:11:26.280 --> 0:11:29.480 to the mean, just like in this condiment example. But anyway, 0:11:29.640 --> 0:11:31.600 I thought it would be interesting to talk a bit 0:11:31.679 --> 0:11:34.800 more about regression to the mean today, because it's one 0:11:34.800 --> 0:11:37.079 of those things that, again, once you see it, it's 0:11:37.080 --> 0:11:41.000 it's pretty simple, it's actually actually pretty clear, but understanding 0:11:41.000 --> 0:11:43.440 it can help you have a better sense of how 0:11:43.480 --> 0:11:47.480 good science works and help keep you from drawing hasty 0:11:47.559 --> 0:11:51.200 inferences in everyday life. Yeah, because it is it is 0:11:51.240 --> 0:11:55.120 interesting how kind of an insidious the results can be 0:11:55.200 --> 0:11:59.920 the idea that that again, praise is ultimately punished because 0:12:00.120 --> 0:12:01.960 is there's going to be a regression to the mean, 0:12:02.400 --> 0:12:05.679 to to to to the mean, and then likewise there 0:12:05.720 --> 0:12:10.240 can be this illusion, uh that uh that's screaming at 0:12:10.240 --> 0:12:12.640 pilots and so forth is going to be the successful 0:12:12.640 --> 0:12:15.440 way to go about things. Um. So yeah, this is 0:12:15.480 --> 0:12:17.760 I think this is an important episode to cover because 0:12:17.760 --> 0:12:19.320 it's the kind of thing that it's the kind of 0:12:19.320 --> 0:12:21.640 tool you kind of need tucked in your back pocket, 0:12:22.000 --> 0:12:25.200 even if you're just doing something like like scanning science 0:12:25.240 --> 0:12:28.240 headlines on a you know, a news server or social 0:12:28.280 --> 0:12:32.600 media message board. Yeah, because of course, understanding regression to 0:12:32.640 --> 0:12:36.120 the mean is extremely important in what scientists do when 0:12:36.120 --> 0:12:40.559 they design good experiments. If you don't take into account 0:12:40.559 --> 0:12:43.480 regression to the mean, you can incorrectly believe you have 0:12:43.559 --> 0:12:46.520 discovered some kind of tiger repellent or something. Uh. This 0:12:46.559 --> 0:12:49.319 concern plays a huge role in the history of medicine. 0:12:49.720 --> 0:12:52.080 It's part of the design of good medical research, or 0:12:52.120 --> 0:12:56.360 really any field that seeks to find remedies for problems. 0:12:56.960 --> 0:13:01.080 So consider a very basic hypothetical, uh path medicines, say 0:13:01.120 --> 0:13:03.160 from a hundred years ago. So you know, you have 0:13:03.320 --> 0:13:06.880 you have a foot pain that you've never really had before. Uh, 0:13:07.440 --> 0:13:08.920 you know, you want it to go away. So you 0:13:08.960 --> 0:13:11.280 go to the store and you buy a bottle of 0:13:11.400 --> 0:13:15.280 doctor Field Grades No Fail Pantasy for tumors, ulcers, cramps, 0:13:15.280 --> 0:13:18.640 and rooms, and you you pull the cork out, you 0:13:18.720 --> 0:13:21.440 chug it, and then the next day your foot feels better. 0:13:22.000 --> 0:13:25.319 Now you can conclude from this that the doctor Field 0:13:25.320 --> 0:13:28.679 Grades cured you. But how do you know actually that 0:13:28.720 --> 0:13:31.520 the feelings in your foot didn't just regress to the 0:13:31.559 --> 0:13:34.599 mean because the average is a low amount or no 0:13:34.720 --> 0:13:37.200 amount of foot pain. And if you don't have a 0:13:37.240 --> 0:13:41.240 medication that's tested with control groups and and randomized allocation 0:13:41.280 --> 0:13:44.480 into the groups, then how do you know that that 0:13:44.559 --> 0:13:47.920 the medicine actually did anything at all? Yeah? Yeah, So 0:13:47.960 --> 0:13:50.479 many of the examples you see for this and the applications, 0:13:50.520 --> 0:13:53.600 you're dealing with some sort of situation in the world, 0:13:53.600 --> 0:13:58.439 whether there is fluctuation and or change happening, often separately 0:13:58.520 --> 0:14:00.560 from whatever is being tested. So in this case, yeah, 0:14:00.559 --> 0:14:03.679 the doctor Field grads could have just been like just water. 0:14:03.960 --> 0:14:06.400 It just just you know, but there is the illusion 0:14:06.440 --> 0:14:09.360 that it worked because things got better. But if you 0:14:09.360 --> 0:14:11.520 don't have a control group and to you know, to 0:14:11.600 --> 0:14:13.120 drive home what that is. That would be like if 0:14:13.160 --> 0:14:15.680 you had a had like three different groups and a 0:14:15.720 --> 0:14:19.440 study of doctor Field Greats elixir. Here, one group was 0:14:19.480 --> 0:14:23.720 taking doctor Field graades elixer, another group was taking I 0:14:23.760 --> 0:14:25.840 don't know, let's say a half dose of Feel Grade 0:14:25.960 --> 0:14:29.640 or maybe a competitor's tonic. And in one group the 0:14:29.680 --> 0:14:34.120 control group was taking nothing was or was taking you know, 0:14:34.160 --> 0:14:37.960 just water or something to that effect, something completely innate. Uh. 0:14:38.000 --> 0:14:42.080 And that would be that would be a group that 0:14:42.080 --> 0:14:45.360 you would judge the results of the other categories by, right, 0:14:45.360 --> 0:14:47.880 and you would need to randomly sort the people into 0:14:47.880 --> 0:14:50.280 those groups. So it wasn't just that, you know, the 0:14:50.680 --> 0:14:54.000 only the people with real severe foot pain we're taking 0:14:54.040 --> 0:14:57.080 the doctor Field grades because the more extreme their pain 0:14:57.160 --> 0:15:00.160 to begin with, probably the more likely they are are 0:15:00.200 --> 0:15:03.560 to have that pain be lessened or go away over time, 0:15:03.640 --> 0:15:06.520 just naturally. Right. And uh. And I'm going to have 0:15:06.560 --> 0:15:08.880 a more specific example of this a little later in 0:15:08.880 --> 0:15:10.800 the podcast. So if you if you still don't get it, 0:15:10.840 --> 0:15:13.040 just hang on we'll we'll have another example in a 0:15:13.080 --> 0:15:21.120 bit thank. I was looking at an article in the 0:15:21.120 --> 0:15:24.960 British Medical Journal from nineteen that was just a collection 0:15:25.040 --> 0:15:28.360 of different examples of regression to the mean in real 0:15:28.400 --> 0:15:32.040 life medical research. This was by J. Martin Bland and 0:15:32.120 --> 0:15:36.480 Douglas J. Altman called statistics notes some examples of regression 0:15:36.480 --> 0:15:39.520 towards the mean, and they point out a very common 0:15:39.560 --> 0:15:42.200 type of example. So this will be similar to what 0:15:42.240 --> 0:15:45.600 we just talked about. The author's right. In clinical practice, 0:15:45.640 --> 0:15:49.960 there are many measurements such as weight, serum, cholesterol concentration, 0:15:50.120 --> 0:15:54.480 or blood pressure, for which particularly high or low values 0:15:54.520 --> 0:15:58.240 are signs of underlying disease or risk factors for disease. 0:15:58.840 --> 0:16:01.800 People with extreme values of the measurements such as high 0:16:01.840 --> 0:16:05.280 blood pressure may be treated to bring their values closer 0:16:05.320 --> 0:16:07.960 to the mean. If they are measured again, we will 0:16:08.000 --> 0:16:10.640 observe that the mean of the extreme group is now 0:16:10.720 --> 0:16:13.480 closer to the mean of the whole population. That is, 0:16:13.680 --> 0:16:17.160 it is reduced. This should not be interpreted as showing 0:16:17.200 --> 0:16:20.680 the effect of the treatment. Even if subjects are not treated, 0:16:20.760 --> 0:16:24.120 the mean blood pressure will go down owing to regression 0:16:24.160 --> 0:16:27.320 towards the means. So again something starts with an extreme 0:16:27.480 --> 0:16:30.520 value in certain types of cases, you would just expect 0:16:30.560 --> 0:16:33.800 it to have a less extreme value the next time 0:16:33.920 --> 0:16:38.040 due to random fluctuation. Uh So again, you know this 0:16:38.080 --> 0:16:41.000 could fill you with despair because you might wonder, well, 0:16:41.040 --> 0:16:42.880 then how could you ever know if a treatment was 0:16:42.920 --> 0:16:45.640 effective or not. But again, this is where the standard 0:16:45.680 --> 0:16:49.080 practices of science based medicine come to play. Instead of 0:16:49.160 --> 0:16:52.120 just taking people with some extreme measurement and giving them 0:16:52.120 --> 0:16:56.680 a treatment, you randomize them into test groups and control groups, 0:16:56.680 --> 0:16:58.280 like we were just talking about. So if you have 0:16:58.320 --> 0:17:01.720 a large enough sample, you really randomize the groups. People 0:17:01.760 --> 0:17:05.600 with the extreme starting conditions will somewhat regress towards the mean, 0:17:05.760 --> 0:17:08.280 but they will all regress toward the mean on average 0:17:08.400 --> 0:17:11.640 the same rate, whether they're receiving a real potential treatment 0:17:12.040 --> 0:17:14.440 or they're in the placebo group. But if the treatment 0:17:14.480 --> 0:17:17.600 actually does something helpful, this effect will manifest as the 0:17:17.640 --> 0:17:21.880 difference between the two groups. So good scientific research, good 0:17:21.920 --> 0:17:25.440 medical research has methods for excluding the effects of reversion 0:17:25.480 --> 0:17:28.480 to the mean on their findings. We have the tools, 0:17:28.520 --> 0:17:32.480 but we can still fall into the trap of regression 0:17:32.480 --> 0:17:35.679 to the mean fallacies, especially in our day to day lives. 0:17:35.840 --> 0:17:39.320 Drawing inferences the way that that the pilot and Inconomens 0:17:39.320 --> 0:17:42.159 story did, or or even in science if we're not 0:17:42.240 --> 0:17:45.960 careful and deliberate about designing experiments. And in addition to 0:17:46.119 --> 0:17:49.879 just a methodology design that has you know, a randomized 0:17:49.920 --> 0:17:52.960 groups and control groups, there are also ways of trying 0:17:53.000 --> 0:17:56.320 to counteract regression to the mean, just through statistical methods 0:17:56.400 --> 0:17:59.560 that are maybe less reliable, but there are statistical methods 0:17:59.560 --> 0:18:03.680 people can used to try to apply sort of modifiers 0:18:03.760 --> 0:18:06.800 to data in order to estimate regression to the mean 0:18:07.000 --> 0:18:10.600 and uh and counteract its effects. So again, we have 0:18:10.720 --> 0:18:13.679 tools within scientific research to to figure this out, and 0:18:13.720 --> 0:18:16.760 it's a lot of what science does is trying to 0:18:16.800 --> 0:18:19.560 sort out the difference between regression to the mean and 0:18:19.680 --> 0:18:23.240 actual effects of interventions. But in our day to day lives, 0:18:23.320 --> 0:18:26.080 we still fall for regression to the mean fallacies all 0:18:26.080 --> 0:18:29.240 the time. Yeah, and it's important to realize too that 0:18:29.320 --> 0:18:31.680 it's not just a situation where regression towards the mean 0:18:32.080 --> 0:18:36.000 could create an illusion of something working when it doesn't. Uh. 0:18:36.040 --> 0:18:40.800 You know, sometimes it can just potentially overstate um the 0:18:40.840 --> 0:18:43.879 effects of something. For an example of that that I 0:18:43.920 --> 0:18:46.960 was looking at was that regression towards the mean, or 0:18:46.960 --> 0:18:49.600 the failure to account for it can also overstate the 0:18:49.600 --> 0:18:53.520 effectiveness of something like traffic light cameras. Is it making 0:18:53.560 --> 0:18:57.680 a difference and cutting down on accidents? Perhaps, but any 0:18:57.760 --> 0:19:02.520 actual effectiveness could potentially be overstated by failure to account 0:19:02.680 --> 0:19:05.680 for just regression towards the mean. Oh yeah, so where 0:19:05.680 --> 0:19:09.240 do you tend to install things like that? High acts 0:19:09.400 --> 0:19:12.440 like problem areas? Right, So, if there's like a stretch 0:19:12.520 --> 0:19:15.800 of road that has a lot of problems on people 0:19:15.920 --> 0:19:18.520 really speeding a lot there or crashing a lot there, 0:19:18.920 --> 0:19:22.160 that might be where you stage the intervention. It's possible 0:19:22.200 --> 0:19:26.480 some things like that fluctuate naturally over time in different locations, 0:19:27.200 --> 0:19:29.280 and you put the cameras in place, and it could 0:19:29.280 --> 0:19:31.359 have an effect, but maybe not as much of an 0:19:31.359 --> 0:19:35.159 effect as it looks like it is taking place. Again, 0:19:35.240 --> 0:19:39.560 if you don't factor regression towards the mean into the study. 0:19:40.040 --> 0:19:43.840 Right now, While our TM is a very important phenomenon 0:19:43.880 --> 0:19:46.840 to understand and take into account, it certainly doesn't apply 0:19:47.040 --> 0:19:51.320 to every sequence of values you could repeatedly sample, so 0:19:51.400 --> 0:19:53.840 you also have to be careful not to apply it 0:19:53.880 --> 0:19:57.720 in situations where it isn't warranted. I was you know, 0:19:57.920 --> 0:19:59.879 there are a million examples. You could cite one that 0:20:00.040 --> 0:20:02.919 came to my mind as the orbital decay of a satellite. 0:20:03.240 --> 0:20:06.439 Let's say you've got a communication satellite in lower orbit 0:20:06.880 --> 0:20:09.040 and you get a reading on its altitude and the 0:20:09.119 --> 0:20:13.320 reading is lower than the satellites average altitude. Uh. Now 0:20:14.000 --> 0:20:16.000 you might say, hey, I think this means we need 0:20:16.040 --> 0:20:18.600 to program a reboost to insert it back into the 0:20:19.160 --> 0:20:22.640 orbit where it's supposed to be. And somebody could erroneously 0:20:22.920 --> 0:20:25.960 apply regression to the mean here and say, nah, we 0:20:26.000 --> 0:20:28.240 don't need to do that. The satellite might just return 0:20:28.320 --> 0:20:31.520 to its average altitude. It doesn't apply in this scenario, 0:20:31.600 --> 0:20:34.439 even though you are taking repeated measurements of a value 0:20:34.480 --> 0:20:39.119 over time, because we know things about the physical characteristics 0:20:39.160 --> 0:20:42.840 determining the orbit of satellites and in lower th orbit uh, 0:20:42.880 --> 0:20:46.440 and that due to factors like atmospheric drag, their altitude 0:20:46.440 --> 0:20:50.359 tends to trend steadily downward over time in a consistent 0:20:50.400 --> 0:20:54.320 direction down down, down, So eventually you will need a 0:20:54.359 --> 0:20:56.600 reboost in order to put it back up to the 0:20:56.640 --> 0:21:00.240 correct distance. So regression to the mean apply is to 0:21:00.440 --> 0:21:04.440 certain kinds of data that are repeatedly sampled data where 0:21:04.480 --> 0:21:09.320 there is natural random fluctuation back and forth, not a 0:21:09.440 --> 0:21:12.320 steady trend in the data in one direction on the 0:21:12.359 --> 0:21:15.720 relevant time scale. The other thing that's important to understand 0:21:15.800 --> 0:21:18.800 is that systems where you expect to find regression to 0:21:18.880 --> 0:21:22.639 the mean are systems in which the repeated data values 0:21:22.680 --> 0:21:26.800 you're sampling are to some degree determined by luck or chance. 0:21:27.359 --> 0:21:30.800 If a series of values is influenced almost entirely by 0:21:31.080 --> 0:21:34.720 deterministic influence, like in the satellite example, by like the 0:21:34.800 --> 0:21:39.199 laws of physics, or by some extremely reliable skill with 0:21:39.320 --> 0:21:43.399 little room for variation, values don't really regress towards the 0:21:43.440 --> 0:21:46.040 mean in the same way because there's just less random 0:21:46.080 --> 0:21:49.760 fluctuation back and forth to begin with. The more chance 0:21:49.880 --> 0:21:53.119 and random variation plays a role in the outcome, the 0:21:53.240 --> 0:21:55.960 more you will tend to observe regression towards the mean 0:21:56.040 --> 0:21:58.920 after an extreme sample in in whatever it is you're 0:21:58.960 --> 0:22:02.879 looking at, I've I've read that the progression towards the 0:22:02.880 --> 0:22:05.760 mean is is not to be confused with the law 0:22:05.800 --> 0:22:09.119 of large numbers. For example, uh. This is the the 0:22:09.200 --> 0:22:11.800 law that that states, as a sample size becomes larger, 0:22:12.040 --> 0:22:15.600 the sample mean gets closer to the expected value. So 0:22:15.920 --> 0:22:18.480 a coin flipping example is key here. Flip a coin 0:22:18.840 --> 0:22:21.640 and the random results are going to ultimately average out 0:22:21.960 --> 0:22:25.320 to a point five proportion. But if you only flip 0:22:25.400 --> 0:22:29.000 the coin ten times, you might not see this breakdown. Um. 0:22:29.040 --> 0:22:32.399 And this also applies to say, even odds on the 0:22:32.480 --> 0:22:34.400 rolling of a of a D six of a six 0:22:34.440 --> 0:22:38.520 sided die. Uh So for example, two regular people, that's 0:22:38.560 --> 0:22:42.240 just to die that nerves like us, it's a D six. Yeah. 0:22:42.480 --> 0:22:43.680 D six is what I could get my hands on. 0:22:43.720 --> 0:22:45.040 Because I was like, well, I'm gonna do an example. 0:22:45.040 --> 0:22:47.160