WEBVTT - Regression to the Mean 0:00:03.040 --> 0:00:05.320 Welcome to Stuff to Blow Your Mind, the production of 0:00:05.320 --> 0:00:14.840 My Heart Radio. Hey, welcome to Stuff to Blow Your Mind. 0:00:15.000 --> 0:00:18.040 My name is Robert Lamb, and I'm Joe McCormick. And 0:00:18.160 --> 0:00:21.079 in today's episode, we are going to be focusing on 0:00:21.120 --> 0:00:24.400 a topic that is already something that's very well known 0:00:24.600 --> 0:00:28.680 to people who are familiar with quantitative research and statistics, 0:00:28.760 --> 0:00:32.159 but less known to the general public. And uh and 0:00:32.200 --> 0:00:34.479 I think that's a tragedy because it's an idea that 0:00:34.479 --> 0:00:39.000 should really be part of everybody's basic critical thinking toolkit, 0:00:39.080 --> 0:00:42.240 no matter what your job is. And so in order 0:00:42.280 --> 0:00:44.840 to introduce this concept, I thought it would be best 0:00:44.880 --> 0:00:47.760 to start with a with a direct illustration from the 0:00:47.800 --> 0:00:52.959 real world of people reaching incorrect conclusions by not understanding 0:00:53.000 --> 0:00:55.960 the subject of today's episode. And so the illustration I 0:00:56.000 --> 0:00:59.000 want to start with is an interesting story told by 0:00:59.200 --> 0:01:03.240 the psycholo, just Daniel Kaneman, that's about the illusory power 0:01:03.360 --> 0:01:07.560 of screaming at pilots. Uh. So, the context of the 0:01:07.600 --> 0:01:10.480 story is that Knemon says he was giving a lecture 0:01:10.680 --> 0:01:15.360 about positive reinforcement to a group of flight instructures. I 0:01:15.360 --> 0:01:18.840 think this was in the nineteen sixties, and Kaneman was 0:01:18.959 --> 0:01:22.600 trying to inform them about what he believed at the 0:01:22.600 --> 0:01:25.839 time was the best consensus of scientific research on learning 0:01:25.880 --> 0:01:29.640 and reinforcement, which was at the time that if these 0:01:29.640 --> 0:01:33.280 flight instructors wanted their students to have the best possible outcomes, 0:01:33.600 --> 0:01:36.880 they should focus more on praising the students when they 0:01:36.880 --> 0:01:39.560 did well than on chewing them out when they did 0:01:39.600 --> 0:01:43.880 something wrong. And Kneman says that when he finished his talk, 0:01:44.319 --> 0:01:46.440 one of the flight instructors that he had been giving 0:01:46.440 --> 0:01:49.360 this lecture two got up and tried to dispute him. 0:01:49.360 --> 0:01:52.240 He said, no, you're wrong, And so the direct quote 0:01:52.240 --> 0:01:55.880 Economy gives from the instructor here is on many occasions, 0:01:55.920 --> 0:01:59.360 I have praised flight cadets for clean execution of some 0:01:59.480 --> 0:02:02.560 aero attic maneuver, and in general when they try it 0:02:02.600 --> 0:02:06.040 again they do worse. On the other hand, I've often 0:02:06.160 --> 0:02:10.359 screamed at cadets for bad execution, and in general they 0:02:10.440 --> 0:02:13.280 do better the next time. So please don't tell us 0:02:13.320 --> 0:02:17.320 that reinforcement works and punishment does not, because the opposite 0:02:17.440 --> 0:02:20.160 is the case. So you might think he has a 0:02:20.160 --> 0:02:23.080 good point here if you accept that this flight instructor 0:02:23.160 --> 0:02:26.359 has had a lot of direct experience working with students, 0:02:26.800 --> 0:02:30.320 and you trust him to remember the relative frequency of 0:02:30.360 --> 0:02:33.720 these events pretty well, you might assume that he has 0:02:33.760 --> 0:02:37.320 a meaningful rebuke to ekonom In. Here again, he says 0:02:37.400 --> 0:02:39.800 that most of the time, after a cadet does something 0:02:39.919 --> 0:02:42.600 bad and he screams at them, they do better the 0:02:42.639 --> 0:02:45.560 next time. And after a cadet does something good and 0:02:45.639 --> 0:02:48.600 he praises them, they actually do worse the next time. 0:02:49.040 --> 0:02:52.359 So if he's remembering these experiences correctly, and he's had 0:02:52.400 --> 0:02:55.160 a lot of them, it would really seem like evidence 0:02:55.240 --> 0:02:59.160 that praise has a negative effect on learning, maybe by 0:02:59.280 --> 0:03:03.040 making the student pilots soft and overconfident or something, and 0:03:03.080 --> 0:03:06.639 getting chewed out is good for skill development. I think 0:03:06.680 --> 0:03:08.880 it's quite easy to see the allure of this, this 0:03:09.000 --> 0:03:12.680 false conclusion, right right, And it's and you can also 0:03:12.680 --> 0:03:15.120 easily imagine how you kind of build upon this with 0:03:15.280 --> 0:03:20.240 certain loosely backed up you know, folk ideas about how 0:03:20.280 --> 0:03:22.839 you encourage people and how people learn, and you got 0:03:22.840 --> 0:03:24.360 to stay on them if they if you tell them 0:03:24.360 --> 0:03:27.200 they're doing a good job, they'll get lazy, right, folk wisdom, 0:03:27.280 --> 0:03:30.680 tough guy mentality. Yeah, But Knemon saw something different in 0:03:30.680 --> 0:03:33.920 this response, and he says that he immediately set up 0:03:33.919 --> 0:03:38.040 an experiment on the spot to demonstrate the flaw in 0:03:38.120 --> 0:03:41.040 the flight instructors thinking here, so I want to read 0:03:41.080 --> 0:03:45.560 from Knomen's description, he says, I immediately arranged a demonstration 0:03:45.600 --> 0:03:49.240 in which each participant tossed two coins at a target 0:03:49.320 --> 0:03:53.880 behind his back without any feedback. We measured the distances 0:03:53.920 --> 0:03:56.280 from the target and could see that those who had 0:03:56.320 --> 0:04:01.080 done best the first time had mostly deteriorated their second try, 0:04:01.120 --> 0:04:04.680 and vice versa. But I knew that this demonstration would 0:04:04.680 --> 0:04:10.120 not undo the effects of lifelong exposure to a perverse contingency. 0:04:10.720 --> 0:04:13.440 So to explain this, this experiment a little bit better, right, 0:04:13.440 --> 0:04:16.120 he has people stand with their backs to a target 0:04:16.240 --> 0:04:18.560 so they couldn't see it, and they would take two 0:04:18.560 --> 0:04:21.719 attempts to throw a coin and hit the target without 0:04:21.800 --> 0:04:24.240 any feedback of any kind. So they're not getting praised, 0:04:24.279 --> 0:04:27.839 they're not getting chewed out, nothing. Uh. And after staging 0:04:27.880 --> 0:04:30.560 a number of these, he found again what he suspected, 0:04:30.839 --> 0:04:33.360 that the people who were the closest on the first 0:04:33.400 --> 0:04:36.640 throw did worse on their second throw, and the people 0:04:36.640 --> 0:04:39.839 who were farthest away on their first throw tended to 0:04:39.880 --> 0:04:43.560 do better on the second throw. So what kandiment is 0:04:43.560 --> 0:04:47.200 actually demonstrating here is something that doesn't really have anything 0:04:47.240 --> 0:04:51.320 to do with learning or reinforcement, or really skills or 0:04:51.360 --> 0:04:56.280 even human psychology. Instead, this demonstration is showing the effects 0:04:56.320 --> 0:05:00.640 of chance, luck, and statistics. What he was showing is 0:05:00.720 --> 0:05:05.240 the subject we're talking about today, regression to the mean. Uh. 0:05:05.520 --> 0:05:07.719 You'll you'll see that phrase a lot in in scientific 0:05:07.839 --> 0:05:10.200 literature and in statistics. But if it helps to put 0:05:10.240 --> 0:05:13.880 it in more everyday terms, anytime you see regression to 0:05:13.960 --> 0:05:16.440 the mean, you can translate it in your head as 0:05:16.680 --> 0:05:21.479 trending toward the average, trending toward the average. So, to 0:05:21.560 --> 0:05:26.440 make the coin tossing illustration even clearer, imagine you throw 0:05:26.480 --> 0:05:28.719 the coin not twice, but that you throw the coin 0:05:28.880 --> 0:05:31.920 a hundred times. So you stand there throwing the coin 0:05:32.000 --> 0:05:35.160 a hundred times. And then let's say afterwards you average 0:05:35.200 --> 0:05:39.920 together the distance from the target across all a hundred throws, 0:05:40.320 --> 0:05:42.880 and you'll come up with some kind of average distance 0:05:42.920 --> 0:05:45.520 from target. Uh, just to make up a number for 0:05:45.520 --> 0:05:47.800 the sake of argument. Common doesn't give this. But let's 0:05:47.800 --> 0:05:50.560 say the average distance from the target across all your 0:05:50.600 --> 0:05:54.240 throws is nine centimeters. And remember that you're getting no 0:05:54.440 --> 0:05:57.000 feedback at all. Here, so it's unlikely that you will 0:05:57.040 --> 0:06:00.280 be getting much better as you go on. So even 0:06:00.320 --> 0:06:03.440 that the average distance from the target is nine cimeters, 0:06:03.680 --> 0:06:05.719 if you throw a coin once and it happens to 0:06:05.760 --> 0:06:09.800 be two centimeters from the targets really close, is your 0:06:09.839 --> 0:06:12.760 next throw likely to be about the same as that one, 0:06:13.080 --> 0:06:17.720 better or worse. Obviously, it is overwhelmingly likely that your 0:06:17.720 --> 0:06:21.320 next throw will be worse, just due to chance, probably 0:06:21.440 --> 0:06:25.479 closer to the average of nine cimeters away. And the 0:06:25.520 --> 0:06:27.840 same goes for throws that are really far off. If 0:06:27.839 --> 0:06:31.920 you throw something three hundred centimeters off, your next random toss, 0:06:32.080 --> 0:06:35.840 just by chance, is likely to be much better, much closer. So, 0:06:35.880 --> 0:06:39.799 simply put, most of the time, if you're sampling something 0:06:39.960 --> 0:06:44.120 in a series over time, if one sample produces an 0:06:44.120 --> 0:06:48.520 extreme value, the next one in the series is more 0:06:48.560 --> 0:06:52.039 likely to be closer to the average instead of extreme 0:06:52.120 --> 0:06:55.480 in the same way. In my experience, Uh, this is 0:06:55.600 --> 0:06:58.320 This is why it can sometimes be liberating to start 0:06:58.360 --> 0:07:02.240 off a game of bowling with just a disastrous gutter ball, 0:07:02.720 --> 0:07:06.159 because because I know that I'm good enough that that's 0:07:06.160 --> 0:07:08.320 probably not gonna happen twice in a row, but it's 0:07:08.360 --> 0:07:10.560 definitely going to happen at some point in the game 0:07:10.600 --> 0:07:14.520 because I'm not that good, you know, I like playing 0:07:14.640 --> 0:07:16.920 you know, once a year or even with less frequency 0:07:17.040 --> 0:07:19.960 these days. Oh yeah. And it's also like why I 0:07:20.000 --> 0:07:22.000 think a lot of us have intuitions that when you 0:07:22.040 --> 0:07:24.480 try something for the first time and you do really 0:07:24.520 --> 0:07:27.080 good on the first attempt, that makes you kind of 0:07:27.120 --> 0:07:29.760 nervous because you just know you're probably not gonna live 0:07:29.840 --> 0:07:32.200 up to that repeatedly. Yeah, like if you get you 0:07:32.200 --> 0:07:35.160 get a strike that first time, then that that first 0:07:35.400 --> 0:07:37.400 um what is it round? I can't even remember. This 0:07:37.400 --> 0:07:41.000 is how frequently I bowl. Um, the first role. So 0:07:41.120 --> 0:07:44.880 the first role first, the first column, you know. So 0:07:44.960 --> 0:07:48.480 the tendency of regression to the mean or or trending 0:07:48.520 --> 0:07:51.480 towards the average is pretty obvious when you're dealing with 0:07:51.520 --> 0:07:55.400 something like lots of random coin tosses with no feedback, 0:07:56.080 --> 0:07:59.200 But it becomes much more obscure when you're dealing with, 0:07:59.240 --> 0:08:02.840 say a more a more limited numbers of outcomes. In 0:08:02.880 --> 0:08:08.280 the series you're looking at and introducing possibly influential variables 0:08:08.320 --> 0:08:12.360 like pilot skill and instructor feedback. After all, we would 0:08:12.360 --> 0:08:16.320 expect that some variables having to do with instructor feedback 0:08:16.640 --> 0:08:19.520 should have an effect on pilot skill, right, That's the 0:08:19.560 --> 0:08:22.000 point of teaching is to have an effect over time. 0:08:22.400 --> 0:08:25.400 And after all, in this one scenario, the Konomen describes 0:08:25.480 --> 0:08:29.080 the the instructor believed that his verbal abuse of the 0:08:29.120 --> 0:08:33.120 students was so motivating that it made them instantly better 0:08:33.160 --> 0:08:36.480 on the stick. And you can't necessarily rule that out, 0:08:36.920 --> 0:08:40.720 but it's unlikely. I think. I'm convinced that regression to 0:08:40.760 --> 0:08:45.000 the mean could more easily explain this flight instructor's belief 0:08:45.040 --> 0:08:48.280 that screaming at pilots for screw ups made them better 0:08:48.320 --> 0:08:51.360 at planes, because, again, on average, even in the absence 0:08:51.400 --> 0:08:54.760 of any feedback at all, if a pilot in training 0:08:55.040 --> 0:09:00.000 executes a maneuver perfectly, the random fluctuation from one execut 0:09:00.040 --> 0:09:02.400 usian to the next will tend to mean that their 0:09:02.480 --> 0:09:05.439 next attempt probably won't be as good as that really 0:09:05.440 --> 0:09:08.079 good when the last time. And likewise, if they make 0:09:08.120 --> 0:09:11.640 a major error totally botcha maneuver, they're more likely to 0:09:11.679 --> 0:09:14.600 do better the next time just by chance. Both of 0:09:14.640 --> 0:09:18.160 these tendencies are regression towards the mean. But then Conomon 0:09:18.200 --> 0:09:21.920 actually draws a really interesting observation about about about our 0:09:21.960 --> 0:09:25.880 psychology and about culture from this fact, so, to quote 0:09:25.920 --> 0:09:29.240 him directly, this was a joyous moment in which I 0:09:29.320 --> 0:09:33.160 understood an important truth about the world. Because we tend 0:09:33.200 --> 0:09:36.640 to reward others when they do well and punish them 0:09:36.640 --> 0:09:39.920 when they do badly, and because there is regression to 0:09:40.000 --> 0:09:43.360 the mean, it is part of the human condition that 0:09:43.400 --> 0:09:47.760 we are statistically punished for rewarding others and rewarded for 0:09:47.840 --> 0:09:50.520 punishing them. And that was one of those things that 0:09:50.559 --> 0:09:52.720 when I read it, I was just like, oh my god, 0:09:53.000 --> 0:09:57.800 that's so true. Um, yeah, yeah, And in this specific instance, 0:09:58.280 --> 0:10:00.920 it makes me think about the special fact of reversion 0:10:00.960 --> 0:10:04.880 to the mean, fallacies on motivating belief in the effectiveness 0:10:04.960 --> 0:10:07.520 of of not just screaming at pilots in this one case, 0:10:07.559 --> 0:10:12.319 but all kinds of punishment behaviors, for example, corporal punishment. 0:10:12.760 --> 0:10:15.120 Thankfully you hear this less often these days, but I 0:10:15.120 --> 0:10:18.000 remember when I was younger, I used to hear people 0:10:18.000 --> 0:10:22.480 who would defend the parental practice of spanking children by saying, 0:10:22.480 --> 0:10:24.120 you know, I don't I don't care what the site 0:10:24.160 --> 0:10:26.440 scientists say. I don't care what the research says. I 0:10:26.520 --> 0:10:30.280 know from experience that it works to the extent that 0:10:30.320 --> 0:10:33.400 comments like this were based on any real experience and 0:10:33.440 --> 0:10:36.040 observation and not just sort of a free form, self 0:10:36.120 --> 0:10:39.280 justifying statement that had nothing to do with experience. I 0:10:39.320 --> 0:10:43.080 bet a lot of it was fallacious inference of causation 0:10:43.240 --> 0:10:46.080 actually based on regression to the mean, just like in 0:10:46.120 --> 0:10:49.040 this commument example. But anyway, I thought it would be 0:10:49.080 --> 0:10:51.960 interesting to talk a bit more about regression to the 0:10:52.000 --> 0:10:54.400 mean today because it's one of those things that, again, 0:10:54.480 --> 0:10:57.600 once you see it, it's it's pretty simple, it's actually 0:10:57.679 --> 0:11:01.080 actually pretty clear. But understanding it can help you have 0:11:01.120 --> 0:11:04.000 a better sense of how good science works and help 0:11:04.160 --> 0:11:08.960 keep you from drawing hasty inferences in everyday life. Yeah, 0:11:09.040 --> 0:11:12.240 because it is. It is interesting how kind of an 0:11:12.320 --> 0:11:15.480 insidious the results can be, the idea that that again, 0:11:16.240 --> 0:11:19.719 praise is ultimately punished because there's going to be a 0:11:19.800 --> 0:11:22.240 regression to the mean, to to to to the mean, 0:11:22.600 --> 0:11:26.560 and then likewise there can be this illusion, uh that 0:11:27.640 --> 0:11:30.360 uh that's screaming at pilots and so forth is going 0:11:30.400 --> 0:11:33.840 to be the successful way to go about things. Um So, yeah, 0:11:33.880 --> 0:11:35.599 this is I think this is an important episode to 0:11:35.640 --> 0:11:37.679 cover because it's the kind of thing that it's the 0:11:37.760 --> 0:11:39.720 kind of tool you kind of need tucked in your 0:11:39.720 --> 0:11:42.559 back pocket, even if you're just doing something like like 0:11:42.640 --> 0:11:45.800 scanning science headlines on a you know, a news server 0:11:46.120 --> 0:11:50.640 or social media message board. Yeah, because of course, understanding 0:11:50.640 --> 0:11:54.480 regression to the mean is extremely important in what scientists 0:11:54.480 --> 0:11:58.640 do when they design good experiments. If you don't take 0:11:58.679 --> 0:12:01.960 into account regression to the mean, you can incorrectly believe 0:12:02.000 --> 0:12:05.079 you have discovered some kind of tiger repellent or something. Uh. 0:12:05.120 --> 0:12:08.000 This concern plays a huge role in the history of medicine. 0:12:08.400 --> 0:12:10.760 It's part of the design of good medical research, or 0:12:10.840 --> 0:12:15.040 really any field that seeks to find remedies for problems. 0:12:15.679 --> 0:12:19.719 So consider a very basic hypothetical, uh patent medicine, say 0:12:19.800 --> 0:12:21.880 from a hundred years ago. So you know, you have 0:12:22.000 --> 0:12:25.600 you have a foot pain that you've never really had before. Uh. 0:12:26.120 --> 0:12:27.600 You know, you want it to go away. So you 0:12:27.679 --> 0:12:29.960 go to the store and you buy a bottle of 0:12:30.080 --> 0:12:34.000 Doctor Field Grades No Fail Pantasy for tumors, ulcers, cramps, 0:12:34.000 --> 0:12:37.319 and rooms, and you you pull the cork out, you 0:12:37.400 --> 0:12:40.120 chug it, and then the next day your foot feels better. 0:12:40.720 --> 0:12:44.040 Now you can conclude from this that the Doctor Field 0:12:44.040 --> 0:12:47.320 Grades cured you. But how do you know actually that 0:12:47.400 --> 0:12:50.440 the feelings in your foot didn't just regress to the mean, 0:12:50.640 --> 0:12:53.800 because the average is a low amount or no amount 0:12:53.800 --> 0:12:56.480 of foot pain. And if you don't have a medication 0:12:56.520 --> 0:13:00.240 that's tested with control groups and and randomized allocation into 0:13:00.240 --> 0:13:03.320 the groups, then how do you know that that the 0:13:03.360 --> 0:13:06.800 medicine actually did anything at all? Yeah? Yeah, So many 0:13:06.840 --> 0:13:09.199 of the examples you see for this and the applications, 0:13:09.240 --> 0:13:12.240 you're dealing with some sort of situation in the world 0:13:12.320 --> 0:13:17.120 where there is fluctuation and or change happening, often separately 0:13:17.200 --> 0:13:19.200 from whatever is being tested. So in this case, yeah, 0:13:19.200 --> 0:13:22.400 the Doctor Field Greats could have just been like just water, 0:13:22.640 --> 0:13:25.080 It just just you know, but there is the illusion 0:13:25.120 --> 0:13:28.040 that it worked because things got better. But if you 0:13:28.040 --> 0:13:30.199 don't have a control group and to you know, to 0:13:30.280 --> 0:13:31.800 drive home what that is, that would be like if 0:13:31.840 --> 0:13:34.360 you had a had like three different groups and a 0:13:34.400 --> 0:13:38.120 study of Doctor Field Greats elixir. Here, one group was 0:13:38.200 --> 0:13:42.360 taking Doctor Feel Greats elixer, another group was taking I 0:13:42.440 --> 0:13:44.480 don't know, let's say a half dose of Feel Grade 0:13:44.640 --> 0:13:48.280 or maybe a competitor's tonic. And then one group, the 0:13:48.360 --> 0:13:52.760 control group was taking nothing was or was taking you know, 0:13:52.880 --> 0:13:56.600 just water or something to that effect something completely innate. Uh. 0:13:56.720 --> 0:14:00.680 And that would be that would be a a group 0:14:00.720 --> 0:14:04.000 that you would judge the results of the other categories by, right, 0:14:04.080 --> 0:14:06.520 And you would need to randomly sort the people into 0:14:06.559 --> 0:14:08.959 those groups. So it wasn't just that, you know, the 0:14:09.360 --> 0:14:12.640 only the people with real severe foot pain. We're taking 0:14:12.720 --> 0:14:15.760 the doctor field grades, because the more extreme their pain 0:14:15.840 --> 0:14:18.880 to begin with, probably the more likely they are to 0:14:19.160 --> 0:14:22.200 have that pain be lessened or go away over time, 0:14:22.360 --> 0:14:25.160 just naturally. Right. And uh. And I'm going to have 0:14:25.240 --> 0:14:27.560 a more specific example of this a little later in 0:14:27.600 --> 0:14:29.400 the podcast. So if you if you still don't get it, 0:14:29.520 --> 0:14:31.880 just hang on. We'll we'll have another example in a bit. 0:14:38.280 --> 0:14:40.520 I was looking at an article in the British Medical 0:14:40.600 --> 0:14:44.200 Journal from nine that was just a collection of different 0:14:44.240 --> 0:14:48.240 examples of regression to the mean in real life medical research. 0:14:48.360 --> 0:14:51.840 This was by j Martin Bland and Douglas J. Altman 0:14:52.480 --> 0:14:55.880 called statistics notes some examples of regression towards the mean, 0:14:56.560 --> 0:14:59.040 and they point out a very common type of example, 0:14:59.160 --> 0:15:01.120 so the this will be similar to what we just 0:15:01.160 --> 0:15:04.600 talked about. The author's right. In clinical practice, there are 0:15:04.680 --> 0:15:09.680 many measurements such as weight, serum cholesterol concentration, or blood pressure, 0:15:10.120 --> 0:15:13.840 for which particularly high or low values are signs of 0:15:14.000 --> 0:15:18.480 underlying disease or risk factors for disease. People with extreme 0:15:18.680 --> 0:15:21.800 values of the measurements, such as high blood pressure may 0:15:21.880 --> 0:15:24.440 be treated to bring their values closer to the mean. 0:15:25.000 --> 0:15:27.400 If they are measured again, we will observe that the 0:15:27.520 --> 0:15:30.120 mean of the extreme group is now closer to the 0:15:30.200 --> 0:15:33.280 mean of the whole population. That is, it is reduced. 0:15:33.800 --> 0:15:36.520 This should not be interpreted as showing the effect of 0:15:36.560 --> 0:15:39.760 the treatment. Even if subjects are not treated, the mean 0:15:39.840 --> 0:15:43.560 blood pressure will go down owing to regression towards the means. 0:15:43.600 --> 0:15:47.040 So again, something starts with an extreme value in certain 0:15:47.080 --> 0:15:49.600 types of cases, you would just expect it to have 0:15:49.720 --> 0:15:54.280 a less extreme value the next time due to random fluctuation. 0:15:55.280 --> 0:15:57.480 Uh So again, you know this could fill you with 0:15:57.600 --> 0:16:00.320 despair because you might wonder, well, then how could you 0:16:00.400 --> 0:16:03.320 ever know if a treatment was effective or not. But again, 0:16:03.400 --> 0:16:06.160 this is where the standard practices of science based medicine 0:16:06.200 --> 0:16:09.040 come to play. Instead of just taking people with some 0:16:09.200 --> 0:16:13.400 extreme measurement and giving them a treatment, you randomize them 0:16:13.520 --> 0:16:15.800 into test groups and control groups like we were just 0:16:15.880 --> 0:16:18.000 talking about. So if you have a large enough sample, 0:16:18.360 --> 0:16:21.520 you properly randomize the groups. People with the extreme starting 0:16:21.560 --> 0:16:24.840 conditions will somewhat regress toward the mean, but they will 0:16:24.880 --> 0:16:27.800 all regress toward the mean on average the same rate 0:16:27.920 --> 0:16:31.120 whether they're receiving a real potential treatment or they're in 0:16:31.160 --> 0:16:34.480 the placebo group. But if the treatment actually does something helpful, 0:16:34.680 --> 0:16:38.080 this effect will manifest as the difference between the two groups. 0:16:38.760 --> 0:16:42.320 So good scientific research good medical research has methods for 0:16:42.400 --> 0:16:45.360 excluding the effects of reversion to the mean on their findings. 0:16:45.480 --> 0:16:49.160 We have the tools, but we can still fall into 0:16:49.240 --> 0:16:53.080 the trap of regression to the mean fallacies, especially in 0:16:53.120 --> 0:16:56.040 our day to day lives drawing inferences the way that 0:16:56.240 --> 0:16:59.520 that the pilot and in common story did, or or 0:16:59.600 --> 0:17:03.280 even science if we're not careful and deliberate about designing experiments. 0:17:03.800 --> 0:17:07.120 And in addition to just a methodology design that has 0:17:07.520 --> 0:17:10.840 you know, randomized groups and control groups, there are also 0:17:10.920 --> 0:17:13.600 ways of trying to counteract regression to the mean, just 0:17:13.760 --> 0:17:17.119 through statistical methods that are maybe less reliable, But there 0:17:17.119 --> 0:17:20.800 are statistical methods people can use to try to apply 0:17:21.080 --> 0:17:24.960 sort of modifiers to data in order to estimate regression 0:17:25.040 --> 0:17:28.639 to the mean and UH and counteract its effects. So 0:17:28.680 --> 0:17:31.920 again we have tools within scientific research to to figure 0:17:31.960 --> 0:17:34.320 this out, and it's a lot of what science does 0:17:34.960 --> 0:17:37.639 is trying to sort out the difference between regression to 0:17:37.720 --> 0:17:41.159 the mean and actual effects of interventions. But in our 0:17:41.240 --> 0:17:43.560 day to day lives, we still fall for regression to 0:17:43.640 --> 0:17:46.640 the mean fallacies all the time. Yeah, and it's important 0:17:46.680 --> 0:17:49.240 to realize too that it's not just a situation where 0:17:49.320 --> 0:17:53.200 regression towards the mean could create an illusion of something 0:17:53.320 --> 0:17:55.600 working when it doesn't. Uh, you know, sometimes it can 0:17:55.680 --> 0:18:02.040 just potentially overstate um the effects something. For an example 0:18:02.119 --> 0:18:04.800 of that that I was looking at was that regression 0:18:04.840 --> 0:18:06.879 towards the mean or the failure to account for it 0:18:07.160 --> 0:18:11.040 can also overstate the effectiveness of something like traffic light cameras. 0:18:11.720 --> 0:18:15.360 Is it making a difference and cutting down on accidents perhaps, 0:18:15.840 --> 0:18:20.520 but any actual effectiveness could potentially be overstated by failure 0:18:20.600 --> 0:18:23.520 to account for just regression towards the mean. Oh yeah, 0:18:23.680 --> 0:18:26.480 So where do you tend to install things like that? 0:18:27.440 --> 0:18:30.520 High acts like problem areas? Right, So, if there's like 0:18:30.560 --> 0:18:33.800 a stretch of road that has a lot of problems 0:18:33.920 --> 0:18:36.640 on it, people really speeding a lot there or crashing 0:18:36.680 --> 0:18:39.760 a lot there, that might be where you stage the intervention. 0:18:40.160 --> 0:18:43.840 It's possible some things like that fluctuate naturally over time 0:18:43.920 --> 0:18:47.160 in different locations. Yeah, and you put the cameras in place, 0:18:47.280 --> 0:18:49.520 and it could have an effect, but maybe not as 0:18:49.600 --> 0:18:52.600 much of an effect as it looks like it is 0:18:52.680 --> 0:18:56.720 taking place. Again, if you don't factor regression towards the 0:18:56.760 --> 0:19:01.040 mean into the study. Right now, While our TM is 0:19:01.080 --> 0:19:04.040 a very important phenomenon to understand and take into account, 0:19:04.119 --> 0:19:07.960 it certainly doesn't apply to every sequence of values you 0:19:08.040 --> 0:19:11.359 could repeatedly sample, So you also have to be careful 0:19:11.480 --> 0:19:15.000 not to apply it in situations where it isn't warranted. 0:19:15.760 --> 0:19:17.920 I was you know, there are a million examples you 0:19:17.960 --> 0:19:20.000 could cite. One that came to my mind is the 0:19:20.160 --> 0:19:22.600 orbital decay of a satellite. Let's say you've got a 0:19:22.600 --> 0:19:26.280 communication satellite in lower orbit and you get a reading 0:19:26.320 --> 0:19:29.040 on its altitude and the reading is lower than the 0:19:29.119 --> 0:19:33.560 satellites average altitude. Uh. Now, you might say, hey, I 0:19:33.840 --> 0:19:36.040 think this means we need to program a reboost to 0:19:36.240 --> 0:19:38.920 insert it back into the orbit where it's supposed to be. 0:19:39.640 --> 0:19:43.360 And somebody could erroneously apply regression to the mean here 0:19:43.800 --> 0:19:45.639 and say, no, we don't need to do that. The 0:19:45.680 --> 0:19:49.000 satellite might just return to its average altitude. It doesn't 0:19:49.040 --> 0:19:51.920 apply in this scenario, even though you are taking repeated 0:19:52.000 --> 0:19:55.359 measurements of a value over time, because we know things 0:19:55.480 --> 0:19:59.800 about the physical characteristics determining the orbit of satellites and 0:20:00.000 --> 0:20:03.080 in lower thorbit uh and that due to factors like 0:20:03.160 --> 0:20:07.840 atmospheric drag, their altitude tends to trend steadily downward over 0:20:07.960 --> 0:20:11.880 time in a consistent direction down, down, down, So eventually 0:20:12.240 --> 0:20:14.440 you will need a reboost in order to put it 0:20:14.520 --> 0:20:17.879 back up to the correct distance. So regression to the 0:20:17.960 --> 0:20:21.520 mean applies to certain kinds of data that are repeatedly 0:20:21.640 --> 0:20:26.919 sampled data where there is natural random fluctuation back and forth, 0:20:27.440 --> 0:20:30.680 not a steady trend in the data in one direction 0:20:30.800 --> 0:20:33.800 on the relevant time scale. The other thing that's important 0:20:33.840 --> 0:20:36.760 to understand is that systems where you expect to find 0:20:36.880 --> 0:20:40.560 regression to the mean are systems in which the repeated 0:20:40.680 --> 0:20:44.040 data values you're sampling are to some degree determined by 0:20:44.280 --> 0:20:48.080 luck or chance. If a series of values is influenced 0:20:48.119 --> 0:20:52.919 almost entirely by deterministic influence, like in the satellite example, 0:20:53.000 --> 0:20:56.400 by like the laws of physics, or by some extremely 0:20:56.520 --> 0:21:01.120 reliable skill with little room for variation, values don't really 0:21:01.280 --> 0:21:03.640 regress towards the mean in the same way, because there's 0:21:03.680 --> 0:21:06.800 just less random fluctuation back and forth to begin with. 0:21:07.480 --> 0:21:10.680 The more chance and random variation plays a role in 0:21:10.720 --> 0:21:14.040 the outcome, the more you will tend to observe regression 0:21:14.080 --> 0:21:17.159 towards the mean after an extreme sample in in whatever 0:21:17.240 --> 0:21:20.560 it is you're looking at. Yeah, I've read that the 0:21:20.720 --> 0:21:23.520 progression towards the mean is is not to be confused 0:21:23.640 --> 0:21:26.800 with the law of large numbers. For example, Uh, this 0:21:27.000 --> 0:21:29.399 is the the law that that states as a sample 0:21:29.480 --> 0:21:32.639 size becomes larger, the sample mean gets closer to the 0:21:32.720 --> 0:21:36.440 expected value. So a coin flipping example is key here. 0:21:36.520 --> 0:21:39.040 Flip a coin and the random results are going to 0:21:39.160 --> 0:21:43.320 ultimately average out to a point five proportion. But if 0:21:43.359 --> 0:21:45.760 you only flip the coin ten times, you might not 0:21:45.920 --> 0:21:49.440 see this breakdown. Um. And this also applies to say, 0:21:49.480 --> 0:21:52.119 even odds on the rolling of a of a D 0:21:52.320 --> 0:21:55.680 six of a six sided die. Uh So for example, 0:21:56.200 --> 0:21:59.000 too regular people, that's just to die. That is nerves 0:21:59.080 --> 0:22:01.600 like us, it's a D six. Yeah, D six is 0:22:01.600 --> 0:22:02.919 what I could get my hands on because I was like, well, 0:22:02.920 --> 0:22:04.639 I'm gonna do an example. I'm gonna try it myself. 0:22:04.720 --> 0:22:06.680 So while I was putting together notes for this, I 0:22:06.720 --> 0:22:10.520 went ahead and rolled ten times, and I got even 0:22:10.600 --> 0:22:14.240 even odd even odd even even even even odd. So 0:22:14.440 --> 0:22:17.560 that's that's seven to three in favor of even. So 0:22:17.760 --> 0:22:19.919 it might make you wonder, well, is this die broken? 0:22:20.200 --> 0:22:22.639 Does this D six need to go away? Because it 0:22:22.720 --> 0:22:26.840 can't be trusted to roll? Uh? You know a balanced 0:22:27.480 --> 0:22:30.080 array of odd and even numbers. Well, no, that's not 0:22:30.200 --> 0:22:32.639 the case. Uh. And if I were to roll this, 0:22:32.920 --> 0:22:37.080 say another hundred times, another thousand times, I would see 0:22:37.119 --> 0:22:40.840