WEBVTT - Research Bias: Sort It Out, Science 0:00:01.480 --> 0:00:04.280 Welcome to Stuff You Should Know, a production of I 0:00:04.360 --> 0:00:13.640 Heart Radio. Hey, and welcome to the podcast. I'm Josh Clark, 0:00:13.720 --> 0:00:16.880 and there's Charles w Chuck, Bryan, and Jerry's here. Jerry's back. 0:00:16.920 --> 0:00:22.159 Everybody looking well rested and sun kissed and everything, and 0:00:22.280 --> 0:00:25.079 this is stuff you should know. She's like a beautiful, 0:00:25.640 --> 0:00:31.400 juicy orange. That's right. That's right, Chuck, that's a really 0:00:31.440 --> 0:00:34.159 apt description. Ready to be squaws. I wish I could 0:00:34.200 --> 0:00:38.040 squeeze her, but we're still not squeezing. Uh no, not 0:00:38.159 --> 0:00:43.680 in this pandemic. Are you crazy? You out of your mind? No, 0:00:43.960 --> 0:00:48.519 no squeezing. Um Yeah, even Robert, Robert Plant wouldn't let 0:00:48.560 --> 0:00:53.720 you anywhere near him. Okay, check figure out that joke. 0:00:53.880 --> 0:01:01.360 Robert the squeeze my my lemon, the lemon song, No 0:01:01.800 --> 0:01:05.480 until the juice runs down my leg. Yeah, that's the 0:01:05.560 --> 0:01:10.680 lemon song. Right. No, I don't think so. I think 0:01:10.680 --> 0:01:14.640 it is. I don't think it is, Okay, I think 0:01:14.680 --> 0:01:17.080 it's a I don't think it is the lemon song, man. 0:01:17.319 --> 0:01:18.800 I think it's a whole lot of love. It's whole 0:01:18.840 --> 0:01:22.520 lot of love, all right. It's maybe the dirtiest thing 0:01:22.520 --> 0:01:28.679 that was ever said in like a top ten song. Okay, regardless, 0:01:29.240 --> 0:01:31.680 I'll just let the emails take care of this, I 0:01:31.800 --> 0:01:36.840 really think. No, the Lemon song is ums I love 0:01:36.959 --> 0:01:40.800 those lemons, right, No, the Lemon song is all about 0:01:41.640 --> 0:01:45.480 how how how you have friends, like you want to 0:01:45.520 --> 0:01:50.280 have friends, and like friends are good to have. Okay, yeah, 0:01:50.560 --> 0:01:52.120 may be all wrong. No, I think it's a whole 0:01:52.160 --> 0:01:55.280 lot of love. Yeah it is. I'm a dcent sure buddy. 0:01:56.920 --> 0:01:58.960 All right, well I encourage you not to google the lyrics. 0:01:59.040 --> 0:02:02.160 Then well we um we could ask our good friend 0:02:02.240 --> 0:02:05.560 and um stuff. You should know writer Ed Grabmanowski the 0:02:05.600 --> 0:02:09.040 grab Stir. Look at that because he is in a 0:02:09.080 --> 0:02:11.480 band and has been for a while. We've mentioned it before, 0:02:11.600 --> 0:02:16.000 space Lord, which has just a super cool Zeppelin esque 0:02:16.120 --> 0:02:21.760 sound to them. Um and they just there and um 0:02:21.800 --> 0:02:24.840 they just released a new single, which you can find 0:02:24.880 --> 0:02:30.360 on band camp by searching space Lord. Not the space Lords. No, 0:02:30.760 --> 0:02:34.080 not Vinny in the space Lords. Yeah space Lord. Just 0:02:34.080 --> 0:02:38.239 look for space Lord with some cool uh graphics, and 0:02:38.320 --> 0:02:41.399 that'll you know that's Ed. You'll know it's the Grabster 0:02:41.840 --> 0:02:46.240 but uh yeah, good stuff. We also have a game 0:02:46.280 --> 0:02:48.720 out that Trivial Pursuit made. Yeah, we should plug our 0:02:48.760 --> 0:02:51.840 own stuff every now and then. We just did. Yes, 0:02:51.960 --> 0:02:55.240 it is a co branded game with Trivial Pursuit from Hasbro, 0:02:55.480 --> 0:02:59.680 and it is not a trivial pursuit game that you 0:02:59.720 --> 0:03:02.320 are used to. It is a stuff you should know 0:03:02.400 --> 0:03:07.560 game that Trivial Pursuit was happy to UH co brandwith. 0:03:07.639 --> 0:03:10.160 So just what, I don't want his emails that are 0:03:10.160 --> 0:03:12.320 like this, this this is a Trivial Pursuit. This is some 0:03:12.639 --> 0:03:16.040 other different game. You're always worried about the emails, aren't you. 0:03:16.800 --> 0:03:19.639 Don't you just ignore them? Let him roll off, roll 0:03:19.680 --> 0:03:23.040 off my back, Like, I'm disappointed in you guys for this. 0:03:23.760 --> 0:03:26.080 I haven't even listened to the episode, but I'm disappointed 0:03:26.080 --> 0:03:27.600 about that. I just got one of those. Did you 0:03:27.639 --> 0:03:30.720 see that It just rolls off your back? Yeah, yeah, 0:03:30.760 --> 0:03:33.040 those are those are always great. I didn't listen, But 0:03:34.960 --> 0:03:37.600 here's what was wrong. I read that person back. Actually, 0:03:37.600 --> 0:03:40.520 I was like, we actually kind of did exactly what 0:03:40.560 --> 0:03:42.360 you hoped we would do. And they're like, oh, sorry 0:03:42.400 --> 0:03:48.720 for being presumptuous anyway, Oh all is forgiven. So, um, 0:03:49.320 --> 0:03:53.440 we're talking today about bias Chuck, and I want to 0:03:53.480 --> 0:03:57.840 set the scene a little bit because you know, um, 0:03:57.880 --> 0:04:00.000 one of the things that I'm always like harping up 0:04:00.000 --> 0:04:04.040 about is like the death of expertise, right, And it's 0:04:04.040 --> 0:04:07.480 a real problem, like this idea that science can't be trusted, 0:04:07.560 --> 0:04:10.720 that people who go and spend a decade or more 0:04:10.800 --> 0:04:14.280 learning about a specific thing that they go out and 0:04:14.320 --> 0:04:18.480 become an expert in or that's their profession, that's their training, 0:04:19.040 --> 0:04:21.880 um that those people what they have to say is 0:04:22.640 --> 0:04:26.640 basically meaningless, or that it's it's no better than somebody 0:04:26.680 --> 0:04:29.840 on the internet's opinion about so that specific subject that 0:04:29.839 --> 0:04:32.160 that person spent ten or twelve years being trained to 0:04:32.520 --> 0:04:34.760 be an expert in. Like that kind of stuff to 0:04:34.800 --> 0:04:38.000 me is like super dangerous, Like it it's it's in 0:04:38.080 --> 0:04:40.480 a there's an erosion of something, and it's a an 0:04:40.480 --> 0:04:43.760 erosion of intelligence to start with, but it's also an 0:04:43.760 --> 0:04:48.400 erosion of just believing in facts and knowing that you're 0:04:48.400 --> 0:04:51.960 not being taken for a ride or hustled. And is 0:04:51.960 --> 0:04:55.599 it a huge enormous problem that we're just beginning to 0:04:55.600 --> 0:04:57.680 wake up too and is still unfolding. It's not like 0:04:57.760 --> 0:05:00.760 it happened and now we're like real from it. It's 0:05:00.800 --> 0:05:04.400 still happening in real time, and it is a massive, 0:05:05.240 --> 0:05:09.440 huge issue one of the biggest issues that that humanity faces, 0:05:09.480 --> 0:05:14.320 I think because it encompasses so many other large issues 0:05:14.360 --> 0:05:20.039 like climate change, existential risks, the pandemic, um politics, all 0:05:20.080 --> 0:05:23.359 of them kind of fall under this this erosion of 0:05:23.480 --> 0:05:26.800 belief and facts and that there are people out there 0:05:26.839 --> 0:05:30.680 who know more than you do. Um, it's a big problem. Yeah, 0:05:30.960 --> 0:05:35.000 imagine being someone who studied and researched something intensely for 0:05:35.040 --> 0:05:40.120 ten or fifteen years, uh that with when presenting facts 0:05:40.120 --> 0:05:43.880 to be met with. I don't know about that. That's 0:05:43.880 --> 0:05:47.040 a response I hear a lot in the South. Yeah, 0:05:47.240 --> 0:05:50.120 or that they saw something on YouTube that flatly contradicts that, 0:05:50.160 --> 0:05:53.479 and it's like that it doesn't matter what you just said. 0:05:53.600 --> 0:05:57.960 Is ridiculous that somebody posted something on YouTube and that 0:05:57.960 --> 0:06:01.479 that like that has as much weight is what somebody 0:06:01.520 --> 0:06:04.240 who spent ten or twelve year studying this very thing 0:06:04.680 --> 0:06:07.320 has to say about it, like knows exactly what they're 0:06:07.320 --> 0:06:11.520 talking about, has to say about it. It's it's maddening. Yeah, 0:06:11.520 --> 0:06:14.720 there's there's something about people from the South in general, 0:06:14.720 --> 0:06:17.560 and I think that are in this group that I 0:06:17.600 --> 0:06:20.599 have literally heard that response from a lot of different 0:06:20.600 --> 0:06:22.159 people when I've been like oh no, no no, no, here 0:06:22.160 --> 0:06:26.040 the facts actually, and then when presented with something that 0:06:26.080 --> 0:06:28.360 they can't refute, they say, I don't know about that, 0:06:29.279 --> 0:06:31.240 and like that's it. That's the end of the conversation. 0:06:31.400 --> 0:06:33.719 That's different than the people I've encountered. The people I 0:06:33.839 --> 0:06:36.719 encountered like their brow furrows and they start pointing fingers 0:06:36.720 --> 0:06:39.280 and their their tone goes up, like you are you 0:06:39.320 --> 0:06:42.520 hanging out at the country club or something. I think 0:06:42.520 --> 0:06:45.880 it's different types of people that, you know, there's ignorance, 0:06:45.920 --> 0:06:48.960 and then there's also people that actually think they're better 0:06:49.040 --> 0:06:52.520 informed that will fire back with YouTube clips. Right. So 0:06:53.320 --> 0:06:56.479 the reason I brought that up is because and one 0:06:56.480 --> 0:06:59.040 of the reasons that that is being allowed to exist, 0:06:59.040 --> 0:07:01.680 that that does exist, I think it's a it's a 0:07:01.760 --> 0:07:05.640 reaction to something else that's going on simultaneously, which is 0:07:07.040 --> 0:07:09.320 there are a lot of experts out there who are 0:07:09.320 --> 0:07:17.160 performing really sloppy science, sometimes outright fraudulent science, and they're 0:07:18.000 --> 0:07:23.680 frittering away whatever faith the general public or society has 0:07:23.760 --> 0:07:26.240 in their expertise and in their profession. And there are 0:07:26.480 --> 0:07:29.240 a ton of scientists out there. I would say the 0:07:29.360 --> 0:07:35.120 vast majority, by far of scientists are legitimate upstanding, upright 0:07:35.600 --> 0:07:39.520 dedicants to science, right, That's where they that's where they 0:07:39.600 --> 0:07:41.760 place there, that's where they hang their hat, that's where 0:07:41.760 --> 0:07:44.840 their heart is, that's that's what they believe in, and 0:07:44.880 --> 0:07:48.320 that's what they work to support. But science has like 0:07:48.400 --> 0:07:52.480 kind of a problem, Chuck in that it's allowing way 0:07:52.520 --> 0:07:56.280 too much for bias, which is what we're gonna talk about, 0:07:56.560 --> 0:08:01.000 to creep into science, um and your mind science and 0:08:01.080 --> 0:08:05.120 basically produce papers that are just useless and trash. And 0:08:05.160 --> 0:08:07.560 there's a whole lot of reasons for it, but it's 0:08:07.560 --> 0:08:10.760 a it's something that needs to be addressed if we're 0:08:10.760 --> 0:08:13.000 ever going to get back on a footing with a 0:08:13.120 --> 0:08:16.600 faith in experts and expertise and just facts that there 0:08:16.640 --> 0:08:19.480 are such things as objective facts. Yeah, I mean a 0:08:19.520 --> 0:08:23.840 lot of times it's financially related, whether it's a lack 0:08:23.880 --> 0:08:27.080 of funding, a desire for more funding, a desire just 0:08:27.120 --> 0:08:31.160 to keep your your uh, your lab running and people 0:08:31.240 --> 0:08:35.120 paid on staff. Which you know, all this stuff is understandable. 0:08:35.400 --> 0:08:38.560 You want to keep doing this work, but you can't 0:08:38.640 --> 0:08:41.160 let that get in the way. It's like it's like 0:08:41.200 --> 0:08:45.280 in Rushmore at the end when Margaret Yang faked the 0:08:45.280 --> 0:08:49.000 results of that science experiment because she didn't want it 0:08:49.040 --> 0:08:52.760 to be wrong. You know, I don't remember what um, 0:08:52.880 --> 0:08:55.600 I don't remember that part was that like a deleted scene? No? No, no, 0:08:55.720 --> 0:08:57.800 was it in the end when they meet up and 0:08:57.840 --> 0:09:00.320 he's flying the Uh. I think he's flying the kite 0:09:00.360 --> 0:09:04.120 with Dirk and she's talking about her science fair project 0:09:04.200 --> 0:09:05.880 and he was really impressed with it, and she was like, 0:09:06.320 --> 0:09:09.400 I fake the results. And the reason why was because 0:09:09.480 --> 0:09:12.480 because she didn't want to be wrong. Uh. And I 0:09:12.480 --> 0:09:14.960 think a lot of times people will get into a 0:09:15.000 --> 0:09:19.640 certain body of research or data too because they want 0:09:19.720 --> 0:09:22.640 to prove a certain thing and if they can't, it 0:09:22.720 --> 0:09:24.360 might be really hard to live with that. So that 0:09:24.400 --> 0:09:28.800 weighs into it. Uh, money for personal gain, uh, advancing 0:09:28.800 --> 0:09:31.920 your career, you know, publisher, parish, that whole thing. Like, 0:09:31.920 --> 0:09:33.880 we're gonna talk about all this, but there are a 0:09:33.960 --> 0:09:36.640 lot of reasons that it's been allowed to creep in, 0:09:37.160 --> 0:09:39.920 But all of it is at the disservice of their 0:09:40.480 --> 0:09:43.280 the fundamentals of what they base their careers on to 0:09:43.320 --> 0:09:46.400 begin with. Yeah, it's at the it's at the disservice 0:09:46.440 --> 0:09:49.679 of science itself, right, because the whole point of science 0:09:49.679 --> 0:09:53.720 and then the scientific publishing, the whole publishing industry, um 0:09:53.840 --> 0:09:58.120 is to to basically create a hypothesis, test your hypothesis, 0:09:58.160 --> 0:09:59.800 and then share the results with the world. And that's 0:09:59.840 --> 0:10:02.880 I deally what would happen because you're building this body 0:10:02.920 --> 0:10:07.120 of scientific knowledge. But money and corporate interests and academic 0:10:07.160 --> 0:10:11.079 publishing have all kind of come in and taken control 0:10:11.160 --> 0:10:14.280 of this whole thing, and as a result, a lot 0:10:14.320 --> 0:10:17.240 of the stuff that gets published are trash papers that 0:10:17.240 --> 0:10:20.160 shouldn't be published. A lot of the really good papers 0:10:20.200 --> 0:10:24.520 that don't come up with sexy results don't get published. 0:10:25.040 --> 0:10:29.440 And then, like you said, um, people using science for 0:10:29.559 --> 0:10:36.520 personal gain. There are a very small chadra of thoroughly 0:10:36.640 --> 0:10:41.160 evil people who are willing to use their scientific credentials 0:10:41.640 --> 0:10:45.319 to create doubt in the general public, to to prevent 0:10:46.520 --> 0:10:51.320 like people from understanding that climate change was real for 0:10:51.480 --> 0:10:56.479 twenty years, or um, that fossil fuels actually do contribute 0:10:56.480 --> 0:11:00.880 to to anthropogenic climate change. But what remains focusing on 0:11:01.080 --> 0:11:05.520 is like bias in the in the sense that people 0:11:05.559 --> 0:11:11.120 carrying out studies are human beings, and human beings are flawed. 0:11:11.200 --> 0:11:14.400 We're just flawed, and we bring those flaws to our studies, 0:11:14.880 --> 0:11:19.120 and that you really have to work hard at rooting 0:11:19.160 --> 0:11:22.679 those flaws and those biases out to produce a really good, 0:11:23.120 --> 0:11:27.600 thorough scientific study with good reliable results that can be 0:11:27.640 --> 0:11:31.960 reproduced by anybody using the same methods. Um and that 0:11:32.679 --> 0:11:35.120 science is just starting to wake up to the idea 0:11:35.200 --> 0:11:38.040 that it is really biased and it needs to take 0:11:38.080 --> 0:11:41.240 these into account in order to to progress forward from 0:11:41.240 --> 0:11:44.000 the point that it's at right now, which is tenuous, 0:11:44.200 --> 0:11:48.360 I think, perhaps more tenuous than ever the point that 0:11:48.400 --> 0:11:52.000 science is at. I think. So science isn't going away. 0:11:52.000 --> 0:11:54.040 It's not going anywhere. It's probably the greatest of course 0:11:54.360 --> 0:11:57.480 humans have ever come up with. Right, It's not going anywhere, 0:11:57.480 --> 0:12:00.200 But it is a terrible position that it's in, and 0:12:00.240 --> 0:12:04.959 it's going to take some genuine leadership in the scientific 0:12:04.960 --> 0:12:07.120 community from a bunch of different quarters in a bunch 0:12:07.160 --> 0:12:09.600 of different fields to basically step up and be like, guys, 0:12:09.600 --> 0:12:13.160 this is really bad and we need to change it now, 0:12:13.440 --> 0:12:15.160 and a lot of people need to be called out. 0:12:15.160 --> 0:12:19.559 In science typically shies away from naming names and calling 0:12:19.559 --> 0:12:24.520 out by name fraudulent scientists because scientists seem to like 0:12:24.640 --> 0:12:28.080 to um suppose the best in people, which is not 0:12:28.200 --> 0:12:31.079 always the case, right, And having said all of this, 0:12:31.400 --> 0:12:34.520 there could we could root out every bias and and 0:12:34.520 --> 0:12:40.040 and really clean up the scientific publishing community. And there's 0:12:40.080 --> 0:12:42.400 still a set a certain set of people in this 0:12:42.480 --> 0:12:44.840 country and in the world who that wouldn't matter to 0:12:45.200 --> 0:12:49.280 and would still shut down facts and because it doesn't 0:12:49.320 --> 0:12:52.800 fit their narrative, so for sure, But the people have 0:12:52.960 --> 0:12:56.480 always they've always been there, right, and they're always going 0:12:56.520 --> 0:12:59.160 to be there. There's always it's just countrarians that they 0:12:59.240 --> 0:13:01.760 are you can call, and free thinkers you can call them, stubborn, 0:13:01.880 --> 0:13:07.560 you can call purposefully, purposefully ignorant who knows they're always 0:13:07.600 --> 0:13:11.760 going to exist. The problem that this crisis that science 0:13:11.800 --> 0:13:15.360 finds itself in right now is that it's allowed that 0:13:15.360 --> 0:13:18.120 that population to grow and grow, and like people who 0:13:18.160 --> 0:13:20.960 otherwise didn't never really question science have been allowed to 0:13:21.040 --> 0:13:24.360 kind of trickle into that fold. And that those are 0:13:24.360 --> 0:13:26.400 the people that we should be worried about, the ones 0:13:26.440 --> 0:13:30.600 who would would know better if they believed in science again, right, 0:13:31.240 --> 0:13:33.520 And our way into this is to talk about different 0:13:33.559 --> 0:13:36.840 kinds of biases in true stuff. You should know fashion 0:13:36.920 --> 0:13:39.240 a top ten that is not a top ten. That's 0:13:39.280 --> 0:13:41.880 exactly right. We ate into at least three in this 0:13:42.000 --> 0:13:44.880 intro and hopefully shining a light on some of this stuff. 0:13:45.240 --> 0:13:49.040 People at least be more aware of different biases. And uh, well, yeah, 0:13:49.080 --> 0:13:52.040 you know. The first one is is good old confirmation bias. 0:13:52.320 --> 0:13:55.560 I mean, these aren't ranked because confirmation bias would probably 0:13:55.559 --> 0:13:59.200 be number one as far as people's awareness of it. 0:13:59.280 --> 0:14:03.760 But there are different examples, um that people use for 0:14:03.840 --> 0:14:06.760 confirmation bias. And I kind of enjoyed the one from 0:14:06.760 --> 0:14:09.199 the house Stuff Works article even though it was from 0:14:09.240 --> 0:14:13.640 N three. After X rays were discovered in Germany, there 0:14:13.760 --> 0:14:18.760 was a French scientist named Renee Blond Blonde Lot. Yeah, 0:14:18.880 --> 0:14:20.880 he looked at as the X rays he said, wow, 0:14:21.720 --> 0:14:25.240 Well who said, ay, I see N rays I've discovered 0:14:25.280 --> 0:14:28.560 in rays And everyone's like, what's an N ray? He said, well, 0:14:28.600 --> 0:14:32.320 it's like a corona when electricity discharges from a crystal 0:14:32.320 --> 0:14:34.800 and you can only see it in your peripheral vision. 0:14:35.760 --> 0:14:40.000 And American Robert Wood laid the wood and said, I'm 0:14:40.000 --> 0:14:43.400 gonna come to your lab and check this out and 0:14:43.600 --> 0:14:47.120 secretly remove the crystals during one of the experiments, and 0:14:48.360 --> 0:14:52.480 Blonde Lot still saw these N rays and so that's 0:14:52.520 --> 0:14:54.920 confirmation bias. He wanted to see those in rays. And 0:14:54.960 --> 0:14:58.520 then later, even though it was disproved, other French scientists 0:14:58.960 --> 0:15:02.680 supposedly published papers or published papers based on that research 0:15:02.920 --> 0:15:05.360 because they wanted it to be true. So that's what 0:15:05.440 --> 0:15:09.880 confirmation biases is when you're starting out with a hypothesis 0:15:10.440 --> 0:15:14.040 that is going to shape the methodology methodology of your 0:15:14.080 --> 0:15:17.280 study to to confirm it right. And then it can 0:15:17.320 --> 0:15:23.440 also occur where you're um. You you're interpreting info um 0:15:23.480 --> 0:15:26.680 to fit your hypothesis, so you're seeking out stuff that 0:15:26.760 --> 0:15:29.640 supports your hypothesis, and then the stuff that you is 0:15:29.800 --> 0:15:31.480 that's just there in front of you, the results, they 0:15:31.480 --> 0:15:33.400 are there in front of your like, ah, this thing 0:15:33.520 --> 0:15:37.640 proves that those end rays actually exist, um or this 0:15:37.760 --> 0:15:41.240 phenomenon cannot be due to anything but end raise. Therefore 0:15:41.400 --> 0:15:43.800 end Raise exists all of its confirmation bias. And that, 0:15:43.960 --> 0:15:46.360 like you said, that's number one, because that's not just 0:15:46.400 --> 0:15:49.640 a scientific bias. I mean, like that is that's like 0:15:50.400 --> 0:15:54.840 every human uses confirmation bias, and that it's two fold. 0:15:54.840 --> 0:15:59.640 We we avoid contradictory information because we UM I don't 0:15:59.680 --> 0:16:03.120 like to be wrong, and we find information that confirms 0:16:03.120 --> 0:16:04.760 our point of view because we like to be right. 0:16:04.800 --> 0:16:10.160 That's it's confirmation bias, and it's everywhere among everyone. That's right, 0:16:10.480 --> 0:16:13.200 Although I will say I know it happens a lot politically, 0:16:14.440 --> 0:16:19.480 but myself and the people that I congregate with, uh 0:16:19.800 --> 0:16:23.920 question their own leaders as much as they do leaders 0:16:23.960 --> 0:16:27.360 from the other parties. Oh, it's good, it's very good 0:16:27.360 --> 0:16:29.760 to do. And I don't know, there shouldn't be sacred 0:16:29.800 --> 0:16:32.480 calves and politics. That's a bad jam, well know. And 0:16:32.520 --> 0:16:35.800 it's like I've always being been like at the forefront 0:16:35.840 --> 0:16:40.200 of calling out my own parties wrongs and saying no, no, no, 0:16:40.360 --> 0:16:43.080 that's you need to do better than that, whereas I 0:16:43.120 --> 0:16:46.320 see a lot of other people in other situations truly 0:16:46.360 --> 0:16:50.520 bury and ignore those things because they don't you know, 0:16:50.560 --> 0:16:53.240 I just don't want to face that. Yeah. And it's 0:16:53.240 --> 0:16:55.600 not like it's not even like I don't want to 0:16:55.640 --> 0:16:59.920 face it. It just doesn't fit their worldview, so they 0:17:00.120 --> 0:17:02.680 just don't include it. It It just gets tossed out. But 0:17:02.720 --> 0:17:09.520 the point is is like it's not an active process necessarily, right, 0:17:10.080 --> 0:17:12.560 I think we should probably check our first break. I 0:17:12.600 --> 0:17:14.239 think so too. Chi All right, we'll be right back 0:17:14.240 --> 0:17:38.120 and talk about sampling bias right after this alright, Chuck, 0:17:38.160 --> 0:17:41.640 we're back, and we're coming back with UM, something called 0:17:41.720 --> 0:17:45.280 sampling bias, which is it turns out a sub type 0:17:45.280 --> 0:17:48.520 of a larger thing called selection bias, and one other 0:17:48.560 --> 0:17:50.399 thing we should say. We kind of got into it 0:17:50.440 --> 0:17:53.439 before I could say this. There are different stages in 0:17:53.840 --> 0:17:57.560 a study where bias can occur. It can happen in 0:17:57.640 --> 0:18:02.359 like the planning the pre study phase, UM and and uh, 0:18:02.359 --> 0:18:05.600 it can happen during the actual study, and then it 0:18:05.640 --> 0:18:08.199 can happen after the study as well. And so when 0:18:08.200 --> 0:18:12.320 we're talking about any kind of selection bias, including sampling bias, 0:18:12.359 --> 0:18:16.600 this is pre study bias where when you're actually setting 0:18:16.680 --> 0:18:19.200 up the study, this bias is where this is where 0:18:19.200 --> 0:18:20.720 the bias is going. Yeah, and you know what, I 0:18:20.800 --> 0:18:23.439 think it also bears saying that biases you have to 0:18:23.480 --> 0:18:26.600 work really hard to avoid it because it's it's almost 0:18:26.680 --> 0:18:30.520 like a disease that's always trying to get involved. And 0:18:30.560 --> 0:18:34.440 it's not like just do better, everybody and quit being biased. 0:18:34.520 --> 0:18:36.720 It's like it's way more complicated than that, because it 0:18:36.800 --> 0:18:39.320 is always knocking at the door, like you said, in 0:18:39.359 --> 0:18:42.199 all three phases, trying to sneak in there, and it 0:18:42.240 --> 0:18:44.760 takes a lot of work in all three phases to 0:18:44.800 --> 0:18:47.600 avoid it. So it's not as I don't want it 0:18:47.600 --> 0:18:49.879 to come across this as easy as us just saying 0:18:49.920 --> 0:18:54.000 like you shouldn't do that, stop it. No, But the 0:18:54.080 --> 0:18:56.879 first step is to recognizing that there's a lot of 0:18:56.880 --> 0:19:00.359 bias and different kinds of bias that are just sitting 0:19:00.359 --> 0:19:03.320 there waiting for a scientist. And then if you start 0:19:03.320 --> 0:19:05.359 admitting that it's there, you can start being on the 0:19:05.359 --> 0:19:07.280 lookout for it, and you can start adjusting for it, 0:19:07.440 --> 0:19:10.200 and then other people who read your papers or here 0:19:10.240 --> 0:19:12.280 you know, read news articles about your papers, can be 0:19:12.520 --> 0:19:16.240 on the lookout for that kind of thing. Yeah, so exactly, Uh, 0:19:16.359 --> 0:19:19.920 sampling bias is your you know, your sample set not 0:19:20.080 --> 0:19:23.080 being accurate and a good representation of the whole. A 0:19:23.119 --> 0:19:28.360 lot of times you'll find this um in either studies 0:19:28.359 --> 0:19:31.439 that are really small scale because you don't have a 0:19:31.520 --> 0:19:34.240 large sample and you don't have the kind of money 0:19:34.280 --> 0:19:36.480 like near you. Like maybe you work for university, so 0:19:36.560 --> 0:19:39.600 you work with university students as your first sample set, 0:19:39.600 --> 0:19:43.480 who are not indicative of anything, but you know, people 0:19:43.640 --> 0:19:47.080 eighteen to twenty one years old or so. Now, remember 0:19:47.119 --> 0:19:52.040 we talked about weird Western educated and realized rich and democrats. 0:19:52.359 --> 0:19:54.600 That's exactly the thing. It's like, I mean, it's a 0:19:54.840 --> 0:19:57.359 it's a decent place to start if you don't have 0:19:57.720 --> 0:19:59.600 much money and you want to get the ball rolling. 0:20:00.080 --> 0:20:02.159 It's not like, oh, you shouldn't do university studies at 0:20:02.200 --> 0:20:07.080 all and using students, but those findings definitely don't represent 0:20:07.119 --> 0:20:10.680 the wide nation and it needs to grow and get 0:20:10.680 --> 0:20:13.040 more funding if you want to actually have a legitimate 0:20:13.080 --> 0:20:17.439 claim to something. Another way that sampling bias can come 0:20:17.480 --> 0:20:20.760 up is from like the group that you're recruiting from. 0:20:20.760 --> 0:20:25.400 Like if you're doing a strictly online survey, but you're 0:20:25.440 --> 0:20:29.160 trying to apply your findings to the wider society, that's 0:20:29.200 --> 0:20:32.000 just not gonna happen because there's so many people who 0:20:32.080 --> 0:20:35.760 aren't Internet savvy enough to take an Internet survey. Like, 0:20:35.880 --> 0:20:38.480 by by nature, you are a little savvier than the 0:20:38.520 --> 0:20:40.879 average person if you're hanging out on the Internet and 0:20:40.920 --> 0:20:44.439 taking a survey. And then also kind of tangential that 0:20:44.480 --> 0:20:47.280 I like to tell myself that at least um and 0:20:47.320 --> 0:20:50.600 then tangential that is something called self selection bias, which 0:20:50.640 --> 0:20:53.640 is where the people who say, let's say you're doing 0:20:53.680 --> 0:20:58.480 a study on wellness and you know what eating tuna 0:20:58.640 --> 0:21:02.720 can do for your health. Um, people who are interested 0:21:02.760 --> 0:21:05.640 in wellness and health are going to be much more 0:21:05.720 --> 0:21:09.240 likely to volunteer for that study than people who couldn't 0:21:09.280 --> 0:21:11.800 care less about health and have no desire whatsoever to 0:21:11.880 --> 0:21:15.000 further sciences understanding of what makes you healthy. So you 0:21:15.040 --> 0:21:18.080 would have to go out and find those people and 0:21:18.119 --> 0:21:20.919 recruit them rather than just relying on the people who 0:21:21.080 --> 0:21:23.439 volunteered based on the flyer you put up in the 0:21:23.480 --> 0:21:28.680 student right, or you know, study all financial demographics or 0:21:28.800 --> 0:21:33.240 pull all financial demographics rather than just and you know, 0:21:33.280 --> 0:21:36.439 sometimes it's a methodology in which they do try and 0:21:36.520 --> 0:21:40.199 recruit people steers them in that direction unknowingly that I know. 0:21:40.320 --> 0:21:45.120 In the article they've talked about the presidential campaign with 0:21:45.440 --> 0:21:49.720 Roosevelt and alf Landon Republican alf Landon, they were doing 0:21:49.760 --> 0:21:54.240 polling with like country clubs, rosters and uh, people who 0:21:54.320 --> 0:21:56.199 drove cars and stuff. At the time that was kind 0:21:56.200 --> 0:21:58.600 of a luxury, so it was all out of whack. 0:21:58.640 --> 0:22:01.680 Everyone's like Landing's gonna win in a landslide. It's because 0:22:01.760 --> 0:22:06.399 you just kind of basically stuck your polling to uh, 0:22:06.480 --> 0:22:08.400 you know, I don't know about wealthy individuals, but people 0:22:08.400 --> 0:22:10.800 who are a little more well off. And I think, um, 0:22:10.880 --> 0:22:13.960 we talked about that in our polling episode, that is 0:22:14.040 --> 0:22:18.200 that fiasco with polling. I also saw one more two 0:22:18.200 --> 0:22:19.600 that I want to mention because that has a really 0:22:19.600 --> 0:22:23.600 great anecdote attached to It's called survivorship bias, where when 0:22:23.600 --> 0:22:28.120 you're studying something, say like business or something, you're you're 0:22:28.160 --> 0:22:30.880 probably going to just be looking at the extant businesses, 0:22:30.880 --> 0:22:33.720 the businesses that have survived twenty years, thirty years, fifty 0:22:33.800 --> 0:22:35.760 years or something like that, and you're not taking into 0:22:35.960 --> 0:22:38.399 account all of the failures. So when you put together 0:22:38.480 --> 0:22:41.520 like a prognosis for business in America, it might have 0:22:41.560 --> 0:22:44.120 a son your outlook than it should, because all you're 0:22:44.119 --> 0:22:46.959 looking at are the ones that managed to survive and thrive. 0:22:47.400 --> 0:22:50.320 And that's survivorship bias. And did you see that The 0:22:50.400 --> 0:22:53.119 anecdote about the World War Two fighter pilots. It was 0:22:53.160 --> 0:22:57.640 actually pretty funny because they studied uh planes that had 0:22:57.680 --> 0:23:01.000 been returned, that had been fired upon managed to get 0:23:01.000 --> 0:23:03.520 back safely, and they're like, well, let's look at all 0:23:03.520 --> 0:23:06.440 these different bullet holes and where this plane was hit, 0:23:07.040 --> 0:23:09.840 and let's beef up all those areas on the body, 0:23:10.040 --> 0:23:14.760 and a mathematician named Abraham Wald said, uh, no, those 0:23:14.760 --> 0:23:17.639 are the places where they got shot and did Okay, 0:23:17.680 --> 0:23:19.919 what you should really do is find these planes that 0:23:19.960 --> 0:23:23.000 actually went down and beef up those sections of the planet. 0:23:23.840 --> 0:23:26.200 And that survivorship bias. It's just it's failing to take 0:23:26.240 --> 0:23:28.719 into account the failures and that have to do with 0:23:28.760 --> 0:23:34.399 what you're trying to study. What about channeling bias. Channeling 0:23:34.400 --> 0:23:37.200 bias is another kind of selection bias. Did you get 0:23:37.200 --> 0:23:40.320 this one? It wasn't the best example of channeling bias. Yeah, 0:23:40.359 --> 0:23:45.320 I mean that's I got it, all right? Did you 0:23:45.320 --> 0:23:47.720 not get it? I got it, but it took a 0:23:47.720 --> 0:23:50.160 lot of work before I finally did. Well. It's basically 0:23:50.160 --> 0:23:53.720 when let's say you have a patient and they're like, 0:23:54.600 --> 0:23:58.399