WEBVTT - How Critical Thinking Works 0:00:04.120 --> 0:00:07.200 Get in tech with technology with tech Stuff from half 0:00:07.200 --> 0:00:13.480 stuff works dot com. Hey there, and welcome to tech 0:00:13.520 --> 0:00:17.640 Stuffs one thousand episode. I am your host, who has 0:00:17.640 --> 0:00:24.080 been on every single of the thousand one thousand. Hi. 0:00:24.160 --> 0:00:27.520 I'm Jonathan Strickland. I'm an executive producer at how Stuff Works. 0:00:27.760 --> 0:00:34.199 I've done a thousand of these, one one thousand of these? 0:00:34.840 --> 0:00:39.040 How crazy is that? And I debated for a while 0:00:39.280 --> 0:00:42.360 on what I should cover for the one thousand episode 0:00:42.440 --> 0:00:45.519 of text Stuff because I've already covered a lot of 0:00:45.560 --> 0:00:47.960 my favorite tech companies and a lot of my favorite 0:00:48.000 --> 0:00:52.280 tech topics, whether it's a specific technology or a person, 0:00:52.920 --> 0:00:55.120 A lot of my favorites are ones that I've already 0:00:55.160 --> 0:01:00.000 focused on in the past. Plus, choosing one company or 0:01:00.080 --> 0:01:03.160 technology or person over all the others seems like a 0:01:03.160 --> 0:01:08.600 pretty difficult decision. How could I say that one company 0:01:08.720 --> 0:01:12.039 or one technology deserved it more than all the others. 0:01:12.920 --> 0:01:15.960 And besides, assigning significance to the number one thousand is 0:01:16.000 --> 0:01:19.160 sort of an arbitrary concept anyway when you think about it. 0:01:19.240 --> 0:01:22.280 So ultimately, I decided to skip all that and cover 0:01:22.360 --> 0:01:25.640 something I think is important that relates to everything in 0:01:25.680 --> 0:01:29.399 our lives, not just tech, but also tech, and that 0:01:29.440 --> 0:01:33.560 would be the concepts of skepticism and critical thinking. These 0:01:33.600 --> 0:01:37.920 are incredibly important tools we should familiarize ourselves with and 0:01:38.120 --> 0:01:43.480 use properly and frequently. First, let's define the terms, and 0:01:43.520 --> 0:01:45.679 I'm going to start with skepticism because I want to 0:01:45.680 --> 0:01:48.760 be clear about what I mean when I use the term, 0:01:48.800 --> 0:01:54.560 as opposed to other usages, including common usages of skepticism. Now, 0:01:54.600 --> 0:01:59.240 philosophically speaking, skepticism refers to the belief that true knowledge 0:01:59.440 --> 0:02:03.760 is difficult and likely impossible for humans to achieve, and 0:02:03.760 --> 0:02:06.480 I'm pretty much on board with that. I think it's 0:02:06.480 --> 0:02:10.400 impossible for us to know everything. Thousands of years ago, 0:02:10.919 --> 0:02:14.680 we had to rely almost solely on our own senses 0:02:14.760 --> 0:02:18.480 to figure out how the world works. But our senses 0:02:18.680 --> 0:02:22.600 are limited. Even the most keenly cited human can't see 0:02:22.600 --> 0:02:25.639 as well at a distance as a falcon, for example, 0:02:26.360 --> 0:02:29.680 and we aren't able to see light in the ultra 0:02:29.760 --> 0:02:34.560 violet or infrared ranges. We're incapable of directly perceiving electro 0:02:34.639 --> 0:02:38.840 magnetic radiation outside of the visible spectrum. The same is 0:02:38.840 --> 0:02:41.720 true for our other senses. That we have a limit 0:02:42.160 --> 0:02:45.560 uh and what we can perceive sounds below twenty hurts 0:02:45.639 --> 0:02:49.440 or above twenty kilo hurts are essentially beyond the range 0:02:49.440 --> 0:02:52.799 of human hearing. Our sense of smell is nowhere near 0:02:52.919 --> 0:02:55.920 sensitive as dogs, so we are limited by the extent 0:02:56.000 --> 0:02:59.040 to which our senses can take in information around us. 0:02:59.320 --> 0:03:02.040 There's a lot of information out there that we are 0:03:02.200 --> 0:03:05.840 incapable of sensing. Now, as time is past, we have 0:03:05.919 --> 0:03:10.480 created technologies that augment our senses and increase our ability 0:03:10.520 --> 0:03:13.720 to detect things we otherwise would never notice. But even so, 0:03:14.160 --> 0:03:18.160 we still don't know how much we don't know. Cosmological 0:03:18.200 --> 0:03:21.640 theories state that dark matter has to exist based on 0:03:21.680 --> 0:03:25.440 our observations of how galaxies move in our universe, and 0:03:25.480 --> 0:03:28.440 that this matter must make up the majority of all 0:03:28.560 --> 0:03:32.239 matter in our reality, but we can't detect it directly. 0:03:32.960 --> 0:03:35.240 That leads me to believe that we will never possess 0:03:35.360 --> 0:03:38.800 true knowledge of the universe, because I think there's always 0:03:38.800 --> 0:03:41.920 going to be something more that's outside our ability to 0:03:41.960 --> 0:03:45.920 perceive and understand. Now, that should not stop us from trying. 0:03:46.360 --> 0:03:49.680 We should just be aware that there's always more to discover, 0:03:49.800 --> 0:03:52.080 and if anything, I think that should make us more 0:03:52.120 --> 0:03:56.480 determined to learn. Frequently we will use the word skeptic 0:03:56.520 --> 0:03:59.920 to refer to someone who doubts or denies the existence 0:04:00.040 --> 0:04:02.280 of something. But when I say it's good to be 0:04:02.280 --> 0:04:06.080 a skeptic, I do not mean you should doubt or 0:04:06.160 --> 0:04:10.080 deny everything. Rather, I use the word to refer to 0:04:10.160 --> 0:04:14.280 someone who applies critical thinking to subjects. They ask questions, 0:04:14.360 --> 0:04:17.479 they look for evidence to support claims. They do not 0:04:17.600 --> 0:04:21.920 accept a statement without supporting proof, and they look to 0:04:21.960 --> 0:04:24.360 make sure that any proof that is offered is of 0:04:24.440 --> 0:04:29.400 sound quality. This is why I won't call anyone a 0:04:29.560 --> 0:04:32.880 climate change skeptic. In that case, I would use the 0:04:33.000 --> 0:04:37.479 term denier rather than skeptic, because there is a mountain 0:04:37.800 --> 0:04:42.440 of research and there's a scientific consensus that climate change 0:04:42.480 --> 0:04:45.679 is a real thing. It is one thing to question 0:04:45.720 --> 0:04:48.880 a topic and demand evidence that I think you should 0:04:48.920 --> 0:04:53.880 totally do. However, it's another thing to dismiss the conclusion 0:04:54.480 --> 0:04:59.479 despite the evidence. If you don't accept evidence despite a 0:04:59.600 --> 0:05:04.480 sidentific a consensus, that's pretty much denial. A skeptic who 0:05:04.480 --> 0:05:07.600 would be confronted by sound evidence should be able to 0:05:07.640 --> 0:05:12.080 accept that information. So, for example, let's move away from 0:05:12.080 --> 0:05:15.280 climate change. Some people argue that that shouldn't count. I 0:05:15.320 --> 0:05:18.920 think they're wrong, but let's take something totally different. Let's 0:05:18.960 --> 0:05:22.520 say let's talk about ghosts. I do not believe in 0:05:22.640 --> 0:05:25.880 ghosts at all. If you came up to me and 0:05:25.920 --> 0:05:28.479 you told me a story about how you once saw 0:05:28.560 --> 0:05:32.599 a ghost, I might humor you, but I wouldn't believe 0:05:32.640 --> 0:05:35.600 you actually saw a ghost. I might believe that you 0:05:35.680 --> 0:05:38.440 believe it, but I would be pretty sure that what 0:05:38.560 --> 0:05:43.920 you experienced wasn't anything supernatural. I would suspect that something 0:05:43.960 --> 0:05:47.120 more mundane had happened, and that over time you had 0:05:47.160 --> 0:05:51.880 formulated a narrative that was reinforced with retellings, perhaps even 0:05:51.920 --> 0:05:55.080 to the extent that the version you tell no longer 0:05:55.200 --> 0:05:59.680 even remotely resembles the thing that actually happened. Because we 0:05:59.760 --> 0:06:02.920 you as we do that when we remember stuff. It's 0:06:02.960 --> 0:06:06.120 not like our brains are acting like a computer. Right 0:06:06.160 --> 0:06:08.640 with a computer, you tell a computer retrieve a file, 0:06:09.000 --> 0:06:10.800 it's gonna pull up a file, and that file is 0:06:10.800 --> 0:06:12.880 going to be exactly the same as it was the 0:06:12.960 --> 0:06:16.520 last time you saved the file. It will be identical 0:06:16.800 --> 0:06:20.560 to the last time you saved it. It's preserved perfectly, 0:06:20.720 --> 0:06:23.919 assuming that there's not something wrong with your computer. But 0:06:24.600 --> 0:06:27.719 that's not how our brains work. When we remember something, 0:06:27.760 --> 0:06:32.560 our brains actually form neural pathways. They create connections between 0:06:32.560 --> 0:06:38.000 neurons in our brains right, and that pathway resembles the 0:06:38.040 --> 0:06:41.400 one that happened when we first experienced the situation that 0:06:41.520 --> 0:06:45.440 formed the memory in the first place. So we experience something, 0:06:45.960 --> 0:06:50.000 our brain creates a connection that is related to that something. 0:06:50.080 --> 0:06:53.880 When we remember, our brain tries to recreate that connection. 0:06:54.839 --> 0:06:59.119 But our brains aren't perfect at recreating those pathways. Detours happen. 0:06:59.520 --> 0:07:03.200 Our memory areas are faulty. To us. It seems like 0:07:03.360 --> 0:07:07.440 the things were remembering our perfect recreations, but in reality, 0:07:07.760 --> 0:07:11.280 we are recreating a scenario in our heads, and our 0:07:11.320 --> 0:07:14.600 brains are not perfect, so we get things wrong and 0:07:14.680 --> 0:07:19.320 sometimes we reinforce the wrong parts. We make those detours 0:07:19.360 --> 0:07:22.960 more permanent parts of a pathway. So I'm not necessarily 0:07:22.960 --> 0:07:24.880 going to think you're trying to hoodwink me. I'm not 0:07:24.920 --> 0:07:27.200 thinking you're lying if you tell me you saw a ghost. 0:07:27.680 --> 0:07:29.840 But still I'm not going to believe your account that 0:07:29.920 --> 0:07:32.960 you saw a ghost. I'm going to think something else 0:07:33.040 --> 0:07:35.840 happened that made you think you saw a ghost, but 0:07:36.880 --> 0:07:41.360 in reality it was something more natural and mundane. However, 0:07:42.400 --> 0:07:46.960 if someone were to provide to me real evidence of 0:07:47.040 --> 0:07:52.200 the existence of ghosts, not an anecdote, but real, incontrovertible 0:07:52.240 --> 0:07:56.280 evidence of ghosts. I would have to reevaluate my beliefs. 0:07:56.680 --> 0:08:00.360 I would need to reconcile my disbelief in ghosts and 0:08:00.360 --> 0:08:04.800 then incorporate a new belief into my worldview. That's my 0:08:04.920 --> 0:08:08.360 responsibility as a skeptic. If you came up to me 0:08:08.440 --> 0:08:11.200 and you said, here's the proof, and you laid it 0:08:11.200 --> 0:08:13.600 out and I look at it and it is undeniable, 0:08:13.640 --> 0:08:15.480 I would have to say, you know what I was wrong, 0:08:15.800 --> 0:08:19.800 You are right, there are ghosts. This probably would not 0:08:19.880 --> 0:08:22.080 be easy for me to do, but I would like 0:08:22.160 --> 0:08:25.040 to think I would do it because I would be 0:08:25.080 --> 0:08:30.120 confronted by evidence. However, the claim that ghosts exist is 0:08:30.160 --> 0:08:35.440 an extraordinary claim, so it requires extraordinary proof. So if 0:08:35.440 --> 0:08:39.960 you are confronted with evidence and the evidence seems reliable, 0:08:40.840 --> 0:08:45.480 then you should take whatever the claim is and think 0:08:45.960 --> 0:08:50.080 this may very well have merit. Climate change deniers do 0:08:50.200 --> 0:08:54.439 not do this. They deny this mountain of evidence and 0:08:54.480 --> 0:08:58.679 a scientific consensus, so they do not take that and 0:08:58.720 --> 0:09:03.600 they incorporated. They instead say this does not align with 0:09:03.800 --> 0:09:08.520 my worldview, and so I reject it. That is not skepticism, 0:09:08.559 --> 0:09:12.080 that is denialism. Now, one of my favorite episodes of 0:09:12.120 --> 0:09:16.160 Tech Stuff was all about ghost hunting technology, and that's 0:09:16.200 --> 0:09:19.199 because Chris Palette and I went through the equipment that 0:09:19.440 --> 0:09:23.199 is frequently used by people in the ghost hunting professions 0:09:23.600 --> 0:09:27.400 and we explained why that technology did not actually prove 0:09:27.480 --> 0:09:31.680 the existence of ghosts. For one thing, ghost hunters tend 0:09:31.679 --> 0:09:34.920 to use equipment that was intended to do other work. 0:09:35.840 --> 0:09:40.200 One example would be a meter to detect electromagnetic fluctuations, 0:09:40.240 --> 0:09:43.120 which you would use in order to look for faulty 0:09:43.200 --> 0:09:46.280 wiring in a house. For example, if wiring didn't have 0:09:46.320 --> 0:09:50.040 proper insulation, then whenever a current runs through that wire, 0:09:50.080 --> 0:09:53.920 it would generate an electromagnetic field and you would be 0:09:53.960 --> 0:09:55.840 able to pick it up and that would tell you, hey, 0:09:55.880 --> 0:09:58.360 maybe we need to fix the wiring here. That's what 0:09:58.559 --> 0:10:03.080 those meters are full. Or however, ghost hunters would say, uh, 0:10:03.280 --> 0:10:06.560 this meter shows fluctuations. That means there's a ghost here, 0:10:07.000 --> 0:10:09.960 But that puts the cart about a mile in front 0:10:10.000 --> 0:10:12.720 of the horse. Because at no point that the ghost 0:10:12.800 --> 0:10:18.600 hunter actually provide evidence that ghosts exist. They're saying, here 0:10:18.720 --> 0:10:21.960 is an effect the cause is this other thing that 0:10:22.080 --> 0:10:26.600 I have yet to prove exists. That's not how science works. 0:10:26.920 --> 0:10:30.760 You have to prove that a ghost exists first. Then 0:10:30.840 --> 0:10:33.640 you have to prove that a ghost can affect or 0:10:33.720 --> 0:10:38.319 generate electromagnetic fields. Then and only then can you say 0:10:38.520 --> 0:10:43.680 I just found a fluctuating electromagnetic field. One possible explanation 0:10:43.800 --> 0:10:48.880 is ghosts. You can't just say because this needle moved, 0:10:49.120 --> 0:10:52.079 ghosts exist. That's not how science works. You have to 0:10:52.840 --> 0:10:56.840 establish the existence first, then established that they can in 0:10:56.920 --> 0:11:00.520 fact produce the effect you claim. Then and you can 0:11:00.679 --> 0:11:07.520 use uh actual instances of that effect to potentially detect 0:11:08.240 --> 0:11:12.880 the presence of such a thing. So that's the way 0:11:12.920 --> 0:11:15.679 that would have to work. Ghost hunting does not do that. 0:11:15.760 --> 0:11:19.920 It ignores that. It skips It presupposes the existence of 0:11:19.960 --> 0:11:23.840 the thing they're looking for without first establishing that actually 0:11:23.960 --> 0:11:28.120 is a thing. Now, I want to stress that I 0:11:28.160 --> 0:11:33.839 don't mean to say ghosts definitely do not exist. What 0:11:33.880 --> 0:11:38.040 I'm saying is I do not believe they exist. But 0:11:38.120 --> 0:11:40.040 as I said at the beginning, I think it is 0:11:40.080 --> 0:11:44.320 impossible for humans to know everything, so I could be 0:11:44.480 --> 0:11:48.280 wrong about this. However, I have seen no evidence that 0:11:48.440 --> 0:11:52.720 cannot be attributed to much more mundane causes than ghosts, 0:11:53.440 --> 0:11:55.640 and that would be necessary for me to change my mind. 0:11:55.640 --> 0:11:59.680 I would need that extraordinary evidence. I do not deny 0:11:59.760 --> 0:12:03.640 the existence of ghosts outright, I just don't believe in them, 0:12:03.679 --> 0:12:08.199 because skepticism and denial are two different things. Now, some 0:12:08.280 --> 0:12:12.800 skeptics do deny certain things that might be climate change. 0:12:12.840 --> 0:12:15.520 There are skeptics who don't believe in it that might 0:12:15.600 --> 0:12:19.800 be the supernatural. And some skeptics believe in certain things 0:12:19.800 --> 0:12:22.840 that are not supported by evidence as a matter of faith. 0:12:23.320 --> 0:12:28.280 They might be uh devout in a religion, and other 0:12:28.360 --> 0:12:30.720 skeptics would say, oh, you're not a skeptic because you 0:12:30.760 --> 0:12:34.880 actually believe in this religious faith and there's no evidence 0:12:34.920 --> 0:12:39.199 to support your beliefs. I don't want to say there's 0:12:39.240 --> 0:12:42.480 only one way to be a skeptic. I do think, however, 0:12:42.840 --> 0:12:46.480 the application of critical thinking and the scientific method is 0:12:46.559 --> 0:12:50.439 really important for everyone, whether you consider yourself a skeptic 0:12:50.720 --> 0:12:53.920 or otherwise. Now, when we come back, I'm gonna go 0:12:53.960 --> 0:12:57.240 into the scientific method, how it works, and why I 0:12:57.280 --> 0:13:01.240 think it's so important, And I'm also going to probably 0:13:01.240 --> 0:13:04.960 talk about some other stuff like anti intellectualism and why 0:13:04.960 --> 0:13:07.960 it's so demoralizing. Maybe I'll even get around to explaining 0:13:07.960 --> 0:13:11.280 why I think the term Boffin's is incredibly insulting. So 0:13:11.640 --> 0:13:14.240 tune in Brits because I got a bone to pick 0:13:14.280 --> 0:13:17.800 with you guys. But first, let's take a quick break 0:13:18.120 --> 0:13:28.840 and thank our sponsor. The scientific method is a process 0:13:28.960 --> 0:13:33.040 by which we test ideas to ascertain their validity. So, 0:13:33.080 --> 0:13:35.560 in other words, it's how we make sure the stuff 0:13:35.600 --> 0:13:39.680 we believe in has some basis in actual reality. The 0:13:39.760 --> 0:13:43.280 history of the scientific method is complex. I can't really 0:13:43.320 --> 0:13:46.480 go into all of it because you could do a 0:13:46.600 --> 0:13:51.199 full podcast series just about the evolution of the scientific method. 0:13:51.280 --> 0:13:55.600 It is fascinating, but it is also exhaustive. Cultures from 0:13:55.600 --> 0:13:59.280 all around the world developed approaches to testing ideas, so 0:13:59.559 --> 0:14:02.600 this was in something that arose specifically out of one 0:14:02.640 --> 0:14:05.920 place in the world. Lots of different cultures came up 0:14:05.920 --> 0:14:09.880 with different ways to take this approach. However, generally speaking, 0:14:10.360 --> 0:14:14.719 we trace the scientific method as we understand it today 0:14:14.720 --> 0:14:17.920 back to the Greek philosopher Aristotle. He gets the credit 0:14:17.920 --> 0:14:20.400 for laying out the basics of the scientific method. He 0:14:20.560 --> 0:14:27.440 proposed combining empirical measurements and observations with inductive reasoning. Now, 0:14:27.960 --> 0:14:31.640 a lot of Greek philosophers before Aristotle essentially said, all 0:14:31.680 --> 0:14:33.600 you need is to be able to think. And if 0:14:33.640 --> 0:14:36.160 you think, and you think good enough, if you're really 0:14:36.160 --> 0:14:38.720 good at the thinking, then you can think the things 0:14:38.760 --> 0:14:41.280 that you never thought before and figure out how the 0:14:41.280 --> 0:14:44.840 world works. All it takes is just pure thought. Aristotle said, 0:14:45.120 --> 0:14:50.160 hang on, what if we see stuff and we measure stuff, 0:14:50.400 --> 0:14:54.080 and we take those empirical measurements and we incorporate that 0:14:54.120 --> 0:14:57.520 into what we think, so that we can actually see 0:14:57.520 --> 0:15:00.640 how the world really works, and by is our knowledge 0:15:00.640 --> 0:15:06.720 off of that. This was pretty rebellious among Greek philosophers 0:15:06.760 --> 0:15:09.400 at the time. Inductive reasoning, by the way, is where 0:15:09.400 --> 0:15:13.160 you take specific points of data and you make generalizations 0:15:13.200 --> 0:15:17.520 from those specific points. Contrast that with deductive reasoning. That's 0:15:17.520 --> 0:15:20.240 where you start with a general idea and then you 0:15:20.280 --> 0:15:24.200 work your way towards specifics. Now, typically scientists use both 0:15:24.240 --> 0:15:27.480 inductive and deductive reasoning in order to form hypotheses and 0:15:27.520 --> 0:15:30.480 then test them to make sure they are true. The 0:15:30.600 --> 0:15:35.600 Islamic scholar even al hath them further developed the scientific method. 0:15:35.680 --> 0:15:38.600 His basic steps are pretty much the foundation of the 0:15:38.640 --> 0:15:41.840 scientific method today. He said, well, first you have to 0:15:42.240 --> 0:15:47.000 state and explicit problem. This would essentially be a claim, 0:15:47.440 --> 0:15:51.320 usually based off observations and experimentation. Then you test a 0:15:51.400 --> 0:15:54.680 hypothesis through various experiments to see if it holds validity. 0:15:54.840 --> 0:15:57.680 You interpret the data from those experiments, and then you 0:15:57.720 --> 0:16:02.160 conclude whether the hypothesis held up under experimentation or whether 0:16:02.360 --> 0:16:05.800 the experimentation didn't support the hypothesis, and then you publish 0:16:05.960 --> 0:16:08.320 what you found so that other people can read it. 0:16:08.840 --> 0:16:13.120 The scientific method developed further throughout the Renaissance, also developed 0:16:13.120 --> 0:16:15.600 in the Enlightenment and the Modern Age of science, and 0:16:16.160 --> 0:16:19.840 basically the principles have remained fairly stable, although the particulars 0:16:19.840 --> 0:16:24.440 have obviously changed over time. Now you can't use the 0:16:24.480 --> 0:16:28.440 scientific method for absolutely everything, but it does provide a 0:16:28.480 --> 0:16:32.800 great model to follow as a skeptic. Basically, you take 0:16:32.880 --> 0:16:36.120 a testable claim, and it is important that the claim 0:16:36.360 --> 0:16:41.040 is testable. It's a claim that can be tested for accuracy. 0:16:41.120 --> 0:16:43.760 If a claim is not testable, then it does not 0:16:43.920 --> 0:16:46.680 belong to the realm of science. We used to call 0:16:46.760 --> 0:16:50.480 this falsifiable. By the way, if a claim is falsifiable, 0:16:51.040 --> 0:16:54.640 then it is scientific. That means if the claim is 0:16:54.680 --> 0:16:58.120 such that you could design a test where the claim 0:16:58.240 --> 0:17:01.000 is proven to be untrue, then you would say it's 0:17:01.040 --> 0:17:05.080 falsifiable and thus scientific. If it's a claim where you 0:17:05.160 --> 0:17:09.120 cannot come up with a test that would have an 0:17:09.160 --> 0:17:12.480 outcome where the claim could be untrue, it is not 0:17:12.640 --> 0:17:16.600 part of science. So, for example, if I claim there 0:17:16.760 --> 0:17:21.600 is a pixie that follows you around and it's always 0:17:21.640 --> 0:17:26.080 just over your right shoulder, however it is completely undetectable 0:17:26.119 --> 0:17:30.000 by any means that we know, that claim is not testable. 0:17:30.040 --> 0:17:32.840 There's nothing you can do to see if there's a 0:17:32.840 --> 0:17:36.600 pixie there, because I just told you it's completely undetectable. 0:17:36.840 --> 0:17:40.480 It's there, but you can't detect it. Well, there's no 0:17:40.560 --> 0:17:43.320 experiment you can conduct to test that claim. It does 0:17:43.359 --> 0:17:46.679 not fall in the realm of science. But if I 0:17:46.720 --> 0:17:50.200 said there's a pixie that floats over your right shoulder 0:17:50.440 --> 0:17:54.639 that can only be detected by some particularly expensive piece 0:17:54.640 --> 0:17:59.679 of scanning equipment, that claim is testable because you could 0:17:59.720 --> 0:18:03.120 go out and secure the appropriate scanning equipment and scan 0:18:03.240 --> 0:18:06.239 yourself and look for the evidence, and if there's no 0:18:06.320 --> 0:18:08.560 evidence of a pixie there, it would show that my 0:18:08.640 --> 0:18:13.159 claim was not substantiated. There was no pixie and I 0:18:13.160 --> 0:18:15.359 had told you, hey, if you get this expensive scanner, 0:18:16.080 --> 0:18:18.280 then you'll find a pixie. If I'm the one selling 0:18:18.320 --> 0:18:21.360 you the expensive scanner, it just means I scammed you, 0:18:21.720 --> 0:18:24.320 and I probably changed my claim slightly, because that's how 0:18:24.359 --> 0:18:27.520 things often go. It's called moving the goal posts. Where 0:18:28.000 --> 0:18:32.240 I make a claim, someone tests the claim, the claim 0:18:32.320 --> 0:18:35.439 does not hold up to the test, so I change 0:18:35.520 --> 0:18:38.639 the claims slightly in order to make it seem like 0:18:38.680 --> 0:18:42.800 it's still valid. That's moving the goal posts. That's not 0:18:42.920 --> 0:18:47.480 good cricket, y'all. Anyway, Let's say I make a testable claim, 0:18:47.520 --> 0:18:51.600 such as, this extremely expensive sound system can playback a 0:18:51.720 --> 0:18:55.520 recording at such high fidelity that it leaves all other 0:18:55.600 --> 0:18:59.760 sound systems behind. Now, you could perhaps measure the performance 0:19:00.000 --> 0:19:04.520 of this sound system with very precise sensitive measurement devices, 0:19:05.280 --> 0:19:08.960 and it might even show that, according to those devices, 0:19:09.560 --> 0:19:13.280 my system is outperforming other systems. But it may not 0:19:13.440 --> 0:19:18.920 actually matter. And that's because again, our senses are limited, right. 0:19:19.080 --> 0:19:23.320 Human senses have limitations. So there comes a point where 0:19:23.720 --> 0:19:28.640 replication will be meaningless because we humans are physically incapable 0:19:28.720 --> 0:19:32.880 of detecting anything at that level of fidelity or resolution. 0:19:33.119 --> 0:19:36.400 So with a claim, like that the way we perceive 0:19:36.520 --> 0:19:40.040 sound might matter more than the empirical measurements we gather 0:19:40.119 --> 0:19:43.640 from the device. And not only that, but the psychology 0:19:43.840 --> 0:19:46.720 of listening to a sound system might matter more. If 0:19:46.760 --> 0:19:50.000 I believe I'm going to get a better experience from 0:19:50.040 --> 0:19:53.280 a particular sound system, my perception might be that I 0:19:53.320 --> 0:19:56.720 did get a better experience, even if objectively, if I 0:19:56.760 --> 0:20:00.840 had no knowledge of the quality or back thereof of 0:20:00.920 --> 0:20:03.719 the sound system, I might not ever have noticed it. 0:20:04.800 --> 0:20:07.679 Brains are funny things. So how can you test a 0:20:07.800 --> 0:20:11.160 claim that says this sound system is better than all 0:20:11.240 --> 0:20:14.239 others so that it makes sense to us that we 0:20:14.280 --> 0:20:17.439 can actually see if there's something there. Well, this is 0:20:17.480 --> 0:20:22.040 where formulating a proper experiment is incredibly important. So a 0:20:22.080 --> 0:20:26.840 blind experiment helps reduce bias in a process. In a 0:20:26.880 --> 0:20:30.480 blind experiment, the subject does not know if he or 0:20:30.520 --> 0:20:34.480 she is in a test group or a control group. Uh. 0:20:34.520 --> 0:20:37.399 And with a sound system issue, you might be in 0:20:37.400 --> 0:20:40.240 a control group sometimes in a test group other times, 0:20:40.280 --> 0:20:44.080 and you're recording your responses to a questionnaire. Let's say 0:20:44.080 --> 0:20:46.800 this design to see if you detect a difference in 0:20:46.880 --> 0:20:49.960 sound quality. Uh. So it might mean that if I 0:20:50.000 --> 0:20:54.119 were to design an experiment to test the claims someone 0:20:54.160 --> 0:20:57.080 makes about a particular sound system. I might make it 0:20:57.160 --> 0:21:00.680 like this. Uh. You are invited to participate in a 0:21:00.680 --> 0:21:04.080 test to see if a particular sound system is perceptibly 0:21:04.200 --> 0:21:08.120 better than other sound systems. So you're led to a room. 0:21:08.200 --> 0:21:11.080 You cannot see the system, or the speakers or anything 0:21:11.080 --> 0:21:13.880 in this room. The room is is designed in such 0:21:13.920 --> 0:21:17.440 way where the entire sound system is hidden from your view. 0:21:18.160 --> 0:21:21.119 You're told to listen to a recording. You do, so, 0:21:21.240 --> 0:21:23.400 you listen to the recording. Maybe you make some notes. 0:21:24.119 --> 0:21:27.200 Then you're led to a second room and you repeat 0:21:27.240 --> 0:21:30.359 the procedure. You you can't see the sound system in 0:21:30.400 --> 0:21:33.200 this room. You don't know if you're listening to an 0:21:33.200 --> 0:21:36.399 identical sound system or a totally different sound system. You 0:21:36.480 --> 0:21:39.240 make notes. Maybe you go to several rooms and you're 0:21:39.240 --> 0:21:42.040 asked to rate the audio quality experience in each of 0:21:42.040 --> 0:21:44.359 the rooms you visit. Some of the rooms might have 0:21:44.440 --> 0:21:48.879 duplicate systems calibrated precisely the same way for their respective 0:21:48.920 --> 0:21:53.639 spaces to give you the same sort of uh, replicated sound, 0:21:54.280 --> 0:21:56.080 but you don't know that. You don't know that going 0:21:56.080 --> 0:21:58.800 from room to room, which one it is and there 0:21:58.840 --> 0:22:02.000 might be as many variables eliminated as possible. So you're 0:22:02.040 --> 0:22:04.719 trying to, you know, keep the rooms the exact same size, 0:22:04.760 --> 0:22:08.199 with the exact same uh sound proofing. Everything needs to 0:22:08.200 --> 0:22:11.400 be as close to identical as possible to eliminate variables. 0:22:12.400 --> 0:22:15.640 And it might be that rooms two and three out 0:22:15.680 --> 0:22:20.560 of seven have the supposedly remarkable system in them, but 0:22:20.640 --> 0:22:25.000 the other five have regular sound systems in them. Now, you, 0:22:25.160 --> 0:22:27.200 as the subject, don't know that. You're just going through 0:22:27.240 --> 0:22:31.200 the experiment. The experimenters will repeat this process with as 0:22:31.240 --> 0:22:34.680 many test subjects as they possibly can. The more subjects 0:22:34.680 --> 0:22:37.760 you have, the better it increases the sample size. It 0:22:37.800 --> 0:22:42.119 helps account for outliers. Then the experimenters collate all the data, 0:22:42.520 --> 0:22:46.520 they analyze it, and they draw conclusions. And perhaps the 0:22:46.560 --> 0:22:50.040 super relative system really is superior, and people are regularly 0:22:50.160 --> 0:22:53.439 and reliably picking it out. They're saying rooms two and 0:22:53.520 --> 0:22:56.479 three sounded really good, like better than all the others. 0:22:57.000 --> 0:22:59.040 If everyone is saying that, or if enough people are 0:22:59.040 --> 0:23:02.600 saying that reliable, then you might say he the claims 0:23:02.680 --> 0:23:06.080 that were made seem to hold true people are able 0:23:06.119 --> 0:23:10.360 to identify it, or you might find that there's no 0:23:10.400 --> 0:23:12.920 conclusive support for the claim, not enough people were able 0:23:12.960 --> 0:23:15.760 to tell a difference, or maybe some other people said, 0:23:15.960 --> 0:23:18.840 you know, rooms two and six sounded better than all 0:23:18.840 --> 0:23:21.680 the other rooms, and you're thinking to yourself, huh, only 0:23:21.720 --> 0:23:25.040 two and three had the sound system that was really 0:23:25.040 --> 0:23:27.720 good in them. The other rooms had regular sound systems 0:23:27.720 --> 0:23:30.280 in them. So if you say two and six, that's 0:23:30.280 --> 0:23:33.800 not conclusive. Or if you say five and six, now 0:23:33.840 --> 0:23:35.720 that's a real problem because you've picked out two of 0:23:35.720 --> 0:23:38.359 the sound systems that we're not supposed to be as good. 0:23:38.800 --> 0:23:41.800 So that's one way of testing it. Now, I would 0:23:41.800 --> 0:23:44.760 actually suggest you go a step further in your experiment 0:23:44.800 --> 0:23:48.520 design and you create what is called a double blind experiment. 0:23:49.000 --> 0:23:52.359 In a double blind experiment, the person in charge of 0:23:52.840 --> 0:23:57.040 conducting the test is unaware of which subjects are in 0:23:57.040 --> 0:24:00.199 a control group or a test group as well. So, 0:24:00.520 --> 0:24:03.480 in other words, the person who leads you the subject 0:24:03.560 --> 0:24:06.280 from room the room also does not know which rooms 0:24:06.320 --> 0:24:09.160 have the really good system in them and which rooms 0:24:09.200 --> 0:24:12.880 have normal systems in them. They have no idea. This way, 0:24:13.040 --> 0:24:15.639 the person who's leading you from room to room can't 0:24:16.560 --> 0:24:23.160 unconsciously introduce bias into the experiment by perhaps giving away 0:24:23.240 --> 0:24:25.919 that rooms two and three have the really good system 0:24:25.960 --> 0:24:29.600 in them because they don't know. Double Blind experiments, in 0:24:29.600 --> 0:24:32.720 my opinion, tend to be the way to go for 0:24:32.800 --> 0:24:36.280 testable claims, especially claims that are on the extraordinary side. 0:24:36.320 --> 0:24:39.360 You want to design a test in such a way 0:24:39.440 --> 0:24:42.679 that you don't influence the subjects to give you the 0:24:42.720 --> 0:24:47.320 result you want to get. Whether you want a test 0:24:47.359 --> 0:24:50.280 to come out positive or negative, you want to avoid 0:24:50.400 --> 0:24:54.600 introducing bias into the actual experiment. Uh, these tests where 0:24:54.640 --> 0:24:58.120 you're testing something that's subjective, like the quality of sound, 0:24:58.920 --> 0:25:02.440 those are particularly tricky because it's not like it's just 0:25:02.680 --> 0:25:05.760 a measurement you can read off a meter. It's the 0:25:05.840 --> 0:25:11.440 experience someone is having, and subjective experiences are very difficult 0:25:11.480 --> 0:25:15.320