WEBVTT - The Black Box Problem 0:00:04.240 --> 0:00:07.240 Welcome to tech Stuff, a production of I Heart Radios, 0:00:07.320 --> 0:00:13.840 How Stuff Works. Hey there, and welcome to tech Stuff. 0:00:13.880 --> 0:00:16.640 I'm your host, Jonathan Strickling. I'm an executive producer with 0:00:16.640 --> 0:00:18.480 How Stuff Works and my Heart Radio and I love 0:00:18.560 --> 0:00:22.560 all things tech. I always have to qualify that after 0:00:22.600 --> 0:00:25.360 I say it, because then I end up covering topics 0:00:25.400 --> 0:00:29.560 like this one today where I don't love all things tech. 0:00:30.000 --> 0:00:33.320 I guess it's it's being a little a little disingenuous 0:00:33.360 --> 0:00:37.760 to make that claim. But recently, critical thinking has really 0:00:38.040 --> 0:00:41.400 been on my mind a lot. I always want to 0:00:41.479 --> 0:00:45.159 be a critical thinker, though like most humans, I do 0:00:45.280 --> 0:00:49.040 have lapses. Sometimes I encounter a message that is so 0:00:49.120 --> 0:00:52.400 appealing that my desire for it to be true can 0:00:52.479 --> 0:00:56.160 override my skepticism, and I'll fail to ask myself or 0:00:56.440 --> 0:00:59.760 anyone else for that matter, important questions to make sure 0:00:59.800 --> 0:01:03.880 that what is being promised is in fact realistic and achievable. 0:01:04.280 --> 0:01:07.280 My goal is to minimize the number of times I 0:01:07.360 --> 0:01:09.840 go along with a pitch simply because it was a 0:01:09.880 --> 0:01:13.679 really good pitch. But technology sometimes makes that really hard. 0:01:14.120 --> 0:01:16.559 And I want to talk about this challenge because it's 0:01:16.560 --> 0:01:21.880 one that we all encounter. Technology is undeniably amazing. Just 0:01:21.920 --> 0:01:25.399 think about how much humans have achieved in a very 0:01:25.480 --> 0:01:28.600 short amount of time. For most of human history, our 0:01:28.720 --> 0:01:33.959 technological advancement was really slow. We completed some monumental achievements 0:01:33.959 --> 0:01:37.760 in art and architecture, and sadly, in finding new ways 0:01:37.800 --> 0:01:40.600 to kill each other, but apart from some early experiments 0:01:40.600 --> 0:01:43.720 and steam power, and a few interesting ideas from geniuses 0:01:43.760 --> 0:01:48.760 like Leonardo da Vinci, we really didn't see incredibly rapid advances. 0:01:49.160 --> 0:01:52.560 Then we get to the nineteenth century, when a combination 0:01:52.600 --> 0:01:56.800 of factors led to the Industrial Revolution. That revolution increased 0:01:56.800 --> 0:01:59.720 productivity and lead to conditions that made it possible for 0:01:59.800 --> 0:02:03.960 more innovators to experiment and expand our knowledge, understanding, and 0:02:04.080 --> 0:02:07.800 ability to exploit the world around us. Then we get 0:02:07.840 --> 0:02:11.080 to the twentieth century and the development of computers and 0:02:11.120 --> 0:02:16.280 the transistor, manaturization, mass produced plastic, and some other important 0:02:16.320 --> 0:02:21.919 innovations that would allow for truly rapid technological evolution. Consider this, 0:02:22.280 --> 0:02:25.440 When I was a kid, there was no public internet, 0:02:25.560 --> 0:02:28.160 cell phones were pretty much restricted to R and D labs. 0:02:28.480 --> 0:02:32.240 Personal computers had just entered the market. Many of today's 0:02:32.400 --> 0:02:36.440 fastest growing companies couldn't have existed because the business they 0:02:36.440 --> 0:02:40.240 are based off of didn't have a platform yet. For 0:02:40.280 --> 0:02:43.560 a while, it remained possible for the average person to 0:02:43.680 --> 0:02:47.320 know enough about the technology they encountered to deal with 0:02:47.360 --> 0:02:50.120 it when things went wrong, at least for most of 0:02:50.160 --> 0:02:53.360 the technology. Some like television sets and stuff, we're already 0:02:53.360 --> 0:02:56.560 well beyond the understanding of the average person. But this 0:02:56.680 --> 0:03:00.359 actually got harder to do as technology advanced, and we've 0:03:00.400 --> 0:03:03.640 seen it manifest in many ways. A good example of 0:03:03.680 --> 0:03:08.160 this is in the automotive industry. Classic cars can be complicated, 0:03:08.200 --> 0:03:11.440 but with some training and practice and owner can learn 0:03:11.480 --> 0:03:13.720 how to do maintenance and repairs on a lot of 0:03:13.760 --> 0:03:17.040 different parts of the car by themselves. There's a learning 0:03:17.040 --> 0:03:19.919 curve there, but it's totally possible, and there are a 0:03:20.000 --> 0:03:22.840 lot of people who love to take old cars and 0:03:22.880 --> 0:03:27.160 restore them as sort of a passion project. Today cars 0:03:27.200 --> 0:03:29.560 tend to have components in them that are high tech 0:03:30.080 --> 0:03:32.200 and sealed away in such a way as to make 0:03:32.240 --> 0:03:36.720 it difficult or impossible to access without proprietary tools and 0:03:36.840 --> 0:03:39.400 a deep understanding of how they work. You might not 0:03:39.440 --> 0:03:41.720 be able to tell what's wrong with a car without 0:03:41.760 --> 0:03:46.280 a special diagnosis scanning tool. And after learning what's wrong, 0:03:46.520 --> 0:03:48.840 you might not be able to do anything about it yourself. 0:03:49.240 --> 0:03:53.120 As cars get more advanced with features like various sensors 0:03:53.160 --> 0:03:56.880 and systems for stuff like lane assist, adaptive cruise control, 0:03:57.040 --> 0:04:00.960 parking assist, and more, they become harder to maintained by 0:04:00.960 --> 0:04:04.920 the owner. They're turning into what folks call a black box, 0:04:05.320 --> 0:04:08.680 in which you have a type of technology where the 0:04:08.720 --> 0:04:12.600 inner workings are hidden away from the user. It doesn't 0:04:12.760 --> 0:04:15.720 literally have to be hidden from sight, it can just 0:04:15.840 --> 0:04:19.960 be so complicated that the average person finds it inaccessible. 0:04:20.400 --> 0:04:22.880 And this leads us to a real challenge. As we 0:04:22.920 --> 0:04:26.600 have learned more about the universe, we've specialized in our 0:04:26.640 --> 0:04:29.760 focus we had to. We quickly reached a point in 0:04:29.760 --> 0:04:32.480 which it's pretty much impossible to have a deep understanding 0:04:32.720 --> 0:04:36.800 of all subjects. Then of our understanding has grown, we've 0:04:36.839 --> 0:04:40.760 pushed back the boundaries of ignorance. But now there's just 0:04:40.800 --> 0:04:43.680 too much to know for any one person to know 0:04:43.760 --> 0:04:47.320 it all. And the stuff we've built has capitalized on 0:04:47.400 --> 0:04:51.360 this focused understanding, but it also means that it's created 0:04:51.400 --> 0:04:54.640 a barrier for us. We might know that something works, 0:04:54.920 --> 0:04:58.920 but we don't necessarily understand how it works because it's 0:04:58.960 --> 0:05:02.360 based off a principle that's alien to us, and this 0:05:02.440 --> 0:05:05.840 was bound to happen, but it creates a dangerous situation. 0:05:06.120 --> 0:05:10.480 It's dangerous for a few different reasons. First, we as 0:05:10.520 --> 0:05:14.400 consumers can grow complacent. We expect stuff to work, and 0:05:14.440 --> 0:05:18.000 when that stuff doesn't work, we're frustrated. Worse, because we 0:05:18.080 --> 0:05:22.080 probably don't understand how that stuff works, we don't really 0:05:22.120 --> 0:05:25.119 know how to go about fixing it. On a positive note, 0:05:25.400 --> 0:05:28.800 that usually means there's opportunity for people with expertise to 0:05:28.839 --> 0:05:32.359 make a living as a troubleshooter or repair professional, but 0:05:32.440 --> 0:05:35.200 it means that as consumers we have less ability to 0:05:35.240 --> 0:05:39.240 work with the stuff we actually consume. Second, our technology 0:05:39.279 --> 0:05:42.880 is approaching the point where it could be really dangerous 0:05:42.960 --> 0:05:46.480 if we don't understand exactly how it's working. As machine 0:05:46.600 --> 0:05:51.280 learning models and artificial intelligence become more sophisticated, it becomes 0:05:51.279 --> 0:05:54.120 more important for us to understand how these systems are 0:05:54.160 --> 0:05:57.800 coming to conclusions. It's pretty cool to say, hey, I 0:05:57.880 --> 0:06:00.560 built this machine learning model and I train AI on 0:06:00.600 --> 0:06:03.240 how to recognize a person's face. Maybe you built the 0:06:03.279 --> 0:06:06.960 model to work with a camera manufacturer so that the 0:06:07.040 --> 0:06:10.640 cameras they make automatically detective face and then focus on it. 0:06:11.000 --> 0:06:14.000 But that same technology could potentially be used in tracking 0:06:14.080 --> 0:06:18.080 and identification systems, and if that system was being used 0:06:18.080 --> 0:06:21.760 by say, law enforcement, you want to understand exactly how 0:06:21.800 --> 0:06:24.640 the system is identifying people so that you can audit 0:06:24.680 --> 0:06:27.200 it and make sure that it's being accurate and not 0:06:27.720 --> 0:06:30.800 having a lot of false flags. Otherwise you run the 0:06:30.880 --> 0:06:34.760 risk of having the machine mistakenly identify people during an investigation, 0:06:35.040 --> 0:06:38.320 which at the very least could be disruptive. Or to 0:06:38.400 --> 0:06:42.800 go back to cars for a second, consider driverless cars. Now. 0:06:42.920 --> 0:06:46.960 I am still optimistic about driverless cars, but I've tempered 0:06:47.040 --> 0:06:50.719 my expectations on when we might see them. In my mind, 0:06:51.240 --> 0:06:54.080 I was thinking, well, a car decked out with sensors 0:06:54.200 --> 0:06:56.520 and a really fast computer system would be able to 0:06:56.560 --> 0:06:59.800 detect potential problems and react to them far faster and 0:06:59.800 --> 0:07:03.760 more logically than a human. I even thought a computer 0:07:03.839 --> 0:07:08.039 system could be able to monitor completely around a vehicle, 0:07:08.279 --> 0:07:11.360 whereas a human is typically focused on whatever is directly 0:07:11.400 --> 0:07:14.520 in front of him or her, or perhaps in a mirror, 0:07:14.800 --> 0:07:17.880 but can't pay attention to all directions at the same time. 0:07:18.200 --> 0:07:21.120 And sure machines can react in a fraction of the 0:07:21.120 --> 0:07:23.680 time it takes humans to do it. But machines are 0:07:23.840 --> 0:07:28.440 really good at handling routine situations and then responding appropriately. 0:07:28.920 --> 0:07:32.360 The more unusual an event, the harder it is for 0:07:32.400 --> 0:07:37.120 the machine to cope with it. Machines typically aren't terribly adaptive, 0:07:37.560 --> 0:07:40.880 and so with many millions of cars on the road, 0:07:41.280 --> 0:07:47.240 plus bicyclists, pedestrians, animals, debris, weather events, and other factors, 0:07:47.480 --> 0:07:50.800 it's pretty rare for any drive of a significant length 0:07:50.880 --> 0:07:54.960 to be completely quote unquote normal. So we need to 0:07:55.000 --> 0:07:59.440 design autonomous cars that can adapt to situations. But that 0:07:59.600 --> 0:08:02.360 also means we need to understand what decisions a car 0:08:02.400 --> 0:08:05.600 will make, or the very least, determine why a car 0:08:05.760 --> 0:08:10.119 behaved in one way versus an alternative. And so there's 0:08:10.160 --> 0:08:13.320 a move in AI and artificial neural network circles to 0:08:13.400 --> 0:08:17.640 make these processes as transparent as possible so that we're 0:08:17.680 --> 0:08:21.400 not caught off guard when a machine takes a particular action. 0:08:21.920 --> 0:08:26.520 And third, advanced technology has given us unrealistic expectations of 0:08:26.600 --> 0:08:30.400 exactly what was possible. After all, two hundred years ago, 0:08:30.920 --> 0:08:35.360 we wouldn't really dream of going up into space. A 0:08:35.480 --> 0:08:38.959 century ago, we might dream of it, but we still 0:08:39.000 --> 0:08:41.840 had no real understanding of how we would accomplish it. 0:08:42.360 --> 0:08:45.920 Then within another five decades we were sending people up 0:08:45.960 --> 0:08:48.800 to space, then to the moon, and now we have 0:08:48.920 --> 0:08:53.120 private companies designing launch vehicles that can return to Earth 0:08:53.160 --> 0:08:56.200 to be refurbished and used again in future launches. That's 0:08:56.240 --> 0:09:00.720 pretty incredible. We also have seen technology go from enormous 0:09:01.040 --> 0:09:04.280 to the very tiny. In the nineteen forties, a computer 0:09:04.360 --> 0:09:07.280 took up a lot of space, maybe the entire floor 0:09:07.320 --> 0:09:10.120 of a building, and it's processing power would be a 0:09:10.120 --> 0:09:13.160 fraction of what you'd find in the average smartphone. These days, 0:09:13.679 --> 0:09:17.160 managorization and Moore's Law have conditioned us to think that 0:09:17.240 --> 0:09:20.920 technology is capable of pretty much anything. I mean, it 0:09:20.960 --> 0:09:23.640 has to be. We wouldn't have assumed that it would 0:09:23.679 --> 0:09:25.920 be easy to get your hands on a portable computer, 0:09:26.080 --> 0:09:29.640 capable of acting like a camera, a communications device, and 0:09:29.720 --> 0:09:32.920 a direct link to the world's largest repository of human knowledge, 0:09:33.640 --> 0:09:36.280 even if most of that knowledge seems to be centered 0:09:36.320 --> 0:09:40.240 on cats. But that means when someone comes forward with 0:09:40.320 --> 0:09:43.719 extraordinary claims, it's easier for us to take them at 0:09:43.760 --> 0:09:48.439 face value. Technology has created an environment in which what 0:09:48.520 --> 0:09:54.240 was impossible yesterday becomes a mundane everyday task tomorrow, and 0:09:54.280 --> 0:09:57.960 this means that people can leverage that to our disadvantage. 0:09:58.320 --> 0:10:01.280 In some cases, you might be with an outright snake 0:10:01.320 --> 0:10:04.720 oil salesman type, someone who knows very well that the 0:10:04.880 --> 0:10:08.520 dream they're peddling isn't based in reality. But in other 0:10:08.600 --> 0:10:12.320 cases you might have sincere people who truly believe they've 0:10:12.360 --> 0:10:15.880 either cracked the code on something that was previously thought 0:10:15.920 --> 0:10:19.720 impossible or they feel they're right on the cusp and 0:10:19.720 --> 0:10:22.760 if they can just get enough funding to cover costs, 0:10:23.040 --> 0:10:25.080 they'll get the rest of the way there. Now. In 0:10:25.080 --> 0:10:27.200 a way, this could be a good thing, as it 0:10:27.240 --> 0:10:30.760 means that innovators have more access to resources than ever before, 0:10:30.880 --> 0:10:34.120 and it could lead to great discoveries. But in other 0:10:34.200 --> 0:10:38.040 cases it can lead to frustration, financial hardship, and worse. 0:10:38.440 --> 0:10:40.760 When we come back after this quick break, I'll give 0:10:40.800 --> 0:10:52.400 an example that's in everyone's minds right now. There's probably 0:10:52.720 --> 0:10:55.600 no better company to point to when you're talking about 0:10:55.640 --> 0:10:59.439 the dangers of wishful thinking than Theopness. And I know 0:10:59.720 --> 0:11:02.720 it's been in the news a lot. You've probably heard 0:11:03.280 --> 0:11:06.559 tons about it, maybe you've seen the documentary about it, 0:11:06.600 --> 0:11:10.520 maybe you've seen the the various articles or listen to 0:11:10.559 --> 0:11:13.240 the podcasts about it. But we're going to talk about 0:11:13.280 --> 0:11:16.920 a little bit more in this context and just to 0:11:16.920 --> 0:11:19.560 give you an overview in case you haven't encountered this. 0:11:20.360 --> 0:11:23.760 A woman named Elizabeth Holmes founded the company after she 0:11:23.840 --> 0:11:27.360 dropped out of Stanford. I do not know her. I 0:11:27.440 --> 0:11:30.559 do not know whether or not she sincerely believed or 0:11:30.640 --> 0:11:33.600 believes that the text she was seeking to invent could 0:11:33.600 --> 0:11:36.800 really work. But I do know that as of right now, 0:11:37.360 --> 0:11:40.720 it isn't working, and that's a big problem for those 0:11:40.760 --> 0:11:43.800 of you who are unfamiliar with the story. Let's give 0:11:43.920 --> 0:11:48.280 a quick summary. Elizabeth Holmes wanted to create tech that 0:11:48.320 --> 0:11:51.800 could disrupt the health care industry. It would, in theory, 0:11:52.160 --> 0:11:55.800 give more control an agency to consumers who could learn 0:11:55.880 --> 0:11:58.160 much more about their health on their own without the 0:11:58.200 --> 0:12:01.120 need to make an appointment with a doctor and undergo 0:12:01.240 --> 0:12:05.240 numerous blood draws to have various blood tests performed. The 0:12:05.280 --> 0:12:08.600 basic idea was that Sara no Nos would develop a 0:12:08.640 --> 0:12:12.520 device capable of taking a very small sample of blood, 0:12:12.800 --> 0:12:15.520 small enough to be drawn from the tip of a finger. 0:12:16.160 --> 0:12:18.880 They would then run a battery of tests to look 0:12:18.920 --> 0:12:23.079 for indicators of different conditions and diseases within a relatively 0:12:23.120 --> 0:12:26.480 short time. It would produce the results, giving the user 0:12:26.520 --> 0:12:29.640 more information about their health, which in theory, would help 0:12:29.720 --> 0:12:32.839 that person have meaningful conversations with a physician if there 0:12:32.840 --> 0:12:36.920 were any markers that raised concern. And it's a very powerful, 0:12:37.200 --> 0:12:41.280 very compelling idea. There are technologies like labs on a 0:12:41.400 --> 0:12:45.640 chip and microprocessors designed to detect the presence of certain 0:12:45.720 --> 0:12:49.320 markers that indicate the presence of illness, but this would 0:12:49.320 --> 0:12:52.600 wrap all of that up into a single package. In fact, 0:12:52.840 --> 0:12:56.000 Homes worked on a project related to this. After her 0:12:56.080 --> 0:12:59.560 first year at Stanford. She joined the Genome Institute in 0:12:59.600 --> 0:13:04.600 single Or and was overseeing blood tests. She first envisioned 0:13:04.800 --> 0:13:08.120 a sort of arm band that would use micro needles, 0:13:08.160 --> 0:13:12.160 and those micro needles would both draw small blood samples 0:13:12.200 --> 0:13:16.599 and would also administer medication on an as needed basis. 0:13:16.640 --> 0:13:19.120 She later led a team to work on a machine 0:13:19.160 --> 0:13:22.520 that would accept a small capsule containing a blood sample 0:13:22.960 --> 0:13:26.480 an attempt to run multiple tests on that sample. This 0:13:26.600 --> 0:13:29.920 was in stark contrast with the normal medical procedure in 0:13:29.960 --> 0:13:33.160 which a doctor or nurse would draw numerous vials of 0:13:33.200 --> 0:13:36.600 blood for testing send those samples off to one of 0:13:36.800 --> 0:13:40.559 two major lab testing companies in the United States. If 0:13:40.600 --> 0:13:45.160 Sarinos's technology worked, the company could totally up end that 0:13:45.320 --> 0:13:49.000 system in the US. Patients could go to a clinic 0:13:49.120 --> 0:13:51.439 to have a test run and get the results back 0:13:51.559 --> 0:13:53.720 in hours, rather than having to take a trip to 0:13:53.760 --> 0:13:56.880 the doctor's office, sit for a blood draw, and wait 0:13:56.920 --> 0:13:59.880 several days. They could end up being cheaper than the 0:14:00.000 --> 0:14:03.640 old approach. Sarainos executives like Homes stressed that this would 0:14:03.640 --> 0:14:07.080 put patients in control of their own health information and 0:14:07.120 --> 0:14:10.080 could provide many benefits, such as a heads up for 0:14:10.160 --> 0:14:14.080 possible problems in the future or catching something early enough 0:14:14.120 --> 0:14:16.880 to treat it before it became too severe. But the 0:14:16.920 --> 0:14:20.560 problem is that this depended upon that if Thara Noss 0:14:20.600 --> 0:14:26.200 technology worked. Thing. As it turned out, the tech wasn't working. 0:14:26.480 --> 0:14:28.720 At least, it wasn't working at the level of the 0:14:28.760 --> 0:14:32.800 company was striving for the engineers that thereas were trying 0:14:33.000 --> 0:14:37.160 really hard to create a diagnostic device that could take 0:14:37.280 --> 0:14:39.720 that small blood sample and run it through a lot 0:14:39.760 --> 0:14:44.200 of tests. But as it turns out, that's actually incredibly complicated. 0:14:44.560 --> 0:14:47.960 Using such a small sample was already a huge challenge. 0:14:48.320 --> 0:14:50.200 On top of that, you have issues you have to 0:14:50.200 --> 0:14:54.520 worry about, like contamination. A contaminated sample could give off 0:14:54.640 --> 0:14:58.040 false positives, creating a situation in which a patient believes 0:14:58.040 --> 0:15:01.440 they might have a particular disease or a condition when 0:15:01.440 --> 0:15:04.800 that isn't really the case, or it could mask something 0:15:04.840 --> 0:15:08.240 that the patient would need to know about, but because 0:15:08.240 --> 0:15:12.640 the results would be inconclusive, they wouldn't know about it. Now. 0:15:12.640 --> 0:15:15.040 Perhaps the hope was that the company would be able 0:15:15.040 --> 0:15:18.360 to develop the technology rapidly with the help of large 0:15:18.360 --> 0:15:21.360 investments in the company. And sure enough, there were a 0:15:21.360 --> 0:15:25.960 lot of folks with deep pockets who poured money into thoroughness, 0:15:26.040 --> 0:15:29.240 and you can sort of understand why. If it all worked, 0:15:29.520 --> 0:15:32.360 it would be a revolution in medicine. The company would 0:15:32.480 --> 0:15:36.280 stand to gain billions of dollars, The cause appeared to 0:15:36.280 --> 0:15:38.520 be noble, and the outcome looked like it would be 0:15:38.560 --> 0:15:42.880 incredibly profitable. It was a very tempting opportunity, and many 0:15:43.680 --> 0:15:47.640 didn't resist that temptation. On top of all that, it 0:15:47.720 --> 0:15:50.880 was dependent upon technology, and as I mentioned, we've come 0:15:50.880 --> 0:15:53.040 to a point where we believe technology can do just 0:15:53.120 --> 0:15:56.080 about anything. So it didn't seem outside the realm of 0:15:56.120 --> 0:16:01.280 possibility that a microchip inside a Suffici CICAD machine would 0:16:01.280 --> 0:16:04.120 be able to run a series of tests on the 0:16:04.240 --> 0:16:07.760 small blood sample and come up with meaningful results. But 0:16:07.880 --> 0:16:11.040 flash forward a few years and a bombshell of an 0:16:11.160 --> 0:16:14.400 article revealing that there was really a shell game going 0:16:14.440 --> 0:16:17.520 on at Sarais tells us that now we know the 0:16:17.520 --> 0:16:21.120 technology was a failure, that THEOS was depending upon the 0:16:21.240 --> 0:16:24.080 same sort of machines that the company was purporting to 0:16:24.120 --> 0:16:28.480 replace with its innovative approach, that hundreds or thousands of 0:16:28.520 --> 0:16:32.600 patients in trial locations were potentially at risk due to 0:16:32.920 --> 0:16:36.800 unreliable results, and that the company used some pretty draconian 0:16:36.920 --> 0:16:40.320 tactics to keep employees in line and prevent them speaking 0:16:40.360 --> 0:16:42.960 out about what was going on. It's about as bad 0:16:42.960 --> 0:16:46.280 an outcome as you could imagine. What's more, there are 0:16:46.320 --> 0:16:49.280 those who say that even if everything had worked, the 0:16:49.320 --> 0:16:52.880 whole enterprise was misguided in the first place. A Piece 0:16:53.000 --> 0:16:56.680 and Wired by Gnoam Cohen cites a couple of those people. 0:16:56.960 --> 0:17:00.080 The piece has the title the other Big lesson we 0:17:00.120 --> 0:17:04.920 should learn from thoroughness. Cohen mentions Faye Flam, who wrote 0:17:04.960 --> 0:17:07.920 a piece in Bloomberg that argued thoroness was tapping into 0:17:08.000 --> 0:17:12.200 another deep human desire, the illusion of controlling our own 0:17:12.240 --> 0:17:16.120 destiny through thous We could end up getting our own 0:17:16.119 --> 0:17:19.960 test results and then apply our own interpretations to them. 0:17:20.000 --> 0:17:22.080 Perhaps we would interpret them in a way that is 0:17:22.119 --> 0:17:25.040 most comforting to us, or one that seems to align 0:17:25.080 --> 0:17:29.800 with our preconceived ideas about our health. This isn't exactly 0:17:29.840 --> 0:17:33.320 the best way to handle a medical issue. Surely it 0:17:33.359 --> 0:17:37.320 makes more sense to have a trained medical professional provide 0:17:37.320 --> 0:17:41.679 an unbiased, objective interpretation of test results that gives you 0:17:41.720 --> 0:17:44.360 the best chance to take appropriate actions that will help 0:17:44.400 --> 0:17:48.439 you lead a better, healthier life. So yeah, it was 0:17:48.480 --> 0:17:52.200 a really compelling sales pitch, no wonder so many people 0:17:52.240 --> 0:17:55.000 were on board. If it worked, it would be cheaper, 0:17:55.359 --> 0:17:58.960 less painful, more convenient, and at least on the surface, 0:17:59.040 --> 0:18:02.600 more empowering than the established method. It was the sort 0:18:02.600 --> 0:18:05.800 of thing we'd want to believe in, so people did. 0:18:06.720 --> 0:18:10.359 I think autonomous cars are following a similar trajectory. Now, 0:18:10.359 --> 0:18:12.840 to be clear, I feel that a lot of great 0:18:12.840 --> 0:18:16.480 work has been done in autonomous cars. They are much 0:18:16.640 --> 0:18:20.680 further along than a mythical blood testing device that only 0:18:20.720 --> 0:18:23.440 ever got approval for performing one type of blood test 0:18:23.800 --> 0:18:25.560 when it was supposed to be able to run more 0:18:25.600 --> 0:18:28.000 than one hundred of them, but we still have a 0:18:28.040 --> 0:18:32.320 long way to go. Unfortunately, because of our experiences dealing 0:18:32.320 --> 0:18:36.000 with truly amazing tech and the expectation that, of course, 0:18:36.040 --> 0:18:38.760 technology can take care of the problem, we've had some 0:18:38.880 --> 0:18:44.919 high profile accidents that prove this isn't the most reliable philosophy. Again, 0:18:45.240 --> 0:18:47.600 it would be understandable to put a lot of faith 0:18:47.800 --> 0:18:51.680 in the tech. Google's self driving cars, which have been 0:18:51.760 --> 0:18:56.040 pioneers in the field, famously operated at first in secret 0:18:56.160 --> 0:19:00.280 and then openly for hundreds of driving miles without a 0:19:00.440 --> 0:19:03.639 single accident, or at least that was the official story. 0:19:04.119 --> 0:19:07.000 There have been a few accidents, most of which were 0:19:07.040 --> 0:19:10.240 likely the fault of a human driver, either the safety 0:19:10.280 --> 0:19:14.360 operator in the autonomous vehicle or the driver of another car, 0:19:14.800 --> 0:19:18.200 but later reports suggested that there were some serious accidents, 0:19:18.440 --> 0:19:20.159 at least a few of which were caused by the 0:19:20.200 --> 0:19:24.480 autonomous driving system behaving in an unexpected way. The company 0:19:24.600 --> 0:19:28.320 kept these accidents quiet, and so there was an unearned 0:19:28.400 --> 0:19:33.639 expectation of safety with this tech. Then enter Tesla and 0:19:33.680 --> 0:19:37.159 the autopilot feature in the company's electric vehicles. While the 0:19:37.200 --> 0:19:40.000 company issued a statement that made it clear that this 0:19:40.040 --> 0:19:43.639 feature wasn't supposed to replace a human driver. That didn't 0:19:43.680 --> 0:19:46.480 stop people from trying it out that way. Most of 0:19:46.520 --> 0:19:50.119 those people didn't have any problems, but in at least 0:19:50.160 --> 0:19:53.199 a couple of cases, drivers using autopilot ended up in 0:19:53.320 --> 0:19:57.360 tragic situations. One of those was the case of Joshua 0:19:57.440 --> 0:20:01.720 Brown in two thousand sixteen, his s Tesla crashed into 0:20:01.720 --> 0:20:05.159 a semi truck. Brown had been using the autopilot feature. 0:20:05.440 --> 0:20:08.919 According to the vehicle's datalogues. Out of the thirty seven 0:20:08.960 --> 0:20:13.000 minutes Brown had the autopilot feature turned on, his hands 0:20:13.000 --> 0:20:16.320 were on the wheel for just twenty five seconds total, 0:20:16.880 --> 0:20:20.040 in direct violation of the policy that Tesla had set. 0:20:20.440 --> 0:20:24.240 The company stressed the autopilot isn't meant to replace human 0:20:24.320 --> 0:20:27.320 drivers and that the cars driver should have had their 0:20:27.320 --> 0:20:30.359 hands on the wheel at all times. The second fatal 0:20:30.400 --> 0:20:34.159 incident happened in March two thousand eighteen, when way Walter 0:20:34.359 --> 0:20:38.920 Huang's Model X veered into a highway safety barrier. Recently, 0:20:39.160 --> 0:20:42.439 his family sued Tesla, alleging the company was aware of 0:20:42.480 --> 0:20:45.400 the dangers of the features, that Huang had been operating 0:20:45.400 --> 0:20:48.920 the vehicle within the parameters of autopilot, and that Tesla 0:20:48.960 --> 0:20:52.320 had been using drivers like Wong to beta test changes 0:20:52.600 --> 0:20:55.439 to the feature in the wild. That suit is just 0:20:55.480 --> 0:20:58.520 getting started as I record this episode, and I don't 0:20:58.520 --> 0:21:01.000 mean to pick on Tesla, after all. I started this 0:21:01.040 --> 0:21:03.879 by talking about how Google kept several accidents on the 0:21:03.960 --> 0:21:07.480 q T. One of Uber's self driving cars in its 0:21:07.520 --> 0:21:10.600 beta test program in Arizona, struck and killed a pedestrian 0:21:10.640 --> 0:21:14.200 in March two thousand eighteen. State prosecutors decided that Uber 0:21:14.359 --> 0:21:17.680 isn't liable for the accident, but that the safety operator 0:21:17.680 --> 0:21:20.320 who was in the car might bear some responsibility for 0:21:20.400 --> 0:21:23.760 failing to act before the accident. According to venture Beat, 0:21:24.080 --> 0:21:27.159 the operator was streaming a video of the voice and 0:21:27.200 --> 0:21:31.080 watching that rather than the road. It's quite possible that 0:21:31.160 --> 0:21:34.520 companies pushing autonomous car technology are doing their best to 0:21:34.600 --> 0:21:38.200 keep incidents quiet in an effort to avoid government regulations 0:21:38.280 --> 0:21:42.360 and interference which could threaten the profitability of such a pursuit. 0:21:42.880 --> 0:21:45.679 But at the same time, these high profile incidents have 0:21:45.840 --> 0:21:49.480 dealt a blow to consumer confidence about the technology in general, 0:21:49.600 --> 0:21:52.880 and they really reinforce that driving is more complicated than 0:21:52.920 --> 0:21:56.359 just staying within your lane and breaking if something is 0:21:56.400 --> 0:21:59.679 in the way. Before I wrap up this section, I 0:21:59.760 --> 0:22:03.000 do want to also mention that, at least according to Tesla, 0:22:03.240 --> 0:22:06.560 the autopilot feature has proven to be safer than human 0:22:06.680 --> 0:22:11.720 drivers operating vehicles unassistant. According to Tesla's report, there was 0:22:11.800 --> 0:22:15.919 one accident per two point eight seven million miles driven 0:22:16.200 --> 0:22:20.119 where autopilot was engaged and one accident per one point 0:22:20.320 --> 0:22:24.040 seven six million miles driven when it wasn't, and that 0:22:24.080 --> 0:22:28.159 according to government statistics, the average is an automobile crash 0:22:28.359 --> 0:22:33.440 every four hundred thirty six thousand miles. However, skeptical researchers 0:22:33.440 --> 0:22:36.479 have found that Tesla hasn't always been honest or at 0:22:36.480 --> 0:22:40.400 the very least correct about safety reports. A firm called 0:22:40.520 --> 0:22:45.080 Quality Control Systems sued the United States National Highway Traffic 0:22:45.200 --> 0:22:49.960 Safety Administration, or in ht s A, in order to 0:22:49.960 --> 0:22:54.640 get the data the agency said had proved that Tesla's 0:22:54.640 --> 0:22:58.440 autopilot had cut back on crashes by fort In fact, 0:22:58.840 --> 0:23:02.200 QCs found at in cases in which all the data 0:23:02.320 --> 0:23:05.400 was available, which was just a fraction of the cases 0:23:05.480 --> 0:23:08.880 the nh t s A had used to come up 0:23:08.920 --> 0:23:13.159 with that mark, the auto steer feature on autopilot actually 0:23:13.200 --> 0:23:17.840 increased crash rates by So what does all this mean. 0:23:18.440 --> 0:23:20.520 I'll get back to that in a second, but first 0:23:20.600 --> 0:23:31.040 let's take a quick break. So am I saying you 0:23:31.080 --> 0:23:35.720 shouldn't believe anyone, Well, that's not quite it. I think 0:23:35.720 --> 0:23:39.120 a better thing to say is don't accept anything at 0:23:39.119 --> 0:23:43.439 face value. Sometimes people can just be wrong about stuff. 0:23:43.840 --> 0:23:45.919 It happens to me more often than I care to admit. 0:23:46.400 --> 0:23:48.840 But the responsible thing to do in those cases is 0:23:48.880 --> 0:23:51.160 to own up to the mistake and correct it where 0:23:51.160 --> 0:23:55.119 you can. I am certain that at least some people 0:23:55.280 --> 0:23:58.359 who thought they were onto some sort of free energy device, 0:23:58.400 --> 0:24:02.480 for example, really did. They were onto something. Others might 0:24:02.480 --> 0:24:05.800 have suspected that what they were pursuing was impossible, but 0:24:05.880 --> 0:24:09.080 they had already invested too much to back out at 0:24:09.119 --> 0:24:12.640 any rate. Whether it's a perpetual motion machine or an 0:24:12.720 --> 0:24:16.280 over unity engine, the simple fact is that these devices 0:24:16.320 --> 0:24:21.240 have never been proven to actually work as described supporters say. 0:24:21.320 --> 0:24:25.200 This is because there are powerful entities like petroleum companies 0:24:25.400 --> 0:24:28.160