WEBVTT - Killer Robots and the Future of War 0:00:15.250 --> 0:00:28.890 Pushkin. You're listening to Brave New Planet, a podcast about 0:00:28.930 --> 0:00:33.090 amazing new technologies that could dramatically improve our world. Or 0:00:33.490 --> 0:00:36.050 if we don't make wise choices, could leave us a 0:00:36.050 --> 0:00:40.730 lot worse off. Utopia or dystopia. It's up to us. 0:00:47.690 --> 0:00:52.610 On September twenty sixth, nineteen eighty three, the world almost 0:00:52.650 --> 0:00:56.570 came to an end. Just three weeks earlier, the Soviet 0:00:56.690 --> 0:01:00.970 Union had shot down Korean Airlines Flight Double O seven, 0:01:01.330 --> 0:01:05.050 a passenger plane with two hundred and sixty nine people aboard. 0:01:05.290 --> 0:01:08.650 I'm coming before you tonight about the Korean Airline massacre. 0:01:09.210 --> 0:01:12.690 President Ronald Reagan addressed the nation the attack by the 0:01:12.730 --> 0:01:16.530 Soviet Union against two hundred and sixty nine innocent men, women, 0:01:16.530 --> 0:01:21.210 and children aboard an unarmed Korean passenger plane. This crime 0:01:21.250 --> 0:01:24.970 against humanity must never be forgotten here or throughout the world. 0:01:25.610 --> 0:01:29.330 Cold War tensions escalated, with the two nuclear powers on 0:01:29.530 --> 0:01:35.890 high alert. World War three felt frighteningly possible. Then, on 0:01:35.930 --> 0:01:39.810 September twenty sixth, in a command center outside of Moscow, 0:01:40.370 --> 0:01:44.890 an alarm sounded. The Soviet Union's early warning system reported 0:01:44.930 --> 0:01:49.210 the launch of multiple intercontinental ballistic missiles from bases in 0:01:49.250 --> 0:01:54.730 the United states. Statislav Petrov, a forty four year old 0:01:54.770 --> 0:01:58.330 member of the Soviet Air Defense Forces, was the duty 0:01:58.370 --> 0:02:02.930 officer that night. His role was to alert Moscow that 0:02:03.010 --> 0:02:08.170 an attack was under way, likely triggering Soviet nuclear retaliation 0:02:08.650 --> 0:02:14.250 and all out war. Petrov spoke with BBC News in 0:02:14.330 --> 0:02:18.610 twenty thirteen. The sirens sounded very loudly, and I just 0:02:18.650 --> 0:02:21.490 sat there for a few seconds, staring at the screen 0:02:21.530 --> 0:02:26.130 with the word launch displayed in bold red letters. A 0:02:26.210 --> 0:02:30.010 minute later, the siren went off again. The second missile 0:02:30.090 --> 0:02:33.570 was launched, then the third, and the fourth, and the fifth. 0:02:34.890 --> 0:02:38.570 The computers changed their alerts from launch to missile strike. 0:02:39.810 --> 0:02:44.090 Petrov's instructions were clear, report the attack on the motherland, 0:02:44.330 --> 0:02:48.450 but something didn't make sense. If the US were attacking, 0:02:48.810 --> 0:02:53.570 why only five missiles rather than an entire fleet? And 0:02:53.610 --> 0:02:57.130 then I made my decision. I would not trust the computer. 0:02:57.810 --> 0:03:01.770 I picked up the telephone handset, spoke to my superiors 0:03:01.810 --> 0:03:06.370 and reported that the alarm was false. But I myself 0:03:07.010 --> 0:03:11.090 was not sure. Until the very last moment. I knew 0:03:11.210 --> 0:03:14.090 perfectly well that nobody would be able to correct my 0:03:14.210 --> 0:03:20.850 mistake if I had made one. Petrov, of course, was right. 0:03:21.290 --> 0:03:23.730 The false alarm was later found to be the result 0:03:23.770 --> 0:03:28.050 of a rare and unanticipated coincidence sunlight glinting off high 0:03:28.050 --> 0:03:32.130 altitude clouds over North Dakota at just the right angle 0:03:32.130 --> 0:03:36.730 to fool the Soviet satellites. Statislav Petrov's story comes up 0:03:36.770 --> 0:03:40.690 again and again in discussions of how far we should 0:03:40.730 --> 0:03:46.130 go and turning over important decisions, especially life and death decisions, 0:03:46.450 --> 0:03:50.610 to artificial intelligence. It's not an easy call. Think about 0:03:50.610 --> 0:03:55.130 the split second decisions and avoiding a highway collision. Who 0:03:55.170 --> 0:03:58.970 will ultimately do better a tire driver or a self 0:03:59.050 --> 0:04:04.090 driving car? Nowhere is the question more fraught than on 0:04:04.090 --> 0:04:08.930 the battlefield. As technology evolves, should weapons systems be given 0:04:08.970 --> 0:04:12.370 the power to make life and death decisions? Or do 0:04:12.410 --> 0:04:16.010 we need to ensure there's always a human a Stanislav 0:04:16.090 --> 0:04:20.850 Petrov in the loop. Some people, including winners of the 0:04:20.850 --> 0:04:24.690 Nobel Peace Prize, say that weapons should never be allowed 0:04:24.730 --> 0:04:28.170 to make their own decisions about who or what to attack. 0:04:29.090 --> 0:04:33.250 They're calling for a ban on what they call killer robots. 0:04:34.290 --> 0:04:40.770 Others think that idea is well meaning but naive. Today's 0:04:40.810 --> 0:04:46.890 big question lethal autonomous weapons. Should they ever be allowed? 0:04:47.810 --> 0:05:00.370 If so, when, if not, can we stop them? My 0:05:00.450 --> 0:05:02.770 name is Eric Lander. I'm a scientist who works on 0:05:02.810 --> 0:05:05.690 ways to improve human health. I helped lead the Human 0:05:05.770 --> 0:05:08.850 Genome Project, and today I lead the Broad Institute of 0:05:09.450 --> 0:05:14.010 and Harvard. In the twenty first century, powerful technologies have 0:05:14.050 --> 0:05:17.690 been appearing at a breathtaking pace, related to the Internet, 0:05:17.810 --> 0:05:23.570 artificial intelligence, genetic engineering, and more. They have amazing potential upsides, 0:05:24.210 --> 0:05:26.930 but we can't ignore the risks that come with them. 0:05:27.010 --> 0:05:31.770 The decisions aren't just up to scientists or politicians, whether 0:05:31.810 --> 0:05:34.650 we like it or not, we all of us are 0:05:34.690 --> 0:05:38.570 the stewards of a brave New Planet. This generation's choices 0:05:38.610 --> 0:05:48.170 will shape the future as never before. Coming up on 0:05:48.210 --> 0:05:53.130 today's episode of Brave New Planet fully autonomous lethal weapons 0:05:53.690 --> 0:05:59.650 or killer robots, we hear from a fighter pilot about 0:05:59.690 --> 0:06:02.290 why it might make sense to have machines in charge 0:06:02.530 --> 0:06:06.810 of some major battlefield decisions. I know people who have 0:06:07.410 --> 0:06:13.210 killed civilians, and in all cases where people made mistakes, 0:06:13.610 --> 0:06:16.570 it was just too much information. Things were happening too fast. 0:06:16.890 --> 0:06:19.570 I speak with one of the world's leading robo ethesis. 0:06:19.810 --> 0:06:23.210 Robots will make mistakes too, but hopefully, if done correctly, 0:06:23.290 --> 0:06:25.970 they will make far far less mistakes than human beings. 0:06:26.730 --> 0:06:30.890 We'll hear about some of the possible consequences of autonomous weapons. 0:06:31.370 --> 0:06:36.370 Algorithms interacting at machine speed faster than humans couldn't respond 0:06:36.930 --> 0:06:40.490 might result in accidents, and that's something like a flash war. 0:06:41.170 --> 0:06:43.970 I'll speak with a leader from Human Rights Watch. The 0:06:44.130 --> 0:06:47.770 campaign to stop Killer Robots is seeking new international lure 0:06:47.930 --> 0:06:51.490 in the form of a new treaty. And we'll talk 0:06:51.530 --> 0:06:55.170 with former Secretary of Defense Ash Carter. Because I'm the 0:06:55.170 --> 0:06:57.050 guy who asked to go out the next morning after 0:06:57.090 --> 0:07:00.450 some women and children have been accidentally killed. As suppose 0:07:00.490 --> 0:07:03.330 I go out there, Eric and I say, oh, I 0:07:03.370 --> 0:07:05.490 don't know how it happened. The machine did it. I 0:07:05.530 --> 0:07:09.970 would be crucified. I should be crucified. So stay with us. 0:07:13.770 --> 0:07:21.690 Chapter one, Stanley the self driving Car. Not long after 0:07:21.730 --> 0:07:25.530 the first general purpose computers were invented in the nineteen forties, 0:07:26.010 --> 0:07:31.090 some people began to dream about fully autonomous robots, machines 0:07:31.130 --> 0:07:35.410 that used their electronic brains to navigate the world, make decisions, 0:07:35.450 --> 0:07:39.690 and take actions. Not surprisingly, some of those dreamers were 0:07:39.690 --> 0:07:44.370 in the US Department of Defense, specifically the Defense Advanced 0:07:44.410 --> 0:07:48.850 Research Projects Agency or DARPA, the visionary unit behind the 0:07:48.890 --> 0:07:52.570 creation of the Internet. They saw a lot of potential 0:07:52.650 --> 0:07:56.250 for automating battlefields, but they knew it might take decades. 0:07:57.050 --> 0:08:01.050 In the nineteen sixties, DARPA funded the Stanford Research inst 0:08:01.570 --> 0:08:05.850 to build Shaky the Robots, a machine that used cameras 0:08:06.210 --> 0:08:09.410 to move about a laboratory. In the nineteen eight ease, 0:08:09.530 --> 0:08:13.690 it supported universities to create vehicles that could follow lanes 0:08:13.770 --> 0:08:18.250 on a road. By the early two thousands, DARPA decided 0:08:18.290 --> 0:08:22.650 that computers had reached the point that fully autonomous vehicles 0:08:23.090 --> 0:08:27.050 able to navigate the real world might finally be feasible. 0:08:27.690 --> 0:08:32.090 To find out, DARPA decided to launch a race. I 0:08:32.210 --> 0:08:35.650 talked to someone who knew a lot about it. My 0:08:35.730 --> 0:08:38.010 name is Sebastian Thrun. I mean the smartest person on 0:08:38.010 --> 0:08:41.850 the planet and the best looking. That's kidding. Sebastian Thrun 0:08:42.010 --> 0:08:46.570 gained recognition when his autonomous car, a modified Volkswagon with 0:08:46.610 --> 0:08:49.530 a computer in the trunk and sensors on the roof, 0:08:50.170 --> 0:08:53.170 was the first to win the DARPA Grand Challenge. A 0:08:53.290 --> 0:08:58.090 dun Challenge was his momentous government sponsors robot raises epic RaSE, 0:08:58.610 --> 0:09:00.890 can you bid a robot that can navigate one hundred 0:09:00.930 --> 0:09:05.210 and thirty punishing miles through the Mohabi Desert and the 0:09:05.290 --> 0:09:08.570 best robot like seven miles and then literally end up 0:09:08.810 --> 0:09:12.250 in many many mini researchers had concluded can't be done. 0:09:12.250 --> 0:09:14.930 In fact, many of my colleagues told me I'm going 0:09:14.970 --> 0:09:17.770 to waste my time and my name if I engaged 0:09:17.770 --> 0:09:19.810 in this kind of super hard race. And that made 0:09:19.850 --> 0:09:22.010 you more interested in doing it, of course, and so 0:09:22.130 --> 0:09:25.730 you built Stanley. Yeah, So my students built Stanley and 0:09:26.050 --> 0:09:29.730 started as a class And Stanford students are great. If 0:09:29.730 --> 0:09:31.450 you tell them go to the moon in two months, 0:09:31.490 --> 0:09:33.890 they're going to go to the moon. So then two 0:09:33.930 --> 0:09:38.770 thousand or five, the actual government sponsored race, how did 0:09:38.850 --> 0:09:43.410 Stanley do. We came in first, so we are focused 0:09:43.730 --> 0:09:46.930 insanely strongly on software and specifically on machine learning, and 0:09:46.970 --> 0:09:49.610 that differentiated as from pretty much every other team that 0:09:49.730 --> 0:09:51.570 focused on hardware. But the way I look at this 0:09:51.610 --> 0:09:54.370 is there were five teams that finished this ruling race 0:09:54.450 --> 0:09:57.250 within one year, and it's the community of the people 0:09:57.410 --> 0:10:01.170 that build all these machines that really won. So nobody 0:10:01.170 --> 0:10:04.810 made it a mile in the first race, and five 0:10:04.930 --> 0:10:07.450 different teams made it more than one hundred and thirty 0:10:07.450 --> 0:10:10.010 miles through the desert, just a year later, Yeah, that's 0:10:10.090 --> 0:10:13.650 kind of amazing to me. That just showed how fast 0:10:13.730 --> 0:10:18.010 this technology can possibly evolve. And what's happened since then? 0:10:18.490 --> 0:10:22.250 I worked at Google for a vile and eventually this guy, 0:10:22.730 --> 0:10:25.130 Larry Page came to me and says, hey, Sebastian, I 0:10:25.170 --> 0:10:27.450 thought about this long and heart. We should build a 0:10:27.450 --> 0:10:29.210 self driving car they can drive on all streets in 0:10:29.210 --> 0:10:32.370 the world. And with my entire authority, I said that 0:10:32.570 --> 0:10:35.930 cannot be done. We just had driven a desert raised 0:10:36.250 --> 0:10:39.050 there was never pedestrians and bicycles and all the other 0:10:39.090 --> 0:10:41.170 people that we could kill in the environment. And for me, 0:10:41.450 --> 0:10:44.290 just the sheer imagination we would drive a self driving 0:10:44.290 --> 0:10:47.530 car to San Francisco sounded always like a crime. So 0:10:47.570 --> 0:10:50.890 you told Larry Page, one of the two co founders 0:10:51.010 --> 0:10:54.050 of Google, that the idea of building a self driving 0:10:54.130 --> 0:10:58.090 car that could navigate anywhere was just not Yeah, feelous. 0:10:58.170 --> 0:11:00.850 Later came back and said, he Sebastian, look, I trust you, 0:11:00.850 --> 0:11:03.170 you're the expert, but I want to explain Eric Schmidt, 0:11:03.210 --> 0:11:05.330 then the Google CEO, and it's my co founder, surgery brain, 0:11:05.970 --> 0:11:08.250 why it can be done. Can you give me the 0:11:09.090 --> 0:11:13.610 technical reason? So I went home in agony, thinking about 0:11:14.690 --> 0:11:17.050 what is the technical reason why it can be done? 0:11:17.730 --> 0:11:19.970 And I got back the next day and I said, so, 0:11:20.050 --> 0:11:23.290 what is it? And I said, I can't think of 0:11:23.370 --> 0:11:27.330 any and Lomi Hoold. Eighteen months later, roughly ten engineers 0:11:27.810 --> 0:11:32.170 we drove pretty much every street in California. Today, autonomous 0:11:32.250 --> 0:11:36.290 technology is changing the transportation industry. About ten percent of 0:11:36.330 --> 0:11:39.410 cars sold in the US are already capable of at 0:11:39.490 --> 0:11:44.010 least partly guiding themselves down the highway. In twenty eighteen, 0:11:44.450 --> 0:11:48.890 Google's self driving car company Weymo launched a self driving 0:11:48.970 --> 0:11:53.690 taxi service in Phoenix, Arizona, initially with human backup drivers 0:11:53.730 --> 0:11:58.330 behind every wheel, but now sometimes even without. I asked 0:11:58.330 --> 0:12:03.210 Sebastian why he thinks this matters. We lose more than 0:12:03.250 --> 0:12:07.610 a million people in traffic accidents every year, almost exclusively 0:12:08.210 --> 0:12:11.130 to us not pay attention. When it was eighteen, my 0:12:11.530 --> 0:12:15.410 best friend died in a traffic accident and it was 0:12:15.450 --> 0:12:19.090 a split second poor decision from his friend who was 0:12:19.170 --> 0:12:22.450 driving in who also died. To me, this is just unacceptable. 0:12:23.530 --> 0:12:28.850 Beyond safety, Sebastian sees many other advantages for autonomy. During 0:12:28.850 --> 0:12:32.610 a commute, you can do something else that means you're 0:12:32.730 --> 0:12:35.610 probably willing to commute further distances. You could sleep, or 0:12:35.650 --> 0:12:38.730 watch the movie, or do email. And then eventually people 0:12:38.770 --> 0:12:42.250 can use cars that today can't operate them blind people, 0:12:42.810 --> 0:12:46.890 old people, children, babies. I mean, there's an entire spectrum 0:12:46.930 --> 0:12:49.050 of people that are kindly excluded. They would now be 0:12:49.090 --> 0:12:57.090 able to be mobile. Chapter two the Tomahawk so darpest 0:12:57.090 --> 0:13:00.130 efforts over the decades helped give rise to the modern 0:13:00.250 --> 0:13:04.970 self driving car industry, which promises to make transportations safer, 0:13:05.330 --> 0:13:10.330 more efficient, and more accessible. But the agencies primary motivation 0:13:10.850 --> 0:13:14.370 was to bring autonomy to a different challenge, the battlefield. 0:13:14.770 --> 0:13:17.530 I traveled to Chapel Hill, North Carolina, to meet with 0:13:17.570 --> 0:13:19.970 someone who spends a lot of time thinking about the 0:13:20.010 --> 0:13:24.490 consequences of autonomous technology. We both serve on a civilian 0:13:24.530 --> 0:13:28.810 advisory board for the Defense Department. My name is Missy Cummings. 0:13:29.050 --> 0:13:33.210 I'm a professor of electrical and computer engineering at Duke University, 0:13:33.770 --> 0:13:36.250 and I think one of the things that people find 0:13:36.410 --> 0:13:39.090 most interesting about me is that I was one of 0:13:39.090 --> 0:13:43.010 the US military's first female fighter pilots in the Navy. 0:13:43.130 --> 0:13:46.250 Did you always want to be a fighter pilot? So 0:13:46.290 --> 0:13:48.890 when I was growing up, I did not know that 0:13:48.970 --> 0:13:52.570 women could be pilots, and indeed, when I was growing up, 0:13:52.570 --> 0:13:55.970 women could it be pilots. And it wasn't until the 0:13:56.850 --> 0:14:01.530 late seventies that women actually became pilots in the military. 0:14:02.450 --> 0:14:06.890 So I went to college. In nineteen eighty four, I 0:14:06.970 --> 0:14:09.290 was at the Naval Academy, and then of course, in 0:14:09.370 --> 0:14:11.490 nineteen eighty six Top Gun came out, and then I 0:14:12.170 --> 0:14:14.010 know who doesn't want to be a pilot After you 0:14:14.050 --> 0:14:17.730 see the movie Top Gun. Missy is tremendously proud of 0:14:17.770 --> 0:14:20.290 the eleven years she spent in the Navy, but she 0:14:20.370 --> 0:14:23.330 also acknowledges the challenges of being part of that first 0:14:23.370 --> 0:14:27.370 generation of woman fighter pilots. It's no secret that the 0:14:27.410 --> 0:14:31.610 reason I left the military was because of the hostile 0:14:31.650 --> 0:14:35.050 attitude towards women. None of the women in that first 0:14:35.050 --> 0:14:37.130 group stayed in to make it a career. The guys 0:14:37.130 --> 0:14:41.130 were very angry that we were there, and I decided 0:14:41.130 --> 0:14:44.370 to leave when they started sabotaging my flight gear. I 0:14:44.490 --> 0:14:47.530 just thought, this is too much. If something really bad happened, 0:14:47.610 --> 0:14:51.370 you know, I would die. When Missy Cummings left the Navy, 0:14:51.610 --> 0:14:55.770 she decided to pursue a PhD in Human Machine interaction. 0:14:56.010 --> 0:14:59.970 In my last three years flying IF eighteens, there were 0:15:00.010 --> 0:15:02.890 about thirty six people I knew that died, about one 0:15:02.930 --> 0:15:06.090 person a month. They were all training accidents. It just 0:15:06.130 --> 0:15:09.850 really struck me how many people were dying because the 0:15:10.130 --> 0:15:13.250 design of the airplane just did not go with the 0:15:13.370 --> 0:15:16.850 human tendencies. And so I decided to go back to 0:15:16.890 --> 0:15:19.570 school to find out what can be done about that. 0:15:20.170 --> 0:15:23.330 So I went to finish my PhD at the University 0:15:23.370 --> 0:15:26.650 of Virginia, and then I spent the next ten years 0:15:26.650 --> 0:15:30.490 at MT learning my craft. The person I am today 0:15:30.730 --> 0:15:34.370 is half because of the Navy and half because of MIGHT. Today, 0:15:34.570 --> 0:15:38.050 Missy is a Duke University where she runs the Humans 0:15:38.050 --> 0:15:42.250 an Autonomy Lab, or for short HOW. It's a nod 0:15:42.290 --> 0:15:46.290 to the sentient computer that goes rogue in Stanley Kubrick's 0:15:46.330 --> 0:15:50.410 film two thousand and one, A Space Odyssey. This mission 0:15:50.530 --> 0:15:53.330 is too important for me to allow you to jeopardize it. 0:15:55.130 --> 0:15:58.450 I don't know what you're talking about. How. I know 0:15:58.530 --> 0:16:01.650 that you and Frank were planning to disconnect me, and 0:16:01.730 --> 0:16:05.570 I'm afraid that's something I cannot allow to happen. And 0:16:05.650 --> 0:16:10.450 so I intentionally named my lab how so that we 0:16:10.450 --> 0:16:13.330 were there to stop that from happening. Right, I had 0:16:13.370 --> 0:16:17.290 seen many friends die, not because the robot became sentient, 0:16:17.450 --> 0:16:21.530 in fact, because the designers of the automation really had 0:16:21.610 --> 0:16:24.610 no clue how people would or would not use this technology. 0:16:25.410 --> 0:16:30.570 It is my life's mission statement to develop human collaborative 0:16:30.610 --> 0:16:34.090 computer systems that work with each other to achieve something 0:16:34.130 --> 0:16:37.370 greater than either would alone. The Humans and Autonomy Lab 0:16:37.450 --> 0:16:42.450 works on the interactions between humans and machines across many fields, 0:16:43.050 --> 0:16:46.930 but given her background, Missy's thought a lot about how 0:16:46.970 --> 0:16:51.850 technology has changed the relationship between humans and their weapons. 0:16:52.250 --> 0:16:56.490 There's a long history of us distancing ourselves from our actions. 0:16:57.330 --> 0:17:00.090 We want to shoot somebody, we wanted to shoot them 0:17:00.090 --> 0:17:03.090 with bows and arrows. We wanted to drop bombs from 0:17:03.130 --> 0:17:05.970 five miles over a target. We want cruise muscles that 0:17:06.010 --> 0:17:09.050 can kill you from another country. Right, it is human 0:17:09.290 --> 0:17:14.250 nature to back that distance up, Missy season inherent tension. 0:17:15.050 --> 0:17:19.810 On one hand, technology distances ourselves from killing. On the 0:17:19.810 --> 0:17:23.370 other hand, technology is letting us design weapons that are 0:17:23.410 --> 0:17:28.410 more accurate and less indiscriminate in their killing. Missy rotor 0:17:28.530 --> 0:17:32.530 PhD thesis about the Tomahawk missile, an early precursor of 0:17:32.610 --> 0:17:37.130 the autonomous weapons systems being developed today. The Tomahawk missile 0:17:37.210 --> 0:17:41.250 has these stored maps in its brain, and as its 0:17:41.250 --> 0:17:44.250 skimming along the nap of the Earth, it compares the 0:17:44.290 --> 0:17:46.930 pictures that it's taking with its pictures and its database 0:17:47.330 --> 0:17:50.530 to decide how to get to its target. This Tomahawk 0:17:50.610 --> 0:17:52.450 was kind of a set it and forget it kind 0:17:52.450 --> 0:17:55.290 of thing. Once you launched it, it would follow its 0:17:55.290 --> 0:17:57.850 map to the right place and there was nobody looking 0:17:57.890 --> 0:18:02.370 over its shoulders. Well, so the Tomahawk missile that we 0:18:02.410 --> 0:18:04.610 saw in the Gulf War, that was a fire and 0:18:04.650 --> 0:18:08.730 forget missile that a target would be programmed into the 0:18:09.290 --> 0:18:11.210 sole and then it would be fired and that's where 0:18:11.210 --> 0:18:15.810 it would go. Later, around two thousand, two thousand and three, 0:18:16.690 --> 0:18:20.010 then GPS technology was coming online, and that's when we 0:18:20.050 --> 0:18:22.610 got the tactical Tomahawk, which had the ability to be 0:18:22.690 --> 0:18:27.090 redirected in flight. That success with GPS and the Tomahawk 0:18:27.170 --> 0:18:31.970 opened the military's eyes to the ability to use them 0:18:32.010 --> 0:18:36.330 in drones. Today's precision guided weapons are far more accurate 0:18:36.370 --> 0:18:39.970 than the widespread aerial bombing that occurred on all sides 0:18:39.970 --> 0:18:44.450 in World War Two, where some cities were almost entirely leveled, 0:18:44.810 --> 0:18:49.210 resulting in huge numbers of civilian casualties. In the Gulf War, 0:18:49.650 --> 0:18:56.170 Tomahawk missile attacks came to be called surgical strikes. I 0:18:56.250 --> 0:19:02.330 know people who have killed civilians, and I know people 0:19:02.370 --> 0:19:05.330 who have killed friendlies. They have dropped bombs on our 0:19:05.330 --> 0:19:10.530 own forces and killed our own people. And in all 0:19:10.610 --> 0:19:15.170 cases where people made mistakes, it was just too much information. 0:19:15.410 --> 0:19:18.730 Things were happening too fast. You've seen some pictures that 0:19:18.770 --> 0:19:21.130 you've got in a brief hours ago, and you're supposed 0:19:21.170 --> 0:19:24.330 to know that what you're seeing now through this grainy 0:19:24.410 --> 0:19:27.210 image thirty five thousand feet over a target is the 0:19:27.250 --> 0:19:30.490 same image that you're being asked to bob. The Tomahawk 0:19:30.970 --> 0:19:35.610 never missed its target. It never made a mistake unless 0:19:35.610 --> 0:19:39.250 it was programmed as a mistake. And that's old autonomy, 0:19:39.370 --> 0:19:46.530 and it's only gotten better over time. Chapter three, Kicking 0:19:46.610 --> 0:19:52.730 down Doors. The Tomahawk was just a baby step toward automation. 0:19:53.250 --> 0:19:56.130 With the ability to read maps, it could correct its course, 0:19:56.650 --> 0:20:00.570 but it couldn't make sophisticated decisions. But what happens when 0:20:00.610 --> 0:20:04.410 you start adding modern artificial intelligence? So where do you 0:20:04.410 --> 0:20:08.010 see autonomous weapons going? If you could kind of map 0:20:08.050 --> 0:20:09.930 out where are we today and where do you think 0:20:10.010 --> 0:20:13.570 we'll be ten twenty years from now. So, in terms 0:20:13.610 --> 0:20:18.130 of autonomy and weapons, by today's standards, the Tomahawk missile 0:20:18.250 --> 0:20:20.090 is still one of the best ones that we have, 0:20:20.330 --> 0:20:23.490 and it's also still one of the most advanced. Certainly, 0:20:23.530 --> 0:20:26.170 there are research arms of the military who are trying 0:20:26.290 --> 0:20:31.370 very hard to come up with new forms of autonomy. 0:20:31.930 --> 0:20:35.010 There was the Predicts that came out of Lincoln Lab, 0:20:35.050 --> 0:20:39.690 and this was basically a swarm of really tiny UAVs 0:20:39.690 --> 0:20:45.450 that could coordinate together. A ua V, an unmanned aerial 0:20:45.570 --> 0:20:50.490 vehicle is military speak for a drone. The Predicts the 0:20:50.730 --> 0:20:54.090 drones that Missy was referring to. They were commissioned by 0:20:54.090 --> 0:20:57.210 the Strategic Capabilities Office of the US Department of Defense. 0:20:57.810 --> 0:21:00.810 These tiny flying robots are able to communicate with each 0:21:00.850 --> 0:21:04.250 other and make split second decisions about how to move 0:21:04.250 --> 0:21:07.130 as a group. Many researchers, have you been using bio 0:21:07.250 --> 0:21:12.210 inspired methods? Be right? So bees have local and global intelligence. 0:21:12.530 --> 0:21:15.370 Like a group of bees, these drones are called a 0:21:15.450 --> 0:21:20.650 swarm collective intelligence on a shared mission. A human can 0:21:20.690 --> 0:21:24.250 make the big picture decision and the swarm of microdrones 0:21:24.290 --> 0:21:27.450 can then collectively decide on the most efficient way to 0:21:27.530 --> 0:21:30.810 carry out the order in the moment. I wanted to 0:21:30.850 --> 0:21:35.370 know why exactly this technology is necessary, so I went 0:21:35.410 --> 0:21:38.450 to speak to someone who I was pretty sure would know. 0:21:38.890 --> 0:21:43.010 I'm Ash Carter. Most people will probably have heard my 0:21:43.090 --> 0:21:48.250 name as the Secretary of Defense who proceeded Gimatus. You 0:21:48.290 --> 0:21:50.610 will know me in part from the fact that we 0:21:50.730 --> 0:21:53.210 knew one another way back in Oxford when we were 0:21:53.250 --> 0:21:55.890 both young scientists, and I guess agents start there. I'm 0:21:55.930 --> 0:21:58.530 a physicist. When you were doing your PhD in physics, 0:21:58.570 --> 0:22:01.450 I was doing my PhD in mathematics at Oxford. What 0:22:01.610 --> 0:22:06.170 was your thesis on? It was on quantum chrominynamics. That 0:22:06.330 --> 0:22:09.810 was the theory of quarks and gluons. And how in 0:22:09.850 --> 0:22:14.330 the world is somebody who's an expert in quantum chromodynamics 0:22:14.450 --> 0:22:18.730 become the Secretary of Defense. It's an interesting story. The 0:22:18.970 --> 0:22:24.610 people who were the seniors in my field of physics, 0:22:25.010 --> 0:22:28.250 the mentors, so to speak, were all members of the 0:22:28.330 --> 0:22:33.890 Manhattan Project generation. They had built the bomb during World 0:22:33.930 --> 0:22:38.330 War Two, and they were proud of what they'd done 0:22:38.890 --> 0:22:42.530 because they believed that it had ended the war with 0:22:42.650 --> 0:22:45.970 fewer casualties than otherwise there would have been in a 0:22:46.010 --> 0:22:49.130 full scale of invasion of Japan, and also that it 0:22:49.170 --> 0:22:51.170 had kept the peace through the Cold War, so they 0:22:51.170 --> 0:22:53.610 were proud of it. However, they knew there was a 0:22:53.690 --> 0:22:57.570 dark side, and they conveyed to me that it was 0:22:58.170 --> 0:23:03.810 my responsibility as a scientist to be involved in these matters. 0:23:04.450 --> 0:23:08.690 And the technology doesn't determine what the balance of good 0:23:08.690 --> 0:23:12.330 and bad is. We human beings do. That was the lesson, 0:23:12.730 --> 0:23:16.130 and so that's what got me started, and then my 0:23:16.250 --> 0:23:19.250 very first Pentagon job, which was in nineteen eighty one, 0:23:19.570 --> 0:23:22.330 right through until the last time I walked out of 0:23:22.370 --> 0:23:27.010 the Pentagon of Sectary Defense, which was January of twenty seventeen. Now, 0:23:27.330 --> 0:23:32.330 when you were secretary, there was a Strategic Capabilities Office 0:23:32.650 --> 0:23:38.330 that it's been publicly reported, was experimenting with using drones 0:23:38.890 --> 0:23:43.410 to make swarms of drones that could do things, communicate 0:23:43.450 --> 0:23:46.890 with each other, make formations. Why would you want such things? 0:23:46.890 --> 0:23:49.210 So it's a good question. Here's what you do with 0:23:49.250 --> 0:23:52.130 the drone like that, You put a jammer on it, 0:23:52.210 --> 0:23:56.210 a little radio beacon, and you fly it right into 0:23:56.250 --> 0:24:01.450 the eye of a enemy radar. So all that radar 0:24:01.570 --> 0:24:06.130 c's is the energy emitted by that little drone, and 0:24:06.170 --> 0:24:10.730 it's essentially dazzled or blinded. If there's one big drone, 0:24:11.330 --> 0:24:14.770 that radar is precious enough that the defenders going to 0:24:14.850 --> 0:24:18.250 shoot that drone down. But if you have so many 0:24:18.290 --> 0:24:21.690 out there, the enemy can't afford to shoot them all down. 0:24:22.450 --> 0:24:26.530 And since they are flying right up to the radar, 0:24:26.850 --> 0:24:29.770 they don't have to be very powerful. So there's an 0:24:29.770 --> 0:24:34.810 application where lots of little drones can have the effect 0:24:34.970 --> 0:24:39.250 of nullifying enemy radar. That's a pretty big deal for 0:24:39.290 --> 0:24:42.810 a few little, little microdrones. To learn more, I went 0:24:42.890 --> 0:24:45.610 to speak with Paul Shari. Paul's the director of the 0:24:45.610 --> 0:24:49.250 Technology and National Security Program at the Center for a 0:24:49.290 --> 0:24:52.850 New American Security. Before that, he worked for Ash Carter 0:24:52.930 --> 0:24:57.290 at the Pentagon studying autonomous weapons and a recently authored 0:24:57.290 --> 0:25:01.570 a book called Army of None, Autonomous Weapons and the 0:25:01.650 --> 0:25:05.810 Future of War. Paul's interest in autonomous weapons began when 0:25:05.810 --> 0:25:09.610 he served in the Army. I enlisted in the Army 0:25:09.850 --> 0:25:12.850 to become an Army ranger. That was June of two 0:25:12.850 --> 0:25:15.850 thousand and one, did a number of tours overseas and 0:25:15.970 --> 0:25:19.410 the wars and a rock Afghanistan. So I'll say one 0:25:19.450 --> 0:25:22.450 moment that stuck out for me where I really sort 0:25:22.450 --> 0:25:24.490 of the light bulb went on about the power of 0:25:24.650 --> 0:25:28.370 robotics in warfare. I was in a rock in two 0:25:28.410 --> 0:25:31.690 thousand and seven. We were on a patrol, driving along 0:25:31.730 --> 0:25:35.410 in a Striker armored vehicle. Came across an ID improvised 0:25:35.450 --> 0:25:38.730 explosive device makeshift road type bomb, and so we called 0:25:38.810 --> 0:25:41.810 up bomb disposal folks. So they show up and I'm 0:25:41.850 --> 0:25:44.730 expecting the bomb tech to come out in that big 0:25:44.770 --> 0:25:46.450 bomb suit that you might have seen in the movie 0:25:46.450 --> 0:25:50.130 The hurt Locker, for example, and instead out rolls out 0:25:50.290 --> 0:25:53.730 a little robot and I kind of went, oh, that 0:25:53.770 --> 0:25:56.930 makes a lot of sense. Have the robot diffused the bomb. Well, 0:25:56.930 --> 0:25:58.370 it turns out there's a lot of things in war 0:25:58.410 --> 0:26:01.210 that are super dangerous where it makes sense to have 0:26:01.330 --> 0:26:04.570 robots out on the front lines, getting people better stand 0:26:04.610 --> 0:26:08.010 off a little bit more separation from potential threats. The 0:26:08.090 --> 0:26:11.850 bomb diffusing robe are still remote controlled by a technician, 0:26:12.330 --> 0:26:15.010 but ashe Carter wants to take the idea of robots 0:26:15.010 --> 0:26:18.530 doing the most dangerous work a step further somewhere in 0:26:18.570 --> 0:26:21.690 the future. But I'm certain will occur. Is I think 0:26:21.730 --> 0:26:25.250 there will be robots who will be part of infantry 0:26:25.290 --> 0:26:28.410 squads and that will do some of the most dangerous 0:26:28.530 --> 0:26:32.770 jobs in an infantry squad, like kicking down the door 0:26:32.850 --> 0:26:35.090 of a building and being the first one to run 0:26:35.170 --> 0:26:39.610 in and clear the building of terrorists or whatever. That's 0:26:39.610 --> 0:26:42.890 a job that doesn't sound like something I would like 0:26:43.050 --> 0:26:46.050