1 00:00:03,120 --> 00:00:07,880 Speaker 1: Bloomberg Audio Studios, Podcasts, radio News. 2 00:00:08,480 --> 00:00:11,240 Speaker 2: Hey, it's Sarah today on The Big Take. We're bringing 3 00:00:11,280 --> 00:00:13,680 Speaker 2: you an episode of The Big Take DC hosted by 4 00:00:13,680 --> 00:00:17,439 Speaker 2: my colleague, Senior Washington correspondent Salia Mosen. She's got a 5 00:00:17,440 --> 00:00:19,880 Speaker 2: great story for you about the promise and peril of 6 00:00:19,920 --> 00:00:25,079 Speaker 2: applying AI technology to the battlefield. I'll be back tomorrow. 7 00:00:25,360 --> 00:00:26,079 Speaker 2: Here's Seleia. 8 00:00:26,960 --> 00:00:29,600 Speaker 3: For a long time, I had a job that wasn't acknowledged, 9 00:00:29,640 --> 00:00:32,280 Speaker 3: and my wife didn't know what I did. We're living 10 00:00:32,320 --> 00:00:34,240 Speaker 3: other than there was a paycheck coming in. 11 00:00:34,800 --> 00:00:37,760 Speaker 4: This is Will Roper. Back in the mid twenty tens. 12 00:00:38,080 --> 00:00:40,560 Speaker 4: He was in charge of a then secret office within 13 00:00:40,600 --> 00:00:44,400 Speaker 4: the Defense Department called the Strategic Capabilities. 14 00:00:43,640 --> 00:00:47,959 Speaker 3: Office that meant developing new systems and new strategies building 15 00:00:48,120 --> 00:00:49,279 Speaker 3: the future of warfare. 16 00:00:49,640 --> 00:00:52,920 Speaker 4: Ruper was fascinated by how big tech was using artificial 17 00:00:52,920 --> 00:00:56,280 Speaker 4: intelligence to scan photos and identify objects and what it 18 00:00:56,320 --> 00:00:57,960 Speaker 4: could do for the US military. 19 00:00:58,240 --> 00:01:00,520 Speaker 3: A lot of what industry was trying to do. Being 20 00:01:00,560 --> 00:01:05,520 Speaker 3: able to identify buildings and roads and objects, it seemed 21 00:01:05,880 --> 00:01:09,280 Speaker 3: very traceable to battlefield analogs. 22 00:01:09,840 --> 00:01:12,160 Speaker 4: Roper would go on to propose what would become a 23 00:01:12,360 --> 00:01:16,480 Speaker 4: powerful piece of AI technology for the military Project Maven. 24 00:01:16,920 --> 00:01:18,880 Speaker 4: But the journey to build Maven and bring it to 25 00:01:18,920 --> 00:01:22,400 Speaker 4: the battlefield hasn't been easy. There are still big questions 26 00:01:22,440 --> 00:01:25,440 Speaker 4: about whether the US military should even be using algorithms 27 00:01:25,440 --> 00:01:26,040 Speaker 4: in warfare. 28 00:01:26,400 --> 00:01:28,280 Speaker 1: They use this language that will always be a human 29 00:01:28,319 --> 00:01:30,120 Speaker 1: in the loop. But at the same time, the whole 30 00:01:30,160 --> 00:01:33,240 Speaker 1: point of using AI is to speed up, and people 31 00:01:33,280 --> 00:01:36,039 Speaker 1: talk about decisions speeding up so false that that false 32 00:01:36,160 --> 00:01:37,399 Speaker 1: than a human can catch up with. 33 00:01:42,200 --> 00:01:45,600 Speaker 4: That was Bloomberg reporter Katrina Manson today. On the show, 34 00:01:45,920 --> 00:01:48,800 Speaker 4: she takes us inside the promise in the peril of 35 00:01:48,920 --> 00:01:52,880 Speaker 4: AI in the US military. We go inside Project Maven, 36 00:01:53,080 --> 00:01:58,200 Speaker 4: the Pentagon's flagship artificial intelligence effort. You'll also hear from 37 00:01:58,200 --> 00:02:01,560 Speaker 4: the US intelligence official who receives much of Maven, and 38 00:02:01,640 --> 00:02:05,720 Speaker 4: the CTO of Scentcom, the US Central Command whose troops 39 00:02:05,720 --> 00:02:09,840 Speaker 4: are actually using it in battle. From Bloomberg's Washington Bureau. 40 00:02:10,200 --> 00:02:13,720 Speaker 4: This is the Big Take DC podcast. I'm Salaiah Mosen. 41 00:02:18,639 --> 00:02:21,880 Speaker 4: My colleague Katrina Manson covers cyber and emerging tech with 42 00:02:21,960 --> 00:02:25,240 Speaker 4: a focus on national security. In the past few years, 43 00:02:25,480 --> 00:02:28,080 Speaker 4: she's been hearing a lot about how AI could be 44 00:02:28,120 --> 00:02:31,200 Speaker 4: the biggest thing to transform warfare since the invention of 45 00:02:31,240 --> 00:02:34,000 Speaker 4: the radio or the machine gun. But she says the 46 00:02:34,120 --> 00:02:37,359 Speaker 4: US military's big effort to bring AI to the battlefield 47 00:02:37,440 --> 00:02:40,680 Speaker 4: really kicked off in twenty seventeen with Project Maven. 48 00:02:40,840 --> 00:02:43,200 Speaker 1: The Defense Department now likes to say they've been using 49 00:02:43,240 --> 00:02:47,160 Speaker 1: AI for sixty years, but it was really that announcement 50 00:02:47,200 --> 00:02:51,320 Speaker 1: that said we want to look at AI machine learning algorithms, 51 00:02:51,320 --> 00:02:54,959 Speaker 1: and we want to find battlefield applications for it. As 52 00:02:55,200 --> 00:02:56,960 Speaker 1: Will rope In, one of the people who came up 53 00:02:57,000 --> 00:02:59,760 Speaker 1: with the idea, said he always saw it as a 54 00:02:59,760 --> 00:03:04,000 Speaker 1: way to find a better way of doing automatic target recognition. 55 00:03:04,720 --> 00:03:09,720 Speaker 4: Automatic target recognition, in this case, scanning satellite imagery and 56 00:03:09,800 --> 00:03:13,720 Speaker 4: other data inputs to identify potential threats and enemy locations 57 00:03:13,720 --> 00:03:17,760 Speaker 4: on a battlefield and making sure the US doesn't misidentify 58 00:03:17,919 --> 00:03:21,160 Speaker 4: non threats as targets and end up deploying a weapon 59 00:03:21,200 --> 00:03:25,760 Speaker 4: against a hospital instead of a weapons factory. Until this point, 60 00:03:25,800 --> 00:03:28,760 Speaker 4: the military relied on people to comb through hours of 61 00:03:28,840 --> 00:03:32,520 Speaker 4: video footage and satellite imagery to identify these targets. But 62 00:03:32,720 --> 00:03:36,640 Speaker 4: Roper thought a computer could do this faster, maybe even better. 63 00:03:36,920 --> 00:03:39,320 Speaker 3: In a real battle, there's so much to see and 64 00:03:39,360 --> 00:03:41,840 Speaker 3: observe to try to get down to the handful of 65 00:03:41,920 --> 00:03:45,960 Speaker 3: targets that you need to prosecute with weapons, and it 66 00:03:46,040 --> 00:03:48,840 Speaker 3: was clear that processing had gotten to be so good 67 00:03:48,880 --> 00:03:51,800 Speaker 3: and there was so much data available that you could 68 00:03:51,840 --> 00:03:54,040 Speaker 3: build these very deep neural nets. 69 00:03:54,400 --> 00:03:57,760 Speaker 4: But in twenty seventeen, the US military wasn't familiar with 70 00:03:57,840 --> 00:04:01,040 Speaker 4: deep neural nets and it definitely couldn't build them. 71 00:04:01,200 --> 00:04:04,080 Speaker 3: The Internet of Things did not happen to the US 72 00:04:04,120 --> 00:04:07,560 Speaker 3: military when it happened to everyone else. The US military 73 00:04:07,600 --> 00:04:12,320 Speaker 3: has amazing hardware, but the connectedness between it is not. 74 00:04:12,920 --> 00:04:16,359 Speaker 3: So military members leave their lives where they're connected to 75 00:04:16,360 --> 00:04:19,760 Speaker 3: almost everything, and they inner military lives where they're connected 76 00:04:19,800 --> 00:04:23,159 Speaker 3: to almost nothing. What most people, I think expect is 77 00:04:23,200 --> 00:04:27,320 Speaker 3: that aircraft and ships connect like our smart devices do, 78 00:04:27,440 --> 00:04:29,480 Speaker 3: and they don't not even close. 79 00:04:30,000 --> 00:04:33,440 Speaker 4: So Rouper proposed the military make a major push focused 80 00:04:33,480 --> 00:04:36,400 Speaker 4: on automatic target recognition, But when he got into a 81 00:04:36,480 --> 00:04:39,039 Speaker 4: room with senior defense officials to pitch his idea for 82 00:04:39,120 --> 00:04:42,359 Speaker 4: what would become Project Maven, he didn't exactly get the 83 00:04:42,400 --> 00:04:43,440 Speaker 4: response he wanted. 84 00:04:43,800 --> 00:04:47,240 Speaker 3: Well, skeptics are pretty much what filled the halls of 85 00:04:47,240 --> 00:04:51,240 Speaker 3: the Pentagon. It's not a place of great creativity and innovation, 86 00:04:51,920 --> 00:04:56,040 Speaker 3: and I remember still having to put together a tutorial 87 00:04:56,200 --> 00:05:01,200 Speaker 3: presentation about why this would feasibly work, why this was 88 00:05:01,240 --> 00:05:05,240 Speaker 3: a big trend in commercial industry, and why we needed 89 00:05:05,240 --> 00:05:07,240 Speaker 3: to be on the right side of the trend. 90 00:05:07,920 --> 00:05:10,800 Speaker 4: Like any government bureaucracy, the Pentagon has a lot of 91 00:05:10,839 --> 00:05:14,000 Speaker 4: red tape. Its approaches, nothing like the move fast and 92 00:05:14,040 --> 00:05:17,800 Speaker 4: break things mentality of Silicon Valley. That's in part for 93 00:05:17,920 --> 00:05:20,839 Speaker 4: good reason. The stakes are incredibly high when it comes 94 00:05:20,880 --> 00:05:24,520 Speaker 4: to anything the US government decides to back, but especially 95 00:05:24,640 --> 00:05:27,520 Speaker 4: when it comes to warfare. But some of those Pentagon 96 00:05:27,560 --> 00:05:30,599 Speaker 4: officials did take a chance on Roper's idea, and in 97 00:05:30,640 --> 00:05:35,000 Speaker 4: April twenty seventeen, the Defense Department announced Project Mavin, a 98 00:05:35,080 --> 00:05:38,880 Speaker 4: way to use AI and data for actionable intelligence. What 99 00:05:38,920 --> 00:05:43,000 Speaker 4: that meant was developing machine learning algorithms and then constantly 100 00:05:43,080 --> 00:05:46,719 Speaker 4: retraining those algorithms with up to date, well labeled data. 101 00:05:47,320 --> 00:05:50,320 Speaker 4: But all that outdated military tech that Roper was dealing 102 00:05:50,320 --> 00:05:54,440 Speaker 4: with made that hard. My colleague Katrina spoke to one 103 00:05:54,440 --> 00:05:57,040 Speaker 4: of the experts who worked on testing and evaluation in 104 00:05:57,080 --> 00:05:58,680 Speaker 4: the early days of MAVEN. 105 00:05:58,480 --> 00:06:01,360 Speaker 1: And she said it was really not very good at all. 106 00:06:01,720 --> 00:06:04,239 Speaker 1: They didn't have the data labeled. There was no sense 107 00:06:04,279 --> 00:06:06,960 Speaker 1: of everything that we'd been doing. When we click online 108 00:06:07,000 --> 00:06:08,800 Speaker 1: and say, yes, this is a traffic light, Yes this 109 00:06:08,920 --> 00:06:12,000 Speaker 1: is a cat. There wasn't that same store of images 110 00:06:12,080 --> 00:06:15,000 Speaker 1: for tanks or weapons factories. 111 00:06:15,320 --> 00:06:17,839 Speaker 4: So Roper and his team had people go through images 112 00:06:17,880 --> 00:06:19,159 Speaker 4: and manually label them. 113 00:06:19,440 --> 00:06:24,120 Speaker 3: There were thousands of people participating and labeling data so 114 00:06:24,160 --> 00:06:25,680 Speaker 3: that we could train algorithms. 115 00:06:25,880 --> 00:06:29,160 Speaker 4: They also needed some expertise to develop the program. The 116 00:06:29,240 --> 00:06:32,839 Speaker 4: Pentagon partnered with some commercial tech companies, ones that were 117 00:06:32,960 --> 00:06:35,360 Speaker 4: leaps and bounds ahead of the military when it came 118 00:06:35,440 --> 00:06:38,839 Speaker 4: to AI, but in twenty eighteen that became a problem. 119 00:06:39,080 --> 00:06:41,520 Speaker 1: Google was involved in the early days of Project MABN, 120 00:06:41,560 --> 00:06:45,640 Speaker 1: and in twenty eighteen thousands of Google workers protested. The 121 00:06:45,720 --> 00:06:49,480 Speaker 1: key question was is just applying that ability to recognize 122 00:06:49,480 --> 00:06:53,160 Speaker 1: a cat to a tank? How much moral culpability did 123 00:06:53,200 --> 00:06:56,159 Speaker 1: those workers then feel? Because what might happen next is 124 00:06:56,279 --> 00:06:58,600 Speaker 1: the tank might be exploded, and if it made a mistake, 125 00:06:58,680 --> 00:07:02,279 Speaker 1: a human might be killed, or the targets may eventually 126 00:07:02,360 --> 00:07:06,240 Speaker 1: involve places where humans are or civilians are, And in 127 00:07:06,279 --> 00:07:08,880 Speaker 1: a letter to their seniors said we do not want 128 00:07:08,880 --> 00:07:10,720 Speaker 1: to work on warfare technology. 129 00:07:10,880 --> 00:07:14,200 Speaker 4: It brought Project me even into the spotlight splashed across 130 00:07:14,240 --> 00:07:15,880 Speaker 4: the news quote, we. 131 00:07:15,920 --> 00:07:19,040 Speaker 5: Believe Google should not be in the business of war. Therefore, 132 00:07:19,080 --> 00:07:22,280 Speaker 5: we asked that Project Mayven be canceled, that Google enforced 133 00:07:22,280 --> 00:07:26,200 Speaker 5: a clear policy stating that neither Google nor its contractors 134 00:07:26,240 --> 00:07:28,480 Speaker 5: will ever build warfare technology. 135 00:07:28,920 --> 00:07:32,239 Speaker 4: Did the protests give people inside the military any pause? 136 00:07:32,840 --> 00:07:35,160 Speaker 1: I think they did. I think the military also realized 137 00:07:35,200 --> 00:07:38,280 Speaker 1: it really had to grapple with, well, what are we 138 00:07:38,320 --> 00:07:40,520 Speaker 1: actually asking big tech to do? These are not people 139 00:07:40,560 --> 00:07:43,040 Speaker 1: who signed up to serve the flag and the way 140 00:07:43,080 --> 00:07:46,160 Speaker 1: that they have the missions are completely different, the cultures 141 00:07:46,160 --> 00:07:49,840 Speaker 1: are different. At the time, you had senior defense officials 142 00:07:49,880 --> 00:07:53,000 Speaker 1: flying over to the West Coast to try and understand 143 00:07:53,000 --> 00:07:56,640 Speaker 1: what this completely different culture was about. And since then 144 00:07:57,440 --> 00:07:59,880 Speaker 1: the people in charge of Project Mayven had to pivot 145 00:08:00,000 --> 00:08:03,320 Speaker 1: and find other partners that already were partners besides Google. 146 00:08:03,720 --> 00:08:06,280 Speaker 1: But I think the Pentagon went through a renewed kind 147 00:08:06,280 --> 00:08:08,120 Speaker 1: of outreach to Silicon Valley. 148 00:08:07,880 --> 00:08:10,360 Speaker 4: And with the help of those other tech partners and 149 00:08:10,480 --> 00:08:14,320 Speaker 4: thousands of workers labeling the data, the algorithms behind Project 150 00:08:14,360 --> 00:08:18,640 Speaker 4: Maven got more and more accurate. By twenty twenty, an 151 00:08:18,840 --> 00:08:22,720 Speaker 4: Army colonel named Joseph Callahan, who runs weapons fires, started 152 00:08:22,760 --> 00:08:25,680 Speaker 4: taking an interest in Maven. He was asked to start 153 00:08:25,720 --> 00:08:28,840 Speaker 4: experimenting with it in his unit, the eighteenth Airborne Core 154 00:08:28,960 --> 00:08:33,200 Speaker 4: in North Carolina. He ran live ammunition exercises, using the 155 00:08:33,240 --> 00:08:37,000 Speaker 4: computer algorithm to identify a target like a tank, and 156 00:08:37,040 --> 00:08:40,480 Speaker 4: then having a human deploy a weapon to explode that target. 157 00:08:41,559 --> 00:08:44,120 Speaker 4: Testing the tech is a crucial part of the process. 158 00:08:44,520 --> 00:08:47,800 Speaker 4: The whole point of machine learning is feedback, trying out 159 00:08:47,840 --> 00:08:50,160 Speaker 4: an algorithm and then telling it when it's right or 160 00:08:50,200 --> 00:08:52,520 Speaker 4: wrong so it can learn and do better next time. 161 00:08:53,200 --> 00:08:55,800 Speaker 4: But just because the boss wants his workers to start 162 00:08:55,920 --> 00:08:59,000 Speaker 4: using a shiny new tool doesn't mean everyone will fall 163 00:08:59,040 --> 00:09:03,640 Speaker 4: into line. Katrina told me about one officer named Joey Temple. 164 00:09:03,840 --> 00:09:08,960 Speaker 1: The very battle hardened targeting officer served five rounds in Iraq, 165 00:09:09,040 --> 00:09:12,079 Speaker 1: and he was offered a demonstration of what's called MAVEN 166 00:09:12,120 --> 00:09:13,120 Speaker 1: Smart System. 167 00:09:12,960 --> 00:09:15,839 Speaker 4: The platform that houses me even in other data streams, and. 168 00:09:15,760 --> 00:09:17,600 Speaker 1: He said, no, I don't want a demo. I don't 169 00:09:17,600 --> 00:09:20,000 Speaker 1: want a demo. And he told me that he had 170 00:09:20,000 --> 00:09:21,839 Speaker 1: a sense that stuff like this doesn't work. He didn't 171 00:09:21,840 --> 00:09:25,440 Speaker 1: need another tool. He was fuddled by hearing so much 172 00:09:25,440 --> 00:09:28,120 Speaker 1: about AI when he'd really never worked with AI before 173 00:09:28,120 --> 00:09:30,800 Speaker 1: and he was an expert. He is an expert targeter. 174 00:09:30,920 --> 00:09:33,200 Speaker 1: He's done this his entire career. 175 00:09:33,440 --> 00:09:37,480 Speaker 4: Initially, Temple wasn't interested, but the next month, in August 176 00:09:37,480 --> 00:09:42,920 Speaker 4: of twenty twenty one, something changed his mind, coming up 177 00:09:43,320 --> 00:09:47,400 Speaker 4: how Project Maven won over battle hardened soldiers like Joey Temple, 178 00:09:47,720 --> 00:09:59,880 Speaker 4: and how it's being used on actual battlefields today. At first, 179 00:10:00,120 --> 00:10:04,240 Speaker 4: Senior targeting Officer Joey Temple wasn't interested in using Project Mavin, 180 00:10:04,840 --> 00:10:07,360 Speaker 4: but that changed in August twenty twenty. 181 00:10:07,080 --> 00:10:10,800 Speaker 6: One chaotic scenes as US troops at Kabul Airport strove 182 00:10:10,840 --> 00:10:13,520 Speaker 6: to evacuate Americans as well as Afghan civilians. 183 00:10:13,640 --> 00:10:17,079 Speaker 4: It was the evacuation of US troops from Afghanistan. The 184 00:10:17,120 --> 00:10:20,080 Speaker 4: military had to airlift one hundred and twenty thousand people 185 00:10:20,120 --> 00:10:24,600 Speaker 4: out of Kabble under very dangerous conditions, and Temple was 186 00:10:24,600 --> 00:10:26,760 Speaker 4: put in front of a Maven scream that mapped out 187 00:10:26,800 --> 00:10:28,679 Speaker 4: threats on the ground in real time. 188 00:10:29,360 --> 00:10:32,800 Speaker 1: He could see people walking around the roots of planes. 189 00:10:32,920 --> 00:10:35,839 Speaker 1: For him, seeing Maven's small system operate like that was 190 00:10:35,880 --> 00:10:36,960 Speaker 1: a bit of a light bulb moment. 191 00:10:37,880 --> 00:10:40,360 Speaker 4: The folks in charge of Maven are relying on those 192 00:10:40,400 --> 00:10:44,040 Speaker 4: sorts of light bulb moments. The military needs officers willing 193 00:10:44,080 --> 00:10:46,559 Speaker 4: to use MAIVEN to train its algorithms to get better, 194 00:10:47,000 --> 00:10:49,640 Speaker 4: and to link it up to data and censors and 195 00:10:49,760 --> 00:10:52,520 Speaker 4: actually make it operational in the way the military runs 196 00:10:52,520 --> 00:10:55,200 Speaker 4: its battles. Otherwise it's pretty much useless. 197 00:10:55,320 --> 00:10:58,520 Speaker 1: You do tend to get experiences like that of Joey 198 00:10:58,559 --> 00:11:03,000 Speaker 1: Temple who are reticent at the start. And he even 199 00:11:03,040 --> 00:11:06,160 Speaker 1: told me he has people under him who say, no, 200 00:11:06,200 --> 00:11:08,520 Speaker 1: I don't want to use this. I mean, he calls 201 00:11:08,640 --> 00:11:11,800 Speaker 1: himself a skeptic. He is constantly going to play devil's 202 00:11:11,840 --> 00:11:14,080 Speaker 1: advocate because what's so important for him is to know 203 00:11:14,520 --> 00:11:17,600 Speaker 1: is this system really reliable? That's what I found so interesting. 204 00:11:18,040 --> 00:11:20,600 Speaker 1: How you start integrating what military folks like to call 205 00:11:20,720 --> 00:11:23,640 Speaker 1: human machine teaming. What is that actually like From the 206 00:11:23,679 --> 00:11:27,280 Speaker 1: perspective of the person on the ground who's been told here, 207 00:11:27,520 --> 00:11:30,560 Speaker 1: there's a newfangled toy, use this. Those people have very 208 00:11:30,679 --> 00:11:34,600 Speaker 1: very deep relationships with their fellow combatants, who the people 209 00:11:34,640 --> 00:11:36,760 Speaker 1: they go to war with they trust. That's a really 210 00:11:36,840 --> 00:11:41,320 Speaker 1: fundamental idea of US warfare, of any warfare. What if 211 00:11:41,360 --> 00:11:42,679 Speaker 1: you've got to trust an algorithm? 212 00:11:42,840 --> 00:11:46,040 Speaker 4: The military is now asking soldiers of the eighteenth Airborne 213 00:11:46,080 --> 00:11:49,200 Speaker 4: Corps to trust that algorithm, at least in experiments on 214 00:11:49,240 --> 00:11:53,160 Speaker 4: their base. It's now the largest operational test bed for MAVEN. 215 00:11:53,800 --> 00:11:56,520 Speaker 4: Last year, Katrina flew to the base in North Carolina 216 00:11:56,559 --> 00:11:58,000 Speaker 4: to see it in action herself. 217 00:11:58,160 --> 00:12:02,000 Speaker 1: I was finally given a classied demonstration of it, and 218 00:12:02,320 --> 00:12:04,960 Speaker 1: it was incredibly revealing to see it because it wasn't 219 00:12:05,000 --> 00:12:07,560 Speaker 1: quite what I expected, and they did a mock up 220 00:12:07,600 --> 00:12:12,760 Speaker 1: for me using unclassified satellite imagery, but it represented a 221 00:12:12,960 --> 00:12:16,760 Speaker 1: real operation that the US had conducted several months before. 222 00:12:17,120 --> 00:12:19,360 Speaker 1: And it's very much like looking at a map on 223 00:12:19,400 --> 00:12:23,360 Speaker 1: your phone, only this had yellow boxes and blue boxes, 224 00:12:23,520 --> 00:12:26,280 Speaker 1: and the blue boxes meant don't hit this, this is 225 00:12:26,320 --> 00:12:29,280 Speaker 1: not a target. We've identified this as a school or 226 00:12:29,320 --> 00:12:32,680 Speaker 1: a hospital or friendly forces, and yellow boxes meant what 227 00:12:32,720 --> 00:12:34,920 Speaker 1: they call points of interest. So in this case, if 228 00:12:34,960 --> 00:12:37,720 Speaker 1: you weren't asking the algorithm to find ships, it would 229 00:12:38,080 --> 00:12:40,840 Speaker 1: box out all the ships, and then you might be 230 00:12:40,920 --> 00:12:44,240 Speaker 1: able to ask it to find a specific kind of ship, 231 00:12:44,280 --> 00:12:47,440 Speaker 1: maybe a warship, and they can layer that in using 232 00:12:47,480 --> 00:12:49,719 Speaker 1: lots and lots of different data feed sources. 233 00:12:49,840 --> 00:12:52,640 Speaker 4: Not just the satellite photos and videos that MAVEN was 234 00:12:52,679 --> 00:12:56,400 Speaker 4: originally trained on. Now the program incorporates and analyzes so 235 00:12:56,559 --> 00:12:59,560 Speaker 4: much more data. Katrina got to hear about that from 236 00:12:59,559 --> 00:13:02,960 Speaker 4: Mark Monell. He's the director of Data and Digital Innovation 237 00:13:03,080 --> 00:13:07,240 Speaker 4: at the National Geospatial Intelligence Agency, which since last year 238 00:13:07,360 --> 00:13:10,320 Speaker 4: has run most of MAVEN. The agency has been working 239 00:13:10,360 --> 00:13:14,000 Speaker 4: closely with troops to test field and improve MAVIN on 240 00:13:14,040 --> 00:13:14,480 Speaker 4: the ground. 241 00:13:14,800 --> 00:13:18,520 Speaker 7: We also use radio waves bouncing off of objects, and 242 00:13:19,120 --> 00:13:24,040 Speaker 7: we also use infrared to see heat light, and we 243 00:13:24,080 --> 00:13:27,920 Speaker 7: also use multi spectral remote sensing, so not just the 244 00:13:28,000 --> 00:13:31,880 Speaker 7: visible light spectrum, but also the invisible light spectrum. 245 00:13:32,080 --> 00:13:34,319 Speaker 4: Muncell told her the way he sees it, there are 246 00:13:34,320 --> 00:13:37,720 Speaker 4: two big advantages of using MAVEN and military targeting. 247 00:13:38,240 --> 00:13:41,360 Speaker 7: So speed is very important, right Obviously, the faster that 248 00:13:41,360 --> 00:13:44,560 Speaker 7: we can take an image, the faster that we can 249 00:13:44,880 --> 00:13:48,319 Speaker 7: run an algorithm and get the results is very important. 250 00:13:49,080 --> 00:13:52,400 Speaker 7: But I will say the biggest advantage would be scale. 251 00:13:52,440 --> 00:13:55,079 Speaker 7: The fact that we can do it on many images, 252 00:13:55,280 --> 00:13:57,440 Speaker 7: you know, as fast as we can is really where 253 00:13:57,440 --> 00:13:59,520 Speaker 7: we're providing the advantage. 254 00:13:58,960 --> 00:14:01,800 Speaker 4: An advantage that was just recently proven, not just in 255 00:14:01,840 --> 00:14:03,920 Speaker 4: a test, but on the actual battlefield. 256 00:14:04,280 --> 00:14:05,960 Speaker 6: October seventh, everything changed. 257 00:14:06,320 --> 00:14:10,200 Speaker 4: That's Skyler Moore, the Chief Technology Officer of US Central Command, 258 00:14:10,440 --> 00:14:13,760 Speaker 4: often referred to as Sentcom. Katrina managed to get her 259 00:14:13,760 --> 00:14:15,120 Speaker 4: on the phone earlier this month. 260 00:14:15,400 --> 00:14:18,400 Speaker 6: We immediately shifted into high gear in a much higher 261 00:14:18,400 --> 00:14:21,760 Speaker 6: operational tempo than we had previously. And the reason that 262 00:14:21,840 --> 00:14:23,880 Speaker 6: I think we were able to and had a pretty 263 00:14:23,880 --> 00:14:27,560 Speaker 6: seamless shift of using NAVAN was because we'd done about 264 00:14:27,560 --> 00:14:30,440 Speaker 6: twelve months worth of digital exercises leading up to that. 265 00:14:31,320 --> 00:14:31,720 Speaker 1: More tool. 266 00:14:31,800 --> 00:14:34,640 Speaker 4: Katrina that the US also used ME even to identify 267 00:14:34,680 --> 00:14:37,960 Speaker 4: and narrow down dozens of targets that US air strikes 268 00:14:38,000 --> 00:14:39,960 Speaker 4: then fired against Iraq and Syria. 269 00:14:40,520 --> 00:14:42,320 Speaker 1: That for me is the first time that I've really 270 00:14:42,360 --> 00:14:46,040 Speaker 1: heard someone talk about using AI to find something and 271 00:14:46,080 --> 00:14:48,720 Speaker 1: then deliver a weapon to it that actually belonged to 272 00:14:48,720 --> 00:14:53,200 Speaker 1: the enemy and damage it destroyed some there were casualties, 273 00:14:53,600 --> 00:14:57,000 Speaker 1: and so that cycle is really being operationalized. 274 00:14:57,120 --> 00:14:59,920 Speaker 6: We've certainly had more opportunities to target in a law 275 00:15:00,160 --> 00:15:03,000 Speaker 6: sixty to ninety days. We have a lot of examples 276 00:15:03,000 --> 00:15:05,480 Speaker 6: of the who he's presenting those types of threats. 277 00:15:05,800 --> 00:15:10,760 Speaker 1: They're finding rocket launches in Yemen, they're finding vessels in 278 00:15:10,800 --> 00:15:13,920 Speaker 1: the Red Sea, and Maven has become an intrinsic part 279 00:15:14,160 --> 00:15:17,400 Speaker 1: of that decision cycle to try and find out where 280 00:15:17,480 --> 00:15:19,960 Speaker 1: is there a threat, what do we want to do 281 00:15:20,080 --> 00:15:22,120 Speaker 1: about it, and how can we be sure? 282 00:15:23,000 --> 00:15:25,680 Speaker 4: The tech is being used but it hasn't been perfected 283 00:15:25,760 --> 00:15:29,280 Speaker 4: yet well. Humans at the eighteenth Airborne cor can correctly 284 00:15:29,320 --> 00:15:32,040 Speaker 4: identify a tank about eighty four percent of the time. 285 00:15:32,560 --> 00:15:35,600 Speaker 4: Mayven gets it closer to sixty percent, and experts have 286 00:15:35,640 --> 00:15:38,640 Speaker 4: told Katrina that that number can dip even lower, as 287 00:15:38,720 --> 00:15:41,200 Speaker 4: low as thirty percent when there's snow or cloud cover 288 00:15:41,240 --> 00:15:43,520 Speaker 4: in an image, or if it's searching for a new 289 00:15:43,560 --> 00:15:44,360 Speaker 4: type of object. 290 00:15:44,840 --> 00:15:48,160 Speaker 1: A lot of the US military imagery comes from the 291 00:15:48,200 --> 00:15:51,600 Speaker 1: Middle East. When you move to somewhere like Ukraine, there's snow, 292 00:15:51,680 --> 00:15:54,880 Speaker 1: there's cloud cover, there's rain, so the conditions change a 293 00:15:54,880 --> 00:15:58,280 Speaker 1: lot and the algorithms start performing far less well. 294 00:15:58,680 --> 00:16:01,400 Speaker 4: These challenges are some of the things that critics point 295 00:16:01,440 --> 00:16:04,440 Speaker 4: to when they express concern about the US military using 296 00:16:04,440 --> 00:16:08,239 Speaker 4: this technology. Another big thing that comes up is backlash 297 00:16:08,280 --> 00:16:11,480 Speaker 4: against the idea of using autonomous machines, which can use 298 00:16:11,520 --> 00:16:14,880 Speaker 4: AI on the battlefield at all. There's a coalition of 299 00:16:14,960 --> 00:16:18,440 Speaker 4: human rights and expert groups called Stop Killer Robots that's 300 00:16:18,520 --> 00:16:22,120 Speaker 4: dedicated to keeping lethal autonomous machines that can use AI 301 00:16:22,400 --> 00:16:26,800 Speaker 4: out of warfare. Even UN Secretary General Antonio Guterrez has 302 00:16:26,800 --> 00:16:31,080 Speaker 4: called for a ban on autonomous weapons systems. Here's Katrina again. 303 00:16:31,600 --> 00:16:35,080 Speaker 1: An algorithm itself can be trained using data that was 304 00:16:35,280 --> 00:16:40,800 Speaker 1: incorrect or incomplete. It can also be targeted by adversary attacks. 305 00:16:40,800 --> 00:16:44,440 Speaker 1: The data could be changed without knowing. The algorithm could 306 00:16:44,440 --> 00:16:47,320 Speaker 1: be spoofed, it can be poisoned. It can lose accuracy 307 00:16:47,360 --> 00:16:50,160 Speaker 1: over time. That's a very natural thing for an algorithm 308 00:16:50,160 --> 00:16:52,360 Speaker 1: to do. It stops being as effective. So you have 309 00:16:52,400 --> 00:16:54,520 Speaker 1: to work out at what point do you want to 310 00:16:54,600 --> 00:16:57,840 Speaker 1: change retrain work with that algorithm. And then, of course, 311 00:16:58,120 --> 00:17:00,960 Speaker 1: when you've got a life and death scenarios where you're 312 00:17:01,000 --> 00:17:03,840 Speaker 1: deciding how much trust should be put in this algorithm, 313 00:17:03,840 --> 00:17:06,400 Speaker 1: what should we do with it. It might be feasible 314 00:17:06,440 --> 00:17:08,720 Speaker 1: to use an algorithm that makes mistakes if it's just 315 00:17:08,760 --> 00:17:10,240 Speaker 1: going to blow up water, you know it's trying to 316 00:17:10,240 --> 00:17:12,160 Speaker 1: get a shit, but instead it gets water, a big deal. 317 00:17:12,400 --> 00:17:15,119 Speaker 1: But if you're trying to get something on land and 318 00:17:15,160 --> 00:17:18,560 Speaker 1: if you miss your hitting civilians or any humans, of 319 00:17:18,560 --> 00:17:20,280 Speaker 1: course the risks are extremely high. 320 00:17:20,720 --> 00:17:23,520 Speaker 4: The US military says that this is why it relies 321 00:17:23,520 --> 00:17:27,120 Speaker 4: on human machine teaming. They say that the algorithm itself 322 00:17:27,160 --> 00:17:30,280 Speaker 4: will never make the decision to execute an airstrike. It 323 00:17:30,400 --> 00:17:33,919 Speaker 4: just identifies a target for a human commander. But MAVEN 324 00:17:34,200 --> 00:17:37,719 Speaker 4: is making those decisions and those links to weapons faster. 325 00:17:38,040 --> 00:17:40,760 Speaker 7: We want to shorten that kill chain, you know, as 326 00:17:40,800 --> 00:17:41,600 Speaker 7: fast as we can. 327 00:17:41,840 --> 00:17:45,159 Speaker 4: Mark Mansell again, he's the agency official who oversees the 328 00:17:45,200 --> 00:17:46,359 Speaker 4: office that runs MAVEN. 329 00:17:46,560 --> 00:17:49,240 Speaker 7: So instead of a human having to open up an image, 330 00:17:49,400 --> 00:17:51,520 Speaker 7: scan the image, find everything on the image, and then 331 00:17:51,600 --> 00:17:54,680 Speaker 7: report that out to a combat and commander, we want 332 00:17:54,680 --> 00:17:57,640 Speaker 7: to shorten that by using the algorithms. 333 00:17:57,080 --> 00:17:59,639 Speaker 1: To do that, so the combatant commander gets the choice 334 00:17:59,680 --> 00:18:02,480 Speaker 1: of what to execute fison and that much faster. 335 00:18:02,800 --> 00:18:05,240 Speaker 4: But as Katrina points out, speeding up the pace at 336 00:18:05,280 --> 00:18:08,160 Speaker 4: which targets can be identified and fired on that can 337 00:18:08,200 --> 00:18:08,960 Speaker 4: be complicated. 338 00:18:09,119 --> 00:18:10,800 Speaker 1: And of course there's been a huge amount of work 339 00:18:10,840 --> 00:18:14,320 Speaker 1: on confirmation bias and an algorithm, and I think you 340 00:18:14,400 --> 00:18:17,800 Speaker 1: have to wonder if an algorithm says this is something 341 00:18:17,840 --> 00:18:20,000 Speaker 1: that we should be looking at as a target, does 342 00:18:20,000 --> 00:18:22,720 Speaker 1: that make the person who's there to say yes or 343 00:18:22,760 --> 00:18:24,879 Speaker 1: no to the algorithm more likely to say yes or 344 00:18:24,920 --> 00:18:26,640 Speaker 1: less likely to say yes, the. 345 00:18:26,680 --> 00:18:30,000 Speaker 4: US military expects this kind of technology to improve, but 346 00:18:30,040 --> 00:18:32,960 Speaker 4: in the meantime they're continuing to use it, and other 347 00:18:33,040 --> 00:18:35,680 Speaker 4: countries around the world like Israel and Ukraine are using 348 00:18:35,720 --> 00:18:39,760 Speaker 4: similar technology. That raises the question is the world ready 349 00:18:39,800 --> 00:18:40,760 Speaker 4: for what that might mean? 350 00:18:41,200 --> 00:18:43,679 Speaker 3: We're not ready for that. In the US military, that 351 00:18:43,800 --> 00:18:46,840 Speaker 3: a war of algorithms, a war of software. 352 00:18:46,760 --> 00:18:49,160 Speaker 4: Keeping up is the reason Will Roper wanted to bring 353 00:18:49,200 --> 00:18:51,679 Speaker 4: AI to the US military in the first place. 354 00:18:52,040 --> 00:18:56,640 Speaker 3: We've always had this human advantage in the US military 355 00:18:56,640 --> 00:19:01,000 Speaker 3: because we're always operating that our people are are experienced 356 00:19:01,040 --> 00:19:04,720 Speaker 3: and trained and they're ready to go, and we've always 357 00:19:04,760 --> 00:19:06,760 Speaker 3: been able to say that as a way if we're 358 00:19:06,760 --> 00:19:09,520 Speaker 3: feeling a little nervous about scenario, is that we've got 359 00:19:09,560 --> 00:19:13,280 Speaker 3: the people who are ready well. With AI, it may 360 00:19:13,280 --> 00:19:17,560 Speaker 3: be the first technology where you can undercut human advantage. 361 00:19:17,680 --> 00:19:19,879 Speaker 4: That's a concern to folks of the Pentagon and in 362 00:19:19,920 --> 00:19:23,480 Speaker 4: government who fear a war with say China, could put 363 00:19:23,520 --> 00:19:26,480 Speaker 4: the US on the back foot. But war doesn't wait 364 00:19:26,520 --> 00:19:29,880 Speaker 4: on software updates. Katrina learned that in twenty twenty two, 365 00:19:30,240 --> 00:19:32,879 Speaker 4: the US helped Ukraine by using Maven to share so 366 00:19:33,040 --> 00:19:37,080 Speaker 4: called points of interest where Russian equipment was located. Here's 367 00:19:37,119 --> 00:19:40,600 Speaker 4: this stat that really stuck with me. During that process, 368 00:19:40,720 --> 00:19:44,639 Speaker 4: the Maven smart system platform underwent more than fifty rounds 369 00:19:44,640 --> 00:19:48,560 Speaker 4: of improvement in just the first ten months. Roper thinks 370 00:19:48,600 --> 00:19:50,760 Speaker 4: that the people within the military are going to need 371 00:19:50,800 --> 00:19:53,720 Speaker 4: to change the way they think about algorithms, because like 372 00:19:53,760 --> 00:19:56,359 Speaker 4: it or not, at this point, there's no going back. 373 00:19:56,600 --> 00:19:58,879 Speaker 3: The military is going to have to think about AI 374 00:19:59,680 --> 00:20:02,240 Speaker 3: not as much like a piece of computer software, but 375 00:20:02,320 --> 00:20:05,280 Speaker 3: almost like a member of the military that you train 376 00:20:05,920 --> 00:20:08,720 Speaker 3: and that you trust with a certain amount of authority 377 00:20:08,840 --> 00:20:12,840 Speaker 3: based on its training and pedigree. Hopefully those lessons will 378 00:20:12,840 --> 00:20:14,080 Speaker 3: be learned before they're needed. 379 00:20:17,280 --> 00:20:19,720 Speaker 4: Thanks for listening to The Big Take DC podcast from 380 00:20:19,720 --> 00:20:23,399 Speaker 4: Bloomberg News. I'm Salaiah Mosen. This episode was produced by 381 00:20:23,480 --> 00:20:26,320 Speaker 4: Julia Press and Naomi Shaven. It was fact checked by 382 00:20:26,359 --> 00:20:31,600 Speaker 4: Alex Sugia. Special thanks to Katrina Manson, Margaret Sutherland, Vicki Vergalina, 383 00:20:31,640 --> 00:20:35,200 Speaker 4: and Rebecca Shassen. Ben O'Brien is our mix engineer. Our 384 00:20:35,240 --> 00:20:38,880 Speaker 4: story editors are Caitlin Kenney, Wendy Benjaminson, and Michael Shephard. 385 00:20:39,280 --> 00:20:42,679 Speaker 4: Nicole Beemsterborer is our executive producer. Sage Bauman is our 386 00:20:42,720 --> 00:20:46,920 Speaker 4: head of podcasts. If you like what you heard, please subscribe, rate, 387 00:20:46,960 --> 00:20:49,520 Speaker 4: and review the show. It helps other listeners find us. 388 00:20:49,840 --> 00:20:53,960 Speaker 4: Thanks for tuning in. I'll be back next week.