WEBVTT - Iran War Spotlights the Pentagon’s AI Strategy

0:00:00.240 --> 0:00:07.200
<v Speaker 1>Bloomberg Audio Studios, podcasts, radio news.

0:00:08.720 --> 0:00:11.920
<v Speaker 2>The Iran War showed little signs of de escalating this week,

0:00:12.119 --> 0:00:15.760
<v Speaker 2>with Israeli forces targeting nuclear development sites in Iran, the

0:00:15.880 --> 0:00:19.120
<v Speaker 2>US attacking Iranian mind laying vessels in the Strait of Hormuz,

0:00:19.480 --> 0:00:24.040
<v Speaker 2>and Iranian drones striking Dubai International Airport. President Trump and

0:00:24.079 --> 0:00:27.440
<v Speaker 2>Iran's new Supreme leader each vowed Thursday to keep up

0:00:27.440 --> 0:00:30.840
<v Speaker 2>their respective military campaigns amid mounting oil prices.

0:00:31.000 --> 0:00:33.920
<v Speaker 1>Every day we lose about twenty percent of global oil supplies.

0:00:34.040 --> 0:00:36.080
<v Speaker 3>We are just shy of one hundred dollars a barrel

0:00:36.200 --> 0:00:36.919
<v Speaker 3>on Brent now.

0:00:37.280 --> 0:00:39.760
<v Speaker 2>In a post on truth Social Trump wrote that preventing

0:00:39.760 --> 0:00:42.599
<v Speaker 2>Iran from threatening the Middle East and having nuclear weapons

0:00:42.800 --> 0:00:45.160
<v Speaker 2>was far more important to him than the cost of oil.

0:00:45.680 --> 0:00:50.040
<v Speaker 2>Iran's new Supreme Leader, Iotolamus Stabahamane, said Iran would continue

0:00:50.040 --> 0:00:52.920
<v Speaker 2>its blockade of the Strait of Hormuz, adding that he'd

0:00:52.960 --> 0:00:56.680
<v Speaker 2>consider opening other fronts if the war continues. But the

0:00:56.720 --> 0:00:59.840
<v Speaker 2>war has already expanded it to an important new front.

0:01:00.240 --> 0:01:03.640
<v Speaker 2>How it's being waged with AI enabled weaponry. In just

0:01:03.680 --> 0:01:06.080
<v Speaker 2>the first twenty four hours of the Iran War, the

0:01:06.200 --> 0:01:09.319
<v Speaker 2>US hit more than one thousand targets. According to the

0:01:09.319 --> 0:01:12.399
<v Speaker 2>commander of US Forces, that is double the scale of

0:01:12.400 --> 0:01:14.880
<v Speaker 2>the assault that opened the Iraq War in two thousand

0:01:14.880 --> 0:01:15.160
<v Speaker 2>and three.

0:01:15.520 --> 0:01:19.440
<v Speaker 3>The decision to wage a campaign like this is intimately

0:01:19.520 --> 0:01:24.080
<v Speaker 3>linked to how remotely the US can pursue this war.

0:01:24.360 --> 0:01:27.520
<v Speaker 2>That's Bloomberg National security and tech reporter Katrina Manson.

0:01:27.680 --> 0:01:31.720
<v Speaker 3>It's clearly really important for Donald Trump not to send

0:01:31.840 --> 0:01:34.720
<v Speaker 3>US troops in on the ground, so conducting an air

0:01:34.760 --> 0:01:39.959
<v Speaker 3>war with the help of AI suddenly becomes critical to

0:01:40.120 --> 0:01:42.160
<v Speaker 3>his own pursuit of a war.

0:01:43.680 --> 0:01:46.880
<v Speaker 2>The US's first strikes on Iran came hours after the

0:01:46.920 --> 0:01:50.680
<v Speaker 2>Pentagon severed its contract with Anthropic inn AI company that

0:01:50.800 --> 0:01:53.880
<v Speaker 2>had been working directly with the US military. The Pentagon

0:01:53.920 --> 0:01:57.920
<v Speaker 2>has formally notified Anthropic that it's deemed the artificial intelligence

0:01:57.960 --> 0:02:01.040
<v Speaker 2>company and its products a supply chain risk to the

0:02:01.120 --> 0:02:03.840
<v Speaker 2>United States. Pentagoon has said it needs the ability to

0:02:03.960 --> 0:02:09.280
<v Speaker 2>use Claude for quote all lawful purposes. Anthropic, which is

0:02:09.360 --> 0:02:12.560
<v Speaker 2>best known for its clawed AI tool, wanted assurances from

0:02:12.600 --> 0:02:15.920
<v Speaker 2>the US government that its technology would not make lethal

0:02:15.960 --> 0:02:18.920
<v Speaker 2>decisions on its own or help to conduct mass surveillance

0:02:18.960 --> 0:02:22.400
<v Speaker 2>on Americans. Anthropics ouster and a move by its rival

0:02:22.480 --> 0:02:25.400
<v Speaker 2>Open Ai to start working with the Department of Defense

0:02:25.680 --> 0:02:29.440
<v Speaker 2>have struck a nerve. Some Americans are a plauding Anthropics

0:02:29.440 --> 0:02:33.360
<v Speaker 2>decision Claude went down last week because of unprecedented demand.

0:02:34.000 --> 0:02:36.760
<v Speaker 2>It's the latest turn in a long simmering debate over

0:02:36.800 --> 0:02:39.840
<v Speaker 2>how the US military works with Silicon Valley companies and

0:02:39.919 --> 0:02:43.040
<v Speaker 2>how those tech titans contribute to the development of AI

0:02:43.120 --> 0:02:46.520
<v Speaker 2>weapons systems. Katrina took a deep dive into that story,

0:02:46.680 --> 0:02:49.000
<v Speaker 2>and it's a big part of her forthcoming book on

0:02:49.040 --> 0:02:50.680
<v Speaker 2>the US military and AI.

0:02:51.000 --> 0:02:53.800
<v Speaker 3>AI may be good for scale, it may be good

0:02:53.840 --> 0:02:58.000
<v Speaker 3>for speed, but is it as good for accuracy and

0:02:58.200 --> 0:03:01.280
<v Speaker 3>saving lives as these claims have been made in the past.

0:03:01.760 --> 0:03:02.880
<v Speaker 3>We don't know yet.

0:03:06.480 --> 0:03:08.280
<v Speaker 2>I'm David Gera and this is the big take from

0:03:08.320 --> 0:03:11.400
<v Speaker 2>Bloomberg News today on the show, as Anthropic and the

0:03:11.400 --> 0:03:15.000
<v Speaker 2>Pentagon feud. How is the US integrating AI into its

0:03:15.000 --> 0:03:18.679
<v Speaker 2>war fighting machine, How is AI enabled weaponry being regulated?

0:03:19.040 --> 0:03:21.040
<v Speaker 2>And is the tech as good as it needs to

0:03:21.080 --> 0:03:29.560
<v Speaker 2>be with so many lives at risk? About five years ago,

0:03:29.840 --> 0:03:33.000
<v Speaker 2>Dario Amide and his sister left their jobs at OpenAI

0:03:33.120 --> 0:03:36.400
<v Speaker 2>to found Anthropic, an AI company meant to enforce strong

0:03:36.440 --> 0:03:41.840
<v Speaker 2>safety guardrails. Since then, Anthropic has achieved mainstream success with Claude.

0:03:42.200 --> 0:03:45.120
<v Speaker 2>It's even managed to partner with the US government, but

0:03:45.240 --> 0:03:47.840
<v Speaker 2>that all changed at the start of the year. Bloomberg

0:03:47.920 --> 0:03:51.000
<v Speaker 2>Senior editor Mike Shepherd has been following the saga closely.

0:03:51.360 --> 0:03:56.480
<v Speaker 1>The Pentagon began reviewing how it wanted to roll out

0:03:56.960 --> 0:04:02.120
<v Speaker 1>artificial intelligence across the Armed Force, and in January they

0:04:02.200 --> 0:04:06.320
<v Speaker 1>put out an order from Defense Secretary Pete Hegseth saying

0:04:06.360 --> 0:04:10.600
<v Speaker 1>that they needed to accelerate adoption across the Defense Department,

0:04:10.680 --> 0:04:13.720
<v Speaker 1>which he refers to as the Department of War. He

0:04:13.800 --> 0:04:17.080
<v Speaker 1>even put up posters around the building saying the government

0:04:17.120 --> 0:04:20.480
<v Speaker 1>wants you to use AI as part of it. They

0:04:20.480 --> 0:04:24.680
<v Speaker 1>included language that said they did not want to be

0:04:24.760 --> 0:04:29.240
<v Speaker 1>bound by any usage restrictions that might come from an

0:04:29.279 --> 0:04:32.520
<v Speaker 1>AI provider of any stripe. They didn't single out Anthropic,

0:04:32.880 --> 0:04:36.200
<v Speaker 1>but Anthropic is not only one of the leading AI

0:04:36.480 --> 0:04:39.440
<v Speaker 1>developers and providers in the world, they are one that

0:04:39.480 --> 0:04:42.640
<v Speaker 1>has stood out for what many see as its safety

0:04:42.680 --> 0:04:46.440
<v Speaker 1>first stance and its adherence to principles of trying to

0:04:46.880 --> 0:04:50.359
<v Speaker 1>develop AI with a mind toward avoiding some of the

0:04:50.360 --> 0:04:53.880
<v Speaker 1>worst case scenarios. Now. For its part, Aanthropic has expressed

0:04:53.920 --> 0:04:58.239
<v Speaker 1>continued interest in working with the military so long as

0:04:58.520 --> 0:05:03.360
<v Speaker 1>the Pentagon A military officials abide by those usage restrictions.

0:05:03.360 --> 0:05:06.039
<v Speaker 1>And the two that have really been a redline for

0:05:06.080 --> 0:05:09.960
<v Speaker 1>the company concern the use of its AI technology for

0:05:10.400 --> 0:05:16.080
<v Speaker 1>mass surveillance of Americans domestically and then also fully autonomous

0:05:16.120 --> 0:05:19.640
<v Speaker 1>weapons the climate they as the developers of the technology,

0:05:19.680 --> 0:05:23.200
<v Speaker 1>insist that they don't have full confidence that it's ready

0:05:23.240 --> 0:05:24.599
<v Speaker 1>for those uses safely.

0:05:25.839 --> 0:05:28.120
<v Speaker 2>Katrina, you have a book coming out later this month

0:05:28.160 --> 0:05:31.480
<v Speaker 2>on the Pentagon's efforts to integrate AI into warfare. I'm

0:05:31.520 --> 0:05:34.159
<v Speaker 2>curious when that initiative started. When did the Pentagon begin

0:05:34.839 --> 0:05:36.160
<v Speaker 2>looking into this an interest.

0:05:36.560 --> 0:05:39.520
<v Speaker 3>The pentagonice to say it's been developing AI for sixty years.

0:05:39.560 --> 0:05:42.080
<v Speaker 3>But the real project that I think we can set

0:05:42.120 --> 0:05:46.200
<v Speaker 3>the timerby is Project Maven, which started in twenty seventeen,

0:05:46.560 --> 0:05:49.320
<v Speaker 3>and it was an effort that really spoke to America's

0:05:49.320 --> 0:05:53.159
<v Speaker 3>concern about falling behind China. The US began to realize

0:05:53.200 --> 0:05:55.640
<v Speaker 3>it was using old tech and that this new age

0:05:55.640 --> 0:06:00.839
<v Speaker 3>of warfare was coming that would require robots, autonomy, and AI,

0:06:01.360 --> 0:06:05.239
<v Speaker 3>and Project Maven was this effort to experiment with AI.

0:06:05.600 --> 0:06:08.480
<v Speaker 3>At the time, the cutting edge was computer vision, things

0:06:08.520 --> 0:06:12.360
<v Speaker 3>that could identify objects on a video feed taken by

0:06:12.440 --> 0:06:16.880
<v Speaker 3>drones and process that quicker. But the people I spoke

0:06:16.920 --> 0:06:19.960
<v Speaker 3>to also explained to me that all along they imagined

0:06:20.080 --> 0:06:21.560
<v Speaker 3>that this would help with targeting.

0:06:22.080 --> 0:06:24.720
<v Speaker 2>You write about the software that comes out of Project

0:06:24.720 --> 0:06:28.120
<v Speaker 2>man Maven smart System, which is made by Palenteer and

0:06:28.160 --> 0:06:32.240
<v Speaker 2>incorporated technology from Amazon and Microsoft and others. Is this

0:06:32.240 --> 0:06:35.040
<v Speaker 2>something the Defense Department thought they could do in house

0:06:35.200 --> 0:06:37.160
<v Speaker 2>or was it always something that they needed the cooperation

0:06:37.200 --> 0:06:38.279
<v Speaker 2>of private industry.

0:06:38.760 --> 0:06:41.520
<v Speaker 3>It's a real core point that obviously the Pentagon is

0:06:41.560 --> 0:06:45.960
<v Speaker 3>still struggling with. For this very advanced tech, This recognition

0:06:46.040 --> 0:06:48.159
<v Speaker 3>that has been happening over the past ten years that

0:06:48.600 --> 0:06:51.640
<v Speaker 3>warfare is going to be what some call software defined,

0:06:52.200 --> 0:06:56.600
<v Speaker 3>they need commercial companies, and so when Project Maven happened

0:06:57.080 --> 0:06:59.080
<v Speaker 3>for the first time, really they were going outside the

0:06:59.120 --> 0:07:02.880
<v Speaker 3>traditional primes as they're known, the traditional defense contractors and

0:07:02.920 --> 0:07:07.080
<v Speaker 3>looking at companies like Google, Microsoft, and Amazon, but other

0:07:07.160 --> 0:07:10.240
<v Speaker 3>tiny startups as well who were just beginning to experiment

0:07:10.320 --> 0:07:15.200
<v Speaker 3>with fascinating algorithms that could detect images on wedding cakes

0:07:16.240 --> 0:07:19.520
<v Speaker 3>and then reapplying that for the tools of war, and

0:07:19.920 --> 0:07:25.360
<v Speaker 3>that transition was incredibly uncomfortable, clunky, difficult, and they're backfired

0:07:25.360 --> 0:07:29.040
<v Speaker 3>at one point rather spectacularly when in twenty eighteen, Google

0:07:29.080 --> 0:07:32.480
<v Speaker 3>workers discovered that their company was working on Project Maven,

0:07:32.520 --> 0:07:36.440
<v Speaker 3>and they feared that their tech and their traditions for

0:07:36.600 --> 0:07:39.320
<v Speaker 3>them could be subverted into what they ended up calling

0:07:39.400 --> 0:07:42.920
<v Speaker 3>in a protest letter the business of war, and the

0:07:42.960 --> 0:07:45.920
<v Speaker 3>parallel with Anthropic today isn't quite the same. The CEO

0:07:46.040 --> 0:07:49.080
<v Speaker 3>of Anthropic all Along has said he believes in national

0:07:49.080 --> 0:07:52.800
<v Speaker 3>security work more than any other AI lab. Anthropic has

0:07:52.880 --> 0:07:55.920
<v Speaker 3>lent into classified work, and the classified cloud is where

0:07:56.280 --> 0:08:00.680
<v Speaker 3>Penticon does its fighting, so they've been involved with lethal

0:08:00.720 --> 0:08:05.720
<v Speaker 3>operations in some way for more than a year. What

0:08:05.840 --> 0:08:08.680
<v Speaker 3>I think is different is he had these red lines,

0:08:08.760 --> 0:08:12.160
<v Speaker 3>and as he is trying to uphold them, tech workers

0:08:12.160 --> 0:08:14.360
<v Speaker 3>in other companies are really paying note now.

0:08:14.640 --> 0:08:19.240
<v Speaker 2>Katrina ethics are so central to this program, and I'm

0:08:19.320 --> 0:08:23.200
<v Speaker 2>curious how defense officials addressed those ethics of seeding life

0:08:23.280 --> 0:08:27.240
<v Speaker 2>or death decision making to machines to artificial intelligence.

0:08:27.840 --> 0:08:30.840
<v Speaker 3>They would argue, I think very current now to the

0:08:30.880 --> 0:08:35.960
<v Speaker 3>operations that are using AI tools against Iran, that humans

0:08:36.000 --> 0:08:40.120
<v Speaker 3>are still making the decisions. But in a very interesting

0:08:40.480 --> 0:08:44.520
<v Speaker 3>presentation by the Commander of US Forces just this week,

0:08:45.000 --> 0:08:48.960
<v Speaker 3>they've made clear that they've hit five thousand, five hundred targets.

0:08:49.440 --> 0:08:54.280
<v Speaker 3>That speed is exactly why they want AI and that scope,

0:08:55.000 --> 0:08:57.600
<v Speaker 3>and the Pentagon has had to work very hard thinking

0:08:57.600 --> 0:09:01.319
<v Speaker 3>through precision. The claim, of course is that US weapons

0:09:01.679 --> 0:09:04.120
<v Speaker 3>are the most precise in the world. But where you

0:09:04.280 --> 0:09:07.719
<v Speaker 3>decide to put them, that is where America has had

0:09:07.760 --> 0:09:11.880
<v Speaker 3>problems and the scale at which it's shooting at areas

0:09:11.920 --> 0:09:15.360
<v Speaker 3>where we know civilians are present, because Scentcom is warning

0:09:15.400 --> 0:09:18.240
<v Speaker 3>civilians to stay home, telling them to stay away from ports.

0:09:19.160 --> 0:09:22.080
<v Speaker 3>There is a margin of error. There's a decision to

0:09:22.240 --> 0:09:24.960
<v Speaker 3>accept or not accept a certain amount of collateral damage.

0:09:24.960 --> 0:09:27.880
<v Speaker 3>They haven't made public what margin they set on that

0:09:27.920 --> 0:09:31.679
<v Speaker 3>collateral damage, but these are key ethical decisions in The

0:09:32.240 --> 0:09:35.600
<v Speaker 3>targeting process is a multi stage effort where people feed

0:09:35.640 --> 0:09:39.280
<v Speaker 3>in and eventually a commander makes a decision to sign off.

0:09:39.720 --> 0:09:42.840
<v Speaker 3>But as you involve AI, that process speeds up. The

0:09:42.920 --> 0:09:49.360
<v Speaker 3>decision making time is reduced, and fears for things like

0:09:49.760 --> 0:09:54.360
<v Speaker 3>automation bias or the algorithms themselves going wrong. Hallucinating or

0:09:54.440 --> 0:09:58.640
<v Speaker 3>drifting algorithms tend to naturally get worse over time. All

0:09:58.679 --> 0:10:01.480
<v Speaker 3>of that has really yet to be worked out. Now

0:10:01.600 --> 0:10:04.000
<v Speaker 3>the US has got a policy for that. It's a directive.

0:10:04.400 --> 0:10:06.480
<v Speaker 3>Some people to me frame it more as a process

0:10:06.920 --> 0:10:09.640
<v Speaker 3>than a rule, but it does say that there needs

0:10:09.679 --> 0:10:13.000
<v Speaker 3>to be human judgment over the appropriate use of force. Well,

0:10:13.000 --> 0:10:16.720
<v Speaker 3>that's supervision, that's not necessarily making a decision. And so

0:10:16.840 --> 0:10:20.040
<v Speaker 3>I think from all the people I've spoken to, that role,

0:10:20.120 --> 0:10:24.240
<v Speaker 3>that human role is reducing, and Sentcom has been particularly

0:10:24.320 --> 0:10:27.280
<v Speaker 3>proud this week the commander to say that AI is

0:10:27.320 --> 0:10:31.600
<v Speaker 3>helping them reduce operations decisions from what used to be

0:10:32.200 --> 0:10:36.160
<v Speaker 3>days and hours two seconds.

0:10:37.000 --> 0:10:39.960
<v Speaker 2>After the break. How good is this technology really? We

0:10:40.080 --> 0:10:42.719
<v Speaker 2>hear about one test where things went awry, and we

0:10:42.840 --> 0:10:56.920
<v Speaker 2>dig into how lawmakers are responding. Proponents of AI enabled

0:10:56.920 --> 0:10:59.760
<v Speaker 2>weapons say the technology can improve decision making on the

0:10:59.760 --> 0:11:03.600
<v Speaker 2>back and put fewer troops at risk, which could save lives.

0:11:04.080 --> 0:11:07.400
<v Speaker 2>Its promise of speed makes it imperative to national security

0:11:07.440 --> 0:11:10.600
<v Speaker 2>experts who see it as essential in the widening AI

0:11:10.760 --> 0:11:14.280
<v Speaker 2>arms race with China, but Bloomberg's Katrina Manson points out

0:11:14.520 --> 0:11:16.160
<v Speaker 2>it also comes with risks.

0:11:16.520 --> 0:11:21.800
<v Speaker 3>AI is this fundamentally unpredictable black box technology. So it's

0:11:21.960 --> 0:11:24.560
<v Speaker 3>brilliant at bringing a lot of data to bear. It

0:11:24.679 --> 0:11:26.720
<v Speaker 3>just might be the wrong data, and it might be

0:11:26.840 --> 0:11:29.320
<v Speaker 3>organized in the wrong way. And if you're not giving

0:11:29.440 --> 0:11:32.400
<v Speaker 3>enough time to checking, or if you don't understand where

0:11:32.440 --> 0:11:34.880
<v Speaker 3>it can make mistakes and you put it into critical

0:11:34.920 --> 0:11:40.160
<v Speaker 3>a place in your own system, you've made yourself incredibly fallible. Now,

0:11:40.240 --> 0:11:43.360
<v Speaker 3>from the US perspective, that brings a couple of problems.

0:11:43.440 --> 0:11:46.360
<v Speaker 3>One is you may end up killing civilians, which is

0:11:46.400 --> 0:11:51.440
<v Speaker 3>against US policy. Certainly deliberate targeting inadvertent targeting under the

0:11:51.559 --> 0:11:54.840
<v Speaker 3>rule of law has to be proportionate. But the second

0:11:54.880 --> 0:11:56.560
<v Speaker 3>thing is you can also hurt your own troops and

0:11:56.640 --> 0:12:00.160
<v Speaker 3>allied troops. And the US has a history of grappling

0:12:00.600 --> 0:12:05.319
<v Speaker 3>with what are called friendly fire incidents, and AI was

0:12:05.400 --> 0:12:08.360
<v Speaker 3>meant to help clear up this kind of claus witsy

0:12:08.440 --> 0:12:10.680
<v Speaker 3>and idea of the fog of war. One of the

0:12:10.679 --> 0:12:13.920
<v Speaker 3>ways they've integrated AI into the battlefield picture is through

0:12:13.920 --> 0:12:17.640
<v Speaker 3>the system that emerges from Project Man called Maven Smart System.

0:12:18.240 --> 0:12:20.360
<v Speaker 3>For most of US, it's a really basic idea. It's

0:12:20.440 --> 0:12:23.800
<v Speaker 3>essentially just a digital map. It's Google Maps for war.

0:12:24.240 --> 0:12:27.040
<v Speaker 3>It's really hard for the military to create that because

0:12:27.040 --> 0:12:28.760
<v Speaker 3>they have data in all sorts of places, they haven't

0:12:28.840 --> 0:12:31.480
<v Speaker 3>labeled their data. They have spent the last ten years

0:12:31.559 --> 0:12:35.200
<v Speaker 3>really getting their equipment, their tech, their connectivity up to

0:12:35.280 --> 0:12:39.040
<v Speaker 3>scale where something like this could work, but then integrating

0:12:39.080 --> 0:12:42.199
<v Speaker 3>AI into it is really down to what checks they

0:12:42.240 --> 0:12:45.920
<v Speaker 3>make for it the algorithms themselves. I found cases where

0:12:46.320 --> 0:12:49.120
<v Speaker 3>the quality of the algorithm, the ability of the algorithm

0:12:49.160 --> 0:12:51.679
<v Speaker 3>to detect what it's meant to be identifying, can change

0:12:51.679 --> 0:12:54.520
<v Speaker 3>if the weather changes, if you move an algorithm from

0:12:54.559 --> 0:12:57.640
<v Speaker 3>a hot area to a green area, picking up something

0:12:57.640 --> 0:13:01.000
<v Speaker 3>as simple as a truck or a tank, andinguishing between

0:13:01.360 --> 0:13:03.440
<v Speaker 3>a man or a woman or a child, which is

0:13:03.600 --> 0:13:08.440
<v Speaker 3>really critical to those life and death decisions and the

0:13:08.520 --> 0:13:11.920
<v Speaker 3>rules under which the US military operates, really isn't at

0:13:11.920 --> 0:13:13.880
<v Speaker 3>the stage that you can give that over to algorithms.

0:13:14.000 --> 0:13:18.160
<v Speaker 3>So the risk is if they are not sufficiently tire

0:13:18.280 --> 0:13:21.120
<v Speaker 3>kicking their own algorithms when it comes to computer vision,

0:13:21.600 --> 0:13:24.360
<v Speaker 3>or when it comes to using lms and chatbots to

0:13:24.400 --> 0:13:27.760
<v Speaker 3>speed up their processes and analyze data, you could be

0:13:27.800 --> 0:13:30.720
<v Speaker 3>going in the wrong direction for quite some time before

0:13:30.760 --> 0:13:31.319
<v Speaker 3>you notice it.

0:13:32.800 --> 0:13:34.080
<v Speaker 2>Katran I wonder if you coul tell us a story

0:13:34.080 --> 0:13:36.040
<v Speaker 2>of what happens when this technology doesn't work as it's

0:13:36.040 --> 0:13:38.400
<v Speaker 2>intended to. And you write about a test that took

0:13:38.440 --> 0:13:40.280
<v Speaker 2>place back in June of twenty twenty five, maybe you

0:13:40.280 --> 0:13:41.920
<v Speaker 2>could just walk us through what happened then and what

0:13:42.040 --> 0:13:45.240
<v Speaker 2>lessons we can draw from how that unintended result could

0:13:45.240 --> 0:13:46.560
<v Speaker 2>pretend difficulties down the road.

0:13:46.880 --> 0:13:50.959
<v Speaker 3>The teams behind Maven had made computer vision and then

0:13:50.960 --> 0:13:52.920
<v Speaker 3>this was the next iteration. This is where a program

0:13:53.000 --> 0:13:56.840
<v Speaker 3>called replicator, so it would be to identify a target

0:13:57.200 --> 0:13:59.520
<v Speaker 3>and then the drone could go after the target and

0:13:59.559 --> 0:14:06.199
<v Speaker 3>then identify it and explode. And developing automatic target recognition

0:14:06.559 --> 0:14:09.280
<v Speaker 3>is one thing that relies on creating algorithms that can

0:14:09.320 --> 0:14:13.520
<v Speaker 3>find things. The other is this swarming technology coordinating between drones.

0:14:13.760 --> 0:14:16.559
<v Speaker 3>Of course they were testing this. It's expected to fail,

0:14:16.640 --> 0:14:19.000
<v Speaker 3>that's why they do it in test conditions. And the

0:14:19.040 --> 0:14:24.680
<v Speaker 3>aim was to deliver multiple thousands of drones by a

0:14:24.720 --> 0:14:27.360
<v Speaker 3>certain date. And two months before that date, there was

0:14:27.400 --> 0:14:31.560
<v Speaker 3>a test in California of some drone boats and in

0:14:31.560 --> 0:14:34.760
<v Speaker 3>this one experiment, the drone was towed out to see

0:14:34.800 --> 0:14:37.360
<v Speaker 3>before it was to be switched into autonomy mode. That

0:14:37.400 --> 0:14:39.720
<v Speaker 3>in itself was meant to be a safety measure What

0:14:39.840 --> 0:14:43.920
<v Speaker 3>happened was, inadvertently a command was sent from the dock

0:14:44.560 --> 0:14:47.720
<v Speaker 3>to the drone boat, and when the drone boat was

0:14:47.760 --> 0:14:51.600
<v Speaker 3>activated into autonomy mode without anyone realizing it, it started

0:14:51.640 --> 0:14:55.600
<v Speaker 3>trying to get away from the captain who was towing it.

0:14:56.040 --> 0:15:00.560
<v Speaker 3>The boat started accelerating, decelerating multiple times between zero and

0:15:00.600 --> 0:15:04.120
<v Speaker 3>six knots at pace semi circling action. The rope goes

0:15:04.400 --> 0:15:07.720
<v Speaker 3>taught and the captain is capsized. At that point, he's

0:15:07.720 --> 0:15:10.400
<v Speaker 3>in the water, and then the drone boat turns and

0:15:10.440 --> 0:15:13.880
<v Speaker 3>comes toward him. That's a very dangerous moment because the

0:15:14.080 --> 0:15:17.680
<v Speaker 3>rope could strangle him underwater, he could be submerged. There's

0:15:17.680 --> 0:15:20.400
<v Speaker 3>a runaway boat coming for him. A third boat was

0:15:20.440 --> 0:15:24.800
<v Speaker 3>able to intervene and save him, and he was okay.

0:15:24.960 --> 0:15:28.600
<v Speaker 3>Then what actually happened in the investigation that ensued was

0:15:28.600 --> 0:15:32.280
<v Speaker 3>that someone had mistakenly sent was called a zero command

0:15:32.800 --> 0:15:36.640
<v Speaker 3>just by pressing enter on a command line, deployed the

0:15:36.640 --> 0:15:39.360
<v Speaker 3>boat into autonomy mode, and then all of this ensued,

0:15:39.520 --> 0:15:41.960
<v Speaker 3>so they fixed it. This is a very early stage

0:15:41.960 --> 0:15:44.680
<v Speaker 3>of testing. They're meant to be putting explosives on these boats.

0:15:44.720 --> 0:15:47.160
<v Speaker 3>They're meant for these boats to be all cooperating together

0:15:47.520 --> 0:15:50.320
<v Speaker 3>and able to defend an island such as Taiwan in

0:15:50.320 --> 0:15:52.640
<v Speaker 3>the event of an invasion, And I think it showed

0:15:52.640 --> 0:15:55.520
<v Speaker 3>that the tech is really hard to deliver and just

0:15:55.600 --> 0:15:56.600
<v Speaker 3>simply not ready.

0:15:57.000 --> 0:15:59.840
<v Speaker 2>In a statement for Katrina's book, a Navy spokesperson said

0:15:59.880 --> 0:16:03.080
<v Speaker 2>that safety is always their top priority, that they have

0:16:03.160 --> 0:16:06.480
<v Speaker 2>backup systems to prevent danger. The lessons they learned during

0:16:06.480 --> 0:16:10.720
<v Speaker 2>these events drive improvements in their systems. Mike, these conversations

0:16:10.760 --> 0:16:13.880
<v Speaker 2>about ethics are happening among the Defense Department and these

0:16:13.880 --> 0:16:17.880
<v Speaker 2>private companies. Where are lawmakers? Is Congress showcasing any interest

0:16:17.920 --> 0:16:20.000
<v Speaker 2>in engaging with the subject.

0:16:20.200 --> 0:16:23.400
<v Speaker 1>Showcasing is a good word, because they do like to

0:16:23.480 --> 0:16:27.920
<v Speaker 1>showcase their interest, but actually advancing a proposal that would

0:16:28.400 --> 0:16:35.160
<v Speaker 1>codify some regulation or law even on how AI is

0:16:35.200 --> 0:16:37.880
<v Speaker 1>deployed in warfare, we are a long way from that.

0:16:38.040 --> 0:16:41.600
<v Speaker 1>There's discussion of attaching some sort of amendment to the

0:16:41.680 --> 0:16:47.200
<v Speaker 1>Annual Defense Authorization Measure, which every year usually gets caught

0:16:47.280 --> 0:16:50.520
<v Speaker 1>up in the fight of the moment, and that could

0:16:50.600 --> 0:16:54.400
<v Speaker 1>be where this ultimately ends up, But it's unclear whether

0:16:54.960 --> 0:16:59.120
<v Speaker 1>there would be enough consensus between both parties to really

0:16:59.160 --> 0:17:02.160
<v Speaker 1>come up with language. They all could agree on and

0:17:02.200 --> 0:17:06.960
<v Speaker 1>that the Trump administration would not try to torpedo somehow itself.

0:17:07.000 --> 0:17:10.119
<v Speaker 1>Remember the tech industry, a lot of the big tech

0:17:10.119 --> 0:17:15.600
<v Speaker 1>companies have really moved to align themselves with President Donald

0:17:15.640 --> 0:17:19.280
<v Speaker 1>Trump in a lot of different ways, and it's difficult

0:17:19.280 --> 0:17:21.480
<v Speaker 1>for them and a lot of other business fronts to

0:17:21.680 --> 0:17:24.080
<v Speaker 1>challenge the president, including on this.

0:17:24.800 --> 0:17:27.160
<v Speaker 2>Katrina, I would ask you lastly, just about how much

0:17:27.240 --> 0:17:28.879
<v Speaker 2>the horse is out of the barn here. I think

0:17:28.880 --> 0:17:31.520
<v Speaker 2>there'll be a lot of people listening to this who

0:17:32.000 --> 0:17:34.879
<v Speaker 2>will be impressed by how far this field has advanced,

0:17:34.920 --> 0:17:38.320
<v Speaker 2>that is a integration into warfare, and may wonder if

0:17:38.320 --> 0:17:41.280
<v Speaker 2>there's any recourse or anything that they can do they

0:17:41.320 --> 0:17:43.800
<v Speaker 2>as citizens to slow down this process.

0:17:44.000 --> 0:17:47.080
<v Speaker 3>People talk about a costless war, We've already seen that

0:17:47.119 --> 0:17:49.880
<v Speaker 3>from the US side alone. It's not a costless war.

0:17:50.240 --> 0:17:53.760
<v Speaker 3>AI is meant to give you a riskless war. And

0:17:54.560 --> 0:17:58.840
<v Speaker 3>probably where civilians and citizens come involved, are trying to

0:17:58.920 --> 0:18:01.800
<v Speaker 3>understand the contours of is there such a thing as

0:18:01.840 --> 0:18:04.639
<v Speaker 3>a riskless war? And who is harmed by that? And

0:18:04.720 --> 0:18:08.320
<v Speaker 3>if in any way AI isn't saving civilians or even

0:18:08.560 --> 0:18:11.919
<v Speaker 3>there are misfires that involve AI, then really you have

0:18:12.000 --> 0:18:15.440
<v Speaker 3>to re examine if it makes war more likely that

0:18:15.520 --> 0:18:19.080
<v Speaker 3>in itself is a change for the way that the

0:18:19.240 --> 0:18:23.359
<v Speaker 3>US had maybe been thinking about conducting wars under President

0:18:23.359 --> 0:18:26.840
<v Speaker 3>Trump himself, who of course is the main political leader

0:18:26.880 --> 0:18:28.959
<v Speaker 3>who has said I don't want any more wars.

0:18:29.680 --> 0:18:33.120
<v Speaker 2>Katrina Manson's book Project Mavin, A Marine Colonel, His Team

0:18:33.200 --> 0:18:36.200
<v Speaker 2>and the Dawn of AI Warfare comes out later this month.

0:18:39.720 --> 0:18:42.240
<v Speaker 2>This is the Big Take from Bloomberg News. I'm David Gura.

0:18:42.520 --> 0:18:44.800
<v Speaker 2>To get more from The Big Take and unlimited access

0:18:44.840 --> 0:18:47.800
<v Speaker 2>to all of Bloomberg dot com, subscribe today at Bloomberg

0:18:47.840 --> 0:18:51.399
<v Speaker 2>dot com slash podcast offer. If you like this episode,

0:18:51.440 --> 0:18:53.480
<v Speaker 2>make sure to follow and review The Big Take wherever

0:18:53.480 --> 0:18:55.880
<v Speaker 2>you listen to podcasts. It helps people find the show.

0:18:56.320 --> 0:18:58.160
<v Speaker 2>Thanks for listening. We'll be back tomorrow.