WEBVTT - Anthropic, the Pentagon, and the Future of Autonomous Weapons 0:00:02.720 --> 0:00:16.400 Bloomberg Audio Studios, Podcasts, Radio News. 0:00:18.079 --> 0:00:21.759 Hello and welcome to another episode of the Odd Lots Podcast. 0:00:21.880 --> 0:00:24.280 I'm Joeisenthal and I'm Tracy Alloway. 0:00:24.640 --> 0:00:27.440 So Tracy recording this March twenty fourth, And of course 0:00:27.480 --> 0:00:29.800 almost all of our episodes. 0:00:29.440 --> 0:00:31.680 Lately have been about the war in Iran. 0:00:32.280 --> 0:00:34.879 But what's interesting, or what's a little weird is that 0:00:35.240 --> 0:00:38.520 just prior to the war, literally days or maybe hours, 0:00:38.880 --> 0:00:42.320 the biggest story in the world was actually about defense 0:00:42.400 --> 0:00:44.760 and you know the DoD that's right. 0:00:44.800 --> 0:00:48.239 So you are referring to anthrop a. Yeah, and it's 0:00:48.520 --> 0:00:51.280 disagreement to put it mildly in the Department of War. 0:00:51.680 --> 0:00:52.440 Yeah, exactly. 0:00:52.479 --> 0:00:55.160 This was the biggest story going right up on the 0:00:55.200 --> 0:00:57.920 eve of the war in Iran. And of course obviously 0:00:58.040 --> 0:01:02.240 there was this contract topics technology was used by the 0:01:02.240 --> 0:01:05.280 Defense Department. So it was not a disagreement about the 0:01:05.360 --> 0:01:08.920 use of AI per se in war, but the question 0:01:09.000 --> 0:01:12.440 of the degree to which AI could be used for 0:01:12.520 --> 0:01:15.360 autonomous weapons systems on their own without a human in the. 0:01:15.360 --> 0:01:18.720 Loop and surveillance and surveillance this was another key element. 0:01:18.840 --> 0:01:22.479 But you're right, so we've heard this expression autonomous weapons, yeah, 0:01:22.520 --> 0:01:25.640 pop up more and more especially in recent days. And 0:01:25.680 --> 0:01:28.880 I've a lot of questions over what exactly that means, 0:01:29.000 --> 0:01:31.880 because my impression is the US military certainly has been 0:01:31.959 --> 0:01:33.320 using AI for some time. 0:01:33.440 --> 0:01:35.480 Yes, and so we're really talking about. 0:01:35.560 --> 0:01:40.360 Degrees here of autonomy, right, And so if you think 0:01:40.360 --> 0:01:43.319 about an autonomous weapon, I think your mind could go 0:01:43.560 --> 0:01:46.760 fully terminator and you know there's like a murder robot 0:01:46.800 --> 0:01:50.680 out there that's making its own decisions on which. 0:01:50.600 --> 0:01:52.720 People or places to target. 0:01:53.200 --> 0:01:55.880 And then you get levels below that, right where AI 0:01:56.040 --> 0:01:59.920 is kind of helping humans to come up with strategic decisions. 0:02:00.160 --> 0:02:00.360 Right. 0:02:00.480 --> 0:02:03.000 So, if there is a missile coming and you have 0:02:03.040 --> 0:02:06.200 a missile defense system, I don't think you want a 0:02:06.280 --> 0:02:08.400 human in the loop. Is like, Okay, here are the 0:02:08.440 --> 0:02:11.200 coordinates X, y Z that we think it's going to hit. 0:02:11.639 --> 0:02:13.360 At this point in time, we think the missile will 0:02:13.360 --> 0:02:14.880 be here, Are you cool with firing it? 0:02:15.040 --> 0:02:16.240 I think everyone's. 0:02:15.840 --> 0:02:18.960 Probably okay with that level of autonomy. But I have 0:02:19.000 --> 0:02:22.799 a feeling that to your point exactly a lot of 0:02:22.840 --> 0:02:26.320 this discussion, and maybe it's core to what Anthropic and 0:02:26.360 --> 0:02:28.680 the Department of Defense who are disagreeing on, I have 0:02:28.720 --> 0:02:30.720 a feeling a lot of this is going to come 0:02:30.760 --> 0:02:33.880 down to definitions. My guess is that there is not 0:02:34.200 --> 0:02:38.440 one shared agreement of this is an autonomous weapon system 0:02:38.720 --> 0:02:39.560 and this one is. 0:02:39.520 --> 0:02:42.440 Not absolutely And of course there are also questions over 0:02:42.600 --> 0:02:47.440 exactly how places like the Department of Defense, not only 0:02:47.440 --> 0:02:51.120 how they define it, but once they have those definitions, 0:02:51.240 --> 0:02:55.120 whether or not certain companies trust them. Yes, so stick 0:02:55.160 --> 0:02:57.400 to those policies, because the US will say, well, our 0:02:57.520 --> 0:03:01.400 policy is not to survey are citizens at the moment, 0:03:01.520 --> 0:03:03.720 So if you're anthropic, you don't need to worry about that. 0:03:04.080 --> 0:03:07.600 Clearly anthropic feels otherwiseer say they do. So there are 0:03:07.600 --> 0:03:11.799 all these really interesting thematic questions that pop up from all. 0:03:11.639 --> 0:03:12.840 Of this totally. 0:03:12.880 --> 0:03:15.320 And then there's the question of Okay, here's a technology 0:03:15.360 --> 0:03:17.639 and the government says, we believe that we can use 0:03:17.680 --> 0:03:19.720 this to make the country safe. For what, You're not 0:03:19.760 --> 0:03:22.200 going to let us do it like private corporation. There's 0:03:22.240 --> 0:03:25.000 some very interesting questions about the role of corporate power 0:03:25.080 --> 0:03:26.800 visa vi the government and so forth. 0:03:26.840 --> 0:03:29.359 Anyway, this is something that has. 0:03:29.240 --> 0:03:32.120 Become even more timely in the reports, even in the 0:03:32.160 --> 0:03:35.640 early days of the Iran War, about these AI systems 0:03:35.640 --> 0:03:38.480 having been used perhaps in target selection, but we don't 0:03:38.520 --> 0:03:40.200 really know. None of the reporting is like that. 0:03:40.440 --> 0:03:40.720 Clear. 0:03:40.800 --> 0:03:44.400 I don't think that they're going out and advertising this strike. 0:03:44.600 --> 0:03:46.440 This is exactly how we're using AI. 0:03:46.600 --> 0:03:48.680 This is exactly how we're using AI, and so forth. 0:03:48.760 --> 0:03:51.920 But this is obviously a huge debate and war aside. 0:03:51.920 --> 0:03:54.760 It's totally going to grow, and just as an AI 0:03:54.880 --> 0:03:57.840 is going to grow, it seems in so many different areas. Anyway, 0:03:57.840 --> 0:04:00.280 I'm really excited to say, we really do have the 0:04:00.320 --> 0:04:02.680 perfect guest, someone who's been writing and thinking about this 0:04:02.760 --> 0:04:05.440 stuff for a long time. When we talked to an 0:04:05.480 --> 0:04:08.520 AI expert, I marked the dividing line AND's like, where 0:04:08.640 --> 0:04:12.120 you talking about AI prior to when Chad Gipt was released, 0:04:12.160 --> 0:04:14.280 as like, I take a little bit more seriously the 0:04:14.280 --> 0:04:17.200 people who are in this space prior to November twenty 0:04:17.320 --> 0:04:20.120 twenty two. Anyway, I'm very excited to say we're gonna 0:04:20.120 --> 0:04:22.440 be speaking with Paul Shari. He's the executive vice president 0:04:22.520 --> 0:04:25.120 at the Center for a New American Security and he's 0:04:25.160 --> 0:04:27.520 the author of two books related to this, one as 0:04:27.680 --> 0:04:30.240 the most recent for Battlegrounds Power in the Age of 0:04:30.320 --> 0:04:33.040 Artificial Intelligence, and then prior to that, he is the 0:04:33.080 --> 0:04:36.159 author of Army of non autonomous weapons and the future 0:04:36.240 --> 0:04:39.080 of war. He was previously in the Office of the 0:04:39.120 --> 0:04:42.760 Secretary of Defense, also a previous Army ranger, so truly 0:04:42.800 --> 0:04:45.080 the perfect guest. So, Paul, thank you so much for 0:04:45.279 --> 0:04:46.640 coming on odlots. 0:04:47.240 --> 0:04:49.440 Oh thank you for having me. Very excited to be here. 0:04:49.800 --> 0:04:50.760 What do you we start? 0:04:50.880 --> 0:04:53.880 I mentioned I had a feeling that maybe the definition 0:04:53.960 --> 0:04:57.360 of an autonomous weapon is a contested one. But if 0:04:57.400 --> 0:04:59.640 I say to you, what's an autonomous weapon, what's an 0:04:59.640 --> 0:05:00.559 autonom this weapon? 0:05:01.520 --> 0:05:03.280 So I think you're right from the beginning that there 0:05:03.320 --> 0:05:07.320 is not a unified definition that everyone agrees on. The 0:05:07.360 --> 0:05:11.080 Defense Department has their definition that's written in their policy. 0:05:11.480 --> 0:05:14.800 I think conceptually, I think the distinction really is a 0:05:14.839 --> 0:05:18.599 weapon that is choosing its own targets on the battlefield, 0:05:18.800 --> 0:05:20.920 and it's not where we are today. Right now, today 0:05:21.000 --> 0:05:24.240 people are choosing those targets. But it is kind of 0:05:24.240 --> 0:05:26.839 a spectrum because we do have examples of weapons that 0:05:26.920 --> 0:05:30.919 have some measure of autonomy. A good analogy might be 0:05:30.920 --> 0:05:35.160 self driving cars, where conceptually, like, okay, a self driving 0:05:35.200 --> 0:05:36.720 car would be where the AI is driving the car, 0:05:36.960 --> 0:05:39.640 but then you get into an actual car today and 0:05:39.680 --> 0:05:43.200 a lot of them have intelligence cruise control, automatic breaking, 0:05:43.760 --> 0:05:47.040 automated self parking, They have all these like automated features 0:05:47.040 --> 0:05:50.120 that are kind of creeping you in this direction where 0:05:50.160 --> 0:05:53.400 the AI is taking over more and more control for 0:05:53.520 --> 0:05:56.240 what the vehicle can do. And it's actually pretty similar 0:05:56.240 --> 0:05:57.560 thing in the military space as well. 0:05:58.080 --> 0:06:00.880 So Joe mentioned that had we been having this conversation 0:06:00.960 --> 0:06:03.920 even a month ago now, it probably would have had 0:06:04.160 --> 0:06:09.760 fewer concrete examples of AI enabled weaponry, let's say, or strategy. 0:06:10.320 --> 0:06:14.240 When the Pentagon talks about its advanced AI tools that 0:06:14.279 --> 0:06:18.360 it's deploying for the Iran conflict, what are some examples 0:06:18.400 --> 0:06:21.760 that you're seeing right now that are different to say, 0:06:22.080 --> 0:06:25.000 maybe just a year ago when we had another Iran conflict. 0:06:26.440 --> 0:06:29.080 Right, So, there's a couple ways in which the Pentagon 0:06:29.279 --> 0:06:33.200 is using AI right now. One is narrow AI systems 0:06:33.200 --> 0:06:36.440 that have around for over a decade now that do 0:06:36.760 --> 0:06:41.400 image classification. For example, So this was the military's original 0:06:41.400 --> 0:06:45.520 project Maven almost a decade ago, where they took machine 0:06:45.560 --> 0:06:49.320 learning image classifiers to sift through drone video feeds and 0:06:49.400 --> 0:06:53.080 saddle images to identify objects. Okay, here's a building, here's 0:06:53.120 --> 0:06:56.839 a person, here's a vehicle, that's pretty mature technology. Now, 0:06:56.960 --> 0:06:58.880 what's come out in just the last couple of weeks 0:06:59.000 --> 0:07:02.400 that's really quite interesting is that in the midst of 0:07:02.440 --> 0:07:06.160 this huge, messy public breakup between Anthropic and the Pentagon, 0:07:06.600 --> 0:07:10.000 we found out that in fact, Anthropics AI tools are 0:07:10.040 --> 0:07:13.840 being used by the US military to help plan the 0:07:13.880 --> 0:07:17.640 war against Iran. That's obviously a different kind of AI tool, 0:07:18.040 --> 0:07:22.120 AI Large language models, AI being used to write code 0:07:22.200 --> 0:07:25.040 AI agents, and that's being used in a different ways, 0:07:25.160 --> 0:07:29.640 really helping intel analysts sift through just the massive amounts 0:07:29.640 --> 0:07:33.080 of data that the US military has. And so you 0:07:33.080 --> 0:07:36.160 can imagine the problem that the military is facing right 0:07:36.200 --> 0:07:39.760 now when they're looking at targets in Iran. US military 0:07:39.760 --> 0:07:44.560 has flown over six thousand sorties against Iran. The Iranian 0:07:44.600 --> 0:07:47.480 military architecture is degraded in a lot of ways. US 0:07:47.520 --> 0:07:50.880 militaries are bombed a lot of targets. There are mobile targets, 0:07:51.160 --> 0:07:56.360 senior Iranian commanders, mobile missile launchers and air defense systems 0:07:56.480 --> 0:07:59.600 and drone launchers. US militter has got to bring off 0:07:59.640 --> 0:08:02.720 it in f together and find out where are these 0:08:02.720 --> 0:08:05.520 targets right now and where is there an aircraft that 0:08:05.640 --> 0:08:07.720 is the right bombs on it to take these targets out. 0:08:08.040 --> 0:08:10.600 And that's how AI is being used to help basically 0:08:11.040 --> 0:08:12.960 process and understand all that information. 0:08:13.240 --> 0:08:16.720 When I think about the description that you gave for that, 0:08:17.240 --> 0:08:19.200 I sometimes think like, could it be that? 0:08:19.760 --> 0:08:19.880 Now? 0:08:19.920 --> 0:08:23.120 I don't think that using anthropics technology mean so that 0:08:23.120 --> 0:08:26.520 they go into Claude dot AI and say, give us 0:08:26.520 --> 0:08:31.000 a list of suitable targets for sorties. But could it 0:08:31.040 --> 0:08:33.120 be something like that? But I'm sure there's a different 0:08:33.120 --> 0:08:36.839 interface and so forth. But is that a completely ridiculous 0:08:36.880 --> 0:08:40.800 way of essentially framing the service that AI is providing 0:08:40.880 --> 0:08:41.320 right now? 0:08:42.640 --> 0:08:45.800 So the way that these AI tools are being integrated 0:08:45.960 --> 0:08:49.559 are through a existing system called the Maven Smart System okay, 0:08:49.559 --> 0:08:52.880 which is built by Palenteer, and that fuses all this 0:08:53.000 --> 0:08:56.600 data together. So you basically have an existing architecture for 0:08:56.800 --> 0:09:00.680 data management for Intel analysts that the Milita already has 0:09:01.040 --> 0:09:03.120 that brings together all these different forms of data. You 0:09:03.200 --> 0:09:08.280 might have satellite imagery, geolocation, data, signals, intelligence, other forms 0:09:08.280 --> 0:09:12.960 of information. That's pretty great for Intel analysts, but that's 0:09:13.000 --> 0:09:16.520 also really unwieldy because how does a human understand all 0:09:16.559 --> 0:09:20.120 that data and process it? And that's where the large 0:09:20.160 --> 0:09:22.880 language model tools, whether it's Clawed or other companies can 0:09:22.880 --> 0:09:26.960 be valuable. Is that could be a way for a 0:09:27.080 --> 0:09:31.040 human to interact with that data, to basically task a 0:09:31.120 --> 0:09:33.280 large language model to say, Okay, here's a bunch of 0:09:33.400 --> 0:09:36.160 data I'm giving you. I want you to look for 0:09:36.720 --> 0:09:39.320 intersections and things, right. I want you to look for 0:09:39.360 --> 0:09:44.160 a place where we have satellite imagery and some other 0:09:44.280 --> 0:09:47.960 forms of intelligence that can help identify the location of 0:09:48.440 --> 0:09:51.800 you know, some missile launcher for example, and then humans 0:09:51.840 --> 0:09:55.040 can look at that and help one just like find 0:09:55.200 --> 0:09:58.920 where are all these targets, and then it helpfully planning too. 0:09:58.960 --> 0:10:01.320 A human could say, okay, here's this list of potential 0:10:01.360 --> 0:10:04.199 targets that I have. Now they're scattered all over run 0:10:04.240 --> 0:10:06.440 around is a really big country. I want to map 0:10:06.480 --> 0:10:10.760 these two locations for US aircraft at different bases across 0:10:10.760 --> 0:10:14.400 the region. What are available aircraft and what are available 0:10:14.600 --> 0:10:17.760 munitions on those aircraft? The commused to take out those 0:10:17.760 --> 0:10:21.280 targets to help build a strike package. So like, the 0:10:21.320 --> 0:10:24.800 AI is definitely being used to help understand the battlespace 0:10:24.800 --> 0:10:27.800 and to plan operations. But in I would say ways 0:10:27.800 --> 0:10:31.559 that are pretty narrowly directed by people It's not quite 0:10:31.559 --> 0:10:33.680 as simple as like, dump all this data into a 0:10:33.679 --> 0:10:37.000 context winder for LM and say, oh, AI figure it out. Okay, 0:10:37.000 --> 0:10:40.040 people are asking the AI some really specific questions, So. 0:10:40.280 --> 0:10:43.880 I'm thinking how to phrase this question diplomatically, But I 0:10:43.920 --> 0:10:47.480 get that the difference between fully autonomous weapons is, you know, 0:10:47.520 --> 0:10:51.640 the human as a decision maker in the current setup, 0:10:51.800 --> 0:10:55.520 how meaningful is the human actually? Like, what's your sense 0:10:55.559 --> 0:10:58.360 of it? Because I'm imagining if you're an intelligence officer 0:10:58.400 --> 0:11:01.920 and you're getting realms and reams data from Iran and 0:11:01.960 --> 0:11:04.880 you ask the AI to pick out certain patterns or 0:11:04.960 --> 0:11:09.440 identify potential strategic targets, how much due diligence are you 0:11:09.480 --> 0:11:12.560 actually doing on what that model spits out. Because of course, 0:11:12.640 --> 0:11:16.960 the tendency when a lot of people use LLM certainly 0:11:17.080 --> 0:11:19.440 is just you accept what it shows you on the screen. 0:11:19.559 --> 0:11:22.360 And just to add on to Tracy's question, because there's 0:11:22.440 --> 0:11:24.199 on to go, which is then in the early. 0:11:24.000 --> 0:11:26.120 Days of the war, we hit that school. 0:11:26.360 --> 0:11:29.520 Yeah, and I'm reading a New York Times report and 0:11:29.559 --> 0:11:33.880 that was the result of quote outdated data provided by 0:11:33.880 --> 0:11:37.319 the Defense Intelligence Agency. Now we don't know exactly what 0:11:37.360 --> 0:11:41.120 that means, but Okay, various outputs come out, then what happens, 0:11:41.200 --> 0:11:44.280 like how much is the human layer currently in terms 0:11:44.280 --> 0:11:46.959 of Okay, here are targets, here are ships that are 0:11:46.960 --> 0:11:50.200 on a battleship. This could be plausible. What do you 0:11:50.360 --> 0:11:53.000 think or what do you know about the level of 0:11:53.120 --> 0:11:57.000 human decision making that happens between some output and then 0:11:57.040 --> 0:11:59.760 the ultimate call for a strike on whatever it is. 0:12:00.920 --> 0:12:02.079 Yeah, I mean, I think, first of all, I think 0:12:02.080 --> 0:12:04.480 it's a really important question because it is one of 0:12:04.520 --> 0:12:08.040 the possible failure modes, if you will, of AI and 0:12:08.080 --> 0:12:10.199 how we use it, because you can end up in 0:12:10.240 --> 0:12:13.480 a place where humans are nominally in the loop and 0:12:13.520 --> 0:12:15.800 you could say, well, it's not an autonous weapon humans 0:12:15.840 --> 0:12:18.640 making these decisions. But if the human is not meaningfully 0:12:18.679 --> 0:12:22.040 engaged and they're just kind of rubber stamping some kind 0:12:22.040 --> 0:12:24.520 of decision, and that's not really what we're looking for. 0:12:24.600 --> 0:12:28.040 So I think that's it's been a longstanding concern for 0:12:28.120 --> 0:12:31.319 many years about people worried about autonous weapons. I think 0:12:31.320 --> 0:12:34.400 that's a very real risk with how AI is used now. 0:12:34.440 --> 0:12:37.480 Based on my understanding of how the AI technology is 0:12:37.559 --> 0:12:41.000 used in Maven today and based on what I've seen 0:12:41.040 --> 0:12:43.760 of demonstrations of it, because I have seen some demonstrations 0:12:43.800 --> 0:12:46.800 of this in action. I think humans are pretty involved 0:12:46.880 --> 0:12:50.600 right now in terms of actually looking at the output 0:12:50.679 --> 0:12:54.920 from AI, giving pretty specific guidance to the AI systems. 0:12:55.360 --> 0:12:59.440 I do think there is an underlying challenge that the 0:12:59.559 --> 0:13:02.920 strike on the school highlights, which is, when you're talking 0:13:03.000 --> 0:13:06.640 about thousands and thousands of targets, what's the degree of 0:13:06.800 --> 0:13:11.120 vetting that's gone into all of that information, both in 0:13:11.160 --> 0:13:14.360 the run up to the war, which in this case, 0:13:14.400 --> 0:13:16.839 that school was a fixed object, and so that's likely 0:13:16.960 --> 0:13:20.000 something that should have clearly been much more vetted prior 0:13:20.120 --> 0:13:23.760 to the we're kicking off. That someone could have identified 0:13:24.400 --> 0:13:27.760 that that building that was struck had actually been at 0:13:27.800 --> 0:13:30.120 one point in time part of an Iranian military compound. 0:13:30.480 --> 0:13:33.920 But you could see based on public available satellite imagery 0:13:34.320 --> 0:13:36.000 that it had been moved out of that compound some 0:13:36.080 --> 0:13:38.560 time ago and it had been converted to a school, 0:13:39.000 --> 0:13:41.680 and it would appear, based on what's been reported in 0:13:41.720 --> 0:13:44.880 the Times, that that information had never been updated in 0:13:44.920 --> 0:13:49.079 this DEIA targeting database. Now, I would hope that we'll 0:13:49.120 --> 0:13:51.600 get more information in the future and some investigation about 0:13:51.679 --> 0:13:54.079 exactly where that went wrong. But I think that does 0:13:54.160 --> 0:13:57.200 speak to this underlying challenge of how good is the 0:13:57.360 --> 0:14:00.679 data going into this AI system and how thoroughly are 0:14:00.720 --> 0:14:03.760 people vetting it? And again, in principle, AI might be 0:14:03.840 --> 0:14:05.800 able to help you with those things, but you got 0:14:05.840 --> 0:14:07.520 to use it the right way and people still have 0:14:07.559 --> 0:14:09.520 to be meaningfully engaging THECC. 0:14:25.760 --> 0:14:28.200 Why don't we back up for a second, tell us 0:14:28.200 --> 0:14:31.600 about the work that you've done in this area, really 0:14:31.640 --> 0:14:34.440 several years ahead of the curve, and talk about this stuff, 0:14:34.480 --> 0:14:36.600 planning for this stuff. Give us a little bit of 0:14:36.680 --> 0:14:40.120 sort of your background and what got you on this 0:14:40.240 --> 0:14:43.320 train again several years before Chad Shept. 0:14:44.880 --> 0:14:49.120 Yeah, so really over a decade ago now, around say 0:14:49.440 --> 0:14:53.320 twenty eleven, I let an effort inside the Pentagon when 0:14:53.320 --> 0:14:55.720 I worked at the Office the Sectary of Defense on 0:14:56.360 --> 0:14:59.920 developing the Pentagon's policy on the role of a ton 0:15:00.040 --> 0:15:02.400 I meet in weapons, the one that's still in effect 0:15:02.440 --> 0:15:06.960 today in fact, and that was really part of at 0:15:06.960 --> 0:15:10.120 the time. We weren't at all where the military is 0:15:10.160 --> 0:15:12.760 now in terms of integrating AI tools. I mean, these 0:15:12.800 --> 0:15:15.200 types of large language models just did exist at the time, 0:15:15.920 --> 0:15:19.680 but the military had kind of woken up to what 0:15:19.720 --> 0:15:23.600 I would call this accidental robotics revolution during the wars 0:15:23.600 --> 0:15:26.680 in the Rock and Afghanistan, where the military had deployed 0:15:26.720 --> 0:15:30.720 thousands of air and ground robots, drones in the air 0:15:30.760 --> 0:15:34.160 and ground robots for diffusing bombs, and the military was 0:15:34.200 --> 0:15:36.640 starting to think through where is this going in the future, 0:15:37.160 --> 0:15:39.040 And one of the things that everyone could see would 0:15:39.080 --> 0:15:42.320 be valuable would be having more autonomy in these systems, 0:15:42.360 --> 0:15:45.400 the ability to not be tortally reliant on a human 0:15:45.480 --> 0:15:48.160 remotely controlling them, which was really the case at the time, 0:15:48.760 --> 0:15:51.640 but that raised all these obviously the any questions about 0:15:51.640 --> 0:15:54.960 like well, how much autonomy should they have and what 0:15:55.040 --> 0:15:57.800 are the legal and ethical implications of that, And that 0:15:57.920 --> 0:16:00.600 was actually the topic of a lot of discussion among 0:16:00.720 --> 0:16:04.080 people in the military at the time and in the 0:16:04.080 --> 0:16:06.640 Pentagon for people working on these issues, and so that 0:16:06.760 --> 0:16:10.720 ultimately led to that policy directive that's still in place 0:16:10.720 --> 0:16:13.040 on the role of autonomy and weapons. And then when 0:16:13.040 --> 0:16:15.680 I left the government, I continued to work on this topic. 0:16:15.840 --> 0:16:20.080 Is we've seen discussions internationally through the United Nations as 0:16:20.080 --> 0:16:23.560 we've seen the technology evolve in really amazing ways but 0:16:23.640 --> 0:16:26.160 also ones out of risks with artificial intelligence. 0:16:26.760 --> 0:16:28.760 So when you were doing that job. 0:16:28.800 --> 0:16:31.000 I get that you're on the policy side, but did 0:16:31.000 --> 0:16:34.240 you ever see anything on the contractor side similar to 0:16:34.240 --> 0:16:36.800 what we're seeing with Anthropic right now. Was there ever 0:16:36.840 --> 0:16:40.680 a contractor who said, actually, no, I'm really uncomfortable with 0:16:40.720 --> 0:16:44.160 the way that the department wants to use this particular tech. 0:16:45.040 --> 0:16:48.640 Not at that time. Now, a few years later, after 0:16:48.720 --> 0:16:52.240 the US military launched Project mavin there was a big 0:16:52.480 --> 0:16:55.440 dust up when it came out publicly that Google had 0:16:55.480 --> 0:16:57.280 been a part of Product Maven, and a number of 0:16:57.280 --> 0:17:01.080 Google employees signed and open letter protesting and Google eventually 0:17:01.120 --> 0:17:04.480 discontinue their work on Project Mavane. And you know, it's 0:17:04.520 --> 0:17:08.200 not exact replica here of what's going on, but there's 0:17:08.200 --> 0:17:12.040 certainly some similarities in terms of a disconnect between how 0:17:12.080 --> 0:17:15.000 some people in the AI community are thinking about how 0:17:15.080 --> 0:17:17.439 their technology ought to be used in war and how 0:17:17.480 --> 0:17:19.880 the military is thinking about it. And I think part 0:17:19.880 --> 0:17:23.240 of that's like there's this underlying challenge of AI is 0:17:23.359 --> 0:17:27.000 really different than a lot of traditional military technologies because 0:17:27.000 --> 0:17:29.200 it's coming out of the commercial sector. So in a way, 0:17:29.200 --> 0:17:31.800 it's kind of like the opposite of stealth technology that 0:17:31.960 --> 0:17:34.600 was invented in secret defense labs and doesn't have a 0:17:34.640 --> 0:17:38.520 lot of commercial applications. AI is all of these different applications. 0:17:38.560 --> 0:17:41.280 It's not being invented by the military. The military is 0:17:41.280 --> 0:17:44.240 having to be import it in and there are a 0:17:44.280 --> 0:17:48.399 lot of debates about how AI should be used, you know, 0:17:48.440 --> 0:17:50.320 in the military and more broadly in society. 0:17:50.520 --> 0:17:53.920 Actually, on that note, I think this is really interesting 0:17:54.000 --> 0:17:56.320 and definitely a pivotal point in I guess, the history 0:17:56.320 --> 0:18:00.360 of the military industrial complex. But why can't the US government, 0:18:00.600 --> 0:18:04.399 with all its resources, actually develop AI in house and 0:18:04.480 --> 0:18:07.719 just avoid the seeming complication of having to deal with 0:18:07.760 --> 0:18:08.800 a commercial enterprise. 0:18:09.520 --> 0:18:13.840 Part ofly, it doesn't have the technical skills the AI 0:18:13.920 --> 0:18:19.399 scientists and engineers are. Really there's a fierce competition for 0:18:19.440 --> 0:18:22.359 talent in the AI space, and so the military just 0:18:22.359 --> 0:18:25.400 like can't buy that talent they don't have it. And 0:18:25.600 --> 0:18:27.720 the government spends a lot of money, hundreds of billions 0:18:27.760 --> 0:18:31.560 of dollars annually on defense. But we've seen actually in 0:18:31.560 --> 0:18:34.399 the last few years that private enterprise is able to 0:18:34.400 --> 0:18:38.040 mobile massive amounts of capital towards building data centers, to 0:18:38.320 --> 0:18:42.720 training AI models. And partly because the commercial applications for 0:18:42.720 --> 0:18:47.159 this technology are much bigger than the defense applications, and 0:18:47.240 --> 0:18:50.639 so for a lot of these tech companies there's some 0:18:51.400 --> 0:18:54.280 at least maybe not in this particular instance, but in 0:18:54.280 --> 0:18:57.200 the past there could offer be some prestige associated with saying, oh, 0:18:57.520 --> 0:19:01.280 you know, the Air Force uses our AI system or 0:19:01.320 --> 0:19:04.680 the Navy users our technology, but the defense sector is 0:19:04.680 --> 0:19:07.320 actually kind of small for them as a customer. I mean, 0:19:07.359 --> 0:19:10.000 the dollar amount that's been talked about publicly for the 0:19:10.000 --> 0:19:12.840 Anthropic contract, it's two hundred million dollars. That's not a 0:19:12.880 --> 0:19:15.320 lot of money for these AI companies. Yeah, and so 0:19:15.400 --> 0:19:18.640 I think that actually we've seen the defense sectors struggle 0:19:18.720 --> 0:19:21.480 to just keep pace with the amount of investment that's 0:19:21.480 --> 0:19:22.280 needed in the space. 0:19:22.960 --> 0:19:26.040 Cherioucy, I think it's a good question. And then you remember, well, 0:19:26.200 --> 0:19:29.600 the government couldn't build a good healthcare website to sign up. 0:19:29.560 --> 0:19:30.400 For health insurance. 0:19:30.640 --> 0:19:32.600 And I hate to bring that up because it's old, 0:19:33.000 --> 0:19:34.960 but it's true. Right, So it's like, are they gonna 0:19:35.000 --> 0:19:38.399 build a world class LLM or can a government build 0:19:38.440 --> 0:19:42.720 a good employment insurance website. We've done multiple episodes the 0:19:43.160 --> 0:19:45.680 answer continues to be not the case. I do find 0:19:45.680 --> 0:19:49.680 it fascinating. However, your point about there is this novelty. 0:19:50.200 --> 0:19:53.960 It is impossible to imagine and say Lockheed Martin inventing 0:19:54.000 --> 0:19:56.000 get technology and they's like, no, you can't use it. 0:19:56.040 --> 0:19:59.840 Because Lockheed Martin's entire raise on death Right is building. 0:19:59.600 --> 0:20:00.639 Technolog for the government. 0:20:00.680 --> 0:20:03.639 It is inconceivable what that would be. But it is 0:20:03.920 --> 0:20:07.720 sort of novel when you're getting these defend technologies. And 0:20:07.800 --> 0:20:09.920 you know the Google was also an example of Google 0:20:09.920 --> 0:20:13.840 obviously had technology that did not originally serve a purpose 0:20:14.080 --> 0:20:17.960 of defense. We saw the we remember the employee revolt. 0:20:18.240 --> 0:20:22.359 Let's talk more about that disagreement though, between Anthropic and 0:20:22.440 --> 0:20:26.360 the Department of Defense. In your mind, where does Peter 0:20:26.800 --> 0:20:30.760 heg Seth want to go with this technology? And is 0:20:30.800 --> 0:20:34.080 that deviate from some of the policies and the directives 0:20:34.080 --> 0:20:36.919 that you were working on when you were when you 0:20:36.960 --> 0:20:37.879 were working on this stuff. 0:20:38.720 --> 0:20:42.439 So it's kind of crazy about this whole dispute is 0:20:42.440 --> 0:20:46.360 particularly an issue of autonomous weapons. Literally everyone I've spoken 0:20:46.440 --> 0:20:50.000 with has said that there's no intention by the military 0:20:50.359 --> 0:20:53.800 to use AI to make fully autonomous weapons today. Okay, 0:20:53.840 --> 0:20:56.280 I think anybody that's actually worked with a large language 0:20:56.280 --> 0:20:58.840 model with any kind of chat bought, whether it's Claude 0:20:58.920 --> 0:21:02.600 or Gemini or chat Chie knows that if you use 0:21:02.640 --> 0:21:05.159 these to write an email, you need to double check it. Like, 0:21:05.200 --> 0:21:08.200 in no way, shape or form are they reliable enough 0:21:08.240 --> 0:21:10.520 to make life and death decisions. I don't think the 0:21:10.520 --> 0:21:14.160 military actually wants to do that. What's at dispute here 0:21:14.240 --> 0:21:17.520 is a more fundamental disagreement about well, who sets the 0:21:17.640 --> 0:21:20.840 rules and so the origins of this really was that 0:21:21.240 --> 0:21:23.960 when the Pentagon came out with a new strategy for 0:21:24.040 --> 0:21:27.240 AI in January, one of the things in their strategy 0:21:27.320 --> 0:21:30.320 was that going forward, they wanted their contracts with AI 0:21:30.359 --> 0:21:33.800 companies to allow the military to use their AD tools 0:21:33.800 --> 0:21:37.760 for any lawful use. Basically, look, anything that's legal, we 0:21:37.760 --> 0:21:39.880 want the ability to do it. And that has conflicted 0:21:39.920 --> 0:21:41.439 with how a lot of these tech companies have been 0:21:41.440 --> 0:21:44.320 thinking about their AI tools. They're very nervous, many of 0:21:44.320 --> 0:21:47.880 these companies about harms from AI. They're conscious of these risks, 0:21:48.080 --> 0:21:50.000 and so a lot of them have various use policies 0:21:50.080 --> 0:21:52.879 in place. You can't use AI, you know, to launch 0:21:52.920 --> 0:21:56.200 offensive cyber attacks for example. That's kind of thing that 0:21:56.359 --> 0:21:59.159 actually like the government might want to do. And so 0:21:59.840 --> 0:22:02.480 this is a This was really the rub with the 0:22:02.480 --> 0:22:06.440 government was like who sets the rules rather than necessarily 0:22:06.640 --> 0:22:09.800 like a near term question of fully autonomous weapons. 0:22:09.960 --> 0:22:13.600 So what we've already seen is Anthropic has this disagreement 0:22:13.920 --> 0:22:18.439 with the government, and then OpenAI steps in and raises 0:22:18.480 --> 0:22:20.639 its hand and says, okay, Anthropic doesn't want to do it, 0:22:20.640 --> 0:22:23.480 We'll do it happily. Does this just leave us in 0:22:23.520 --> 0:22:26.680 a situation where it's sort of a race to the bottom, Right, 0:22:26.720 --> 0:22:29.840 It's like the lab with maybe the least amount of 0:22:29.920 --> 0:22:34.080 safety concern or the least amount of reputational concern is 0:22:34.119 --> 0:22:36.520 able to do this, and so we still wind up 0:22:36.560 --> 0:22:39.399 in a situation where the government is using AI. 0:22:40.680 --> 0:22:43.280 Well, I think what's unfortunate here is that when you 0:22:43.280 --> 0:22:46.320 think about would be optimal for the government one, I 0:22:46.359 --> 0:22:47.959 think it would be ideal for the government to have 0:22:47.960 --> 0:22:51.600 access to this technology and to have access to all 0:22:51.680 --> 0:22:55.080 of the best in class models available, because they are 0:22:55.160 --> 0:22:58.920 good at slightly different things sometimes, and it's much healthier 0:22:58.920 --> 0:23:01.359 for the government to have access to a number of 0:23:01.400 --> 0:23:04.760 different providers so that there is healthy competition in the market. 0:23:04.800 --> 0:23:07.120 You don't get locked in with one vendor. But also, 0:23:07.720 --> 0:23:10.800 if the AI scientists are saying, hey, it's not reliable 0:23:10.800 --> 0:23:13.600 for this, you to listen like that seems like a 0:23:13.680 --> 0:23:16.520 thing you want to hear them like out about right, 0:23:16.680 --> 0:23:19.560 And so I think like in order to use AI 0:23:19.600 --> 0:23:23.400 and ways that actually are effective for the US military, 0:23:23.640 --> 0:23:26.680 we've got to have a healthy dialogue between the AI 0:23:26.760 --> 0:23:29.639 community and people in the military profession. But what the 0:23:29.680 --> 0:23:33.919 technology can and cannot do. And I think it's unfortunate 0:23:33.960 --> 0:23:35.960 that we've seen that dialogue break down in such a 0:23:36.000 --> 0:23:37.320 dramatic way over the speed. 0:23:37.880 --> 0:23:40.359 Just going back to the idea, who actually makes the 0:23:40.440 --> 0:23:43.920 rules you mentioned earlier that you know you can't use 0:23:44.400 --> 0:23:47.600 claude to hack into to illegally hack into a system. 0:23:47.600 --> 0:23:49.879 Supposedly it is unable to do that. It has like 0:23:49.920 --> 0:23:52.760 a kill switch within itself that prevents it from doing that. 0:23:53.000 --> 0:23:55.560 If you're anthropic, could you not just hardcode some of 0:23:55.600 --> 0:23:58.320 these systems and say you're not going to be able 0:23:58.320 --> 0:24:03.080 to be used for domestic surveillance of Americans or for 0:24:03.359 --> 0:24:04.120 war crimes. 0:24:05.000 --> 0:24:06.879 So yeah, So this is where it gets a little 0:24:07.000 --> 0:24:09.440 more technical. It has to do with some of the 0:24:09.480 --> 0:24:13.080 ways in which the companies may be providing their technology 0:24:13.119 --> 0:24:15.160 to the government. So there's a couple of different ways 0:24:15.160 --> 0:24:17.439 in which an AI company can put safeguards in place 0:24:17.800 --> 0:24:21.320 to make sure that their models not being abused. One 0:24:21.400 --> 0:24:24.639 is training the model itself to refuse certain requests. So 0:24:24.680 --> 0:24:26.639 if you ask the model to do something, it's just 0:24:26.640 --> 0:24:28.280 going to say, like, I'm not going to do that. 0:24:28.280 --> 0:24:30.640 That's not consistent with the guidance that I've been given, 0:24:30.640 --> 0:24:32.680 and the model has been trained to do that response. 0:24:33.480 --> 0:24:37.800 Another way is that the company can put classifiers on 0:24:37.840 --> 0:24:41.160 the input and or the output of a model, where 0:24:41.800 --> 0:24:44.400 the model might give you an answer, but then there's 0:24:44.440 --> 0:24:47.320 like another aisystem that's checking that answer or checking what 0:24:47.359 --> 0:24:50.000 you ask of it and saying, well, that's not acceptable. 0:24:50.280 --> 0:24:52.800 And then a third and I've want into that actually 0:24:52.840 --> 0:24:55.240 myself in my own research because the nature of the 0:24:55.240 --> 0:24:57.520 things that I work on a security things, and I've 0:24:57.520 --> 0:25:01.480 had situations where I asked Claude help me understand this issue. 0:25:01.640 --> 0:25:05.120 Claud actually generates response and then it gets deleted. It's 0:25:05.119 --> 0:25:07.439 really interesting to see. And then the other way an 0:25:07.440 --> 0:25:10.840 anthropic has actually talked about this in response to countering 0:25:11.359 --> 0:25:15.960 some use of Claude by Chinese hackers who are using 0:25:15.960 --> 0:25:21.720 it for cyber attacks, is the company monitors use through 0:25:22.280 --> 0:25:25.320 that people are doing, and so people are doing things 0:25:25.359 --> 0:25:28.080 that seem suspicious. Maybe they're logging in from an IP 0:25:28.200 --> 0:25:30.920 address that's known to be associated with cyber criminals or 0:25:30.960 --> 0:25:33.840 a hacking group that to try and find ways to 0:25:33.880 --> 0:25:36.560 get around some of these protections. The company can also 0:25:36.600 --> 0:25:38.440 find ways to try to catch that, and so there's 0:25:38.440 --> 0:25:41.040 a couple different ways to do it that might not 0:25:41.240 --> 0:25:45.000 all be in place. If you're thinking about military use 0:25:45.040 --> 0:25:48.600 where if depending on how that relationship is structure between 0:25:48.600 --> 0:25:51.320 the company. If the model is, for example, that hosted 0:25:51.520 --> 0:25:55.040 on a different cloud infrastructure and the military is direct 0:25:55.040 --> 0:25:57.359 access to it, the company may not have the same 0:25:57.600 --> 0:26:01.879 ways to actually shape what or not the technology is 0:26:01.920 --> 0:26:05.040 being used according to their principles, which is partly why 0:26:05.040 --> 0:26:07.560 the contract details do matter of like what is the 0:26:07.600 --> 0:26:10.640 agreement between the company the government about what the military 0:26:10.640 --> 0:26:28.520 can and cannot use the technology trace. 0:26:28.600 --> 0:26:31.040 I think your point about like this sort of seemingly 0:26:31.680 --> 0:26:34.560 safety or safety race at the bottom is very real 0:26:34.600 --> 0:26:36.640 and it's one that I think about a lot. When 0:26:37.400 --> 0:26:41.760 LMS or AI was basically just synonymous with open AI, 0:26:42.160 --> 0:26:44.080 they could set the pace of development. 0:26:43.760 --> 0:26:44.600 Right, they could do it. 0:26:44.800 --> 0:26:47.960 As soon as this became a hyper competitive space where 0:26:47.960 --> 0:26:50.239 you have open AI and you have anthropic, and you 0:26:50.240 --> 0:26:54.120 have Gemini and a thousand open source AI models out 0:26:54.160 --> 0:26:58.560 of China, et cetera. The tempo of release has really heightened, 0:26:58.600 --> 0:27:01.680 and the degree which it feels like they have no 0:27:01.800 --> 0:27:05.560 choice but to accelerate just for the commercial imperative feels 0:27:05.560 --> 0:27:07.960 like a very real dynamic in which like, I don't 0:27:08.000 --> 0:27:10.600 know where that leaves AI safety well totally. 0:27:10.640 --> 0:27:13.560 And also you mentioned China. Then it's not just domestic 0:27:13.600 --> 0:27:17.119 competition became, you know, open AI versus anthropic, it's a 0:27:17.160 --> 0:27:20.000 competition between international actors where it's like, Okay, well, the 0:27:20.080 --> 0:27:22.800 US might want to have safeguards on technology or say 0:27:22.800 --> 0:27:25.760 that it does, but maybe Russia or China don't care. 0:27:26.080 --> 0:27:28.160 Right tell you you know, it's funny all you mentioned 0:27:28.200 --> 0:27:30.280 where you like see the output for one second and 0:27:30.280 --> 0:27:33.560 then deletes, Like when deep Seak came out doing some 0:27:33.680 --> 0:27:37.679 experiments about like figuring out its censorship and I was 0:27:37.720 --> 0:27:40.600 trying to do some adversarial prompting and I was like, 0:27:41.080 --> 0:27:43.880 historians like to talk about a period in the twentieth 0:27:43.880 --> 0:27:48.480 century where it failed. Attempt at extreme rapid industrialization happened 0:27:48.480 --> 0:27:50.399 and it led to famine. And then you see the 0:27:50.480 --> 0:27:53.760 output and it said Okay, what happened in the twentieth century? 0:27:53.960 --> 0:27:56.280 Where did this famine? Well, there's something called the Great 0:27:56.359 --> 0:27:59.040 Leap Forward, and then immediately it just as soon as 0:27:59.080 --> 0:27:59.640 the chain of. 0:27:59.560 --> 0:28:01.840 Thought hit the Great Leap Forward, it just like disappeared. 0:28:01.880 --> 0:28:04.159 So I'm always like very amused by like when the 0:28:04.240 --> 0:28:07.439 system recognizes that the system has gone too far. Anyway, 0:28:08.080 --> 0:28:12.480 we've been talking about quote large language models, but actually 0:28:12.520 --> 0:28:15.760 AI is beyond large language models, including the image stuff, 0:28:15.760 --> 0:28:18.520 and that actually lllm's at this point. It's a very 0:28:18.560 --> 0:28:22.040 twenty twenty three sort of term. And when and I 0:28:22.040 --> 0:28:24.000 think this is important because when we get to the 0:28:24.040 --> 0:28:28.720 intersection of AI and robotics and so forth, or AI 0:28:28.920 --> 0:28:32.119 and target, we're talking about something but beyond large language model, 0:28:32.320 --> 0:28:34.640 but we might still be talking about generative AI. 0:28:35.200 --> 0:28:36.719 Where do you see this going? 0:28:36.960 --> 0:28:41.720 And what are the weapon systems that aren't currently You said, currently, 0:28:41.760 --> 0:28:45.520 no one is actually talking about true autonomous weapons. But 0:28:45.560 --> 0:28:47.840 if that were the case, then there wouldn't be a controversy. 0:28:47.880 --> 0:28:50.000 There was, so there was clearly something just beyond the 0:28:50.000 --> 0:28:52.920 horizon that could come into the picture of a true 0:28:52.960 --> 0:28:56.200 autonomous weapon system where the technology is building towards that. 0:28:56.680 --> 0:28:58.400 If this weren't the case, there would be no dispute, 0:28:58.440 --> 0:29:01.240 you wouldn't have two books written about this subject. So 0:29:01.280 --> 0:29:05.200 what are these weapons systems that would classify as autonomous 0:29:05.240 --> 0:29:09.120 weapons today that the technology is building towards right now? 0:29:10.480 --> 0:29:13.040 Yeah, I mean I certainly the trends are taking us there. 0:29:13.040 --> 0:29:14.640 And one of the things that you see in the 0:29:14.680 --> 0:29:17.120 Pentagon's position in this issue, for example, is they want 0:29:17.120 --> 0:29:19.920 to preserve that option. Yeah, for there's certainly not interested 0:29:19.920 --> 0:29:22.360 in tying their hands. I think you could see that 0:29:22.480 --> 0:29:26.160 evolving in a couple ways. One trend we're clearly seeing 0:29:26.280 --> 0:29:30.280 with the largest and most capable AI systems is they're 0:29:30.320 --> 0:29:33.440 increasingly multimodal. They're bringing in lots of different forms of data, 0:29:33.480 --> 0:29:35.760 of course, and they're increasing the general purpose they could 0:29:35.760 --> 0:29:38.080 just like do a variety of different kinds of things. 0:29:38.080 --> 0:29:40.440 They become more capable of that, and so that's like 0:29:40.600 --> 0:29:45.680 one way in which you could see AI being used 0:29:46.120 --> 0:29:50.040 in ways that might sort of slowly pulls humans out 0:29:50.040 --> 0:29:53.120 of the loop, where instead of person giving an AI 0:29:53.240 --> 0:29:56.680 like really narrow tasks to do in a planning process, 0:29:56.880 --> 0:29:58.720 maybe the AI is able to take on more, bring 0:29:58.840 --> 0:30:02.280 in more data, take on more like sophisticated longer term tasks, 0:30:02.280 --> 0:30:04.320 and we're certainly seeing this in other areas like coding, 0:30:04.720 --> 0:30:07.000 where the task length that an AI system could do 0:30:07.120 --> 0:30:11.080 is growing exponentially over time. Another sort of way that 0:30:11.120 --> 0:30:13.480 we might see this look is just we see a 0:30:13.520 --> 0:30:17.440 network of AI agents that are interacting with different pieces 0:30:17.480 --> 0:30:20.560 of data, doing different types of things, and that the 0:30:20.600 --> 0:30:24.200 net effect of that is that maybe humans are again 0:30:24.240 --> 0:30:27.160 sort of like nominally looking at these targets but not 0:30:27.200 --> 0:30:30.560 actually approving them in some meaningful way. And then there's 0:30:30.640 --> 0:30:32.400 like a more separate I would almost think of like 0:30:32.400 --> 0:30:35.680 an embodied form of AI in robotics, which could be 0:30:35.720 --> 0:30:40.200 a drone or munition or robotic system that has some 0:30:40.280 --> 0:30:45.040 kind of onboard autonomy that might be partly a distilled 0:30:45.280 --> 0:30:47.480 model so that it can be operating at the edge 0:30:47.520 --> 0:30:51.680 on lower computing on this actual munition or drone, or 0:30:51.800 --> 0:30:54.720 it might be some hybrid system that has partly machine 0:30:54.760 --> 0:30:57.920 learning but also just a lot of hand coded code. 0:30:58.240 --> 0:31:01.240 It's more like expert level system that's going out into 0:31:01.320 --> 0:31:04.560 the battle space and hunting targets directly and attacking them. 0:31:04.640 --> 0:31:06.840 So something kind of like the low cost drones that 0:31:06.840 --> 0:31:10.760 we're seeing Iran launch, but once the can loiter and 0:31:11.040 --> 0:31:12.720 identify targets that attack them. 0:31:12.880 --> 0:31:16.440 Whether that doesn't exist today. We don't have drones loiter 0:31:16.920 --> 0:31:19.959 that are just hanging out there. And then when something flat, 0:31:20.080 --> 0:31:21.920 then there's a system is like this looks like a 0:31:21.960 --> 0:31:24.920 target attacks That actually doesn't exist currently as far as you. 0:31:24.880 --> 0:31:29.760 Know, well, I mean, they're not in widespread use. So 0:31:29.800 --> 0:31:33.200 there have been some narrow examples, I would say historically 0:31:33.480 --> 0:31:38.000 dating back to the eighties, in fact of lording munitions 0:31:38.400 --> 0:31:41.320 that could search over wider and would queue off of radars, 0:31:41.720 --> 0:31:44.600 and so radars are what the military would call a 0:31:44.640 --> 0:31:48.360 cooperative target that when they're emitting in the electromaenetic spectrum, 0:31:48.680 --> 0:31:50.680 if you know the signature of the radar you're looking for, 0:31:50.840 --> 0:31:52.120 you could see it. You can just home in on 0:31:52.120 --> 0:31:54.600 that radar. Now, if they turn off, that's different than 0:31:54.920 --> 0:31:57.480 kidden and they're harder to find. But there have been 0:31:57.520 --> 0:32:01.440 some examples. A system that the US Navy had in 0:32:01.440 --> 0:32:04.560 the eighties called the Tomahawk anti shit missile, not actually 0:32:04.640 --> 0:32:07.480 the same Tomahawk cruise missile that the military is using now, 0:32:07.720 --> 0:32:10.400 a different one that was designed to fly a search 0:32:10.480 --> 0:32:13.760 pattern and hunt Soviet chips. There was an Israeli system 0:32:13.760 --> 0:32:16.120 called the Harpy drone that was designed to go after 0:32:16.240 --> 0:32:18.440 radars that would loorder for a period of time. But 0:32:18.520 --> 0:32:21.479 these lording musicians have never really been in widespread use 0:32:21.520 --> 0:32:22.280 by militaries. 0:32:22.440 --> 0:32:24.560 We got to invent one of those, like high pitched 0:32:24.640 --> 0:32:28.800 alarms to deter the loitering drones from hanging out outside targets. 0:32:28.800 --> 0:32:30.479 I guess I mean we have electrical jammings. 0:32:30.560 --> 0:32:31.400 Yeah, that does exist. 0:32:31.720 --> 0:32:34.480 Okay, So when I think about as we move towards 0:32:34.640 --> 0:32:39.080 more autonomous weaponry, I think about bots basically interacting with 0:32:39.160 --> 0:32:41.560 bots at that point, and then I think back to 0:32:41.640 --> 0:32:44.800 previous examples of bots interacting with bots, and there are 0:32:45.160 --> 0:32:47.640 numerous ones where things tend to go off the rails. 0:32:47.720 --> 0:32:49.640 They just start debating the meaning of life, right or. 0:32:49.640 --> 0:32:52.080 They start talking in like a language that no one 0:32:52.160 --> 0:32:53.680 understands accept them. 0:32:53.520 --> 0:32:54.200 Stuff like that. 0:32:54.840 --> 0:33:01.120 Does the possibility of undesired escalation go up the more 0:33:01.200 --> 0:33:03.480 we move towards fully autonomous weaponry. 0:33:05.280 --> 0:33:08.080 I think that is a very serious risk. And so 0:33:08.800 --> 0:33:11.800 the mental model that I have for this are things 0:33:11.840 --> 0:33:16.000 like flash crashes that we've seen in financial markets due 0:33:16.000 --> 0:33:20.320 to the interactions of different algorithms that are executing trades 0:33:20.640 --> 0:33:23.400 where you get these emergent properties of how the algorithms 0:33:23.440 --> 0:33:26.680 might interact in the market, and it's a competitive environment. 0:33:27.000 --> 0:33:28.800 Companies aren't going to share the details of what their 0:33:28.800 --> 0:33:32.040 algorithms are doing, and you can do strange behaviors. Now, 0:33:32.080 --> 0:33:34.960 the way that financial markets have dealt with this problem 0:33:35.120 --> 0:33:38.120 is regulators have installed circuit breakers that take stocks offline 0:33:38.120 --> 0:33:40.560 if the price moves too quickly. There's no referee to 0:33:40.560 --> 0:33:42.800 call time out in war, and so I think that's like, 0:33:42.880 --> 0:33:46.520 particularly in cyberspace, one can envision a future where that 0:33:46.680 --> 0:33:48.840 is a risk, where things are happening at machine speed 0:33:49.400 --> 0:33:53.720 and you have autonomous offensive cyber operations. You need to 0:33:53.760 --> 0:33:57.239 defend against that. You need some measure of autonomy on 0:33:57.280 --> 0:34:00.840 the defensive side to defend it. Machines space need and 0:34:00.880 --> 0:34:04.080 you could get situations where you get weird interactions that 0:34:04.160 --> 0:34:06.840 might escalate the conflict, or it could also happen between 0:34:06.920 --> 0:34:11.640 drones interacting in some kind of crisis situation. Now, a 0:34:11.640 --> 0:34:14.000 situation where like there's a big shooting war in the 0:34:14.080 --> 0:34:16.640 way people are already attacking, there might be less of 0:34:16.680 --> 0:34:20.320 a concern, although you still could worry about escalation geographically 0:34:20.480 --> 0:34:23.600 against bringing new countries into a conflict, or maybe attacking 0:34:24.040 --> 0:34:27.200 really sensitive sites that are tied to nuclear command and 0:34:27.200 --> 0:34:29.480 control that you'd rather not go after. So I think 0:34:29.480 --> 0:34:31.520 that's a very real risk when we think about how 0:34:31.560 --> 0:34:33.320 this technology might be employed going forward. 0:34:33.800 --> 0:34:39.719 What about AI in really difficult ethical questions strikes where 0:34:39.719 --> 0:34:42.399 we know that civilians, for example, are going to be killed, 0:34:42.440 --> 0:34:45.440 which that happens all the time in war and presumably 0:34:45.840 --> 0:34:50.080 tries to be minimized, but war planners will find some 0:34:51.040 --> 0:34:56.040 level of acceptable they called collateral damage. Is AI playing 0:34:56.160 --> 0:34:58.799 a role or do you expect it to play a 0:34:59.000 --> 0:35:02.120 role in some of these strikes? 0:35:02.160 --> 0:35:03.360 That may be great areas. 0:35:04.400 --> 0:35:06.400 I think you could envision in ways that a I 0:35:06.440 --> 0:35:09.160 would be used that would make warfare more precise and 0:35:09.200 --> 0:35:11.200 more human and ethical, and ways that it could be 0:35:11.280 --> 0:35:13.239 used that would not and what can be opposite. So, 0:35:13.280 --> 0:35:17.840 for example, if you had an AI system that could 0:35:17.880 --> 0:35:21.719 look over all this targeting data and then identify if 0:35:21.719 --> 0:35:24.759 a strike is within a certain distance using you know, 0:35:24.880 --> 0:35:29.080 musis of a certain size of protected targets, whether it's 0:35:29.120 --> 0:35:34.319 schools or hospitals or critical civilian infrastructure, and say hey, well, 0:35:34.560 --> 0:35:38.120 like warning here, you should not carry out the strike 0:35:38.320 --> 0:35:41.800 or it's a higher level of approval, or maybe you 0:35:41.800 --> 0:35:44.520 should use small or more precise munitions. That would be 0:35:44.520 --> 0:35:47.239 a really beneficial use of AI, particularly when you're talking 0:35:47.239 --> 0:35:50.080 about a military campaign that hits a lot of targets 0:35:50.080 --> 0:35:52.080 in a short period of time, That could be really 0:35:52.120 --> 0:35:56.200 valuable and may reduce civilian casualties. You know, the risk 0:35:56.280 --> 0:35:58.160 of all of this is you could end up in 0:35:58.160 --> 0:36:03.080 a world where humans are just less engaged in this process, right, 0:36:03.160 --> 0:36:05.880 and so there's both mistakes that humans miss or humans 0:36:05.920 --> 0:36:09.040 just don't feel as morally responsible, which I think is 0:36:09.080 --> 0:36:13.200 like a really tricky thing to think about morally because 0:36:14.000 --> 0:36:17.560 on the one hand, as a democratic society we make 0:36:17.600 --> 0:36:19.759 a decision as a nation to go to war. It's 0:36:19.760 --> 0:36:21.680 a very small number of people that have to carry 0:36:21.680 --> 0:36:24.879 that burden. And if someone if you could say, well, look, 0:36:25.000 --> 0:36:28.800 what's the benefit to someone having like PTSD years after 0:36:28.840 --> 0:36:31.600 a conflict that they're haunted mess having happened, that doesn't 0:36:31.600 --> 0:36:34.360 seem great. Maybe we could reduce that. On the other hand, 0:36:34.560 --> 0:36:37.520 if we thought of war and nobody felt morally responsible 0:36:37.560 --> 0:36:40.040 for the killing that occurred, that doesn't seem good either, 0:36:40.320 --> 0:36:43.760 and that could lead to more suffering and civilian casualties 0:36:43.760 --> 0:36:46.320 and war. So I think that's certainly a concern we 0:36:46.360 --> 0:36:47.799 think about how to use the technology. 0:36:48.080 --> 0:36:51.680 Yeah, this is very enders game coded, right, where you 0:36:51.760 --> 0:36:54.120 have someone who's basically playing like a video game and 0:36:54.160 --> 0:36:56.960 wiping out entire civilizations and they think it's just a 0:36:57.040 --> 0:36:59.120 video game, just an exercise, but it turns out it's 0:36:59.160 --> 0:37:02.160 actual warfare. And we're seeing some degree of that in 0:37:02.200 --> 0:37:05.520 the way that the Department of War is portraying this 0:37:05.600 --> 0:37:09.880 conflict so far. It's very video game, yes. 0:37:09.200 --> 0:37:10.480 Especially in public presentation. 0:37:10.760 --> 0:37:13.960 Yeah, literal animated gifts of video games exactly. 0:37:14.320 --> 0:37:18.240 So, Paul, you mentioned something you mentioned the word circuit breaker, 0:37:18.760 --> 0:37:22.200 and circuit breakers are nice things to have in markets. 0:37:22.280 --> 0:37:24.319 I think they'd be even nicer things to have in 0:37:24.520 --> 0:37:27.600 armed conflict and war. Is there any possibility that you 0:37:27.680 --> 0:37:32.360 could design something like that for a major conflict. 0:37:32.680 --> 0:37:34.719 I think it's possible, like at a technical level, to 0:37:34.719 --> 0:37:37.200 figure out how you would do that and where you 0:37:37.239 --> 0:37:39.759 put protections on your side in the military, and what 0:37:39.800 --> 0:37:43.080 you would do with even maybe cooperatively with an enemy. 0:37:43.320 --> 0:37:45.560 The challenge is how do you avoid what we were 0:37:45.560 --> 0:37:48.040 talking about earlier, a race to the bottom on safety. Yeah, 0:37:48.080 --> 0:37:50.719 and right, we're seeing this in the private sector between 0:37:50.719 --> 0:37:52.640 the AA companies as the Russian to get products out 0:37:52.680 --> 0:37:56.360 the market. I think it's especially hard in the military space, 0:37:56.360 --> 0:38:00.680 where countries are investing in the military because they're worried 0:38:00.719 --> 0:38:03.359 about what some other adversary might do and then want 0:38:03.400 --> 0:38:05.719 to get a leg up on them. And so it's 0:38:05.760 --> 0:38:09.920 not that cooperation in the midst of conflict never happens. 0:38:09.960 --> 0:38:15.120 It does, and countries have agreed to take certain weapons 0:38:15.239 --> 0:38:19.400 off the table chemical and biological weapons, for example. It 0:38:19.400 --> 0:38:22.840 doesn't mean that they're never used, but most civilized countries 0:38:22.840 --> 0:38:25.240 that were not going to use them. But those examples 0:38:25.239 --> 0:38:28.120 are pretty rare and it's pretty hard to do, and 0:38:28.200 --> 0:38:30.920 some think that dynamic is the really challenging one. It's like, 0:38:30.960 --> 0:38:34.280 how do you find ways to cooperate with your enemies 0:38:34.600 --> 0:38:36.480 to avoid some of the biggest dangers here. 0:38:37.200 --> 0:38:39.799 So I think there's the last question for me. You know, 0:38:39.880 --> 0:38:43.160 you mentioned that drones are kind of robot and there 0:38:43.160 --> 0:38:46.320 are other robots that have been existence in either national 0:38:46.320 --> 0:38:50.680 security or police work for a while. I think there 0:38:50.719 --> 0:38:53.400 are robots on the subway sometimes that seem to be 0:38:53.520 --> 0:38:55.480 really yeah, but I don't think they really. 0:38:55.280 --> 0:38:57.279 See the robots at the grocery store and they end 0:38:57.360 --> 0:38:59.480 up like chasing me while I'm trying to buy like 0:38:59.680 --> 0:39:01.920 rare or something. Yeah, the ones that sweep the floors 0:39:01.960 --> 0:39:02.360 and stuff. 0:39:02.440 --> 0:39:03.480 Yeah, there's the robots. 0:39:03.600 --> 0:39:06.480 Yeah, I think there is a Eric Adams did a 0:39:06.520 --> 0:39:08.600 contract with some company that was doing like subway for 0:39:08.760 --> 0:39:10.880 cards or something like that. But these are really different 0:39:11.280 --> 0:39:13.759 ai as we talk about it in robots are two 0:39:13.760 --> 0:39:17.520 different technological trees, but they are going to merge, and 0:39:17.560 --> 0:39:21.400 there's the possibility of their ultimate merger. Do you foresee 0:39:21.440 --> 0:39:25.839 a world in which essentially we don't have human soldiers 0:39:26.080 --> 0:39:30.520 and wars are fought with who has the most advanced 0:39:30.560 --> 0:39:33.839 autonomous robots? We know China is investing a lot in 0:39:33.920 --> 0:39:37.560 humanoid robotics. Do you foresee a world in which that 0:39:37.760 --> 0:39:41.279 is the nature of a ground invasion as you and 0:39:41.520 --> 0:39:43.840 it happens with robots or various other sorts, talk to 0:39:43.920 --> 0:39:45.319 us about how far that could go. 0:39:46.520 --> 0:39:48.919 Yeah, So, I mean, look, I think where we see 0:39:49.000 --> 0:39:52.600 robots whose more and were for absolutely the long arc 0:39:52.680 --> 0:39:56.160 of technology in war, from the first time someone picked 0:39:56.200 --> 0:39:58.160 up a rocket through it at somebody else has been 0:39:58.680 --> 0:40:04.120 towards great distance between adversaries moving up through bows and 0:40:04.239 --> 0:40:07.239 arrows and rifles and intercontinental ballistic missiles. And I think 0:40:07.520 --> 0:40:09.880 robotics will be the next evolution of this trend of 0:40:09.960 --> 0:40:12.879 finding ways to find the enemy strike the enemy without 0:40:12.880 --> 0:40:15.760 putting yourself at risk. And there's certainly a role for 0:40:16.280 --> 0:40:19.040 robotics out on the battlefield. I think a vision of 0:40:19.200 --> 0:40:23.000 like future wars of just robots fighting robots, there's no 0:40:23.080 --> 0:40:26.040 humans in wealth. That's not realistic for a couple of reasons. 0:40:26.080 --> 0:40:29.080 One is I think militaries are going to need people 0:40:29.239 --> 0:40:34.440 relatively forward deployed to execute command and control for robotic systems. 0:40:35.200 --> 0:40:37.880 The US military right now can fly drones remotely from 0:40:37.920 --> 0:40:42.279 the United States in a relatively uncontested environment against more 0:40:42.320 --> 0:40:45.759 sophisticated adversaries who could jam your communications like and we see, 0:40:45.800 --> 0:40:47.680 for example, like a lot of jamming on the front 0:40:47.719 --> 0:40:50.399 lines in Ukraine. That's one of the ways you should 0:40:50.400 --> 0:40:53.439 go after these drones. Then you need people close by 0:40:53.920 --> 0:40:58.080 because it is easier to have shorter range protected communications. 0:40:58.200 --> 0:41:00.000 When you go to a longer distances, that's just hard 0:41:00.200 --> 0:41:02.439 to do. So I think given be people relatively close 0:41:02.480 --> 0:41:04.840 for that reason. I think if you want to control territory, 0:41:05.080 --> 0:41:07.680 you have to put people there eventually to get out 0:41:07.680 --> 0:41:10.200 of a vehicle and walk around and control it. But 0:41:10.239 --> 0:41:12.200 I think the other reason is maybe a little dark, 0:41:12.239 --> 0:41:15.680 which I think realistically, in order for wars to end, 0:41:16.360 --> 0:41:19.120 there will have to be some human price that's paid. 0:41:19.160 --> 0:41:22.440 I think that's an unfortunate reality that if it's just 0:41:22.680 --> 0:41:26.239 machines that are being destroyed, that we may not get 0:41:26.280 --> 0:41:28.960 to the place where one side or the other is 0:41:29.040 --> 0:41:32.759 willing to suit for peace. And I think, unfortunately, war 0:41:32.840 --> 0:41:36.880 is likely to involve people and human costs for a 0:41:36.960 --> 0:41:37.640 very long time. 0:41:38.400 --> 0:41:40.319 I have one more question as well, and I guess 0:41:40.360 --> 0:41:42.319 it's a thought experiment, but if we think back to 0:41:42.600 --> 0:41:46.239 sort of pivotal moments in military history and their intersection 0:41:46.440 --> 0:41:49.400 with technology, one of them that comes up is the 0:41:49.480 --> 0:41:53.279 Russian officer who decided not to press the button in 0:41:53.360 --> 0:41:56.879 response to the US and thereby, you know, supposedly save 0:41:56.960 --> 0:42:01.960 the world from nuclear disaster. Happen in a fully autonomous 0:42:02.000 --> 0:42:04.560 military environment nowadays, I. 0:42:04.560 --> 0:42:06.880 Mean today, it would still happen because there's people involved. Right, 0:42:06.920 --> 0:42:11.200 So this incident standis la Petrov, He's sitting at a 0:42:11.280 --> 0:42:14.960 terminal and it gets this warning that there's a ballistic 0:42:15.000 --> 0:42:17.920 missile launched from the United states against the Soviet Union, 0:42:18.320 --> 0:42:21.839 and then another missile, another five missiles coming in. And 0:42:22.239 --> 0:42:25.279 the thing that's interesting about this is when Petrov talked 0:42:25.280 --> 0:42:27.759 about it afterwards, and we could hear what he said 0:42:27.800 --> 0:42:30.400 because we all lived because he made the right decision. 0:42:30.400 --> 0:42:33.480 Here is he talked about how he said he had 0:42:33.480 --> 0:42:35.880 a funny feeling in his gut and that he knew 0:42:35.920 --> 0:42:38.279 that the Russian system, the Russians had just deployed, or 0:42:38.280 --> 0:42:41.359 said the Soviets rather just deployed a new satellite based 0:42:41.400 --> 0:42:44.960 early warning system to detect us ICBM launches. That it 0:42:45.000 --> 0:42:46.360 was new, and he knew that a lot of the 0:42:46.400 --> 0:42:49.000 Soviet technology doesn't work that great at first, so he 0:42:49.080 --> 0:42:51.680 was skeptical of it. Turns out it was in fact faulty. 0:42:51.760 --> 0:42:54.359 He was detecting the reflection of sunlight off the top 0:42:54.400 --> 0:42:57.320 of clouds and the system was identifying that as a 0:42:57.640 --> 0:43:01.160 missile launch, and that's what it was. And he went 0:43:01.280 --> 0:43:04.359 and then called the early warning radar stations and staid, 0:43:04.360 --> 0:43:06.239 are you seeing his missiles come over the horizon and said, no, 0:43:06.320 --> 0:43:08.640 there's not missiles. So he reported up the chain that 0:43:08.640 --> 0:43:12.040 the system was malfunctioning. I think the scary question here 0:43:12.120 --> 0:43:14.040 is like if that was an AI, what would the 0:43:14.080 --> 0:43:17.000 AI have done? And it's kind of like whatever it 0:43:17.040 --> 0:43:20.520 was programmed to wherever it was trained to do. And 0:43:21.360 --> 0:43:25.040 obviously we're seeing more general purpose AI systems like large 0:43:25.080 --> 0:43:29.000 language models, have the ability to bring together more information, 0:43:29.239 --> 0:43:32.960 to understand better context, to have just like a more 0:43:33.000 --> 0:43:35.759 contextual understanding of the questions that you're asking of it. 0:43:36.280 --> 0:43:39.360 But it still doesn't know the stakes of a conference. 0:43:39.400 --> 0:43:42.160 It still doesn't know, like in some visceral level, what 0:43:42.200 --> 0:43:44.800 the consequences are. And so I think that's a strong 0:43:44.880 --> 0:43:48.160 compelling reason why we need to have humans involved these decisions. 0:43:48.600 --> 0:43:51.359 Even as the AI becomes more capable, there's still gonna 0:43:51.360 --> 0:43:53.960 be things we want humans to do because humans understand 0:43:53.960 --> 0:43:54.680 why it matters. 0:43:55.719 --> 0:43:59.080 I started the conversation by mentioning that you know, it's 0:43:59.160 --> 0:44:02.600 not very common traversial to say, have an anti missile 0:44:02.640 --> 0:44:06.080 system fire a missile when there's one coming in, but 0:44:06.440 --> 0:44:08.040 that could be wrong, and you want to make sure 0:44:08.040 --> 0:44:10.600 that it is in fact a missile and not a 0:44:10.640 --> 0:44:13.520 civilian air jet or something like that. Be So even 0:44:13.680 --> 0:44:16.880 there where it seems like a canonical example, if you 0:44:16.920 --> 0:44:19.759 just want to have the missile system go off, you 0:44:19.760 --> 0:44:23.080 would want to have human safeguards and human oversight and 0:44:23.160 --> 0:44:27.840 human understanding of the system such that it is in 0:44:28.000 --> 0:44:32.879 fact shooting down missile anyway, Paul Sharie, fascinating conversation. Really 0:44:32.920 --> 0:44:35.040 appreciate you coming out on the up lots and talking 0:44:35.040 --> 0:44:35.560 about your work. 0:44:36.360 --> 0:44:37.720 Thank you. Really joyed the discussion. 0:44:38.239 --> 0:44:41.239 Thanks so much, Paul. That was depressing and fascinating. 0:44:41.360 --> 0:44:44.160 Yeah at the time, Yeah, no, it was great. It 0:44:44.200 --> 0:44:45.000 was Thank you so much. 0:44:45.400 --> 0:44:46.359 Yeah, Facebook cover. 0:44:59.080 --> 0:44:59.680 I kind of get. 0:44:59.640 --> 0:45:01.880 Choked up at the end thinking about that decision that 0:45:02.080 --> 0:45:05.000 saved humanity at the end, And. 0:45:05.000 --> 0:45:06.680 It's actually it's a crazy story. 0:45:06.719 --> 0:45:09.000 It's a crazy story. It's one of those stories that like, 0:45:09.080 --> 0:45:10.720 why doesn't everybody know. 0:45:10.920 --> 0:45:12.479 That that person's name? 0:45:12.640 --> 0:45:14.799 I mean, when you think about how many people, I 0:45:14.840 --> 0:45:15.960 couldn't remember it either. 0:45:16.280 --> 0:45:18.480 But shout out to another podcast if you want to 0:45:18.520 --> 0:45:21.759 learn more about this. Dan Carlin's Hardcore History has at 0:45:21.880 --> 0:45:27.680 least one, possibly two episodes on narrow aversions of nuclear disaster. 0:45:27.920 --> 0:45:31.200 So very good to listen to, if not terrifying. 0:45:31.320 --> 0:45:34.040 You know, there's another point in that exact story that 0:45:34.120 --> 0:45:36.600 I think is really interesting and it's something I've been 0:45:36.680 --> 0:45:40.680 thinking about a lot across AI because There's something similar 0:45:40.719 --> 0:45:45.200 about humans and AI, which is that there is definitely 0:45:45.320 --> 0:45:48.799 a gap between what we know and what we can articulate. 0:45:49.040 --> 0:45:51.600 And this is certainly true with AI. Right, so the 0:45:51.640 --> 0:45:54.480 bot makes some decision, or it determines something, it does 0:45:54.520 --> 0:45:56.440 not mean it's going to be able to spit out 0:45:56.480 --> 0:45:58.960 in words, how it arrived at that decision. 0:45:59.160 --> 0:46:01.080 But that's true for humans as well. 0:46:01.120 --> 0:46:04.200 And so the idea that like, okay, maybe we do 0:46:04.239 --> 0:46:07.560 get funny feelings about or take you know, like, again, 0:46:08.000 --> 0:46:11.439 we're still pretty good at determining the difference between AI 0:46:11.480 --> 0:46:15.280 generated texts and human generated text can we but often 0:46:15.680 --> 0:46:17.799 I mean we're still like often could get it right, 0:46:18.719 --> 0:46:21.560 but could we write down exactly what we saw that 0:46:21.600 --> 0:46:25.959 we understood? There is that gap, and when we're talking 0:46:25.960 --> 0:46:31.040 about life or death decisions being made, it is scary 0:46:31.160 --> 0:46:34.480 to think about that role of instinct that we can't 0:46:34.600 --> 0:46:37.480 articulate having been taken out of the decision. Well. 0:46:37.520 --> 0:46:40.480 I think also technology is very good at pattern recognition, 0:46:40.760 --> 0:46:44.319 right and responding to patterns and preset paths. It's been 0:46:44.360 --> 0:46:47.600 programmed to do certain things, and I think in a 0:46:47.640 --> 0:46:50.719 war environment, that's one of the most uncertain environments that 0:46:50.719 --> 0:46:53.399 you can possibly imagine, and so you have to think 0:46:53.480 --> 0:46:57.400 that there should be some element of flexibility in your response, 0:46:57.440 --> 0:47:00.600 But I don't know how you actually encode that into 0:47:00.640 --> 0:47:03.800 a thing that like runs on rigid numbers and lines 0:47:04.160 --> 0:47:06.360 lines of code. The other thing I was thinking was 0:47:06.560 --> 0:47:09.960 the anthropic situation and just how new that is from 0:47:10.000 --> 0:47:12.440 a sort of military history perspective, in the sense that 0:47:12.520 --> 0:47:16.879 here we have this really important pivotal piece of technology 0:47:16.920 --> 0:47:21.120 that hasn't come out of like actual military demand. Right 0:47:21.280 --> 0:47:24.520 to Paul's point, it's a commercial product. Its commercial uses 0:47:24.560 --> 0:47:27.920 are arguably a lot more profitable than its military ones, 0:47:28.160 --> 0:47:31.799 and so seeing that now interact with the Pentagon and 0:47:31.880 --> 0:47:34.080 the Department of War really interesting. 0:47:34.200 --> 0:47:35.279 It's been flipped right. 0:47:35.560 --> 0:47:35.840 Yeah. 0:47:35.880 --> 0:47:39.040 The closest example actually that comes to mind, there is 0:47:39.080 --> 0:47:42.760 one example that's in fairly recent history, and it's starlink 0:47:42.840 --> 0:47:44.920 oh yeah co And of course that was developed for 0:47:44.920 --> 0:47:47.960 commercial internet purposes, but it played a role in Ukraine 0:47:48.280 --> 0:47:51.280 and so forth, and at one point, if I recall, 0:47:51.680 --> 0:47:54.839 there was a tension point about the degree to which 0:47:54.880 --> 0:47:58.040 the Ukrainians could use and so I do think that 0:47:58.320 --> 0:48:00.759 is sort of an interesting parallel here. 0:48:00.960 --> 0:48:02.480 The other thing that we didn't. 0:48:02.239 --> 0:48:04.600 Get to this and I this is gonna be a 0:48:04.640 --> 0:48:07.439 little cynical, but I think it's right, which is that 0:48:08.200 --> 0:48:12.600 there's another element I believe to the Anthropic situation, which is, 0:48:12.680 --> 0:48:15.040 like Anthropic is the last. 0:48:14.800 --> 0:48:16.000 Big lib tech company. 0:48:16.239 --> 0:48:19.239 We're perceived as such, right, and we know that there's 0:48:19.280 --> 0:48:22.120 been this fairly sort of right word turn in Silicon 0:48:22.200 --> 0:48:25.920 Valley over the years. And I don't think like Anthropic 0:48:26.040 --> 0:48:28.120 is like totally part of that. I think there's still 0:48:28.200 --> 0:48:31.600 sort of lib coded. I also think it's incidentally why 0:48:31.640 --> 0:48:33.319 a bunch of people who probably like work in media 0:48:33.360 --> 0:48:35.400 are like end up using Claude even though they're all 0:48:35.440 --> 0:48:37.880 kind of the same. Like, I do think there's something 0:48:37.920 --> 0:48:41.080 there and that they have this thing they say, we're 0:48:41.120 --> 0:48:41.880 not going to ever. 0:48:41.880 --> 0:48:44.720 Have ads, and we know that. 0:48:44.600 --> 0:48:47.319 Like Andreas and Horowitz, Mark Andresen just talked about ads 0:48:47.360 --> 0:48:50.600 are good. Ads democratize the internet. Ads enabled the Internet 0:48:50.640 --> 0:48:53.080 to be spread to everyone. There are some other politics 0:48:53.080 --> 0:48:56.800 at play. Because again, like from my understanding, and it 0:48:56.840 --> 0:48:58.480 would take a lawyer, it's like, I don't think that 0:48:58.560 --> 0:49:02.040 the agreement that open as Ey sign was that different 0:49:03.080 --> 0:49:06.040 than what the agreement that Anthropic had. There is probably 0:49:06.040 --> 0:49:08.000 a little bit of difference. I just think there's some 0:49:08.080 --> 0:49:13.720 other politics and play here. Per to Paul's point, maybe 0:49:13.760 --> 0:49:19.600 nobody is talking about currently autonomous weapons right now fully autonomous, 0:49:19.680 --> 0:49:23.000 but it can't be long, and I think this is 0:49:23.040 --> 0:49:24.040 going to be a real tension. 0:49:24.120 --> 0:49:24.959 Sooner resan later. 0:49:25.040 --> 0:49:26.480 Can I say one thing and I'm going to be 0:49:26.560 --> 0:49:30.000 slightly facetious, but also not Can you be slightly facetious. 0:49:30.040 --> 0:49:30.560 I'm going to be. 0:49:30.480 --> 0:49:33.160 Facetious, which is I have a solution to modern warfare. 0:49:33.760 --> 0:49:34.399 Don't do it? 0:49:35.960 --> 0:49:38.319 Okay, but no for real? 0:49:38.680 --> 0:49:41.480 Yeah, if we're just gonna have bots fighting points and 0:49:41.520 --> 0:49:43.440 it's going to cost a lot of money and result 0:49:43.480 --> 0:49:49.000 in people's deaths, every country should have to build its biggest, best, 0:49:49.239 --> 0:49:52.719 most technologically advanced robot and just have them fight it 0:49:52.760 --> 0:49:56.400 out gladiatorial. And my twist is to Paul's point about 0:49:56.400 --> 0:50:00.840 war always having to be painful in some way, Everyone 0:50:00.960 --> 0:50:04.240 in that particular society has to be engaged and dedicate 0:50:04.320 --> 0:50:08.640 some amount of time or money to building that particular robot, 0:50:09.040 --> 0:50:11.720 and you just have to iterate on the robot forever 0:50:11.880 --> 0:50:14.759 until you feel comfortable to have them fight, and that 0:50:14.840 --> 0:50:17.840 way everyone shares the pain, but without the loss of 0:50:17.920 --> 0:50:19.320 human life. Am I high. 0:50:19.400 --> 0:50:20.640 I don't think so well. 0:50:20.840 --> 0:50:22.279 I think you should write a book. No, I don't 0:50:22.280 --> 0:50:23.520 think you should write a sci fi book. 0:50:23.640 --> 0:50:24.839 All right, shall we leave it there. 0:50:24.920 --> 0:50:25.640 Let's leave it there. 0:50:25.800 --> 0:50:27.960 This has been another episode of the Authoughts podcast. 0:50:28.040 --> 0:50:31.080 I'm Tracy Alloway. You can follow me at Tracy Alloway. 0:50:30.800 --> 0:50:33.560 And I'm Joe Wisenthal. You can follow me at the Stalwart. 0:50:33.680 --> 0:50:37.120 Follow our guest Paul Shari. He's at Paul Underscore Shari. 0:50:37.440 --> 0:50:40.600 Follow our producers Carmen Rodriguez at Carmen armat dash, Ol 0:50:40.600 --> 0:50:42.000 Bennett at Dashbot. 0:50:41.680 --> 0:50:43.520 And kill Brooks at Kelbrooks. 0:50:43.800 --> 0:50:46.080 For more Odd Lots content, go to Bloomberg dot com 0:50:46.080 --> 0:50:48.239 slash odd Lots, where the daily newsletter and all of 0:50:48.280 --> 0:50:50.279 our episodes, and you can chaut about all of these 0:50:50.320 --> 0:50:53.439 topics twenty four to seven in our discord Discord dot 0:50:53.520 --> 0:50:54.880 gg slash onlines. 0:50:54.920 --> 0:50:56.719 And if you enjoy all thoughts, if you like it 0:50:56.760 --> 0:50:59.399 when we talk about the future of autonomous weapons, then 0:50:59.400 --> 0:51:02.640 please leave us a positive review on your favorite podcast platform. 0:51:02.760 --> 0:51:05.239 And remember, if you are a Bloomberg subscriber, you can 0:51:05.280 --> 0:51:07.960 listen to all of our episodes absolutely ad free. 0:51:08.080 --> 0:51:10.600 All you need to do is find the Bloomberg. 0:51:10.120 --> 0:51:12.799 Channel on Apple Podcasts and follow the instructions there. 0:51:13.200 --> 0:51:14.040 Thanks for listening.