WEBVTT - why is the government fighting with claude? 0:00:09.840 --> 0:00:12.880 Yo, what's going on? It's Texter. So technology does not 0:00:13.039 --> 0:00:15.920 slow down for anybody. And sometimes I wish that I 0:00:15.920 --> 0:00:18.520 could just talk to y'all live about what's happening right now. 0:00:19.160 --> 0:00:21.120 But then we were talking about it, we realized, wait 0:00:21.120 --> 0:00:24.000 a second, we can talk to you live. So we're 0:00:24.000 --> 0:00:26.319 gonna try something new here. Every once in a while, 0:00:26.320 --> 0:00:28.639 we're gonna talk about something that's happening right now and 0:00:28.720 --> 0:00:31.400 let people jump into the audience to ask questions, and 0:00:31.440 --> 0:00:33.720 we'll bring a guest along. And so what you're about 0:00:33.760 --> 0:00:36.240 to listen to is our first episode that we live 0:00:36.280 --> 0:00:39.239 streamed on Wednesday, March third. So if you want to 0:00:39.240 --> 0:00:41.440 catch the next one as we're recording it and maybe 0:00:41.440 --> 0:00:44.199 even ask some questions, you can subscribe to our YouTube 0:00:44.240 --> 0:00:46.280 page and you can catch the links to that in 0:00:46.360 --> 0:00:57.200 the description. So everyone, this is we're doing something different here. 0:00:57.240 --> 0:01:01.319 We're doing the first live episode of kill Switch. We're 0:01:01.360 --> 0:01:03.720 gonna probably try to do these maybe once a month, 0:01:04.200 --> 0:01:06.480 just to give people opportunity to actually jump into the 0:01:06.560 --> 0:01:10.560 chat and talk about stuff that is happening right now. 0:01:10.640 --> 0:01:13.360 So thanks for bearing with us while we experiment a 0:01:13.400 --> 0:01:17.160 little bit. But today what I'd like to talk about 0:01:17.520 --> 0:01:21.160 is what is happening with Anthropic, what's going on with 0:01:21.200 --> 0:01:23.760 open Ai, what's going on with the US government. To 0:01:23.800 --> 0:01:26.440 talk about it today, our guest is Will Knight, who's 0:01:26.480 --> 0:01:29.360 a senior writer at Wired. He's been writing about AI 0:01:29.480 --> 0:01:32.360 for a while. He's got a weekly AI newsletter called 0:01:32.400 --> 0:01:35.040 AI Lab. Will Welcome to kill Switch. 0:01:35.080 --> 0:01:37.280 Welcome in, Thank you for having me. I'm wanting to 0:01:37.280 --> 0:01:40.120 be on the first experimental live stream. 0:01:40.480 --> 0:01:43.679 Yeah, hey, look, we'll see. Everything's in experiment at this point, 0:01:43.720 --> 0:01:48.080 for better or for worse. Sure, So I want to 0:01:48.160 --> 0:01:51.520 start this off with a timeline, Okay. On the twenty seventh, 0:01:52.080 --> 0:01:55.280 Donald Trump gets on truth Social and says that Claude 0:01:55.320 --> 0:01:59.480 is a radical left woke company. Okay, yeah, out of nowhere, 0:02:00.040 --> 0:02:02.720 People like, what why is he saying this? Pete heg 0:02:02.800 --> 0:02:06.840 Saith gets on Twitter and says that Claude Anthropic, the 0:02:06.840 --> 0:02:10.200 company that makes Claude, is a supply chain risk, which 0:02:10.200 --> 0:02:12.320 most people have never thought about what that even means. 0:02:13.000 --> 0:02:17.040 The next day, Open Ai, who makes chat GPT, announces 0:02:17.080 --> 0:02:19.680 that they're working with the Department of War, and then 0:02:19.800 --> 0:02:23.880 Donald Trump announces that we're bombing Iran. As I say 0:02:23.919 --> 0:02:27.480 that that sounds like a completely unhinged sequence of events. 0:02:28.360 --> 0:02:30.880 Please tell me where I'm wrong, because I must be 0:02:30.919 --> 0:02:31.640 missing something. 0:02:32.120 --> 0:02:34.320 I mean, I think I think there's a lot that's 0:02:34.960 --> 0:02:37.840 that's right in there and a lot that's unhinged. But 0:02:38.639 --> 0:02:40.639 I mean to sort of set the scene a little bit, 0:02:41.040 --> 0:02:43.920 you know. So this goes back. So the use of 0:02:44.240 --> 0:02:49.440 Claude in classified military systems has its origins last year, 0:02:49.520 --> 0:02:52.799 last summer, Anthropic was the first to get involved in 0:02:53.160 --> 0:02:55.480 working with the military. You know, they've pursued a very 0:02:55.880 --> 0:02:58.519 business focus, and I think that sort of led them 0:02:58.520 --> 0:03:00.440 to focus on the d D, which is like one 0:03:00.480 --> 0:03:02.840 of the biggest businesses in the US. So they ended 0:03:02.919 --> 0:03:05.880 up supplying Claude their model, a version of that which 0:03:05.960 --> 0:03:11.400 runs on the Pentagon's own systems for military use on 0:03:11.520 --> 0:03:15.480 classified systems. In many ways, this is totally remarkable because 0:03:16.160 --> 0:03:18.320 just a few years ago, everybody in the AI industry 0:03:18.440 --> 0:03:20.480 was saying, one thing we don't want to do is 0:03:20.520 --> 0:03:22.800 work with the military, and people were staging, you know, 0:03:22.880 --> 0:03:26.120 walkouts over this, right, So it shows you how the 0:03:26.160 --> 0:03:29.000 tech world has really pivoted to that. What happened more 0:03:29.040 --> 0:03:33.800 recently was that its themes and you know, this is 0:03:33.840 --> 0:03:38.360 somewhat disputed but there's been reports that when President Maduro 0:03:38.520 --> 0:03:42.280 was captured, Claude was used, and somebody at Anthropic may 0:03:42.320 --> 0:03:46.520 have raised questions about that. This seems like it snowballed. 0:03:46.840 --> 0:03:49.760 And one of the carve outs in the original contract 0:03:49.840 --> 0:03:53.680 signed last year there were two. One was against mass 0:03:53.680 --> 0:03:56.800 surveillance of US citizen and the others was that the 0:03:56.840 --> 0:04:00.160 model shouldn't be used to build fully autonomous weapons. And 0:04:01.280 --> 0:04:06.200 the DoD or W said that they wanted to change 0:04:06.240 --> 0:04:08.520 the contract essentially, which is an unusual thing to do, 0:04:08.760 --> 0:04:11.760 but they wanted to allow all your lawful use. And 0:04:12.080 --> 0:04:15.200 to some degree, this dispute seems like it's a little 0:04:15.200 --> 0:04:17.520 bit like people, you know, each side talking past each 0:04:17.520 --> 0:04:19.920 other to some degree. And I think that the DOW 0:04:20.080 --> 0:04:24.160 the Department of War became very concerned about a tech 0:04:24.200 --> 0:04:27.200 company saying how it could and couldn't use the technology. 0:04:27.400 --> 0:04:30.360 And to be fair, that is quite an unusual idea, right. 0:04:30.400 --> 0:04:33.680 It's like somebody selling a missile or a jet fighter 0:04:33.720 --> 0:04:35.760 to the Pentagon and then later on saying we don't 0:04:35.760 --> 0:04:37.320 want you to use it in this way or that way. 0:04:38.080 --> 0:04:40.680 But then again, ai is is a really different technology, 0:04:40.760 --> 0:04:43.760 is very new, it's very experimental and I believe anthropic, 0:04:44.200 --> 0:04:46.719 especially among AI companies, it's very focused on safety and 0:04:46.720 --> 0:04:49.920 they have concerns just about how it could end up 0:04:49.960 --> 0:04:50.640 being used. 0:04:50.960 --> 0:04:54.240 Yeah, well, so let's jump into this so anthropic you know, 0:04:54.640 --> 0:04:57.680 this is a statement that is put on anthropic dot com. 0:04:57.760 --> 0:05:00.400 So this is their own, you know, pr statement here 0:05:00.920 --> 0:05:03.000 and the title is statement from Daria Ama Day on 0:05:03.040 --> 0:05:06.320 our discussions with the Department of War, and it starts 0:05:06.360 --> 0:05:09.720 out the first line, I believe deeply in the existential 0:05:09.720 --> 0:05:12.800 importance of using AI to defend the United States and 0:05:12.839 --> 0:05:17.840 other democracies and to defeat our autocratic adversaries. So already 0:05:17.839 --> 0:05:20.240 out of the gate, that's kind of an interesting thing 0:05:20.440 --> 0:05:23.320 for again a company in Silicon Valley to be saying, 0:05:23.480 --> 0:05:27.560 even just in the past couple of years, really, but 0:05:27.680 --> 0:05:30.360 it seems like the two big points of contention here, 0:05:30.440 --> 0:05:34.600 there's two. He says, using these systems for mass domestic 0:05:34.800 --> 0:05:39.560 surveillance is incompatible with democratic values. So spying out other people, 0:05:39.880 --> 0:05:42.640 we're okay with that. Spying on people in the US, 0:05:42.960 --> 0:05:45.320 We're not okay with that. Right, And then the next 0:05:45.320 --> 0:05:49.400 thing is, quote frontier AI systems are simply not reliable 0:05:49.520 --> 0:05:54.400 enough to power fully autonomous weapons both of those sound 0:05:54.760 --> 0:05:59.239 totally okay and reasonable. Not only do they sound okay 0:05:59.279 --> 0:06:02.120 and reasonable, but I mean, look, I've spent a whole 0:06:02.120 --> 0:06:05.600 lot of time around people on the pretty far rate 0:06:06.680 --> 0:06:09.200 and you know, for work, right, and they will tell 0:06:09.240 --> 0:06:12.919 you I don't want no government spying on me, and 0:06:13.000 --> 0:06:16.680 I don't trust these computers. Right, where's the beef? What's 0:06:16.720 --> 0:06:17.520 the problem here? 0:06:18.200 --> 0:06:20.640 That's a great question. You know. I've spent a lot 0:06:20.680 --> 0:06:24.919 of time talking to people in the Pentagon and military 0:06:25.520 --> 0:06:29.520 defense companies, and nobody believes that those frontier models are 0:06:29.520 --> 0:06:32.679 ready were even remotely near ready being used in autonomous weapons, 0:06:33.360 --> 0:06:35.760 And consistently the government said we have no desire to 0:06:35.839 --> 0:06:39.160 do either of those things. I think the way what 0:06:39.240 --> 0:06:42.200 it really came down to was that they didn't just 0:06:43.120 --> 0:06:46.280 you know, take a knee and agree to change the 0:06:46.360 --> 0:06:48.960 terms of their contract. And as you say, you know, 0:06:49.720 --> 0:06:54.120 it culminated in Trump and Hegxeth and others putting out 0:06:54.120 --> 0:06:58.080 these very very you know, as often happens, they were 0:06:58.080 --> 0:06:59.680 sort of classifying them all of a sudden as the 0:06:59.720 --> 0:07:03.359 radical left, which is it's pretty incredible for company supplying 0:07:03.360 --> 0:07:06.400 technology that's working on classified US systems. 0:07:06.520 --> 0:07:10.080 Right, So this is the post that Donald Trump made 0:07:10.120 --> 0:07:13.680 on his social media network Truth Social and this I'm 0:07:13.720 --> 0:07:16.600 reading from the top, and I'm reading verbatim. The United 0:07:16.600 --> 0:07:20.120 States of America will never allow a radical left woke 0:07:20.280 --> 0:07:25.280 company to dictate how a great military fights and wins wars. 0:07:25.480 --> 0:07:28.120 That decision belongs to your commander in chief and the 0:07:28.120 --> 0:07:31.400 tremendous leaders I appoint to run are military. And then 0:07:31.480 --> 0:07:33.200 he finally gets to what he's talking about. He says, 0:07:33.200 --> 0:07:37.760 the left wing nut jobs Anthropic have made a disastrous 0:07:37.800 --> 0:07:41.160 mistake trying to strong arm the Department of War and 0:07:41.200 --> 0:07:43.520 force them to obey their terms of service instead of 0:07:43.520 --> 0:07:44.320 our constitution. 0:07:44.760 --> 0:07:45.320 Yeah. 0:07:45.400 --> 0:07:47.400 I mean, like a lot of that doesn't really even 0:07:47.920 --> 0:07:51.120 make sense. I think Anthropic is not a radical left 0:07:51.720 --> 0:07:56.160 organization by a stretch. It's since come out that that 0:07:56.200 --> 0:07:59.240 should The DoD was negotiating with that for they were 0:07:59.240 --> 0:08:01.880 trying to get this war odding kind of massalage into 0:08:01.880 --> 0:08:04.040 a format that everybody was happy with. And it doesn't 0:08:04.080 --> 0:08:06.840 seem like that would have been impossible. It feels like 0:08:07.560 --> 0:08:10.880 some people within the administration lost patients, And I think 0:08:10.880 --> 0:08:12.960 a lot of it came down to just power and 0:08:14.360 --> 0:08:18.560 the Department of Defense and the leadership not being able 0:08:18.640 --> 0:08:20.960 to handle being denied that by a tech company that 0:08:21.000 --> 0:08:23.720 they could very easily see as as you know, part 0:08:23.760 --> 0:08:26.800 of this sort of you know, coastal elite and in 0:08:26.840 --> 0:08:30.160 the case of Anthropic, woke and liberal. I mean, it's 0:08:30.200 --> 0:08:32.880 another reflection of how AI is just is it sort 0:08:32.920 --> 0:08:36.640 of also a cultural lightning rod, right, it becomes suddenly 0:08:36.640 --> 0:08:41.559 it becomes this kind of left versus right. So, I mean, 0:08:41.760 --> 0:08:46.199 you know that that post kind of does unfortunately sum 0:08:46.280 --> 0:08:49.480 up the entire situation, which is slightly bewildering if you 0:08:49.920 --> 0:08:52.800 know that now they've they've taken the action to try 0:08:52.800 --> 0:08:56.440 and to classify, as you said, Anthropic a supply chain risk, 0:08:56.880 --> 0:08:59.040 which is something that's only ever been to foreign nations. 0:08:59.040 --> 0:09:02.319 And we're doing this to one of the foremost AI 0:09:02.440 --> 0:09:05.280 companies in the United States, one of them I supposedly, 0:09:05.360 --> 0:09:09.200 you know, should be our national champions. It it doesn't 0:09:09.240 --> 0:09:13.560 sound like to me they're really promoting American technology. If 0:09:13.559 --> 0:09:15.319 you if you're too keen to do that, I think 0:09:15.360 --> 0:09:16.920 you don't. You know, it's a real danger you're going 0:09:17.000 --> 0:09:21.160 to harm some of you know, this this nascent, very 0:09:21.160 --> 0:09:24.439 important American company also maybe dissuade other companies from working 0:09:24.480 --> 0:09:26.880 with the government or the Pentagon if they feel that 0:09:26.880 --> 0:09:27.839 that's going to happen. 0:09:27.640 --> 0:09:32.480 Right, Yeah, So the supply chain risk thing again that 0:09:32.480 --> 0:09:35.720 that's the language that Pete Hegseth used. What does it 0:09:35.800 --> 0:09:40.319 mean to call anthropic a supply chain risk? What does 0:09:40.360 --> 0:09:41.960 that even mean? Yeah? 0:09:42.000 --> 0:09:43.599 I mean, you know, well, the first thing is to 0:09:43.640 --> 0:09:49.760 say that this supply chain risk designation is incredibly unusual, extraordinary, 0:09:49.880 --> 0:09:53.040 and just the kind of thing that I think the 0:09:53.080 --> 0:09:56.480 sort of left field thinking that we're become accustomed to 0:09:56.600 --> 0:09:59.440 with the administration because it's usually applied to, you know, 0:10:00.040 --> 0:10:02.400 a company that you want to remove from the US 0:10:02.520 --> 0:10:05.439 supply chain because you're worried that their software or their 0:10:05.440 --> 0:10:09.800 hardware might contain backdoors, might be a threat to national 0:10:09.800 --> 0:10:13.000 security because it could be used by an adversary. So 0:10:13.120 --> 0:10:14.839 this is the very sort of all or nothing. You 0:10:15.280 --> 0:10:18.160 either agree with everything we say, or you are the enemy. 0:10:18.920 --> 0:10:21.880 I mean, that's the kind of tenor which sounds familiar 0:10:21.880 --> 0:10:26.000 to me generally. Often the most extreme version of that designation, 0:10:26.120 --> 0:10:27.760 and we haven't seen the full details of it because 0:10:27.760 --> 0:10:30.000 they haven't produced it yet, but would mean that any 0:10:30.360 --> 0:10:33.240 company who's working with the d id couldn't use your technology, 0:10:33.240 --> 0:10:36.280 which is a vast I mean, Amazon, which has invested 0:10:36.320 --> 0:10:39.920 billions in Anthropy, has massive contracts with the DoD so 0:10:39.960 --> 0:10:43.000 it's an enormous thing to designate them. It's also doesn't 0:10:43.000 --> 0:10:45.400 really make sense because you know, they were also saying 0:10:45.440 --> 0:10:47.880 we may invoke the Defense Production Act, which has been 0:10:48.000 --> 0:10:51.000 used in wartimes or in actually also during the Cold 0:10:51.080 --> 0:10:56.559 War to force companies to prioritize production and supply because 0:10:56.760 --> 0:11:01.240 what they were making is so important. So it's it's wait, 0:11:01.600 --> 0:11:03.400 I think this may be what, Yeah, this may be 0:11:03.400 --> 0:11:06.480 a legal problem that the d ID has, and I 0:11:06.520 --> 0:11:09.160 think you know Anthropic is going to as a signal 0:11:09.200 --> 0:11:11.760 that's going to take some legal action, So you can't 0:11:11.800 --> 0:11:14.800 it's difficult to argue. On one hand, it's so vital 0:11:14.840 --> 0:11:17.520 we may invoke the Defense Production Act, but then we 0:11:17.559 --> 0:11:19.880 may also designate you a supply chain risk. 0:11:20.400 --> 0:11:24.600 Yeah, like you're too dangerous for us to use, but 0:11:25.200 --> 0:11:28.840 you're so useful that we want to use your product. 0:11:28.559 --> 0:11:31.640 But we want to force you to supply it big one. 0:11:32.280 --> 0:11:35.600 Yeah, like nobody can have it, but we want it. Okay, 0:11:35.880 --> 0:11:49.480 what are we doing here? So basically the next day 0:11:50.480 --> 0:11:54.640 open Ai signs something with the government. How does that 0:11:54.720 --> 0:11:57.120 sequence of events? Like, what kind of sense are you 0:11:57.160 --> 0:11:57.840 able to make of that? 0:11:58.160 --> 0:12:02.720 Yeah, it is very interesting that anthropics chief rival. And 0:12:03.000 --> 0:12:06.040 quick reminder that Anthropic was founded by people who left 0:12:06.400 --> 0:12:10.079 OpenAI because they said that company was not developing AI 0:12:10.559 --> 0:12:15.600 with enough focus on safety. And they're really really bitter rivals. 0:12:15.640 --> 0:12:18.360 And as it was going on, Sam al Mun was 0:12:18.400 --> 0:12:21.960 working through an alternative deal. The deal that they signed 0:12:22.000 --> 0:12:25.760 seems extremely similar to the one that Anthropic we're trying 0:12:25.760 --> 0:12:28.520 to hash out, has similar carve outs, so it's sort 0:12:28.520 --> 0:12:31.360 of doesn't actually carve them out, but it specifies that 0:12:31.360 --> 0:12:36.000 those things that massurbillance is unlawful and that nobody wants 0:12:36.040 --> 0:12:37.240 to build autonomous weapons. 0:12:37.559 --> 0:12:41.880 So are the details or the terms of the contract 0:12:41.880 --> 0:12:45.880 that OpenAI signed with the Department of War, are they 0:12:45.920 --> 0:12:51.199 materially different from what Anthropic did? Because this is from CNBC, 0:12:51.679 --> 0:12:55.040 Altman said, and I want to read this here quote. 0:12:55.120 --> 0:12:57.959 I believe that we will hopefully have the best models 0:12:57.960 --> 0:13:00.440 that were encouraged the government to be willing to work 0:13:00.480 --> 0:13:03.679 with us, even if our safety stack annoys them. But 0:13:03.679 --> 0:13:05.800 there will be at least one other actor, which I 0:13:05.840 --> 0:13:09.520 assume will be x Ai, which effectively will say, will 0:13:09.559 --> 0:13:14.040 do whatever you want. So the vibe here seems to 0:13:14.080 --> 0:13:17.280 be like, okay, well listen, at least worthy adults in 0:13:17.280 --> 0:13:19.880 the room. They're gonna let Grop do whatever Groc wants 0:13:19.880 --> 0:13:22.280 to do, whatever, whatever Trump wants to do. They're just 0:13:22.320 --> 0:13:24.719 gonna let it do it, drop bombs wherever. At least 0:13:24.760 --> 0:13:26.240 we're going to be some adults in the room. But 0:13:26.960 --> 0:13:30.360 is Opening Eye doing something fundamentally different. Did they seed 0:13:30.400 --> 0:13:33.480 more grounded than Anthropic did or is it the same thing? 0:13:35.280 --> 0:13:36.920 You know, Yeah, there's a lot going on there. I 0:13:37.200 --> 0:13:41.120 think that they they did see slightly more ground in 0:13:41.160 --> 0:13:45.200 that they basically agreed to a contract that specified all 0:13:45.280 --> 0:13:49.560 lawful use. They changed the contractors make Oh but you know, 0:13:49.640 --> 0:13:51.800 to be clear, here's what's what is lawful and what 0:13:51.840 --> 0:13:54.880 the deity doesn't want to do. But I think there 0:13:54.880 --> 0:13:56.760 isn't a huge amount of difference. I mean, I think 0:13:56.800 --> 0:14:00.000 what you're seeing with that that internal statement was really, 0:14:01.200 --> 0:14:04.600 you know, Sam open Ai willing to take a bit 0:14:04.640 --> 0:14:09.720 of flat because it's just incredibly important to get a 0:14:09.720 --> 0:14:11.920 win over Anthropic, to be seen to be very on 0:14:12.000 --> 0:14:14.040 board with the US government because you want more and 0:14:14.120 --> 0:14:17.840 more involvement, more and more contracts and they've since you know, 0:14:17.880 --> 0:14:20.000 open a I since got quite a lot of blowback. 0:14:20.640 --> 0:14:22.920 Yeah, well, let's talk about the blowback here. I mean 0:14:23.120 --> 0:14:27.360 this is on tech crunch us app. Uninstalls of chat 0:14:27.400 --> 0:14:31.440 gbt's mobile app went up almost three hundred per so 0:14:31.600 --> 0:14:40.640 people were uninstalling chat gbt and started installing Claude Anthropics Claude. 0:14:40.720 --> 0:14:44.120 And so there's a weird way in which there's this 0:14:45.040 --> 0:14:47.280 and I really hesitate to call this a game, but 0:14:47.280 --> 0:14:49.800 there's there's a weird, really weird way in which there's 0:14:49.840 --> 0:14:53.520 this game being played over who gets to be the 0:14:53.600 --> 0:14:57.160 contractor for the Department of War. But then also we're 0:14:57.240 --> 0:14:59.720 still fighting for consumers here. 0:15:00.720 --> 0:15:03.800 Definitely, And it is interesting. It's the sort of perception 0:15:03.840 --> 0:15:06.120 of who's the good guy and you had, you know, 0:15:06.200 --> 0:15:10.920 Katie Perry was subscribing to claud pro and posting about 0:15:10.920 --> 0:15:14.120 it and this, you know, the sort of interesting cultural moment. 0:15:14.160 --> 0:15:17.040 And I think the thing that was that struck me 0:15:17.160 --> 0:15:20.560 was like, the people really know how Claude is being 0:15:20.640 --> 0:15:23.640 used by the military, Do they know what Anthropic was 0:15:23.680 --> 0:15:26.920 willing to do? I think it's gotten a little oversimplified 0:15:26.960 --> 0:15:29.040 to my mind, and I think that one of the 0:15:29.160 --> 0:15:32.680 risks there is that we ought to be rather necessarily 0:15:32.760 --> 0:15:36.320 uninstalling something or posting about who you think is best, 0:15:37.120 --> 0:15:39.080 really pushing for more of an explanation about how the 0:15:39.160 --> 0:15:42.400 technology is being used and developed and tested, like what 0:15:42.520 --> 0:15:45.240 is it mean that it's used on classified systems? There 0:15:45.240 --> 0:15:48.200 should be more accountability there. And the fact of the 0:15:48.200 --> 0:15:51.160 matter is there is nothing legally to putting the issue 0:15:51.160 --> 0:15:53.440 of the bonus to once nothing prohibiting the development of 0:15:53.480 --> 0:15:57.640 autonomous weapons, and there isn't really a legal framework around 0:15:57.720 --> 0:16:00.520 that sort. If they, you know, civilians were affected by 0:16:00.520 --> 0:16:03.600 fully autonomous weapons, you suddenly have a situation where nobody 0:16:03.680 --> 0:16:06.640 is legally accountable. And so these are things that really 0:16:06.640 --> 0:16:09.560 need to be figured out. And that doesn't mean a 0:16:09.600 --> 0:16:12.720 fully autonomous web that could be your automated part of 0:16:12.760 --> 0:16:15.280 the so called kill chain and so decisions were kind 0:16:15.280 --> 0:16:17.960 of automated away. I think that is really important to 0:16:18.000 --> 0:16:20.800 have more focus on this and to have more discussion 0:16:20.840 --> 0:16:24.120 of it. And you know, one of the reasons, honestly, 0:16:24.440 --> 0:16:27.680 as someone who's looked at the DoD and the work 0:16:27.680 --> 0:16:31.080 going on around autonomy, is that unfortunately it is the 0:16:31.120 --> 0:16:33.040 case that we're just going to have more and more autonomy, 0:16:33.400 --> 0:16:36.160 and most analysts expect that they're going to be swarms 0:16:36.200 --> 0:16:39.440 of drones attacking systems, and the truth is you just 0:16:39.480 --> 0:16:41.600 won't be able to have humans in the loop always. 0:16:41.760 --> 0:16:43.520 So this is going to become a really big issue. 0:16:43.520 --> 0:16:47.080 And it's not necessarily about anthropics models in those situations, 0:16:47.080 --> 0:16:50.120 but the question of who's accountable, how do we know 0:16:50.160 --> 0:16:52.600 that the systems are reliable? That is going to become 0:16:52.680 --> 0:16:54.200 more and more important. So I think it's good that 0:16:54.200 --> 0:16:57.000 the situation is getting some attention. I do think that 0:16:57.040 --> 0:17:00.320 the positive thing is that it is important for people 0:17:00.320 --> 0:17:03.520 who understand this technology really really well. Do you maybe 0:17:03.560 --> 0:17:06.680 have a role in at least influencing how it's used. 0:17:06.840 --> 0:17:08.560 And that doesn't mean, you know, like you get to 0:17:08.600 --> 0:17:10.920 say yes or no to a mission. It just means 0:17:10.920 --> 0:17:15.119 perhaps that you say, have some input on how you 0:17:15.160 --> 0:17:17.640 ought to try and deploy it within the Pentagon I. 0:17:17.600 --> 0:17:20.720 Think, I mean, how does that functionally work? Though maybe 0:17:20.720 --> 0:17:23.240 that's the question here, That is the question, Yeah, can 0:17:23.280 --> 0:17:28.240 you have a contract with the government and tell them no, 0:17:28.400 --> 0:17:31.320 you can't do this? And I'm asking that partially because 0:17:31.720 --> 0:17:34.320 certainly there is a lot of talk that will wait 0:17:34.320 --> 0:17:37.520 a second, The iron Strake. Was Claude used there. 0:17:38.000 --> 0:17:41.119 Yeah. To answer the first question, I think that is 0:17:41.200 --> 0:17:43.919 a really good question. How do companies, how does the 0:17:43.920 --> 0:17:47.719 public understand how things are developed, what the rules are 0:17:47.720 --> 0:17:51.920 around them. And there are specified rules around autonomous weapons, 0:17:51.960 --> 0:17:55.439 around surveillance, but this is also a technology that's different, 0:17:55.840 --> 0:17:58.600 that can misbehave in surprising ways and can be kind 0:17:58.640 --> 0:18:01.080 vulnerable in surprising ways. So I mean that it needs 0:18:01.119 --> 0:18:05.000 a bit more scrutiny. The Washington Posters has reported. I 0:18:05.000 --> 0:18:08.440 think it's that it seems very likely true that Claude 0:18:08.520 --> 0:18:12.080 was absolutely used in the Iran operation as part of 0:18:12.080 --> 0:18:14.960 this system called Maize and which was built by Palanteer, 0:18:15.080 --> 0:18:18.160 which includes a bunch of things, not just a language model. 0:18:18.200 --> 0:18:21.119 It can do you know, analyzed maps and images and 0:18:21.760 --> 0:18:24.679 mission plan and so on. But yeah, as part of that. 0:18:24.760 --> 0:18:26.840 And so now, one of the things that was kind 0:18:26.840 --> 0:18:29.280 of crazy with Trump's initial statement was that he said, 0:18:29.720 --> 0:18:33.560 I'm ordering everybody to stop working with Anthropic because they're 0:18:33.760 --> 0:18:36.000 all these terrible people. But there'll be a six month 0:18:36.040 --> 0:18:38.560 period of what we carry on doing. Right, it can't 0:18:38.600 --> 0:18:41.960 be that bad, and we're going to use We're about 0:18:42.000 --> 0:18:45.040 to use them in this you know, incredible operation in 0:18:45.800 --> 0:18:46.520 the Middle East. 0:18:47.000 --> 0:18:51.479 So yeah, I mean we're laughing. It's terrible. I mean 0:18:51.520 --> 0:18:54.760 it's ridiculous and also terrible. I suppose it can be 0:18:54.760 --> 0:18:59.080 two things at the same time. And because we're doing 0:18:59.119 --> 0:19:02.520 this live, I wanted to take a few questions. And 0:19:02.840 --> 0:19:06.040 one of the questions that I got actually previously I 0:19:06.080 --> 0:19:09.399 posted on Instagram does anybody have any questions? And somebody posted, 0:19:09.480 --> 0:19:12.160 and I think they were kind of joking, but they said, hey, 0:19:12.160 --> 0:19:15.520 can we just make all the country's Ais fight against 0:19:15.520 --> 0:19:17.600 each other? Can we just have them fight instead of 0:19:17.640 --> 0:19:21.080 the real war. But it's kind of funny that they 0:19:21.119 --> 0:19:24.200 say that because there's I don't know if you've seen 0:19:24.200 --> 0:19:29.960 this article in the New Scientist, but basically different AIS, 0:19:30.000 --> 0:19:32.960 and the headline here kind of says it, all AIS 0:19:33.040 --> 0:19:37.359 can't stop recommending nuclear strikes in wargames simulations, and the 0:19:37.400 --> 0:19:42.320 subhead leading AIS from Open Ai, Anthropic and Google opted 0:19:42.359 --> 0:19:46.040 to use nuclear weapons in simulated wargames in ninety five 0:19:46.080 --> 0:19:47.440 percent of cases. 0:19:48.520 --> 0:19:52.000 Right, Yeah, So I read that paper. It's important to 0:19:52.080 --> 0:19:55.840 know that this person was just trying to understand what 0:19:55.880 --> 0:19:57.720 the kind of behavior of models would be. Well, I'm 0:19:57.720 --> 0:19:59.959 not advocating to do that, and I don't think anybody. 0:20:00.240 --> 0:20:02.520 I think actually this is one of the like not 0:20:02.680 --> 0:20:06.520 letting AI have a role in nuclear weapon decision making 0:20:06.640 --> 0:20:08.960 is one thing that the US and China have actually 0:20:09.000 --> 0:20:11.480 agreed on, which is extremely unusual these days. 0:20:11.520 --> 0:20:11.680 Right. 0:20:11.720 --> 0:20:14.840 So, but it's unsurprising to me that if you have 0:20:14.880 --> 0:20:17.879 a model trade on the entire Internet, it's going to 0:20:18.400 --> 0:20:23.760 tend toward escalation and not be very very expert in 0:20:24.440 --> 0:20:27.440 how to avoid you know, it's not going to be 0:20:27.840 --> 0:20:30.040 someone who spent their life worrying about that sort of thing. 0:20:30.080 --> 0:20:33.160 And on the question of whether ais could fight each other, 0:20:33.359 --> 0:20:37.679 the truth is, like you are already seeing in certain conflicts, 0:20:38.080 --> 0:20:41.560 you know, autonomous systems, semi autonomous systems are remotely operated 0:20:42.160 --> 0:20:43.960 fighting other semi autonomous systems. 0:20:44.000 --> 0:20:44.119 Right. 0:20:44.119 --> 0:20:47.040 And some people may argue that that would remove people 0:20:47.080 --> 0:20:50.800 from harm's way. That is not the reality of a conflict. 0:20:50.840 --> 0:20:52.640 There's going to be plenty of people in harm's way. 0:20:53.240 --> 0:20:55.639 And the question is, you know, how do you do 0:20:55.720 --> 0:20:59.200 that in ways that are reliable and don't make mistakes. 0:20:59.280 --> 0:21:02.040 And there has been talking in recent years of these 0:21:02.080 --> 0:21:04.679 sort of things like existential risks, but long before that, 0:21:04.720 --> 0:21:08.200 there are lots of systemic dangers of using this technology 0:21:08.200 --> 0:21:12.040 in any critical system. So you know, just as you 0:21:12.080 --> 0:21:14.719 have hallucinations if you're using a chat, or just as 0:21:14.760 --> 0:21:17.840 it gets simple reasoning wrong in a very convincing way, 0:21:18.240 --> 0:21:19.880 that is what we should be worried about. I think 0:21:19.920 --> 0:21:23.439 in how this and other models language models are used 0:21:23.480 --> 0:21:24.800 by the military, by the Pentagon. 0:21:25.200 --> 0:21:25.520 You know what. 0:21:25.640 --> 0:21:29.320 I once talked to the excuse and naval officer in 0:21:29.320 --> 0:21:32.120 the US Navy whose job it was to, you know, 0:21:32.440 --> 0:21:34.920 be in command of these nuclear submarines, and he said 0:21:35.200 --> 0:21:37.120 one of the first things he did when he got 0:21:37.119 --> 0:21:38.960 the job was get permission to go and talk to 0:21:38.960 --> 0:21:42.400 his counterpart in China, go meet him and spend time there. 0:21:42.560 --> 0:21:45.760 The reason being because if something went wrong, if one 0:21:45.760 --> 0:21:48.320 of the systems went wrong with some a person made 0:21:48.359 --> 0:21:52.159 a mistake, you know, or say there's a collision between 0:21:52.200 --> 0:21:54.719