WEBVTT - Debating the Morality of Dario Amodei 0:00:18.600 --> 0:00:19.400 Daniel. 0:00:20.360 --> 0:00:21.040 Hey. How are you? 0:00:21.560 --> 0:00:22.720 Good. How are you doing? 0:00:23.400 --> 0:00:24.360 Not too bad. 0:00:24.960 --> 0:00:26.880 Great. Thanks for doing this, by the way. 0:00:27.360 --> 0:00:27.840 Totally. 0:00:27.880 --> 0:00:29.640 Yeah. Yeah. It was. It was. You were the one 0:00:29.640 --> 0:00:31.240 who said we should do it on camera, so we're 0:00:31.240 --> 0:00:33.960 doing it on camera. Uh, I think there's a lot 0:00:33.960 --> 0:00:36.880 of people excited about it, so, uh. I'm happy. I 0:00:36.880 --> 0:00:39.600 think we have, uh, so the chat is apparently a 0:00:39.600 --> 0:00:42.440 little bit messed up in terms of like time delay 0:00:42.440 --> 0:00:46.279 and stuff. Uh, but we'll just kind of ignore the 0:00:46.280 --> 0:00:49.519 chat for a bit and just assume there's an audience 0:00:49.520 --> 0:00:53.479 who cares. And yeah, just do this between us. Uh, sound. Okay. 0:00:53.960 --> 0:00:54.840 Sounds perfect. 0:00:55.360 --> 0:00:57.560 Great. Uh, nice to meet you. By the way, you 0:00:57.560 --> 0:01:00.440 and I have never actually talked on a call before. Like, 0:01:00.480 --> 0:01:03.600 we've only interacted on the internet, so. Good to meet you. 0:01:03.840 --> 0:01:04.800 Yeah, good to meet you. 0:01:05.319 --> 0:01:08.600 Cool. Okay, so what I thought we'd do is just. 0:01:08.600 --> 0:01:11.039 I kind of want to start off by just giving like, 0:01:11.080 --> 0:01:13.640 a little bit of a summary for people of what, 0:01:13.640 --> 0:01:16.600 what the interaction was. Uh, and then I'll just let 0:01:16.600 --> 0:01:20.310 you kind of give your take on what your opinion was, 0:01:20.350 --> 0:01:23.350 and then we'll just go and have a discussion from there. Cool. 0:01:24.229 --> 0:01:27.509 Um, or maybe you can give the background of like 0:01:27.550 --> 0:01:30.269 what your opinion was that I responded to. 0:01:30.590 --> 0:01:33.190 Yeah, yeah. So I'm going to start from where I, yeah. 0:01:33.430 --> 0:01:36.149 So okay. So for anyone who is like just new 0:01:36.150 --> 0:01:39.030 to this or okay, so what has happened is I 0:01:39.430 --> 0:01:43.750 made a video, uh, of 18 minutes of me just 0:01:43.750 --> 0:01:49.150 trashing Dario Amodei. Okay. Who was the CEO of anthropic? Uh, 0:01:49.150 --> 0:01:53.389 just saying. Basically I think he's a bad person. Okay. Or, um, 0:01:53.430 --> 0:01:55.230 bad person is maybe not the word I use, but 0:01:55.270 --> 0:01:57.590 it was, it was along those lines. And I basically said, 0:01:57.870 --> 0:02:00.990 the problem is, is that what? And I walked through 0:02:01.030 --> 0:02:06.070 in this video, uh, Dario's, uh, position on AI safety. 0:02:06.070 --> 0:02:09.950 And for anyone who doesn't, um, understand some of the context, 0:02:09.990 --> 0:02:12.470 this can only give a little more information. I am 0:02:12.510 --> 0:02:15.150 a little bit fortunate in that I happen to be 0:02:15.150 --> 0:02:18.940 when I was doing my master's, I was at Oxford, 0:02:18.940 --> 0:02:21.660 which is where Nick Bostrom was. And Nick Bostrom is 0:02:21.660 --> 0:02:23.740 a person who was like very early in or like 0:02:23.780 --> 0:02:26.740 a big name in the AI safety debates. So I 0:02:26.739 --> 0:02:29.060 went and I heard him speak at some event. I 0:02:29.060 --> 0:02:32.020 forget what it was exactly. Uh, and so, and then 0:02:32.019 --> 0:02:33.980 that made me read up on all of the AI 0:02:34.019 --> 0:02:37.019 safety discussions. And this was like 2016. So I'm like 0:02:37.060 --> 0:02:40.740 a little bit like more well versed in AI safety as, um, 0:02:40.780 --> 0:02:43.540 than your average person would be just because I just 0:02:43.540 --> 0:02:45.660 happen to be at the place where they were talking 0:02:45.660 --> 0:02:47.460 about it, like where it was kind of was a 0:02:47.460 --> 0:02:48.980 big idea at the time. They have like a whole 0:02:48.980 --> 0:02:51.780 thing called like the future of humanity. Um, and so 0:02:51.780 --> 0:02:54.540 I make this video where I basically walk through what 0:02:54.579 --> 0:02:58.139 Dario believes and then walk through what Dario wants. And 0:02:58.139 --> 0:03:03.139 then I say basically, uh, no, uh, fuck that. We're 0:03:03.139 --> 0:03:06.300 not doing that. Uh, everything Dario wants is bad. And 0:03:06.299 --> 0:03:08.260 I think that we should all reject it, and we 0:03:08.260 --> 0:03:11.419 should all just completely reject Dario completely because he's trying 0:03:11.419 --> 0:03:13.880 to lead society down a path that is bad. And 0:03:13.880 --> 0:03:16.639 specifically my argument comes down to this, which is that 0:03:16.919 --> 0:03:21.000 Dario believes that AI has a 25% chance of going 0:03:21.040 --> 0:03:24.560 extremely poorly for society. That's his position on it. Okay. 0:03:24.840 --> 0:03:28.520 And in that world, we want to he wants to 0:03:28.560 --> 0:03:31.999 make sure it goes well. And his basically, his proposal 0:03:32.000 --> 0:03:36.400 is like, help me regulate all of my competitors. And, um, 0:03:36.840 --> 0:03:39.280 let me continue to run AI so that he gets 0:03:39.280 --> 0:03:42.400 put in a position where he runs the safety of 0:03:42.880 --> 0:03:45.760 AI and he can keep you all safe. Uh, make 0:03:45.800 --> 0:03:47.880 sure that you're all safe by not giving you access 0:03:47.880 --> 0:03:50.840 to anything dangerous. Um, only he gets access to that. 0:03:50.840 --> 0:03:52.360 And so I basically called that out and made fun 0:03:52.360 --> 0:03:55.840 of it for 17 straight minutes. Uh, and then Daniel, 0:03:55.880 --> 0:03:58.440 you wrote a post and then a number. We went 0:03:58.440 --> 0:04:00.360 back and forth a bit. Do you want to explain 0:04:00.560 --> 0:04:02.600 what you said, what your position is on this? 0:04:03.080 --> 0:04:05.920 Yeah, I don't really remember what I said, but I 0:04:06.440 --> 0:04:09.000 but what, what you said just now reminds me of 0:04:09.000 --> 0:04:13.510 like what my position is. So, um, so I guess 0:04:13.550 --> 0:04:18.630 at a high level, my position on this, um, is 0:04:18.630 --> 0:04:23.430 that what you're saying? Sounds perfectly logical. And I, I 0:04:23.470 --> 0:04:27.790 actually think we probably agree 80 to 95% on like 0:04:27.830 --> 0:04:32.270 pretty much everything here. Um, and I kind of just 0:04:32.270 --> 0:04:34.350 have a way of thinking about this, which is a 0:04:34.350 --> 0:04:39.910 little bit different. Um, so I essentially think that if 0:04:40.510 --> 0:04:44.310 we take certain things into account about what we want 0:04:44.310 --> 0:04:46.150 to happen in the world and what we don't want 0:04:46.150 --> 0:04:50.190 to happen in the world, the question then becomes, okay, 0:04:50.230 --> 0:04:52.990 what do we do about it? Okay. So, so let's 0:04:52.990 --> 0:04:57.110 just step through these because I think we probably actually agree. Um, 0:04:57.190 --> 0:05:00.590 do we agree that we don't want like a government, um, 0:05:01.590 --> 0:05:06.150 authoritarian government who has like AGI, ASI or whatever that 0:05:06.150 --> 0:05:08.510 just like subjugates the population. 0:05:09.310 --> 0:05:11.780 Yeah. Subjugation of the population is bad. So I think. 0:05:11.820 --> 0:05:13.980 Yeah, that's bad. Yeah. Um, so we don't want that 0:05:13.980 --> 0:05:16.740 from the government, but we also don't want in your 0:05:16.740 --> 0:05:19.540 mind like a Dario type or like a, you know, 0:05:19.580 --> 0:05:23.540 super rich, ultra rich, like oligarchy, who's on the corporate 0:05:23.540 --> 0:05:27.780 side doing the same thing. Right. Both, both like government 0:05:27.780 --> 0:05:30.780 or corporate. It doesn't matter. Like either way, if it 0:05:30.780 --> 0:05:35.700 ends up with us subjugated, that's bad. So 1,000% agree 0:05:35.700 --> 0:05:39.219 with you there. So I agree with what you're kind 0:05:39.220 --> 0:05:42.180 of like attacking. I think you're attacking the wrong person 0:05:42.180 --> 0:05:45.300 in the wrong entity. Um all right. So let's keep going. 0:05:45.660 --> 0:05:49.260 Do you agree that it would be bad if the 0:05:49.260 --> 0:05:58.380 public suddenly received like, um. Let's say mythos or further level, um, 0:05:58.380 --> 0:06:06.060 ability to, um, create bioweapons, um, planned terrorist attacks, uh, 0:06:06.060 --> 0:06:10.130 create end to end like vulnerability research all the way 0:06:10.290 --> 0:06:11.650 to exploitation. 0:06:13.010 --> 0:06:14.930 So I think you have to break that. So if 0:06:14.930 --> 0:06:17.410 we're talking the cyber capabilities. No, I don't think that'd 0:06:17.410 --> 0:06:19.290 be bad. I think that'd be good. I think like 0:06:19.530 --> 0:06:22.970 from a cyber perspective, if the whole public got access 0:06:22.970 --> 0:06:26.330 to like mythos plus plus, let's call it like mythos seven, 0:06:26.330 --> 0:06:30.370 and it's like the greatest hacker of all time. Like 0:06:30.370 --> 0:06:32.970 I still think that's net positive for the world because 0:06:33.010 --> 0:06:36.210 like cybersecurity is not magic. So like, there are still 0:06:36.210 --> 0:06:39.530 ways to protect, like, it's not like, you know, a 0:06:39.570 --> 0:06:43.529 mythos seven can not just hack anyone, even if like, 0:06:43.570 --> 0:06:47.170 there has to be some issue to hack. Right? Um, 0:06:47.210 --> 0:06:48.410 so I'm not really, I don't. 0:06:48.450 --> 0:06:51.330 Okay, well, maybe this is our, maybe this is our 0:06:51.330 --> 0:06:57.730 first level of disagreement. Um, uh, let's see. Okay, I 0:06:57.770 --> 0:06:59.930 guess we can go tactically down into here. So it's 0:06:59.930 --> 0:07:02.330 not my main point, but but maybe this is part 0:07:02.330 --> 0:07:05.570 of the main point. So you're saying that it would 0:07:05.570 --> 0:07:08.990 not be bad for Society. Because keep in mind all 0:07:08.990 --> 0:07:12.310 attackers who have been attacking us this whole time. They 0:07:12.310 --> 0:07:14.710 would also have this capability, right? 0:07:14.750 --> 0:07:15.150 Sure. 0:07:15.950 --> 0:07:19.230 Because the whole public does. Everyone has it. Um, you're 0:07:19.230 --> 0:07:20.630 saying this would not be a problem? 0:07:21.910 --> 0:07:23.990 No, no, I mean, it would not. A problem is 0:07:23.990 --> 0:07:25.630 a bit of an understatement. Like, obviously it would be 0:07:25.630 --> 0:07:28.550 inconvenient and it would be like a little painful. Um, 0:07:28.870 --> 0:07:32.750 but like, is it a thing that I would go 0:07:32.750 --> 0:07:35.910 to bed worrying about? Like, no, not really and I don't. Okay. 0:07:35.950 --> 0:07:38.430 I think that's what's true of a lot of people 0:07:38.430 --> 0:07:40.830 in cybersecurity, as we're pointing out that like, I kind 0:07:40.870 --> 0:07:43.630 of asked that question of like, how bad would it 0:07:43.630 --> 0:07:46.550 be if mythos level capabilities were available to everyone? And 0:07:46.550 --> 0:07:49.550 the answer was, like most people in cybersecurity that I, 0:07:49.870 --> 0:07:52.470 of course, biased by the people who follow me, but 0:07:52.470 --> 0:07:56.110 like it's a pretty big pool. Uh, we're like, yeah, 0:07:56.230 --> 0:07:58.270 it'd be a little painful in the short run. I 0:07:58.270 --> 0:08:01.870 think that there's no reason to take actions. And I 0:08:01.870 --> 0:08:04.030 think this is sort of comes back to the original 0:08:04.030 --> 0:08:06.260 position I have, which is there's no reason to take 0:08:06.300 --> 0:08:10.820 actions based on those types of fears. If the actions 0:08:10.820 --> 0:08:14.580 themselves require us to do things that are like wholly illiberal. Right? 0:08:15.540 --> 0:08:17.540 In a generic sense. Liberal, right. 0:08:17.580 --> 0:08:21.380 Okay. Okay. So a couple of things there. Um, so I, 0:08:21.740 --> 0:08:24.140 I think we're very far apart on how bad this 0:08:24.140 --> 0:08:26.580 would be. Um, so let me ask you this. Do 0:08:26.580 --> 0:08:32.260 you not believe that most of our software and, and 0:08:32.260 --> 0:08:35.579 mythos is essentially like showing us this? And it's not 0:08:35.580 --> 0:08:38.579 just mythos, like a good AI stack on a previous 0:08:38.700 --> 0:08:40.980 level model can actually do the same thing. I've been 0:08:40.980 --> 0:08:46.620 doing this forever. Um, that it can reveal the fact 0:08:46.620 --> 0:08:50.180 that our entire infrastructure, basically everything that our society, by 0:08:50.180 --> 0:08:52.100 the way, your video's frozen for me. I don't know 0:08:52.100 --> 0:08:55.100 if it's frozen for other people, it's probably probably just 0:08:55.100 --> 0:08:59.780 my computer. Um, but just in case it's not, um, basic, 0:08:59.820 --> 0:09:03.700 basically my argument is, um, and, and just like you 0:09:03.770 --> 0:09:06.170 were saying, people agree with you about this. There's lots 0:09:06.170 --> 0:09:11.010 of people agree with me about this. Um, our infrastructure 0:09:11.010 --> 0:09:16.130 is basically a giant soup sandwich. It is a complete mess. Um, 0:09:16.170 --> 0:09:21.010 Linux is massively still vulnerable. Windows is massively still vulnerable. 0:09:21.010 --> 0:09:24.450 All the critical infrastructure runs on software that is full 0:09:24.450 --> 0:09:27.930 of holes, which we simply haven't taken the time to 0:09:27.970 --> 0:09:29.890 look for. And if we did take the time to 0:09:29.930 --> 0:09:32.650 look for it, we would find serious issues. So if 0:09:32.650 --> 0:09:35.410 the whole world, including all the bad guys, suddenly get 0:09:35.410 --> 0:09:39.290 the ability to uncover those, that is not an inconvenience. 0:09:39.290 --> 0:09:43.130 That is seriously bad. Especially if you do it all 0:09:43.130 --> 0:09:45.770 at once and you just release the model if this 0:09:45.770 --> 0:09:47.370 suddenly becomes good. 0:09:47.410 --> 0:09:49.170 But your your take on it was that this is 0:09:49.170 --> 0:09:52.130 not just mythos thing. It's also true of things like, 0:09:52.370 --> 0:09:55.290 you know, if you're running a lot of stacks with like, okay, 0:09:55.370 --> 0:09:58.250 other models that you can use to do, but those, 0:09:58.290 --> 0:10:01.410 those do exist. And it's not like nothing bad has 0:10:01.410 --> 0:10:01.930 happened yet. 0:10:02.040 --> 0:10:07.400 No, no. But they require extraordinary. Sorry. They require extraordinary 0:10:07.400 --> 0:10:11.600 skill to create and manage. And only a few people 0:10:11.600 --> 0:10:15.160 have them, right. Me and my buddies, we have them lots. 0:10:15.200 --> 0:10:18.240 Lots of other hacker buddies. Like we have these stacks. 0:10:18.520 --> 0:10:21.319 And so it is a you know, the top of 0:10:21.320 --> 0:10:23.800 the top has these stacks and has been able to 0:10:23.800 --> 0:10:27.199 do this type of stuff. And the models like improve them. 0:10:27.200 --> 0:10:30.960 But the more you go mythos mythos plus plus, it's 0:10:30.960 --> 0:10:34.440 like a single prompt. I want you to achieve this. 0:10:35.040 --> 0:10:38.440 Believe me. Yeah, yeah. Well, I don't think we, I 0:10:38.480 --> 0:10:40.720 don't think we disagree on the premise here. I just 0:10:40.720 --> 0:10:43.200 think that basically what happens as a result of it 0:10:43.200 --> 0:10:46.520 is if you were to do something like, let's say, 0:10:46.720 --> 0:10:49.680 hold these models off for release to try to like 0:10:49.720 --> 0:10:54.000 solve some problem in advance, like try to gate access, whatever, 0:10:54.040 --> 0:10:56.880 like the solution to that might be, it's just going 0:10:56.920 --> 0:10:59.360 to be so much worse than just saying like, let's 0:10:59.360 --> 0:11:02.140 just rip the band aid off. Some problems will happen. 0:11:02.140 --> 0:11:05.460 We'll fix the problems like no, nothing is going to 0:11:05.500 --> 0:11:08.460 like the world will go on in very normal ways. 0:11:08.500 --> 0:11:11.100 Like I think that if you release those capabilities into 0:11:11.100 --> 0:11:13.580 the world, the world will go on and it will 0:11:13.580 --> 0:11:15.420 just be like, yeah, some bad stuff happened along the 0:11:15.420 --> 0:11:17.340 way and we fixed it and we're good. Okay. And 0:11:17.340 --> 0:11:21.339 so like, I don't think that there's any major reason 0:11:21.340 --> 0:11:24.780 to be like in a panic about that besides, like, 0:11:25.220 --> 0:11:27.860 you know, compared to the types of especially the types 0:11:27.860 --> 0:11:29.860 of proposals we're talking about, which would be like, okay, well, 0:11:29.860 --> 0:11:32.179 now we need to limit what open rate models can 0:11:32.180 --> 0:11:34.380 be put into the public, right? We need like, these 0:11:34.380 --> 0:11:39.700 are extremely draconian measures to prevent something I think is 0:11:39.700 --> 0:11:40.780 a shrug, you know? 0:11:40.820 --> 0:11:45.580 Okay, well, a shrug, okay, a shrug. Okay. All of 0:11:45.580 --> 0:11:49.660 our financial systems are based on this software. All of 0:11:49.660 --> 0:11:53.700 our economic systems are based on those financial systems. Um, 0:11:53.700 --> 0:11:58.500 our critical infrastructure for running, uh, water, energy, everything, everything 0:11:58.500 --> 0:12:02.170 is based on running that same exact shitty software, which 0:12:02.210 --> 0:12:04.689 it sounds like we agree. We know it's shitty and 0:12:04.690 --> 0:12:08.570 we are now releasing into open source models or or 0:12:08.770 --> 0:12:12.730 openly releasing mythos from closed source models, whatever. We're just 0:12:12.730 --> 0:12:15.410 giving it to the entire world. And you believe that 0:12:15.410 --> 0:12:16.170 is a shrug. 0:12:16.570 --> 0:12:19.610 I believe it is like, yeah, effectively, I think that 0:12:19.610 --> 0:12:20.690 basically we will have. 0:12:20.730 --> 0:12:23.890 Well, I think we found I think we found the disagreement. Uh, 0:12:23.890 --> 0:12:26.050 I think that is just wrong. 0:12:26.770 --> 0:12:29.090 Okay. But like, but that I think gets actually to 0:12:29.130 --> 0:12:32.250 like the deeper point, which is like, okay, so imagine 0:12:32.250 --> 0:12:35.570 that you're sitting there going like, okay, I believe that 0:12:35.570 --> 0:12:38.250 like all of these terrible, that mythos is this amazing 0:12:38.250 --> 0:12:39.930 model and that if we release it to the public, 0:12:39.930 --> 0:12:44.010 like lots of terrible, terrible things happen, right? The, the 0:12:44.170 --> 0:12:47.330 follow on steps and actions from that are like, okay, 0:12:47.370 --> 0:12:49.010 so what do you do? Like what is, what's your 0:12:49.010 --> 0:12:50.650 proposed plan to that? 0:12:51.810 --> 0:12:55.170 Totally. 100% agree. So let me just say this. I 0:12:55.210 --> 0:12:56.810 want to I want to make this very sort of 0:12:56.960 --> 0:13:02.160 agreement based. I agree with you that if this were 0:13:02.160 --> 0:13:06.120 a minor situation, okay, if this were a minor situation 0:13:06.120 --> 0:13:08.680 and it wasn't that big of a deal, and then 0:13:08.680 --> 0:13:12.200 you have someone like anthropic or someone like Sam or 0:13:12.240 --> 0:13:14.880 Dario or whoever, and they're like, mm, I want to 0:13:14.880 --> 0:13:18.000 control everything. And like, we can't let this get released. 0:13:18.280 --> 0:13:21.640 And they're clearly doing this for many reasons. Then I 0:13:21.640 --> 0:13:23.800 see that the same way it seems like you're seeing it, 0:13:23.800 --> 0:13:26.360 which is these guys are just trying to control things 0:13:26.400 --> 0:13:28.920 like this is nasty. And I guess what I would 0:13:28.920 --> 0:13:33.640 make a 17 minute video bashing them just like you did. 0:13:34.040 --> 0:13:37.560 The difference is I don't believe that is what's happening. 0:13:37.560 --> 0:13:42.000 I believe what's happening is they believe, just like I do, 0:13:42.280 --> 0:13:44.400 and tons of people in the military do, and tons 0:13:44.400 --> 0:13:48.360 of people who, in my opinion, who are close to 0:13:48.400 --> 0:13:51.520 this are like, no, that would be bad. That would 0:13:51.520 --> 0:13:52.680 be seriously bad. 0:13:52.880 --> 0:13:55.360 Well, to clarify, I believe Dario believes that too. So 0:13:55.510 --> 0:13:57.630 of course, one of the things that makes me diverge 0:13:57.630 --> 0:14:00.550 from some of the people who just believe it's regulatory 0:14:00.590 --> 0:14:04.070 capture play is that I fully believe Dario believes like, 0:14:04.070 --> 0:14:06.870 so I don't I don't for any moment suggest that 0:14:06.870 --> 0:14:09.230 Dario is doing it all just to exploit something for 0:14:09.230 --> 0:14:12.469 his own personal gain. I think Dario is is deeply 0:14:12.470 --> 0:14:15.630 versed in the world of like, you know, less wrong 0:14:15.630 --> 0:14:18.030 ideology of, you know, less wrong being a blog for 0:14:18.030 --> 0:14:19.830 anyone who doesn't know. It's not a. 0:14:19.870 --> 0:14:20.550 Rationalist. 0:14:20.550 --> 0:14:23.470 Movement. Yeah. You know, e world. Like I think he's 0:14:23.470 --> 0:14:25.350 so deep into that he believes it fully. So I 0:14:25.350 --> 0:14:28.230 don't accuse him of not believing, uh, what I think 0:14:28.230 --> 0:14:33.790 is that basically, um, the belief that he has about 0:14:33.790 --> 0:14:36.910 the risks, right? And then the measures he wants to 0:14:36.910 --> 0:14:40.750 take in order to address those risks are like completely 0:14:40.750 --> 0:14:42.550 out of weight from each other. And for one of 0:14:42.550 --> 0:14:44.510 the reasons we found we disagree is that, of course, 0:14:44.510 --> 0:14:47.510 I don't think that releasing mythos publicly would have been 0:14:47.510 --> 0:14:49.910 the end of the world. But at the second time. 0:14:49.910 --> 0:14:52.590 On the other hand, I also think the measures he's 0:14:52.590 --> 0:14:58.650 proposing can lead to some extraordinarily terrible things, right? You know, 0:14:58.970 --> 0:15:02.050 the measures he's proposing is basically to say, I want 0:15:02.050 --> 0:15:04.730 to make sure that China does not have capability to 0:15:04.770 --> 0:15:10.090 develop models. Okay. Good models at least, I guess. Do you? 0:15:10.130 --> 0:15:12.170 I guess the question is, do you agree with his 0:15:12.170 --> 0:15:15.730 proposal there? Like, should we be blocking distillation and should 0:15:15.730 --> 0:15:18.410 we be blocking open weight models? Like what do you 0:15:18.450 --> 0:15:19.690 what do you think there? 0:15:20.170 --> 0:15:23.770 Right. So this is completely consistent with the belief that 0:15:23.770 --> 0:15:26.050 I have, which I believe is the same one that 0:15:26.050 --> 0:15:30.970 he has that mass release at extremely high pace of 0:15:30.970 --> 0:15:33.650 all this stuff is very bad. So we're trying to 0:15:33.690 --> 0:15:37.850 like control the flow and the speed to like get 0:15:37.850 --> 0:15:42.770 society ready. Um, so I don't believe we should give 0:15:42.770 --> 0:15:45.250 this to the CCP immediately. I don't believe we should 0:15:45.250 --> 0:15:49.050 give all these capabilities to the public immediately. I think 0:15:49.050 --> 0:15:53.920 it needs to be done slowly, um, and carefully and thoughtfully. Um. 0:15:54.040 --> 0:15:57.600 I don't believe that he is. People will characterize him 0:15:58.000 --> 0:16:00.800 as being like, oh, he hates open source. He hates 0:16:00.840 --> 0:16:03.280 all this stuff. I think what he's saying, and I 0:16:03.280 --> 0:16:05.840 think is clear from all his interviews, what he's saying 0:16:05.840 --> 0:16:08.960 is like, we got to be careful. Okay. So so 0:16:09.320 --> 0:16:12.200 let me ask you this. What would a person who 0:16:12.200 --> 0:16:14.920 agreed with what I just said, what would they do 0:16:14.960 --> 0:16:17.160 relative to a person who is saying, I want to 0:16:17.160 --> 0:16:20.000 control everything and I don't want to give to anyone? 0:16:20.240 --> 0:16:22.600 No, they would do, of course, exactly what Dario's doing 0:16:22.600 --> 0:16:24.960 because I don't believe that it's about I don't believe 0:16:24.960 --> 0:16:27.120 that Dario's doing it because he wants to control it 0:16:27.120 --> 0:16:29.760 for his own good. I believe Dario is doing it, 0:16:29.760 --> 0:16:31.760 and I know a lot of people in cybersecurity disagree 0:16:31.760 --> 0:16:33.880 with me, but I believe that Dario is doing it 0:16:33.880 --> 0:16:35.960 because he wants to control it. For your good and 0:16:35.960 --> 0:16:39.520 my good and all of our goods. Right. Um, Dario 0:16:39.520 --> 0:16:41.440 believes this is for the betterment of everyone for the 0:16:41.440 --> 0:16:44.600 reasons you're stating, like. Right. The problem I have with 0:16:44.600 --> 0:16:48.680 it is that the what he's basically proposing is that 0:16:48.680 --> 0:16:51.790 we trust him and that we take a hit on 0:16:51.790 --> 0:16:53.230 our side. So, I mean, if you think about it 0:16:53.230 --> 0:16:55.830 this way, if we had done what anthropic had asked 0:16:55.830 --> 0:16:58.510 for us and has been asking for for years and 0:16:58.510 --> 0:17:00.990 years and years and all of their policy proposals and stuff, 0:17:01.030 --> 0:17:04.230 especially for the last year, uh, we would have to 0:17:04.270 --> 0:17:08.070 pay opus level pricing for an opus level model. Okay. 0:17:08.109 --> 0:17:10.109 Whereas as of a couple of days ago, now I 0:17:10.149 --> 0:17:12.590 can pay one fourth of the opus pricing to get 0:17:12.590 --> 0:17:16.109 an opus. Basically an opus clone like GLM 5.2 is 0:17:16.109 --> 0:17:19.190 an opus clone, okay. And it costs one fifth of 0:17:19.190 --> 0:17:23.470 opus 4.8. And that is like a huge net benefit 0:17:23.469 --> 0:17:27.230 to society and to every American, like basically to every 0:17:27.230 --> 0:17:31.630 American company, to every, you know, Western company, to every company. Uh, 0:17:31.669 --> 0:17:34.670 that's huge. That's a hugely beneficial thing that we would 0:17:34.709 --> 0:17:38.109 have not allowed had we followed anthropic policy proposal. 0:17:39.669 --> 0:17:44.550 Uh, is he saying we should limit things like GLM 5.2? 0:17:45.230 --> 0:17:47.350 He is very clear that we need to limit distillation. 0:17:47.350 --> 0:17:49.299 He wants the US government to make an attempt to 0:17:49.340 --> 0:17:52.100 limit distillation more. Okay, so he's like, that's one of 0:17:52.100 --> 0:17:54.019 their policy proposals from like six months ago. 0:17:54.619 --> 0:17:58.939 Well, isn't distillation just isn't distillation? Isn't he trying to 0:17:58.980 --> 0:18:04.659 limit like the CCP's ability to distill from our top models? 0:18:05.060 --> 0:18:07.980 Yes. And that's what GLM did, obviously. GLM is like 0:18:07.980 --> 0:18:10.179 clearly an opus clone. Like, I think it would be 0:18:10.379 --> 0:18:15.019 very hard to believe that they trained GLM without hitting, 0:18:15.540 --> 0:18:20.019 without basically training it on Opus. Like they, they behave identically. 0:18:20.300 --> 0:18:22.259 Yeah. I feel like this is a little bit of 0:18:22.260 --> 0:18:25.419 a diversion, but I'm happy to go down. So basically 0:18:25.939 --> 0:18:30.780 I think it's, it's blatantly okay. Do you agree that 0:18:30.899 --> 0:18:36.780 the CCP's China's overall strategy of just blatantly, blatantly just 0:18:36.780 --> 0:18:40.979 stealing from everyone on the planet to better their country 0:18:40.980 --> 0:18:42.340 is like not a good thing? 0:18:43.459 --> 0:18:46.179 Well, well, if they're releasing it to the public, like 0:18:46.219 --> 0:18:48.730 as an open weight model the way that they have been. 0:18:48.929 --> 0:18:50.369 As long as that is there. 0:18:50.609 --> 0:18:53.969 For the for the purpose of damaging the United States, 0:18:55.330 --> 0:18:57.570 that is, the strategic goal is to. 0:18:58.010 --> 0:19:00.689 Yeah. But how how did they damage the United States 0:19:00.689 --> 0:19:03.369 for releasing GLM 5.2? Like that's just it just doesn't 0:19:03.409 --> 0:19:03.730 I don't. 0:19:03.730 --> 0:19:09.449 See no, no, the overall Chinese AI strategy. Um, and 0:19:09.490 --> 0:19:13.129 to just to get my bias here. So I'm like ex-military, uh, 0:19:13.129 --> 0:19:15.770 I wasn't like super elite or anything like that. But 0:19:15.770 --> 0:19:18.929 my bias is that like, I see, I see things 0:19:18.929 --> 0:19:24.369 in terms of like, what, what happens if the adversaries win? 0:19:24.689 --> 0:19:27.330 What happens if we win? What does it mean to 0:19:27.530 --> 0:19:31.050 want somebody to win? Why does it matter? And ultimately, like, 0:19:31.090 --> 0:19:33.369 I'm a Star Trek liberal. So like, I think we 0:19:33.369 --> 0:19:36.210 should all just get along anyway. But in the meantime, 0:19:36.409 --> 0:19:39.369 the CCP is stealing everything, trying to win because they 0:19:39.369 --> 0:19:42.889 just want to win everything. So the strategy for China 0:19:43.290 --> 0:19:49.790 is to completely underpin US dominance via AI, by harvesting 0:19:49.790 --> 0:19:53.149 everything through distillation and making it open source and getting 0:19:53.149 --> 0:19:54.989 it into open source as fast as possible. 0:19:55.030 --> 0:19:57.429 But that's but you think about it from my perspective 0:19:57.469 --> 0:19:59.389 as just like an American citizen, which by the way, 0:19:59.389 --> 0:20:00.750 I know a lot of people on the internet think 0:20:00.750 --> 0:20:02.709 I'm not. But like American citizen here, I. 0:20:02.709 --> 0:20:04.509 Was trying to figure that out. I'm like, you do 0:20:04.510 --> 0:20:07.350 an amazing American accent. Yeah. For an Oxford guy. 0:20:07.750 --> 0:20:11.389 Yeah, I'm American. Just happened to move around too much. Um, 0:20:11.709 --> 0:20:14.749 you know, uh, the thing is, as like, an American 0:20:14.750 --> 0:20:17.749 running like a company and, like, living my life and 0:20:17.750 --> 0:20:22.990 all of that, it feels a very much like China's 0:20:22.990 --> 0:20:27.629 helping me, okay? They're giving me great models at low prices. Right? Um, 0:20:28.070 --> 0:20:31.149 open weight stopping these. Okay. And then the question and 0:20:31.149 --> 0:20:33.229 then the question becomes, if you want me to believe 0:20:33.230 --> 0:20:35.830 that it's all like, this is a plan from China 0:20:35.830 --> 0:20:37.949 to hurt Americans and all of that. It's like you 0:20:37.949 --> 0:20:41.949 have to be willing to give something quite, quite strong there. 0:20:41.949 --> 0:20:45.820 Otherwise you're asking like people like me to give up 0:20:45.820 --> 0:20:48.619 all of the benefits that we get from this in 0:20:48.619 --> 0:20:51.899 exchange for like, some global foreign policy that really doesn't 0:20:51.899 --> 0:20:54.219 seem to affect us much at all. Like, I don't 0:20:54.219 --> 0:20:57.780 notice these things in our normal life. I notice like 0:20:58.020 --> 0:21:00.540 the fact that, okay, I now can't even use this 0:21:00.540 --> 0:21:03.139 model or I can't do that. Like for my daily 0:21:03.139 --> 0:21:07.699 life and for my work. Chinese models giving me like 0:21:08.020 --> 0:21:11.340 open weight models is like just positive for most Americans. 0:21:12.260 --> 0:21:14.899 Yeah. And they're subsidizing it. So it's cheaper too. It's 0:21:14.899 --> 0:21:17.500 like very nice of them. So, so basically it's true. 0:21:17.540 --> 0:21:20.060 Wait, but that's not true because from the perspective of inference. 0:21:20.060 --> 0:21:22.419 So if you meant inference there, that's not true because 0:21:22.459 --> 0:21:25.820 like Google Cloud's pricing is, is like $1 per input token, 0:21:25.820 --> 0:21:28.459 which would be compared to like $4 per input token 0:21:28.459 --> 0:21:31.780 for opus. And of course, Google Cloud's like, that's just 0:21:31.780 --> 0:21:34.100 Google cloud. It's not China subsidizing that. Of course, you 0:21:34.100 --> 0:21:36.619 could be saying like that they capitalizing the cost into the. 0:21:36.659 --> 0:21:37.500 No, I'm saying like. 0:21:37.500 --> 0:21:38.259 They really are. 0:21:38.500 --> 0:21:41.580 I'm talking about like EC2, like a whole bunch of 0:21:41.580 --> 0:21:44.970 Chinese models that are available that people can use as 0:21:44.969 --> 0:21:49.289 alternatives to US models. They are super cheap, and you're 0:21:49.290 --> 0:21:51.609 literally taking all of the stuff in your harness and 0:21:51.609 --> 0:21:53.770 all your cool ideas, and it's going off to China. 0:21:53.810 --> 0:21:55.770 I'm sure they're doing nothing bad with it, right? 0:21:55.810 --> 0:21:57.450 But that's not true for like, but that's not true. 0:21:57.449 --> 0:21:59.650 If you're running it on like Google Cloud. So if 0:21:59.689 --> 0:22:01.129 you run any of these. 0:22:01.169 --> 0:22:02.129 Oh, sure sure, sure. 0:22:02.169 --> 0:22:04.769 Hosted. Right. And they're still cheaper. So the point is 0:22:04.770 --> 0:22:07.330 it's not about stealing our data because they're not getting 0:22:07.330 --> 0:22:10.529 the data. It's not about that. They're forcing us to 0:22:10.530 --> 0:22:13.409 use their AI APIs because they're not. And that's actually 0:22:13.409 --> 0:22:15.650 one of the things is that that's very hard for 0:22:15.689 --> 0:22:17.449 me to accept, I think very hard for a lot 0:22:17.449 --> 0:22:19.970 of people to accept, which is like Anthropic's plan is 0:22:19.969 --> 0:22:22.850 to say, uh, we and maybe a few other labs 0:22:22.850 --> 0:22:26.369 in America will have the best models. You will have 0:22:26.369 --> 0:22:29.409 to go through us for your own safety. Um, we 0:22:29.409 --> 0:22:31.809 will block the companies that were willing to give it 0:22:31.810 --> 0:22:35.690 to you cheaper and better because for your own safety. And, um, 0:22:35.730 --> 0:22:37.809 and no, we don't have any intention or plan of 0:22:37.889 --> 0:22:41.040 giving you cheaper models, right. We don't have any intention 0:22:41.040 --> 0:22:42.959 or plan of giving you better models that will be 0:22:43.000 --> 0:22:45.560 able that will serve your actual purposes. So you see 0:22:45.600 --> 0:22:47.679 why that's such a hard sell. Like politically. 0:22:48.080 --> 0:22:51.520 I hear you, I hear you. That sounds it sounds awesome. 0:22:51.520 --> 0:22:56.200 Sounds amazing. Um, it sounds amazing. It really does. Like 0:22:56.199 --> 0:23:00.879 I'm not being facetious there. Um, the, the issue is that, okay, 0:23:01.119 --> 0:23:06.999 let me continue to propose this, this training strategy, um, the, 0:23:07.000 --> 0:23:13.160 the concept of, um, the US companies having a stranglehold, 0:23:13.159 --> 0:23:16.119 like super strong control, the best ice that just keeps 0:23:16.119 --> 0:23:19.000 getting better and better. And China is kind of behind 0:23:19.040 --> 0:23:22.600 that is not okay for China. So they decided, I 0:23:22.600 --> 0:23:24.280 don't know when like a year and a half ago, 0:23:24.280 --> 0:23:27.800 three years, who knows to go the open source model, 0:23:27.800 --> 0:23:32.840 just really everything public, public, public. Um, now, so I 0:23:32.879 --> 0:23:35.280 happen to study China quite a bit. I've read like 0:23:35.320 --> 0:23:38.520 dozens of books about it. They, they just clearly state 0:23:38.520 --> 0:23:41.460 what their strategy is, which is to win, and for 0:23:41.459 --> 0:23:44.540 the U.S. to become a minor power with them being 0:23:45.020 --> 0:23:47.820 a major power like this is not conjecture. Like, they 0:23:47.820 --> 0:23:50.379 just write about it. They talk about it openly because 0:23:50.379 --> 0:23:53.340 they think no one is is actually reading anything. They're 0:23:53.340 --> 0:23:57.540 writing because it's in Chinese. So it's very clear what 0:23:57.540 --> 0:24:01.899 they're actually trying to do. Right. Um, and the fact 0:24:01.899 --> 0:24:04.500 that they're giving out, oh, we're just free and open. 0:24:04.820 --> 0:24:07.179 It's not free and open in China. It's not free 0:24:07.179 --> 0:24:09.739 and open in Hong Kong, which they took over. Um, 0:24:09.740 --> 0:24:11.419 it's not like they're going to give us all these 0:24:11.419 --> 0:24:13.260 models and then we're going to be like, yeah, so 0:24:13.260 --> 0:24:16.500 what are alternatives to Chinese government? And they're like, yeah, 0:24:16.500 --> 0:24:20.100 that's a perfectly, you know, natural question you're allowed to ask. 0:24:20.459 --> 0:24:22.700 It is allowed. We are allowed to ask it in 0:24:22.699 --> 0:24:24.619 the US. We're not allowed to ask it in China. 0:24:25.219 --> 0:24:28.980 So like, I don't think I mean, you're you're you're 0:24:28.980 --> 0:24:31.700 kind of jumping ahead to a world in which I 0:24:31.699 --> 0:24:34.739 agree with you. Okay. Both of you and I agree 0:24:34.980 --> 0:24:37.369 in the Star Trek world. What are you talking about? 0:24:37.369 --> 0:24:40.929 All this should be open source. I 1,000% agree with you. 0:24:41.169 --> 0:24:43.210 This should all be open source. We shouldn't have these 0:24:43.209 --> 0:24:47.850 crazy governments like me. And you are probably Star Trek liberals. 0:24:47.850 --> 0:24:50.330 I'm talking about how we get from here to there. 0:24:50.889 --> 0:24:55.010 And I don't want an extra step where CCP runs 0:24:55.010 --> 0:24:58.569 all of this. Um, now, now, let me ask you this. 0:25:00.010 --> 0:25:03.449 You're saying that the actions anthropic is taking is like really, 0:25:03.449 --> 0:25:07.250 really bad. Wasn't his action to go and get all 0:25:07.290 --> 0:25:11.769 of his competitors and get them all on board and 0:25:11.770 --> 0:25:14.810 do this shared community release of like, we're going to 0:25:14.810 --> 0:25:17.449 give this to hundreds of companies, they're going to come back, 0:25:17.490 --> 0:25:20.449 we're going to roll it out slowly for the purpose 0:25:20.449 --> 0:25:23.409 of doing the thing that you said, which is you 0:25:23.449 --> 0:25:26.969 kind of like, do the controlled burns in the, uh, 0:25:27.010 --> 0:25:30.290 in the forest. You burn things down, you clean things up, 0:25:30.290 --> 0:25:33.129 then you start releasing it. And now I would agree 0:25:33.129 --> 0:25:36.119 with you, once you've done that, maybe it's okay now 0:25:36.119 --> 0:25:39.719 we can release more, but you don't just drop it. Um, 0:25:40.119 --> 0:25:43.760 because here's the thing, dude, they don't they don't hate money. 0:25:43.760 --> 0:25:46.480 They like money. They would have made so much more 0:25:46.480 --> 0:25:48.280 money if they just released mythos. 0:25:48.560 --> 0:25:50.919 Of course. And, and, and believe me, it's because I 0:25:50.919 --> 0:25:53.560 believe that they're believers. Like, I think it comes down 0:25:53.560 --> 0:25:55.639 back to that now on the China point, like it's 0:25:55.639 --> 0:25:57.679 important to understand. Like, so I actually, I studied at 0:25:57.679 --> 0:25:59.920 the University of Hong Kong law school for a year, uh, 0:25:59.919 --> 0:26:05.759 the year prior to Hong Kong, basically getting completely, you know, done, uh, 0:26:05.800 --> 0:26:08.159 and that the protests that kicked off actually kicked off 0:26:08.159 --> 0:26:09.959 from the University of Hong Kong law school. So I 0:26:09.959 --> 0:26:12.920 was like at the epicenter of this point. Um, and 0:26:12.919 --> 0:26:15.880 I have tons of friends who are, uh, from mainland China, 0:26:15.879 --> 0:26:18.520 including one who by all accounts, I will, you know, 0:26:18.560 --> 0:26:21.840 can pretty openly say he is like clearly a spy, uh, 0:26:21.879 --> 0:26:24.359 for China. He works for in the US for a 0:26:24.359 --> 0:26:27.760 tech company. Um, and I happen to know his background. Right. 0:26:27.800 --> 0:26:29.480 Well enough to know that he would be a spy 0:26:29.520 --> 0:26:32.239 for China. Um, and so, but the fact of the 0:26:32.240 --> 0:26:33.960 matter is, and like, no, of course I don't think 0:26:33.990 --> 0:26:37.869 that's great. Um, you know, at the end of the day, 0:26:38.189 --> 0:26:40.990 I'm sitting here going, yeah, but they are giving us 0:26:40.990 --> 0:26:43.710 models that. No. Yes. If I asked about Tiananmen Square, 0:26:43.709 --> 0:26:47.109 it would deny that. Right. Like it will deny that request. Okay. 0:26:47.149 --> 0:26:49.989 But I don't typically have to ask my AI models 0:26:49.990 --> 0:26:52.869 about Tiananmen Square. Right? Like it's not it's not a 0:26:52.869 --> 0:26:55.629 thing that comes up when I'm doing like any type 0:26:55.629 --> 0:26:59.030 of cybersecurity work, right? Those types of like, can they 0:26:59.030 --> 0:27:03.310 influence it with pro-Chinese bias is such like an abstract 0:27:03.310 --> 0:27:07.709 concept of how you might influence a population in comparison 0:27:07.709 --> 0:27:11.429 to like, what if only Dario controlled it? And now 0:27:11.429 --> 0:27:13.230 when you mentioned it like, okay, but that it was 0:27:13.230 --> 0:27:15.550 that he went and got all of his competitors together 0:27:15.550 --> 0:27:17.830 to this is just what we call like collusion, like 0:27:17.830 --> 0:27:20.590 it's illegal for a reason and normal circumstances that would 0:27:20.590 --> 0:27:23.149 be a crime. Uh, we're willing to accept it because 0:27:23.149 --> 0:27:25.990 we pass it off through the safety exception. But in 0:27:26.030 --> 0:27:29.909 normal circumstances, putting together, like all of the leaders of 0:27:29.909 --> 0:27:33.169 all of the major companies of a Get competitors and saying, 0:27:33.169 --> 0:27:35.250 how about we cooperate on this would just would just 0:27:35.250 --> 0:27:37.889 be illegal because that would be a cartel, right? So 0:27:37.889 --> 0:27:41.129 we don't want cooperation. I do not want cooperation between 0:27:41.129 --> 0:27:42.689 these labs. I want them to compete. 0:27:44.129 --> 0:27:47.489 Okay. But they're competing. You're familiar with like the Malek, 0:27:47.770 --> 0:27:49.530 like you're chasing this thing that you know is going 0:27:49.530 --> 0:27:52.730 to kill you and you're you're aggressively chasing a thing. 0:27:52.770 --> 0:27:55.289 Everyone knows if we get there, it's going to kill us, 0:27:55.649 --> 0:27:57.770 but we can't let the other person win. So we 0:27:57.770 --> 0:27:59.489 accelerate like that. 0:27:59.490 --> 0:28:02.090 That's that's what they live in. That's what they believe. Right? 0:28:02.689 --> 0:28:04.530 Well, no, but that's what they would do if they 0:28:04.530 --> 0:28:08.609 weren't collaborating in some sort of way towards a positive outcome. Right. 0:28:08.649 --> 0:28:11.650 So so they literally said, and Sam has also said 0:28:11.649 --> 0:28:14.650 this earlier, um, I've a whole, whole nother talk show 0:28:14.649 --> 0:28:19.969 on Sam, but, um, they're not just saying, let's collaborate. 0:28:19.969 --> 0:28:23.529 They're saying, let's collaborate with this thing, check this out. 0:28:23.929 --> 0:28:26.609 This concept of government is the idea that the people 0:28:26.609 --> 0:28:31.040 are involved. So it's them, all of them getting together 0:28:31.040 --> 0:28:33.119 with the government to say, hey, what is the best 0:28:33.119 --> 0:28:36.280 thing that we should do? That is literally the consortium 0:28:36.480 --> 0:28:40.760 that that he is arguing for and, and saying he's saying, look, 0:28:40.760 --> 0:28:43.800 we need to be regulated. People need to be involved. 0:28:43.800 --> 0:28:46.280 We want all of our competitors to be involved. We're 0:28:46.280 --> 0:28:49.239 trying to control the speed that this comes out. And 0:28:49.240 --> 0:28:52.000 that's literally what he's proposed and. 0:28:52.000 --> 0:28:54.519 Done, because I don't want them to cooperate on the 0:28:54.520 --> 0:28:56.440 speed with which it comes out, because that means that 0:28:56.440 --> 0:28:58.840 they can both say, hey, because what I thought when 0:28:58.840 --> 0:29:00.600 they didn't, by the way, in the chat, there is 0:29:00.600 --> 0:29:02.200 a guy named Daniel that I worked with as a 0:29:02.200 --> 0:29:04.239 developer for years ago. That's why I laughed because like, yo, 0:29:04.240 --> 0:29:07.120 what's up Daniel? Uh, just like it's been like eight 0:29:07.120 --> 0:29:09.999 years since I talked to that guy. Uh, but anyways, um, 0:29:10.080 --> 0:29:12.840 back on the point if we go back to that though, 0:29:12.880 --> 0:29:15.640 is that if you think that like, uh, when mythos 0:29:15.640 --> 0:29:17.200 came out and they said, we're not going to release 0:29:17.200 --> 0:29:21.040 mythos to anyone, I was like, watch it. Because what 0:29:21.040 --> 0:29:23.600 happens is, of course, if Sam Altman releases his version 0:29:23.600 --> 0:29:25.999 of mythos and like, then of course it kind of 0:29:26.040 --> 0:29:30.430 forces Dario to release mythos, Right. So the game like 0:29:30.469 --> 0:29:34.870 the game here of competition forces them into the world 0:29:34.870 --> 0:29:36.789 that like I want to live in, which is that like, 0:29:36.830 --> 0:29:38.630 we just get the best models and we get the 0:29:38.630 --> 0:29:41.350 best prices and we get it the fastest. That's the 0:29:41.350 --> 0:29:44.470 world that I want. And so the, the collusion they're 0:29:44.469 --> 0:29:46.590 trying to do is to avoid that world, but it's 0:29:46.590 --> 0:29:49.390 coming at a cost and the cost is not on them. 0:29:49.390 --> 0:29:52.670 The cost is on us. They're already trillionaires. Right. It's 0:29:52.670 --> 0:29:55.470 like the permanent underclass that loses from this. It's all 0:29:55.469 --> 0:29:58.550 the people who are not trillionaires and do not control 0:29:58.550 --> 0:30:01.469 the AI, you know, or are not friends with anthropic, 0:30:01.469 --> 0:30:02.110 because of course. 0:30:02.270 --> 0:30:05.470 We still have. We still have. I'm sorry. 0:30:05.750 --> 0:30:08.350 Yeah, but but they but but opus being much more 0:30:08.350 --> 0:30:10.469 expensive than what we got from the labs that he 0:30:10.469 --> 0:30:13.989 doesn't want to have it now, but also importantly, um, 0:30:14.110 --> 0:30:17.509 you know, you mentioned what I was about to say there, uh, 0:30:17.510 --> 0:30:18.950 I forget whatever that was it. 0:30:20.830 --> 0:30:26.950 Okay. So I, I think, I think I'm hearing something 0:30:27.380 --> 0:30:31.620 strange in the argument. The reason I reacted so viscerally 0:30:31.620 --> 0:30:35.340 to your video is because you were saying Daria was 0:30:35.340 --> 0:30:38.100 a bad person. Now you're telling me I agree with you. 0:30:38.140 --> 0:30:41.740 He's a good person. But the output, the output that 0:30:41.740 --> 0:30:45.100 he is asking for would be bad for us. No, no, no. 0:30:45.140 --> 0:30:46.500 I don't think he's a good. I don't think he's 0:30:46.500 --> 0:30:48.700 a good person just because he genuinely believes. I mean, 0:30:48.700 --> 0:30:50.779 like I said, I think in the video there are 0:30:50.780 --> 0:30:53.460 people who have believed in like ethnic cleansing and they, like, 0:30:53.459 --> 0:30:56.900 genuinely believe in it. Uh, but it's like still evil. Right. 0:30:56.900 --> 0:30:58.740 And I think it's the same as that to he 0:30:58.780 --> 0:31:01.540 genuinely believes he's keeping us safe and that it's better 0:31:01.540 --> 0:31:03.740 for everyone to keep us safe and that he's doing 0:31:03.740 --> 0:31:06.420 that for safety. Um, the problem is, of course, is 0:31:06.420 --> 0:31:08.820 that he is willing by doing that, he is willing 0:31:08.860 --> 0:31:12.820 to basically create a society in which very few people 0:31:12.820 --> 0:31:16.420 are benefiting because at least they didn't die. Okay. I think, uh, what. 0:31:16.420 --> 0:31:19.660 Are you talking. He still wants to release all these models. 0:31:19.700 --> 0:31:21.380 He still wants to release them. 0:31:21.380 --> 0:31:23.620 No, no, no, he wants to release the current level 0:31:23.620 --> 0:31:25.920 of models. He's very clear on that. The next. and 0:31:25.920 --> 0:31:28.719 the next generations of models we will never receive. Okay, 0:31:28.760 --> 0:31:30.600 he is very clear on the fact that he believes. 0:31:30.600 --> 0:31:33.000 What are you talking about? They're in development. They're going 0:31:33.000 --> 0:31:35.080 to come out. What? What are you talking about? 0:31:35.280 --> 0:31:38.320 It's his quotes. He literally says, in the future, you know, 0:31:38.360 --> 0:31:40.360 we might have to treat this like nuclear weapons, not 0:31:40.360 --> 0:31:43.120 as a safety risk, but rather as a risk to 0:31:43.160 --> 0:31:46.800 society and to humanity, and that we would need tighter controls. 0:31:46.840 --> 0:31:49.160 You know, basically the point being then it is not 0:31:49.160 --> 0:31:51.400 a public good. It becomes, you know, not not a 0:31:51.400 --> 0:31:53.160 public good is the wrong term there. It is not 0:31:53.160 --> 0:31:54.640 a good that he would be willing to release to 0:31:54.680 --> 0:31:57.520 the public ever. I think Dario would not deny that. 0:31:57.520 --> 0:31:58.080 I think. 0:31:58.120 --> 0:31:59.760 But if you look at the context. 0:31:59.800 --> 0:32:03.039 Of that, Dario does not want to release all future 0:32:03.040 --> 0:32:05.160 models if they become sufficiently smart. 0:32:05.480 --> 0:32:09.719 He's talking about in the case where it becomes certain 0:32:09.719 --> 0:32:12.360 amount of dangerous. And like you shouldn't just drop it 0:32:12.360 --> 0:32:14.680 on the world, which is my point. Okay. Which I 0:32:14.680 --> 0:32:17.839 think we disagree about. Yeah. But we disagree about the 0:32:17.840 --> 0:32:21.400 potential impact of that. He believes, as I do, that 0:32:21.400 --> 0:32:24.789 the potential impact of dropping. Let me give you a 0:32:24.790 --> 0:32:28.350 better example. Okay, let's call this mythos six or mythos 0:32:28.350 --> 0:32:33.190 seven or whatever. Okay. Um, give me I don't even 0:32:33.190 --> 0:32:37.670 know if I should say this prompted. Um, yeah. Okay. 0:32:37.709 --> 0:32:42.670 Imagine the creation of a wet lab combined with doing 0:32:42.670 --> 0:32:48.110 the research for creating, uh, uh, different, uh, pathogens that 0:32:48.110 --> 0:32:51.190 combine the worst of the worst. And the problem with 0:32:51.190 --> 0:32:53.269 this is that the knowledge has been there. Like if 0:32:53.270 --> 0:32:57.270 you could jailbreak or whatever potentially been there, but as 0:32:57.270 --> 0:33:02.269 you start to move into AGI, ASI maybe mythos 6728 0:33:02.670 --> 0:33:05.029 who knows when it's going to happen? It starts to 0:33:05.030 --> 0:33:07.830 be like, oh, actually, if you make a cut here 0:33:07.830 --> 0:33:12.270 in this particular virus and you add this, um, okay, 0:33:12.310 --> 0:33:16.509 this will kill everyone. This will effectively do what your goal, 0:33:16.510 --> 0:33:19.030 you did a slash goal. And guess what? I made 0:33:19.030 --> 0:33:21.830 it for you. The problem has always been, okay, that 0:33:21.830 --> 0:33:25.299 knowledge is fine. Kind of like with nuclear. That's fine. 0:33:25.340 --> 0:33:28.180 The knowledge is there. But you don't have a lab. Well, 0:33:28.180 --> 0:33:30.219 guess what? I'm going to give you all the stuff 0:33:30.219 --> 0:33:32.580 to go and make this in a lab. And I'm 0:33:32.580 --> 0:33:34.740 going to help you automate the process. And now we've 0:33:34.740 --> 0:33:38.180 actually built it. This little canister here now has the 0:33:38.180 --> 0:33:42.500 thing you believe that is safe to provide that level 0:33:42.500 --> 0:33:45.340 of knowledge to the world, the entire world. Give it 0:33:45.340 --> 0:33:47.580 to CCP, give it to Joe next door. 0:33:47.980 --> 0:33:51.100 Importantly, here, this this kind of moves into the world 0:33:51.100 --> 0:33:53.459 of what we consider like, I think Nick Bostrom refers 0:33:53.459 --> 0:33:54.940 to it as like this idea that you can make 0:33:54.940 --> 0:33:57.860 nukes like in your microwave, right? If there is some 0:33:57.860 --> 0:34:01.980 piece of technology that like we discover that would, whether 0:34:01.980 --> 0:34:04.900 that's through AI or something else that would allow like 0:34:04.940 --> 0:34:09.660 every random person to go make like civilization ending technology 0:34:09.820 --> 0:34:14.100 at home easily, right? That would be catastrophically bad for 0:34:14.100 --> 0:34:17.620 the existence of humanity. That would be like probably civilization 0:34:17.620 --> 0:34:20.300 ending if it got out. Like that's what we're talking 0:34:20.300 --> 0:34:23.009 about there. The problem is, of course, I don't believe this. Okay. 0:34:23.050 --> 0:34:25.770 I think that we've not seen sufficient evidence that AI 0:34:25.810 --> 0:34:27.890 will get to a level that will allow that to 0:34:27.930 --> 0:34:31.569 be possible. And with that being the case, what happens is, 0:34:31.569 --> 0:34:34.410 is that we're starting to be asked to accept, again, 0:34:34.450 --> 0:34:37.289 like fairly liberal positions of like, we will allow for 0:34:37.290 --> 0:34:40.209 the AI labs to control larger and larger. I hear 0:34:40.210 --> 0:34:43.049 you get the best models. All in favor of this 0:34:43.049 --> 0:34:46.410 theoretical possibility. And now I'm sitting here going, well, let's 0:34:46.410 --> 0:34:49.170 say that Dario says the counter to this is, well, Zach, 0:34:49.210 --> 0:34:52.009 you don't have the access to the information like I. 0:34:52.049 --> 0:34:54.890 Dario do. Dario. I'm Dario and I can see the 0:34:54.890 --> 0:34:56.930 model does it. The problem is of course I don't 0:34:56.930 --> 0:34:59.130 want Dario to be able to see this either. Because 0:34:59.130 --> 0:35:00.890 if it is at that level. Of course I think 0:35:00.890 --> 0:35:03.689 it should be like completely expropriated, you know, expropriated out 0:35:03.690 --> 0:35:06.050 of his hands and like the government should just roll in. 0:35:06.049 --> 0:35:07.810 But he doesn't want that either, which is kind of 0:35:07.850 --> 0:35:10.529 a good signal that this is not that. Right. This 0:35:10.529 --> 0:35:12.250 is not telling people how to. 0:35:12.730 --> 0:35:14.729 What do you mean? He is literally saying we should 0:35:14.730 --> 0:35:17.009 be regulated. This stuff is super dangerous. 0:35:17.049 --> 0:35:19.229 Regulation is not what you would do if you believed 0:35:19.230 --> 0:35:21.910 that you had civilization ending technology in your hands. And 0:35:21.910 --> 0:35:22.270 I think that. 0:35:22.390 --> 0:35:25.350 He believes it's it's about to come. He believes it's 0:35:25.350 --> 0:35:28.870 starting like it's very clear from his statements. 0:35:29.150 --> 0:35:31.470 Do you think that Dario. It's a very interesting question, 0:35:31.469 --> 0:35:34.709 is if I believed. So this is one area where 0:35:34.710 --> 0:35:37.029 I do think it becomes very clear where Dario and 0:35:37.029 --> 0:35:40.270 I disagree. The strongest, besides just about what we believe 0:35:40.270 --> 0:35:43.549 comes in the future. If I believed that that was 0:35:43.549 --> 0:35:46.069 coming in the future, there would be one and only 0:35:46.069 --> 0:35:48.190 one appropriate answer, which is to hand it over to 0:35:48.190 --> 0:35:50.469 the government and say, look, guys, this is now a 0:35:50.469 --> 0:35:52.430 matter of your this is on, you. 0:35:52.469 --> 0:35:54.790 Know, now we're now we're getting to it. 0:35:54.830 --> 0:35:57.430 No, no, no, but not as a matter of regulation, 0:35:57.870 --> 0:36:01.430 but as just full on nationalization, because most people would 0:36:01.430 --> 0:36:05.230 accept that you should not allow civilization ending technology to 0:36:05.230 --> 0:36:08.270 be developed in a private lab, uh, and making trillions 0:36:08.270 --> 0:36:11.430 of dollars. But Dario referred to that as communism. Okay. 0:36:11.469 --> 0:36:14.230 So Dario very openly said, this is communist. Okay. 0:36:14.430 --> 0:36:16.710 So say that again. Say that again. What did he 0:36:16.710 --> 0:36:17.540 say was communist? 0:36:17.900 --> 0:36:20.900 Uh, nationalization. So when he said that some people argue 0:36:20.940 --> 0:36:23.100 now that because of the risks that it should be nationalized, 0:36:23.100 --> 0:36:26.460 he said, guys, you're yo yoing between, uh, full, um, 0:36:26.500 --> 0:36:30.379 basically full libertarianism and full communism. And so is of course, 0:36:30.380 --> 0:36:33.060 the implication being that nationalization was the communist position. He 0:36:33.060 --> 0:36:37.380 did not he if if, if Dario came out tomorrow 0:36:37.380 --> 0:36:39.899 and said, I think that the US government should nationalize 0:36:39.900 --> 0:36:44.180 all AI labs, I'd say I'm sorry to Dario because 0:36:44.219 --> 0:36:46.459 at least he has. I would say he's wrong because 0:36:46.460 --> 0:36:48.740 of course, I don't believe in the pre. The premise 0:36:48.739 --> 0:36:51.220 that this is going to become make a wet lab 0:36:51.219 --> 0:36:54.100 at home, kill everyone technology. But I would at least 0:36:54.100 --> 0:36:57.299 say he's drawn the correct conclusions from the series of 0:36:57.299 --> 0:37:00.020 things he believes the problem. The biggest problem I have 0:37:00.020 --> 0:37:02.939 with Dario is that he wants to believe that, uh, 0:37:02.940 --> 0:37:05.819 this is civilization ending technology and that he should be 0:37:05.819 --> 0:37:07.580 able to make $1 trillion off of it. And that's 0:37:07.580 --> 0:37:09.620 like super weird. Like, why would anyone think that? 0:37:10.339 --> 0:37:12.939 What do you mean? So, so, so I think our 0:37:12.940 --> 0:37:17.249 positions are extremely close here, Extremely close. I think this 0:37:17.250 --> 0:37:19.969 is a framing issue. Okay. So first of all, the 0:37:19.969 --> 0:37:23.129 government has no ability to do anything. They they have 0:37:23.130 --> 0:37:28.049 no AI expertise. They can't make anything. We can't nationalize crap. Um, 0:37:28.210 --> 0:37:33.330 guess what an actual nationalization looks like. An actual nationalization. 0:37:33.330 --> 0:37:36.690 Looks like the labs get together and they're like, hey, 0:37:36.730 --> 0:37:39.809 we're inviting the government in to be a huge part 0:37:39.810 --> 0:37:42.209 of this because this is super important, which happens to 0:37:42.210 --> 0:37:44.450 be the exact thing that he's asking for. 0:37:44.850 --> 0:37:48.850 No one. So if I went and asked my parents 0:37:48.850 --> 0:37:51.570 who don't know about AI or my brother who's a lawyer, 0:37:51.569 --> 0:37:54.450 or any one of my friends who aren't in technology 0:37:54.489 --> 0:37:57.050 to explain that premise to me, which is that this 0:37:57.049 --> 0:38:00.290 technology is being developed and it might kill everyone in 0:38:00.290 --> 0:38:03.609 the world, right? It might cause basically mass, mass, mass extinction, 0:38:03.610 --> 0:38:06.049 which is the belief that he has. Right. And then 0:38:06.049 --> 0:38:08.450 he said, okay, but they don't believe that this should 0:38:08.450 --> 0:38:11.969 be nationalized and they just believe it should be regulated. 0:38:12.210 --> 0:38:15.640 All of the people, all of those normal non-tech people 0:38:15.640 --> 0:38:19.759 would go, what? That's absurd. Obviously, we don't want to 0:38:19.759 --> 0:38:23.120 see people, uh, having any sort of. You know, you're 0:38:23.120 --> 0:38:25.560 basically it's a negative externality that you're basically saying, what 0:38:25.560 --> 0:38:28.000 if we killed everyone and made $1 trillion in the meantime? 0:38:28.040 --> 0:38:29.479 I mean, like, what are you talking about? Of course 0:38:29.480 --> 0:38:30.839 you want to support that. 0:38:30.880 --> 0:38:34.399 What does nationalization mean? What I'm arguing to you is 0:38:34.400 --> 0:38:39.160 that there is a smooth spectrum between the government having 0:38:39.160 --> 0:38:45.519 control and oversight and transparency into all U.S. companies who 0:38:45.520 --> 0:38:47.999 are pushing towards the same goal. And there's like some 0:38:48.000 --> 0:38:52.000 sort of internal consortium there with the government actually having 0:38:52.000 --> 0:38:56.239 veto power and control. And and actually the labs are 0:38:56.239 --> 0:38:58.680 the ones doing the work because the government doesn't there 0:38:58.719 --> 0:39:02.359 is no nationalization there. Like, there's not a government thing 0:39:02.360 --> 0:39:05.879 that is just like anthropic and just like Google. And 0:39:05.880 --> 0:39:06.839 it's just as good. 0:39:07.080 --> 0:39:09.120 It would be anthropic. You would expropriate it, it would 0:39:09.120 --> 0:39:11.559 be yours. Now you own 100% of the shares. Game over. Right? 0:39:11.600 --> 0:39:12.830 It would be anthropic. 0:39:13.549 --> 0:39:17.149 Yeah. Okay. So imagine you're Dario. Do you think that's 0:39:17.150 --> 0:39:20.910 a good idea right now, given our administration? 0:39:21.589 --> 0:39:23.870 Of course. Dario does not believe this. 0:39:24.190 --> 0:39:26.509 Okay. Do you believe it's a good idea? Okay. Let 0:39:26.509 --> 0:39:28.189 me ask you this. Do you think it's a great 0:39:28.190 --> 0:39:34.549 idea to hand all the power of of anthropic and Sam? 0:39:34.589 --> 0:39:38.149 Sam and Dario should go over and meet with Hegseth 0:39:38.589 --> 0:39:41.310 and Trump and say, hey, this is all yours. Um, 0:39:41.469 --> 0:39:43.470 we just got a piece out of this game because 0:39:43.469 --> 0:39:46.670 you guys are super you guys are super talented in this, 0:39:46.670 --> 0:39:48.470 and it's all yours. Take it. 0:39:48.830 --> 0:39:50.669 Wait, I, I absolutely don't believe that because of course, 0:39:50.670 --> 0:39:53.550 I don't believe the things that Dario believes about the world. 0:39:53.549 --> 0:39:56.310 But if I was sitting in Dario's shoes believing the 0:39:56.310 --> 0:39:59.390 things Dario believes, yes, I would, because I think that 0:39:59.390 --> 0:40:01.310 even though we might say like, oh, well, we don't 0:40:01.310 --> 0:40:03.750 think that the Trump administration would do a good job 0:40:03.750 --> 0:40:06.390 or whatever you might believe. Uh, at the end of 0:40:06.390 --> 0:40:08.910 the day, a public process is like, that is the 0:40:08.910 --> 0:40:11.729 best system we have, what they're saying. And the reason 0:40:11.730 --> 0:40:14.129 that I have such objections to what Dario's position is, 0:40:14.170 --> 0:40:17.690 is to say, you can't trust this government which was elected. 0:40:17.890 --> 0:40:20.690 You can trust Dario, who wasn't elected. As long as 0:40:20.690 --> 0:40:23.129 we get a light level of regulation from the government. No, 0:40:23.130 --> 0:40:25.049 I think that absolutely, if you were to move to 0:40:25.089 --> 0:40:27.530 this fake extreme world, which I do not believe in, 0:40:27.569 --> 0:40:30.450 you would believe in in just full nationalization. No one 0:40:30.450 --> 0:40:34.490 should be profiting off of taking a 25% risk of 0:40:34.489 --> 0:40:37.530 everyone dying. Okay. So like, that's a very basic premise, 0:40:37.569 --> 0:40:40.250 which is like, if you believe there's a 25% chance 0:40:40.250 --> 0:40:42.810 of extinction risk, which admittedly is not what Dario said. 0:40:42.850 --> 0:40:45.649 It said 25% chance of very, very, very poorly. He 0:40:45.650 --> 0:40:48.330 didn't call it extinction risk. But if you believe 25% 0:40:48.330 --> 0:40:51.010 chance of going very, very, very poorly, I don't believe 0:40:51.009 --> 0:40:52.770 that we should be driving that as a for profit 0:40:52.770 --> 0:40:55.770 profit enterprise. That would be basically just saying if if 0:40:55.770 --> 0:40:58.729 it goes well, we become your new trillionaire overlords. If 0:40:58.730 --> 0:41:00.609 it goes poorly, you all die and who cares, right? 0:41:00.650 --> 0:41:03.009 That's that's just not an acceptable state of the world. 0:41:03.049 --> 0:41:06.930 I hear you. So so I think you're talking about 0:41:07.529 --> 0:41:11.080 worlds that don't exist. Zach. That's what I think you're 0:41:11.080 --> 0:41:11.479 talking about. 0:41:11.520 --> 0:41:14.799 100% agree because I'm talking about the world's Dario has 0:41:14.799 --> 0:41:15.440 put forward. 0:41:15.680 --> 0:41:18.120 No no no. I think you're talking about worlds that 0:41:18.120 --> 0:41:24.120 don't exist. Okay. I believe that this progress is is happening. Um, 0:41:24.440 --> 0:41:27.800 I believe Dario when he says they were surprised by mythos. 0:41:27.799 --> 0:41:28.720 Do you agree with that? 0:41:28.960 --> 0:41:29.759 Of course. Yes. 0:41:30.239 --> 0:41:32.680 Okay. So he was surprised. He's like, Holy crap, this 0:41:32.680 --> 0:41:34.400 is way better than I thought it was going to be. 0:41:34.719 --> 0:41:36.640 This is happening faster than I thought it was going 0:41:36.640 --> 0:41:40.000 to happen. Okay. Do you agree that most AI experts 0:41:40.000 --> 0:41:43.600 are worried about the future and quickly being able to 0:41:43.600 --> 0:41:45.160 have this kind of capability? 0:41:46.360 --> 0:41:48.920 Yeah. Most AI experts, yes. But that's that's a narrow 0:41:48.920 --> 0:41:49.799 pool of technology. 0:41:49.839 --> 0:41:52.720 No, no. But what what I'm what I'm saying to you, Zach, 0:41:52.719 --> 0:41:56.959 is that there is a consensus. Everyone knows. Every book 0:41:56.960 --> 0:42:02.640 written knows. Holy crap. We are going into something very strange. Uh, 0:42:02.640 --> 0:42:06.080 we're at the point where things like auto research can 0:42:06.080 --> 0:42:09.510 just make AI better, right? And it's slowly happening. Like, 0:42:09.549 --> 0:42:11.549 this is such an inevitable thing. 0:42:11.589 --> 0:42:14.590 Yes. Okay. Okay. But yeah. Go on. 0:42:14.870 --> 0:42:18.269 So so it's it's inevitable. We agree on that if 0:42:18.270 --> 0:42:21.029 it's inevitable and. 0:42:21.069 --> 0:42:21.629 Well I. 0:42:21.630 --> 0:42:23.549 Don't. And hand in handing it. 0:42:23.710 --> 0:42:24.670 Yes. Yeah. 0:42:25.710 --> 0:42:30.750 Handing it to. Well I don't I don't think anthropic. 0:42:30.750 --> 0:42:33.509 If they shut down tomorrow they literally turned everything off 0:42:33.509 --> 0:42:36.870 and went home. That would not. It would be a blip. 0:42:37.150 --> 0:42:38.230 It would be a blip. 0:42:38.270 --> 0:42:38.390 Of. 0:42:38.390 --> 0:42:39.310 Course. And overall. 0:42:39.310 --> 0:42:41.549 Progress. Anthropic should shut down. I'm you know. 0:42:41.549 --> 0:42:44.550 Right, right. So this is why I'm saying that that 0:42:44.870 --> 0:42:48.310 we're not arguing in a realistic way. If we're saying 0:42:48.310 --> 0:42:53.830 this is inevitable and anthropic shutting down tomorrow would would 0:42:53.830 --> 0:42:56.470 only be a blip in the progress. Then the reality 0:42:56.509 --> 0:42:59.749 we're dealing with is this is happening and we need 0:42:59.750 --> 0:43:03.510 to try to do the best we possibly can. Yes. Okay. 0:43:04.150 --> 0:43:06.859 And one, one thing that is not the best we 0:43:06.860 --> 0:43:12.219 can possibly do. One option is handing it over to 0:43:12.259 --> 0:43:15.540 the current US administration, and walking away and going home 0:43:15.540 --> 0:43:18.380 with your billions and starting a farm. That would be 0:43:18.819 --> 0:43:19.939 super cowardly. 0:43:20.660 --> 0:43:22.620 But but okay, but let's get back to the point. 0:43:22.660 --> 0:43:24.779 What I believe the best thing to do is that 0:43:24.779 --> 0:43:28.420 they just keep on building these models in a competitive way. 0:43:28.420 --> 0:43:31.620 Where OpenAI builds their models and anthropic builds their models, 0:43:31.620 --> 0:43:34.900 and maybe Google decides to build models again, and China 0:43:34.900 --> 0:43:37.060 builds their models. And we do not do anything as 0:43:37.060 --> 0:43:40.779 the US government to regulate China so that the American 0:43:40.779 --> 0:43:43.660 people can get the best models at the best prices. Right. 0:43:43.660 --> 0:43:47.140 That's what I believe. Okay. Uh, fundamentally, because if we 0:43:47.140 --> 0:43:49.979 accept the worlds of like runaway AI, that might kill 0:43:49.980 --> 0:43:52.859 everyone and all of these very much extreme positions that like, 0:43:52.900 --> 0:43:55.219 if you believe all of this is true, uh, it 0:43:55.259 --> 0:43:57.379 leads you to conclusions that are still different from what 0:43:57.420 --> 0:43:59.899 Dario is doing. Okay. Which would be like, you should 0:43:59.900 --> 0:44:02.299 believe in nationalization at that point. Like if you are 0:44:02.299 --> 0:44:04.219 publicly minded, right? Uh, well. 0:44:04.279 --> 0:44:06.280 But right now, he wouldn't want to give it to them. 0:44:06.279 --> 0:44:07.839 He wouldn't want to give it to them. 0:44:07.839 --> 0:44:10.680 Who gave? But who gave Dario the choice to make 0:44:10.680 --> 0:44:13.239 that call? If it is what he says it is, 0:44:13.239 --> 0:44:16.400 Dario has no right. Okay. It doesn't matter that I 0:44:16.400 --> 0:44:18.600 don't I don't want you know, it's like it doesn't matter. 0:44:18.600 --> 0:44:19.680 It's like at the end of the day. 0:44:20.000 --> 0:44:20.799 He has no right. 0:44:20.839 --> 0:44:21.560 To say yes. 0:44:21.960 --> 0:44:24.719 He has no right to do what I don't understand 0:44:24.719 --> 0:44:26.080 what he shouldn't be allowed to do. 0:44:26.440 --> 0:44:29.879 To decide to say, if you, like, believe all of 0:44:29.880 --> 0:44:32.000 these things and you're actually sitting in the shoes of 0:44:32.000 --> 0:44:34.200 saying it's going to kill everyone, or it might kill everyone, 0:44:34.200 --> 0:44:37.879 whatever it may be, unless somebody really takes leadership and 0:44:37.880 --> 0:44:40.160 Dario decides that Dario should be the person to take 0:44:40.160 --> 0:44:42.719 that leadership instead of, for example, the public, like the 0:44:42.719 --> 0:44:43.960 US government, right? 0:44:44.000 --> 0:44:48.840 Like Dario, Dario is taking a pro-u.s. position, which is 0:44:48.839 --> 0:44:50.359 the US should get there first. 0:44:50.880 --> 0:44:52.799 Yeah, but but Dario is not doing that through any 0:44:52.799 --> 0:44:55.280 sort of public apparatus. Dario is doing that through a private. 0:44:55.319 --> 0:44:56.759 Yes he is. Yes he is. 0:44:57.040 --> 0:44:58.560 He's a private company. 0:44:58.600 --> 0:45:04.510 He's attacking Jensen uh Wong publicly against everyone else, saying 0:45:04.509 --> 0:45:08.069 we should not be shipping our best GPUs to China. 0:45:08.430 --> 0:45:10.590 I know, but the point is, is that he's taking 0:45:10.589 --> 0:45:12.670 a pro-American position that also benefits. 0:45:12.710 --> 0:45:15.989 It's not because he benefits. No, it's not because he benefits. 0:45:16.029 --> 0:45:17.390 It's because he believes. 0:45:17.750 --> 0:45:19.710 It's because he believes that it will make him remain 0:45:19.710 --> 0:45:22.230 in power so that he can lead the world to 0:45:22.270 --> 0:45:24.230 the best path of the future. Like I think he 0:45:24.230 --> 0:45:26.350 does believe that he would be the best person to 0:45:26.390 --> 0:45:28.509 lead us to not dying. Okay. I think he does 0:45:28.509 --> 0:45:30.149 believe that. The problem is, is that. 0:45:30.190 --> 0:45:32.790 He believes he's one of the best people. One. 0:45:32.830 --> 0:45:35.350 Exactly. The point is, I find the whole premise like 0:45:35.350 --> 0:45:37.910 kind of repulsive, which is that like, if I were 0:45:37.910 --> 0:45:40.029 sitting like, I believe just to say it from this angle, 0:45:40.029 --> 0:45:42.230 like I believe very much in myself, right? I believe 0:45:42.230 --> 0:45:45.110 like I am a very good guy and all of that. 0:45:45.110 --> 0:45:47.189 But if I was sitting in the shoes of saying like, okay, 0:45:47.190 --> 0:45:49.390 I have this great technology, blah, blah, blah, and if 0:45:49.390 --> 0:45:51.469 I were saying like, and I trust myself more than 0:45:51.469 --> 0:45:54.509 the public system, I would still hand it over. Okay. 0:45:54.549 --> 0:45:56.310 And I would hand it over because I believe that 0:45:56.310 --> 0:45:58.189 there's like a process issue here, which is that like, 0:45:58.190 --> 0:46:01.100 no one picked me to decide that, okay, even if 0:46:01.100 --> 0:46:04.100 I pick me. Dario was saying pick me and this can. 0:46:04.140 --> 0:46:05.940 If you think about this future state of the world, 0:46:05.940 --> 0:46:08.540 imagine that. Like you believe in this recursive take off, like, 0:46:08.580 --> 0:46:11.660 you know, self-improvement, you know, at each point. Dario could 0:46:11.660 --> 0:46:14.180 keep saying, well, I'm actually the best person to be 0:46:14.180 --> 0:46:15.739 the dictator and I'm actually the best person to be 0:46:15.739 --> 0:46:17.620 in charge. And I'm actually the best person to be. Right. 0:46:17.620 --> 0:46:19.819 We don't want that. I want to make sure that 0:46:19.819 --> 0:46:22.259 that doesn't happen. I want to make sure that if 0:46:22.259 --> 0:46:26.100 this fast take off thing happens and that self-improvement happens, 0:46:26.100 --> 0:46:29.460 it is not controlled by somebody who thinks that they 0:46:29.460 --> 0:46:30.900 should be in charge of it. 0:46:31.900 --> 0:46:35.180 Yeah. So this is fascinating that we're talking about this. 0:46:35.460 --> 0:46:39.299 I'm reading the culture series right now. Um, I just 0:46:39.299 --> 0:46:42.379 got done with the games one and, uh, whole bunch 0:46:42.380 --> 0:46:46.020 of conversation about this. It's, it's crazy. So it got 0:46:46.020 --> 0:46:50.259 me to reading, uh, the Machiavelli stuff because, um, that 0:46:50.259 --> 0:46:53.539 just came out on Dwarkesh podcast and that got me 0:46:53.540 --> 0:46:58.539 to reading, um, Hobbes Leviathan, which is like, which goes 0:46:58.540 --> 0:47:03.210 back to Plato, which is philosopher king. And, um, what 0:47:03.210 --> 0:47:05.730 if someone believes that they're the right person or whatever, 0:47:05.730 --> 0:47:07.569 and they just think they're the best and you want 0:47:07.569 --> 0:47:09.729 to give them all the power? So I see the 0:47:09.730 --> 0:47:12.569 thing that you're getting at, which is like you're saying, 0:47:12.569 --> 0:47:15.129 it's kind of gross when someone is like, I am 0:47:15.130 --> 0:47:18.410 the good guy, I'm going to do my best. Right. 0:47:18.969 --> 0:47:22.049 Yeah. Do my best individually without, you know, obviously everyone should. 0:47:22.089 --> 0:47:25.370 No no no no no. But see, here's, here's here's 0:47:25.370 --> 0:47:30.770 my bigger point. Okay. I think there is a really 0:47:30.770 --> 0:47:36.609 gross narrative which says that if somebody believes something in 0:47:36.610 --> 0:47:39.489 a positive future, they believe that they can help in 0:47:39.489 --> 0:47:43.129 some way. They believe they can contribute and they they 0:47:43.170 --> 0:47:46.169 go into the arena and they fight for what they 0:47:46.170 --> 0:47:50.049 believe in, that people like us should sit on the 0:47:50.049 --> 0:47:53.290 side and we should be cynical. We should assume the 0:47:53.290 --> 0:47:57.130 worst of them. We should attack them. And I think 0:47:57.130 --> 0:47:59.630 that is super wrong. I think it's super fucked up. 0:47:59.989 --> 0:48:01.910 But I don't assume the worst of Dario. I assume 0:48:01.910 --> 0:48:04.910 the best. Dario. Dario is terrible for it. 0:48:05.390 --> 0:48:07.910 No no no. You are assuming the worst of Dario 0:48:07.950 --> 0:48:10.469 that your video is assuming the worst of Dario. 0:48:10.710 --> 0:48:12.910 Which thing do you think that I believe about Dario. 0:48:12.950 --> 0:48:16.110 That's not true because I, I thought in my view 0:48:16.150 --> 0:48:20.070 that I expressed every opinion Dario has the way Dario 0:48:20.069 --> 0:48:23.750 has them, right. Like I, I tried to um you know, 0:48:23.790 --> 0:48:26.550 I'm not making any assumptions about his intention that is different. 0:48:26.549 --> 0:48:28.709 I tried to say this is his intention is what 0:48:28.710 --> 0:48:31.509 I really believe is true. If, if I don't think 0:48:31.509 --> 0:48:34.470 there's anything in that video from that perspective that Dario 0:48:34.509 --> 0:48:37.310 would say was wrong, at least besides the part where 0:48:37.310 --> 0:48:38.910 I call him shitty. Right? 0:48:39.190 --> 0:48:41.550 Um, no, no, but but what? You're what you're saying. 0:48:41.790 --> 0:48:42.670 I'm sorry. Go ahead. 0:48:43.110 --> 0:48:45.270 The next piece of that, though, is to say just 0:48:45.270 --> 0:48:47.709 because somebody is like you're saying, okay, they're out there, 0:48:47.710 --> 0:48:49.550 they're trying to make the world better. They're trying to, 0:48:49.589 --> 0:48:52.030 you know, like I said, there are people who've committed like, 0:48:52.190 --> 0:48:55.629 like genocide under those terms, right? They genuinely believe that like, 0:48:55.670 --> 0:48:58.620 you know, you know something, some group is after them 0:48:58.620 --> 0:49:01.220 or whatever. And then and they're out in the arena 0:49:01.259 --> 0:49:04.100 trying to tell people not to, you know, trust this 0:49:04.100 --> 0:49:06.739 ethnic minority group. And then people are critical of them. 0:49:06.739 --> 0:49:08.379 And we say that they, you know, you see, it 0:49:08.380 --> 0:49:11.140 doesn't it doesn't change anything. It's like it doesn't change 0:49:11.140 --> 0:49:14.339 the in conclusion, which is that like, in the end, 0:49:14.380 --> 0:49:18.660 what Dario wants is bad for all of us. And 0:49:18.660 --> 0:49:21.459 he wants it kind of knowing that he kind of 0:49:21.500 --> 0:49:23.660 knows it's bad for all of us because he doesn't 0:49:23.660 --> 0:49:25.939 believe it's as bad as what could happen, which would 0:49:25.940 --> 0:49:28.780 be like extinction or like genocide or things like that. Right. 0:49:28.819 --> 0:49:31.739 Okay. So it sounds like you're arguing against E. 0:49:32.540 --> 0:49:34.380 Oh, yeah. Oh, 100%. Yeah. 0:49:34.819 --> 0:49:37.380 Well, so, so what I haven't studied is how close 0:49:37.380 --> 0:49:40.900 he is to e my read on him is like 0:49:40.900 --> 0:49:43.980 he doesn't care at all about that shit. He literally 0:49:43.980 --> 0:49:47.660 is going in a different direction. Um, I hear, okay, 0:49:47.660 --> 0:49:51.219 let me just I'm going to steal your position. Um, 0:49:51.380 --> 0:49:54.019 he thinks he understands the world. He thinks he's trying 0:49:54.020 --> 0:49:56.609 to do the right thing and he's wrong in some 0:49:56.610 --> 0:49:59.810 sort of way, but he is mostly good and he 0:49:59.810 --> 0:50:03.250 just is like Jesus like syndrome, where it's like, I'm, 0:50:03.290 --> 0:50:06.010 I'm going to save the world or whatever, and he's wrong. 0:50:06.009 --> 0:50:08.890 And if too many people give him the power, um, 0:50:08.930 --> 0:50:10.969 he'll end up doing bad things for us because he 0:50:10.969 --> 0:50:13.290 doesn't want us to have all the toys or whatever. 0:50:13.569 --> 0:50:15.249 Not, not quite what I would say. I would say 0:50:15.250 --> 0:50:17.490 it's not just because that would not make him bad. 0:50:17.489 --> 0:50:19.370 That wouldn't make him a bad person. Okay, so being 0:50:19.370 --> 0:50:22.529 wrong doesn't make someone bad, right? Um, what it is, 0:50:22.529 --> 0:50:24.729 is that he is wrong in a specific way, which 0:50:24.730 --> 0:50:28.169 leads him to a conclusion which is, uh, on its 0:50:28.170 --> 0:50:31.169 own grounds, immoral, which is basically to say that he 0:50:31.170 --> 0:50:33.330 should be trusted with an amount of power that no 0:50:33.330 --> 0:50:35.649 one entrusted him with, and that he can seize that 0:50:35.650 --> 0:50:38.930 power for himself unilaterally. Okay. Because he happens to control 0:50:38.930 --> 0:50:41.090 AI because he happens to be the CEO of anthropic. 0:50:41.130 --> 0:50:42.489 But he's not doing that. 0:50:43.009 --> 0:50:45.690 But he is, because he's trying to maintain private control 0:50:45.690 --> 0:50:49.370 over AI while believing in this self recursive self improvement world. 0:50:49.370 --> 0:50:51.089 Like that's just not how I would play it if 0:50:51.089 --> 0:50:53.400 I were in his shoes. Now, of course, I don't 0:50:53.400 --> 0:50:55.799 believe any of this, so I would not I'm not 0:50:55.799 --> 0:50:58.160 asking Dario to hand over to the government. I'm asking 0:50:58.160 --> 0:51:01.359 Dario to just compete. Just play the game so we 0:51:01.360 --> 0:51:04.520 can keep this ball moving. You know, keep the game rolling. 0:51:04.839 --> 0:51:08.399 But he is competing and he's inviting collaboration from his 0:51:08.400 --> 0:51:10.880 competitors and inviting the government. 0:51:10.920 --> 0:51:13.879 So what I'm asking is like of people listening to this, 0:51:13.880 --> 0:51:16.960 what I want is for you to basically not vote 0:51:16.960 --> 0:51:20.279 for politicians that would believe in supporting the types of 0:51:20.279 --> 0:51:22.920 policies that Dario wants. Okay. I think we should not 0:51:22.920 --> 0:51:29.160 support Dario's policies are bad for most people. Uh, unless 0:51:29.200 --> 0:51:33.040 you believe in extinction risk. If you believe in extinction risk, okay, fine. 0:51:33.040 --> 0:51:34.560 We can play a different game. But if you don't 0:51:34.560 --> 0:51:36.040 believe in extinction risk, how could not? 0:51:36.080 --> 0:51:38.720 How could you not believe in extinction risk if most 0:51:38.759 --> 0:51:40.320 AI experts believe in it? 0:51:40.880 --> 0:51:43.520 Well, okay, so that's like there have been plenty of 0:51:43.520 --> 0:51:46.839 historical examples of like people believing, like groups of people 0:51:46.880 --> 0:51:48.640 believing things that aren't true. Right? 0:51:48.680 --> 0:51:50.439 No, no, but but, but I, I gave you an 0:51:50.440 --> 0:51:55.180 example of the wet lab and the the advancements of AI. 0:51:55.219 --> 0:51:57.980 So so let me just pitch this to you. Um, 0:51:57.980 --> 0:52:02.180 it will not be long before open source models and 0:52:02.180 --> 0:52:04.859 closed source models will be able to do that end 0:52:04.860 --> 0:52:08.060 to end and lab, uh, wet lab thing and the 0:52:08.100 --> 0:52:12.019 end to end cyber, um, zero days that that will 0:52:12.020 --> 0:52:14.140 not be long. Do you agree with that? 0:52:14.819 --> 0:52:17.060 No, no, no, no, I, I don't I don't necessarily 0:52:17.060 --> 0:52:19.900 have a strong opinion of course end in zero days. Yeah. Sure. 0:52:19.900 --> 0:52:22.660 That's not that's not that's already kind of there. Um okay. 0:52:22.739 --> 0:52:26.420 Do I believe that the extinction risk I mean zero 0:52:26.420 --> 0:52:29.100 day doesn't cause extinction risk by and large. Okay. 0:52:29.660 --> 0:52:32.460 Can I, can I have a quick technical sort of 0:52:32.500 --> 0:52:39.339 breakout here? Um, the, the reason that mythos advanced, in 0:52:39.339 --> 0:52:43.459 my opinion, based on being, uh, somewhat of an AI 0:52:43.500 --> 0:52:47.180 expert compared to a true like AI researcher, I'm basically 0:52:47.180 --> 0:52:51.730 a four year old, but, um, My understanding of this, okay, 0:52:51.770 --> 0:52:55.129 is that mythos and all the good models, the way 0:52:55.130 --> 0:52:58.290 the good models are getting better is because they're better 0:52:58.290 --> 0:53:02.290 at pursuing goals long term. There is no difference between 0:53:02.290 --> 0:53:06.370 cyber and building a wet lab when you generalize. This 0:53:06.370 --> 0:53:09.410 is all the same shit. And the the speed in 0:53:09.410 --> 0:53:12.530 which these are improving. Go ahead. 0:53:12.890 --> 0:53:14.769 But there's like other parts of this, like, of course, 0:53:14.770 --> 0:53:16.930 I am not a biology expert, but this is one 0:53:16.930 --> 0:53:19.049 of the problems is that like, neither are any of 0:53:19.049 --> 0:53:21.930 these AI experts either. And so what happens is, is 0:53:21.930 --> 0:53:23.770 that like, and I can give a really great example 0:53:23.770 --> 0:53:25.370 of this. When I was at the University of Hong Kong, 0:53:25.370 --> 0:53:26.890 one of the professors there happens to be one of 0:53:26.890 --> 0:53:30.690 the leading experts in nuclear non-proliferation. Uh, he was cited 0:53:30.690 --> 0:53:33.130 in the I c, J blah, blah, blah. Okay. And 0:53:33.130 --> 0:53:35.530 I remember I tried to get him involved in AI 0:53:35.569 --> 0:53:37.569 safety stuff because of course, it has a lot of 0:53:37.569 --> 0:53:40.770 parallels to nuclear non-proliferation. And he said that when he 0:53:40.770 --> 0:53:43.410 tried to discuss it with his AI, you know, people 0:53:43.410 --> 0:53:46.649 from the AI part of HKU, basically they said, you 0:53:46.650 --> 0:53:49.080 don't know what you're talking about on nuclear non-proliferation. So 0:53:49.080 --> 0:53:51.040 he said, okay, well, I'm now. Right. And so what 0:53:51.040 --> 0:53:53.439 happens is, is that like the AI safety people can 0:53:53.440 --> 0:53:55.919 genuinely convince themselves, they have logic out all of the 0:53:55.920 --> 0:53:58.239 way in which these wet labs might be created and stuff. 0:53:58.239 --> 0:54:01.239 But like AI experts are not an expert in wet 0:54:01.279 --> 0:54:05.160 lab creation and like biological weapons and things of that nature. 0:54:05.160 --> 0:54:07.080 We haven't had some sort of discourse on this. And 0:54:07.080 --> 0:54:09.200 then we also don't know what happens as a result. 0:54:09.239 --> 0:54:12.360 Like what if AI also produces some sort of like 0:54:12.400 --> 0:54:16.280 biology protection mechanism, right? Like we don't know. And so 0:54:16.319 --> 0:54:18.479 it's like, these are risks that I think are just like, 0:54:18.520 --> 0:54:21.799 very like, uh, they, they're kind of like, we don't 0:54:21.799 --> 0:54:25.000 really know that's true. And yet we're being asked to 0:54:25.040 --> 0:54:27.399 accept things now. And I think so we have to 0:54:27.440 --> 0:54:28.680 kind of ramp up. But like, I just want to 0:54:28.680 --> 0:54:31.799 say it from this angle, if we want to believe that, 0:54:31.839 --> 0:54:34.239 like if Dario really believes this, there's also this piece 0:54:34.239 --> 0:54:37.080 of advice I have, which is to say, currently AI 0:54:37.120 --> 0:54:41.600 safety ism is losing the political, um, war, right? Like 0:54:41.640 --> 0:54:44.680 I think that the position that I'm expressing in one 0:54:44.680 --> 0:54:46.950 reason that some people like yourself might get frustrated is 0:54:46.950 --> 0:54:51.549 because like, my argumentation is actually quite popular and it's 0:54:51.549 --> 0:54:54.750 quite easy for like people to accept and latch on to, 0:54:54.790 --> 0:54:58.069 whereas to accept the safety as an argument, you have 0:54:58.069 --> 0:54:59.430 to like kind of go a lot deeper and believe 0:54:59.469 --> 0:55:03.189 a bunch of things. If you believe the safety ism position, 0:55:03.190 --> 0:55:05.749 you need to switch the political framing of it to 0:55:05.790 --> 0:55:09.950 give people something positive other than just, we'll protect you 0:55:09.950 --> 0:55:11.790 from the risk that, you know, you don't believe in 0:55:11.830 --> 0:55:12.710 in the first place. 0:55:12.750 --> 0:55:13.350 I hear you. 0:55:13.390 --> 0:55:15.630 Right now, they're not doing that right now. They're basically 0:55:15.630 --> 0:55:19.109 saying you give up everything for your own safety so 0:55:19.110 --> 0:55:20.870 that we can control you. No one's gonna accept that. 0:55:20.870 --> 0:55:22.470 It's just not going to work. So, like, their plan 0:55:22.469 --> 0:55:23.390 doesn't work. 0:55:23.589 --> 0:55:28.630 Yeah. So so the thing is, Zach. Yes, your argument 0:55:28.630 --> 0:55:32.310 is very popular. Your argument is basically you see these 0:55:32.310 --> 0:55:36.029 guys like they're, oh, man, they're just making all this 0:55:36.029 --> 0:55:39.029 stuff and they're trying to control us. And, um, they've 0:55:39.029 --> 0:55:40.709 got all the best stuff and we should have all 0:55:40.710 --> 0:55:44.580 the best stuff too. And Dario was like, guys, this 0:55:44.580 --> 0:55:47.459 is fucking bad. This is really fucking bad. We have 0:55:47.460 --> 0:55:50.180 to be very careful here. Let's get the government involved. 0:55:50.180 --> 0:55:53.060 Let's get everyone involved. We need to regulate this. This 0:55:53.060 --> 0:55:54.980 is like I'm still going to do it. Like I 0:55:54.980 --> 0:55:57.060 still have a company and I still think I have 0:55:57.060 --> 0:55:59.539 a good path going forward. But we need to be 0:55:59.540 --> 0:56:02.500 super careful about this. They were the first people to 0:56:02.540 --> 0:56:06.819 release their own dirty laundry. The guy has consistently been 0:56:06.819 --> 0:56:10.660 releasing all, all the sort of flaws in their shit. 0:56:10.660 --> 0:56:14.739 And describing this consistently put his company on the line 0:56:14.739 --> 0:56:19.780 multiple times, um, financially to, to stand up for, for 0:56:19.779 --> 0:56:24.259 ethical red lines and ultimately, like the stuff is going 0:56:24.259 --> 0:56:27.100 to happen and he is not trying to say is 0:56:27.140 --> 0:56:30.580 so this, I disagree with you. He's trying to produce 0:56:30.580 --> 0:56:33.180 some negative thing in the future. What he's trying to 0:56:33.180 --> 0:56:37.380 do is saying this is bad, it's happening. Everyone's going there. 0:56:37.380 --> 0:56:40.700 It's going to happen no matter what. Let's be careful 0:56:40.700 --> 0:56:44.240 and release slowly and make sure that the US wins 0:56:44.239 --> 0:56:46.880 before CCP. That's the argument. 0:56:47.000 --> 0:56:49.279 But also, when we say the US wins, like let's 0:56:49.279 --> 0:56:51.360 keep in mind this is not the, you know, unlike 0:56:51.360 --> 0:56:53.919 the CCP, which is like, okay, obviously can control any 0:56:53.920 --> 0:56:57.319 Chinese company. Like there were people were already suggesting like, 0:56:57.319 --> 0:56:59.920 what if anthropic just moved to Europe? Right? It's not like, 0:56:59.960 --> 0:57:02.000 you know, also at another example, it's like when the 0:57:02.000 --> 0:57:04.000 Department of War said, we want to use your models 0:57:04.000 --> 0:57:05.919 in this way. Dario kind of refused. So it's not 0:57:05.920 --> 0:57:09.040 like it's controlled by the US government to the same degree. 0:57:09.040 --> 0:57:11.680 So saying America wins is a bit unfair because like, 0:57:11.680 --> 0:57:14.239 it's not an American model. It's just a model made 0:57:14.239 --> 0:57:16.880 by an American company, which of course is like a 0:57:16.880 --> 0:57:19.560 fairly floating concept. I mean, you could, you know, but. 0:57:20.200 --> 0:57:23.640 Let's make an American consortium to try to do this. 0:57:23.640 --> 0:57:24.840 So we win before China. 0:57:24.880 --> 0:57:27.120 Yes. But he's not giving. So this is the issue. 0:57:27.160 --> 0:57:30.280 He's not giving anything. He's asking us to believe in 0:57:30.280 --> 0:57:32.600 all those risks, which I don't believe in. And he's 0:57:32.640 --> 0:57:35.800 asking us to believe in that and do things that 0:57:35.800 --> 0:57:38.479 will hurt us, like all of the people who are 0:57:38.480 --> 0:57:42.750 not him. Right. And his group. Uh, he's not giving 0:57:42.750 --> 0:57:46.030 us anything. And so you sit there going, ah, I'd 0:57:46.030 --> 0:57:49.669 rather the Chinese model than believing that this risk, you know, 0:57:49.710 --> 0:57:51.709 this is how most. I mean, isn't this like, I've 0:57:51.710 --> 0:57:54.310 not watched this Star Wars thing, but isn't this the 0:57:54.310 --> 0:57:57.470 whole like the Chancellor took over to protect the people 0:57:57.470 --> 0:58:00.550 from some made up risk or some shit like this 0:58:00.550 --> 0:58:02.910 happens all the time. Like Napoleon did it. Exactly right. 0:58:03.030 --> 0:58:05.030 That's a common story, Zach. 0:58:05.270 --> 0:58:08.870 The the emperor was an evil person. Like just straight 0:58:08.870 --> 0:58:11.550 up pure evil. Like wanted to do evil things for 0:58:11.550 --> 0:58:12.350 evil people. 0:58:12.710 --> 0:58:14.669 You met Sam Altman, like, I mean, come on. Like, 0:58:14.670 --> 0:58:15.950 people talk about this all the time. 0:58:16.190 --> 0:58:19.030 That's a different talk show. Sam Altman is not Dario. 0:58:19.070 --> 0:58:20.070 They are the opposite. 0:58:20.550 --> 0:58:22.990 I to some degree. Yeah, sure. I concede that. Of 0:58:22.990 --> 0:58:24.670 course I believe that there's a difference there. 0:58:24.670 --> 0:58:28.350 Um, so so here's, here's my thing, Zach. I think 0:58:28.830 --> 0:58:32.590 the situation is I believe Dario is a moral actor. 0:58:32.590 --> 0:58:35.110 I don't think it's perfect. Like I'm super pissed about 0:58:35.110 --> 0:58:37.910 my my prices. I wish I had more stuff in 0:58:37.910 --> 0:58:41.380 my subscription. Like I've been pissed off at anthropic plenty 0:58:41.380 --> 0:58:45.340 of times. The issue is, you're not explaining to me 0:58:45.820 --> 0:58:50.980 how a moral actor which you agree that that Dario is, 0:58:50.980 --> 0:58:54.460 he's behaving as a moral actor. You think he's ultimately 0:58:54.460 --> 0:58:57.540 going to lead somehow to an immoral thing? I'm saying 0:58:57.540 --> 0:59:00.540 I can't see any difference between how I would behave 0:59:00.780 --> 0:59:04.100 and how Dario would behave, given my model of the world. 0:59:04.140 --> 0:59:06.140 Yeah, but I would hope that if you were in 0:59:06.140 --> 0:59:09.580 Dario's position and actually sitting there, you would turn around 0:59:09.580 --> 0:59:12.980 and go, oh, God, and believe the things Dario believes. 0:59:12.980 --> 0:59:15.940 You would go, oh, God. Actually, probably it shouldn't be 0:59:15.980 --> 0:59:20.620 Daniel deciding it should be the government. Okay. But importantly, 0:59:20.940 --> 0:59:24.780 which he referred to as communism. Uh, but importantly, I 0:59:24.820 --> 0:59:27.860 don't believe any of these things. And so therefore, I 0:59:27.900 --> 0:59:30.140 don't believe in a world where Americans are made worse 0:59:30.180 --> 0:59:31.940 off or not just Americans like the whole, you know, 0:59:31.980 --> 0:59:34.500 I live in Norway. I don't want my Norwegian family 0:59:34.500 --> 0:59:37.260 made worse off. I don't like I don't want anyone 0:59:37.260 --> 0:59:39.770 made worse off to protect from a risk that I 0:59:39.810 --> 0:59:42.770 have not seen playing out or seen proven in any 0:59:42.770 --> 0:59:45.410 meaningful sense. Well, I do not think we should be 0:59:45.410 --> 0:59:47.770 willing to accept that trade off. And if that is 0:59:47.770 --> 0:59:49.970 the trade off that we're being asked to accept, how 0:59:49.970 --> 0:59:52.290 about if Dario believes this is the end of the world? 0:59:52.290 --> 0:59:54.930 He makes some concessions on his side to give us 0:59:54.930 --> 0:59:57.130 things back, as opposed to just saying we'll block the 0:59:57.130 --> 0:59:59.690 model that would do, you know, give you cheaper prices 0:59:59.690 --> 1:00:02.250 and things of that nature and not just prices, but 1:00:02.250 --> 1:00:05.249 also better capabilities. We will block that one and you 1:00:05.250 --> 1:00:07.730 get nothing. I mean, he's got to come if he 1:00:07.730 --> 1:00:09.689 wants to save the world. That's a trade he's got 1:00:09.690 --> 1:00:12.130 to be willing to make, surely. But he's not doing it. 1:00:12.450 --> 1:00:14.610 Dude, it's only been a few months. What are you 1:00:14.610 --> 1:00:19.730 talking about? We're talking. We're talking about. Okay, potentially. Okay. 1:00:19.770 --> 1:00:21.970 This is we just disagree on this. I just think 1:00:22.010 --> 1:00:25.010 you're wrong about the actual threats to the world. But 1:00:25.210 --> 1:00:28.570 I'm going to ask you to empathize and just accept 1:00:28.570 --> 1:00:31.010 it for the sake of argument, right? For the sake 1:00:31.010 --> 1:00:34.490 of argument, if we are correct, me and Dario and 1:00:34.530 --> 1:00:37.709 these other AI experts or whatever. If we are right 1:00:37.710 --> 1:00:41.230 about that, then he is behaving as a as a 1:00:41.230 --> 1:00:41.870 good actor. 1:00:42.070 --> 1:00:44.590 No, you should if you. That's my point is that 1:00:44.590 --> 1:00:46.590 if he if I was sitting in his shoes and 1:00:46.590 --> 1:00:47.710 I believed that thing. 1:00:47.750 --> 1:00:47.910 What. 1:00:47.910 --> 1:00:50.829 Would you do? He. I would basically say this needs 1:00:50.830 --> 1:00:52.790 to be nationalized because this is not something that should 1:00:52.790 --> 1:00:53.590 be privately owned. 1:00:55.430 --> 1:01:01.110 Right. But nationalization doesn't. Nationalization doesn't solve anything. It doesn't 1:01:01.110 --> 1:01:01.990 solve anything. 1:01:02.030 --> 1:01:04.190 That's. But that's the point. That's not for us to decide. 1:01:04.190 --> 1:01:06.510 That's that's that's my point is that that's the thing. 1:01:06.550 --> 1:01:08.350 It is up to us to decide. 1:01:08.390 --> 1:01:10.709 If not us in the sense of Dario. I mean, 1:01:10.710 --> 1:01:12.390 it's up for us in the sense of like American 1:01:12.390 --> 1:01:14.470 people who can vote and stuff like that. But it's 1:01:14.470 --> 1:01:17.390 not us up to Dario to be making that call. 1:01:17.430 --> 1:01:17.870 And so. 1:01:18.230 --> 1:01:18.709 You have. 1:01:18.710 --> 1:01:18.950 To. 1:01:19.710 --> 1:01:22.269 You have to look at the actual thing that would 1:01:22.270 --> 1:01:25.270 happen if he did that, if he walked off the. 1:01:25.270 --> 1:01:27.830 Field. That's not up to him. That's not that's my 1:01:27.830 --> 1:01:29.310 point is that at the end of the day, there's 1:01:29.310 --> 1:01:31.669 a public system that needs to be played. He is making. 1:01:31.710 --> 1:01:33.990 That's my point is he is he is acting like 1:01:33.990 --> 1:01:37.500 somebody picked him to make that call and no one did. Okay. 1:01:37.540 --> 1:01:40.500 If you found out tomorrow that a company in Alabama 1:01:40.500 --> 1:01:43.780 was making nuclear weapons in their garage, right? You would 1:01:43.780 --> 1:01:47.660 immediately say, go see, like, send the troops in. Cease 1:01:47.660 --> 1:01:48.900 that shit, Zach. 1:01:48.900 --> 1:01:51.820 All of us are are supposed to be making that call. 1:01:51.980 --> 1:01:54.100 Every single one of us starting with just happens to 1:01:54.100 --> 1:01:54.820 have a company. 1:01:55.180 --> 1:01:56.459 No, but as a it's in a. 1:01:56.900 --> 1:01:59.980 Different than me. I would make the same decision. Anyone 1:01:59.980 --> 1:02:02.180 in that position should make the same decision. 1:02:02.460 --> 1:02:04.260 No, you should make the decision to hand over to 1:02:04.260 --> 1:02:05.780 the public system. That's like my position. 1:02:05.820 --> 1:02:08.180 And just leave and not participate anymore. 1:02:08.340 --> 1:02:10.900 Participate? Sure. But like it's not up to you to 1:02:10.940 --> 1:02:13.580 decide it then. So like to basically say this becomes 1:02:13.580 --> 1:02:16.540 a thing that is controlled entirely by the government. Okay. 1:02:16.540 --> 1:02:18.340 You can still go to work every day, help make 1:02:18.340 --> 1:02:19.500 things safer, whatever you want to. Do. 1:02:19.540 --> 1:02:23.459 You think if he thought things were this bad, he 1:02:23.460 --> 1:02:26.780 should completely give it to the government and just hand 1:02:26.780 --> 1:02:27.740 over full control? 1:02:27.740 --> 1:02:29.380 The same we would accept. But this is the same 1:02:29.380 --> 1:02:31.300 thing we would accept if someone was making nuclear weapons, 1:02:31.300 --> 1:02:33.130 which is what he compares it to. If you found 1:02:33.130 --> 1:02:35.330 out that a company in Alabama was making nuclear weapons 1:02:35.330 --> 1:02:38.570 in their garage, you would say, this is unacceptable and 1:02:38.570 --> 1:02:40.370 we should go take it. Of course, like you would 1:02:40.410 --> 1:02:42.530 believe that. I don't think anyone would think. 1:02:42.570 --> 1:02:45.970 No, but but but tactically but tactically the way it 1:02:45.970 --> 1:02:46.530 would work. 1:02:47.010 --> 1:02:49.490 That's the problem. The tactics are not up for us. Right. 1:02:49.490 --> 1:02:51.490 That's that's that's something that should be handed to a public. 1:02:51.650 --> 1:02:56.970 But that nationalization in that model, why wouldn't it be? Okay. Okay. 1:02:56.970 --> 1:02:58.490 So let's say I'm in charge of the government now. 1:02:58.490 --> 1:03:02.250 I'm in charge of the nationalization program. My first step. 1:03:02.650 --> 1:03:05.970 Get all these AI companies in here. All the US 1:03:06.010 --> 1:03:09.209 American AI companies. Get them all in here. This is 1:03:09.210 --> 1:03:13.530 now nationalized. We're using these resources of these companies to 1:03:13.570 --> 1:03:14.410 move forward. 1:03:14.450 --> 1:03:16.410 Yeah. That's of course what you would. That's how nationalization 1:03:16.410 --> 1:03:17.810 would work of course. But it's. 1:03:17.850 --> 1:03:21.330 Very. How different is that from what he is proposing? 1:03:21.330 --> 1:03:22.650 I'm arguing it's private. 1:03:22.650 --> 1:03:26.410 Ownership that he's going to IPO. Like who cares about ownership? 1:03:26.450 --> 1:03:28.970 Who cares? Who cares about shares and ownership debate? 1:03:28.970 --> 1:03:30.970 That's the whole debate. It's like, should you be able 1:03:30.970 --> 1:03:33.480 to build nuclear weapons and have private ownership? 1:03:33.680 --> 1:03:37.160 That's not the debate. The debate is whether or not 1:03:37.200 --> 1:03:39.680 the government is involved in controlling. 1:03:39.880 --> 1:03:42.439 I don't want the government, and I don't want the 1:03:42.480 --> 1:03:46.160 FDA for nuclear weapons. Right. Like, it's like it's a silly. 1:03:46.480 --> 1:03:49.080 You're asking for complete nationalization. 1:03:49.120 --> 1:03:51.479 No I'm not. I'm asking for competition. It's just if 1:03:51.480 --> 1:03:53.440 you believe the crazy things Dario believes, I would be 1:03:53.440 --> 1:03:57.280 asking for nationalization. I believe that people should be that 1:03:57.280 --> 1:03:59.360 all of these models need to compete so that we 1:03:59.400 --> 1:04:01.680 don't run into a world where all of. I think 1:04:01.680 --> 1:04:04.440 the biggest risk for AI is that it gets controlled 1:04:04.440 --> 1:04:07.160 by a few small companies, and that's why I want 1:04:07.200 --> 1:04:10.400 everyone to be competing openly. I want us to make 1:04:10.400 --> 1:04:12.959 it so that distillation is like a right. You get 1:04:12.960 --> 1:04:15.800 to distill models, okay? Because I want there to be 1:04:15.800 --> 1:04:18.640 a ton of labs all competing in parallel. And that 1:04:18.640 --> 1:04:23.040 is because I don't believe in like the existential risk here. Okay. 1:04:23.080 --> 1:04:27.360 I believe the biggest risk is Dario's company becoming king, right? 1:04:27.400 --> 1:04:30.140 That's horrible for society. It would be horrible to have 1:04:30.140 --> 1:04:33.780 $150 trillion company and everyone else be unemployed. That's what 1:04:33.780 --> 1:04:34.340 I don't want. 1:04:34.380 --> 1:04:36.820 If that were he were trying to do, he would 1:04:36.820 --> 1:04:37.820 not have to. 1:04:37.860 --> 1:04:39.340 It's just that could happen. 1:04:39.540 --> 1:04:43.940 No, but I'm saying he's explicitly like doing the opposite. 1:04:45.100 --> 1:04:48.220 No, I mean, his company's gone from whatever to one 1:04:48.220 --> 1:04:49.580 point whatever trillion. I mean. 1:04:49.620 --> 1:04:52.459 No, but he's the one that started glasswing. He's the 1:04:52.460 --> 1:04:53.419 one that did all of this. 1:04:53.460 --> 1:04:55.260 But yeah, but glass, this was the other point I 1:04:55.260 --> 1:04:57.499 forgot to make, which is like Glasswing didn't even have 1:04:57.500 --> 1:04:59.820 any app set companies in it. Why didn't they give 1:04:59.820 --> 1:05:01.020 it to them is because they're going to try to 1:05:01.020 --> 1:05:03.540 compete with apps. So obviously that was like there were 1:05:03.540 --> 1:05:06.939 aspects of Glasswing that were not purely altruistic, right? Like, 1:05:06.940 --> 1:05:08.700 why would you have not given it to an application 1:05:08.700 --> 1:05:11.500 security company that has more access to code than they do? 1:05:11.540 --> 1:05:13.100 I mean, just because they don't want to compete. 1:05:13.140 --> 1:05:15.740 They gave it to the people with the actual infrastructure 1:05:15.780 --> 1:05:16.820 that they were worried about. 1:05:16.860 --> 1:05:20.580 There are plenty, like there are plenty of application security 1:05:20.580 --> 1:05:24.180 companies that would have the ability to use mythos to 1:05:24.220 --> 1:05:27.140 make a large lot of code safer. Right. You could 1:05:27.180 --> 1:05:28.770 have done that. They chose not to. 1:05:29.490 --> 1:05:33.530 Yeah. I mean, that's an implementation thing, right? But I mean. 1:05:34.530 --> 1:05:36.370 But it's an implement. It's kind of the point here 1:05:36.370 --> 1:05:38.570 in the comments that Glasswing is like becomes when it's 1:05:38.570 --> 1:05:43.410 an implementation that doesn't reflect the full details of safety ism, 1:05:43.410 --> 1:05:45.530 you start to go like, it's not really that wasn't 1:05:45.530 --> 1:05:49.530 Glasswing was not really a safety thing. Okay. Glasswing large. 1:05:49.570 --> 1:05:53.010 How could it be anything? Okay. Okay. If someone who 1:05:53.010 --> 1:05:57.930 believes they're behaving morally comes up and says, hey, I 1:05:57.930 --> 1:06:00.530 don't want to release this immediately. I need to give 1:06:00.530 --> 1:06:04.090 it to people who have infrastructure so they can defend. Yes. Like, 1:06:04.130 --> 1:06:05.970 how is that not a moral action? 1:06:06.170 --> 1:06:09.370 That part's normal. That's normal. Okay. And then what happens? 1:06:09.370 --> 1:06:11.450 They go, I need to give it to these people. Um, 1:06:11.490 --> 1:06:13.610 can you come up with a list? And then the 1:06:13.610 --> 1:06:15.530 marketing department comes up with a list and they say, oh, 1:06:15.530 --> 1:06:18.050 we should probably not include the appsec people, because. 1:06:18.330 --> 1:06:20.530 Now you're back to assuming the worst of Sam or 1:06:20.530 --> 1:06:21.570 of Dario. 1:06:21.610 --> 1:06:23.090 No, no, no, Dario is not even in the room 1:06:23.090 --> 1:06:25.689 for that discussion. Right. It's like that's what happens because 1:06:25.690 --> 1:06:28.840 it's consistent with safety ism. He's not released it to 1:06:28.840 --> 1:06:30.640 the world. So Dario's not going to have a problem 1:06:30.640 --> 1:06:33.280 with this, right? But like, so that's the one thing 1:06:33.280 --> 1:06:35.960 to understand is that I argue that Dario genuinely believes 1:06:35.960 --> 1:06:38.320 in safety, blah, blah, blah, but it doesn't mean that 1:06:38.320 --> 1:06:42.519 every action Dario takes is only about safety. It just 1:06:42.520 --> 1:06:46.519 means it's consistent with safety. So Glasswing is consistent with 1:06:46.520 --> 1:06:49.240 safety because he didn't release it to the world. Right. 1:06:49.280 --> 1:06:54.040 But it is not only about safety, okay? There was 1:06:54.040 --> 1:06:56.800 choices made in there that were not about safety, the 1:06:56.800 --> 1:06:58.080 choice to use fire. 1:06:58.120 --> 1:07:00.919 Well, he's also running. He's also running a company. He's 1:07:00.920 --> 1:07:04.760 also trying to make money. Sure, sure, sure. Who else 1:07:04.760 --> 1:07:06.680 would be in the position to do this other than 1:07:06.720 --> 1:07:09.919 someone who has the ability to start a company and 1:07:09.920 --> 1:07:10.360 run one? 1:07:10.400 --> 1:07:13.080 Say it to damn him. I just said it to say, 1:07:13.120 --> 1:07:14.479 you know, when we look at it and say yes, 1:07:14.480 --> 1:07:16.200 but he did the right thing with Glasswing like I 1:07:16.200 --> 1:07:19.400 think he did like a thing with Glasswing, right? He 1:07:19.400 --> 1:07:21.320 did like the 5050 of it. It's like, no, I'm 1:07:21.320 --> 1:07:22.760 not going to give him a huge amount of credit 1:07:22.760 --> 1:07:26.590 for Glasswing. Right. I'm not going to sit there and 1:07:26.590 --> 1:07:29.750 go like, round of applause for giving it to CrowdStrike, 1:07:29.750 --> 1:07:30.710 but not to, you. 1:07:30.710 --> 1:07:31.789 Know, I'm just saying it's. 1:07:31.790 --> 1:07:33.350 Extreme. Give it to you. 1:07:33.350 --> 1:07:37.510 Know, I'm just saying it's extremely consistent with the view 1:07:37.550 --> 1:07:38.750 that I've presented here. 1:07:38.790 --> 1:07:40.390 Of course. And I, I agree, and this is one 1:07:40.390 --> 1:07:42.310 of the things is that you can view almost everything 1:07:42.350 --> 1:07:45.830 anthropic does is consistent with safety ism, right? Uh, and 1:07:45.830 --> 1:07:48.749 I think that, that, that's one reason why I have 1:07:48.750 --> 1:07:51.470 come to believe between that and Dario's, the fact that 1:07:51.470 --> 1:07:54.510 Dario does much like the Chinese do, writes everything he 1:07:54.510 --> 1:07:57.630 thinks and plans down in long winded essays that he 1:07:57.630 --> 1:07:59.350 thinks no one will read. And then you read them. 1:07:59.350 --> 1:08:01.110 It's like, you know, there are. When I prepared for 1:08:01.110 --> 1:08:04.269 the video, I mean, I probably read like 20,000 words 1:08:04.270 --> 1:08:05.670 written by the guy, right? 1:08:05.750 --> 1:08:07.269 Uh, he's read all that stuff. 1:08:07.470 --> 1:08:10.950 He's prolific at writing these things. Okay. Um, yeah. You know, 1:08:10.990 --> 1:08:13.189 and this is why I believe that Dario believes all this, 1:08:13.190 --> 1:08:16.350 but what Dario, there is a part of AI safety ism, 1:08:16.350 --> 1:08:18.070 which is that like, you know, if you go back 1:08:18.070 --> 1:08:21.870 to the origins of like, if you think like Nick Bostrom, like, um, superintelligence, 1:08:21.870 --> 1:08:24.689 which would be like 2015 book, right. Uh, which would 1:08:24.730 --> 1:08:27.010 have been basically around the time when yod would have 1:08:27.010 --> 1:08:29.850 been writing all this stuff. You know, basically one of 1:08:29.850 --> 1:08:34.850 the positions that's argued is like that AI is aligned, 1:08:34.890 --> 1:08:38.769 fully aligned with our goals. But, uh, given that it's aligned, 1:08:38.810 --> 1:08:42.930 whoever controls it has like immense power, right? And this 1:08:42.930 --> 1:08:44.570 is kind of your concern of like, what if the 1:08:44.570 --> 1:08:46.730 CCP had it right? That'd be like really bad, right? 1:08:46.770 --> 1:08:49.130 If like, if like the Chinese Communist Party has the ability, 1:08:49.170 --> 1:08:51.729 has this recursive self-improving thing, and then you get there first. 1:08:51.730 --> 1:08:55.050 And Dario believes in this world where like whoever, if, 1:08:55.090 --> 1:08:56.930 if we can control it, which I think he thinks 1:08:56.930 --> 1:08:59.250 is a little bit unclear, but as he assumes is like, 1:08:59.290 --> 1:09:01.930 let's just assume it's true because there's some probability where 1:09:01.930 --> 1:09:04.810 it's true. Um, you know, there was this whole wing 1:09:04.850 --> 1:09:07.810 of safety ism, which is like, but then what if 1:09:07.810 --> 1:09:10.970 the company who's in charge of it decides to do 1:09:10.970 --> 1:09:14.450 something evil with it? Right? Okay. And, and like, that's 1:09:14.450 --> 1:09:16.890 a part of this, which is like, seriously, like, if 1:09:16.930 --> 1:09:19.290 you believe in this recursive self improvement plan where like 1:09:19.290 --> 1:09:22.240 suddenly whoever gets the best models will be permanently ahead. 1:09:22.280 --> 1:09:24.760 You have to be concerned about what the company who 1:09:24.800 --> 1:09:27.120 controls it would do with it, right? 1:09:27.640 --> 1:09:30.320 So here's my argument to you. So if you've read 1:09:30.320 --> 1:09:33.760 all his essays and stuff like that, what's clear to 1:09:33.760 --> 1:09:36.560 me from his essays and you can't always know the 1:09:36.560 --> 1:09:39.760 mind of someone, right? My, in my view, what Dario 1:09:39.800 --> 1:09:43.960 would want the absolute most, okay, is we get to 1:09:44.000 --> 1:09:47.400 the other side of this risk situation. Either it burns down, 1:09:47.400 --> 1:09:50.040 we build it back up, or we mitigate it. Somehow. 1:09:50.280 --> 1:09:53.800 We get to the other side. It. We have a 1:09:54.280 --> 1:09:57.760 we have ACI. We're now building up humanity like all 1:09:57.760 --> 1:10:00.760 the good things are happening or whatever. There's still companies. 1:10:00.760 --> 1:10:03.479 He's still one of the companies. He's making tons of money. 1:10:03.920 --> 1:10:07.120 There's tons of open source models and they're doing great 1:10:07.120 --> 1:10:10.960 to everyone has tons of AI and he's still making 1:10:10.960 --> 1:10:14.600 money and the company is great and thriving. That is 1:10:14.600 --> 1:10:17.840 from his essays, the thing he's trying to build. 1:10:18.000 --> 1:10:20.070 I have never once heard him say that he wants 1:10:20.070 --> 1:10:21.910 there to be open source models. If there's a quote 1:10:21.910 --> 1:10:24.390 on that, like send it to me because I haven't 1:10:24.430 --> 1:10:24.870 heard him say it. 1:10:24.910 --> 1:10:28.630 Yeah. I mean, there's no indication to me that he 1:10:28.630 --> 1:10:32.430 doesn't want open source models other than he doesn't want 1:10:32.790 --> 1:10:36.470 the most dangerous stuff to go into open source models immediately, 1:10:36.470 --> 1:10:38.670 because that would be bad for the reasons we talked about. 1:10:38.710 --> 1:10:41.710 No, because he's talked to pretty extensively about how he 1:10:41.710 --> 1:10:44.150 believes one of the problems with open weight models is 1:10:44.150 --> 1:10:47.390 that you can remove any guardrails that might exist. So 1:10:47.430 --> 1:10:49.670 I guess like so from that perspective, it'd be hard 1:10:49.670 --> 1:10:51.710 for him to argue for open weight models because then 1:10:51.710 --> 1:10:53.070 you could just remove the guardrails. 1:10:53.870 --> 1:10:56.270 Right? Well, I mean, that's saying the same thing I'm saying, 1:10:56.870 --> 1:11:01.070 assuming that that we're something we could potentially solve. He's not. 1:11:01.070 --> 1:11:04.150 In other words, he's not trying to limit the intelligence 1:11:04.150 --> 1:11:07.510 that people have access to other than for the reason 1:11:07.630 --> 1:11:10.510 of this risk thing. So he's not trying to get 1:11:10.870 --> 1:11:12.750 he's not trying to go to a place. He's not 1:11:12.750 --> 1:11:15.430 trying to go to a place where him and his 1:11:15.430 --> 1:11:19.060 rich buddies control intelligence and people have to go through him. 1:11:19.060 --> 1:11:20.620 That's not what he's trying to do. 1:11:20.660 --> 1:11:22.060 No, I think he is. I think he's doing it 1:11:22.060 --> 1:11:24.900 for safety reasons, though, like he very much would love 1:11:24.940 --> 1:11:27.260 if I think if you told Dario tomorrow that like 1:11:27.620 --> 1:11:31.980 all AI goes through anthropic, right? Um, he would say 1:11:31.980 --> 1:11:34.300 that's great because it allows him to ensure that everyone 1:11:34.300 --> 1:11:34.820 is safe. 1:11:35.580 --> 1:11:37.020 No, but what I'm saying. 1:11:37.060 --> 1:11:39.180 Is surely you agree he would say that that that 1:11:39.180 --> 1:11:41.260 would be good because it would allow him to keep 1:11:41.260 --> 1:11:42.060 everyone safe. 1:11:43.460 --> 1:11:45.940 Yeah, I don't know. I think he would probably like 1:11:45.980 --> 1:11:48.660 other people trying the same thing at the same time. Yeah, 1:11:48.660 --> 1:11:49.100 I agree. 1:11:49.300 --> 1:11:50.460 They would probably. Yes. 1:11:50.580 --> 1:11:54.540 Yeah. Yeah. Agreed. Agreed. So what what I would challenge 1:11:54.540 --> 1:11:57.620 you on is I think um, and maybe we could 1:11:57.620 --> 1:12:00.620 find quotes. Maybe we could use AI to do this. Um, 1:12:00.620 --> 1:12:03.820 I think you're wrong about the, the final thing he's 1:12:03.820 --> 1:12:06.180 trying to get to, I think he wants there to 1:12:06.220 --> 1:12:11.300 be open models, lots of competition. He's not the only company. 1:12:11.340 --> 1:12:14.860 Everyone's safe, everyone's fine. And everyone has access to tons 1:12:14.860 --> 1:12:17.480 of intelligence, one of which is him. 1:12:17.920 --> 1:12:19.560 Yeah, but I'm not gonna. I'm not going to let 1:12:19.560 --> 1:12:22.960 us play out this world where, like, we have to 1:12:23.000 --> 1:12:26.320 first hand over all this power to these guys, and 1:12:26.320 --> 1:12:28.600 then one day they will cede it back and make 1:12:28.640 --> 1:12:31.120 sure that everyone lives in harmony. Like, I think it's 1:12:31.120 --> 1:12:33.760 just the best course of action. And I think, sorry, 1:12:33.760 --> 1:12:35.240 I probably have to wrap up, but like, and I'll 1:12:35.240 --> 1:12:37.280 let you say something after this. But the best course 1:12:37.280 --> 1:12:42.000 of action is fundamentally, everyone's competing for building the best 1:12:42.040 --> 1:12:44.880 AI models so that that way, like we don't end 1:12:44.880 --> 1:12:46.600 up in a world where it's controlled by one or 1:12:46.600 --> 1:12:49.600 2 or 3 people that maybe we shouldn't have in control, right? 1:12:49.640 --> 1:12:53.360 Let them all compete, let everyone have access. And then like, 1:12:53.400 --> 1:12:55.720 if it turns out that there's some sort of like, 1:12:55.880 --> 1:12:57.800 if when we get closer to some of these really 1:12:57.800 --> 1:13:01.040 tangible risks, like, okay, like let's have a discussion maybe, 1:13:01.040 --> 1:13:02.920 but like right now, I'm not, I'm not willing to 1:13:03.680 --> 1:13:06.680 cave on any of this until I, until there's like 1:13:06.680 --> 1:13:09.040 some sort of reason to, why would we hand over 1:13:09.040 --> 1:13:12.960 to the, you know, Napoleon, just so that about some fear, 1:13:12.960 --> 1:13:14.080 we don't know if it's true. 1:13:15.030 --> 1:13:19.510 We are extremely close. It is happening. Look, Dario just 1:13:19.510 --> 1:13:22.510 got surprised by this. Anthropic just got surprised by this. 1:13:22.830 --> 1:13:24.750 This is happening a lot faster than I think. You 1:13:24.750 --> 1:13:29.870 think it's happening. Which would explain his behavior. So, um. Yeah, 1:13:29.870 --> 1:13:34.550 I would just say, uh, great conversation. Uh, I'm glad 1:13:34.550 --> 1:13:38.030 we did it. And, uh, I think this really does 1:13:38.070 --> 1:13:41.510 illuminate the differences, which, like I said before, we're not 1:13:41.510 --> 1:13:44.150 disagreeing on what we're trying to achieve. I think it's 1:13:44.150 --> 1:13:47.630 just this, uh, this middle path that we're disagreeing on. 1:13:48.150 --> 1:13:50.710 Yeah. No, I think this was good. Um, I think obviously, 1:13:50.710 --> 1:13:53.910 I think anyone listening probably now can form. I think 1:13:53.910 --> 1:13:56.350 there's very little left to ambiguity. Like people can now 1:13:56.390 --> 1:13:58.230 decide which, which way they think. 1:13:58.270 --> 1:13:59.310 Well, that's a success. 1:13:59.590 --> 1:14:03.270 Yeah, exactly. So thanks for doing this. Um, yeah, totally. 1:14:03.790 --> 1:14:05.750 I will then just stop the stream. I guess I 1:14:05.750 --> 1:14:07.790 will leave, you'll be on for one second. So okay, 1:14:07.790 --> 1:14:10.470 for everyone listening, see you guys. Uh, I'll probably be 1:14:10.470 --> 1:14:13.350 on Twitter, so I'll probably post some stuff. So, uh. 1:14:13.350 --> 1:14:13.910 See y'all.