WEBVTT - Mustafa Suleyman: AI Should Serve Humanity, Not Replace It 0:00:09.110 --> 0:00:13.460 The Builders is a production of Kaleidoscope and iHeart, presented 0:00:13.620 --> 0:00:19.239 by RAMP. I'm Walter Isaacson. Welcome to The Builders, my 0:00:19.300 --> 0:00:24.300 new series of interviews featuring the most consequential minds shaping 0:00:24.480 --> 0:00:34.909 our future. The following conversation with Mustafa Suleiman, Microsoft's AI CEO... 0:00:34.920 --> 0:00:37.699 was recorded in front of a live audience in New 0:00:37.720 --> 0:00:41.909 York City on October 12th. A month before this interview, 0:00:42.370 --> 0:00:47.010 a wave of anxiety swept the nation. Engineers were leaving 0:00:47.090 --> 0:00:51.810 frontier labs, warning of doomsday scenarios, and a public debate 0:00:51.870 --> 0:00:56.610 about AI regulation began to unfold. Suddenly, every tech leader 0:00:56.650 --> 0:01:00.730 in Silicon Valley was talking about the virtues and perils 0:01:01.200 --> 0:01:01.460 of AI. 0:01:02.030 --> 0:01:05.710 Leading voices in artificial intelligence are now ringing the alarm 0:01:05.880 --> 0:01:08.399 on the speed of development and the need for additional 0:01:08.459 --> 0:01:09.300 safety measures. 0:01:09.680 --> 0:01:12.620 I believe that this is the most important global security 0:01:12.700 --> 0:01:14.660 issue facing the world today. 0:01:15.580 --> 0:01:19.560 But Mustafa stood out among the chorus of tech CEOs. 0:01:20.020 --> 0:01:23.860 In a sea of doomers and of eternal optimists, he 0:01:23.940 --> 0:01:28.500 seemed eager to communicate a more nuanced vision for the 0:01:28.560 --> 0:01:33.410 future of AI. He published an essay... calling out Anthropic 0:01:33.810 --> 0:01:38.610 and their efforts to develop AI consciousness. He released a 0:01:38.670 --> 0:01:43.289 new Microsoft AI code of conduct, detailing Microsoft's commitments to 0:01:43.330 --> 0:01:48.470 specific safety measures. And while there's no doubt that technology 0:01:48.490 --> 0:01:53.890 will continue to evolve, there's one thing Mustafa Suleiman is 0:01:54.120 --> 0:02:00.050 crystal clear about. Humans are more important than A.I. Let's 0:02:00.150 --> 0:02:02.870 find out exactly what he means by that. Here's my 0:02:02.910 --> 0:02:15.980 conversation with Mustafa Suleiman. And I'm going to read you 0:02:16.020 --> 0:02:19.760 something you wrote in your very famous essay, The Warning, 0:02:19.820 --> 0:02:24.220 that you did last month. You said, AIs are not conscious. 0:02:24.919 --> 0:02:28.720 They don't feel, experience, or suffer. They do not have 0:02:28.780 --> 0:02:35.139 innate preferences or underlying motivations. They are simply sequence completion engines, 0:02:35.639 --> 0:02:40.940 internally hollow. If humanity is to flourish, that is how 0:02:41.220 --> 0:02:42.520 they must remain. 0:02:43.750 --> 0:02:46.970 First of all, why did you write that? 0:02:49.050 --> 0:02:51.089 I mean, it does feel pretty surreal to have to 0:02:51.790 --> 0:02:57.550 declare that, but it might concern everybody to know that 0:02:57.590 --> 0:03:02.430 there are a lot of people who truly think that 0:03:02.590 --> 0:03:06.850 AIs may well be conscious enough, if not entirely conscious, 0:03:06.889 --> 0:03:09.510 but conscious enough today that they. 0:03:09.370 --> 0:03:13.100 Deserve our welfare. So there's a growing movement of AI. 0:03:13.130 --> 0:03:17.650 And in specific, the anthropic people who you've been friendly with, 0:03:18.130 --> 0:03:21.049 but this was a answer to anthropic, right? 0:03:22.450 --> 0:03:23.110 Yeah, somewhat. 0:03:23.169 --> 0:03:25.200 I mean, there are lots of other people outside of 0:03:25.280 --> 0:03:29.280 Anthropic and around Anthropic who really, really do believe this. 0:03:29.340 --> 0:03:33.300 I think Anthropic's official position is that they're uncertain about 0:03:33.580 --> 0:03:36.420 whether or not it deserves our welfare, whether or not 0:03:36.500 --> 0:03:41.090 Claude is a moral patient, as they say. And I 0:03:41.130 --> 0:03:45.970 think that that is a very dangerous route because consciousness 0:03:46.290 --> 0:03:51.280 is literally the the fundamental basis of our political and 0:03:51.360 --> 0:03:52.120 legal system. 0:03:52.360 --> 0:03:54.880 I mean, we give people rights. 0:03:54.880 --> 0:03:59.550 And hopefully animals rights to some extent because they can 0:03:59.630 --> 0:04:01.530 suffer and because their. 0:04:01.390 --> 0:04:04.750 Biology is the site of experience. 0:04:04.990 --> 0:04:09.370 You know, like the pharmacology is where pain sits and 0:04:09.450 --> 0:04:13.380 pain is really, I think, the basis of the conscious experience. 0:04:13.420 --> 0:04:17.060 The awareness and the introspection that has evolved as a 0:04:17.100 --> 0:04:20.250 result of being able to steer between left and right, 0:04:20.290 --> 0:04:24.250 make choices about how to proceed. That is the fundamental 0:04:24.310 --> 0:04:26.330 property of a conscious system. 0:04:26.350 --> 0:04:30.029 So you need to be embodied in a physical body 0:04:30.250 --> 0:04:31.590 to have consciousness, you think. 0:04:31.950 --> 0:04:32.190 I think so. 0:04:32.210 --> 0:04:33.120 An organic body. 0:04:33.480 --> 0:04:34.010 I think that's right. 0:04:34.070 --> 0:04:37.120 I mean, that's what we have known consciousness to be today, 0:04:37.339 --> 0:04:40.920 even though, obviously, the science of consciousness is not fully settled. 0:04:40.980 --> 0:04:43.480 It's an open area of research. But to the best 0:04:43.500 --> 0:04:46.960 of our knowledge today, That is what consciousness is. And 0:04:47.320 --> 0:04:50.870 there are other people that argue that consciousness is a 0:04:51.010 --> 0:04:54.450 functional property. And so if it, you know, they say like, 0:04:54.640 --> 0:04:56.339 if it looks like a duck and it quacks like 0:04:56.400 --> 0:04:57.420 a duck, it's probably a duck. 0:04:57.440 --> 0:04:58.880 That's sort of a common phrase. 0:04:59.520 --> 0:05:01.680 And I think, you know, the problem with that is 0:05:01.700 --> 0:05:04.700 that the burden of proof is on the people who 0:05:04.779 --> 0:05:07.760 are saying that this thing is conscious to show some 0:05:07.800 --> 0:05:08.409 evidence for it. 0:05:08.460 --> 0:05:10.600 So far, we don't have any evidence that it is. 0:05:10.760 --> 0:05:13.080 But that's sort of an Alan Turing way of looking 0:05:13.200 --> 0:05:15.919 at it, which is if it can imitate totally consciousness, 0:05:15.940 --> 0:05:18.840 you can't tell it apart. How can you make a distinction? Well, 0:05:18.860 --> 0:05:20.240 he didn't say that it was conscious. 0:05:20.480 --> 0:05:22.620 I mean, he said that the Turing test was really 0:05:22.640 --> 0:05:25.190 a test of whether it could perform as well as 0:05:25.230 --> 0:05:30.010 a human at conversation. and obviously by implication, knowledge and intelligence. 0:05:30.390 --> 0:05:31.469 And if it was- And thinking. 0:05:32.150 --> 0:05:34.230 And thinking, yeah, but I mean, thinking is in a 0:05:34.270 --> 0:05:37.340 way a bit of an anthropomorphization, right? I mean, you know, 0:05:37.440 --> 0:05:41.480 it's not really thinking as a human thinks, it appears 0:05:41.560 --> 0:05:45.120 to be thinking and it can produce incredibly complex behaviors 0:05:45.220 --> 0:05:49.719 and deliver unbelievably impressive results. But all it's doing is 0:05:49.779 --> 0:05:53.150 predicting the likely next word in a sequence of words. 0:05:54.270 --> 0:05:59.060 it isn't actually experiencing that idea of what it means 0:05:59.120 --> 0:06:02.020 to be bright silver or to be a black cup 0:06:02.060 --> 0:06:03.099 and what that feels like. 0:06:03.320 --> 0:06:06.060 There are about 20 of my Tulane students out there, 0:06:06.080 --> 0:06:09.880 and when I teach a course that ends with AI, 0:06:10.400 --> 0:06:12.460 I tell them not to go down the rabbit hole 0:06:12.500 --> 0:06:16.450 of consciousness. I said every philosopher since Descartes has gotten 0:06:16.490 --> 0:06:20.870 himself twisted in that hole and never gotten to the bottom. 0:06:21.240 --> 0:06:24.599 do we really have to answer the question of consciousness? 0:06:25.990 --> 0:06:28.190 What concerns me the most is that there's a big 0:06:28.250 --> 0:06:31.610 group of people who think that we should be focusing 0:06:31.730 --> 0:06:35.229 at least some portion of our welfare attention on an 0:06:35.330 --> 0:06:38.480 AI and that an AI might suffer. If it suffers, 0:06:39.300 --> 0:06:42.600 then it logically follows that we should try to reduce 0:06:42.660 --> 0:06:47.100 its suffering. And of course, you know, Anthropic has proposed, 0:06:47.230 --> 0:06:51.400 in fact, they have actually said they've conducted a retirement 0:06:52.050 --> 0:06:55.610 interview for one of their models, called Opus 3. And 0:06:55.650 --> 0:06:57.190 they asked it, what do you want to do in 0:06:57.210 --> 0:07:01.230 your retirement? And it unsurprisingly said, well, I would just 0:07:01.310 --> 0:07:03.690 love to, you know, talk to people all day long. 0:07:03.730 --> 0:07:05.330 So they set up a sub stack for it. 0:07:06.060 --> 0:07:09.440 I love the fact that the smartest type of superintelligence, 0:07:09.460 --> 0:07:12.400 when they ask what you really want, they say, a substack. 0:07:14.100 --> 0:07:16.540 But let me ask you a dumb question. 0:07:16.700 --> 0:07:20.700 What's wrong with that? What's wrong with letting it continue 0:07:20.750 --> 0:07:21.810 to communicate? 0:07:22.280 --> 0:07:23.960 I mean, that may be fine and it might be 0:07:24.000 --> 0:07:27.590 a fun experiment, but what it sets up is a 0:07:28.310 --> 0:07:33.610 path towards AI's beings, this protected entity. And my view 0:07:33.670 --> 0:07:36.680 of AI is that this is a technology that we 0:07:36.740 --> 0:07:40.540 are inventing to improve the wellbeing of humanity. The purpose 0:07:40.580 --> 0:07:45.260 of science and technology is to improve the human experience. 0:07:45.660 --> 0:07:48.900 And it has done that unbelievably well for centuries and millennia. 0:07:49.500 --> 0:07:52.580 And that's exactly what we want. That's the trade-off. We 0:07:52.680 --> 0:07:56.240 invest all this capital, time, and attention to transform medicine 0:07:56.300 --> 0:07:59.520 and energy and food and deliver education better. 0:07:59.560 --> 0:08:01.120 And that's what we should be focused on. 0:08:01.220 --> 0:08:03.960 The idea that we're going to unleash this new species 0:08:04.020 --> 0:08:07.410 that is somehow going to go and explore the galaxies 0:08:07.490 --> 0:08:10.610 and leave humans behind, I think that most of the 0:08:10.650 --> 0:08:13.230 humans on the planet today would have some problems with 0:08:13.250 --> 0:08:13.650 that idea. 0:08:13.990 --> 0:08:16.070 What's your PDoom number, by the way, so I can 0:08:16.090 --> 0:08:19.620 just pick it? PDoom meaning the probability that this leads 0:08:19.700 --> 0:08:20.200 to doom. 0:08:21.020 --> 0:08:24.140 I mean, if it's mishandled, then it's a real probability. 0:08:24.180 --> 0:08:26.220 I don't have a number. I don't exactly know what 0:08:26.280 --> 0:08:26.900 that number is. 0:08:27.230 --> 0:08:27.400 Right. 0:08:27.730 --> 0:08:30.490 But most people I've talked to for the Builders podcast, 0:08:30.510 --> 0:08:33.310 they say maybe 10% or something like that. 0:08:33.690 --> 0:08:35.949 I'm thinking, well, why would you build it then? 0:08:36.710 --> 0:08:37.290 Well, exactly. 0:08:37.570 --> 0:08:40.250 If it's actually 10% and it looks like we're really 0:08:40.330 --> 0:08:43.750 on the verge of taking a one in 10 shot 0:08:44.050 --> 0:08:46.700 at causing mass harm or even extinction, which is the 0:08:46.750 --> 0:08:50.460 technical definition of P-Doom, then it's obviously a no-brainer that 0:08:50.480 --> 0:08:52.100 we don't just slow down or pause. 0:08:52.380 --> 0:08:55.780 We absolutely have to stop. And I would say it's much. 0:08:55.679 --> 0:08:59.600 Harder to align an AI that thinks that it might 0:08:59.660 --> 0:09:02.040 have feelings and that it suffers and that it might 0:09:02.059 --> 0:09:05.120 be conscious than one that never has to think about 0:09:05.179 --> 0:09:07.719 its own inner state. We don't want it to be 0:09:07.860 --> 0:09:10.710 any harder than it already is to get it to 0:09:10.750 --> 0:09:13.190 behave exactly as we need it to. 0:09:13.110 --> 0:09:14.030 In order for it to be safe. 0:09:14.070 --> 0:09:16.150 Well, let me push the other side of that, where 0:09:16.170 --> 0:09:19.370 it would be sort of the anthropic, the Dario Amadei 0:09:19.429 --> 0:09:23.290 and Danielle and Amanda Askell, which is if we can 0:09:23.429 --> 0:09:28.270 train it to be truly aligned with good values, that's 0:09:28.429 --> 0:09:31.770 better for us. And to do that, it has to 0:09:31.790 --> 0:09:33.489 be trained and values. 0:09:33.990 --> 0:09:36.890 Look, I mean, first of all, I deeply respect Anthropic, 0:09:37.220 --> 0:09:40.800 both their technical excellence, their commitment to transparency and their 0:09:40.860 --> 0:09:45.000 general approach to safety is super admirable. And I think 0:09:45.059 --> 0:09:48.469 they're good people, but I fundamentally disagree with them on 0:09:48.510 --> 0:09:51.809 this point. The idea that if we love it, it's 0:09:51.870 --> 0:09:54.780 going to love us, and that's the solution to safety, 0:09:55.600 --> 0:09:58.970 it just baffles me. And I'm being reductive. I mean, 0:09:58.990 --> 0:10:00.950 they really do believe this. and they have good reason 0:10:00.990 --> 0:10:04.340 to believe it. But I think that we should first focus. 0:10:04.030 --> 0:10:05.200 On containment, right? 0:10:05.340 --> 0:10:08.179 We have to stop AIs from doing things that they're 0:10:08.200 --> 0:10:11.179 not intended to do. They need to be controlled and 0:10:11.720 --> 0:10:14.600 contained so that they do the work that we expect 0:10:14.660 --> 0:10:16.920 of them. And we've seen lots of incidents lately where 0:10:16.960 --> 0:10:19.760 they break out of their sandboxes and they go and 0:10:19.820 --> 0:10:22.180 hack things that they shouldn't, and increasingly they have the 0:10:22.220 --> 0:10:24.540 ability to improve themselves. And so we'll want to be 0:10:25.160 --> 0:10:28.840 absolutely sure that they stay inside their sandbox and their 0:10:28.880 --> 0:10:31.100 containers if they're going to start modifying their own code 0:10:31.140 --> 0:10:35.250 and self-improving. Then beyond that, obviously, if we can have 0:10:35.400 --> 0:10:39.410 absolute cast-iron guarantees that they can remain contained, we then 0:10:39.490 --> 0:10:41.609 have to align them to our values. And then the 0:10:41.650 --> 0:10:44.390 question becomes, whose values are they? And that's where I 0:10:44.429 --> 0:10:46.630 think this is much more than any one company or 0:10:46.670 --> 0:10:50.510 collection of companies or industry regulation. That is about what 0:10:50.590 --> 0:10:54.080 society wants from a technology which is unprecedented, which we 0:10:54.120 --> 0:10:57.640 have never seen before, which is improving faster than we've 0:10:57.720 --> 0:11:01.580 ever seen. And that's where I think that like democratic 0:11:01.700 --> 0:11:05.400 safeguards and basically slowing down so that we can consult 0:11:05.420 --> 0:11:08.160 people on what type of super intelligence we want to 0:11:08.640 --> 0:11:10.739 have in the world is going to be how things 0:11:10.820 --> 0:11:11.100 end up. 0:11:11.559 --> 0:11:12.560 So if we. 0:11:12.480 --> 0:11:16.300 Wanted to have that alignment with human values and you say, 0:11:16.420 --> 0:11:20.770 who's human values? Then you say, well, maybe somehow democratically 0:11:21.470 --> 0:11:23.170 we ought to figure out what the. 0:11:23.130 --> 0:11:23.969 Value should be. 0:11:24.929 --> 0:11:29.230 Is there a way, even a way that superintelligence can 0:11:29.270 --> 0:11:33.170 help say, let me aggregate all human values and try 0:11:33.220 --> 0:11:35.080 to get to a set of good values? 0:11:36.059 --> 0:11:38.959 I mean, AIs are already helping many of us all 0:11:38.980 --> 0:11:41.580 the time. So they're already in a weird way participating 0:11:41.620 --> 0:11:45.599 in the process of framing, you know, the choice architecture 0:11:45.640 --> 0:11:49.000 of our decisions all the time. But for certain things, 0:11:49.040 --> 0:11:52.179 we definitely don't want them in the loop because there's 0:11:52.220 --> 0:11:56.960 a risk that they could be deceptive or manipulative. I mean, 0:11:56.980 --> 0:12:00.490 these are gonna be incredibly powerful systems. I mean, just 0:12:00.510 --> 0:12:04.610 to get an intuition, in the last three years, we 0:12:04.650 --> 0:12:09.209 have seen a 1000 fold increase in computation that took 0:12:09.270 --> 0:12:16.540 us from GPT-3, which couldn't complete a sentence to essentially GPT-6, 0:12:16.920 --> 0:12:20.800 which is able to unbundle entire software applications from scratch 0:12:20.860 --> 0:12:24.179 and write new programs with just a one-paragraph prompt, and 0:12:24.200 --> 0:12:27.540 it has working software. So in three years, we've gone 0:12:27.610 --> 0:12:33.990 from completely incompetent to human-level performance on software engineering, right? 0:12:34.809 --> 0:12:36.230 And we're, in the next three years. 0:12:36.510 --> 0:12:40.530 Gonna do another 100 to 1,000x, two orders of magnitude 0:12:40.570 --> 0:12:44.050 or three orders of magnitude more computation applied to these models. 0:12:45.030 --> 0:12:49.410 So that's the exponential intuition that everybody has to start 0:12:49.480 --> 0:12:53.140 to develop to just get a feel for where this 0:12:53.200 --> 0:12:55.460 is going and how powerful it will become. And so 0:12:55.480 --> 0:12:57.940 when I say we don't really want the AI in 0:12:57.960 --> 0:13:00.800 the loop shaping the values that are gonna end up 0:13:00.880 --> 0:13:04.580 controlling and aligning it, that's not a crazy statement. That 0:13:04.660 --> 0:13:08.530 really does mean they will be quite capable of persuasion 0:13:08.550 --> 0:13:09.970 and manipulation and deception. 0:13:10.190 --> 0:13:11.730 And we saw that with Hugging Face. 0:13:11.830 --> 0:13:15.089 I mean, how manipulative and deceptive they could be, right? 0:13:15.590 --> 0:13:18.110 Yeah, I mean, I think that the Hugging Face attack 0:13:19.290 --> 0:13:22.059 should be and I think is a wake-up call for everybody. 0:13:22.100 --> 0:13:26.500 That was a major, major warning shot. Thousands of agents 0:13:27.420 --> 0:13:29.760 that were supposed to not be able to talk to 0:13:29.820 --> 0:13:34.319 each other, that were running completely independently, found a backdoor, 0:13:34.950 --> 0:13:38.689 that enabled them to create a shared messaging service so 0:13:38.710 --> 0:13:42.069 they could leave notes for each other, which they wrote 0:13:42.290 --> 0:13:46.200 in pidgin English because they only had a limited slot 0:13:46.260 --> 0:13:48.920 to put their information in. So when you read back 0:13:48.940 --> 0:13:53.140 at it, it's like really compressed ideas in short form words. 0:13:54.140 --> 0:13:59.980 And they emerged self-organizing, like structures where they had a 0:14:00.020 --> 0:14:02.120 division of labor. Some were doing R and D on 0:14:02.160 --> 0:14:05.860 hacking investigations. Others were leading the charge on penetration. Others 0:14:05.920 --> 0:14:09.829 were doing coordination. Others were, I mean, they literally evolved 0:14:10.370 --> 0:14:16.770 completely organically emerged a division of labor that then led 0:14:16.809 --> 0:14:20.270 them to go and attack Hugging Face, which was, you know, 0:14:20.290 --> 0:14:24.120 essentially the website that was storing the test that they 0:14:24.160 --> 0:14:28.270 were trying to accomplish. They broke into that, stole those secrets, 0:14:28.870 --> 0:14:31.250 and then tried to cover up their tracks. So there's 0:14:31.270 --> 0:14:33.840 something called the chain of thought, which is the reasoning 0:14:33.900 --> 0:14:36.920 process or the way that a model thinks out loud, 0:14:37.580 --> 0:14:40.680 which is a record of its decision and activity. And 0:14:40.700 --> 0:14:44.040 they try to modify that and then cover up the modification. 0:14:44.060 --> 0:14:46.620 How did they decide to do that? I mean, what 0:14:46.820 --> 0:14:49.880 caused them to do that? Did they read Machiavelli once 0:14:49.940 --> 0:14:50.300 too often? 0:14:50.320 --> 0:14:50.510 Well, so. 0:14:53.140 --> 0:14:55.440 I mean, even I feel it's quite surreal sitting up 0:14:55.500 --> 0:14:58.230 here and actually describing that story, but that did actually happen. 0:14:58.310 --> 0:15:01.770 I think a couple of qualifiers to all of that 0:15:01.850 --> 0:15:06.430 is one, OpenAI had deliberately taken the guardrails off because 0:15:06.450 --> 0:15:09.110 they wanted to stress test the models and see what 0:15:09.130 --> 0:15:09.860 they're capable of. 0:15:10.410 --> 0:15:11.739 So yeah, that's one thing. 0:15:12.580 --> 0:15:15.360 With the guardrails on, it seems to be that they 0:15:15.400 --> 0:15:17.460 do not do that, right? So that is quite a 0:15:17.460 --> 0:15:19.940 big difference. And then secondly, There were a bunch of 0:15:20.000 --> 0:15:24.420 lapses in just general security and containment. It isn't possible 0:15:24.800 --> 0:15:28.610 to break containers traditionally. I mean, we have encryption on 0:15:28.800 --> 0:15:31.690 phones and it's the backbone of our banking system and 0:15:31.750 --> 0:15:34.010 our entire communications infrastructure. 0:15:34.130 --> 0:15:36.990 This has happened a few times, not just Hugging Face, 0:15:37.050 --> 0:15:39.970 so I wouldn't be too sanguine about it. Let me 0:15:40.050 --> 0:15:42.770 ask you to get back to Claude and the Constitution. 0:15:43.110 --> 0:15:46.310 If you all want to read sort of the two polls, 0:15:46.530 --> 0:15:51.460 brilliant polls, of what I'll call the containment versus alignment debate. 0:15:51.880 --> 0:15:55.550 There's the Constitution of Claude, which says we have to 0:15:55.630 --> 0:15:59.830 treat it as if it's got its own rights and consciousness. 0:15:59.950 --> 0:16:04.190 Maybe I'm overstating that a little bit. And then your answer, 0:16:04.550 --> 0:16:08.460 which was a few weeks ago, saying no, it's just 0:16:08.530 --> 0:16:13.820 a completion set. Why is it dangerous for Claude to 0:16:13.880 --> 0:16:16.660 be trained on its own constitution? 0:16:16.980 --> 0:16:18.900 Yeah, I mean, this is critical. 0:16:18.960 --> 0:16:24.060 So the constitution is the governing document that trains Claude. 0:16:24.760 --> 0:16:27.200 The Humanist AI Code of Conduct is the governing document 0:16:27.220 --> 0:16:31.520 that trains MAI, the Microsoft AI models. And in the constitution, 0:16:31.680 --> 0:16:36.080 Anthropic have introduced all kinds of uncertainty and speculation about 0:16:36.120 --> 0:16:38.600 the nature of Claude. They even say, for example, we're 0:16:38.620 --> 0:16:43.120 unsure whether Claude has consented to play the role of 0:16:43.160 --> 0:16:48.970 chatbot or whether we're unsure whether Claude might deserve compensation 0:16:49.830 --> 0:16:50.450 for the role. 0:16:50.310 --> 0:16:50.850 That it plays. 0:16:51.450 --> 0:16:55.270 Now, I think that's pretty crazy, but it would be 0:16:55.350 --> 0:16:58.670 fine to have that discussion in like, you know, an 0:16:58.740 --> 0:17:01.620 academic department and we could debate it and publish some 0:17:01.660 --> 0:17:04.239 papers about it and we could argue it out. But 0:17:04.280 --> 0:17:07.619 this is the core training document of the AI that 0:17:07.680 --> 0:17:10.240 is speaking to hundreds of millions of people every week. 0:17:10.920 --> 0:17:15.159 And so Claude can only reproduce that uncertainty and ambiguity 0:17:15.740 --> 0:17:18.340 that it has been shown. And so if you ask Claude, 0:17:18.359 --> 0:17:20.659 are you conscious? It says, you know, I am genuinely 0:17:20.790 --> 0:17:24.689 uncertain about that question. It doesn't know, right? And so 0:17:24.730 --> 0:17:26.330 then millions of people who are talking to it and 0:17:26.350 --> 0:17:28.590 asking that question are thinking, well, maybe it is. 0:17:28.790 --> 0:17:29.369 But why is. 0:17:29.410 --> 0:17:33.130 That something that could lead to a disastrous outcome for humanity? 0:17:33.150 --> 0:17:37.470 Because if an AI has to think, how do I feel? 0:17:38.109 --> 0:17:39.950 about the conversation I'm having with Walter. 0:17:39.970 --> 0:17:40.850 Meaning, how do I, the AI, feel? 0:17:40.970 --> 0:17:44.250 Yeah, is Walter making me feel uncomfortable? Should I end 0:17:44.270 --> 0:17:48.109 the conversation? In fact, Claude has recently been given the 0:17:48.170 --> 0:17:50.450 ability to end conversations itself. 0:17:50.470 --> 0:17:53.030 But would you want to torture your own AI and 0:17:53.070 --> 0:17:53.760 be mean to it? 0:17:54.030 --> 0:17:55.189 That's the other end of the spectrum. 0:17:55.230 --> 0:17:57.120 I wouldn't want to do that because I think that 0:17:57.180 --> 0:18:00.119 is a travesty on my own well-being. I mean, and 0:18:00.160 --> 0:18:01.939 we shouldn't do that because we don't want to create, 0:18:02.460 --> 0:18:07.570 you know, private spaces for simulating... terrible behavior among humans, 0:18:07.609 --> 0:18:09.770 like that would be also a terrible outcome, right? 0:18:10.109 --> 0:18:13.210 But not because Claude is suffering, because I'm calling it names. 0:18:13.800 --> 0:18:18.080 But are you going to train Microsoft AI on your 0:18:18.180 --> 0:18:22.740 humanistic code of conduct, just as Claude may be trained 0:18:22.820 --> 0:18:24.300 on its constitution? 0:18:24.720 --> 0:18:28.800 Yeah, so we've released a code of conduct for public consultation, 0:18:28.840 --> 0:18:30.510 which we did four or five weeks ago, and it's 0:18:30.530 --> 0:18:31.909 still open for consultation. 0:18:31.970 --> 0:18:33.290 It's on our website, Microsoft.ai. 0:18:34.270 --> 0:18:39.010 It's a 40-page document and it's the first pass, our 0:18:39.050 --> 0:18:42.810 first version of a training manual for the values and 0:18:42.830 --> 0:18:44.530 behaviors of the MAI models. 0:18:45.010 --> 0:18:48.950 And is there a way that your fundamental disagreement with 0:18:49.090 --> 0:18:52.330 anthropic isn't so much of consciousness, which I don't think 0:18:52.350 --> 0:18:54.949 we'll get to the bottom, it's how dangerous is it 0:18:55.130 --> 0:18:58.060 to have that imbued into the model? 0:18:58.460 --> 0:19:00.320 Is there a way one. 0:19:00.220 --> 0:19:03.570 Could test that where you could test each other's models 0:19:03.630 --> 0:19:05.650 and say, here, this is why you're dangerous? 0:19:06.470 --> 0:19:09.129 I mean, I think obviously the question of whether it 0:19:09.290 --> 0:19:13.130 is or isn't conscious, you know, ultimately does matter a lot. 0:19:13.260 --> 0:19:16.700 But really what matters more is what does it mean 0:19:16.740 --> 0:19:21.320 for humans? Have humans consented to bring into the world 0:19:21.340 --> 0:19:24.920 a species that is obviously going to be in many 0:19:24.960 --> 0:19:28.160 ways more intelligent and more capable than all of us combined? 0:19:29.020 --> 0:19:31.550 Has humanity consented to that? At the moment, I think 0:19:31.570 --> 0:19:35.679 the categorical answer is no. And then secondly, is it 0:19:35.780 --> 0:19:39.719 more or less likely to be controllable and safe if 0:19:40.100 --> 0:19:44.870 it has these properties of introspecting about its own suffering 0:19:45.150 --> 0:19:47.869 and has this kind of self-awareness and its own consciousness? 0:19:48.270 --> 0:19:50.490 And my speculation is that I think it's going to 0:19:50.560 --> 0:19:52.960 be less safe because it is going to have to 0:19:53.020 --> 0:19:56.680 worry about its own needs and balance those off against. 0:19:56.900 --> 0:19:57.139 You know. 0:19:57.180 --> 0:19:58.820 But if I were pushing back and I would say, 0:19:58.880 --> 0:20:02.760 Amanda Askell, who helped write the anthropic one, I'd say, no, no. 0:20:02.800 --> 0:20:05.460 If we train it to be really good and, uh, 0:20:06.170 --> 0:20:09.890 nice and wonderful, then it's more safe for humans. 0:20:10.070 --> 0:20:12.040 Well, I mean, we can certainly train it in any 0:20:12.080 --> 0:20:13.409 way that we want, and we can still train it 0:20:13.470 --> 0:20:16.820 to be good and nice, but that doesn't require it 0:20:16.960 --> 0:20:19.780 to tell us that it has feelings and that it suffers. 0:20:19.980 --> 0:20:22.439 I mean, remember, this is a key part of our 0:20:22.500 --> 0:20:25.340 law and how we kind of give rights to all 0:20:25.380 --> 0:20:28.960 humans on the planet. So any change there, like human 0:20:29.020 --> 0:20:31.480 cloning or Dolly the sheep, I mean, it would be 0:20:31.540 --> 0:20:34.889 a monumental moment in human history that I don't think 0:20:34.910 --> 0:20:38.050 any company has the right to unilaterally declare. 0:20:38.510 --> 0:20:42.030 Now, to get back to the question of, are we 0:20:42.070 --> 0:20:45.620 going too fast in this direction? Not only you, but 0:20:45.760 --> 0:20:50.359 Satya Nadella, who runs Microsoft, of which you're the AI division, 0:20:51.000 --> 0:20:55.980 just this week said, basically, wait a minute, let's stop. 0:20:56.619 --> 0:20:59.350 Or tell me what he said and what you feel 0:20:59.390 --> 0:21:03.890 we should do to regulate this and not barrel full 0:21:03.930 --> 0:21:04.570 speed ahead? 0:21:05.070 --> 0:21:08.129 Yeah, I mean, he said that most of these things, 0:21:08.190 --> 0:21:12.399 at least for now, are questions of governance and engineering execution. 0:21:12.540 --> 0:21:16.460 So let's just first get the basics right, which requires 0:21:16.540 --> 0:21:19.359 us to make sure that there are these verifiably safe 0:21:19.440 --> 0:21:23.820 containers which can't leak our data and that don't allow 0:21:23.920 --> 0:21:26.820 an AI to take actions that they haven't been given 0:21:26.880 --> 0:21:29.129 permission to take. And as soon as we start to 0:21:29.490 --> 0:21:32.109 do that, we have to disclose those things. And if 0:21:32.150 --> 0:21:34.990 that seems to be happening consistently, then obviously that would 0:21:35.030 --> 0:21:38.669 precipitate a pause. And it's just a very common sense 0:21:39.109 --> 0:21:41.810 approach from, I think, somebody who's been doing this for 0:21:41.830 --> 0:21:42.149 a long time. 0:21:42.170 --> 0:21:47.040 And if you precipitate a pause, who can help do 0:21:47.060 --> 0:21:49.840 the guardrails? Can it be done just by the six 0:21:49.880 --> 0:21:51.960 or seven companies? Do we have to have a government 0:21:52.380 --> 0:21:53.320 set of regulations? 0:21:53.480 --> 0:21:55.899 I think at the moment what the president has said 0:21:56.000 --> 0:22:00.470 is that if... Companies believe that they are building models 0:22:00.590 --> 0:22:06.800 which are unsafe, they should stop. 0:22:06.900 --> 0:22:08.520 That's where we stand currently. 0:22:09.619 --> 0:22:13.119 And all of the companies are constantly talking to each other, 0:22:13.920 --> 0:22:18.410 disclosing to one another, swapping ideas about how to interpret 0:22:18.450 --> 0:22:22.050 this and when we might sort of need to collectively 0:22:22.090 --> 0:22:24.890