WEBVTT - Mustafa Suleyman: AI Should Serve Humanity, Not Replace It 0:00:09.930 --> 0:00:14.270 The Builders is a production of Kaleidoscope and iHeart, presented 0:00:14.430 --> 0:00:19.029 by Ramp. Ramp is a financial operations platform that helps 0:00:19.130 --> 0:00:23.300 ambitious companies manage their finances so their teams can spend 0:00:23.340 --> 0:00:28.620 less time chasing receipts and approvals and more time bringing extraordinary. 0:00:28.060 --> 0:00:29.300 Ideas to life. 0:00:30.680 --> 0:00:34.800 I'm Walter Isaacson. Welcome to The Builders, my new series 0:00:34.870 --> 0:00:39.830 of interviews featuring the most consequential minds Shaping Our Future. 0:00:43.180 --> 0:00:49.860 The following conversation with Mustafa Suleiman, Microsoft's AI CEO, was 0:00:49.900 --> 0:00:52.740 recorded in front of a live audience in New York 0:00:52.800 --> 0:00:57.190 City on October 12th. A month before this interview, a 0:00:57.350 --> 0:01:02.390 wave of anxiety swept the nation. Engineers were leaving frontier 0:01:02.470 --> 0:01:06.840 labs Warning of doomsday scenarios and a public debate about 0:01:06.860 --> 0:01:11.500 AI regulation began to unfold. Suddenly, every tech leader in 0:01:11.600 --> 0:01:16.610 Silicon Valley was talking about the virtues and perils of AI. 0:01:16.830 --> 0:01:20.510 Leading voices in artificial intelligence are now ringing the alarm 0:01:20.650 --> 0:01:23.190 on the speed of development and the need for additional 0:01:23.270 --> 0:01:24.090 safety measures. 0:01:24.470 --> 0:01:27.410 I believe that this is the most important global security 0:01:27.490 --> 0:01:29.410 issue facing the world today. 0:01:30.350 --> 0:01:34.330 But Mustafa stood out among the chorus of tech CEOs. 0:01:34.790 --> 0:01:38.630 In a sea of doomers and of eternal optimists, he 0:01:38.730 --> 0:01:43.280 seemed eager to communicate a more nuanced vision for the 0:01:43.360 --> 0:01:48.200 future of AI. He published an essay calling out Anthropic 0:01:48.600 --> 0:01:53.400 and their efforts to develop AI consciousness. He released a 0:01:53.460 --> 0:01:58.080 new Microsoft AI code of conduct, detailing Microsoft's commitments to 0:01:58.120 --> 0:02:03.250 specific safety measures. And while there's no doubt that technology 0:02:03.290 --> 0:02:08.680 will continue to evolve, there's one thing Mustafa Suleiman is 0:02:08.919 --> 0:02:14.840 crystal clear about. Humans are more important than AI. Let's 0:02:14.940 --> 0:02:17.660 find out exactly what he means by that. Here's my 0:02:17.710 --> 0:02:31.130 conversation with Mustafa Suleiman. I'm going to read you something 0:02:31.169 --> 0:02:34.730 you wrote in your very famous essay, the warning that 0:02:34.750 --> 0:02:38.989 you did last month. You said, AIs are not conscious. 0:02:39.710 --> 0:02:43.510 They don't feel, experience, or suffer. They do not have 0:02:43.570 --> 0:02:49.919 innate preferences or underlying motivations. They are simply sequence completion engines, 0:02:50.440 --> 0:02:55.760 internally hollow. If humanity is to flourish, that is how 0:02:56.000 --> 0:03:01.750 they must remain. First of all, Why did you write that? 0:03:03.830 --> 0:03:05.869 I mean, it does feel pretty surreal to have to 0:03:06.570 --> 0:03:12.359 declare that. But it might concern everybody to know that 0:03:12.380 --> 0:03:17.239 there are a lot of people who truly think that 0:03:17.380 --> 0:03:21.639 AIs may well be conscious enough, if not entirely conscious, 0:03:21.680 --> 0:03:25.910 but conscious enough today that they deserve our welfare. So 0:03:25.930 --> 0:03:28.130 there's a growing movement of AI welfare. 0:03:28.150 --> 0:03:32.440 And in specific, the anthropic people who you've been friendly with, 0:03:32.919 --> 0:03:35.880 but this was a answer to anthropic, right? 0:03:37.240 --> 0:03:37.900 Yeah, somewhat. 0:03:37.960 --> 0:03:39.980 I mean, there are lots of other people outside of 0:03:40.060 --> 0:03:44.060 anthropic and around anthropic who really, really do believe this. 0:03:44.150 --> 0:03:48.090 I think anthropic's official position is that they're uncertain about 0:03:48.370 --> 0:03:51.210 whether or not it deserves our welfare, whether or not 0:03:51.290 --> 0:03:55.810 Claude is a moral patient, as they say. And, you know, 0:03:55.830 --> 0:03:58.280 I think that that is a, is a very dangerous 0:03:58.600 --> 0:04:04.940 route because consciousness is literally the fundamental basis of our 0:04:05.200 --> 0:04:09.330 political and legal system. I mean, we give people rights 0:04:09.670 --> 0:04:14.330 and hopefully animals rights to some extent because they can 0:04:14.410 --> 0:04:19.560 suffer and because their biology is the site of experience, 0:04:19.800 --> 0:04:24.150 you know, like the pharmacology is where pain sits. And 0:04:24.230 --> 0:04:28.150 pain is really, I think, the basis of the conscious experience, 0:04:28.190 --> 0:04:31.850 the awareness and the introspection that has evolved as a 0:04:31.890 --> 0:04:35.040 result of being able to steer between left and right, 0:04:35.080 --> 0:04:39.020 make choices about how to proceed. That is the fundamental 0:04:39.100 --> 0:04:41.120 property of a conscious system. 0:04:41.160 --> 0:04:44.820 So you need to be embodied in a physical body 0:04:45.040 --> 0:04:46.370 to have consciousness, you think. 0:04:46.750 --> 0:04:48.790 I think so. An organic body. I think that's right. 0:04:48.850 --> 0:04:51.909 I mean, that's what we have known consciousness to be today, 0:04:52.110 --> 0:04:55.700 even though obviously the science of consciousness is not fully settled. 0:04:55.760 --> 0:04:58.260 It's an open area of research, but to the best 0:04:58.279 --> 0:05:01.750 of our knowledge today, that is what consciousness is. And 0:05:02.110 --> 0:05:05.670 there are other people that argue that consciousness is a 0:05:05.790 --> 0:05:09.230 functional property. And so if it, you know, they say like, 0:05:09.410 --> 0:05:11.110 if it looks like a duck and it quacks like 0:05:11.170 --> 0:05:12.190 a duck, it's probably a duck. 0:05:12.210 --> 0:05:13.690 That's sort of a common phrase. 0:05:14.320 --> 0:05:16.479 And I think, you know, the problem with that is 0:05:16.500 --> 0:05:19.500 that the burden of proof is on the people who 0:05:19.580 --> 0:05:22.560 are saying that this thing is conscious to show some 0:05:22.600 --> 0:05:23.210 evidence for it. 0:05:23.240 --> 0:05:25.380 So far, we don't have any evidence that it is. 0:05:25.540 --> 0:05:27.869 But that's sort of an Alan Turing way of looking 0:05:27.990 --> 0:05:30.710 at it, which is if it can imitate totally consciousness, 0:05:30.730 --> 0:05:31.770 you can't tell it apart. 0:05:32.110 --> 0:05:34.110 How can you make a distinction? Well, he didn't say 0:05:34.150 --> 0:05:35.030 that it was conscious. 0:05:35.290 --> 0:05:37.390 I mean, he said that the Turing test was really 0:05:37.410 --> 0:05:39.979 a test of whether it could perform as well as 0:05:40.010 --> 0:05:44.800 a human at conversation, and obviously by implication, knowledge and intelligence. 0:05:45.180 --> 0:05:46.260 And thinking. 0:05:46.940 --> 0:05:49.000 And thinking, yeah, but I mean, thinking is in a 0:05:49.060 --> 0:05:51.100 way a bit of an anthropomorphization, right? 0:05:51.160 --> 0:05:54.580 I mean, you know, it's not really thinking as a 0:05:54.640 --> 0:05:55.400 human thinks. 0:05:55.580 --> 0:05:58.780 It appears to be thinking and it can produce incredibly 0:05:58.820 --> 0:06:04.010 complex behaviors and deliver unbelievably impressive results. But all it's 0:06:04.050 --> 0:06:07.470 doing is predicting the likely next word in a sequence 0:06:07.510 --> 0:06:13.380 of words. it isn't actually experiencing that idea of what 0:06:13.400 --> 0:06:16.180 it means to be bright silver or to be a 0:06:16.220 --> 0:06:17.880 black cup and what that feels like. 0:06:18.500 --> 0:06:20.850 There are about 20 of my Tulane students out there, 0:06:20.870 --> 0:06:24.669 and when I teach a course that ends with AI, 0:06:25.190 --> 0:06:27.250 I tell them not to go down the rabbit hole 0:06:27.290 --> 0:06:31.250 of consciousness. I said every philosopher since Descartes has gotten 0:06:31.290 --> 0:06:35.650 himself twisted in that hole and never gotten to the bottom. 0:06:36.040 --> 0:06:39.400 do we really have to answer the question of consciousness? 0:06:40.760 --> 0:06:42.980 What concerns me the most is that there's a big 0:06:43.020 --> 0:06:46.380 group of people who think that we should be focusing 0:06:46.880 --> 0:06:50.000 at least some portion of our welfare attention on an 0:06:50.110 --> 0:06:53.330 AI and that an AI might suffer. If it suffers, 0:06:54.089 --> 0:06:57.409 then it logically follows that we should try to reduce 0:06:57.470 --> 0:07:01.890 its suffering. And of course, you know, Anthropic has proposed, 0:07:02.010 --> 0:07:06.210 in fact, they have actually said they've conducted a retirement 0:07:06.830 --> 0:07:08.409 interview for one of their models. 0:07:08.850 --> 0:07:10.330 Claude Opus 3. 0:07:10.330 --> 0:07:11.850 And they asked it, what do you want to do 0:07:11.910 --> 0:07:15.810 in your retirement? And it unsurprisingly said, well, I would 0:07:15.850 --> 0:07:18.490 just love to talk to people all day long. 0:07:18.510 --> 0:07:20.890 So they set up a sub stack for it. You 0:07:20.930 --> 0:07:21.590 should go and check it out. 0:07:21.630 --> 0:07:24.210 I love the fact that the smartest type of super intelligence, 0:07:24.250 --> 0:07:26.520 when they ask what you really want, they say, a 0:07:26.560 --> 0:07:27.140 sub stack. 0:07:28.880 --> 0:07:31.980 But let me ask you a dumb question. What's wrong 0:07:32.020 --> 0:07:36.620 with that? What's wrong with letting it continue to communicate? 0:07:37.060 --> 0:07:38.760 I mean, that may be fine and it might be 0:07:38.780 --> 0:07:43.010 a fun experiment, but what it sets up is a 0:07:43.090 --> 0:07:48.410 path towards AI's beings, this protected entity. And my view 0:07:48.470 --> 0:07:51.450 of AI is that this is a technology that we 0:07:51.510 --> 0:07:55.310 are inventing to improve the wellbeing of humanity. The purpose 0:07:55.350 --> 0:08:00.040 of science and technology is to improve the human experience. 0:08:00.440 --> 0:08:03.660 And it has done that unbelievably well for centuries and millennia. 0:08:04.280 --> 0:08:07.360 And that's exactly what we want. That's the trade-off. We 0:08:07.460 --> 0:08:11.020 invest all this capital, time and attention to transform medicine 0:08:11.080 --> 0:08:14.800 and energy and food and deliver education better. And that's 0:08:14.820 --> 0:08:16.820 what we should be focused on. The idea that we're 0:08:16.850 --> 0:08:19.910 gonna unleash this new species that is somehow gonna go 0:08:19.950 --> 0:08:24.400 and explore the galaxies leave humans behind, I think that 0:08:24.880 --> 0:08:27.310 most of the humans on the planet today would have 0:08:27.350 --> 0:08:28.470 some problems with that idea. 0:08:28.790 --> 0:08:30.870 What's your PDoom number, by the way, so I can 0:08:30.890 --> 0:08:34.410 just pick it? PDoom meaning the probability that this leads 0:08:34.470 --> 0:08:35.010 to doom. 0:08:35.809 --> 0:08:38.930 I mean, if it's mishandled, then it's a real probability. 0:08:38.970 --> 0:08:41.010 I don't have a number. I don't exactly know what 0:08:41.050 --> 0:08:41.700 that number is. 0:08:42.020 --> 0:08:42.199 Right. 0:08:42.500 --> 0:08:45.080 But most people I've talked to for the Builders podcast 0:08:45.100 --> 0:08:48.100 and all, they say maybe 10% or something like that. 0:08:48.480 --> 0:08:52.080 I'm thinking, well, why would you build it then? Well, exactly. 0:08:52.360 --> 0:08:55.040 If it's actually 10% and it looks like we're really 0:08:55.120 --> 0:08:58.559 on the verge of taking a one in 10 shot 0:08:58.840 --> 0:09:01.480 at causing mass harm or even extinction, which is the 0:09:01.540 --> 0:09:05.240 technical definition of P-Doom, then it's obviously a no-brainer that 0:09:05.280 --> 0:09:06.860 we don't just slow down or pause. 0:09:06.880 --> 0:09:10.559 We absolutely have to stop. And I would say it's much. 0:09:10.480 --> 0:09:14.380 Harder to align an AI that thinks that it might 0:09:14.440 --> 0:09:16.819 have feelings and that it suffers and that it might 0:09:16.840 --> 0:09:19.900 be conscious than one that never has to think about 0:09:19.960 --> 0:09:22.520 its own inner state. We don't want it to be 0:09:22.640 --> 0:09:25.490 any harder than it already is to get it to 0:09:25.550 --> 0:09:28.290 behave exactly as we need it to in order for 0:09:28.330 --> 0:09:28.830 it to be safe. 0:09:28.870 --> 0:09:30.929 Well, let me push the other side of that, where 0:09:30.950 --> 0:09:34.150 it would be sort of the anthropic, the Dario Amadei 0:09:34.210 --> 0:09:38.080 and Daniel and Amanda Askell, which is if we can 0:09:38.240 --> 0:09:43.080 train it to be truly aligned with good values, that's 0:09:43.220 --> 0:09:46.550 better for us. And to do that, it has to 0:09:46.570 --> 0:09:48.290 be trained in values. 0:09:48.790 --> 0:09:51.700 Look, I mean, first of all, I deeply respect Anthropic, 0:09:52.000 --> 0:09:55.579 both their technical excellence, their commitment to transparency, and their 0:09:55.640 --> 0:09:59.770 general approach to safety is super admirable, and I think 0:09:59.790 --> 0:10:03.239 they're good people, but I fundamentally disagree with them on 0:10:03.280 --> 0:10:03.700 this point. 0:10:04.240 --> 0:10:05.560 The idea that if we. 0:10:05.480 --> 0:10:08.270 Love it, it's going to love us, and that's the 0:10:08.309 --> 0:10:12.370 solution to safety, it just baffles me. And like, I'm 0:10:12.390 --> 0:10:15.070 being reductive. I mean, they really do believe this, and 0:10:15.090 --> 0:10:17.480 they have good reason to believe it. But I think 0:10:17.520 --> 0:10:20.340 that we should first focus on containment, right? We have 0:10:20.380 --> 0:10:23.500 to stop AIs from doing things that they're not intended 0:10:23.520 --> 0:10:27.370 to do. They need to be controlled and contained so 0:10:27.390 --> 0:10:29.750 that they do the work that we expect of them. 0:10:30.010 --> 0:10:32.390 And we've seen lots of incidents lately where they break 0:10:32.470 --> 0:10:35.090 out of their sandboxes and they go and hack things 0:10:35.130 --> 0:10:37.490 that they shouldn't. And increasingly they have the ability to 0:10:37.570 --> 0:10:41.000 improve themselves. And so we'll want to be absolutely sure 0:10:41.380 --> 0:10:44.239 that they stay inside their sandbox and their containers if 0:10:44.260 --> 0:10:46.719 they're going to start modifying their own code and self-improving. 0:10:47.360 --> 0:10:51.540 Then beyond that, obviously, if we can have absolute cast-iron 0:10:51.559 --> 0:10:54.520 guarantees that they can remain contained, we then have to 0:10:54.620 --> 0:10:57.080 align them to our values. And then the question becomes, 0:10:57.140 --> 0:10:59.559 whose values are they? And that's where I think this 0:10:59.620 --> 0:11:01.920 is much more than any one company or collection of 0:11:01.980 --> 0:11:06.230 companies or industry regulation. That is about what society wants 0:11:06.280 --> 0:11:09.370 from a technology which is unprecedented, which we have never 0:11:09.429 --> 0:11:12.910 seen before, which is improving faster than we've ever seen, 0:11:13.770 --> 0:11:17.510 And that's where I think that like democratic safeguards and 0:11:17.809 --> 0:11:20.580 basically slowing down so that we can consult people on 0:11:20.620 --> 0:11:23.699 what type of super intelligence we want to have in 0:11:23.720 --> 0:11:25.880 the world is going to be how things end up. 0:11:26.340 --> 0:11:29.880 So if we wanted to have that alignment with human 0:11:29.940 --> 0:11:33.360 values and you say, who's human values? Then you say, well, 0:11:33.420 --> 0:11:37.980 maybe somehow democratically we ought to figure out what the. 0:11:37.920 --> 0:11:38.680 Value should be. 0:11:39.700 --> 0:11:44.030 Is there a way, even a way that superintelligence can 0:11:44.070 --> 0:11:47.970 help say, let me aggregate all human values and try 0:11:47.990 --> 0:11:49.870 to get to a set of good values? 0:11:50.830 --> 0:11:53.730 I mean, AIs are already helping many of us all 0:11:53.770 --> 0:11:56.350 the time. So they're already in a weird way participating 0:11:56.390 --> 0:12:01.320 in the process of framing. 0:11:59.380 --> 0:12:01.780 The choice architecture of our decisions all the time. 0:12:02.480 --> 0:12:05.580 But for certain things, we definitely don't want them in 0:12:05.600 --> 0:12:08.599 the loop because there's a risk that they could be, 0:12:08.620 --> 0:12:12.040 you know, deceptive or manipulative. I mean, these are going 0:12:12.059 --> 0:12:15.760 to be incredibly powerful systems. I mean, just to get 0:12:15.780 --> 0:12:19.920 an intuition, in the last three years, we have seen 0:12:19.940 --> 0:12:25.370 a 1000 fold increase in computation that took us from GPT-3, 0:12:26.030 --> 0:12:32.120 which couldn't complete a sentence to essentially GPT-6, which is 0:12:32.420 --> 0:12:36.280 able to unbundle entire software applications from scratch and write 0:12:36.350 --> 0:12:39.030 new programs with just a one paragraph prompt and it 0:12:39.070 --> 0:12:42.569 has working software. So in three years, we've gone from 0:12:42.670 --> 0:12:48.770 completely incompetent to human level performance on software engineering, right? 0:12:49.590 --> 0:12:51.270 And we're in the next three years. 0:12:51.290 --> 0:12:54.989 Gonna do another 100 to 1000 X, two orders of 0:12:55.010 --> 0:12:57.800 magnitude or three orders of magnitude more computation applied to 0:12:58.340 --> 0:13:03.830 these models. That's the exponential intuition that everybody has to 0:13:03.880 --> 0:13:07.730 start to develop to just get a feel for where 0:13:07.770 --> 0:13:08.150 this is. 0:13:08.110 --> 0:13:09.690 Going and how powerful it will become. 0:13:10.070 --> 0:13:12.270 And so when I say we don't really want the 0:13:12.330 --> 0:13:15.070 AI in the loop shaping the values that are going 0:13:15.090 --> 0:13:18.459 to end up controlling and aligning it, that's not a 0:13:18.500 --> 0:13:21.780 crazy statement. That really does mean they will be quite 0:13:21.920 --> 0:13:24.780 capable of persuasion and manipulation and deception. 0:13:25.000 --> 0:13:27.059 And we saw that with Hugging Face. I mean, how 0:13:27.080 --> 0:13:29.750 manipulative and deceptive they could be, right? 0:13:30.380 --> 0:13:32.900 Yeah, I mean, I think that the hugging face attack 0:13:34.059 --> 0:13:36.300 should be, and I think is a wake up call 0:13:36.320 --> 0:13:40.420 for everybody. That was a major, major warning shot. Thousands 0:13:40.500 --> 0:13:44.090 of agents that were supposed to not be able to 0:13:44.150 --> 0:13:48.429 talk to each other, that were running completely independently, found 0:13:48.470 --> 0:13:52.189 a backdoor that enabled them to create a shared messaging 0:13:52.250 --> 0:13:56.270 service so they could leave notes for each other, which 0:13:56.309 --> 0:13:59.949 they wrote in pidgin English, because they only had a 0:14:00.010 --> 0:14:03.230 limited slot to put their information in. So when you 0:14:03.270 --> 0:14:06.410 read back at it, it's like really compressed ideas in 0:14:06.559 --> 0:14:14.640 short form words. And they emerged self-organizing structures where they. 0:14:14.580 --> 0:14:17.300 Had a division of labor. Some were doing R & D. 0:14:16.840 --> 0:14:20.380 On hacking investigations. Others were leading the charge on penetration. 0:14:20.440 --> 0:14:25.710 Others were doing coordination. I mean, they literally evolved, completely 0:14:25.830 --> 0:14:31.730 organically emerged a... division of labor that then led them 0:14:31.790 --> 0:14:35.570 to go and attack Hugging Face, which was essentially the 0:14:35.590 --> 0:14:39.360 website that was storing the test that they were trying 0:14:39.400 --> 0:14:43.760 to accomplish. They broke into that, stole those secrets, and 0:14:43.780 --> 0:14:46.359 then try to cover up their tracks. So there's something 0:14:46.400 --> 0:14:49.200 called the chain of thought, which is the reasoning process 0:14:49.280 --> 0:14:52.520 or the way that a model thinks out loud, which 0:14:52.560 --> 0:14:55.640 is a record of its decision and activity. And they 0:14:55.840 --> 0:14:58.720 try to modify that and then cover up the modification. 0:14:58.740 --> 0:15:01.400 How did they decide to do that? I mean, what 0:15:01.610 --> 0:15:04.670 caused them to do that? Did they read Machiavelli once 0:15:04.730 --> 0:15:05.190 too often? 0:15:07.920 --> 0:15:10.229 I mean, even I feel it's quite surreal sitting up 0:15:10.290 --> 0:15:13.010 here and actually describing that story, but that did actually happen. 0:15:13.090 --> 0:15:16.550 I think a couple of qualifiers to all of that 0:15:16.650 --> 0:15:21.220 is one, OpenAI had deliberately taken the guardrails off because 0:15:21.240 --> 0:15:23.900 they wanted to stress test the models and see what 0:15:23.920 --> 0:15:24.640 they're capable of. 0:15:25.220 --> 0:15:26.520 So yeah, that's one thing. 0:15:27.360 --> 0:15:30.160 With the guardrails on, it seems to be that they 0:15:30.180 --> 0:15:31.280 do not do that, right? 0:15:31.350 --> 0:15:32.729 So that is quite a big difference. 0:15:32.750 --> 0:15:35.620 And then secondly, There were a bunch of lapses in 0:15:35.660 --> 0:15:40.130 just general security and containment. It isn't possible to break 0:15:40.310 --> 0:15:44.210 containers traditionally. I mean, we have encryption on phones and 0:15:44.250 --> 0:15:47.390 it's the backbone of our banking system and our entire 0:15:47.450 --> 0:15:48.790 communications infrastructure. 0:15:48.910 --> 0:15:51.790 This has happened a few times, not just Hugging Face, 0:15:51.830 --> 0:15:54.750 so I wouldn't be too sanguine about it. Let me 0:15:54.830 --> 0:15:57.550 ask you to get back to Claude and the Constitution. 0:15:57.890 --> 0:16:01.100 If you all want to read sort of the two polls, 0:16:01.320 --> 0:16:06.250 brilliant polls, of what I'll call the containment versus alignment debate. 0:16:06.670 --> 0:16:10.330 There's the Constitution of Claude, which says we have to 0:16:10.410 --> 0:16:14.630 treat it as if it's got its own rights and consciousness. 0:16:14.730 --> 0:16:18.980 Maybe I'm overstating that a little bit. And then your answer, 0:16:19.340 --> 0:16:23.240 which was a few weeks ago, saying no, it's just 0:16:23.320 --> 0:16:28.600 a completion set. Why is it dangerous for Claude to 0:16:28.660 --> 0:16:31.440 be trained on its own constitution? 0:16:31.760 --> 0:16:33.700 Yeah, I mean, this is critical. 0:16:33.740 --> 0:16:38.840 So the constitution is the governing document that trains Claude. 0:16:39.540 --> 0:16:41.980 The Humanist AI Code of Conduct is the governing document 0:16:42.000 --> 0:16:46.300 that trains MAI, the Microsoft AI models. And in the constitution, 0:16:46.470 --> 0:16:50.870 Anthropic have introduced all kinds of uncertainty and speculation about 0:16:50.910 --> 0:16:53.390 the nature of Claude. They even say, for example, we're 0:16:53.410 --> 0:16:57.900 unsure whether Claude has consented to play the role of 0:16:57.970 --> 0:17:02.770 chatbot or whether we are unsure whether Claude might deserve 0:17:02.870 --> 0:17:07.490 compensation for the role that it plays. Now, I think 0:17:07.510 --> 0:17:10.800 that's pretty crazy, but it would be fine to have 0:17:10.840 --> 0:17:14.919 that discussion in like, you know, an academic department and 0:17:15.000 --> 0:17:17.220 we could debate it and publish some papers about it 0:17:17.359 --> 0:17:19.540 and we could argue it out. But this is the 0:17:19.660 --> 0:17:23.109 core training document of the AI that is speaking to 0:17:23.160 --> 0:17:26.560 hundreds of millions of people every week. And so Claude 0:17:26.660 --> 0:17:31.120 can only reproduce that uncertainty and ambiguity that it has 0:17:31.160 --> 0:17:33.850 been shown. And so if you ask Claude, are you conscious? 0:17:33.900 --> 0:17:36.770 It says, you know, I am genuinely uncertain about that question. 0:17:36.970 --> 0:17:40.270 It doesn't know, right? And so then millions of people 0:17:40.290 --> 0:17:42.510 who are talking to it and asking that question are thinking, well, 0:17:42.830 --> 0:17:43.369 maybe it is. 0:17:43.570 --> 0:17:45.690 But why is that something that could lead to a 0:17:45.730 --> 0:17:47.870 disastrous outcome for humanity? 0:17:47.890 --> 0:17:52.260 Because if an AI has to think, how do I feel? 0:17:52.890 --> 0:17:56.990 about the conversation I'm having with Walter. Is Walter making 0:17:57.030 --> 0:18:00.150 me feel uncomfortable? Should I end the conversation? In fact, 0:18:00.450 --> 0:18:05.260 Claude has recently been given the ability to end conversations itself. 0:18:05.280 --> 0:18:07.820 But would you want to torture your own AI and 0:18:07.859 --> 0:18:08.550 be mean to it? 0:18:08.820 --> 0:18:09.980 That's the other end of the spectrum. 0:18:10.030 --> 0:18:11.910 I wouldn't want to do that because I think that 0:18:11.970 --> 0:18:14.929 is a travesty on my own well-being. I mean, and 0:18:14.950 --> 0:18:16.730 we shouldn't do that because we don't want to create, 0:18:17.250 --> 0:18:22.359 you know, private spaces for simulating terrible behavior among humans. 0:18:22.400 --> 0:18:25.220 Like that would be also a terrible outcome, right? But 0:18:25.280 --> 0:18:28.020 not because Claude is suffering because I'm calling it names. 0:18:28.580 --> 0:18:32.880 But are you going to train Microsoft AI on your 0:18:32.960 --> 0:18:37.530 humanistic code of conduct just as Claude may be trained 0:18:37.609 --> 0:18:38.970 on its constitution? 0:18:39.490 --> 0:18:40.130 Yeah. 0:18:40.150 --> 0:18:43.570 So we've released a code of conduct for public consultation, 0:18:43.609 --> 0:18:45.270 which we did four or five weeks ago, and it's 0:18:45.310 --> 0:18:46.669 still open for consultation. 0:18:46.730 --> 0:18:48.470 It's on our website, microsoft.ai. 0:18:49.050 --> 0:18:53.820 It's a 40-page document, and it's the first pass, our 0:18:53.840 --> 0:18:57.600 first version of a training manual for the values and 0:18:57.619 --> 0:18:59.320 behaviors of the MAI models. 0:18:59.800 --> 0:19:03.740 And is there a way that your fundamental disagreement with 0:19:03.880 --> 0:19:07.129 anthropic isn't so much of consciousness, which I don't think 0:19:07.170 --> 0:19:09.730 we'll get to the bottom, it's How dangerous is it 0:19:09.910 --> 0:19:13.870 to have that imbued into the model? Is there a 0:19:14.310 --> 0:19:17.530 way one could test that where you could test each 0:19:17.609 --> 0:19:20.430 other's models and say, here, this is why you're dangerous? 0:19:21.250 --> 0:19:23.909 I mean, I think obviously the question of whether it 0:19:24.080 --> 0:19:28.180 is or isn't conscious, you know, ultimately does matter a lot. 0:19:28.260 --> 0:19:31.480 But really what matters more is what does it mean 0:19:31.520 --> 0:19:36.110 for humans? Have humans consented? to bring into the world 0:19:36.130 --> 0:19:39.940 a species that is obviously gonna be in many ways 0:19:40.020 --> 0:19:42.960 more intelligent and more capable than all of us combined. 0:19:43.800 --> 0:19:46.350 Has humanity consented to that? At the moment, I think 0:19:46.369 --> 0:19:50.480 the categorical answer is no. And then secondly, is it 0:19:50.580 --> 0:19:54.500 more or less likely to be controllable and safe if 0:19:54.880 --> 0:19:59.660 it has these properties of introspecting about its own suffering 0:19:59.920 --> 0:20:02.649 and has this kind of self-awareness and its own consciousness? 0:20:03.050 --> 0:20:05.290 And my speculation is that I think it's going to 0:20:05.350 --> 0:20:07.750 be less safe because it is going to have to 0:20:07.830 --> 0:20:11.780 worry about its own needs and balance those off against. 0:20:11.440 --> 0:20:14.060 You know we're pushing back and i would say amanda 0:20:14.180 --> 0:20:17.350 askle who helped write the anthropic one i'd say no 0:20:17.400 --> 0:20:19.650 no if we train it to be really good and 0:20:20.190 --> 0:20:24.690 uh nice and wonderful then it's more safe for humans 0:20:24.810 --> 0:20:26.649 well i mean we we can certainly train it in 0:20:26.710 --> 0:20:28.129 any way that we want and we can still train 0:20:28.150 --> 0:20:30.649 it to be good and nice but that that doesn't 0:20:30.890 --> 0:20:33.929 require it to tell us that it has feelings and 0:20:33.950 --> 0:20:36.060 that it suffers i mean remember this is a key 0:20:36.100 --> 0:20:38.389 part of of our law and how we kind of 0:20:38.430 --> 0:20:40.950 give rights to, you know, all humans on the planet. 0:20:41.060 --> 0:20:44.480 So any change there, you know, like human cloning or 0:20:44.720 --> 0:20:47.140 Dolly the sheep, I mean, it would be a monumental 0:20:47.220 --> 0:20:50.320 moment in human history that I don't think any company 0:20:50.380 --> 0:20:52.840 has the right to unilaterally declare. 0:20:53.300 --> 0:20:56.830 Now, to get back to the question of are we 0:20:56.869 --> 0:21:00.410 going too fast in this direction, not only you, but 0:21:00.570 --> 0:21:05.090 Satya Nadella, who runs Microsoft, of which you're the AI division. 0:21:06.590 --> 0:21:10.770 This week said basically, wait a minute, let's stop. 0:21:11.369 --> 0:21:14.120 Or tell me what he said and what you feel 0:21:14.150 --> 0:21:18.700 we should do to regulate this and not barrel full 0:21:18.720 --> 0:21:19.359 speed ahead. 0:21:19.859 --> 0:21:22.920 Yeah, I mean, he said that most of these things, 0:21:23.000 --> 0:21:27.210 at least for now, are questions of governance and engineering execution. 0:21:27.330 --> 0:21:31.250 So let's just first get the basics right, which requires 0:21:31.330 --> 0:21:34.149 us to make sure that there are these verifiably safe 0:21:34.230 --> 0:21:38.750 containers which can't leak data and that don't allow an 0:21:38.910 --> 0:21:42.030 AI to take actions that they haven't been given permission 0:21:42.070 --> 0:21:44.670 to take. And as soon as we start to do that, 0:21:44.730 --> 0:21:47.320 we have to disclose those things. And if that seems 0:21:47.359 --> 0:21:51.040 to be happening consistently, then obviously that would precipitate a pause. 0:21:51.820 --> 0:21:54.920 And it's just a very common sense approach from, I think, 0:21:54.960 --> 0:21:56.659 somebody who's been doing this for a long time. 0:21:56.700 --> 0:22:01.820 And if you precipitate a pause, who can help do 0:22:01.840 --> 0:22:04.610 the guardrails? Can it be done just by the Six 0:22:04.670 --> 0:22:06.730 or seven companies? Do we have to have a government 0:22:07.170 --> 0:22:08.150 set of regulations? 0:22:08.250 --> 0:22:10.670 I think at the moment what the president has said 0:22:10.780 --> 0:22:15.280 is that if companies believe that they are building models 0:22:15.400 --> 0:22:17.320 which are unsafe, they should stop. 0:22:19.119 --> 0:22:23.330 That's not... That's where we stand currently. 0:22:24.410 --> 0:22:27.510 And all of the companies are constantly talking to each 0:22:27.590 --> 0:22:32.130 other disclosing to one another, swapping ideas about, you know, 0:22:32.410 --> 0:22:34.870 how to interpret this and when, you know, we might 0:22:35.310 --> 0:22:37.930 sort of need to collectively slow down. I think different 0:22:38.170 --> 0:22:41.729 people have different interpretations about what the harm is today. And, 0:22:42.010 --> 0:22:44.430 you know, I think that the more we do this 0:22:44.490 --> 0:22:47.880