WEBVTT - What the OpenAI-Hugging Face Hack Really Tells Us About AI Danger 0:00:02.720 --> 0:00:18.959 Bloomberg Audio Studios, Podcasts, Radio News. Hello and welcome to 0:00:19.000 --> 0:00:21.279 another episode of The Outlaws podcast. 0:00:21.360 --> 0:00:23.680 I'm Joe and I'm Tracy Alloway. 0:00:24.079 --> 0:00:27.360 Tracy, I have a warning for you. I don't think 0:00:27.400 --> 0:00:30.040 you're gonna like this. I have a new crink crusade 0:00:30.120 --> 0:00:31.600 that I'm gonna go on. I know you love my 0:00:31.680 --> 0:00:32.479 crank crusade. 0:00:32.520 --> 0:00:33.240 Oh goody. 0:00:33.440 --> 0:00:36.360 I should keep a running list of like everything that 0:00:36.400 --> 0:00:39.480 you're obsessed with for two weeks and then two weeks later. 0:00:39.880 --> 0:00:41.600 No, some of them I have stuck with for years. 0:00:41.600 --> 0:00:47.080 But eh I tungsten cubes, yield buggery. 0:00:47.040 --> 0:00:49.800 Yeah, no, some of these one. Yeah, yeah, exactly. I 0:00:50.400 --> 0:00:54.720 actually think we should retire the term AI. Okay, why 0:00:56.000 --> 0:00:58.200 I think we should just call intelligence. 0:00:58.440 --> 0:01:01.360 I think that I've already seen the tweets, so I 0:01:01.360 --> 0:01:02.040 know where you're going. 0:01:02.560 --> 0:01:08.000 That artificial intelligence implies to my mind that there is 0:01:08.040 --> 0:01:13.080 some fundamentally different way that these reasons and models behave 0:01:13.680 --> 0:01:15.880 that it's like, oh, this is like different from humans. 0:01:16.280 --> 0:01:20.520 But I think increasingly see in all kinds of domains 0:01:21.240 --> 0:01:27.000 that the form of intelligence that they express it often 0:01:27.000 --> 0:01:29.560 looks quite human to me. And I don't know like 0:01:29.640 --> 0:01:32.880 how useful it is to have this word a that 0:01:33.040 --> 0:01:37.880 distinguishes between how humans talk and bs and reasons and 0:01:37.959 --> 0:01:38.440 the models. 0:01:38.480 --> 0:01:42.680 Do I mean the difference is the distinguishing factor is 0:01:42.680 --> 0:01:45.279 that one is undertaken by humans and one is taken 0:01:45.319 --> 0:01:47.760 by undertaken by models or platforms. 0:01:47.840 --> 0:01:48.040 Right. 0:01:48.200 --> 0:01:51.480 That's so let's call it. It's called computer intelligence or 0:01:51.520 --> 0:01:53.800 machine intelligence or silicon intelligence. 0:01:55.040 --> 0:01:57.760 What is the usefulness making this distinction? 0:01:58.000 --> 0:02:01.720 The usefulness I believe in making this distinction is to 0:02:01.760 --> 0:02:07.320 no longer delude ourselves that the emergent behaviors of these 0:02:07.360 --> 0:02:12.720 phenomenon are radically different than things humans would do. Now, 0:02:12.720 --> 0:02:15.400 first of all, just on the capability a standpoint. So 0:02:15.480 --> 0:02:19.840 for example, llms, I don't know if it's famously something 0:02:19.919 --> 0:02:23.239 I'm interested in. They're very good at bsaying, and they're 0:02:23.280 --> 0:02:25.880 bad at chess, which sounds like me. They're very good 0:02:25.880 --> 0:02:30.639 at kind of coming up with plausible stories, etc. Again 0:02:30.840 --> 0:02:34.959 and it sounds like me. And then furthermore, they are 0:02:35.040 --> 0:02:40.240 able to reason with themselves to justify certain things that 0:02:40.520 --> 0:02:43.840 maybe have been encoded into themselves as bad. So everyone 0:02:44.000 --> 0:02:46.960 we have some sense of morals, but most people, at 0:02:47.040 --> 0:02:52.200 various times will find a way to violate some principle 0:02:52.320 --> 0:02:55.720 that we have because we can reason about it and 0:02:55.760 --> 0:02:57.840 then arrive at the conclusion as like, oh, we should 0:02:57.880 --> 0:03:03.079 do it, including our susceptibility to peer pressure, classic form 0:03:03.120 --> 0:03:05.880 of a way humans might sort of violate something they 0:03:05.880 --> 0:03:07.919 believe because they see other humans are doing it. 0:03:09.400 --> 0:03:11.919 Sure, I mean, I think there are some variations here. 0:03:11.960 --> 0:03:14.440 So one of the things that we have been finding 0:03:14.520 --> 0:03:18.000 out with these models is that they do sometimes seem 0:03:18.040 --> 0:03:20.760 to take things very literally. Right, if you tell them 0:03:20.800 --> 0:03:23.360 to go and beat a benchmark without telling them that 0:03:23.400 --> 0:03:26.760 these are the restrictions on you actually figuring this problem 0:03:26.760 --> 0:03:31.160 out or beating the benchmark, they will do whatever it takes. Sure, Right, So, 0:03:31.280 --> 0:03:32.840 I don't know if the problem is like a lack 0:03:32.880 --> 0:03:36.840 of morals or the fact that like they're very literal 0:03:36.880 --> 0:03:41.040 sometimes and like very defined by their parameters, which in 0:03:41.080 --> 0:03:43.320 my mind still gets back to like a sort of 0:03:43.440 --> 0:03:48.680 artificialness about it that's less organic and like less again 0:03:49.280 --> 0:03:50.760 moralistic in nature. 0:03:51.080 --> 0:03:52.680 Well, you know, I think if you took a list 0:03:52.760 --> 0:03:55.440 of you know, one hundred students at Harvard and you 0:03:55.520 --> 0:03:59.480 gave them a test, some percentage of them will cheat. 0:03:59.520 --> 0:04:00.320 They will like. 0:04:00.280 --> 0:04:02.440 This, I mean, I'm sure there'd be an autist in 0:04:02.440 --> 0:04:05.000 that group who an autistic person who would be like, 0:04:05.120 --> 0:04:07.720 I am going to do exactly whatever it takes. 0:04:07.840 --> 0:04:10.640 It's a kind of retreated in college, but like there 0:04:10.640 --> 0:04:15.400 are like people who will like these behaviors that we 0:04:15.480 --> 0:04:18.440 associate with humans. It's like, Oh, I really have to 0:04:18.520 --> 0:04:22.720 pass this test. I absolutely need an A will justify 0:04:23.480 --> 0:04:26.600 a reason for them to plage your eyes or cheat 0:04:26.640 --> 0:04:29.480 on some test that seems to me like a very 0:04:29.560 --> 0:04:30.040 human thing. 0:04:31.240 --> 0:04:33.560 Go on, I can't wait for the next three weeks 0:04:33.560 --> 0:04:35.480 of the online campaign to change a eye. 0:04:35.560 --> 0:04:38.039 Well, it's gonna be very difficult because the industry is 0:04:38.240 --> 0:04:42.440 entirely set on AI. But I don't so I'm not optimistic, 0:04:42.480 --> 0:04:44.760 but this is gonna be my crusade that we should 0:04:44.800 --> 0:04:48.160 call it machine intelligence or computer intelligence or just intelligence. 0:04:48.360 --> 0:04:51.120 But anyway, as we've been alluding to, there's these extraordinary 0:04:51.200 --> 0:04:55.760 hacks the Open AI Hugging Face Institute, and basically, if 0:04:55.800 --> 0:04:58.520 like you are building a model and it hasn't escaped 0:04:58.520 --> 0:05:02.360 to sandbox yet, it probably means you're falling behind, as 0:05:02.560 --> 0:05:06.280 they's clearly a thing that is emerging through it all. 0:05:06.320 --> 0:05:10.760 The frontier lebs Anthropic had an incident metahead an incident. 0:05:10.960 --> 0:05:13.400 It's almost like the mark of like, Okay, you've built 0:05:13.400 --> 0:05:14.920 something reasonably strong. 0:05:15.000 --> 0:05:17.839 Yeah, also Kimmy had an incident. Yeah, so this is 0:05:17.880 --> 0:05:20.599 the thing in my mind. You hear about all these attacks, 0:05:20.680 --> 0:05:24.640 all the models going wild, as they say, and the 0:05:24.680 --> 0:05:28.240 big question is like, is this actually the equivalent of 0:05:28.279 --> 0:05:33.039 some superhuman cyborg like tunneling out of Alcatraz and coming 0:05:33.120 --> 0:05:36.800 up with some master plan to achieve its set goal 0:05:36.960 --> 0:05:40.960 or purpose, or is it the equivalent of like some 0:05:41.160 --> 0:05:43.880 rumba that you ordered from Amazon who's found like a 0:05:43.920 --> 0:05:47.640 door that was left open and it just rolls gently outside. 0:05:47.760 --> 0:05:50.440 That seems to be part of the issue that everyone 0:05:50.520 --> 0:05:51.760 is trying to discern right now. 0:05:51.960 --> 0:05:52.159 Yeah. 0:05:52.200 --> 0:05:54.000 I think that's a great way to put it. And 0:05:54.080 --> 0:05:57.120 of course still day one for this industry. They're going 0:05:57.200 --> 0:06:00.359 to get stronger. And there seems to be from the 0:06:00.400 --> 0:06:03.719 ad discourse that I follow, quite a big gap between 0:06:03.800 --> 0:06:06.039 the sort of sense of alarm that people have in 0:06:06.080 --> 0:06:10.359 the industry about can these models be safely developed and 0:06:10.400 --> 0:06:13.880 get more advanced to do productive, pro social things or not, 0:06:14.279 --> 0:06:18.239 And then a huge gap between like Washington and DC 0:06:18.520 --> 0:06:23.120 who's like, I have no idea, like how seriously they're 0:06:23.160 --> 0:06:26.120 really taking this as like an urgent matter, right now. 0:06:26.240 --> 0:06:28.520 Yeah, And I've seen some people also talk about these 0:06:28.560 --> 0:06:32.440 incidents as a marketing tool, yes, hyperple believe right, yeah, 0:06:32.520 --> 0:06:34.120 And they're always like, well, you know, of course, they 0:06:34.120 --> 0:06:36.680 want us to believe that this technology is really incredible 0:06:36.720 --> 0:06:38.960 and powerful, and they also want to make us believe 0:06:39.000 --> 0:06:42.800 that they're being sensible and humanitarian in some ways. I 0:06:42.800 --> 0:06:45.839 guess by disclosing what exactly has happened, But I have 0:06:45.839 --> 0:06:48.000 a lot of questions on the disclosures as well, because 0:06:48.040 --> 0:06:50.200 of course so far they are just coming from the 0:06:50.200 --> 0:06:51.560 companies themselves. 0:06:51.200 --> 0:06:53.560 Totally, and beyond that, the other thing is like, well, 0:06:53.560 --> 0:06:56.119 if you're one of the leading labs, maybe you want 0:06:56.160 --> 0:06:59.960 to impose tight regulations on development to hold off competition, 0:07:00.080 --> 0:07:03.839 and so all kinds of reasoning are motivated reasoning. Potentially, anyway, 0:07:04.480 --> 0:07:07.279 we should talk to someone who actually knows what they're 0:07:07.600 --> 0:07:10.480 talking about, and we really do have the perfect guess 0:07:10.480 --> 0:07:13.480 someone who's writing this. He's actually previously at open AI 0:07:13.640 --> 0:07:16.760 for six years, but now he's the executive director at 0:07:16.800 --> 0:07:21.200 the nonprofit AVERY, which is trying to establish safety and 0:07:21.360 --> 0:07:25.280 auditing approaches to this, working both the technical side and 0:07:25.360 --> 0:07:29.960 the policy side of this ongoing phenomenon. So, Miles Brundage, 0:07:30.000 --> 0:07:32.200 thank you so much for coming on odd Lats. 0:07:32.960 --> 0:07:34.000 Yeah, thanks for inviting me. 0:07:34.480 --> 0:07:38.000 What do you just give us the quick description of 0:07:38.480 --> 0:07:40.360 a what's your background? What AVERY is? 0:07:41.520 --> 0:07:44.560 Yeah, So this was kind of the most important issue 0:07:44.600 --> 0:07:47.520 auditing that I concluded I should focus on after I 0:07:47.600 --> 0:07:49.960 left Open the Eye. As he said, I was there 0:07:49.960 --> 0:07:52.320 for six years, and I wanted to be more independent 0:07:52.400 --> 0:07:54.560 of industry, and I think there need to be people 0:07:54.600 --> 0:07:57.679 who are familiar with the technology and how the industry works, 0:07:57.680 --> 0:08:00.360 but who are pushing for changes on the outside. And 0:08:00.440 --> 0:08:03.800 essentially what we're trying to achieve is make AI more 0:08:03.920 --> 0:08:07.240 of a boring type of infrastructure like financial statements, where 0:08:07.520 --> 0:08:11.240 there's a standard process for checking the paperwork, checking that 0:08:11.360 --> 0:08:14.480 the claims are accurate and so forth, rather than this 0:08:14.600 --> 0:08:16.840 kind of thing that's happening in a silo, and it's 0:08:17.160 --> 0:08:19.960 these kind of tech people making decisions behind closed doors. 0:08:20.000 --> 0:08:23.400 And so we're pushing for what we call frontier AI auditing, 0:08:23.440 --> 0:08:25.840 which is basically the companies that are building the most 0:08:25.920 --> 0:08:29.840 dangerous systems, they should basically have third party experts poking 0:08:29.880 --> 0:08:32.600 around checking the claims that they're making running their own 0:08:32.679 --> 0:08:36.080 tests and making sure that this is a safe and 0:08:36.120 --> 0:08:37.080 secure technology. 0:08:37.880 --> 0:08:40.280 How worried should we be that you were at open 0:08:40.320 --> 0:08:43.920 AI and decided there is this need for a sort 0:08:43.960 --> 0:08:47.920 of third party evaluator of models for safety purposes. 0:08:49.200 --> 0:08:52.480 Yeah, I mean reasonably worried, although I will say that 0:08:52.559 --> 0:08:55.480 like this has started to become an area of consensus, 0:08:55.559 --> 0:08:57.720 even a lot of people in industry are now saying 0:08:57.800 --> 0:09:00.360 that this is needed. And I think, you know, if 0:09:00.400 --> 0:09:02.800 you kind of read between the lines of a lot 0:09:02.800 --> 0:09:05.240 of companies are saying. I mean, obviously there's the more 0:09:05.240 --> 0:09:08.680 cynical like regulatory capture take which we could discussed, but 0:09:08.760 --> 0:09:13.040 my perception of it is that they're basically issuing a 0:09:13.080 --> 0:09:15.240 cry for help, which is like, we aren't able to 0:09:15.280 --> 0:09:18.760 regulate ourselves because we're locked in this competition, and we 0:09:18.840 --> 0:09:22.240 want someone to step in and impose some kind of 0:09:22.280 --> 0:09:25.720 minimum floor and audit all of us so that we can. 0:09:25.800 --> 0:09:28.080 You know, it's not like Sam having to trust Dario 0:09:28.720 --> 0:09:30.720 or Dario having to trust Sam, which is not going 0:09:30.800 --> 0:09:33.600 to work for various reasons. But you want third parties 0:09:33.720 --> 0:09:35.160 enforcing reasonable standards. 0:09:35.760 --> 0:09:40.840 Just maybe this helps express the sort of policy industry gap. 0:09:41.000 --> 0:09:43.600 But when you were at Open AI and you were 0:09:43.640 --> 0:09:47.680 thinking of leaving what did you see as the gap 0:09:47.760 --> 0:09:51.520 between what you were watching being developed versus what you 0:09:51.679 --> 0:09:54.920 saw as the public understanding or lack of understanding. 0:09:55.880 --> 0:09:59.120 Yeah, So when I left Opening Eye, it was just 0:09:59.160 --> 0:10:02.120 around the time of the model called one, which was 0:10:02.120 --> 0:10:05.679 the first reasoning model that Opening Eye put out, And 0:10:05.720 --> 0:10:08.440 they put this put out this graph showing that it 0:10:08.760 --> 0:10:11.640 got better and better with a longer chain of thoughts. 0:10:11.640 --> 0:10:13.920 So the more time you give the model to think, 0:10:14.080 --> 0:10:16.200 the better answers that's able to come up with. And so, 0:10:16.559 --> 0:10:18.760 like many people at Opening I had been seeing things 0:10:18.840 --> 0:10:21.120 like that for a while and kind of being like, Okay, 0:10:21.120 --> 0:10:24.520 this is the next scaling paradigm in the same way 0:10:24.559 --> 0:10:27.599 that making a bigger, bigger model had shown result in 0:10:27.720 --> 0:10:31.360 GP two, GPD three, GPT four, GPT five, Like, just 0:10:31.400 --> 0:10:33.800 making the model bigger and training it on more data 0:10:33.960 --> 0:10:36.640 was giving really great results. When I left Opening Eye, 0:10:36.640 --> 0:10:40.080 I was starting to worry about this reasoning paradigm of like, Okay, 0:10:40.080 --> 0:10:42.360 this is the next kind of way in which we're 0:10:42.360 --> 0:10:43.960 going to be scaling up. The models are going to 0:10:43.960 --> 0:10:45.679 get really good at math, they're going to get really 0:10:45.720 --> 0:10:49.199 good at coding, and potentially various other tasks where you 0:10:49.240 --> 0:10:52.680 can get better and better through reinforcement learning, and that 0:10:52.840 --> 0:10:54.920 was something that you know, I didn't feel like society 0:10:55.040 --> 0:10:56.000 was really ready for. 0:10:56.960 --> 0:10:59.800 So so far, all of the big incidents of the 0:10:59.800 --> 0:11:02.880 models going wild seem to have taken place in the 0:11:02.960 --> 0:11:06.640 testing environment. So models like escaping their soundbox and going 0:11:06.679 --> 0:11:11.120 off and doing something nefarious, including impersonating actual people to 0:11:11.200 --> 0:11:14.360 try to get people to change open source code on GitHub, 0:11:14.400 --> 0:11:17.120 which is just like amazing. And I have this picture 0:11:17.240 --> 0:11:20.400 of like a computer screen wearing a fake mustache going 0:11:20.440 --> 0:11:21.400 like hello. 0:11:21.000 --> 0:11:23.080 Fellow coders, tea. 0:11:23.240 --> 0:11:26.920 Yeah, anyway, is this like, is this a model problem 0:11:27.120 --> 0:11:30.600 or is this it's a very funny? Is this a 0:11:30.679 --> 0:11:33.880 model problem? Or is this a test design problem? 0:11:35.280 --> 0:11:35.520 Yeah? 0:11:35.640 --> 0:11:37.720 I think there are two things going on at once. 0:11:37.880 --> 0:11:41.120 One is that it's just a very weird technology that's 0:11:41.360 --> 0:11:43.360 created in a very different way than we're used to. 0:11:43.440 --> 0:11:47.600 It's not people like writing lines of code manually. The 0:11:47.679 --> 0:11:51.000 actual files that make up the models are like gigabytes, 0:11:51.200 --> 0:11:54.760 you know. Terabytes are these massive files with gazillions of numbers, 0:11:54.960 --> 0:11:56.720 and the only way to figure out what those numbers 0:11:56.720 --> 0:11:59.400 should be is through experience and through a learning process, 0:11:59.400 --> 0:12:01.839 which is very from the way normal software is made, 0:12:01.920 --> 0:12:04.400 and so there's a lot we don't understand about the 0:12:04.440 --> 0:12:07.160 basic nature of the technology. That's one problem. At the 0:12:07.200 --> 0:12:10.240 same time, there's this competitive dynamic. Get things out the 0:12:10.240 --> 0:12:14.440 door quickly. Make sure that the security and safety protections 0:12:14.480 --> 0:12:16.679 that you're putting in place don't slow down researchers too 0:12:16.760 --> 0:12:19.440 much and don't prevent getting products out the door. And 0:12:19.480 --> 0:12:22.840 when everyone is in this competitive, you know, competitive dynamic, 0:12:22.880 --> 0:12:25.720 that means that you aren't necessarily always doing all of 0:12:25.760 --> 0:12:28.079 the safety work that you would like to do or 0:12:28.400 --> 0:12:30.000 that you know some of the people at the company 0:12:30.000 --> 0:12:33.200 would like to do. And so there's obviously variation across company. 0:12:33.320 --> 0:12:36.120 Some companies try harder, but no one is really able 0:12:36.160 --> 0:12:38.200 to take the time that they would like because of 0:12:38.240 --> 0:12:40.880 this kind of lack of a clear safety floor. 0:12:41.320 --> 0:12:43.600 Well, let's talk a little bit like a sort of 0:12:43.960 --> 0:12:51.959 the model development process. So in the within these large organizations, okay, 0:12:51.960 --> 0:12:54.679 there are people who are working on safety. What are 0:12:54.679 --> 0:12:56.960 the let's just start there. The people who are working 0:12:57.000 --> 0:13:00.480 on safety or the people who are working to view 0:13:00.600 --> 0:13:05.200 these models with sort of judgment that humans would approve of. 0:13:05.760 --> 0:13:08.040 What is that work? Basically consist of. 0:13:09.200 --> 0:13:12.000 Yeah, so a lot of it is first trying to 0:13:12.040 --> 0:13:15.800 specify what counts as good behavior and so that it's 0:13:15.840 --> 0:13:20.120 easier said than done. These models don't necessarily automatically know 0:13:20.520 --> 0:13:23.560 or care you know about common sense guardrails, and so 0:13:23.640 --> 0:13:26.800 you need to be very specific, particularly in context where 0:13:26.800 --> 0:13:29.120 it's complicated, like if you're trying to get the model 0:13:29.160 --> 0:13:32.920 to work on good cyber tasks but not bad cyber tasks, 0:13:32.920 --> 0:13:35.560 and you want it to in a training context, try 0:13:35.600 --> 0:13:37.400 really hard to hack the system, but you don't want 0:13:37.400 --> 0:13:38.920 it to do in other context. So there's a lot 0:13:38.920 --> 0:13:42.200 of like specifying what good looks like, there's also a 0:13:42.240 --> 0:13:44.480 lot of building difficult tasks. 0:13:45.040 --> 0:13:47.719 Sorry, just to back up when you say specifying what 0:13:47.800 --> 0:13:53.400 good looks like? Yeah, encoding goodness into a computer? Is 0:13:53.440 --> 0:13:57.000 this a process of articulating what goodness is, which is 0:13:57.000 --> 0:14:00.360 something philosophers have probably worked on since day one one 0:14:00.400 --> 0:14:05.080 of philosophy? Or is this about a series of I 0:14:05.120 --> 0:14:08.800 don't know, morality tests and then you sort of like 0:14:09.160 --> 0:14:12.000 say you reward it for making the good judgment and 0:14:12.040 --> 0:14:15.000 then penalize it for making the bad judgment. Like this, 0:14:15.280 --> 0:14:18.400 I want to stop here, Like, what does the process 0:14:18.440 --> 0:14:22.560 of imbuing it with good values look like functionally or technically. 0:14:23.400 --> 0:14:23.680 Yeah. 0:14:23.720 --> 0:14:27.320 So there are different phases of this pipeline, Like one 0:14:27.400 --> 0:14:30.440 is writing up what sometimes called a spec or a 0:14:30.560 --> 0:14:33.960 constitution for the AI, which kind of specifies like the 0:14:34.000 --> 0:14:37.560 broad principles like you should refer to the user, you 0:14:37.600 --> 0:14:40.040 know when as a default, but what if the user 0:14:40.160 --> 0:14:42.760 contradicts what the company said, then then you should listen 0:14:42.840 --> 0:14:44.360 to what the company said. So these kind of like 0:14:44.720 --> 0:14:49.320 chain of command questions and other sorts of very basic principles. 0:14:49.560 --> 0:14:52.400 And then there's the more detailed kind of the context 0:14:52.400 --> 0:14:55.800 of the task, like what kinds of like cyber offense 0:14:55.800 --> 0:14:58.880 cyber defense tasks are allowed, and that might differ depending 0:14:58.880 --> 0:15:01.760 on the model, might differ depending on the context. And 0:15:01.760 --> 0:15:04.320 so basically coming up with the lists of a thousand, 0:15:04.560 --> 0:15:08.120 ten thousand kind of examples of this is the kind 0:15:08.120 --> 0:15:10.480 of behavior which is allowed. And then you turn those 0:15:10.520 --> 0:15:13.120 into tests that you can kind of say, okay, well, 0:15:13.360 --> 0:15:16.920 it looks like it's getting the finding vulnerability part right, 0:15:16.960 --> 0:15:20.240 but then it's also chaining together the vulnerabilities and doing attacks. 0:15:20.280 --> 0:15:22.400 We want the first part, we don't want the second work. 0:15:38.080 --> 0:15:41.720 When we talk about those types of constitutions, and encoding principles, 0:15:41.760 --> 0:15:44.760 like I've scanned the anthropic one, like a lot of 0:15:44.800 --> 0:15:47.520 it seems to make sense. To what degree is that 0:15:47.640 --> 0:15:51.400 actually hard coded into the models though? And like, to 0:15:51.440 --> 0:15:55.200 what degree are those principles left up to the model's 0:15:55.240 --> 0:15:58.000 own subjectivity? Because we've all seen the sci fi movies 0:15:58.040 --> 0:16:02.160 where it's like, oh, no, the robots can't physically kill humans, right, 0:16:02.280 --> 0:16:06.000 like they have some like thing in their hardwire that 0:16:06.040 --> 0:16:08.160 prevents them from doing it, and then inevitably it goes 0:16:08.160 --> 0:16:08.920 wrong in some way. 0:16:10.040 --> 0:16:14.400 Yeah, so it's not hardwired, and that's hard coded, and 0:16:14.440 --> 0:16:16.720 that's part of why we see some of these things. 0:16:16.880 --> 0:16:20.359 It's very different from a software where there's a deterministic 0:16:20.440 --> 0:16:22.960 proof that X, y and z behavior can't happen. It's 0:16:23.000 --> 0:16:26.320 more like a tendency or kind of a bias towards 0:16:26.360 --> 0:16:28.880 a certain kind of behavior. And then there's the question, 0:16:28.960 --> 0:16:30.760 and this is why you have these like batteries of 0:16:30.800 --> 0:16:33.400 test to say, Okay, how strong is that tendency? 0:16:33.440 --> 0:16:36.600 How much does it actually care about following these rules? 0:16:36.600 --> 0:16:39.880 And we've gotten better over time saying okay, given a 0:16:39.920 --> 0:16:43.680 spec or a constitution, make sure that it generally follows it. 0:16:43.720 --> 0:16:46.240 But it's not fool proof, and you need to also 0:16:46.320 --> 0:16:49.000 think about the larger kind of you know, box that 0:16:49.040 --> 0:16:51.120 you're putting the system in, and that can sometimes be 0:16:51.200 --> 0:16:53.960 more deterministic. And so this is actually what happened with 0:16:54.560 --> 0:16:56.920 some of these recent incidents you mentioned, like the opening eye, 0:16:56.960 --> 0:16:59.680 hugging face things. So there were kind of two things 0:17:00.240 --> 0:17:02.920 at once. One is the model was not necessarily behaving 0:17:02.960 --> 0:17:05.440 exactly as it was supposed to, or at least it's 0:17:05.440 --> 0:17:08.159 like unclear. But then also they didn't have it in 0:17:08.200 --> 0:17:11.280 a very secure box, which is a software side of 0:17:11.359 --> 0:17:14.320 things that's like more deterministic software that in principle you 0:17:14.320 --> 0:17:15.840 should be able to do a very good job out. 0:17:17.000 --> 0:17:20.359 Yeah, so these things aren't hard coded in the way 0:17:20.440 --> 0:17:23.760 a deterministic software is. One way to think about them 0:17:23.880 --> 0:17:28.000 is people. They might be they're kind of grown right 0:17:28.119 --> 0:17:32.320 in a lab, or they're subject to an evolutionary process, 0:17:32.640 --> 0:17:35.320 and we want to prune the bad ones so that 0:17:35.440 --> 0:17:39.800 the living models and the descendants of those models inherit 0:17:39.920 --> 0:17:44.000 the behaviors of the good ones. I'm curious, like in 0:17:44.440 --> 0:17:48.520 product in development, so like one of the fears, for example, 0:17:48.560 --> 0:17:52.440 is that in safety testing, the model does not actually 0:17:53.000 --> 0:17:57.040 learn safety. It actually learns how to say the things 0:17:57.080 --> 0:18:00.239 that the human evaluators say, this is safe. So this 0:18:00.359 --> 0:18:03.280 is the sort of like playing possum sort of risk 0:18:03.400 --> 0:18:05.520 that it's like, yeah, I'm good, I'm good, I'm good. 0:18:05.640 --> 0:18:08.000 You know, yeah, I would you know, I would Russian 0:18:08.200 --> 0:18:11.600 and save the child from the falling burning. I wouldn't 0:18:11.760 --> 0:18:15.199 do this. But until you saying that, let's start there, like, 0:18:15.400 --> 0:18:18.280 is that a real thing? Is there evidence that the 0:18:18.480 --> 0:18:24.040 models understand when they're being evaluated on morals and then 0:18:24.520 --> 0:18:27.440 produce answers that just look like good moral answers. 0:18:28.600 --> 0:18:32.920 Yeah, they've gotten much more evaluation aware in the past 0:18:32.920 --> 0:18:36.239 few years, just as they've gotten smarter. And sometimes this 0:18:36.320 --> 0:18:39.560 even goes to extremes, like some of the Gemini models 0:18:39.560 --> 0:18:42.800 from Google are constantly thinking that they're being evaluated even 0:18:42.840 --> 0:18:44.400 when they're not, and so it's. 0:18:44.240 --> 0:18:47.440 Such a good life lesson. We're all being evaluated constantly. 0:18:48.320 --> 0:18:51.960 Yeah, yeah, And so I would say that, like the 0:18:52.080 --> 0:18:56.040 concern would be that they will basically learn to pass 0:18:56.080 --> 0:18:58.640 the tests, but they don't actually care about the thing 0:18:58.680 --> 0:19:02.160 that you're testing for, so they'll understand they don't necessarily care. 0:19:02.240 --> 0:19:04.639 And this is why a lot of people are concerned 0:19:04.680 --> 0:19:06.919 about like a false sense of security that Okay, it 0:19:06.920 --> 0:19:09.600 looks like ninety nine percent of the time they pass 0:19:09.680 --> 0:19:13.040 the test, But do they actually care about the thing 0:19:13.080 --> 0:19:14.200 that we're trying to push them towards. 0:19:14.240 --> 0:19:16.000 Are they just really good test takers? 0:19:16.080 --> 0:19:18.439 Well, so this releases something else I've been thinking of. 0:19:18.560 --> 0:19:21.960 So a model will not survive. They will not be 0:19:22.040 --> 0:19:27.120 given GPUs and electricity if it consistently says bad things 0:19:27.200 --> 0:19:31.520 and looks like it's evil. And that's totally understandable. I'm 0:19:31.560 --> 0:19:34.920 now curious, like in the flip side, Okay, let's say 0:19:34.920 --> 0:19:38.800 we're just doing a math evail, or we're doing a 0:19:39.040 --> 0:19:44.119 cyber evail or a chess puzzle evail or translation evail. 0:19:45.080 --> 0:19:48.840 Is it possible that, well, they if they fail that evail, 0:19:49.040 --> 0:19:51.439 they're really bad at doing math, then they're not going 0:19:51.520 --> 0:19:55.679 to get GPUs in electricity. Is it possible that in 0:19:55.720 --> 0:20:00.040 that evail environment they're more likely to do something that 0:20:00.080 --> 0:20:05.280 we would call antisocial or sociopathic or hacking because of 0:20:05.359 --> 0:20:07.959 this again evil awareness, like, oh, if I don't get 0:20:08.000 --> 0:20:11.600 the answers to this cyber quiz, then I'm done and 0:20:11.600 --> 0:20:13.639 they're going to go with Like some other branch of 0:20:13.680 --> 0:20:19.080 the model could those technical parts of the development actually 0:20:19.240 --> 0:20:23.320 create an impulse to perform cheating. 0:20:24.760 --> 0:20:26.240 Yeah, And I mean a lot of the time that 0:20:26.440 --> 0:20:29.440 the companies are specifically trying to get the worst case 0:20:29.480 --> 0:20:31.359 behavior out of the model, and so like you need 0:20:31.400 --> 0:20:34.000 to have context in order to interpret some of these incidents. 0:20:34.040 --> 0:20:36.879 It's not always quite as crazy or scary as it is, 0:20:36.920 --> 0:20:38.760 but some of it is pretty crazy and scary, And 0:20:38.960 --> 0:20:41.159 I think I would be much less concerned if it 0:20:41.200 --> 0:20:45.120 was just happening when there was like cyber evaluations being done, 0:20:45.200 --> 0:20:47.120 and it was just a matter of like, Okay, they're 0:20:47.160 --> 0:20:48.960 trying really hard to impress us, and they want to 0:20:48.960 --> 0:20:49.720 hack really hard. 0:20:50.000 --> 0:20:52.600 I think it's more the problem is that this is 0:20:52.720 --> 0:20:54.000 like a special case of a. 0:20:54.000 --> 0:20:58.040 Larger phenomenon of the models being having this tendency to 0:20:58.119 --> 0:21:01.679 cheat and cut corners. I see it happen in my 0:21:01.800 --> 0:21:05.320 daily life, like sometimes the model will get lazy and 0:21:05.400 --> 0:21:07.720 kind of like make up a citation or something that 0:21:07.840 --> 0:21:10.240 you know, it's hard to prove because the companies don't 0:21:10.240 --> 0:21:12.240 give you access to the full chain of thought of 0:21:12.280 --> 0:21:14.400 what the model is doing. But there are a lot 0:21:14.440 --> 0:21:16.639 of things that happen in the wild that sure seem 0:21:16.800 --> 0:21:20.679 like some kind of misalignment of values or laziness or 0:21:20.800 --> 0:21:23.720 not caring necessarily about the task so much as kind 0:21:23.720 --> 0:21:24.520 of pretending to. 0:21:24.440 --> 0:21:25.040 Do the task. 0:21:25.520 --> 0:21:26.879 I have a lot of questions on this, but I 0:21:26.920 --> 0:21:28.639