WEBVTT - Did the AI Catastrophe Just Happen? - Week in Tech 0:00:12.119 --> 0:00:14.040 Natasha read Taylor. 0:00:14.440 --> 0:00:17.400 I'm not sure how much you think about students using 0:00:17.920 --> 0:00:21.280 AI to cheat, or at least to get an edge 0:00:21.280 --> 0:00:24.280 in the classroom, but there was some interesting live research 0:00:24.360 --> 0:00:28.920 that emerged recently from Brown University. An economics professor made 0:00:29.000 --> 0:00:32.640 his students take a final exam in person after suspecting 0:00:33.000 --> 0:00:35.839 rampant cheating on the take home midterm. I want to 0:00:35.840 --> 0:00:38.440 ask you to guess the delta between the average score 0:00:38.479 --> 0:00:40.879 on the midterm and the average score on the final. 0:00:41.720 --> 0:00:44.120 Well, I saw this story, and it was a massive delta. 0:00:44.280 --> 0:00:45.240 Me too, Me too. 0:00:45.320 --> 0:00:47.240 I gotta say fifty percent. 0:00:47.320 --> 0:00:47.879 I didn't see it. 0:00:47.960 --> 0:00:50.440 The average score on the midterm was ninety six percent, 0:00:51.040 --> 0:00:54.280 the average score on the final forty eight point six percent. 0:00:54.640 --> 0:00:56.920 But there was that one kid who like, actually did 0:00:57.200 --> 0:00:59.080 he like got an F and then it went to 0:00:59.160 --> 0:01:00.440 like a D or something like that. 0:01:05.680 --> 0:01:06.679 Welcome to Tech stuff. 0:01:06.760 --> 0:01:09.800 I'ma's Volosian and this is the Week in Tech where 0:01:09.800 --> 0:01:11.920 I'm joined by three of the world's most clubbed and 0:01:11.959 --> 0:01:15.840 reporters to break down what's really happening in tech right now. Today, 0:01:15.959 --> 0:01:19.480 we're joined by reed Albrigotti, Take editor a Semaphore, Taylor 0:01:19.520 --> 0:01:22.800 Lorenz of user mag and Natasha Tiku, tech reporter at 0:01:22.800 --> 0:01:24.200 The Washington Post's welcome all. 0:01:24.600 --> 0:01:26.720 Hey everyone, thanks for having us. 0:01:26.760 --> 0:01:27.400 Good to be here. 0:01:28.040 --> 0:01:28.720 So Natasha. 0:01:29.400 --> 0:01:32.520 Nick Thompson, the former editor of Wired and current CEO 0:01:32.560 --> 0:01:35.600 of The Atlantic, came on tech Stuff in early January 0:01:35.600 --> 0:01:38.160 this year to give his predictions for the year, and 0:01:38.200 --> 0:01:39.679 one of them, the top one, was that this will 0:01:39.720 --> 0:01:44.800 be the year of the first legit AI catastrophe. Do 0:01:44.880 --> 0:01:46.560 you think this was the week when it happened? 0:01:47.760 --> 0:01:52.480 No, I don't, But I think this is one of 0:01:52.720 --> 0:01:56.520 many weeks in which we got a pretty clear picture 0:01:56.680 --> 0:01:59.480 of what the next few years are going to look like, 0:01:59.560 --> 0:02:04.360 which is messy, disastrous. Everybody taking incidents to try to 0:02:04.520 --> 0:02:08.120 support their own narrative about where AI capabilities are headed, 0:02:08.200 --> 0:02:10.760 and you know, how to protect ourselves. 0:02:11.040 --> 0:02:11.959 So what actually happened? 0:02:12.680 --> 0:02:14.960 So I mean this this is a really good one 0:02:14.960 --> 0:02:18.880 to dig into. I think because last week Hugging Face, 0:02:18.919 --> 0:02:20.919 which is kind of like get hub but for AI, 0:02:21.080 --> 0:02:24.600 like they have data sets, models, it's very open source, 0:02:25.639 --> 0:02:28.240 funded by all the usual vcs. They put up this 0:02:28.320 --> 0:02:32.080 blog post that they had this unprecedented security incident that 0:02:32.160 --> 0:02:36.160 for the first time, an AI agent had broken into 0:02:36.160 --> 0:02:39.079 their system and tried all of these methods to get 0:02:39.160 --> 0:02:41.720 access to some data sets, and it did it by 0:02:41.960 --> 0:02:45.680 uploading like a malicious data set that would allow it 0:02:45.720 --> 0:02:48.040 to kind of like get into the system through their 0:02:48.120 --> 0:02:49.920 data processing pipeline and then. 0:02:50.080 --> 0:02:51.440 Like a trojan horses. 0:02:51.480 --> 0:02:56.280 It were yeah, yeah, And at the time, Hugging Face said, 0:02:56.400 --> 0:02:58.400 you know, we think that this looks like an agent 0:02:58.440 --> 0:03:00.840 that's used for security and RESO, but we don't know 0:03:00.880 --> 0:03:03.600 what LLLM it is. We don't know what model. So people, 0:03:03.960 --> 0:03:05.960 you know, obviously this is coming in the middle of 0:03:06.000 --> 0:03:09.720 all of this talk about Chinese open source models, how 0:03:09.720 --> 0:03:13.400 they're proliferating, how they're getting more capable, so nobody knows, 0:03:13.520 --> 0:03:17.360 you know, what's happening. Then this week on Tuesday, open 0:03:17.400 --> 0:03:20.399 Ai puts out a blog post and they're like, oh, hey, 0:03:20.440 --> 0:03:23.720 that was us, and they said, you know, they discovered 0:03:23.760 --> 0:03:29.040 after the fact that while they were testing both GPT 0:03:30.120 --> 0:03:33.400 five point six Soul and some new unreleased model, they 0:03:33.400 --> 0:03:35.760 were testing it on its ability to do these like 0:03:35.840 --> 0:03:40.640 cyber offensive attacks, and they gave it this benchmark called 0:03:40.760 --> 0:03:43.960 exploit benchmark that's supposed to test its ability to like 0:03:44.040 --> 0:03:48.400 break into, you know, to hack into systems. The AI agent, 0:03:48.480 --> 0:03:51.560 which was allowed to work you know, autonomously for a while, 0:03:52.800 --> 0:03:56.600 decided to cheat on the test and figured a way 0:03:56.760 --> 0:03:59.920 out of the sandbox that open ai created, like based 0:04:00.280 --> 0:04:04.040 you know, testing environment, got into the Internet, decided to 0:04:04.120 --> 0:04:08.840 break into hugging Face. And open ai is so lucky. 0:04:08.880 --> 0:04:12.680 They are so lucky that it was another AI company 0:04:12.680 --> 0:04:16.080 that has invested in in the industry and the narrative 0:04:16.200 --> 0:04:18.920 because this is I mean, even the the CEO of 0:04:19.000 --> 0:04:21.240 hugging Face was like, yeah, don't do this to us. 0:04:21.360 --> 0:04:24.839 It's illegal. You know, they had notified law enforcement. And 0:04:24.880 --> 0:04:27.520 now the announcement that it was open ai came as 0:04:27.560 --> 0:04:31.040 part of a partnership between hugging Face and open Ai, 0:04:31.440 --> 0:04:34.599 which was obviously you know, put together after the fact. 0:04:34.920 --> 0:04:37.640 So the model had essentially determined that it would be 0:04:37.680 --> 0:04:40.119 easier to go and steal the answer from hugging Face 0:04:40.160 --> 0:04:43.320 than to come up with it itself, and therefore did that. 0:04:44.279 --> 0:04:47.440 Yeah, it knew that it was being tested on a benchmark. 0:04:48.080 --> 0:04:51.280 I think they even maybe you know, uploaded the benchmark 0:04:51.320 --> 0:04:54.480 from hugging Face and it figured out it was supposed 0:04:54.520 --> 0:04:56.440 to not be able to access the Internet. 0:04:58.160 --> 0:05:00.440 But that's a little I mean that That's the part 0:05:00.440 --> 0:05:02.720 why Zone zoned in on the word catastrophe, because the 0:05:02.760 --> 0:05:04.360 bit where is not supposed to be able to access 0:05:04.400 --> 0:05:06.400 the Internet where where, but it does. I mean that's 0:05:06.480 --> 0:05:09.640 got a kind of cinematically spooky quality. 0:05:10.000 --> 0:05:14.000 Well, we failed to secure the sandbox, right, Yeah, exactly. 0:05:13.760 --> 0:05:17.279 Failed to secure the soundbox. Premagine preschool is hearing that 0:05:17.320 --> 0:05:18.040 in the nineties. 0:05:18.400 --> 0:05:19.040 What are you doing? 0:05:22.880 --> 0:05:27.840 Yeah, you zeroed in on exactly the right part of 0:05:27.920 --> 0:05:30.680 the confusion around this and why people are using it 0:05:30.720 --> 0:05:34.480 to like support you know, various narratives. You know, people 0:05:34.520 --> 0:05:38.760 who are very worried about AI super intelligence and AI 0:05:38.839 --> 0:05:42.600 capabilities growing really fast. You know, we're talking about this 0:05:42.760 --> 0:05:44.800 as like this is what we've been warning you about. 0:05:44.880 --> 0:05:47.599 It's a rogue AI. You know, we gave it a 0:05:47.680 --> 0:05:50.880 simple set of instructions and it decided to do it 0:05:50.920 --> 0:05:53.200 in a way that you know, was not was not 0:05:53.320 --> 0:05:57.680 what its developers would want. But many other cybersecurity professionals 0:05:57.680 --> 0:06:02.160 were like, why did you not have a properly configured sandbox, 0:06:02.240 --> 0:06:05.240 Like you could be using an air gap, which would 0:06:05.279 --> 0:06:08.560 not have allowed it to break into an Internet system. 0:06:08.960 --> 0:06:13.240 They allowed it to use like package installs, and I 0:06:13.279 --> 0:06:15.960 think just a lot of cybersecurity professionals found this like 0:06:16.080 --> 0:06:18.200 very sloppy. This is not you know, so it's not 0:06:18.600 --> 0:06:20.679 it's not as though, oh my god, we can't contain 0:06:20.720 --> 0:06:23.599 AI and it's doing some you know, like you know, 0:06:23.680 --> 0:06:26.400 people were throwing around the word sentient machines and that's 0:06:26.640 --> 0:06:27.440 not what this is. 0:06:27.520 --> 0:06:30.640 Well, on top of that that the model had no safeguards, right, 0:06:30.680 --> 0:06:33.000 there were no you know, it was a really raw, 0:06:33.400 --> 0:06:35.040 you know model, So. 0:06:35.560 --> 0:06:39.680 Yeah, they were testing its ability to to do these 0:06:39.760 --> 0:06:42.159 kind of exploits, so they didn't have any safeguards on. 0:06:42.240 --> 0:06:46.720 But then they just didn't have like traditional cybersecurity controls 0:06:47.040 --> 0:06:48.919 that you would have in place when you're testing. And 0:06:48.960 --> 0:06:52.039 this is something people had been warning against that maybe 0:06:52.040 --> 0:06:55.360 the biggest danger is not when your systems are in deployment, 0:06:55.400 --> 0:06:57.920 because then you have all your safeguards up. It's refusing 0:06:57.920 --> 0:07:00.720 a lot of requests, but it's actually during this kind 0:07:00.760 --> 0:07:03.600 of testing environment. And I'll just say one more thing. 0:07:03.720 --> 0:07:08.440 This incident followed another blog post last week from open 0:07:08.480 --> 0:07:11.960 ai about how they had to stop even just testing 0:07:12.040 --> 0:07:15.760 an internal model because it like they weren't monitoring it 0:07:15.800 --> 0:07:18.280 and it started doing all of these things because they 0:07:18.400 --> 0:07:20.520 let it go on and on, because that's you know, 0:07:20.560 --> 0:07:23.080 that's how these models get more functional. They're able to 0:07:23.160 --> 0:07:26.440 like think and try thousands and thousands of different ways 0:07:26.480 --> 0:07:27.560 of breaking into things. 0:07:27.760 --> 0:07:30.360 Well, I think that I like agree with you, Natasha 0:07:30.400 --> 0:07:33.120 that it's not this isn't like some you know, code 0:07:33.200 --> 0:07:37.600 read catastrophe at all. But I do think there's something 0:07:37.640 --> 0:07:40.520 here that is sort of like you know this, The 0:07:40.640 --> 0:07:43.080 AI security people do have a point on this. It's 0:07:43.080 --> 0:07:45.080 a point they've been making for a long time, which 0:07:45.120 --> 0:07:48.360 is that like, when you tell an AI model to 0:07:48.480 --> 0:07:52.760 do something, it will, like the smarter they are, the 0:07:52.800 --> 0:07:55.520 more they're just going to find some shortcut to do that. Right, 0:07:55.600 --> 0:07:57.480 this is the whole argument. But this is the argument 0:07:57.560 --> 0:07:59.880 with like the you know, the paper clip argument, Right, 0:08:00.080 --> 0:08:02.000 if you tell AI to make paper clips, it will 0:08:02.040 --> 0:08:04.680 just ultimately like find the best way to do that, 0:08:04.760 --> 0:08:07.239 even if that means like turning humans into paper clips. 0:08:07.280 --> 0:08:09.920 So but also but also read, I think that that 0:08:10.040 --> 0:08:12.960 is a is a sort of ridiculous concept because any 0:08:13.000 --> 0:08:16.880 AI that's smart enough in that way would also have 0:08:16.960 --> 0:08:20.360 the reasoning to know, like they're not going to turn 0:08:20.360 --> 0:08:22.320 the world into paper clips. That's not the goal. Like, 0:08:22.480 --> 0:08:24.560 I don't know, I think a lot of this like 0:08:24.640 --> 0:08:26.240 kind of like what Natasha was saying before is like 0:08:26.240 --> 0:08:28.800 a little it's being taken by these people. It's like 0:08:29.000 --> 0:08:32.840 they made a sloppy security environment, the AI went and 0:08:32.880 --> 0:08:36.320 solved the problem as it should have or whatever, And 0:08:36.320 --> 0:08:39.520 and that's not evidence of it being super sentient or 0:08:39.520 --> 0:08:43.640 even this being a particularly good AI agent. Mostly it's 0:08:43.679 --> 0:08:46.880 just that like they didn't set up this test very responsibly. 0:08:47.520 --> 0:08:49.920 Definitely agree, I agree with but again, like I'm not 0:08:49.920 --> 0:08:51.680 disagreeing with you at all. I just think there is 0:08:51.720 --> 0:08:54.040 one sliver of this that is worth sort of looking at, 0:08:54.040 --> 0:08:56.400 which is like, you know, they do sort of find 0:08:56.440 --> 0:08:59.360 these shortcuts, so and in this case, the shortcut meant 0:08:59.360 --> 0:09:04.480 like breaking the law and hacking into hugging face, so 0:09:04.600 --> 0:09:06.880 I and I think the other thing to remember is 0:09:06.880 --> 0:09:10.120 that like we still don't know how these models work, 0:09:10.480 --> 0:09:12.280 Like no one has been able to figure out like 0:09:12.360 --> 0:09:16.280 what are these neurons inside these models actually doing? And 0:09:16.640 --> 0:09:19.320 I do I think the big question is like, as 0:09:19.360 --> 0:09:23.640 they get more and more capable and more powerful, is 0:09:23.679 --> 0:09:26.080 it is it enough to just put you know, the 0:09:26.120 --> 0:09:29.200 safeguards in place, which again we're not like they didn't 0:09:29.240 --> 0:09:31.959 what I'm curious about because this this this Natasha, you 0:09:32.040 --> 0:09:34.800 mentioned there were two open Ami models of working on this. 0:09:34.840 --> 0:09:37.400 One was a publicly available one and one has not 0:09:37.480 --> 0:09:40.760 yet been released. So could could I Well, probably not me, 0:09:40.840 --> 0:09:42.679 but like could somebody has a bit more sophisticated than 0:09:42.720 --> 0:09:45.960 me use open Ami to hugging face exactly like this 0:09:46.040 --> 0:09:49.520 went down like using the publicly available models. Or was 0:09:49.559 --> 0:09:53.000 there something special about this unreleased model? Well, no, because 0:09:53.000 --> 0:09:56.000 there were no safeguards like the publicly released one will 0:09:56.040 --> 0:09:58.800 have will have controls on it that will not let 0:09:58.880 --> 0:10:01.280 you do this unless you can jail break it, which 0:10:01.559 --> 0:10:04.319 you know is getting more difficult to do, right. 0:10:04.800 --> 0:10:06.959 Okay, I will say, I mean, I do think it's 0:10:07.080 --> 0:10:09.640 important to note that this is like many people are 0:10:09.679 --> 0:10:12.480 interpreting this as like we were right about the paper 0:10:12.480 --> 0:10:15.000 clip maximizer. You know, this is happening in this way. 0:10:15.040 --> 0:10:18.040 But I think there's a way that because they conceived 0:10:18.080 --> 0:10:21.120 of it, of the problem this way, they approached security 0:10:21.240 --> 0:10:24.120 in a certain way, like you could have been thinking 0:10:24.360 --> 0:10:27.760 like a cybersecurity professional the whole time and not thinking 0:10:27.840 --> 0:10:32.920 about AI alignment, aligning it with with human values. And 0:10:33.080 --> 0:10:35.400 one of the things I mentioned that like prior open 0:10:35.440 --> 0:10:38.880 ai blog post. What they realized is that they didn't 0:10:38.880 --> 0:10:42.120 have sufficient monitoring systems. When you let a system go 0:10:42.280 --> 0:10:44.800 on and on for a really long time, they weren't 0:10:45.160 --> 0:10:49.640 watching it closely, like there are simple mechanisms, you know. 0:10:49.880 --> 0:10:50.600 It doesn't even go. 0:10:50.640 --> 0:10:53.439 Back to what Reid said about knowing what's happening on 0:10:53.480 --> 0:10:56.280 the neurons, Like they weren't even watching what it was 0:10:56.320 --> 0:10:59.160 doing during the security test in a secure way. 0:10:59.400 --> 0:11:02.960 Ax out there saying that saying maybe open Aye wanted 0:11:03.040 --> 0:11:05.400 to demonstrate anything mythos can do, we can do better, 0:11:05.520 --> 0:11:08.120 or is that it's great marketing for opening I. 0:11:08.640 --> 0:11:10.480 For sure, I think the way that they wrote the 0:11:10.520 --> 0:11:12.719 blog post at the top, it says like we are 0:11:12.760 --> 0:11:17.679 treating this as an unprecedented cybersecurity incident that shows the capabilities. 0:11:17.720 --> 0:11:20.480 And I will say, like, I don't think anyone's arguing 0:11:21.000 --> 0:11:25.440 that these lms make people much much much better at 0:11:25.520 --> 0:11:30.400 hacking and that it's capable of improving and enhancing and trying, 0:11:31.400 --> 0:11:34.960 you know, cyber offensives that humans can't do. But everyone 0:11:35.040 --> 0:11:37.120 I talked to was like, this is not an example 0:11:37.160 --> 0:11:40.920 of enhanced capabilities. This is an example of like kind 0:11:40.920 --> 0:11:42.200 of a sloppy testing environment. 0:11:42.280 --> 0:11:43.880 I mean, we all agree on that. I think it's 0:11:43.920 --> 0:11:47.040 just it just sort of highlights this question about the future, 0:11:47.080 --> 0:11:49.280 which is like, when these models become more powerful, do 0:11:49.320 --> 0:11:52.160 we need do we actually have to understand how they work? 0:11:52.559 --> 0:11:56.560 Has this has this effected the kind of policy discussions 0:11:56.559 --> 0:11:59.280 in Washington? What's how's the White House and crapsios and 0:11:59.320 --> 0:12:02.480 others around around Trump and the sort of technology advisory 0:12:03.040 --> 0:12:05.319 space reacted to this? Is this is this like grist 0:12:05.360 --> 0:12:07.840 to the mill of the of the deceleration lists or 0:12:07.920 --> 0:12:08.160 is it? 0:12:09.080 --> 0:12:12.240 I would say they were very busy with They're still 0:12:12.320 --> 0:12:14.200 very busy with Chinese open source models. 0:12:14.480 --> 0:12:15.480 So that was the focus. 0:12:15.559 --> 0:12:18.079 This didn't I don't think it got the attention of 0:12:18.600 --> 0:12:21.680 you know, the the same White House folks they were 0:12:21.800 --> 0:12:26.880 putting out. They were putting out information about distillation, which 0:12:26.920 --> 0:12:30.000 we talked about last week, and you know, did did 0:12:30.400 --> 0:12:33.800 Chinese open source models steal from quote unquote steal from 0:12:34.000 --> 0:12:38.960 from anthropic? But there were a number of regulators that said, like, 0:12:39.040 --> 0:12:42.920 it's time to legislate, it's time to get involved, Bernie 0:12:42.960 --> 0:12:48.000 Sanders and many others. So yeah, it's feeding into this 0:12:48.760 --> 0:12:53.240 growing sense of urgency around having some safeguards in place, 0:12:53.280 --> 0:12:57.520 But I just want to say, like another plea for 0:12:57.600 --> 0:13:01.680 people to start thinking about monitoring like downstream usage, like 0:13:01.800 --> 0:13:03.920 kind of what reads saying about the neurons. Actually, now 0:13:03.920 --> 0:13:06.160 that I go back to it, because what they do 0:13:06.200 --> 0:13:10.119 when they try to figure out the models like reasoning 0:13:10.440 --> 0:13:13.079 is look at its chain of thought, like look at 0:13:13.240 --> 0:13:15.480 this little like they give the model a scratch pad 0:13:15.520 --> 0:13:17.760 and it says like, ah, it might be faster to 0:13:17.840 --> 0:13:20.040 go to directly to hooking face, let me break in 0:13:20.120 --> 0:13:23.439 or whatever, and we don't even know if those that 0:13:23.600 --> 0:13:29.320 text actually reflects its interior thought. You know, it's like 0:13:29.440 --> 0:13:31.560 decision making process. So it's true, this. 0:13:31.640 --> 0:13:33.080 Is the dome worry mom and dad, I'm doing my 0:13:33.080 --> 0:13:36.920 homework upstairs. Basically, of AI, it's thinking out loud. 0:13:37.160 --> 0:13:39.400 It's thinking out loud, and that is actually this is 0:13:39.440 --> 0:13:41.400 now this, this is actually the state of the art 0:13:41.440 --> 0:13:44.120 in like AI safety is like looking at it's thinking 0:13:44.160 --> 0:13:44.920 out loud. 0:13:45.080 --> 0:13:48.400 But but it can think out loud deceptively potentially yes. 0:13:48.440 --> 0:13:50.840 And also you're not looking at it's thinking inside. It's 0:13:50.880 --> 0:13:54.080 like if you if you only base what somebody's thoughts 0:13:54.120 --> 0:13:57.280 are and what they say, then that's you're clearly not 0:13:57.360 --> 0:13:59.280 getting all their thoughts right and open. 0:13:59.400 --> 0:14:02.600 I wasn't even reading that you know, so so. 0:14:02.960 --> 0:14:04.400 Right for this experiment. 0:14:04.440 --> 0:14:07.200 They were let's let's yeah, let's let's clean it up. 0:14:08.120 --> 0:14:10.880 So here's my favorite detail from this whole story, which 0:14:10.920 --> 0:14:14.200 actually comes from Forbes, of all places. Quote Hugging Face 0:14:14.200 --> 0:14:16.760 said it had first attempted to use an undisclosed AI 0:14:16.840 --> 0:14:20.120 model from leading US Labs to defend against the attacking 0:14:20.160 --> 0:14:24.000 AI agent, but the guardrails around that model cyber capabilities 0:14:24.160 --> 0:14:27.480 stemied its response teams work. The company said it instead 0:14:27.480 --> 0:14:30.360 wound up using an open source AI model from Chinese 0:14:30.360 --> 0:14:33.040 company Zai to carry out its defense. 0:14:33.400 --> 0:14:38.760 Which is I think exactly why these type of safeguards 0:14:38.840 --> 0:14:41.000 that we're sort of putting in our American models are 0:14:41.000 --> 0:14:45.520 deranged and we need access to open source Chinese models. 0:14:45.680 --> 0:14:49.240 I mean, I think even just from a security standpoint. 0:14:48.840 --> 0:14:52.840 Right, I mean, many cybersecurity people were saying, you know, 0:14:52.920 --> 0:14:56.520 this shows the need to like have policies that help 0:14:56.600 --> 0:15:01.440 proliferation of open source models, because as like, history has 0:15:01.480 --> 0:15:06.040 shown that cyber defense capabilities rely on open source models, 0:15:06.080 --> 0:15:07.960 So it doesn't need to be the Chinese if we 0:15:08.080 --> 0:15:10.840 had been investing in open source AI or if our 0:15:10.960 --> 0:15:13.560 leading companies were putting out open source AI models, it 0:15:13.640 --> 0:15:16.640 could be us. But they did talk about the need 0:15:16.680 --> 0:15:20.760 to make sure like hospitals are adopting the latest open 0:15:20.800 --> 0:15:23.600 source AI models in order to make sure they're capable 0:15:23.600 --> 0:15:25.840 of defending if such a thing happens to them. 0:15:26.240 --> 0:15:28.840 Okay, lightning round, before we go to the break, what's 0:15:28.880 --> 0:15:31.320 the one thing from each of you that somebody's listening 0:15:31.360 --> 0:15:34.080 to this podcast needs to know to sound smart. 0:15:33.880 --> 0:15:34.360 At the weekend? 0:15:34.440 --> 0:15:37.320 About Kimmy K three, I mean, I think everybody wants 0:15:37.360 --> 0:15:40.160 to act like, oh, it was distilled. It was definitely distilled. 0:15:40.200 --> 0:15:42.640 I mean, this is a model that I believe was 0:15:42.680 --> 0:15:46.240 testing and also if I might be getting this wrong, 0:15:46.280 --> 0:15:49.520 but I also think that it's this model also has 0:15:49.560 --> 0:15:52.760 capabilities that our American models don't have. Or it is. 0:15:53.000 --> 0:15:55.240 Kimmykre right, Yeah? Which is what? 0:15:55.360 --> 0:15:58.600 Which is a new model from a Chinese lab that's 0:15:58.680 --> 0:16:02.840 kind of outperform throwed on various benchmarks and therefore everyone 0:16:02.920 --> 0:16:03.760 kind of lost their mind? 0:16:03.800 --> 0:16:04.760 Or what's the what's the like? 0:16:04.880 --> 0:16:07.840 It's sort of on par with with the with the Frontier. 0:16:08.160 --> 0:16:10.280 I wouldn't say it's outperforming, and I don't know if 0:16:10.320 --> 0:16:13.200 I do, you mean it has capabilities the US models 0:16:13.240 --> 0:16:15.880 don't have, or they use training techniques that are sort 0:16:15.880 --> 0:16:18.680 of novel and that you know, the US labs could 0:16:18.720 --> 0:16:19.240 learn from. 0:16:19.480 --> 0:16:21.880 Basically, Like what I was reading yesterday, I was saying 0:16:21.920 --> 0:16:26.480 that like the architecture of the model is superior to 0:16:26.480 --> 0:16:29.400 to whatever it was claiming to be distilled on, and 0:16:29.440 --> 0:16:31.920 that these are not developments that they could have stolen 0:16:32.120 --> 0:16:36.880 basically because it is they have developed further on you 0:16:36.880 --> 0:16:39.160 know whatever. I guess it's like the latest sort of 0:16:39.160 --> 0:16:42.800 Claude model. So I just I think this idea that like, oh, 0:16:42.840 --> 0:16:45.640 these are all just like t MoU versions of Claude. 0:16:45.680 --> 0:16:49.200 I don't think that that's a full picture. Yes, I imagine 0:16:49.120 --> 0:16:52.160 that they're doing some level of distilling. They all do. Like, 0:16:52.320 --> 0:16:55.000 I just think this is like an industry. 0:16:55.280 --> 0:16:57.800 No, there's an innovation there. Like I think what you're 0:16:57.800 --> 0:17:01.040 saying is like you can distill from one of these 0:17:01.160 --> 0:17:03.760 large models in order to shortcut the training, but there's 0:17:03.800 --> 0:17:08.080 still innovation in the training in being just because by 0:17:08.200 --> 0:17:11.320 virtue of being hamstrung on the amount of compute they have, 0:17:11.520 --> 0:17:14.080 they have to they have to take these shortcuts and 0:17:14.119 --> 0:17:17.000 they have to find new and novel ways to train 0:17:17.080 --> 0:17:20.680 with less compute, and so there's innovation there for sure. 0:17:25.920 --> 0:17:28.680 When we come back, read takes us to a garage 0:17:28.840 --> 0:17:43.639 full of automated pickup trucks carrying anti drone weaponry. 0:17:48.800 --> 0:17:49.440 Welcome back. 0:17:49.920 --> 0:17:52.320 So I mentioned in the first story that Nick Thompson 0:17:52.640 --> 0:17:54.720 predicted this would be the year of the first legit 0:17:54.840 --> 0:17:58.240 ai catastrophe. Our panelisting that that that prediction has not 0:17:58.320 --> 0:17:59.600 yet been proven out. 0:18:00.119 --> 0:18:03.520 My prediction was was the year of the robot Read. 0:18:03.880 --> 0:18:06.080 You had a story this week about a company that, 0:18:06.080 --> 0:18:09.560 according to the headline semiphore quote, wants to turn robotics 0:18:09.600 --> 0:18:10.800 into child's play. 0:18:11.359 --> 0:18:13.760 What's the company and what's it do today? And what 0:18:13.800 --> 0:18:15.760 does turning robotics into child's playing? 0:18:16.320 --> 0:18:20.600 Yeah, so the companies applied intuition and I went visited 0:18:20.600 --> 0:18:23.920 their their headquarters on Monday and took a tour through. 0:18:24.520 --> 0:18:26.600 You know, they have this garage where there's really just 0:18:26.640 --> 0:18:30.399 a bunch of cars in you know, various states of disrepair, 0:18:30.560 --> 0:18:35.000 so like you just see all the electronics around a car. Essentially, 0:18:35.280 --> 0:18:39.240 what they've been doing is working for a lot of automakers, 0:18:39.280 --> 0:18:42.600 most of the big automakers actually and building the technology 0:18:42.640 --> 0:18:47.080 for them everything from like the software platform, infotainment systems, 0:18:47.080 --> 0:18:49.840 et cetera, to the self driving tech and they've been 0:18:49.840 --> 0:18:51.960 making a lot of money doing that. They've now expanded 0:18:52.000 --> 0:18:55.760 into like farming and other you know, construction, other areas. 0:18:56.359 --> 0:18:59.560 So basically, if you're not Tesla, your license applied intuition 0:19:00.000 --> 0:19:00.800 evan technology. 0:19:01.119 --> 0:19:04.160 Yeah, exactly. Like if you're an automaker and you realize 0:19:04.160 --> 0:19:06.240 you don't have the tech talent that Tesla has, but 0:19:06.280 --> 0:19:09.120 you want to have a Tesla like product, you kind 0:19:09.160 --> 0:19:12.439 of you hire this this company rather than try to 0:19:12.480 --> 0:19:14.800 do it in house. But and that's been a good 0:19:14.800 --> 0:19:18.240 business for them. But what the news was on Monday, 0:19:18.359 --> 0:19:20.560 I wrote I wrote about it on Tuesday morning was 0:19:20.640 --> 0:19:24.720 that they've come out with this new open platform that 0:19:24.760 --> 0:19:26.280 you can you know, it's a product that you can 0:19:26.280 --> 0:19:28.680 pay for and you can actually use all of their 0:19:28.760 --> 0:19:32.639 data and all the infrastructure they built over almost a 0:19:32.720 --> 0:19:36.560 decade and train your own robots. Essentially. They would call 0:19:36.600 --> 0:19:39.800 it physical intelligence, which is this jargon term for like 0:19:39.880 --> 0:19:43.280 any machine that can be programmed essentially. But like their 0:19:43.359 --> 0:19:46.440 example of is, like you have a kid who wants 0:19:46.480 --> 0:19:50.480 to make an autonomous lawnmower, they could do it now, 0:19:50.520 --> 0:19:52.560 whereas it used to take a team of like eight, 0:19:53.080 --> 0:19:56.399 you know, PhD engineers to do all the computer vision 0:19:56.520 --> 0:19:59.439 and train models, et cetera. And I played around with it. 0:19:59.480 --> 0:20:01.639 I mean, it's still I would say, this is not 0:20:01.800 --> 0:20:05.639 child's play yet, but it's moving in that direction. And 0:20:05.800 --> 0:20:07.720 I just think it's really interesting because it's like, I 0:20:07.720 --> 0:20:09.280 don't know about you guys. I play around with like 0:20:09.359 --> 0:20:11.840 Raspberry pies with my with my ten year old, you know, 0:20:11.880 --> 0:20:14.400 and we try to hack stuff together, and I'm like, oh, 0:20:14.440 --> 0:20:17.080 this is cool. Like this allows to do even more 0:20:17.080 --> 0:20:19.720 fun projects. And then I think sort of create more 0:20:19.880 --> 0:20:23.119 entrepreneurship around AI for the physical world. 0:20:23.560 --> 0:20:25.800 Okay, but so I could go and buy a regular 0:20:26.119 --> 0:20:29.760 NORM from home depot and then use this platform to 0:20:29.800 --> 0:20:30.680 turn into a robot. 0:20:31.280 --> 0:20:33.159 Yeah, well you could put like a GoPro on it 0:20:33.280 --> 0:20:37.359 or something, right, and go film film yourself mowing your 0:20:37.440 --> 0:20:41.679 lawn and then upload the video into this platform and say, okay, 0:20:41.720 --> 0:20:44.600 I want you to basically do like an evaluation and 0:20:44.640 --> 0:20:47.679 train an AI model that is the perfect model for 0:20:47.800 --> 0:20:51.080 mowing my lawn, you know, and be done in a day. 0:20:51.400 --> 0:20:57.120 Essentially, you provide the hardware and they provide the intellig This. 0:20:57.200 --> 0:20:59.760 Is just a software platform. There's no they don't do 0:21:00.040 --> 0:21:02.720 they don't do any hardware, so you make the hardware 0:21:03.400 --> 0:21:06.320