WEBVTT - How DeepSeek Showed That Silicon Valley Is Washed 0:00:02.800 --> 0:00:03.960 Zone media. 0:00:05.320 --> 0:00:08.840 Chosen by God, perfected by science. I'm ed Zetron. This 0:00:08.960 --> 0:00:23.400 is better offline. And as I've written about many, many, 0:00:23.440 --> 0:00:26.119 many many times and argued on this very podcast, just 0:00:26.160 --> 0:00:28.920 as often, the large language models run by companies like 0:00:28.960 --> 0:00:33.120 open Ai, Athropic, Google, and Meta are unprofitable and unsustainable, 0:00:33.280 --> 0:00:36.000 and the transformer based architecture they run on as peaked. 0:00:36.159 --> 0:00:38.920 They're running out of training data, and the actual capabilities 0:00:38.920 --> 0:00:40.840 of these models were peaking as far back as March 0:00:40.880 --> 0:00:44.800 twenty twenty four. Nevertheless, I'd assumed, incorrectly, by the way, 0:00:45.040 --> 0:00:47.320 that there would be no way to make them more efficient, 0:00:47.360 --> 0:00:51.160 because I had assumed, also incorrectly that the hyperscalers, along 0:00:51.200 --> 0:00:53.960 with open Ai and Anthropic, would be constantly looking for 0:00:54.000 --> 0:00:56.680 ways to bring down the ruinous cost of their services. 0:00:57.200 --> 0:00:59.920 After all, open Ai lost five billion dollars last year, 0:01:00.000 --> 0:01:02.960 and that's after three point seven billion dollars in revenue too. 0:01:03.160 --> 0:01:06.000 An Anthropic lost just under three billion dollars in twenty 0:01:06.040 --> 0:01:08.600 twenty four. And in the last episode I told you 0:01:08.640 --> 0:01:10.720 a little bit about deep seek. By the way, in 0:01:10.800 --> 0:01:13.119 this one we're going to get into well how fucked 0:01:13.120 --> 0:01:16.280 things might actually be. But what I didn't wager was 0:01:16.280 --> 0:01:19.559 that potentially nobody was actually trying to make these models 0:01:19.640 --> 0:01:22.640 more efficient. My mistake was if you can believe this 0:01:23.240 --> 0:01:26.160 being too generous to the AI companies, assuming that they 0:01:26.200 --> 0:01:29.600 didn't pursue efficiency because they couldn't and not because they 0:01:29.600 --> 0:01:33.240 couldn't be bothered. But then, as I just hinted at, 0:01:33.440 --> 0:01:36.080 a little known Chinese company released a product that was 0:01:36.120 --> 0:01:39.800 broadly equivalent to open AI's latest reasoning models, but cost 0:01:39.800 --> 0:01:42.160 a fraction of the cost to train and run. And 0:01:42.200 --> 0:01:45.080 now the conventional understanding of how generative AI should work 0:01:45.120 --> 0:01:49.120 has been fundamentally upended. You see, the pre Deep Sea 0:01:49.360 --> 0:01:52.120 status quo was one where several truths, and I'd say 0:01:52.120 --> 0:01:54.360 that in the loosest sense of the word, allowed the 0:01:54.400 --> 0:01:56.960 party to keep going. So the first one is that 0:01:56.960 --> 0:02:00.160 these models were incredibly expensive to train. GPT four morrow 0:02:00.240 --> 0:02:02.240 cost one hundred million dollars in the middle of twenty 0:02:02.240 --> 0:02:05.960 twenty four, and future models, according Dario Ama Day of Anthropic, 0:02:06.040 --> 0:02:08.480 might cost as much as one billion or more to train. 0:02:09.160 --> 0:02:11.880 And training future models, by the way, as a result 0:02:11.919 --> 0:02:14.560 of this, would necessitate spending billions of dollars on both 0:02:14.639 --> 0:02:17.520 data centers and the GPUs necessary to keep training these 0:02:17.560 --> 0:02:21.120 bigger huger models. Now, another thing was these models had 0:02:21.160 --> 0:02:23.720 to be large because making them large, pumping them full 0:02:23.720 --> 0:02:26.799 of training data, and throwing masses of compute at them 0:02:26.880 --> 0:02:29.919 would unlock new features such as an AI that helps 0:02:29.960 --> 0:02:32.560 us accomplish much more than we ever could without AI, 0:02:32.680 --> 0:02:36.280 which is Sam Altman quote. And in the words of 0:02:36.320 --> 0:02:39.320 Sam again, you'd be able to get a personal AI 0:02:39.360 --> 0:02:42.120 team full of virtual experts in different areas working together 0:02:42.200 --> 0:02:44.960 to create almost anything we can imagine. I don't know, mate, 0:02:45.000 --> 0:02:48.880 you ever try creating a functional fucking business. Dipshit. Anyway, 0:02:49.040 --> 0:02:52.680 here's another one. These models were incredibly expensive to run. 0:02:52.760 --> 0:02:54.480 There has to be this way, but it was all 0:02:54.480 --> 0:02:57.440 worth it because making these models powerful was way more 0:02:57.480 --> 0:02:59.919 important than making them efficient. Because once the price of 0:03:00.080 --> 0:03:02.400 silicon comes down, and this is a refrain I've heard 0:03:02.440 --> 0:03:04.600 from multiple different people as a defense of the costs 0:03:04.600 --> 0:03:07.720 of generative AI, we would then have these powerful models 0:03:07.720 --> 0:03:13.959 that were cheaper somehow because of silicon. Now you may think, 0:03:14.120 --> 0:03:16.640 ed that sounds like a not a real argument. That 0:03:16.720 --> 0:03:19.120 just sounds like something someone said once, and it is. 0:03:19.200 --> 0:03:21.680 It is something someone said once. Anyone who knows anything 0:03:21.680 --> 0:03:23.640 about chips know how hard it is to make a 0:03:23.680 --> 0:03:26.520 new chip. And remember one of the CES episodes when 0:03:26.520 --> 0:03:28.840 I asked Max Cherney about this, should go back and 0:03:28.880 --> 0:03:33.040 listen to him. Anyway, another thing, another part of this 0:03:33.919 --> 0:03:36.760 was that as a result of this need to make bigger, huger, 0:03:36.920 --> 0:03:40.440 even bigger models, the most powerful ones, these big beautiful 0:03:40.480 --> 0:03:42.400 models that we love them. We look at the big 0:03:42.440 --> 0:03:45.880 beautiful models, we would of course need to keep buying bigger, 0:03:46.000 --> 0:03:49.920 more powerful GPUs, which would continue the American excellence of 0:03:49.960 --> 0:03:52.800 burning a bunch of money on nothing. And by following 0:03:52.840 --> 0:03:56.640 this roadmap, everybody wins. The hyperscalers get the justification they 0:03:56.680 --> 0:03:59.920 needed to create more sprawling data centers and spend massive 0:04:00.000 --> 0:04:02.560 amounts of money and open AI and their alk get 0:04:02.640 --> 0:04:06.040 to continue building powerful models, and also in video continues 0:04:06.080 --> 0:04:09.440 to make money selling GPUs. Remember I've said in the 0:04:09.480 --> 0:04:11.400 past that things were kind of a death cull. This 0:04:11.520 --> 0:04:13.760 is what this is. It's a capitalist death cull. It 0:04:13.840 --> 0:04:17.599 runs on plagiarism and hubris and the assumption that at 0:04:17.640 --> 0:04:21.440 some point all of this would turn into something meaningful. Now, 0:04:21.440 --> 0:04:23.640 I've argued for a while that the latter part of 0:04:23.680 --> 0:04:25.800 the plan was insane, that there was no profitability for 0:04:25.839 --> 0:04:28.320 these large language models. As I believe, there simply wasn't 0:04:28.360 --> 0:04:30.839 a way to make these models more efficient in a 0:04:30.880 --> 0:04:33.919 way I was raimed. The current models developed by both 0:04:33.960 --> 0:04:37.200 the hyperscalers, so Gemini from Google or Lama from Meta, 0:04:37.240 --> 0:04:39.240 and so on and so forth, and the multi billion 0:04:39.279 --> 0:04:41.480 dollar startups, if you can even fucking call them that, 0:04:41.600 --> 0:04:45.159 Open Ai and Anthropic, they're horribly inefficient. And I just 0:04:45.240 --> 0:04:47.200 made the mistake of assuming that they tried to make 0:04:47.200 --> 0:04:50.880 them more efficient, and they couldn't. Now, but what we're 0:04:50.920 --> 0:04:56.839 witnessing right now isn't some sort of weird China situation. 0:04:57.000 --> 0:05:01.360 This isn't China being Chinese and doing scary Chinese things 0:05:01.400 --> 0:05:03.960 to us. Now, what we're witnessing is the American tech 0:05:04.000 --> 0:05:08.120 industry's greatest act of hubris. It's a monument to the 0:05:08.160 --> 0:05:12.120 barely conscious stewards of so called innovation who are incapable 0:05:12.120 --> 0:05:14.920 of breaking the k fab of the fake competition where 0:05:14.960 --> 0:05:18.080 everybody makes the same products, charges about the same amount 0:05:18.080 --> 0:05:22.640 of money, and mostly innovates in the same direction. Somehow, nobody, 0:05:22.920 --> 0:05:26.440 not Google, not Microsoft, not open Ai, not Meta, not Amazon, 0:05:26.480 --> 0:05:29.640 no Oracle thought to try or was capable of creating 0:05:29.720 --> 0:05:33.000 something like deep Seek, Which doesn't mean that deep Seek's 0:05:33.040 --> 0:05:36.480 team is particularly remarkable or found anything super new, but 0:05:36.720 --> 0:05:39.880 that for all the talent, trillions of dollars of market capitalization, 0:05:39.920 --> 0:05:44.120 and supposed expertise in Americans American tech hologarchs, not one 0:05:44.160 --> 0:05:46.799 bright spark thought to try things that deep Seak had tried, 0:05:46.960 --> 0:05:48.880 which appeared to be what if we didn't use as 0:05:48.960 --> 0:05:52.400 much memory, and what if we tried synthetic data? And 0:05:52.440 --> 0:05:55.400 because the cost of model development and inference was so 0:05:55.480 --> 0:05:58.520 astronomical in the case of American models, they never assumed 0:05:58.520 --> 0:06:01.800 that anyone would try to use their position. This is 0:06:01.960 --> 0:06:04.839 especially bad considering that China's folks and AI as a 0:06:04.839 --> 0:06:08.200 strategic part of its industrial priority was really no secret, 0:06:08.560 --> 0:06:11.840 even if the ways it supported domestic companies kind of is. 0:06:12.400 --> 0:06:15.000 In the same way that the automotive industry was blindsided 0:06:15.000 --> 0:06:18.760 by China's EV manufacturers, the same is happening with AI. Fat, 0:06:19.040 --> 0:06:22.200 happy and lazy, and most of all oblivious. America's most 0:06:22.279 --> 0:06:25.840 powerful tech companies sacked back and built bigger, messier models 0:06:25.839 --> 0:06:28.520 powered by sprawling data centers of billions of dollars of 0:06:28.560 --> 0:06:31.800 GPUs from Nvidia, A bacchanalia are spending that strains our 0:06:31.920 --> 0:06:35.000 energy grid and depletes our fucking water reserves. Without it 0:06:35.080 --> 0:06:39.359 appears much consideration of whether an alternative was possible. I 0:06:39.560 --> 0:06:42.400 refuse to believe that none of these companies could have 0:06:42.480 --> 0:06:44.760 done what Deep Seek has done, which means that they 0:06:44.800 --> 0:06:47.880 either chose not to or they were so utterly myopic, 0:06:48.080 --> 0:06:50.720 so excited to burn so much money on so many 0:06:50.800 --> 0:06:53.880 parts of burning the earth, boiling lakes, and stealing from 0:06:53.920 --> 0:06:56.719 people in pursuit of further growth, that they didn't think 0:06:56.760 --> 0:07:00.719 to try. This isn't about China. It's so much fucking 0:07:00.839 --> 0:07:03.960 easier if we let it be about China. No, no, no, no. 0:07:04.160 --> 0:07:09.240 It's about how the American tech industry is incurious, lazy, entitled, directionless, 0:07:09.240 --> 0:07:14.240 and irresponsible open an anthropic of the antithesis of Silicon Valley. 0:07:14.400 --> 0:07:18.040 They're incumbents, public companies wearing startup suits. I'm willing to 0:07:18.040 --> 0:07:21.240 take on real challenges more focused on optics and marketing 0:07:21.280 --> 0:07:24.280 than they are in solving actual fucking problems, even the 0:07:24.320 --> 0:07:27.560 problems that they themselves created with their large language models. 0:07:28.200 --> 0:07:31.280 By making this about China, we ignore the root of 0:07:31.320 --> 0:07:34.280 the problem that the American tech industry is no longer 0:07:34.280 --> 0:07:38.360 interested in making good software that actually helps people. Deep 0:07:38.400 --> 0:07:41.480 seaks shouldn't be scary to Silicon Valley because they that 0:07:41.560 --> 0:07:43.920 Silicon Valley should have come up with this first. It 0:07:44.040 --> 0:07:46.880 uses less memory, fewer resources, and uses several kind of 0:07:46.920 --> 0:07:50.280 quirky workarounds to adapt to the limited compute resources available. 0:07:50.440 --> 0:07:54.560 All things that you'd previously associate with Silicon Valley, except now. 0:07:54.600 --> 0:07:57.040 Silicon Valley's only interest that the rest of the American 0:07:57.040 --> 0:08:01.000 tech industry is the rot economy. It only cares about growing, 0:08:01.280 --> 0:08:04.360 growing at all costs, even if said costs were really 0:08:04.800 --> 0:08:07.360 things you could mitigate, or if the costs themselves were 0:08:07.400 --> 0:08:10.800 self defeating. To be clear, if the alternative is that 0:08:10.880 --> 0:08:13.200 all of these companies simply did not come up with 0:08:13.200 --> 0:08:15.960 this idea that in and of itself is a damning 0:08:16.080 --> 0:08:19.080 indictment of the Valley. Was nobody thinking of this stuff? 0:08:19.240 --> 0:08:21.760 If they were, why didn't Sam Altman or Dario Amadae 0:08:21.840 --> 0:08:24.360 or sach In Adela or anyone else put serious resources 0:08:24.360 --> 0:08:26.920 into efficiency? Was it because there was no reason to? 0:08:27.320 --> 0:08:29.600 Was it because there was, if we're honest, no real 0:08:29.640 --> 0:08:33.800 competition between any of these companies? Did anybody try anything 0:08:33.920 --> 0:08:36.679 other than throwing as much compute and training data at 0:08:36.679 --> 0:08:40.640 the model as possible. It's all just so cynical and 0:08:40.679 --> 0:08:44.440 antithetical to innovation itself. Surely, if any of this shit mattered, 0:08:44.559 --> 0:08:47.440 if generative AI truly was valid and viable in the 0:08:47.480 --> 0:08:50.280 eyes of these companies, they would have actively worked to 0:08:50.320 --> 0:08:53.760 do something like Deep Seeker has done. Don't get me wrong, 0:08:54.200 --> 0:08:56.840 it appears Deep Seak employed all sorts of weird tricks 0:08:56.880 --> 0:08:59.640 to make this work, including taking advantage of distinct parts 0:08:59.640 --> 0:09:02.400 of both CPUs and GPUs to create something called a 0:09:02.400 --> 0:09:06.280 digital processing unit, essentially redefining how data is communicated within 0:09:06.320 --> 0:09:09.080 the servers running training and inference. And just this reminder, 0:09:09.120 --> 0:09:11.800 inference is the thing where when you type something in 0:09:12.120 --> 0:09:15.440 that's it infers the meaning. Just could have specified that earlier. 0:09:16.040 --> 0:09:18.640 Deepseak had to do things that a company with unrestrained 0:09:18.679 --> 0:09:21.200 access to capital and equipment wouldn't have to do, and 0:09:21.240 --> 0:09:25.439 it often used impractical and quirky methods to do so. Nevertheless, 0:09:25.520 --> 0:09:28.160 Open AI and Anthropic both have enough money and hiring 0:09:28.200 --> 0:09:30.959 power to have tried and succeeded in creating a model 0:09:31.280 --> 0:09:34.880 this efficient and capable of running on older GPUs, except 0:09:34.920 --> 0:09:38.199 what they wanted, what they actually wanted was more goddamn 0:09:38.280 --> 0:09:41.320 growth and the chance to build even bigger data centers 0:09:41.360 --> 0:09:44.560 with even more compute that they would own. Open ai 0:09:44.760 --> 0:09:48.559 is as much a lazy, cumbersome incumbent as Google or Microsoft, 0:09:48.679 --> 0:09:51.640 and it's about as innovative too. The launch of its 0:09:51.679 --> 0:09:55.640 Operator agent was a joke, a barely functional product that's 0:09:55.640 --> 0:09:58.720 allegedly meant to control your computer and take distinct actions 0:09:58.760 --> 0:10:01.679 like ordering stuff of instacark, you know, things you could 0:10:01.720 --> 0:10:04.080 do with your hands, but just to be clear, it 0:10:04.080 --> 0:10:07.320 doesn't work. You'll never guess who was really into it. 0:10:07.360 --> 0:10:10.800 Though his name is Casey Newton. He writes a blog 0:10:10.840 --> 0:10:14.080 called Platformer, and he's a man so gratingly credulous that 0:10:14.080 --> 0:10:17.000 it makes me want to fucking scream. And of course 0:10:17.200 --> 0:10:19.839 he wrote the Operator when he used it was a 0:10:19.880 --> 0:10:24.720 compelling demonstration that represented an extraordinary technological achievement. Also, somehow 0:10:24.800 --> 0:10:27.960 was significantly slower, more frustrating, and more expensive than simply 0:10:28.000 --> 0:10:32.040 doing any of these tasks yourself. Casey, of course, not 0:10:32.160 --> 0:10:35.280 to worry, had some extra thoughts about Deep Seek, that 0:10:35.320 --> 0:10:38.079 there were reasons to be worried, but that American AI 0:10:38.200 --> 0:10:40.600 lads were still in the lead, saying that deep seak 0:10:40.679 --> 0:10:43.360 was only optimizing technology that open ai and others had 0:10:43.360 --> 0:10:46.760 invented first, before saying that deep seek was only last 0:10:46.760 --> 0:10:49.120 week that open ai made available to pro plan used 0:10:49.120 --> 0:10:51.520 as a computer that can use itself. This statement is 0:10:51.559 --> 0:10:54.760 bordering on factually incorrect. It is fucking insane that Casey 0:10:54.840 --> 0:10:57.040 is still doing this. I do not want to I 0:10:57.040 --> 0:10:58.800 don't know what to do with this guy. This guy 0:10:59.600 --> 0:11:03.320 just like that. That's a fucking lie. The computer can't 0:11:03.400 --> 0:11:06.680 use itself, This shit can't just to explain what operator is, 0:11:06.720 --> 0:11:09.040 You're meant to type in something like, hey, order me 0:11:09.120 --> 0:11:12.200 some milk, Order me some milk off of vistacar. And 0:11:12.480 --> 0:11:14.719 when Casey tried this, he tried to find milk and 0:11:14.760 --> 0:11:20.120 de moine iowa. Just fucking insane. Just this is how 0:11:20.160 --> 0:11:22.880 these companies have got big. It's people like cases people 0:11:22.880 --> 0:11:25.280 are Casey, who are just like anything they show tore 0:11:25.280 --> 0:11:27.280 like God, damn, that's the most impressive thing I've seen 0:11:27.320 --> 0:11:30.360 in my life. It's a fucking fast. But let's be frank, 0:11:30.840 --> 0:11:34.240 these companies aren't building shit. Open Air and Anthropic are 0:11:34.240 --> 0:11:37.080 both limply throwing around the idea that agents are possible 0:11:37.240 --> 0:11:39.200 in an attempt to raise more money to burn, and 0:11:39.240 --> 0:11:41.199 after the launch of deep Seek, I have to wonder 0:11:41.200 --> 0:11:44.240 what any investor thinks they're investing in other than certain 0:11:44.280 --> 0:11:46.720 ones I'll get into in a bit. And to be clear, 0:11:46.760 --> 0:11:49.800 an agent is meant to be this autonomous thing which 0:11:49.800 --> 0:11:52.239 you say, hey, go and do this action, go and 0:11:52.360 --> 0:11:55.120 sell things for me, and go and email people for me. 0:11:55.320 --> 0:11:58.040 They don't really work. There are some that kind of 0:11:58.080 --> 0:12:00.520 do that are really expensive, but large Lane which model 0:12:00.559 --> 0:12:03.600 is not built for this kind of thing. But let's 0:12:03.640 --> 0:12:06.360 be honest, Deep Seek and as I said in the 0:12:06.440 --> 0:12:09.680 last episode, they've built a more efficient reasoning model. So 0:12:09.800 --> 0:12:14.080 like open aiy zo one And you'd think, well, okay, 0:12:14.360 --> 0:12:17.600 couldn't open Ai simply add on deep Seek to its models. 0:12:18.280 --> 0:12:21.360 Not really. First of all, with the way these models work, 0:12:21.400 --> 0:12:22.960 you can't just like plug it in. It's just not 0:12:23.040 --> 0:12:25.480 how it works. They could train a new model using 0:12:25.520 --> 0:12:29.680 deep Seek's techniques, but the optics of that aren't brilliant. 0:12:29.760 --> 0:12:32.040 It would be a concession. It'd be in the middle 0:12:32.080 --> 0:12:35.120 that open aiyes slipped and needs to catch up, and 0:12:35.120 --> 0:12:38.120 not to its main rival pretend rival, I mean anthropic 0:12:38.440 --> 0:12:40.840 or to like another big tech firm, but to an 0:12:40.960 --> 0:12:44.560 outgrowth of a hedge fund in China, a company that 0:12:44.679 --> 0:12:49.600 few had heard of before December and the like. Really 0:12:49.760 --> 0:12:52.720 not that many people had heard before January twenty fifth. 0:12:53.320 --> 0:12:55.920 It's very embarrassing, and this, in turn, I think will 0:12:55.960 --> 0:12:59.319 make any serious investor think twice about writing the company 0:12:59.360 --> 0:13:01.560 a blank chet. They're going to have to dip into 0:13:01.600 --> 0:13:06.160 some very bothersome pockets, and as I've said ad nauseum, 0:13:06.440 --> 0:13:09.359 this is potentially fatal as open ai needs to continually 0:13:09.440 --> 0:13:12.480 raise money, more money than any startup has ever raised 0:13:12.480 --> 0:13:14.760 in the history of anything, and it really doesn't have 0:13:14.800 --> 0:13:17.720 a path to breaking even even if they copy what 0:13:17.760 --> 0:13:20.960 Deep Sea did, because we still right now, though deep 0:13:20.960 --> 0:13:24.720 Seek is thirty times cheaper than one, we don't know 0:13:24.760 --> 0:13:28.120 if that's profit or we don't know. We haven't found out. 0:13:28.960 --> 0:13:31.120 And if open ai wants to do its own, cheaper, 0:13:31.160 --> 0:13:33.319 more efficient model, it's likely to have to create it 0:13:33.360 --> 0:13:35.240 from scratch, like I said, And while it could do 0:13:35.360 --> 0:13:38.000 distillation to make it kind of more like open ai 0:13:38.200 --> 0:13:40.720 using their own models. By the way, deep Sea taught 0:13:40.760 --> 0:13:43.880 itself using open aies outputs. Like I mentioned in the 0:13:43.920 --> 0:13:46.600 last episode, it's kind of what deep Seak already did. 0:13:46.640 --> 0:13:49.840 It already has been fed open ai bullshit. Even with 0:13:50.000 --> 0:13:53.280 open AI's much larger team and more powerful hardware, it's 0:13:53.280 --> 0:13:55.480 hard to see how creating a smaller, more efficient, and 0:13:55.520 --> 0:13:57.840 almost as powerful version of OH one benefits them in 0:13:57.880 --> 0:14:01.480 any way, because said version has well already been beaten 0:14:01.520 --> 0:14:03.559 to market by deep Seek, and thanks to deep Seek, 0:14:03.800 --> 0:14:06.120 will almost certainly have a great deal of competition for 0:14:06.160 --> 0:14:08.600 a product that to this day lacks any killer apps. 0:14:08.600 --> 0:14:24.440 Anyway, It's just it's very frustrating to me. It's very 0:14:24.440 --> 0:14:27.200 frustrating to me. It drives me a little insane. Reading 0:14:27.200 --> 0:14:29.200 all this stuff makes me feel crazy. I think you 0:14:29.240 --> 0:14:32.440 hear it in my voice, you hear the sanity stripping away. 0:14:32.480 --> 0:14:36.360 But I'm here to podcast and don't worry. But seriously, though, 0:14:37.240 --> 0:14:39.400 anyone can build on top of what deep seek has 0:14:39.400 --> 0:14:44.480 already built. Where is open Aiy's mote exactly? And where's anthropics? 0:14:44.720 --> 0:14:47.040 What are the things that make these companies worth sixty 0:14:47.080 --> 0:14:49.960 billion or one hundred and fifty billion, or oh my god, 0:14:50.000 --> 0:14:51.840 as we'll discuss in a bit, three hundred and forty 0:14:51.880 --> 0:14:52.560 billion dollars? 0:14:53.520 --> 0:14:56.000 What is the technology they own or the talent they 0:14:56.040 --> 0:14:59.040 have that justifies these valuations? Because it's kind of hard 0:14:59.080 --> 0:15:04.320 to argue that their model particularly valuable anymore. Celebrity celebrity 0:15:04.400 --> 0:15:09.840 them CULTI personality. Sam Orman, He's an artful bullshitter, and 0:15:09.880 --> 0:15:11.840 he's built a career out of being in the right 0:15:11.920 --> 0:15:14.080 places at the right times, have in the right connections, 0:15:14.160 --> 0:15:17.880 knowing exactly what to say, especially to credulous tech media 0:15:18.120 --> 0:15:20.640 ponces without the spine or inclination to push back on 0:15:20.680 --> 0:15:24.240 his more stupid claims. And already Aortman has tried to 0:15:24.280 --> 0:15:27.760 shrug off Deep Seaks Rise, admitting that while Deep Seeks 0:15:27.760 --> 0:15:30.520 are one model is impressive, particularly when it comes to 0:15:30.560 --> 0:15:33.800 its efficiency, open Ai will obviously deliver much better models, 0:15:33.800 --> 0:15:36.720 and also it's legit invigorating for open Ai to have 0:15:36.760 --> 0:15:39.960 a new competitor. Yeah, mate, sure, I'll bet your love 0:15:40.000 --> 0:15:43.080 in this. Aortmand ended his tweet where that came from 0:15:43.120 --> 0:15:46.880 with look forward to bringing you all AGI and beyond, 0:15:47.200 --> 0:15:50.000 something which I add has always been close on the 0:15:50.040 --> 0:15:53.800 horizon in Altman's world, but it's never really materialized. Then 0:15:53.840 --> 0:15:56.680 the timeline keeps moving and there's no actual proof they 0:15:56.720 --> 0:15:59.720 can do it. AGI is not fucking happening, And if 0:15:59.720 --> 0:16:02.240 it's possible in any way, it's not coming out of 0:16:02.280 --> 0:16:05.840 probabilistic models. I'm fucking sick of this and Opening I 0:16:06.080 --> 0:16:09.240 can't even lean on its relationship with Microsoft, which on Wednesday, 0:16:09.320 --> 0:16:12.320 January twenty ninth started offering deep Seek's models through its 0:16:12.320 --> 0:16:16.440 own cloud services. Opening I hasn't got shit. Deep Seek 0:16:16.440 --> 0:16:19.800 has commoditized the large language model, publishing both the source 0:16:19.840 --> 0:16:22.240 code and the guide to building your own. Whether or 0:16:22.320 --> 0:16:25.040 not someone chooses to pay deep Seek is largely irrelevant. 0:16:25.280 --> 0:16:27.600 Someone else will take what they have created and build 0:16:27.640 --> 0:16:29.760 their own, or people will start running their own deep 0:16:29.840 --> 0:16:33.200 Seek instances, renting GPUs from one of the various cloud 0:16:33.200 --> 0:16:35.520 computing firms. They don't give a shit. They'll take the 0:16:35.600 --> 0:16:38.280 money and while in video, will always find other ways 0:16:38.280 --> 0:16:41.120 to make money. Jensen Wong is amazing at this. It's 0:16:41.160 --> 0:16:43.240 going to be a hard sell for any hyperscala to 0:16:43.320 --> 0:16:46.360 justify spending billions more on GPUs to markets that now 0:16:46.440 --> 0:16:48.560 know that near identical models can be built for a 0:16:48.600 --> 0:16:51.400 fraction of the cost with older hardware. Why do you 0:16:51.440 --> 0:16:55.120 need Blackwell, which is the latest Nvidia GPU. The narrative 0:16:55.160 --> 0:16:57.720 of this is the only way to build powerful models. 0:16:57.920 --> 0:17:00.800 It doesn't really hold water anymore, and the only other 0:17:00.880 --> 0:17:04.560 selling point is that what if China does something. Well, 0:17:04.680 --> 0:17:07.480 the Chinese did something, and they've now proven that they 0:17:07.560 --> 0:17:10.960 cannot only compete with American AI companies, but doing so 0:17:11.720 --> 0:17:14.440 is possible in an efficient way that can effectively crash 0:17:14.480 --> 0:17:17.800 the market while there's being a recovery. This is still 0:17:17.920 --> 0:17:21.480 very worrying. I also want to address something real quick. 0:17:21.880 --> 0:17:24.160 A few people on Twitter have been suggesting that talking 0:17:24.160 --> 0:17:26.600 about deep Seek positively in any way is some sort 0:17:26.640 --> 0:17:30.120 of Chinese op. If you believe this, you're a fucking moron. 0:17:30.600 --> 0:17:33.960 I really must be clear, take your weirdsenophobia and go 0:17:34.119 --> 0:17:36.840 eat your own shit. I don't fucking care anymore. Yes, 0:17:36.880 --> 0:17:39.920 there are problems with China. Yes, China does something to America. 0:17:40.080 --> 0:17:43.240 This is an open source thing. This is an open 0:17:43.280 --> 0:17:46.000 source thing, and you can remove China from the equation. 0:17:46.080 --> 0:17:48.520 Because it's open source, someone else is going to use this. 0:17:48.800 --> 0:17:52.320 If your only defense is the sneaky Chinese are doing something, 0:17:52.480 --> 0:17:55.000 go to therapy and talk to the therapist about you 0:17:55.119 --> 0:17:57.960 being paranoid or racist, because it's one of them. I 0:17:57.960 --> 0:18:01.119 should also be clear, concerns about China very realistic. The 0:18:01.200 --> 0:18:04.359 Chinese government has tried to interfere with America. It's happened 0:18:04.359 --> 0:18:09.000 many times. Even if China is funding Deep Seeks models, 0:18:09.920 --> 0:18:12.760 the fact that they are open sourced means that anyone 0:18:12.800 --> 0:18:15.520 can run them and anyone can build their own. They 0:18:15.520 --> 0:18:17.960 can look under the hood. We don't have the training data, 0:18:18.000 --> 0:18:21.400 but that is it. You cannot win on a senophobic 0:18:21.520 --> 0:18:25.800 argument here. You can have realistic concerns about another foreign power. 0:18:26.040 --> 0:18:29.080 I'm not saying not to, but what I am saying 0:18:29.520 --> 0:18:32.040 is that you have to look at this realistically, and 0:18:32.040 --> 0:18:34.800 you have to take this seriously. And dismissing this as 0:18:34.920 --> 0:18:41.919 Chinese magic is stupid. It's very goddamn stupid. But like 0:18:41.960 --> 0:18:44.720 I said earlier, it also isn't clear if these models 0:18:44.720 --> 0:18:47.480 are actually going to be profitable. It's unclear who funds 0:18:47.520 --> 0:18:49.639 Deep seg like I just said, and whether it's current 0:18:49.680 --> 0:18:53.119 pricing is actually sustainable. But they're likely going to be 0:18:53.160 --> 0:18:56.040 a damn site more profitable than anything open ai is 0:18:56.080 --> 0:18:59.280 currently selling. After all, open ai loses money on every 0:18:59.280 --> 0:19:02.360 single transaction, even their two hundred dollars a month Chat 0:19:02.480 --> 0:19:05.920 GPT Pro subscription, And if open ai cuts its prices 0:19:05.920 --> 0:19:08.479 to compete with deep sek, its losses are only going 0:19:08.520 --> 0:19:11.040 to deepen. And as I've said again and again this 0:19:11.160 --> 0:19:13.960 is also deeply cynical, because it's obvious that none of 0:19:14.000 --> 0:19:17.119 this was ever about the proliferation of generative AI or 0:19:17.160 --> 0:19:21.040 making sure that generative AI was accessible. Putting aside my 0:19:21.400 --> 0:19:24.520 very obvious personal beliefs for a second, it's fairly obvious 0:19:24.520 --> 0:19:28.000 why these companies, the big hyperscalers and open ai and 0:19:28.080 --> 0:19:31.680 Anthropic wouldn't want to create something like deep seek because 0:19:31.760 --> 0:19:34.560 creating an open source model that uses less resources means 0:19:34.600 --> 0:19:38.960 that open Ai, Anthropic and their associated hyperscalar findom clients 0:19:39.200 --> 0:19:42.600 would lose their soft monopoly on large language models. Now 0:19:42.600 --> 0:19:46.800 what does that mean, I'll explain before deep seek to 0:19:46.840 --> 0:19:50.200 make a competitive large language model like GPT four zero, 0:19:50.680 --> 0:19:53.720 as in one that you can actually commercialize, required exceedingly 0:19:53.800 --> 0:19:56.840 large amounts of capital, and to make larger ones effectively 0:19:56.880 --> 0:19:59.080 required you to kiss the ring or the ass of Microsoft, 0:19:59.160 --> 0:20:01.600 Google or Amazon. While it isn't clear what it cost 0:20:01.640 --> 0:20:04.239 to train open aizo one reasoning model, we know that 0:20:04.280 --> 0:20:06.560 GPT four to ZH cost in excess of one hundred 0:20:06.640 --> 0:20:09.200 million dollars, and oh one is a more complex model 0:20:09.240 --> 0:20:12.280 would likely cost even more. We also know that open 0:20:12.320 --> 0:20:14.719 AI's training and inference costs in twenty twenty four were 0:20:14.760 --> 0:20:18.840 around seven billion dollars, meaning that either refining current models 0:20:18.920 --> 0:20:22.480 or building new ones is quite costly. The mythology of 0:20:22.520 --> 0:20:25.000 both open AI and Anthropic is that these large amounts 0:20:25.040 --> 0:20:27.639 of capital weren't just necessary, but the only way to 0:20:27.720 --> 0:20:31.280 do this. While these companies ostensibly compete, neither of them 0:20:31.320 --> 0:20:34.200 seemed concerned about doing so. As actual businesses that made 0:20:34.240 --> 0:20:37.760 products that were say, cheaper and more efficient to run, 0:20:37.800 --> 0:20:40.600 you know, they made more money than they cost because 0:20:40.600 --> 0:20:43.320 in doing so, these companies would break the illusion that 0:20:43.359 --> 0:20:46.520 the only way to create powerful artificial intelligence was to 0:20:46.560 --> 0:20:49.119 hand billions of dollars to one of two companies and 0:20:49.160 --> 0:20:52.480 build giant data centers to build even larger language models. 0:20:53.119 --> 0:20:57.120 This is aisrot economy, two lumbering companies claiming that their 0:20:57.160 --> 0:20:59.520 startups creating a narrative that the only way to build 0:20:59.520 --> 0:21:02.359 the future to keep growing, to build more data centers, 0:21:02.359 --> 0:21:05.359 to build larger language models, to consume more training data 0:21:05.400 --> 0:21:08.760 with each infusion of capital, GPU purchase, and data center 0:21:08.800 --> 0:21:11.960 build out, creating an infrastructural mote that always leads back 0:21:12.000 --> 0:21:16.159 to one of a few tech hyperscalers. Open an anthropic 0:21:16.320 --> 0:21:18.920 need the narrative to say, buy more GPUs and build 0:21:19.000 --> 0:21:21.480 more data centers, because in doing so they create the 0:21:21.480 --> 0:21:25.720