Speaker 1: Zone media.
Speaker 2: Chosen by God, perfected by science. I'm ed Zetron. This
Speaker 2: is better offline. And as I've written about many, many,
Speaker 2: many many times and argued on this very podcast, just
Speaker 2: as often, the large language models run by companies like
Speaker 2: open Ai, Athropic, Google, and Meta are unprofitable and unsustainable,
Speaker 2: and the transformer based architecture they run on as peaked.
Speaker 2: They're running out of training data, and the actual capabilities
Speaker 2: of these models were peaking as far back as March
Speaker 2: twenty twenty four. Nevertheless, I'd assumed, incorrectly, by the way,
Speaker 2: that there would be no way to make them more efficient,
Speaker 2: because I had assumed, also incorrectly that the hyperscalers, along
Speaker 2: with open Ai and Anthropic, would be constantly looking for
Speaker 2: ways to bring down the ruinous cost of their services.
Speaker 2: After all, open Ai lost five billion dollars last year,
Speaker 2: and that's after three point seven billion dollars in revenue too.
Speaker 2: An Anthropic lost just under three billion dollars in twenty
Speaker 2: twenty four. And in the last episode I told you
Speaker 2: a little bit about deep seek. By the way, in
Speaker 2: this one we're going to get into well how fucked
Speaker 2: things might actually be. But what I didn't wager was
Speaker 2: that potentially nobody was actually trying to make these models
Speaker 2: more efficient. My mistake was if you can believe this
Speaker 2: being too generous to the AI companies, assuming that they
Speaker 2: didn't pursue efficiency because they couldn't and not because they
Speaker 2: couldn't be bothered. But then, as I just hinted at,
Speaker 2: a little known Chinese company released a product that was
Speaker 2: broadly equivalent to open AI's latest reasoning models, but cost
Speaker 2: a fraction of the cost to train and run. And
Speaker 2: now the conventional understanding of how generative AI should work
Speaker 2: has been fundamentally upended. You see, the pre Deep Sea
Speaker 2: status quo was one where several truths, and I'd say
Speaker 2: that in the loosest sense of the word, allowed the
Speaker 2: party to keep going. So the first one is that
Speaker 2: these models were incredibly expensive to train. GPT four morrow
Speaker 2: cost one hundred million dollars in the middle of twenty
Speaker 2: twenty four, and future models, according Dario Ama Day of Anthropic,
Speaker 2: might cost as much as one billion or more to train.
Speaker 2: And training future models, by the way, as a result
Speaker 2: of this, would necessitate spending billions of dollars on both
Speaker 2: data centers and the GPUs necessary to keep training these
Speaker 2: bigger huger models. Now, another thing was these models had
Speaker 2: to be large because making them large, pumping them full
Speaker 2: of training data, and throwing masses of compute at them
Speaker 2: would unlock new features such as an AI that helps
Speaker 2: us accomplish much more than we ever could without AI,
Speaker 2: which is Sam Altman quote. And in the words of
Speaker 2: Sam again, you'd be able to get a personal AI
Speaker 2: team full of virtual experts in different areas working together
Speaker 2: to create almost anything we can imagine. I don't know, mate,
Speaker 2: you ever try creating a functional fucking business. Dipshit. Anyway,
Speaker 2: here's another one. These models were incredibly expensive to run.
Speaker 2: There has to be this way, but it was all
Speaker 2: worth it because making these models powerful was way more
Speaker 2: important than making them efficient. Because once the price of
Speaker 2: silicon comes down, and this is a refrain I've heard
Speaker 2: from multiple different people as a defense of the costs
Speaker 2: of generative AI, we would then have these powerful models
Speaker 2: that were cheaper somehow because of silicon. Now you may think,
Speaker 2: ed that sounds like a not a real argument. That
Speaker 2: just sounds like something someone said once, and it is.
Speaker 2: It is something someone said once. Anyone who knows anything
Speaker 2: about chips know how hard it is to make a
Speaker 2: new chip. And remember one of the CES episodes when
Speaker 2: I asked Max Cherney about this, should go back and
Speaker 2: listen to him. Anyway, another thing, another part of this
Speaker 2: was that as a result of this need to make bigger, huger,
Speaker 2: even bigger models, the most powerful ones, these big beautiful
Speaker 2: models that we love them. We look at the big
Speaker 2: beautiful models, we would of course need to keep buying bigger,
Speaker 2: more powerful GPUs, which would continue the American excellence of
Speaker 2: burning a bunch of money on nothing. And by following
Speaker 2: this roadmap, everybody wins. The hyperscalers get the justification they
Speaker 2: needed to create more sprawling data centers and spend massive
Speaker 2: amounts of money and open AI and their alk get
Speaker 2: to continue building powerful models, and also in video continues
Speaker 2: to make money selling GPUs. Remember I've said in the
Speaker 2: past that things were kind of a death cull. This
Speaker 2: is what this is. It's a capitalist death cull. It
Speaker 2: runs on plagiarism and hubris and the assumption that at
Speaker 2: some point all of this would turn into something meaningful. Now,
Speaker 2: I've argued for a while that the latter part of
Speaker 2: the plan was insane, that there was no profitability for
Speaker 2: these large language models. As I believe, there simply wasn't
Speaker 2: a way to make these models more efficient in a
Speaker 2: way I was raimed. The current models developed by both
Speaker 2: the hyperscalers, so Gemini from Google or Lama from Meta,
Speaker 2: and so on and so forth, and the multi billion
Speaker 2: dollar startups, if you can even fucking call them that,
Speaker 2: Open Ai and Anthropic, they're horribly inefficient. And I just
Speaker 2: made the mistake of assuming that they tried to make
Speaker 2: them more efficient, and they couldn't. Now, but what we're
Speaker 2: witnessing right now isn't some sort of weird China situation.
Speaker 2: This isn't China being Chinese and doing scary Chinese things
Speaker 2: to us. Now, what we're witnessing is the American tech
Speaker 2: industry's greatest act of hubris. It's a monument to the
Speaker 2: barely conscious stewards of so called innovation who are incapable
Speaker 2: of breaking the k fab of the fake competition where
Speaker 2: everybody makes the same products, charges about the same amount
Speaker 2: of money, and mostly innovates in the same direction. Somehow, nobody,
Speaker 2: not Google, not Microsoft, not open Ai, not Meta, not Amazon,
Speaker 2: no Oracle thought to try or was capable of creating
Speaker 2: something like deep Seek, Which doesn't mean that deep Seek's
Speaker 2: team is particularly remarkable or found anything super new, but
Speaker 2: that for all the talent, trillions of dollars of market capitalization,
Speaker 2: and supposed expertise in Americans American tech hologarchs, not one
Speaker 2: bright spark thought to try things that deep Seak had tried,
Speaker 2: which appeared to be what if we didn't use as
Speaker 2: much memory, and what if we tried synthetic data? And
Speaker 2: because the cost of model development and inference was so
Speaker 2: astronomical in the case of American models, they never assumed
Speaker 2: that anyone would try to use their position. This is
Speaker 2: especially bad considering that China's folks and AI as a
Speaker 2: strategic part of its industrial priority was really no secret,
Speaker 2: even if the ways it supported domestic companies kind of is.
Speaker 2: In the same way that the automotive industry was blindsided
Speaker 2: by China's EV manufacturers, the same is happening with AI. Fat,
Speaker 2: happy and lazy, and most of all oblivious. America's most
Speaker 2: powerful tech companies sacked back and built bigger, messier models
Speaker 2: powered by sprawling data centers of billions of dollars of
Speaker 2: GPUs from Nvidia, A bacchanalia are spending that strains our
Speaker 2: energy grid and depletes our fucking water reserves. Without it
Speaker 2: appears much consideration of whether an alternative was possible. I
Speaker 2: refuse to believe that none of these companies could have
Speaker 2: done what Deep Seek has done, which means that they
Speaker 2: either chose not to or they were so utterly myopic,
Speaker 2: so excited to burn so much money on so many
Speaker 2: parts of burning the earth, boiling lakes, and stealing from
Speaker 2: people in pursuit of further growth, that they didn't think
Speaker 2: to try. This isn't about China. It's so much fucking
Speaker 2: easier if we let it be about China. No, no, no, no.
Speaker 2: It's about how the American tech industry is incurious, lazy, entitled, directionless,
Speaker 2: and irresponsible open an anthropic of the antithesis of Silicon Valley.
Speaker 2: They're incumbents, public companies wearing startup suits. I'm willing to
Speaker 2: take on real challenges more focused on optics and marketing
Speaker 2: than they are in solving actual fucking problems, even the
Speaker 2: problems that they themselves created with their large language models.
Speaker 2: By making this about China, we ignore the root of
Speaker 2: the problem that the American tech industry is no longer
Speaker 2: interested in making good software that actually helps people. Deep
Speaker 2: seaks shouldn't be scary to Silicon Valley because they that
Speaker 2: Silicon Valley should have come up with this first. It
Speaker 2: uses less memory, fewer resources, and uses several kind of
Speaker 2: quirky workarounds to adapt to the limited compute resources available.
Speaker 2: All things that you'd previously associate with Silicon Valley, except now.
Speaker 2: Silicon Valley's only interest that the rest of the American
Speaker 2: tech industry is the rot economy. It only cares about growing,
Speaker 2: growing at all costs, even if said costs were really
Speaker 2: things you could mitigate, or if the costs themselves were
Speaker 2: self defeating. To be clear, if the alternative is that
Speaker 2: all of these companies simply did not come up with
Speaker 2: this idea that in and of itself is a damning
Speaker 2: indictment of the Valley. Was nobody thinking of this stuff?
Speaker 2: If they were, why didn't Sam Altman or Dario Amadae
Speaker 2: or sach In Adela or anyone else put serious resources
Speaker 2: into efficiency? Was it because there was no reason to?
Speaker 2: Was it because there was, if we're honest, no real
Speaker 2: competition between any of these companies? Did anybody try anything
Speaker 2: other than throwing as much compute and training data at
Speaker 2: the model as possible. It's all just so cynical and
Speaker 2: antithetical to innovation itself. Surely, if any of this shit mattered,
Speaker 2: if generative AI truly was valid and viable in the
Speaker 2: eyes of these companies, they would have actively worked to
Speaker 2: do something like Deep Seeker has done. Don't get me wrong,
Speaker 2: it appears Deep Seak employed all sorts of weird tricks
Speaker 2: to make this work, including taking advantage of distinct parts
Speaker 2: of both CPUs and GPUs to create something called a
Speaker 2: digital processing unit, essentially redefining how data is communicated within
Speaker 2: the servers running training and inference. And just this reminder,
Speaker 2: inference is the thing where when you type something in
Speaker 2: that's it infers the meaning. Just could have specified that earlier.
Speaker 2: Deepseak had to do things that a company with unrestrained
Speaker 2: access to capital and equipment wouldn't have to do, and
Speaker 2: it often used impractical and quirky methods to do so. Nevertheless,
Speaker 2: Open AI and Anthropic both have enough money and hiring
Speaker 2: power to have tried and succeeded in creating a model
Speaker 2: this efficient and capable of running on older GPUs, except
Speaker 2: what they wanted, what they actually wanted was more goddamn
Speaker 2: growth and the chance to build even bigger data centers
Speaker 2: with even more compute that they would own. Open ai
Speaker 2: is as much a lazy, cumbersome incumbent as Google or Microsoft,
Speaker 2: and it's about as innovative too. The launch of its
Speaker 2: Operator agent was a joke, a barely functional product that's
Speaker 2: allegedly meant to control your computer and take distinct actions
Speaker 2: like ordering stuff of instacark, you know, things you could
Speaker 2: do with your hands, but just to be clear, it
Speaker 2: doesn't work. You'll never guess who was really into it.
Speaker 2: Though his name is Casey Newton. He writes a blog
Speaker 2: called Platformer, and he's a man so gratingly credulous that
Speaker 2: it makes me want to fucking scream. And of course
Speaker 2: he wrote the Operator when he used it was a
Speaker 2: compelling demonstration that represented an extraordinary technological achievement. Also, somehow
Speaker 2: was significantly slower, more frustrating, and more expensive than simply
Speaker 2: doing any of these tasks yourself. Casey, of course, not
Speaker 2: to worry, had some extra thoughts about Deep Seek, that
Speaker 2: there were reasons to be worried, but that American AI
Speaker 2: lads were still in the lead, saying that deep seak
Speaker 2: was only optimizing technology that open ai and others had
Speaker 2: invented first, before saying that deep seek was only last
Speaker 2: week that open ai made available to pro plan used
Speaker 2: as a computer that can use itself. This statement is
Speaker 2: bordering on factually incorrect. It is fucking insane that Casey
Speaker 2: is still doing this. I do not want to I
Speaker 2: don't know what to do with this guy. This guy
Speaker 2: just like that. That's a fucking lie. The computer can't
Speaker 2: use itself, This shit can't just to explain what operator is,
Speaker 2: You're meant to type in something like, hey, order me
Speaker 2: some milk, Order me some milk off of vistacar. And
Speaker 2: when Casey tried this, he tried to find milk and
Speaker 2: de moine iowa. Just fucking insane. Just this is how
Speaker 2: these companies have got big. It's people like cases people
Speaker 2: are Casey, who are just like anything they show tore
Speaker 2: like God, damn, that's the most impressive thing I've seen
Speaker 2: in my life. It's a fucking fast. But let's be frank,
Speaker 2: these companies aren't building shit. Open Air and Anthropic are
Speaker 2: both limply throwing around the idea that agents are possible
Speaker 2: in an attempt to raise more money to burn, and
Speaker 2: after the launch of deep Seek, I have to wonder
Speaker 2: what any investor thinks they're investing in other than certain
Speaker 2: ones I'll get into in a bit. And to be clear,
Speaker 2: an agent is meant to be this autonomous thing which
Speaker 2: you say, hey, go and do this action, go and
Speaker 2: sell things for me, and go and email people for me.
Speaker 2: They don't really work. There are some that kind of
Speaker 2: do that are really expensive, but large Lane which model
Speaker 2: is not built for this kind of thing. But let's
Speaker 2: be honest, Deep Seek and as I said in the
Speaker 2: last episode, they've built a more efficient reasoning model. So
Speaker 2: like open aiy zo one And you'd think, well, okay,
Speaker 2: couldn't open Ai simply add on deep Seek to its models.
Speaker 2: Not really. First of all, with the way these models work,
Speaker 2: you can't just like plug it in. It's just not
Speaker 2: how it works. They could train a new model using
Speaker 2: deep Seek's techniques, but the optics of that aren't brilliant.
Speaker 2: It would be a concession. It'd be in the middle
Speaker 2: that open aiyes slipped and needs to catch up, and
Speaker 2: not to its main rival pretend rival, I mean anthropic
Speaker 2: or to like another big tech firm, but to an
Speaker 2: outgrowth of a hedge fund in China, a company that
Speaker 2: few had heard of before December and the like. Really
Speaker 2: not that many people had heard before January twenty fifth.
Speaker 2: It's very embarrassing, and this, in turn, I think will
Speaker 2: make any serious investor think twice about writing the company
Speaker 2: a blank chet. They're going to have to dip into
Speaker 2: some very bothersome pockets, and as I've said ad nauseum,
Speaker 2: this is potentially fatal as open ai needs to continually
Speaker 2: raise money, more money than any startup has ever raised
Speaker 2: in the history of anything, and it really doesn't have
Speaker 2: a path to breaking even even if they copy what
Speaker 2: Deep Sea did, because we still right now, though deep
Speaker 2: Seek is thirty times cheaper than one, we don't know
Speaker 2: if that's profit or we don't know. We haven't found out.
Speaker 2: And if open ai wants to do its own, cheaper,
Speaker 2: more efficient model, it's likely to have to create it
Speaker 2: from scratch, like I said, And while it could do
Speaker 2: distillation to make it kind of more like open ai
Speaker 2: using their own models. By the way, deep Sea taught
Speaker 2: itself using open aies outputs. Like I mentioned in the
Speaker 2: last episode, it's kind of what deep Seak already did.
Speaker 2: It already has been fed open ai bullshit. Even with
Speaker 2: open AI's much larger team and more powerful hardware, it's
Speaker 2: hard to see how creating a smaller, more efficient, and
Speaker 2: almost as powerful version of OH one benefits them in
Speaker 2: any way, because said version has well already been beaten
Speaker 2: to market by deep Seek, and thanks to deep Seek,
Speaker 2: will almost certainly have a great deal of competition for
Speaker 2: a product that to this day lacks any killer apps.
Speaker 1: Anyway, It's just it's very frustrating to me. It's very
Speaker 1: frustrating to me. It drives me a little insane. Reading
Speaker 1: all this stuff makes me feel crazy. I think you
Speaker 1: hear it in my voice, you hear the sanity stripping away.
Speaker 1: But I'm here to podcast and don't worry. But seriously, though,
Speaker 1: anyone can build on top of what deep seek has
Speaker 1: already built. Where is open Aiy's mote exactly? And where's anthropics?
Speaker 1: What are the things that make these companies worth sixty
Speaker 1: billion or one hundred and fifty billion, or oh my god,
Speaker 1: as we'll discuss in a bit, three hundred and forty
Speaker 1: billion dollars?
Speaker 2: What is the technology they own or the talent they
Speaker 2: have that justifies these valuations? Because it's kind of hard
Speaker 2: to argue that their model particularly valuable anymore. Celebrity celebrity
Speaker 2: them CULTI personality. Sam Orman, He's an artful bullshitter, and
Speaker 2: he's built a career out of being in the right
Speaker 2: places at the right times, have in the right connections,
Speaker 2: knowing exactly what to say, especially to credulous tech media
Speaker 2: ponces without the spine or inclination to push back on
Speaker 2: his more stupid claims. And already Aortman has tried to
Speaker 2: shrug off Deep Seaks Rise, admitting that while Deep Seeks
Speaker 2: are one model is impressive, particularly when it comes to
Speaker 2: its efficiency, open Ai will obviously deliver much better models,
Speaker 2: and also it's legit invigorating for open Ai to have
Speaker 2: a new competitor. Yeah, mate, sure, I'll bet your love
Speaker 2: in this. Aortmand ended his tweet where that came from
Speaker 2: with look forward to bringing you all AGI and beyond,
Speaker 2: something which I add has always been close on the
Speaker 2: horizon in Altman's world, but it's never really materialized. Then
Speaker 2: the timeline keeps moving and there's no actual proof they
Speaker 2: can do it. AGI is not fucking happening, And if
Speaker 2: it's possible in any way, it's not coming out of
Speaker 2: probabilistic models. I'm fucking sick of this and Opening I
Speaker 2: can't even lean on its relationship with Microsoft, which on Wednesday,
Speaker 2: January twenty ninth started offering deep Seek's models through its
Speaker 2: own cloud services. Opening I hasn't got shit. Deep Seek
Speaker 2: has commoditized the large language model, publishing both the source
Speaker 2: code and the guide to building your own. Whether or
Speaker 2: not someone chooses to pay deep Seek is largely irrelevant.
Speaker 2: Someone else will take what they have created and build
Speaker 2: their own, or people will start running their own deep
Speaker 2: Seek instances, renting GPUs from one of the various cloud
Speaker 2: computing firms. They don't give a shit. They'll take the
Speaker 2: money and while in video, will always find other ways
Speaker 2: to make money. Jensen Wong is amazing at this. It's
Speaker 2: going to be a hard sell for any hyperscala to
Speaker 2: justify spending billions more on GPUs to markets that now
Speaker 2: know that near identical models can be built for a
Speaker 2: fraction of the cost with older hardware. Why do you
Speaker 2: need Blackwell, which is the latest Nvidia GPU. The narrative
Speaker 2: of this is the only way to build powerful models.
Speaker 2: It doesn't really hold water anymore, and the only other
Speaker 2: selling point is that what if China does something. Well,
Speaker 2: the Chinese did something, and they've now proven that they
Speaker 2: cannot only compete with American AI companies, but doing so
Speaker 2: is possible in an efficient way that can effectively crash
Speaker 2: the market while there's being a recovery. This is still
Speaker 2: very worrying. I also want to address something real quick.
Speaker 2: A few people on Twitter have been suggesting that talking
Speaker 2: about deep Seek positively in any way is some sort
Speaker 2: of Chinese op. If you believe this, you're a fucking moron.
Speaker 2: I really must be clear, take your weirdsenophobia and go
Speaker 2: eat your own shit. I don't fucking care anymore. Yes,
Speaker 2: there are problems with China. Yes, China does something to America.
Speaker 2: This is an open source thing. This is an open
Speaker 2: source thing, and you can remove China from the equation.
Speaker 2: Because it's open source, someone else is going to use this.
Speaker 2: If your only defense is the sneaky Chinese are doing something,
Speaker 2: go to therapy and talk to the therapist about you
Speaker 2: being paranoid or racist, because it's one of them. I
Speaker 2: should also be clear, concerns about China very realistic. The
Speaker 2: Chinese government has tried to interfere with America. It's happened
Speaker 2: many times. Even if China is funding Deep Seeks models,
Speaker 2: the fact that they are open sourced means that anyone
Speaker 2: can run them and anyone can build their own. They
Speaker 2: can look under the hood. We don't have the training data,
Speaker 2: but that is it. You cannot win on a senophobic
Speaker 2: argument here. You can have realistic concerns about another foreign power.
Speaker 2: I'm not saying not to, but what I am saying
Speaker 2: is that you have to look at this realistically, and
Speaker 2: you have to take this seriously. And dismissing this as
Speaker 2: Chinese magic is stupid. It's very goddamn stupid. But like
Speaker 2: I said earlier, it also isn't clear if these models
Speaker 2: are actually going to be profitable. It's unclear who funds
Speaker 2: Deep seg like I just said, and whether it's current
Speaker 2: pricing is actually sustainable. But they're likely going to be
Speaker 2: a damn site more profitable than anything open ai is
Speaker 2: currently selling. After all, open ai loses money on every
Speaker 2: single transaction, even their two hundred dollars a month Chat
Speaker 2: GPT Pro subscription, And if open ai cuts its prices
Speaker 2: to compete with deep sek, its losses are only going
Speaker 2: to deepen. And as I've said again and again this
Speaker 2: is also deeply cynical, because it's obvious that none of
Speaker 2: this was ever about the proliferation of generative AI or
Speaker 2: making sure that generative AI was accessible. Putting aside my
Speaker 2: very obvious personal beliefs for a second, it's fairly obvious
Speaker 2: why these companies, the big hyperscalers and open ai and
Speaker 2: Anthropic wouldn't want to create something like deep seek because
Speaker 2: creating an open source model that uses less resources means
Speaker 2: that open Ai, Anthropic and their associated hyperscalar findom clients
Speaker 2: would lose their soft monopoly on large language models. Now
Speaker 2: what does that mean, I'll explain before deep seek to
Speaker 2: make a competitive large language model like GPT four zero,
Speaker 2: as in one that you can actually commercialize, required exceedingly
Speaker 2: large amounts of capital, and to make larger ones effectively
Speaker 2: required you to kiss the ring or the ass of Microsoft,
Speaker 2: Google or Amazon. While it isn't clear what it cost
Speaker 2: to train open aizo one reasoning model, we know that
Speaker 2: GPT four to ZH cost in excess of one hundred
Speaker 2: million dollars, and oh one is a more complex model
Speaker 2: would likely cost even more. We also know that open
Speaker 2: AI's training and inference costs in twenty twenty four were
Speaker 2: around seven billion dollars, meaning that either refining current models
Speaker 2: or building new ones is quite costly. The mythology of
Speaker 2: both open AI and Anthropic is that these large amounts
Speaker 2: of capital weren't just necessary, but the only way to
Speaker 2: do this. While these companies ostensibly compete, neither of them
Speaker 2: seemed concerned about doing so. As actual businesses that made
Speaker 2: products that were say, cheaper and more efficient to run,
Speaker 2: you know, they made more money than they cost because
Speaker 2: in doing so, these companies would break the illusion that
Speaker 2: the only way to create powerful artificial intelligence was to
Speaker 2: hand billions of dollars to one of two companies and
Speaker 2: build giant data centers to build even larger language models.
Speaker 2: This is aisrot economy, two lumbering companies claiming that their
Speaker 2: startups creating a narrative that the only way to build
Speaker 2: the future to keep growing, to build more data centers,
Speaker 2: to build larger language models, to consume more training data
Speaker 2: with each infusion of capital, GPU purchase, and data center
Speaker 2: build out, creating an infrastructural mote that always leads back
Speaker 2: to one of a few tech hyperscalers. Open an anthropic
Speaker 2: need the narrative to say, buy more GPUs and build
Speaker 2: more data centers, because in doing so they create the
Speaker 2: conditions of that infrastructural monopoly. Because the terms forget about
Speaker 2: building software that does stuff for a second, were implicitly
Speaker 2: that smaller players cannot enter the market because the market
Speaker 2: is defined as large language models that cost hundreds of
Speaker 2: millions of dollars and require access to more compute than
Speaker 2: any startup could ever reasonably access without the infrastructure that
Speaker 2: a public tech company delivers. Remember, neither Anthropic nor open
Speaker 2: AI has ever marketed themselves based on the products they
Speaker 2: actually build. Large language models are in and of themselves,
Speaker 2: fairly bland software products, which is why we've yet to
Speaker 2: see any killer apps. This isn't a particularly exciting pick
Speaker 2: to investors or the public markets because there's no product, innovation,
Speaker 2: or business model to point to, and if they'd actually
Speaker 2: tried to productize it and turn it into a business,
Speaker 2: it's quite obvious at this point that there really isn't
Speaker 2: a multi trillion dollar industry for generative AI. Look at
Speaker 2: Microsoft and their attempts to strong arm Copilot into Microsoft
Speaker 2: three sixty five. Both personally and commercially, Nobody said Wow,
Speaker 2: this is great. When they demanded use copilot in word,
Speaker 2: Lots of people, however, asked why am I being charged
Speaker 2: significantly more for a product that I don't want or
Speaker 2: care about. Open Ai only makes twenty seven percent of
Speaker 2: its revenue from selling access to its models, so allowing
Speaker 2: people to use their models to build products around a
Speaker 2: billion dollars of annual recurring revenue, by the way, with
Speaker 2: the rest of their money coming in about two point
Speaker 2: seven billion dollars last year coming from subscriptions to chat GPT.
Speaker 2: If you ignore the hype, open Air and Anthropic are
Speaker 2: actually deeply boring software businesses with unprofitable, unreliable products prone
Speaker 2: to hallucinations, and then new products such as open Aisaura
Speaker 2: cost way too much money to both run and trained
Speaker 2: to get the results that well, they suck. They're not good.
Speaker 2: Even open a Eyes pushing to the federal government with
Speaker 2: the release of chat GPT, GOV is unlikely to reverse
Speaker 2: its dismal fortunes. Seriously, think about it. I'm sure some
Speaker 2: of you are going to say, well, Trump will just
Speaker 2: give them money. These motherfuckers need way more money than
Speaker 2: Trump is going to give them, and why would Trump
Speaker 2: bet on a loser? Why would Trump be like, oh, yeah,
Speaker 2: I'm going to give more money to this company that
Speaker 2: does the same thing. He doesn't understand it this shit,
Speaker 2: and he probably just looks at Samut and goes, now,
Speaker 2: that's a kind of new money I don't like. But
Speaker 2: to make this more than a deeply boring software business,
Speaker 2: Open iron Anthropic needed larger models, and they needed them
Speaker 2: to get larger generally in perpetuity, and for the story
Speaker 2: to always be that there was only one way to
Speaker 2: build the future, and that the future cost hundreds of
Speaker 2: billions of dollars, and that only the biggest geniuses, who
Speaker 2: all happened to work in the same two or three places,
Speaker 2: were capable of doing it. Post deep Seek, there really
Speaker 2: isn't a compelling argument for investing hundreds of billions of
Speaker 2: dollars of capex in data centers, or buying new GPUs,
Speaker 2: or even pursuing large language models as they currently stand.
Speaker 2: It's possible, and deep Seek, through its research papers, explained
Speaker 2: in detail how to build models competitive with both of
Speaker 2: open AI's leading models, and that's assuming you don't simply
Speaker 2: build on top of the ones that deep Seek released.
Speaker 2: It also seriously calls into question what it is you're
Speaker 2: paying open Ai for in its various subscriptions, most of
Speaker 2: which other than the two hundred dollars a month pro subscription,
Speaker 2: have hard limits on how much you can use their
Speaker 2: most advanced reasoning models. One thing we do know, though,
Speaker 2: is that open ai and Anthropic will now have to
Speaker 2: either drop the price of accessing their models and potentially
Speaker 2: even the cost of their subscriptions too. I'd argue that
Speaker 2: despite the significant price difference between O one and deep
Speaker 2: Seek's are one reasoning model, the real danger to both
Speaker 2: open ai and Anthropic is deep Seek V three, which
Speaker 2: competes with GPT four h, which is their general purpose model.
Speaker 2: By the way, and as on recording this episode, by
Speaker 2: the way, news broke the Alibaba, a bohemoth thie out
Speaker 2: of China in its own right, has created its own
Speaker 2: model that outsperforms Deep Seek and yet definitely dive into it.
Speaker 2: But if it's true, it's only going to pile on
Speaker 2: the price pressure, though I kind of wonder what they
Speaker 2: could possibly do. Is it going to be cheaper, because
Speaker 2: if it's more powerful, that's just like it doesn't really
Speaker 2: change shit anyway. Though Deep Seek's narrative shift isn't just
Speaker 2: about commoditizing lllms at large, but commoditizing the most expensive
Speaker 2: ones run by two monopolists backed by three other monopolists.
Speaker 2: I mean, the magic's died. There's no halo around Sam
Speaker 2: Moltman or Dario ama Day's head anymore, as their only
Speaker 2: real argument was we're the only ones that can do this,
Speaker 2: something that nobody should have believed in the first place.
Speaker 2: Up until this point. People believe that the reason these
Speaker 2: models were so expensive was because they had to be,
Speaker 2: and that we had to build more data centers and
Speaker 2: buy more silicon because that's just how things worked. They
Speaker 2: believe that reasoning models were the future, even if members
Speaker 2: of the media didn't really seem to understand what reasoning
Speaker 2: models did or why they mattered, and that as a result,
Speaker 2: they had to be expensive because open Ai and their
Speaker 2: elk were just so fucking smart, even if it wasn't
Speaker 2: obvious what reasoning men or what it allowed you to do,
Speaker 2: or what the products were. It's just very annoying, and
Speaker 2: now we're going to find out, by the way, because
Speaker 2: reasoning is now commoditized along with large language models in general.
Speaker 2: Funnily enough, the way that deep seek may have been
Speaker 2: trained using at least in part synthetic data. Also pushes
Speaker 2: against the paradigm that these companies even need to use
Speaker 2: other people's training data, though their argument, of course will
Speaker 2: be that they need more training data always. We also
Speaker 2: don't know the environmental effects, by the way, with deep SEEK,
Speaker 2: because even if it's cheaper, these models still require those
Speaker 2: energy guzzling GPUs to run, and they're running at full tilt.
Speaker 2: In any case, if I had to guess, the result
Speaker 2: would be that the markets are going to be far
Speaker 2: less tolerant of generative AI, and the idea that generative
Speaker 2: AI is the future. Open AI and anthropic no longer
Speaker 2: really have motes. Unless, well, there's another idea. What if
Speaker 2: there was a huge fucking idiot with a lot of money.
Speaker 2: How about billionaire dipshit Massioshi's son, the CEO of soft Bank,
Speaker 2: a multinational investment firm that's rumored to be investing anywhere
Speaker 2: from fifteen to twenty five billion dollars in Open AI
Speaker 2: in a round that values the company and astonishing three
Speaker 2: hundred and forty billion dollars as part of a round
Speaker 2: of up to forty billion dollars. Now you may think, damn,
Speaker 2: this is a sign that open ai is going to
Speaker 2: make it. But I must remind you how bad soft
Speaker 2: Bank is at investing. They put sixteen billion dollars into
Speaker 2: famously awful real estate company we Work and managed to
Speaker 2: lose I think eight hundred million dollars on the DoorDash
Speaker 2: Ipo and one point eight billion dollars on their investment
Speaker 2: in Uber, and in both cases did so because they
Speaker 2: were desperate to bandwagon on to supposedly your fire bets,
Speaker 2: either just before they crashed away after an investment made sense.
Speaker 2: According to the Wall Street Journal, SoftBank would lead this
Speaker 2: insane forty billion dollar round in the company and would
Speaker 2: and I quote help assemble investors for the rest of
Speaker 2: the round. In doing so, SoftBank would also become open
Speaker 2: AI's largest investor, replacing Microsoft. And yes, SoftBank was the
Speaker 2: largest investor in whe Work before it went tits up.
Speaker 2: It's also important to remember the open ai is pledged
Speaker 2: to put eighteen to nineteen billion dollars to fund the
Speaker 2: Stargate data center project. Along with you guessed SoftBank will
Speaker 2: be committing the same amount. This is on some level,
Speaker 2: soft Bank handing money to itself to invest in data
Speaker 2: centers to prop up an industry that's dying. Now. This
Speaker 2: is a developing story, but it's hard to imagine any
Speaker 2: serious contribution from any respectable investor at this point. My
Speaker 2: money is on a few VC firms desperately scraping at
Speaker 2: the bottom and the barrel quarter of a billion here,
Speaker 2: quarter of a billion there. Maybe in Vidia chucks in
Speaker 2: some and on the subject of barrels of stuff, I
Speaker 2: also expect money from the Kingdom of Saudi Arabia and
Speaker 2: its associated venture arms. You're going to see a few
Speaker 2: maybe Andrews and Horowitz gets involved. They don't think much.
Speaker 2: It's just very fucking silly, and I don't know how
Speaker 2: this works out. Open AI burns money, and even if
Speaker 2: they somehow make more efficient models, the actual total addressable
Speaker 2: market of generative AI is actually pretty small. Microsoft said
Speaker 2: in their recent earnings they made twelve to thirteen billion
Speaker 2: dollars of arr on AI. Just to be clear, that's
Speaker 2: not profit and that's not a business unit. There's no
Speaker 2: AI business unit, which means that that is just spread
Speaker 2: across delivering cloud compute for AI copilot on Microsoft three
Speaker 2: sixty five products, which by the way, no one likes.
Speaker 2: They're having trouble selling and other associated copilot products they sell.
Speaker 2: I don't think and like twelve to thirteen billion dollars
Speaker 2: across four quarters. That's not actually good at all. It's
Speaker 2: just very silly. All of this is so silly, and
Speaker 2: when I think about it, too hard to feel a
Speaker 2: little crazy. Open Ai makes three seven billion dollars in revenue,
Speaker 2: and they do so, as I mentioned, primarily from chad
Speaker 2: GPT subscriptions. Even if that somehow and it won't by
Speaker 2: the way, turns into three point seven billion dollars of
Speaker 2: profit or even ten billion dollars of profit a year,
Speaker 2: that's less than the profits in a single quarter of
Speaker 2: any given hyperscaler. It would be respectable, sure, But three
Speaker 2: hundred and forty billion dollar valuation, I guess makes sense
Speaker 2: if it was profit. It doesn't make sense if it's
Speaker 2: not profitable. Though it also isn't obvious how open ai
Speaker 2: would actually provide any liquidity to investors, by which I
Speaker 2: mean allow them to sell their stock beyond selling shares
Speaker 2: of people that work at open ai, like people who
Speaker 2: work there who have been given stock runts selling that
Speaker 2: to another investor, a really dumb guy maybe, And as
Speaker 2: in the side, SoftBank bought one point five billion dollars
Speaker 2: of stock from open ai employees and the tender at
Speaker 2: the end of November twenty twenty four. Just a note
Speaker 2: for you. They could also take the company public, but
Speaker 2: with the unit economics of this fucking company, which boil
Speaker 2: down by the way to our products, lose billions of
Speaker 2: dollars and are extremely compoditized. I'm not really sure what
Speaker 2: the plan is here. The fundamental problems that open ai
Speaker 2: has are not solved by throwing more money at the problem.
Speaker 2: This hasn't worked before and it won't work this time.
Speaker 2: They're burning cash and in soft banks case, it isn't
Speaker 2: obvious what it is they're getting from open ai. Is
Speaker 2: it the chance to continue an industry wide con the
Speaker 2: chance to participate in a catalyst death cult. I don't know.
Speaker 2: Maybe it's the chance to burn money at a faster
Speaker 2: rate than we work ever could dream of. Will this
Speaker 2: be the time that Microsoft, Amazon, and Google just drop
Speaker 2: open ai and Anthropic and make their own models based
Speaker 2: on deep Seek's work. What incentive is there for the
Speaker 2: hyperscalers to keep funding open ai and Anthropic. They hold
Speaker 2: all the cards. The GPU is the infrastructure, and in
Speaker 2: the case of Microsoft, non revocable licenses that permit them
Speaker 2: unfettered use and access to open AI's tech, and there's
Speaker 2: little stopping the hyperscalers from building their own models and
Speaker 2: just dumping them entirely. In fact, Microsoft might actually be
Speaker 2: a little glad to see soft bare become the biggest
Speaker 2: investor and pick up the tab for open AI's expenses.
Speaker 2: I can imagine satching Adella texting Semilt and being like, no,
Speaker 2: don't take that money. Hell, well, don't do oh I'd
Speaker 2: hate that. I'd hate if this was someone else's problem.
Speaker 2: And the stargate thing, by the way, is an attempt
Speaker 2: to that up to five hundred billion thing. It's just
Speaker 2: bollocks whatever. It's an attempt to remove themselves from Microsoft.
Speaker 2: And Microsoft actually allowed open ai to alter their deal
Speaker 2: so that they could get cloud compute from others. Now
Speaker 2: at the time, people were like, yeah, man, this is
Speaker 2: a good thing. This shows open ai will be independent. No,
Speaker 2: it doesn't. It just means that they're gonna be under
Speaker 2: Masayoshi's son, now the funniest dumbest man in investing. I
Speaker 2: love Massa Yoshi's son. I think it's nice that we
Speaker 2: have an insane guy who isn't instantly murderous in our
Speaker 2: lives anyway. Anyway, though, as I've said before, I believe
Speaker 2: we're at PKI and now that generative AI has been commoditized,
Speaker 2: the only thing that open AI and Anthropy have left
Speaker 2: other than a pile of cash is their ability to innovate,
Speaker 2: and I don't think they're capable of doing so. And
Speaker 2: because we sit in the ruins of Silicon Valley with
Speaker 2: our biggest startups all doing the same thing in the
Speaker 2: least efficient way possible, living at the beck and call
Speaker 2: of public companies with multi trillion dollar market caps, everyone
Speaker 2: is trying to do the same thing in the same way,
Speaker 2: based on the fantastical marketing nonsense of a succession of
Speaker 2: directionless rich guys that all want to create Americas next
Speaker 2: top monopoly. It's time to wake up and accept that
Speaker 2: there was never any kind of AI arms race, and
Speaker 2: that the only reason that Hyperscale has built so many
Speaker 2: data centers and bought so many GPUs is because they're
Speaker 2: run by people that don't experience real problems and thus
Speaker 2: don't know what problems real people face. Generator AI does
Speaker 2: not solve any trillion dollar problems, nor does it create
Speaker 2: outcomes that are profitable for any particular business. Deep Seeks
Speaker 2: models are cheaper to run. But the real magic trick
Speaker 2: they pulled is that they showed how utterly replaceable a
Speaker 2: company like open AI and buy extension any LLM company
Speaker 2: really is. There Really isn't anything special about any of
Speaker 2: these companies. They have no moat, their infrastructural advantages moot,
Speaker 2: and their hordes of talent are relatively irrelevant. What Deep
Speaker 2: Seek is proven isn't just technological, it's philosophical. It shows
Speaker 2: that the scrappy spirit of Silicon Valley builders is dead,
Speaker 2: replaced by a series of different management consultants that lead
Speaker 2: teams of engineers to do things based on vibes. You
Speaker 2: may ask if all of this means generative AI suddenly
Speaker 2: gets more prevalent after all. Sech in Adella of Microsoft
Speaker 2: quoted Yvonne's paradox, which posits that when resources are made
Speaker 2: more efficient, their use increases. Sadly, I hypothesize that something
Speaker 2: else happens right now. I do not believe that there
Speaker 2: are companies that are stymied by the pricing the Open
Speaker 2: AI and their ILK coffer, Nor do I think there
Speaker 2: are many companies or use cases that don't exist because
Speaker 2: large language models are too expensive. AI companies took up
Speaker 2: a third of all venture capital funding last year, and
Speaker 2: on top of that, it's fairly try reasoning models like
Speaker 2: O one and make a proof of concept without having
Speaker 2: to make an entire operational company. Sheer open ai barely
Speaker 2: has one. I don't think anyone has been on the
Speaker 2: sidelines of generative II due to costs, and remember, few
Speaker 2: seem to be able to come up with great use
Speaker 2: case for one or other reasoning models anyway, and deep
Speaker 2: Seek's models, while cheaper, don't have any new functionality. As
Speaker 2: a result, I don't really see anything changing beyond the
Speaker 2: eventual collapse of the API market, which is the way
Speaker 2: you plug these models into things. For companies like Anthropic
Speaker 2: and open AI, large language models and reasoning models the niche.
Speaker 2: The only reason that chat GPT became such a big
Speaker 2: deal is because the tech industry is no other growth ideas,
Speaker 2: and despite the entire industry and public markets screaming about it,
Speaker 2: I can't think of any mass market product that really matters.
Speaker 2: Even if deep Seek doesn't land the fatal blow, it
Speaker 2: could set the foundations from another company to drag open
Speaker 2: AI's carcass out behind the barn and hit it with
Speaker 2: a big stick. One way in which this entire fast
Speaker 2: could fall as if nasty Mark Zuckerberg decides he wants
Speaker 2: to splay destroy the entire market for llms. Meta has
Speaker 2: already formed four separate war rooms to break down how
Speaker 2: Deep Seep did it and apparently to quote the information
Speaker 2: in pursuing Lama, which is their large language model. CEO
Speaker 2: Mark Zuckerberg wants to commoditize AI models so that the
Speaker 2: applications that use such models, including metas, generate more money
Speaker 2: than the sales of the AI models themselves. That could
Speaker 2: hurt metas AI rivals such as open ai and Anthropic,
Speaker 2: which were on pace to generate billions of dollars in
Speaker 2: revenue from such sales and lose billions. Fucking hell. I
Speaker 2: love the information, but can you add the most important bit?
Speaker 2: But I could absolutely see Meta releasing its own version
Speaker 2: of deep Seek's models. They've got the GPUs and Mark
Speaker 2: Zuckerberg can never be fired, meaning that if he simply
Speaker 2: decided to throw billions of dollars into specifically creating his
Speaker 2: own deep discounted alms to wipe out open ai he
Speaker 2: absolutely could. After all, a few weeks back, Mark Zuckerberg
Speaker 2: said that Meta would spend between sixty and sixty five
Speaker 2: billion dollars in CAPITALGICS spenditures in twenty twenty five. And
Speaker 2: this was before the deep Seek situation. And I imagine
Speaker 2: the markets would love a more modest proposal that involves
Speaker 2: Meta offering a chat GPT be to simply to fuck
Speaker 2: over Sam Altman. And that's the thing. Chat GPT is
Speaker 2: big because everybody's talking about AI, and chat GPT is
Speaker 2: the big brand in AI. It is not essential, and
Speaker 2: it's only been treated as such because the media and
Speaker 2: the markets ran away with a narrative that they barely understood.
Speaker 2: Deep Seek pierced that narrative because believing said narrative also
Speaker 2: required you to believe that Sam Moortman is a magician
Speaker 2: versus an extremely shitty CEO that burned a bunch of money.
Speaker 2: And I don't believe that, even before Deep Seek, that
Speaker 2: Altman's peers really bought into the hype. Sure you can
Speaker 2: argue that deep Seek just built on top of software
Speaker 2: that already existed thanks to open Ai. Thank you Casey,
Speaker 2: by the way, but this beg's a fairly obvious question
Speaker 2: why didn't open ai build on top of software invented
Speaker 2: by open ai? And here's another question, why is it
Speaker 2: goddamn matter? In any case, the massive expense of running
Speaker 2: generative models hasn't been the limiter on their deployment, of
Speaker 2: their success. You can blame that on the fact that
Speaker 2: they as a piece of technology and neither artificial intelligence
Speaker 2: nor capable of providing the kind of meaningful outcomes that
Speaker 2: would make them the next smartphone or cloud computing. Honestly,
Speaker 2: it's all been a con It's been a painfully obvious
Speaker 2: one one had been screaming about since February, since when
Speaker 2: I started this podcast, trying to explain that beneath the
Speaker 2: hype was an industry that provided monister at best outcomes
Speaker 2: rather than any kind of next big thing. Without reasoning
Speaker 2: is its magical new creation. Open ai really doesn't have
Speaker 2: anything left. Agents aren't coming, large language models aren't going
Speaker 2: to build them. AGI isn't coming either. There's no proof
Speaker 2: it's possible. All of this is fucking flim flamm to
Speaker 2: cover up how mediocre and unreliable the fundament of the
Speaker 2: supposed AI revolution really was. All of this money, all
Speaker 2: of this energy, and all of this talent was wasted
Speaker 2: thanks to markets that don't actually do anything, markets that
Speaker 2: don't make for good companies, just growth hogs, and a
Speaker 2: media industry that fails to hold the powerful to account.
Speaker 2: And it looks like everything got broken by some random
Speaker 2: outgrowth of a Chinese hedge fund. It's so ridiculous, it's
Speaker 2: so sickening. I can't believe it. While I can totally
Speaker 2: believe it, I'm actually surprised they didn't come up with
Speaker 2: this idea myself, Just the idea that someone could do
Speaker 2: this cheaper. It makes me go insane. And what's more
Speaker 2: insane is that Open Eye is still going to be
Speaker 2: able to raise that round. But I think we're approaching
Speaker 2: the end of days. I'm not calling the end of
Speaker 2: the bubble yet. I refuse to do that. I'm not
Speaker 2: going to do that. What I am going to say
Speaker 2: is it's deflating, and I am going to say I
Speaker 2: have no idea how they reinflate it. A bunch more
Speaker 2: money isn't going to change anything. These companies are washed.
Speaker 2: Sam Morton's washed. He's the Mark Sanchez of the tech industry,
Speaker 2: and he's so sickening. All of them are so sickening.
Speaker 2: Imagine if this money had gone anywhere else. Imagine if
Speaker 2: it had gone into batteries. Imagine if it had gone
Speaker 2: into climate stuff. Imagine if gone somewhere useful. Imagine if
Speaker 2: instead of spending billions on this dog shit, they actually
Speaker 2: fix their problems, they actually fix the products that they've
Speaker 2: made worse. But the problem is that the rot economy
Speaker 2: is in control. That the growth at all costs mindset
Speaker 2: is all that you see in the tech industry, and
Speaker 2: Silicon Valley needs to repent. Silicon Valley needs to change
Speaker 2: its ways because when the bubble bursts, and I really
Speaker 2: think it will, the destruction that follows will be horrifying
Speaker 2: and it will hit workers. It will hit tens of
Speaker 2: thousands of tech workers, and it will affect the markets.
Speaker 2: And after that, the markets are going to realize something.
Speaker 2: The tech industry doesn't have anything left, they don't have
Speaker 2: another growth market. They're out. They're all out. And I
Speaker 2: look forward to telling you how I look forward to
Speaker 2: talking about it when it happens. And I'm so grateful
Speaker 2: for you listening. Thank you for listening to Better Offline.
Speaker 3: The editor and composer of the Better Offline theme song,
Speaker 3: is Matasowski. You can check out more of his music
Speaker 3: and audio projects at Matasowski dot com, M A T
Speaker 3: T O S O W s ki dot com. You
Speaker 3: can email me at easy at better offline dot com
Speaker 3: or visit better offline dot com to find more podcast
Speaker 3: links and of course, my newsletter. I also really recommend
Speaker 3: you go to chat dot where's youreaed dot at to
Speaker 3: visit the discord, and go to our slash Better Offline
Speaker 3: to check out our reddit. Thank you so much for listening.
Speaker 3: Better Offline is a production of cool Zone Media. For
Speaker 3: more from cool Zone Media, visit our website cool Zonemedia
Speaker 3: dot com, or check us out on the iHeartRadio app,
Speaker 3: Apple Podcasts, or wherever you get your podcasts.