Speaker 1: Ze Media.
Speaker 2: Hello one, Welcome to Better Offline. I'm your host ed
Speaker 2: Zi Trun. This is part two of our three parts
Speaker 2: serious on how to argue with an AI booster. When
Speaker 2: we last left off, I'd started talking about some of
Speaker 2: the most common and vacuous talking points used by those
Speaker 2: who defend the generative AI industry and why a lot
Speaker 2: of them are wholly without merit. These are the booster quips,
Speaker 2: assertions that if you don't know much, sound convincing but
Speaker 2: are easily disproven with the right information. And in that
Speaker 2: last episode we addressed the quips that say were in
Speaker 2: the early days of AI and that people doubted smartphones
Speaker 2: and the internet. Things they didn't do just like they
Speaker 2: did generative AI, which they should do in the cycle
Speaker 2: of grief. That's the denial stage. Now we're going to
Speaker 2: move on to bargaining. This is just that the dot
Speaker 2: com boom, even if of this collapses, the overcapacity will
Speaker 2: be practical for the market like the fiber boom was.
Speaker 2: All right, folks, time for a little history. You know me,
Speaker 2: I'll love me some mystery. The fiber boom began after
Speaker 2: the Telecommunications Act of nineteen ninety six deregulated large parts
Speaker 2: of America's communications infrastructure, creating a massive boom, a five
Speaker 2: hundred billion dollars one to be precise, primarily funded with debt. Obviously,
Speaker 2: we're still using the infrastructure bought during that boom, and
Speaker 2: this fact is used as a defense of the insane
Speaker 2: capex spending surrounding generative AI. High speed Internet is useful, right, sure,
Speaker 2: But the fiber optic boom period was also defined by
Speaker 2: a gluttony of overinvestment, ridiculous valuations, and genuine, outright fraud.
Speaker 2: In any case, this is not remotely the same thing,
Speaker 2: and anyone making this point needs to learn the very
Speaker 2: fucking basics of technology. Let's get going now. The fiber
Speaker 2: optic cable of this era is mostly owned by a
Speaker 2: few companies. Forty two percent of Nvidia's revenue is from
Speaker 2: the Magnificent seven, and the companies buying these gps are
Speaker 2: for the most part not going to go bust once
Speaker 2: the AI bubble bursts. You can also already get the
Speaker 2: cheap fiber of this era too cheap aigpus already here.
Speaker 2: GPUs are depreciating assets, meaning that the good deals are
Speaker 2: already happening. I found an in Vidia a one hundred
Speaker 2: for two or three thousand dollars multiple times on eBay,
Speaker 2: and you can get the h one hundreds which are
Speaker 2: more powerful for well, I think thirty grand and those
Speaker 2: things go forty five thousand retails, So not brilliant. Aigpus
Speaker 2: also do not have a variety of use cases and
Speaker 2: are limited by Kuda, in Vidia's programming libraries and APIs.
Speaker 2: Aigpus are integrated into applications using this language Kuda, and
Speaker 2: this is specifically in Vidia's programming language. While there are
Speaker 2: other use cases scientific simulations, image and video processing, data
Speaker 2: science and analytics, medical imaging, and so on. Kuder is
Speaker 2: not a one size fits or digital panacea. While fiber
Speaker 2: optic cable was, and it was also put everywhere, it
Speaker 2: truly did set up the future. What are the these
Speaker 2: GPUs setting up exactly? Also, widespread access to cheaper GPUs
Speaker 2: has already happened, and what new use cases are there?
Speaker 2: What are the new innovative things we can do? As
Speaker 2: a result of the AI bubble, there are now many, many, many, many,
Speaker 2: many different vendors to get access to GPUs. You can
Speaker 2: pay at an hourly rate. Who knows if it's probitable,
Speaker 2: but you can do it, and sometimes you can get
Speaker 2: them for as little as one dollars an hour, which
Speaker 2: is really not good. It definitely isn't making them money
Speaker 2: but putting the financial collapse aside. While they might be
Speaker 2: cheaper when the AI bubble bursts, does cheaper actually enable
Speaker 2: people to do new stuff? Is costs the problem because
Speaker 2: I think the costs are going to go up. But
Speaker 2: even if they weren't going up, what are the things
Speaker 2: that you could do that a new What is the
Speaker 2: prohibitive cost? No one can actually answer this question because
Speaker 2: the answer isn't fun. GPUs are built to shove massive
Speaker 2: amounts of compute into one specific function, again and again
Speaker 2: and again, like generating the output of model, which remember,
Speaker 2: mostly boils down to complex maths. Unlike CPUs, a GPU
Speaker 2: can't easily changed tasks or handle many little distinct operations,
Speaker 2: meaning that these things aren't going to be adopted for
Speaker 2: another mass market use case because there probably isn't one.
Speaker 2: In simpler terms, this was not an infrastructure built out.
Speaker 2: The GPU boom is a heavily centralized, capital expenditure funded
Speaker 2: asset bubble where a bunch of chips will sit in
Speaker 2: warehouses or kind of fallow data centers waiting for somebody
Speaker 2: to make up a use case for them. And if
Speaker 2: an endearing one existed, we'd already have it, because we
Speaker 2: already have all the fucking GPUs. Now here's a really
Speaker 2: big boost e quip and I have been looking forward to.
Speaker 2: I get a lot of people asking you about this.
Speaker 2: I'm ed, you're so stupid. Why am I stupid? Exactly? Well,
Speaker 2: five really smart guys got together and wrote AI twenty
Speaker 2: twenty seven, which is a very real sounding extrapolation that
Speaker 2: shut the fuck up, shut up, shut up. AI twenty
Speaker 2: twenty seven is fan fiction. If you were scared by this,
Speaker 2: and you're not a booster, you shouldn't feel bad. By
Speaker 2: the way this was written to scare you. By the way,
Speaker 2: if you don't know what it is I'm talking about,
Speaker 2: you should consider yourself lucky. It's essentially a piece of
Speaker 2: speculative fiction that describes where GENAI companies get fatter models
Speaker 2: that get exponentially better, and the US and China are
Speaker 2: in brailed in an AI arms race. It's really silly.
Speaker 2: It's so very silly, and I call it fan fiction
Speaker 2: because it is. If we're thinking about this in purely
Speaker 2: intellectual terms. It's up there with my immortal and no,
Speaker 2: I'm not explaining that you can google that one for yourselves.
Speaker 2: It doesn't matter if all the people writing the fan
Speaker 2: fiction are scientists or that they have the right credentials.
Speaker 2: They themselves said that AI twenty twenty seven is a
Speaker 2: guess an extrapolation, which means guess with expert feedback, which
Speaker 2: means someone editing your fan fiction and involves experience that
Speaker 2: open AI. There are people that worked on the shows
Speaker 2: they write fan fiction about. We're not even insulting fan fiction.
Speaker 2: By the way, go nuts, you're more You are one
Speaker 2: hundred times more ethically positive than these people. At least
Speaker 2: you admits fan fiction could knuckles get pregnant. I'm sure
Speaker 2: somebody's found out. I'm not going to go line by
Speaker 2: line and cut this any more than I'm going to
Speaker 2: go and do a lengthy takedown of someone's erotic Bancho
Speaker 2: Kazoui's story, because both are fictional. The entire premise of
Speaker 2: this nonsense is that at one point someone invents a
Speaker 2: self learning agent that teaches itself stuff, and it does
Speaker 2: a bunch of other stuff requiring a Brazilian compute points
Speaker 2: with different agents with different numbers after them. There is
Speaker 2: no proof that this is possible. Nobody has done it,
Speaker 2: and nobody will do it. AA twenty twenty seven was
Speaker 2: written specifically to fool people that want to be fooled,
Speaker 2: with big chants and the right technical terms used to
Speaker 2: lull the credulus into a wet dream and a New
Speaker 2: York Times column where one of the writers folds their
Speaker 2: hands and looks worried. It was also written to scare
Speaker 2: people that are already scared. It makes big, scary proclamations
Speaker 2: with tons of links to stuff that looks really legitimate,
Speaker 2: but when you piece it all together, is literally just
Speaker 2: fan fection, except really not that endearing. My personal favorite
Speaker 2: part is mid twenty twenty six China Wakes Up, which
Speaker 2: involves China's intelligence agents. He's trying to steal Open Brains
Speaker 2: agent no idea who this companicably referring to please email
Speaker 2: if you can work it out to I don't care
Speaker 2: at business dot org before the headline of AI take
Speaker 2: some jobs. After Open Brain releases a model. Oh God,
Speaker 2: I'm so bored even fucking talking about this now. Sarah
Speaker 2: lyonce puts this well, arguing that AI twenty twenty seven
Speaker 2: and AI in general is no different from the spurious
Speaker 2: spectral evidence used to accuse someone of being a witch
Speaker 2: during the Salem witch trials, and I quote and the
Speaker 2: evidence is spectral. What is the real evidence in AI
Speaker 2: twenty twenty seven beyond trust us and vibes? People who
Speaker 2: wrote it site themselves in the piece, do not demand
Speaker 2: I take this seriously. This is so clearly a marketing
Speaker 2: device to scare people into buying your product before this
Speaker 2: imaginary window closes. Don't call me stupid for not falling
Speaker 2: for your spectral evidence. My whole life, people have been
Speaker 2: saying artificial intelligence is around the corner, and it never arrives.
Speaker 2: I simply do not believe a chatbot will ever be
Speaker 2: more than a chat pot, and until you show me
Speaker 2: it doing that, I will not believe it anyway. AI
Speaker 2: twenty twenty seven is fan fiction nothing more. Just because
Speaker 2: it's full of fancy words and has five different grifters
Speaker 2: on its byline doesn't mean a goddamn thing. Now now, now, now, now, folks,
Speaker 2: we've all been waiting for this moment, and here's the
Speaker 2: ultimate booster quip the cust of inference is coming down.
Speaker 2: This proves that things are getting cheaper. And here's a
Speaker 2: bonus trick for you before I get to my ben
Speaker 2: Here we go, ask them to explain whether things have
Speaker 2: actually got cheaper, and if they say they have, ask
Speaker 2: them why there are no profitable AI companies. If they
Speaker 2: say they're in the growth stage, ask them why there
Speaker 2: are no profitable AI companies. Again, I'd say it's been
Speaker 2: several years and not got one. At this point they
Speaker 2: should try and kill you. But really, I'm about to
Speaker 2: be petty. I'm about to be petty for a fucking
Speaker 2: reason though. In an interview on a podcast from earlier
Speaker 2: this year that I will not even quote because the
Speaker 2: journalist in question did not back me up and it
Speaker 2: pisses me off, Journalist Casey Newton said the following about
Speaker 2: my work.
Speaker 1: You don't think that that kind of flies in the
Speaker 1: face of same altman saying that we need billions of
Speaker 1: dollars for years. No, not at all. And I think
Speaker 1: that's why it's so important when you're reading about AI
Speaker 1: to read people who actually interview people who work at
Speaker 1: these companies and understand how the technology works. Because the
Speaker 1: entire industry has been on this curve where they are
Speaker 1: trying to find micro innovations that reduce the cost of
Speaker 1: training the models and to reduce the cost of what
Speaker 1: they call inference, which is when you actually enter aquarium
Speaker 1: the chat GBT and if you plotted the curve of
Speaker 1: how the cost has been following over time, Deep Seek
Speaker 1: is on that curve. Right, So everything that Deep Seek
Speaker 1: did it was expected by the AI labs that someone
Speaker 1: would be able to do. The novelty was just that
Speaker 1: a Chinese company did it. So to say that it
Speaker 1: like up ends expectations of how AI would be built
Speaker 1: is just purely false and the opinion of somebody who
Speaker 1: does not know what he's talking about.
Speaker 2: Newton then says several octaves higher, which shows you exactly
Speaker 2: how mad he isn't that he thought what he said
Speaker 2: was very civil, and that there are things that are
Speaker 2: true and there are things that are false, like you
Speaker 2: can choose which ones you want to believe. I'm not
Speaker 2: going to be so civil. Other than the fact that
Speaker 2: Casey refers to micro innovations, the fuck are you talking about?
Speaker 2: And Deep Seak being on a curve that was expected,
Speaker 2: he makes, as many do, two very big mistakes and personally.
Speaker 2: If I was doing this, I personally would not have
Speaker 2: said these things in a sentence that began with me
Speaker 2: suggesting that I be in case and Newton in this
Speaker 2: example knew how the technology works. Now here's the case
Speaker 2: in Newton wib inference, which is when you actually enter
Speaker 2: a query into chat GPT. This statement is false. It's
Speaker 2: not what inference means. Inference and I've gotten this wrong
Speaker 2: in the past too. I'm being accountable. Is everything that
Speaker 2: happens when you put in a prompt to generate an output.
Speaker 2: It's when an AI based on your infers meaning. To
Speaker 2: be more specific, in quoting Google machine learning, inference is
Speaker 2: the process of running data points into a machine learning
Speaker 2: model to calculate an output, such as a single numerical score.
Speaker 2: Except that's what these things are bad at. But nevertheless,
Speaker 2: Casey will try and weasel out of this one and
Speaker 2: say this is what he meant. It wasn't. He also said,
Speaker 2: if he planted the curve of how the cost of
Speaker 2: inference has been falling over time, well that's wrong, Casey,
Speaker 2: that's wrong the man. The cost of inference has gone
Speaker 2: up over time. Now, Casey, like many people who talk
Speaker 2: about stuff without learning about it first is likely referring
Speaker 2: to the fact that the price of tokens for some
Speaker 2: models has gone down in some cases. But you know what, folks,
Speaker 2: let's establish and facts about inference. I'm doing the train.
Speaker 2: I'm pulling the big horn on the invisible train. I'm
Speaker 2: cooking now. Inference is a thing that costs money, is
Speaker 2: entirely different to the price of tokens, and conflating the
Speaker 2: two is journalistic malpractice. The cost of inference would be
Speaker 2: the price of running the GPU and the associated architecture.
Speaker 2: Of course, we do not at this point have any
Speaker 2: real insight into token prices are set by the people
Speaker 2: who sell access to the tokens, such as open ai
Speaker 2: and Anthropic. For example, open ai dropped the price of
Speaker 2: its O three models token costs almost immediately after the
Speaker 2: launch of Claude Opus four. Do you think it did
Speaker 2: that because the price of serving the models got cheaper.
Speaker 2: If you do, I don't know how you possibly put
Speaker 2: your trousers on every morning without cutting yourself in half. Now,
Speaker 2: the cost of inference conversation comes from articles that say
Speaker 2: that we now have models that are cheaper that can
Speaker 2: now hit higher benchmark scores. Though the article I'm referring to,
Speaker 2: which will be in the show notes, is from November
Speaker 2: twenty twenty four, and the comparison it makes is between
Speaker 2: GPT three, which is from November twenty twenty one, and
Speaker 2: LAMA three point two to three b September twenty twenty four. Now,
Speaker 2: the suggestion is in any case, that the cost of
Speaker 2: inference is going down ten x year over year. The
Speaker 2: problem is, however, that these are raw token costs, not
Speaker 2: actual expressions of evaluations of token burn in a practical setting.
Speaker 2: And to really I realized that it was a bit technical.
Speaker 2: These are just what it costs to do something. It
Speaker 2: doesn't actually tell you how how many tokens will be
Speaker 2: burned at what volume they will be burned, because that
Speaker 2: would change things. And well, wouldn't you know it, the
Speaker 2: cost of inference actually went up as a result. In
Speaker 2: an excellent blog from Killer Code, and I did not
Speaker 2: get the chance to find out the pronunciation of this
Speaker 2: second name, so I'm just going to call her. It
Speaker 2: is ewasyz sz Ka. I am so sorry. I would
Speaker 2: rather spell it out, miss than actually mispronounce it. I
Speaker 2: hate when people say z tron wrong. Great blog anyway,
Speaker 2: let me quote, application inference costs increase for two reasons.
Speaker 2: The frontier models cost per token stayed constant, and the
Speaker 2: token consumption per application grew a lot. Token consumption per
Speaker 2: application grew a lot because models allowed for longer context
Speaker 2: windows and bigger suggestions from the models. The combination of
Speaker 2: a steady price per token and more token consumption caused
Speaker 2: that inference cost to grow about ten times over the
Speaker 2: past two years. To explain that in really simple terms,
Speaker 2: while the costs of old models may have decreased, new models,
Speaker 2: which you need to do most things, cost about the same,
Speaker 2: and the reasoning that these new models use do actually
Speaker 2: burn way way more tokens. When these new models reason,
Speaker 2: they break the user's input down and break it into
Speaker 2: component parts, then run inference on each of those parts.
Speaker 2: When you plug an L and M into an AI
Speaker 2: coding environment, it will naturally burn an absolute shit ton
Speaker 2: of tokens, in part because of the large amount of
Speaker 2: information you have to load into the prompt and the
Speaker 2: context window, or the amount of information you can load
Speaker 2: in at once, and in part because generatingcode is inference
Speaker 2: intensive and also breaking down all those coding tasks. At
Speaker 2: each of those tasks requiring a coding tool and taking
Speaker 2: a bunch of inference themselves. It's really bad. In fact,
Speaker 2: the inference costs are so severe. The Killer Code says
Speaker 2: that a combination of a steady price for token and
Speaker 2: more token consumption caused app inference costs to grow about
Speaker 2: ten x over the last two years. I'm repeating myself.
Speaker 2: I realized, But I really need you to get one thing,
Speaker 2: which is that the cost of inference went up. But
Speaker 2: I'm not done. I refuse to let this point go
Speaker 2: because people love to say the cost of inference is
Speaker 2: going down when the cost of inference has increased, and
Speaker 2: they do so to a national audience, all while suggesting
Speaker 2: I'm wrong somehow and acting superior. I don't like being
Speaker 2: made to feel this way. I don't think it's nice
Speaker 2: to do this to people. And if you're gonna do it,
Speaker 2: if you have the temerity to call someone out directly,
Speaker 2: at least be fucking right. I'm not wrong, You're wrong.
Speaker 2: In fact, software developer influencer Theo Brown recently put out
Speaker 2: a video called I was wrong about AI costs They
Speaker 2: keep going up, which he breaks down as follows, reasoning
Speaker 2: models are significantly increasing the amount of output tokens being generated.
Speaker 2: These tokens are also more expensive. In one example, Brown
Speaker 2: finds that Grockfor's reasoning mode uses six hundred and three
Speaker 2: tokens to generate two words. This was a problem across
Speaker 2: every single reasoning model, as even cheap reasoning models would
Speaker 2: do the same thing. As a result, tasks are taking
Speaker 2: longer and burning more tokens. Another writer called Ethan Deing
Speaker 2: noted a few months ago that reasoning models burn so
Speaker 2: many tokens that there is no flat subscrips price that
Speaker 2: works in this new world. As the number of tokens
Speaker 2: they consume to an absolutely nuclear the price drops have
Speaker 2: also for the most part stopped. You cannot at this
Speaker 2: point fairly evaluate whether a model is cheaper just based
Speaker 2: on its cost per tokens, because reasoning models inherently burn
Speaker 2: and are built to inherently burn more tokens to create
Speaker 2: an output. Reasoning models are also the only way that
Speaker 2: model developers have been able to improve the efficacy of
Speaker 2: new models, using something called test time compute to burn
Speaker 2: extra tokens to complete a task, and in basically anything
Speaker 2: you're using today, there's going to be some sort of
Speaker 2: reasoning model, especially if you're coding, the cost of inference
Speaker 2: has gone up. Statements otherwise are purely false and are
Speaker 2: the opinion of somebody who does not know what he's
Speaker 2: talking about. But you ask, could the costs of inference
Speaker 2: go down? Maybe it sure isn't trending that way, nor
Speaker 2: has it gone down yet. I also predict that there's
Speaker 2: going to be some sort of sudden realization in the
Speaker 2: media that inference is going up, which is kind of
Speaker 2: already started. The Information had a piece on it in
Speaker 2: late August where they note that into it paide twenty
Speaker 2: million dollars to as your last year, primarily to access
Speaker 2: open AI's models, and it's on track to spend thirty
Speaker 2: million this year, which outpaces the company's revenue growth in
Speaker 2: the same period, raising questions about how sustainable the spending
Speaker 2: is and how much of the cost it can pass
Speaker 2: along to customers. Christopher Mims and The Wall Street Journal
Speaker 2: also had a piece about the costs going up. Do
Speaker 2: not be mad at Chris. Chris and I chatted before
Speaker 2: he submitted that piece, like he literally on Blue Sky
Speaker 2: called me out if fucking rocks. By the way, big
Speaker 2: up to Chris Mims because it's nice to see the
Speaker 2: mainstream media actually engaging with these things, even though it's
Speaker 2: dangerous to the bubble. But you know what, the truth
Speaker 2: must win out, and the problem here is that the
Speaker 2: architecture underlying large language models is inherently unreliable. I imagine open
Speaker 2: AI's introduction of the router to chat GPT five as
Speaker 2: an attempt to moderate both the costs of the model
Speaker 2: chosen and reduce the amount of exposure to reasoning models
Speaker 2: for simple queries. Though Sam Moltman was boasting on August
Speaker 2: tenth about the significant increase in both free and paid
Speaker 2: users exposure to reasoning models, they don't teach you this
Speaker 2: in business school. Still, A study written up by VentureBeat
Speaker 2: found that open weight models burn between one point five
Speaker 2: to four times more tokens, in part due to a
Speaker 2: lack of token efficiency and in part thanks to you
Speaker 2: guessed it reasoning models. I quote the finding's challenge of
Speaker 2: prevailing assumption in the AI industry that open source models
Speaker 2: offer a clear economic advantages over proprietary alternatives. While open
Speaker 2: source models typically cost less per token to run, the
Speaker 2: study suggests that this advantage could be and I quote
Speaker 2: the study easily offset if they require more tokens to
Speaker 2: reason about a given problem, and models keep getting bigger
Speaker 2: and more expensive too. So why did this happen? Well,
Speaker 2: it's because model developers hit a wall of diminishing returns
Speaker 2: and the only way to make models do more was
Speaker 2: to make them burn more tokens to generate a more
Speaker 2: accurate response, which is a very simple way of describing
Speaker 2: reasoning a thing that opening I launched in September twenty
Speaker 2: twenty four, and others followed. As a result, all the
Speaker 2: gains from powerful new models come from burning more and
Speaker 2: more tokens. The cost per million token number is no
Speaker 2: longer an accurate measure of the actual cost of generative
Speaker 2: a because it's much much, much much harder to tell
Speaker 2: how many tokens of reasoning model may burn, and it
Speaker 2: varies as the boint the O Boying, I'm keeping that
Speaker 2: all right. You get the real cuts as the O
Speaker 2: Brown noted from model to model. In any case, there
Speaker 2: really is no changing this path. These companies are out
Speaker 2: of ideas now another another one of my favorite ultimate
Speaker 2: booster gripts. This is a classic and I still get
Speaker 2: this on social media. I'm I have people yapping in
Speaker 2: my ear saying open air and Anthropic are just like
Speaker 2: Uber because Uber bent twenty five billion dollars over the
Speaker 2: course of fifteen or so years and look look edward,
Speaker 2: they're now profitable. Why are you calling me Airport? Shut up?
Speaker 2: This proves the open Ai, a totally different company with
Speaker 2: different economics, will be totally fine. So I've heard this
Speaker 2: argument maybe fifty times in the last year, to the
Speaker 2: point that I had to talk about it in my
Speaker 2: piece how does open Ai Survive, which I also turned
Speaker 2: into a podcast around July twenty twenty four. Go back
Speaker 2: and link a link to it in the piece. Yaddy yaddy, yadda. Nevertheless,
Speaker 2: people make a few points by Uber and AI that
Speaker 2: I think are fundamentally incorrect, and I'm going to break
Speaker 2: them down for you now. They claim that AI is
Speaker 2: making itself too big to fail and betting itself everywhere
Speaker 2: and becoming essential, and none of these things are the case.
Speaker 2: I've heard this argument a lot, by the way, and
Speaker 2: it's one that's both ahistorical and alarmingly ignorant of the
Speaker 2: very basics of society. But ed the government, no no, no, no, no, no,
Speaker 2: you've heard, you've heard. OpenAI got a two hundred million
Speaker 2: dollar Defense contract with an estimated completion date of July
Speaker 2: twenty twenty six. And just to be clear, that's up
Speaker 2: to two hundred million dollars, and that they're selling chat
Speaker 2: GBT Enterprise to the US government for a dollar a year,
Speaker 2: along with Anthropic doing the same thing, and even Google's
Speaker 2: doing it, except they're doing forty cents for a year. Now,
Speaker 2: you're probably hearing this and thinking, ah shit, this means
Speaker 2: the government's paid them. They're never going away. And I
Speaker 2: cannot be clear enough that you believing this is the
Speaker 2: very intention of these deals. They are built specifically to
Speaker 2: make you feel like these things are never going away.
Speaker 2: This is also an attempt to get in with the
Speaker 2: government at a rate that makes train these models a
Speaker 2: no brainer. At which point I ask, and the government
Speaker 2: is going to have cheap access to AI software does
Speaker 2: not mean that the government relies on m every member
Speaker 2: of the government having access to chat GPT, something that
Speaker 2: is not even necessarily the case, does not make this
Speaker 2: software useful, let alone essential. And if open ai burns
Speaker 2: a bunch of money making it work for them, it
Speaker 2: still won't be essential because large language models are not
Speaker 2: actually that useful for doing stuff now let's talk Uber.
Speaker 2: Uber was and is useful, which eventually made it essential.
Speaker 2: Uber used lobbyist Bradley Tusk to steam roll local governments
Speaker 2: into allowing Uber to operate in their cities, but Tasks
Speaker 2: did not have to convince local governments that Uber was
Speaker 2: useful or have to train people how to use Uber.
Speaker 2: Uber's too big to fail moment was that local cabs
Speaker 2: kind of fucking sucked just about everywhere. You ever try
Speaker 2: and take a yellow cab from downtown Manhattan to Hoboken,
Speaker 2: New Jersey, or Brooklyn or Queen's Do you ever try
Speaker 2: and pay with a credit card? How about trying to
Speaker 2: get a cab outside a major metropolitan area. Do you
Speaker 2: remember how bad it was? It was really awful. I
Speaker 2: don't think people realize or remember how bad it was.
Speaker 2: And I'm not saying that Uber is good. I'm not
Speaker 2: glorifying Uber in any way. But the experience that Uber
Speaker 2: replaced was very, very bad. As a result, Uber did
Speaker 2: become too big to fail because people now rely on
Speaker 2: it because the old system sucked. Uber used its masses
Speaker 2: of venture capital to keep prices low to get people
Speaker 2: used to it too, but the fundamental experience was better
Speaker 2: than calling a cab company and hoping they showed up.
Speaker 2: I also want to be clear that this is not
Speaker 2: me condoning Uber take public transport, if you can to
Speaker 2: be clear. Uber has created a new kind of horrifying,
Speaker 2: extractive labor practice which deprives people of benefits and dignity,
Speaker 2: paying off academics to help the media gloss over the
Speaker 2: horrors of their platform, and also now having to increase
Speaker 2: prices so that they reached profitability by doing that. That
Speaker 2: isn't something that's going to happen with genitive AI. Just
Speaker 2: the costs are too high, They're way too high. But anyway,
Speaker 2: what is essential about generative AI? What exactly, and be specific,
Speaker 2: is the essential experience of generative AI? What are we
Speaker 2: if chat, GPT disappeared tomorrow, what actually disappears? And on
Speaker 2: an enterprise or governmental level, what exactly are these tools
Speaker 2: doing for governments that would make removing them so painful?
Speaker 2: What use cases, what outcomes? If your answer here is
Speaker 2: to say, well, they're putting it in and they're choosing,
Speaker 2: they're choosing which people to cut out of benefits, and please, goddamn,
Speaker 2: this is what they want you to do. They want
Speaker 2: you to be scared so they can feel powerful. They're
Speaker 2: not doing that. You notice that we get all these
Speaker 2: horrible stories by the way of internal government things, shoving
Speaker 2: stuff into olms. You know what, we don't get another
Speaker 2: thing we don't get, oh and then have It's just
Speaker 2: they're doing this scary, bad thing that they shouldn't be.
Speaker 2: This shouldn't be putting people's private information into anyway. I'm rambling.
Speaker 2: Uber's essentral nature is that millions of people use it
Speaker 2: in place of regular taxis, and it effectively replaced de
Speaker 2: krepit of exploitative systems like the yellow cab Medallions in
Speaker 2: New York with its own tech enabled exploitation system that
Speaker 2: nevertheless worked far better for the user. Okay, I also
Speaker 2: want to do a side note just to acknowledge that
Speaker 2: the disruption from Uber brought something to the medallion system
Speaker 2: that was genuinely horrendous. The consequences were horrifying for the
Speaker 2: owners of the medallions, some of who had paid more
Speaker 2: than a million dollars for the privilege of driving a
Speaker 2: New York cab and were burdened under mountains of debt.
Speaker 2: That our system is so fucking evil. I think it's horrifying,
Speaker 2: and I think the payday loan people involved should all
Speaker 2: be in fucking prison, worst scum of the world. The
Speaker 2: people who are taking advantage of people come to this
Speaker 2: country to drive a fucking cab that they have to
Speaker 2: take out massive loans to buy. That is evil. Uber
Speaker 2: is also just to be clear, but that also is
Speaker 2: That's the point I'm trying to make. Should feel sorry
Speaker 2: for the victims of that system. That system was a
Speaker 2: kind of corruption unto itself anyway, getting back to the thing,
Speaker 2: because I don't know, I feel I actually feel a
Speaker 2: lot for the people who are the victims of the
Speaker 2: medallion system. It's fucking rough, and every time I think
Speaker 2: of it, I feel very sad inside. But let's get
Speaker 2: back to the episode. I don't want to think about
Speaker 2: it any longer. There really are no essential use cases
Speaker 2: for Chat, GPT, or really any Genai system. You cannot
Speaker 2: point to one use case that is anywhere near as
Speaker 2: necessary as cabs in cities, And indeed the biggest use cases,
Speaker 2: things like brainstorming and search, are either easily replaced by
Speaker 2: any other commoditized The lam will already exist in the
Speaker 2: case of Google Search. Now let's do another boost quip
Speaker 2: data centers are important economic growth vehicles and now helping
Speaker 2: drive innovation and jobs throughout America. Having data centers promotes innovation,
Speaker 2: making open AI and AI data centers essential. And the
Speaker 2: answer to there is no no. Sorry, this is a
Speaker 2: really simple one. These data centers are not in and
Speaker 2: of themselves driving much economic growth other than the costs
Speaker 2: of building them, which I went into last episode. As
Speaker 2: I've discussed again and again, there's maybe forty billion dollars
Speaker 2: in revenue and no profit coming out of AI companies.
Speaker 2: There isn't any economic growth. They're not holding up anything
Speaker 2: other than the massive, massive infrastructure built to make them
Speaker 2: make no money and lose billions. There's no great loss
Speaker 2: associated with the death of large language models or the
Speaker 2: death of this era. Taking away Ober would be genuinely
Speaker 2: catastrophic with some people's ability to get places and people's jobs,
Speaker 2: even if they are horrifyingly underpaid. But here's another booster, quipped.
Speaker 2: Uber burned a lot of money twenty five billion dollars
Speaker 2: or more to get where it is today. Ooh, mister Zichron,
Speaker 2: mister Zitchron, You're dead. And my response is the open
Speaker 2: AI and anthropic are both separately burned more than four
Speaker 2: times as much money since the beginning of twenty twenty
Speaker 2: four as Uber did in its entire existence. So the
Speaker 2: classic and wrong argument about open ai and companies like
Speaker 2: open ai is that Uber burned a bunch of money,
Speaker 2: is now cash flow positive or profitable. I want to
Speaker 2: be clear that Uber's costs are nothing like large language models,
Speaker 2: and making this comparison is ridiculous and desperate. But let's
Speaker 2: talk about raw losses, shall we, and where people are
Speaker 2: making this assumption. So Uber lost twenty four point nine
Speaker 2: billion dollars in the space of four years from twenty
Speaker 2: nineteen to twenty twenty two, in part because of the
Speaker 2: billions it was spending on sales and marketing in R
Speaker 2: and D four point six billion dollars and four point
Speaker 2: eight billion dollars respectively in twenty nineteen alone. It also
Speaker 2: massively subsidized the cost of rights, which is why prices
Speaker 2: had to increase, and spent heavily on driver recruitment, burning
Speaker 2: cash to get scale, you know, the classic Silicon Valley way.
Speaker 2: This is absolutely nothing like how large language models are growing.
Speaker 2: And I'm tired of defending this point, but defended I
Speaker 2: shall open AI and Anthropic burn money primarily through compute
Speaker 2: costs and specialized talent. These costs are increasing, especially with
Speaker 2: the rush to hire every single AI scientists at the
Speaker 2: most expensive price possible. There are also essential immovable costs
Speaker 2: that neither open AI or Anthropic have to shoulder. The
Speaker 2: construction of the data centers necessary to train and run
Speaker 2: inference for their models, and of course the GPU is
Speaker 2: inside them, which I will get to in a little bit. Yes,
Speaker 2: Uber raised thirty three point five billion dollars through multiple
Speaker 2: rounds of posting IPO dam though it raised about twenty
Speaker 2: five billion dollars in actual funding. Yes, Uber burned an
Speaker 2: absolutely as ton of money. Yes, Uber a scale, but
Speaker 2: Uber has not burned money as a means of making
Speaker 2: its product functional or useful. Uber worked immediately. I mean
Speaker 2: was twenty twelve. I think I used it for the
Speaker 2: first time. Maybe earlier. No, no, it would have been
Speaker 2: twenty ten. It worked immediately. You used it, You're like, wow, this,
Speaker 2: I can just put in my address. I don't have
Speaker 2: to say my address three times because I have a
Speaker 2: British accent and nobody can fucking understand me. Sometimes you can,
Speaker 2: though you're special. Yeah, it was really obvious that it worked,
Speaker 2: and also the costs associate with Uber and its capital
Speaker 2: expenditures from twenty nineteen through twenty twenty four were around
Speaker 2: two point two billion dollars, by the way, on miniscule
Speaker 2: compared to the actual real costs of open ai and Anthropic.
Speaker 2: Both open Ai and Anthropic around five billion dollars each
Speaker 2: in twenty twenty four, but their infrastructure was entirely paid
Speaker 2: for by either Microsoft, Google, or Amazon. And by which
Speaker 2: I mean the building of it and the expansion they're
Speaker 2: in what we don't know how much of this infrastructure
Speaker 2: is specifically for open ai or Anthropic. As the largest
Speaker 2: model developers, it's fair to assume that a large chunk
Speaker 2: at least thirty percent of Amazon and Microsoft's capital expenditures
Speaker 2: have been to support these loads. Great sentence to cut
Speaker 2: and listen to again. I also leave out Google, as
Speaker 2: it's unclear whether it's expanded its infrastructure for Anthropic, but
Speaker 2: we know Amazon has done so. As a result, the
Speaker 2: true cost of open ai and Anthropic is at least
Speaker 2: ten times what uberburned. Amazon spent eighty three billion dollars
Speaker 2: in capital expenditures in twenty twenty four and expects one
Speaker 2: hundred and five billion dollars are the fuckers in twenty
Speaker 2: twenty five. Microsoft spent fifty five point six billion dollars
Speaker 2: in twenty twenty four and expects to spend eighty billion
Speaker 2: dollars this year. I'm actually confident most of that is
Speaker 2: open Ai, but based on my conservative calculations, the true
Speaker 2: cost of open ai is at least eighty two billion dollars,
Speaker 2: and that only includes capex twenty twenty four onwards. Based
Speaker 2: on thirty percent of Microsoft's capex. It's not everything has
Speaker 2: been invested yet in twenty twenty five, and open Ai
Speaker 2: might not be all of the capex, and also the
Speaker 2: forty one point four billion dollars of funding that open
Speaker 2: ai has received so far. The true cost of Anthropic
Speaker 2: is around seventy seven point one billion dollars, and that's
Speaker 2: not including the thirteen billion they just raised, but it
Speaker 2: does include all their previous funding and thirty percent of
Speaker 2: Amazon's capex in the beginning of twenty twenty four. Now
Speaker 2: these are in exact comparisons, but the classic argument is
Speaker 2: that Uber burned lots of money and worked out okay,
Speaker 2: when in fact the combined couple expenditures from twenty twenty
Speaker 2: four onwards that are necessary to make open ai and
Speaker 2: Anthropic worker each on their own four times what Uber
Speaker 2: burned in over a decade. I also believe these numbers
Speaker 2: are conservative. There's a good chance that open ai and
Speaker 2: Anthropic dominate the capex of Amazon, Google, and Microsoft in
Speaker 2: part because of what the fuck else are they buying
Speaker 2: all these GPUs for as their own AI services don't
Speaker 2: appear to be making much money at all anyway. To
Speaker 2: put it real simple, AI has burned way more in
Speaker 2: the last two years than Uber burned in ten. Uber
Speaker 2: didn't burn money in the same way, didn't burn much
Speaker 2: in the way of capital expenditures, didn't require massive amounts
Speaker 2: of infrastructure, and isn't remotely the same in any way,
Speaker 2: shape or form other than that it burned a lot
Speaker 2: of money. And that burning wasn't because it was trying
Speaker 2: to build the core product. It was trying to scale.
Speaker 2: It's all so stupid, And you know what, I'm not
Speaker 2: even done. Our next and final AI booster episode will
Speaker 2: breeze through the dumbest of the dumb arguments, and I'll
Speaker 2: say why I'm finally drawing a line under these arguments
Speaker 2: for real, because it needs to be said. We need
Speaker 2: to say something. I hope you've enjoyed this, see you tomorrow, godspeed.
Speaker 2: Thank you for listening to Better Offline. The editor and
Speaker 2: composer of the Better Offline theme song is Matasowski. You
Speaker 2: can check out more of his music and audio projects
Speaker 2: at Matasowski dot com, M A T T O S
Speaker 2: O W s ki dot com. You can email me
Speaker 2: at easy at Better offline dot com or visit Better
Speaker 2: Offline dot com to find more podcast links and of course,
Speaker 2: my newsletter. I also really recommend you go to chat
Speaker 2: dot Where's youreaed dot at to visit the discord, and
Speaker 2: go to our slash Better Offline to check out our reddit.
Speaker 2: Thank you so much for listening. Better Offline is a
Speaker 2: production of cool Zone Media. For more from cool Zone Media,
Speaker 2: visit our website cool Zonemedia dot com, or check us
Speaker 2: out on the iHeartRadio app, Apple Podcasts, or wherever you
Speaker 2: get your podcasts.