Speaker 1: Zone Media.
Speaker 2: Hello, and welcome to Better Offline. I'm your surly yet
Speaker 2: lovable host ed Zitron. Today I'm going to kick off
Speaker 2: by reading something I wrote in March twenty twenty four
Speaker 2: and talked about in the episode PKI. What if what
Speaker 2: we're seeing today isn't a glimpse of the future but
Speaker 2: the new terms of the present. What if artificial intelligence
Speaker 2: isn't actually capable of doing much more than what we're
Speaker 2: seeing today, and what if there's no clear timeline when
Speaker 2: it will be able to do more? What if this
Speaker 2: entire hype cycle has been built on hot air ghost
Speaker 2: by a compliant media, ready and willing to take career
Speaker 2: embellishes at their word. Reading that back, well, I think
Speaker 2: I might have been right, And that's kind of what
Speaker 2: I'm going to get at today. I don't want to
Speaker 2: scream mustard. I'm not going to get smug about it.
Speaker 2: But this is what we're getting into today and in
Speaker 2: the next episode that will come out on Friday. Now,
Speaker 2: I'll be linking to some articles, so check the episode
Speaker 2: notes if you want to read them. But I'm going
Speaker 2: to get a lot into the spoken word.
Speaker 3: So I warned you in February.
Speaker 2: That generative AI has no killer apps and had no
Speaker 2: way of justifying its valuations. I also warned you in
Speaker 2: March that generative AI had already peaked, and I pleaded
Speaker 2: with the tech industry in April to consider an eventuality
Speaker 2: where the jump between GPT four, which is the most
Speaker 2: current model, or GPT four to zero to GPT five
Speaker 2: was not significant, in part due to a lack of
Speaker 2: training data, one of the more obvious things. I shared
Speaker 2: more concerns in July that the transformer based architecture underpinning
Speaker 2: generative AI things like chad GPT was a dead end
Speaker 2: and that there were really not many ways we'd progress
Speaker 2: past the products we'd already seen back then, impart due
Speaker 2: to the limits of training data are convention and the
Speaker 2: limits of the models that use said training data. In August,
Speaker 2: I summarized the pale horses of the AI apocalypse events
Speaker 2: many that have now come to pass. Unafraid that would
Speaker 2: signify the end well being nigh, though it's not quite
Speaker 2: here yet and it's not obvious when it will be.
Speaker 2: But this can't last forever. But I also add that
Speaker 2: GPT five would not change the game enough to matter,
Speaker 2: let alone add a new architecture to build future and
Speaker 2: more capable models or products of any kind. Now, throughout
Speaker 2: the things I've written and the things have spoken, I've
Speaker 2: repeatedly made the point that separate to any core value proposition,
Speaker 2: training data, drought or unsustainable economics that I've gone over
Speaker 2: quite a lot, general IFAI is a dead end due
Speaker 2: to the limitations of a probabilistic model that hallucinates. Now,
Speaker 2: just to be clear with what that means, it's guessing
Speaker 2: what the next thing might be, and it's quite good
Speaker 2: at it, but quite good is actually.
Speaker 3: Kind of shit.
Speaker 2: And hallucinations, of of course, are whether you authoritatively state
Speaker 2: things that aren't true, like when chat GPT tells you
Speaker 2: something like I don't know there are two hours in strawberry.
Speaker 2: The hallucination problem is one that is nowhere closer to
Speaker 2: being solved. You may remember a few months ago when
Speaker 2: you had every tech executive, you had Tim Cooks and
Speaker 2: it's satching Adella Sun Dapashai. We'll deal with the hallucination problem.
Speaker 2: It'll be all right. But I want to be clear,
Speaker 2: they have not solved it, they have not really mitigated
Speaker 2: and there's no fixing it, at least with the current technology,
Speaker 2: it's not going anywhere, and it makes all of this
Speaker 2: stuff kind of a non starter for many business tasks.
Speaker 2: I have since March expressed great dismay about the credulousness
Speaker 2: of the media about this, and they're weird acceptance of
Speaker 2: this inevitable way in which generative AI will change society,
Speaker 2: despite the fact there's not really a meaningful product that
Speaker 2: might justify any of this bullshit, This environmentally destructive nonsense
Speaker 2: led by a company that burns more than five billion
Speaker 2: dollars a year in big tech firms that are spending
Speaker 2: two hundred billion dollars on data centers for products that
Speaker 2: people don't want or even potentially use. And you're going
Speaker 2: to need context for everything I'm saying today. So it's
Speaker 2: worth going over how these models work and how they're trained.
Speaker 2: And I must be clear the reason I'm repeating myself
Speaker 2: on so many levels here that it's it's just really
Speaker 2: important for you to know how obvious the problems of
Speaker 2: generative AI have been since the beginning. It's really important.
Speaker 2: Let's go over how they work real quick. A transformer
Speaker 2: based generative AI models such as GPT, which is the
Speaker 2: technology behind chat. GPT generates answers using inference, which means
Speaker 2: it draws conclusions based off of its training, which requires
Speaker 2: feeding it masses of training data, mostly text and images
Speaker 2: straight from the Internet. And both of these processes require
Speaker 2: you to use high end GPUs graphics processing units, and
Speaker 2: lots of them, tens hundreds of thousands of them, well
Speaker 2: over one hundred thousand. I'll get to that next episode. Now,
Speaker 2: the theory was and might still be that the more
Speaker 2: training data and compute you throw out these models, the
Speaker 2: better they get. And I've hypothesized for a while that
Speaker 2: we'd have diminishing returns both and running out of training
Speaker 2: data and based on the limitations of transformer based models.
Speaker 3: And wouldn't you know it, I was bloody right.
Speaker 2: I'm not going to do many of these, but this one, really,
Speaker 2: this one I'm right on. A few weeks ago, Bloomberg
Speaker 2: reported the open AI. Google and Anthropic are struggling to
Speaker 2: build more advanced AI and the open ais A right
Speaker 2: model otherwise known as GPT five did not hit the
Speaker 2: company's desired performance. And that and I quote again, Orion
Speaker 2: is so far not considered to be a bigger step
Speaker 2: up as it was from GPT three point five. To
Speaker 2: GPT four its current model. You will be shocked to
Speaker 2: hear that the reason is that it's become increasingly difficult
Speaker 2: to find new, untapped sources of high quality human made
Speaker 2: training data that can be used to build more advanced
Speaker 2: AI systems, something that I said in March. I said
Speaker 2: it would happen in March and pissed off that people
Speaker 2: said I was a pessimist. Well, who's a pessimist now me?
Speaker 2: I guess I don't know. But they also added one
Speaker 2: other thing, which is that they believe and I quote,
Speaker 2: that the AGI bubble is bursting a little bit, which
Speaker 2: is something I said in July. AGI isn't coming out
Speaker 2: of this shit. Let's just be honest. And I also
Speaker 2: want to stop and stare really hard at one particular point,
Speaker 2: and I quote again from Bloomberg. These issues challenged the
Speaker 2: gospel that's taken hold in Silicon Valley in recent years,
Speaker 2: particularly since OpenAI released chat GPT two years ago. Much
Speaker 2: of the tech industry is bet on so called scaling
Speaker 2: laws that say more computing power, data, and larger models
Speaker 2: will inevitably pave the way the greater leaps forward in
Speaker 2: the power of AI. The only people taking this as gospel.
Speaker 2: Have been members of the media unwilling to ask the
Speaker 2: tough questions, and AI founders that don't know what the
Speaker 2: fuck they're talking about, or they intend to mislead you.
Speaker 2: Generative AI's products have effectively been trapped in amber for
Speaker 2: over a year. It's been blatantly obvious if you fucking
Speaker 2: use them and pissed off, I shouldn't swear so much.
Speaker 2: There have been no meaningful, industry defining products out of
Speaker 2: this because, and I quote Darreness and Mooglu, the economist
Speaker 2: that MIT back in May, more powerful models do not
Speaker 2: unlock new features or really change the experience, Nor what
Speaker 2: you can build with transform based models is really a
Speaker 2: worthwhile product. Or put another way, a slightly better white
Speaker 2: elephant is still a white elephant. Despite the billions of
Speaker 2: dollars burned and thousands of glossy headlines, it's difficult to
Speaker 2: point to any truly important generative AI product, even Apple Intelligence,
Speaker 2: the only thing that Apple really had to add to
Speaker 2: the latest iPhone. It sucks, it's not useful. I can
Speaker 2: make a special emoji. Now I now get summaries of
Speaker 2: my texts that are completely or vaguely incorrect or just
Speaker 2: summarize a giant, meaningful paragraph into a blob of a sentence.
Speaker 2: It's so stupid. And just as a side question, what
Speaker 2: the hell is Apple going to put in the next iPhone?
Speaker 2: I buy one of these every year. I'm a little
Speaker 2: big oincoin CoInc but still I don't even know why
Speaker 2: ad upgrade again. The camera is already about as good
Speaker 2: as it's going to get.
Speaker 3: Anyway.
Speaker 2: There are people that use chat GPT two hundred million
Speaker 2: for them a week, allegedly losing the company money with
Speaker 2: every prompt, by the way, but there's little to suggest
Speaker 2: that there's widespread adoption of actual generative AI software. The
Speaker 2: Information reported in September that between zero point one percent
Speaker 2: and one percent of the four hundred and forty million
Speaker 2: of Microsoft's business customers were willing to pay for its
Speaker 2: AI powered Copilot, and in late October, Microsoft claimed that
Speaker 2: it was on pace to make AI a ten billion
Speaker 2: dollar a year business, which sounds really good until you
Speaker 2: think about it for roughly ten seconds. First of all,
Speaker 2: Microsoft does not have an AI business unit, which means
Speaker 2: that this annual ten billion dollars or two and a
Speaker 2: half billion a quarter revenue figure is split across providing
Speaker 2: cloud compute services, and Azure selling Copilot. The dumb people
Speaker 2: with Microsoft three sixty five subscriptions selling git Hub, Copilot,
Speaker 2: and basically anything else with AI on it. Microsoft is
Speaker 2: cherry picking a number based on nonspecific criteria and claiming
Speaker 2: it's a big deal when it's actually pretty pathetic, considering
Speaker 2: that Microsoft's capital expenditures will likely hit over sixty billion
Speaker 2: dollars in twenty twenty four with no sign they're going
Speaker 2: to slow down. No, that's sticky word. Revenue not profit.
Speaker 2: Those are two very different things. How much is Microsoft
Speaker 2: spending to make ten billion dollars a year? Open ai
Speaker 2: currently spends two dollars and thirty five cents to make
Speaker 2: a dollar, and Microsoft CFO Amyhood said that open ai
Speaker 2: would cut into Microsoft profits in their last earning score,
Speaker 2: losing it a remarkable one point five billion dollars, mainly
Speaker 2: because of the expected loss from a company that has
Speaker 2: only ever lost money now a year ago. In October
Speaker 2: twenty twenty three, The Wall Street Journal reported that Microsoft
Speaker 2: was losing an average of twenty dollars per user per
Speaker 2: month on GitHub Copilot, a product with over a million users.
Speaker 2: If this is true, by the way, this suggests losses
Speaker 2: of at least two hundred million a year. They have
Speaker 2: one point eight million users. Allegedly, this is based on
Speaker 2: documents have reviewed. It's not great either way. That two
Speaker 2: hundred million dollars is a lot of money to lose.
Speaker 2: I would personally like to make two hundred million dollars
Speaker 2: rather than lose it. Don't ask me, though I don't
Speaker 2: run Microsoft.
Speaker 3: Now.
Speaker 2: Microsoft is still yet to break out exactly how much
Speaker 2: generative AI is increasing revenue in the specific business units
Speaker 2: they have. Generally, if a company's doing well at something,
Speaker 2: they take great pains to make that clear. Instead, Microsoft
Speaker 2: chose in August to revamp its reporting structure to give
Speaker 2: better visibility into cloud consumption revenue, which is something you
Speaker 2: do if you say, anticipate you're going to have your
Speaker 2: worst day of trading in year after your next earnings,
Speaker 2: as Microsoft did in October. It's all very good, it's
Speaker 2: all going well. Now it must be clear that every
Speaker 2: single one of these investments and products, as I've been
Speaker 2: hyped with the whisper, that they would get exponentially better
Speaker 2: over time, and that eventually the two hundred billion dollars
Speaker 2: in capital expenditures would spit out this remarkable productivity improvement,
Speaker 2: this crazy new product that would change our lives, fascinating
Speaker 2: new things that consumers and enterprise of buying droves and
Speaker 2: talk about how much they loved. Instead, Big tech has
Speaker 2: found itself peddling increasingly more expensive iterations of near identical
Speaker 2: large language models and shitty products attached to them, a
Speaker 2: direct result of all of them having to use the
Speaker 2: same training data, which they're now running out of. But
Speaker 2: if you're running out of stuff and you can't find
Speaker 2: stuff to buy, I really recommend the following advertisement. I'm
Speaker 2: sure it will totally gel with my beliefs. The things
Speaker 2: I'm talking about right now won't be embarrassing at all,
Speaker 2: and we're back now. There's another assumption that people have
Speaker 2: about these so called scaling laws. That's been by simply
Speaker 2: building bigger data centers with even bigger, more powerful GPUs,
Speaker 2: the expensive power hungry graphics processing units that use to
Speaker 2: both train and run these models, and throwing as much
Speaker 2: training data at them as possible.
Speaker 3: They would simply start doing new things.
Speaker 2: They'd have new capabilities, despite their being little proof that
Speaker 2: they would do so in any way, shape or form. Microsoft, Meta, Amazon,
Speaker 2: and Google of all burn billions, and the assumption that
Speaker 2: doing so would create something, you know, a thing, a
Speaker 2: good thing, like a human level artificial general intelligence, or
Speaker 2: a product that made more money than it cost that
Speaker 2: people liked. It's become kind of obvious that that isn't
Speaker 2: going to happen. As we speak, members of the media
Speaker 2: who should know better are already desperately trying to prove
Speaker 2: that this is not a problem the information in a
Speaker 2: similar story to Bloomberg's attempted to put lipstick on the
Speaker 2: pig of generative AI, framing the lack of meaningful progress
Speaker 2: of GPT fivers fine because open ai can now combine
Speaker 2: its GPT five model with its one reasoning model, which
Speaker 2: is the one that can't count the number of ours
Speaker 2: and the strawberry by the way, which will then do something,
Speaker 2: something good. Something's gonna happen. Like Sam Altman said, it
Speaker 2: could write a lot more very difficult code. You know, Samultman,
Speaker 2: the career liar who intimated the GPT five may function
Speaker 2: like a virtual brain in may like these people are liars.
Speaker 2: They're liars, they're lying to you. They were lying then
Speaker 2: they're lying now now I couldn't possibly leave out chief
Speaker 2: Ellly cheerleader Casey Newton, who wrote on platform Or a
Speaker 2: few weeks ago that diminishing returns in training models may
Speaker 2: not matter as much as you would guess, with his
Speaker 2: evidence being the anthropic, who he also claims has not
Speaker 2: been prone to hyperbole, do not think the scaling laws
Speaker 2: are ending now. The original scaling law's paper, partly written
Speaker 2: by Dario Amadesi of Anthropic important to know and to
Speaker 2: be clear, in a fourteen thousand word ophed that Casey
Speaker 2: Newton for no reason wrote two pieces about Anthropic CEO Dario.
Speaker 3: He said that, and I quote AI.
Speaker 2: Accelerated neuroscience is likely to vastly improve treatments for or
Speaker 2: even cure most mental illness, which is the kind of
Speaker 2: hyperbole that should be Have you tired and feathered and
Speaker 2: put in a jail? I'm not seriously saying you put
Speaker 2: him in jail? But why are we Why are we
Speaker 2: trusting these people? Why are we listening to them? Why
Speaker 2: are we treating them as if they're telling the truth
Speaker 2: or even that they know what's going on? But let's
Speaker 2: summarize the main technology behind the entire and I say
Speaker 2: this in quotation marks by the way, artificial intelligence boom
Speaker 2: is generative AI transformed based models like open AI's GPT
Speaker 2: four and soon GPT five, and said technology has speaked
Speaker 2: with diminishing returns from the only ways of making them better,
Speaker 2: feeding them, training data, and throwing tons of compute at them,
Speaker 2: suggesting that we may have, as I said before, reached PKI.
Speaker 2: Generative AI is incredibly unprofitable. Open Ai, the biggest player
Speaker 2: in the industry, is on course to lose more than
Speaker 2: five billion dollars this year, with competitor Anthropic, which also
Speaker 2: makes its own transformer based model, clawed, on course to
Speaker 2: lose more than two point seven billion dollars this year.
Speaker 2: They just raised another four billion. Every single big tech
Speaker 2: company has thrown billions of dollars, as much as seventy
Speaker 2: five billion dollars in Amazon's case in twenty twenty four alone.
Speaker 2: Are building the data centers and acquiring the GPUs to
Speaker 2: populate said data centers, specifically so they can train their
Speaker 2: models and other people's models, or serve customers that would
Speaker 2: integrate Generativai into their businesses, something that does not appear
Speaker 2: to be happening at scale, and these investments could theoretically
Speaker 2: be used for other products but these data cent as
Speaker 2: a heavily focused on GENERATIVIAI. Business Insider reports that Microsoft
Speaker 2: intends to amass one point eight million GPUs by the
Speaker 2: end of this year, costing it tens of billions of dollars.
Speaker 2: Worse still, many of these companies integrating GENERATIVIAI do so
Speaker 2: by connecting to models made by either Open AI or Anthropic,
Speaker 2: both of whom are running unprofitable businesses and likely charging
Speaker 2: nowhere near enough to cover their costs. As I've said
Speaker 2: before in my article the sub MAI Crisis, in the
Speaker 2: event that these companies start charging what they actually need
Speaker 2: to their real costs, I hypothesize that it will multiply
Speaker 2: the cost of their customers to the point that they
Speaker 2: can't afford to run their businesses, or at the very least,
Speaker 2: we'll have to remove or scale back generative AI functionality
Speaker 2: in their products. It's just it's such a waste. The
Speaker 2: entire tech industry has become orientated around this dead end
Speaker 2: technology that requires burning billions and billions of dollars to
Speaker 2: provide in essential products that cost them more money to
Speaker 2: serve than anybody ever would pay. Their big strategy has
Speaker 2: to be to throw more money at the problem until
Speaker 2: one of these transformer based models created something useful. Despite
Speaker 2: the fact that every iteration of GPT in other models
Speaker 2: has been well iterative, and it's weird, you think at
Speaker 2: some point that goes, shit, do we actually.
Speaker 3: Have the ability to build products with this? What are
Speaker 3: the products?
Speaker 2: Maybe we should work out the products first before we
Speaker 2: throw all the capex at it. But wait, no, oh,
Speaker 2: over yonder, I couldn't possibly not do this because the
Speaker 2: other big tech company that also has no ideas they're
Speaker 2: doing this. And if I don't do this, my investors
Speaker 2: are going to be angry at me. And then what
Speaker 2: will I do? Oh no, oh no, what could I
Speaker 2: possibly do? If the investors I don't fucking know, that's
Speaker 2: your problem? Why waste this much money? It's just there's
Speaker 2: never been any proof other than these benchmarks that are
Speaker 2: really easy to gain and also only showed just this
Speaker 2: vague power of these models. It's been obvious that GPT
Speaker 2: or other models wouldn't become conscious that they're not going
Speaker 2: to do more than they do today or three months
Speaker 2: ago or even a year ago. Hesitate to give Gary
Speaker 2: Marcus credit, but in twenty twenty three he was saying
Speaker 2: this if not earlier. Many people have as well, and
Speaker 2: it's just really really, really really frustrating. Better Offline isn't
Speaker 2: even a year old. But when we put out our
Speaker 2: PKI episode, I got so much flak. I got so
Speaker 2: much shit for being a hater. I didn't really understand things.
Speaker 2: That my fly was open in my Instagram picture, that
Speaker 2: I didn't get it, and that in mere months I
Speaker 2: would be proven wrong.
Speaker 3: Well here we are. How wrong am I? Now?
Speaker 2: What happens next? Exactly where do all these hundreds of
Speaker 2: billions of dollars go? What happens to Open AI when
Speaker 2: it collapses? What does Microsoft do with all of these GPUs?
Speaker 2: Because you can't just move them into other shit? You
Speaker 2: know from what I hear, they don't really have a plan.
Speaker 2: And that's the scariest thing, because what happens to a
Speaker 2: stock market that's dependent on big tech companies for growth
Speaker 2: when the big tech companies can't work out a way
Speaker 2: to grow anymore, and in fact, their big path to
Speaker 2: try and to grow more was to burn a shit
Speaker 2: ton of money on things that people hate, that destroy
Speaker 2: our environment. I know, I know, I'm angry. I know
Speaker 2: I should calm down. I should, But as I said
Speaker 2: in the Rot Society, this money could go elsewhere, more
Speaker 2: things could be done. It would enter a fallow period
Speaker 2: of tech. But we don't just have to burn all
Speaker 2: this money. We don't have to do that. Why not
Speaker 2: make the products you have already better? Because stapling generative
Speaker 2: AI on them, I think it makes them worse. But
Speaker 2: there are more problems ahead. There are problems around the infrastructure.
Speaker 2: And in the next episode, I'm gonna break down these
Speaker 2: worrying problems and I'm gonna kind of tell you what
Speaker 2: happens next is best I can. I really appreciate your
Speaker 2: faith in me. And there are many people who also
Speaker 2: contacted me and said, no, you bang on, keep going.
Speaker 2: I'm glad they did. I'm very grateful for your audience.
Speaker 2: I love you all much like you said in the menu,
Speaker 2: thank you for listening to Better Offline. The editor and
Speaker 2: composer of the Better Offline theme song is Metasowski. You
Speaker 2: can check out more of his music and audio projects
Speaker 2: at Matasowski dot com, M A T T O. S
Speaker 2: O w Ski dot com. You can email me at
Speaker 2: easy at Better offline dot com, or visit Better Offline
Speaker 2: dot com to find more podcast links and of course
Speaker 2: my newsletter. I also really recommend you go to chat
Speaker 2: dot Where's youread dot at to visit the discord, and
Speaker 2: go to our slash Better Offline to check out our reddit.
Speaker 2: Thank you so much for listening.
Speaker 1: Better Offline is a production of cool Zone Media. For
Speaker 1: more from cool Zone Media, visit our website cool Zonemedia
Speaker 1: dot com, or check us out on the iHeartRadio app,
Speaker 1: Apple Podcasts, or wherever you get your podcasts.