WEBVTT - Yep, We're At Peak AI

0:00:02.840 --> 0:00:03.560
<v Speaker 1>Zone Media.

0:00:05.440 --> 0:00:08.240
<v Speaker 2>Hello, and welcome to Better Offline. I'm your surly yet

0:00:08.280 --> 0:00:22.720
<v Speaker 2>lovable host ed Zitron. Today I'm going to kick off

0:00:22.720 --> 0:00:25.160
<v Speaker 2>by reading something I wrote in March twenty twenty four

0:00:25.400 --> 0:00:28.800
<v Speaker 2>and talked about in the episode PKI. What if what

0:00:28.800 --> 0:00:30.920
<v Speaker 2>we're seeing today isn't a glimpse of the future but

0:00:31.040 --> 0:00:34.240
<v Speaker 2>the new terms of the present. What if artificial intelligence

0:00:34.280 --> 0:00:36.959
<v Speaker 2>isn't actually capable of doing much more than what we're

0:00:36.960 --> 0:00:39.440
<v Speaker 2>seeing today, and what if there's no clear timeline when

0:00:39.479 --> 0:00:41.320
<v Speaker 2>it will be able to do more? What if this

0:00:41.560 --> 0:00:43.960
<v Speaker 2>entire hype cycle has been built on hot air ghost

0:00:43.960 --> 0:00:46.600
<v Speaker 2>by a compliant media, ready and willing to take career

0:00:46.640 --> 0:00:51.880
<v Speaker 2>embellishes at their word. Reading that back, well, I think

0:00:51.880 --> 0:00:53.440
<v Speaker 2>I might have been right, And that's kind of what

0:00:53.440 --> 0:00:55.120
<v Speaker 2>I'm going to get at today. I don't want to

0:00:55.120 --> 0:00:57.400
<v Speaker 2>scream mustard. I'm not going to get smug about it.

0:00:57.960 --> 0:01:00.080
<v Speaker 2>But this is what we're getting into today and in

0:01:00.120 --> 0:01:02.400
<v Speaker 2>the next episode that will come out on Friday. Now,

0:01:02.440 --> 0:01:04.560
<v Speaker 2>I'll be linking to some articles, so check the episode

0:01:04.640 --> 0:01:05.880
<v Speaker 2>notes if you want to read them. But I'm going

0:01:05.920 --> 0:01:07.280
<v Speaker 2>to get a lot into the spoken word.

0:01:08.880 --> 0:01:10.399
<v Speaker 3>So I warned you in February.

0:01:10.040 --> 0:01:12.520
<v Speaker 2>That generative AI has no killer apps and had no

0:01:12.600 --> 0:01:15.360
<v Speaker 2>way of justifying its valuations. I also warned you in

0:01:15.400 --> 0:01:18.120
<v Speaker 2>March that generative AI had already peaked, and I pleaded

0:01:18.120 --> 0:01:21.399
<v Speaker 2>with the tech industry in April to consider an eventuality

0:01:21.440 --> 0:01:23.920
<v Speaker 2>where the jump between GPT four, which is the most

0:01:23.920 --> 0:01:26.920
<v Speaker 2>current model, or GPT four to zero to GPT five

0:01:27.080 --> 0:01:29.679
<v Speaker 2>was not significant, in part due to a lack of

0:01:29.720 --> 0:01:33.080
<v Speaker 2>training data, one of the more obvious things. I shared

0:01:33.080 --> 0:01:36.560
<v Speaker 2>more concerns in July that the transformer based architecture underpinning

0:01:36.600 --> 0:01:39.360
<v Speaker 2>generative AI things like chad GPT was a dead end

0:01:39.560 --> 0:01:42.520
<v Speaker 2>and that there were really not many ways we'd progress

0:01:42.560 --> 0:01:45.200
<v Speaker 2>past the products we'd already seen back then, impart due

0:01:45.240 --> 0:01:47.400
<v Speaker 2>to the limits of training data are convention and the

0:01:47.440 --> 0:01:50.800
<v Speaker 2>limits of the models that use said training data. In August,

0:01:50.880 --> 0:01:53.880
<v Speaker 2>I summarized the pale horses of the AI apocalypse events

0:01:54.280 --> 0:01:56.960
<v Speaker 2>many that have now come to pass. Unafraid that would

0:01:57.000 --> 0:02:00.720
<v Speaker 2>signify the end well being nigh, though it's not quite

0:02:00.760 --> 0:02:03.120
<v Speaker 2>here yet and it's not obvious when it will be.

0:02:03.320 --> 0:02:06.000
<v Speaker 2>But this can't last forever. But I also add that

0:02:06.040 --> 0:02:08.560
<v Speaker 2>GPT five would not change the game enough to matter,

0:02:08.800 --> 0:02:11.120
<v Speaker 2>let alone add a new architecture to build future and

0:02:11.240 --> 0:02:15.000
<v Speaker 2>more capable models or products of any kind. Now, throughout

0:02:15.000 --> 0:02:16.680
<v Speaker 2>the things I've written and the things have spoken, I've

0:02:16.680 --> 0:02:20.280
<v Speaker 2>repeatedly made the point that separate to any core value proposition,

0:02:20.520 --> 0:02:23.320
<v Speaker 2>training data, drought or unsustainable economics that I've gone over

0:02:23.400 --> 0:02:25.720
<v Speaker 2>quite a lot, general IFAI is a dead end due

0:02:25.720 --> 0:02:29.240
<v Speaker 2>to the limitations of a probabilistic model that hallucinates. Now,

0:02:29.320 --> 0:02:31.360
<v Speaker 2>just to be clear with what that means, it's guessing

0:02:31.360 --> 0:02:33.639
<v Speaker 2>what the next thing might be, and it's quite good

0:02:33.720 --> 0:02:35.720
<v Speaker 2>at it, but quite good is actually.

0:02:35.520 --> 0:02:36.240
<v Speaker 3>Kind of shit.

0:02:36.639 --> 0:02:40.200
<v Speaker 2>And hallucinations, of of course, are whether you authoritatively state

0:02:40.280 --> 0:02:42.600
<v Speaker 2>things that aren't true, like when chat GPT tells you

0:02:42.680 --> 0:02:45.639
<v Speaker 2>something like I don't know there are two hours in strawberry.

0:02:46.360 --> 0:02:49.080
<v Speaker 2>The hallucination problem is one that is nowhere closer to

0:02:49.120 --> 0:02:51.840
<v Speaker 2>being solved. You may remember a few months ago when

0:02:51.880 --> 0:02:54.120
<v Speaker 2>you had every tech executive, you had Tim Cooks and

0:02:54.120 --> 0:02:57.880
<v Speaker 2>it's satching Adella Sun Dapashai. We'll deal with the hallucination problem.

0:02:58.000 --> 0:03:00.000
<v Speaker 2>It'll be all right. But I want to be clear,

0:03:00.480 --> 0:03:03.280
<v Speaker 2>they have not solved it, they have not really mitigated

0:03:04.000 --> 0:03:06.400
<v Speaker 2>and there's no fixing it, at least with the current technology,

0:03:06.840 --> 0:03:09.800
<v Speaker 2>it's not going anywhere, and it makes all of this

0:03:09.840 --> 0:03:12.440
<v Speaker 2>stuff kind of a non starter for many business tasks.

0:03:13.120 --> 0:03:17.120
<v Speaker 2>I have since March expressed great dismay about the credulousness

0:03:17.120 --> 0:03:20.240
<v Speaker 2>of the media about this, and they're weird acceptance of

0:03:20.280 --> 0:03:23.320
<v Speaker 2>this inevitable way in which generative AI will change society,

0:03:23.600 --> 0:03:26.080
<v Speaker 2>despite the fact there's not really a meaningful product that

0:03:26.160 --> 0:03:30.080
<v Speaker 2>might justify any of this bullshit, This environmentally destructive nonsense

0:03:30.360 --> 0:03:32.360
<v Speaker 2>led by a company that burns more than five billion

0:03:32.400 --> 0:03:34.600
<v Speaker 2>dollars a year in big tech firms that are spending

0:03:34.600 --> 0:03:37.600
<v Speaker 2>two hundred billion dollars on data centers for products that

0:03:37.640 --> 0:03:42.160
<v Speaker 2>people don't want or even potentially use. And you're going

0:03:42.200 --> 0:03:45.400
<v Speaker 2>to need context for everything I'm saying today. So it's

0:03:45.440 --> 0:03:48.520
<v Speaker 2>worth going over how these models work and how they're trained.

0:03:49.080 --> 0:03:51.240
<v Speaker 2>And I must be clear the reason I'm repeating myself

0:03:51.480 --> 0:03:54.800
<v Speaker 2>on so many levels here that it's it's just really

0:03:54.880 --> 0:03:56.960
<v Speaker 2>important for you to know how obvious the problems of

0:03:57.000 --> 0:04:00.480
<v Speaker 2>generative AI have been since the beginning. It's really important.

0:04:00.880 --> 0:04:03.920
<v Speaker 2>Let's go over how they work real quick. A transformer

0:04:03.960 --> 0:04:06.520
<v Speaker 2>based generative AI models such as GPT, which is the

0:04:06.600 --> 0:04:11.040
<v Speaker 2>technology behind chat. GPT generates answers using inference, which means

0:04:11.080 --> 0:04:14.360
<v Speaker 2>it draws conclusions based off of its training, which requires

0:04:14.400 --> 0:04:18.000
<v Speaker 2>feeding it masses of training data, mostly text and images

0:04:18.040 --> 0:04:21.600
<v Speaker 2>straight from the Internet. And both of these processes require

0:04:21.640 --> 0:04:24.880
<v Speaker 2>you to use high end GPUs graphics processing units, and

0:04:25.040 --> 0:04:28.640
<v Speaker 2>lots of them, tens hundreds of thousands of them, well

0:04:28.680 --> 0:04:32.039
<v Speaker 2>over one hundred thousand. I'll get to that next episode. Now,

0:04:32.040 --> 0:04:34.560
<v Speaker 2>the theory was and might still be that the more

0:04:34.600 --> 0:04:36.960
<v Speaker 2>training data and compute you throw out these models, the

0:04:36.960 --> 0:04:39.800
<v Speaker 2>better they get. And I've hypothesized for a while that

0:04:39.960 --> 0:04:42.320
<v Speaker 2>we'd have diminishing returns both and running out of training

0:04:42.400 --> 0:04:45.840
<v Speaker 2>data and based on the limitations of transformer based models.

0:04:46.240 --> 0:04:48.520
<v Speaker 3>And wouldn't you know it, I was bloody right.

0:04:48.640 --> 0:04:51.440
<v Speaker 2>I'm not going to do many of these, but this one, really,

0:04:51.839 --> 0:04:54.840
<v Speaker 2>this one I'm right on. A few weeks ago, Bloomberg

0:04:54.920 --> 0:04:57.600
<v Speaker 2>reported the open AI. Google and Anthropic are struggling to

0:04:57.680 --> 0:05:00.000
<v Speaker 2>build more advanced AI and the open ais A right

0:05:00.240 --> 0:05:03.000
<v Speaker 2>model otherwise known as GPT five did not hit the

0:05:03.040 --> 0:05:06.640
<v Speaker 2>company's desired performance. And that and I quote again, Orion

0:05:06.720 --> 0:05:08.880
<v Speaker 2>is so far not considered to be a bigger step

0:05:09.000 --> 0:05:10.919
<v Speaker 2>up as it was from GPT three point five. To

0:05:10.960 --> 0:05:14.760
<v Speaker 2>GPT four its current model. You will be shocked to

0:05:14.800 --> 0:05:18.040
<v Speaker 2>hear that the reason is that it's become increasingly difficult

0:05:18.040 --> 0:05:20.760
<v Speaker 2>to find new, untapped sources of high quality human made

0:05:20.760 --> 0:05:23.280
<v Speaker 2>training data that can be used to build more advanced

0:05:23.320 --> 0:05:26.320
<v Speaker 2>AI systems, something that I said in March. I said

0:05:26.320 --> 0:05:28.320
<v Speaker 2>it would happen in March and pissed off that people

0:05:28.320 --> 0:05:32.080
<v Speaker 2>said I was a pessimist. Well, who's a pessimist now me?

0:05:32.200 --> 0:05:34.840
<v Speaker 2>I guess I don't know. But they also added one

0:05:34.880 --> 0:05:37.400
<v Speaker 2>other thing, which is that they believe and I quote,

0:05:37.480 --> 0:05:39.800
<v Speaker 2>that the AGI bubble is bursting a little bit, which

0:05:39.800 --> 0:05:43.240
<v Speaker 2>is something I said in July. AGI isn't coming out

0:05:43.240 --> 0:05:46.039
<v Speaker 2>of this shit. Let's just be honest. And I also

0:05:46.120 --> 0:05:49.000
<v Speaker 2>want to stop and stare really hard at one particular point,

0:05:49.040 --> 0:05:52.320
<v Speaker 2>and I quote again from Bloomberg. These issues challenged the

0:05:52.320 --> 0:05:55.080
<v Speaker 2>gospel that's taken hold in Silicon Valley in recent years,

0:05:55.160 --> 0:05:58.960
<v Speaker 2>particularly since OpenAI released chat GPT two years ago. Much

0:05:59.000 --> 0:06:01.280
<v Speaker 2>of the tech industry is bet on so called scaling

0:06:01.360 --> 0:06:04.200
<v Speaker 2>laws that say more computing power, data, and larger models

0:06:04.200 --> 0:06:06.760
<v Speaker 2>will inevitably pave the way the greater leaps forward in

0:06:06.800 --> 0:06:10.520
<v Speaker 2>the power of AI. The only people taking this as gospel.

0:06:10.520 --> 0:06:12.279
<v Speaker 2>Have been members of the media unwilling to ask the

0:06:12.320 --> 0:06:14.520
<v Speaker 2>tough questions, and AI founders that don't know what the

0:06:14.520 --> 0:06:16.960
<v Speaker 2>fuck they're talking about, or they intend to mislead you.

0:06:17.480 --> 0:06:20.279
<v Speaker 2>Generative AI's products have effectively been trapped in amber for

0:06:20.320 --> 0:06:22.600
<v Speaker 2>over a year. It's been blatantly obvious if you fucking

0:06:22.720 --> 0:06:24.680
<v Speaker 2>use them and pissed off, I shouldn't swear so much.

0:06:26.400 --> 0:06:29.400
<v Speaker 2>There have been no meaningful, industry defining products out of

0:06:29.400 --> 0:06:32.279
<v Speaker 2>this because, and I quote Darreness and Mooglu, the economist

0:06:32.360 --> 0:06:35.839
<v Speaker 2>that MIT back in May, more powerful models do not

0:06:35.960 --> 0:06:39.680
<v Speaker 2>unlock new features or really change the experience, Nor what

0:06:39.720 --> 0:06:42.680
<v Speaker 2>you can build with transform based models is really a

0:06:42.720 --> 0:06:46.640
<v Speaker 2>worthwhile product. Or put another way, a slightly better white

0:06:46.640 --> 0:06:50.919
<v Speaker 2>elephant is still a white elephant. Despite the billions of

0:06:50.920 --> 0:06:53.640
<v Speaker 2>dollars burned and thousands of glossy headlines, it's difficult to

0:06:53.640 --> 0:06:57.640
<v Speaker 2>point to any truly important generative AI product, even Apple Intelligence,

0:06:57.640 --> 0:06:59.760
<v Speaker 2>the only thing that Apple really had to add to

0:06:59.800 --> 0:07:05.120
<v Speaker 2>the latest iPhone. It sucks, it's not useful. I can

0:07:05.200 --> 0:07:08.200
<v Speaker 2>make a special emoji. Now I now get summaries of

0:07:08.200 --> 0:07:11.560
<v Speaker 2>my texts that are completely or vaguely incorrect or just

0:07:11.840 --> 0:07:15.440
<v Speaker 2>summarize a giant, meaningful paragraph into a blob of a sentence.

0:07:15.640 --> 0:07:20.240
<v Speaker 2>It's so stupid. And just as a side question, what

0:07:20.280 --> 0:07:22.120
<v Speaker 2>the hell is Apple going to put in the next iPhone?

0:07:22.200 --> 0:07:23.960
<v Speaker 2>I buy one of these every year. I'm a little

0:07:23.960 --> 0:07:27.480
<v Speaker 2>big oincoin CoInc but still I don't even know why

0:07:27.520 --> 0:07:30.160
<v Speaker 2>ad upgrade again. The camera is already about as good

0:07:30.160 --> 0:07:30.920
<v Speaker 2>as it's going to get.

0:07:32.160 --> 0:07:32.560
<v Speaker 3>Anyway.

0:07:32.960 --> 0:07:35.800
<v Speaker 2>There are people that use chat GPT two hundred million

0:07:35.920 --> 0:07:38.520
<v Speaker 2>for them a week, allegedly losing the company money with

0:07:38.560 --> 0:07:40.680
<v Speaker 2>every prompt, by the way, but there's little to suggest

0:07:40.720 --> 0:07:45.000
<v Speaker 2>that there's widespread adoption of actual generative AI software. The

0:07:45.000 --> 0:07:47.920
<v Speaker 2>Information reported in September that between zero point one percent

0:07:47.960 --> 0:07:50.000
<v Speaker 2>and one percent of the four hundred and forty million

0:07:50.240 --> 0:07:52.600
<v Speaker 2>of Microsoft's business customers were willing to pay for its

0:07:52.600 --> 0:07:56.360
<v Speaker 2>AI powered Copilot, and in late October, Microsoft claimed that

0:07:56.440 --> 0:07:59.960
<v Speaker 2>it was on pace to make AI a ten billion

0:08:00.200 --> 0:08:03.680
<v Speaker 2>dollar a year business, which sounds really good until you

0:08:03.840 --> 0:08:08.400
<v Speaker 2>think about it for roughly ten seconds. First of all,

0:08:08.600 --> 0:08:12.000
<v Speaker 2>Microsoft does not have an AI business unit, which means

0:08:12.040 --> 0:08:14.480
<v Speaker 2>that this annual ten billion dollars or two and a

0:08:14.480 --> 0:08:17.440
<v Speaker 2>half billion a quarter revenue figure is split across providing

0:08:17.560 --> 0:08:21.119
<v Speaker 2>cloud compute services, and Azure selling Copilot. The dumb people

0:08:21.120 --> 0:08:24.680
<v Speaker 2>with Microsoft three sixty five subscriptions selling git Hub, Copilot,

0:08:24.800 --> 0:08:27.920
<v Speaker 2>and basically anything else with AI on it. Microsoft is

0:08:28.000 --> 0:08:31.040
<v Speaker 2>cherry picking a number based on nonspecific criteria and claiming

0:08:31.080 --> 0:08:33.640
<v Speaker 2>it's a big deal when it's actually pretty pathetic, considering

0:08:33.679 --> 0:08:37.080
<v Speaker 2>that Microsoft's capital expenditures will likely hit over sixty billion

0:08:37.120 --> 0:08:39.720
<v Speaker 2>dollars in twenty twenty four with no sign they're going

0:08:39.760 --> 0:08:44.560
<v Speaker 2>to slow down. No, that's sticky word. Revenue not profit.

0:08:44.720 --> 0:08:48.400
<v Speaker 2>Those are two very different things. How much is Microsoft

0:08:48.440 --> 0:08:51.520
<v Speaker 2>spending to make ten billion dollars a year? Open ai

0:08:51.640 --> 0:08:53.880
<v Speaker 2>currently spends two dollars and thirty five cents to make

0:08:53.920 --> 0:08:56.559
<v Speaker 2>a dollar, and Microsoft CFO Amyhood said that open ai

0:08:56.600 --> 0:08:59.120
<v Speaker 2>would cut into Microsoft profits in their last earning score,

0:08:59.440 --> 0:09:02.600
<v Speaker 2>losing it a remarkable one point five billion dollars, mainly

0:09:02.640 --> 0:09:04.840
<v Speaker 2>because of the expected loss from a company that has

0:09:04.920 --> 0:09:07.959
<v Speaker 2>only ever lost money now a year ago. In October

0:09:08.000 --> 0:09:10.760
<v Speaker 2>twenty twenty three, The Wall Street Journal reported that Microsoft

0:09:10.760 --> 0:09:13.400
<v Speaker 2>was losing an average of twenty dollars per user per

0:09:13.480 --> 0:09:17.000
<v Speaker 2>month on GitHub Copilot, a product with over a million users.

0:09:17.480 --> 0:09:20.240
<v Speaker 2>If this is true, by the way, this suggests losses

0:09:20.280 --> 0:09:22.760
<v Speaker 2>of at least two hundred million a year. They have

0:09:22.880 --> 0:09:25.439
<v Speaker 2>one point eight million users. Allegedly, this is based on

0:09:25.480 --> 0:09:29.719
<v Speaker 2>documents have reviewed. It's not great either way. That two

0:09:29.840 --> 0:09:32.319
<v Speaker 2>hundred million dollars is a lot of money to lose.

0:09:32.480 --> 0:09:35.240
<v Speaker 2>I would personally like to make two hundred million dollars

0:09:35.320 --> 0:09:37.480
<v Speaker 2>rather than lose it. Don't ask me, though I don't

0:09:37.520 --> 0:09:38.280
<v Speaker 2>run Microsoft.

0:09:39.360 --> 0:09:39.560
<v Speaker 3>Now.

0:09:39.600 --> 0:09:41.920
<v Speaker 2>Microsoft is still yet to break out exactly how much

0:09:41.960 --> 0:09:44.839
<v Speaker 2>generative AI is increasing revenue in the specific business units

0:09:44.880 --> 0:09:47.520
<v Speaker 2>they have. Generally, if a company's doing well at something,

0:09:47.600 --> 0:09:50.600
<v Speaker 2>they take great pains to make that clear. Instead, Microsoft

0:09:50.679 --> 0:09:53.079
<v Speaker 2>chose in August to revamp its reporting structure to give

0:09:53.120 --> 0:09:56.200
<v Speaker 2>better visibility into cloud consumption revenue, which is something you

0:09:56.280 --> 0:09:58.839
<v Speaker 2>do if you say, anticipate you're going to have your

0:09:58.840 --> 0:10:01.240
<v Speaker 2>worst day of trading in year after your next earnings,

0:10:01.240 --> 0:10:04.240
<v Speaker 2>as Microsoft did in October. It's all very good, it's

0:10:04.280 --> 0:10:07.040
<v Speaker 2>all going well. Now it must be clear that every

0:10:07.080 --> 0:10:09.480
<v Speaker 2>single one of these investments and products, as I've been

0:10:09.559 --> 0:10:12.400
<v Speaker 2>hyped with the whisper, that they would get exponentially better

0:10:12.440 --> 0:10:15.160
<v Speaker 2>over time, and that eventually the two hundred billion dollars

0:10:15.200 --> 0:10:18.839
<v Speaker 2>in capital expenditures would spit out this remarkable productivity improvement,

0:10:19.080 --> 0:10:22.000
<v Speaker 2>this crazy new product that would change our lives, fascinating

0:10:22.000 --> 0:10:24.480
<v Speaker 2>new things that consumers and enterprise of buying droves and

0:10:24.520 --> 0:10:27.720
<v Speaker 2>talk about how much they loved. Instead, Big tech has

0:10:27.720 --> 0:10:31.360
<v Speaker 2>found itself peddling increasingly more expensive iterations of near identical

0:10:31.400 --> 0:10:34.120
<v Speaker 2>large language models and shitty products attached to them, a

0:10:34.160 --> 0:10:36.040
<v Speaker 2>direct result of all of them having to use the

0:10:36.080 --> 0:10:39.520
<v Speaker 2>same training data, which they're now running out of. But

0:10:39.600 --> 0:10:42.040
<v Speaker 2>if you're running out of stuff and you can't find

0:10:42.040 --> 0:10:45.520
<v Speaker 2>stuff to buy, I really recommend the following advertisement. I'm

0:10:45.559 --> 0:10:48.400
<v Speaker 2>sure it will totally gel with my beliefs. The things

0:10:48.440 --> 0:10:51.200
<v Speaker 2>I'm talking about right now won't be embarrassing at all,

0:11:02.000 --> 0:11:05.679
<v Speaker 2>and we're back now. There's another assumption that people have

0:11:05.920 --> 0:11:10.480
<v Speaker 2>about these so called scaling laws. That's been by simply

0:11:10.480 --> 0:11:14.319
<v Speaker 2>building bigger data centers with even bigger, more powerful GPUs,

0:11:14.440 --> 0:11:17.400
<v Speaker 2>the expensive power hungry graphics processing units that use to

0:11:17.440 --> 0:11:19.840
<v Speaker 2>both train and run these models, and throwing as much

0:11:19.880 --> 0:11:21.880
<v Speaker 2>training data at them as possible.

0:11:22.000 --> 0:11:24.000
<v Speaker 3>They would simply start doing new things.

0:11:24.000 --> 0:11:26.920
<v Speaker 2>They'd have new capabilities, despite their being little proof that

0:11:26.960 --> 0:11:31.439
<v Speaker 2>they would do so in any way, shape or form. Microsoft, Meta, Amazon,

0:11:31.480 --> 0:11:33.480
<v Speaker 2>and Google of all burn billions, and the assumption that

0:11:33.559 --> 0:11:37.320
<v Speaker 2>doing so would create something, you know, a thing, a

0:11:37.360 --> 0:11:40.800
<v Speaker 2>good thing, like a human level artificial general intelligence, or

0:11:40.840 --> 0:11:44.600
<v Speaker 2>a product that made more money than it cost that

0:11:44.679 --> 0:11:48.760
<v Speaker 2>people liked. It's become kind of obvious that that isn't

0:11:48.800 --> 0:11:52.080
<v Speaker 2>going to happen. As we speak, members of the media

0:11:52.080 --> 0:11:55.360
<v Speaker 2>who should know better are already desperately trying to prove

0:11:55.400 --> 0:11:58.040
<v Speaker 2>that this is not a problem the information in a

0:11:58.080 --> 0:12:00.640
<v Speaker 2>similar story to Bloomberg's attempted to put lipstick on the

0:12:00.679 --> 0:12:03.720
<v Speaker 2>pig of generative AI, framing the lack of meaningful progress

0:12:03.760 --> 0:12:07.440
<v Speaker 2>of GPT fivers fine because open ai can now combine

0:12:07.480 --> 0:12:10.800
<v Speaker 2>its GPT five model with its one reasoning model, which

0:12:10.800 --> 0:12:12.760
<v Speaker 2>is the one that can't count the number of ours

0:12:12.760 --> 0:12:17.280
<v Speaker 2>and the strawberry by the way, which will then do something,

0:12:16.840 --> 0:12:22.080
<v Speaker 2>something good. Something's gonna happen. Like Sam Altman said, it

0:12:22.160 --> 0:12:26.760
<v Speaker 2>could write a lot more very difficult code. You know, Samultman,

0:12:26.840 --> 0:12:29.560
<v Speaker 2>the career liar who intimated the GPT five may function

0:12:29.640 --> 0:12:33.959
<v Speaker 2>like a virtual brain in may like these people are liars.

0:12:33.960 --> 0:12:35.920
<v Speaker 2>They're liars, they're lying to you. They were lying then

0:12:35.920 --> 0:12:39.120
<v Speaker 2>they're lying now now I couldn't possibly leave out chief

0:12:39.120 --> 0:12:41.959
<v Speaker 2>Ellly cheerleader Casey Newton, who wrote on platform Or a

0:12:42.000 --> 0:12:44.600
<v Speaker 2>few weeks ago that diminishing returns in training models may

0:12:44.640 --> 0:12:46.680
<v Speaker 2>not matter as much as you would guess, with his

0:12:46.720 --> 0:12:49.520
<v Speaker 2>evidence being the anthropic, who he also claims has not

0:12:49.559 --> 0:12:52.400
<v Speaker 2>been prone to hyperbole, do not think the scaling laws

0:12:52.400 --> 0:12:55.680
<v Speaker 2>are ending now. The original scaling law's paper, partly written

0:12:55.679 --> 0:12:59.720
<v Speaker 2>by Dario Amadesi of Anthropic important to know and to

0:12:59.800 --> 0:13:03.280
<v Speaker 2>be clear, in a fourteen thousand word ophed that Casey

0:13:03.320 --> 0:13:07.080
<v Speaker 2>Newton for no reason wrote two pieces about Anthropic CEO Dario.

0:13:07.400 --> 0:13:09.280
<v Speaker 3>He said that, and I quote AI.

0:13:09.080 --> 0:13:12.760
<v Speaker 2>Accelerated neuroscience is likely to vastly improve treatments for or

0:13:12.800 --> 0:13:15.520
<v Speaker 2>even cure most mental illness, which is the kind of

0:13:15.600 --> 0:13:18.000
<v Speaker 2>hyperbole that should be Have you tired and feathered and

0:13:18.040 --> 0:13:20.520
<v Speaker 2>put in a jail? I'm not seriously saying you put

0:13:20.559 --> 0:13:22.880
<v Speaker 2>him in jail? But why are we Why are we

0:13:22.920 --> 0:13:25.959
<v Speaker 2>trusting these people? Why are we listening to them? Why

0:13:26.000 --> 0:13:29.360
<v Speaker 2>are we treating them as if they're telling the truth

0:13:29.679 --> 0:13:32.800
<v Speaker 2>or even that they know what's going on? But let's

0:13:32.800 --> 0:13:36.640
<v Speaker 2>summarize the main technology behind the entire and I say

0:13:36.640 --> 0:13:39.920
<v Speaker 2>this in quotation marks by the way, artificial intelligence boom

0:13:40.080 --> 0:13:43.360
<v Speaker 2>is generative AI transformed based models like open AI's GPT

0:13:43.480 --> 0:13:46.760
<v Speaker 2>four and soon GPT five, and said technology has speaked

0:13:46.920 --> 0:13:49.440
<v Speaker 2>with diminishing returns from the only ways of making them better,

0:13:49.800 --> 0:13:52.199
<v Speaker 2>feeding them, training data, and throwing tons of compute at them,

0:13:52.360 --> 0:13:55.840
<v Speaker 2>suggesting that we may have, as I said before, reached PKI.

0:13:56.640 --> 0:13:59.800
<v Speaker 2>Generative AI is incredibly unprofitable. Open Ai, the biggest player

0:13:59.800 --> 0:14:01.719
<v Speaker 2>in the industry, is on course to lose more than

0:14:01.760 --> 0:14:05.000
<v Speaker 2>five billion dollars this year, with competitor Anthropic, which also

0:14:05.000 --> 0:14:07.760
<v Speaker 2>makes its own transformer based model, clawed, on course to

0:14:07.800 --> 0:14:10.080
<v Speaker 2>lose more than two point seven billion dollars this year.

0:14:10.480 --> 0:14:13.880
<v Speaker 2>They just raised another four billion. Every single big tech

0:14:13.880 --> 0:14:16.440
<v Speaker 2>company has thrown billions of dollars, as much as seventy

0:14:16.480 --> 0:14:19.440
<v Speaker 2>five billion dollars in Amazon's case in twenty twenty four alone.

0:14:19.520 --> 0:14:21.880
<v Speaker 2>Are building the data centers and acquiring the GPUs to

0:14:21.880 --> 0:14:24.840
<v Speaker 2>populate said data centers, specifically so they can train their

0:14:24.880 --> 0:14:27.520
<v Speaker 2>models and other people's models, or serve customers that would

0:14:27.520 --> 0:14:30.800
<v Speaker 2>integrate Generativai into their businesses, something that does not appear

0:14:30.840 --> 0:14:34.080
<v Speaker 2>to be happening at scale, and these investments could theoretically

0:14:34.120 --> 0:14:35.920
<v Speaker 2>be used for other products but these data cent as

0:14:35.960 --> 0:14:39.880
<v Speaker 2>a heavily focused on GENERATIVIAI. Business Insider reports that Microsoft

0:14:39.960 --> 0:14:42.720
<v Speaker 2>intends to amass one point eight million GPUs by the

0:14:42.800 --> 0:14:45.360
<v Speaker 2>end of this year, costing it tens of billions of dollars.

0:14:46.240 --> 0:14:49.760
<v Speaker 2>Worse still, many of these companies integrating GENERATIVIAI do so

0:14:49.800 --> 0:14:53.200
<v Speaker 2>by connecting to models made by either Open AI or Anthropic,

0:14:53.320 --> 0:14:56.240
<v Speaker 2>both of whom are running unprofitable businesses and likely charging

0:14:56.240 --> 0:14:58.640
<v Speaker 2>nowhere near enough to cover their costs. As I've said

0:14:58.640 --> 0:15:01.600
<v Speaker 2>before in my article the sub MAI Crisis, in the

0:15:01.600 --> 0:15:04.080
<v Speaker 2>event that these companies start charging what they actually need

0:15:04.120 --> 0:15:07.800
<v Speaker 2>to their real costs, I hypothesize that it will multiply

0:15:07.840 --> 0:15:09.640
<v Speaker 2>the cost of their customers to the point that they

0:15:09.640 --> 0:15:12.240
<v Speaker 2>can't afford to run their businesses, or at the very least,

0:15:12.280 --> 0:15:14.840
<v Speaker 2>we'll have to remove or scale back generative AI functionality

0:15:14.880 --> 0:15:19.840
<v Speaker 2>in their products. It's just it's such a waste. The

0:15:20.000 --> 0:15:22.880
<v Speaker 2>entire tech industry has become orientated around this dead end

0:15:22.880 --> 0:15:25.840
<v Speaker 2>technology that requires burning billions and billions of dollars to

0:15:25.840 --> 0:15:28.440
<v Speaker 2>provide in essential products that cost them more money to

0:15:28.480 --> 0:15:32.440
<v Speaker 2>serve than anybody ever would pay. Their big strategy has

0:15:32.480 --> 0:15:34.760
<v Speaker 2>to be to throw more money at the problem until

0:15:34.800 --> 0:15:38.400
<v Speaker 2>one of these transformer based models created something useful. Despite

0:15:38.400 --> 0:15:40.920
<v Speaker 2>the fact that every iteration of GPT in other models

0:15:40.920 --> 0:15:46.000
<v Speaker 2>has been well iterative, and it's weird, you think at

0:15:46.000 --> 0:15:49.520
<v Speaker 2>some point that goes, shit, do we actually.

0:15:49.240 --> 0:15:53.040
<v Speaker 3>Have the ability to build products with this? What are

0:15:53.040 --> 0:15:53.640
<v Speaker 3>the products?

0:15:53.680 --> 0:15:57.520
<v Speaker 2>Maybe we should work out the products first before we

0:15:57.600 --> 0:16:00.120
<v Speaker 2>throw all the capex at it. But wait, no, oh,

0:16:00.480 --> 0:16:04.640
<v Speaker 2>over yonder, I couldn't possibly not do this because the

0:16:04.720 --> 0:16:07.880
<v Speaker 2>other big tech company that also has no ideas they're

0:16:07.880 --> 0:16:10.000
<v Speaker 2>doing this. And if I don't do this, my investors

0:16:10.040 --> 0:16:12.000
<v Speaker 2>are going to be angry at me. And then what

0:16:12.120 --> 0:16:15.800
<v Speaker 2>will I do? Oh no, oh no, what could I

0:16:15.880 --> 0:16:19.480
<v Speaker 2>possibly do? If the investors I don't fucking know, that's

0:16:19.560 --> 0:16:25.640
<v Speaker 2>your problem? Why waste this much money? It's just there's

0:16:25.640 --> 0:16:28.680
<v Speaker 2>never been any proof other than these benchmarks that are

0:16:28.760 --> 0:16:31.720
<v Speaker 2>really easy to gain and also only showed just this

0:16:31.840 --> 0:16:35.760
<v Speaker 2>vague power of these models. It's been obvious that GPT

0:16:35.920 --> 0:16:39.560
<v Speaker 2>or other models wouldn't become conscious that they're not going

0:16:39.600 --> 0:16:41.400
<v Speaker 2>to do more than they do today or three months

0:16:41.440 --> 0:16:44.120
<v Speaker 2>ago or even a year ago. Hesitate to give Gary

0:16:44.160 --> 0:16:46.360
<v Speaker 2>Marcus credit, but in twenty twenty three he was saying

0:16:46.440 --> 0:16:49.400
<v Speaker 2>this if not earlier. Many people have as well, and

0:16:49.440 --> 0:16:53.120
<v Speaker 2>it's just really really, really really frustrating. Better Offline isn't

0:16:53.160 --> 0:16:54.960
<v Speaker 2>even a year old. But when we put out our

0:16:55.000 --> 0:16:58.480
<v Speaker 2>PKI episode, I got so much flak. I got so

0:16:58.640 --> 0:17:01.040
<v Speaker 2>much shit for being a hater. I didn't really understand things.

0:17:01.080 --> 0:17:03.040
<v Speaker 2>That my fly was open in my Instagram picture, that

0:17:03.120 --> 0:17:05.200
<v Speaker 2>I didn't get it, and that in mere months I

0:17:05.200 --> 0:17:06.240
<v Speaker 2>would be proven wrong.

0:17:06.880 --> 0:17:08.680
<v Speaker 3>Well here we are. How wrong am I? Now?

0:17:08.720 --> 0:17:11.439
<v Speaker 2>What happens next? Exactly where do all these hundreds of

0:17:11.440 --> 0:17:14.000
<v Speaker 2>billions of dollars go? What happens to Open AI when

0:17:14.040 --> 0:17:17.240
<v Speaker 2>it collapses? What does Microsoft do with all of these GPUs?

0:17:17.359 --> 0:17:19.560
<v Speaker 2>Because you can't just move them into other shit? You

0:17:19.600 --> 0:17:23.400
<v Speaker 2>know from what I hear, they don't really have a plan.

0:17:24.359 --> 0:17:27.760
<v Speaker 2>And that's the scariest thing, because what happens to a

0:17:27.800 --> 0:17:30.479
<v Speaker 2>stock market that's dependent on big tech companies for growth

0:17:30.600 --> 0:17:32.960
<v Speaker 2>when the big tech companies can't work out a way

0:17:33.000 --> 0:17:35.320
<v Speaker 2>to grow anymore, and in fact, their big path to

0:17:35.400 --> 0:17:37.200
<v Speaker 2>try and to grow more was to burn a shit

0:17:37.280 --> 0:17:39.640
<v Speaker 2>ton of money on things that people hate, that destroy

0:17:39.680 --> 0:17:45.119
<v Speaker 2>our environment. I know, I know, I'm angry. I know

0:17:45.480 --> 0:17:48.920
<v Speaker 2>I should calm down. I should, But as I said

0:17:48.960 --> 0:17:52.880
<v Speaker 2>in the Rot Society, this money could go elsewhere, more

0:17:52.920 --> 0:17:55.439
<v Speaker 2>things could be done. It would enter a fallow period

0:17:55.480 --> 0:17:57.919
<v Speaker 2>of tech. But we don't just have to burn all

0:17:57.960 --> 0:18:02.000
<v Speaker 2>this money. We don't have to do that. Why not

0:18:02.119 --> 0:18:05.760
<v Speaker 2>make the products you have already better? Because stapling generative

0:18:05.800 --> 0:18:09.399
<v Speaker 2>AI on them, I think it makes them worse. But

0:18:09.440 --> 0:18:12.920
<v Speaker 2>there are more problems ahead. There are problems around the infrastructure.

0:18:13.320 --> 0:18:15.480
<v Speaker 2>And in the next episode, I'm gonna break down these

0:18:15.480 --> 0:18:19.280
<v Speaker 2>worrying problems and I'm gonna kind of tell you what

0:18:19.359 --> 0:18:22.959
<v Speaker 2>happens next is best I can. I really appreciate your

0:18:22.960 --> 0:18:25.320
<v Speaker 2>faith in me. And there are many people who also

0:18:25.359 --> 0:18:28.240
<v Speaker 2>contacted me and said, no, you bang on, keep going.

0:18:28.480 --> 0:18:31.360
<v Speaker 2>I'm glad they did. I'm very grateful for your audience.

0:18:31.960 --> 0:18:33.920
<v Speaker 2>I love you all much like you said in the menu,

0:18:42.359 --> 0:18:44.800
<v Speaker 2>thank you for listening to Better Offline. The editor and

0:18:44.840 --> 0:18:48.000
<v Speaker 2>composer of the Better Offline theme song is Metasowski. You

0:18:48.040 --> 0:18:50.280
<v Speaker 2>can check out more of his music and audio projects

0:18:50.440 --> 0:18:53.879
<v Speaker 2>at Matasowski dot com, M A T T O. S

0:18:54.000 --> 0:18:58.199
<v Speaker 2>O w Ski dot com. You can email me at

0:18:58.240 --> 0:19:01.080
<v Speaker 2>easy at Better offline dot com, or visit Better Offline

0:19:01.119 --> 0:19:03.200
<v Speaker 2>dot com to find more podcast links and of course

0:19:03.240 --> 0:19:06.360
<v Speaker 2>my newsletter. I also really recommend you go to chat

0:19:06.359 --> 0:19:09.000
<v Speaker 2>dot Where's youread dot at to visit the discord, and

0:19:09.040 --> 0:19:11.760
<v Speaker 2>go to our slash Better Offline to check out our reddit.

0:19:12.520 --> 0:19:13.879
<v Speaker 2>Thank you so much for listening.

0:19:14.720 --> 0:19:17.439
<v Speaker 1>Better Offline is a production of cool Zone Media. For

0:19:17.520 --> 0:19:20.680
<v Speaker 1>more from cool Zone Media, visit our website cool Zonemedia

0:19:20.760 --> 0:19:23.600
<v Speaker 1>dot com, or check us out on the iHeartRadio app,

0:19:23.640 --> 0:19:26.359
<v Speaker 1>Apple Podcasts, or wherever you get your podcasts.