1
00:00:02,840 --> 00:00:03,560
Speaker 1: Zone Media.

2
00:00:05,440 --> 00:00:08,240
Speaker 2: Hello, and welcome to Better Offline. I'm your surly yet

3
00:00:08,280 --> 00:00:22,720
Speaker 2: lovable host ed Zitron. Today I'm going to kick off

4
00:00:22,720 --> 00:00:25,160
Speaker 2: by reading something I wrote in March twenty twenty four

5
00:00:25,400 --> 00:00:28,800
Speaker 2: and talked about in the episode PKI. What if what

6
00:00:28,800 --> 00:00:30,920
Speaker 2: we're seeing today isn't a glimpse of the future but

7
00:00:31,040 --> 00:00:34,240
Speaker 2: the new terms of the present. What if artificial intelligence

8
00:00:34,280 --> 00:00:36,959
Speaker 2: isn't actually capable of doing much more than what we're

9
00:00:36,960 --> 00:00:39,440
Speaker 2: seeing today, and what if there's no clear timeline when

10
00:00:39,479 --> 00:00:41,320
Speaker 2: it will be able to do more? What if this

11
00:00:41,560 --> 00:00:43,960
Speaker 2: entire hype cycle has been built on hot air ghost

12
00:00:43,960 --> 00:00:46,600
Speaker 2: by a compliant media, ready and willing to take career

13
00:00:46,640 --> 00:00:51,880
Speaker 2: embellishes at their word. Reading that back, well, I think

14
00:00:51,880 --> 00:00:53,440
Speaker 2: I might have been right, And that's kind of what

15
00:00:53,440 --> 00:00:55,120
Speaker 2: I'm going to get at today. I don't want to

16
00:00:55,120 --> 00:00:57,400
Speaker 2: scream mustard. I'm not going to get smug about it.

17
00:00:57,960 --> 00:01:00,080
Speaker 2: But this is what we're getting into today and in

18
00:01:00,120 --> 00:01:02,400
Speaker 2: the next episode that will come out on Friday. Now,

19
00:01:02,440 --> 00:01:04,560
Speaker 2: I'll be linking to some articles, so check the episode

20
00:01:04,640 --> 00:01:05,880
Speaker 2: notes if you want to read them. But I'm going

21
00:01:05,920 --> 00:01:07,280
Speaker 2: to get a lot into the spoken word.

22
00:01:08,880 --> 00:01:10,399
Speaker 3: So I warned you in February.

23
00:01:10,040 --> 00:01:12,520
Speaker 2: That generative AI has no killer apps and had no

24
00:01:12,600 --> 00:01:15,360
Speaker 2: way of justifying its valuations. I also warned you in

25
00:01:15,400 --> 00:01:18,120
Speaker 2: March that generative AI had already peaked, and I pleaded

26
00:01:18,120 --> 00:01:21,399
Speaker 2: with the tech industry in April to consider an eventuality

27
00:01:21,440 --> 00:01:23,920
Speaker 2: where the jump between GPT four, which is the most

28
00:01:23,920 --> 00:01:26,920
Speaker 2: current model, or GPT four to zero to GPT five

29
00:01:27,080 --> 00:01:29,679
Speaker 2: was not significant, in part due to a lack of

30
00:01:29,720 --> 00:01:33,080
Speaker 2: training data, one of the more obvious things. I shared

31
00:01:33,080 --> 00:01:36,560
Speaker 2: more concerns in July that the transformer based architecture underpinning

32
00:01:36,600 --> 00:01:39,360
Speaker 2: generative AI things like chad GPT was a dead end

33
00:01:39,560 --> 00:01:42,520
Speaker 2: and that there were really not many ways we'd progress

34
00:01:42,560 --> 00:01:45,200
Speaker 2: past the products we'd already seen back then, impart due

35
00:01:45,240 --> 00:01:47,400
Speaker 2: to the limits of training data are convention and the

36
00:01:47,440 --> 00:01:50,800
Speaker 2: limits of the models that use said training data. In August,

37
00:01:50,880 --> 00:01:53,880
Speaker 2: I summarized the pale horses of the AI apocalypse events

38
00:01:54,280 --> 00:01:56,960
Speaker 2: many that have now come to pass. Unafraid that would

39
00:01:57,000 --> 00:02:00,720
Speaker 2: signify the end well being nigh, though it's not quite

40
00:02:00,760 --> 00:02:03,120
Speaker 2: here yet and it's not obvious when it will be.

41
00:02:03,320 --> 00:02:06,000
Speaker 2: But this can't last forever. But I also add that

42
00:02:06,040 --> 00:02:08,560
Speaker 2: GPT five would not change the game enough to matter,

43
00:02:08,800 --> 00:02:11,120
Speaker 2: let alone add a new architecture to build future and

44
00:02:11,240 --> 00:02:15,000
Speaker 2: more capable models or products of any kind. Now, throughout

45
00:02:15,000 --> 00:02:16,680
Speaker 2: the things I've written and the things have spoken, I've

46
00:02:16,680 --> 00:02:20,280
Speaker 2: repeatedly made the point that separate to any core value proposition,

47
00:02:20,520 --> 00:02:23,320
Speaker 2: training data, drought or unsustainable economics that I've gone over

48
00:02:23,400 --> 00:02:25,720
Speaker 2: quite a lot, general IFAI is a dead end due

49
00:02:25,720 --> 00:02:29,240
Speaker 2: to the limitations of a probabilistic model that hallucinates. Now,

50
00:02:29,320 --> 00:02:31,360
Speaker 2: just to be clear with what that means, it's guessing

51
00:02:31,360 --> 00:02:33,639
Speaker 2: what the next thing might be, and it's quite good

52
00:02:33,720 --> 00:02:35,720
Speaker 2: at it, but quite good is actually.

53
00:02:35,520 --> 00:02:36,240
Speaker 3: Kind of shit.

54
00:02:36,639 --> 00:02:40,200
Speaker 2: And hallucinations, of of course, are whether you authoritatively state

55
00:02:40,280 --> 00:02:42,600
Speaker 2: things that aren't true, like when chat GPT tells you

56
00:02:42,680 --> 00:02:45,639
Speaker 2: something like I don't know there are two hours in strawberry.

57
00:02:46,360 --> 00:02:49,080
Speaker 2: The hallucination problem is one that is nowhere closer to

58
00:02:49,120 --> 00:02:51,840
Speaker 2: being solved. You may remember a few months ago when

59
00:02:51,880 --> 00:02:54,120
Speaker 2: you had every tech executive, you had Tim Cooks and

60
00:02:54,120 --> 00:02:57,880
Speaker 2: it's satching Adella Sun Dapashai. We'll deal with the hallucination problem.

61
00:02:58,000 --> 00:03:00,000
Speaker 2: It'll be all right. But I want to be clear,

62
00:03:00,480 --> 00:03:03,280
Speaker 2: they have not solved it, they have not really mitigated

63
00:03:04,000 --> 00:03:06,400
Speaker 2: and there's no fixing it, at least with the current technology,

64
00:03:06,840 --> 00:03:09,800
Speaker 2: it's not going anywhere, and it makes all of this

65
00:03:09,840 --> 00:03:12,440
Speaker 2: stuff kind of a non starter for many business tasks.

66
00:03:13,120 --> 00:03:17,120
Speaker 2: I have since March expressed great dismay about the credulousness

67
00:03:17,120 --> 00:03:20,240
Speaker 2: of the media about this, and they're weird acceptance of

68
00:03:20,280 --> 00:03:23,320
Speaker 2: this inevitable way in which generative AI will change society,

69
00:03:23,600 --> 00:03:26,080
Speaker 2: despite the fact there's not really a meaningful product that

70
00:03:26,160 --> 00:03:30,080
Speaker 2: might justify any of this bullshit, This environmentally destructive nonsense

71
00:03:30,360 --> 00:03:32,360
Speaker 2: led by a company that burns more than five billion

72
00:03:32,400 --> 00:03:34,600
Speaker 2: dollars a year in big tech firms that are spending

73
00:03:34,600 --> 00:03:37,600
Speaker 2: two hundred billion dollars on data centers for products that

74
00:03:37,640 --> 00:03:42,160
Speaker 2: people don't want or even potentially use. And you're going

75
00:03:42,200 --> 00:03:45,400
Speaker 2: to need context for everything I'm saying today. So it's

76
00:03:45,440 --> 00:03:48,520
Speaker 2: worth going over how these models work and how they're trained.

77
00:03:49,080 --> 00:03:51,240
Speaker 2: And I must be clear the reason I'm repeating myself

78
00:03:51,480 --> 00:03:54,800
Speaker 2: on so many levels here that it's it's just really

79
00:03:54,880 --> 00:03:56,960
Speaker 2: important for you to know how obvious the problems of

80
00:03:57,000 --> 00:04:00,480
Speaker 2: generative AI have been since the beginning. It's really important.

81
00:04:00,880 --> 00:04:03,920
Speaker 2: Let's go over how they work real quick. A transformer

82
00:04:03,960 --> 00:04:06,520
Speaker 2: based generative AI models such as GPT, which is the

83
00:04:06,600 --> 00:04:11,040
Speaker 2: technology behind chat. GPT generates answers using inference, which means

84
00:04:11,080 --> 00:04:14,360
Speaker 2: it draws conclusions based off of its training, which requires

85
00:04:14,400 --> 00:04:18,000
Speaker 2: feeding it masses of training data, mostly text and images

86
00:04:18,040 --> 00:04:21,600
Speaker 2: straight from the Internet. And both of these processes require

87
00:04:21,640 --> 00:04:24,880
Speaker 2: you to use high end GPUs graphics processing units, and

88
00:04:25,040 --> 00:04:28,640
Speaker 2: lots of them, tens hundreds of thousands of them, well

89
00:04:28,680 --> 00:04:32,039
Speaker 2: over one hundred thousand. I'll get to that next episode. Now,

90
00:04:32,040 --> 00:04:34,560
Speaker 2: the theory was and might still be that the more

91
00:04:34,600 --> 00:04:36,960
Speaker 2: training data and compute you throw out these models, the

92
00:04:36,960 --> 00:04:39,800
Speaker 2: better they get. And I've hypothesized for a while that

93
00:04:39,960 --> 00:04:42,320
Speaker 2: we'd have diminishing returns both and running out of training

94
00:04:42,400 --> 00:04:45,840
Speaker 2: data and based on the limitations of transformer based models.

95
00:04:46,240 --> 00:04:48,520
Speaker 3: And wouldn't you know it, I was bloody right.

96
00:04:48,640 --> 00:04:51,440
Speaker 2: I'm not going to do many of these, but this one, really,

97
00:04:51,839 --> 00:04:54,840
Speaker 2: this one I'm right on. A few weeks ago, Bloomberg

98
00:04:54,920 --> 00:04:57,600
Speaker 2: reported the open AI. Google and Anthropic are struggling to

99
00:04:57,680 --> 00:05:00,000
Speaker 2: build more advanced AI and the open ais A right

100
00:05:00,240 --> 00:05:03,000
Speaker 2: model otherwise known as GPT five did not hit the

101
00:05:03,040 --> 00:05:06,640
Speaker 2: company's desired performance. And that and I quote again, Orion

102
00:05:06,720 --> 00:05:08,880
Speaker 2: is so far not considered to be a bigger step

103
00:05:09,000 --> 00:05:10,919
Speaker 2: up as it was from GPT three point five. To

104
00:05:10,960 --> 00:05:14,760
Speaker 2: GPT four its current model. You will be shocked to

105
00:05:14,800 --> 00:05:18,040
Speaker 2: hear that the reason is that it's become increasingly difficult

106
00:05:18,040 --> 00:05:20,760
Speaker 2: to find new, untapped sources of high quality human made

107
00:05:20,760 --> 00:05:23,280
Speaker 2: training data that can be used to build more advanced

108
00:05:23,320 --> 00:05:26,320
Speaker 2: AI systems, something that I said in March. I said

109
00:05:26,320 --> 00:05:28,320
Speaker 2: it would happen in March and pissed off that people

110
00:05:28,320 --> 00:05:32,080
Speaker 2: said I was a pessimist. Well, who's a pessimist now me?

111
00:05:32,200 --> 00:05:34,840
Speaker 2: I guess I don't know. But they also added one

112
00:05:34,880 --> 00:05:37,400
Speaker 2: other thing, which is that they believe and I quote,

113
00:05:37,480 --> 00:05:39,800
Speaker 2: that the AGI bubble is bursting a little bit, which

114
00:05:39,800 --> 00:05:43,240
Speaker 2: is something I said in July. AGI isn't coming out

115
00:05:43,240 --> 00:05:46,039
Speaker 2: of this shit. Let's just be honest. And I also

116
00:05:46,120 --> 00:05:49,000
Speaker 2: want to stop and stare really hard at one particular point,

117
00:05:49,040 --> 00:05:52,320
Speaker 2: and I quote again from Bloomberg. These issues challenged the

118
00:05:52,320 --> 00:05:55,080
Speaker 2: gospel that's taken hold in Silicon Valley in recent years,

119
00:05:55,160 --> 00:05:58,960
Speaker 2: particularly since OpenAI released chat GPT two years ago. Much

120
00:05:59,000 --> 00:06:01,280
Speaker 2: of the tech industry is bet on so called scaling

121
00:06:01,360 --> 00:06:04,200
Speaker 2: laws that say more computing power, data, and larger models

122
00:06:04,200 --> 00:06:06,760
Speaker 2: will inevitably pave the way the greater leaps forward in

123
00:06:06,800 --> 00:06:10,520
Speaker 2: the power of AI. The only people taking this as gospel.

124
00:06:10,520 --> 00:06:12,279
Speaker 2: Have been members of the media unwilling to ask the

125
00:06:12,320 --> 00:06:14,520
Speaker 2: tough questions, and AI founders that don't know what the

126
00:06:14,520 --> 00:06:16,960
Speaker 2: fuck they're talking about, or they intend to mislead you.

127
00:06:17,480 --> 00:06:20,279
Speaker 2: Generative AI's products have effectively been trapped in amber for

128
00:06:20,320 --> 00:06:22,600
Speaker 2: over a year. It's been blatantly obvious if you fucking

129
00:06:22,720 --> 00:06:24,680
Speaker 2: use them and pissed off, I shouldn't swear so much.

130
00:06:26,400 --> 00:06:29,400
Speaker 2: There have been no meaningful, industry defining products out of

131
00:06:29,400 --> 00:06:32,279
Speaker 2: this because, and I quote Darreness and Mooglu, the economist

132
00:06:32,360 --> 00:06:35,839
Speaker 2: that MIT back in May, more powerful models do not

133
00:06:35,960 --> 00:06:39,680
Speaker 2: unlock new features or really change the experience, Nor what

134
00:06:39,720 --> 00:06:42,680
Speaker 2: you can build with transform based models is really a

135
00:06:42,720 --> 00:06:46,640
Speaker 2: worthwhile product. Or put another way, a slightly better white

136
00:06:46,640 --> 00:06:50,919
Speaker 2: elephant is still a white elephant. Despite the billions of

137
00:06:50,920 --> 00:06:53,640
Speaker 2: dollars burned and thousands of glossy headlines, it's difficult to

138
00:06:53,640 --> 00:06:57,640
Speaker 2: point to any truly important generative AI product, even Apple Intelligence,

139
00:06:57,640 --> 00:06:59,760
Speaker 2: the only thing that Apple really had to add to

140
00:06:59,800 --> 00:07:05,120
Speaker 2: the latest iPhone. It sucks, it's not useful. I can

141
00:07:05,200 --> 00:07:08,200
Speaker 2: make a special emoji. Now I now get summaries of

142
00:07:08,200 --> 00:07:11,560
Speaker 2: my texts that are completely or vaguely incorrect or just

143
00:07:11,840 --> 00:07:15,440
Speaker 2: summarize a giant, meaningful paragraph into a blob of a sentence.

144
00:07:15,640 --> 00:07:20,240
Speaker 2: It's so stupid. And just as a side question, what

145
00:07:20,280 --> 00:07:22,120
Speaker 2: the hell is Apple going to put in the next iPhone?

146
00:07:22,200 --> 00:07:23,960
Speaker 2: I buy one of these every year. I'm a little

147
00:07:23,960 --> 00:07:27,480
Speaker 2: big oincoin CoInc but still I don't even know why

148
00:07:27,520 --> 00:07:30,160
Speaker 2: ad upgrade again. The camera is already about as good

149
00:07:30,160 --> 00:07:30,920
Speaker 2: as it's going to get.

150
00:07:32,160 --> 00:07:32,560
Speaker 3: Anyway.

151
00:07:32,960 --> 00:07:35,800
Speaker 2: There are people that use chat GPT two hundred million

152
00:07:35,920 --> 00:07:38,520
Speaker 2: for them a week, allegedly losing the company money with

153
00:07:38,560 --> 00:07:40,680
Speaker 2: every prompt, by the way, but there's little to suggest

154
00:07:40,720 --> 00:07:45,000
Speaker 2: that there's widespread adoption of actual generative AI software. The

155
00:07:45,000 --> 00:07:47,920
Speaker 2: Information reported in September that between zero point one percent

156
00:07:47,960 --> 00:07:50,000
Speaker 2: and one percent of the four hundred and forty million

157
00:07:50,240 --> 00:07:52,600
Speaker 2: of Microsoft's business customers were willing to pay for its

158
00:07:52,600 --> 00:07:56,360
Speaker 2: AI powered Copilot, and in late October, Microsoft claimed that

159
00:07:56,440 --> 00:07:59,960
Speaker 2: it was on pace to make AI a ten billion

160
00:08:00,200 --> 00:08:03,680
Speaker 2: dollar a year business, which sounds really good until you

161
00:08:03,840 --> 00:08:08,400
Speaker 2: think about it for roughly ten seconds. First of all,

162
00:08:08,600 --> 00:08:12,000
Speaker 2: Microsoft does not have an AI business unit, which means

163
00:08:12,040 --> 00:08:14,480
Speaker 2: that this annual ten billion dollars or two and a

164
00:08:14,480 --> 00:08:17,440
Speaker 2: half billion a quarter revenue figure is split across providing

165
00:08:17,560 --> 00:08:21,119
Speaker 2: cloud compute services, and Azure selling Copilot. The dumb people

166
00:08:21,120 --> 00:08:24,680
Speaker 2: with Microsoft three sixty five subscriptions selling git Hub, Copilot,

167
00:08:24,800 --> 00:08:27,920
Speaker 2: and basically anything else with AI on it. Microsoft is

168
00:08:28,000 --> 00:08:31,040
Speaker 2: cherry picking a number based on nonspecific criteria and claiming

169
00:08:31,080 --> 00:08:33,640
Speaker 2: it's a big deal when it's actually pretty pathetic, considering

170
00:08:33,679 --> 00:08:37,080
Speaker 2: that Microsoft's capital expenditures will likely hit over sixty billion

171
00:08:37,120 --> 00:08:39,720
Speaker 2: dollars in twenty twenty four with no sign they're going

172
00:08:39,760 --> 00:08:44,560
Speaker 2: to slow down. No, that's sticky word. Revenue not profit.

173
00:08:44,720 --> 00:08:48,400
Speaker 2: Those are two very different things. How much is Microsoft

174
00:08:48,440 --> 00:08:51,520
Speaker 2: spending to make ten billion dollars a year? Open ai

175
00:08:51,640 --> 00:08:53,880
Speaker 2: currently spends two dollars and thirty five cents to make

176
00:08:53,920 --> 00:08:56,559
Speaker 2: a dollar, and Microsoft CFO Amyhood said that open ai

177
00:08:56,600 --> 00:08:59,120
Speaker 2: would cut into Microsoft profits in their last earning score,

178
00:08:59,440 --> 00:09:02,600
Speaker 2: losing it a remarkable one point five billion dollars, mainly

179
00:09:02,640 --> 00:09:04,840
Speaker 2: because of the expected loss from a company that has

180
00:09:04,920 --> 00:09:07,959
Speaker 2: only ever lost money now a year ago. In October

181
00:09:08,000 --> 00:09:10,760
Speaker 2: twenty twenty three, The Wall Street Journal reported that Microsoft

182
00:09:10,760 --> 00:09:13,400
Speaker 2: was losing an average of twenty dollars per user per

183
00:09:13,480 --> 00:09:17,000
Speaker 2: month on GitHub Copilot, a product with over a million users.

184
00:09:17,480 --> 00:09:20,240
Speaker 2: If this is true, by the way, this suggests losses

185
00:09:20,280 --> 00:09:22,760
Speaker 2: of at least two hundred million a year. They have

186
00:09:22,880 --> 00:09:25,439
Speaker 2: one point eight million users. Allegedly, this is based on

187
00:09:25,480 --> 00:09:29,719
Speaker 2: documents have reviewed. It's not great either way. That two

188
00:09:29,840 --> 00:09:32,319
Speaker 2: hundred million dollars is a lot of money to lose.

189
00:09:32,480 --> 00:09:35,240
Speaker 2: I would personally like to make two hundred million dollars

190
00:09:35,320 --> 00:09:37,480
Speaker 2: rather than lose it. Don't ask me, though I don't

191
00:09:37,520 --> 00:09:38,280
Speaker 2: run Microsoft.

192
00:09:39,360 --> 00:09:39,560
Speaker 3: Now.

193
00:09:39,600 --> 00:09:41,920
Speaker 2: Microsoft is still yet to break out exactly how much

194
00:09:41,960 --> 00:09:44,839
Speaker 2: generative AI is increasing revenue in the specific business units

195
00:09:44,880 --> 00:09:47,520
Speaker 2: they have. Generally, if a company's doing well at something,

196
00:09:47,600 --> 00:09:50,600
Speaker 2: they take great pains to make that clear. Instead, Microsoft

197
00:09:50,679 --> 00:09:53,079
Speaker 2: chose in August to revamp its reporting structure to give

198
00:09:53,120 --> 00:09:56,200
Speaker 2: better visibility into cloud consumption revenue, which is something you

199
00:09:56,280 --> 00:09:58,839
Speaker 2: do if you say, anticipate you're going to have your

200
00:09:58,840 --> 00:10:01,240
Speaker 2: worst day of trading in year after your next earnings,

201
00:10:01,240 --> 00:10:04,240
Speaker 2: as Microsoft did in October. It's all very good, it's

202
00:10:04,280 --> 00:10:07,040
Speaker 2: all going well. Now it must be clear that every

203
00:10:07,080 --> 00:10:09,480
Speaker 2: single one of these investments and products, as I've been

204
00:10:09,559 --> 00:10:12,400
Speaker 2: hyped with the whisper, that they would get exponentially better

205
00:10:12,440 --> 00:10:15,160
Speaker 2: over time, and that eventually the two hundred billion dollars

206
00:10:15,200 --> 00:10:18,839
Speaker 2: in capital expenditures would spit out this remarkable productivity improvement,

207
00:10:19,080 --> 00:10:22,000
Speaker 2: this crazy new product that would change our lives, fascinating

208
00:10:22,000 --> 00:10:24,480
Speaker 2: new things that consumers and enterprise of buying droves and

209
00:10:24,520 --> 00:10:27,720
Speaker 2: talk about how much they loved. Instead, Big tech has

210
00:10:27,720 --> 00:10:31,360
Speaker 2: found itself peddling increasingly more expensive iterations of near identical

211
00:10:31,400 --> 00:10:34,120
Speaker 2: large language models and shitty products attached to them, a

212
00:10:34,160 --> 00:10:36,040
Speaker 2: direct result of all of them having to use the

213
00:10:36,080 --> 00:10:39,520
Speaker 2: same training data, which they're now running out of. But

214
00:10:39,600 --> 00:10:42,040
Speaker 2: if you're running out of stuff and you can't find

215
00:10:42,040 --> 00:10:45,520
Speaker 2: stuff to buy, I really recommend the following advertisement. I'm

216
00:10:45,559 --> 00:10:48,400
Speaker 2: sure it will totally gel with my beliefs. The things

217
00:10:48,440 --> 00:10:51,200
Speaker 2: I'm talking about right now won't be embarrassing at all,

218
00:11:02,000 --> 00:11:05,679
Speaker 2: and we're back now. There's another assumption that people have

219
00:11:05,920 --> 00:11:10,480
Speaker 2: about these so called scaling laws. That's been by simply

220
00:11:10,480 --> 00:11:14,319
Speaker 2: building bigger data centers with even bigger, more powerful GPUs,

221
00:11:14,440 --> 00:11:17,400
Speaker 2: the expensive power hungry graphics processing units that use to

222
00:11:17,440 --> 00:11:19,840
Speaker 2: both train and run these models, and throwing as much

223
00:11:19,880 --> 00:11:21,880
Speaker 2: training data at them as possible.

224
00:11:22,000 --> 00:11:24,000
Speaker 3: They would simply start doing new things.

225
00:11:24,000 --> 00:11:26,920
Speaker 2: They'd have new capabilities, despite their being little proof that

226
00:11:26,960 --> 00:11:31,439
Speaker 2: they would do so in any way, shape or form. Microsoft, Meta, Amazon,

227
00:11:31,480 --> 00:11:33,480
Speaker 2: and Google of all burn billions, and the assumption that

228
00:11:33,559 --> 00:11:37,320
Speaker 2: doing so would create something, you know, a thing, a

229
00:11:37,360 --> 00:11:40,800
Speaker 2: good thing, like a human level artificial general intelligence, or

230
00:11:40,840 --> 00:11:44,600
Speaker 2: a product that made more money than it cost that

231
00:11:44,679 --> 00:11:48,760
Speaker 2: people liked. It's become kind of obvious that that isn't

232
00:11:48,800 --> 00:11:52,080
Speaker 2: going to happen. As we speak, members of the media

233
00:11:52,080 --> 00:11:55,360
Speaker 2: who should know better are already desperately trying to prove

234
00:11:55,400 --> 00:11:58,040
Speaker 2: that this is not a problem the information in a

235
00:11:58,080 --> 00:12:00,640
Speaker 2: similar story to Bloomberg's attempted to put lipstick on the

236
00:12:00,679 --> 00:12:03,720
Speaker 2: pig of generative AI, framing the lack of meaningful progress

237
00:12:03,760 --> 00:12:07,440
Speaker 2: of GPT fivers fine because open ai can now combine

238
00:12:07,480 --> 00:12:10,800
Speaker 2: its GPT five model with its one reasoning model, which

239
00:12:10,800 --> 00:12:12,760
Speaker 2: is the one that can't count the number of ours

240
00:12:12,760 --> 00:12:17,280
Speaker 2: and the strawberry by the way, which will then do something,

241
00:12:16,840 --> 00:12:22,080
Speaker 2: something good. Something's gonna happen. Like Sam Altman said, it

242
00:12:22,160 --> 00:12:26,760
Speaker 2: could write a lot more very difficult code. You know, Samultman,

243
00:12:26,840 --> 00:12:29,560
Speaker 2: the career liar who intimated the GPT five may function

244
00:12:29,640 --> 00:12:33,959
Speaker 2: like a virtual brain in may like these people are liars.

245
00:12:33,960 --> 00:12:35,920
Speaker 2: They're liars, they're lying to you. They were lying then

246
00:12:35,920 --> 00:12:39,120
Speaker 2: they're lying now now I couldn't possibly leave out chief

247
00:12:39,120 --> 00:12:41,959
Speaker 2: Ellly cheerleader Casey Newton, who wrote on platform Or a

248
00:12:42,000 --> 00:12:44,600
Speaker 2: few weeks ago that diminishing returns in training models may

249
00:12:44,640 --> 00:12:46,680
Speaker 2: not matter as much as you would guess, with his

250
00:12:46,720 --> 00:12:49,520
Speaker 2: evidence being the anthropic, who he also claims has not

251
00:12:49,559 --> 00:12:52,400
Speaker 2: been prone to hyperbole, do not think the scaling laws

252
00:12:52,400 --> 00:12:55,680
Speaker 2: are ending now. The original scaling law's paper, partly written

253
00:12:55,679 --> 00:12:59,720
Speaker 2: by Dario Amadesi of Anthropic important to know and to

254
00:12:59,800 --> 00:13:03,280
Speaker 2: be clear, in a fourteen thousand word ophed that Casey

255
00:13:03,320 --> 00:13:07,080
Speaker 2: Newton for no reason wrote two pieces about Anthropic CEO Dario.

256
00:13:07,400 --> 00:13:09,280
Speaker 3: He said that, and I quote AI.

257
00:13:09,080 --> 00:13:12,760
Speaker 2: Accelerated neuroscience is likely to vastly improve treatments for or

258
00:13:12,800 --> 00:13:15,520
Speaker 2: even cure most mental illness, which is the kind of

259
00:13:15,600 --> 00:13:18,000
Speaker 2: hyperbole that should be Have you tired and feathered and

260
00:13:18,040 --> 00:13:20,520
Speaker 2: put in a jail? I'm not seriously saying you put

261
00:13:20,559 --> 00:13:22,880
Speaker 2: him in jail? But why are we Why are we

262
00:13:22,920 --> 00:13:25,959
Speaker 2: trusting these people? Why are we listening to them? Why

263
00:13:26,000 --> 00:13:29,360
Speaker 2: are we treating them as if they're telling the truth

264
00:13:29,679 --> 00:13:32,800
Speaker 2: or even that they know what's going on? But let's

265
00:13:32,800 --> 00:13:36,640
Speaker 2: summarize the main technology behind the entire and I say

266
00:13:36,640 --> 00:13:39,920
Speaker 2: this in quotation marks by the way, artificial intelligence boom

267
00:13:40,080 --> 00:13:43,360
Speaker 2: is generative AI transformed based models like open AI's GPT

268
00:13:43,480 --> 00:13:46,760
Speaker 2: four and soon GPT five, and said technology has speaked

269
00:13:46,920 --> 00:13:49,440
Speaker 2: with diminishing returns from the only ways of making them better,

270
00:13:49,800 --> 00:13:52,199
Speaker 2: feeding them, training data, and throwing tons of compute at them,

271
00:13:52,360 --> 00:13:55,840
Speaker 2: suggesting that we may have, as I said before, reached PKI.

272
00:13:56,640 --> 00:13:59,800
Speaker 2: Generative AI is incredibly unprofitable. Open Ai, the biggest player

273
00:13:59,800 --> 00:14:01,719
Speaker 2: in the industry, is on course to lose more than

274
00:14:01,760 --> 00:14:05,000
Speaker 2: five billion dollars this year, with competitor Anthropic, which also

275
00:14:05,000 --> 00:14:07,760
Speaker 2: makes its own transformer based model, clawed, on course to

276
00:14:07,800 --> 00:14:10,080
Speaker 2: lose more than two point seven billion dollars this year.

277
00:14:10,480 --> 00:14:13,880
Speaker 2: They just raised another four billion. Every single big tech

278
00:14:13,880 --> 00:14:16,440
Speaker 2: company has thrown billions of dollars, as much as seventy

279
00:14:16,480 --> 00:14:19,440
Speaker 2: five billion dollars in Amazon's case in twenty twenty four alone.

280
00:14:19,520 --> 00:14:21,880
Speaker 2: Are building the data centers and acquiring the GPUs to

281
00:14:21,880 --> 00:14:24,840
Speaker 2: populate said data centers, specifically so they can train their

282
00:14:24,880 --> 00:14:27,520
Speaker 2: models and other people's models, or serve customers that would

283
00:14:27,520 --> 00:14:30,800
Speaker 2: integrate Generativai into their businesses, something that does not appear

284
00:14:30,840 --> 00:14:34,080
Speaker 2: to be happening at scale, and these investments could theoretically

285
00:14:34,120 --> 00:14:35,920
Speaker 2: be used for other products but these data cent as

286
00:14:35,960 --> 00:14:39,880
Speaker 2: a heavily focused on GENERATIVIAI. Business Insider reports that Microsoft

287
00:14:39,960 --> 00:14:42,720
Speaker 2: intends to amass one point eight million GPUs by the

288
00:14:42,800 --> 00:14:45,360
Speaker 2: end of this year, costing it tens of billions of dollars.

289
00:14:46,240 --> 00:14:49,760
Speaker 2: Worse still, many of these companies integrating GENERATIVIAI do so

290
00:14:49,800 --> 00:14:53,200
Speaker 2: by connecting to models made by either Open AI or Anthropic,

291
00:14:53,320 --> 00:14:56,240
Speaker 2: both of whom are running unprofitable businesses and likely charging

292
00:14:56,240 --> 00:14:58,640
Speaker 2: nowhere near enough to cover their costs. As I've said

293
00:14:58,640 --> 00:15:01,600
Speaker 2: before in my article the sub MAI Crisis, in the

294
00:15:01,600 --> 00:15:04,080
Speaker 2: event that these companies start charging what they actually need

295
00:15:04,120 --> 00:15:07,800
Speaker 2: to their real costs, I hypothesize that it will multiply

296
00:15:07,840 --> 00:15:09,640
Speaker 2: the cost of their customers to the point that they

297
00:15:09,640 --> 00:15:12,240
Speaker 2: can't afford to run their businesses, or at the very least,

298
00:15:12,280 --> 00:15:14,840
Speaker 2: we'll have to remove or scale back generative AI functionality

299
00:15:14,880 --> 00:15:19,840
Speaker 2: in their products. It's just it's such a waste. The

300
00:15:20,000 --> 00:15:22,880
Speaker 2: entire tech industry has become orientated around this dead end

301
00:15:22,880 --> 00:15:25,840
Speaker 2: technology that requires burning billions and billions of dollars to

302
00:15:25,840 --> 00:15:28,440
Speaker 2: provide in essential products that cost them more money to

303
00:15:28,480 --> 00:15:32,440
Speaker 2: serve than anybody ever would pay. Their big strategy has

304
00:15:32,480 --> 00:15:34,760
Speaker 2: to be to throw more money at the problem until

305
00:15:34,800 --> 00:15:38,400
Speaker 2: one of these transformer based models created something useful. Despite

306
00:15:38,400 --> 00:15:40,920
Speaker 2: the fact that every iteration of GPT in other models

307
00:15:40,920 --> 00:15:46,000
Speaker 2: has been well iterative, and it's weird, you think at

308
00:15:46,000 --> 00:15:49,520
Speaker 2: some point that goes, shit, do we actually.

309
00:15:49,240 --> 00:15:53,040
Speaker 3: Have the ability to build products with this? What are

310
00:15:53,040 --> 00:15:53,640
Speaker 3: the products?

311
00:15:53,680 --> 00:15:57,520
Speaker 2: Maybe we should work out the products first before we

312
00:15:57,600 --> 00:16:00,120
Speaker 2: throw all the capex at it. But wait, no, oh,

313
00:16:00,480 --> 00:16:04,640
Speaker 2: over yonder, I couldn't possibly not do this because the

314
00:16:04,720 --> 00:16:07,880
Speaker 2: other big tech company that also has no ideas they're

315
00:16:07,880 --> 00:16:10,000
Speaker 2: doing this. And if I don't do this, my investors

316
00:16:10,040 --> 00:16:12,000
Speaker 2: are going to be angry at me. And then what

317
00:16:12,120 --> 00:16:15,800
Speaker 2: will I do? Oh no, oh no, what could I

318
00:16:15,880 --> 00:16:19,480
Speaker 2: possibly do? If the investors I don't fucking know, that's

319
00:16:19,560 --> 00:16:25,640
Speaker 2: your problem? Why waste this much money? It's just there's

320
00:16:25,640 --> 00:16:28,680
Speaker 2: never been any proof other than these benchmarks that are

321
00:16:28,760 --> 00:16:31,720
Speaker 2: really easy to gain and also only showed just this

322
00:16:31,840 --> 00:16:35,760
Speaker 2: vague power of these models. It's been obvious that GPT

323
00:16:35,920 --> 00:16:39,560
Speaker 2: or other models wouldn't become conscious that they're not going

324
00:16:39,600 --> 00:16:41,400
Speaker 2: to do more than they do today or three months

325
00:16:41,440 --> 00:16:44,120
Speaker 2: ago or even a year ago. Hesitate to give Gary

326
00:16:44,160 --> 00:16:46,360
Speaker 2: Marcus credit, but in twenty twenty three he was saying

327
00:16:46,440 --> 00:16:49,400
Speaker 2: this if not earlier. Many people have as well, and

328
00:16:49,440 --> 00:16:53,120
Speaker 2: it's just really really, really really frustrating. Better Offline isn't

329
00:16:53,160 --> 00:16:54,960
Speaker 2: even a year old. But when we put out our

330
00:16:55,000 --> 00:16:58,480
Speaker 2: PKI episode, I got so much flak. I got so

331
00:16:58,640 --> 00:17:01,040
Speaker 2: much shit for being a hater. I didn't really understand things.

332
00:17:01,080 --> 00:17:03,040
Speaker 2: That my fly was open in my Instagram picture, that

333
00:17:03,120 --> 00:17:05,200
Speaker 2: I didn't get it, and that in mere months I

334
00:17:05,200 --> 00:17:06,240
Speaker 2: would be proven wrong.

335
00:17:06,880 --> 00:17:08,680
Speaker 3: Well here we are. How wrong am I? Now?

336
00:17:08,720 --> 00:17:11,439
Speaker 2: What happens next? Exactly where do all these hundreds of

337
00:17:11,440 --> 00:17:14,000
Speaker 2: billions of dollars go? What happens to Open AI when

338
00:17:14,040 --> 00:17:17,240
Speaker 2: it collapses? What does Microsoft do with all of these GPUs?

339
00:17:17,359 --> 00:17:19,560
Speaker 2: Because you can't just move them into other shit? You

340
00:17:19,600 --> 00:17:23,400
Speaker 2: know from what I hear, they don't really have a plan.

341
00:17:24,359 --> 00:17:27,760
Speaker 2: And that's the scariest thing, because what happens to a

342
00:17:27,800 --> 00:17:30,479
Speaker 2: stock market that's dependent on big tech companies for growth

343
00:17:30,600 --> 00:17:32,960
Speaker 2: when the big tech companies can't work out a way

344
00:17:33,000 --> 00:17:35,320
Speaker 2: to grow anymore, and in fact, their big path to

345
00:17:35,400 --> 00:17:37,200
Speaker 2: try and to grow more was to burn a shit

346
00:17:37,280 --> 00:17:39,640
Speaker 2: ton of money on things that people hate, that destroy

347
00:17:39,680 --> 00:17:45,119
Speaker 2: our environment. I know, I know, I'm angry. I know

348
00:17:45,480 --> 00:17:48,920
Speaker 2: I should calm down. I should, But as I said

349
00:17:48,960 --> 00:17:52,880
Speaker 2: in the Rot Society, this money could go elsewhere, more

350
00:17:52,920 --> 00:17:55,439
Speaker 2: things could be done. It would enter a fallow period

351
00:17:55,480 --> 00:17:57,919
Speaker 2: of tech. But we don't just have to burn all

352
00:17:57,960 --> 00:18:02,000
Speaker 2: this money. We don't have to do that. Why not

353
00:18:02,119 --> 00:18:05,760
Speaker 2: make the products you have already better? Because stapling generative

354
00:18:05,800 --> 00:18:09,399
Speaker 2: AI on them, I think it makes them worse. But

355
00:18:09,440 --> 00:18:12,920
Speaker 2: there are more problems ahead. There are problems around the infrastructure.

356
00:18:13,320 --> 00:18:15,480
Speaker 2: And in the next episode, I'm gonna break down these

357
00:18:15,480 --> 00:18:19,280
Speaker 2: worrying problems and I'm gonna kind of tell you what

358
00:18:19,359 --> 00:18:22,959
Speaker 2: happens next is best I can. I really appreciate your

359
00:18:22,960 --> 00:18:25,320
Speaker 2: faith in me. And there are many people who also

360
00:18:25,359 --> 00:18:28,240
Speaker 2: contacted me and said, no, you bang on, keep going.

361
00:18:28,480 --> 00:18:31,360
Speaker 2: I'm glad they did. I'm very grateful for your audience.

362
00:18:31,960 --> 00:18:33,920
Speaker 2: I love you all much like you said in the menu,

363
00:18:42,359 --> 00:18:44,800
Speaker 2: thank you for listening to Better Offline. The editor and

364
00:18:44,840 --> 00:18:48,000
Speaker 2: composer of the Better Offline theme song is Metasowski. You

365
00:18:48,040 --> 00:18:50,280
Speaker 2: can check out more of his music and audio projects

366
00:18:50,440 --> 00:18:53,879
Speaker 2: at Matasowski dot com, M A T T O. S

367
00:18:54,000 --> 00:18:58,199
Speaker 2: O w Ski dot com. You can email me at

368
00:18:58,240 --> 00:19:01,080
Speaker 2: easy at Better offline dot com, or visit Better Offline

369
00:19:01,119 --> 00:19:03,200
Speaker 2: dot com to find more podcast links and of course

370
00:19:03,240 --> 00:19:06,360
Speaker 2: my newsletter. I also really recommend you go to chat

371
00:19:06,359 --> 00:19:09,000
Speaker 2: dot Where's youread dot at to visit the discord, and

372
00:19:09,040 --> 00:19:11,760
Speaker 2: go to our slash Better Offline to check out our reddit.

373
00:19:12,520 --> 00:19:13,879
Speaker 2: Thank you so much for listening.

374
00:19:14,720 --> 00:19:17,439
Speaker 1: Better Offline is a production of cool Zone Media. For

375
00:19:17,520 --> 00:19:20,680
Speaker 1: more from cool Zone Media, visit our website cool Zonemedia

376
00:19:20,760 --> 00:19:23,600
Speaker 1: dot com, or check us out on the iHeartRadio app,

377
00:19:23,640 --> 00:19:26,359
Speaker 1: Apple Podcasts, or wherever you get your podcasts.