1
00:00:10,160 --> 00:00:14,480
Speaker 1: Hello, and welcome to another episode of The Odd Blocks podcast.

2
00:00:14,560 --> 00:00:16,280
Speaker 1: I'm Joe Wisenthal.

3
00:00:15,840 --> 00:00:16,880
Speaker 2: And I'm Tracy Alloway.

4
00:00:17,239 --> 00:00:21,160
Speaker 1: Tracy, I'm not sure if you've heard anyone talking about

5
00:00:21,200 --> 00:00:22,800
Speaker 1: it or anything, but have you heard about like this

6
00:00:22,920 --> 00:00:24,920
Speaker 1: sort of AI thing people have been discussing?

7
00:00:24,960 --> 00:00:27,720
Speaker 2: Oh, you know what, I discovered this really cool new

8
00:00:27,800 --> 00:00:29,240
Speaker 2: thing called chat gps.

9
00:00:29,280 --> 00:00:31,520
Speaker 1: Oh yeah, I saw that website too. Yeah.

10
00:00:31,560 --> 00:00:32,440
Speaker 2: Have you tried it?

11
00:00:32,960 --> 00:00:35,080
Speaker 1: I tried it. Yeah, I kind of like write a

12
00:00:35,080 --> 00:00:38,840
Speaker 1: poem for me. She's pretty cool technology. We should probably

13
00:00:38,880 --> 00:00:39,639
Speaker 1: learn more about it.

14
00:00:39,880 --> 00:00:42,199
Speaker 2: Yeah, I think we should know. Okay, all right, obviously

15
00:00:42,320 --> 00:00:46,920
Speaker 2: we're being facetious and joking, but everyone has been talking

16
00:00:47,159 --> 00:00:51,920
Speaker 2: about AI and these new sort of natural language interfaces

17
00:00:52,000 --> 00:00:56,440
Speaker 2: that allow you to ask questions or generate all different

18
00:00:56,480 --> 00:00:59,320
Speaker 2: types of texts and things like that. It feels like

19
00:00:59,440 --> 00:01:02,280
Speaker 2: everyone is very excited about that space.

20
00:01:02,160 --> 00:01:06,240
Speaker 1: Every almost every time. Mile Like I went out with

21
00:01:06,280 --> 00:01:08,399
Speaker 1: some friends that I hadn't seen in a long time,

22
00:01:08,480 --> 00:01:10,720
Speaker 1: Like I was at a bar last night, and like

23
00:01:10,800 --> 00:01:13,840
Speaker 1: the conversation like turned to AI within like two minutes.

24
00:01:13,880 --> 00:01:16,120
Speaker 1: Never got to talk about the experiments they did. But yes,

25
00:01:16,240 --> 00:01:18,959
Speaker 1: there is a lot. It's basically like this, like wall

26
00:01:19,040 --> 00:01:22,240
Speaker 1: of noise and everyone's been talking about actually but us,

27
00:01:22,280 --> 00:01:24,280
Speaker 1: because I don't think we have done as far as

28
00:01:24,319 --> 00:01:27,400
Speaker 1: I can recall, like an AI episode. We don't want

29
00:01:27,400 --> 00:01:30,240
Speaker 1: to just add to the noise and get another sort

30
00:01:30,280 --> 00:01:33,240
Speaker 1: of chin stroke around. But obviously there's a lot there

31
00:01:33,280 --> 00:01:33,679
Speaker 1: for us.

32
00:01:33,560 --> 00:01:36,320
Speaker 2: To discuss totally, and I'm sure this will be the

33
00:01:36,319 --> 00:01:39,720
Speaker 2: first of many episodes. But one of the ways that

34
00:01:39,760 --> 00:01:43,640
Speaker 2: it fits into sort of classic odd lots lore is

35
00:01:44,000 --> 00:01:45,360
Speaker 2: via semiconductors.

36
00:01:45,480 --> 00:01:45,640
Speaker 3: Right.

37
00:01:45,840 --> 00:01:49,480
Speaker 2: If you think about what chat GPT, for instance, is doing,

38
00:01:49,680 --> 00:01:55,000
Speaker 2: it's taking words and transforming them into numbers and then

39
00:01:55,240 --> 00:01:57,920
Speaker 2: spitting those words back out at you. And the thing

40
00:01:58,000 --> 00:02:01,520
Speaker 2: that enables it to do that semiconductors chips.

41
00:02:01,800 --> 00:02:04,560
Speaker 1: Right, So here's like the four things I think I

42
00:02:04,600 --> 00:02:08,440
Speaker 1: know about this and so this is that A. Training

43
00:02:08,480 --> 00:02:10,680
Speaker 1: the AI models so that they can do that is

44
00:02:10,680 --> 00:02:16,119
Speaker 1: a computationally intensive process. B. Each query is much more

45
00:02:16,120 --> 00:02:18,680
Speaker 1: computationally intensive than say a Google search.

46
00:02:19,400 --> 00:02:19,680
Speaker 3: Three.

47
00:02:20,360 --> 00:02:23,880
Speaker 1: The company that's absolutely crushing the space and printing money

48
00:02:24,000 --> 00:02:27,840
Speaker 1: because of this is in Nvidia. Yeah, And four there's

49
00:02:27,880 --> 00:02:31,920
Speaker 1: a general scarcity of computing powers, so that even if

50
00:02:31,960 --> 00:02:35,200
Speaker 1: you and I like were brilliant mathematicians and AI theorists,

51
00:02:35,200 --> 00:02:38,440
Speaker 1: et cetera. If we wanted to start a chat GPT competitor,

52
00:02:39,200 --> 00:02:42,400
Speaker 1: just getting access to the computing power in order to

53
00:02:42,480 --> 00:02:44,960
Speaker 1: do that would not be trivial, even if we had

54
00:02:45,000 --> 00:02:46,040
Speaker 1: tons of money outside of it.

55
00:02:46,120 --> 00:02:49,200
Speaker 2: I'm going to buy an out of business cryptomne and

56
00:02:49,240 --> 00:02:49,799
Speaker 2: take all the.

57
00:02:50,280 --> 00:02:53,280
Speaker 1: They've already been bought. Someone got that. But that's that's

58
00:02:53,360 --> 00:02:57,240
Speaker 1: basically the extent of my understanding of the nexus between

59
00:02:57,360 --> 00:03:01,320
Speaker 1: this AI and chips, and I suspect there's more to know.

60
00:03:01,400 --> 00:03:05,120
Speaker 2: They're just well. I also think having a conversation about

61
00:03:05,160 --> 00:03:09,280
Speaker 2: semiconductors and AI is a really good way to understand

62
00:03:09,480 --> 00:03:12,720
Speaker 2: the underlying technology of both those things. So that's what

63
00:03:12,760 --> 00:03:14,280
Speaker 2: I'm hoping for out of this conversation.

64
00:03:14,320 --> 00:03:16,320
Speaker 1: All right, Well, you mentioned we've been doing We've done

65
00:03:16,360 --> 00:03:18,560
Speaker 1: lots of Chips episodes in the past, so we're going

66
00:03:18,639 --> 00:03:22,040
Speaker 1: to go back to the future or something like that.

67
00:03:22,080 --> 00:03:23,960
Speaker 1: We're going to go back to our first episode, our

68
00:03:24,000 --> 00:03:27,240
Speaker 1: first guest, where we started exploring Chips episodes. I think

69
00:03:27,240 --> 00:03:29,720
Speaker 1: it was the first one that we did sometime maybe

70
00:03:29,760 --> 00:03:32,760
Speaker 1: in early twenty twenty one. We're going to be speaking

71
00:03:32,800 --> 00:03:36,320
Speaker 1: with Stacey Raskin, Managing director and senior analyst of US

72
00:03:36,360 --> 00:03:40,960
Speaker 1: Semiconductors and Semiconductor Capital Equipment at Bernstein Research, someone who's

73
00:03:41,040 --> 00:03:43,280
Speaker 1: great at breaking all this stuff down has been doing

74
00:03:43,320 --> 00:03:46,280
Speaker 1: a lot of research on this question now. So Stacy,

75
00:03:46,680 --> 00:03:48,760
Speaker 1: thank you so much for coming back on odd lots.

76
00:03:49,680 --> 00:03:51,520
Speaker 3: I am so happy to be back. Thank you so

77
00:03:51,640 --> 00:03:52,560
Speaker 3: much for having me right.

78
00:03:52,560 --> 00:03:54,560
Speaker 1: So I'm going to start with just sort of like

79
00:03:54,880 --> 00:03:58,560
Speaker 1: not even a business question, but a sort of semiconductor

80
00:03:58,600 --> 00:04:03,280
Speaker 1: design question, which is this company in video Like for

81
00:04:03,440 --> 00:04:05,480
Speaker 1: years I just sort of knew them. Is like they

82
00:04:05,480 --> 00:04:08,680
Speaker 1: were the company that made graphics cards for video games,

83
00:04:08,720 --> 00:04:10,880
Speaker 1: and then for a while they got there like oh,

84
00:04:10,920 --> 00:04:13,960
Speaker 1: and they're also good for crypto mining, and they were

85
00:04:14,040 --> 00:04:16,880
Speaker 1: very popular for a while in ethereum mining when it

86
00:04:17,000 --> 00:04:20,279
Speaker 1: used roof of work. And now my understanding is everyone

87
00:04:20,320 --> 00:04:22,800
Speaker 1: wants their chips for AI purposes. And we'll get into

88
00:04:22,839 --> 00:04:25,760
Speaker 1: all that, but just to start, what is it about

89
00:04:25,839 --> 00:04:29,920
Speaker 1: the design of their chips that makes them naturally suited

90
00:04:29,960 --> 00:04:32,200
Speaker 1: for these other things? A company that started in graphics

91
00:04:32,240 --> 00:04:35,440
Speaker 1: cards that makes them naturally suited for these things like

92
00:04:35,560 --> 00:04:39,240
Speaker 1: AI in a way apparently that other chip makers, like

93
00:04:39,279 --> 00:04:43,400
Speaker 1: saying Intel, their chips do not seem to be as

94
00:04:43,720 --> 00:04:44,640
Speaker 1: used for this space.

95
00:04:46,160 --> 00:04:48,560
Speaker 3: Yeah, so let me step back.

96
00:04:48,640 --> 00:04:52,040
Speaker 1: Yeah, sure, if the question, if the question is totally

97
00:04:52,120 --> 00:04:54,320
Speaker 1: flawed in its premise, then feel free to say your

98
00:04:54,400 --> 00:04:56,320
Speaker 1: question is totally let me step back.

99
00:04:56,360 --> 00:05:00,279
Speaker 3: So sure, I'd say the idea of like using cute

100
00:05:00,360 --> 00:05:02,599
Speaker 3: and artificial intelligence has obviously been around for a long

101
00:05:02,880 --> 00:05:05,120
Speaker 3: long time, and actually the AI industry has been through

102
00:05:05,120 --> 00:05:08,240
Speaker 3: a number of what they call AI winters over the years,

103
00:05:08,279 --> 00:05:10,760
Speaker 3: where people would get really excited about this and then

104
00:05:10,760 --> 00:05:12,279
Speaker 3: they would do work, and then it would just turn

105
00:05:12,320 --> 00:05:15,640
Speaker 3: out it wasn't working, and pretty much it was just

106
00:05:15,680 --> 00:05:19,839
Speaker 3: because the compute capacity and capabilities of the hardware at

107
00:05:19,880 --> 00:05:21,720
Speaker 3: the time doesn't really wasn't really up to the task,

108
00:05:21,760 --> 00:05:24,080
Speaker 3: and so interest would wane and you'd go through this

109
00:05:24,160 --> 00:05:27,560
Speaker 3: winter period, and a while back, I don't know, ten

110
00:05:27,720 --> 00:05:29,719
Speaker 3: fifteen years ago, whenever it was, it was sort of

111
00:05:29,760 --> 00:05:35,520
Speaker 3: discovered that the types of calculations that are used for

112
00:05:35,839 --> 00:05:38,280
Speaker 3: neural networks and machine learning, it turns out they are

113
00:05:38,440 --> 00:05:41,080
Speaker 3: very similar to the kinds of application the kinds of

114
00:05:41,200 --> 00:05:45,479
Speaker 3: mathematics that are used for graphics process processing and graphics rendering.

115
00:05:45,520 --> 00:05:48,960
Speaker 3: As it turns out it's primarily matrix multiplication and we'll

116
00:05:48,960 --> 00:05:51,000
Speaker 3: probably get into this call on this call a little

117
00:05:51,040 --> 00:05:53,960
Speaker 3: bit in terms of how these machine learning models and

118
00:05:53,960 --> 00:05:55,680
Speaker 3: everything actually work. But at the end of the day,

119
00:05:55,800 --> 00:05:59,520
Speaker 3: really it comes down to like really really large amounts

120
00:05:59,520 --> 00:06:02,840
Speaker 3: of matrix multiplication and parallel operations. And as it turned out,

121
00:06:03,600 --> 00:06:07,200
Speaker 3: the GPU, the graphics of processing unit was was quite suitable.

122
00:06:07,640 --> 00:06:10,400
Speaker 1: Before you go on then and maybe we'll get into

123
00:06:10,440 --> 00:06:13,159
Speaker 1: this an hour three of this conversation. No, we're not

124
00:06:13,160 --> 00:06:15,599
Speaker 1: going to go down on but what is matrix multiplication?

125
00:06:17,000 --> 00:06:18,599
Speaker 3: Yeah? So, I don't know how many of you are

126
00:06:18,640 --> 00:06:21,880
Speaker 3: our listeners here have had linear algebra or anything, but

127
00:06:22,120 --> 00:06:24,000
Speaker 3: a matrix is just like an array of numbers, like

128
00:06:24,120 --> 00:06:27,279
Speaker 3: thinking about like a square array of numbers, okay, okay,

129
00:06:27,320 --> 00:06:29,800
Speaker 3: and matrix multiplications. I've got two of these arrays and

130
00:06:29,839 --> 00:06:32,960
Speaker 3: I'm multiplying them together, and it's it's not as simple

131
00:06:33,000 --> 00:06:35,800
Speaker 3: as the kind of math or multiplication that maybe you're

132
00:06:35,960 --> 00:06:39,880
Speaker 3: typically used to, but it can be done. And it

133
00:06:39,960 --> 00:06:42,240
Speaker 3: turns out there are some of these characteristics of these

134
00:06:42,320 --> 00:06:44,520
Speaker 3: kinds of matrix' number of these matrix can be really big,

135
00:06:44,560 --> 00:06:46,680
Speaker 3: and there's like lots and lots of operations that need

136
00:06:46,760 --> 00:06:49,000
Speaker 3: to happen, and this stuff needs to happen like like

137
00:06:49,080 --> 00:06:52,520
Speaker 3: quite rapidly. And again I'm grossly simplifying here for the listeners,

138
00:06:53,279 --> 00:06:56,360
Speaker 3: But when when you're working through these kinds of machine

139
00:06:56,440 --> 00:06:58,960
Speaker 3: learning models, that that's really what you're doing. It's it's

140
00:06:58,960 --> 00:07:02,000
Speaker 3: a bunch of different makes, a bunch of different arrays

141
00:07:02,720 --> 00:07:06,080
Speaker 3: of numbers that contain all of the different parameters and things.

142
00:07:06,279 --> 00:07:08,120
Speaker 3: But we should probably step up a bit and talk

143
00:07:08,160 --> 00:07:11,200
Speaker 3: about what we actually mean when we talk about machine

144
00:07:11,240 --> 00:07:14,720
Speaker 3: learning and models and all kinds of things. But at

145
00:07:14,760 --> 00:07:16,440
Speaker 3: the end of the day, you have these really large

146
00:07:16,480 --> 00:07:19,560
Speaker 3: arrays of numbers that have to get multiplied together in

147
00:07:19,600 --> 00:07:21,760
Speaker 3: many cases, over and over again, many many times, and

148
00:07:21,800 --> 00:07:26,000
Speaker 3: it turns into a very very large compute problem. And

149
00:07:26,040 --> 00:07:30,000
Speaker 3: it's something that the GPU architecture can actually can do

150
00:07:30,120 --> 00:07:33,800
Speaker 3: really really efficiently, much more efficiently than you could say

151
00:07:33,840 --> 00:07:37,760
Speaker 3: on a traditional CPU. And so, as it turns out,

152
00:07:37,760 --> 00:07:40,200
Speaker 3: the GPU has become a good architecture for this. Now

153
00:07:40,200 --> 00:07:41,640
Speaker 3: when a video has done on top of this, not

154
00:07:41,640 --> 00:07:44,160
Speaker 3: only with having the hardware is they've also built a

155
00:07:44,240 --> 00:07:48,160
Speaker 3: really massive software ecosystem around all of this. They have

156
00:07:48,360 --> 00:07:51,240
Speaker 3: their software is called Kuta. Think about it as kind

157
00:07:51,280 --> 00:07:54,440
Speaker 3: of like the software of the programming and environment, like

158
00:07:54,440 --> 00:07:57,440
Speaker 3: the parallel programming environment for these gps, and they've layered

159
00:07:57,480 --> 00:08:01,120
Speaker 3: on all kinds of other libraries, stks and everything on

160
00:08:01,440 --> 00:08:05,480
Speaker 3: top of that that actually makes this relatively easy to

161
00:08:05,640 --> 00:08:07,600
Speaker 3: use and to deploy and to deliver. And so they've

162
00:08:07,640 --> 00:08:09,800
Speaker 3: built up not just the hardware bus of the software

163
00:08:09,800 --> 00:08:12,160
Speaker 3: around this, and it's given them a really really sort

164
00:08:12,160 --> 00:08:15,520
Speaker 3: of like like like massive gap versus like a lot

165
00:08:15,520 --> 00:08:17,480
Speaker 3: of the other competitors that are now trying to get

166
00:08:17,480 --> 00:08:19,960
Speaker 3: into this market as well. And so and it's FUNNYO

167
00:08:20,000 --> 00:08:22,720
Speaker 3: if you look at Nvidia as a stock I mean today,

168
00:08:22,760 --> 00:08:24,320
Speaker 3: I mean this morning, it's about a lot of a

169
00:08:24,320 --> 00:08:26,640
Speaker 3: two hundred and sixty or two hundred and seventy dollars

170
00:08:26,680 --> 00:08:29,920
Speaker 3: a share. This was a ten to twenty dollars stock forever,

171
00:08:30,000 --> 00:08:33,319
Speaker 3: and they did a four to one s stock split recently,

172
00:08:33,400 --> 00:08:35,200
Speaker 3: so that'd be more like, you know, like a two

173
00:08:35,240 --> 00:08:37,880
Speaker 3: dollars and fifty cent to five dollars stock on today's

174
00:08:37,880 --> 00:08:40,560
Speaker 3: basis for for years and years and years. And just

175
00:08:40,600 --> 00:08:44,640
Speaker 3: the magnitude of the growth that we've had with these

176
00:08:44,640 --> 00:08:47,000
Speaker 3: guys over over the last like five or ten years,

177
00:08:47,000 --> 00:08:51,040
Speaker 3: particularly around their data center business and artificial intelligence. Everything

178
00:08:51,240 --> 00:08:54,000
Speaker 3: has just been quite remarkable, and so the earnings have

179
00:08:54,040 --> 00:08:56,959
Speaker 3: gone through the roof, and clearly the multiple that you're

180
00:08:57,000 --> 00:08:59,280
Speaker 3: placing on those earnings has gone through the roof, because

181
00:08:59,440 --> 00:09:01,400
Speaker 3: you know, the the view is that the opportunity here

182
00:09:01,440 --> 00:09:02,960
Speaker 3: is massive and that we're early and there's a lot

183
00:09:02,960 --> 00:09:05,000
Speaker 3: of runway ahead of us and the stocks. I mean,

184
00:09:05,000 --> 00:09:07,000
Speaker 3: it's had it tops and downs, but in general it's

185
00:09:07,000 --> 00:09:07,640
Speaker 3: been a home run.

186
00:09:08,200 --> 00:09:10,240
Speaker 2: I definitely want to ask you about where we are

187
00:09:10,280 --> 00:09:14,800
Speaker 2: in the sort of semiconductor stock price cycle. But before

188
00:09:14,840 --> 00:09:17,560
Speaker 2: we get into that, you know, I will also bite

189
00:09:17,640 --> 00:09:21,240
Speaker 2: on the really basic question that you already alluded to,

190
00:09:21,400 --> 00:09:26,560
Speaker 2: but how does machine learning slash AI actually work. You

191
00:09:26,640 --> 00:09:29,560
Speaker 2: mentioned this idea of I guess processing a bunch of

192
00:09:29,640 --> 00:09:34,199
Speaker 2: data in parallel versus I guess old style computing where

193
00:09:34,240 --> 00:09:36,960
Speaker 2: it would be sequential. But like, talk to us about

194
00:09:37,000 --> 00:09:40,280
Speaker 2: what is actually happening here and how does it fit

195
00:09:40,480 --> 00:09:42,200
Speaker 2: into the semiconductor space.

196
00:09:43,360 --> 00:09:45,120
Speaker 3: You bet? You bet? So let me let me first

197
00:09:45,160 --> 00:09:47,679
Speaker 3: abstract this up and I'll give you a really contrived

198
00:09:47,720 --> 00:09:50,959
Speaker 3: example just sort of simplistically about what's going on, and

199
00:09:51,000 --> 00:09:52,319
Speaker 3: then we can go a little bit more into the

200
00:09:52,360 --> 00:09:55,199
Speaker 3: actual details of what's happening. But let's imagine you want

201
00:09:55,200 --> 00:09:58,079
Speaker 3: to have some kind of a neural net. But the

202
00:09:58,280 --> 00:10:01,079
Speaker 3: machine learning is typically done with something called a neural network,

203
00:10:01,480 --> 00:10:03,600
Speaker 3: and I'll talk about what that is in a moment.

204
00:10:03,600 --> 00:10:05,680
Speaker 3: And let's let's just imagine, for example, you want to

205
00:10:05,679 --> 00:10:09,720
Speaker 3: build a an artificial intelligence a neural network to recognize

206
00:10:09,760 --> 00:10:13,040
Speaker 3: pictures of casts. It's just saying, okay, let's imagine I've

207
00:10:13,080 --> 00:10:15,040
Speaker 3: got this black box sitting in front of me, and

208
00:10:15,280 --> 00:10:17,680
Speaker 3: it's got a slots on one side where I'm taking

209
00:10:17,720 --> 00:10:20,800
Speaker 3: pictures and I'm feeding them in. It's got to display

210
00:10:20,880 --> 00:10:22,800
Speaker 3: on the other side which tells me, yes, it's a

211
00:10:22,840 --> 00:10:25,360
Speaker 3: cat or no it's not. And on the side of

212
00:10:25,400 --> 00:10:30,080
Speaker 3: the box there are a billion knobs that you can turn, okay,

213
00:10:30,679 --> 00:10:34,160
Speaker 3: and and they'll change various parameters of this model that

214
00:10:34,280 --> 00:10:36,520
Speaker 3: right now are inside the black box. Don't worry about

215
00:10:36,520 --> 00:10:38,920
Speaker 3: what those parameters are, but there's there's knobs that can

216
00:10:39,000 --> 00:10:41,760
Speaker 3: change them, and so effectively what you're doing when you're

217
00:10:42,480 --> 00:10:43,880
Speaker 3: training the thing. And by the way, when you have

218
00:10:43,920 --> 00:10:45,440
Speaker 3: the artificion does what you have is you have this

219
00:10:45,480 --> 00:10:48,320
Speaker 3: big black box. You need to train it to do

220
00:10:48,400 --> 00:10:50,600
Speaker 3: a specific task. That's what I'm going to talk about

221
00:10:50,600 --> 00:10:53,760
Speaker 3: in a moment. That's called training, and then once it's trained,

222
00:10:53,800 --> 00:10:56,800
Speaker 3: you need to use it for whatever task you've traded for.

223
00:10:57,080 --> 00:10:59,280
Speaker 3: That task is called inference. So you got to do

224
00:10:59,520 --> 00:11:02,040
Speaker 3: the training inference. So the training here's where we have.

225
00:11:02,280 --> 00:11:04,160
Speaker 3: I got my box with a slot and the display

226
00:11:04,160 --> 00:11:06,920
Speaker 3: and a billion knobs. Okay, So what I do for

227
00:11:06,960 --> 00:11:09,360
Speaker 3: the training process effectively is I take a picture and

228
00:11:10,440 --> 00:11:12,400
Speaker 3: a known picture okay, so I know if it's a

229
00:11:12,440 --> 00:11:15,599
Speaker 3: catter or not. I feed it into the box and

230
00:11:15,720 --> 00:11:18,400
Speaker 3: I look at the display and it tells me yes

231
00:11:18,440 --> 00:11:20,240
Speaker 3: it's a catteror yes it's not, and it probably gets

232
00:11:20,280 --> 00:11:21,640
Speaker 3: it wrong. And so then what I do is I

233
00:11:21,679 --> 00:11:25,240
Speaker 3: turn some of the knobs and I feed another picture in,

234
00:11:26,160 --> 00:11:27,920
Speaker 3: and then I turned some of the knobs, and I'm

235
00:11:27,920 --> 00:11:31,440
Speaker 3: basically tuning all of the parameters and sort of measuring

236
00:11:31,559 --> 00:11:35,280
Speaker 3: how accurate is this network at doing this tasket recognizing

237
00:11:35,360 --> 00:11:36,679
Speaker 3: is this a picture of a cat or is it not?

238
00:11:37,400 --> 00:11:42,200
Speaker 3: And I keep feeding pictures in known pictures known data set,

239
00:11:42,679 --> 00:11:45,080
Speaker 3: and I keep playing with all the knobs until the

240
00:11:45,120 --> 00:11:47,040
Speaker 3: accuracy of the thing is wherever I want it to be.

241
00:11:47,120 --> 00:11:50,480
Speaker 3: So yes, it's decided that that now it's very good

242
00:11:50,520 --> 00:11:52,840
Speaker 3: at recognizing is this a picture of a catteror is

243
00:11:52,840 --> 00:11:55,600
Speaker 3: it not. At that point, my model, my box is trained.

244
00:11:56,240 --> 00:11:58,280
Speaker 3: I now lock all of those knobs in place, I

245
00:11:58,280 --> 00:12:00,720
Speaker 3: don't move them anymore, and I use it now I

246
00:12:00,720 --> 00:12:02,839
Speaker 3: can just feed in pictures and it'll tell me yes,

247
00:12:02,880 --> 00:12:05,360
Speaker 3: it's a category, yes it's not. And so the process

248
00:12:05,400 --> 00:12:07,920
Speaker 3: of training this model is what that's really what it's about.

249
00:12:07,920 --> 00:12:11,079
Speaker 3: It's about varying all of the parameters. And by the way,

250
00:12:11,120 --> 00:12:14,480
Speaker 3: these models can have billions or hundreds of billions or

251
00:12:14,480 --> 00:12:17,679
Speaker 3: even more of parameters that they can be changed. And

252
00:12:17,720 --> 00:12:20,920
Speaker 3: that's the process of training. You're basically trying to optimize

253
00:12:20,960 --> 00:12:24,240
Speaker 3: this this sort of situation. I'm changing the parameters a

254
00:12:24,280 --> 00:12:26,960
Speaker 3: little bit at a time such that I can optimize

255
00:12:27,000 --> 00:12:29,040
Speaker 3: the response of this thing such sus that I can

256
00:12:29,080 --> 00:12:33,280
Speaker 3: get the performance of it, the accuracy of the network

257
00:12:33,320 --> 00:12:36,040
Speaker 3: to be high. So that's the training process, and it

258
00:12:36,120 --> 00:12:39,040
Speaker 3: is very very compute intensive, because you can imagine, if

259
00:12:39,040 --> 00:12:41,480
Speaker 3: I've got a billion different knobs on turning, I'm trying

260
00:12:41,520 --> 00:12:43,640
Speaker 3: to optimize the output, that takes a lot of compute.

261
00:12:43,960 --> 00:12:47,280
Speaker 3: The inference process once all that is much less compute

262
00:12:47,280 --> 00:12:50,640
Speaker 3: intensive because I'm not changing anything. I'm just applying the

263
00:12:50,679 --> 00:12:53,559
Speaker 3: network as it is to whatever data that I'm feeding

264
00:12:53,559 --> 00:12:55,480
Speaker 3: in at that But I'm not changing anything. But I

265
00:12:55,559 --> 00:12:57,240
Speaker 3: may be doing a lot more that the difference of

266
00:12:57,320 --> 00:12:58,679
Speaker 3: the inference. I may be using it all the time,

267
00:12:58,720 --> 00:13:01,280
Speaker 3: whereas once I've trained the model trained it. So it's

268
00:13:01,280 --> 00:13:04,000
Speaker 3: more like a one and done versus like a continual

269
00:13:04,080 --> 00:13:04,679
Speaker 3: use sort of thing.

270
00:13:05,160 --> 00:13:07,160
Speaker 1: Since you talk said, we're getting into sort of the

271
00:13:07,240 --> 00:13:12,199
Speaker 1: economics of training versus inference. A is there sort of

272
00:13:12,240 --> 00:13:14,440
Speaker 1: any way to get a sense of Like let's say

273
00:13:14,679 --> 00:13:18,000
Speaker 1: Tracy and me start odd Lodge GPT. It's a competitor

274
00:13:18,080 --> 00:13:21,000
Speaker 1: to chat, a competitor to open AI, Like, what are

275
00:13:21,040 --> 00:13:23,199
Speaker 1: we thinking of in terms of just that scale? How

276
00:13:23,280 --> 00:13:27,400
Speaker 1: much we're spending to compute on the training part? Then

277
00:13:27,440 --> 00:13:30,520
Speaker 1: how much are recurring costs in terms of inference are?

278
00:13:30,920 --> 00:13:33,280
Speaker 1: And then I'm also just curious, like also, like I

279
00:13:33,640 --> 00:13:36,280
Speaker 1: know you said the inference is much cheaper, but how

280
00:13:36,360 --> 00:13:41,120
Speaker 1: much cheaper is it versus say, asking Google question? How

281
00:13:41,200 --> 00:13:43,960
Speaker 1: much more expensive is it? How much more expensive is

282
00:13:44,000 --> 00:13:47,320
Speaker 1: a Chad GPT query or an odd Lodge GPT query

283
00:13:47,520 --> 00:13:49,520
Speaker 1: versus just a normal Google search?

284
00:13:50,000 --> 00:13:52,080
Speaker 3: Yeah, now you get and by the wahen I say cheaper.

285
00:13:52,080 --> 00:13:54,800
Speaker 3: It's like for any given given single use right again,

286
00:13:54,840 --> 00:13:56,480
Speaker 3: if I've got if I'm if I've got like one

287
00:13:56,520 --> 00:13:58,719
Speaker 3: hundred billion different inference activities, maybe it's not.

288
00:13:58,880 --> 00:13:59,840
Speaker 1: It's still expensive.

289
00:14:00,360 --> 00:14:02,400
Speaker 3: Yeah, But I first want to talk about it, just

290
00:14:02,400 --> 00:14:04,160
Speaker 3: just really quickly about like so that this is my

291
00:14:04,200 --> 00:14:07,760
Speaker 3: big abstract, contrived example about what's going on. If if

292
00:14:07,800 --> 00:14:10,000
Speaker 3: I go just a little bit deeper about what what

293
00:14:10,040 --> 00:14:11,880
Speaker 3: this thing is, like, let's talk just briefly about a

294
00:14:11,920 --> 00:14:13,959
Speaker 3: neural network, and then I will get true question, but

295
00:14:14,559 --> 00:14:17,120
Speaker 3: it kind of influences it. So think what is a

296
00:14:17,160 --> 00:14:19,640
Speaker 3: neural If I was to draw like a representation of

297
00:14:19,640 --> 00:14:21,160
Speaker 3: a neural network for you, what I would do is

298
00:14:21,200 --> 00:14:24,000
Speaker 3: I have a bunch of circles. Each of the circles

299
00:14:24,000 --> 00:14:25,760
Speaker 3: would be a neuron, and I wish I was there.

300
00:14:25,760 --> 00:14:28,200
Speaker 3: I could draw a picture for you. But imagine like send.

301
00:14:27,960 --> 00:14:30,680
Speaker 1: A picture after you're done, send a picture and we'll

302
00:14:30,720 --> 00:14:31,840
Speaker 1: run it with the episode.

303
00:14:31,840 --> 00:14:34,200
Speaker 3: We'll run it with the Okay, okay, I can I

304
00:14:34,200 --> 00:14:34,480
Speaker 3: can do?

305
00:14:34,520 --> 00:14:38,760
Speaker 1: There your a hand drawn explanation of these are varies.

306
00:14:39,400 --> 00:14:42,680
Speaker 3: These are varies and fine, but anyways, but imagine like

307
00:14:42,720 --> 00:14:44,720
Speaker 3: I've got like a group of circles. I've got like

308
00:14:44,760 --> 00:14:47,720
Speaker 3: a column, you know, in column one with like three circles,

309
00:14:47,720 --> 00:14:50,160
Speaker 3: and then column two, I've got i don't know, three

310
00:14:50,200 --> 00:14:52,520
Speaker 3: or four circles, and column three, I've got some circles.

311
00:14:52,760 --> 00:14:55,160
Speaker 3: These are my neurons. And imagine I've got arrows that

312
00:14:55,200 --> 00:14:58,960
Speaker 3: are connecting each circle to the circles in one row,

313
00:14:59,000 --> 00:15:00,720
Speaker 3: to all of the circles in the next throw. Those

314
00:15:00,760 --> 00:15:03,280
Speaker 3: are my connections between my neurons. So you can see

315
00:15:03,280 --> 00:15:05,880
Speaker 3: it looks like kind of a net or a network. Okay.

316
00:15:06,520 --> 00:15:09,960
Speaker 3: And so within each circle, I've got some which what's

317
00:15:10,000 --> 00:15:12,480
Speaker 3: called activation function. So what each circle does is it

318
00:15:12,520 --> 00:15:16,120
Speaker 3: takes an input the arrow that's coming into it, and

319
00:15:16,160 --> 00:15:18,720
Speaker 3: it has to decide based on those inputs, do I

320
00:15:18,800 --> 00:15:22,520
Speaker 3: send an output out out the other side or not? Right,

321
00:15:22,840 --> 00:15:25,960
Speaker 3: So there's some certain threshold. If the inputs reach some

322
00:15:26,040 --> 00:15:28,200
Speaker 3: amount of threshold, the neuron will fire, just just like

323
00:15:28,240 --> 00:15:31,760
Speaker 3: the neuron in your brain. Okay. Each each neuron can

324
00:15:31,800 --> 00:15:33,800
Speaker 3: have more than one input coming in from from more

325
00:15:33,840 --> 00:15:36,480
Speaker 3: than one neuron in the previous These are called layers.

326
00:15:36,480 --> 00:15:38,840
Speaker 3: By the way, these rows of circles can have more

327
00:15:38,840 --> 00:15:41,360
Speaker 3: than one input from the different neurons in the previous layer,

328
00:15:41,640 --> 00:15:44,600
Speaker 3: and that the neuron can weight those those different inputs

329
00:15:44,640 --> 00:15:46,720
Speaker 3: differently good, So it can say, you know, from from

330
00:15:46,920 --> 00:15:48,600
Speaker 3: this one neuron, I'm going to give that a fifty

331
00:15:48,640 --> 00:15:50,680
Speaker 3: percent weight, and from the other neural only weight at

332
00:15:50,680 --> 00:15:52,640
Speaker 3: twenty percent. I'm not going to take the full signal.

333
00:15:53,040 --> 00:15:57,400
Speaker 3: So those are called the weights of the network. And

334
00:15:57,440 --> 00:16:01,160
Speaker 3: so each neuron has inputs coming in and outputs going out,

335
00:16:01,200 --> 00:16:02,760
Speaker 3: and each of those inputs and outputs will have a

336
00:16:02,760 --> 00:16:04,960
Speaker 3: weight associated with it. So those those are where I

337
00:16:05,000 --> 00:16:08,320
Speaker 3: talk about those knobs. Those parameters. Yeah, those weights are

338
00:16:08,400 --> 00:16:11,800
Speaker 3: are one set of parameters. And then within each neuron

339
00:16:12,000 --> 00:16:15,600
Speaker 3: there's there's basically there's a certain threshold with all those

340
00:16:15,640 --> 00:16:17,760
Speaker 3: all those signals coming in when you add them up,

341
00:16:17,760 --> 00:16:20,560
Speaker 3: if they reach a certain threshold, then the neuron fires. Okay,

342
00:16:20,720 --> 00:16:23,080
Speaker 3: So that that threshold is called the bias, and you

343
00:16:23,120 --> 00:16:25,520
Speaker 3: can tune that. Like I can have a really sensitive

344
00:16:25,560 --> 00:16:28,080
Speaker 3: neuron where if the bias doesn't I don't need a

345
00:16:28,080 --> 00:16:29,920
Speaker 3: lot of signal coming in to make it fire. I

346
00:16:29,920 --> 00:16:32,200
Speaker 3: can have a neuron that's less sensitive. I need a

347
00:16:32,200 --> 00:16:35,560
Speaker 3: lot of signal coming into portal fire. That's called a bias.

348
00:16:35,600 --> 00:16:37,520
Speaker 3: That that that's also a parameter. So those are the

349
00:16:37,560 --> 00:16:41,440
Speaker 3: parameters that you're setting. The structure of the network itself,

350
00:16:41,480 --> 00:16:43,640
Speaker 3: the number of neurons and the number of layers and

351
00:16:43,640 --> 00:16:46,640
Speaker 3: everything that's that's sort of set, and then you're trying

352
00:16:46,680 --> 00:16:50,160
Speaker 3: to determine these weights and biases and again just just

353
00:16:50,200 --> 00:16:53,160
Speaker 3: the level set you check GPT, which you haven't getting

354
00:16:53,160 --> 00:16:56,360
Speaker 3: excited about as one hundred and seventy five billion separate

355
00:16:56,400 --> 00:17:00,400
Speaker 3: parameters that they get set during their during the training press. Okay,

356
00:17:00,640 --> 00:17:02,640
Speaker 3: So that's that's kind of what's what's going on.

357
00:17:19,440 --> 00:17:21,640
Speaker 2: Before you talk about the economics. Can I just ask

358
00:17:21,800 --> 00:17:24,920
Speaker 2: so one of the things about the technology is it's

359
00:17:24,960 --> 00:17:28,360
Speaker 2: sort of it's supposed to be iterative, right, like it's

360
00:17:28,480 --> 00:17:31,440
Speaker 2: learning as it goes along. Can you talk just briefly

361
00:17:31,480 --> 00:17:36,760
Speaker 2: maybe about how it's incorporating like new inputs as it develops.

362
00:17:37,880 --> 00:17:40,639
Speaker 3: Yeah, So when when you when you training, let's talk

363
00:17:40,640 --> 00:17:43,760
Speaker 3: about training now. So when you train the network, it

364
00:17:43,880 --> 00:17:47,000
Speaker 3: happens on a static data set. Okay, so you have

365
00:17:47,080 --> 00:17:49,359
Speaker 3: to start with a data set, right, and in terms

366
00:17:49,359 --> 00:17:53,159
Speaker 3: of check GPT, that is you know, it has a

367
00:17:53,400 --> 00:17:56,000
Speaker 3: large corpus of data that it was trained on. It

368
00:17:56,040 --> 00:17:58,399
Speaker 3: was there's a lot of data from the Internet and

369
00:17:58,400 --> 00:17:59,680
Speaker 3: from other sources.

370
00:17:59,359 --> 00:18:02,439
Speaker 1: Right, basically trained the smart like all of the Internet,

371
00:18:03,200 --> 00:18:06,920
Speaker 1: but also a lot of Reddit. So it's like we've right,

372
00:18:07,080 --> 00:18:09,120
Speaker 1: like is it like we've trained just like the greatest

373
00:18:09,119 --> 00:18:11,120
Speaker 1: brain of all time is like reddit pill.

374
00:18:11,800 --> 00:18:13,880
Speaker 2: Now it talks like a seventeen year old boy.

375
00:18:14,400 --> 00:18:16,440
Speaker 3: So there's a lot of data and and so yes,

376
00:18:16,560 --> 00:18:18,639
Speaker 3: I sort of how does that data get get you know,

377
00:18:19,560 --> 00:18:22,760
Speaker 3: incorporated into I don't want to get too short of

378
00:18:22,760 --> 00:18:24,480
Speaker 3: getting too complet I don't want to get too complicated.

379
00:18:24,760 --> 00:18:26,600
Speaker 3: Let me talk about how to standard training works, and

380
00:18:26,600 --> 00:18:28,400
Speaker 3: then we can talk about chat GPT because that uses

381
00:18:28,440 --> 00:18:30,760
Speaker 3: a different kind of model. It's called a transformer model.

382
00:18:30,840 --> 00:18:33,639
Speaker 3: But anyways, but when when I'm training this, so, so

383
00:18:33,680 --> 00:18:35,800
Speaker 3: what happens is is I feed this stuff that there's

384
00:18:35,840 --> 00:18:38,600
Speaker 3: a there's a process called it's called back propagation. Basically

385
00:18:38,680 --> 00:18:42,879
Speaker 3: what you do is you sort of feed this stuff

386
00:18:42,920 --> 00:18:46,679
Speaker 3: through through this through the network itself, and then you

387
00:18:46,720 --> 00:18:48,680
Speaker 3: work it backwards and you're basically what you're doing is

388
00:18:48,720 --> 00:18:51,480
Speaker 3: you're measuring the output against a known response. I want

389
00:18:51,480 --> 00:18:54,480
Speaker 3: to sort of you know, that's my my cat picture.

390
00:18:54,560 --> 00:18:56,080
Speaker 3: Is it a cat or is it not a cat, right,

391
00:18:56,119 --> 00:18:58,160
Speaker 3: I'm trying to minimize the difference between because I want

392
00:18:58,160 --> 00:19:00,080
Speaker 3: to be accurate. Right, So what you sort of to

393
00:19:00,160 --> 00:19:03,280
Speaker 3: do is you roll a certain step through the network, right,

394
00:19:03,320 --> 00:19:06,040
Speaker 3: You measure the output against the against the known what

395
00:19:06,200 --> 00:19:08,400
Speaker 3: it should be. And then there's a process that's called

396
00:19:08,480 --> 00:19:11,200
Speaker 3: back propagation, where what you're doing you're actually what you're

397
00:19:11,200 --> 00:19:14,160
Speaker 3: calculate what's called the gradients of all of these things.

398
00:19:14,160 --> 00:19:16,119
Speaker 3: You're basically looking at sort of like the sort of

399
00:19:16,119 --> 00:19:19,720
Speaker 3: like the rate of change of of these different parameters,

400
00:19:19,720 --> 00:19:23,000
Speaker 3: and you sort of work the network backwards, and that

401
00:19:23,160 --> 00:19:25,400
Speaker 3: gradient that you're calculating kind of tells you how much

402
00:19:25,440 --> 00:19:28,560
Speaker 3: to adjust each parameter. So you work it back and

403
00:19:28,600 --> 00:19:30,280
Speaker 3: then you work it forward again, and then you work

404
00:19:30,280 --> 00:19:31,879
Speaker 3: it backward, and then you work at forward and you

405
00:19:31,920 --> 00:19:35,720
Speaker 3: work at backward, and then you do that until you've

406
00:19:35,760 --> 00:19:38,800
Speaker 3: converged like that that the that the network itself is

407
00:19:39,000 --> 00:19:41,359
Speaker 3: accurate to to wherever you want it to be to

408
00:19:41,400 --> 00:19:45,320
Speaker 3: be accurate at. That's so that's again I'm I'm I'm

409
00:19:45,359 --> 00:19:47,760
Speaker 3: grossly simplifying here. I'm trying to keep this as high

410
00:19:47,840 --> 00:19:50,720
Speaker 3: level as possible, but that's kind of what you're and

411
00:19:50,800 --> 00:19:52,320
Speaker 3: just in terms of the amount of can be sort

412
00:19:52,359 --> 00:19:55,720
Speaker 3: of train check GPT and and checking we can do.

413
00:19:55,760 --> 00:19:57,919
Speaker 3: They've actually released all the details of the network, like

414
00:19:57,960 --> 00:20:01,119
Speaker 3: how many layers and what's the dimension, I parameters all

415
00:20:01,119 --> 00:20:02,920
Speaker 3: this stuff, so we can do this math. It turns

416
00:20:02,960 --> 00:20:05,000
Speaker 3: out to take about three times ten to the twenty

417
00:20:05,040 --> 00:20:07,800
Speaker 3: third operations to train it. And so just just that's

418
00:20:07,880 --> 00:20:13,080
Speaker 3: three hundred sex tillion operations it took to train chat GPT.

419
00:20:14,080 --> 00:20:16,680
Speaker 3: Now in terms of how much it costs, so CHATTYB

420
00:20:16,880 --> 00:20:19,320
Speaker 3: was was they kind of said this, It was trained

421
00:20:19,320 --> 00:20:23,040
Speaker 3: on ten thousand in video what they called the V

422
00:20:23,119 --> 00:20:25,000
Speaker 3: one hundred. That's that's the Volta chip. That's a chip

423
00:20:25,040 --> 00:20:27,240
Speaker 3: that's several years old for in video. But it was

424
00:20:27,280 --> 00:20:29,760
Speaker 3: trained on supposedly about ten thousand of these. And we

425
00:20:29,800 --> 00:20:31,760
Speaker 3: did some of this math ourselves. I was coming out

426
00:20:31,760 --> 00:20:33,840
Speaker 3: more like three or four thousand, but there's a ton

427
00:20:33,840 --> 00:20:35,560
Speaker 3: of another assumptions you have to make it here, ten

428
00:20:35,560 --> 00:20:37,760
Speaker 3: thousand seems to be the right order of magnitude for

429
00:20:37,880 --> 00:20:41,000
Speaker 3: that part. That part of the time cost about you know,

430
00:20:41,080 --> 00:20:43,720
Speaker 3: I don't know, eight thousand bucks. And so the number

431
00:20:43,760 --> 00:20:45,280
Speaker 3: that was kind of tossed up with something like eighty

432
00:20:45,320 --> 00:20:48,600
Speaker 3: million dollars to train chat GPT one time.

433
00:20:49,160 --> 00:20:51,480
Speaker 1: I think on some of the it doesn't seem like

434
00:20:51,480 --> 00:20:54,080
Speaker 1: that much to me. Well, so this is like did

435
00:20:54,080 --> 00:20:55,119
Speaker 1: I get it, but like there are a lot of

436
00:20:55,160 --> 00:20:57,200
Speaker 1: companies that could spend that have eighty millions.

437
00:20:57,400 --> 00:20:59,679
Speaker 3: I actually agree with it. We're jumping ahead. But my

438
00:21:00,040 --> 00:21:02,440
Speaker 3: take is that for for large language models, and we

439
00:21:02,480 --> 00:21:05,439
Speaker 3: can talk about these different things, but for large language

440
00:21:05,440 --> 00:21:08,320
Speaker 3: almost chat CHIPD, I actually think inference is a bigger opportunity,

441
00:21:08,680 --> 00:21:10,119
Speaker 3: and you're kind of getting to the heart of it.

442
00:21:10,119 --> 00:21:13,200
Speaker 3: It's because inference scales directly the more queries I run.

443
00:21:14,960 --> 00:21:17,159
Speaker 1: Trained once and that's done, and that's eighty one, or

444
00:21:17,160 --> 00:21:17,600
Speaker 1: even if.

445
00:21:17,480 --> 00:21:20,520
Speaker 3: You're training more than once and again to your question, Tracy,

446
00:21:20,600 --> 00:21:22,440
Speaker 3: like you can add to the to the data set

447
00:21:22,480 --> 00:21:24,959
Speaker 3: and retrain it. But if I've already got the info,

448
00:21:25,080 --> 00:21:28,320
Speaker 3: let's say I'm training it every two weeks, Okay, yeah,

449
00:21:28,400 --> 00:21:30,440
Speaker 3: that'd be training it like twenty four to twenty five

450
00:21:30,480 --> 00:21:32,840
Speaker 3: times a year. But I've I've got the infrastructure that

451
00:21:32,920 --> 00:21:35,919
Speaker 3: is in place already right to do that, and so

452
00:21:36,440 --> 00:21:41,000
Speaker 3: the training TAM will be more around how many different

453
00:21:41,400 --> 00:21:44,680
Speaker 3: entities actually develop these models and how many models each

454
00:21:44,760 --> 00:21:47,480
Speaker 3: do they develop and how often do they train those models,

455
00:21:47,520 --> 00:21:49,280
Speaker 3: and importantly how big do the models get, Because this

456
00:21:49,359 --> 00:21:52,040
Speaker 3: is one of the things. Chat GPD is is big,

457
00:21:52,080 --> 00:21:54,920
Speaker 3: but GPT four, which they've released, that is even bigger.

458
00:21:54,920 --> 00:21:57,679
Speaker 3: They haven't they haven't talked about specs, but I wouldn't

459
00:21:57,680 --> 00:22:00,240
Speaker 3: be surprised. CHATCHIPD four is room to have over billion

460
00:22:00,280 --> 00:22:03,040
Speaker 3: parameters like a very well mighte and you have. We're

461
00:22:03,160 --> 00:22:05,199
Speaker 3: very early into this, like these these models are going

462
00:22:05,240 --> 00:22:07,080
Speaker 3: to keep getting bigger and bigger and bigger. And so

463
00:22:07,119 --> 00:22:10,200
Speaker 3: that's how I think the training market, the training tam

464
00:22:10,320 --> 00:22:12,760
Speaker 3: will be growing. It it's a function of the of

465
00:22:12,840 --> 00:22:15,199
Speaker 3: the number of trainings of all these models we're doing

466
00:22:15,240 --> 00:22:16,760
Speaker 3: every year, in the size of these models, and the

467
00:22:16,760 --> 00:22:17,520
Speaker 3: model will get big.

468
00:22:18,280 --> 00:22:20,480
Speaker 1: So let's get it. But in your view, the big

469
00:22:20,560 --> 00:22:22,439
Speaker 1: money is going to be made on the inference, So

470
00:22:22,560 --> 00:22:23,320
Speaker 1: let's talk about it.

471
00:22:23,440 --> 00:22:24,200
Speaker 3: I think.

472
00:22:24,400 --> 00:22:28,720
Speaker 1: So think that's talk about what happens then and your

473
00:22:28,760 --> 00:22:31,320
Speaker 1: sort of sense of the side. I don't know, Yeah,

474
00:22:31,359 --> 00:22:33,840
Speaker 1: just talk to us about the inference part and the economics.

475
00:22:34,200 --> 00:22:37,280
Speaker 3: You bet, Chat CHPT in these large language models, it's

476
00:22:37,320 --> 00:22:39,680
Speaker 3: a it's a new type of model's called a transformer model,

477
00:22:39,680 --> 00:22:42,919
Speaker 3: and there's a bunch of compute steps that have to happen.

478
00:22:43,600 --> 00:22:45,760
Speaker 3: There's also a step in there that helps it map

479
00:22:45,800 --> 00:22:49,320
Speaker 3: the relation, capture the relationship between you. You know, by

480
00:22:49,320 --> 00:22:51,320
Speaker 3: the way, if you if you've ever used chatcha, you know,

481
00:22:51,320 --> 00:22:54,560
Speaker 3: you type in like a querry into a box and

482
00:22:54,560 --> 00:22:57,480
Speaker 3: it and it returns to respond, so that querry is

483
00:22:57,480 --> 00:23:00,199
Speaker 3: broken into what are called tokens. It's basically thinking do

484
00:23:00,240 --> 00:23:03,080
Speaker 3: you think about token is kind of like a word

485
00:23:03,160 --> 00:23:05,760
Speaker 3: or a group of words sort of. But the transformer

486
00:23:05,800 --> 00:23:08,880
Speaker 3: model has something it's it's called a self attention mechanism,

487
00:23:09,359 --> 00:23:11,879
Speaker 3: and what that does is it captures the relationship between

488
00:23:11,880 --> 00:23:14,600
Speaker 3: those different tokens and the input sequence based on the

489
00:23:14,640 --> 00:23:16,560
Speaker 3: training data that it has. And that's how it knows

490
00:23:16,680 --> 00:23:20,320
Speaker 3: what it's really doing. It's predictive text. It knows based

491
00:23:20,359 --> 00:23:22,320
Speaker 3: on this query, I'm going to start the response with

492
00:23:22,400 --> 00:23:25,240
Speaker 3: this word, and based on this word and this query

493
00:23:25,280 --> 00:23:27,959
Speaker 3: and my data said, I know, these other words typically follow,

494
00:23:28,440 --> 00:23:31,679
Speaker 3: and it kind of constructs the response from that. And

495
00:23:31,720 --> 00:23:35,760
Speaker 3: so our math suggests that for like a typical query

496
00:23:35,840 --> 00:23:38,280
Speaker 3: response called like you know, five hundred tokens or maybe

497
00:23:38,320 --> 00:23:42,800
Speaker 3: two thousand words, it was something like four hundred quadrillion

498
00:23:43,000 --> 00:23:46,679
Speaker 3: operations needed to accomplish something like that. And so you

499
00:23:46,720 --> 00:23:49,760
Speaker 3: can size this up because I know, for like an

500
00:23:49,840 --> 00:23:52,080
Speaker 3: Nvidia GPU, and you can do it for different GPUs.

501
00:23:52,119 --> 00:23:55,520
Speaker 3: I know how many operations per second each GPU can run,

502
00:23:57,000 --> 00:23:59,879
Speaker 3: and I know how much these GPS ballpark kind of costs.

503
00:24:00,040 --> 00:24:02,080
Speaker 3: And so then you know, you got I assume like, well, okay,

504
00:24:02,080 --> 00:24:03,440
Speaker 3: how many queries per day are you going to do?

505
00:24:03,480 --> 00:24:06,200
Speaker 3: And you can come up with a number, and I mean, frankly,

506
00:24:06,200 --> 00:24:07,720
Speaker 3: the number can be as big as you want. It

507
00:24:07,760 --> 00:24:10,160
Speaker 3: depends on how many queries. But I think a tam

508
00:24:10,200 --> 00:24:12,200
Speaker 3: you know, at least in the multiple tens of billions

509
00:24:12,200 --> 00:24:16,080
Speaker 3: of dollars is not unreasonable, if not more, and just

510
00:24:16,080 --> 00:24:18,120
Speaker 3: the level set I mean, I guess to your Google question,

511
00:24:18,160 --> 00:24:20,239
Speaker 3: Google does about ten billion searches a day and give

512
00:24:20,320 --> 00:24:23,000
Speaker 3: or take. I think a lot of people have been

513
00:24:23,040 --> 00:24:25,719
Speaker 3: looking at at that level as part of like you know,

514
00:24:25,840 --> 00:24:28,000
Speaker 3: like the end all bill for where this could go.

515
00:24:28,720 --> 00:24:32,280
Speaker 3: I'll be honest, like, I understand why people are, especially

516
00:24:32,280 --> 00:24:34,760
Speaker 3: the Internet investors, are concerned that large language models and

517
00:24:34,800 --> 00:24:38,080
Speaker 3: things like chat GPD can start to disrupt search. I'm

518
00:24:38,119 --> 00:24:40,680
Speaker 3: not exactly sure that search is the right proxy person.

519
00:24:40,800 --> 00:24:42,760
Speaker 3: It feels kind of limiting to me. I mean, you

520
00:24:42,760 --> 00:24:45,720
Speaker 3: could imagine I've watched a little too much Star Trek,

521
00:24:45,760 --> 00:24:47,240
Speaker 3: I guess, but I mean you could imagine, you know,

522
00:24:47,280 --> 00:24:49,040
Speaker 3: when you have like a virtual assist in the ceiling,

523
00:24:49,080 --> 00:24:51,680
Speaker 3: I'm calling out to it, and you know, it doesn't

524
00:24:51,760 --> 00:24:54,000
Speaker 3: have to be just search on my screen. I could

525
00:24:54,080 --> 00:24:56,879
Speaker 3: have it in my car, right, I could have you know,

526
00:24:56,920 --> 00:24:59,280
Speaker 3: I call up American Airlines that change my airline tickets

527
00:24:59,320 --> 00:25:02,480
Speaker 3: and it's a checkbo that the CHET bought that's talking

528
00:25:02,520 --> 00:25:04,040
Speaker 3: to me. So this could be very big and by

529
00:25:04,040 --> 00:25:06,400
Speaker 3: the way, I think to get by the way, the

530
00:25:06,440 --> 00:25:08,439
Speaker 3: one problem with this start to a calculation that's kind

531
00:25:08,440 --> 00:25:11,160
Speaker 3: of static, Like the cost is sort of an output

532
00:25:11,240 --> 00:25:15,160
Speaker 3: rather than an input. I think to drive adoption, cost

533
00:25:15,200 --> 00:25:17,960
Speaker 3: will come down, and we've already seen that. Like Video

534
00:25:17,960 --> 00:25:20,639
Speaker 3: has a new product it's called Hopper, which is like

535
00:25:20,680 --> 00:25:23,040
Speaker 3: two generations past those V one hundreds that I was

536
00:25:23,080 --> 00:25:26,320
Speaker 3: talking about, past the Volta generation. The cost per query

537
00:25:26,400 --> 00:25:28,640
Speaker 3: to do this or the cost for training on Hopper

538
00:25:28,920 --> 00:25:31,120
Speaker 3: is much lower than a Bolta because it's much more efficient.

539
00:25:31,160 --> 00:25:34,560
Speaker 3: Part that's a good thing, though it's camacreed if it

540
00:25:34,560 --> 00:25:36,240
Speaker 3: will drive adoption, and.

541
00:25:36,320 --> 00:25:40,080
Speaker 4: Video actually has specific products specifically designed to do this

542
00:25:40,280 --> 00:25:43,720
Speaker 4: this kind of thing, and Hopper has specific blocks on

543
00:25:43,760 --> 00:25:46,080
Speaker 4: it that actually helped with with the training and inference

544
00:25:46,080 --> 00:25:47,440
Speaker 4: on these kind of large language models.

545
00:25:47,480 --> 00:25:50,200
Speaker 3: And so I actually think over time, is the efficiency

546
00:25:50,200 --> 00:25:52,439
Speaker 3: gets better and better, you're going to drive adoption more

547
00:25:52,480 --> 00:25:54,160
Speaker 3: and more. I think this is a big thing. And

548
00:25:54,200 --> 00:25:56,480
Speaker 3: I remember we're still really early. Chatchp deal only showed

549
00:25:56,520 --> 00:25:57,240
Speaker 3: up in November.

550
00:25:57,680 --> 00:25:59,720
Speaker 1: Yeah, it's crazy, it's really early.

551
00:25:59,760 --> 00:26:00,280
Speaker 3: Still.

552
00:26:00,000 --> 00:26:04,639
Speaker 2: Well, just on that note, can you draw directly the

553
00:26:04,680 --> 00:26:08,760
Speaker 2: connection between the software and the hardware you're here, because

554
00:26:08,800 --> 00:26:11,920
Speaker 2: I think it at this point probably everyone listening has

555
00:26:11,960 --> 00:26:14,960
Speaker 2: tried chat GPT, and you're used to seeing it as

556
00:26:15,000 --> 00:26:17,159
Speaker 2: a sort of you know, it's an interface on the

557
00:26:17,200 --> 00:26:19,560
Speaker 2: Internet and you type stuff into it and it spits

558
00:26:19,640 --> 00:26:24,440
Speaker 2: something out. But like, where do the semiconductors actually come

559
00:26:24,480 --> 00:26:28,600
Speaker 2: in when we're talking about crunching these enormous data sets

560
00:26:28,760 --> 00:26:31,119
Speaker 2: and what makes us You kind of touched on this

561
00:26:31,160 --> 00:26:33,480
Speaker 2: a little bit with Nvidio, but what makes a semiconductor

562
00:26:34,040 --> 00:26:38,840
Speaker 2: better at doing AI versus more traditional computational processes?

563
00:26:39,200 --> 00:26:41,359
Speaker 3: Yeah, yeah, you bet. So. To answer that second question,

564
00:26:41,480 --> 00:26:44,120
Speaker 3: I think AI is really much more around parallel processing,

565
00:26:44,160 --> 00:26:46,760
Speaker 3: and in particular thing it's this kind of MAPP matrix map.

566
00:26:48,160 --> 00:26:54,159
Speaker 3: It's a single class of calculations that these things do

567
00:26:54,400 --> 00:26:57,040
Speaker 3: very very efficiently and do very very well, and they

568
00:26:57,040 --> 00:26:58,920
Speaker 3: do them much more efficiently than a CPO that that

569
00:26:59,000 --> 00:27:02,560
Speaker 3: performs a little more really versus parallel. You just couldn't

570
00:27:02,600 --> 00:27:05,000
Speaker 3: run this stuff on CPUs. But don't get me wrong,

571
00:27:05,200 --> 00:27:07,440
Speaker 3: you do some of we've been talking about inference on

572
00:27:08,760 --> 00:27:12,000
Speaker 3: large language models. There's there's all kinds of inference. Inference

573
00:27:12,040 --> 00:27:15,600
Speaker 3: workloads range from very simplistic to very very complex like

574
00:27:15,680 --> 00:27:19,480
Speaker 3: and my my, you know, cat recognition example was very simplistic,

575
00:27:20,359 --> 00:27:23,159
Speaker 3: something like this, or fakly something like autonomous driving that

576
00:27:23,560 --> 00:27:26,360
Speaker 3: is an inference activity, but is a hugely computationally intense

577
00:27:26,920 --> 00:27:29,400
Speaker 3: inference activity. And so there's still a lot of inference

578
00:27:29,440 --> 00:27:32,000
Speaker 3: today that actually happens. In fact, most inference today actually

579
00:27:32,040 --> 00:27:35,600
Speaker 3: happens on CPUs. But i'd say the types of things

580
00:27:35,640 --> 00:27:38,040
Speaker 3: that you're trying to do are getting more and more complex,

581
00:27:38,600 --> 00:27:41,760
Speaker 3: and CPUs are getting less and less viable for that

582
00:27:41,800 --> 00:27:43,520
Speaker 3: for that kind of that kind of anth and so

583
00:27:43,920 --> 00:27:46,560
Speaker 3: that's kind of the difference between GPUs and other types

584
00:27:46,600 --> 00:27:50,440
Speaker 3: of parallel offerings versus like a CPU. I should say,

585
00:27:50,440 --> 00:27:52,000
Speaker 3: by the way, GPUs are not the only way to

586
00:27:52,000 --> 00:27:54,639
Speaker 3: do this. Google, for example, has their own an I chips.

587
00:27:54,680 --> 00:27:57,000
Speaker 3: They call them a TPU tensor processing unit.

588
00:27:57,520 --> 00:27:59,760
Speaker 1: One thing I write like about talking to Stacey to

589
00:28:00,080 --> 00:28:03,080
Speaker 1: things is a I think he comes up with better

590
00:28:03,200 --> 00:28:05,720
Speaker 1: versions of our questions than we do.

591
00:28:05,840 --> 00:28:08,760
Speaker 2: Which it's like one thing about the question is just ask.

592
00:28:09,880 --> 00:28:11,560
Speaker 1: He's always like, all right, that's a good question, but

593
00:28:11,640 --> 00:28:14,919
Speaker 1: let me actually reframe the question to get a better response.

594
00:28:14,920 --> 00:28:18,720
Speaker 1: So I appreciate that, and he also anticipates because I literally,

595
00:28:18,880 --> 00:28:21,479
Speaker 1: like on my computer right now, I had Google Cloud

596
00:28:21,520 --> 00:28:24,080
Speaker 1: tensor processing units because that was my next question. And

597
00:28:24,160 --> 00:28:27,879
Speaker 1: also important because I think yesterday the information reported that

598
00:28:27,960 --> 00:28:30,840
Speaker 1: Microsoft is also So why don't you talk to us

599
00:28:30,880 --> 00:28:34,480
Speaker 1: about that these other and what the competing directly?

600
00:28:36,240 --> 00:28:39,680
Speaker 3: Yeah, yeah, yeah, you got so Google's good. By the good,

601
00:28:39,720 --> 00:28:41,200
Speaker 3: this is not new. Google has been doing their own

602
00:28:41,240 --> 00:28:44,160
Speaker 3: chips for seven or eight years. It is not new, right,

603
00:28:44,200 --> 00:28:46,160
Speaker 3: And but they have what they call TPU and they

604
00:28:46,200 --> 00:28:50,480
Speaker 3: use it extensively for their own internal workloads. Absolutely, Amazon

605
00:28:50,560 --> 00:28:52,840
Speaker 3: has their own chips. They have a training chip. It's

606
00:28:52,960 --> 00:28:55,520
Speaker 3: that's called you know kind of hysterically. It's called tranium.

607
00:28:56,200 --> 00:29:00,000
Speaker 3: They have an inference chip. It's called Interferentia. Microsoft apparently

608
00:29:00,120 --> 00:29:03,680
Speaker 3: is working on their own. My feeling is every hyperscaler

609
00:29:03,760 --> 00:29:06,760
Speaker 3: is working on their own chat, particularly for their own

610
00:29:06,800 --> 00:29:09,320
Speaker 3: internal workloads. And that is an area we talked about

611
00:29:09,320 --> 00:29:12,560
Speaker 3: in Vida software remote like Google doesn't need in video

612
00:29:12,680 --> 00:29:15,800
Speaker 3: software mode, they're not running Kuda. They're they're just running

613
00:29:15,840 --> 00:29:19,240
Speaker 3: tensorflock right and and doing their their thing. They don't

614
00:29:19,240 --> 00:29:23,000
Speaker 3: need Kuda anything. However, that is facing an end customer,

615
00:29:23,040 --> 00:29:25,200
Speaker 3: like an enterprise like end customer, like on a public cloud,

616
00:29:25,280 --> 00:29:28,479
Speaker 3: like like a customer going to AWS and ranting, you know,

617
00:29:28,880 --> 00:29:32,440
Speaker 3: compute power, that tends to be GPUs because customers don't

618
00:29:32,480 --> 00:29:36,720
Speaker 3: have Google's just sophistication. They really do need the software

619
00:29:36,960 --> 00:29:40,360
Speaker 3: ecosystem that's built around they use. So for example, I

620
00:29:40,360 --> 00:29:42,520
Speaker 3: can go to Google Cloud, I can actually rent a

621
00:29:42,640 --> 00:29:46,920
Speaker 3: TPU instance. It can be done. Nobody really doesn't. And

622
00:29:46,960 --> 00:29:49,400
Speaker 3: actually if you look how they're priced, typically it's actually

623
00:29:49,400 --> 00:29:52,120
Speaker 3: more expensive usually even than than have the way that

624
00:29:52,120 --> 00:29:56,320
Speaker 3: Google's pricing GPUs on on on Google Cloud. It's it's

625
00:29:56,320 --> 00:29:59,360
Speaker 3: similar for Amazon and others, And so I do think

626
00:29:59,400 --> 00:30:01,200
Speaker 3: that all the hyper feelers are working on their own

627
00:30:01,280 --> 00:30:03,640
Speaker 3: and there is a certain certainly a place for that,

628
00:30:03,760 --> 00:30:06,600
Speaker 3: especially for their own internal workloads, anything that's facing a

629
00:30:06,640 --> 00:30:09,680
Speaker 3: customer that that in Video GPO ecosystem is really kind.

630
00:30:09,520 --> 00:30:12,560
Speaker 1: Of yeah, this is so, this is so Actually these

631
00:30:12,960 --> 00:30:15,200
Speaker 1: just to clarify, because that point is really interesting that

632
00:30:15,280 --> 00:30:18,680
Speaker 1: for like, if again Tracy and I want to launch

633
00:30:18,760 --> 00:30:23,000
Speaker 1: odd launch GPT, part of the issue would be not

634
00:30:23,160 --> 00:30:28,920
Speaker 1: necessarily the hardware, this sort of the silicon, but actually

635
00:30:29,480 --> 00:30:33,000
Speaker 1: that in Video's software suite built around it would make

636
00:30:33,040 --> 00:30:36,239
Speaker 1: it much easier for us to sort of start and

637
00:30:36,360 --> 00:30:38,280
Speaker 1: use on in Video for training our model.

638
00:30:38,360 --> 00:30:41,320
Speaker 3: Yeah, yes, it was, and they've built a lot of

639
00:30:41,360 --> 00:30:44,160
Speaker 3: It's funny. You can go listen to Video's announcements in

640
00:30:44,200 --> 00:30:46,000
Speaker 3: their analyst dys and things, and there as much about

641
00:30:46,000 --> 00:30:48,840
Speaker 3: software as they are about hardware. So not only have

642
00:30:48,920 --> 00:30:52,680
Speaker 3: they continue to extend like the basic like like COUDA ecosystem,

643
00:30:52,680 --> 00:30:56,040
Speaker 3: they've layered all kinds of other application specific things on

644
00:30:56,560 --> 00:30:58,400
Speaker 3: top of it. So they've got what they call RAPIDS,

645
00:30:58,400 --> 00:31:01,920
Speaker 3: which is for enterprise Machine Learn. They've got a library

646
00:31:01,920 --> 00:31:04,760
Speaker 3: package called ISAACS, which is for automation robotics, They've got

647
00:31:04,760 --> 00:31:08,080
Speaker 3: a package called Clara, which is specifically for medical imaging

648
00:31:08,120 --> 00:31:11,520
Speaker 3: and diagnostics. They've got something called cou Quantum, which is

649
00:31:11,520 --> 00:31:15,600
Speaker 3: actually for quantum computer simulations. They've got something for drug discovery.

650
00:31:15,960 --> 00:31:20,000
Speaker 3: So they're layering all these things on top, right depending

651
00:31:20,040 --> 00:31:22,520
Speaker 3: on your application. They've got internal teams that are working

652
00:31:22,520 --> 00:31:24,760
Speaker 3: on it's not just throwing the software out there. They've

653
00:31:24,800 --> 00:31:27,040
Speaker 3: got people there that can actually like help you work

654
00:31:27,080 --> 00:31:30,040
Speaker 3: or work and come along with it. They're doing other

655
00:31:30,080 --> 00:31:32,480
Speaker 3: things easier, you know. So they actually just launched a

656
00:31:32,520 --> 00:31:35,200
Speaker 3: cloud service, and this is with Google and Oracle and

657
00:31:35,200 --> 00:31:37,480
Speaker 3: Google and Microsoft ware. You can almost they'll do like

658
00:31:37,520 --> 00:31:41,680
Speaker 3: a fully provisioned in Vidia AI supercomputer in the cloud.

659
00:31:41,800 --> 00:31:43,880
Speaker 3: So because like you, they sell these AI servers and

660
00:31:43,920 --> 00:31:46,800
Speaker 3: they can cost hundreds of thousands of dollars apiece. If

661
00:31:46,840 --> 00:31:48,960
Speaker 3: you want now you can just go to Oracle Cloud

662
00:31:49,040 --> 00:31:50,840
Speaker 3: or Google Cloud or whatever. You can sort of rent

663
00:31:50,880 --> 00:31:54,840
Speaker 3: they fully provisioned in Vidia supercomputer sitting in the cloud

664
00:31:54,840 --> 00:31:56,960
Speaker 3: that they'll all you got to u is access it

665
00:31:57,000 --> 00:32:00,000
Speaker 3: right for a web browser. This kind of get super easy.

666
00:32:00,160 --> 00:32:02,000
Speaker 2: This is going to be my next question actually because

667
00:32:02,240 --> 00:32:06,040
Speaker 2: so I take the point about software, but like what

668
00:32:06,160 --> 00:32:11,120
Speaker 2: do the AI supercomputers actually look like nowadays, Like is

669
00:32:11,160 --> 00:32:14,760
Speaker 2: there a physical thing in a giant data center somewhere? Yeah,

670
00:32:14,840 --> 00:32:17,960
Speaker 2: they mostly like cloud based or what does this look like?

671
00:32:17,960 --> 00:32:21,520
Speaker 3: Like? Walk astro so video sells, and Video sells something

672
00:32:21,560 --> 00:32:24,000
Speaker 3: they called a DGX. It's a it's a box. I

673
00:32:24,040 --> 00:32:26,280
Speaker 3: mean it's I don't know it's when it's a two peat,

674
00:32:26,320 --> 00:32:27,720
Speaker 3: but I don't know what the dimensions are two peak

675
00:32:27,760 --> 00:32:30,200
Speaker 3: by two pet or something like that. It's got eight

676
00:32:30,280 --> 00:32:33,760
Speaker 3: GPUs and two CPUs and a bunch of memory and

677
00:32:33,760 --> 00:32:35,840
Speaker 3: a bunch of networking. They've got their own like you know,

678
00:32:35,840 --> 00:32:37,960
Speaker 3: they bought a company called Melanox a while back that

679
00:32:38,040 --> 00:32:41,320
Speaker 3: did networking hardware. So it's got a bunch of proprietary

680
00:32:41,360 --> 00:32:43,400
Speaker 3: network because that's but that's something else we haven't talked about.

681
00:32:43,440 --> 00:32:45,680
Speaker 3: It's not just enough to have the computer the compute.

682
00:32:46,000 --> 00:32:48,440
Speaker 3: These models are so big they don't fit on a

683
00:32:48,480 --> 00:32:50,160
Speaker 3: single c GPU. So you have to be able to

684
00:32:50,200 --> 00:32:53,400
Speaker 3: network all this stuff together, right, And so they've got

685
00:32:53,480 --> 00:32:56,080
Speaker 3: networking in there, and they have this this box, and

686
00:32:56,120 --> 00:32:57,720
Speaker 3: then you can you can stack a whole bunch of

687
00:32:57,720 --> 00:33:01,520
Speaker 3: boxes together, like and Video has their own internal supercomputer.

688
00:33:01,560 --> 00:33:03,600
Speaker 3: It's it's fairly a high on the top five hundred less.

689
00:33:03,600 --> 00:33:06,720
Speaker 3: They call it Selene. It's a bunch of these DGX

690
00:33:06,840 --> 00:33:10,080
Speaker 3: like servers that they make, all just like stacked together effectively,

691
00:33:10,520 --> 00:33:13,800
Speaker 3: and they sell for the older generation. Their prior generation

692
00:33:13,920 --> 00:33:16,000
Speaker 3: was called Ampeer and that box sold for one hundred

693
00:33:16,000 --> 00:33:18,800
Speaker 3: and ninety nine thousand dollars. I don't believe they've released

694
00:33:18,840 --> 00:33:20,960
Speaker 3: pricing on the Hopper version, but I know for the

695
00:33:20,960 --> 00:33:25,080
Speaker 3: Hopper GPU it costs two to three x what Amper

696
00:33:25,160 --> 00:33:26,880
Speaker 3: costs the prior generation.

697
00:33:27,000 --> 00:33:32,440
Speaker 1: So this really is a separate question to me, which is, Okay,

698
00:33:32,520 --> 00:33:34,880
Speaker 1: there's the price, and it exists, and you could go

699
00:33:34,920 --> 00:33:38,320
Speaker 1: to you could theoretically go and use Google's tensor based

700
00:33:38,320 --> 00:33:42,160
Speaker 1: cloud or is it available or is it because I

701
00:33:42,240 --> 00:33:44,680
Speaker 1: sort of get the impression that, like for some of

702
00:33:44,720 --> 00:33:47,920
Speaker 1: the technology that people want to use, it's not available

703
00:33:47,960 --> 00:33:50,760
Speaker 1: at any price, and that there is actually is that

704
00:33:50,800 --> 00:33:51,320
Speaker 1: real or not?

705
00:33:52,080 --> 00:33:54,440
Speaker 3: It seems to be so we're the like. So their

706
00:33:54,520 --> 00:33:57,600
Speaker 3: new generation, which is called Hopper, which like I said,

707
00:33:57,720 --> 00:34:01,400
Speaker 3: has characteristics of it maked very attractive, especially for these

708
00:34:01,440 --> 00:34:04,120
Speaker 3: kind of like chat GPT large language models, is in

709
00:34:04,200 --> 00:34:05,840
Speaker 3: tighted to play. Were at the very beginning of that

710
00:34:05,880 --> 00:34:08,399
Speaker 3: product cycle. They just launched it like in the last

711
00:34:08,560 --> 00:34:11,399
Speaker 3: couple of quarters, and so that ramp up takes time,

712
00:34:11,440 --> 00:34:15,160
Speaker 3: and it does seem like they are seeing accelerated demand

713
00:34:15,320 --> 00:34:17,120
Speaker 3: because of this kinds of stuff, and so yeah, I

714
00:34:17,160 --> 00:34:20,880
Speaker 3: think supply is tight. We've heard stories about GPU shortages

715
00:34:20,960 --> 00:34:23,719
Speaker 3: at Microsoft and the cloud vendors, and I think there

716
00:34:23,760 --> 00:34:25,279
Speaker 3: was a Bloomberg store the other day that said these

717
00:34:25,280 --> 00:34:27,400
Speaker 3: things were selling for like forty thousand dollars on eBay.

718
00:34:27,400 --> 00:34:30,040
Speaker 3: Its a thing, right, I took a look at some

719
00:34:30,040 --> 00:34:31,759
Speaker 3: of those listings. They looked a little shady to me,

720
00:34:31,920 --> 00:34:33,839
Speaker 3: But yeah, it's tight. You have to remember, these parts

721
00:34:33,880 --> 00:34:36,279
Speaker 3: are very complicated, so the lead times to actually have

722
00:34:36,360 --> 00:34:38,240
Speaker 3: more made it takes a while.

723
00:34:38,480 --> 00:34:41,680
Speaker 2: Wait, so just on this snow. I joked about this

724
00:34:41,760 --> 00:34:44,839
Speaker 2: in the intro, But you know, could I buy like

725
00:34:45,360 --> 00:34:50,360
Speaker 2: a bitcoin mining facility and take all that computer processing

726
00:34:50,440 --> 00:34:53,239
Speaker 2: power and like convert it into something that could be

727
00:34:53,320 --> 00:34:55,120
Speaker 2: used for AI. Is that a possibility?

728
00:34:55,360 --> 00:34:57,960
Speaker 3: You could? The big point stuff at least a lot

729
00:34:58,000 --> 00:35:00,520
Speaker 3: of the big point stuff was done that was with gps.

730
00:35:00,760 --> 00:35:03,960
Speaker 3: Those were still mostly gaming GPUs. People are buying gaming

731
00:35:03,960 --> 00:35:07,160
Speaker 3: gps and purposing them for a bitcoin and the theory

732
00:35:07,320 --> 00:35:10,080
Speaker 3: mostly etherory of mining. Yeah, they're they're not nearly as

733
00:35:10,080 --> 00:35:13,319
Speaker 3: compute efficient as the data center parts, right, but I

734
00:35:13,320 --> 00:35:15,520
Speaker 3: mean in theory, yeah, you could get you know, gaming

735
00:35:15,600 --> 00:35:17,759
Speaker 3: GPUs if you could and stringly get but it would

736
00:35:17,800 --> 00:35:20,440
Speaker 3: be prohibitive, right, And even now most of that stuff's

737
00:35:20,440 --> 00:35:23,160
Speaker 3: cleared out. I think as as Joe said, but the

738
00:35:23,200 --> 00:35:27,640
Speaker 3: math is somewhat similar, I'd say for these kinds of models, though, again,

739
00:35:27,760 --> 00:35:30,440
Speaker 3: like a hopper in Video's new data center product has,

740
00:35:30,520 --> 00:35:32,760
Speaker 3: they have something that they call it a transformer engine.

741
00:35:33,520 --> 00:35:35,400
Speaker 3: What it really does is it allows you to do

742
00:35:35,480 --> 00:35:38,440
Speaker 3: the training at a slightly lower precision than unless you

743
00:35:38,480 --> 00:35:41,000
Speaker 3: do it at eight bit floating point versus sixteen bit

744
00:35:41,400 --> 00:35:44,319
Speaker 3: it'll so it lets you get higher performance. And then

745
00:35:44,360 --> 00:35:47,200
Speaker 3: there's another process. There's like a conversion process. Sometimes it

746
00:35:47,280 --> 00:35:49,880
Speaker 3: has to go when you go from training to inference.

747
00:35:49,920 --> 00:35:53,040
Speaker 3: It's something of quantization, and with these transformer engines you

748
00:35:53,080 --> 00:35:55,120
Speaker 3: don't have to do that. So it increases the efficiency

749
00:35:55,480 --> 00:35:58,040
Speaker 3: which you wouldn't get by picking some random GPUs.

750
00:35:58,080 --> 00:35:59,640
Speaker 1: Where is Intel in this story?

751
00:36:00,360 --> 00:36:03,000
Speaker 3: Well, so let's let's talk about the other competitive options

752
00:36:03,000 --> 00:36:05,319
Speaker 3: that we're out there. Okay, So we talked about some

753
00:36:05,400 --> 00:36:08,920
Speaker 3: of the captive silicon and hyperscalers that is there, and

754
00:36:08,960 --> 00:36:10,680
Speaker 3: it is real, and they're all building their own and

755
00:36:10,680 --> 00:36:12,760
Speaker 3: they've been doing it forever and it hasn't slowed anything

756
00:36:12,760 --> 00:36:14,839
Speaker 3: down on the slightest because we're still early, and then

757
00:36:14,880 --> 00:36:17,080
Speaker 3: the opportunity is big. By the way, I will say,

758
00:36:17,320 --> 00:36:19,920
Speaker 3: I don't worry to lead with it. I don't worry

759
00:36:19,920 --> 00:36:23,719
Speaker 3: so much about competition at this point because think about it.

760
00:36:23,719 --> 00:36:25,719
Speaker 3: In Videa's run rating their data center business right now,

761
00:36:25,719 --> 00:36:28,080
Speaker 3: it's something like fifteen billion dollars a year. That's where

762
00:36:28,080 --> 00:36:29,920
Speaker 3: it is. It's growing, but that's where it is. So

763
00:36:30,120 --> 00:36:33,200
Speaker 3: Jensen in Video CEO likes to throw out big numbers,

764
00:36:33,200 --> 00:36:36,040
Speaker 3: and he threw out I think he said for silicon

765
00:36:36,120 --> 00:36:38,160
Speaker 3: and hardware TAM in the data center, and he thought

766
00:36:38,160 --> 00:36:41,520
Speaker 3: that their TAM overtime is three hundred billion dollars, and

767
00:36:41,600 --> 00:36:43,680
Speaker 3: it seemed kind of crazy. Although I would say, like

768
00:36:43,719 --> 00:36:46,000
Speaker 3: it's seeming a little less and less crazy every day.

769
00:36:46,680 --> 00:36:49,120
Speaker 3: But if you thought the TAM was three hundred billion

770
00:36:49,320 --> 00:36:51,960
Speaker 3: or two or one hundred billion or like whatever, and

771
00:36:52,000 --> 00:36:54,400
Speaker 3: they're run rating at fifteen billion dollars, there's tons of

772
00:36:54,440 --> 00:36:57,160
Speaker 3: headrooms competition doesn't really matter, and that's what we've seen.

773
00:36:57,200 --> 00:37:01,439
Speaker 3: We've seen competition, but there's so much opportunity like who

774
00:37:01,480 --> 00:37:03,520
Speaker 3: cares right versus like if you thought it was a

775
00:37:03,560 --> 00:37:05,880
Speaker 3: twenty billion dollar ten like they would have a problem

776
00:37:05,960 --> 00:37:08,640
Speaker 3: like already today. So that's why I don't worry too

777
00:37:08,680 --> 00:37:11,359
Speaker 3: much because I think the opportunity is still very very

778
00:37:11,440 --> 00:37:15,080
Speaker 3: large relative to where they're running into business today. In

779
00:37:15,120 --> 00:37:17,520
Speaker 3: terms of other competitors, though, sayes so you mentioned let's

780
00:37:17,520 --> 00:37:20,959
Speaker 3: talk about AMD first, because A and D actually makes GPUs,

781
00:37:21,360 --> 00:37:23,439
Speaker 3: they make data center GPUs. They don't sell very many

782
00:37:23,480 --> 00:37:25,640
Speaker 3: of them. Their current product is something called the Mi

783
00:37:25,680 --> 00:37:30,560
Speaker 3: I two fifty and they've sold deminimus basically. And in fact,

784
00:37:30,560 --> 00:37:33,400
Speaker 3: you know, when the China sanctions were put on, and

785
00:37:33,520 --> 00:37:35,040
Speaker 3: you know, we didn't talk about that, but the US

786
00:37:35,120 --> 00:37:38,480
Speaker 3: has stopped allowing like high end aichips from being shipped

787
00:37:38,480 --> 00:37:41,200
Speaker 3: to China. The MI two to fifty eighties part was

788
00:37:41,200 --> 00:37:42,480
Speaker 3: on the list, but it didn't affect them at all

789
00:37:42,480 --> 00:37:45,080
Speaker 3: because they weren't selling anything. Hey, so their sales were zero.

790
00:37:45,320 --> 00:37:47,680
Speaker 3: They've got another product coming out at the following that's

791
00:37:47,680 --> 00:37:49,560
Speaker 3: called the Mi I three hundred, and people have been

792
00:37:49,560 --> 00:37:51,279
Speaker 3: getting kind of excited about A and B. They've been

793
00:37:51,360 --> 00:37:52,640
Speaker 3: sort of looking to play it as kind of like

794
00:37:52,640 --> 00:37:55,359
Speaker 3: the Foreman's and Video. I'll be honest, I don't think

795
00:37:55,360 --> 00:37:57,480
Speaker 3: it's the Foreman's in video and video is doing, you know,

796
00:37:57,640 --> 00:38:00,600
Speaker 3: close to four billion dollars a quarter in data revenues.

797
00:38:01,040 --> 00:38:02,799
Speaker 3: I don't know that I see anything like that with

798
00:38:02,840 --> 00:38:05,160
Speaker 3: the mi I three hundred figure they in AMD as

799
00:38:05,200 --> 00:38:07,480
Speaker 3: far as i fell, has not even released any sort

800
00:38:07,520 --> 00:38:10,480
Speaker 3: of specifications for what it looks like at this point. So,

801
00:38:10,600 --> 00:38:13,160
Speaker 3: but that is an option, and some people would say

802
00:38:13,160 --> 00:38:15,520
Speaker 3: there's maybe some truth to this is you know, if

803
00:38:15,520 --> 00:38:19,120
Speaker 3: you want an alternative, AV will present an alternative. And

804
00:38:19,120 --> 00:38:20,880
Speaker 3: if the opportunity is really that they they'll get some.

805
00:38:21,000 --> 00:38:23,320
Speaker 3: They'll they'll probably get some. Do you have that? You

806
00:38:23,400 --> 00:38:27,640
Speaker 3: have Intel? So Intel's got a few things on their CPUs.

807
00:38:27,680 --> 00:38:31,680
Speaker 3: Their current version is called Sapphire Rapids. It has AI

808
00:38:31,800 --> 00:38:34,839
Speaker 3: specific accelerate, is four core inference not not so much

809
00:38:34,840 --> 00:38:38,560
Speaker 3: maybe for this kind of stuff, but for general inference activities.

810
00:38:39,080 --> 00:38:41,800
Speaker 3: They're trying to play at the capabilities of their CPU

811
00:38:42,560 --> 00:38:44,600
Speaker 3: on that fine, and why are they doing that. It's

812
00:38:44,640 --> 00:38:47,520
Speaker 3: because their accelerator roadmap isn't so good. So they have

813
00:38:47,600 --> 00:38:51,200
Speaker 3: a GPU roadmap. The code name for it was ponta Vecchio,

814
00:38:52,239 --> 00:38:54,720
Speaker 3: and they've kind of gutted that roadmap. So the follow

815
00:38:54,800 --> 00:38:57,680
Speaker 3: on product was something called rialto Bridge that they've since canceled,

816
00:38:58,560 --> 00:39:01,800
Speaker 3: and one of the Pontaventio products recently they just canceled,

817
00:39:02,680 --> 00:39:06,040
Speaker 3: and a Pajaveci originally was designed for the Area supercomputer

818
00:39:06,080 --> 00:39:09,759
Speaker 3: and it was massively late. I mean so like they

819
00:39:09,800 --> 00:39:11,560
Speaker 3: took a much was it was something like a three

820
00:39:11,640 --> 00:39:15,160
Speaker 3: hundred million dollar charge. I think it was the at

821
00:39:15,160 --> 00:39:16,759
Speaker 3: the end of twenty twenty one. It was either the

822
00:39:16,840 --> 00:39:19,040
Speaker 3: end of twenty or g twenty twenty one where they're

823
00:39:19,080 --> 00:39:21,520
Speaker 3: they basically they gave it away. It was so late,

824
00:39:21,880 --> 00:39:23,759
Speaker 3: So that's that's how late they were. They also have

825
00:39:23,840 --> 00:39:28,440
Speaker 3: another product. They bought an Israeli AI company called Habana,

826
00:39:29,239 --> 00:39:31,840
Speaker 3: and Habana has a product called Goudi. It's not a

827
00:39:31,880 --> 00:39:36,759
Speaker 3: GPU exactly, but it's like a specific accelerator technology. And

828
00:39:36,880 --> 00:39:39,040
Speaker 3: Amazon bought some of them and they sell a little bit,

829
00:39:39,040 --> 00:39:42,000
Speaker 3: but again it versus Intel's total revenues. It's the Minimus,

830
00:39:42,360 --> 00:39:44,839
Speaker 3: so they're not really there. There's also a bunch of

831
00:39:44,840 --> 00:39:48,520
Speaker 3: startups and the problem with most of the startups is

832
00:39:48,560 --> 00:39:51,080
Speaker 3: their their their story tends to be something like, you know,

833
00:39:51,080 --> 00:39:53,160
Speaker 3: we have a product that's ten times as good as Nvidia,

834
00:39:53,200 --> 00:39:56,160
Speaker 3: and the issue is with every generation, in Vidia has

835
00:39:56,160 --> 00:39:57,960
Speaker 3: something that's ten times as good as in video, and

836
00:39:58,000 --> 00:40:00,600
Speaker 3: they have the software ecosystem that goes with it. Neither

837
00:40:00,640 --> 00:40:02,760
Speaker 3: a m D, nor Intel, nor most of the startups

838
00:40:02,760 --> 00:40:05,960
Speaker 3: have anything remotely resembling in video software. So that's another

839
00:40:06,040 --> 00:40:08,479
Speaker 3: huge issue right that all of them are facing. There's

840
00:40:08,520 --> 00:40:11,520
Speaker 3: a few startups that have some niche success. One of

841
00:40:11,560 --> 00:40:14,359
Speaker 3: the one that's probably gotten the most attention is called

842
00:40:14,400 --> 00:40:18,240
Speaker 3: Servius or Cerebraus, and their whole thing. They make a chip.

843
00:40:18,400 --> 00:40:21,560
Speaker 3: It's imaginating a three hundred millimeters silicon wafer and it's

844
00:40:21,600 --> 00:40:25,000
Speaker 3: inscribing a square on it. That's their chip. It's like

845
00:40:25,040 --> 00:40:27,759
Speaker 3: one chip per wafer, and so you can put very

846
00:40:27,920 --> 00:40:30,960
Speaker 3: large models onto these chips, and they've been deploying them

847
00:40:30,960 --> 00:40:34,040
Speaker 3: for those kinds of things. But again the software becomes

848
00:40:34,200 --> 00:40:35,880
Speaker 3: an issue. But they've had a little bit of success.

849
00:40:36,400 --> 00:40:38,640
Speaker 3: There's some other names that that you know, You've got

850
00:40:38,760 --> 00:40:41,000
Speaker 3: Groc and some others I think that are still out there.

851
00:40:41,000 --> 00:40:43,560
Speaker 3: And then there's a company called Tends toward which is

852
00:40:43,640 --> 00:40:47,160
Speaker 3: interesting not because of so far what they're doing because

853
00:40:47,200 --> 00:40:49,360
Speaker 3: it's early, but it's run now by Jim Keller. And

854
00:40:49,360 --> 00:40:52,520
Speaker 3: do you guys know who Jim Keller is. Jim Keller

855
00:40:52,680 --> 00:40:55,280
Speaker 3: was was He's sort of like a star chip designer.

856
00:40:55,320 --> 00:40:59,239
Speaker 3: He designed Apple's first custom processor. He designed A and

857
00:40:59,320 --> 00:41:01,520
Speaker 3: ds as and and epic road NEPs that they've been

858
00:41:01,600 --> 00:41:02,920
Speaker 3: that they've been taking a lot of share with. He

859
00:41:03,040 --> 00:41:05,000
Speaker 3: was even at Tesla for a while and at Intel,

860
00:41:05,480 --> 00:41:07,719
Speaker 3: and so he's now running tense to it and they

861
00:41:07,760 --> 00:41:10,319
Speaker 3: do it's a risk five. Risk five is another type

862
00:41:10,320 --> 00:41:13,239
Speaker 3: of architecture, and they do they do an AI chap,

863
00:41:13,280 --> 00:41:14,120
Speaker 3: So Jim is running that.

864
00:41:14,520 --> 00:41:16,960
Speaker 2: So can I just ask based on that? I mean,

865
00:41:17,120 --> 00:41:22,439
Speaker 2: how like capex intensive is developing chips that are well

866
00:41:22,480 --> 00:41:26,600
Speaker 2: suited for AI versus other types of chips. And then secondly,

867
00:41:26,760 --> 00:41:32,040
Speaker 2: like where do the improvements come from or what are

868
00:41:32,120 --> 00:41:36,279
Speaker 2: the like improvements focused on? Is it speed or like

869
00:41:36,600 --> 00:41:40,799
Speaker 2: scale given the data sets involved in the parallel processes

870
00:41:40,800 --> 00:41:41,600
Speaker 2: that you described.

871
00:41:42,480 --> 00:41:43,960
Speaker 3: Yeah, so it's a few thing so in terms of

872
00:41:44,000 --> 00:41:46,480
Speaker 3: Capex intents, and these are mostly design companies, so they

873
00:41:46,480 --> 00:41:48,440
Speaker 3: don't have a lot of Capex. It's certainly r and

874
00:41:48,520 --> 00:41:51,640
Speaker 3: D intensive, So maybe maybe that's that's what you're getting

875
00:41:51,640 --> 00:41:53,800
Speaker 3: on in video spends like many billions of dollars a

876
00:41:53,920 --> 00:41:56,160
Speaker 3: year on R and D and and VIDA has a

877
00:41:56,160 --> 00:41:58,160
Speaker 3: little bit of advantage too because it's it's effectively the

878
00:41:58,200 --> 00:42:01,319
Speaker 3: same architecture between day center in gaming, so they've got

879
00:42:01,360 --> 00:42:04,759
Speaker 3: other other volume effectively to sort of amortize some of

880
00:42:04,760 --> 00:42:07,440
Speaker 3: those investments over although now I mean this year, I mean,

881
00:42:07,480 --> 00:42:10,120
Speaker 3: data center's probably sixty percent of in videous revenues now,

882
00:42:10,120 --> 00:42:11,879
Speaker 3: so I mean in video is sort of the center

883
00:42:11,920 --> 00:42:13,719
Speaker 3: of data center is a center of gravity for in

884
00:42:13,800 --> 00:42:16,480
Speaker 3: video now, but it's very R and D intensive and

885
00:42:16,560 --> 00:42:18,920
Speaker 3: probably getting more so. And you've got folks all up

886
00:42:18,920 --> 00:42:20,879
Speaker 3: and down the value chain that are investing or both

887
00:42:20,880 --> 00:42:23,719
Speaker 3: the silicon guys you know, and the cloud guys and

888
00:42:23,760 --> 00:42:25,839
Speaker 3: the customers and everything else. But I mean, that's that's

889
00:42:25,920 --> 00:42:28,000
Speaker 3: kind of where we are in terms of what you're

890
00:42:28,080 --> 00:42:30,960
Speaker 3: you're looking for. So there's a few things you're looking for.

891
00:42:31,080 --> 00:42:33,800
Speaker 3: Performance and on training, quite often that comes down to

892
00:42:33,920 --> 00:42:36,759
Speaker 3: like time to train. So I've got a model, Like

893
00:42:36,800 --> 00:42:38,520
Speaker 3: some of these models, I mean, you could imagine it

894
00:42:38,520 --> 00:42:43,640
Speaker 3: could take weeks or months historically to train right, and

895
00:42:43,880 --> 00:42:46,279
Speaker 3: that's a problem. You want it to be faster, so

896
00:42:46,320 --> 00:42:48,400
Speaker 3: I can get that down you know, two weeks or

897
00:42:48,440 --> 00:42:50,440
Speaker 3: you know too days or hours that would be better.

898
00:42:50,920 --> 00:42:52,640
Speaker 3: So that's one thing clearly that they work on.

899
00:42:53,040 --> 00:42:53,560
Speaker 1: I don't want to.

900
00:42:53,719 --> 00:42:56,080
Speaker 3: It's something notice, yeah, go ahead.

901
00:42:56,160 --> 00:42:58,480
Speaker 1: No finish your thought that I have a slightly oh yeah.

902
00:42:58,760 --> 00:43:00,000
Speaker 3: The other think I was talking about that there's something

903
00:43:00,040 --> 00:43:02,359
Speaker 3: where I'm like like scale out. So basically, remember I said,

904
00:43:02,360 --> 00:43:05,400
Speaker 3: you're you're connecting lots and lots of these chips together.

905
00:43:06,320 --> 00:43:08,840
Speaker 3: So for example, if if I if I increase the

906
00:43:08,920 --> 00:43:12,040
Speaker 3: number of chips by ten X, does my trading time

907
00:43:12,080 --> 00:43:13,880
Speaker 3: go back down by like a factor of ten or

908
00:43:13,920 --> 00:43:16,040
Speaker 3: is it like by factor of two? So yeah, ideally

909
00:43:16,040 --> 00:43:18,480
Speaker 3: you would want like linear scaling, right, I want, like

910
00:43:18,680 --> 00:43:20,920
Speaker 3: I add resources, it scaled linearly.

911
00:43:21,080 --> 00:43:23,080
Speaker 1: So this is kind of gonna was going to get

912
00:43:23,080 --> 00:43:25,759
Speaker 1: into my next question. Actually, and you know, we can

913
00:43:25,920 --> 00:43:29,520
Speaker 1: talk to another with someone else about certain like AI

914
00:43:29,680 --> 00:43:30,399
Speaker 1: fantasy doom.

915
00:43:30,840 --> 00:43:33,440
Speaker 3: But I think, but I'm not an AI. I'm not

916
00:43:33,480 --> 00:43:36,400
Speaker 3: an AI architecture X. But I'm a down past here.

917
00:43:36,480 --> 00:43:38,120
Speaker 3: So I could just say you may want to get aged,

918
00:43:38,200 --> 00:43:38,520
Speaker 3: no I.

919
00:43:38,480 --> 00:43:41,480
Speaker 1: Know somebody, but I am curious though, because I do

920
00:43:41,560 --> 00:43:44,080
Speaker 1: think it relates to this question, which is that okay,

921
00:43:44,200 --> 00:43:46,600
Speaker 1: like with each one like GPT five and they're going

922
00:43:46,640 --> 00:43:49,200
Speaker 1: to keep adding more knobs on the box, et cetera,

923
00:43:49,440 --> 00:43:54,520
Speaker 1: like and is your perception that this sort of quality

924
00:43:54,560 --> 00:43:58,080
Speaker 1: of the output is growing exponentially or is it the

925
00:43:58,160 --> 00:44:01,960
Speaker 1: kind of thing where it's like GPT four, you know,

926
00:44:02,080 --> 00:44:04,120
Speaker 1: there's a lot more knobs and they got a big

927
00:44:04,200 --> 00:44:08,320
Speaker 1: jump from GPT three. GPT five will be way more knobs,

928
00:44:08,520 --> 00:44:10,880
Speaker 1: but like is it going to be marginally better? Like

929
00:44:11,080 --> 00:44:12,960
Speaker 1: what is this sort of like where are we in

930
00:44:13,000 --> 00:44:14,680
Speaker 1: the sort of like what does the shape of the

931
00:44:14,719 --> 00:44:17,440
Speaker 1: output curve look like? And this sort of like cost

932
00:44:17,680 --> 00:44:21,640
Speaker 1: of you know, these chip developments of getting there. I

933
00:44:21,640 --> 00:44:23,360
Speaker 1: don't know, it's kind of so there's a couple of things.

934
00:44:23,400 --> 00:44:25,759
Speaker 3: So, so, first of all, when you're talking about large

935
00:44:25,800 --> 00:44:28,840
Speaker 3: language where it was accuracy, it's sort of a nebulous

936
00:44:28,920 --> 00:44:30,839
Speaker 3: term because it's not just accuracy. It's like like case,

937
00:44:30,840 --> 00:44:34,399
Speaker 3: it's also capability, like what could it do? What chat

938
00:44:34,440 --> 00:44:36,719
Speaker 3: GPT and GPD four can do. And also, like I

939
00:44:36,760 --> 00:44:38,360
Speaker 3: think as you're going forward and you talk about the

940
00:44:38,400 --> 00:44:42,640
Speaker 3: trajectors here, it's not just text right, we're talking text

941
00:44:42,680 --> 00:44:45,200
Speaker 3: to texture, but there's also text images and anybody like

942
00:44:45,200 --> 00:44:48,600
Speaker 3: with like Dolly where words. You know, it's generating images

943
00:44:48,719 --> 00:44:51,320
Speaker 3: from a text prompt and now we've got like video

944
00:44:52,000 --> 00:44:54,400
Speaker 3: what it was it mid was it midsummer? Is that

945
00:44:54,440 --> 00:44:57,160
Speaker 3: what it's called big journey? Journey can't mid journey? Yeah,

946
00:44:57,160 --> 00:44:59,799
Speaker 3: so it's it's it's creating like video prompts. I mean,

947
00:44:59,800 --> 00:45:03,040
Speaker 3: so like the like text is de scrapped as just

948
00:45:03,239 --> 00:45:05,360
Speaker 3: the tip of the iceberg, I think in terms of

949
00:45:05,400 --> 00:45:08,360
Speaker 3: what we're going to need, but they're.

950
00:45:08,200 --> 00:45:10,919
Speaker 1: Never they're never going to get to where they could

951
00:45:10,920 --> 00:45:14,560
Speaker 1: have three people having a conversation with voices sound like Tracy,

952
00:45:14,640 --> 00:45:17,520
Speaker 1: Joe and Stacy. Right, No, I'm just kidding, No, I mean,

953
00:45:18,440 --> 00:45:21,840
Speaker 1: I'm just kidding. It feels like, yeah, this job.

954
00:45:21,719 --> 00:45:24,759
Speaker 3: Now one of the dangerous clearly, and maybe this gets

955
00:45:24,800 --> 00:45:27,160
Speaker 3: the capabilities. So what one thing with chat GPT is

956
00:45:27,200 --> 00:45:29,879
Speaker 3: it's very very good. This why I should worry about

957
00:45:29,920 --> 00:45:32,239
Speaker 3: my job because it's very good about that. That's it

958
00:45:32,400 --> 00:45:34,439
Speaker 3: sounding like it knows what it's talking about, where maybe

959
00:45:34,480 --> 00:45:37,120
Speaker 3: it doesn't hate, So maybe I should be worried about

960
00:45:37,160 --> 00:45:39,680
Speaker 3: my job, you know, And accuracy, I think is a

961
00:45:39,680 --> 00:45:41,279
Speaker 3: big issue, but you have to remember it.

962
00:45:41,360 --> 00:45:44,640
Speaker 1: So, but like on this accuracy question, like I assume,

963
00:45:44,719 --> 00:45:47,919
Speaker 1: you know, like self driving cars, like when people were

964
00:45:47,920 --> 00:45:50,600
Speaker 1: really hyped about them ten years ago, they're like, oh,

965
00:45:50,600 --> 00:45:52,959
Speaker 1: it's ninety five percent solid, we just have a little

966
00:45:52,960 --> 00:45:56,040
Speaker 1: bit more, and then it's solid ten years later. Yeah,

967
00:45:56,160 --> 00:45:58,399
Speaker 1: ten years later, it feels like they haven't made any

968
00:45:58,440 --> 00:45:59,880
Speaker 1: progress on that final five percent.

969
00:46:00,040 --> 00:46:01,719
Speaker 3: Yeah. I mean, these things are always a power law.

970
00:46:01,840 --> 00:46:06,280
Speaker 1: So this is my question when we talk about accuracy

971
00:46:06,360 --> 00:46:08,840
Speaker 1: or these things, like are we at the point where

972
00:46:08,920 --> 00:46:10,080
Speaker 1: like is it going to be the kind of thing

973
00:46:10,080 --> 00:46:13,160
Speaker 1: where it's like, yeah, GPT five will definitely be better

974
00:46:13,160 --> 00:46:16,319
Speaker 1: than GBT four, but it will be like ninety six

975
00:46:16,360 --> 00:46:17,359
Speaker 1: percent of the way there.

976
00:46:18,000 --> 00:46:21,080
Speaker 3: Well, again, let me separate out. Let me separate an

977
00:46:21,120 --> 00:46:25,200
Speaker 3: accuracy from capability again. So there's an accuracy you have

978
00:46:25,280 --> 00:46:28,960
Speaker 3: to remember, like it the model has no idea what

979
00:46:29,120 --> 00:46:32,480
Speaker 3: accurate even means. It doesn't remember. These things are not

980
00:46:32,560 --> 00:46:34,600
Speaker 3: actually intelligent. I know there's a lot of worry about

981
00:46:34,640 --> 00:46:36,800
Speaker 3: like what they go like like like agi like artifice

982
00:46:36,840 --> 00:46:39,319
Speaker 3: with general intelligence. Right, I don't think this is it.

983
00:46:39,400 --> 00:46:42,359
Speaker 3: This is predictive text. That's all. The model doesn't know

984
00:46:42,400 --> 00:46:44,960
Speaker 3: if it's if it's viewing bull crap or truth. It

985
00:46:44,960 --> 00:46:46,880
Speaker 3: has no idea, it's just predicting the next word in

986
00:46:47,160 --> 00:46:49,560
Speaker 3: the the thing. And it's because of what it's trained on.

987
00:46:49,600 --> 00:46:52,880
Speaker 3: So you need to add on maybe other kinds of

988
00:46:52,880 --> 00:46:55,200
Speaker 3: things to ensure accuracy, maybe to put guard rails or

989
00:46:55,239 --> 00:46:57,560
Speaker 3: things things like that. You may need to very carefully,

990
00:46:57,640 --> 00:47:00,040
Speaker 3: like more harsh like your input like data sets and

991
00:47:00,120 --> 00:47:02,360
Speaker 3: things like that. I think that's a problem now. I

992
00:47:02,400 --> 00:47:05,440
Speaker 3: think it'll get solved. There's enough date. But like and

993
00:47:05,520 --> 00:47:07,759
Speaker 3: this has already been an issue and you got you

994
00:47:07,800 --> 00:47:09,319
Speaker 3: can take it like the other like the I don't

995
00:47:09,320 --> 00:47:10,520
Speaker 3: know if it's the converse of it or not, but

996
00:47:10,600 --> 00:47:12,920
Speaker 3: things like deep fakes, people are deliberately trying to use

997
00:47:13,280 --> 00:47:15,680
Speaker 3: AI to deceive. I mean, this is just human nature.

998
00:47:15,719 --> 00:47:17,719
Speaker 3: This is this is why we have problems. But I

999
00:47:17,760 --> 00:47:20,000
Speaker 3: think they can work through that just in terms of

1000
00:47:20,040 --> 00:47:23,640
Speaker 3: capabilities now, I think it's it's really interesting to look

1001
00:47:23,640 --> 00:47:27,440
Speaker 3: at like like sort of similar like a response like

1002
00:47:27,440 --> 00:47:30,600
Speaker 3: to a similar prompt between like chat GPT and GPT four,

1003
00:47:30,680 --> 00:47:33,279
Speaker 3: and like what people are getting out of GPD four.

1004
00:47:33,280 --> 00:47:35,239
Speaker 3: It's it's it's miles ahead of like some of the

1005
00:47:35,280 --> 00:47:37,480
Speaker 3: stuff that that that chat GPT, which was trained on

1006
00:47:37,920 --> 00:47:40,719
Speaker 3: GPT three of them, all that than what it was,

1007
00:47:40,840 --> 00:47:44,600
Speaker 3: what is delivering in terms of nuance, right, and color

1008
00:47:44,680 --> 00:47:46,839
Speaker 3: and every and everything else. I mean, and I think

1009
00:47:46,880 --> 00:47:49,480
Speaker 3: that's going to continue. I wouldn't be And already you're

1010
00:47:49,480 --> 00:47:51,000
Speaker 3: on the boat where these things can already pass the

1011
00:47:51,000 --> 00:47:53,760
Speaker 3: turning tests. Oh yeah, right, it can be very difficult

1012
00:47:53,760 --> 00:47:55,560
Speaker 3: to know if it's if I'm put in the question

1013
00:47:55,600 --> 00:47:58,320
Speaker 3: of accuracy aside perment, it's very difficult to know for

1014
00:47:58,440 --> 00:47:59,960
Speaker 3: some of these things if if you didn't know any

1015
00:48:00,040 --> 00:48:02,040
Speaker 3: better whether it was coming from a real person or not.

1016
00:48:02,200 --> 00:48:04,840
Speaker 3: And I think it's going to get like harder and

1017
00:48:04,920 --> 00:48:07,520
Speaker 3: harder to tell, like whether you know even if it's

1018
00:48:07,520 --> 00:48:10,040
Speaker 3: not you know, quote unquote really thinking it's going to

1019
00:48:10,080 --> 00:48:11,440
Speaker 3: be hard for us to tell what's really going on.

1020
00:48:11,520 --> 00:48:14,080
Speaker 3: That is sort of like other interesting you know, implications

1021
00:48:14,400 --> 00:48:17,279
Speaker 3: or for what this might be over the next five

1022
00:48:17,360 --> 00:48:33,280
Speaker 3: years or ten years.

1023
00:48:35,440 --> 00:48:37,759
Speaker 2: Just going back to the stock prices, I mean, we

1024
00:48:37,800 --> 00:48:40,680
Speaker 2: mentioned the Nvidia chart, which is up quite a lot,

1025
00:48:40,680 --> 00:48:43,799
Speaker 2: although not it hasn't reached its its peak back in

1026
00:48:43,840 --> 00:48:48,360
Speaker 2: twenty twenty one. The Socks Index is recovering, but you know,

1027
00:48:48,440 --> 00:48:51,479
Speaker 2: still below an intel. I mean, I won't even mention,

1028
00:48:51,760 --> 00:48:55,839
Speaker 2: but like, where are we in the semiconductor cycle, because

1029
00:48:55,880 --> 00:48:59,040
Speaker 2: it feels like, on the one hand there's talk about

1030
00:48:59,040 --> 00:49:01,960
Speaker 2: excess capacity and orders starting to fall, but on the

1031
00:49:02,000 --> 00:49:04,720
Speaker 2: other hand, there is this real excitement about the future

1032
00:49:04,880 --> 00:49:05,920
Speaker 2: in the form of AI.

1033
00:49:06,960 --> 00:49:11,080
Speaker 3: Yes. Yes, So seventies in general were pretty lousy last year.

1034
00:49:11,160 --> 00:49:14,120
Speaker 3: They've had a very strong year to date performance and

1035
00:49:14,160 --> 00:49:17,480
Speaker 3: the sectors up, which is sectors up, you know, twenty

1036
00:49:17,600 --> 00:49:20,120
Speaker 3: twenty two percent year to date, quite a bit above

1037
00:49:20,120 --> 00:49:23,640
Speaker 3: the overall market. And the reason is, to your point,

1038
00:49:23,800 --> 00:49:25,680
Speaker 3: we've been in a cycle. Numbers have been coming down.

1039
00:49:25,680 --> 00:49:27,120
Speaker 3: And we may have talked about this last time. I

1040
00:49:27,120 --> 00:49:30,200
Speaker 3: don't remember, but semi conutter investors, that turns out the

1041
00:49:30,200 --> 00:49:33,200
Speaker 3: best friend to buy stocks in general is after numbers

1042
00:49:33,239 --> 00:49:35,000
Speaker 3: come down, but before they hit bottoms, Like if you

1043
00:49:35,040 --> 00:49:38,200
Speaker 3: could buy them right before the last cut, if you

1044
00:49:38,239 --> 00:49:40,759
Speaker 3: could have perfect foresight. You never know when that is.

1045
00:49:40,840 --> 00:49:42,480
Speaker 3: But I mean numbers of cut. But numbers have come

1046
00:49:42,520 --> 00:49:45,240
Speaker 3: down the laws so estimates forward estimates for the industry

1047
00:49:45,280 --> 00:49:49,080
Speaker 3: peaked last June and they are down over thirty percent,

1048
00:49:49,160 --> 00:49:51,000
Speaker 3: like thirty five percent since that when it's actually the

1049
00:49:51,280 --> 00:49:56,040
Speaker 3: largest negative earnings revision we've had probably since the financial crisis. Wow,

1050
00:49:57,120 --> 00:50:00,320
Speaker 3: and people are looking for you know, playing the ottoming

1051
00:50:00,400 --> 00:50:02,520
Speaker 3: theme and that hopefully things get better into the second half.

1052
00:50:02,560 --> 00:50:06,319
Speaker 3: You know, we get hope, hopefully China reopening, and you've

1053
00:50:06,360 --> 00:50:08,520
Speaker 3: got markets like and this relates to Intel like like

1054
00:50:08,600 --> 00:50:12,560
Speaker 3: PCs and things where you know, we've now corrected kind

1055
00:50:12,600 --> 00:50:14,399
Speaker 3: of we're back like more on a pre COVID run

1056
00:50:14,440 --> 00:50:17,520
Speaker 3: rate for PCs versus where we were, and the CPUs

1057
00:50:17,520 --> 00:50:20,400
Speaker 3: which were massively overshipping at the peak, they're now undershipping.

1058
00:50:20,400 --> 00:50:23,280
Speaker 3: And so we're in that inventory flushed part of the cycle,

1059
00:50:23,960 --> 00:50:25,960
Speaker 3: and so people have been sort of playing the space

1060
00:50:26,480 --> 00:50:28,960
Speaker 3: for that like second half recovery. Not now. All that

1061
00:50:28,960 --> 00:50:31,560
Speaker 3: being said, if you look at the overall industry, if

1062
00:50:31,560 --> 00:50:33,439
Speaker 3: you look at numbers in the second half, they're actually

1063
00:50:33,440 --> 00:50:35,680
Speaker 3: above seasonal. So people are starting to bake in that

1064
00:50:35,760 --> 00:50:39,960
Speaker 3: cyclical recovery to the numbers. And if you look at inventories,

1065
00:50:40,040 --> 00:50:42,919
Speaker 3: it just overall in the space they are ludicrously high.

1066
00:50:42,920 --> 00:50:46,040
Speaker 3: I've actually never seen them this five before. So we've

1067
00:50:46,040 --> 00:50:48,160
Speaker 3: had some inventory correction, but we may we may have not,

1068
00:50:48,960 --> 00:50:50,920
Speaker 3: we may just be getting started there. And if you

1069
00:50:50,920 --> 00:50:53,479
Speaker 3: look at valuations. I think the sector's trading. It's something

1070
00:50:53,520 --> 00:50:55,279
Speaker 3: like a thirty percent of premium to the S and

1071
00:50:55,280 --> 00:50:58,560
Speaker 3: P five hundred, which is the largest premium we've had again,

1072
00:50:58,680 --> 00:51:01,200
Speaker 3: probably since things normal life after the tech bubble or

1073
00:51:01,360 --> 00:51:04,640
Speaker 3: after the financial crisis at least, so people have been

1074
00:51:04,680 --> 00:51:07,279
Speaker 3: playing this backup recoverary. But yeah, we better get it

1075
00:51:09,120 --> 00:51:11,000
Speaker 3: as as as it relates to some of the other

1076
00:51:11,080 --> 00:51:13,520
Speaker 3: some of the individual stocks, like you mentioned Intel, It's funny.

1077
00:51:13,520 --> 00:51:14,759
Speaker 3: I think you guys may not know this. I just

1078
00:51:14,880 --> 00:51:20,200
Speaker 3: upgraded Intel. Oh. The title of the note was we

1079
00:51:20,280 --> 00:51:26,239
Speaker 3: hate this call, and I meant I desperately would like

1080
00:51:26,280 --> 00:51:28,840
Speaker 3: the standard prom It was and it was not a

1081
00:51:28,920 --> 00:51:31,080
Speaker 3: we like an Intel call. It was just I think

1082
00:51:31,120 --> 00:51:33,759
Speaker 3: that they that they're now under shipping in PCs by

1083
00:51:33,760 --> 00:51:35,560
Speaker 3: a wide margin, and I think for the first time

1084
00:51:35,560 --> 00:51:38,680
Speaker 3: in a while, the second half street numbers might actually

1085
00:51:38,680 --> 00:51:41,320
Speaker 3: be too low. So that's it's not like a super

1086
00:51:41,360 --> 00:51:44,640
Speaker 3: compelling call. But I felt uncomfortable Push although they were

1087
00:51:44,640 --> 00:51:46,920
Speaker 3: port earning next week, I make I may be kicking myself,

1088
00:51:46,920 --> 00:51:50,120
Speaker 3: like we'll still see in Vidia, however, so it's clearly

1089
00:51:50,200 --> 00:51:52,759
Speaker 3: you know you're ready. It hasn't reached its prior peak

1090
00:51:52,760 --> 00:51:55,120
Speaker 3: from a stock price base, and the reason the numbers

1091
00:51:55,120 --> 00:51:57,080
Speaker 3: have come down a lot. I mean, let's be honest,

1092
00:51:57,560 --> 00:52:01,040
Speaker 3: the gaining you know, business was was in plated significantly

1093
00:52:01,040 --> 00:52:04,240
Speaker 3: by crypto, right, and so that's all come out right.

1094
00:52:04,320 --> 00:52:06,440
Speaker 3: And then you know with data center, you had some

1095
00:52:06,560 --> 00:52:10,360
Speaker 3: impacts from from China. China general was weak, and then

1096
00:52:10,400 --> 00:52:12,080
Speaker 3: we had some of the export controls that they had

1097
00:52:12,080 --> 00:52:15,080
Speaker 3: to work their way around, and see had some issues there. Now,

1098
00:52:15,120 --> 00:52:18,560
Speaker 3: all of that being said, graphics cards in gaming, we

1099
00:52:18,760 --> 00:52:21,600
Speaker 3: talked about some of these inventory corrections. Graphics cards actually

1100
00:52:21,640 --> 00:52:23,840
Speaker 3: corrected the most and the most rapidly. So those have

1101
00:52:23,880 --> 00:52:25,799
Speaker 3: already hit bottom and they're growing again. And in VideA

1102
00:52:25,840 --> 00:52:27,759
Speaker 3: has got a product cycle there that they just kicked off.

1103
00:52:27,920 --> 00:52:30,400
Speaker 3: The new cards are called Lovelace and they and they

1104
00:52:30,480 --> 00:52:32,480
Speaker 3: look really good and especially behind and they're starting to

1105
00:52:32,480 --> 00:52:34,280
Speaker 3: fill out like the rest of the stack. So gaming

1106
00:52:34,360 --> 00:52:37,040
Speaker 3: is okay. And then in data centering again this you know,

1107
00:52:37,080 --> 00:52:40,200
Speaker 3: this generative AI has really caught everybody's fancy. And in

1108
00:52:40,320 --> 00:52:42,920
Speaker 3: Vivia had a data center of and they're saying that

1109
00:52:42,960 --> 00:52:44,800
Speaker 3: they were at the beginning of a product cycle in

1110
00:52:44,880 --> 00:52:47,440
Speaker 3: data centerm and you know, they had an advantage. A

1111
00:52:47,440 --> 00:52:49,319
Speaker 3: couple of weeks ago, they're their GtC event where they

1112
00:52:49,320 --> 00:52:53,360
Speaker 3: actually basically and directly said we're seeing upside from generative

1113
00:52:53,360 --> 00:52:56,480
Speaker 3: AI even now, right, So people have been buying in

1114
00:52:56,600 --> 00:52:59,480
Speaker 3: VideA on on on those on that thesis, and like

1115
00:52:59,520 --> 00:53:01,560
Speaker 3: the last the stock hit the peak at these peaks,

1116
00:53:01,560 --> 00:53:04,120
Speaker 3: at least in terms of valuation. The issue is we

1117
00:53:04,120 --> 00:53:07,200
Speaker 3: were at the peak of their product cycles and numbers

1118
00:53:07,239 --> 00:53:10,160
Speaker 3: came down. This time, valuations kind of went back to

1119
00:53:10,160 --> 00:53:12,720
Speaker 3: where they were at those peaks, but were the skinning

1120
00:53:12,719 --> 00:53:14,640
Speaker 3: of the product cycles, and numbers are probably going up

1121
00:53:14,680 --> 00:53:15,399
Speaker 3: knock knock down.

1122
00:53:15,719 --> 00:53:19,080
Speaker 1: So that's that's why Stacy I joked at the beginning

1123
00:53:19,160 --> 00:53:21,560
Speaker 1: that we could talk about about this for three hours,

1124
00:53:21,560 --> 00:53:24,920
Speaker 1: and I'm sure we could. Sure there's such a deep area,

1125
00:53:25,000 --> 00:53:28,120
Speaker 1: but that was a great overview of just like the

1126
00:53:28,160 --> 00:53:30,239
Speaker 1: state of competition, of the state of play, and the

1127
00:53:30,320 --> 00:53:32,799
Speaker 1: economics of this a very good way for us to

1128
00:53:32,840 --> 00:53:36,359
Speaker 1: sort of enter talking about AI. Stum more broadly, thank

1129
00:53:36,400 --> 00:53:38,760
Speaker 1: you so much for coming back on online.

1130
00:53:39,960 --> 00:53:42,120
Speaker 3: My pleasure. Anytime you guys want me here, just let

1131
00:53:42,120 --> 00:53:42,799
Speaker 3: me now, all right.

1132
00:53:42,719 --> 00:53:46,759
Speaker 1: We'll have you back next week for Intel take care

1133
00:53:46,800 --> 00:54:04,760
Speaker 1: of State. I really like talking to Stacey. He's really

1134
00:54:04,760 --> 00:54:07,399
Speaker 1: good at explaining complicated Yeah.

1135
00:54:07,400 --> 00:54:09,040
Speaker 2: I know, he made a point of saying that he's

1136
00:54:09,080 --> 00:54:11,040
Speaker 2: not an AI expert, but I thought he did a

1137
00:54:11,040 --> 00:54:13,480
Speaker 2: pretty good job of explaining it. I do think the

1138
00:54:13,640 --> 00:54:17,920
Speaker 2: trajectory of how all this, I mean, this is such

1139
00:54:17,960 --> 00:54:19,560
Speaker 2: an obvious thing to say, but it's going to be

1140
00:54:19,600 --> 00:54:23,719
Speaker 2: really interesting to watch and how businesses adapt to this,

1141
00:54:23,920 --> 00:54:26,640
Speaker 2: and we're what's kind of fascinating to me is that

1142
00:54:26,640 --> 00:54:30,680
Speaker 2: we're already seeing that differentiation play out in the market,

1143
00:54:30,800 --> 00:54:33,160
Speaker 2: with in video shares up quite a bit and Intel,

1144
00:54:33,239 --> 00:54:35,800
Speaker 2: which is seen as not as competitive in the space,

1145
00:54:36,000 --> 00:54:36,799
Speaker 2: down quite a bit.

1146
00:54:37,400 --> 00:54:40,359
Speaker 1: I was really interested in some of his points about

1147
00:54:40,480 --> 00:54:44,960
Speaker 1: software in particular, and so I have realized that, Yeah,

1148
00:54:45,040 --> 00:54:47,960
Speaker 1: like I mean, I you know, like sometimes I see

1149
00:54:48,160 --> 00:54:49,799
Speaker 1: like someone will post on Twitter it's like, look at

1150
00:54:49,800 --> 00:54:52,200
Speaker 1: this cool thing and video just rolled out where they

1151
00:54:52,200 --> 00:54:54,200
Speaker 1: can make your face look like something else or whatever.

1152
00:54:54,800 --> 00:54:59,040
Speaker 1: But thinking about like how important that is in terms

1153
00:54:59,080 --> 00:55:01,239
Speaker 1: of like, Okay, you and I want to start an

1154
00:55:01,280 --> 00:55:04,759
Speaker 1: AI company and idea for a large language model or

1155
00:55:04,760 --> 00:55:08,040
Speaker 1: something specifically have a model to train. There's going to

1156
00:55:08,080 --> 00:55:10,319
Speaker 1: be a big advantage going with the company that has

1157
00:55:10,360 --> 00:55:13,440
Speaker 1: this huge like wealth of like libraries and code bases

1158
00:55:13,480 --> 00:55:17,680
Speaker 1: and specific tools around specific industries as opposed to it

1159
00:55:17,760 --> 00:55:20,480
Speaker 1: seems like where some of the other competitors are, or

1160
00:55:20,480 --> 00:55:24,080
Speaker 1: it's just much more technically challenging to even like use

1161
00:55:24,200 --> 00:55:27,719
Speaker 1: the chips if they exist, like Google's.

1162
00:55:27,200 --> 00:55:32,239
Speaker 2: TPUs totally the other thing that caught my attention, And

1163
00:55:32,320 --> 00:55:34,760
Speaker 2: I know these are very different spaces in many ways,

1164
00:55:34,800 --> 00:55:38,399
Speaker 2: but there's so much of the terminology and like that's

1165
00:55:38,520 --> 00:55:41,880
Speaker 2: very reminiscent of crypto. So just the idea of like

1166
00:55:41,920 --> 00:55:45,239
Speaker 2: an AI winter and a crypto winter, and you can see,

1167
00:55:45,280 --> 00:55:47,680
Speaker 2: I mean, you can see the pivot happening right now

1168
00:55:47,719 --> 00:55:50,600
Speaker 2: from like crypto people moving into AI. So that's going

1169
00:55:50,640 --> 00:55:53,359
Speaker 2: to be interesting to watch play out. Like how much

1170
00:55:53,360 --> 00:55:56,760
Speaker 2: of it is hype classic sort of gartment hype cycle

1171
00:55:57,200 --> 00:55:58,399
Speaker 2: versus the real thing.

1172
00:55:58,560 --> 00:56:01,120
Speaker 1: But you know, two things, it's absolutely you know, so

1173
00:56:01,200 --> 00:56:02,840
Speaker 1: two things I think would be interesting. It'd be interesting

1174
00:56:02,920 --> 00:56:06,279
Speaker 1: to go back to like past AI summers, like what

1175
00:56:06,320 --> 00:56:08,400
Speaker 1: were some past periods which people thought we made this

1176
00:56:08,440 --> 00:56:10,120
Speaker 1: break through and then what happened? So that might be

1177
00:56:10,120 --> 00:56:12,600
Speaker 1: an interesting And then the other thing is like, look

1178
00:56:12,680 --> 00:56:16,719
Speaker 1: like you know, in twenty twenty three, I have never

1179
00:56:16,840 --> 00:56:20,200
Speaker 1: actually like found a reason I've ever felt compelled to

1180
00:56:20,280 --> 00:56:23,120
Speaker 1: like need to use a blockchain for something. And I

1181
00:56:23,160 --> 00:56:26,920
Speaker 1: get use out of chad GPT on something like almost

1182
00:56:26,960 --> 00:56:31,040
Speaker 1: every day. And so for example, we recently did an

1183
00:56:31,160 --> 00:56:33,759
Speaker 1: episode you know, yeah, look, we'll do an episode now

1184
00:56:33,800 --> 00:56:35,040
Speaker 1: of a question. At the end, they're like, oh, what

1185
00:56:35,160 --> 00:56:37,960
Speaker 1: is the difference Like yesterday, you know, we recently did

1186
00:56:37,960 --> 00:56:40,560
Speaker 1: an episode on like lending, and so it's like, oh,

1187
00:56:40,600 --> 00:56:44,479
Speaker 1: what's the difference sort of structurally between the leverage loan

1188
00:56:44,480 --> 00:56:46,239
Speaker 1: market and the private debt market. It's like, this might

1189
00:56:46,239 --> 00:56:48,600
Speaker 1: be an interesting question for a chat GPT, and like

1190
00:56:48,640 --> 00:56:52,160
Speaker 1: I got this like very useful, clear answer from it

1191
00:56:52,239 --> 00:56:55,000
Speaker 1: that like I couldn't have gotten perhaps as easily from

1192
00:56:55,040 --> 00:56:57,520
Speaker 1: a Google search. So I do think like some of

1193
00:56:57,560 --> 00:57:00,120
Speaker 1: these hype cycles like are really useful, But like I

1194
00:57:00,160 --> 00:57:04,040
Speaker 1: am already in my daily life and very already getting

1195
00:57:04,239 --> 00:57:06,000
Speaker 1: use out of this technology in a way that I

1196
00:57:06,040 --> 00:57:08,600
Speaker 1: cannot say for anything related like web three. No, that

1197
00:57:08,760 --> 00:57:09,760
Speaker 1: is very true.

1198
00:57:09,840 --> 00:57:11,799
Speaker 2: And you know the fact that this only came out

1199
00:57:11,880 --> 00:57:14,520
Speaker 2: a few months ago and everyone has been talking about

1200
00:57:14,520 --> 00:57:17,200
Speaker 2: it and experimenting with it kind of speaks for itself.

1201
00:57:17,520 --> 00:57:19,120
Speaker 1: Shall we leave it there? Let's leave it there.

1202
00:57:19,280 --> 00:57:22,640
Speaker 2: This has been another episode of the Oddlots podcast. I'm

1203
00:57:22,640 --> 00:57:25,920
Speaker 2: Tracy Alloway. You can follow me on Twitter at Tracy Alloway.

1204
00:57:26,000 --> 00:57:29,040
Speaker 1: And I'm Joe Wisenthal. You can follow me on Twitter

1205
00:57:29,120 --> 00:57:32,720
Speaker 1: at the Stalwart. Follow our guest Stacey Raskin. He's at

1206
00:57:33,000 --> 00:57:37,360
Speaker 1: s Raskin. Follow our producers Carmen Rodriguez at Carmen Arman

1207
00:57:37,480 --> 00:57:40,520
Speaker 1: and Dash o' bennett at dashbot. And check out all

1208
00:57:40,560 --> 00:57:44,240
Speaker 1: of our podcasts at Bloomberg under the handle at podcasts,

1209
00:57:44,280 --> 00:57:47,479
Speaker 1: and for more Oddlots content, go to Bloomberg dot com

1210
00:57:47,480 --> 00:57:51,000
Speaker 1: slash odd Lots. We blog, we post transcripts, we have

1211
00:57:51,000 --> 00:57:54,760
Speaker 1: a newsletter, and check out the Odd Loots Discord people

1212
00:57:54,880 --> 00:57:57,240
Speaker 1: listeners chatting twenty four to seven about all the things

1213
00:57:57,280 --> 00:57:59,680
Speaker 1: we talk about here. We even have an AI specific

1214
00:57:59,720 --> 00:58:03,360
Speaker 1: world that's really fun and set and the semiconductor room,

1215
00:58:03,520 --> 00:58:05,760
Speaker 1: and so people chatting about these things. I even so

1216
00:58:05,840 --> 00:58:09,000
Speaker 1: listened to some questions for today from that group, so

1217
00:58:09,160 --> 00:58:11,520
Speaker 1: it's really fun. I like hanging out there. To go

1218
00:58:11,560 --> 00:58:16,000
Speaker 1: to Discord dot gg slash pop. Thanks for listening