1
00:00:03,160 --> 00:00:18,520
Speaker 1: Bloomberg Audio Studios, Podcasts, radio News.

2
00:00:20,079 --> 00:00:23,959
Speaker 2: Hello and welcome to another episode of the Odd Lots podcast.

3
00:00:24,040 --> 00:00:25,680
Speaker 2: I'm Jill Wisenthal.

4
00:00:25,360 --> 00:00:26,439
Speaker 3: And I'm Tracy Alloway.

5
00:00:26,720 --> 00:00:30,880
Speaker 2: Tracy, here's something I know about AI. I don't know much,

6
00:00:30,920 --> 00:00:31,920
Speaker 2: but here's something.

7
00:00:31,680 --> 00:00:32,080
Speaker 4: I do know.

8
00:00:32,240 --> 00:00:33,600
Speaker 3: How to log into chat GPT.

9
00:00:33,920 --> 00:00:35,680
Speaker 2: No, I'm good at it. I'm good at that. I'm

10
00:00:35,680 --> 00:00:38,479
Speaker 2: good at logging into chat GPT and claude, and I'm

11
00:00:38,520 --> 00:00:41,680
Speaker 2: reasonably good at asking questions. Now, here's actually something about

12
00:00:41,680 --> 00:00:44,280
Speaker 2: the actually about the business of AI that I know.

13
00:00:44,520 --> 00:00:44,840
Speaker 3: Okay.

14
00:00:45,120 --> 00:00:45,879
Speaker 4: I know that in.

15
00:00:46,080 --> 00:00:50,120
Speaker 2: Video is making a ton of money and the stock

16
00:00:50,159 --> 00:00:53,280
Speaker 2: has gone to the moon, and that other companies would

17
00:00:53,280 --> 00:00:54,560
Speaker 2: like a slice of that pie.

18
00:00:55,160 --> 00:00:57,560
Speaker 3: Yes, yes, that's a good thing to know.

19
00:00:58,000 --> 00:01:00,360
Speaker 2: It's like a basic, simple thing, which is that when

20
00:01:00,360 --> 00:01:04,080
Speaker 2: people think about AI chips, there's literally one company that

21
00:01:04,160 --> 00:01:08,280
Speaker 2: comes to mind. I know others are involved. AMD has stuff,

22
00:01:08,440 --> 00:01:11,800
Speaker 2: Intel obviously wants to play others, but there is obviously

23
00:01:11,840 --> 00:01:15,840
Speaker 2: that one gigantic pile of cash that's flowing to this

24
00:01:15,840 --> 00:01:18,120
Speaker 2: one company. I don't know if it's still but at

25
00:01:18,120 --> 00:01:20,240
Speaker 2: one point, is the biggest company in the world is

26
00:01:20,440 --> 00:01:21,160
Speaker 2: pulled back.

27
00:01:21,000 --> 00:01:21,520
Speaker 4: A little bit.

28
00:01:22,080 --> 00:01:24,640
Speaker 2: Well, I would say two things. One, other companies would

29
00:01:24,680 --> 00:01:27,920
Speaker 2: like that a piece of that pie. And b companies

30
00:01:27,959 --> 00:01:31,639
Speaker 2: that are in the business of building AI models would

31
00:01:31,680 --> 00:01:35,039
Speaker 2: like to find a way to get cheaper, more efficient,

32
00:01:35,360 --> 00:01:38,640
Speaker 2: less energy intensive chips so that they don't have to

33
00:01:38,680 --> 00:01:40,160
Speaker 2: always pay the Nvidia tax.

34
00:01:40,440 --> 00:01:43,240
Speaker 3: Do you want to know what I know about AI

35
00:01:43,319 --> 00:01:46,320
Speaker 3: and semiconductors, Let's go for it. Okay, here's the one

36
00:01:46,360 --> 00:01:49,160
Speaker 3: thing that I know, which is that whenever you have

37
00:01:49,280 --> 00:01:52,800
Speaker 3: this conversation about in Nvidia, the one word that always

38
00:01:52,800 --> 00:01:54,080
Speaker 3: comes up is moat.

39
00:01:54,400 --> 00:01:55,440
Speaker 2: Oh yes, moat yeah.

40
00:01:55,520 --> 00:01:59,400
Speaker 3: So, like you're either talking about like medieval castles or

41
00:01:59,440 --> 00:02:02,280
Speaker 3: you're talking about semiconductor manufacturing. That's when you hear the

42
00:02:02,320 --> 00:02:05,360
Speaker 3: word mote because over and over again people will say

43
00:02:05,400 --> 00:02:07,480
Speaker 3: it is expensive to make the chips. You need a

44
00:02:07,480 --> 00:02:10,040
Speaker 3: lot of money for research and development and to set

45
00:02:10,120 --> 00:02:12,480
Speaker 3: up the fabs, and you need a lot of first

46
00:02:12,520 --> 00:02:16,080
Speaker 3: person expertise in building them. And then there's also the

47
00:02:16,120 --> 00:02:20,160
Speaker 3: network effect. So a company like Nvidia has this huge

48
00:02:20,200 --> 00:02:23,560
Speaker 3: moat around its business. The question, of course, is whether

49
00:02:23,680 --> 00:02:26,520
Speaker 3: or not, getting back to the medieval castle analogy, it

50
00:02:26,600 --> 00:02:28,560
Speaker 3: is unassailable, that's right.

51
00:02:28,720 --> 00:02:32,519
Speaker 2: If semiconductor seems to be mote after MOTI, after mode,

52
00:02:32,520 --> 00:02:36,840
Speaker 2: because there's ASML's moat, and then there's Taiwan Semiconductor's moat,

53
00:02:37,440 --> 00:02:41,000
Speaker 2: and then there's Nvidia's moat, and so yes, it's like

54
00:02:41,040 --> 00:02:44,880
Speaker 2: there's a series of moats, and if someone could overcome

55
00:02:45,000 --> 00:02:46,960
Speaker 2: these moats or make find a way to build a

56
00:02:47,000 --> 00:02:50,800
Speaker 2: bridge over one of these moats and enter this proverbial castle,

57
00:02:51,080 --> 00:02:53,760
Speaker 2: that would be very lucrative. We know that many are

58
00:02:53,880 --> 00:02:57,919
Speaker 2: trying to enter these moats, but it's incredibly costly and

59
00:02:58,080 --> 00:03:01,680
Speaker 2: capital intensive and difficult. There are just not many people

60
00:03:01,680 --> 00:03:04,080
Speaker 2: who know how to do any of this stuff, and

61
00:03:04,200 --> 00:03:06,840
Speaker 2: so the question of whether these modes can be overcome.

62
00:03:07,200 --> 00:03:09,480
Speaker 2: But again, there are many businesses that would love to

63
00:03:09,480 --> 00:03:13,320
Speaker 2: see more robust competition in the space so that their

64
00:03:13,400 --> 00:03:15,160
Speaker 2: payment is not a attack.

65
00:03:15,520 --> 00:03:18,359
Speaker 3: You know, one thing I don't know, and I don't

66
00:03:18,400 --> 00:03:21,120
Speaker 3: think we've ever done an episode purely on this, but

67
00:03:21,200 --> 00:03:25,040
Speaker 3: I don't really understand the different designs of chips. So

68
00:03:25,200 --> 00:03:28,720
Speaker 3: I know that some chips, specifically in videos, are supposed

69
00:03:28,760 --> 00:03:33,040
Speaker 3: to be better at AI. They're better at running lots

70
00:03:33,120 --> 00:03:36,400
Speaker 3: of little calculations all at the same time. And I

71
00:03:36,440 --> 00:03:40,200
Speaker 3: know there's basic chips that go into your refrigerator or

72
00:03:40,200 --> 00:03:42,360
Speaker 3: your car or whatever. But I don't really know the

73
00:03:42,400 --> 00:03:46,560
Speaker 3: difference between what a chip that was designed specifically to

74
00:03:46,720 --> 00:03:50,120
Speaker 3: run a large language model would look like compared to

75
00:03:50,560 --> 00:03:52,080
Speaker 3: a standard basic chip.

76
00:03:52,320 --> 00:03:54,400
Speaker 2: I don't know anything about chip design. I just sort

77
00:03:54,400 --> 00:03:58,760
Speaker 2: of imagined someone on like using some CADS software, etching

78
00:03:58,880 --> 00:04:02,520
Speaker 2: little lines in the thing and drawing some sort of

79
00:04:02,560 --> 00:04:05,560
Speaker 2: like circuitry or you know, put it place in the trains.

80
00:04:06,040 --> 00:04:08,520
Speaker 3: You know, A chip design game would be really fun,

81
00:04:08,600 --> 00:04:10,400
Speaker 3: now that I think about it. Yeah, you could just

82
00:04:10,520 --> 00:04:13,360
Speaker 3: draw little things on the square. Okay. Anyway, Well, we

83
00:04:13,400 --> 00:04:13,800
Speaker 3: are going.

84
00:04:13,760 --> 00:04:17,200
Speaker 2: To learn about how chip design works. We are going

85
00:04:17,279 --> 00:04:21,200
Speaker 2: to learn about what makes a chip particularly good for

86
00:04:21,279 --> 00:04:25,320
Speaker 2: the task of training and running inference on these AI models.

87
00:04:25,600 --> 00:04:27,479
Speaker 2: And I have to say, I really do believe we

88
00:04:27,600 --> 00:04:31,400
Speaker 2: have the two perfect guests because they are both veterans

89
00:04:31,480 --> 00:04:34,400
Speaker 2: in this space, and they are both active in the

90
00:04:34,680 --> 00:04:38,120
Speaker 2: attempt to bridge some of these motes and enter the

91
00:04:38,160 --> 00:04:41,479
Speaker 2: space and bring competition to the industry. We are going

92
00:04:41,520 --> 00:04:44,320
Speaker 2: to be speaking with yin Or Pope, co founder and

93
00:04:44,440 --> 00:04:47,400
Speaker 2: CEO of Medex, as well as Mike Gunter, co founder

94
00:04:47,400 --> 00:04:50,679
Speaker 2: and CTO of Madex. It's a new company that's trying

95
00:04:50,720 --> 00:04:55,960
Speaker 2: to build chips specifically for the purpose of large language models.

96
00:04:56,279 --> 00:04:58,839
Speaker 2: Both of them have a lot of experience in the

97
00:04:58,880 --> 00:05:01,440
Speaker 2: space we're going to we get our hands dirty, so

98
00:05:01,520 --> 00:05:04,560
Speaker 2: to speak, and understand how you build the hardware for

99
00:05:04,560 --> 00:05:06,800
Speaker 2: all this stuff and what makes it win and whether

100
00:05:06,839 --> 00:05:09,400
Speaker 2: it's even a winnable game. Ryan Or and Mike, thank

101
00:05:09,440 --> 00:05:11,080
Speaker 2: you so much for coming on Outlaws.

102
00:05:11,440 --> 00:05:14,040
Speaker 5: Thanks, happy to be here, pleasure to be here.

103
00:05:14,160 --> 00:05:16,839
Speaker 2: So what do you tell us? What does a chip

104
00:05:16,880 --> 00:05:20,640
Speaker 2: designer do? I know, I have this completely cartoonish view

105
00:05:20,680 --> 00:05:23,880
Speaker 2: in my head that cannot possibly be right of someone

106
00:05:23,960 --> 00:05:27,200
Speaker 2: on a big screen using some CAD software to sort of,

107
00:05:27,279 --> 00:05:28,880
Speaker 2: you know, figure out what's going to be etched in

108
00:05:28,920 --> 00:05:31,560
Speaker 2: that way for of silicon. What is the job of

109
00:05:31,640 --> 00:05:32,280
Speaker 2: chip design?

110
00:05:33,200 --> 00:05:35,440
Speaker 5: So maybe this is best told by what is the

111
00:05:35,480 --> 00:05:38,520
Speaker 5: story of chip development from the beginning of a project

112
00:05:38,560 --> 00:05:41,000
Speaker 5: to the end of it. So there's a range of

113
00:05:41,000 --> 00:05:42,360
Speaker 5: different ways this can go, but there's a lot of

114
00:05:42,400 --> 00:05:46,000
Speaker 5: things that are in common. So generally a chip design

115
00:05:46,200 --> 00:05:49,880
Speaker 5: team is at the low end, maybe thirty people, up

116
00:05:49,920 --> 00:05:52,560
Speaker 5: to many many thousands of people at the high end,

117
00:05:53,000 --> 00:05:56,479
Speaker 5: and it as the project typically runs for somewhere in

118
00:05:56,480 --> 00:05:58,800
Speaker 5: the range of three to five years from conception to

119
00:05:58,880 --> 00:06:02,160
Speaker 5: actually shipping to customer, and so over that time what

120
00:06:02,160 --> 00:06:04,760
Speaker 5: we see in the life cycle is we tend to

121
00:06:04,800 --> 00:06:07,840
Speaker 5: start with a small team of architects. If you think

122
00:06:07,839 --> 00:06:10,080
Speaker 5: of designing a house, the team of architects are the

123
00:06:10,080 --> 00:06:12,440
Speaker 5: people who decide what rooms go in here, or how

124
00:06:12,440 --> 00:06:14,880
Speaker 5: many bedrooms, how many bathrooms, what are the flows between them,

125
00:06:14,880 --> 00:06:16,640
Speaker 5: how do people walk through the corridors, and so on,

126
00:06:17,000 --> 00:06:19,840
Speaker 5: the coarse grained design of the chip, in the chip itself,

127
00:06:19,880 --> 00:06:22,080
Speaker 5: that is, you know what kinds of components at the

128
00:06:22,360 --> 00:06:26,160
Speaker 5: high level we have, and then after that initial exploration,

129
00:06:26,680 --> 00:06:29,039
Speaker 5: this moves then over to the micro architects. These are

130
00:06:29,080 --> 00:06:31,200
Speaker 5: the people who are designing the individual rooms. What are

131
00:06:31,240 --> 00:06:34,320
Speaker 5: the components that go in the individual rooms. So at

132
00:06:34,360 --> 00:06:36,760
Speaker 5: that point everything we've done so far is a design

133
00:06:36,839 --> 00:06:41,040
Speaker 5: stage thing. This is done in documents, spreadsheets, and it's

134
00:06:41,080 --> 00:06:44,080
Speaker 5: a verbal and human communication form. But beyond that, that's

135
00:06:44,080 --> 00:06:46,160
Speaker 5: when it starts to actually touch the computer in a

136
00:06:46,520 --> 00:06:49,839
Speaker 5: more meaningful sense. And so the micro architects will hand

137
00:06:49,880 --> 00:06:52,760
Speaker 5: over to the logic designers. They are the people who

138
00:06:52,800 --> 00:06:55,200
Speaker 5: are actually writing code. So even though you think of

139
00:06:55,240 --> 00:06:58,080
Speaker 5: chips as being this very physical thing where there's wires

140
00:06:58,120 --> 00:07:00,320
Speaker 5: and gates and everything. The way we try to admit

141
00:07:00,320 --> 00:07:02,400
Speaker 5: this information to the computer is actually writing code. We

142
00:07:02,440 --> 00:07:05,760
Speaker 5: write verolog that expresses the design of the chip. So

143
00:07:06,120 --> 00:07:10,040
Speaker 5: that's what the logic designers are doing. That's an extended

144
00:07:10,080 --> 00:07:12,400
Speaker 5: period of time building out all of the different you know,

145
00:07:12,680 --> 00:07:16,320
Speaker 5: matrix multiplies, memories, circuitry that connects to the outside world,

146
00:07:16,360 --> 00:07:18,800
Speaker 5: and so on. And then the output of all of

147
00:07:18,840 --> 00:07:21,920
Speaker 5: them is this verolog piece of software code that gets

148
00:07:22,080 --> 00:07:25,000
Speaker 5: then compiled by a computer down to a set of

149
00:07:25,080 --> 00:07:27,960
Speaker 5: gates which are logic gates and or gates and so on.

150
00:07:28,040 --> 00:07:29,800
Speaker 5: And then why is that connect them together? That's the

151
00:07:29,840 --> 00:07:33,560
Speaker 5: netlist this file. Then there's a few more stages still

152
00:07:33,560 --> 00:07:36,960
Speaker 5: coming here. This file gets handed off to physical designers,

153
00:07:37,000 --> 00:07:39,760
Speaker 5: who again work with CAD tools to convert this kind

154
00:07:39,760 --> 00:07:40,600
Speaker 5: of logical discussion.

155
00:07:40,640 --> 00:07:42,560
Speaker 2: Was right, Someone is using CAD tools.

156
00:07:43,480 --> 00:07:46,040
Speaker 5: Absolutely, there's a CAD tool, but it's it's only out

157
00:07:46,080 --> 00:07:50,040
Speaker 5: of the job. Okay, So the physical designers are converting

158
00:07:50,080 --> 00:07:52,800
Speaker 5: the sort of logical description into a physical placement. So

159
00:07:53,240 --> 00:07:55,560
Speaker 5: where do each of these gates go? Now there's two

160
00:07:55,640 --> 00:07:58,000
Speaker 5: hundred billion logic gates on a chip, so a human

161
00:07:58,040 --> 00:07:59,760
Speaker 5: is not going to be placing all of those manually.

162
00:08:00,040 --> 00:08:03,120
Speaker 5: So there's a huge amount of software assistance here. But

163
00:08:03,160 --> 00:08:05,240
Speaker 5: what the human is doing is providing oversights through this

164
00:08:05,280 --> 00:08:07,520
Speaker 5: process and saying, I've done this a ton of times before.

165
00:08:07,640 --> 00:08:10,560
Speaker 5: This placement kind of looks wrong, it doesn't match my heuristics,

166
00:08:10,600 --> 00:08:12,760
Speaker 5: and so I can probably do a better job here.

167
00:08:13,160 --> 00:08:15,360
Speaker 5: So that's the physical designers, and the output of their

168
00:08:15,400 --> 00:08:18,920
Speaker 5: work is actually eventually you get a polygons, so basically

169
00:08:18,920 --> 00:08:21,600
Speaker 5: an image saying here is the thing that is going

170
00:08:21,680 --> 00:08:26,160
Speaker 5: to get etched onto a piece of silicon. So that

171
00:08:26,600 --> 00:08:29,640
Speaker 5: file is ultimately a huge, like really big image in

172
00:08:29,640 --> 00:08:32,000
Speaker 5: some form a bunch of polygons on it. It gets

173
00:08:32,040 --> 00:08:36,439
Speaker 5: handed over to a manufacturing company such as TSMC. They

174
00:08:36,440 --> 00:08:41,040
Speaker 5: spend maybe four or five months initially creating a mask set,

175
00:08:41,120 --> 00:08:43,760
Speaker 5: so those are like the templates or the stencils that

176
00:08:43,800 --> 00:08:46,160
Speaker 5: will be used to stamp out many many copies of

177
00:08:46,160 --> 00:08:48,679
Speaker 5: the chip, and then stamps up many copies of the chip.

178
00:08:48,720 --> 00:08:51,840
Speaker 5: You get a chip back. This is typically about two

179
00:08:51,920 --> 00:08:54,160
Speaker 5: or three years after you started the project. You get

180
00:08:54,200 --> 00:08:57,000
Speaker 5: chips back, and now you have a bring up team

181
00:08:57,000 --> 00:09:00,520
Speaker 5: who puts this chip into a whole board and connected

182
00:09:00,559 --> 00:09:02,680
Speaker 5: to what to power and electricity and starts testing it,

183
00:09:03,240 --> 00:09:05,760
Speaker 5: and then after another six to twelve months or maybe

184
00:09:05,760 --> 00:09:08,800
Speaker 5: even more, eventually you actually can hand this over to customers.

185
00:09:09,160 --> 00:09:10,920
Speaker 5: There's maybe just one or two other things which are

186
00:09:10,920 --> 00:09:13,920
Speaker 5: not in that flow but very essential to call out too.

187
00:09:14,360 --> 00:09:18,040
Speaker 5: Are because of this whole process taking so long, especially

188
00:09:18,040 --> 00:09:21,440
Speaker 5: the manufacturing, we also have like very large teams of

189
00:09:21,600 --> 00:09:24,920
Speaker 5: verification people. So these are the people who before we

190
00:09:24,920 --> 00:09:27,160
Speaker 5: actually send it to manufacturing and pay twenty to thirty

191
00:09:27,160 --> 00:09:31,480
Speaker 5: million dollars of manufacturing, we have a substantial team doing

192
00:09:31,480 --> 00:09:33,640
Speaker 5: a lot of testing. And this is software based testing,

193
00:09:33,720 --> 00:09:36,120
Speaker 5: so writing tests in the same way a software engineer

194
00:09:36,160 --> 00:09:39,600
Speaker 5: might to make sure that the functionality actually works as intended.

195
00:09:39,920 --> 00:09:44,240
Speaker 6: To underlying the comparison to ordinary software, which Reiner touched

196
00:09:44,280 --> 00:09:47,760
Speaker 6: on it, we're writing code, but it's on super hard mode.

197
00:09:48,160 --> 00:09:50,600
Speaker 6: So if you have a if you have a software

198
00:09:50,640 --> 00:09:54,000
Speaker 6: that's deployed the website, you can fix a bug and

199
00:09:54,120 --> 00:09:57,880
Speaker 6: you know, ten minutes at basically zero cost. Whereas in

200
00:09:57,920 --> 00:09:59,880
Speaker 6: our case, the reason that we have a large team

201
00:10:00,080 --> 00:10:03,280
Speaker 6: people doing verification making sure that what we've done is

202
00:10:03,320 --> 00:10:07,439
Speaker 6: correct is that it's potentially four months and thirty million

203
00:10:07,440 --> 00:10:11,079
Speaker 6: dollars for every mistake that you let through. Likewise, there

204
00:10:11,120 --> 00:10:14,280
Speaker 6: is software, but it's a relatively small fraction of software

205
00:10:14,320 --> 00:10:16,719
Speaker 6: that's very performance critical where you want the code to

206
00:10:16,760 --> 00:10:19,400
Speaker 6: run as fast as possible. But in some sense, every

207
00:10:19,480 --> 00:10:22,120
Speaker 6: line of code that you write in hardware has an

208
00:10:22,120 --> 00:10:25,480
Speaker 6: impact on the overall performance of the product, because every

209
00:10:25,520 --> 00:10:28,400
Speaker 6: line of code ends up getting embodied in silicon, and

210
00:10:28,440 --> 00:10:31,280
Speaker 6: every line of code affects the eventual performance. So it's

211
00:10:31,360 --> 00:10:34,080
Speaker 6: kind of coding, but on hard mode.

212
00:10:34,800 --> 00:10:40,520
Speaker 3: So I intuitively understand the importance of getting the software right.

213
00:10:40,679 --> 00:10:45,360
Speaker 3: But why does placement on the actual chip or wayfer

214
00:10:45,480 --> 00:10:48,280
Speaker 3: Why does that matter? Are you trying to make it

215
00:10:48,280 --> 00:10:51,280
Speaker 3: more efficient, are you trying to reduce the rise time?

216
00:10:51,440 --> 00:10:53,640
Speaker 3: Or why does it matter where the little bits and

217
00:10:53,679 --> 00:10:56,679
Speaker 3: bobs are placed? To use the scientific.

218
00:10:56,200 --> 00:11:00,400
Speaker 6: Term, Yeah, you're right that reducing the right time is

219
00:11:00,640 --> 00:11:04,320
Speaker 6: a massive issue. And you know, fundamentally the issue is

220
00:11:04,320 --> 00:11:07,520
Speaker 6: that chips, you know, at a very abstract level, are

221
00:11:07,960 --> 00:11:11,480
Speaker 6: composed of were at a somewhat content concrete level, really

222
00:11:11,800 --> 00:11:16,000
Speaker 6: are composed of transistors and wires, and the placement has

223
00:11:16,000 --> 00:11:19,720
Speaker 6: a dramatic effect on the link through the wires, which

224
00:11:19,720 --> 00:11:22,199
Speaker 6: has a dramatic effect on both the performance of the

225
00:11:22,240 --> 00:11:24,760
Speaker 6: chip and how much you can fit. In terms of

226
00:11:24,800 --> 00:11:27,679
Speaker 6: the impact that this has on the quality of chip

227
00:11:27,720 --> 00:11:32,080
Speaker 6: that you produce, wires have over time not been shrinking

228
00:11:32,200 --> 00:11:36,559
Speaker 6: in the same way that transistors have, and so getting

229
00:11:36,800 --> 00:11:39,560
Speaker 6: the wearing right, which usually means getting the placement right,

230
00:11:39,679 --> 00:11:41,560
Speaker 6: has become more and more important over time.

231
00:11:57,960 --> 00:12:01,160
Speaker 3: Can chips be beautiful? I know code can be elegant,

232
00:12:01,720 --> 00:12:04,160
Speaker 3: and some people will say certain code is beautiful, But

233
00:12:04,320 --> 00:12:07,120
Speaker 3: have you ever looked at a semiconductor and been like, oh, wow,

234
00:12:07,320 --> 00:12:09,680
Speaker 3: that's really nicely put together.

235
00:12:10,520 --> 00:12:12,640
Speaker 5: For me, I mean I think absolutely yes. This is

236
00:12:12,679 --> 00:12:14,320
Speaker 5: like why I work in this space is I just

237
00:12:14,400 --> 00:12:16,560
Speaker 5: really like geeking out on the design of things. But

238
00:12:16,800 --> 00:12:19,000
Speaker 5: to me, what beautiful for a chip means is that

239
00:12:19,280 --> 00:12:21,439
Speaker 5: it kind of does exactly what it was designed to do,

240
00:12:21,960 --> 00:12:24,679
Speaker 5: and no more and no less. I mean, obviously less

241
00:12:24,720 --> 00:12:27,720
Speaker 5: would be a bit of a disappointment, but often if

242
00:12:27,720 --> 00:12:29,600
Speaker 5: it does more, do you think, well, maybe I designed

243
00:12:29,600 --> 00:12:31,600
Speaker 5: it for slightly the wrong purpose or something like that.

244
00:12:32,000 --> 00:12:35,240
Speaker 2: I think this is a good seg into getting into

245
00:12:35,360 --> 00:12:39,120
Speaker 2: your business specifically, so we all know that so much

246
00:12:39,120 --> 00:12:42,720
Speaker 2: of this AI is powered by these in video GPUs,

247
00:12:43,240 --> 00:12:46,520
Speaker 2: but in video GPUs have been used for a long

248
00:12:46,559 --> 00:12:49,480
Speaker 2: time for many things that do not have anything to

249
00:12:49,559 --> 00:12:53,880
Speaker 2: do with large language models or the specific AI applications

250
00:12:53,880 --> 00:12:56,120
Speaker 2: that people are excited about today in twenty twenty four.

251
00:12:56,640 --> 00:12:58,960
Speaker 2: So for a while they were, well, the video games

252
00:12:59,000 --> 00:13:01,400
Speaker 2: is obviously the big one for decades and decades, and

253
00:13:01,440 --> 00:13:03,560
Speaker 2: then there was like five minutes where people got really

254
00:13:03,600 --> 00:13:07,520
Speaker 2: excited to use them for ethereum mining, and now everyone's

255
00:13:07,559 --> 00:13:11,600
Speaker 2: really excited about their use for artificial intelligence and large

256
00:13:11,679 --> 00:13:14,920
Speaker 2: language models and some of these other generative AI applications

257
00:13:14,960 --> 00:13:18,440
Speaker 2: that people are excited about right now, Why don't you

258
00:13:18,559 --> 00:13:21,920
Speaker 2: tell us maybe the sort of idea behind maddex, but

259
00:13:22,040 --> 00:13:25,640
Speaker 2: specifically what you were both doing when you were at

260
00:13:25,679 --> 00:13:29,440
Speaker 2: alphabet or Google, which you know it has its own chips.

261
00:13:29,480 --> 00:13:33,319
Speaker 2: I believe it has something called TPUs. What was the

262
00:13:33,440 --> 00:13:38,160
Speaker 2: project at Google? Why did Google find it necessary or

263
00:13:38,280 --> 00:13:40,600
Speaker 2: a good business to start building their own chips for

264
00:13:40,640 --> 00:13:43,520
Speaker 2: in house purposes? And then why did you feel the

265
00:13:43,559 --> 00:13:46,960
Speaker 2: need to then leave to build what you're building now

266
00:13:47,040 --> 00:13:48,400
Speaker 2: for LLM specifically?

267
00:13:48,960 --> 00:13:52,760
Speaker 6: Yeah, So what Google was seeing, and this was at

268
00:13:52,760 --> 00:13:56,640
Speaker 6: this point sometime back more than a decade ago, they

269
00:13:56,679 --> 00:14:01,439
Speaker 6: were seeing that the use of artific intelligence lllms were

270
00:14:01,440 --> 00:14:04,160
Speaker 6: not a thing at that point, was going up, and

271
00:14:04,440 --> 00:14:08,520
Speaker 6: they were worried about how much money they would have

272
00:14:08,720 --> 00:14:11,960
Speaker 6: to spend on traditional it would be it would have

273
00:14:12,000 --> 00:14:16,040
Speaker 6: been GPUs at that time, and so they built a

274
00:14:16,160 --> 00:14:21,040
Speaker 6: very specialized chip to do neural nets, and that chips

275
00:14:21,400 --> 00:14:27,240
Speaker 6: specialize on matrix multiplication. So they put in a structure

276
00:14:27,280 --> 00:14:31,520
Speaker 6: called a systolic array, which they definitely didn't invent. It existed,

277
00:14:32,120 --> 00:14:35,400
Speaker 6: has existed from the seventies that is especially good at

278
00:14:35,400 --> 00:14:39,920
Speaker 6: doing matrix multiplication. Now after that, Nvidia has added a

279
00:14:39,960 --> 00:14:44,680
Speaker 6: similar structure into their chips. And the initial Google TPU

280
00:14:45,000 --> 00:14:47,600
Speaker 6: was an inference focused only chip, and then they have

281
00:14:47,840 --> 00:14:51,360
Speaker 6: subsequently made chips that can be used for both training

282
00:14:51,360 --> 00:14:54,480
Speaker 6: and inference. And I guess now is a good point

283
00:14:54,520 --> 00:14:56,920
Speaker 6: to So the very last thing that I was doing

284
00:14:56,920 --> 00:14:59,440
Speaker 6: at Google was I was on the TPU team and

285
00:14:59,480 --> 00:15:02,120
Speaker 6: Reiner was on the large language model team. And it's

286
00:15:02,120 --> 00:15:04,680
Speaker 6: probably good to have him sort of tell free from here.

287
00:15:05,040 --> 00:15:07,320
Speaker 5: So I mean, what we were seeing and this this

288
00:15:07,400 --> 00:15:09,320
Speaker 5: is what we personally were seeing, but Google was seeing

289
00:15:09,360 --> 00:15:12,120
Speaker 5: more generally as well. Is just large language models were

290
00:15:12,120 --> 00:15:14,400
Speaker 5: a thing. There was this period of time between GPT

291
00:15:14,560 --> 00:15:17,480
Speaker 5: three and chat GIPT coming out. GPT three came out

292
00:15:17,480 --> 00:15:20,440
Speaker 5: in twenty twenty, and so people who were very plugged

293
00:15:20,480 --> 00:15:24,560
Speaker 5: into the field recognized the importance of it all at

294
00:15:24,640 --> 00:15:26,720
Speaker 5: least to some extent, recognized the importance of it back then,

295
00:15:27,280 --> 00:15:30,080
Speaker 5: and so there was this push to you know, everyone

296
00:15:30,120 --> 00:15:32,600
Speaker 5: wanted to create their own large language model that was

297
00:15:32,640 --> 00:15:35,800
Speaker 5: better than GPT three, and so, I mean, at the time,

298
00:15:35,840 --> 00:15:38,280
Speaker 5: I was on the Large Language Model team. We helped

299
00:15:38,320 --> 00:15:41,440
Speaker 5: training Google Palm, and we were using thousands of TPUs

300
00:15:41,480 --> 00:15:44,240
Speaker 5: for that, and one of the things we were saying is, well,

301
00:15:44,240 --> 00:15:47,240
Speaker 5: look what does it cost to deploy this? In Google Search?

302
00:15:47,360 --> 00:15:49,280
Speaker 5: There's quite a lot of search querers. I think it's

303
00:15:49,320 --> 00:15:51,200
Speaker 5: the public estimates thro about one hundred thousand of them

304
00:15:51,200 --> 00:15:54,600
Speaker 5: per second. If you multiply out how much each querer costs,

305
00:15:54,720 --> 00:15:56,400
Speaker 5: and if you want to run that on large language models,

306
00:15:56,400 --> 00:15:58,680
Speaker 5: that's a lot more expensive. And then also I just

307
00:15:58,720 --> 00:16:00,680
Speaker 5: if I want to train a model that's times bigger

308
00:16:00,680 --> 00:16:03,840
Speaker 5: than my current model or one hundred times bigger, suddenly

309
00:16:04,280 --> 00:16:07,120
Speaker 5: these models have just moved from costing you know, a

310
00:16:07,160 --> 00:16:09,640
Speaker 5: million dollars or one hundred thousand dollars to train to

311
00:16:10,000 --> 00:16:12,040
Speaker 5: tens of millions and hundreds of millions of dollars, and

312
00:16:12,120 --> 00:16:16,000
Speaker 5: so the overall goal was can we make it cheaper

313
00:16:16,000 --> 00:16:18,440
Speaker 5: by any way possible. So, of course there's algorithmic approaches.

314
00:16:18,480 --> 00:16:21,440
Speaker 5: There's a lot of opportunity on the algorithm and research side.

315
00:16:21,480 --> 00:16:23,560
Speaker 5: But then the other really big lever is just making

316
00:16:23,560 --> 00:16:25,840
Speaker 5: better hardware. So one of the things we were looking

317
00:16:25,880 --> 00:16:29,440
Speaker 5: at was trying to make Google's TPUs better for large

318
00:16:29,480 --> 00:16:32,000
Speaker 5: language models. What led us, actually, i mean this is

319
00:16:32,040 --> 00:16:33,760
Speaker 5: personally about Mike and me in this case, or what

320
00:16:33,840 --> 00:16:36,440
Speaker 5: led us to leave Google to make medics was we

321
00:16:36,480 --> 00:16:38,640
Speaker 5: saw that there was We believe that there is some

322
00:16:38,720 --> 00:16:42,400
Speaker 5: opportunity to make chips substantially better if you're only looking

323
00:16:42,400 --> 00:16:45,160
Speaker 5: to focus on large language models. And so the chips

324
00:16:45,160 --> 00:16:49,560
Speaker 5: that were designed pre GPT three and especially pre chat

325
00:16:49,600 --> 00:16:52,560
Speaker 5: GPT try to do a really good job on really

326
00:16:52,560 --> 00:16:54,440
Speaker 5: good job on small models as well as a really

327
00:16:54,480 --> 00:16:56,840
Speaker 5: good job on large models. And so what you find

328
00:16:56,880 --> 00:16:59,040
Speaker 5: is that the circuitry in those chips, there's a bit

329
00:16:59,080 --> 00:17:01,120
Speaker 5: of circuitry for what you need for small models, there's

330
00:17:01,120 --> 00:17:03,080
Speaker 5: a bit of secretry for what you need for large models.

331
00:17:03,120 --> 00:17:05,760
Speaker 5: Also for maybe embedding look ups. There's three or four

332
00:17:05,760 --> 00:17:08,560
Speaker 5: different kinds of workloads, and all of them take some

333
00:17:08,640 --> 00:17:11,640
Speaker 5: of the real estate in your cellica. And so if

334
00:17:11,640 --> 00:17:13,280
Speaker 5: you really want to make the best use of the

335
00:17:13,280 --> 00:17:15,119
Speaker 5: real estate, you should just focus on the thing you

336
00:17:15,160 --> 00:17:17,520
Speaker 5: care about most and hope that there's a big market there.

337
00:17:17,640 --> 00:17:20,639
Speaker 5: So that the game and or what we decided to

338
00:17:20,680 --> 00:17:22,600
Speaker 5: do when we see some others deciding to do as well,

339
00:17:22,720 --> 00:17:25,680
Speaker 5: is to really try and focus on just the one

340
00:17:25,680 --> 00:17:27,639
Speaker 5: workload that seems like it's going to become a one

341
00:17:27,680 --> 00:17:30,320
Speaker 5: hundred billion dollar or a trendion dollar industry.

342
00:17:30,680 --> 00:17:33,160
Speaker 2: I know there's always this sort of cliche when talking

343
00:17:33,160 --> 00:17:36,480
Speaker 2: about techno. Oh, Google and Facebook, they can just build

344
00:17:36,480 --> 00:17:38,760
Speaker 2: this and they'll destroy your little startup because they have

345
00:17:38,840 --> 00:17:42,000
Speaker 2: infinites amounts of money. Except that doesn't actually seem to

346
00:17:42,200 --> 00:17:44,840
Speaker 2: happen in the real world as much as people on

347
00:17:44,880 --> 00:17:48,400
Speaker 2: Twitter expect it to happen. But can you just sort

348
00:17:48,400 --> 00:17:51,639
Speaker 2: of give a sense of maybe the business and organizational

349
00:17:52,200 --> 00:17:57,960
Speaker 2: incentives for why a company like Google doesn't say, oh,

350
00:17:58,040 --> 00:18:00,159
Speaker 2: this is one hundred billion dollar market in video is

351
00:18:00,200 --> 00:18:02,320
Speaker 2: worth three and a half trillion or three trillion dollars,

352
00:18:02,440 --> 00:18:06,240
Speaker 2: let's build our own LM specific chips. Why doesn't that

353
00:18:06,880 --> 00:18:11,159
Speaker 2: happen at these large, hyperscaler companies that presumably have all

354
00:18:11,200 --> 00:18:12,520
Speaker 2: the talent and money to do it.

355
00:18:13,920 --> 00:18:20,919
Speaker 6: So Google's TPUs are primarily built to serve their internal customers,

356
00:18:21,520 --> 00:18:25,320
Speaker 6: and Google's revenue for the most part comes from Google

357
00:18:25,359 --> 00:18:28,960
Speaker 6: Search that Google Search, and in particular from Google Search ads.

358
00:18:29,400 --> 00:18:34,280
Speaker 6: Google Search ads. Is you know, a customer of the TPUs,

359
00:18:34,040 --> 00:18:38,720
Speaker 6: It's a relatively difficult thing to say that hundreds of

360
00:18:38,800 --> 00:18:41,480
Speaker 6: billions of dollars of revenue that we're making, we're going

361
00:18:41,520 --> 00:18:44,359
Speaker 6: to make a chip that doesn't really support that particularly well,

362
00:18:44,400 --> 00:18:47,400
Speaker 6: and focuses on this at this point unproven in terms

363
00:18:47,440 --> 00:18:51,840
Speaker 6: of revenue market and it's not just ads, but they

364
00:18:51,880 --> 00:18:54,320
Speaker 6: are you know, a variety of other customers. For instance,

365
00:18:54,560 --> 00:18:57,359
Speaker 6: you know, you may have noticed how Google is pretty

366
00:18:57,359 --> 00:19:01,679
Speaker 6: good at identifying good photos and doing a whole variety

367
00:19:01,760 --> 00:19:04,359
Speaker 6: of other things that are supported in many cases by

368
00:19:04,400 --> 00:19:05,000
Speaker 6: the TPUs.

369
00:19:06,280 --> 00:19:08,240
Speaker 5: I think one of the other things too, that we

370
00:19:08,320 --> 00:19:11,760
Speaker 5: see in all chip companies in general, or companies producing chips,

371
00:19:11,840 --> 00:19:14,919
Speaker 5: is because producing chips is so expensive, you end up

372
00:19:14,960 --> 00:19:16,600
Speaker 5: in this place where you really want to put all

373
00:19:16,640 --> 00:19:21,320
Speaker 5: your resources behind one chip effort. And so just because

374
00:19:21,400 --> 00:19:23,520
Speaker 5: the thinking is that there's a huge amount of return

375
00:19:23,600 --> 00:19:25,879
Speaker 5: on investment in making this one thing better rather than

376
00:19:25,920 --> 00:19:28,199
Speaker 5: fragmenting your efforts. Really, what you'd like to do in

377
00:19:28,200 --> 00:19:30,880
Speaker 5: this situation where there's a new emerging field that might

378
00:19:30,960 --> 00:19:33,600
Speaker 5: be huge or might not, but it's hard to say yet,

379
00:19:33,720 --> 00:19:35,399
Speaker 5: what you'd like to do is maybe spin up a

380
00:19:35,440 --> 00:19:37,760
Speaker 5: second effort on the side and have like a skunk works. Yeah,

381
00:19:37,880 --> 00:19:38,439
Speaker 5: that's work, right.

382
00:19:38,440 --> 00:19:41,199
Speaker 2: That would be just to let Ryan er and just

383
00:19:41,320 --> 00:19:43,280
Speaker 2: let the two of you go have your own little

384
00:19:43,280 --> 00:19:44,160
Speaker 2: office somewhere else.

385
00:19:44,560 --> 00:19:48,199
Speaker 5: Yeah, just organizationally that it's often challenging to do, and

386
00:19:48,240 --> 00:19:50,720
Speaker 5: we see this across all companies. Every chip company really

387
00:19:50,720 --> 00:19:54,760
Speaker 5: has essentially only one mainstream chip product that is that

388
00:19:54,800 --> 00:19:57,120
Speaker 5: they're iterating on and making better and better over time.

389
00:19:58,200 --> 00:20:03,000
Speaker 3: To what degree is to design driven by the customer?

390
00:20:03,119 --> 00:20:05,440
Speaker 3: And what I mean by that is, so the TPUs

391
00:20:05,480 --> 00:20:09,639
Speaker 3: at Google were developed to handle Google's internal workloads, but

392
00:20:09,920 --> 00:20:13,920
Speaker 3: at other chip designers, to what degree will customers come

393
00:20:13,960 --> 00:20:16,600
Speaker 3: and like basically do a reverse inquiry and ask for

394
00:20:16,640 --> 00:20:20,320
Speaker 3: a specific chip or what does the dialogue between customers

395
00:20:20,400 --> 00:20:23,320
Speaker 3: and the big chip designers actually look like.

396
00:20:24,080 --> 00:20:27,040
Speaker 5: Yeah, it's a fun interplay of I want my provider

397
00:20:27,080 --> 00:20:28,479
Speaker 5: to do a good job, but I also don't want

398
00:20:28,520 --> 00:20:31,880
Speaker 5: to leak my IP too much. So you can see

399
00:20:31,920 --> 00:20:34,640
Speaker 5: this how this played out in so Mike was talking

400
00:20:34,680 --> 00:20:37,880
Speaker 5: about through the development of the TPUs which were publicly

401
00:20:37,920 --> 00:20:41,439
Speaker 5: announced in twenty sixteen and around the same time in

402
00:20:41,520 --> 00:20:44,119
Speaker 5: videos first GPU with the tens ocos, So that was

403
00:20:44,160 --> 00:20:46,520
Speaker 5: the first GPU that was really focused on matrix multiplication.

404
00:20:46,800 --> 00:20:49,320
Speaker 5: That was the vaulted generation came out at about the

405
00:20:49,359 --> 00:20:52,479
Speaker 5: same time. And some of this actually was a result

406
00:20:52,520 --> 00:20:56,680
Speaker 5: of when Google had this recognition of look, matrix multiplication

407
00:20:56,760 --> 00:20:58,600
Speaker 5: is so important, we need to make it really better.

408
00:20:58,800 --> 00:21:01,800
Speaker 5: They simultaneously work themselves but also went to Nvideo and

409
00:21:01,840 --> 00:21:04,399
Speaker 5: said we're not telling you much, but can you do

410
00:21:04,440 --> 00:21:07,879
Speaker 5: better at matrix multification? And so that was enough for

411
00:21:08,000 --> 00:21:10,760
Speaker 5: Nvidia to go on the first generation. They made a

412
00:21:10,760 --> 00:21:12,720
Speaker 5: pretty good attempt. But if you talk to people that

413
00:21:12,800 --> 00:21:15,199
Speaker 5: in video, I'll say that actually the second generation of

414
00:21:15,240 --> 00:21:17,760
Speaker 5: the tensacle which was in the MPa generation, was where

415
00:21:17,760 --> 00:21:20,399
Speaker 5: they really nailed it. So when it's big enough, you

416
00:21:20,440 --> 00:21:22,760
Speaker 5: sometimes see these customers coming and saying what they want,

417
00:21:22,800 --> 00:21:26,040
Speaker 5: but they'll maybe they'll try and disguise what they're asking

418
00:21:26,080 --> 00:21:28,240
Speaker 5: for or not giving you the absolute minimum amount of

419
00:21:28,280 --> 00:21:31,600
Speaker 5: information to help a vendor make what they want without

420
00:21:31,600 --> 00:21:32,760
Speaker 5: revealing too much about their.

421
00:21:32,640 --> 00:21:50,240
Speaker 4: IB Let's get to maddex.

422
00:21:50,680 --> 00:21:54,280
Speaker 2: Tell us the product that you're designing and how it

423
00:21:54,720 --> 00:21:59,040
Speaker 2: fundamentally will differ from the offerings on the market, most

424
00:21:59,080 --> 00:22:00,000
Speaker 2: notably from VideA.

425
00:22:00,040 --> 00:22:00,240
Speaker 4: Yeah.

426
00:22:01,240 --> 00:22:04,320
Speaker 5: Yeah, So we make chips and in fact racks and

427
00:22:04,320 --> 00:22:08,840
Speaker 5: clusters for large language models. So when you look at

428
00:22:09,160 --> 00:22:11,719
Speaker 5: in videos GPUs, you already talked about all of this,

429
00:22:12,000 --> 00:22:15,679
Speaker 5: the original background in gaming, this brief movement in ethereum,

430
00:22:15,920 --> 00:22:18,280
Speaker 5: and then even within AI they're doing small models of

431
00:22:18,400 --> 00:22:22,760
Speaker 5: large models. So what that translates to in you can

432
00:22:22,760 --> 00:22:24,560
Speaker 5: think of it as the rooms of the house or something.

433
00:22:24,680 --> 00:22:27,000
Speaker 5: They have a different room for each of each of

434
00:22:27,040 --> 00:22:29,880
Speaker 5: those different use cases, so different circuitry in the chip

435
00:22:29,920 --> 00:22:32,840
Speaker 5: for all of these use cases. And the fundamental bet

436
00:22:32,920 --> 00:22:35,919
Speaker 5: is that if you say, look, I don't care about that,

437
00:22:35,960 --> 00:22:37,720
Speaker 5: I'm going to do a lousy job if you try

438
00:22:37,720 --> 00:22:38,919
Speaker 5: and run a game on me, or I'm going to

439
00:22:38,960 --> 00:22:41,280
Speaker 5: do a lousy job if you want to run a

440
00:22:41,280 --> 00:22:44,479
Speaker 5: convolutional network on me. But if you give me a

441
00:22:44,560 --> 00:22:47,240
Speaker 5: large model with very large matrices, I'm going to crush it.

442
00:22:47,640 --> 00:22:50,440
Speaker 5: That's the bet that we're making amatix, so we spend

443
00:22:50,440 --> 00:22:52,520
Speaker 5: as much of our silicon as we can on making

444
00:22:52,600 --> 00:22:54,959
Speaker 5: this work. There's a lot of detail in making all

445
00:22:54,960 --> 00:22:56,480
Speaker 5: of this work out, because you need not just the

446
00:22:56,480 --> 00:22:58,960
Speaker 5: matrix multiplication, but the all of the memory bandwidths and

447
00:22:58,960 --> 00:23:02,639
Speaker 5: communication bandwidths and the actual engineering things to make a

448
00:23:02,640 --> 00:23:05,280
Speaker 5: pen out. But that's the core bette.

449
00:23:05,200 --> 00:23:09,040
Speaker 3: And why can't Invidia do this? So you know, in

450
00:23:09,160 --> 00:23:11,359
Speaker 3: Video has a lot of resources, It has that big

451
00:23:11,400 --> 00:23:13,959
Speaker 3: moat as we were discussing in the intro, and it

452
00:23:14,000 --> 00:23:17,119
Speaker 3: has the GPUs that are already in production and working

453
00:23:17,240 --> 00:23:20,159
Speaker 3: on new ones. But why couldn't it start designing an

454
00:23:20,480 --> 00:23:22,680
Speaker 3: LM focused chip from scratch?

455
00:23:23,800 --> 00:23:27,200
Speaker 6: Right? So you talked about in Vidia's mode and that

456
00:23:27,680 --> 00:23:30,919
Speaker 6: moat has two components. One component is that they build

457
00:23:31,000 --> 00:23:34,040
Speaker 6: the very best hardware, and I think you know that

458
00:23:34,200 --> 00:23:38,520
Speaker 6: is the result of having a very large team that

459
00:23:38,680 --> 00:23:42,720
Speaker 6: executes extremely well and making good choices about how to

460
00:23:42,760 --> 00:23:46,479
Speaker 6: serve their market. They also have a tremendous software mode.

461
00:23:46,840 --> 00:23:48,800
Speaker 6: And you know, both of these moats are important to

462
00:23:48,840 --> 00:23:53,040
Speaker 6: different sets of customers, so they're tremendous software mode. They

463
00:23:53,040 --> 00:23:57,359
Speaker 6: have a very broad, deep software ecosystem based on Kuda

464
00:23:57,880 --> 00:23:58,879
Speaker 6: that allows it.

465
00:23:59,040 --> 00:24:01,080
Speaker 3: Oh yeah, I remember this came up in our discussion

466
00:24:01,080 --> 00:24:01,720
Speaker 3: with core Weave.

467
00:24:02,000 --> 00:24:06,439
Speaker 6: Yeah yeah. And so that allows customers who are not

468
00:24:06,560 --> 00:24:11,720
Speaker 6: very sophisticated, who don't have gigantic engineering budgets themselves, to

469
00:24:11,880 --> 00:24:15,560
Speaker 6: use those chips and using videos chips and be efficient

470
00:24:15,640 --> 00:24:18,960
Speaker 6: at that. So the thing about a mote is not

471
00:24:19,000 --> 00:24:21,840
Speaker 6: only does it in some sense keep other people out,

472
00:24:21,880 --> 00:24:25,560
Speaker 6: it also keeps you in. So insofar as they want

473
00:24:25,600 --> 00:24:29,040
Speaker 6: to keep their software mode, their Kuda mote, they have

474
00:24:29,080 --> 00:24:34,240
Speaker 6: to remain compatible with Kuda and compatibilility with that software mode.

475
00:24:34,440 --> 00:24:39,640
Speaker 6: Compatibilility with Kuda requires certain hardware structures. So in Videos

476
00:24:40,080 --> 00:24:43,080
Speaker 6: has lots and lots of threads, they have a very

477
00:24:43,119 --> 00:24:47,080
Speaker 6: flexible memory system. These things are great for being able

478
00:24:47,119 --> 00:24:50,479
Speaker 6: to flexibly address a whole bunch of different types of

479
00:24:50,560 --> 00:24:53,760
Speaker 6: neural net problems, but they all cost in terms of hardware,

480
00:24:53,840 --> 00:24:58,160
Speaker 6: and they're not necessarily those The choices to have those

481
00:24:58,160 --> 00:25:01,040
Speaker 6: sorts of things are not necessarily the in fact, not

482
00:25:01,119 --> 00:25:03,399
Speaker 6: the choices that you would want to make if you

483
00:25:03,440 --> 00:25:06,879
Speaker 6: were aiming specifically at an LM. So in order to

484
00:25:06,920 --> 00:25:11,400
Speaker 6: be you know, fully competitive with a chip that's specialized

485
00:25:11,440 --> 00:25:14,080
Speaker 6: for LLMS, they would have to give up all of that.

486
00:25:14,600 --> 00:25:18,120
Speaker 6: And you know, Jensen himself has said that the one

487
00:25:18,680 --> 00:25:21,320
Speaker 6: non negotiable rule in our company is that we have

488
00:25:21,359 --> 00:25:22,520
Speaker 6: to be compatible with kuda.

489
00:25:23,480 --> 00:25:27,240
Speaker 2: This is interesting. So the challenge for them of spinning

490
00:25:27,280 --> 00:25:31,280
Speaker 2: out something totally different is that it would be outside

491
00:25:31,359 --> 00:25:35,359
Speaker 2: the family. And so it's outside the Kudah family, so

492
00:25:35,400 --> 00:25:35,800
Speaker 2: to speak.

493
00:25:35,880 --> 00:25:38,640
Speaker 3: And meanwhile, you already have like pie, Torch and Triton

494
00:25:38,680 --> 00:25:39,600
Speaker 3: waiting in the wings.

495
00:25:39,640 --> 00:25:42,920
Speaker 2: I guess, so why don't you tell us a little

496
00:25:42,920 --> 00:25:46,680
Speaker 2: bit more about the business of LLLM chips specifically, because

497
00:25:46,680 --> 00:25:50,320
Speaker 2: there's a lot of questions, Like, you know, one question

498
00:25:50,440 --> 00:25:52,920
Speaker 2: is you have all these people in Silicon Valley who

499
00:25:52,960 --> 00:25:57,040
Speaker 2: seem motivated by the idea of like agi that that's

500
00:25:57,080 --> 00:26:00,280
Speaker 2: the goal, that we're going to have super intelligence one day,

501
00:26:00,320 --> 00:26:03,720
Speaker 2: maybe thousands IQs and the hundreds of thousands one day.

502
00:26:03,720 --> 00:26:06,200
Speaker 2: That'll make us all seem very dumb, et cetera. Are

503
00:26:06,200 --> 00:26:09,240
Speaker 2: you implicitly making a bet by your company that it'll

504
00:26:09,280 --> 00:26:12,480
Speaker 2: be lllms that we'll get there, Because as you mentioned,

505
00:26:12,520 --> 00:26:15,240
Speaker 2: there are other algorithmic ideas, There are other ideas for

506
00:26:15,320 --> 00:26:18,800
Speaker 2: how you might be able to expand intelligent. How much

507
00:26:18,920 --> 00:26:22,080
Speaker 2: of your company's bet is the idea that the future

508
00:26:22,240 --> 00:26:26,119
Speaker 2: of generative AI or as we know it, is going

509
00:26:26,160 --> 00:26:27,960
Speaker 2: to be along the LLLM pathway.

510
00:26:28,520 --> 00:26:30,440
Speaker 5: One of the core things. I think there's two core

511
00:26:30,600 --> 00:26:34,760
Speaker 5: ingredients of the LM pathway. Yeah, one so far is

512
00:26:34,920 --> 00:26:37,840
Speaker 5: the transformer architecture, which is a model architecture and was

513
00:26:37,880 --> 00:26:41,040
Speaker 5: substantially better than the things that came before. But the

514
00:26:41,080 --> 00:26:43,919
Speaker 5: other one, and that actually has a much longer history,

515
00:26:44,080 --> 00:26:48,000
Speaker 5: is the scaling hypothesis in hypothesis in general sore, But

516
00:26:48,200 --> 00:26:51,359
Speaker 5: that's the there's a general observation which has been widely

517
00:26:51,400 --> 00:26:56,040
Speaker 5: recognized for a decade or more that if I am sorry,

518
00:26:56,160 --> 00:26:58,840
Speaker 5: I'm training in neural net or some kind of AI model,

519
00:26:59,200 --> 00:27:01,080
Speaker 5: if I want to make its quality better and make

520
00:27:01,119 --> 00:27:03,680
Speaker 5: it bigger, and so what does bigger mean? Bigger means

521
00:27:03,680 --> 00:27:06,359
Speaker 5: I have to spend more compute training it. Bigger means

522
00:27:06,359 --> 00:27:09,800
Speaker 5: I have more neurons. Tho's are the loosely analogous to

523
00:27:09,880 --> 00:27:12,439
Speaker 5: the sort of processing power in a human brain, although

524
00:27:12,800 --> 00:27:15,520
Speaker 5: analogy is weak. If I make my model bigger, I

525
00:27:15,520 --> 00:27:18,160
Speaker 5: get better quality. That's a sort of simple qualitative thing

526
00:27:18,160 --> 00:27:20,800
Speaker 5: to say, and that's been true for a really long

527
00:27:20,840 --> 00:27:25,199
Speaker 5: time in these models. So the advantage of that, or

528
00:27:25,400 --> 00:27:27,480
Speaker 5: the thing that we've seen really recently is we've seen

529
00:27:27,480 --> 00:27:31,760
Speaker 5: this turned up to eleven. So around the time when

530
00:27:31,800 --> 00:27:34,280
Speaker 5: GPT three came out, So in twenty twenty, a paper

531
00:27:34,359 --> 00:27:37,880
Speaker 5: was published called the Scaling Laws, and so this took

532
00:27:37,920 --> 00:27:41,879
Speaker 5: this qualitative observation and made it quantitative and said, actually,

533
00:27:41,920 --> 00:27:43,680
Speaker 5: we can even fit an equation to it, and so

534
00:27:43,760 --> 00:27:46,760
Speaker 5: that gave people a lot more conviction to it. And

535
00:27:46,960 --> 00:27:50,159
Speaker 5: this is what led to the people saying, well, if

536
00:27:50,200 --> 00:27:52,240
Speaker 5: I have a better model, I can solve more problems

537
00:27:52,240 --> 00:27:54,440
Speaker 5: with AI than I could before. And so every time

538
00:27:54,480 --> 00:27:57,360
Speaker 5: I spend ten times as much training on it, I

539
00:27:57,480 --> 00:28:00,879
Speaker 5: unlock new use cases. And so that's what to this craze.

540
00:28:00,920 --> 00:28:03,840
Speaker 5: And the remarkable thing is that while there are these

541
00:28:03,840 --> 00:28:05,920
Speaker 5: diminishing returns, I have to spend ten times as much

542
00:28:05,920 --> 00:28:09,639
Speaker 5: computing power to get some improvement beyond that sort of

543
00:28:09,800 --> 00:28:13,679
Speaker 5: logarithmic scale. We don't see as yet any plateau an

544
00:28:13,840 --> 00:28:16,880
Speaker 5: so it seems like there continues to be opportunity here.

545
00:28:17,160 --> 00:28:19,879
Speaker 5: So the key thing is this scaling hypothesis or scaling

546
00:28:19,960 --> 00:28:21,960
Speaker 5: laws in general that are causing these models to grow.

547
00:28:22,480 --> 00:28:24,600
Speaker 5: And then I mean as a hardware provider, what you

548
00:28:24,680 --> 00:28:26,520
Speaker 5: might look at is you might say, that's the thing

549
00:28:26,520 --> 00:28:28,000
Speaker 5: I really want to bet on. I want to bet

550
00:28:28,040 --> 00:28:30,840
Speaker 5: on the growth of models, and I mean, now it's

551
00:28:30,840 --> 00:28:33,320
Speaker 5: a little more in the details, but the thing you

552
00:28:33,359 --> 00:28:35,399
Speaker 5: actually have to bet on is the growth of matrix sites,

553
00:28:35,600 --> 00:28:37,920
Speaker 5: which is very strongly correlated with the growth of models.

554
00:28:38,560 --> 00:28:42,280
Speaker 3: Just to hammer this point home, if more AI was

555
00:28:42,360 --> 00:28:47,560
Speaker 3: learning from stuff like self play or synthetic data rather

556
00:28:47,640 --> 00:28:51,520
Speaker 3: than scraping the internet, would the design of the chips

557
00:28:51,720 --> 00:28:54,480
Speaker 3: have to take that into account, Like, how would the

558
00:28:54,560 --> 00:28:58,360
Speaker 3: chips vary between those different learning styles.

559
00:28:59,080 --> 00:29:02,240
Speaker 5: Yeah, so in general, when you're building a chip, you

560
00:29:02,520 --> 00:29:04,560
Speaker 5: have to make it programmable because you're going to make

561
00:29:04,560 --> 00:29:06,080
Speaker 5: this chip and you will ship a new version every

562
00:29:06,120 --> 00:29:07,920
Speaker 5: two years, but what people want to do with the

563
00:29:08,000 --> 00:29:10,239
Speaker 5: chip is going to change every month or so, so

564
00:29:10,280 --> 00:29:12,760
Speaker 5: it has to be programmable to some extent. So that's

565
00:29:12,760 --> 00:29:15,520
Speaker 5: true for all of the chips that anyone ships, and

566
00:29:15,560 --> 00:29:19,360
Speaker 5: so there's different scales of programmability and what kinds of

567
00:29:19,440 --> 00:29:23,160
Speaker 5: changes you need to adapt to, So changes in kind

568
00:29:23,200 --> 00:29:25,880
Speaker 5: of the way you feed it data that's maybe on

569
00:29:25,920 --> 00:29:28,480
Speaker 5: the very very outer layers of doesn't affect much of

570
00:29:28,520 --> 00:29:30,360
Speaker 5: the core of the chip, and so those kinds of

571
00:29:30,480 --> 00:29:33,120
Speaker 5: changes tend to be some of the easier changes to

572
00:29:33,120 --> 00:29:35,560
Speaker 5: adapt to. The things that then become a little harder

573
00:29:35,560 --> 00:29:39,200
Speaker 5: to adapt to is if I'm substantially changing my model architecture.

574
00:29:39,320 --> 00:29:41,600
Speaker 5: So a small change might be maybe I change the

575
00:29:41,680 --> 00:29:44,640
Speaker 5: number of layers, or I reorder some of the layers

576
00:29:44,640 --> 00:29:47,240
Speaker 5: in my model, or maybe I use the same ingredients

577
00:29:47,320 --> 00:29:49,800
Speaker 5: but shuffle them around in some way. A bigger change

578
00:29:49,840 --> 00:29:51,560
Speaker 5: would be that say, Okay, I'm actually going to throw

579
00:29:51,560 --> 00:29:53,320
Speaker 5: out all of these ingredients and use a completely different

580
00:29:53,320 --> 00:29:56,600
Speaker 5: set of primitives. And that's often that's that last step

581
00:29:56,640 --> 00:29:58,560
Speaker 5: is the one that that would really kill you if

582
00:29:58,600 --> 00:30:01,280
Speaker 5: you're betting very much on it. Partic a set of ingredients.

583
00:30:01,920 --> 00:30:05,400
Speaker 6: So an example of a potential different set of primitives

584
00:30:05,440 --> 00:30:08,920
Speaker 6: that are used in other models that aren't used in

585
00:30:09,320 --> 00:30:13,120
Speaker 6: llms are we made mention of these embedding things that

586
00:30:13,160 --> 00:30:16,800
Speaker 6: are used in recommender and ad models. So Facebook has

587
00:30:17,000 --> 00:30:20,240
Speaker 6: talked about building special purpose hardware to support inference on

588
00:30:20,280 --> 00:30:24,160
Speaker 6: those kind of models. Those are they have much less

589
00:30:24,200 --> 00:30:31,080
Speaker 6: emphasis relative emphasis, particularly on matrix multiply. Another possible direction

590
00:30:31,200 --> 00:30:35,280
Speaker 6: that model architecture could go. That would be different and

591
00:30:35,680 --> 00:30:38,880
Speaker 6: bad for a chip designed for current LLMS, would be

592
00:30:39,400 --> 00:30:43,560
Speaker 6: instead of having very large matrices in about one hundred layers,

593
00:30:43,600 --> 00:30:46,800
Speaker 6: you could have much smaller matrices but ten thousand layers,

594
00:30:47,280 --> 00:30:50,920
Speaker 6: and that would demand a different sort of design to

595
00:30:51,000 --> 00:30:54,520
Speaker 6: be good at that kind of model. So a bet

596
00:30:54,560 --> 00:30:58,880
Speaker 6: that looks good given the modern history of neural nets

597
00:30:58,960 --> 00:31:01,280
Speaker 6: is that matrices will get larger over time.

598
00:31:01,960 --> 00:31:04,280
Speaker 2: You know, you're talking about scaling laws, and so everyone

599
00:31:04,320 --> 00:31:09,000
Speaker 2: talks about okay, computation, power, energy efficiency, et cetera, and

600
00:31:09,120 --> 00:31:11,520
Speaker 2: I never know if they're true. But then sometimes you

601
00:31:11,600 --> 00:31:14,520
Speaker 2: read these stories they're like Sam Altman wants to go

602
00:31:14,560 --> 00:31:18,400
Speaker 2: around the world and raise like five trillion dollars to

603
00:31:18,480 --> 00:31:22,600
Speaker 2: like build his own semiconductor fabs and have the entire architecture,

604
00:31:22,600 --> 00:31:25,400
Speaker 2: because that's like what it's going to take. What about

605
00:31:25,400 --> 00:31:27,920
Speaker 2: the data side, because this is another thing people talk about,

606
00:31:27,920 --> 00:31:30,520
Speaker 2: the data wall that you know, there's only one Internet

607
00:31:30,600 --> 00:31:33,480
Speaker 2: to scrape, and then you know, after that, what if

608
00:31:33,480 --> 00:31:36,320
Speaker 2: you're not there at AGI yet again, I know you're

609
00:31:36,360 --> 00:31:39,520
Speaker 2: solving for the hardware side, but when you think about

610
00:31:39,640 --> 00:31:44,400
Speaker 2: risks going forward along the LLM pathway, what's your perspective

611
00:31:44,600 --> 00:31:47,880
Speaker 2: on well, what happens when we've just we've ingested all

612
00:31:47,880 --> 00:31:49,000
Speaker 2: the data.

613
00:31:49,080 --> 00:31:51,760
Speaker 5: So there's two ways you can make a model better.

614
00:31:51,960 --> 00:31:54,000
Speaker 5: One of them is by training on more data, and

615
00:31:54,040 --> 00:31:56,280
Speaker 5: the other one is making a bigger model. And these

616
00:31:56,320 --> 00:31:59,520
Speaker 5: two effects work in a really complimentary way. So you

617
00:31:59,520 --> 00:32:01,520
Speaker 5: can think of a like having a bigger brain and

618
00:32:01,560 --> 00:32:03,840
Speaker 5: then practicing more and so both of these are going

619
00:32:03,920 --> 00:32:06,720
Speaker 5: to help to some extent. So there's a risk that

620
00:32:06,760 --> 00:32:09,680
Speaker 5: we hit a data wall. In general, there's been a

621
00:32:09,720 --> 00:32:13,440
Speaker 5: long history of people predicting walls in different kinds of

622
00:32:13,480 --> 00:32:18,040
Speaker 5: walls in techno training and then ingenuity overcoming this, and

623
00:32:18,080 --> 00:32:22,280
Speaker 5: so I wouldn't I would bet that there's a fairly

624
00:32:22,360 --> 00:32:26,000
Speaker 5: large amount of mileage to continue here. Tracy mentioned self

625
00:32:26,040 --> 00:32:30,840
Speaker 5: training and generating new data. That's the vibe in the

626
00:32:30,840 --> 00:32:33,640
Speaker 5: industry is that this is a promising direction for sure.

627
00:32:34,040 --> 00:32:36,520
Speaker 5: But even if you don't bet on that, there's mileage,

628
00:32:36,520 --> 00:32:38,800
Speaker 5: and it's less attractive mileage, but there is mileage in

629
00:32:38,800 --> 00:32:42,480
Speaker 5: making the models bigger. So I believe, and I think

630
00:32:42,480 --> 00:32:46,000
Speaker 5: this is shared by many people insiders in the industry

631
00:32:46,000 --> 00:32:48,520
Speaker 5: as well, is that there's at least a few more

632
00:32:48,600 --> 00:32:50,880
Speaker 5: orders of magnitude available here before we run out of

633
00:32:51,080 --> 00:32:54,200
Speaker 5: easy engineering knobs to turn. But of course, one of

634
00:32:54,240 --> 00:32:56,800
Speaker 5: the limiting factors here is just the dollars you spend.

635
00:32:57,200 --> 00:33:01,200
Speaker 5: So you have some amount of budge that I'm willing

636
00:33:01,240 --> 00:33:03,440
Speaker 5: to spend. And I mean, maybe Sam can raise five

637
00:33:03,440 --> 00:33:05,880
Speaker 5: trillion dollars, I don't think necessarily everyone else can raise

638
00:33:05,960 --> 00:33:08,360
Speaker 5: that amount of money to train a model. And so

639
00:33:08,400 --> 00:33:10,120
Speaker 5: if you've got a fixed amount of dollars that you

640
00:33:10,160 --> 00:33:11,920
Speaker 5: want to spend, and you want to train the best model,

641
00:33:12,280 --> 00:33:14,240
Speaker 5: you want to make the best use of the multipliers,

642
00:33:14,440 --> 00:33:15,840
Speaker 5: you want to make the best use of the dollars

643
00:33:15,880 --> 00:33:18,320
Speaker 5: you spend, and so that means fundamentally, what you're paying

644
00:33:18,320 --> 00:33:21,200
Speaker 5: for is the flops, which flops is a floating point operation,

645
00:33:21,760 --> 00:33:24,840
Speaker 5: so the number of multipliers you can do. And then

646
00:33:24,880 --> 00:33:27,320
Speaker 5: every time I increase my model size or increase the

647
00:33:27,360 --> 00:33:29,920
Speaker 5: amount of training data I've got, I'm spending more flops,

648
00:33:30,040 --> 00:33:34,320
Speaker 5: and so flops converts into intelligence. And then if I've

649
00:33:34,320 --> 00:33:36,200
Speaker 5: got a fixed budget, really what I want to maximize

650
00:33:36,280 --> 00:33:37,280
Speaker 5: is my flops per dollar.

651
00:33:38,680 --> 00:33:41,840
Speaker 3: I find this so fascinating because there are so many

652
00:33:41,880 --> 00:33:45,560
Speaker 3: different directions that you could theoretically go in, and so

653
00:33:45,680 --> 00:33:49,760
Speaker 3: many decisions that need to be made, you know, do

654
00:33:49,800 --> 00:33:52,440
Speaker 3: you go after of that scale? How do you tailor

655
00:33:52,520 --> 00:33:55,479
Speaker 3: the design for different methods of data input? Although, as

656
00:33:55,480 --> 00:33:57,680
Speaker 3: you said earlier, maybe that's one of the easiest things

657
00:33:57,840 --> 00:34:01,280
Speaker 3: to respond to. But then there are other trade offs

658
00:34:01,320 --> 00:34:04,680
Speaker 3: that you have to think about between speed and power

659
00:34:04,720 --> 00:34:09,440
Speaker 3: consumption and I guess area utilization or the placement of

660
00:34:09,480 --> 00:34:11,840
Speaker 3: all the bits and bobs that we were discussing earlier,

661
00:34:11,880 --> 00:34:16,200
Speaker 3: and cost effectiveness too. How do you balance all those

662
00:34:16,320 --> 00:34:19,600
Speaker 3: elements and are there particular things that you're willing to

663
00:34:19,760 --> 00:34:21,520
Speaker 3: sacrifice for others.

664
00:34:22,719 --> 00:34:27,000
Speaker 6: So different people can choose different targets to go after

665
00:34:27,480 --> 00:34:31,400
Speaker 6: in the market, and so one one target, which you

666
00:34:31,440 --> 00:34:35,640
Speaker 6: could argue in VideA is winning on currently and one

667
00:34:35,680 --> 00:34:37,719
Speaker 6: of the reasons that their chips their products are so

668
00:34:37,760 --> 00:34:41,400
Speaker 6: popular is, as Rayner said, just the amount of flops

669
00:34:41,440 --> 00:34:43,520
Speaker 6: you can get out out of a chip, and if

670
00:34:43,719 --> 00:34:46,440
Speaker 6: all the chips are roughly the same to make, that

671
00:34:46,800 --> 00:34:52,080
Speaker 6: translates into two flops flops per dollar. So another target

672
00:34:52,160 --> 00:34:55,440
Speaker 6: you could also go after would be the time to

673
00:34:55,520 --> 00:34:58,279
Speaker 6: respond to one user so to get the answer back.

674
00:34:58,560 --> 00:35:01,960
Speaker 6: One approach is maxim the throughput that you can have

675
00:35:02,040 --> 00:35:05,520
Speaker 6: and others minimizing the latency, So kind of the difference

676
00:35:05,560 --> 00:35:09,040
Speaker 6: between a seven forty seven flying a group of passengers

677
00:35:09,080 --> 00:35:12,719
Speaker 6: across the country versus an SR seventy one getting there

678
00:35:13,000 --> 00:35:15,720
Speaker 6: in a couple hours but only bringing one or two people.

679
00:35:16,120 --> 00:35:18,799
Speaker 2: Let's talk about the business itself. So you know, in

680
00:35:18,840 --> 00:35:22,000
Speaker 2: the old you know, ten years ago, someone starting a

681
00:35:22,360 --> 00:35:26,080
Speaker 2: tech startup, they you know, get three or four people

682
00:35:26,080 --> 00:35:28,120
Speaker 2: in an office and then they write something up. But

683
00:35:28,160 --> 00:35:30,319
Speaker 2: then they have a code and it doesn't maybe they

684
00:35:30,360 --> 00:35:32,719
Speaker 2: don't even have to raise any money to do it,

685
00:35:32,760 --> 00:35:35,600
Speaker 2: and they certainly don't have to depend on whether Taiwan

686
00:35:35,680 --> 00:35:39,319
Speaker 2: Semiconductor has any capacity at their fab or anything like

687
00:35:39,360 --> 00:35:42,600
Speaker 2: this walk us through the sort of nuts and bolts

688
00:35:42,760 --> 00:35:46,280
Speaker 2: of what it actually takes to build a chip business

689
00:35:46,320 --> 00:35:49,799
Speaker 2: from the ground up, both in terms of costs and

690
00:35:50,200 --> 00:35:52,440
Speaker 2: time and what you have to rely on. You know,

691
00:35:52,480 --> 00:35:55,560
Speaker 2: we've talked about some of the design element, what are

692
00:35:55,560 --> 00:35:58,239
Speaker 2: the business side requirements and what will it take to

693
00:35:58,280 --> 00:35:59,080
Speaker 2: actually succeed.

694
00:35:59,800 --> 00:36:05,200
Speaker 6: So fortunately we've kind of referred to this in multiple places.

695
00:36:05,520 --> 00:36:10,239
Speaker 6: There's a huge ecosystem around designing chips. So there's a

696
00:36:10,280 --> 00:36:12,440
Speaker 6: portion you have to do yourself, and there's a portion

697
00:36:12,520 --> 00:36:15,520
Speaker 6: that you can buy, so the placement of Tracy's bits

698
00:36:15,560 --> 00:36:18,280
Speaker 6: and bobs and also the testing that we've talked about.

699
00:36:18,800 --> 00:36:24,080
Speaker 6: There are DA electronic design automation companies that build those tools,

700
00:36:24,680 --> 00:36:28,600
Speaker 6: like there are companies that do just manufacturing, so TSMC

701
00:36:29,800 --> 00:36:34,480
Speaker 6: and their suppliers, and then there are many other other companies.

702
00:36:34,520 --> 00:36:39,440
Speaker 6: So most companies don't go directly to TSMC. So so

703
00:36:39,640 --> 00:36:45,360
Speaker 6: very sophisticated companies like Apple or Nvidia interface directly with them,

704
00:36:45,400 --> 00:36:49,279
Speaker 6: but most other companies go through ACIC vendors. And so

705
00:36:49,440 --> 00:36:52,400
Speaker 6: you know, the prominent companies in the most prominent companies

706
00:36:52,440 --> 00:36:56,399
Speaker 6: in that space are Broadcom and Marvel, and then there

707
00:36:56,400 --> 00:36:59,040
Speaker 6: are a bunch of smaller companies. A couple that are

708
00:36:59,520 --> 00:37:04,319
Speaker 6: close to TSMC are all Chip and GUC and so

709
00:37:04,560 --> 00:37:08,040
Speaker 6: they'll do a lot of the work of taking your

710
00:37:08,160 --> 00:37:11,799
Speaker 6: code and actually getting it placed on the chip. That's

711
00:37:11,800 --> 00:37:15,600
Speaker 6: often a very good thing to outsource because it's the

712
00:37:15,640 --> 00:37:18,160
Speaker 6: work is somewhat seasonal. You're only ready to do that

713
00:37:18,239 --> 00:37:21,880
Speaker 6: placement when you're near the end of this three year project,

714
00:37:22,360 --> 00:37:25,279
Speaker 6: and so you kind of don't have work unless you're

715
00:37:25,280 --> 00:37:29,680
Speaker 6: a massive company for people the whole time. So while

716
00:37:29,840 --> 00:37:32,040
Speaker 6: that ecosystem means that you don't have to hire a

717
00:37:32,040 --> 00:37:35,600
Speaker 6: ton of a huge number of people yourself. All of

718
00:37:35,600 --> 00:37:39,040
Speaker 6: those people have to get paid, and so you do

719
00:37:39,120 --> 00:37:40,800
Speaker 6: have to raise a fair bit of money. And another

720
00:37:40,840 --> 00:37:43,520
Speaker 6: big element of actually thing that you end up spending

721
00:37:43,640 --> 00:37:46,400
Speaker 6: money on is there are parts of the chip that

722
00:37:46,920 --> 00:37:51,680
Speaker 6: are very special, difficult to design and take multiple iterations

723
00:37:51,719 --> 00:37:54,520
Speaker 6: of taping things out and seeing if they work. So

724
00:37:54,960 --> 00:37:58,200
Speaker 6: the very high speed interconnect the connects to get connects

725
00:37:58,239 --> 00:38:02,400
Speaker 6: together chips is an example that. So those are designed

726
00:38:02,400 --> 00:38:06,239
Speaker 6: by yet another set of companies, and the design is

727
00:38:06,239 --> 00:38:08,840
Speaker 6: difficult and fairly expensive because of the need to do

728
00:38:08,920 --> 00:38:13,359
Speaker 6: multiple tapeouts, and so it's very fairly expensive to buy

729
00:38:13,440 --> 00:38:17,279
Speaker 6: that IP. So when you add up the cost of

730
00:38:17,320 --> 00:38:21,560
Speaker 6: the IP, the cost of the ASK vendors services, and

731
00:38:21,600 --> 00:38:27,680
Speaker 6: then the mask fees that TSMC charges using ASMLS and

732
00:38:27,719 --> 00:38:31,799
Speaker 6: ASK creation software, you're talking about tens of millions of

733
00:38:31,800 --> 00:38:34,960
Speaker 6: dollars to bring a state of the art chip to market.

734
00:38:35,000 --> 00:38:38,239
Speaker 6: It's the numbers are much lower for a simpler chip

735
00:38:38,280 --> 00:38:41,080
Speaker 6: on it without the very high speed iOS and on

736
00:38:41,120 --> 00:38:44,799
Speaker 6: an older node, but for an advanced node it's a

737
00:38:44,840 --> 00:38:46,440
Speaker 6: pretty expensive process.

738
00:38:46,680 --> 00:38:48,480
Speaker 3: When do you think you'll be able to bring your

739
00:38:48,520 --> 00:38:49,240
Speaker 3: chips to market.

740
00:38:49,760 --> 00:38:52,879
Speaker 5: Generally, we see these projects taking three to five years

741
00:38:53,440 --> 00:38:56,040
Speaker 5: across most companies. We started on this seriously at the

742
00:38:56,080 --> 00:38:58,680
Speaker 5: beginning of twenty four, so about three years from there

743
00:38:58,760 --> 00:38:59,839
Speaker 5: is likely for us.

744
00:39:00,719 --> 00:39:04,520
Speaker 2: Tell us about what customers because I've heard this, you know,

745
00:39:04,920 --> 00:39:08,520
Speaker 2: we're all trying to find some alternative to video, whether

746
00:39:08,560 --> 00:39:12,800
Speaker 2: it's to reduce energy costs or just reduce costs in general,

747
00:39:13,040 --> 00:39:16,480
Speaker 2: or be able to even access chips at all, since

748
00:39:16,520 --> 00:39:18,480
Speaker 2: not everyone can get them because there are only so

749
00:39:18,480 --> 00:39:20,680
Speaker 2: many chips getting made. But when you talk to like

750
00:39:20,880 --> 00:39:25,759
Speaker 2: theoretical customers, A, who do you imagine as your customers?

751
00:39:25,880 --> 00:39:28,239
Speaker 2: Is it the open eyes of the world, is it

752
00:39:28,360 --> 00:39:31,440
Speaker 2: the metas of the world. Is it labs that we

753
00:39:31,560 --> 00:39:34,600
Speaker 2: haven't heard of yet that could only get into this

754
00:39:35,040 --> 00:39:38,000
Speaker 2: if there were sort of more focused, lower cost options.

755
00:39:38,600 --> 00:39:40,719
Speaker 2: And then b what are they asking for? What do

756
00:39:40,800 --> 00:39:43,360
Speaker 2: they say, like, you know what we're using in video

757
00:39:43,520 --> 00:39:45,840
Speaker 2: right now, but we would really like X or Y

758
00:39:46,120 --> 00:39:47,440
Speaker 2: in the ideal world.

759
00:39:48,160 --> 00:39:50,760
Speaker 5: So there's a range of possible customers in the world.

760
00:39:50,920 --> 00:39:53,440
Speaker 5: The way that we see or away you divide them up,

761
00:39:53,560 --> 00:39:55,799
Speaker 5: and how we choose to do that is what is

762
00:39:55,840 --> 00:39:58,160
Speaker 5: the ratio of engineering time they're putting into their work

763
00:39:58,239 --> 00:40:01,319
Speaker 5: versus the amount of computers spent that they're putting in.

764
00:40:01,800 --> 00:40:05,600
Speaker 5: So the ideal customer in general for a hardware vendor

765
00:40:05,640 --> 00:40:08,920
Speaker 5: who's trying to make the absolute best, but not necessarily

766
00:40:08,960 --> 00:40:12,680
Speaker 5: easiest to use hardware is a company that is spending

767
00:40:12,719 --> 00:40:14,400
Speaker 5: a lot more on their computing power than they are

768
00:40:14,400 --> 00:40:16,680
Speaker 5: spending on the engineering type, because then that makes a

769
00:40:16,719 --> 00:40:18,680
Speaker 5: really good trade off of maybe I can spend a

770
00:40:18,719 --> 00:40:20,759
Speaker 5: bit more engineering time to make your hardware work, but

771
00:40:20,800 --> 00:40:23,839
Speaker 5: I get a big saving on my computing costs. So

772
00:40:24,360 --> 00:40:27,359
Speaker 5: companies like open ai would be obviously a slam dunk.

773
00:40:27,640 --> 00:40:30,640
Speaker 5: There's many more companies as well. So the companies that

774
00:40:30,680 --> 00:40:34,440
Speaker 5: meet this criteria of spending many times more on compute

775
00:40:34,600 --> 00:40:38,359
Speaker 5: than on engineering. There's actually a set of maybe ten

776
00:40:38,360 --> 00:40:41,040
Speaker 5: to fifteen large language model labs that are not as

777
00:40:41,080 --> 00:40:44,719
Speaker 5: well known as open ai, but you might think character Ai, Coheer,

778
00:40:44,760 --> 00:40:48,719
Speaker 5: and many other companies like that in mistrial. So the

779
00:40:48,800 --> 00:40:51,120
Speaker 5: common thing that we hear from those companies, all of

780
00:40:51,120 --> 00:40:53,960
Speaker 5: those are spending hundreds of millions of dollars on compute,

781
00:40:55,239 --> 00:40:59,480
Speaker 5: is I just want better flops for dollar. That's actually

782
00:40:59,480 --> 00:41:03,040
Speaker 5: the single deciding factor, And that's primarily the reason they're

783
00:41:03,040 --> 00:41:07,040
Speaker 5: deciding on today, deciding on in videos products rather than

784
00:41:07,080 --> 00:41:09,280
Speaker 5: some of the other products in the market, is because

785
00:41:09,280 --> 00:41:11,440
Speaker 5: the flops for dollar of those products is the best

786
00:41:11,520 --> 00:41:13,799
Speaker 5: you can buy. But when you give them a spec

787
00:41:13,840 --> 00:41:15,600
Speaker 5: sheet and the first thing they're going to look at

788
00:41:15,680 --> 00:41:17,719
Speaker 5: is just what's the most floating point operations I can

789
00:41:17,760 --> 00:41:20,400
Speaker 5: run on my chip? And then you can rule out

790
00:41:20,440 --> 00:41:22,640
Speaker 5: ninety percent of products there on the basis of okay,

791
00:41:22,760 --> 00:41:25,880
Speaker 5: just doesn't meet that far. But then after that you

792
00:41:25,960 --> 00:41:28,720
Speaker 5: then go through the more detailed analysis of saying, okay, well,

793
00:41:28,880 --> 00:41:31,799
Speaker 5: I've got these floating point operations, but is the rest

794
00:41:31,840 --> 00:41:33,640
Speaker 5: going to work out? Do I have the memory bandwidth

795
00:41:33,719 --> 00:41:36,600
Speaker 5: and the interconnect? But for sure, the number one criteria

796
00:41:36,719 --> 00:41:38,200
Speaker 5: is that top line flops.

797
00:41:38,600 --> 00:41:42,120
Speaker 2: When we talk about delivering more flops per dollar, what

798
00:41:42,160 --> 00:41:46,000
Speaker 2: are you aiming for? What is current benchmark flops per dollar?

799
00:41:46,360 --> 00:41:48,000
Speaker 2: And then are we talking like can it be done

800
00:41:48,120 --> 00:41:51,600
Speaker 2: like ninety percent cheaper? What do you think is realistic

801
00:41:51,640 --> 00:41:54,600
Speaker 2: in terms of coming to market with something meaningfully better

802
00:41:54,640 --> 00:41:55,480
Speaker 2: on that metric?

803
00:41:56,280 --> 00:42:00,120
Speaker 5: So in videos, Blackwell in their FP four format offers

804
00:42:00,680 --> 00:42:03,399
Speaker 5: ten pet of flops in their chip, and that chip

805
00:42:03,440 --> 00:42:08,840
Speaker 5: sells for Bullpark thirty to fifty thousand, depends on many factors.

806
00:42:09,360 --> 00:42:12,800
Speaker 5: That is about a factor of two to four better

807
00:42:13,080 --> 00:42:15,440
Speaker 5: than the previous generation and video chip, which is the

808
00:42:15,480 --> 00:42:18,359
Speaker 5: Hopper chip. So part of that factor is coming from

809
00:42:18,360 --> 00:42:21,040
Speaker 5: going to lower precision, going from eight bit precision to

810
00:42:21,080 --> 00:42:24,480
Speaker 5: four bit precision. In general, precision is in one of

811
00:42:24,520 --> 00:42:27,640
Speaker 5: the best ways to improve the flops you can pack

812
00:42:27,680 --> 00:42:30,040
Speaker 5: into a certain amount of silicon, and then some of

813
00:42:30,040 --> 00:42:31,960
Speaker 5: it is also coming from other factors such as cost

814
00:42:32,000 --> 00:42:35,000
Speaker 5: productions that in Vidia has been deployed. So that's a

815
00:42:35,000 --> 00:42:37,480
Speaker 5: benchmark for ware inn video is that now you need

816
00:42:37,520 --> 00:42:40,120
Speaker 5: to be at least a few integer multiples better than

817
00:42:40,120 --> 00:42:42,160
Speaker 5: that in order to compete with the incumbent. So at

818
00:42:42,239 --> 00:42:45,240
Speaker 5: least you know, two or three times better on that metric,

819
00:42:45,280 --> 00:42:47,520
Speaker 5: we would say. But then, of course, if you're designing

820
00:42:47,520 --> 00:42:49,359
Speaker 5: for the future, you have to compete against the next

821
00:42:49,400 --> 00:42:51,960
Speaker 5: generation after that too, and so you want to be

822
00:42:52,280 --> 00:42:54,839
Speaker 5: many times better than the future chip, which isn't down yet,

823
00:42:54,880 --> 00:42:56,360
Speaker 5: And so that's the thing you aim for.

824
00:42:57,000 --> 00:42:59,760
Speaker 2: Is there anything else that we should sort of understand

825
00:43:00,080 --> 00:43:02,360
Speaker 2: about this business that we haven't touched on that you

826
00:43:02,400 --> 00:43:03,359
Speaker 2: think is important?

827
00:43:03,560 --> 00:43:06,400
Speaker 6: One thing, given that this is odd lots that I

828
00:43:06,440 --> 00:43:09,360
Speaker 6: think the reason that sam Altman is going around the

829
00:43:09,360 --> 00:43:12,839
Speaker 6: world talking about trillions of dollars of spend is that

830
00:43:12,920 --> 00:43:16,120
Speaker 6: he wants to move the expectations of all of the

831
00:43:16,160 --> 00:43:20,719
Speaker 6: suppliers up. So as you have we've observed in the

832
00:43:20,800 --> 00:43:26,160
Speaker 6: semiconductor shortage, if the suppliers are preparing for a certain

833
00:43:26,200 --> 00:43:28,600
Speaker 6: amount of demand and demand you know, in the case

834
00:43:29,640 --> 00:43:33,240
Speaker 6: famously of the auto manufacturers as a result of COVID

835
00:43:33,719 --> 00:43:37,040
Speaker 6: canceled their orders and then they found that demand was much, much,

836
00:43:37,160 --> 00:43:40,279
Speaker 6: much larger than they expected. It took a very long

837
00:43:40,360 --> 00:43:44,640
Speaker 6: time to catch up. A similar thing happened with the

838
00:43:45,000 --> 00:43:48,600
Speaker 6: in videos H one hundred. So TSMC was actually perfectly

839
00:43:48,640 --> 00:43:51,839
Speaker 6: capable of keeping up with demand for the chips themselves.

840
00:43:52,280 --> 00:43:57,279
Speaker 6: But the chips for these AI products are use a

841
00:43:57,360 --> 00:44:01,040
Speaker 6: very special kind of packaging which puts the compute chips

842
00:44:01,120 --> 00:44:03,239
Speaker 6: very close to the memory chips and hence allows them

843
00:44:03,280 --> 00:44:08,719
Speaker 6: to communicate very quickly, called coos, And the capacity for

844
00:44:08,840 --> 00:44:14,040
Speaker 6: coos was limited because TSMC built with a particular expectation

845
00:44:14,160 --> 00:44:17,520
Speaker 6: of demand, and when H one hundred became such a

846
00:44:17,560 --> 00:44:22,520
Speaker 6: monster product, their coosts capacity wasn't able to keep pace

847
00:44:22,680 --> 00:44:26,440
Speaker 6: with demand. So, you know, supply chain tends to be

848
00:44:26,560 --> 00:44:31,200
Speaker 6: really good if you predict accurately, and if you predict badly,

849
00:44:31,480 --> 00:44:33,920
Speaker 6: you know, on on the low side, then you end

850
00:44:34,000 --> 00:44:37,520
Speaker 6: up with these shortages. But on the other hand, these companies,

851
00:44:37,680 --> 00:44:41,600
Speaker 6: because the manufacturing companies have very high capex, they are

852
00:44:41,640 --> 00:44:44,759
Speaker 6: fairly lows to it, predict badly on the high side

853
00:44:44,800 --> 00:44:48,040
Speaker 6: because that leads them to having spent a bunch of

854
00:44:48,040 --> 00:44:50,920
Speaker 6: money on capital capex that they're unable to recover.

855
00:44:51,520 --> 00:44:54,879
Speaker 2: So, yeah, this is very interesting, this idea that in

856
00:44:54,920 --> 00:44:58,960
Speaker 2: some part it's a signal we're not slowing down. We're

857
00:44:59,040 --> 00:45:01,000
Speaker 2: you know, we have more and more that we want

858
00:45:01,040 --> 00:45:04,759
Speaker 2: to do. So if you're anywhere along the semiconductor supply chain,

859
00:45:05,280 --> 00:45:08,359
Speaker 2: don't start, you know, curbing your expectations or curbing your

860
00:45:08,360 --> 00:45:10,960
Speaker 2: production because we want to build a lot more. I'm

861
00:45:11,000 --> 00:45:14,120
Speaker 2: curious one last question, I guess for both of you.

862
00:45:14,120 --> 00:45:15,880
Speaker 2: You know, you hear a lot of people in the

863
00:45:15,960 --> 00:45:18,840
Speaker 2: industry you say, like, we might just be three or

864
00:45:18,880 --> 00:45:24,400
Speaker 2: four years away from AGI or super intelligence, however that's defined,

865
00:45:24,800 --> 00:45:27,000
Speaker 2: and then you get into a lot of these philosophical

866
00:45:27,120 --> 00:45:30,000
Speaker 2: questions and ethical questions about you know, whatever is the

867
00:45:30,080 --> 00:45:32,440
Speaker 2: AI going to, well, it's gonna be the role for

868
00:45:32,520 --> 00:45:34,520
Speaker 2: humans or is it gonna kill us all? Or whatever

869
00:45:35,160 --> 00:45:37,640
Speaker 2: you know, fear scenario you want. But the two of

870
00:45:37,680 --> 00:45:39,879
Speaker 2: you like, how do you see that question? Like could

871
00:45:39,960 --> 00:45:42,680
Speaker 2: we hit it in just a few short years where

872
00:45:43,200 --> 00:45:45,840
Speaker 2: we have something that people agree is oh, this is

873
00:45:45,960 --> 00:45:49,360
Speaker 2: agi Like are you is it short runway or just

874
00:45:49,440 --> 00:45:51,200
Speaker 2: a couple of years away from this or does it

875
00:45:51,200 --> 00:45:53,719
Speaker 2: feel like no, that's still quite a few years out.

876
00:45:53,960 --> 00:45:56,560
Speaker 5: If ever, I think what we have what's your.

877
00:46:00,560 --> 00:46:06,480
Speaker 6: Approximately zero to be blunt? Thank you, my p great things.

878
00:46:06,640 --> 00:46:09,480
Speaker 6: I mean, I think we kind of already have great things,

879
00:46:09,480 --> 00:46:12,960
Speaker 6: and we've just gotten the models of this level of

880
00:46:13,000 --> 00:46:15,440
Speaker 6: quality recently and we're learning how to use them, and

881
00:46:15,480 --> 00:46:19,520
Speaker 6: the quality is going up. The you know, the fact

882
00:46:19,520 --> 00:46:21,960
Speaker 6: that we can get a computer to write code pretty

883
00:46:21,960 --> 00:46:26,160
Speaker 6: well is fairly amazing to me. That you can ask

884
00:46:26,200 --> 00:46:28,520
Speaker 6: it to tell a good joke in the style of

885
00:46:28,560 --> 00:46:32,760
Speaker 6: a particular person and it can do that is also amazing. Yeah.

886
00:46:32,840 --> 00:46:36,160
Speaker 2: Well, uh, I'm glad, You're I'm glad, you're I'm glad

887
00:46:36,200 --> 00:46:39,239
Speaker 2: your odds of total doom and annihilation are zero. That

888
00:46:39,320 --> 00:46:41,799
Speaker 2: makes me feel a little bit better. Ryan or and Mike,

889
00:46:41,840 --> 00:46:43,560
Speaker 2: thank you so much for coming on odd laws.

890
00:46:43,600 --> 00:47:02,600
Speaker 7: I learned as from that conversation there's a pleasure.

891
00:46:58,840 --> 00:46:59,240
Speaker 4: Tracy.

892
00:46:59,239 --> 00:47:02,920
Speaker 2: There was obviously ton that was really interesting in that conversation,

893
00:47:03,000 --> 00:47:07,000
Speaker 2: but I particularly like the part about incentives of large

894
00:47:07,200 --> 00:47:11,360
Speaker 2: legacy incumbents about entering a totally new business. So for

895
00:47:11,400 --> 00:47:15,720
Speaker 2: a company like Google, the primary purpose of their chips

896
00:47:16,120 --> 00:47:19,720
Speaker 2: is going to be serving an in house business purpose.

897
00:47:19,760 --> 00:47:21,600
Speaker 2: And even with all the money that they have, and

898
00:47:21,640 --> 00:47:24,920
Speaker 2: even with the engineering talent, there's still a sort of

899
00:47:25,080 --> 00:47:28,359
Speaker 2: trade off question involved of how much do we want

900
00:47:28,440 --> 00:47:31,839
Speaker 2: to build chips for some other purpose, for some sort

901
00:47:31,880 --> 00:47:33,000
Speaker 2: of external service.

902
00:47:33,120 --> 00:47:36,120
Speaker 3: Yeah, and I also thought the point about why Sam

903
00:47:36,160 --> 00:47:39,120
Speaker 3: Altman is going around talking about how, you know, how

904
00:47:39,160 --> 00:47:42,319
Speaker 3: many billions he's going to spend was really interesting and

905
00:47:42,480 --> 00:47:44,960
Speaker 3: it kind of makes sense in the aftermath of the

906
00:47:45,000 --> 00:47:49,000
Speaker 3: pandemic and semiconductors. I'm sure you remember this. I think

907
00:47:49,040 --> 00:47:51,600
Speaker 3: that was actually where we first learned about the bullwhip

908
00:47:51,640 --> 00:47:55,040
Speaker 3: effect and this idea that very small changes in one

909
00:47:55,280 --> 00:47:57,880
Speaker 3: end of the supply chain, which would be customer demand,

910
00:47:58,160 --> 00:48:02,000
Speaker 3: can end up reverberate, you know, all the way through

911
00:48:02,040 --> 00:48:05,440
Speaker 3: the supply chain. And so when you had carmakers start

912
00:48:05,440 --> 00:48:08,280
Speaker 3: to cut back on their orders. That had a much

913
00:48:08,320 --> 00:48:12,000
Speaker 3: bigger and longer impact than you might have anticipated. And

914
00:48:12,040 --> 00:48:15,000
Speaker 3: so it's interesting to see companies coming at it from

915
00:48:15,040 --> 00:48:17,799
Speaker 3: the other end and saying like, no, we have all

916
00:48:17,840 --> 00:48:19,640
Speaker 3: this money and we're going to be here for a

917
00:48:19,680 --> 00:48:20,240
Speaker 3: long time.

918
00:48:20,480 --> 00:48:23,480
Speaker 2: We're not slowing down. We are going to agi. And

919
00:48:23,560 --> 00:48:25,680
Speaker 2: so if you think like, oh, we're gonna come out

920
00:48:25,719 --> 00:48:28,360
Speaker 2: with GPT five and then we're going to focus on

921
00:48:28,480 --> 00:48:31,719
Speaker 2: just like commercializing that and selling it to airlines to

922
00:48:31,719 --> 00:48:34,560
Speaker 2: do customer support after that, and just go into glide

923
00:48:34,600 --> 00:48:37,120
Speaker 2: mode and take business like they want to signal that

924
00:48:37,160 --> 00:48:39,600
Speaker 2: they're like building more and more and more. I thought

925
00:48:39,600 --> 00:48:42,400
Speaker 2: that was interesting. I thought it was interesting the point

926
00:48:42,480 --> 00:48:47,160
Speaker 2: about Nvidia and Kuda and the idea that, Okay, yes,

927
00:48:47,520 --> 00:48:51,400
Speaker 2: the Kuda software ecosystem is perceived to be this mote

928
00:48:51,400 --> 00:48:55,000
Speaker 2: that makes it harder for other semiconductor companies to break

929
00:48:55,080 --> 00:48:58,680
Speaker 2: into the same business, but it's also constraining from an

930
00:48:58,680 --> 00:49:01,839
Speaker 2: in video perspective, the idea that, Okay, if they want

931
00:49:01,920 --> 00:49:06,280
Speaker 2: everything to be Kuda compatible or be within the same

932
00:49:06,480 --> 00:49:10,799
Speaker 2: family of software usage, then that also constrains the potential

933
00:49:11,160 --> 00:49:13,400
Speaker 2: sidelines that they might get into right.

934
00:49:13,280 --> 00:49:16,200
Speaker 3: And opens up space for competitors. But I don't know

935
00:49:16,239 --> 00:49:20,080
Speaker 3: why I haven't really like internalized this lesson before, because

936
00:49:20,120 --> 00:49:24,360
Speaker 3: it comes up in every conversation we do on semiconductors.

937
00:49:24,440 --> 00:49:26,919
Speaker 3: But I think there's still a perception, or at least

938
00:49:26,960 --> 00:49:29,520
Speaker 3: maybe I still have this perception that the moat around

939
00:49:29,600 --> 00:49:32,319
Speaker 3: Nvidia is like the actual hardware. Yes, but it's not.

940
00:49:32,640 --> 00:49:34,560
Speaker 3: It's the software. It's Kuda.

941
00:49:34,840 --> 00:49:35,800
Speaker 2: It seems like it's both.

942
00:49:36,080 --> 00:49:39,160
Speaker 3: Well, yeah, but I think I'm starting to appreciate how

943
00:49:39,239 --> 00:49:41,120
Speaker 3: much of it is Kuda is what I'm.

944
00:49:40,960 --> 00:49:43,960
Speaker 2: Saying it certainly, it certainly seems to come up over

945
00:49:44,120 --> 00:49:47,400
Speaker 2: and over again. How much the fact that this is

946
00:49:47,440 --> 00:49:50,920
Speaker 2: what people use. It's the software that makes it easy

947
00:49:51,000 --> 00:49:56,360
Speaker 2: for less sophistic less sophisticated customers to use the applications.

948
00:49:56,520 --> 00:49:59,439
Speaker 2: It seems extremely powerful. It's also interesting to hear about

949
00:49:59,480 --> 00:50:05,880
Speaker 2: like the ecosystem of businesses around semiconductor design. And you know,

950
00:50:06,120 --> 00:50:10,120
Speaker 2: he mentioned Broadcom. Ryner mentioned Broadcom, which is a company

951
00:50:10,160 --> 00:50:12,759
Speaker 2: that I don't think we've ever really talked about very

952
00:50:12,840 --> 00:50:15,880
Speaker 2: much on the show. But if you look at that stock,

953
00:50:16,280 --> 00:50:18,880
Speaker 2: I mean, it looks kind of like you're looking at

954
00:50:18,920 --> 00:50:20,960
Speaker 2: a chart of in video like that has been a

955
00:50:21,040 --> 00:50:25,640
Speaker 2: gigantic winner over the last few years. Back in twenty twenty,

956
00:50:25,800 --> 00:50:27,600
Speaker 2: it was a thirty one dollars stock. Now it's one

957
00:50:27,680 --> 00:50:29,800
Speaker 2: hundred and forty six dollars stock. Okay, I tell you

958
00:50:29,800 --> 00:50:32,560
Speaker 2: a five back or so, maybe not quite in video returns.

959
00:50:33,000 --> 00:50:35,799
Speaker 3: And this idea that, like how in Vidia has just

960
00:50:35,880 --> 00:50:39,719
Speaker 3: skewed like I know what's expected of every stock, it's like,

961
00:50:40,080 --> 00:50:41,600
Speaker 3: but this is on a different plane.

962
00:50:41,719 --> 00:50:46,040
Speaker 2: And this idea that a semiconductor startup doesn't necessarily interface

963
00:50:46,120 --> 00:50:50,440
Speaker 2: directly with TSMC like that really for the most sophisticated advance,

964
00:50:50,480 --> 00:50:52,560
Speaker 2: and then there are some of these companies in the middle.

965
00:50:52,640 --> 00:50:54,080
Speaker 2: I thought that was extremely interesting.

966
00:50:54,320 --> 00:50:57,160
Speaker 3: Uh you know what, Joe, I asked chat GPT, what

967
00:50:57,280 --> 00:51:03,080
Speaker 3: the most beautiful semiconductor is. Yeah, it says Gallium arsenide

968
00:51:03,440 --> 00:51:09,080
Speaker 3: is considered beautiful for several reasons. It's crystal structure is

969
00:51:09,160 --> 00:51:12,600
Speaker 3: often admired for its clarity and elegance. Wow, So I

970
00:51:12,600 --> 00:51:15,399
Speaker 3: guess semiconductors may solum arsenide, So.

971
00:51:16,320 --> 00:51:19,600
Speaker 2: There's beauty at the molecular level. Yeah, But actually I thought,

972
00:51:19,680 --> 00:51:22,000
Speaker 2: you know, I thought when you asked that question, it's like, oh,

973
00:51:22,040 --> 00:51:25,440
Speaker 2: it's just sort of a you know, philosophical, you know,

974
00:51:25,600 --> 00:51:29,960
Speaker 2: fun whimsical question, but this idea of like doing the

975
00:51:30,040 --> 00:51:33,360
Speaker 2: minimum required or not building a bunch of extra rooms

976
00:51:33,440 --> 00:51:36,239
Speaker 2: in the house that you don't really need. And as

977
00:51:36,280 --> 00:51:38,959
Speaker 2: we know, I mean, it's just objectively true that even

978
00:51:38,960 --> 00:51:41,080
Speaker 2: if in video chips are the best in the world

979
00:51:41,280 --> 00:51:44,879
Speaker 2: for AI, they do other stuff beyond AI, and they

980
00:51:44,920 --> 00:51:47,840
Speaker 2: do ethereum mining, or they used to, and that was

981
00:51:48,160 --> 00:51:50,479
Speaker 2: based on proof of work back in the old days.

982
00:51:50,520 --> 00:51:52,320
Speaker 2: And of course they're for video games. But if you

983
00:51:52,400 --> 00:51:55,560
Speaker 2: really just want a computer, or if you really just

984
00:51:55,600 --> 00:51:59,520
Speaker 2: want a model that can speak in English or write code,

985
00:52:00,200 --> 00:52:04,240
Speaker 2: or can just think without doing video games and chip mining,

986
00:52:04,560 --> 00:52:06,440
Speaker 2: then perhaps there are a bunch of rooms in the

987
00:52:06,440 --> 00:52:07,920
Speaker 2: house that are totally unnecessary.

988
00:52:08,080 --> 00:52:11,480
Speaker 3: Yeah, And I mean there's efficiency costs to that efficiency cost. Yeah,

989
00:52:11,680 --> 00:52:14,759
Speaker 3: you're trying to streamline it as much as possible. All right,

990
00:52:14,800 --> 00:52:15,520
Speaker 3: shall we leave it there.

991
00:52:15,600 --> 00:52:16,279
Speaker 2: Let's leave it there.

992
00:52:16,480 --> 00:52:19,280
Speaker 3: This has been another episode of the All Thoughts podcast.

993
00:52:19,360 --> 00:52:22,760
Speaker 3: I'm Tracy Alloway. You can follow me at Tracy Alloway.

994
00:52:22,480 --> 00:52:25,040
Speaker 2: And I'm Jill Wisenthal. You can follow me at the Stalwart.

995
00:52:25,280 --> 00:52:28,200
Speaker 2: Follow our guests Rein or Pope. He's at rein Or

996
00:52:28,280 --> 00:52:32,400
Speaker 2: Pope and Mike Gunter. He's Mike Gunter Underscore. Follow our

997
00:52:32,400 --> 00:52:35,920
Speaker 2: producers Carmen Rodriguez at Carman Ermann dash Oll, Bennett at

998
00:52:36,000 --> 00:52:39,520
Speaker 2: dashbot In Kelbrooks at Kelbrooks. Thank you to our producer

999
00:52:39,560 --> 00:52:42,919
Speaker 2: Moses Ondam. For more Oddlots content, go to Bloomberg dot

1000
00:52:42,920 --> 00:52:46,040
Speaker 2: com slash odd Lots, where we have transcripts, a blog,

1001
00:52:46,120 --> 00:52:48,400
Speaker 2: and a newsletter and you can chat about all of

1002
00:52:48,440 --> 00:52:51,480
Speaker 2: these topics twenty four to seven in the discord Discord

1003
00:52:51,520 --> 00:52:55,239
Speaker 2: dot gg slash odd Laws. There's even a semiconductor room

1004
00:52:55,239 --> 00:52:57,480
Speaker 2: in there, so you can just go there and just

1005
00:52:57,520 --> 00:52:59,040
Speaker 2: talk about chips all day if you want.

1006
00:53:00,040 --> 00:53:02,399
Speaker 3: If you enjoy All Lots, if you like it when

1007
00:53:02,440 --> 00:53:05,359
Speaker 3: we talk about what the most beautiful semiconductor is, then

1008
00:53:05,400 --> 00:53:09,200
Speaker 3: please leave us a positive review on your favorite podcast platform.

1009
00:53:09,560 --> 00:53:12,359
Speaker 3: And remember, if you're a Bloomberg subscriber, you can listen

1010
00:53:12,400 --> 00:53:15,520
Speaker 3: to all of our episodes absolutely ad free. All you

1011
00:53:15,600 --> 00:53:18,960
Speaker 3: need to do is connect your Bloomberg account with Apple Podcasts.

1012
00:53:19,239 --> 00:53:21,640
Speaker 3: In order to do that, just find the Bloomberg channel

1013
00:53:21,719 --> 00:53:41,200
Speaker 3: on the platform and follow the instructions there. Thanks for listening.