1
00:00:02,680 --> 00:00:19,919
Speaker 1: Bloomberg Audio Studios, Podcasts, radio News. Hello and welcome to

2
00:00:19,960 --> 00:00:23,600
Speaker 1: another episode of the Odd Blots podcast. I'm Tracy Alloway.

3
00:00:23,320 --> 00:00:24,520
Speaker 2: And I'm Joe Wisenthal.

4
00:00:24,800 --> 00:00:28,920
Speaker 1: Joe, what's been your favorite chat GPT or claude prompt

5
00:00:29,040 --> 00:00:29,400
Speaker 1: so far?

6
00:00:31,320 --> 00:00:33,040
Speaker 3: You know, it's funny because I have a lot of

7
00:00:33,040 --> 00:00:37,040
Speaker 3: fun with them, and also I use them for serious things.

8
00:00:37,040 --> 00:00:40,560
Speaker 3: So I'll like upload conference call transcripts and say, tell

9
00:00:40,600 --> 00:00:44,440
Speaker 3: me what this company said about labor market indicators or

10
00:00:44,520 --> 00:00:46,880
Speaker 3: something like that, and that'll be extremely useful for that.

11
00:00:47,159 --> 00:00:49,519
Speaker 1: Wait, do you actually find that more efficient than just

12
00:00:49,520 --> 00:00:52,720
Speaker 1: doing a word search for like labor or working? I don't.

13
00:00:52,760 --> 00:00:54,760
Speaker 1: I hate uploading stuff because you can only do it

14
00:00:54,760 --> 00:00:55,840
Speaker 1: in like fragments.

15
00:00:56,120 --> 00:01:01,960
Speaker 3: No, what, Tracy, Oh, let me, I'll show you how prompt? Okay, No,

16
00:01:02,160 --> 00:01:05,040
Speaker 3: I get a lot of professional use out of the

17
00:01:05,160 --> 00:01:08,080
Speaker 3: various AI tools, but I also, you know, have a

18
00:01:08,120 --> 00:01:11,039
Speaker 3: lot of fun with them. And there's even a song

19
00:01:11,160 --> 00:01:13,480
Speaker 3: and I'm not going to say which one that I wrote.

20
00:01:13,880 --> 00:01:16,840
Speaker 3: I didn't use the lyrics. No, I did not like

21
00:01:16,880 --> 00:01:18,800
Speaker 3: because it's very good. Wait what did you use?

22
00:01:18,920 --> 00:01:21,080
Speaker 1: Did he give you an actual melody? What happened?

23
00:01:21,160 --> 00:01:21,280
Speaker 4: No?

24
00:01:21,560 --> 00:01:24,720
Speaker 3: So there was a song that I liked, okay, and

25
00:01:24,840 --> 00:01:28,600
Speaker 3: the song title sort of rested upon a pun okay,

26
00:01:28,959 --> 00:01:31,800
Speaker 3: and so I asked chat GPT to come up with

27
00:01:31,920 --> 00:01:38,040
Speaker 3: another song that sort of like had a similar twist

28
00:01:38,240 --> 00:01:40,880
Speaker 3: based on the headline of that song. I needed basically

29
00:01:40,920 --> 00:01:43,000
Speaker 3: a song prompt idea.

30
00:01:43,240 --> 00:01:45,800
Speaker 1: This opens up a whole can of worms. No, this

31
00:01:45,920 --> 00:01:48,600
Speaker 1: is actually the perfect segue into what we're going to

32
00:01:48,680 --> 00:01:52,400
Speaker 1: talk about today, because for you and I, using something

33
00:01:53,040 --> 00:01:56,400
Speaker 1: like a chat GPT, we don't really have the same

34
00:01:56,520 --> 00:02:01,800
Speaker 1: concerns that a proper company or large which corporation would have, Like,

35
00:02:01,840 --> 00:02:04,640
Speaker 1: it doesn't really matter to us if the answer is wrong.

36
00:02:04,720 --> 00:02:06,680
Speaker 1: I mean, ideally you would like it to be correct,

37
00:02:06,680 --> 00:02:10,000
Speaker 1: but if I'm just asking some silly question, it doesn't

38
00:02:10,000 --> 00:02:12,560
Speaker 1: really matter what chat gpt spits out at me. And

39
00:02:12,680 --> 00:02:16,119
Speaker 1: also copyright kind of doesn't matter, so we don't care

40
00:02:16,320 --> 00:02:18,400
Speaker 1: what it spits out in terms of who owns it,

41
00:02:18,440 --> 00:02:21,040
Speaker 1: and also we don't care what we're putting in in

42
00:02:21,160 --> 00:02:23,920
Speaker 1: terms of who owns that. That's right, But if you

43
00:02:23,960 --> 00:02:27,919
Speaker 1: are a company you are thinking about generative AI very differently.

44
00:02:28,280 --> 00:02:30,040
Speaker 2: I just want to say one thing, which is.

45
00:02:29,960 --> 00:02:32,200
Speaker 1: That your defense Okay, defend yourself.

46
00:02:32,240 --> 00:02:35,320
Speaker 3: No, No, I'm not even trying to defend myself. If I upload, say,

47
00:02:35,760 --> 00:02:38,000
Speaker 3: you know, the McDonald's earning transcript, and I say, what

48
00:02:38,040 --> 00:02:40,720
Speaker 3: does McDonald say about the labor market, then there's some quote.

49
00:02:41,040 --> 00:02:43,440
Speaker 3: I always go back and check that that quote is

50
00:02:43,520 --> 00:02:45,799
Speaker 3: actually in there. So I do very good, you know,

51
00:02:45,840 --> 00:02:49,040
Speaker 3: I'm not just blindly relying on it. I do also

52
00:02:49,320 --> 00:02:52,120
Speaker 3: do my own work and everything. But yeah, it's very true.

53
00:02:52,160 --> 00:02:54,480
Speaker 3: Like so I can say I get a tremendous amount

54
00:02:54,480 --> 00:02:56,840
Speaker 3: of use from chat, GPT or Claude or whatever, and

55
00:02:56,880 --> 00:03:00,600
Speaker 3: it is very useful to me. But it makes mistakes sometimes,

56
00:03:00,880 --> 00:03:03,920
Speaker 3: and if you think about deploying AI in the sort

57
00:03:03,960 --> 00:03:08,720
Speaker 3: of enterprise world, then maybe like a one percent mistake

58
00:03:08,840 --> 00:03:11,799
Speaker 3: raid or a one percent hallucination or you ever want

59
00:03:11,800 --> 00:03:14,960
Speaker 3: to call them, is just completely unacceptable and a level

60
00:03:15,000 --> 00:03:19,880
Speaker 3: of risk that makes it almost unusual for professional purposes.

61
00:03:19,919 --> 00:03:22,480
Speaker 1: Absolutely. And of course the other thing with AI is

62
00:03:22,639 --> 00:03:25,880
Speaker 1: there is still this ongoing, very heated debate about how

63
00:03:25,919 --> 00:03:29,200
Speaker 1: transformational it's actually going to be. So you and I

64
00:03:29,400 --> 00:03:32,160
Speaker 1: are using it as you know, a productivity hack in

65
00:03:32,240 --> 00:03:35,880
Speaker 1: some cases, or maybe to generate song lyrics or even

66
00:03:36,280 --> 00:03:41,720
Speaker 1: songs in some cases, but what is the true use

67
00:03:41,840 --> 00:03:44,480
Speaker 1: case for this particular technology. There's still a lot of

68
00:03:44,480 --> 00:03:47,800
Speaker 1: debate about that, and so I'm very pleased to say

69
00:03:47,840 --> 00:03:50,400
Speaker 1: we do, in fact have the perfect guest. We're going

70
00:03:50,440 --> 00:03:54,280
Speaker 1: to be speaking to someone who is implementing AI at

71
00:03:54,280 --> 00:03:57,080
Speaker 1: a very, very large financial institution. We're going to be

72
00:03:57,080 --> 00:04:02,080
Speaker 1: speaking with Marco Urgenti, the chief information officer at Goldman Sachs. Marco,

73
00:04:02,160 --> 00:04:03,760
Speaker 1: thank you so much for coming on of thoughts.

74
00:04:04,200 --> 00:04:05,200
Speaker 4: Thank you for having me.

75
00:04:05,600 --> 00:04:08,680
Speaker 1: Marco tell us what a chief information officer does at

76
00:04:08,680 --> 00:04:11,920
Speaker 1: Goldman Sachs. Whenever I see CIO, I always think chief

77
00:04:11,920 --> 00:04:15,240
Speaker 1: investment officer, as it's very confusing. Yeah, so what does

78
00:04:15,280 --> 00:04:16,640
Speaker 1: the other CIO do?

79
00:04:17,480 --> 00:04:19,880
Speaker 4: So last week I was in Italy visiting my mother.

80
00:04:20,200 --> 00:04:23,320
Speaker 4: She's eighty three, and she obviously doesn't know much about

81
00:04:23,360 --> 00:04:26,800
Speaker 4: technology or banking, and so she said, what do you

82
00:04:26,880 --> 00:04:29,080
Speaker 4: do with Coleman? And I said, you know, I just

83
00:04:29,120 --> 00:04:31,320
Speaker 4: tried to simplify. I say, make sure that the printers

84
00:04:31,320 --> 00:04:36,880
Speaker 4: don't run out of And interestingly, the CIO job has

85
00:04:36,920 --> 00:04:40,359
Speaker 4: been traditionally associated with the word it.

86
00:04:41,080 --> 00:04:42,360
Speaker 2: Okay and it.

87
00:04:42,640 --> 00:04:45,200
Speaker 4: I tell you, talk to any technologist, they don't want

88
00:04:45,200 --> 00:04:46,400
Speaker 4: to be classified as IT.

89
00:04:47,320 --> 00:04:49,880
Speaker 3: Right, because those are you associated with those are the

90
00:04:49,920 --> 00:04:51,960
Speaker 3: people who like, see if the ethernet cable with.

91
00:04:51,960 --> 00:04:54,159
Speaker 4: Those are the ones who tell you that those that

92
00:04:54,600 --> 00:04:56,480
Speaker 4: you know, I mean, I have a lot of respect

93
00:04:56,520 --> 00:04:59,000
Speaker 4: for it, but generally you go to the IT department

94
00:04:59,000 --> 00:05:02,440
Speaker 4: when something doesn't work, okay, And so it's very back

95
00:05:02,520 --> 00:05:06,400
Speaker 4: office and something that attracted me to this job. I've

96
00:05:06,440 --> 00:05:08,080
Speaker 4: been here for five years and this is the first

97
00:05:08,120 --> 00:05:10,240
Speaker 4: time that I do like a CIO job. Before I

98
00:05:10,279 --> 00:05:12,880
Speaker 4: was doing more like, you know, creating technology, et cetera,

99
00:05:12,920 --> 00:05:15,200
Speaker 4: and service. I can talk about that, but is the

100
00:05:15,240 --> 00:05:17,080
Speaker 4: fact that the role of a CEO has actually changed

101
00:05:17,160 --> 00:05:21,599
Speaker 4: quite a bit, and now it's about really asking the question,

102
00:05:21,800 --> 00:05:26,760
Speaker 4: you know, how do we implement technology in order to

103
00:05:26,960 --> 00:05:30,480
Speaker 4: achieve our strategic objectives and actually to be differentiated, And

104
00:05:30,520 --> 00:05:33,520
Speaker 4: it's really sitting at the strategic table of the firm.

105
00:05:33,560 --> 00:05:33,880
Speaker 2: Okay.

106
00:05:34,760 --> 00:05:37,440
Speaker 4: So today we live in a world where obviously a

107
00:05:37,480 --> 00:05:39,279
Speaker 4: lot of the things that we want to do, or

108
00:05:39,360 --> 00:05:42,400
Speaker 4: every company wants to do, are really kind of determined

109
00:05:42,400 --> 00:05:45,599
Speaker 4: by how good you are at technology. And so I

110
00:05:45,640 --> 00:05:48,080
Speaker 4: think the role of the CIO has changed quite a bit.

111
00:05:48,200 --> 00:05:50,680
Speaker 4: And now, you know, I would define it as in general,

112
00:05:51,279 --> 00:05:54,159
Speaker 4: defining the technology strategy of a firm and also making

113
00:05:54,160 --> 00:05:56,520
Speaker 4: sure that you have the right culture in the engineering

114
00:05:56,560 --> 00:05:58,240
Speaker 4: team in order to execute on that.

115
00:05:58,600 --> 00:06:00,880
Speaker 3: What's the day to day look like? Like, what's the

116
00:06:00,920 --> 00:06:03,360
Speaker 3: typical day you get into the office and then what.

117
00:06:03,320 --> 00:06:03,599
Speaker 2: Do you do?

118
00:06:04,120 --> 00:06:06,880
Speaker 4: Well? I mean I get into the office, and I generally,

119
00:06:07,160 --> 00:06:09,440
Speaker 4: like everybody else, you know, I talk to people every

120
00:06:09,520 --> 00:06:11,599
Speaker 4: day all day, and so I talk to people. You know,

121
00:06:11,640 --> 00:06:13,640
Speaker 4: we have a bunch of meetings one after the other. End.

122
00:06:13,640 --> 00:06:16,600
Speaker 4: I have teams coming to me with either regularly scheduled

123
00:06:16,600 --> 00:06:19,800
Speaker 4: meetings or meetings that have been requested to discuss a

124
00:06:19,839 --> 00:06:23,440
Speaker 4: certain topic. And you know, we just go through is

125
00:06:23,480 --> 00:06:27,159
Speaker 4: there a whiteboard? Well right now in the age of Zoom,

126
00:06:27,360 --> 00:06:29,880
Speaker 4: I guess still. You know, we have a globally distributed

127
00:06:29,880 --> 00:06:31,479
Speaker 4: team and so a lot of our people are not

128
00:06:31,600 --> 00:06:33,880
Speaker 4: in the same office, and so we use virtual whiteboards

129
00:06:33,920 --> 00:06:36,640
Speaker 4: like everybody else. But I would say, you know, one

130
00:06:36,680 --> 00:06:39,280
Speaker 4: of the things that I tried to do while joining Golma,

131
00:06:39,320 --> 00:06:41,960
Speaker 4: which was part of sort of the cultural agen that

132
00:06:42,160 --> 00:06:48,760
Speaker 4: was emphasizing the importance of narratives and written world versus

133
00:06:48,800 --> 00:06:51,920
Speaker 4: you know, PowerPoint and talking. Okay, so, which is kind

134
00:06:51,920 --> 00:06:54,000
Speaker 4: of what I learned that Amazon over the years. Okay,

135
00:06:54,080 --> 00:06:56,880
Speaker 4: all right, w I was a REDWS and one of

136
00:06:56,880 --> 00:06:59,200
Speaker 4: the things you learned there as soon as you join Amazon,

137
00:06:59,279 --> 00:07:03,680
Speaker 4: in any part of Amazon, like the first few meetings

138
00:07:03,680 --> 00:07:07,160
Speaker 4: are kind of shocking because nobody talks. Everybody starts reading.

139
00:07:07,560 --> 00:07:10,559
Speaker 4: You start reading for like sometimes thirty minutes or forty

140
00:07:10,600 --> 00:07:14,480
Speaker 4: five minutes, and if you're the author of the document,

141
00:07:15,000 --> 00:07:18,000
Speaker 4: you're just sitting there basically, and you just try to

142
00:07:18,080 --> 00:07:20,760
Speaker 4: look at people's faces and understand what they think about

143
00:07:20,800 --> 00:07:22,840
Speaker 4: your document. And sometimes, you know, if you're with Jeff

144
00:07:22,880 --> 00:07:25,720
Speaker 4: Bezos or others, you know, at that time it can

145
00:07:25,760 --> 00:07:29,480
Speaker 4: be pretty pretty terrifying. And so this kind of shift

146
00:07:29,760 --> 00:07:34,240
Speaker 4: from a culture of people talk, people comment on a PowerPoint,

147
00:07:34,280 --> 00:07:37,720
Speaker 4: and the discussion sometimes get you know, driven by who

148
00:07:37,720 --> 00:07:40,520
Speaker 4: has the stronger personality versus, you know, who has the

149
00:07:40,520 --> 00:07:43,600
Speaker 4: greatest ideas. One of the things that I try to

150
00:07:43,720 --> 00:07:45,280
Speaker 4: change is that a lot of the meetings that we

151
00:07:45,360 --> 00:07:49,120
Speaker 4: do today actually start the same way by reading a document.

152
00:07:49,880 --> 00:07:51,920
Speaker 4: So I now read a lot of documents like I

153
00:07:52,000 --> 00:07:54,240
Speaker 4: used to in Amazon. You know, I would say maybe

154
00:07:54,400 --> 00:07:56,800
Speaker 4: thirty forty percent of the meeting are starting that way,

155
00:07:57,520 --> 00:08:00,280
Speaker 4: and I think people love it because it breaks the

156
00:08:00,320 --> 00:08:02,560
Speaker 4: barrier of language for someone like me, that English is

157
00:08:02,600 --> 00:08:05,720
Speaker 4: obviously not my first language, breaks the Sometimes some of

158
00:08:05,760 --> 00:08:08,040
Speaker 4: the people are more shy than others, et cetera. So

159
00:08:08,040 --> 00:08:11,120
Speaker 4: people see that as a mechanism for inclusion. So back

160
00:08:11,160 --> 00:08:14,400
Speaker 4: to your question, let's say thirty forty percent of my

161
00:08:14,520 --> 00:08:17,840
Speaker 4: meetings actually now start by us reading a document together

162
00:08:17,920 --> 00:08:19,920
Speaker 4: and then commenting on that and making decisions.

163
00:08:20,000 --> 00:08:22,800
Speaker 3: Can I just say, Tracy, I've always thought more meetings

164
00:08:23,000 --> 00:08:24,920
Speaker 3: you should start with just reading. Because you go to

165
00:08:25,000 --> 00:08:27,920
Speaker 3: you hear like a quarterly call or a FED event,

166
00:08:28,280 --> 00:08:30,680
Speaker 3: and someone just reads out of prepared text. It's like,

167
00:08:30,880 --> 00:08:33,080
Speaker 3: just let everyone read it and just jump straight into like,

168
00:08:33,160 --> 00:08:34,400
Speaker 3: let everyone do the reading first.

169
00:08:34,440 --> 00:08:35,320
Speaker 2: You don't need someone.

170
00:08:35,160 --> 00:08:38,440
Speaker 3: Standing up there talking about what's on a written piece

171
00:08:38,480 --> 00:08:39,240
Speaker 3: of paper somewhere.

172
00:08:39,240 --> 00:08:43,880
Speaker 1: Anyway, I agree that we could reduce the time of meetings. Yes, okay,

173
00:08:43,880 --> 00:08:47,400
Speaker 1: So speaking of meetings and the decision making process, then

174
00:08:47,760 --> 00:08:52,160
Speaker 1: talk to us about how Goldman Sachs decided to approach

175
00:08:52,520 --> 00:08:56,320
Speaker 1: generative AI. What was the decision making process? Like there

176
00:08:56,440 --> 00:08:59,680
Speaker 1: the development process, and you know, we'll get to what

177
00:08:59,720 --> 00:09:02,720
Speaker 1: you're developing, but like, how did you initially approach it?

178
00:09:03,160 --> 00:09:07,679
Speaker 4: So I think our initial approach was really to realize

179
00:09:07,880 --> 00:09:10,480
Speaker 4: that there were so many more things that we didn't

180
00:09:10,520 --> 00:09:13,080
Speaker 4: know compared to the things that we knew, because it's

181
00:09:13,120 --> 00:09:15,760
Speaker 4: a really new thing, and even for companies like us

182
00:09:15,760 --> 00:09:19,240
Speaker 4: that have been working on machine learning and traditionally I

183
00:09:19,400 --> 00:09:23,920
Speaker 4: for literally decades, this felt like a very different thing.

184
00:09:24,400 --> 00:09:26,920
Speaker 1: What sort of timeframe are we talking about? Like, was

185
00:09:26,960 --> 00:09:29,760
Speaker 1: there a sort of like big realization that this is

186
00:09:29,800 --> 00:09:31,199
Speaker 1: something that we need to focus on.

187
00:09:31,679 --> 00:09:35,000
Speaker 4: Yes, because I was lucky enough that I got into

188
00:09:35,280 --> 00:09:40,480
Speaker 4: the very very early version of GPT, even before it

189
00:09:40,520 --> 00:09:44,079
Speaker 4: was called chat GIBT. So the very first version was

190
00:09:44,200 --> 00:09:49,440
Speaker 4: essentially completing a sentence. It wasn't even allowing you to

191
00:09:49,440 --> 00:09:52,440
Speaker 4: do interactive chat. You would just paste a text and

192
00:09:52,520 --> 00:09:55,240
Speaker 4: that will just complete that text. And so I started

193
00:09:55,240 --> 00:09:56,920
Speaker 4: to do that with a bunch of stuff, and then

194
00:09:56,960 --> 00:09:59,439
Speaker 4: I was saying that the quality which this will continue

195
00:10:00,400 --> 00:10:03,200
Speaker 4: was pretty much indistinguishable with the part that you actually

196
00:10:03,200 --> 00:10:05,800
Speaker 4: put in that. And so we started to obviously talk

197
00:10:05,880 --> 00:10:08,559
Speaker 4: between ourselves but also among other people in the industry,

198
00:10:08,600 --> 00:10:12,560
Speaker 4: and we all realized very soon that this would be

199
00:10:12,880 --> 00:10:15,880
Speaker 4: something very different, but be also something that could have

200
00:10:15,920 --> 00:10:18,120
Speaker 4: a pretty profound impact in what we do. Because at

201
00:10:18,160 --> 00:10:21,599
Speaker 4: the end of the day, we are a purely digital business.

202
00:10:21,760 --> 00:10:24,200
Speaker 4: We don't bend metal, we don't you know, like use

203
00:10:24,280 --> 00:10:26,960
Speaker 4: high temperatures. We don't really have physics. So it's all

204
00:10:27,000 --> 00:10:29,679
Speaker 4: about how we service our clients. It's all about how

205
00:10:29,720 --> 00:10:33,000
Speaker 4: smart we are. It's all about how we can process

206
00:10:33,160 --> 00:10:36,839
Speaker 4: incredible amount of information. It's all about, you know, how

207
00:10:36,880 --> 00:10:40,200
Speaker 4: we analyze data in a very sometimes opinionated way. We

208
00:10:40,280 --> 00:10:43,160
Speaker 4: form our own views on the market, we form our

209
00:10:43,280 --> 00:10:47,000
Speaker 4: views of investments, et cetera. And so given that this

210
00:10:47,240 --> 00:10:52,880
Speaker 4: AI showed very early sign of being able to synthesize

211
00:10:53,000 --> 00:10:57,719
Speaker 4: and summarize very complex set of information but also identify patterns,

212
00:10:58,320 --> 00:11:00,679
Speaker 4: we thought that could be something that we definitely need

213
00:11:00,720 --> 00:11:03,920
Speaker 4: to pay attention to. So given that, one of the

214
00:11:03,960 --> 00:11:06,640
Speaker 4: things that we decided to do very early on was

215
00:11:06,840 --> 00:11:09,920
Speaker 4: to put a structure and I can say that more

216
00:11:09,960 --> 00:11:13,120
Speaker 4: about that, put a structure around this so that we

217
00:11:13,160 --> 00:11:17,640
Speaker 4: could experiment but in a sort of safe and controlled way.

218
00:11:18,080 --> 00:11:21,760
Speaker 1: Right, So you decided to develop your own Goldman Sachs

219
00:11:21,880 --> 00:11:26,199
Speaker 1: AI model versus you know, use a chat, GPT or

220
00:11:26,320 --> 00:11:27,640
Speaker 1: clod or getting something off the show.

221
00:11:27,679 --> 00:11:30,679
Speaker 4: Actually, initially we kind of thought about that, but then

222
00:11:30,800 --> 00:11:34,120
Speaker 4: very quickly. We decided that our time was spent much

223
00:11:34,160 --> 00:11:37,960
Speaker 4: better with using existing models, which by the way, we're

224
00:11:37,960 --> 00:11:41,920
Speaker 4: iterating really really quickly, but then put them in a

225
00:11:41,960 --> 00:11:44,960
Speaker 4: condition so that they would be safe to use and

226
00:11:45,000 --> 00:11:48,240
Speaker 4: also they would actually give us the most reliable information,

227
00:11:48,360 --> 00:11:51,960
Speaker 4: because taken as they are, you can't just drop a

228
00:11:52,040 --> 00:11:55,560
Speaker 4: model in an environment like Goldman and then, like you know,

229
00:11:55,640 --> 00:11:57,960
Speaker 4: to your earlier point of a one percent in accuracy,

230
00:11:58,200 --> 00:12:02,800
Speaker 4: zero point one percent in accuracy completely an acceptable class.

231
00:12:03,520 --> 00:12:06,520
Speaker 4: There are a lot of potential issues related to you know,

232
00:12:06,559 --> 00:12:09,240
Speaker 4: what data has it been used to train? And you know,

233
00:12:09,280 --> 00:12:12,240
Speaker 4: there is a lot of uncertainty with regards to you know,

234
00:12:12,320 --> 00:12:15,079
Speaker 4: like what are the boundaries between what you can safely

235
00:12:15,160 --> 00:12:17,560
Speaker 4: use and what you can And so what we decided

236
00:12:17,600 --> 00:12:23,080
Speaker 4: to do was instead to build a platform around the model.

237
00:12:23,160 --> 00:12:25,120
Speaker 4: So think of that almost as if you had a

238
00:12:25,200 --> 00:12:28,840
Speaker 4: nuclear reactor. You know that now you have invented fission

239
00:12:28,920 --> 00:12:30,880
Speaker 4: or fusion, and there is a lot of power that

240
00:12:30,920 --> 00:12:33,280
Speaker 4: can be generated from that, but then you need to

241
00:12:33,320 --> 00:12:36,080
Speaker 4: contain it and direct it in a certain way. And

242
00:12:36,080 --> 00:12:40,280
Speaker 4: so we build this GSAI platform, which essentially takes a

243
00:12:40,360 --> 00:12:43,200
Speaker 4: variety of models that we select, puts them in the

244
00:12:43,240 --> 00:12:47,840
Speaker 4: condition of being completely segregated and completely secluded and completely

245
00:12:47,880 --> 00:12:51,800
Speaker 4: safe from an information a security standpoint. Abstract some of

246
00:12:51,840 --> 00:12:54,559
Speaker 4: the ways to use the model, so that our developers

247
00:12:54,559 --> 00:12:58,360
Speaker 4: can use the models interchangeably, and then creates a set

248
00:12:58,400 --> 00:13:03,480
Speaker 4: of standardized way, for example, improve the accuracy using retrieval,

249
00:13:03,520 --> 00:13:09,199
Speaker 4: a granted generation, access external or internal data sources, applying

250
00:13:09,960 --> 00:13:13,160
Speaker 4: entitlement so that someone is on the private side, you know,

251
00:13:13,160 --> 00:13:15,160
Speaker 4: I've got to see different information that someone is on

252
00:13:15,200 --> 00:13:18,240
Speaker 4: the public side. And then on top of that, build

253
00:13:18,320 --> 00:13:21,920
Speaker 4: a developer environment so that people will very easily be

254
00:13:22,000 --> 00:13:25,760
Speaker 4: able to embed that AI in their own applications. So

255
00:13:25,880 --> 00:13:28,880
Speaker 4: imagine this, we got a great engine and we decided

256
00:13:28,920 --> 00:13:30,320
Speaker 4: to build a great car around that.

257
00:13:45,960 --> 00:13:47,440
Speaker 2: What are you putting in the model?

258
00:13:47,440 --> 00:13:49,840
Speaker 3: Because I have to imagine at a bank like Goldman,

259
00:13:50,080 --> 00:13:51,600
Speaker 3: you know, you have a lot of data, but you

260
00:13:51,679 --> 00:13:54,720
Speaker 3: must have just an extraordinary amount of unstructured data. There's

261
00:13:54,800 --> 00:13:59,880
Speaker 3: conversations that bankers have with clients. There's other sort of meeting,

262
00:14:00,000 --> 00:14:02,320
Speaker 3: the meetings you have, and there's words that are said

263
00:14:02,400 --> 00:14:05,840
Speaker 3: during that meeting that could be synthesized in some way.

264
00:14:06,280 --> 00:14:11,200
Speaker 3: In these early iterations, you know, I upload a conference

265
00:14:11,200 --> 00:14:12,960
Speaker 3: called transcript and I ask a question, what do you

266
00:14:13,040 --> 00:14:16,040
Speaker 3: upload it? What is the unstructured data that you have

267
00:14:16,800 --> 00:14:19,240
Speaker 3: or the questions or these yeah, what are you what

268
00:14:19,280 --> 00:14:22,240
Speaker 3: are you putting into it from your reams of knowledge

269
00:14:22,240 --> 00:14:23,280
Speaker 3: that you must have internally.

270
00:14:23,720 --> 00:14:26,200
Speaker 4: So one of the first things that we did was

271
00:14:26,680 --> 00:14:30,320
Speaker 4: use the platform and the models to extract information from

272
00:14:30,520 --> 00:14:34,280
Speaker 4: publicly available documents. That's kind of the safest way public

273
00:14:34,320 --> 00:14:36,720
Speaker 4: filing all the case or the queues and you know,

274
00:14:36,760 --> 00:14:40,600
Speaker 4: and obviously earnings, and put our bankers in a condition

275
00:14:40,720 --> 00:14:45,480
Speaker 4: to be able to ask very very sophisticated multi dimensional

276
00:14:45,600 --> 00:14:50,960
Speaker 4: questions around what was reported, cross refit with previous reports,

277
00:14:51,320 --> 00:14:55,560
Speaker 4: cross refit with any announcement, any earnings, called transcripts, all

278
00:14:55,640 --> 00:14:57,880
Speaker 4: things that are out there but just are difficult to

279
00:14:57,880 --> 00:15:00,920
Speaker 4: bring together. And so that as a involved into a

280
00:15:01,000 --> 00:15:04,600
Speaker 4: tool that physically we use and we're rolling it out

281
00:15:04,680 --> 00:15:08,520
Speaker 4: right now as an assistant to our bankers so that

282
00:15:08,680 --> 00:15:11,360
Speaker 4: they can you know, service their client or answer client

283
00:15:11,480 --> 00:15:14,720
Speaker 4: questions or even their wrong questions. In a time there

284
00:15:14,760 --> 00:15:17,280
Speaker 4: is a fraction of what you used to take even

285
00:15:17,400 --> 00:15:21,720
Speaker 4: generate documents that then can be you know, shared the

286
00:15:21,760 --> 00:15:23,720
Speaker 4: clients and so on and so forth. And obviously we

287
00:15:23,800 --> 00:15:27,360
Speaker 4: always have as a rule, like when you drive a

288
00:15:27,400 --> 00:15:29,920
Speaker 4: car that has some autonomous capability, that you always keep

289
00:15:30,000 --> 00:15:31,840
Speaker 4: the hands on the wheel. Our rule is that there

290
00:15:31,840 --> 00:15:33,720
Speaker 4: always needs to be a human in the loop. Okay,

291
00:15:34,200 --> 00:15:37,240
Speaker 4: And so the way that works is actually interesting because

292
00:15:37,320 --> 00:15:40,720
Speaker 4: we found out that you can't just shove something into

293
00:15:40,720 --> 00:15:42,960
Speaker 4: a model and then pretend that the model is going

294
00:15:43,000 --> 00:15:47,560
Speaker 4: to give you the answer right away. Why well, because models,

295
00:15:47,600 --> 00:15:51,800
Speaker 4: by themselves, you know, they essentially apply a stochastic or

296
00:15:51,840 --> 00:15:54,560
Speaker 4: a statistical way to understand what is the next world

297
00:15:54,560 --> 00:15:57,240
Speaker 4: that they need to say. So, no matter how good

298
00:15:57,560 --> 00:16:00,440
Speaker 4: is the material that you put in, there's always going

299
00:16:00,480 --> 00:16:03,160
Speaker 4: to be some level of variability. There is almost like

300
00:16:03,240 --> 00:16:06,000
Speaker 4: the intersection between the documents that you insert and what

301
00:16:06,200 --> 00:16:09,160
Speaker 4: is I call it like the shadow of all the

302
00:16:09,200 --> 00:16:11,720
Speaker 4: knowledge of all the things that the model has seen before.

303
00:16:12,520 --> 00:16:15,280
Speaker 4: And so we really perfected this. You know, there are

304
00:16:15,280 --> 00:16:19,680
Speaker 4: two techniques that are widely used to improve the accuracy

305
00:16:19,680 --> 00:16:23,920
Speaker 4: of the answers. One is working on the way those

306
00:16:24,000 --> 00:16:28,960
Speaker 4: models represent knowledge, which is called embeddings technically, and the

307
00:16:29,000 --> 00:16:31,760
Speaker 4: concept of embeddings by the way, everybody talks about embeddings,

308
00:16:31,760 --> 00:16:34,720
Speaker 4: but then for very few people actually it took me

309
00:16:34,760 --> 00:16:38,320
Speaker 4: a while to understand that well. And embedding is simply

310
00:16:38,520 --> 00:16:41,920
Speaker 4: a way for the model to parameterize and create a

311
00:16:42,040 --> 00:16:45,040
Speaker 4: description of what they're seeing. So if I see a phone,

312
00:16:45,080 --> 00:16:47,280
Speaker 4: for example, in front of me, the embeddings of a

313
00:16:47,320 --> 00:16:50,920
Speaker 4: phone could be it's a piece of electronic Yes, one,

314
00:16:51,000 --> 00:16:54,840
Speaker 4: it's definitely a piece of electronics. It's edible. Zero. You

315
00:16:54,880 --> 00:16:56,840
Speaker 4: can't really eat it, you know, And then you have

316
00:16:56,880 --> 00:17:00,760
Speaker 4: all these parameters. Is almost like twenty questions. I give

317
00:17:00,800 --> 00:17:02,720
Speaker 4: you all these questions and then you finally understand that

318
00:17:02,800 --> 00:17:04,960
Speaker 4: it's a phone, and that's what the embeddings is almost

319
00:17:04,960 --> 00:17:08,000
Speaker 4: like the twenty questions of the reality instead of twenty

320
00:17:08,080 --> 00:17:11,479
Speaker 4: is like twenty twenty thousands. And then you have DRAG,

321
00:17:11,560 --> 00:17:14,560
Speaker 4: which is the retrieval augmented generation, which is actually interesting

322
00:17:14,600 --> 00:17:18,720
Speaker 4: because you tell the model that instead of using its

323
00:17:18,760 --> 00:17:20,919
Speaker 4: on internal knowledge in order to give you an answer,

324
00:17:20,960 --> 00:17:23,640
Speaker 4: which sometimes, as I said, is like a representation of reality,

325
00:17:23,680 --> 00:17:26,840
Speaker 4: but it's often not accurate, you point them to the

326
00:17:26,920 --> 00:17:30,520
Speaker 4: right sections of the document that actually is more likely

327
00:17:30,560 --> 00:17:33,280
Speaker 4: to answer your question. Okay, and that's the key. It

328
00:17:33,320 --> 00:17:35,480
Speaker 4: needs to point to the right sections and then you

329
00:17:35,520 --> 00:17:38,560
Speaker 4: get the citations back. So that took a lot of effort.

330
00:17:39,040 --> 00:17:41,880
Speaker 4: But we're using that in many many cases because then

331
00:17:41,920 --> 00:17:45,399
Speaker 4: we expanded the use case from purely like banker assistant

332
00:17:45,440 --> 00:17:48,840
Speaker 4: in a way to more like okay, document management. You know,

333
00:17:48,880 --> 00:17:53,280
Speaker 4: we process millions of documents. Think of that credit confirmation

334
00:17:53,600 --> 00:17:59,439
Speaker 4: implements confirmation. Every document has a task called entity strauction.

335
00:17:59,560 --> 00:18:02,639
Speaker 4: So you need to extract stuff from the document and

336
00:18:02,680 --> 00:18:05,320
Speaker 4: then digitize it and then model it in a certain way.

337
00:18:05,880 --> 00:18:09,600
Speaker 4: And so the use of general TVii there does a

338
00:18:09,680 --> 00:18:14,600
Speaker 4: great job at extracting information. And this is an interesting

339
00:18:14,640 --> 00:18:19,760
Speaker 4: concept because you don't have to actually tell a fixed pattern.

340
00:18:20,000 --> 00:18:22,520
Speaker 4: You can just say, give a lot of examples, and

341
00:18:22,560 --> 00:18:24,840
Speaker 4: then the AI will figure out from that pattern. One

342
00:18:24,840 --> 00:18:27,520
Speaker 4: of my favorite example is the following. Let's say that

343
00:18:27,600 --> 00:18:31,480
Speaker 4: my phone number is five five three two one three

344
00:18:31,640 --> 00:18:35,439
Speaker 4: h five oh, and someone writes in the document instead

345
00:18:35,440 --> 00:18:39,600
Speaker 4: of with zero rights an oh. Okay. You can test

346
00:18:39,640 --> 00:18:42,840
Speaker 4: yourself even with GPT, if you give a number with

347
00:18:42,920 --> 00:18:45,480
Speaker 4: an O instead of zero, and you ask GPT, what's

348
00:18:45,800 --> 00:18:50,320
Speaker 4: likely wrong with this entity? GPT is gonna tell you, well,

349
00:18:50,960 --> 00:18:53,520
Speaker 4: it looks like a phone number that is an all,

350
00:18:53,600 --> 00:18:56,280
Speaker 4: which general is not in phone numbers. Most likely this

351
00:18:56,320 --> 00:19:00,600
Speaker 4: is the correct phone number. Now, nobody has written software

352
00:19:01,040 --> 00:19:04,080
Speaker 4: to do a pattern match in there. And imagine if

353
00:19:04,119 --> 00:19:06,800
Speaker 4: in the tradition, in traditional way of doing antity instruction,

354
00:19:06,920 --> 00:19:10,280
Speaker 4: there were developers that were writing rules. They were saying, okay, numbers,

355
00:19:10,480 --> 00:19:13,560
Speaker 4: it needs to be ten digits and blah blah blah.

356
00:19:13,760 --> 00:19:16,000
Speaker 4: The AI figures.

357
00:19:15,560 --> 00:19:18,520
Speaker 2: Out their own rules.

358
00:19:17,960 --> 00:19:20,600
Speaker 4: That are the most likely. So this is the key thing.

359
00:19:20,760 --> 00:19:24,240
Speaker 4: It has common sense. And that common sense when you're

360
00:19:24,359 --> 00:19:28,840
Speaker 4: dealing with millions of documents that contain all bunch of

361
00:19:28,960 --> 00:19:32,320
Speaker 4: ways that you must might have written those things, and

362
00:19:32,400 --> 00:19:34,439
Speaker 4: imagine the complexity of all the rules that you need

363
00:19:34,480 --> 00:19:37,560
Speaker 4: to write. And every bank has the same problem. This

364
00:19:37,720 --> 00:19:42,680
Speaker 4: simplifies things tremendously because it's able to figure out what's

365
00:19:42,800 --> 00:19:48,240
Speaker 4: most likely by itself. And so that thing evolved into

366
00:19:48,560 --> 00:19:52,080
Speaker 4: a tremendous time saving for everybody in the bank that

367
00:19:52,119 --> 00:19:54,639
Speaker 4: has to do with the workflow documents. And so that

368
00:19:55,119 --> 00:19:57,359
Speaker 4: was a very interesting finding that we did early on.

369
00:19:57,440 --> 00:20:02,119
Speaker 4: And so again to summarize more, those are raw material

370
00:20:02,240 --> 00:20:05,800
Speaker 4: of intelligence. You know you need to somehow direct them,

371
00:20:05,840 --> 00:20:07,960
Speaker 4: you need to guide them, you need to instruct them,

372
00:20:07,960 --> 00:20:10,040
Speaker 4: you need to put them in an environment that actually

373
00:20:10,080 --> 00:20:12,000
Speaker 4: gets the most out of that, and that's what we've

374
00:20:12,000 --> 00:20:12,800
Speaker 4: been focusing on.

375
00:20:13,119 --> 00:20:16,240
Speaker 1: So going back to the analogy that you used previously,

376
00:20:16,280 --> 00:20:18,800
Speaker 1: this idea of a nuclear reactor and sort of building

377
00:20:18,840 --> 00:20:22,400
Speaker 1: the containment casing or the protective casing around it. I

378
00:20:22,400 --> 00:20:26,359
Speaker 1: imagine one of the complications of being Goldman Sachs and

379
00:20:26,400 --> 00:20:29,960
Speaker 1: working with AI is that you're a regulated financial entity.

380
00:20:30,560 --> 00:20:35,480
Speaker 1: How does that added complexity affect your use of AI.

381
00:20:35,640 --> 00:20:39,680
Speaker 1: Are there additional data considerations or additional infosec considerations.

382
00:20:40,240 --> 00:20:44,520
Speaker 4: I think that's a great question, because obviously we live

383
00:20:44,560 --> 00:20:47,040
Speaker 4: in a regulated world, and in fact, I have to

384
00:20:47,040 --> 00:20:49,880
Speaker 4: tell you that in this case, regulation actually helps us

385
00:20:50,000 --> 00:20:53,520
Speaker 4: think through all the possible unknown now, something that, as

386
00:20:53,520 --> 00:20:56,080
Speaker 4: I said, is something that is still largely something that

387
00:20:56,119 --> 00:20:59,600
Speaker 4: nobody really completely understands. And so what we did was

388
00:20:59,680 --> 00:21:03,240
Speaker 4: to put but governance around the usage of the models

389
00:21:03,280 --> 00:21:05,760
Speaker 4: and also governance with regards to the use cases that

390
00:21:05,800 --> 00:21:09,119
Speaker 4: we can implement on the models. Every bank has a

391
00:21:09,160 --> 00:21:12,639
Speaker 4: function called model risk, which, in the traditional sense, a

392
00:21:12,720 --> 00:21:18,040
Speaker 4: model is any decision or any algorithm that is running

393
00:21:18,080 --> 00:21:21,040
Speaker 4: automatically to do for example, pricing or you know, there

394
00:21:21,119 --> 00:21:24,400
Speaker 4: is a lot of that tradition in every bank risk calculation, etc.

395
00:21:24,760 --> 00:21:27,920
Speaker 4: So that's the traditional model risk. We use that very

396
00:21:27,960 --> 00:21:31,000
Speaker 4: well established pattern. That is also you know, that has

397
00:21:31,040 --> 00:21:35,280
Speaker 4: its own second and third line like controls and supervision

398
00:21:35,840 --> 00:21:38,840
Speaker 4: also to validate what we do on the AI side.

399
00:21:38,880 --> 00:21:41,600
Speaker 4: So there is a governance part which we really set

400
00:21:41,680 --> 00:21:44,359
Speaker 4: up very early on. We have an AI committee that

401
00:21:44,400 --> 00:21:47,479
Speaker 4: looks at the business case should we do this? And

402
00:21:47,520 --> 00:21:50,840
Speaker 4: then we have an AI control and risk committee that

403
00:21:50,880 --> 00:21:52,720
Speaker 4: looks at, okay, how are we going to do that?

404
00:21:52,840 --> 00:21:54,840
Speaker 4: And then the two of them need to actually come

405
00:21:54,880 --> 00:21:57,719
Speaker 4: together before we can release a use case. And then

406
00:21:57,760 --> 00:21:59,879
Speaker 4: of course we did a lot of work with regards

407
00:21:59,880 --> 00:22:05,080
Speaker 4: to the let's say accuracy lineage and in a way,

408
00:22:05,200 --> 00:22:08,040
Speaker 4: the way you connect the output to where does the

409
00:22:08,119 --> 00:22:10,840
Speaker 4: data come from and who can actually see that what

410
00:22:10,920 --> 00:22:14,159
Speaker 4: we call entitlements, and we did that in lockstep with

411
00:22:14,160 --> 00:22:17,320
Speaker 4: the regulators, so that I think, you know, you know,

412
00:22:17,359 --> 00:22:18,920
Speaker 4: in a world, I think we put a sort of

413
00:22:19,000 --> 00:22:22,399
Speaker 4: what we like to call responsible AI first since the

414
00:22:22,480 --> 00:22:24,680
Speaker 4: very beginning, and it really helped us. The fact that

415
00:22:24,760 --> 00:22:28,080
Speaker 4: you know, we embedded all those controls into a single platform.

416
00:22:28,160 --> 00:22:31,320
Speaker 4: This is how our people use AI inside the inside goal.

417
00:22:31,760 --> 00:22:33,919
Speaker 1: This is something I'm really interested in just from a

418
00:22:33,960 --> 00:22:36,800
Speaker 1: technical perspective, But can you talk a little bit more

419
00:22:37,000 --> 00:22:41,000
Speaker 1: about that interoperability aspect. So you have a pool of

420
00:22:41,119 --> 00:22:43,960
Speaker 1: data that is gold pins that you presumably don't really

421
00:22:44,000 --> 00:22:46,720
Speaker 1: want to share with outside entities, So how do you

422
00:22:46,760 --> 00:22:50,520
Speaker 1: plug that into an AI model if you're working with

423
00:22:50,680 --> 00:22:52,920
Speaker 1: you know, Chat, GPT or clod or something like that.

424
00:22:53,160 --> 00:22:55,399
Speaker 4: So there are two ways that we do that. We

425
00:22:55,560 --> 00:22:59,639
Speaker 4: use the sort of a large proprietary models in a

426
00:22:59,680 --> 00:23:02,719
Speaker 4: way that we worked with Microsoft, we work on Google.

427
00:23:02,760 --> 00:23:07,680
Speaker 4: We have very strong partnerships, so that essentially there are

428
00:23:07,760 --> 00:23:11,040
Speaker 4: controls that guarantee that nobody has access to the data

429
00:23:11,080 --> 00:23:13,680
Speaker 4: that we put into the model, that the data leaves

430
00:23:13,760 --> 00:23:17,679
Speaker 4: no side effects, so it's not saved anywhere, it's the

431
00:23:17,680 --> 00:23:21,560
Speaker 4: only stays in memory. The model is completely stateless, meaning

432
00:23:21,640 --> 00:23:24,199
Speaker 4: that the state of the model doesn't change after the

433
00:23:24,280 --> 00:23:26,159
Speaker 4: data comes through, so there is no training, there is

434
00:23:26,200 --> 00:23:29,520
Speaker 4: nothing down on that data. And also that operator access

435
00:23:29,960 --> 00:23:32,720
Speaker 4: meaning who can actually access the memory or those machines

436
00:23:32,960 --> 00:23:35,879
Speaker 4: is restricted and controlled and needs to be agreed with us.

437
00:23:36,040 --> 00:23:39,720
Speaker 4: So imagine secure in putting a vault around those models.

438
00:23:39,760 --> 00:23:44,560
Speaker 4: But even then, what's really really sort of secret, source, proprietory, etc.

439
00:23:44,920 --> 00:23:48,160
Speaker 4: We like to use also different approach to use open

440
00:23:48,200 --> 00:23:52,439
Speaker 4: source models that we can run on our own environment. Okay,

441
00:23:53,320 --> 00:23:55,439
Speaker 4: and we like a lot of open source models. I

442
00:23:55,480 --> 00:23:58,040
Speaker 4: have to say that. One we particularly like Islama and

443
00:23:58,080 --> 00:24:01,359
Speaker 4: actually Lama tree and Lama tree point one especially as.

444
00:24:01,280 --> 00:24:02,560
Speaker 2: No one developed by Facebook.

445
00:24:03,040 --> 00:24:06,800
Speaker 4: Oh yeah, so they recently announced Lama three point one,

446
00:24:07,320 --> 00:24:10,080
Speaker 4: which has a version that is four hundred and five

447
00:24:10,119 --> 00:24:14,320
Speaker 4: million billion parameters. So it's pretty large and it seems

448
00:24:14,359 --> 00:24:17,320
Speaker 4: to be performing. You know, the gap with those big

449
00:24:17,359 --> 00:24:20,600
Speaker 4: fundational models is now very very narrow. So for that,

450
00:24:20,760 --> 00:24:23,080
Speaker 4: we run it in our own sort of a private cloud,

451
00:24:23,200 --> 00:24:26,920
Speaker 4: call it that way, with GPUs that we own, and

452
00:24:26,960 --> 00:24:29,640
Speaker 4: that we train it with data that stays in that environment.

453
00:24:29,680 --> 00:24:32,000
Speaker 4: So imagine that. You know, our approach is okay, there

454
00:24:32,040 --> 00:24:34,960
Speaker 4: is a sort of arrating of sensitivity of this data.

455
00:24:35,680 --> 00:24:38,960
Speaker 4: Every data needs to be protected. Therefore we use those

456
00:24:39,000 --> 00:24:42,960
Speaker 4: safeties all throughout regardless. But then for the super super

457
00:24:42,960 --> 00:24:45,560
Speaker 4: super secret stuff, you know, we like to do it

458
00:24:45,600 --> 00:24:47,000
Speaker 4: in our own embod.

459
00:24:46,760 --> 00:24:49,239
Speaker 3: Since you're talking about building your own environment, and this

460
00:24:49,320 --> 00:24:51,600
Speaker 3: is something we've talked a lot about on the podcast.

461
00:24:52,000 --> 00:24:57,239
Speaker 3: Hardware constraints, energy constraints, things like that, how does that

462
00:24:57,359 --> 00:25:01,040
Speaker 3: manifest in your world some of these physical, real world

463
00:25:01,119 --> 00:25:06,119
Speaker 3: constraints to building out the compute platform at Goldman sax Well.

464
00:25:06,200 --> 00:25:09,199
Speaker 4: Initially we thought maybe we can host those GPUs in

465
00:25:09,240 --> 00:25:13,000
Speaker 4: our own data centers, and then immediately you run into

466
00:25:13,000 --> 00:25:15,840
Speaker 4: considerations such as a first of all, they develop a

467
00:25:15,880 --> 00:25:18,679
Speaker 4: lot of heat. Secondly, they consume a lot of power.

468
00:25:19,160 --> 00:25:21,720
Speaker 4: Tree there is a decent chance that they might fail

469
00:25:21,880 --> 00:25:24,439
Speaker 4: because you know, of all those considerations if you're not

470
00:25:24,920 --> 00:25:29,680
Speaker 4: properly addressed. And then d they need very special for example,

471
00:25:29,680 --> 00:25:32,280
Speaker 4: interconnect and high speed bandwidth between them. And so the

472
00:25:32,359 --> 00:25:35,199
Speaker 4: decision what we ended up doing is actually to have

473
00:25:35,280 --> 00:25:38,640
Speaker 4: them hosted into some of the hyperscalers that we use,

474
00:25:39,200 --> 00:25:42,560
Speaker 4: but use them in their own virtual private clouds. So

475
00:25:42,640 --> 00:25:46,639
Speaker 4: those racks are basically only ours. And if you're asking

476
00:25:46,640 --> 00:25:49,280
Speaker 4: me the more general question, which is, hey, where is

477
00:25:49,320 --> 00:25:52,600
Speaker 4: the world going with regards of that? Okay, so right

478
00:25:52,640 --> 00:25:57,320
Speaker 4: now there are two really rapidly competing forces. One is

479
00:25:57,400 --> 00:26:01,000
Speaker 4: pushing towards more and more consumption and one is pushing

480
00:26:01,000 --> 00:26:03,560
Speaker 4: for more and more optimization. Okay, and I can talk

481
00:26:03,600 --> 00:26:06,959
Speaker 4: about that for a couple of minutes. For the more consumption,

482
00:26:07,440 --> 00:26:10,119
Speaker 4: I mean, really the two dimensions for scaling a model

483
00:26:10,720 --> 00:26:12,959
Speaker 4: is one of the most important. One is obviously the

484
00:26:13,000 --> 00:26:16,000
Speaker 4: size of the prompt or the context. Okay, and there

485
00:26:16,040 --> 00:26:19,160
Speaker 4: is pretty good evidence that the larger the context, which

486
00:26:19,200 --> 00:26:21,479
Speaker 4: is really like the memory of those models, and the

487
00:26:21,480 --> 00:26:23,680
Speaker 4: more you can get out in terms of the ability

488
00:26:23,680 --> 00:26:27,480
Speaker 4: to reason on your data. That has already gone up

489
00:26:27,520 --> 00:26:30,679
Speaker 4: from thousands to tens of thousands to now millions. And

490
00:26:30,720 --> 00:26:33,240
Speaker 4: there is a prediction, you know, you heard some very

491
00:26:33,280 --> 00:26:36,439
Speaker 4: prominent people saying that there could be the trillion prompt

492
00:26:36,480 --> 00:26:40,160
Speaker 4: and the power scales quadratically with the prompt, so that

493
00:26:40,320 --> 00:26:43,280
Speaker 4: points to a consumption of energy and GPU power which

494
00:26:43,359 --> 00:26:46,040
Speaker 4: is going to continue to raise exponentially. At the same time,

495
00:26:46,560 --> 00:26:51,400
Speaker 4: we've seen great results with optimization techniques such as quantitization,

496
00:26:51,960 --> 00:26:55,280
Speaker 4: reducing from sixteen bits to eight bit to four bit precision,

497
00:26:56,000 --> 00:27:00,520
Speaker 4: having even smaller models using what's called window that which

498
00:27:00,560 --> 00:27:03,120
Speaker 4: means that you know that you can only pay more

499
00:27:03,160 --> 00:27:05,760
Speaker 4: attention to some of the parts of the context intell

500
00:27:05,840 --> 00:27:08,760
Speaker 4: of all of it, and so you need a smaller one.

501
00:27:08,920 --> 00:27:11,640
Speaker 4: And so I'm seeing those two kind of going into

502
00:27:11,680 --> 00:27:13,919
Speaker 4: two opposite directions. It's going to be very interesting to

503
00:27:13,920 --> 00:27:17,040
Speaker 4: see how that evolves. I would say for the short term.

504
00:27:17,520 --> 00:27:20,680
Speaker 4: I see that definitely that trend is going to continue

505
00:27:20,720 --> 00:27:23,600
Speaker 4: to go up. And one of the things that fascinates

506
00:27:23,640 --> 00:27:26,800
Speaker 4: me the most is that from one version to another,

507
00:27:27,520 --> 00:27:31,800
Speaker 4: the most striking difference is the ability to reason and

508
00:27:31,840 --> 00:27:36,000
Speaker 4: the ability to actually come up with logical step by

509
00:27:36,080 --> 00:27:40,840
Speaker 4: step instructions or step by step chains of thought of

510
00:27:40,880 --> 00:27:44,280
Speaker 4: what the output is going to be. So we decided, okay,

511
00:27:44,320 --> 00:27:45,639
Speaker 4: first of all, we need to get access to the

512
00:27:45,680 --> 00:27:49,000
Speaker 4: most powerful GPUs, secondary we need to host them into

513
00:27:49,000 --> 00:27:52,920
Speaker 4: an environment that actually allows for the most optimal functioning

514
00:27:52,960 --> 00:27:55,160
Speaker 4: in terms of bandit, in terms of power consumption, etc.

515
00:27:56,359 --> 00:27:58,399
Speaker 4: And then at the same time, we've been focusing a

516
00:27:58,480 --> 00:28:00,840
Speaker 4: lot on optimizing the algorithm so that you know, we

517
00:28:00,880 --> 00:28:03,120
Speaker 4: can really got we could really get the most out

518
00:28:03,160 --> 00:28:03,359
Speaker 4: of that.

519
00:28:04,160 --> 00:28:06,119
Speaker 1: Just to press you on this point, what are the

520
00:28:06,160 --> 00:28:10,200
Speaker 1: conversations actually like with cloud providers at the moment when

521
00:28:10,200 --> 00:28:14,200
Speaker 1: you're trying to get more compute or more space, more racks, whatever.

522
00:28:14,680 --> 00:28:17,000
Speaker 1: Is it maybe different for you because you were at AWS.

523
00:28:17,040 --> 00:28:18,879
Speaker 1: Maybe you can just call someone up there and be like,

524
00:28:18,960 --> 00:28:22,959
Speaker 1: we would like some more servers, or have you found

525
00:28:23,000 --> 00:28:26,359
Speaker 1: yourselves at times maybe limited in what you can do

526
00:28:26,600 --> 00:28:28,320
Speaker 1: by the amount of power available to you.

527
00:28:29,640 --> 00:28:31,200
Speaker 4: Well, I wish that would be the case, but I

528
00:28:31,480 --> 00:28:34,880
Speaker 4: cannot just pick up the phone and get whatever I want.

529
00:28:35,000 --> 00:28:37,919
Speaker 4: But I think so far. I mean obviously because we

530
00:28:38,000 --> 00:28:40,920
Speaker 4: are a really good client of those companies in general,

531
00:28:41,160 --> 00:28:43,720
Speaker 4: but also because we've been very selective in the use

532
00:28:43,800 --> 00:28:46,200
Speaker 4: cases that we put in production. I have to say,

533
00:28:46,240 --> 00:28:48,880
Speaker 4: like I said before, think about that, if you look

534
00:28:48,880 --> 00:28:52,640
Speaker 4: at the consumption of resources today, those who consume more

535
00:28:52,680 --> 00:28:55,360
Speaker 4: resources are people that actually do the training of their

536
00:28:55,360 --> 00:29:00,880
Speaker 4: own models. Okay, and it initially everybody was trying to

537
00:29:00,880 --> 00:29:03,400
Speaker 4: do full training from scratch, which was taken like the

538
00:29:03,480 --> 00:29:06,720
Speaker 4: absolutely if that's one hundred, we do fine tuning, which

539
00:29:06,800 --> 00:29:10,040
Speaker 4: is adaptation of existing models that could be one to

540
00:29:10,120 --> 00:29:12,920
Speaker 4: one hundred or less in terms of consumption or resources.

541
00:29:12,920 --> 00:29:15,280
Speaker 4: So because of the techniques they were using, and because

542
00:29:15,280 --> 00:29:18,120
Speaker 4: of the fact that we decided to really focus on

543
00:29:18,200 --> 00:29:21,840
Speaker 4: fine tuning or RAG versus full training, we haven't really

544
00:29:22,120 --> 00:29:24,640
Speaker 4: hit any caps. And also have to be honest, you know,

545
00:29:24,720 --> 00:29:28,479
Speaker 4: we bought our GPUs pretty well early, so probably there

546
00:29:28,520 --> 00:29:31,360
Speaker 4: wasn't as much craziness as there is today, and so

547
00:29:31,400 --> 00:29:47,160
Speaker 4: that's turned out probably to be a good idea.

548
00:29:48,560 --> 00:29:49,760
Speaker 2: You know, in videos huge.

549
00:29:49,880 --> 00:29:52,360
Speaker 3: Everyone would like to have some of in Video's market

550
00:29:52,440 --> 00:29:55,800
Speaker 3: cap be their market cap. I have offering some cheaper product.

551
00:29:56,200 --> 00:30:00,400
Speaker 3: We interviewed some guys who have a semiconductor started that's

552
00:30:00,440 --> 00:30:04,000
Speaker 3: just going to be LLLM focused startups. We know that Google,

553
00:30:04,080 --> 00:30:08,000
Speaker 3: for example, has TPUs their own chips. Can you envision

554
00:30:08,040 --> 00:30:12,040
Speaker 3: as a roadmap some alternative where GPUs are not the

555
00:30:12,200 --> 00:30:14,320
Speaker 3: dominant hardware for AI?

556
00:30:14,640 --> 00:30:17,200
Speaker 4: Well, that's literally like you know the trillion dollar question.

557
00:30:17,240 --> 00:30:18,520
Speaker 2: Yeah, well that's I'm asking you.

558
00:30:18,600 --> 00:30:21,040
Speaker 4: Yeah, but I'm not an analyst and I'm just a technogy.

559
00:30:21,040 --> 00:30:23,480
Speaker 4: Remember I'm the guy that makes sure that I.

560
00:30:23,440 --> 00:30:26,000
Speaker 3: Would say, you're probably a better person to ask than

561
00:30:26,040 --> 00:30:28,200
Speaker 3: an analyst because you're actually the one who's going to

562
00:30:28,200 --> 00:30:29,440
Speaker 3: be making So I'm.

563
00:30:29,560 --> 00:30:31,640
Speaker 4: Okay, so they're going to ask it to you. So

564
00:30:32,240 --> 00:30:35,400
Speaker 4: you have to distinguish between There are actually two dimensions

565
00:30:35,440 --> 00:30:37,440
Speaker 4: that we need to consider. One is training and the

566
00:30:37,440 --> 00:30:41,720
Speaker 4: other one is inferenced. Okay, that's the first dichotomy. For training.

567
00:30:42,200 --> 00:30:45,959
Speaker 4: At the moment, there's most likely nothing better than GPU's okay,

568
00:30:46,080 --> 00:30:50,400
Speaker 4: because when you train a model, the software or Pythons

569
00:30:50,480 --> 00:30:53,720
Speaker 4: or whatever framework needs to see all your GPUs as one.

570
00:30:54,320 --> 00:30:58,040
Speaker 4: As a cluster, and it's not just the GPU itself,

571
00:30:58,080 --> 00:31:00,360
Speaker 4: but it's the what Nvidia has been doing a great

572
00:31:00,440 --> 00:31:03,680
Speaker 4: job at is actually to make them work in unison

573
00:31:04,160 --> 00:31:07,400
Speaker 4: with the virtualization software called Kuda, which runs on and

574
00:31:07,560 --> 00:31:11,320
Speaker 4: video GPUs, which is a extraordinary piece of software and

575
00:31:11,440 --> 00:31:15,480
Speaker 4: it became the standard for that. And also because you know,

576
00:31:15,600 --> 00:31:19,000
Speaker 4: the performance premium that you have on those GPUs when

577
00:31:19,000 --> 00:31:22,840
Speaker 4: you're trying to train those incredibly large models is something

578
00:31:22,880 --> 00:31:25,360
Speaker 4: that you really really want. And so the training part,

579
00:31:25,520 --> 00:31:27,360
Speaker 4: I'm pretty sure that it's going to be dominated by

580
00:31:27,440 --> 00:31:30,200
Speaker 4: GPUs for a while. But then you know, as those

581
00:31:30,200 --> 00:31:34,760
Speaker 4: models get used, obviously the pendulum swings towards inference, which

582
00:31:34,800 --> 00:31:36,920
Speaker 4: is the actual Now you have a model which is

583
00:31:36,920 --> 00:31:38,880
Speaker 4: a bunch of weights and you just need to calculate

584
00:31:38,880 --> 00:31:43,240
Speaker 4: a bunch of matrix multiplications on that. I think accelerators

585
00:31:43,280 --> 00:31:47,000
Speaker 4: and specialized chips are actually going to have a really

586
00:31:47,040 --> 00:31:49,680
Speaker 4: big role to play. So you may imagine that you

587
00:31:49,760 --> 00:31:52,880
Speaker 4: go from a world where everybody builds the cars and

588
00:31:52,920 --> 00:31:55,240
Speaker 4: not too many people drive the cars to a world

589
00:31:55,240 --> 00:31:57,560
Speaker 4: where most people are going to drive cars. And then

590
00:31:57,600 --> 00:32:00,880
Speaker 4: there is another two dimensions, which is models that are

591
00:32:00,920 --> 00:32:04,720
Speaker 4: hosted by the client and models that are hosted by

592
00:32:04,880 --> 00:32:08,680
Speaker 4: a hyperscale. So, as you know today, I can take

593
00:32:08,720 --> 00:32:10,720
Speaker 4: a model like Lamma, I can put it in my

594
00:32:10,800 --> 00:32:13,920
Speaker 4: own environ, I can run it on a MacBook, or

595
00:32:13,920 --> 00:32:16,040
Speaker 4: I can run it in my own data center and

596
00:32:16,080 --> 00:32:19,800
Speaker 4: with my own GPUs. And given that I'm used to GPUs,

597
00:32:20,240 --> 00:32:22,160
Speaker 4: given that those are the ones that we can buy,

598
00:32:22,240 --> 00:32:25,280
Speaker 4: given that Kuda is what developers know, etc. I'm most

599
00:32:25,320 --> 00:32:27,280
Speaker 4: likely going to use that. That's a good part for

600
00:32:27,440 --> 00:32:30,560
Speaker 4: Nvidia for that. But then there is another way to

601
00:32:30,640 --> 00:32:33,720
Speaker 4: use those models, which is to have someone host them

602
00:32:33,760 --> 00:32:36,840
Speaker 4: for me and I just access them to an API.

603
00:32:37,280 --> 00:32:40,760
Speaker 4: That's what services like Amazon Bedrock does. You basically choose

604
00:32:40,760 --> 00:32:42,920
Speaker 4: your own model and then you serve it through them.

605
00:32:43,160 --> 00:32:46,040
Speaker 4: When you do that, you don't really know what's underneath.

606
00:32:46,440 --> 00:32:48,120
Speaker 4: You don't know if it's a VP, or if it

607
00:32:48,160 --> 00:32:51,120
Speaker 4: is an accelerator, if it is Amazon's own chips or

608
00:32:51,320 --> 00:32:54,840
Speaker 4: Google's own chips, etc. So now the real question, that's

609
00:32:54,840 --> 00:32:58,040
Speaker 4: why the trillion dollar question is are most people going

610
00:32:58,120 --> 00:33:03,160
Speaker 4: to use those models through hosted environments where the hyperscaler

611
00:33:03,200 --> 00:33:04,920
Speaker 4: will have a lot of freedom with regards to what

612
00:33:05,000 --> 00:33:07,920
Speaker 4: they use underneath, and most likely they will vertically integrate

613
00:33:08,600 --> 00:33:11,400
Speaker 4: or are they going to use them you know, themselves

614
00:33:11,560 --> 00:33:13,720
Speaker 4: in a more more like you know, in a self

615
00:33:13,760 --> 00:33:16,760
Speaker 4: service way, And in that case it's less likely that

616
00:33:17,040 --> 00:33:20,960
Speaker 4: those accelerators are going to dominate. We currently are in

617
00:33:20,960 --> 00:33:23,520
Speaker 4: a sort of a you know, balanced way because we

618
00:33:23,600 --> 00:33:25,680
Speaker 4: have our own that we use like I described, and

619
00:33:25,720 --> 00:33:28,440
Speaker 4: also we use you know, the hosted models. And so

620
00:33:28,720 --> 00:33:31,160
Speaker 4: where is this going to go? It's hard to say,

621
00:33:31,240 --> 00:33:34,360
Speaker 4: because I think it depends on the evolution of the models,

622
00:33:34,400 --> 00:33:36,400
Speaker 4: and it depends which models are going to be made

623
00:33:36,400 --> 00:33:39,080
Speaker 4: available as an open source that you can actually host yourself.

624
00:33:39,880 --> 00:33:42,040
Speaker 4: And I think right now one of the greatest questions

625
00:33:42,160 --> 00:33:45,480
Speaker 4: is are the open source models are going to be

626
00:33:45,600 --> 00:33:49,160
Speaker 4: in absolutely on parer alternative to the to the hosted model,

627
00:33:49,200 --> 00:33:53,120
Speaker 4: to the to the foundational proprietary models, and that given

628
00:33:53,160 --> 00:33:55,960
Speaker 4: Glama three point one, that answer seems to be more likely.

629
00:33:56,120 --> 00:33:59,280
Speaker 1: Yes, I had a question about this actually, which is

630
00:33:59,440 --> 00:34:03,440
Speaker 1: do you think Wall Street's attitudes towards open source have

631
00:34:03,640 --> 00:34:06,280
Speaker 1: changed over time? And the reason I ask is because

632
00:34:06,320 --> 00:34:09,799
Speaker 1: nowadays it seems like a fact of life. Everyone uses

633
00:34:09,920 --> 00:34:12,520
Speaker 1: open source, whether you're a Goldman or somewhere else. But

634
00:34:12,600 --> 00:34:16,480
Speaker 1: I remember, you know, like back in as recently as

635
00:34:16,719 --> 00:34:20,480
Speaker 1: like twenty twelve. I remember Deutsche Bank had like this

636
00:34:20,600 --> 00:34:25,640
Speaker 1: open source project called the Loadstone Foundation, where they were like, oh,

637
00:34:25,680 --> 00:34:29,000
Speaker 1: we should all stop wasting our own resources developing our

638
00:34:29,040 --> 00:34:31,279
Speaker 1: own code and our own software. We should all pool

639
00:34:31,320 --> 00:34:34,239
Speaker 1: our resources together and do open source. And they had

640
00:34:34,239 --> 00:34:38,239
Speaker 1: to actually lobby. It was unsuccessful ultimately, but they were

641
00:34:38,239 --> 00:34:40,840
Speaker 1: trying to get all the banks on Wall Street to

642
00:34:40,880 --> 00:34:44,479
Speaker 1: work together for open source. Nowadays, it seems like there's

643
00:34:44,520 --> 00:34:47,040
Speaker 1: been this significant cultural shift, it's not even a question.

644
00:34:47,719 --> 00:34:51,080
Speaker 4: So in general, my direction, my guide as to you know,

645
00:34:51,120 --> 00:34:55,080
Speaker 4: my team is a don't build anything unless you have to.

646
00:34:57,000 --> 00:34:59,480
Speaker 4: Don't think that just because you're a smart person you

647
00:34:59,520 --> 00:35:02,439
Speaker 4: can build software better than anybody else. Maybe you can,

648
00:35:03,080 --> 00:35:05,200
Speaker 4: but it's a good thing that we focus on building

649
00:35:05,200 --> 00:35:09,080
Speaker 4: things that are actually differentiating for us. And then I

650
00:35:09,120 --> 00:35:11,480
Speaker 4: think the use of open source software, which we very

651
00:35:11,520 --> 00:35:15,839
Speaker 4: much endorse, is also really good hedge with regards to

652
00:35:16,200 --> 00:35:19,320
Speaker 4: you know, which vendors to use, because it really heavily

653
00:35:19,360 --> 00:35:23,800
Speaker 4: reduces the vendor lock in. Of course, open source software,

654
00:35:24,000 --> 00:35:26,800
Speaker 4: as you know, is a tremendous long tail. There's millions

655
00:35:26,840 --> 00:35:29,520
Speaker 4: of that, and so I think there are best practices

656
00:35:29,560 --> 00:35:33,200
Speaker 4: around the use of open source, and those best practices are,

657
00:35:33,400 --> 00:35:35,360
Speaker 4: you know, like you know you need to run reviews

658
00:35:35,360 --> 00:35:38,280
Speaker 4: on open source tech or tech risk reviews or security

659
00:35:38,320 --> 00:35:41,799
Speaker 4: reviews or anything as I've almost built it yourself. And

660
00:35:41,840 --> 00:35:46,759
Speaker 4: then secondly tending to concentrate on the larger, very well

661
00:35:46,840 --> 00:35:50,279
Speaker 4: supported by the community type of open source. And so

662
00:35:50,840 --> 00:35:53,360
Speaker 4: my philosophy is yes to open source, but then you

663
00:35:53,480 --> 00:35:57,000
Speaker 4: need to own it in in truest way because you

664
00:35:57,040 --> 00:35:59,600
Speaker 4: are actually going to be generally the one that actually

665
00:35:59,719 --> 00:36:02,520
Speaker 4: needs to support that as or really building knowledge around that.

666
00:36:02,680 --> 00:36:05,279
Speaker 1: And now you can ask AI to run the code

667
00:36:05,360 --> 00:36:06,200
Speaker 1: for you and check it.

668
00:36:06,239 --> 00:36:09,120
Speaker 4: For yeah, okay. That of course leads to probably what

669
00:36:09,440 --> 00:36:12,560
Speaker 4: if you ask everybody where did you get so far?

670
00:36:13,040 --> 00:36:16,280
Speaker 4: The biggest bank for the back for AI? Most CIOs

671
00:36:16,280 --> 00:36:19,360
Speaker 4: are going to tell you on developer productivity. And I

672
00:36:19,400 --> 00:36:21,880
Speaker 4: think it's something that for us was the first project

673
00:36:21,880 --> 00:36:23,960
Speaker 4: that we actually expanded at scale. I have to say

674
00:36:23,960 --> 00:36:27,240
Speaker 4: that today virtually every developer in Goma SACS is equipped

675
00:36:27,239 --> 00:36:30,040
Speaker 4: to with generative coding tools, and you know we have

676
00:36:30,080 --> 00:36:33,200
Speaker 4: twelve thousand of that. So we didn't enable yet the

677
00:36:33,239 --> 00:36:36,840
Speaker 4: ones that are using our own proprietary language called slang,

678
00:36:36,920 --> 00:36:39,720
Speaker 4: but everybody else has an AI tool and the resulso

679
00:36:39,800 --> 00:36:41,279
Speaker 4: be pretty extraordinary.

680
00:36:41,440 --> 00:36:43,360
Speaker 2: How do you measure that? What are what are some numbers?

681
00:36:43,440 --> 00:36:44,440
Speaker 2: Or how would you describe the right?

682
00:36:44,560 --> 00:36:48,000
Speaker 4: So we measure it according to a number of metrics,

683
00:36:48,000 --> 00:36:51,040
Speaker 4: such as the time that it takes from let's say

684
00:36:51,040 --> 00:36:53,400
Speaker 4: when you start the sprint, when you actually commit the code,

685
00:36:53,680 --> 00:36:56,000
Speaker 4: or when you complete your task. We measure it by

686
00:36:56,120 --> 00:36:58,680
Speaker 4: number of commits, meaning how many times you actually put

687
00:36:58,680 --> 00:37:01,720
Speaker 4: code into production. We measure it by a number of defects,

688
00:37:01,760 --> 00:37:04,880
Speaker 4: which in this case is like, for example, deployment related errors.

689
00:37:05,040 --> 00:37:08,319
Speaker 4: So there are more like velocity and quality metrics. At

690
00:37:08,360 --> 00:37:14,000
Speaker 4: the same time, we have seen a wide range ranging

691
00:37:14,040 --> 00:37:18,479
Speaker 4: from ten to forty percent productivity increase. I would say

692
00:37:18,480 --> 00:37:22,880
Speaker 4: that today we are probably on average seeing twenty percent. Now,

693
00:37:23,040 --> 00:37:25,880
Speaker 4: developers don't spend one hundred percent of their time coding.

694
00:37:26,640 --> 00:37:29,040
Speaker 4: They maybe spend fifty percent of their time coding. So

695
00:37:29,480 --> 00:37:31,759
Speaker 4: your question is what are they doing with half of

696
00:37:31,840 --> 00:37:35,160
Speaker 4: their times where there is a lot of other activities

697
00:37:35,239 --> 00:37:39,440
Speaker 4: such as documenting code, such as doing deployment, doing deployment scripts,

698
00:37:39,480 --> 00:37:41,799
Speaker 4: doing you know, buntio tests, et cetera, et cetera. So

699
00:37:41,840 --> 00:37:45,880
Speaker 4: what's called generally the software development life cycle. Okay, and

700
00:37:45,960 --> 00:37:49,120
Speaker 4: so we see net of ten percent. But then the

701
00:37:49,160 --> 00:37:51,320
Speaker 4: cool thing is that those AIS and the things that

702
00:37:51,360 --> 00:37:55,000
Speaker 4: we're building around that are starting to go beyond coding.

703
00:37:55,440 --> 00:37:57,799
Speaker 4: They're starting to help you write the right tests, write

704
00:37:57,800 --> 00:38:01,360
Speaker 4: the right documentation. They are even figure out algorithms or

705
00:38:01,400 --> 00:38:06,719
Speaker 4: even for example, reducing or minimizing the likelihood of deployment

706
00:38:06,800 --> 00:38:10,520
Speaker 4: issues writing deployment scripts for you. So as that expands,

707
00:38:10,800 --> 00:38:12,719
Speaker 4: we're going to be closer to one hundred percent, and

708
00:38:12,760 --> 00:38:14,920
Speaker 4: therefore we're going to be closer probably to twenty percent,

709
00:38:15,000 --> 00:38:17,120
Speaker 4: which you know, for an organization of our side, is

710
00:38:17,160 --> 00:38:18,680
Speaker 4: a pretty massive efficiency play.

711
00:38:18,800 --> 00:38:20,839
Speaker 3: Can I ask a question about hiring developers? So I've

712
00:38:20,840 --> 00:38:23,759
Speaker 3: probably read one hundred articles over the years about Wall

713
00:38:23,800 --> 00:38:26,640
Speaker 3: Street competing with tech companies to hire developers, like, oh,

714
00:38:26,680 --> 00:38:28,360
Speaker 3: they got a ping pong Lloyd Blank.

715
00:38:28,120 --> 00:38:30,040
Speaker 1: Fine used to say, they are a technology company.

716
00:38:30,160 --> 00:38:32,279
Speaker 3: Yeah, you gotta have your ping pong tables and your

717
00:38:32,320 --> 00:38:34,640
Speaker 3: free lunches and let people are sneakers and I have

718
00:38:34,719 --> 00:38:37,560
Speaker 3: all that stuff. But now it seems with AI, there's

719
00:38:37,680 --> 00:38:41,320
Speaker 3: a number of people interested in who are truly believing

720
00:38:41,400 --> 00:38:44,080
Speaker 3: that within a few years they might build the digital

721
00:38:44,120 --> 00:38:47,719
Speaker 3: god that's ten thousand times smarter than any human, and

722
00:38:47,800 --> 00:38:50,719
Speaker 3: that they approach the task with messianic fervor. And I

723
00:38:50,719 --> 00:38:53,799
Speaker 3: imagine it, right if you're at Goldman and you're trying

724
00:38:53,840 --> 00:38:57,360
Speaker 3: to help a banker answer a question to a client

725
00:38:57,520 --> 00:39:00,600
Speaker 3: about something in the chemical industry, like maybe that's not

726
00:39:00,680 --> 00:39:03,120
Speaker 3: like the thing that gets you out of bed the way,

727
00:39:03,280 --> 00:39:06,520
Speaker 3: sort of like metaphysical realms about what is the nature

728
00:39:06,560 --> 00:39:09,879
Speaker 3: of consciousness and things like that that people talk. Does

729
00:39:09,920 --> 00:39:13,200
Speaker 3: that present any challenges or anything when trying to hire

730
00:39:13,480 --> 00:39:15,080
Speaker 3: talented a developers.

731
00:39:15,560 --> 00:39:19,960
Speaker 4: I think developers love to solve real problems. And one

732
00:39:19,960 --> 00:39:22,520
Speaker 4: of the things also that attracted me in the first place,

733
00:39:23,080 --> 00:39:24,800
Speaker 4: Not that it matters, but I'm saying, you know, I

734
00:39:24,880 --> 00:39:28,160
Speaker 4: tell you my own personal experience is that working in

735
00:39:28,160 --> 00:39:31,920
Speaker 4: a technology company is absolutely fantastic, but you're always like

736
00:39:31,960 --> 00:39:35,000
Speaker 4: one step removed from the business or from the application.

737
00:39:35,160 --> 00:39:36,799
Speaker 4: So I have to you know, let's say you are

738
00:39:36,840 --> 00:39:40,160
Speaker 4: the bank and I'm the technology company. I need to

739
00:39:40,239 --> 00:39:41,920
Speaker 4: sell you a tool that then you're going to use

740
00:39:42,040 --> 00:39:45,200
Speaker 4: to run your business or improve your business. We are

741
00:39:45,280 --> 00:39:48,520
Speaker 4: kind of one degree of separation. Less I were right

742
00:39:48,560 --> 00:39:52,760
Speaker 4: there in a digital business there is fast, huge amounts

743
00:39:52,760 --> 00:39:55,920
Speaker 4: of data, huge amounts of flaws, immediate results, and that's

744
00:39:56,000 --> 00:40:00,360
Speaker 4: kind of addictive. And so developers, especially when a AIS

745
00:40:00,400 --> 00:40:02,839
Speaker 4: are starting to do all those magical things that we're

746
00:40:02,840 --> 00:40:06,320
Speaker 4: talking about, you know, they can see the impact on

747
00:40:06,360 --> 00:40:08,960
Speaker 4: the business right away, and then I think is kind

748
00:40:09,000 --> 00:40:10,799
Speaker 4: of attracting a lot of people. In fact, that there

749
00:40:10,880 --> 00:40:13,480
Speaker 4: is more and more people that are moving into the

750
00:40:13,560 --> 00:40:20,279
Speaker 4: industries oil and gas, transportation, chemical, medical, finance because you know,

751
00:40:20,320 --> 00:40:22,640
Speaker 4: this is new and there's nothing more exciting than seeing

752
00:40:22,640 --> 00:40:24,680
Speaker 4: it in action. And so there is so much action

753
00:40:24,840 --> 00:40:27,640
Speaker 4: going on that I think is actually really really interesting.

754
00:40:27,680 --> 00:40:29,680
Speaker 4: I think another question that maybe you haven't asked me,

755
00:40:29,680 --> 00:40:31,640
Speaker 4: but it's kind of part of this question, is what

756
00:40:31,760 --> 00:40:34,080
Speaker 4: kind of developers? How is the profession of being a

757
00:40:34,120 --> 00:40:35,400
Speaker 4: developers actually changed?

758
00:40:35,480 --> 00:40:37,719
Speaker 1: Oh wait, I had a related question. It's not quite

759
00:40:37,800 --> 00:40:40,680
Speaker 1: that question, but you can certainly answer that too. But Okay,

760
00:40:41,040 --> 00:40:45,000
Speaker 1: to my knowledge, Goldman Sachs doesn't have a job title

761
00:40:45,160 --> 00:40:49,920
Speaker 1: specifically with the words prompt engineer in it. So, looking

762
00:40:50,000 --> 00:40:53,799
Speaker 1: at the impact of AI on your business overall, is

763
00:40:53,880 --> 00:41:00,640
Speaker 1: AI a net hiring positive or a net hiring negative

764
00:41:01,239 --> 00:41:03,280
Speaker 1: for gold Men's employees overall?

765
00:41:05,120 --> 00:41:07,560
Speaker 4: Well, meaning, are we going to hire more or less development?

766
00:41:07,600 --> 00:41:10,000
Speaker 1: Yeah, it doesn't lead to more jobs because you're doing

767
00:41:10,080 --> 00:41:13,400
Speaker 1: more things and productivity increases. Or does it lead to

768
00:41:13,440 --> 00:41:16,080
Speaker 1: fewer jobs because now you can automate a bunch of stuffs.

769
00:41:16,120 --> 00:41:18,440
Speaker 4: Well, listen, there is so many things that we would

770
00:41:18,480 --> 00:41:20,920
Speaker 4: like to do if we had more resources that I

771
00:41:20,960 --> 00:41:23,920
Speaker 4: think this is going to be leading to more things

772
00:41:23,920 --> 00:41:26,600
Speaker 4: that we can do. You know, some people tell me sometimes,

773
00:41:26,600 --> 00:41:29,080
Speaker 4: so you're gonna maybe hire less or have less developers.

774
00:41:29,719 --> 00:41:32,240
Speaker 4: I don't know. I've been in it quote and quote

775
00:41:32,280 --> 00:41:35,279
Speaker 4: for like literally almost forty years, and I've never ever

776
00:41:35,360 --> 00:41:39,560
Speaker 4: seen that go down. But I've seen inflection points where

777
00:41:39,840 --> 00:41:42,600
Speaker 4: you can actually get developers to do way more and

778
00:41:42,760 --> 00:41:46,960
Speaker 4: worry about way less. There is not related to a

779
00:41:47,040 --> 00:41:49,759
Speaker 4: business outcome, and so I think it's more like how

780
00:41:49,800 --> 00:41:52,879
Speaker 4: the profession is going to change. In my opinion, we're

781
00:41:52,920 --> 00:41:56,880
Speaker 4: going to be less low level and more Hey, I

782
00:41:56,920 --> 00:41:59,440
Speaker 4: need to really understand the business problem. Hey, I really

783
00:41:59,520 --> 00:42:02,360
Speaker 4: need to think outcome driven. Ay, I need to have

784
00:42:02,400 --> 00:42:04,480
Speaker 4: a crisp mental model and I need to be able

785
00:42:04,480 --> 00:42:06,719
Speaker 4: to describe it in words. So the profession is going

786
00:42:06,800 --> 00:42:10,160
Speaker 4: to change, and there are tasks that I think are

787
00:42:10,239 --> 00:42:14,520
Speaker 4: so repetitive that the automation of those is actually going

788
00:42:14,600 --> 00:42:18,120
Speaker 4: to help developers, you know, really kind of feeling really

789
00:42:18,160 --> 00:42:21,239
Speaker 4: really connected with the business and with the strategy, and

790
00:42:21,239 --> 00:42:24,000
Speaker 4: that will attract people that are generally curious, that are

791
00:42:24,040 --> 00:42:27,239
Speaker 4: generally interested in understanding what we actually do. So the

792
00:42:27,239 --> 00:42:30,160
Speaker 4: focus kind of shifts from the how to do what

793
00:42:30,520 --> 00:42:33,080
Speaker 4: and to the why, which is really kind of the heart.

794
00:42:33,640 --> 00:42:35,799
Speaker 4: Or think of this evolution of technology over the years

795
00:42:35,800 --> 00:42:38,600
Speaker 4: from the back office of it, which doesn't even know

796
00:42:38,600 --> 00:42:40,440
Speaker 4: what you're doing, but as long as your monitor is

797
00:42:40,440 --> 00:42:44,520
Speaker 4: actually working to hey, I'm actually able to take a

798
00:42:44,520 --> 00:42:47,400
Speaker 4: business problem and break it down into pieces that then

799
00:42:47,440 --> 00:42:50,440
Speaker 4: even an AI can write code for. So to your

800
00:42:50,440 --> 00:42:54,759
Speaker 4: specific question, I think this might maybe potentially for some

801
00:42:54,800 --> 00:42:57,399
Speaker 4: companies are going to try to realize some of those

802
00:42:57,440 --> 00:43:01,319
Speaker 4: efficiencies by curbing the growth or even sometimes reducing it.

803
00:43:01,719 --> 00:43:05,520
Speaker 4: For companies like us that are extremely competitive, for companies

804
00:43:05,520 --> 00:43:08,160
Speaker 4: that have lots of ambition, this race at the end

805
00:43:08,160 --> 00:43:09,600
Speaker 4: of the day, and I think we're going to go

806
00:43:09,719 --> 00:43:12,279
Speaker 4: for you know, trying to get even more out of

807
00:43:12,320 --> 00:43:14,919
Speaker 4: our developers and actually like you know, trying to turn

808
00:43:14,960 --> 00:43:18,040
Speaker 4: them more into something that makes them feel super super connected.

809
00:43:18,080 --> 00:43:18,919
Speaker 4: To the business.

810
00:43:19,640 --> 00:43:23,480
Speaker 3: What about non developer roles, non tech roles, And you know, again,

811
00:43:23,520 --> 00:43:26,200
Speaker 3: I guess a company like Goldman doesn't have you know,

812
00:43:26,200 --> 00:43:29,120
Speaker 3: probably a lot of like low level customers support things

813
00:43:29,160 --> 00:43:31,000
Speaker 3: for in a window is like oh, I need to

814
00:43:31,160 --> 00:43:33,960
Speaker 3: change my plane ticket, et cetera. But you know, a

815
00:43:34,000 --> 00:43:38,440
Speaker 3: lot of modern work is essentially just answering somebody's basic question.

816
00:43:39,040 --> 00:43:41,480
Speaker 3: Are the roles within a bank that are going to

817
00:43:41,560 --> 00:43:44,880
Speaker 3: either fundamentally change or go away due to sort of

818
00:43:45,280 --> 00:43:46,880
Speaker 3: agentic or generative AI.

819
00:43:48,000 --> 00:43:50,800
Speaker 4: I think a lot of the work there is about

820
00:43:51,120 --> 00:43:56,279
Speaker 4: content production or content summarization will actually be streamlined quite

821
00:43:56,320 --> 00:44:00,359
Speaker 4: a bit, like, for example, taking an earnings report, making

822
00:44:00,400 --> 00:44:02,880
Speaker 4: it into ten different sauces in order to wear for

823
00:44:02,960 --> 00:44:05,960
Speaker 4: different channels of distribution. Here's the one for internal people,

824
00:44:05,960 --> 00:44:07,640
Speaker 4: here's the one for the client, here's the one for

825
00:44:07,680 --> 00:44:10,960
Speaker 4: the website, et cetera, et cetera. Imagine the creation of

826
00:44:11,000 --> 00:44:13,480
Speaker 4: pitch books for clients where you take ten plates, you

827
00:44:13,480 --> 00:44:15,480
Speaker 4: put a bunch of data, you go out and do research,

828
00:44:15,560 --> 00:44:17,680
Speaker 4: you take logos, you take this, you take that. There

829
00:44:17,760 --> 00:44:21,480
Speaker 4: is a lot of that machinery and factory, which you know,

830
00:44:21,480 --> 00:44:24,120
Speaker 4: we have thousands of people doing that I'm sure there's a.

831
00:44:24,080 --> 00:44:26,799
Speaker 1: Lot of junior analyst who would be maybe glad to

832
00:44:26,880 --> 00:44:29,160
Speaker 1: hear that some of making a pitchbox is going to

833
00:44:29,239 --> 00:44:29,440
Speaker 1: be on.

834
00:44:29,760 --> 00:44:31,640
Speaker 4: But I think that's a good thing. It takes away

835
00:44:31,719 --> 00:44:33,880
Speaker 4: some of the toil. And so I think at the

836
00:44:33,960 --> 00:44:36,960
Speaker 4: end of the day, listen right now, have you noticed

837
00:44:36,960 --> 00:44:40,480
Speaker 4: that everything is kind of converging to words and concepts,

838
00:44:40,600 --> 00:44:43,040
Speaker 4: no matter if you're a developer, if you're a knowledge worker,

839
00:44:43,320 --> 00:44:47,640
Speaker 4: those jobs are candle colliding. And I'm absolutely developers have

840
00:44:47,719 --> 00:44:51,880
Speaker 4: seen that first. Why well, because it's a low hanging fruit.

841
00:44:51,960 --> 00:44:54,400
Speaker 4: The developers deal with the vocabulary. There is no fifty

842
00:44:54,440 --> 00:44:57,480
Speaker 4: thousand words. There's like two three hundred keywords for language,

843
00:44:57,480 --> 00:44:59,480
Speaker 4: and so of course that works extremely well, and of

844
00:44:59,480 --> 00:45:01,879
Speaker 4: course that's the first thing to go. But I think

845
00:45:01,920 --> 00:45:05,359
Speaker 4: eventually the knowledge worker is going to be, you know,

846
00:45:05,480 --> 00:45:07,400
Speaker 4: the one that is really benefit and no matter if

847
00:45:07,440 --> 00:45:09,839
Speaker 4: you are a developer or or or if you are

848
00:45:10,080 --> 00:45:11,960
Speaker 4: working on a pitch book, or if you're working on

849
00:45:11,960 --> 00:45:14,440
Speaker 4: a summarization of a meeting or the action items, or

850
00:45:14,480 --> 00:45:16,920
Speaker 4: you're working on a strategy, et cetera, et cetera. And

851
00:45:16,960 --> 00:45:21,640
Speaker 4: I think overall this will elevate the quality of the work,

852
00:45:21,800 --> 00:45:24,920
Speaker 4: which then everybody says a happy worker or a happy

853
00:45:24,920 --> 00:45:27,799
Speaker 4: developer is a productive developer. I think you're happy when

854
00:45:27,800 --> 00:45:29,840
Speaker 4: you're actually doing something that allows you to do your

855
00:45:29,920 --> 00:45:33,480
Speaker 4: best work. And I'm hoping that if AI allows all

856
00:45:33,520 --> 00:45:36,440
Speaker 4: of us to do more of our best work, I

857
00:45:36,480 --> 00:45:38,280
Speaker 4: think it's going to be, you know, probably the biggest

858
00:45:38,320 --> 00:45:39,319
Speaker 4: effect that we can have.

859
00:45:39,719 --> 00:45:41,319
Speaker 1: I know, we just have a couple more minutes. So

860
00:45:41,400 --> 00:45:44,320
Speaker 1: one very quick question, what makes a good prompt?

861
00:45:45,160 --> 00:45:49,239
Speaker 4: Well, believe it or not. Empathy. You need to be empathic,

862
00:45:49,280 --> 00:45:51,080
Speaker 4: and you need to be gentle, and you need to

863
00:45:51,120 --> 00:45:54,080
Speaker 4: be kind, and you need to kind of, you know, just.

864
00:45:55,560 --> 00:45:57,040
Speaker 1: Like empathetic.

865
00:46:00,120 --> 00:46:01,760
Speaker 2: She makes fun of me for how empathement.

866
00:46:01,840 --> 00:46:04,320
Speaker 1: You know, I've said, it's very sweet that you say.

867
00:46:04,960 --> 00:46:07,480
Speaker 4: You need to take the AI literally by the hand

868
00:46:07,560 --> 00:46:09,600
Speaker 4: and take it where you want to go. And I

869
00:46:09,680 --> 00:46:11,759
Speaker 4: tell you that, you know, one of my interesting, more

870
00:46:11,800 --> 00:46:15,080
Speaker 4: interesting experience with prompts is the following. You know how

871
00:46:15,120 --> 00:46:17,520
Speaker 4: hard it is to get an AI to say I

872
00:46:17,560 --> 00:46:21,120
Speaker 4: don't know. It's almost impossible. You're always going to get

873
00:46:21,120 --> 00:46:24,080
Speaker 4: an ass And so one time I decided I want

874
00:46:24,120 --> 00:46:26,120
Speaker 4: to get it to the point, and so I had

875
00:46:26,160 --> 00:46:30,160
Speaker 4: to navigate the prompt and the AI to understand that

876
00:46:30,239 --> 00:46:33,040
Speaker 4: it was safe and okay to say I don't know.

877
00:46:33,800 --> 00:46:36,000
Speaker 4: And so then at the end I prompted it, what's

878
00:46:36,040 --> 00:46:39,840
Speaker 4: the capital of you know, the United States? Okay? And

879
00:46:39,880 --> 00:46:42,400
Speaker 4: then I said that you know, what's the weather going

880
00:46:42,440 --> 00:46:44,200
Speaker 4: to be like tomorrow? And I got an answer, and

881
00:46:44,200 --> 00:46:46,200
Speaker 4: then I said what's the weather going to be in

882
00:46:46,239 --> 00:46:50,000
Speaker 4: a year, and it's simply I don't know. And then

883
00:46:50,200 --> 00:46:53,200
Speaker 4: at one point, you know, I even decided what to say.

884
00:46:53,239 --> 00:46:55,719
Speaker 4: It's like, is there a role for humans in a

885
00:46:55,840 --> 00:46:57,400
Speaker 4: world of a eye?

886
00:46:59,400 --> 00:46:59,960
Speaker 2: I don't want to know?

887
00:47:04,400 --> 00:47:07,640
Speaker 1: Okay, Well, everyone's going to be off on chat GPT

888
00:47:07,840 --> 00:47:09,560
Speaker 1: now trying to get it to say I don't know.

889
00:47:09,800 --> 00:47:12,440
Speaker 1: Marco Argenti from Goldman Sachs, thank you so much. That

890
00:47:12,480 --> 00:47:12,920
Speaker 1: was good fun.

891
00:47:12,960 --> 00:47:14,520
Speaker 2: Thank you, Johing, thank you so much.

892
00:47:14,520 --> 00:47:28,959
Speaker 1: Thank you so much, Joe. That was a lot of fun.

893
00:47:29,000 --> 00:47:30,600
Speaker 1: And I have to say I do not make fun

894
00:47:30,600 --> 00:47:32,640
Speaker 1: of you for saying please and thank you to Chat GPT.

895
00:47:32,880 --> 00:47:35,240
Speaker 1: I have I'm going to repeat it. I've said it's endearing,

896
00:47:35,400 --> 00:47:38,560
Speaker 1: it's very sweet, and I've tried to follow your example.

897
00:47:38,600 --> 00:47:40,160
Speaker 1: And I now I don't say thank you because I

898
00:47:40,239 --> 00:47:42,239
Speaker 1: usually move on to the next question. But I do

899
00:47:42,280 --> 00:47:42,800
Speaker 1: say please.

900
00:47:43,080 --> 00:47:44,359
Speaker 2: I've heard this though.

901
00:47:44,400 --> 00:47:46,400
Speaker 3: It's funny that you said that, because I actually have

902
00:47:46,680 --> 00:47:50,640
Speaker 3: heard this that there does seem to be quantitative evidence

903
00:47:51,200 --> 00:47:54,000
Speaker 3: that words like please and thank you, et cetera do

904
00:47:54,360 --> 00:47:59,600
Speaker 3: actually improve really well, yeah, mad Buseegan, who you know

905
00:47:59,640 --> 00:48:02,640
Speaker 3: we've known on Twitter forever, has posted about this. So

906
00:48:03,400 --> 00:48:05,880
Speaker 3: there's a good reason to do it besides just the

907
00:48:06,000 --> 00:48:08,360
Speaker 3: habit the all entities you talk to, you should be

908
00:48:08,360 --> 00:48:09,160
Speaker 3: in the habit of flight.

909
00:48:09,239 --> 00:48:11,960
Speaker 1: Oh yeah, that was your argument, right, yeah, yeah, yeah, Okay,

910
00:48:11,960 --> 00:48:13,520
Speaker 1: Well I thought that was fascinating.

911
00:48:13,600 --> 00:48:14,000
Speaker 2: Yeah.

912
00:48:14,040 --> 00:48:16,399
Speaker 1: We've been talking a lot about AI and the sort

913
00:48:16,400 --> 00:48:19,800
Speaker 1: of potential use cases and the chips that are driving

914
00:48:19,840 --> 00:48:21,839
Speaker 1: the technology and things like that, but it was nice

915
00:48:21,840 --> 00:48:25,279
Speaker 1: to hear from someone who's actually making the purchasing decisions, yes,

916
00:48:25,360 --> 00:48:27,719
Speaker 1: and implementing them at a large institution.

917
00:48:28,440 --> 00:48:29,040
Speaker 2: Absolutely.

918
00:48:29,040 --> 00:48:31,880
Speaker 3: That was probably one of my favorite AI conversations we

919
00:48:31,960 --> 00:48:36,320
Speaker 3: had for precisely that reason, because it was interesting hearing

920
00:48:36,400 --> 00:48:39,399
Speaker 3: him talk about this idea that right now, like these

921
00:48:39,440 --> 00:48:43,080
Speaker 3: open source models, particularly like the latest version of LAMA,

922
00:48:43,239 --> 00:48:46,920
Speaker 3: is getting really close to sort of the core proprietary models.

923
00:48:47,160 --> 00:48:50,879
Speaker 3: That was striking the fact that he sees, perhaps particularly

924
00:48:50,920 --> 00:48:55,479
Speaker 3: on the inference side of model usage, an opportunity for

925
00:48:55,600 --> 00:48:58,400
Speaker 3: greater use of different types of hardware.

926
00:48:58,440 --> 00:49:01,480
Speaker 1: Also very interesting, that's right, And we're so used to

927
00:49:01,520 --> 00:49:04,560
Speaker 1: talking about the massive amounts of power and energy that

928
00:49:04,600 --> 00:49:07,200
Speaker 1: AI will consume, and we you and I have had

929
00:49:07,280 --> 00:49:09,480
Speaker 1: a lot of conversations about how we're going to power

930
00:49:09,600 --> 00:49:13,279
Speaker 1: all these servers and things. But what's gotten far less

931
00:49:13,360 --> 00:49:17,800
Speaker 1: attention is just optimizing the way you use AI such

932
00:49:17,840 --> 00:49:20,160
Speaker 1: that you don't need to consume as much power, So

933
00:49:20,280 --> 00:49:23,840
Speaker 1: maybe doing less training, leaving training to the big like

934
00:49:23,960 --> 00:49:26,800
Speaker 1: hyperscalers or whatever, and then just doing the inference.

935
00:49:27,080 --> 00:49:28,840
Speaker 3: In the end, it's going to be both, right, because

936
00:49:28,880 --> 00:49:31,439
Speaker 3: in the end, like there's both, it's going to happen.

937
00:49:31,440 --> 00:49:35,160
Speaker 3: People are gonna find algorithmic techniques and Marco described some

938
00:49:35,280 --> 00:49:39,440
Speaker 3: of them to lessen the sort of pressure and stress

939
00:49:39,480 --> 00:49:42,280
Speaker 3: that you're putting on the hardware, but of course that's

940
00:49:42,360 --> 00:49:44,920
Speaker 3: just going to mean you're going to use it more.

941
00:49:45,040 --> 00:49:46,799
Speaker 3: And then also people are going to have to solve

942
00:49:46,800 --> 00:49:49,719
Speaker 3: the power consumptions. That kind of like all of economic

943
00:49:49,800 --> 00:49:53,400
Speaker 3: history in general, in which we're always finding new ways

944
00:49:53,440 --> 00:49:56,440
Speaker 3: to get more out of the same you know, gigajewel

945
00:49:56,640 --> 00:49:59,680
Speaker 3: of energy but also using more energy at the same time.

946
00:49:59,760 --> 00:50:00,240
Speaker 4: Yeah.

947
00:50:00,280 --> 00:50:02,600
Speaker 1: Absolutely, well, shall we leave it there.

948
00:50:02,680 --> 00:50:03,359
Speaker 2: Let's leave it there.

949
00:50:03,520 --> 00:50:06,360
Speaker 1: This has been another episode of the aud Thoughts podcast.

950
00:50:06,440 --> 00:50:09,760
Speaker 1: I'm Tracy Alloway. You can follow me at Tracy Alloway.

951
00:50:09,360 --> 00:50:12,239
Speaker 3: And I'm Jill Wisenthal. You can follow me at the Stalwart.

952
00:50:12,480 --> 00:50:16,000
Speaker 3: Follow our producers Carman Rodriguez at Carman Ermann dash O,

953
00:50:16,040 --> 00:50:19,560
Speaker 3: Bennett at Dashbot, and kel Brooks at Kelbrooks. Thank you

954
00:50:19,600 --> 00:50:22,720
Speaker 3: to our producer Moses ONEm and from our Odd Lots content.

955
00:50:22,760 --> 00:50:25,879
Speaker 3: Go to Bloomberg dot com slash od loss. We have transcripts,

956
00:50:25,880 --> 00:50:28,440
Speaker 3: a blog, and a newsletter, and you can chat about

957
00:50:28,440 --> 00:50:30,759
Speaker 3: all of these topics in our discord where we even

958
00:50:30,800 --> 00:50:34,160
Speaker 3: have an AI channel. Great stuff in their discord dot

959
00:50:34,200 --> 00:50:35,440
Speaker 3: gg slash Odlins.

960
00:50:35,719 --> 00:50:38,200
Speaker 1: And if you enjoy Odd Loots, if you like our

961
00:50:38,280 --> 00:50:41,960
Speaker 1: continuing series of AI conversations, then please leave us a

962
00:50:42,000 --> 00:50:45,880
Speaker 1: positive review on your favorite podcast platform. And remember, if

963
00:50:45,920 --> 00:50:48,279
Speaker 1: you are a Bloomberg subscriber, you can listen to all

964
00:50:48,320 --> 00:50:51,360
Speaker 1: of our episodes absolutely ad free. All you need to

965
00:50:51,400 --> 00:50:54,840
Speaker 1: do is connect your Bloomberg account with Apple Podcasts. In

966
00:50:54,960 --> 00:50:57,080
Speaker 1: order to do that, just find the Bloomberg channel on

967
00:50:57,160 --> 00:51:00,760
Speaker 1: Apple Podcasts and follow the instructions there. Thanks for listening.