1
00:00:02,720 --> 00:00:17,079
Speaker 1: Bloomberg Audio Studios, Podcasts, Radio News.

2
00:00:18,040 --> 00:00:20,760
Speaker 2: Hello and welcome to another episode of the Odd Thoughts podcast.

3
00:00:20,840 --> 00:00:23,640
Speaker 3: I'm Tracy Alloway and I'm Joe. Why isn't thal Joe?

4
00:00:23,960 --> 00:00:28,240
Speaker 2: The thing about AI? I feel like it's accelerated all

5
00:00:28,280 --> 00:00:31,680
Speaker 2: of our timelines, right, Like it's phenomenal to me to

6
00:00:31,720 --> 00:00:34,279
Speaker 2: think back that, like back to the days of chat

7
00:00:34,400 --> 00:00:36,239
Speaker 2: gept and when did that come out?

8
00:00:36,280 --> 00:00:39,600
Speaker 4: Twenty twenty, twenty twenty two, twenty number two, twenty two.

9
00:00:39,760 --> 00:00:41,959
Speaker 2: That's just crazy to think, and.

10
00:00:41,920 --> 00:00:45,559
Speaker 4: That's really unbelievable, the gap. And I've been thinking about

11
00:00:45,600 --> 00:00:49,280
Speaker 4: this like just a the explosion of capabilities. Yeah, And

12
00:00:49,320 --> 00:00:51,159
Speaker 4: the thing I've been thinking about is that, you know,

13
00:00:51,200 --> 00:00:52,959
Speaker 4: after the first year or so when it came out,

14
00:00:53,080 --> 00:00:55,360
Speaker 4: we talked to executives and it's like, how are you

15
00:00:55,440 --> 00:00:58,960
Speaker 4: using AI and your workflow? Everyone's experimenting. It's great, everyone

16
00:00:59,120 --> 00:01:01,320
Speaker 4: is using cha GP two. It's always very vague. And

17
00:01:01,360 --> 00:01:03,640
Speaker 4: now in twenty twenty six, to the story is that

18
00:01:03,680 --> 00:01:06,200
Speaker 4: AI is so powerful that it's going to destroy all

19
00:01:06,240 --> 00:01:09,039
Speaker 4: these legacy software companies. So what I would say is

20
00:01:09,440 --> 00:01:12,720
Speaker 4: we must be past the age of experimentation. I think

21
00:01:12,760 --> 00:01:16,280
Speaker 4: that any of the cases using it better have some

22
00:01:16,440 --> 00:01:20,280
Speaker 4: example of like here's a workflow where we're using well exactly.

23
00:01:20,280 --> 00:01:22,679
Speaker 2: And to this point, now that we're past the age

24
00:01:22,680 --> 00:01:27,200
Speaker 2: of experimentation, I'm very curious how executives and managers are

25
00:01:27,200 --> 00:01:31,000
Speaker 2: actually evaluating the return on investment in AI and what

26
00:01:31,080 --> 00:01:33,600
Speaker 2: they actually want to see from it at this point. So,

27
00:01:33,720 --> 00:01:36,279
Speaker 2: you know, are you going to replace all your third

28
00:01:36,319 --> 00:01:40,440
Speaker 2: party SaaS contractors with internal coders and what does that

29
00:01:40,480 --> 00:01:44,040
Speaker 2: look like from an actual headcount perspective, from a cost

30
00:01:44,080 --> 00:01:48,000
Speaker 2: savings perspective. We can actually get some concrete details on

31
00:01:48,040 --> 00:01:50,400
Speaker 2: this now, So I'm very excited to say we do,

32
00:01:50,480 --> 00:01:52,840
Speaker 2: in fact have the perfect guest, someone who we had

33
00:01:52,920 --> 00:01:57,280
Speaker 2: on before to talk generally about AI and someone at

34
00:01:57,320 --> 00:01:59,840
Speaker 2: a company that has been doing you know, they got

35
00:01:59,880 --> 00:02:02,680
Speaker 2: it it pretty fast. The last time we spoke to

36
00:02:02,680 --> 00:02:05,760
Speaker 2: this person was in twenty twenty four, and even since

37
00:02:05,800 --> 00:02:08,440
Speaker 2: then a few years ago, I gues, it just feels

38
00:02:08,480 --> 00:02:11,320
Speaker 2: like light years in an AI time.

39
00:02:11,400 --> 00:02:14,640
Speaker 4: So just one last thing on the last few years

40
00:02:14,639 --> 00:02:17,720
Speaker 4: of AI, which is that when chagbt I used when

41
00:02:17,760 --> 00:02:19,799
Speaker 4: it came out, I played around with hell a lot

42
00:02:19,840 --> 00:02:21,960
Speaker 4: when I hadn't write poems and all this stuff, and

43
00:02:22,000 --> 00:02:23,919
Speaker 4: then I bet if you actually looked at my AI

44
00:02:24,000 --> 00:02:26,840
Speaker 4: usage went through a trough, whereas like I wasn't really

45
00:02:26,840 --> 00:02:30,040
Speaker 4: getting any productivity, there was nothing it really could do

46
00:02:30,160 --> 00:02:32,519
Speaker 4: that I needed. It still sort of seemed like a toy.

47
00:02:32,600 --> 00:02:34,320
Speaker 4: So I had this like a tense burst of use

48
00:02:34,639 --> 00:02:37,520
Speaker 4: for the first several months, and then this trough. And

49
00:02:37,560 --> 00:02:40,919
Speaker 4: now these days, with the expansion of capabilities, particularly claud code,

50
00:02:40,919 --> 00:02:42,840
Speaker 4: I'm finding all kinds of new things. So there is

51
00:02:42,880 --> 00:02:44,640
Speaker 4: like we're out coming out of the trough. I think

52
00:02:44,639 --> 00:02:46,760
Speaker 4: a lot of people are actually finding things, at least

53
00:02:47,120 --> 00:02:49,000
Speaker 4: if I can generalize from my own experience.

54
00:02:49,200 --> 00:02:49,519
Speaker 5: Yeah.

55
00:02:49,560 --> 00:02:52,000
Speaker 2: Absolutely, So we do, in fact have the perfect guest

56
00:02:52,200 --> 00:02:55,680
Speaker 2: we've brought back. Marco Argenti is, of course, the chief

57
00:02:55,680 --> 00:02:59,080
Speaker 2: information officer over at Goldman Sachs, someone we had on

58
00:02:59,160 --> 00:03:02,600
Speaker 2: the podcast back in August of twenty twenty four. So Marco,

59
00:03:02,680 --> 00:03:04,079
Speaker 2: thank you so much for coming back on.

60
00:03:04,880 --> 00:03:05,720
Speaker 3: Thank you for having me.

61
00:03:06,120 --> 00:03:08,960
Speaker 2: How much have things changed for you? Does twenty twenty

62
00:03:09,000 --> 00:03:12,320
Speaker 2: four seem like twenty years ago now in AI time?

63
00:03:12,639 --> 00:03:15,240
Speaker 5: Yeah, I barely remember even what happened back then.

64
00:03:16,720 --> 00:03:18,320
Speaker 2: That's a nice way of saying you forgot what we

65
00:03:18,360 --> 00:03:19,400
Speaker 2: talked about on the podcast.

66
00:03:19,480 --> 00:03:20,240
Speaker 3: That's funny with.

67
00:03:20,160 --> 00:03:23,360
Speaker 5: That, but literally, like things are really changing on a

68
00:03:23,360 --> 00:03:25,880
Speaker 5: weekly basis, almost right now. And if I look at

69
00:03:25,919 --> 00:03:29,360
Speaker 5: the evolution not only since a year ago, but even

70
00:03:29,400 --> 00:03:32,560
Speaker 5: since six months ago, I think it has been nothing

71
00:03:32,680 --> 00:03:36,080
Speaker 5: short than a revolutionary. A year ago, we barely talked

72
00:03:36,080 --> 00:03:39,880
Speaker 5: about agents or the world almost didn't exist. We were

73
00:03:40,000 --> 00:03:42,600
Speaker 5: using AI's like a chat companions.

74
00:03:42,760 --> 00:03:44,720
Speaker 3: Yeah, there was such function.

75
00:03:44,920 --> 00:03:47,640
Speaker 5: Yeah, it was telling you, oh, I'm sorry, I don't

76
00:03:47,680 --> 00:03:50,160
Speaker 5: know who's the president of the United States because my

77
00:03:50,280 --> 00:03:52,680
Speaker 5: cutoff date is like a year and a half before,

78
00:03:52,800 --> 00:03:56,040
Speaker 5: or things of that nature. Now you can say, hey,

79
00:03:56,520 --> 00:03:58,840
Speaker 5: as a person, you can say, hey, my plan just

80
00:03:58,840 --> 00:04:01,200
Speaker 5: got canceled, and this going to redo all your plans,

81
00:04:01,280 --> 00:04:03,400
Speaker 5: is going to check for like available flights, it's going

82
00:04:03,440 --> 00:04:06,440
Speaker 5: to do all these capabilities of personal assistance, and that

83
00:04:06,520 --> 00:04:09,640
Speaker 5: translates in corporations also in a lot of utility that

84
00:04:09,680 --> 00:04:13,080
Speaker 5: you can see in everyday's tasks. So I would say,

85
00:04:13,120 --> 00:04:15,640
Speaker 5: you know what I like to say to my people,

86
00:04:15,720 --> 00:04:17,679
Speaker 5: but also in general, let's say, this is not a drill.

87
00:04:17,880 --> 00:04:20,120
Speaker 5: This is real. You know, it's not the age of

88
00:04:20,160 --> 00:04:23,360
Speaker 5: experimentation anymore. There is a tool that now can do

89
00:04:23,600 --> 00:04:26,360
Speaker 5: a lot for you. And so we put it at work,

90
00:04:26,600 --> 00:04:28,839
Speaker 5: and we put it at work starting from developers, but

91
00:04:28,880 --> 00:04:33,039
Speaker 5: don't expanding in many many other areas. So I would say, actually,

92
00:04:33,080 --> 00:04:36,320
Speaker 5: if I look at the increase of capabilities of these models,

93
00:04:36,760 --> 00:04:39,240
Speaker 5: what we've seen in the last six months or so,

94
00:04:39,360 --> 00:04:42,760
Speaker 5: with really the evolution of this advanced reasoning capabilities that

95
00:04:42,920 --> 00:04:46,360
Speaker 5: came out, I think that finally got us the confidence

96
00:04:46,440 --> 00:04:49,840
Speaker 5: that you can use AIS for everyday's work with the

97
00:04:49,920 --> 00:04:53,520
Speaker 5: right supervision, and also in many cases for mission critical applications.

98
00:04:54,080 --> 00:04:56,480
Speaker 5: It's not a toy anymore. It's something that you can

99
00:04:56,560 --> 00:05:00,320
Speaker 5: expect results from, and I think that's the biggest change today.

100
00:05:00,360 --> 00:05:03,520
Speaker 5: I would say that there is almost nobody in Goldman

101
00:05:03,600 --> 00:05:06,000
Speaker 5: that is not touched in a way or another by AIS.

102
00:05:06,080 --> 00:05:10,320
Speaker 5: We gave our GSI or GSA system to forty seven

103
00:05:10,400 --> 00:05:14,320
Speaker 5: thousand people, most of them use it every day, most

104
00:05:14,360 --> 00:05:17,839
Speaker 5: of them used multiple times a day. And what's interesting

105
00:05:18,000 --> 00:05:20,719
Speaker 5: is it's like the first time you see a tool

106
00:05:20,800 --> 00:05:24,160
Speaker 5: like I don't know, Microsoft Excel, you can almost not

107
00:05:24,240 --> 00:05:26,200
Speaker 5: predict what people are going to do with that. Okay,

108
00:05:26,200 --> 00:05:29,680
Speaker 5: maybe it's born for doing some form of accounting, and

109
00:05:29,680 --> 00:05:32,000
Speaker 5: then people write entire applications on top of that. Or

110
00:05:32,080 --> 00:05:35,200
Speaker 5: use it for project managers, management or things of that nature.

111
00:05:36,200 --> 00:05:39,599
Speaker 5: And AI is kind of turning that way. If I

112
00:05:39,640 --> 00:05:43,360
Speaker 5: look at what people do with that, it's really things

113
00:05:43,360 --> 00:05:45,800
Speaker 5: that surprises every day because.

114
00:05:46,040 --> 00:05:48,520
Speaker 4: Well, what do you give us some examples? So in

115
00:05:48,680 --> 00:05:53,320
Speaker 4: production right now, what are some workflows or novel things

116
00:05:53,360 --> 00:05:57,080
Speaker 4: that were not workflows before that you see within Goldman

117
00:05:57,240 --> 00:05:59,599
Speaker 4: that AI is doing for people today.

118
00:06:00,040 --> 00:06:05,520
Speaker 5: Let's start from the GS assystem that can answer really

119
00:06:05,560 --> 00:06:12,760
Speaker 5: complex questions based on external and internal data that generally

120
00:06:12,800 --> 00:06:17,000
Speaker 5: before used to take sometimes hours or even days, sometimes

121
00:06:17,040 --> 00:06:20,360
Speaker 5: weeks to answer. Okay, it can do very complex research

122
00:06:20,440 --> 00:06:23,240
Speaker 5: for you in topics. You know, for example, we can

123
00:06:23,279 --> 00:06:26,440
Speaker 5: ask questions that come from clients such as, hey, how

124
00:06:26,480 --> 00:06:29,719
Speaker 5: does the recent geopolitical events on the hormone strait actually

125
00:06:29,760 --> 00:06:31,039
Speaker 5: impact the portfolio?

126
00:06:31,080 --> 00:06:33,120
Speaker 3: What could be a potential rebalancing strategy?

127
00:06:33,640 --> 00:06:36,599
Speaker 5: Or you could ask intersection of like I don't know,

128
00:06:36,640 --> 00:06:40,240
Speaker 5: how does the certain FED decision or interest rate actually

129
00:06:40,240 --> 00:06:43,280
Speaker 5: impact the volatility of certain assets? So you ask these

130
00:06:43,600 --> 00:06:49,320
Speaker 5: multi dimensional questions, and what the saassystem does calls out

131
00:06:49,320 --> 00:06:54,000
Speaker 5: a model, retrieves the relevant information, creates a plan to

132
00:06:54,120 --> 00:06:57,560
Speaker 5: answer that question. That's kind of the key because this AI,

133
00:06:57,720 --> 00:07:01,600
Speaker 5: is they really plan before responding rather than just giving

134
00:07:01,640 --> 00:07:04,360
Speaker 5: you the first thing that comes to mind. And so

135
00:07:04,400 --> 00:07:07,080
Speaker 5: that's what kind of at the very surface thinks that

136
00:07:07,120 --> 00:07:09,640
Speaker 5: one of the most common use cases, which is we

137
00:07:09,800 --> 00:07:13,120
Speaker 5: really enhance the client experience by being able to answer

138
00:07:13,320 --> 00:07:16,520
Speaker 5: questions internally and external in a much much faster way.

139
00:07:16,560 --> 00:07:19,520
Speaker 5: But really complex question not simple questions. We had to

140
00:07:19,560 --> 00:07:23,760
Speaker 5: wire up hundreds of data sources and also, most importantly,

141
00:07:23,800 --> 00:07:26,320
Speaker 5: which is something that I tell everybody that asks me, Hey,

142
00:07:26,400 --> 00:07:29,640
Speaker 5: give me some advice on how to implement AI in

143
00:07:29,640 --> 00:07:34,920
Speaker 5: a corporation, data quality is really the determinant between.

144
00:07:34,560 --> 00:07:36,520
Speaker 3: Good AI and not so good AI.

145
00:07:37,320 --> 00:07:39,040
Speaker 5: So we did a lot of work to not only

146
00:07:39,080 --> 00:07:42,040
Speaker 5: take a bunch of data but also making it understandable

147
00:07:42,080 --> 00:07:42,840
Speaker 5: to the AI.

148
00:07:43,040 --> 00:07:44,280
Speaker 3: So, for example, just to.

149
00:07:44,240 --> 00:07:47,520
Speaker 5: Go a little bit deeper, we have a tool called

150
00:07:47,560 --> 00:07:50,840
Speaker 5: legend AI, which is our lake house, which allows you

151
00:07:50,880 --> 00:07:55,080
Speaker 5: to go from query to MCP server connected to GSA

152
00:07:55,160 --> 00:07:58,320
Speaker 5: assistant IE, from data to answers. You can wire it

153
00:07:58,400 --> 00:08:01,320
Speaker 5: up literally in two or three clicks, and it does

154
00:08:01,400 --> 00:08:04,000
Speaker 5: all of that for you. So the quality of the data,

155
00:08:04,040 --> 00:08:06,240
Speaker 5: the quantity of the data, not only it's not just

156
00:08:06,760 --> 00:08:09,040
Speaker 5: the bitter lesson here, but it's also the lesson of

157
00:08:09,240 --> 00:08:12,600
Speaker 5: you need to curate your data, you get better answer disproportionately.

158
00:08:12,640 --> 00:08:13,880
Speaker 3: That's something that has driven that.

159
00:08:14,400 --> 00:08:18,720
Speaker 5: So there is kind of the knowledge aspect of AI,

160
00:08:18,840 --> 00:08:21,920
Speaker 5: which is I would say the most widespread because every

161
00:08:21,960 --> 00:08:24,239
Speaker 5: single one in the forma has that and it's.

162
00:08:24,080 --> 00:08:24,840
Speaker 3: The highest uses.

163
00:08:24,880 --> 00:08:27,320
Speaker 5: We are like way above a million prompts per month

164
00:08:27,680 --> 00:08:31,160
Speaker 5: and it's growing really really fast. And then of course

165
00:08:31,200 --> 00:08:33,880
Speaker 5: you know you're asking me like real impact in production.

166
00:08:34,559 --> 00:08:38,720
Speaker 5: Every developer in Goldman is enabled with agentic AI. Okay,

167
00:08:38,800 --> 00:08:40,560
Speaker 5: so we were probably one of the first, if not

168
00:08:40,559 --> 00:08:43,079
Speaker 5: the first, to launch a DEVIN almost a year ago,

169
00:08:43,720 --> 00:08:47,679
Speaker 5: which is the fully Edgentic Developer Assistant. We have cloud

170
00:08:47,720 --> 00:08:50,560
Speaker 5: code that, we have many other tools geitub copilot, agent,

171
00:08:50,679 --> 00:08:51,120
Speaker 5: et cetera.

172
00:08:51,240 --> 00:08:54,320
Speaker 3: But on that you really see the step change.

173
00:08:54,360 --> 00:08:57,240
Speaker 5: There is no question that there is changing the way

174
00:08:57,320 --> 00:09:01,400
Speaker 5: developers work. And by the way, it's not just about

175
00:09:01,480 --> 00:09:05,520
Speaker 5: doing the exact same things more efficient. It's changing the

176
00:09:05,520 --> 00:09:09,000
Speaker 5: way developers actually do their work. And that is very

177
00:09:09,160 --> 00:09:12,520
Speaker 5: very easy to see how that kind of changes the

178
00:09:12,559 --> 00:09:15,760
Speaker 5: paradigm of what a developer does. You're much more of

179
00:09:15,800 --> 00:09:18,319
Speaker 5: a product manager, you're much more of a planner, you're

180
00:09:18,400 --> 00:09:21,120
Speaker 5: much more of an idea. Generally, the most important thing

181
00:09:21,160 --> 00:09:23,200
Speaker 5: for the developer today is to be able to explain

182
00:09:23,280 --> 00:09:27,320
Speaker 5: things rather than jumping on other things.

183
00:09:28,400 --> 00:09:29,520
Speaker 3: I don't know you want to know that.

184
00:09:29,840 --> 00:09:31,640
Speaker 4: I'm just going to say that resonates because I've been

185
00:09:31,760 --> 00:09:33,720
Speaker 4: like vibe coding, but I can't explain how any of

186
00:09:33,720 --> 00:09:35,599
Speaker 4: it works. So someone is like, you know, I like

187
00:09:35,640 --> 00:09:38,120
Speaker 4: the little like toy apps and stuff that I get

188
00:09:38,120 --> 00:09:40,320
Speaker 4: really anxious. I couldn't explain. That's why I'm not a

189
00:09:40,440 --> 00:09:42,440
Speaker 4: software Yeah, well, just.

190
00:09:42,360 --> 00:09:44,680
Speaker 2: On this note. I mean, people tend to talk in

191
00:09:44,720 --> 00:09:49,160
Speaker 2: generalities when it comes to AI boosting productivity, or maybe

192
00:09:49,559 --> 00:09:51,720
Speaker 2: AI changes the way we work, or it leads to

193
00:09:51,800 --> 00:09:56,240
Speaker 2: some new ideas. From your seat at Goldman, you're a manager.

194
00:09:56,640 --> 00:09:59,520
Speaker 2: You're looking at the bottom line of like all these businesses,

195
00:10:00,360 --> 00:10:04,000
Speaker 2: what exactly is the outcome, the specific outcome that you

196
00:10:04,000 --> 00:10:07,920
Speaker 2: would like to see from your developers using something like

197
00:10:08,000 --> 00:10:08,720
Speaker 2: cloud code.

198
00:10:10,040 --> 00:10:12,840
Speaker 5: It's really about increasing the output, so I want to

199
00:10:12,840 --> 00:10:15,320
Speaker 5: see I was actually having this discussion this morning.

200
00:10:15,760 --> 00:10:16,920
Speaker 3: I was looking at some of.

201
00:10:16,880 --> 00:10:19,760
Speaker 5: The reports on some of the deliverables for our cloud migration,

202
00:10:19,920 --> 00:10:22,600
Speaker 5: which is a very important thing for us, and I

203
00:10:22,679 --> 00:10:25,959
Speaker 5: was looking at this really big project that was saying

204
00:10:26,920 --> 00:10:29,040
Speaker 5: it was not only green, it was like two months

205
00:10:29,080 --> 00:10:32,840
Speaker 5: ahead of schedule, And I was saying, this is how

206
00:10:32,880 --> 00:10:34,920
Speaker 5: we know when things are going to work. You're going

207
00:10:35,000 --> 00:10:38,960
Speaker 5: to consistently start seeing projects that are actually finishing ahead

208
00:10:39,000 --> 00:10:41,880
Speaker 5: of schedule, which means that then people are ambitious, they

209
00:10:41,920 --> 00:10:44,160
Speaker 5: want to do more, and therefore you end up with

210
00:10:44,280 --> 00:10:46,840
Speaker 5: output that is much higher than what you had before.

211
00:10:47,400 --> 00:10:50,600
Speaker 5: And listen with developers, obviously, the biggest question that everybody

212
00:10:50,600 --> 00:10:52,360
Speaker 5: asks is Okay, what are you going to do? Are

213
00:10:52,360 --> 00:10:56,600
Speaker 5: you gonna cut developers this and that? So, first of all,

214
00:10:57,440 --> 00:10:59,480
Speaker 5: with all the innovation that I've seen in the last

215
00:10:59,480 --> 00:11:01,640
Speaker 5: three years, so I kind of never seen a moment

216
00:11:01,720 --> 00:11:04,760
Speaker 5: where really people were reducing the number of developers. Because

217
00:11:04,760 --> 00:11:06,720
Speaker 5: if I look at the things that we're not doing

218
00:11:07,040 --> 00:11:09,719
Speaker 5: in a certain year because of budget reasons, because of

219
00:11:09,760 --> 00:11:13,040
Speaker 5: complexity reason because of prioritization, the stuff that is below

220
00:11:13,600 --> 00:11:16,679
Speaker 5: the cut of the backlog, it's a lot and a

221
00:11:16,760 --> 00:11:19,040
Speaker 5: lot of that is really driving the growth of the business.

222
00:11:19,559 --> 00:11:21,959
Speaker 5: So it's good to have the optionality to do it.

223
00:11:22,200 --> 00:11:23,839
Speaker 5: Do you have the optionality of say, now I have

224
00:11:23,880 --> 00:11:25,760
Speaker 5: one hundred and twenty percent of my capacity. I have

225
00:11:25,760 --> 00:11:27,760
Speaker 5: one hundred and thirty percent of my capacity. Do I

226
00:11:27,800 --> 00:11:29,520
Speaker 5: want to do one hundred and thirty percent more?

227
00:11:30,000 --> 00:11:30,320
Speaker 3: Great?

228
00:11:30,679 --> 00:11:33,120
Speaker 5: If I don't, I have the option to reduce. So

229
00:11:33,160 --> 00:11:36,080
Speaker 5: that's really how we measure it. It's really the impact

230
00:11:36,200 --> 00:11:37,600
Speaker 5: on the timelines of delivery.

231
00:11:37,679 --> 00:11:38,360
Speaker 3: It's output.

232
00:11:38,600 --> 00:11:43,240
Speaker 5: It's basical equality and timeline becoming. Quality gets actually better

233
00:11:44,400 --> 00:11:45,600
Speaker 5: and the timelines gets short.

234
00:11:46,080 --> 00:11:47,320
Speaker 3: So that's what we mentioned.

235
00:12:03,000 --> 00:12:06,040
Speaker 4: Obviously, one of the big questions for the market this

236
00:12:06,200 --> 00:12:10,280
Speaker 4: year is what is the impact of AI on legacy

237
00:12:10,320 --> 00:12:13,120
Speaker 4: software providers? And there's various theories about how they could

238
00:12:13,120 --> 00:12:17,120
Speaker 4: be disrupted. There are reports I think about Anthropic having

239
00:12:17,200 --> 00:12:21,720
Speaker 4: quote four deployed engineers inside Goldman Sachs, so Entropic employees

240
00:12:21,720 --> 00:12:25,520
Speaker 4: building out AI systems internally. Maybe that could or place

241
00:12:25,559 --> 00:12:30,560
Speaker 4: some legacy software right now. Can you say like there

242
00:12:30,720 --> 00:12:33,320
Speaker 4: is a change in the balance of power when there

243
00:12:33,520 --> 00:12:36,480
Speaker 4: is a given piece of software up for negotiation.

244
00:12:37,040 --> 00:12:40,000
Speaker 5: I think generally there is. Okay, if first of all,

245
00:12:40,720 --> 00:12:43,720
Speaker 5: there is kind of always been that tension because imagine,

246
00:12:43,760 --> 00:12:47,840
Speaker 5: for example, imagine when there were software they didn't run

247
00:12:47,880 --> 00:12:49,520
Speaker 5: on the cloud, and then all of a sudden, a

248
00:12:49,520 --> 00:12:51,520
Speaker 5: bunch of new vendors are coming to you and say, hey,

249
00:12:51,600 --> 00:12:53,600
Speaker 5: wait a cycle. Why are you running that on your

250
00:12:53,640 --> 00:12:55,920
Speaker 5: mainframe or your own prem Why don't you run it

251
00:12:55,960 --> 00:12:58,200
Speaker 5: in the cloud? Or remember when software needed to be

252
00:12:58,240 --> 00:13:02,199
Speaker 5: installed then everything became browser based and scess. So there's

253
00:13:02,240 --> 00:13:04,840
Speaker 5: always been a little bit of a cycle and a renewal.

254
00:13:04,840 --> 00:13:07,000
Speaker 5: What I say is that today that cycle of renewal

255
00:13:07,080 --> 00:13:07,680
Speaker 5: is much faster.

256
00:13:07,760 --> 00:13:08,800
Speaker 3: That's really what it is.

257
00:13:09,600 --> 00:13:13,679
Speaker 5: And I would say I generally resist making like really

258
00:13:13,720 --> 00:13:17,559
Speaker 5: broad categorizations. AI is, by the way, the largest possible

259
00:13:17,840 --> 00:13:20,640
Speaker 5: to me. It's like saying computers, okay, yeah, but even

260
00:13:20,720 --> 00:13:24,880
Speaker 5: software is very broad. And so within the software category,

261
00:13:24,960 --> 00:13:27,280
Speaker 5: I think there are winners for losers. There are winners

262
00:13:27,320 --> 00:13:29,120
Speaker 5: in the long term and losers in the long term.

263
00:13:29,160 --> 00:13:31,120
Speaker 5: But it's really like people tend to make it a

264
00:13:31,160 --> 00:13:33,199
Speaker 5: category and then tend maybe.

265
00:13:33,080 --> 00:13:34,640
Speaker 3: Through the baby with the bad water.

266
00:13:35,160 --> 00:13:37,600
Speaker 5: So here's an example to me, the question that I

267
00:13:37,600 --> 00:13:41,240
Speaker 5: ask myself with regards to which vendors am I going to?

268
00:13:41,280 --> 00:13:42,880
Speaker 5: Which software am I going to have like a few

269
00:13:42,920 --> 00:13:47,480
Speaker 5: years down the road, is software generally is attached to

270
00:13:48,240 --> 00:13:51,559
Speaker 5: a process or certain ways of working. Okay, it does

271
00:13:51,600 --> 00:13:53,960
Speaker 5: something for you, and he puts it in the form

272
00:13:54,000 --> 00:13:57,440
Speaker 5: of an application that you use. The real question is

273
00:13:57,600 --> 00:14:00,480
Speaker 5: that process and or ways of working going to be

274
00:14:00,520 --> 00:14:02,680
Speaker 5: the same or is it going to change in five years?

275
00:14:03,360 --> 00:14:08,000
Speaker 5: Then you can determine what is basically the likelihood that

276
00:14:08,040 --> 00:14:09,679
Speaker 5: the software is going to be robust to that or

277
00:14:09,760 --> 00:14:14,280
Speaker 5: no example, is accounting or closing the books going to

278
00:14:14,320 --> 00:14:16,120
Speaker 5: be very different five years from now.

279
00:14:16,720 --> 00:14:17,400
Speaker 3: I don't think so.

280
00:14:18,400 --> 00:14:21,480
Speaker 5: Really, he hasn't really changed. I mean, everything changes, but

281
00:14:21,520 --> 00:14:22,760
Speaker 5: he hasn't really changed much.

282
00:14:22,800 --> 00:14:23,880
Speaker 3: And so if you're.

283
00:14:23,720 --> 00:14:27,760
Speaker 5: Operating in the general Ledger type of category, I don't

284
00:14:27,760 --> 00:14:30,080
Speaker 5: think all of a sudden you take a GPT or

285
00:14:30,120 --> 00:14:32,320
Speaker 5: a cloud that is going to close your books magically.

286
00:14:32,360 --> 00:14:34,760
Speaker 5: You know the counting, and you still need to do

287
00:14:34,800 --> 00:14:37,880
Speaker 5: a lot of that, and it's very regulated importantly, right,

288
00:14:37,920 --> 00:14:43,200
Speaker 5: it's extremely regulated jurisdiction by jurisdiction, country by country, product

289
00:14:43,200 --> 00:14:46,360
Speaker 5: by product, industry by industry. So that part is kind

290
00:14:46,400 --> 00:14:49,640
Speaker 5: of to me in kind of the safe mode.

291
00:14:50,240 --> 00:14:51,200
Speaker 3: And then you go to the.

292
00:14:51,600 --> 00:14:54,840
Speaker 5: Other end of the spectrum, and you have sometimes software

293
00:14:54,880 --> 00:14:56,960
Speaker 5: that cand of is aligned to the way people do

294
00:14:57,080 --> 00:15:00,440
Speaker 5: things today, like software being one of them. Like if

295
00:15:00,480 --> 00:15:03,240
Speaker 5: you look at the software developer life cycle, a lot

296
00:15:03,320 --> 00:15:07,040
Speaker 5: of that is changing developers are developing software by developing

297
00:15:07,080 --> 00:15:09,320
Speaker 5: specs today, and so if you're too much in the

298
00:15:09,360 --> 00:15:12,400
Speaker 5: weeds down there in that mechanics and you don't adapt

299
00:15:12,520 --> 00:15:16,400
Speaker 5: for AIS or agents doing that work software development, life

300
00:15:16,480 --> 00:15:20,680
Speaker 5: cycle deployments, rollbacks, monitoring, observability and all that, I think

301
00:15:20,720 --> 00:15:24,200
Speaker 5: that part will be very much disrupted. Or if you're

302
00:15:24,200 --> 00:15:27,680
Speaker 5: adding a sort of a ux on things, that's another class.

303
00:15:27,800 --> 00:15:30,400
Speaker 5: You have a very simple process, I don't know, you're

304
00:15:30,440 --> 00:15:34,480
Speaker 5: doing surveys or expense reports or whatever, and now people

305
00:15:34,480 --> 00:15:38,880
Speaker 5: are going to start expecting their personal assistance or agents

306
00:15:38,920 --> 00:15:41,600
Speaker 5: to kind of do all them mechanics for them, And

307
00:15:41,640 --> 00:15:43,800
Speaker 5: so I think that part is probably something that has

308
00:15:43,840 --> 00:15:46,360
Speaker 5: a bigger question mark on top. And so I always

309
00:15:46,360 --> 00:15:50,520
Speaker 5: ask myself the process question first, the process transformation section

310
00:15:50,760 --> 00:15:54,240
Speaker 5: question first, and then consequently was the tool that is

311
00:15:54,280 --> 00:15:54,960
Speaker 5: going to support that?

312
00:15:55,680 --> 00:15:57,960
Speaker 2: Just to press on this point, though, have you replaced

313
00:15:58,040 --> 00:16:02,560
Speaker 2: any third party software where providers with something that's been

314
00:16:02,560 --> 00:16:04,040
Speaker 2: developed internally through AI?

315
00:16:04,440 --> 00:16:06,760
Speaker 3: We have, we have terminated contracts already.

316
00:16:06,840 --> 00:16:07,040
Speaker 5: Yes.

317
00:16:07,080 --> 00:16:07,560
Speaker 3: Absolute.

318
00:16:08,360 --> 00:16:10,040
Speaker 5: Now I'm not going to ask you your follow up

319
00:16:10,080 --> 00:16:18,520
Speaker 5: questions again because your name, but overall yes, absolutely absolutely.

320
00:16:18,560 --> 00:16:22,240
Speaker 5: You know the thing is like the whole buy versus built. Okay, yeah,

321
00:16:22,320 --> 00:16:23,760
Speaker 5: the equation has changed quite a bit.

322
00:16:24,520 --> 00:16:25,480
Speaker 3: Buy versus build.

323
00:16:25,680 --> 00:16:27,960
Speaker 5: Was always like, okay, guys, how long does it take

324
00:16:28,000 --> 00:16:30,000
Speaker 5: to do to build this? And you get an answer

325
00:16:30,080 --> 00:16:31,680
Speaker 5: which is, well, we can do it in like X

326
00:16:31,680 --> 00:16:32,480
Speaker 5: amount of years and.

327
00:16:32,560 --> 00:16:34,040
Speaker 3: X amount of millions of dollars.

328
00:16:34,680 --> 00:16:37,120
Speaker 5: Now I'm starting to see people coming to me and say,

329
00:16:37,120 --> 00:16:39,680
Speaker 5: by the way, I had some time, you know, this weekend,

330
00:16:39,720 --> 00:16:43,440
Speaker 5: and here is a perfectly working application. So the cost

331
00:16:44,040 --> 00:16:47,960
Speaker 5: or at least for simple applications, the cost of kind

332
00:16:48,000 --> 00:16:53,760
Speaker 5: of build from a time perspective, from an actual cost perspective,

333
00:16:53,760 --> 00:16:58,480
Speaker 5: has gone down quite dramatically. So right now the little

334
00:16:58,520 --> 00:17:01,840
Speaker 5: things are most likely going to be build the very big,

335
00:17:02,040 --> 00:17:06,560
Speaker 5: large software that is to be deployed that scale across

336
00:17:06,560 --> 00:17:09,600
Speaker 5: thousands of people and et cetera, that big complexity. As

337
00:17:09,640 --> 00:17:12,160
Speaker 5: you know from we all do toy stuff with our

338
00:17:12,200 --> 00:17:15,280
Speaker 5: clock coats at home and whatever. You know, there's still

339
00:17:15,280 --> 00:17:17,199
Speaker 5: some rough edges, you know, and so it's hard to

340
00:17:17,280 --> 00:17:19,760
Speaker 5: think that all of a sudden the big applications are

341
00:17:19,760 --> 00:17:22,680
Speaker 5: going to disappear. And so that's really what I'm saying

342
00:17:22,680 --> 00:17:24,840
Speaker 5: that if I look at the applications that I buy today,

343
00:17:24,920 --> 00:17:27,679
Speaker 5: there is a lot of small applications, I'm saying, and

344
00:17:27,720 --> 00:17:30,520
Speaker 5: so the build is kind of the pendulum is starting

345
00:17:30,520 --> 00:17:33,040
Speaker 5: to swing back towards the build at least for that

346
00:17:33,160 --> 00:17:34,119
Speaker 5: category for sure.

347
00:17:34,840 --> 00:17:37,520
Speaker 4: What is the forward deployed engineer? I know that's like

348
00:17:37,560 --> 00:17:39,679
Speaker 4: one of the hot buzzwords of twenty twenty six, and

349
00:17:39,720 --> 00:17:43,199
Speaker 4: I saw headlines that were entropic forward deployed engineers at Goldman.

350
00:17:43,359 --> 00:17:45,439
Speaker 4: I have no idea what that means. Actually, yeah, you know,

351
00:17:45,600 --> 00:17:47,720
Speaker 4: like what is that term? I think what they do

352
00:17:47,760 --> 00:17:48,440
Speaker 4: when they got there?

353
00:17:48,600 --> 00:17:51,560
Speaker 5: Okay, so I think listen, that name has changed quite

354
00:17:51,600 --> 00:17:54,879
Speaker 5: a bit. No no, no, no, no, I mean that is

355
00:17:54,920 --> 00:17:58,000
Speaker 5: the latest, So you are fully you're in the in

356
00:17:58,080 --> 00:18:01,520
Speaker 5: the the latest of that. But remember, I mean there

357
00:18:01,600 --> 00:18:04,160
Speaker 5: was a time where you used to call them solution architects, right,

358
00:18:04,240 --> 00:18:08,440
Speaker 5: and so the point is right now. I think one

359
00:18:08,520 --> 00:18:10,880
Speaker 5: trend that I see, which is also kind of true

360
00:18:10,920 --> 00:18:13,920
Speaker 5: for us, is when things change so much and so rapidly,

361
00:18:14,400 --> 00:18:16,439
Speaker 5: you kind of want to go to the origin of

362
00:18:16,520 --> 00:18:19,760
Speaker 5: who produces this new thing. Okay, So the least intermediaries

363
00:18:19,840 --> 00:18:22,639
Speaker 5: you have, and probably the faster you can go. And

364
00:18:22,680 --> 00:18:25,760
Speaker 5: so going and working directly with the model providers is

365
00:18:25,800 --> 00:18:28,960
Speaker 5: generally good idea, because if you're putting someone in the middle,

366
00:18:29,119 --> 00:18:31,600
Speaker 5: this company is going to have to be trained, that's

367
00:18:31,640 --> 00:18:33,200
Speaker 5: going to have to be you know, there is a

368
00:18:33,240 --> 00:18:36,800
Speaker 5: cycle which at this point of very rapid change is

369
00:18:36,840 --> 00:18:39,800
Speaker 5: going to slow you down. And so the first thing

370
00:18:39,840 --> 00:18:42,719
Speaker 5: that that term means is those are people that are

371
00:18:42,760 --> 00:18:47,280
Speaker 5: actually normally building the product. They're normally building the cloud

372
00:18:47,400 --> 00:18:51,480
Speaker 5: or GPT or X or so that's the first differentiation.

373
00:18:51,640 --> 00:18:54,480
Speaker 5: They're straight to the source of the AI production in

374
00:18:54,520 --> 00:18:59,240
Speaker 5: a way. And second is that they are generally product people,

375
00:18:59,400 --> 00:19:03,840
Speaker 5: so people that have actually built those tools rather than

376
00:19:03,960 --> 00:19:08,120
Speaker 5: people that are more like support than deployment. And so

377
00:19:08,760 --> 00:19:12,480
Speaker 5: this characterization is you take the classic sales support team

378
00:19:12,600 --> 00:19:16,600
Speaker 5: or solutions support team, which was mostly doing integration, and

379
00:19:16,640 --> 00:19:19,960
Speaker 5: when things are so rapid, it's like, you know, imagine

380
00:19:19,960 --> 00:19:22,160
Speaker 5: if there is like something like I don't know if

381
00:19:22,280 --> 00:19:25,560
Speaker 5: the clothes style works changes so fast, instead of a

382
00:19:25,600 --> 00:19:28,040
Speaker 5: fashion assistant, you want to talk to the tailor because

383
00:19:28,040 --> 00:19:30,840
Speaker 5: they can actually make it. Things are changing so fast,

384
00:19:31,000 --> 00:19:32,240
Speaker 5: So those are the tailors.

385
00:19:32,480 --> 00:19:35,000
Speaker 2: So on this note, one of the things we heard

386
00:19:35,280 --> 00:19:38,480
Speaker 2: in support of SAS was this idea that while integration

387
00:19:38,640 --> 00:19:41,120
Speaker 2: is still going to be really important, and that's really

388
00:19:41,119 --> 00:19:43,480
Speaker 2: going to be like the major hurdle for a lot

389
00:19:43,520 --> 00:19:45,840
Speaker 2: of this stuff. Have you found that AI is making

390
00:19:45,920 --> 00:19:49,480
Speaker 2: integration even faster at this point. Has that basically become

391
00:19:49,840 --> 00:19:50,880
Speaker 2: irrelevant nowadays?

392
00:19:51,640 --> 00:19:56,080
Speaker 5: No. I think integration is extremely important, especially for the

393
00:19:56,119 --> 00:19:58,600
Speaker 5: industry caust like systems of records. So when you do

394
00:19:58,680 --> 00:20:01,160
Speaker 5: something like you know, when you do a process, then

395
00:20:01,160 --> 00:20:03,600
Speaker 5: you have a source of data like your CRM systems

396
00:20:03,600 --> 00:20:05,640
Speaker 5: could be a system of record, or you have your

397
00:20:05,720 --> 00:20:08,600
Speaker 5: client system of record, your accounting system or record, and

398
00:20:08,640 --> 00:20:12,120
Speaker 5: those when they become the authoritative source of an answer,

399
00:20:12,840 --> 00:20:15,200
Speaker 5: they need to integrate with the rest of the firm

400
00:20:15,480 --> 00:20:17,560
Speaker 5: and the rest of the data, the rest of the applications.

401
00:20:17,720 --> 00:20:20,320
Speaker 5: So I can see that those vendors that sit on

402
00:20:20,400 --> 00:20:23,760
Speaker 5: top of those, we can argue that they will implement

403
00:20:23,920 --> 00:20:26,800
Speaker 5: there's nobody that is better position than them to implement AI.

404
00:20:26,880 --> 00:20:29,280
Speaker 3: They will kind of reach outwards and actually do that

405
00:20:29,400 --> 00:20:30,200
Speaker 3: kind of integration.

406
00:20:30,320 --> 00:20:32,639
Speaker 5: So I think those who will evolve so that you

407
00:20:32,720 --> 00:20:35,280
Speaker 5: still get the same level of automatism and you still

408
00:20:35,359 --> 00:20:37,960
Speaker 5: get the same benefit of speed, but it kind of

409
00:20:38,000 --> 00:20:40,720
Speaker 5: comes from within. I think that part is probably something

410
00:20:40,760 --> 00:20:43,919
Speaker 5: that that would remain very valuable and so in general,

411
00:20:43,960 --> 00:20:46,679
Speaker 5: I don't have anything against it. Again, I don't have

412
00:20:46,680 --> 00:20:50,520
Speaker 5: anything against the SaaS category at all overall, but I have,

413
00:20:50,640 --> 00:20:54,159
Speaker 5: as I said, different opinions of who actually is going

414
00:20:54,200 --> 00:20:56,640
Speaker 5: to adopt to the future and adapt to the future

415
00:20:56,680 --> 00:20:57,200
Speaker 5: and those.

416
00:20:57,000 --> 00:20:59,840
Speaker 2: Who are You mentioned this idea of people have a

417
00:21:00,080 --> 00:21:02,720
Speaker 2: you extra hours over the weekend and they come in

418
00:21:02,760 --> 00:21:04,600
Speaker 2: the morning and they're like, Oh, I had some extra

419
00:21:04,640 --> 00:21:05,000
Speaker 2: time and.

420
00:21:04,960 --> 00:21:06,119
Speaker 5: I decided to do this.

421
00:21:06,520 --> 00:21:10,159
Speaker 2: What's the coolest or most novel example of something that

422
00:21:10,160 --> 00:21:13,040
Speaker 2: people basically vibe coded in a limited amount of time

423
00:21:13,160 --> 00:21:15,639
Speaker 2: that wouldn't have happened, say two years ago.

424
00:21:16,280 --> 00:21:20,359
Speaker 5: So I've seen people doing a cloud migrations of legacy

425
00:21:20,359 --> 00:21:23,440
Speaker 5: applications that were on premise once they have been enabled

426
00:21:23,440 --> 00:21:26,400
Speaker 5: with those tools literally in a matter of hours. I've

427
00:21:26,440 --> 00:21:30,320
Speaker 5: seen someone build a complete travel assistant for corporate, a

428
00:21:30,359 --> 00:21:33,879
Speaker 5: travel assistant that looks at your calendar, it looks at

429
00:21:33,920 --> 00:21:36,720
Speaker 5: the flight delays, and look at rebooking stuff literally like

430
00:21:37,200 --> 00:21:39,440
Speaker 5: doing a meeting where they were not paying attention. So

431
00:21:42,320 --> 00:21:43,399
Speaker 5: those are some of the things.

432
00:21:43,480 --> 00:21:45,520
Speaker 2: That's what I'm doing right now and while we're doing

433
00:21:45,520 --> 00:21:46,120
Speaker 2: this podcast.

434
00:21:46,560 --> 00:21:49,679
Speaker 4: But actually that brings me the executive or wanted to

435
00:21:49,800 --> 00:21:53,480
Speaker 4: go next, which is I'm curious, like, do a large

436
00:21:53,520 --> 00:21:56,919
Speaker 4: corporations have a token budget the way they would have

437
00:21:56,960 --> 00:21:59,320
Speaker 4: a dollar budget in the past, So like I would

438
00:21:59,400 --> 00:22:03,840
Speaker 4: love to have unlimited access to coding models and whatever

439
00:22:04,000 --> 00:22:05,840
Speaker 4: and actually just play around and try to work.

440
00:22:05,720 --> 00:22:07,520
Speaker 3: Out all that. It's one of my favorite questions.

441
00:22:07,520 --> 00:22:10,440
Speaker 4: But I'm curious, like how you think about token allocation

442
00:22:10,680 --> 00:22:15,119
Speaker 4: within the firm and whether there's intra firm competition for compute.

443
00:22:15,320 --> 00:22:19,080
Speaker 2: Yeah, token allocation could be like included in your performance, right,

444
00:22:19,680 --> 00:22:20,400
Speaker 2: well you got more.

445
00:22:20,320 --> 00:22:22,879
Speaker 4: Different teams and stuff like that, Whether that's part of

446
00:22:22,920 --> 00:22:24,160
Speaker 4: what you think about for planning.

447
00:22:24,960 --> 00:22:27,920
Speaker 5: Absolutely so. A few months ago I did that. I

448
00:22:28,480 --> 00:22:30,919
Speaker 5: spoke about traditions for twenty six and one thing that

449
00:22:31,000 --> 00:22:32,359
Speaker 5: I said was it was going to be the birth

450
00:22:32,359 --> 00:22:34,520
Speaker 5: of the personal assistant, and that kind of happened with

451
00:22:34,640 --> 00:22:36,960
Speaker 5: open Kloh and all that stuff kind of early on.

452
00:22:37,320 --> 00:22:39,199
Speaker 5: And then the one was there's going to be a

453
00:22:39,240 --> 00:22:41,639
Speaker 5: token sticker shock for CFOs, right that all of a

454
00:22:41,680 --> 00:22:44,080
Speaker 5: sudden they're going to start seeing bills that they absolutely

455
00:22:44,160 --> 00:22:44,880
Speaker 5: did not expect.

456
00:22:44,960 --> 00:22:48,199
Speaker 4: Jensen Wongs had an interview today, sorry not today, in

457
00:22:48,280 --> 00:22:51,720
Speaker 4: recent weeks something about like if I'm paying an engineer

458
00:22:51,840 --> 00:22:54,439
Speaker 4: five hundred thousand dollars. I hope that he's spending at

459
00:22:54,480 --> 00:22:57,119
Speaker 4: least two hundred and fifty thousand dollars on tokens. Now, again,

460
00:22:57,160 --> 00:23:00,199
Speaker 4: as many people pointed out, that's like the barber saying, oh,

461
00:23:00,280 --> 00:23:03,440
Speaker 4: you really need to get haircuts every week. Nonetheless, we're

462
00:23:03,440 --> 00:23:05,359
Speaker 4: talking about some pretty big numbers, a lot more than

463
00:23:05,400 --> 00:23:08,080
Speaker 4: there's like a cloud Marx plan for two hundred right

464
00:23:08,160 --> 00:23:09,520
Speaker 4: now to talk about.

465
00:23:09,320 --> 00:23:12,679
Speaker 5: That right now. Okay, So first of all, lesson number

466
00:23:12,720 --> 00:23:17,320
Speaker 5: one is you need to centralize the access to models

467
00:23:17,960 --> 00:23:21,280
Speaker 5: so that you can monitor, miter it, and then optimize it. Okay,

468
00:23:21,400 --> 00:23:23,959
Speaker 5: So the wild west of everybody goes and calls an

469
00:23:23,960 --> 00:23:26,880
Speaker 5: API and starts consuming tokens and then you find out

470
00:23:26,960 --> 00:23:29,160
Speaker 5: later on is a big problem. And so that's why

471
00:23:29,160 --> 00:23:33,119
Speaker 5: we built this GSAI platform, which is what's called the

472
00:23:33,160 --> 00:23:38,240
Speaker 5: Model Gateway, and the model Gateway intelligently routes requests to

473
00:23:38,760 --> 00:23:43,000
Speaker 5: the combination the pareto frontier of quality and cost. Okay,

474
00:23:43,080 --> 00:23:46,520
Speaker 5: so you got to centralize that. It's not a one

475
00:23:46,560 --> 00:23:49,440
Speaker 5: size fits all because many cases, if you're asking what's

476
00:23:49,480 --> 00:23:52,160
Speaker 5: the weather, you don't need to call cloud Opus four

477
00:23:52,200 --> 00:23:55,159
Speaker 5: point six you can ask it to live in a

478
00:23:55,200 --> 00:23:57,600
Speaker 5: local model that you run very cheaply on prem and

479
00:23:57,640 --> 00:24:01,600
Speaker 5: so there is there are ways to opt is way

480
00:24:01,640 --> 00:24:05,120
Speaker 5: before you start even having the conversation, you're consuming too much.

481
00:24:05,359 --> 00:24:08,359
Speaker 4: Yeah, so this is very interesting to me. Is a

482
00:24:08,400 --> 00:24:11,119
Speaker 4: big part of the problem that you're trying to solve.

483
00:24:11,880 --> 00:24:16,840
Speaker 4: And we know, like chedgbt, they intelligently route they do

484
00:24:16,920 --> 00:24:19,520
Speaker 4: some on there. You go to chedgbt dot com and

485
00:24:19,520 --> 00:24:21,359
Speaker 4: they'll try to route it to the best model, and

486
00:24:21,400 --> 00:24:23,679
Speaker 4: there might even be some conflict of interest because they

487
00:24:23,720 --> 00:24:26,560
Speaker 4: probably want to route it to the cheapest model. The

488
00:24:26,640 --> 00:24:29,359
Speaker 4: user wants the most performance model. But how much of

489
00:24:29,400 --> 00:24:32,320
Speaker 4: the work of your senior engineers is essentially solving this

490
00:24:32,440 --> 00:24:36,639
Speaker 4: problem of the right query going to the pereto optimal model.

491
00:24:36,760 --> 00:24:39,159
Speaker 5: It is a big part of the time of the

492
00:24:39,560 --> 00:24:42,399
Speaker 5: spent by the AI central.

493
00:24:42,040 --> 00:24:43,400
Speaker 3: Group, Okay, platform group.

494
00:24:43,480 --> 00:24:46,800
Speaker 5: The platform group worries a lot about where do I

495
00:24:46,840 --> 00:24:49,359
Speaker 5: get the right data for example, for this question, and

496
00:24:49,400 --> 00:24:51,720
Speaker 5: which model do I write route it too. That's a

497
00:24:51,760 --> 00:24:55,040
Speaker 5: big you know, because again I spoke about para frontier,

498
00:24:55,160 --> 00:24:58,240
Speaker 5: meaning the optimization between a quality which we don't want

499
00:24:58,240 --> 00:25:01,080
Speaker 5: to compromise and the actual cost. And you can be

500
00:25:01,240 --> 00:25:04,680
Speaker 5: iso quality of very different price points, because not all

501
00:25:04,760 --> 00:25:08,200
Speaker 5: questions require the most expensive token. So that's point number one.

502
00:25:08,240 --> 00:25:11,480
Speaker 5: So what I'm trying to say is my philosophy is

503
00:25:11,520 --> 00:25:16,600
Speaker 5: to try to isolate the developer or the user from

504
00:25:16,720 --> 00:25:22,000
Speaker 5: the talken anxiety. It's a little bit like we'll electric cars. Okay,

505
00:25:22,040 --> 00:25:24,200
Speaker 5: at one point, if you have eight team miles of range,

506
00:25:24,200 --> 00:25:26,760
Speaker 5: you're always optimizing routes and maybe I'm not going to

507
00:25:26,960 --> 00:25:30,480
Speaker 5: go there. I don't need this ice cream today, have

508
00:25:30,600 --> 00:25:33,679
Speaker 5: it later yourself limiting in ways, there are kind of

509
00:25:33,880 --> 00:25:37,320
Speaker 5: really no useful optimy micro optimizations. We don't want people

510
00:25:37,359 --> 00:25:39,639
Speaker 5: to go there yet, at least right now. It's the

511
00:25:39,720 --> 00:25:42,160
Speaker 5: time where people need to really find the best way

512
00:25:42,200 --> 00:25:44,960
Speaker 5: to kind of do more and do the best possible

513
00:25:45,040 --> 00:25:48,639
Speaker 5: work with the eye and let us, meaning internally in

514
00:25:48,680 --> 00:25:51,520
Speaker 5: the sort of a central team, optimize it in.

515
00:25:51,480 --> 00:25:53,760
Speaker 3: A way that we're going to make it economical.

516
00:25:54,160 --> 00:25:58,040
Speaker 5: And I think reducing the talking anxiety is a big challenge,

517
00:25:58,080 --> 00:26:01,040
Speaker 5: but I think it really frees up create and what

518
00:26:01,080 --> 00:26:03,679
Speaker 5: you can do with the eye. It's also like there

519
00:26:03,720 --> 00:26:08,119
Speaker 5: are certain problems that you don't want to optimize too early, Okay, Okay,

520
00:26:08,240 --> 00:26:11,240
Speaker 5: So for example, yeah, how much time do you want

521
00:26:11,280 --> 00:26:14,520
Speaker 5: to optimize now? For remember we used to kind of

522
00:26:14,560 --> 00:26:16,959
Speaker 5: optimize the way to web pages because they were too

523
00:26:17,040 --> 00:26:19,200
Speaker 5: slow to load. And then at one point the editors

524
00:26:19,280 --> 00:26:21,639
Speaker 5: or whatever say why can't I put yet another image?

525
00:26:21,680 --> 00:26:24,040
Speaker 5: And then people are starting to say, okay, why don't

526
00:26:24,080 --> 00:26:25,840
Speaker 5: you do it? And then on the back end, I'm

527
00:26:25,880 --> 00:26:28,240
Speaker 5: going to work in optimizing your images rather than asking

528
00:26:28,240 --> 00:26:29,960
Speaker 5: you what the most you can put three images on

529
00:26:30,000 --> 00:26:33,040
Speaker 5: the homepage? Right, So that's the approach people right now.

530
00:26:33,119 --> 00:26:36,919
Speaker 5: I would rather have them err on the side of

531
00:26:37,080 --> 00:26:40,800
Speaker 5: usage and let me worry about optimization. And the other

532
00:26:40,880 --> 00:26:45,760
Speaker 5: point is really, at the end, human hours always tend

533
00:26:45,800 --> 00:26:48,800
Speaker 5: to be the most expensive cost, and so as long

534
00:26:48,880 --> 00:26:51,720
Speaker 5: as your token cost per hour is less than your

535
00:26:51,800 --> 00:26:54,840
Speaker 5: wage per hour, there is a kind of a positive ory.

536
00:26:55,160 --> 00:26:57,639
Speaker 2: And so at that point it's fine, Well, what's your

537
00:26:57,640 --> 00:27:00,879
Speaker 2: feeling about future costs of tokens, whether they're going up

538
00:27:00,960 --> 00:27:03,400
Speaker 2: or down? Because you hear different things on this one

539
00:27:03,400 --> 00:27:05,760
Speaker 2: of the things you hear is that, again going back

540
00:27:05,760 --> 00:27:08,800
Speaker 2: to the beginning of this conversation, AI has improved so

541
00:27:09,040 --> 00:27:12,159
Speaker 2: quickly in the course of months, if not weeks, that

542
00:27:12,640 --> 00:27:15,000
Speaker 2: those costs are destined to come down. But on the

543
00:27:15,040 --> 00:27:18,159
Speaker 2: other hand, we know that the hyperscalers are still losing

544
00:27:18,200 --> 00:27:23,000
Speaker 2: money handover fist for power users such as yourself at Goldman.

545
00:27:23,119 --> 00:27:24,879
Speaker 2: So where do you think those are going over time?

546
00:27:27,240 --> 00:27:30,800
Speaker 5: My personal view is talking cost is going to go

547
00:27:30,880 --> 00:27:34,679
Speaker 5: down quite a bit, but talken numbers are going to

548
00:27:34,720 --> 00:27:37,760
Speaker 5: go up thro a year more and so total talking

549
00:27:37,800 --> 00:27:39,439
Speaker 5: cost is going to Actually we're going to have to

550
00:27:39,480 --> 00:27:43,240
Speaker 5: accept that is going to be a major item of

551
00:27:43,359 --> 00:27:47,520
Speaker 5: cost in any organization, and it's to be compared to

552
00:27:47,680 --> 00:27:49,840
Speaker 5: the cost of people and not to be compared to

553
00:27:49,880 --> 00:27:52,800
Speaker 5: the cost of the ORTICIPIP packets or computer or.

554
00:27:52,760 --> 00:27:53,320
Speaker 3: Any of that.

555
00:27:54,200 --> 00:27:56,479
Speaker 5: If you look at just the number of talks being

556
00:27:56,600 --> 00:27:59,639
Speaker 5: used for the same use case, if you go the

557
00:27:59,680 --> 00:28:02,600
Speaker 5: reasons an route, or you're gonna don't go the reasoning route,

558
00:28:02,640 --> 00:28:05,600
Speaker 5: if you go the agentic route, or not the agentic route,

559
00:28:05,640 --> 00:28:08,239
Speaker 5: if you go the open floor route, where you know

560
00:28:08,240 --> 00:28:11,239
Speaker 5: it checks every you know, starts having these tasks that

561
00:28:11,320 --> 00:28:13,359
Speaker 5: are firing one after the other and then you have

562
00:28:13,400 --> 00:28:16,000
Speaker 5: to start to very fires, et cetera, et cetera. So

563
00:28:16,359 --> 00:28:19,640
Speaker 5: I think the trend will continue with regards to more

564
00:28:19,680 --> 00:28:22,320
Speaker 5: and more of those, but the cost the per unit

565
00:28:22,440 --> 00:28:25,040
Speaker 5: cost of talking I pretty I'm pretty sure that it's

566
00:28:25,040 --> 00:28:28,280
Speaker 5: gonna go down. Also because the GPUs are becoming more powerful,

567
00:28:28,359 --> 00:28:30,959
Speaker 5: the cost for what hopefully is gonna go down. And

568
00:28:31,000 --> 00:28:34,120
Speaker 5: then also, like to be fair, I mean, these hyperscalers

569
00:28:34,160 --> 00:28:36,400
Speaker 5: are doing a lot of optimizations to try to run

570
00:28:36,560 --> 00:28:39,760
Speaker 5: those stacks on their own hard right, which will potentially

571
00:28:39,800 --> 00:28:45,120
Speaker 5: can also generate some economists. Okay, h.

572
00:28:58,000 --> 00:29:02,480
Speaker 4: Can Golvin employees run open claw on their work computers?

573
00:29:02,600 --> 00:29:05,720
Speaker 4: And I'm curious, like about the degree to which you

574
00:29:05,800 --> 00:29:07,719
Speaker 4: have people who like want to I want to install

575
00:29:07,800 --> 00:29:09,600
Speaker 4: this or this seems really cool, and then think about

576
00:29:09,880 --> 00:29:13,480
Speaker 4: the security imperative and how you handle that aspect, not

577
00:29:13,600 --> 00:29:16,040
Speaker 4: the token anxiety, but the sort of I want to

578
00:29:16,040 --> 00:29:17,160
Speaker 4: install this, this is awesome.

579
00:29:18,640 --> 00:29:20,640
Speaker 5: As you know, like as a bank, we're pretty locked

580
00:29:20,640 --> 00:29:22,760
Speaker 5: down in terms of what you can install. You cannot

581
00:29:22,760 --> 00:29:25,200
Speaker 5: install stuff that is not you know, in the corporate

582
00:29:25,280 --> 00:29:27,280
Speaker 5: in the corporate app store.

583
00:29:27,000 --> 00:29:28,360
Speaker 3: In a way, and so there's no way.

584
00:29:28,960 --> 00:29:30,920
Speaker 2: But do you feel there's definitely no way though, because

585
00:29:31,080 --> 00:29:34,480
Speaker 2: can't you just ask Claude how to install itself?

586
00:29:34,520 --> 00:29:36,520
Speaker 5: But it's not going to be able to execute, it's

587
00:29:36,560 --> 00:29:38,760
Speaker 5: not going to be able to create the actually executable.

588
00:29:38,760 --> 00:29:41,400
Speaker 5: It can actually even gsa as system today can spit

589
00:29:41,440 --> 00:29:45,240
Speaker 5: out a lot of code, but it spits out source code. Now,

590
00:29:45,280 --> 00:29:49,040
Speaker 5: our source code doesn't run executable, so it needs to

591
00:29:49,080 --> 00:29:51,560
Speaker 5: be built and it needs to be turned into an executable.

592
00:29:51,600 --> 00:29:54,000
Speaker 5: It needs to be signed, otherwise the operating system is

593
00:29:54,040 --> 00:29:55,760
Speaker 5: going to refuse to run it, and so it just

594
00:29:55,800 --> 00:29:57,480
Speaker 5: doesn't run unless you have that.

595
00:29:57,600 --> 00:30:01,040
Speaker 4: But do you feel an anxiety where start? They're probably

596
00:30:01,200 --> 00:30:03,720
Speaker 4: and there's not a ton of startup investment banks, but

597
00:30:03,760 --> 00:30:05,800
Speaker 4: there are various FinTechs and other you don't want to

598
00:30:05,840 --> 00:30:08,280
Speaker 4: chip away at parts of your business, and they can

599
00:30:08,360 --> 00:30:10,520
Speaker 4: run perhaps faster, and they can be a little bit

600
00:30:10,560 --> 00:30:13,320
Speaker 4: more liberal about what their employees are allowed to do,

601
00:30:13,400 --> 00:30:17,120
Speaker 4: et cetera. Do you feel like you have to keep

602
00:30:17,160 --> 00:30:21,480
Speaker 4: a certain cadence of expanding the list of those executables

603
00:30:21,520 --> 00:30:23,080
Speaker 4: that are able to be run.

604
00:30:23,760 --> 00:30:24,720
Speaker 3: So I'll give you two answers.

605
00:30:24,720 --> 00:30:25,880
Speaker 5: So, first of all, I want to make sure that

606
00:30:25,920 --> 00:30:28,160
Speaker 5: I answer your first question, which is we're not using

607
00:30:28,160 --> 00:30:33,040
Speaker 5: open claw Okay, okay, but some of the properties of

608
00:30:33,080 --> 00:30:36,680
Speaker 5: open call as actually have informed the way we are

609
00:30:36,720 --> 00:30:42,640
Speaker 5: building our agentic platform. Okay, agents today because of open

610
00:30:42,680 --> 00:30:44,719
Speaker 5: claw have actually changed.

611
00:30:44,920 --> 00:30:46,560
Speaker 3: If you break down what open claw.

612
00:30:46,480 --> 00:30:49,520
Speaker 5: Is, there are This is my own interpretation and I

613
00:30:49,520 --> 00:30:54,720
Speaker 5: actually never even spoke about this. There are three characteristics

614
00:30:54,920 --> 00:30:58,600
Speaker 5: that make open clob what it is. One is, it's

615
00:30:58,600 --> 00:31:01,800
Speaker 5: a constant loop, so it's basically what you know in

616
00:31:02,200 --> 00:31:05,200
Speaker 5: information theory you can call an observer pattern. It's something

617
00:31:05,200 --> 00:31:09,200
Speaker 5: that continues to run and observe, So it is that

618
00:31:09,320 --> 00:31:13,200
Speaker 5: it's a constant observer, so it runs constantly. The other

619
00:31:13,240 --> 00:31:17,600
Speaker 5: one is it can schedule events every seven am, do this,

620
00:31:17,800 --> 00:31:20,360
Speaker 5: or you know, like in personal life, we all have

621
00:31:20,680 --> 00:31:22,920
Speaker 5: something like that that sends me the news in the

622
00:31:22,960 --> 00:31:25,240
Speaker 5: morning and all that, So there is a schedule ability

623
00:31:25,280 --> 00:31:26,080
Speaker 5: of tasks.

624
00:31:26,440 --> 00:31:27,560
Speaker 3: The third one is.

625
00:31:28,400 --> 00:31:30,840
Speaker 5: You can instruct it to kind of change its own

626
00:31:30,880 --> 00:31:35,040
Speaker 5: behavior because it has these files dot MD files solvet

627
00:31:35,200 --> 00:31:38,000
Speaker 5: MD the way you so you can say things like, hey,

628
00:31:38,440 --> 00:31:40,520
Speaker 5: I would like you to never use this term or

629
00:31:40,600 --> 00:31:43,080
Speaker 5: PRIs change or change the way you filter news. And

630
00:31:43,080 --> 00:31:46,000
Speaker 5: so it kind of writes its own software to do

631
00:31:46,080 --> 00:31:50,360
Speaker 5: things for you without you even seeing what's behind the scenes.

632
00:31:51,120 --> 00:31:55,160
Speaker 5: And so instead of letting people install open clod on

633
00:31:55,160 --> 00:31:58,480
Speaker 5: their computers, what we do is we incorporate some of

634
00:31:58,480 --> 00:32:02,240
Speaker 5: those characteristics to regantic platform so that it does things

635
00:32:02,240 --> 00:32:06,520
Speaker 5: that are more similar to open So that's the second

636
00:32:06,600 --> 00:32:09,880
Speaker 5: question was interesting because basically, if I read behind the questions,

637
00:32:09,960 --> 00:32:11,800
Speaker 5: are you asking whether there is a sort of a

638
00:32:11,880 --> 00:32:16,840
Speaker 5: velocity disadvantage with regard to us versus others. I often

639
00:32:16,880 --> 00:32:19,719
Speaker 5: say that there is a difference between speed and velocity.

640
00:32:20,080 --> 00:32:24,760
Speaker 5: Speed is almost like you have a certain sprint, okay,

641
00:32:25,280 --> 00:32:27,280
Speaker 5: but then at some point you're gonna hit the wall,

642
00:32:27,600 --> 00:32:30,920
Speaker 5: security wall, excalability wall. There's gonna be a bug, you

643
00:32:30,920 --> 00:32:32,760
Speaker 5: don't know what you're doing, and it's gonna forst sooner

644
00:32:32,840 --> 00:32:35,480
Speaker 5: or literally gonna be hit by that. It would be

645
00:32:35,560 --> 00:32:40,080
Speaker 5: like most airplanes are in auto autopilot that can do everything.

646
00:32:40,120 --> 00:32:42,240
Speaker 5: So theoretically you and I could go on the cockpit

647
00:32:42,280 --> 00:32:44,360
Speaker 5: and for a long time during the flight we.

648
00:32:44,280 --> 00:32:45,920
Speaker 3: Will feel pretty good about that, right.

649
00:32:45,960 --> 00:32:48,440
Speaker 5: We will drink some soft drinks or your tea, we

650
00:32:48,760 --> 00:32:50,120
Speaker 5: might maybe watch some videos.

651
00:32:50,120 --> 00:32:51,040
Speaker 3: We will be very happy.

652
00:32:51,360 --> 00:32:55,000
Speaker 6: So great is one point in time in the flight

653
00:32:55,160 --> 00:32:57,160
Speaker 6: there's gonna be some storm and there's gonna be an

654
00:32:57,160 --> 00:32:59,800
Speaker 6: autopilot disconnect and you and I are going to look

655
00:32:59,840 --> 00:33:03,800
Speaker 6: at child that I'm gonna say, right, where's the pilot?

656
00:33:03,880 --> 00:33:05,640
Speaker 4: Can I just say recently, I was on a flight

657
00:33:05,720 --> 00:33:07,479
Speaker 4: I tell you this that I was going to Newark.

658
00:33:08,040 --> 00:33:10,080
Speaker 4: We circled three times, we tried to land. It was

659
00:33:10,160 --> 00:33:13,400
Speaker 4: during a storm. They kept not landing. Everyone's like starting

660
00:33:13,400 --> 00:33:14,840
Speaker 4: to get pretty annoyed because we were up there for

661
00:33:14,840 --> 00:33:18,240
Speaker 4: a while. And then the flight attendant comes on and says, unprompted,

662
00:33:18,560 --> 00:33:20,959
Speaker 4: by the way, we have plenty of gas, and everyone

663
00:33:21,040 --> 00:33:23,320
Speaker 4: started figure out, No, it's like, this is a question,

664
00:33:23,480 --> 00:33:25,880
Speaker 4: this is an answer that no one had been nobody anyway,

665
00:33:25,920 --> 00:33:29,640
Speaker 4: I'm sorry. Then everyone got really nervous. Okay, by the way,

666
00:33:29,640 --> 00:33:30,440
Speaker 4: we have plenty of gas.

667
00:33:30,520 --> 00:33:32,720
Speaker 5: But you know what, I was almost fearing that you

668
00:33:32,720 --> 00:33:34,000
Speaker 5: would say, is there a pilot?

669
00:33:35,400 --> 00:33:37,000
Speaker 4: And then we did land in Washington, DC.

670
00:33:37,120 --> 00:33:39,320
Speaker 3: Okay, oh you did? Yeah from Newark. No, No, that's

671
00:33:39,320 --> 00:33:39,720
Speaker 3: not good.

672
00:33:39,800 --> 00:33:40,480
Speaker 4: But that makes sense.

673
00:33:40,800 --> 00:33:41,000
Speaker 1: Yeah.

674
00:33:41,000 --> 00:33:43,720
Speaker 5: And so so that's why I mean by velocities really

675
00:33:43,760 --> 00:33:47,040
Speaker 5: like it's like the marathon is sustained speed for a

676
00:33:47,080 --> 00:33:50,360
Speaker 5: long time in a certent direction. I don't think by

677
00:33:50,440 --> 00:33:53,920
Speaker 5: randomizing that you actually gained velocity. You gain an instant

678
00:33:53,960 --> 00:33:57,000
Speaker 5: speed of some sort. And so I'm kind of optimizing

679
00:33:57,040 --> 00:33:57,760
Speaker 5: for velocity.

680
00:33:58,600 --> 00:34:02,320
Speaker 2: But related to SHOs point, though, you are a regulated bank,

681
00:34:02,360 --> 00:34:05,000
Speaker 2: absolutely right, and so there are restrictions on what you

682
00:34:05,040 --> 00:34:07,600
Speaker 2: can do in terms of technology. I am very very

683
00:34:07,640 --> 00:34:11,200
Speaker 2: curious what your discussions with regulators are right now, because

684
00:34:11,840 --> 00:34:14,160
Speaker 2: a lot of regulators this is still pretty new to them.

685
00:34:14,600 --> 00:34:19,120
Speaker 2: A lot of the models basically are black boxes. How

686
00:34:19,160 --> 00:34:22,400
Speaker 2: do you convince them that like they're running as they should,

687
00:34:22,640 --> 00:34:26,120
Speaker 2: that they're spitting out the correct output, that you understand

688
00:34:26,400 --> 00:34:28,440
Speaker 2: how they're actually functioning.

689
00:34:29,000 --> 00:34:32,879
Speaker 5: So this is not the first time that banks use

690
00:34:32,960 --> 00:34:36,640
Speaker 5: neural networks. Okay, these are just much larger neural networks.

691
00:34:36,680 --> 00:34:40,680
Speaker 5: But we've been using neural networks for like decade plus,

692
00:34:41,320 --> 00:34:45,320
Speaker 5: and so every bank has already gone through the motions

693
00:34:45,360 --> 00:34:49,800
Speaker 5: of explaining that neural networks don't have perfect explicability. Therefore

694
00:34:49,800 --> 00:34:52,719
Speaker 5: you need to change the control system around them. You

695
00:34:52,760 --> 00:34:55,799
Speaker 5: need to look at what actions can they actually do,

696
00:34:55,840 --> 00:34:58,480
Speaker 5: and then you leamit to the actions. Okay, there is

697
00:34:58,640 --> 00:35:01,279
Speaker 5: functions called the like mode, the risk management, which is

698
00:35:01,320 --> 00:35:05,120
Speaker 5: a very standardized function within every bank that for each

699
00:35:05,120 --> 00:35:07,759
Speaker 5: of those neural networks, you need to have an inventory,

700
00:35:07,880 --> 00:35:10,120
Speaker 5: you need to have a risk tiering, and you need

701
00:35:10,160 --> 00:35:13,960
Speaker 5: to put controls around them. So it is not really

702
00:35:14,640 --> 00:35:17,400
Speaker 5: that much of a new thing is more of an evolution,

703
00:35:17,560 --> 00:35:19,600
Speaker 5: where now you have things that are much faster and

704
00:35:19,640 --> 00:35:22,680
Speaker 5: much more powerful. But the basic pattern and the basic

705
00:35:22,760 --> 00:35:25,319
Speaker 5: discussion with the regulators is kind of the same, which

706
00:35:25,400 --> 00:35:29,520
Speaker 5: is are you classifying the risk tiering of the application?

707
00:35:29,680 --> 00:35:29,919
Speaker 3: Right?

708
00:35:30,400 --> 00:35:32,799
Speaker 5: And then which controls are you putting and are you

709
00:35:32,840 --> 00:35:36,040
Speaker 5: putting human supervision and human in the loop. So, for example,

710
00:35:36,080 --> 00:35:39,800
Speaker 5: for code, we don't allow ais to ought to approve

711
00:35:39,840 --> 00:35:43,560
Speaker 5: their own code. Okay, all they can do is publish

712
00:35:43,560 --> 00:35:46,560
Speaker 5: what's called the pull request or a merge request the

713
00:35:46,640 --> 00:35:49,719
Speaker 5: same way as a developer would do. And we kind

714
00:35:49,760 --> 00:35:52,080
Speaker 5: of have a sort of zero trust model there because

715
00:35:52,120 --> 00:35:54,759
Speaker 5: we don't assume that maybe a junior developer is going

716
00:35:54,840 --> 00:35:57,480
Speaker 5: to be less more bug freedom than an AI, right,

717
00:35:57,880 --> 00:36:02,719
Speaker 5: And so we have several controls in place. For example,

718
00:36:03,320 --> 00:36:05,759
Speaker 5: there needs to be a human smare senior than you

719
00:36:05,960 --> 00:36:09,440
Speaker 5: that actually looks at the code and then certifies and approves.

720
00:36:09,600 --> 00:36:12,560
Speaker 5: And then after that, before it goes to the production

721
00:36:12,680 --> 00:36:14,719
Speaker 5: it goes to something that is called CICD or the

722
00:36:15,360 --> 00:36:19,759
Speaker 5: continuous Integration continues deployment pipeline where when it goes through

723
00:36:19,760 --> 00:36:22,840
Speaker 5: the build phase, etc. There is a lot of checks

724
00:36:22,840 --> 00:36:25,520
Speaker 5: that are injected into that. There are security checks, there

725
00:36:25,560 --> 00:36:28,719
Speaker 5: are tech risk checks. So I don't think at the

726
00:36:28,800 --> 00:36:30,800
Speaker 5: end of the day you really lose too much velocity

727
00:36:30,880 --> 00:36:32,759
Speaker 5: or at all. You just need to invest more in

728
00:36:32,800 --> 00:36:35,200
Speaker 5: those kinds of things. And the relators. I think if

729
00:36:35,200 --> 00:36:37,799
Speaker 5: you bring them back into sort of a familiar territory

730
00:36:38,239 --> 00:36:40,160
Speaker 5: and you're also honest on things that you know and

731
00:36:40,200 --> 00:36:41,759
Speaker 5: things that you don't know, and for the things that

732
00:36:41,840 --> 00:36:44,000
Speaker 5: you don't know, you kind of put higher protections.

733
00:36:44,000 --> 00:36:46,200
Speaker 3: I think the conversation is generally very positive.

734
00:36:47,080 --> 00:36:50,160
Speaker 4: We were having an episode that we recorded several weeks

735
00:36:50,160 --> 00:36:52,920
Speaker 4: ago that we still haven't released. I don't know exactly

736
00:36:53,000 --> 00:36:54,880
Speaker 4: the timing of that. One of this one, we interviewed

737
00:36:54,960 --> 00:36:58,479
Speaker 4: Scott balk, the former CEO of Greenhill, the boutique investment bank,

738
00:36:58,520 --> 00:37:00,800
Speaker 4: And for the reason we had that conversation was because

739
00:37:00,800 --> 00:37:02,680
Speaker 4: we want to know, like if AI is going to

740
00:37:02,760 --> 00:37:05,440
Speaker 4: someday disrupt banking as we know, like what was banking

741
00:37:05,480 --> 00:37:06,920
Speaker 4: as we know? And so we talked about the history

742
00:37:06,920 --> 00:37:09,440
Speaker 4: of investment banking. But one of the things that he

743
00:37:09,560 --> 00:37:12,440
Speaker 4: talked about was that a big advantage that the banks

744
00:37:12,440 --> 00:37:15,200
Speaker 4: had was this sort of information asymmetry and that they

745
00:37:15,200 --> 00:37:17,640
Speaker 4: would know a lot more about their industries and so

746
00:37:17,680 --> 00:37:21,000
Speaker 4: forth than their clients, and this was profitable. Now, going

747
00:37:21,040 --> 00:37:24,120
Speaker 4: back to your answer the very first question, You're like, Okay,

748
00:37:24,520 --> 00:37:27,040
Speaker 4: a client might call Goldman and they say, what does

749
00:37:27,120 --> 00:37:29,520
Speaker 4: the straight up Horrmon's closure mean for this portfolio of

750
00:37:29,560 --> 00:37:30,360
Speaker 4: shock et cetera.

751
00:37:30,480 --> 00:37:32,080
Speaker 2: I was going to ask this exact question.

752
00:37:32,200 --> 00:37:34,319
Speaker 4: Yeah, I kind of think I could do that. I

753
00:37:34,320 --> 00:37:36,600
Speaker 4: think I could. I mean, I have no offense. I'm

754
00:37:36,600 --> 00:37:39,759
Speaker 4: sure your platform is a little bit better than what like,

755
00:37:39,800 --> 00:37:41,520
Speaker 4: but I think I could get ninety percent of the

756
00:37:41,560 --> 00:37:42,960
Speaker 4: way there, and I bet I could, like, with a

757
00:37:43,000 --> 00:37:45,680
Speaker 4: little bit of data, build a basket that says I

758
00:37:45,760 --> 00:37:50,080
Speaker 4: want helium shortage basket, which country? Which companies would I short?

759
00:37:50,080 --> 00:37:52,000
Speaker 4: If I think the helium shortage is going to get worse,

760
00:37:52,080 --> 00:37:54,120
Speaker 4: I could build a basket the way a trading desk

761
00:37:54,160 --> 00:37:57,759
Speaker 4: would do. You think long term, like that AI erodes

762
00:37:57,760 --> 00:38:01,680
Speaker 4: a certain structural source of profit for bents andtter.

763
00:38:01,719 --> 00:38:04,200
Speaker 5: I think you can get to the ninety percent, but

764
00:38:04,280 --> 00:38:05,920
Speaker 5: I think clients are really paying.

765
00:38:05,680 --> 00:38:09,399
Speaker 3: Us for that extra ten percent. Okay, so I think

766
00:38:09,440 --> 00:38:10,080
Speaker 3: that's the endswer.

767
00:38:10,800 --> 00:38:13,160
Speaker 2: So what is the extra ten percent? In that context?

768
00:38:13,200 --> 00:38:16,320
Speaker 2: Is it? Your models are slightly better or is it also.

769
00:38:16,040 --> 00:38:18,040
Speaker 5: The day we have access to you know, we buy

770
00:38:18,080 --> 00:38:20,440
Speaker 5: a lot of data that you know, is very expensive

771
00:38:20,480 --> 00:38:23,120
Speaker 5: and it's a massive quantities and it's very up to

772
00:38:23,239 --> 00:38:24,520
Speaker 5: date and very real time.

773
00:38:24,640 --> 00:38:26,919
Speaker 3: So we have a little bit of a data advantage.

774
00:38:27,800 --> 00:38:31,279
Speaker 5: We operate across multiple asset classes, so we see the

775
00:38:31,320 --> 00:38:35,239
Speaker 5: trading side, we see the asset management side, so we

776
00:38:35,320 --> 00:38:38,799
Speaker 5: have a sort of correlation between assets. Advantage that we

777
00:38:38,840 --> 00:38:42,399
Speaker 5: see those because generally rates move, interest rates can move,

778
00:38:42,560 --> 00:38:44,480
Speaker 5: yields that can move, you know, there is a correlation

779
00:38:44,560 --> 00:38:47,640
Speaker 5: between all those indicators. So there is another advantage. We

780
00:38:47,680 --> 00:38:49,960
Speaker 5: have a global advantage. We have people on the ground

781
00:38:50,080 --> 00:38:52,879
Speaker 5: in one hundred plus countries and these people have relationships

782
00:38:52,920 --> 00:38:56,920
Speaker 5: and information travels through those channels. And also, you know,

783
00:38:57,040 --> 00:39:00,440
Speaker 5: like we generally deal with very complex portfolio. Not you

784
00:39:00,480 --> 00:39:02,800
Speaker 5: and I may be having three stocks or four or five.

785
00:39:02,880 --> 00:39:07,040
Speaker 3: This is like very complex multi assets with complex.

786
00:39:06,640 --> 00:39:10,439
Speaker 5: Products like swaps or swaptions or exotic products, et cetera,

787
00:39:10,520 --> 00:39:14,000
Speaker 5: et cetera. And so that's really the ten percent that

788
00:39:14,200 --> 00:39:16,400
Speaker 5: the clients that we have really value and what we

789
00:39:16,480 --> 00:39:18,200
Speaker 5: really need to get. It's like, at the end of

790
00:39:18,200 --> 00:39:21,280
Speaker 5: the day, listen, look at Formula One, okay. The difference

791
00:39:21,360 --> 00:39:27,000
Speaker 5: per time per lap between the Mercedes and take you know,

792
00:39:27,040 --> 00:39:31,000
Speaker 5: your favorite last team, it's sometimes one second after two

793
00:39:31,040 --> 00:39:34,160
Speaker 5: million out of two minutes. And that is the difference

794
00:39:34,160 --> 00:39:37,200
Speaker 5: between you know, getting one hundred million dollars a year

795
00:39:37,200 --> 00:39:42,239
Speaker 5: sponsorship or a ten thousand dollars sponsorship. So for sophisticated clients,

796
00:39:43,080 --> 00:39:45,880
Speaker 5: that ten percent is really where the money is and

797
00:39:45,920 --> 00:39:47,399
Speaker 5: that's really what people are paying us for.

798
00:39:48,200 --> 00:39:51,680
Speaker 2: So actually, you mentioned all the different businesses at Colman,

799
00:39:51,800 --> 00:39:54,000
Speaker 2: and there are a bunch of them like asset management,

800
00:39:54,000 --> 00:39:57,640
Speaker 2: there's banking, there's trading. A lot of those businesses aren't

801
00:39:57,760 --> 00:40:00,520
Speaker 2: supposed to talk to each other in various ways. And

802
00:40:00,560 --> 00:40:02,920
Speaker 2: so when it comes to the data, is there like

803
00:40:02,960 --> 00:40:06,240
Speaker 2: a data leakage issue where you might have a model

804
00:40:06,480 --> 00:40:10,440
Speaker 2: that's in house, like GSAI that's pulling data from different

805
00:40:10,480 --> 00:40:14,279
Speaker 2: sides of the company in ways that maybe it shouldn't be.

806
00:40:14,800 --> 00:40:17,920
Speaker 2: Maybe it's really hard to tell given the complexity of

807
00:40:17,960 --> 00:40:19,799
Speaker 2: the model. Is that something you have to pay attention to.

808
00:40:20,080 --> 00:40:20,799
Speaker 3: Absolutely.

809
00:40:20,880 --> 00:40:23,400
Speaker 5: So we have the concept of info barriers, okay, and

810
00:40:23,440 --> 00:40:27,680
Speaker 5: the info barriers are enforced throughout the entire system, okay,

811
00:40:28,400 --> 00:40:32,600
Speaker 5: and they're linked to your idea or your account. Okay,

812
00:40:32,680 --> 00:40:36,200
Speaker 5: so if I'm on the private side, I can only

813
00:40:36,200 --> 00:40:38,839
Speaker 5: see certain information. If I am on a public side,

814
00:40:38,840 --> 00:40:41,640
Speaker 5: I can only see certain information and I cannot even

815
00:40:42,360 --> 00:40:44,560
Speaker 5: know about the information. On the other side, I can

816
00:40:44,880 --> 00:40:46,520
Speaker 5: I don't have access to the file.

817
00:40:46,560 --> 00:40:47,960
Speaker 3: So to the fold, there's nothing.

818
00:40:48,280 --> 00:40:51,640
Speaker 5: Each AI or each agent or each application that's the

819
00:40:51,680 --> 00:40:54,560
Speaker 5: beauty of this centralized platform needs to get an idea

820
00:40:54,640 --> 00:40:57,680
Speaker 5: or a badge, and that badge is attached to the

821
00:40:57,760 --> 00:41:01,080
Speaker 5: exact same info barriers as any application or any computers.

822
00:41:01,120 --> 00:41:03,320
Speaker 3: And so this are enforced basically at the source.

823
00:41:03,440 --> 00:41:06,520
Speaker 5: So even if it is the same type of model,

824
00:41:07,040 --> 00:41:10,120
Speaker 5: but that particular use of the model, that particular session

825
00:41:10,160 --> 00:41:12,640
Speaker 5: of the model, they needs to get a ticket or

826
00:41:12,680 --> 00:41:15,439
Speaker 5: a badge, and that badge or those keys just take

827
00:41:15,480 --> 00:41:16,920
Speaker 5: them to a sort of place. And so this is

828
00:41:16,960 --> 00:41:19,319
Speaker 5: one of these been It took us almost two years

829
00:41:19,360 --> 00:41:22,279
Speaker 5: to build a GSEI platform. These are this back to

830
00:41:22,320 --> 00:41:25,720
Speaker 5: the reason why you can't be casual about these things.

831
00:41:25,719 --> 00:41:28,440
Speaker 5: This thing is not being built by some random vibe coders.

832
00:41:28,520 --> 00:41:30,640
Speaker 5: Because you need to worry about cyber you need to

833
00:41:30,640 --> 00:41:32,960
Speaker 5: worry about infobarriers, you need to worry about all that

834
00:41:33,080 --> 00:41:35,719
Speaker 5: and so when I talk about there are places where

835
00:41:35,719 --> 00:41:38,239
Speaker 5: you can leverage and do correlations, but there are others

836
00:41:38,239 --> 00:41:40,360
Speaker 5: where you absolutely can't. And this is kind of that

837
00:41:40,480 --> 00:41:42,520
Speaker 5: is foundational to the fact that you need to be

838
00:41:42,560 --> 00:41:43,120
Speaker 5: ready for it.

839
00:41:43,320 --> 00:41:44,879
Speaker 3: You can't be casual about.

840
00:41:45,160 --> 00:41:48,239
Speaker 4: So I take your point that there's never been a

841
00:41:48,280 --> 00:41:51,760
Speaker 4: technology that you've seen in your career that has actually

842
00:41:51,840 --> 00:41:55,120
Speaker 4: reduced the need for software engineers, and that the nature

843
00:41:55,160 --> 00:41:57,959
Speaker 4: of the job of software engineers change and maybe gets

844
00:41:57,960 --> 00:42:02,840
Speaker 4: more high level and whatever. Setting aside that volume question,

845
00:42:03,200 --> 00:42:06,560
Speaker 4: setting aside the pure head level of headcount question. Is

846
00:42:06,640 --> 00:42:11,080
Speaker 4: AI changing right now across anything technology or otherwise the

847
00:42:11,160 --> 00:42:14,000
Speaker 4: types of person you're looking for, or changing something about

848
00:42:14,040 --> 00:42:16,560
Speaker 4: the nature of the type of talent?

849
00:42:17,000 --> 00:42:20,120
Speaker 5: Yeah, absolutely, great question. So I think in this day

850
00:42:20,160 --> 00:42:25,880
Speaker 5: and age, almost nobody is an individual contributor really, because

851
00:42:25,880 --> 00:42:29,359
Speaker 5: when you're working with agents, you need to have at

852
00:42:29,480 --> 00:42:33,239
Speaker 5: least three fundamental characteristics. One is you need to be

853
00:42:33,239 --> 00:42:37,560
Speaker 5: able to explain what you want to get done. The

854
00:42:37,600 --> 00:42:40,680
Speaker 5: second one is you need to be able to delegate work.

855
00:42:41,239 --> 00:42:43,520
Speaker 5: Guess what, because you're going to have multiple agents. One

856
00:42:43,520 --> 00:42:46,719
Speaker 5: is specialized for example, in doing I don't know DCF calculations,

857
00:42:46,719 --> 00:42:49,359
Speaker 5: and one is specialized in doing research, so you need

858
00:42:49,400 --> 00:42:52,080
Speaker 5: to be able to break down the work into chunks

859
00:42:52,120 --> 00:42:55,960
Speaker 5: that can be executed in parallel in some way. And

860
00:42:56,000 --> 00:42:58,719
Speaker 5: then three, you need to have the ability to supervise.

861
00:42:58,920 --> 00:43:01,200
Speaker 5: You need to actually look at the output and say, Okay,

862
00:43:01,360 --> 00:43:02,120
Speaker 5: I'm good with this.

863
00:43:02,239 --> 00:43:05,200
Speaker 3: Or go back. It turns out that those three.

864
00:43:05,080 --> 00:43:10,560
Speaker 5: Things I explain, delegate, and supervise are kind of the

865
00:43:10,600 --> 00:43:13,160
Speaker 5: one on one of managers. Managers need to have those

866
00:43:13,160 --> 00:43:16,439
Speaker 5: three otherwise they can't manage a team. And so AI

867
00:43:16,560 --> 00:43:19,120
Speaker 5: is kind of turning everybody a little bit into a manager.

868
00:43:19,840 --> 00:43:21,719
Speaker 5: And those are kind of the skills that we are

869
00:43:21,719 --> 00:43:25,799
Speaker 5: actually looking for. People that they know that they're going

870
00:43:25,880 --> 00:43:28,799
Speaker 5: to have agency on tools that at some point are

871
00:43:28,800 --> 00:43:31,759
Speaker 5: going to be even more proficient and specific specialized than

872
00:43:31,800 --> 00:43:32,239
Speaker 5: they are.

873
00:43:32,640 --> 00:43:34,839
Speaker 3: And so the most important thing is.

874
00:43:34,800 --> 00:43:39,200
Speaker 5: Really the ability to id eight, to explain, to delegate,

875
00:43:39,280 --> 00:43:42,160
Speaker 5: and then to really know what good looks like. And

876
00:43:42,239 --> 00:43:44,000
Speaker 5: I think that is a big change, and I don't

877
00:43:44,000 --> 00:43:47,000
Speaker 5: think everybody is going to actually rapidly go through that.

878
00:43:47,080 --> 00:43:49,799
Speaker 5: And I think we're doing a combination of training. There

879
00:43:49,840 --> 00:43:52,920
Speaker 5: is a combination of exposing them to other people. Like

880
00:43:53,000 --> 00:43:55,719
Speaker 5: one of the advantages of having forward deployment engineers. Is

881
00:43:55,760 --> 00:43:57,640
Speaker 5: also that there is a little bit of classer culture

882
00:43:57,680 --> 00:44:00,120
Speaker 5: that is happening around the table, and so people think

883
00:44:00,120 --> 00:44:04,800
Speaker 5: really really differently, and that pushes people outside their comfort zone.

884
00:44:04,880 --> 00:44:07,040
Speaker 5: That's why I'm saying that there is a little bit

885
00:44:07,040 --> 00:44:10,200
Speaker 5: of a metamorphosis happening there. It's not just about Deficiency's

886
00:44:10,239 --> 00:44:12,440
Speaker 5: really thinking about is my job going to stay the same? No,

887
00:44:12,520 --> 00:44:13,799
Speaker 5: it's actually changing quite a bit.

888
00:44:14,400 --> 00:44:18,560
Speaker 2: So I'm thinking how to frame this question, But what's

889
00:44:18,640 --> 00:44:23,040
Speaker 2: work life balance like now for a developer at Goldman

890
00:44:23,120 --> 00:44:26,880
Speaker 2: because you have this existential angst about jobs potentially changing

891
00:44:27,239 --> 00:44:29,920
Speaker 2: at the same time you have AI tools that enable

892
00:44:30,040 --> 00:44:34,040
Speaker 2: more productivity, and you also have this thing happening where

893
00:44:34,080 --> 00:44:36,880
Speaker 2: I feel like, Joe, maybe you know more about this

894
00:44:36,920 --> 00:44:38,400
Speaker 2: than I do, but I feel like a lot of

895
00:44:38,480 --> 00:44:40,520
Speaker 2: Vibe coders like it's addictive.

896
00:44:40,719 --> 00:44:40,959
Speaker 4: Yeah.

897
00:44:41,000 --> 00:44:44,640
Speaker 2: Right, It's like you're pressing the button of a slot machine.

898
00:44:44,920 --> 00:44:47,760
Speaker 2: You're interacting with Claude, and you're seeing what it spits

899
00:44:47,800 --> 00:44:50,080
Speaker 2: back out over and over again until you get that

900
00:44:50,160 --> 00:44:53,799
Speaker 2: big win. And so I've heard people talk about burnout

901
00:44:53,920 --> 00:44:56,920
Speaker 2: among developers who are just doing so much with this

902
00:44:57,200 --> 00:44:59,120
Speaker 2: right now that they're just hitting that button over.

903
00:44:59,120 --> 00:45:01,840
Speaker 4: It was a good discussion in the Odd Lage discord

904
00:45:01,920 --> 00:45:06,239
Speaker 4: recently about exactly this. Some engineers and semiconductors feel like

905
00:45:06,280 --> 00:45:09,279
Speaker 4: that their job has become less satisfying. And I think

906
00:45:09,320 --> 00:45:10,840
Speaker 4: it's sort of what you're getting at. There's sort of

907
00:45:10,880 --> 00:45:14,120
Speaker 4: slot machines where it's like, oh, you're gonna hit the problem. Okay,

908
00:45:14,120 --> 00:45:16,200
Speaker 4: this is the great output. Then it's like they feel

909
00:45:16,200 --> 00:45:18,520
Speaker 4: the work is like less satisfying and stuff like that

910
00:45:18,600 --> 00:45:19,880
Speaker 4: than actually like writing code.

911
00:45:19,880 --> 00:45:22,799
Speaker 5: So yeah, I mean, listen again, this is where kind

912
00:45:22,800 --> 00:45:24,440
Speaker 5: of the fact that I'm a little bit older than

913
00:45:24,480 --> 00:45:28,080
Speaker 5: most here engineers, cand I've seen that the first time

914
00:45:28,120 --> 00:45:32,080
Speaker 5: people had Excel and for the first time people had Python,

915
00:45:32,800 --> 00:45:34,520
Speaker 5: Oh my god, I don't need to know job. And

916
00:45:34,560 --> 00:45:36,360
Speaker 5: then the kids start to code, and there is the

917
00:45:36,520 --> 00:45:38,760
Speaker 5: whole coding movement, and then you get to you start

918
00:45:39,320 --> 00:45:42,120
Speaker 5: creating your applications. I've seen the first time people have

919
00:45:42,239 --> 00:45:44,840
Speaker 5: mobile stuff and you know, mobile apps, and so I

920
00:45:44,880 --> 00:45:46,960
Speaker 5: think a little bit of that is because it's new,

921
00:45:47,040 --> 00:45:49,520
Speaker 5: to be perfectly honest, and I think, yes, there is

922
00:45:49,560 --> 00:45:51,720
Speaker 5: a little bit of that, but there is a little

923
00:45:51,760 --> 00:45:54,000
Speaker 5: a lot of novelty to that. And then I've seen

924
00:45:54,000 --> 00:45:56,320
Speaker 5: that people have been using those tools for a couple

925
00:45:56,320 --> 00:45:58,480
Speaker 5: of years, they're taking them a little bit more like, Okay,

926
00:45:58,520 --> 00:46:00,880
Speaker 5: it's a professionalist tool that I'm going to use it

927
00:46:00,920 --> 00:46:03,480
Speaker 5: for what I actually need rather than just trying to discover.

928
00:46:03,880 --> 00:46:06,759
Speaker 5: One thing that I've seen is that because maybe of that,

929
00:46:06,960 --> 00:46:09,480
Speaker 5: but also because of what you can get, there is

930
00:46:09,520 --> 00:46:12,879
Speaker 5: a sort of in a way reward cycle.

931
00:46:12,560 --> 00:46:15,720
Speaker 3: That is pretty quick. Yeah, people are very excited.

932
00:46:15,760 --> 00:46:18,800
Speaker 5: Actually, there's some sort of a joy of the profession

933
00:46:18,840 --> 00:46:21,960
Speaker 5: that is actually coming out, as if engineers were feeling

934
00:46:22,000 --> 00:46:23,920
Speaker 5: like this job is new again, because a lot of

935
00:46:23,960 --> 00:46:26,920
Speaker 5: engineers have seen the same patterns sometimes for three decades,

936
00:46:27,560 --> 00:46:30,759
Speaker 5: so there has been something that I observed. There is

937
00:46:30,800 --> 00:46:33,080
Speaker 5: also a lot of peer pressure. There is a lot

938
00:46:33,120 --> 00:46:35,640
Speaker 5: of fear of missing out. So people rather than it's

939
00:46:35,680 --> 00:46:38,160
Speaker 5: no longer me trying to push the car up here,

940
00:46:38,200 --> 00:46:40,440
Speaker 5: it's more people are actually looking at their peers and

941
00:46:40,480 --> 00:46:42,480
Speaker 5: they're looking at oh my god, how could you do that?

942
00:46:42,600 --> 00:46:44,919
Speaker 5: And so it's kind of spreading horizontally quite a bit,

943
00:46:45,000 --> 00:46:48,439
Speaker 5: which is really nice to see. And so so far

944
00:46:48,480 --> 00:46:51,960
Speaker 5: I have to say that it's being positive, positive change.

945
00:46:52,000 --> 00:46:54,480
Speaker 5: And also one other thing that we're talking about burnout.

946
00:46:55,080 --> 00:46:57,400
Speaker 5: I see that a lot of people get fatigued. I

947
00:46:57,719 --> 00:47:00,479
Speaker 5: want to talk about burnout, but they get fatigue when

948
00:47:01,080 --> 00:47:04,240
Speaker 5: there are a lot of repetitive tasks, especially for a developer.

949
00:47:04,280 --> 00:47:07,720
Speaker 5: Here's an example. Let's say you go from a version

950
00:47:07,760 --> 00:47:11,960
Speaker 5: of the Java library or springboot to another version and

951
00:47:12,000 --> 00:47:14,160
Speaker 5: then all of a sudden you compile and you get

952
00:47:14,200 --> 00:47:16,200
Speaker 5: to rebuild, and you get all these errors.

953
00:47:16,200 --> 00:47:17,480
Speaker 3: That says you need to upgrade.

954
00:47:18,080 --> 00:47:22,040
Speaker 5: Honestly, upgrading libraries is not the most fun draw and

955
00:47:22,120 --> 00:47:24,440
Speaker 5: if you need to do it one hundred times or

956
00:47:24,600 --> 00:47:26,440
Speaker 5: is like someone says, by the way, guys, we have

957
00:47:26,480 --> 00:47:30,319
Speaker 5: this new design, new logo, new collors implemented on like

958
00:47:30,400 --> 00:47:33,160
Speaker 5: two hundred websites. It might be fun the first ten

959
00:47:33,600 --> 00:47:37,040
Speaker 5: and then it becomes a drug, and so I think

960
00:47:37,120 --> 00:47:40,640
Speaker 5: taking that away kind of they focus more and more

961
00:47:40,680 --> 00:47:43,520
Speaker 5: on the plan. For example, and so right now, let's

962
00:47:43,560 --> 00:47:46,120
Speaker 5: do a migration plan to the cloud of a complex application.

963
00:47:46,239 --> 00:47:48,879
Speaker 5: They spend maybe seventy percent of their time going back

964
00:47:48,880 --> 00:47:52,080
Speaker 5: and forth with the very powerful set of ais to

965
00:47:52,120 --> 00:47:52,800
Speaker 5: really get.

966
00:47:52,600 --> 00:47:53,279
Speaker 3: The plan right.

967
00:47:54,160 --> 00:47:57,640
Speaker 5: They feel a little bit more elevated, and the mechanical

968
00:47:57,719 --> 00:48:01,399
Speaker 5: part it's kind of left to the the same way.

969
00:48:01,680 --> 00:48:02,400
Speaker 3: I mean listen.

970
00:48:02,520 --> 00:48:06,160
Speaker 5: I started developing when I was literally flipping switches okay,

971
00:48:06,480 --> 00:48:08,279
Speaker 5: and then pressing a button that we should move the

972
00:48:08,280 --> 00:48:12,920
Speaker 5: register up one. And then came some languages. They were like, see,

973
00:48:13,040 --> 00:48:15,600
Speaker 5: oh my god, Now I don't have to flip switches anymore.

974
00:48:15,640 --> 00:48:17,800
Speaker 5: But I guess what I need to do memory management?

975
00:48:17,840 --> 00:48:20,040
Speaker 5: I need to do pointers. I mean there's a lot

976
00:48:20,080 --> 00:48:22,200
Speaker 5: of heavy lifting. Oh, I have a memory leak. I'm

977
00:48:22,200 --> 00:48:24,960
Speaker 5: gonna spend a week before I actually finally identify that.

978
00:48:25,520 --> 00:48:29,439
Speaker 5: And then it comes Java. Oh, garbage collection, I don't

979
00:48:29,480 --> 00:48:31,160
Speaker 5: have to worry more memory leaks anymore.

980
00:48:31,200 --> 00:48:31,799
Speaker 3: Fantastic.

981
00:48:31,880 --> 00:48:35,360
Speaker 5: And then comes Python, which is, oh, all that rigidity,

982
00:48:35,719 --> 00:48:39,160
Speaker 5: so much easier to be type free and so forth,

983
00:48:39,760 --> 00:48:42,040
Speaker 5: And so every time you can keep raising the bar

984
00:48:42,200 --> 00:48:44,760
Speaker 5: and a lot of the candle mechanics kind of goes away.

985
00:48:44,880 --> 00:48:47,760
Speaker 5: I think this has been like a ten years jump

986
00:48:47,800 --> 00:48:50,800
Speaker 5: in a metter of two years. But I think overall,

987
00:48:51,280 --> 00:48:54,839
Speaker 5: nobody really likes to have that toil and that mechanical work.

988
00:48:54,880 --> 00:48:56,800
Speaker 5: And I'm actually quite happy that people are going to

989
00:48:56,880 --> 00:48:59,759
Speaker 5: spend that, maybe initially more time because they're excited, but

990
00:49:00,080 --> 00:49:02,880
Speaker 5: thinks that I'm enjoying rather than thinks that this is

991
00:49:03,000 --> 00:49:03,479
Speaker 5: a thread.

992
00:49:04,320 --> 00:49:06,399
Speaker 2: All right, well, Marco, we'll have to have you back

993
00:49:06,400 --> 00:49:08,520
Speaker 2: on the podcast in another year and a half, I

994
00:49:08,520 --> 00:49:13,160
Speaker 2: guess in discussion again on the reduced AI timeline. Thank

995
00:49:13,200 --> 00:49:15,040
Speaker 2: you so much for coming back on OFF.

996
00:49:16,600 --> 00:49:31,040
Speaker 3: Thank you so much so, Joe.

997
00:49:31,080 --> 00:49:33,279
Speaker 2: That was great to catch up. One thing I thought

998
00:49:33,280 --> 00:49:36,520
Speaker 2: was really interesting was his point about the discussions with

999
00:49:36,560 --> 00:49:39,480
Speaker 2: the regulators and framing it like very similar to previous

1000
00:49:39,560 --> 00:49:44,400
Speaker 2: technological advances, where you're not necessarily explaining exactly how the

1001
00:49:44,440 --> 00:49:47,359
Speaker 2: models are coming to certain conclusions. Yeah, but you're more

1002
00:49:47,400 --> 00:49:50,480
Speaker 2: focused on actually limiting the risks and making sure that

1003
00:49:50,480 --> 00:49:52,719
Speaker 2: they're in the right bucket for risk assessment.

1004
00:49:53,000 --> 00:49:53,040
Speaker 5: No.

1005
00:49:53,200 --> 00:49:55,400
Speaker 4: I thought that was really interesting, just that some of

1006
00:49:55,440 --> 00:49:58,640
Speaker 4: these technologies, the black box Yeah, lms are not the

1007
00:49:58,719 --> 00:50:01,600
Speaker 4: first black box. Sure, We've actually been talking about black

1008
00:50:01,640 --> 00:50:05,960
Speaker 4: box trading for years in finance before. So the idea

1009
00:50:06,000 --> 00:50:08,040
Speaker 4: of like, Okay, there are these things that are happening,

1010
00:50:08,120 --> 00:50:11,360
Speaker 4: we can't articulate them and whatever, it's not the first

1011
00:50:11,440 --> 00:50:14,400
Speaker 4: rodeo for finance is really interesting. I'm also, you know,

1012
00:50:14,440 --> 00:50:18,279
Speaker 4: I thought the whole conversation about token budgets and allocations

1013
00:50:18,280 --> 00:50:20,680
Speaker 4: are interesting. The idea of like, okay, part of the

1014
00:50:20,760 --> 00:50:22,600
Speaker 4: job here is you have a bunch of different models.

1015
00:50:23,000 --> 00:50:26,120
Speaker 4: Everyone in theory wants the most performance model. But how

1016
00:50:26,120 --> 00:50:28,080
Speaker 4: do you find that optimization where you get the best

1017
00:50:28,120 --> 00:50:31,400
Speaker 4: performance relative to price? It sounds like a pretty interesting,

1018
00:50:31,680 --> 00:50:32,960
Speaker 4: like engineering problem.

1019
00:50:33,080 --> 00:50:36,000
Speaker 2: Yeah, I would actually love to do more on that question.

1020
00:50:36,000 --> 00:50:40,399
Speaker 2: I would do because it's such an interesting question of incentives, right,

1021
00:50:40,440 --> 00:50:43,560
Speaker 2: and like how does how do you actually like prioritize?

1022
00:50:43,600 --> 00:50:46,799
Speaker 4: How do you know what constitutes a good output? And

1023
00:50:46,920 --> 00:50:49,920
Speaker 4: how when you sacrifice a little bit of quality for

1024
00:50:50,040 --> 00:50:53,399
Speaker 4: like ten x less token budget or whatever like these said,

1025
00:50:53,400 --> 00:50:55,799
Speaker 4: this would be it would be very interesting to talk

1026
00:50:55,840 --> 00:50:58,920
Speaker 4: about how that problem specifically gets solved in inside of

1027
00:50:58,920 --> 00:50:59,480
Speaker 4: an organ.

1028
00:50:59,360 --> 00:51:03,799
Speaker 2: So ECONOL or efficiency optimization. Yeah, well, we'll have Marco

1029
00:51:03,880 --> 00:51:07,120
Speaker 2: back on very soon to talk about all the new

1030
00:51:07,120 --> 00:51:08,920
Speaker 2: things that AI is doing. But for now, shall we

1031
00:51:09,000 --> 00:51:09,279
Speaker 2: leave it there.

1032
00:51:09,360 --> 00:51:10,040
Speaker 3: Let's leave it there.

1033
00:51:10,120 --> 00:51:12,400
Speaker 2: This has been another episode of the Odd Lots podcast.

1034
00:51:12,440 --> 00:51:15,359
Speaker 2: I'm Tracy Alloway. You can follow me at Tracy Alloway.

1035
00:51:15,200 --> 00:51:18,120
Speaker 4: And I'm Jill Wisenthal. You can follow me at the Stalwart.

1036
00:51:18,160 --> 00:51:21,200
Speaker 4: Follow our producers Carmen Rodriguez at Kerman Arman dash Ol

1037
00:51:21,200 --> 00:51:24,640
Speaker 4: Bennett at Dashbot and Klebrooks at Kalebrooks and for more

1038
00:51:24,640 --> 00:51:27,160
Speaker 4: Odd Lots content, go to Bloomberg dot com slash odd

1039
00:51:27,160 --> 00:51:29,560
Speaker 4: Lots or a daily newsletter and all of our episodes,

1040
00:51:29,840 --> 00:51:31,719
Speaker 4: and you can chat about all these topics twenty four

1041
00:51:31,760 --> 00:51:34,839
Speaker 4: to seven in our discord with fellow listeners discord dot

1042
00:51:34,880 --> 00:51:36,160
Speaker 4: gg slash odlines.

1043
00:51:36,400 --> 00:51:38,759
Speaker 2: And if you enjoy Odd Lots, if you like it

1044
00:51:38,840 --> 00:51:41,040
Speaker 2: when we talk to Goldman Sachs about how they're actually

1045
00:51:41,040 --> 00:51:43,719
Speaker 2: deploying AI across the company, then please leave us a

1046
00:51:43,760 --> 00:51:47,040
Speaker 2: positive review on your favorite podcast platform. And remember, if

1047
00:51:47,040 --> 00:51:49,359
Speaker 2: you are a Bloomberg subscriber, you can listen to all

1048
00:51:49,360 --> 00:51:52,319
Speaker 2: of our episodes absolutely ad free. All you need to

1049
00:51:52,320 --> 00:51:54,960
Speaker 2: do is find the Bloomberg channel on Apple Podcasts and

1050
00:51:55,040 --> 00:52:22,960
Speaker 2: follow the instructions there. Thanks for listening in in