1
00:00:01,080 --> 00:00:04,640
Speaker 1: We've talked about many of the ways AI is working

2
00:00:04,680 --> 00:00:09,319
Speaker 1: itself into our lives at the office in Hollywood, even

3
00:00:09,400 --> 00:00:14,000
Speaker 1: in the US presidential campaign today. Another place where robots

4
00:00:14,080 --> 00:00:18,000
Speaker 1: are moving in Wall Street. Of course, traders have long

5
00:00:18,079 --> 00:00:21,040
Speaker 1: used data to try to divine which way markets will move.

6
00:00:21,520 --> 00:00:26,720
Speaker 1: Quantitative traders are quants already build sophisticated mathematical models and

7
00:00:26,760 --> 00:00:31,680
Speaker 1: amass huge amounts of information to identify opportunities others have missed.

8
00:00:32,200 --> 00:00:35,360
Speaker 1: But AI can conceivably do it even better by pulling

9
00:00:35,400 --> 00:00:39,720
Speaker 1: in and analyzing vastly more data and much faster. But

10
00:00:40,080 --> 00:00:43,519
Speaker 1: is AI better at beating the market than human stock pickers.

11
00:00:44,040 --> 00:00:47,680
Speaker 1: Bloomberg's Justina Lee and Sam Potter say so far, the

12
00:00:47,800 --> 00:00:49,080
Speaker 1: results are mixed.

13
00:00:49,760 --> 00:00:52,960
Speaker 2: Part of the issue is when you're using machine learning algorithm,

14
00:00:53,200 --> 00:00:57,000
Speaker 2: you lose the explainability of it. So when you're losing money,

15
00:00:57,080 --> 00:00:59,320
Speaker 2: your clients might be like, why did you decide to

16
00:00:59,360 --> 00:00:59,720
Speaker 2: trade that?

17
00:01:00,480 --> 00:01:03,600
Speaker 3: I wonder if someone will discover an achilles heel to

18
00:01:03,680 --> 00:01:06,959
Speaker 3: these machines in the way the machines work that will

19
00:01:07,000 --> 00:01:10,960
Speaker 3: provide an unintended market advantage for those who are looking

20
00:01:10,959 --> 00:01:11,240
Speaker 3: for it.

21
00:01:12,080 --> 00:01:14,640
Speaker 1: Justina and Sam are here to tell us how investment

22
00:01:14,760 --> 00:01:18,679
Speaker 1: firms are using AI and how it actually works, And

23
00:01:18,800 --> 00:01:22,399
Speaker 1: later we speak to Renee Yao, whose firm is leveraging

24
00:01:22,440 --> 00:01:23,480
Speaker 1: this new technology.

25
00:01:24,000 --> 00:01:29,120
Speaker 4: We're not just saying we're using AI to generate supure returns.

26
00:01:29,200 --> 00:01:35,720
Speaker 4: We're using AI combined with good risk controls.

27
00:01:40,840 --> 00:01:44,279
Speaker 1: I'm wes Kasova today on the big take. Can AI

28
00:01:44,800 --> 00:01:55,919
Speaker 1: supercharge your returns? Justina, We're hearing all the time about

29
00:01:55,960 --> 00:02:00,000
Speaker 1: how AI is encroaching on every little aspect of our lives.

30
00:02:00,000 --> 00:02:02,200
Speaker 1: If I suppose it was only a matter of time

31
00:02:02,320 --> 00:02:06,560
Speaker 1: before the markets wanted to use AI to make money,

32
00:02:06,920 --> 00:02:09,200
Speaker 1: and you found there are so many big brains working

33
00:02:09,240 --> 00:02:10,240
Speaker 1: on this question.

34
00:02:10,720 --> 00:02:12,919
Speaker 2: Yeah, I mean maybe we can start with a certain

35
00:02:12,960 --> 00:02:16,120
Speaker 2: category of big brains on Wall Street, which are these

36
00:02:16,160 --> 00:02:20,200
Speaker 2: people we call quants. What quants stand for is quantitative

37
00:02:20,280 --> 00:02:24,120
Speaker 2: traders or strategies or analysts, and they're basically people who

38
00:02:24,200 --> 00:02:26,799
Speaker 2: are using computer models to figure out how to make

39
00:02:26,880 --> 00:02:30,160
Speaker 2: money in the markets. Like usually it's about analyzing some

40
00:02:30,320 --> 00:02:33,760
Speaker 2: pattern and trying to profit from that. And in recent years,

41
00:02:33,840 --> 00:02:37,440
Speaker 2: quants have been especially interested in using a certain kind

42
00:02:37,440 --> 00:02:41,400
Speaker 2: of AI called machine learning, which basically just means that

43
00:02:42,080 --> 00:02:46,000
Speaker 2: they're using very sophisticated techniques, you know, the kind that

44
00:02:46,120 --> 00:02:50,200
Speaker 2: is powering chat GPT, that is powering driverless cars. They're

45
00:02:50,280 --> 00:02:54,079
Speaker 2: using those kinds of techniques to try to figure out exactly,

46
00:02:54,200 --> 00:02:56,600
Speaker 2: you know, where the S and P five hundred is going,

47
00:02:56,680 --> 00:02:58,720
Speaker 2: or where the Apple Stock is going, and trying to

48
00:02:58,720 --> 00:02:59,560
Speaker 2: make money from that.

49
00:03:00,280 --> 00:03:02,840
Speaker 1: And Sam, that's a really good point because Wall Street

50
00:03:02,840 --> 00:03:06,440
Speaker 1: has been using computers and modeling in a really sophisticated

51
00:03:06,480 --> 00:03:09,320
Speaker 1: way long before any of us were talking about AI

52
00:03:09,440 --> 00:03:10,600
Speaker 1: and chat gyput.

53
00:03:11,080 --> 00:03:13,720
Speaker 3: That's absolutely right. Whereas and I think one of the

54
00:03:13,720 --> 00:03:16,920
Speaker 3: things that Justina and I talked about through the process

55
00:03:16,919 --> 00:03:19,239
Speaker 3: of this story was when you look back over the

56
00:03:19,280 --> 00:03:22,640
Speaker 3: past ten years or so, especially places like Bloomberg News

57
00:03:22,680 --> 00:03:26,000
Speaker 3: and other platforms have repeatedly said, you know, AI is

58
00:03:26,080 --> 00:03:29,400
Speaker 3: coming further jobs on Wall Street, Computers are going to

59
00:03:29,440 --> 00:03:31,919
Speaker 3: replace traders. You know, humans are going to be a thing.

60
00:03:31,760 --> 00:03:32,280
Speaker 1: Of the past.

61
00:03:32,520 --> 00:03:35,560
Speaker 3: The AI revolution is upon us. We see those stories

62
00:03:35,600 --> 00:03:38,280
Speaker 3: time and time again. But one question that we had

63
00:03:38,320 --> 00:03:41,560
Speaker 3: through this story was where is it then, especially in

64
00:03:41,600 --> 00:03:44,960
Speaker 3: a year when chat GPT has blown up, everyone's in

65
00:03:45,000 --> 00:03:47,840
Speaker 3: a frenzy about it. It's powered a lot of the

66
00:03:47,880 --> 00:03:51,160
Speaker 3: stock gains this year has been this excitement over stocks

67
00:03:51,200 --> 00:03:54,840
Speaker 3: like Navidia who are building AI technology, and yet wall

68
00:03:54,880 --> 00:03:59,120
Speaker 3: streets and the business of investing seems no closer, Like

69
00:03:59,200 --> 00:04:03,800
Speaker 3: AI has not displace the humans yet. Yes, quants are

70
00:04:03,840 --> 00:04:06,760
Speaker 3: out there and they're using the latest tech, but where's

71
00:04:06,760 --> 00:04:10,440
Speaker 3: the revolution? Is it still happening or are the human

72
00:04:10,480 --> 00:04:11,680
Speaker 3: trade as say for a while?

73
00:04:11,760 --> 00:04:17,000
Speaker 1: Yeah, Justina, what is the difference between what I guess

74
00:04:17,000 --> 00:04:20,320
Speaker 1: you would call traditional quants, that people who are digging

75
00:04:20,360 --> 00:04:24,720
Speaker 1: into numbers and making models what they do, and what

76
00:04:24,920 --> 00:04:27,520
Speaker 1: AI platforms are now trying to do.

77
00:04:28,360 --> 00:04:31,200
Speaker 2: Yeah, that really is a great question, and it's mostly

78
00:04:31,560 --> 00:04:35,920
Speaker 2: a difference in statistical technique. So for traditional quants, a

79
00:04:35,920 --> 00:04:38,920
Speaker 2: lot of the time, if we simplify it, what they're

80
00:04:38,920 --> 00:04:43,040
Speaker 2: trying to figure out is the relationship between stock returns

81
00:04:43,120 --> 00:04:46,039
Speaker 2: and some variable. Right, So for instance, you might look

82
00:04:46,080 --> 00:04:49,560
Speaker 2: at Apple's profit margins and decide that it's a great buy.

83
00:04:50,120 --> 00:04:52,840
Speaker 2: So quants are doing that, but they're doing it statistically,

84
00:04:53,480 --> 00:04:56,360
Speaker 2: and to simplify a little bit, usually they're using what

85
00:04:56,400 --> 00:05:00,599
Speaker 2: we call linear models, which means they're saying this leads

86
00:05:00,600 --> 00:05:03,600
Speaker 2: to this, which is a bit simplistic, but at least

87
00:05:03,600 --> 00:05:06,240
Speaker 2: you can understand it and you can kind of understand

88
00:05:06,279 --> 00:05:09,280
Speaker 2: why that does not always work out because there are

89
00:05:09,320 --> 00:05:12,720
Speaker 2: so many variables moving a stock performance every single day,

90
00:05:13,040 --> 00:05:16,240
Speaker 2: and what machine learning algos generally try to do is

91
00:05:16,520 --> 00:05:20,000
Speaker 2: it can take in hundreds of variables, all the information

92
00:05:20,080 --> 00:05:22,800
Speaker 2: you can find, and try to figure out what the

93
00:05:22,880 --> 00:05:25,960
Speaker 2: relationship between all of them is and how they predict

94
00:05:26,120 --> 00:05:29,440
Speaker 2: the price performance. The downside of that is that you

95
00:05:29,480 --> 00:05:33,440
Speaker 2: don't really understand why the machine has decided to tell

96
00:05:33,480 --> 00:05:36,000
Speaker 2: you to buy a stock that day. But the flip

97
00:05:36,040 --> 00:05:38,960
Speaker 2: side of that is that it kind of comes closer

98
00:05:39,080 --> 00:05:41,160
Speaker 2: to just how complicated the world is.

99
00:05:41,920 --> 00:05:44,480
Speaker 1: And so SAM is part of the idea that the

100
00:05:44,600 --> 00:05:48,560
Speaker 1: models that quants use are based on a certain number

101
00:05:48,720 --> 00:05:51,640
Speaker 1: of these variables that they put together to try to

102
00:05:51,640 --> 00:05:54,840
Speaker 1: make predictions. But these AI models are able to take

103
00:05:54,880 --> 00:05:58,320
Speaker 1: in many, many, many more of these and paint a

104
00:05:58,360 --> 00:06:02,239
Speaker 1: more sophisticated picture of what's happening in the market.

105
00:06:02,880 --> 00:06:06,400
Speaker 3: I would say that YESAI is taking into account many

106
00:06:06,920 --> 00:06:11,360
Speaker 3: more variables than the traditional quant methods. Traditional quants, many

107
00:06:11,400 --> 00:06:14,159
Speaker 3: of them, are trying to adopt these adapt these layer

108
00:06:14,200 --> 00:06:16,920
Speaker 3: them onto the theories, the academic theories that they built

109
00:06:16,960 --> 00:06:19,440
Speaker 3: all those years ago, and there's kind of debates within

110
00:06:19,480 --> 00:06:21,520
Speaker 3: the industry over how effective it is to do that.

111
00:06:22,400 --> 00:06:24,800
Speaker 2: And you know, we always hear that the world is

112
00:06:24,880 --> 00:06:28,080
Speaker 2: producing so much more data these days. Quants are also

113
00:06:28,160 --> 00:06:31,120
Speaker 2: looking at our tweets, for instance, or our news headlines.

114
00:06:31,360 --> 00:06:35,000
Speaker 2: I mean they're not actually reading them directly, but they

115
00:06:35,000 --> 00:06:38,520
Speaker 2: are using a kind of machine learning algorithm called natural

116
00:06:38,560 --> 00:06:42,719
Speaker 2: language processing, where they try to turn all the material

117
00:06:42,800 --> 00:06:45,440
Speaker 2: in the world into numbers that they can analyze.

118
00:06:45,920 --> 00:06:48,359
Speaker 1: And I guess that's really sort of the holy grail

119
00:06:48,440 --> 00:06:52,800
Speaker 1: of this right is trying to understand the human behavior

120
00:06:52,960 --> 00:06:56,920
Speaker 1: that underlies all the decisions that go into markets moving

121
00:06:57,000 --> 00:06:57,520
Speaker 1: up or down.

122
00:06:58,240 --> 00:07:01,040
Speaker 2: Yeah, exactly. I mean that's sort of hardest part. I mean,

123
00:07:01,040 --> 00:07:03,159
Speaker 2: if you can predict where the S and P five

124
00:07:03,240 --> 00:07:05,279
Speaker 2: hundred goes every day, I mean, you would be a

125
00:07:05,360 --> 00:07:09,320
Speaker 2: rich person. And that is the hardest part, especially in

126
00:07:09,480 --> 00:07:12,160
Speaker 2: the US thought market, for instance, which is so efficient.

127
00:07:12,480 --> 00:07:15,160
Speaker 2: So even in my story, when I spoke to a

128
00:07:15,160 --> 00:07:17,760
Speaker 2: lot of these quants who are so optimistic about the

129
00:07:17,800 --> 00:07:20,640
Speaker 2: progress in AI, they always say, you know, we're just

130
00:07:20,680 --> 00:07:22,600
Speaker 2: looking for a small edge, But a small edge on

131
00:07:22,640 --> 00:07:24,880
Speaker 2: Wall Street can mean a lot more money. So even

132
00:07:24,880 --> 00:07:26,880
Speaker 2: if you're not right every day, if you're right, more

133
00:07:26,960 --> 00:07:28,800
Speaker 2: than half the time. I mean, that's good enough.

134
00:07:30,400 --> 00:07:32,320
Speaker 1: Justin in your story, you spoke to a lot of

135
00:07:32,320 --> 00:07:34,400
Speaker 1: the people who are doing this work. Are trying to

136
00:07:34,440 --> 00:07:37,440
Speaker 1: incorporate AI into trading. Who are they?

137
00:07:38,120 --> 00:07:40,600
Speaker 2: Yeah, obviously a lot of quants will say they use

138
00:07:40,760 --> 00:07:44,040
Speaker 2: machine learning, but there are some hedge funds that are

139
00:07:44,160 --> 00:07:47,560
Speaker 2: known for the AI prowess. So, for instance, I spoke

140
00:07:47,600 --> 00:07:52,440
Speaker 2: to Michael Kurrichanoff, who's the co founder of Volon, which

141
00:07:52,480 --> 00:07:54,720
Speaker 2: is one of the few hedge funds known for AI.

142
00:07:55,200 --> 00:07:58,120
Speaker 2: And he's a guy, you know who was a physicist,

143
00:07:58,240 --> 00:08:01,280
Speaker 2: you know, who has a computer science PhD. After a

144
00:08:01,360 --> 00:08:05,360
Speaker 2: period in working in Silicon Valley, he went to Wall Street.

145
00:08:05,800 --> 00:08:08,880
Speaker 2: He started bolly On about sixteen years ago with the

146
00:08:08,960 --> 00:08:12,840
Speaker 2: idea of using machine learning to try to predict where

147
00:08:12,960 --> 00:08:15,560
Speaker 2: the equity market will go. And if you talk to

148
00:08:15,560 --> 00:08:18,680
Speaker 2: people like him, one interesting thing is they almost sometimes

149
00:08:18,680 --> 00:08:22,120
Speaker 2: sound a little bit scornful of all other quants, in

150
00:08:22,200 --> 00:08:25,280
Speaker 2: that they really see themselves as doing something that's different

151
00:08:25,360 --> 00:08:28,160
Speaker 2: from the last generation of quants, and their pitch to

152
00:08:28,240 --> 00:08:31,680
Speaker 2: investors is we can give you something that doesn't look

153
00:08:31,720 --> 00:08:34,440
Speaker 2: like anything else out there. So even when your other

154
00:08:34,520 --> 00:08:37,839
Speaker 2: quant strategies fail, we will be making money and.

155
00:08:37,840 --> 00:08:41,520
Speaker 1: Sam exactly what is it that they're telling potential clients

156
00:08:41,559 --> 00:08:44,680
Speaker 1: that they're going to be able to do with this technology.

157
00:08:45,440 --> 00:08:48,320
Speaker 3: I think one interesting thing about Boleion is that the

158
00:08:48,360 --> 00:08:52,520
Speaker 3: perfect example of this run by a data scientist who

159
00:08:52,600 --> 00:08:56,439
Speaker 3: used to do nuclear physics, and they based themselves near

160
00:08:56,440 --> 00:08:59,000
Speaker 3: the Berkeley campus so that they're close to a center

161
00:08:59,040 --> 00:09:04,160
Speaker 3: of world machine learning. The people trying to harness this

162
00:09:04,280 --> 00:09:08,640
Speaker 3: AI and beat Wall Street and not your classic finance types.

163
00:09:08,920 --> 00:09:12,880
Speaker 3: They are data people, they are statisticians, they are programmers,

164
00:09:13,160 --> 00:09:16,280
Speaker 3: and I think that is an interesting dynamic that this

165
00:09:16,360 --> 00:09:19,280
Speaker 3: sort of power based on Wall Street is potentially moving

166
00:09:19,360 --> 00:09:26,439
Speaker 3: away from the classic finance economist types and towards mathematicians. Basically,

167
00:09:26,880 --> 00:09:30,360
Speaker 3: certainly for Volion, they're presenting something that is uncorrelated. You

168
00:09:30,400 --> 00:09:33,760
Speaker 3: imagine you're a giant investment firm and you're placing your

169
00:09:33,800 --> 00:09:36,079
Speaker 3: money just directly in the stock market or in a

170
00:09:36,160 --> 00:09:40,760
Speaker 3: track of fund and in some bonds. You basically create

171
00:09:40,840 --> 00:09:43,360
Speaker 3: a risk exposure that is the same as the market.

172
00:09:43,600 --> 00:09:46,120
Speaker 3: You are vulnerable to the same things. So a lot

173
00:09:46,120 --> 00:09:48,920
Speaker 3: of what people are looking for on Wall Street is uncorrelated.

174
00:09:49,080 --> 00:09:51,640
Speaker 3: It's why we see a lot of activity in occasionally

175
00:09:51,720 --> 00:09:55,079
Speaker 3: very random commodities funds and things like that. Because these

176
00:09:55,120 --> 00:09:57,640
Speaker 3: big investment houses are trying to spread out their risk,

177
00:09:58,000 --> 00:10:01,800
Speaker 3: they're trying to be smart about it. Cracked, beating the market,

178
00:10:01,840 --> 00:10:05,880
Speaker 3: beating the benchmark doesn't seem so just yet, they do

179
00:10:05,960 --> 00:10:08,360
Speaker 3: say they are providing a return that is going to

180
00:10:08,360 --> 00:10:10,840
Speaker 3: behave differently to anything else, and that's very appealing to

181
00:10:10,920 --> 00:10:11,960
Speaker 3: certain money managers.

182
00:10:13,360 --> 00:10:16,000
Speaker 1: So is the idea that when the herd of the

183
00:10:16,040 --> 00:10:19,080
Speaker 1: market is zigging, that AI will allow these firms to

184
00:10:19,200 --> 00:10:21,520
Speaker 1: zag and take advantage of opportunities that the rest of

185
00:10:21,520 --> 00:10:22,880
Speaker 1: the market just doesn't see.

186
00:10:23,280 --> 00:10:26,520
Speaker 2: Yeah, I think that's exactly right. And the reasoning behind

187
00:10:26,600 --> 00:10:29,720
Speaker 2: that is probably that they are taking in so many

188
00:10:29,880 --> 00:10:34,440
Speaker 2: variables from everything that they can get into a more sophisticated,

189
00:10:34,840 --> 00:10:39,199
Speaker 2: superhuman view of what markets are really doing. And one

190
00:10:39,200 --> 00:10:42,680
Speaker 2: thing that they really emphasized is they don't have a model.

191
00:10:42,720 --> 00:10:44,679
Speaker 2: I mean, they don't have an economic theory, but they

192
00:10:44,760 --> 00:10:46,960
Speaker 2: just trust the data. You know, in the sense that

193
00:10:47,120 --> 00:10:49,520
Speaker 2: you know, chat GPT is reading so much Shakespeare that

194
00:10:49,559 --> 00:10:52,200
Speaker 2: it can write Shakespeare. So it's the same idea, which

195
00:10:52,240 --> 00:10:55,520
Speaker 2: is that the machines are looking at so many years

196
00:10:55,640 --> 00:10:58,560
Speaker 2: of market data that they can tell you based on

197
00:10:58,640 --> 00:11:00,800
Speaker 2: what the market looks like right now, here's where prices

198
00:11:00,800 --> 00:11:03,320
Speaker 2: are going to go. And even if you cannot reason

199
00:11:03,360 --> 00:11:05,319
Speaker 2: it as a human, it will tell you the right

200
00:11:05,360 --> 00:11:07,359
Speaker 2: answer after.

201
00:11:07,120 --> 00:11:10,400
Speaker 1: The break, are these AI models beating the market?

202
00:11:17,840 --> 00:11:18,040
Speaker 3: Same?

203
00:11:18,120 --> 00:11:20,600
Speaker 1: I guess on the flip side, taking advantage of all

204
00:11:20,720 --> 00:11:25,679
Speaker 1: this information may not necessarily lead to a clearer picture.

205
00:11:26,000 --> 00:11:28,200
Speaker 1: We always try to figure out why the stock market

206
00:11:28,320 --> 00:11:30,400
Speaker 1: went up or down on a given day, but often

207
00:11:30,520 --> 00:11:34,040
Speaker 1: it's not really clear. So does all this data that

208
00:11:34,120 --> 00:11:38,720
Speaker 1: says what happens in the past necessarily predict what's happening now.

209
00:11:39,280 --> 00:11:41,080
Speaker 3: I've got a smile on my face because you're talking

210
00:11:41,080 --> 00:11:44,199
Speaker 3: to a guy who ran Bloomberg's Global Wrap for about

211
00:11:44,240 --> 00:11:47,559
Speaker 3: three years. So my job every day was to define

212
00:11:47,559 --> 00:11:50,560
Speaker 3: a narrative. You know, why why stocks are up, why

213
00:11:50,640 --> 00:11:53,040
Speaker 3: bonds are down, et cetera. And it's true. And this

214
00:11:53,120 --> 00:11:55,360
Speaker 3: is where the kind of rubber meets the road in

215
00:11:55,400 --> 00:11:58,680
Speaker 3: the story, the AI and the machines are finding it

216
00:11:58,800 --> 00:12:02,000
Speaker 3: just as tough as everyone else to figure out what's

217
00:12:02,040 --> 00:12:04,240
Speaker 3: the driver and where it's going next. One of the

218
00:12:04,280 --> 00:12:08,959
Speaker 3: problems is regime shifts happen in markets from nowhere when

219
00:12:09,000 --> 00:12:11,600
Speaker 3: the pandemic hit, we obviously had a massive crash. For

220
00:12:11,679 --> 00:12:16,360
Speaker 3: anyone to see that coming machine or otherwise was near impossible.

221
00:12:16,640 --> 00:12:20,760
Speaker 3: And then afterwards, with the various pandemic support and emergency

222
00:12:20,800 --> 00:12:24,079
Speaker 3: aid from governments, we got a rapid rebound and some

223
00:12:24,160 --> 00:12:27,000
Speaker 3: of the AI funds were caught out by that. Basically,

224
00:12:27,040 --> 00:12:30,319
Speaker 3: the AI the machine. It can look at the complete

225
00:12:30,480 --> 00:12:35,160
Speaker 3: history of all stop moves. Ever, something can happen, something external,

226
00:12:35,280 --> 00:12:39,960
Speaker 3: something macro, that makes tomorrow different, tomorrow a thing that

227
00:12:40,080 --> 00:12:43,480
Speaker 3: never happened before, and that's probably there. Arguably the biggest

228
00:12:43,480 --> 00:12:46,240
Speaker 3: shortcoming is you can learn all you want from history,

229
00:12:46,280 --> 00:12:48,840
Speaker 3: but it doesn't tell you what's going to happen next.

230
00:12:49,280 --> 00:12:52,760
Speaker 1: Justine, you also talked about another fun called Man Group. Yeah.

231
00:12:52,800 --> 00:12:56,400
Speaker 2: So Man Group is the biggest listed hatch fund in

232
00:12:56,400 --> 00:12:59,280
Speaker 2: the world and they're a huge company based in London.

233
00:13:00,000 --> 00:13:03,280
Speaker 2: And what I found interesting about their story is they

234
00:13:03,280 --> 00:13:06,520
Speaker 2: actually started looking into applying machine learning back in two

235
00:13:06,520 --> 00:13:09,679
Speaker 2: thousand and nine, but they didn't really start using it

236
00:13:09,760 --> 00:13:13,680
Speaker 2: and actual trading until twenty fourteen. And that just kind

237
00:13:13,679 --> 00:13:16,880
Speaker 2: of tells you how much research and work goes into

238
00:13:17,000 --> 00:13:20,200
Speaker 2: getting it right and making sure it actually helps and

239
00:13:20,240 --> 00:13:23,360
Speaker 2: how even though we've seen so many breakthroughs in Silicon Valley,

240
00:13:23,400 --> 00:13:26,920
Speaker 2: the financial industry can feel like it's many years behind.

241
00:13:27,559 --> 00:13:30,760
Speaker 2: Part of the issue is when you're using machine learning algorithm,

242
00:13:31,000 --> 00:13:34,920
Speaker 2: you lose the explainability of it. So when you're losing money,

243
00:13:35,000 --> 00:13:37,240
Speaker 2: your clients might be like, why did you decide to

244
00:13:37,280 --> 00:13:40,280
Speaker 2: tred that you can't exactly point to a formula or

245
00:13:40,320 --> 00:13:43,880
Speaker 2: point to your reasoning because machine learning, I mean, the

246
00:13:43,920 --> 00:13:46,880
Speaker 2: beauty of it is that it can figure out all

247
00:13:46,960 --> 00:13:51,120
Speaker 2: these complex relationships that you cannot exactly reverse engineer rationally,

248
00:13:51,559 --> 00:13:53,800
Speaker 2: and so it's really hard to explain and that is

249
00:13:53,920 --> 00:13:56,640
Speaker 2: a bigger issue in finance than it might be. You know,

250
00:13:56,679 --> 00:13:58,640
Speaker 2: for chat GPT for instance.

251
00:14:00,040 --> 00:14:02,680
Speaker 1: Same is there any downside to putting so much into

252
00:14:02,800 --> 00:14:05,800
Speaker 1: these AI models? When you look at something like chat GPT,

253
00:14:06,040 --> 00:14:07,960
Speaker 1: one of the big problems is what they call the

254
00:14:08,040 --> 00:14:11,760
Speaker 1: hallucination rate, where it just gets it completely wrong and

255
00:14:11,840 --> 00:14:14,839
Speaker 1: yet asserts it with total confidence. How do we know

256
00:14:14,920 --> 00:14:18,480
Speaker 1: that these models are getting it right? We don't.

257
00:14:18,559 --> 00:14:20,920
Speaker 3: And that's where the real debate in the field is

258
00:14:20,960 --> 00:14:25,320
Speaker 3: coming from those who say we need to overlay some

259
00:14:25,520 --> 00:14:29,080
Speaker 3: economic theory so that they know they have some idea

260
00:14:29,080 --> 00:14:32,360
Speaker 3: what they're looking for. What the advocates of just turning

261
00:14:32,360 --> 00:14:34,320
Speaker 3: the machines loose would tell you is that if you

262
00:14:34,400 --> 00:14:36,440
Speaker 3: let them loose on the data, they will find the

263
00:14:36,520 --> 00:14:39,560
Speaker 3: old school quant rules at the same time that they

264
00:14:39,600 --> 00:14:40,640
Speaker 3: find everything else.

265
00:14:41,520 --> 00:14:44,160
Speaker 2: And I think that's also why a lot of hatch

266
00:14:44,200 --> 00:14:48,000
Speaker 2: funds and financial firms aren't using it to predict prices

267
00:14:48,040 --> 00:14:50,360
Speaker 2: and trade based on that just yet, because they really

268
00:14:50,400 --> 00:14:54,120
Speaker 2: want to be sure that's better than everything they've ever done.

269
00:14:54,400 --> 00:14:58,320
Speaker 1: After hearing this and seeing how so many firms and

270
00:14:58,440 --> 00:15:02,240
Speaker 1: smart people are in are being put into this question,

271
00:15:03,000 --> 00:15:06,000
Speaker 1: is it working? Do these AI models beat the market?

272
00:15:06,040 --> 00:15:08,480
Speaker 1: Do they do better than the old fashioned and new

273
00:15:08,520 --> 00:15:10,440
Speaker 1: fashioned ways of trading?

274
00:15:11,080 --> 00:15:13,320
Speaker 2: I really want to josh the question and.

275
00:15:13,240 --> 00:15:15,240
Speaker 1: Say the cury is still out.

276
00:15:15,520 --> 00:15:17,720
Speaker 2: I mean, if you look at the track record, the

277
00:15:17,800 --> 00:15:20,960
Speaker 2: answer definitely is not mind blowing. So in the story,

278
00:15:21,080 --> 00:15:24,560
Speaker 2: I cite this academic paper that looked at mutual funds

279
00:15:24,600 --> 00:15:27,720
Speaker 2: that use AI and what have found was that most

280
00:15:27,720 --> 00:15:30,760
Speaker 2: of them still did not beat their benchmark, even if

281
00:15:30,800 --> 00:15:33,920
Speaker 2: they were a bit better than the human managed funds.

282
00:15:34,240 --> 00:15:37,240
Speaker 2: And in the exchange traded fund market in the US

283
00:15:37,480 --> 00:15:40,360
Speaker 2: that pick stocks based on an AI that has a

284
00:15:40,400 --> 00:15:43,800
Speaker 2: pretty long track record, and that also has not done

285
00:15:43,920 --> 00:15:45,880
Speaker 2: that well. And I think if you look at the

286
00:15:45,960 --> 00:15:49,640
Speaker 2: more sophisticated AI hatch funds, what people would say is,

287
00:15:49,840 --> 00:15:52,040
Speaker 2: you know, they've done pretty well, but it's not going

288
00:15:52,120 --> 00:15:54,520
Speaker 2: to be like number one on the leaderboard and it's

289
00:15:54,560 --> 00:15:57,200
Speaker 2: not going to like blow your mind. What was clear

290
00:15:57,280 --> 00:16:00,800
Speaker 2: to me is that the use of these methods is increasing,

291
00:16:01,040 --> 00:16:03,640
Speaker 2: but a lot of the time they're using it but

292
00:16:03,760 --> 00:16:06,600
Speaker 2: not necessarily letting it go wild just yet.

293
00:16:07,400 --> 00:16:10,840
Speaker 3: What we've seen with quantum besters and quant strategies of

294
00:16:10,880 --> 00:16:13,440
Speaker 3: the old school is that when they found something that worked,

295
00:16:13,840 --> 00:16:16,280
Speaker 3: everyone quickly piled into the trade, and then suddenly it

296
00:16:16,320 --> 00:16:18,840
Speaker 3: didn't work anymore because there are too many people doing it.

297
00:16:18,840 --> 00:16:21,320
Speaker 3: The gap gets closed very quickly and they can't make money.

298
00:16:21,720 --> 00:16:26,240
Speaker 3: As these AI programs get more sophisticated, more embedded in

299
00:16:26,280 --> 00:16:29,720
Speaker 3: the market, any inefficiencies they find, they're going to disappear faster.

300
00:16:29,840 --> 00:16:32,680
Speaker 3: The market actually could end up way more efficient and

301
00:16:32,800 --> 00:16:37,000
Speaker 3: making excess profit, excess return. Beating other people in the

302
00:16:37,000 --> 00:16:39,680
Speaker 3: market might just get harder and harder because the computers

303
00:16:39,720 --> 00:16:40,520
Speaker 3: get so good at it.

304
00:16:41,560 --> 00:16:45,480
Speaker 1: So we know that this technology looks promising, isn't quite

305
00:16:45,600 --> 00:16:48,640
Speaker 1: beating the market yet overall, But when you look down

306
00:16:48,640 --> 00:16:53,000
Speaker 1: the road, given how quickly this technology in AI in

307
00:16:53,080 --> 00:16:56,280
Speaker 1: general is advancing, what do you see, where do things

308
00:16:56,320 --> 00:16:59,080
Speaker 1: go from here? In how it's used in markets.

309
00:16:59,360 --> 00:17:02,920
Speaker 2: In terms of actually using machine learning to predict prices

310
00:17:03,160 --> 00:17:05,639
Speaker 2: and decide what to trade. I think that's probably going

311
00:17:05,720 --> 00:17:08,760
Speaker 2: to happen, but at a slower pace, just because it's

312
00:17:08,880 --> 00:17:13,639
Speaker 2: much harder to make that decision to hand the reins

313
00:17:13,680 --> 00:17:16,880
Speaker 2: of your money management to a machine that you don't

314
00:17:16,960 --> 00:17:20,080
Speaker 2: understand that well.

315
00:17:19,040 --> 00:17:21,919
Speaker 1: Sam Justina, thanks so much for sharing your reporting. I

316
00:17:21,960 --> 00:17:24,440
Speaker 1: have a feeling we'll be talking about this again.

317
00:17:25,000 --> 00:17:26,280
Speaker 2: Thank you so much for having us.

318
00:17:27,560 --> 00:17:29,960
Speaker 1: When we come back, we'll talk with a trader who's

319
00:17:30,080 --> 00:17:41,480
Speaker 1: all in on AI. Now, let's hear from someone who's

320
00:17:41,480 --> 00:17:45,399
Speaker 1: actually putting these AI models to work. Renee Yao is

321
00:17:45,400 --> 00:17:49,040
Speaker 1: the founder of neo Ivy Capital. It's a quantitative hedge

322
00:17:49,040 --> 00:17:54,280
Speaker 1: fund that uses artificial intelligence in its investment decisions. You

323
00:17:54,320 --> 00:17:58,920
Speaker 1: are a quantitative trader. Exactly what is that and how

324
00:17:58,960 --> 00:18:01,920
Speaker 1: does it differ from AI that's not being used.

325
00:18:02,840 --> 00:18:08,200
Speaker 4: So from a normal person's standpoint of view, market might

326
00:18:08,280 --> 00:18:14,920
Speaker 4: be full of noise or chaos, but behind those chaos

327
00:18:15,000 --> 00:18:23,040
Speaker 4: or randomness market actually does appear certain patterns, and AI

328
00:18:23,720 --> 00:18:30,080
Speaker 4: works better as a noise cancelor compared to traditional mesters.

329
00:18:30,680 --> 00:18:34,199
Speaker 4: If you look at the history of QUANTU investment, it

330
00:18:34,359 --> 00:18:40,399
Speaker 4: actually undergone three different generations of big changes. The first

331
00:18:40,400 --> 00:18:45,160
Speaker 4: generation quants probably dates back to nineteen eighties nineties, where

332
00:18:45,280 --> 00:18:50,919
Speaker 4: people like Solomon Brothers are trying to mimic what the

333
00:18:50,960 --> 00:18:55,280
Speaker 4: financial analysts we're trying to do and they use simple

334
00:18:55,480 --> 00:19:01,240
Speaker 4: implementation methods like Excel spreadsheet to do this their calculations.

335
00:19:01,560 --> 00:19:06,399
Speaker 4: And later the second generation quant they realize they not

336
00:19:06,520 --> 00:19:10,320
Speaker 4: only need to have a forecast of each of the stocks,

337
00:19:10,680 --> 00:19:15,840
Speaker 4: they need to have different forecasts that has uniqueness that

338
00:19:16,000 --> 00:19:20,160
Speaker 4: others don't have, which is why they're trying to hire

339
00:19:20,400 --> 00:19:23,720
Speaker 4: hundreds of thousands of people to try to achieve that

340
00:19:24,000 --> 00:19:25,679
Speaker 4: unique edge.

341
00:19:25,800 --> 00:19:28,600
Speaker 1: And they're all trying to create computer models and other

342
00:19:28,760 --> 00:19:31,960
Speaker 1: methods to look at the market and find holes in

343
00:19:32,000 --> 00:19:33,760
Speaker 1: the market that other people don't see.

344
00:19:34,720 --> 00:19:39,359
Speaker 4: Yes, they're trying to do data minings, pattern recognitions from

345
00:19:39,520 --> 00:19:42,400
Speaker 4: historic data and trying to see if there's certainly pattern

346
00:19:42,680 --> 00:19:45,880
Speaker 4: that happened in the historic data and hopefully that's going

347
00:19:45,920 --> 00:19:49,880
Speaker 4: to happen again in the future. And then what happens

348
00:19:50,000 --> 00:19:52,800
Speaker 4: is we do we consider ourselves to be the third

349
00:19:52,840 --> 00:19:56,359
Speaker 4: generation quant which is we're not trying to do data

350
00:19:56,480 --> 00:20:01,080
Speaker 4: mining pattern recognition. Instead, we use over AI I augorism

351
00:20:01,600 --> 00:20:06,000
Speaker 4: to trying to come up with fast new ideas automatically

352
00:20:06,040 --> 00:20:10,760
Speaker 4: for us, and they look at leading indicators to predict

353
00:20:10,840 --> 00:20:14,520
Speaker 4: the future instead of the lagging indicators of what happened

354
00:20:14,560 --> 00:20:15,560
Speaker 4: already in the past.

355
00:20:16,520 --> 00:20:19,840
Speaker 1: So you no longer need the hundreds or thousands of

356
00:20:19,960 --> 00:20:23,800
Speaker 1: individual incredibly smart people all working away on computers to

357
00:20:23,840 --> 00:20:26,439
Speaker 1: find patterns. You just let AI do it for you.

358
00:20:27,480 --> 00:20:33,280
Speaker 4: Yes, in some scenario, I think our AI allg has

359
00:20:33,560 --> 00:20:37,959
Speaker 4: replaced the functionality of the human researchers.

360
00:20:38,560 --> 00:20:41,040
Speaker 1: That's really interesting where you put it that there's o

361
00:20:41,160 --> 00:20:44,000
Speaker 1: this noise. What's an example of noise? And how does

362
00:20:44,040 --> 00:20:46,120
Speaker 1: it get in the way of figuring out which way

363
00:20:46,119 --> 00:20:47,160
Speaker 1: a market is going to move?

364
00:20:48,280 --> 00:20:52,240
Speaker 4: So solving the market direction is like try to solve

365
00:20:52,359 --> 00:20:55,480
Speaker 4: a crime see problem, Like we're trying to figure out

366
00:20:56,080 --> 00:20:59,479
Speaker 4: who is a suspect. If we have a video showing

367
00:20:59,560 --> 00:21:03,040
Speaker 4: what happen happen exactly at the crime scene, that will

368
00:21:03,080 --> 00:21:07,240
Speaker 4: be like perfect signals because it can leads us to

369
00:21:07,280 --> 00:21:12,439
Speaker 4: directly who the suspect is. However, in most of the

370
00:21:12,520 --> 00:21:15,200
Speaker 4: time that's not the case. We don't have a video

371
00:21:15,280 --> 00:21:18,359
Speaker 4: capture what happened back then. So what we can do

372
00:21:18,520 --> 00:21:22,680
Speaker 4: is we look at things that are relevant that will

373
00:21:22,720 --> 00:21:26,840
Speaker 4: give us certain indications as to what the suspect looks like.

374
00:21:27,240 --> 00:21:30,199
Speaker 4: For example, certain things might lead to hey, the expect

375
00:21:30,240 --> 00:21:33,520
Speaker 4: late is a male or female, and so AI it's

376
00:21:33,600 --> 00:21:40,400
Speaker 4: good at capturing rules, little small pieces and put them together, and.

377
00:21:40,400 --> 00:21:44,000
Speaker 1: So AI is able to sort through all the different

378
00:21:44,040 --> 00:21:46,760
Speaker 1: things that might be causing a market to move, and

379
00:21:46,800 --> 00:21:49,040
Speaker 1: it's always difficult to tell which one of the many

380
00:21:49,080 --> 00:21:51,800
Speaker 1: many things it might be in zero in on the

381
00:21:51,800 --> 00:21:56,359
Speaker 1: ones that might be the most important. Yes, exactly, and

382
00:21:56,400 --> 00:21:57,440
Speaker 1: how does it do that?

383
00:21:58,000 --> 00:22:02,440
Speaker 4: A good example of traditional how machine learning organism will

384
00:22:02,520 --> 00:22:06,479
Speaker 4: be IBM's The Blue All Go, where IBM built and

385
00:22:06,520 --> 00:22:09,480
Speaker 4: released in the nineteen nineties and they use that to

386
00:22:09,480 --> 00:22:13,440
Speaker 4: be the Russian chess champion. Now, the reason we can

387
00:22:13,560 --> 00:22:17,240
Speaker 4: use traditional machine learning for chess game is because chess

388
00:22:17,240 --> 00:22:21,439
Speaker 4: game has so many well defined rules, like queens can

389
00:22:21,480 --> 00:22:23,680
Speaker 4: only move in a certain way in knights can move

390
00:22:23,720 --> 00:22:26,639
Speaker 4: in a certain way, So after each movement you have

391
00:22:26,760 --> 00:22:32,080
Speaker 4: only finite possible solutions for next step. That's why, as

392
00:22:32,119 --> 00:22:35,440
Speaker 4: long as you have enough computing power, you can literally

393
00:22:35,480 --> 00:22:39,280
Speaker 4: tell your machine to go through every possible scenarios of

394
00:22:39,359 --> 00:22:44,560
Speaker 4: the next steps and handpick you the highest winning hand. However,

395
00:22:44,960 --> 00:22:48,840
Speaker 4: we don't have that luxury in real market problem right,

396
00:22:48,920 --> 00:22:51,040
Speaker 4: Like we don't know where sp is going to land

397
00:22:51,080 --> 00:22:51,920
Speaker 4: by the year end.

398
00:22:52,520 --> 00:22:55,520
Speaker 1: And there's a million rules and so many participants, and

399
00:22:55,600 --> 00:23:00,440
Speaker 1: so it's not a finite set of variables. Precisely, how

400
00:23:00,480 --> 00:23:03,440
Speaker 1: effective has it been? Are you finding that these models

401
00:23:03,520 --> 00:23:07,080
Speaker 1: are much better than previous models without AI?

402
00:23:08,119 --> 00:23:12,399
Speaker 4: I definitely think so. If you only look at lagging

403
00:23:12,560 --> 00:23:15,880
Speaker 4: nicular historic information and then you hope the market will

404
00:23:15,920 --> 00:23:21,040
Speaker 4: be in a history repeat itself pattern, then that's probably

405
00:23:21,040 --> 00:23:24,120
Speaker 4: going to have a lot of constraints because we live

406
00:23:24,160 --> 00:23:28,600
Speaker 4: in a day where every day like something unexpected happened, Like,

407
00:23:28,680 --> 00:23:32,520
Speaker 4: for example, just the past five years, we experienced COVID,

408
00:23:33,040 --> 00:23:37,760
Speaker 4: we experienced the sudden collapse of Silicon Valley Bank, one

409
00:23:37,840 --> 00:23:41,680
Speaker 4: of the largest regional banks in the US. Those are

410
00:23:41,960 --> 00:23:48,040
Speaker 4: all unprecedented events that didn't happen historically, So naturally that's

411
00:23:48,080 --> 00:23:51,760
Speaker 4: going to post a challenge for the traditional much learning models.

412
00:23:51,800 --> 00:23:57,760
Speaker 4: Where relies on history repeat itself. But for AI, because

413
00:23:57,760 --> 00:24:01,400
Speaker 4: the model is learning in real time like we do,

414
00:24:02,080 --> 00:24:06,320
Speaker 4: then it's able to navigate those different market conditions smartly

415
00:24:06,520 --> 00:24:08,240
Speaker 4: and deliver good returns.

416
00:24:08,840 --> 00:24:12,199
Speaker 1: You're not saying, though, that AI has the ability to predict,

417
00:24:12,440 --> 00:24:15,359
Speaker 1: say the collapse of a Silicon valley bank, but that

418
00:24:15,440 --> 00:24:17,800
Speaker 1: it's able to respond more quickly to figure out what

419
00:24:17,880 --> 00:24:20,080
Speaker 1: to do in the event that that happens.

420
00:24:20,480 --> 00:24:25,600
Speaker 4: Yes, our goal is to respond quickly and smartly as

421
00:24:25,640 --> 00:24:29,119
Speaker 4: to what to react if such unprecedented things happen.

422
00:24:30,800 --> 00:24:32,960
Speaker 1: So you said you're having good success with this, but

423
00:24:33,080 --> 00:24:35,400
Speaker 1: if you look at the overall numbers, I guess that

424
00:24:35,760 --> 00:24:39,720
Speaker 1: these AI algorithms aren't yet doing a great job of

425
00:24:39,880 --> 00:24:42,760
Speaker 1: just beating the overall market, doing better than say the

426
00:24:42,880 --> 00:24:44,000
Speaker 1: S and P five hundred.

427
00:24:44,760 --> 00:24:48,720
Speaker 4: That's actually one of the myths with AI. People's attitude

428
00:24:48,720 --> 00:24:52,160
Speaker 4: towards AI tends to go to the two extremes. Some

429
00:24:52,200 --> 00:24:55,359
Speaker 4: people just don't believe it. They think it's just a

430
00:24:55,520 --> 00:24:59,840
Speaker 4: ring name of a traditional technology that has been existing

431
00:24:59,880 --> 00:25:02,880
Speaker 4: for more than fifteen years. Or some people will say,

432
00:25:02,920 --> 00:25:06,160
Speaker 4: oh AI is God, it can do anything, it can

433
00:25:06,200 --> 00:25:12,520
Speaker 4: do everything. Well, my experience is AI is not stupid

434
00:25:13,160 --> 00:25:19,320
Speaker 4: nor it's god. AI does has its limitations because there

435
00:25:19,400 --> 00:25:24,400
Speaker 4: are periods of time where markets are completely randomness and

436
00:25:24,440 --> 00:25:28,679
Speaker 4: there's just no pattern for you to capture, SOI no

437
00:25:28,800 --> 00:25:33,600
Speaker 4: traditional machine learning will work in those periods of time. However,

438
00:25:34,240 --> 00:25:38,440
Speaker 4: during the period of time when market does have a pattern,

439
00:25:38,840 --> 00:25:42,440
Speaker 4: AI works better as a noise canceler compared to traditional

440
00:25:42,480 --> 00:25:43,040
Speaker 4: mation learning.

441
00:25:44,000 --> 00:25:46,320
Speaker 1: Where do you see all of this heading, do you

442
00:25:46,320 --> 00:25:50,000
Speaker 1: think that AI is going to more and more work

443
00:25:50,040 --> 00:25:53,320
Speaker 1: its way into markets and work its way into the

444
00:25:53,359 --> 00:25:55,080
Speaker 1: way trading is done.

445
00:25:55,800 --> 00:25:58,600
Speaker 4: I definitely think so. Eight years ago, when I first

446
00:25:58,680 --> 00:26:01,879
Speaker 4: came out and becoming a pat then build NEYV not

447
00:26:02,080 --> 00:26:05,840
Speaker 4: many people are interested in muchine learning of AI. But

448
00:26:05,960 --> 00:26:10,840
Speaker 4: now with Tauchipt, with Task last Auto Drive, more and

449
00:26:10,920 --> 00:26:15,240
Speaker 4: more people began to realize how powerful AI is and

450
00:26:15,280 --> 00:26:19,879
Speaker 4: how much more it can be done without just involved

451
00:26:20,480 --> 00:26:23,720
Speaker 4: over human So that's why I believe in the future

452
00:26:24,160 --> 00:26:26,920
Speaker 4: AI is definitely going to play a much more important

453
00:26:27,040 --> 00:26:30,280
Speaker 4: role than today in terms of its applications in the

454
00:26:30,320 --> 00:26:31,280
Speaker 4: financial market.

455
00:26:32,200 --> 00:26:35,800
Speaker 1: Renee, thanks so much for giving us a look inside

456
00:26:36,080 --> 00:26:36,840
Speaker 1: what quants do.

457
00:26:37,119 --> 00:26:37,920
Speaker 4: Thank you so much.

458
00:26:38,000 --> 00:26:40,240
Speaker 1: Wes, thanks for listening to us here at the Big

459
00:26:40,280 --> 00:26:43,200
Speaker 1: Take it's a daily podcast from Bloomberg and iHeart Radio.

460
00:26:43,440 --> 00:26:47,560
Speaker 1: For more shows from iHeartRadio, visit the iHeartRadio app, Apple Podcasts,

461
00:26:47,640 --> 00:26:50,359
Speaker 1: or wherever you listen. And we'd love to hear from you.

462
00:26:50,640 --> 00:26:53,840
Speaker 1: Email us questions or comments to Big Take at Bloomberg

463
00:26:53,840 --> 00:26:56,840
Speaker 1: dot net. The supervising producer of The Big Take is

464
00:26:56,920 --> 00:27:01,240
Speaker 1: Vicky Bergolina. Our senior producer is Catherine Fink. Our producers

465
00:27:01,280 --> 00:27:05,480
Speaker 1: are Mow Barrow and Michael Falero. Kilde Garcia is our engineer.

466
00:27:05,920 --> 00:27:10,040
Speaker 1: Our original music was composed by Leosidrin. I'm Westkasova. We'll

467
00:27:10,040 --> 00:27:11,920
Speaker 1: be back tomorrow with another Big tag