Speaker 1: Bloomberg Audio Studios, Podcasts, radio News.
Speaker 2: Hello and welcome to another episode of the Odd Lots podcast.
Speaker 3: I'm Joe Wisenthal and I'm Tracy Alway.
Speaker 2: Tracy. We did another one of our live shows, this
Speaker 2: time our biggest show ever, biggest show ever.
Speaker 4: It was absolutely amazing. We did it at City Winery
Speaker 4: in New York. I think we had over three hundred people.
Speaker 2: Yeah, three hundred and fifty people were there.
Speaker 3: Yeah.
Speaker 4: And the crazy thing is, I think it was our
Speaker 4: sort of first themed show, and we didn't really plan
Speaker 4: it that way, but it just worked out right.
Speaker 2: I guess like it themed an anti theme at the
Speaker 2: same time, because we're in this moment in which everything
Speaker 2: is just like AI markets, markets, AI, et cetera. But
Speaker 2: you know, there's all kinds of new things to trade,
Speaker 2: and people are fascinated by the trade itself, pro fascinated
Speaker 2: by the way the technology development is affecting the trade.
Speaker 2: So we really wanted to do a kind of future
Speaker 2: of trading show, which is a very broad thing. But
Speaker 2: it did sort of come out that way.
Speaker 4: Yeah, it really did. And you finally fulfilled your longtime
Speaker 4: dream of doing two part episodes with our guests. So
Speaker 4: our first speaker of the evening was actually someone who's
Speaker 4: been on the show before.
Speaker 2: That's right, So we had him on the show last year,
Speaker 2: and we had him on our live show. Listen to
Speaker 2: our episode with Ian Dunning, he is the head of
Speaker 2: AI at Hudson River Trading. Talked about all things implementing AI, GPUs,
Speaker 2: all that stuff within the context of a trading shop.
Speaker 3: Take a listen, Joe, this is.
Speaker 4: Your dream, right, you finally got to do a two parts.
Speaker 2: This is the thing I always think about, which is
Speaker 2: that after every episode we do, I'm like, Oh, there's
Speaker 2: a question I wish I had asked. So we had
Speaker 2: Yan on sometime last.
Speaker 4: Year, so the last round was easy, this one will
Speaker 4: be right.
Speaker 2: Worried about that, well, I was going to start first
Speaker 2: before we you know, talk about what you do, et cetera.
Speaker 2: So here's the question I wish I had asked last time. So,
Speaker 2: Hudson River Trading Shop, you're involved in the AI stuff,
Speaker 2: could you theoretically do what high Flyer did and launch
Speaker 2: an LLM with this tech stack that you have and
Speaker 2: launch a deep seat competitor?
Speaker 3: I think so. I think we're good at training models.
Speaker 3: We have a lot of compute and people are good
Speaker 3: at doing the cycle of research which is required to
Speaker 3: catch up to the sort of frontier. However, I guess
Speaker 3: reaching the frontier is clearly a very daunting task. So
Speaker 3: maybe it's with some effort deep seek, but beyond that,
Speaker 3: it's not a claim I'd be willing to make. It's
Speaker 3: a hugely capital intensive task, clearly.
Speaker 2: And do you ever, like, do people ever chat about that?
Speaker 3: It's we could we think about it?
Speaker 4: You know?
Speaker 3: I mean, I think perhaps we missed our moment to
Speaker 3: do so. It is so many open models now coming
Speaker 3: out from the US as well as like China, that
Speaker 3: there's like a huge array of them. It's kind of
Speaker 3: an interesting shift from that deep Seak moment where it
Speaker 3: felt like it was the first bolt from the blue.
Speaker 3: He's a competitive open model. So now I see so
Speaker 3: many groups releasing them. I don't know what the future
Speaker 3: of open models is. That they're all kind of a
Speaker 3: serious step back in. The frontier is progressing so fast.
Speaker 3: I don't know how you keep up with that, but
Speaker 3: many people believe that it's possible. I'm not so sure.
Speaker 3: I'm one of those people though.
Speaker 4: Okay, so, speaking of things moving so fast, my first
Speaker 4: question is slightly different. I looked up your Twitter feed.
Speaker 4: Oh no, before you came on the show. Your last
Speaker 4: tweet before today was and I quote feel this every day, worry.
Speaker 4: It's some sort of AI induce delirium. But then again,
Speaker 4: various empirical measures are exponential looking, so feels best to
Speaker 4: assume we're hurtling towards some sort of endgame. So first
Speaker 4: of all, please convince us all live on stage that
Speaker 4: you are in fact not suffering from AI induce delarium.
Speaker 4: But secondly, like, what is the endgame that you speak
Speaker 4: of here?
Speaker 3: God, now I saw like a San Francisco pressent I do. I'sist.
Speaker 3: I've been doing AI stuff since around twenty sixteen, and
Speaker 3: that stuted at Deep Mind, and it was a bit
Speaker 3: of a culture shock for me, because there are true
Speaker 3: believers then, and I was most certainly not a true believer,
Speaker 3: and I resisted it, and it was kind of a
Speaker 3: natural skeptic for a long time. But certain empirical measures
Speaker 3: of the pace of progress in the outside world. And
Speaker 3: I also look at our own business, which looks somewhat
Speaker 3: exponentially the amount of compute I'll have next year versus
Speaker 3: this year, and this year versus last year. Looks kind
Speaker 3: of exponentially, and we're doing things today that I didn't
Speaker 3: really I should have dreamt of. I wish I had
Speaker 3: that kind of visionary say, I'm a visionary and I
Speaker 3: can see the future and I'm building towards it. But no,
Speaker 3: I'm I think I'm a pragmatic engineering archetype, and so
Speaker 3: it's been very incremental and I'm like, wow, that happened
Speaker 3: in a year. So what does this mean? It's some
Speaker 3: sort of technological convergence, everything going faster all the time. Well,
Speaker 3: give us any I'm probably, but you give us an.
Speaker 2: Example then, because you know, obviously those of us using
Speaker 2: just the regular models, obviously the improvements and capability from
Speaker 2: one year to another our mind blowing. But from the
Speaker 2: perspective of like, okay, the application of AI within the
Speaker 2: trading context, what is something that you can do in
Speaker 2: twenty twenty six that and say twenty twenty four you
Speaker 2: would not have been able to anticipate.
Speaker 3: Oh, I think it's one way to think is just
Speaker 3: like the amount of compute going into both training a
Speaker 3: model and running a model, and that it's the same
Speaker 3: technology working across every equity, every future, every crypto market,
Speaker 3: every option market across the world with a kind of
Speaker 3: unified approach, and this is something that we're doing. But
Speaker 3: even more interestingly, I would not claim that we are
Speaker 3: some unique people who are only the ones who have
Speaker 3: really made progress in AI and trading. I think many
Speaker 3: of our peers are also investing massively and we're all
Speaker 3: doing it all at the same time. And what does
Speaker 3: that mean, Like, surely you can't just like keep getting
Speaker 3: better at predicting markets Furvace. It's got to be some
Speaker 3: sort of forcing function where you know your your margins
Speaker 3: go to zero as you keep investing.
Speaker 2: What you're saying is you are just getting better and
Speaker 2: better at being able to predict where a market is
Speaker 2: going to go further and further out in the.
Speaker 3: Time frame basically, And we're not the only ones. So
Speaker 3: in the end, can there be some highlander type thing
Speaker 3: like what are we doing? And it's it's this is
Speaker 3: like my scale, And I guess the other thing I
Speaker 3: find interesting is, of course the scale that everyone can
Speaker 3: see with the big labs and what they're doing of computeion.
Speaker 3: It's like it looks awfully exponential to me. We just
Speaker 3: had another model released today from Anthropic and the type
Speaker 3: of spacing between them seems to be compressing. I do
Speaker 3: sound hilarious. I sound feverish, and that's why.
Speaker 2: It's literally everyone in this room.
Speaker 3: Probably I feel everyone must feel the fever to some extent.
Speaker 4: Yeah, I never understood the Highlander there can only be
Speaker 4: one thing because they're already two. They could just coexist,
Speaker 4: that's right. Anyway, Sorry, I'm just picking apart your analogy.
Speaker 4: You mentioned a new model release. When a new model
Speaker 4: gets released, like, what is the first thing you do
Speaker 4: at Hudson River Trading to evaluate it? And how do
Speaker 4: you actually compare them to the existing one?
Speaker 3: So I mean our prime reuse cases as a trading
Speaker 3: and definitely kind of just like accelerating your own research.
Speaker 3: So that can be coding, but it can also be
Speaker 3: coming up with experiment ideas, monitoring experiments. We had a
Speaker 3: sort of a false start with AI I would say
Speaker 3: sometime last year with the Opus four point zero models,
Speaker 3: especially from Anthropic, where a cursory examination made us feel like, well,
Speaker 3: this is the moment we've crossed the dividing line, and
Speaker 3: we had a very feverish week where we felt the
Speaker 3: AGI and we left feeling empty because we realized that
Speaker 3: it was not there and was not able to meaningfully
Speaker 3: augment human researchers. And then we had that same feeling
Speaker 3: again when Opus four point five came out and suddenly
Speaker 3: it was like, oh, wait, no, this is this is
Speaker 3: actually what we thought it was going to be six
Speaker 3: months ago. So in the most recent model releases, the
Speaker 3: differences have been more subtle, but we see, I think
Speaker 3: we have a much better sense of an ever reducing
Speaker 3: set of errors they make, and so we're kind of
Speaker 3: looking for those those of mistakes, and we are We
Speaker 3: spent some time in the past couple of weeks trying
Speaker 3: to come up with objective measures to index them against
Speaker 3: humans in the active quant research ideating signals and things
Speaker 3: quant research used to be, as we talked about a
Speaker 3: little bit like hand crafting indicators and things, why not
Speaker 3: ask AI agents to do that and compare them against
Speaker 3: humans like a little sort of battle. And it's they're like,
Speaker 3: I don't know, intern level AI. Perhaps the thing is like,
Speaker 3: what do I think it'll be in a year? And
Speaker 3: I would not want to make a bold claim, but
Speaker 3: it will still be.
Speaker 2: It'll be wild. So when we think about investing in general,
Speaker 2: even within sort of like classical quant trade, and going
Speaker 2: back decades, there is often it might be quant but
Speaker 2: there's some intuition behind it. Right, Cheap stocks tend to
Speaker 2: do better, and we don't actually totally have agreement why
Speaker 2: they did for a while, but people aren't necessarily surprised
Speaker 2: by that fact, right are we at the point where
Speaker 2: it's like why even bother coming up with the human
Speaker 2: intuitive story and you just skip the part of giving
Speaker 2: an explanation that's owns logical to a person and it's
Speaker 2: just basically pure like rigorous back testing. And then it's like, look,
Speaker 2: here is something that seems to work, and we've back
Speaker 2: tested it a million different ways and it seems to work,
Speaker 2: and we don't even bother coming up with a story
Speaker 2: for why, but we're going to trade it.
Speaker 3: I feel like we're in that world today. It's seat
Speaker 3: of post post post capitalism. When I see IPOs discussed
Speaker 3: for this coming summer, at the valuations they are, I'm like,
Speaker 3: what is a fundamental? Like what is anything? It feels
Speaker 3: like markets are just the cynical thick is everything is gambling,
Speaker 3: and so everything is some sort of like gambling market,
Speaker 3: including public markets. But the joke is flows, it's buying
Speaker 3: and selling, and it's just it's worth what it's worth,
Speaker 3: and it's detached and more biased, and sellers price go up,
Speaker 3: and models are excellent at like pulling that out of data.
Speaker 2: But just like let's say, you know the classic example
Speaker 2: of like a back test, it is like, oh, companies
Speaker 2: with the ticker symbol it starts with P, they do
Speaker 2: well on Tuesdays. And it's like, well, look the data
Speaker 2: says that, but this makes no sense. We're not going
Speaker 2: to trade that. Could it get to the point where
Speaker 2: it's like, look, symbols that starts with P, you do
Speaker 2: well on Tuesdays, And we've run this a bunch of
Speaker 2: times and it seems to work, so we're gonna put
Speaker 2: money beyond us.
Speaker 4: AI Like I.
Speaker 3: Feel like, yes, although it sounds crazy, it sounds like
Speaker 3: AI delirium when I say it, but I feel like
Speaker 3: there's some sense that that could be true at some
Speaker 3: point I can't predict. So at the very short time scale,
Speaker 3: people accept this already, right Like I can't tell you
Speaker 3: the price of like a stock in a minute, and
Speaker 3: no one would really reasonably expect any human to do so,
Speaker 3: even if they had the uder book and spend all
Speaker 3: the time in the world is staring at it. But
Speaker 3: we accept that neural networks can do this, And then
Speaker 3: when does that logic break down? Why should it break
Speaker 3: down at some long time scale. If it's ingesting all
Speaker 3: the data and has everything and it can keep it
Speaker 3: all in the context in way human can't, why should
Speaker 3: I be able to understand it? Yeah, and that is
Speaker 3: a strange thought, a loss of control. It feels like
Speaker 3: a loss of control. But it's like, you know, people
Speaker 3: save us from math. Maybe humans are actually very bad
Speaker 3: at math. So it's not surprising AI as much better
Speaker 3: than humans that these like mass proofs. Humans probably would
Speaker 3: be pretty bad at markets where thousands of tradable instruments
Speaker 3: on like very long time scales. We just kind of
Speaker 3: accepted that we were some people were good at this.
Speaker 3: Maybe that's a temporary state of affairs.
Speaker 4: Well, we talked about this the last time. You were
Speaker 4: on the idea that the models themselves are not very interpretable,
Speaker 4: I guess you would say, but you're comfortable with that
Speaker 4: on a short trading timeframe, which is what you do.
Speaker 4: And then we started joking about magic models, and magic
Speaker 4: is a dangerous word to use on this podcast because
Speaker 4: people start thinking about magic boxes. But anyway, now that
Speaker 4: you've been doing this for another six months since we
Speaker 4: last spoke to you, do you feel like you have
Speaker 4: better insight into what the models are actually doing and
Speaker 4: why they're able to succeed on short timeframes.
Speaker 3: I do think there are diagnostics we have done where
Speaker 3: when we can see things that we can understand. It's
Speaker 3: like looking at some very very complex thing and you
Speaker 3: can look at one facet of it and be like,
Speaker 3: this is the fast that I understand, and that gives
Speaker 3: you some confidence, but it might be illusory because it's
Speaker 3: a very very complex object, and you can if you're
Speaker 3: only taking slices through it and to understand aspects of it.
Speaker 3: You know, we had this emergent phenomenon we saw where
Speaker 3: it felt like the model kind of understood meme stocks
Speaker 3: from first principles like quantum stocks and crypto stocks being
Speaker 3: kind of adjacent in stock space, and of course from
Speaker 3: a fundamental perspective that says, no, there's no meaning to it.
Speaker 3: But we looked at the model and us under a
Speaker 3: certain lens and it clearly felt like they knew they
Speaker 3: were connected. There's some other actual companies that I probably
Speaker 3: won't name. It feels like it's bad form, but you
Speaker 3: know Wall Street BET's favorites, I guess, and they were
Speaker 3: near the cluster too, And this was like just one
Speaker 3: little window. But there were other slices we tried to
Speaker 3: take which just didn't make sense to us. But again
Speaker 3: it's like, who am I to say?
Speaker 2: The model says they're in that vicinity of hyperdimensional space.
Speaker 3: Yeah. The one thing for us though, is that when
Speaker 3: we do have this magical model, it is in a
Speaker 3: lot of safety around it. Because we're doing this higher
Speaker 3: frequency trading. We're trading positions back and forth. There's a
Speaker 3: lot of risk checks that are fully automated in things.
Speaker 3: I don't know how you generalize this logic to long
Speaker 3: term discretionary trading, where the idea of like risk checking
Speaker 3: and that kind of layer of defense around it, it's
Speaker 3: not so obvious to me how you apply that. We
Speaker 3: can apply very strict controls around this model because it's
Speaker 3: a well posed problem. We're not taking giant idiosyncratic risks
Speaker 3: in like one name for months at a time. We
Speaker 3: can sleep at night because of this. I don't know
Speaker 3: how you apply the same thinking to like a fundamental long,
Speaker 3: short thing where you have to put a trade on
Speaker 3: and it's for three months, and you're intentionally taking a
Speaker 3: very large risk in a very sudden direction. That's what's
Speaker 3: the risk management story around the AI if you just
Speaker 3: give up all control to just the magic prediction.
Speaker 2: So you said something on the last time we interviewed you,
Speaker 2: which is very important. First of all, I feel like
Speaker 2: in the quote AI trade unquote, people are obsessed with like,
Speaker 2: what's the bottleneck now, right? And because whatever the bottleneck is,
Speaker 2: you probably solved for a lot more money. You said
Speaker 2: the last time we talked to you, the chips themselves
Speaker 2: are We're not actually a major constraint for you, and
Speaker 2: that it was more like citing the chips and the
Speaker 2: powering the chips, the access to electricity, talk about that.
Speaker 2: What is the state right now? Let's say like I
Speaker 2: assemble approach a bunch of people from Hudson River Trading. Yeah,
Speaker 2: I get a bunch of GPUs. Is it then not
Speaker 2: trivial to find a place to plug those in.
Speaker 3: It's definitely hot to find sites and at shortly times.
Speaker 3: If I went to the market and said I want,
Speaker 3: you know, six thousand Blackwell GPUs in a box somewhere
Speaker 3: in North America for delivery in Q four. I'm not
Speaker 3: sure such an offering exists at any reasonable price. Like
Speaker 3: if it from maybe someone will give up a lease
Speaker 3: and I could snag it. But I think if I
Speaker 3: went to the market and tried to get a.
Speaker 2: Quia sorry just to be clear, the chips are available,
Speaker 2: but not the competity.
Speaker 3: I think if I had power, I could get the
Speaker 3: chips Blackwell chips for delivery this yet, but I do
Speaker 3: not think I could get the whole solution. And then
Speaker 3: if you go into twenty twenty seven for the next
Speaker 3: generation of GPUs, the Ruben GPUs, they at least for
Speaker 3: the first like stretch, are going to be very much
Speaker 3: assault out. And so I think that's like a maybe
Speaker 3: you actually have on a twenty twenty seven delivery. You
Speaker 3: have more like finding a data center shell by then.
Speaker 3: But you need you need to be in cute now
Speaker 3: for those GPUs if you want them early. So those
Speaker 3: things are those things are in demand. I'll say that
Speaker 3: for sure. And one of my greatest failures has been
Speaker 3: you know, part of my skepticism has been predicting how
Speaker 3: many GPUs we would need and a long enough horizon
Speaker 3: and it's punishing because you're constantly playing catchup. And one
Speaker 3: of our competitors put out a podcast this weekend and
Speaker 3: they mentioned somebody along the lines of the fact they
Speaker 3: had one data center and it was the data center,
Speaker 3: and that was their data center, and then as they're hungering,
Speaker 3: hungry for more compute, they had to go out and
Speaker 3: find it wherever they could. And I would say, we
Speaker 3: are in exactly the same boat. You just can't be picky.
Speaker 3: It's like you've got like a mega watt there, I'll
Speaker 3: take it, and it could be you know, not in
Speaker 3: terms that are super favorable to you.
Speaker 4: Because we'll say more about that. How are you actually
Speaker 4: going out and sourcing this stuff, because as you say,
Speaker 4: it seems to be exceptionally competitive. And at the same time,
Speaker 4: don't you guys have an insane data center in like
Speaker 4: Norway or something.
Speaker 3: And it's not enough. Yeah, and it's not enough. Yes.
Speaker 3: We go to the neo clouds, the hyperscalers, everyone, and
Speaker 3: it's a constant dialogue and they're all in competition with
Speaker 3: each other. But in some sense there must be some
Speaker 3: much bigger, shadowy competition going on behind the scenes behind
Speaker 3: these neoclouds because they are all looking for space and
Speaker 3: power and I don't know if that's the true scarce resource,
Speaker 3: and they're a kind of intermediory layer over it. I
Speaker 3: don't know what their process is like for saucing it.
Speaker 3: But yeah, they have come to us and said, this
Speaker 3: lease opened up. Can you please get back to us
Speaker 3: by the end of the day. And for commitments on
Speaker 3: a long term contract, and our contracts are a long term.
Speaker 3: This is not spot compute. This is like eight thousand
Speaker 3: GPUs for three years, four years, five years payment? Do
Speaker 3: you wanna pay half upfront? Do you want to pay
Speaker 3: some per year? A lot of different commercial terms, credit
Speaker 3: risk on both sides. It's complicated stuff.
Speaker 2: Tell us more about the counterparty risk. So it's like
Speaker 2: you come and you say you want capacity and some
Speaker 2: data center. I'm from Hudson River Trading. Who yeah, well,
Speaker 2: this is the kind of thing like this crowd. A
Speaker 2: lot of people know what Hudgson River Trading is, but
Speaker 2: maybe in San Francisco or whatever, that's not a household name,
Speaker 2: et cetera. They want to know for sure that you're
Speaker 2: gonna be good, You're gonna like pay your bills, et cetera.
Speaker 2: How do you establish to the data center that you
Speaker 2: were going to be a reliable I guess tenant.
Speaker 3: Yeah, it's a it's a definitely being a dance. It's
Speaker 3: getting better at this point. I think we've anted it
Speaker 3: enough deals enough people that I think we have that,
Speaker 3: But we've had everything from people being like, oh, you've
Speaker 3: issued bonds, what's the rating on those too? Not wanting
Speaker 3: us to sell too much of one site because if
Speaker 3: we take all their power rights and then go bust,
Speaker 3: they might have a long lead time where they can't
Speaker 3: get into a tenant and fill that. And so there's
Speaker 3: a kind of two potty problems of us where it's
Speaker 3: like they want customers, but there's presumably a lot of customers,
Speaker 3: but maybe not as many customers are willing to do
Speaker 3: the big size and pay more upfront. But you know,
Speaker 3: we're looking at their CDs is on some of these
Speaker 3: ones and thinking about how that affects our You know,
Speaker 3: maybe we should pay you three fifty an hour and
Speaker 3: take out a CDs for ten cents for our equivalent
Speaker 3: of insurance on your heavy leveraged neocloud. You're having a disruption,
Speaker 3: no names, but you know it's I think is reason
Speaker 3: to be cagy on both sides because this has all
Speaker 3: come from nothing like a year ago. We weren't there
Speaker 3: asking for it, and they didn't exist to sell it,
Speaker 3: and so the only rock is Nvidio I guess, an
Speaker 3: extremely well capitalized entity who is not going anywhere, and
Speaker 3: it's making love GPUs and we have a very positive
Speaker 3: relationship with them. I think that is also a material factor.
Speaker 4: How much optionality do you actually have on GPUs now,
Speaker 4: Like if you say you want to prioritize latency or throughput,
Speaker 4: like can you get the chips that you need to
Speaker 4: specify on one of those things? Or like do you
Speaker 4: just take what you can get.
Speaker 3: Or build your own? Yeah you can? You can. Well,
Speaker 3: many people I guess now are working on building their
Speaker 3: own chips for for inference, which is as strictly similar
Speaker 3: technological problem, and ourselves and many of our peer trading
Speaker 3: firms have hardware teams to tackle this and you can
Speaker 3: outsource pots of it process, so it's not as daunting
Speaker 3: as it seems. But it's definitely an active, very investigation
Speaker 3: for us and now clearly everyone because I feel like
Speaker 3: everyone's talking about their partnership with Broadcom or something like this.
Speaker 3: And if someone says partnering your broadcom, it's like that
Speaker 3: they're making an inference chip.
Speaker 2: So it's interesting, right because you hear about like Amazon,
Speaker 2: like them Google tp So we could be in a
Speaker 2: world in which we hear of like a Hudson River
Speaker 2: Trading branded chipise.
Speaker 3: I don't think we'll sell it, but yeah, but yes,
Speaker 3: you're right, that is that is definitely the right model.
Speaker 3: And on the other hand, Jensen Never sleeps and Jensen
Speaker 3: purchased Grock and you know they have got their new
Speaker 3: product lineup from the groc acquisition, which is a very
Speaker 3: compelling product as well. And there are others setups etched
Speaker 3: comes to mind. So the inference space is a smaller
Speaker 3: design space. It's not clear that inhouse solutions will be
Speaker 3: a necessary thing in the future if physic enough people competing.
Speaker 3: But on the training side, I mean, what a mote like,
Speaker 3: there's just a video, it's just and I suppose Google.
Speaker 3: But you know, if you're using TPUs, you're also kind
Speaker 3: of entering a very close relationship with Google, some feeling
Speaker 3: of vandor lockin. It's a complicated thing if you go
Speaker 3: down that past. But if you're compute hungry, like the
Speaker 3: neo labs. I mean, obviously the big labs. You'll take
Speaker 3: what you can get. I think Anthropic takes TPUs, trainiums
Speaker 3: and GPUs. You know they need them all.
Speaker 2: So maybe we'll create a little bit of con controversy
Speaker 2: here because later on in a little bit we're going
Speaker 2: to be speaking with Carmen Lee, the CEU of Compute Exchange,
Speaker 2: which is this, you know, one of the multiple entities
Speaker 2: are trying to build financial markets for compute capacity right
Speaker 2: trade it like oil, so it's like compute futures and
Speaker 2: stuff that right now, could you see a use for
Speaker 2: that of financial instrument that's like on some liquid tradable
Speaker 2: exchange for H one hundred or whatever, some benchmark of
Speaker 2: how much it costs run these chips. Could you see
Speaker 2: that being a useful instrument for you at some point?
Speaker 3: It's it's plausible. It highly relates back to my previously
Speaker 3: stated failure to plan correctly for the future. If in
Speaker 3: some sense I could lock in a price for some
Speaker 3: future date for delivery or something of compute, something that
Speaker 3: is connected to for prices compute in the long term future,
Speaker 3: I think that could be value to that We could
Speaker 3: basically hedge our risk that we wait too long to
Speaker 3: put the order in and it price goes up. I mean,
Speaker 3: in twenty twenty six, the priceive memory has gone up
Speaker 3: so much that we do have concrete specific things. I
Speaker 3: wish I put that order in a month earlier, so
Speaker 3: it's a real real thing. Do I believe that there
Speaker 3: would be a good market with left liquidity for long
Speaker 3: dated compute futures? That I guess remains to be seen.
Speaker 3: I don't know what I would do with a short
Speaker 3: dated compute future. I do think defining what compute is
Speaker 3: is pretty hard, and I have no idea what physical
Speaker 3: delivery would be if that is indeed of interest because
Speaker 3: you know, because of a long term contract and because
Speaker 3: of how much work goes into every site, Like when
Speaker 3: we connect to a neocloud site, we're thinking about how
Speaker 3: to connect it back to our other sites. Everyone's got
Speaker 3: a different networking system, the file system, Like you know,
Speaker 3: visit of a GPUs, which was all the focus, but
Speaker 3: this's also like you know, how is data stored of
Speaker 3: a site or is it stored of itt site at all?
Speaker 3: Is there an adjacent site that all the hard drives
Speaker 3: are in and they're all idiosyncratic, and I can't do
Speaker 3: anything of one hundred and twenty eight GPUs. I need
Speaker 3: thousands of GPUs or bust. That's like my lot size,
Speaker 3: And so it's very hot to see how you could
Speaker 3: kind of break that down into useful units. But maybe
Speaker 3: it's just a spot thing and if it's long dated,
Speaker 3: I don't know, they could be.
Speaker 2: We'll learn more.
Speaker 4: Yeah, I did get a preview and it is pretty cool,
Speaker 4: like the actual program where you can select like the
Speaker 4: type of compute you need from a specific data center
Speaker 4: that has like literally I think dozens, if not maybe
Speaker 4: hundreds of parameters at the moment. So maybe we can
Speaker 4: get a demonstration from Carmen. What is your token spend
Speaker 4: at the moment? Is it bigger than Joe's?
Speaker 1: I have?
Speaker 3: I think I what is my average? I think it's
Speaker 3: on the order of one hundred two hundred dollars a
Speaker 3: day per employee's on my team. I feel like that's
Speaker 3: kind of what I've been seeing lately, and some people
Speaker 3: are more in a thousand a day range, burst bit
Speaker 3: bursty for that.
Speaker 4: Wait, do you like those people because they're supposedly more predictive?
Speaker 3: Definitely not trying to encourage that. I mean, some people
Speaker 3: go through sturges of experimentation slash Ai delarium, which is understandable.
Speaker 3: And I think we are always trying to understand the
Speaker 3: people who are using more like are they doing it
Speaker 3: for something that you even't figured out yet? That's a
Speaker 3: pretty profound new expense to have. It's it's it's not
Speaker 3: at the level that concerns us. Well, it didn't exist
Speaker 3: at all as an expense type. So that's kind of
Speaker 3: interesting to think about.
Speaker 2: Well, I'm curious, like you know, for the consumer models,
Speaker 2: they talk about how psychophantic they are. Does that happen?
Speaker 2: It's like, yes, you're close. This is really smart. You're
Speaker 2: close to cracking the code of the market. Keep pursuing this.
Speaker 2: They just one more this idea is doing or is
Speaker 2: it Claude likes to say, this is doing some real
Speaker 2: work here in this argument.
Speaker 3: It's really good.
Speaker 2: Do you get that in the engineering content?
Speaker 3: I think we do. And it's interesting. We have we
Speaker 3: decided a new internship for the summer, and uh, in
Speaker 3: previous internships, we noticed that, you know, it's quite daunting
Speaker 3: coming into this quent training context. You know, you have
Speaker 3: no there's not much to like read a book, can't
Speaker 3: read a tetbook about it. It's useful, So people ask AI,
Speaker 3: and uh, you know, it would always mention some things
Speaker 3: of like an unusual frequency that maybe an expert in
Speaker 3: netfield wouldn't like focus us on some things. And we
Speaker 3: noticed in our like winter internship program a lot of
Speaker 3: sort of very technical quant finance research terms being mentioned
Speaker 3: a lot by the interns that no full time are used.
Speaker 3: And it's like the original seed of the mind virus
Speaker 3: is AI. So there's a little stuff like that. But
Speaker 3: our token spending is going to go up, but I
Speaker 3: mean that's almost guaranteed, and yeah, we're getting value out
Speaker 3: of it. Maybe not two x productivity, but I talked
Speaker 3: to someone who said that team's fifty percent more productive.
Speaker 3: That's pretty good. I mean, you'd definitely pay one hundred
Speaker 3: dollars a day for that. I just don't understand how
Speaker 3: people who are token poor could keep up with someone
Speaker 3: who's token rich. And that's a gain goes to the
Speaker 3: acceleration feeling. It's like if you have two people who
Speaker 3: are sort of equally resourceful and smart, but someone who's
Speaker 3: basically a co pilot with them, that's giving them a
Speaker 3: fifty percent boost and all they have to do to
Speaker 3: get that is essentially spend money. It creates a have
Speaker 3: have no dynamic that possibly compounds. As you have more success,
Speaker 3: you make more money, you're more willing to eat now
Speaker 3: a thousand dollars a day per pres and for token
Speaker 3: spend you go even faster. And this feeling again of
Speaker 3: compounding acceleration, which might be delirium, but you could make
Speaker 3: an argument for why it could be a real effect
Speaker 3: instead of more winner. Take all contexts where speed of
Speaker 3: improvement is like the key thing. There's a story there,
Speaker 3: I think, or delirium matter well.
Speaker 4: I mean, speaking of the haves and have nots. The
Speaker 4: other big story in AI world is just competition for talent, right,
Speaker 4: and everyone is sort of chasing the same genius engineer.
Speaker 4: I guess how are you finding that At.
Speaker 3: The moment, it's changed a bit. There's a lot of
Speaker 3: dynamics going on. There's still a feeling that if you
Speaker 3: are plucky enough, you can get a VC to fund
Speaker 3: your idea based on very little. You have the right pedigree,
Speaker 3: and that's always been true. I guess some sense this
Speaker 3: is like the YC philosophy. Some sense you go and
Speaker 3: it's just that some of the numbers and the foamo
Speaker 3: feeling and is quite shocking. And so that's actually a
Speaker 3: form of competition, just like why didn't I go create
Speaker 3: a stetup. I don't have any ideas or anything. I'm
Speaker 3: just going to make a startup for the big labs.
Speaker 3: A question of upside remaining upside. You're at a trillian now,
Speaker 3: I guess there are two big ones that are a
Speaker 3: trillion dollar evaluation. Where do you go from there? I
Speaker 3: think that's affecting people's level of like forward looking optimism
Speaker 3: for people who are taking offers now and for people
Speaker 3: who are at those places and looking to leave. Generally,
Speaker 3: it's a question of like, well, they've become big tech,
Speaker 3: They've added people at a vast rate, and the culture
Speaker 3: has shifted, especially at some of the labs a lot,
Speaker 3: and to our favor. For a while, it did feel
Speaker 3: like we're in a very, very fierce competition. And now
Speaker 3: maybe it's now it's maybe more even playing field. But
Speaker 3: I don't know. I talk to a lot of undergrads
Speaker 3: and they don't feel great about the future. They feel
Speaker 3: very worried.
Speaker 2: Basically, what I was gonna just went by way too fast.
Speaker 2: But like you mentioned already, the models are like okay,
Speaker 2: junior level, Yeah, what does talent look like at this point?
Speaker 2: And what are like I've seen some of the anthropic
Speaker 2: interview questions and it's like designing some GPU kernel or
Speaker 2: like optimizing the configuration of GPS within the data center?
Speaker 2: Did what do you want someone to bring to the
Speaker 2: table at this point?
Speaker 3: I mean, I think the first thing is just trying
Speaker 3: to embrace an open book philosophy, like let the interviews
Speaker 3: be done with the aid of AI is something we're
Speaker 3: trying to aspire to do because it's just at some
Speaker 3: point you become it becomes unrealistic to pretend anyone would
Speaker 3: work without that. One of the big things in quant
Speaker 3: is being like this is like archetype of like the
Speaker 3: math theorist or the string theorist or something and they
Speaker 3: go in to Long Island somewhere and they come out
Speaker 3: with alpha. But you know, like our experience has been
Speaker 3: a little bit more mixed because it's like, if you
Speaker 3: can't implement your ideas, how do you how does that happen? Exactly? Well,
Speaker 3: now Claude can presumably implement the ideas, So trying to
Speaker 3: embrace that maybe we do accept more theorists, more dreamers,
Speaker 3: people who can come up with ideas, trusting that the
Speaker 3: implementation work can be done by AI. So I think
Speaker 3: that's our shift. But I've been joking. It's like the
Speaker 3: word cell versus shape rotator type, Like I feel like
Speaker 3: the error of the word cell may be a bonus,
Speaker 3: like if I did prompt. Yeah, I mean prompt engineering
Speaker 3: is kind of a boomer term at this point, but
Speaker 3: there is something to be said for like describing what
Speaker 3: you want clearly and without confounding factors, and that is
Speaker 3: a skill that can be learned, and that's not evenly
Speaker 3: distributed in the population. And I would argue that as
Speaker 3: shot up in value simply because of AI. So I
Speaker 3: like to think of myself as one of these people though,
Speaker 3: So that could be the Delarium'm talking.
Speaker 2: I don't know, all right, Ian Dunning, we could talk
Speaker 2: for two more hours. Thank you so much for joining
Speaker 2: us at.
Speaker 4: That was our conversation with Ian Dunning of Hudson River Trading,
Speaker 4: recorded live at our New York show. I'm Tracy Alloway.
Speaker 4: You can follow me at Tracy Alloway.
Speaker 2: And I'm Joe Wisenthal. You can follow me at the Stalwart.
Speaker 2: Follow Ian at Ian Dunning. Follow our producers Carmen Rodriguez
Speaker 2: at Carmen Arman, Dashil Bennett at Dashbod, Calebrooks at Kalebrooks
Speaker 2: and Kevin Lozano at Kevin Lloyd Lozano. And for more
Speaker 2: odd Laws content, go to Bloomberg dot com slash odd Lots.
Speaker 2: We have a daily newsletter and all of our episodes,
Speaker 2: and you can chat about all these topics twenty four
Speaker 2: to seven in our discord Discord dot gig slash odd Lots.
Speaker 4: And if you enjoy odd Lots, if you like it
Speaker 4: when we do these live shows and talk about how
Speaker 4: trading firms are actually using AI, then please leave us
Speaker 4: a positive review on your favorite podcast platform. And remember,
Speaker 4: if you are a Bloomberg subscriber, you can listen to
Speaker 4: all of our episodes absolutely ad free. All you need
Speaker 4: to do is find the Bloomberg channel on Apple Podcasts
Speaker 4: and follow the instructions there. Thanks for listening. In behead