Speaker 1: This is Master's in Business with Barry Ridholds on Bloomberg Radio.
Speaker 2: This week on the podcast Strap Yourself In, I have
Speaker 2: another extra special guest. John mccauliffe is co founder and
Speaker 2: chief investment officer at the Volleyon Group. They're a five
Speaker 2: billion dollar hedge funds and one of the earliest shops
Speaker 2: to ever use machine learning as it applies to training
Speaker 2: and investment management decisions. It is a full systematic approach
Speaker 2: to using computer horsepower and database and machine learning and
Speaker 2: their own predictive engine to make investments and trades, and
Speaker 2: it's managed to put together quite a track record. Previously,
Speaker 2: John was at d SHAW where he ran statistical arbitrage.
Speaker 2: He is one of the people who worked on the
Speaker 2: Amazon recommendation engine, and he is currently a professor of
Speaker 2: statistics at Berkeley. I don't even know where to begin,
Speaker 2: other than say, if you're interested in AI or machine
Speaker 2: learning or quantitative strategies, this is just a masterclass in
Speaker 2: how it's done by one of the first people in
Speaker 2: the space to not only do this sort of machine
Speaker 2: learning and apply it to investing, but one of the best.
Speaker 2: I think this is a fascinating conversation and I believe
Speaker 2: you will find it to be so. Also, with no
Speaker 2: further ado, my discussion with volleyon groups. John mccauliffe. John mccauliff,
Speaker 2: Welcome to Bloomberg.
Speaker 1: Thanks, Barry. I'm really happy to be here.
Speaker 2: So let's talk a little bit about your academic background. First,
Speaker 2: you start out undergrad computer science and applied mathematics at
Speaker 2: Harvard before you go on to get a PhD from
Speaker 2: California Berkeley. What led to a career in data analysis?
Speaker 2: How early did you know that's what you wanted to do?
Speaker 1: Well, it was a winding path. Actually, I was very
Speaker 1: interested in international relations and foreign languages when I was
Speaker 1: finishing high school. In fact, I spent the last year
Speaker 1: of high school as an exchange student in Germany. And
Speaker 1: so when I got to college, I was expecting to
Speaker 1: major in government and go on to maybe work in
Speaker 1: the foreign service something like that.
Speaker 2: Really, so this is a big shift from your original expectations.
Speaker 1: Yeah, it took about one semester for me to realize
Speaker 1: that none of the questions that were being asked in
Speaker 1: my classes had definitive and correct answers.
Speaker 2: Did that frustrate you?
Speaker 1: It did frustrate me. Yeah, And so I stayed home
Speaker 1: over winter I stayed Excuse me, I didn't go home.
Speaker 1: I stayed at college over winter break to try to
Speaker 1: sort out what the heck I was going to do,
Speaker 1: because I could see that it wasn't My plan was
Speaker 1: in disarray. And I'd always been interested in computers, had
Speaker 1: played around with computers, never done anything very serious, but
Speaker 1: I thought I might as well give it a shot,
Speaker 1: and so in the spring semester I took my first
Speaker 1: computer science course. And when you write software, everything has
Speaker 1: a right answer. It either does what you wanted to
Speaker 1: do or.
Speaker 2: It doesn't, does not compile exactly. So that's really quite
Speaker 2: quite fascinating. So what led you from Berkeley to d
Speaker 2: sure that they're one of the first quand shops. How
Speaker 2: did you get there? What sort of research did Yeah?
Speaker 1: I actually I spent time at d Shot in between
Speaker 1: my undergrad and my PhD program, So it was after
Speaker 1: Harvard that I went to that show.
Speaker 2: Did that light an interest in using machine learning and
Speaker 2: computers applied to finance or what was that experience like?
Speaker 1: Yeah, it made me really interested in and excited about
Speaker 1: using statistical thinking and data analysis to sort of understand
Speaker 1: then amics of securities prices. Machine learning did not play
Speaker 1: really a role at that time, I think, not at
Speaker 1: d SHAW, but you know, probably nowhere it was too
Speaker 1: immature a feel in the nineties. But I had already
Speaker 1: been curious and interested in using these kinds of statistical
Speaker 1: tools in trading and in investing when I was finishing
Speaker 1: college and then at d SHAW. You know, I had
Speaker 1: brilliant colleagues and we were working on hard problems. So
Speaker 1: I really, I really got a lot.
Speaker 2: Of us still one of the top performing hedge funds,
Speaker 2: one of the earliest quant hedge funds, A great a
Speaker 2: great place to absolutely cut your teeth at. So was
Speaker 2: it Harvard d SHAW and then Berkeley?
Speaker 1: Yeah, that's right?
Speaker 2: And then from Berkeley? How did you end up at Amazon?
Speaker 1: I guess I should correct myself. There was a year
Speaker 1: at Amazon after d Eshaw, but before Berkeley.
Speaker 2: And am I reading this correctly? The recommendation engine that
Speaker 2: Amazon uses you helped develop?
Speaker 1: I would say I worked on it. I would you
Speaker 1: know it existed, It was in place when I got there,
Speaker 1: and sort of the things that are familiar about the
Speaker 1: recommendation engine had already been built by my manager and
Speaker 1: his colleagues. But I worked I did research on improvements
Speaker 1: and different ways of forming recommendations. It was funny because
Speaker 1: at the time, the entire database of purchase history for
Speaker 1: all of Amazon fit in one twenty gigabyte file on
Speaker 1: a disc, so I could just load it on my
Speaker 1: computer and run.
Speaker 2: Now I don't think we could do that anymore. We
Speaker 2: could not, so, thank goodness is Amica Zone cloud services,
Speaker 2: so you could put what is it, twenty five years
Speaker 2: and hundreds of billions of dollars of transactions. So my
Speaker 2: assumption is products like that are highly iterative. The first
Speaker 2: version is all right, it does a half decent job,
Speaker 2: and then it gets better, and then it starts to
Speaker 2: get almost spookily good. It's like, oh, how much of
Speaker 2: that is just the size of the database, and how
Speaker 2: much of that is just a clever algorithm.
Speaker 1: Well, that's a great question, because the two are inextricably linked.
Speaker 1: The way that you make algorithms great is by making
Speaker 1: them more powerful, more expressive, able to describe lots of
Speaker 1: different kinds of patterns and relationships. But those kinds of
Speaker 1: approaches need huge amounts of data in order to correctly
Speaker 1: sort out what's signal and what's noise. The more expressive.
Speaker 1: A tool like that is like a recommender system, the
Speaker 1: more prone it is to mistake one time noise for
Speaker 1: persistent signal, and that is a recurring theme in statistical prediction.
Speaker 1: It is really the central problem in statistical predictions. So
Speaker 1: you have it in recommender systems, you have it in
Speaker 1: predicting price action, in the problems that we solve, and elsewhere.
Speaker 2: There was a pretty infamous New York Times article a
Speaker 2: couple of years ago about targets using their own recommender
Speaker 2: system and sending out maternity things to people. A dad
Speaker 2: gets his young teenage daughters what is this and goes
Speaker 2: in to yell at them, and turns out she was
Speaker 2: pregnant and they had pieced it together. How far of
Speaker 2: a leap is it from these systems to much more
Speaker 2: sophisticated machine learning and even large language models.
Speaker 1: It's the answer, it turns out, is that it's a
Speaker 1: question of scale. That wasn't at all obvious before GPT
Speaker 1: three and chat GPT, But it just turned out that
Speaker 1: when you have, for example, GPT is built from a
Speaker 1: database of sentences in English, it's got a trillion words
Speaker 1: in it that database, and when you take a trillion
Speaker 1: words and you use it to fit a model that
Speaker 1: has one hundred and seventy five billion parameters. There is
Speaker 1: apparently a kind of transition where things become, you know,
Speaker 1: frankly astounding. I don't I think, I don't think that
Speaker 1: anybody who isn't astounded is telling the truth.
Speaker 2: Right. It's eerie is in terms of how sophisticated it is,
Speaker 2: but it's also kind of surprising in terms of I
Speaker 2: guess what the programs like to call hallucinations. I guess
Speaker 2: if you're using the Internet as your base model, Hey,
Speaker 2: there's one or two things on the Internet that are wrong,
Speaker 2: so of course that's going to show up in something
Speaker 2: like chap GPT.
Speaker 1: Yeah, you know. Underlyingly, there's this tool GPT three that's
Speaker 1: really the engine that powers jed GPT, and that tool
Speaker 1: it has one goal. It's a simple goal. You show
Speaker 1: at the beginning of a sentence, and it predicts the
Speaker 1: next word in the sentence, and that's all it is
Speaker 1: trained to do. I mean, it really is actually that simple.
Speaker 2: It's a dumb program that looks smart.
Speaker 1: If you like. But the thing about predicting the next
Speaker 1: word in a sentence is whether you know the sequence
Speaker 1: of words that's being output, is leading to something that
Speaker 1: is true or false, is irrelevant. The only thing that
Speaker 1: it is trained to do is make highly accurate predictions
Speaker 1: of next words.
Speaker 2: So when I said it's really very sophisticated, it just
Speaker 2: for what we tend to call this artificial intelligence. But
Speaker 2: I've read a number of people said, hey, this really
Speaker 2: is an AI. This is something a little more rudimentary.
Speaker 1: Yeah, I think, you know, a critic would say that
Speaker 1: artificial intelligence is a complete misnomer. There's sort of nothing
Speaker 1: remotely intelligent in the colloquial sense about these systems. And
Speaker 1: then a common defense in AI research is that artificial
Speaker 1: intelligence is a moving target. As soon as you build
Speaker 1: a system that does something quasi magical that was the
Speaker 1: old yardstick of intelligence, then the goalposts get moved by
Speaker 1: the people who are supplying the evaluations. And I guess
Speaker 1: I would sit somewhere in between. I think the language
Speaker 1: is unfortunate because it's so easily misconstrued. I wouldn't call
Speaker 1: the system dumb, and I wouldn't call it smart. It's
Speaker 1: you know, those are those are not characteristics of these systems.
Speaker 2: But it's complex and sophisticated.
Speaker 1: It certainly is it has one hundred and seventy five
Speaker 1: billion parameters. That doesn't fit your definition of complex you
Speaker 1: know what would?
Speaker 2: Yeah, that works for me. So your in your career line,
Speaker 2: where is aphametrics and what was that recommendation engine? Like?
Speaker 1: Yeah, So that was work I did as a summer
Speaker 1: research intern during my PhD. And that work was about
Speaker 1: what's called the problem is called genotype calling. So genotype calling,
Speaker 1: I'll explain, Barry, do you have an identical twin? I
Speaker 1: do not, Okay, So I can safely say your genome
Speaker 1: is unique in the world. There's no one else who
Speaker 1: has exactly your genome. On the other hand, if you
Speaker 1: were to lay your genome in mind alongside each other
Speaker 1: lined up, they would be ninety nine point nine percent identical.
Speaker 1: About one position in a thousand is different. But those
Speaker 1: differences are what caused you to be you and me
Speaker 1: to be me. So they're obviously of intense kind of
Speaker 1: scientific and applied interest. And so it's very important to
Speaker 1: be able to take a sort of a sample of
Speaker 1: your DNA and quickly produce a profile of all the
Speaker 1: places that have variability what your particular values are, Okay,
Speaker 1: And that problem is the genotyping problem.
Speaker 2: And this used to be a very expensive, very complex
Speaker 2: problem to solve that. We've spent billions of dollars figuring
Speaker 2: out now a lot faster, a lot cheaper.
Speaker 1: A lot faster. In fact, even the technology I worked
Speaker 1: on in two thousand and five two thousand and four
Speaker 1: is multiple generations old and not really what's used anymore.
Speaker 2: So let's talk about what you did at the efficient frontier.
Speaker 2: Explain what real time click prediction rules are and how
Speaker 2: it works for a keyword search.
Speaker 1: Sure, the revenue engine that drives Google is search keyword ads, right,
Speaker 1: So every time you do a search, at the top
Speaker 1: you see ad ad AD, And so how do those
Speaker 1: ads get there? Well, actually it's surprising maybe if you
Speaker 1: don't know about it, but every single time you type
Speaker 1: in a search term on Google and hit return, a
Speaker 1: very fast auction takes place, and a whole bunch of
Speaker 1: companies running software bid electronically to place their ads at
Speaker 1: the top of your search results. And the more or
Speaker 1: less the results that are shown on the page are
Speaker 1: in order of how much they bid. It's not quite true,
Speaker 1: but you could think of it. It's true.
Speaker 2: A rough outline. So the first three sponsored results on
Speaker 2: a Google page, go through that auction process, and I
Speaker 2: think at this point everybody knows what page rank is
Speaker 2: for for the rest of that that's right, And that
Speaker 2: seemed to be Google secret sauce early on.
Speaker 1: Right, Well, you know, to talk about the the ad placement.
Speaker 1: So the people who are supplying the ad, who are
Speaker 1: participating in the auctions, they have a problem, which is
Speaker 1: how much to bid, right, And so how would you
Speaker 1: decide how much to bid? Well, you want to know
Speaker 1: basically the probability that somebody is going to click on
Speaker 1: your ad, and then you would multiply that by how
Speaker 1: much money you make eventually if they click. And that's
Speaker 1: kind of an expectation of how much money you'll make.
Speaker 1: And so then you gear your bid price to make
Speaker 1: sure that it's going to be profitable for you. And
Speaker 1: then so really you have to make a decision about
Speaker 1: what this click through rate is going to be. You
Speaker 1: have to predict the click through probability.
Speaker 2: So I was going to say, this sounds like it's
Speaker 2: a very sophisticated application of computer science probability and statistics.
Speaker 2: And if you do it right, you make money, and
Speaker 2: if you do it wrong, your ad budget is a
Speaker 2: money loser.
Speaker 1: That's right.
Speaker 2: Huh. So tell us a little bit about your doctorate,
Speaker 2: what you wrote about for your PhD at Berkeley.
Speaker 1: Yeah, so we're back to genomes. Actually, this was around
Speaker 1: the time when I was in my first year of
Speaker 1: my PhD program, is when the human genome was published
Speaker 1: in Nature. So it was kind of really the beginning
Speaker 1: of the explosion of work on kind of high throughput,
Speaker 1: large scale genetics research. And one really important question after
Speaker 1: you've sequenced a genome is well, what are all the
Speaker 1: bits of it doing. You can look at a string
Speaker 1: of DNA. It's just made up of these kind of
Speaker 1: four letters, but you don't want to just know the
Speaker 1: four letters. They're kind of a code. And some parts
Speaker 1: of the DNA represent useful stuff that is being turned
Speaker 1: by your cell into proteins and et cetera, and other
Speaker 1: parts of the DNA don't appear to have any function
Speaker 1: at all, and it's really important to know which is
Speaker 1: which as a biology researcher. And so it's you know,
Speaker 1: for a long time before high throughput sequencing, biologists would
Speaker 1: be in the lab and they would very laboriously look
Speaker 1: at very tiny segments of DNA and establish what their
Speaker 1: function was. But now we have the whole human genome
Speaker 1: sitting on disk, and we would like to be able
Speaker 1: to just run an analysis on it and have the
Speaker 1: computer spit out everything that is functional and not functional.
Speaker 1: And so that's the problem I worked on. And a
Speaker 1: really important insight is that you can take advantage of
Speaker 1: the idea of natural selection and the idea of evolution
Speaker 1: to help you. And the way you do that is
Speaker 1: you have the human genome, you sequence a bunch of
Speaker 1: primate genomes nearby relatives of the union, and you lay
Speaker 1: all those genomes on top of each other, and then
Speaker 1: you look for places where all of the genomes agree. Right,
Speaker 1: there hasn't been variation that's happening through mutations. And why
Speaker 1: hasn't there been, Well, the biggest force that throws out
Speaker 1: variation is natural selection. If you get a mutation in
Speaker 1: a part of your genome that really matters, then you're
Speaker 1: kind of on it and you won't have progeny and
Speaker 1: that'll get stamped out. So natural selection is this very
Speaker 1: strong force that's causing DNA not to change. And so
Speaker 1: when you when you make these primate alignments, you can
Speaker 1: really leverage that fact and look for conservation and use
Speaker 1: that as a big signal that something is functional.
Speaker 2: Huh, really really interesting. You mentioned our DNA is ninety
Speaker 2: nine point ninety nine. I don't know how many places
Speaker 2: to the right of the decimal point you would want
Speaker 2: to go, but very similar. How how similar or different
Speaker 2: are we from let's say, a chimpanzee. I've always questioned,
Speaker 2: there's an urban legend that they're practically the same. It
Speaker 2: always seems like it's overstated two percent. So you and
Speaker 2: I have a point one percent different me and the
Speaker 2: average chimp. It's two point zero percent.
Speaker 1: That's exactly right. Yeah, so chimps are essentially our closest
Speaker 1: non human primate relatives.
Speaker 2: Really really quite fascinating. So let's talk a little bit
Speaker 2: about the firm. You guys were one of the earliest
Speaker 2: pioneers of machine learning research. Explain a little bit what
Speaker 2: the firm does.
Speaker 1: Sure, so, we run trading strategies investment strategies that are
Speaker 1: fully automated, so we call them fully systematic, and that
Speaker 1: means that we have software systems that run every day
Speaker 1: during market hours, and they take in information about the
Speaker 1: characteristics of the securities we're trading. Think of stocks and
Speaker 1: then they make predictions of how the prices of each
Speaker 1: security is going to change over time, and then they
Speaker 1: decide on changes in our inventory, changes in held positions
Speaker 1: based on those predictions, and then those desired changes are
Speaker 1: sent into an execution system which automatically carries them out.
Speaker 2: So fully automated. Is there supervision or it's kind of
Speaker 2: running on its own with a couple of checks.
Speaker 1: There's lots of human diagnostic supervision, right, So there are
Speaker 1: people who are watching screens full of instrumentation and telemetry
Speaker 1: about what the systems are doing. But those people are
Speaker 1: not taking any actions, right unless there's a problem, right,
Speaker 1: and then they do.
Speaker 2: So let's talk a little bit about how machines learn
Speaker 2: to identify signals. I'm assuming you start with the giant
Speaker 2: database that is the history of stock prices, volume movement, etc.
Speaker 2: And then bring in a lot of additional things to bear.
Speaker 2: What's the process like developing a particular trading strategy.
Speaker 1: Yeah, so, as you're saying, we begin with a very
Speaker 1: large historical data set of prices and volumes, market data
Speaker 1: that kind, but importantly all kinds of other information about securities,
Speaker 1: financial statement data, textual data, analyst data.
Speaker 2: So it's everything from prices fundamental everything from learnings to
Speaker 2: revenue to sales, etc. I'm assuming the change and the
Speaker 2: delta of the change is going to be very significant
Speaker 2: in that. What about macroeconomic what some people call noise,
Speaker 2: but one would imagine some signal in everything from inflation
Speaker 2: to interest rates to GDPs firm spending. Are those inputs
Speaker 2: worthwhile or how do you think about those?
Speaker 1: So we don't hold portfolios that are exposed to those things.
Speaker 1: So it's really a business decision on our part. We
Speaker 1: are working with institutional investors who already have as much
Speaker 1: exposure as they want to things like the market or
Speaker 1: to well recognized econometric risk factors like value, and so
Speaker 1: they don't need our help to be exposed to those things.
Speaker 1: They are very well equipped to handle that part of
Speaker 1: their investment process. What we're trying to provide is the
Speaker 1: most diversification possible. So we want to give them a
Speaker 1: new return stream which has good and stable returns, but
Speaker 1: on top of that, importantly, is also not correlated with
Speaker 1: any of the other return streams that they already that
Speaker 1: they already have.
Speaker 2: That's interesting. So can I assume that you're applying your
Speaker 2: machine learning methodology across different asset classes or is it
Speaker 2: strictly equities? Oh?
Speaker 1: No, We apply it to UH to equities, to credit,
Speaker 1: to corporate bonds, and we trade futures contracts, and in
Speaker 1: the fullness of time, we hope that we will be
Speaker 1: trading kind of every security in the world.
Speaker 2: So, so currently stocks, bonds, When you say futures, I
Speaker 2: assume commodities, all.
Speaker 1: Kinds of futures contract.
Speaker 2: It's really really interesting. So it could be anything from
Speaker 2: interest rate swaps to commodities to the full gamut. So,
Speaker 2: so how different is this approach from what other quant
Speaker 2: shops do that really focus on equities.
Speaker 1: I think it's kind of the same question as asking, well,
Speaker 1: what do we mean when we say we use machine
Speaker 1: learning or that you know we are our principles are
Speaker 1: our machine learning principles, and so how does that make
Speaker 1: us different than the kind of standard approach in quantitative trading?
Speaker 1: And the answer to the question really comes back to
Speaker 1: this idea we mentioned a little while ago of how
Speaker 1: powerful the tools are that you're using to form predictions. Right,
Speaker 1: So in our business, the thing that we build is
Speaker 1: called a prediction rule. Okay, that's that's our widget and
Speaker 1: What a prediction rule does is it takes in a
Speaker 1: bunch of input, a bunch of information about a stock
Speaker 1: at a moment in time, and it hands you a
Speaker 1: guess about how that stock's price is going to change
Speaker 1: over some future period of time. Okay, and so there
Speaker 1: is one most important question about prediction rules, which is
Speaker 1: how complex are they? How much complexity do they have?
Speaker 1: Complexity is a colloquial term. It's unfortunately another example of
Speaker 1: a place where things can be vague or ambiguous because
Speaker 1: a general purpose word has been borrowed in a technical setting.
Speaker 1: But when you use the word complexity in statistical prediction,
Speaker 1: there's a very specific meaning. It means how much expressive
Speaker 1: power does this prediction rule have? How good a job
Speaker 1: can it do of approximating what's going on in the
Speaker 1: data you show it. Remember, we have these giant historical
Speaker 1: data sets, and every entry in the data set looks
Speaker 1: like this. What was going on with the stock at
Speaker 1: a moment in a certain moment in time, it's price action,
Speaker 1: it's financials analyst information. And then what did its price
Speaker 1: do in the subsequent twenty four hours or the subsequent
Speaker 1: fifteen minutes or whatever. Okay, and so when you talk
Speaker 1: about the amount of complexity that a prediction rule has,
Speaker 1: that means how well is it able to capture the
Speaker 1: relationship between the things that you can show it when
Speaker 1: you ask it for a prediction, and what actually happens
Speaker 1: to the price. And naturally you kind of want to
Speaker 1: use high complexity rules because they have a lot of
Speaker 1: approximating power. They do a good job of describing anything
Speaker 1: that's going on. But there are two There are two
Speaker 1: disadvantages to high complexity. One is it needs a lot
Speaker 1: of data, otherwise it gets fooled into thinking that randomness
Speaker 1: is actually signal. And the other is that it's hard
Speaker 1: to reason about what's going on under the hood. Right,
Speaker 1: when you have very simple prediction rules, you can sort
Speaker 1: of summarize everything that's good that they're doing in a sentence. Right,
Speaker 1: you can look you can look inside them and get
Speaker 1: a complete understanding of how they behave, and that's not
Speaker 1: possible with high complexity prediction rules.
Speaker 2: So I'm glad you brought up the concept of how
Speaker 2: easy it is or how frequently you can fool an
Speaker 2: algorithm or a complex rule, because sometimes the results are
Speaker 2: just random. And it reminds me of the issue of
Speaker 2: back testing. No one ever shows you a bad back test.
Speaker 2: How do you deal with the issue of overfitting and
Speaker 2: back testing that just is geared towards what already happened
Speaker 2: and not what might happen in the future.
Speaker 1: Yeah, that is you know, if you like the million
Speaker 1: dollar question in statistical prediction, Okay, and it might you
Speaker 1: might find it surprising that relatively straightforward ideas go a
Speaker 1: long way here. And so let me let me just
Speaker 1: describe a little scenario of how you deal you can
Speaker 1: deal with this. All right, we agree, we have this
Speaker 1: big historical data set, right, One thing you could do
Speaker 1: is just start analyzing the heck out of that data
Speaker 1: set and find a complicated prediction rule. But you're you've
Speaker 1: already started doing it wrong. The first thing you do
Speaker 1: before or you even look at the data is you
Speaker 1: randomly pick out half of the data and you lock
Speaker 1: it in a drawer. Okay, and that leads you with
Speaker 1: the other half of the data that you haven't locked away.
Speaker 1: On this half, you get to go hogwild. You build
Speaker 1: every kind of prediction rule, simple rules, enormously complicated rules,
Speaker 1: everything in between. Right, and now you can check how
Speaker 1: accurate all of these prediction rules that you've built are
Speaker 1: on the data that they have been looking at, and
Speaker 1: the answer will always be the same. The most complex
Speaker 1: rules will look the best. Of course, they have the
Speaker 1: most expressive power, so naturally they do the best job
Speaker 1: of describing what you showed them. The big problem is
Speaker 1: that what you showed them is a mix of signal
Speaker 1: and noise, and there's no way you can tell to
Speaker 1: what extent a complex rule has found the signal versus
Speaker 1: the noise. All you know is that it's perfectly described
Speaker 1: to the data you showed it. You certainly suspect it
Speaker 1: must be overfitting if it's doing that. Well, okay, so
Speaker 1: now you freeze all those prediction rules. You're not allowed
Speaker 1: to change them in any way anymore. And now you
Speaker 1: unlock the drawer and you pull out all that data
Speaker 1: that you've never looked at. You can't overfit data that
Speaker 1: you never fit, and so you take that data and
Speaker 1: you run it through each of these prediction rules that's frozen,
Speaker 1: that you built. And now it is not the case
Speaker 1: at all that the most complex rules look the best. Instead,
Speaker 1: you'll see a kind of U shaped behavior where the
Speaker 1: very simple rules are too simple. They've missed signal, they
Speaker 1: left signal on the table. The two complex rules are
Speaker 1: also doing badly because they've captured all the signal but
Speaker 1: also lots of noise. And then somewhere in the middle
Speaker 1: is a sweet spot where you've struck the right trade
Speaker 1: off between how much expressive power the prediction rule has
Speaker 1: and how good a job it is doing of avoiding
Speaker 1: the mistaking of noise for signal.
Speaker 2: Really really intriguing. So you guys, have you've built one
Speaker 2: of the largest specialized machine learning research and development teams
Speaker 2: in finance. How do you assndle a team like that
Speaker 2: and how do you get the brain trust to do
Speaker 2: the sort of work that's applicable to managing assets.
Speaker 1: Well, the short answer is, we spend a huge amount
Speaker 1: of energy on recruiting and uh, you know, identifying the
Speaker 1: sort of premier people in the field of machine learning,
Speaker 1: kind of both academic and practitioners, and we exhibit a
Speaker 1: lot of patients. We we wait a really long time
Speaker 1: to be able to find the people who are kind
Speaker 1: of really the best, and that that that matters enormously
Speaker 1: to us, both from the standpoint of the success of
Speaker 1: the firm and also because it's something that you know,
Speaker 1: we value extremely highly just having great colleagues, brilliant colleagues
Speaker 1: that you know, I want to work in a place
Speaker 1: where I can learn from all the people around me.
Speaker 1: And you know, when when my co founder Michael Caratanev
Speaker 1: and I we're talking about starting Bollion, one of the
Speaker 1: reasons that was on our minds is we wanted to
Speaker 1: be in control of who we worked with. You know,
Speaker 1: we really wanted to be able to assemble a group
Speaker 1: of people who were, you know, as brilliant as we
Speaker 1: could find, but also you know, good people, people that
Speaker 1: we liked, people that we were excited to collaborate with.
Speaker 2: So let's talk about some of the fundamental principles Volnon
Speaker 2: is built on. You reference a prediction based approach from
Speaker 2: a paper Leo Briman wrote called two Cultures. Yeah, tell
Speaker 2: us a little bit about what two cultures actually is.
Speaker 1: Yeah. So this this paper was written about twenty years ago.
Speaker 1: Leo Briman was one of the great probabilists and statisticians
Speaker 1: of his generation. Berkeley professor need I say, and you know,
Speaker 1: Leo had been a practitioner in statistical consulting actually for
Speaker 1: quite some time. I'm in between a U. C. L.
Speaker 1: A tenured job and returning to academia at Berkeley, and
Speaker 1: he learned a lot in that time about actually solving
Speaker 1: prediction problems and instead of hypothetically solving them in sort
Speaker 1: of the academic context. And so all of his insights
Speaker 1: about the difference really culminated in this paper from two
Speaker 1: thousand that he wrote.
Speaker 2: The difference between practical use versus academic theory if you like.
Speaker 1: Yeah, And so he identified two schools of thought about
Speaker 1: solving prediction problems, right, and one school is sort of
Speaker 1: model based. The idea is there's some stuff you're going
Speaker 1: to get to observe stock characteristics. Let's say there's a
Speaker 1: thing you wish you knew future price change, let's say,
Speaker 1: and there's a box in nature that turns those inputs
Speaker 1: into the output, right. And in the model based school
Speaker 1: of thought, you try to open that box, reason about
Speaker 1: how it must work, make theories. In our case, these
Speaker 1: would be sort of econometric theories, financial economics theories. And
Speaker 1: then those theories have knobs, not many, and you use
Speaker 1: data to set the knobs, but otherwise you believe the model.
Speaker 1: And he contrasts that with the machine learning school of thought,
Speaker 1: which is also has the idea of Nature's box. The
Speaker 1: inputs go in, the thing you wish you knew comes out.
Speaker 1: But in machine learning, you don't try to open the box.
Speaker 1: You just try to approximate what the box is doing.
Speaker 1: And your measure of success is predictive accuracy, and is
Speaker 1: only predictive accuracy. If you build a gadget and that
Speaker 1: gadget produces predictions that are really accurate they turn out
Speaker 1: to look like the thing that nature produces, then that
Speaker 1: is success. And at the time he wrote the paper,
Speaker 1: his assessment was ninety eight percent of statistics was taking
Speaker 1: the model based approach, two percent was taking the machine
Speaker 1: learning approach.
Speaker 2: And are those statistics still valid today or have we
Speaker 2: shifted quite a bit?
Speaker 1: We shifted quite a bit, And different different arenas of
Speaker 1: prediction problems have different mixes these days. But even in finance,
Speaker 1: I would say it's it's probably more like fifty to.
Speaker 2: Fifty really that much?
Speaker 1: Yeah, I think you know, And if you the logical
Speaker 1: extreme is natural language modeling, which was done for decades
Speaker 1: and decades in the model based approach, where you kind
Speaker 1: of reasoned about linguistic characteristics of how people kind of
Speaker 1: do dialogue and those models had some parameters and you
Speaker 1: fit them with data, and then instead you have, as
Speaker 1: we said, a database of a trillion words and a
Speaker 1: tool with one hundred and seventy five billion parameters, and
Speaker 1: you run that and there is no hope of completely
Speaker 1: understanding what is going on inside of GPD three. But
Speaker 1: nobody complains about that because the results are astounding. The
Speaker 1: thing that you get is incredible. And so that is
Speaker 1: by analogy, the way that we reason about running systematic
Speaker 1: investment strategies. At the end of the day, predictive accuracy
Speaker 1: is what creates returns for investors. Being able to give
Speaker 1: complete descriptions of exactly how the predictions arise does not
Speaker 1: in itself create returns for investors. Now, I'm not against
Speaker 1: interpretability and simplicity all equal. I love interpretability and simplicity,
Speaker 1: but all else is not equal. If you want the
Speaker 1: most accurate predictions, you are going to have to sacrifice
Speaker 1: some amount of simplicity. In fact, this truth is so
Speaker 1: widespread that Leo gave it a name in his paper.
Speaker 1: He called it Accam's dilemma. So Accam's razor is the
Speaker 1: philosophical idea that you should choose the simplest explanation that
Speaker 1: fits the facts. Akam's dilemma is the point that in
Speaker 1: statistical prediction, simplest approach, even though you wish you could
Speaker 1: choose it, is not the most accurate approach if you
Speaker 1: care about predictive accuracy. If you're putting predictive accuracy first,
Speaker 1: then you have to embrace a certain amount of complexity
Speaker 1: and lack of interpretability.
Speaker 2: Huh, that's really quite fascinating. So let's talk a little
Speaker 2: bit about artificial intelligence and large language models. You follow
Speaker 2: d Shaw playing in e commerce and biotech. It seems
Speaker 2: like this approach to using statistics, probability, and computer science
Speaker 2: is applicable to so many different fields.
Speaker 1: It is. Yeah, I think you're talking about prediction problems ultimately,
Speaker 1: So in recommender systems, you can think of the question
Speaker 1: as being well, if I had to predict what thing
Speaker 1: I could show a person that would you be most
Speaker 1: likely to change their behavior and cause them to buy it.
Speaker 1: It's a kind of prediction problem that motivates recommendations. In biotechnology.
Speaker 1: Very often we are trying to make predictions about whether someone,
Speaker 1: let's say, does or doesn't have a condition a disease
Speaker 1: based on lots of information we can gather from high
Speaker 1: throughput diagnostic techniques. These days, the keyword in biology and
Speaker 1: in medicine and biotechnology is high throughput. You're running analyses
Speaker 1: on an individual that are producing hundreds of thousands of numbers,
Speaker 1: and you want to be able to take all of
Speaker 1: that kind of wealth of data and turn it into
Speaker 1: diagnostic information about.
Speaker 2: And we've seen AI get applied to pharmaceutical development in
Speaker 2: ways that people just never really could have imagined just
Speaker 2: a few short years ago. Is there a field that
Speaker 2: AI and large language models are not going to touch
Speaker 2: or is this just the future of everything.
Speaker 1: The kinds of fields where you would expect uptake to
Speaker 1: be slow are where it is hard to assemble large
Speaker 1: data sets of systematically gathered data. And so any field
Speaker 1: where it's relatively easy to at large scale, let's say,
Speaker 1: produce the kinds of the same kinds of informations that
Speaker 1: experts are using to make their decisions, you should expect
Speaker 1: that field to be impacted by these tools if it
Speaker 1: hasn't been already.
Speaker 2: So you're kind of answering my next question, which is
Speaker 2: what led you back to investment management. But it seems
Speaker 2: if there's any field that just generates endless amounts of data.
Speaker 1: It's the markets, that's true. And I had already been
Speaker 1: really interested in the problems of systematic investment strategies from
Speaker 1: my time working in d SHAW, and so my co
Speaker 1: founder Michael Kratanav and I, you know, we were both
Speaker 1: in the Bay Area in the two thousand and four.
Speaker 1: He was there because of a firm that he had founded,
Speaker 1: and I was there finishing my PhD. And we started
Speaker 1: to talk about the idea of using contemporary machine learning
Speaker 1: methods to build strategies that would be, you know, really
Speaker 1: different from strategies that result from classical techniques. And we
Speaker 1: had met at d SHAW in the nineties and been
Speaker 1: less excited about this idea because the methods were pretty immature.
Speaker 1: There wasn't actually a giant diversity of data back in
Speaker 1: the nineties in financial markets, not like there was in
Speaker 1: two thousand and five. And compute was really still quite
Speaker 1: expensive in the nineties, whereas in two thousand and five,
Speaker 1: you know, it had been dropping in the usual More's
Speaker 1: Law way. And this was even before GPUs, and so
Speaker 1: when we looked at the problem in two thousand and five,
Speaker 1: it felt like there was a very live opportunity to
Speaker 1: do something with a lot of promise that would be
Speaker 1: really different. And we had the sense that not a
Speaker 1: lot of people were of the same opinion, and so
Speaker 1: it seemed like something that we should try.
Speaker 2: That there was a void. Nothing nothing in the market
Speaker 2: hates more than a vacuum and intellectual approach. So so
Speaker 2: you mentioned the diversity of various data sources. What what
Speaker 2: don't you consider, like how how far off of price
Speaker 2: and volume do you go in the net you're casting
Speaker 2: for inputs into into your systems.
Speaker 1: Well, I think we're prepared as a you know, as
Speaker 1: a as a research principle, we're prepared to consider any
Speaker 1: data that has some bearing on price formation, like some
Speaker 1: some plausible bearing on how prices are formed. Now, of
Speaker 1: course we're you know, we're a relatively small group of
Speaker 1: people with a lot of ideas and uh, and so
Speaker 1: we have to prioritize so you know, in the event
Speaker 1: we end up pursuing data that you know makes a
Speaker 1: lot of sense, you know, we don't we don't try.
Speaker 2: I mean, can you go as far as politics or
Speaker 2: the weather, like how far off of prices can you
Speaker 2: can you look?
Speaker 1: So, you know, an example would be the weather. You're
Speaker 1: for most securities, you're not going to be very interested
Speaker 1: in the weather, but for commodities future as you might be,
Speaker 1: so that you know, that's the kind of reasoning you
Speaker 1: would apply.
Speaker 2: Right, really really interesting. So let's talk about some of
Speaker 2: the strategies. You guys are running short and mid horizon
Speaker 2: US equities, European equities, Asian equities, mid horizon US credit,
Speaker 2: and then cross assets. So I might to assume all
Speaker 2: of these are machine learning based, and how similar different
Speaker 2: is each approach to each of those asset classes.
Speaker 1: Yeah, they're all machine learning based. The kind of principles
Speaker 1: that I've described of using as much complexity as you
Speaker 1: need to maximize predictive accuracy, et cetera. Those principles underlie
Speaker 1: all the systems. But of course it's trading. Trading corporate
Speaker 1: bonds is very different from trading equities, and so the
Speaker 1: implementations reflect that reality.
Speaker 2: Huh. So let's talk a little bit about the four
Speaker 2: step process that you bring to the systematic approach, and
Speaker 2: this is off of your site, so it's it's data prediction, engine,
Speaker 2: portfolio construction, and execution. Yeah, I'm assuming that is heavily
Speaker 2: computer and machine learning based. At each step along the way.
Speaker 2: Is that is that fair?
Speaker 1: I think that's fair. I mean to different degrees. The
Speaker 1: data gathering that's you know, that's a that's largely a
Speaker 1: software and kind of operations and infrastructure job.
Speaker 2: Do you guys have to spend a lot of time
Speaker 2: cleaning up that data and making sure that because you
Speaker 2: hear between CRISP and s and P and Bloomberg, sometimes
Speaker 2: you'll pull something up and they're just all off a
Speaker 2: little bit from each other because they all bring a
Speaker 2: very different approach to data assembly. How do you make
Speaker 2: sure everything is consistent and there's no errors or errants
Speaker 2: inputs throughout.
Speaker 1: Yeah, through a lot of effort. Essentially, there there we have.
Speaker 1: You know, we have an entire group of people who
Speaker 1: focus on data operations, both for gathering a historical data
Speaker 1: and for the management of the ongoing live data feeds.
Speaker 1: There's no way around that. I mean, that's just work
Speaker 1: that you have to that you have to do.
Speaker 2: You just have to brute force your way through that.
Speaker 2: And then the prediction engine. Sounds like that's the single
Speaker 2: most important part of the machine learning process if I'm
Speaker 2: understanding you correctly, that that's where all the meat of
Speaker 2: the technology is.
Speaker 1: Yeah, I understand the sentiment. I mean, it's worth emphasizing
Speaker 1: that you do not get to a successful systematic strategy
Speaker 1: without all the ingredients. You have to have clean data
Speaker 1: because of the garbage in garbage out. You have to
Speaker 1: have accurate predictions. But you know, predictions don't automatically translate
Speaker 1: into returns for investors. Those predictions are kind of the
Speaker 1: power that drives the portfolio holding part of the system.
Speaker 2: So let's talk about that portfolio construction. Given that you
Speaker 2: have a prediction engine that and good data going into it,
Speaker 2: so you're fairly confident as to the output. How do
Speaker 2: you then take that output and say, here's how I'm
Speaker 2: going to build a portfolio based on what this generates.
Speaker 1: Yeah, so there are three big ingredients in the portfolio construction.
Speaker 1: The predictions what is usually called a risk model in
Speaker 1: this business, which means some understanding of how volatile prices
Speaker 1: are across all the securities you're trading, how correlated they are,
Speaker 1: how you know if they have a if they have
Speaker 1: a big movement, how big that movement will be. That's
Speaker 1: all the risk model. And then the final ingredient is
Speaker 1: what's usually called a market impact model, and that means
Speaker 1: an understanding of how much you are going to push
Speaker 1: prices away from you when you try to trade. This
Speaker 1: is a reality of all trading. You buy a lot
Speaker 1: of a security, you push the price up, you push
Speaker 1: it away from you in the unfavorable direction. And in
Speaker 1: the systems that we run, the predictions that we're trying
Speaker 1: to capture are about the same size as the effect
Speaker 1: that we have on the markets when we trade, and
Speaker 1: so you cannot neglect that impact effect when you're thinking
Speaker 1: about what portfolios to hold.
Speaker 2: So execution becomes really important. If you're not executing well,
Speaker 2: you are moving prices away from your profit.
Speaker 1: That's right, and it is you know, probably the single
Speaker 1: thing that undoes quantitative hedge funds most often is that
Speaker 1: they misunderstand how much they're moving prices. They get too big,
Speaker 1: they start trading too much, and they sort of blowed
Speaker 1: themselves up.
Speaker 2: It's funny that you say that, because as you were
Speaker 2: describing that, the first name that popped into my head
Speaker 2: was long term capital managements trading these really thinly traded
Speaker 2: obscure fixed income products, and everything they bought they sent
Speaker 2: higher because there just wasn't any volume in it. And
Speaker 2: when they needed liquiditly there was none to be had.
Speaker 2: And you know that plus no risk management one hundred
Speaker 2: x leverage equals a kaboom.
Speaker 1: They made a number of mistakes the book. The book
Speaker 1: is good. So when genius fail in, oh absolutely love
Speaker 1: that fantastically fascinating.
Speaker 2: So when you're reading a book like that, somewhere in
Speaker 2: the back of your head are you thinking, hey, this
Speaker 2: is like a what not to do when you're setting
Speaker 2: up a machine learning fund. How influential is something like.
Speaker 1: Well one hundred percent? I mean, look, I think the
Speaker 1: most important adage I've ever heard in my professional life
Speaker 1: is good judgment comes from experience. Experience comes from bad judgment.
Speaker 1: So the extent to which you can get good judgment
Speaker 1: from other people's experience, that is that that is like
Speaker 1: a free tuition. And so we talk a lot about
Speaker 1: all the all the mistakes that that that other people
Speaker 1: have made. And you know, we do not congratulate ourselves
Speaker 1: on having avoided mistakes. We think those people were smart.
Speaker 1: I mean look that you know, you read about these
Speaker 1: events and these people. None of these people were dummies.
Speaker 1: They were sophisticated Nobel laureates.
Speaker 2: Yeah, right, it's they just didn't have a guide book
Speaker 2: on what not to do, which which you guys.
Speaker 1: Do We don't. No, I don't think we do. I
Speaker 1: mean apart from that, apart from reading about right, But
Speaker 1: everybody is undone by a failure that they they didn't
Speaker 1: they did, they didn't think of ever didn't know about yet.
Speaker 1: And we're extremely cognizant of that.
Speaker 2: Huh. That has to be somewhat humbling to come being
Speaker 2: on the lookout for that blind spot that could disrupt everything.
Speaker 1: Yes, yeah, humility is the key ingredient in running in
Speaker 1: running these systems.
Speaker 2: Really quite amazing. So let's talk a little bit about
Speaker 2: how academically focused volling On is. You guys have a
Speaker 2: pretty deep R and D team internally, you teach at Berkeley.
Speaker 2: What does it mean for a Hedge fund to be
Speaker 2: academically focused?
Speaker 1: What I would say probably is kind of evidence based
Speaker 1: rather than academically focused. Saying academically focused gives the impression
Speaker 1: that kind of papers would be the goal or the
Speaker 1: desired output, and that's not the case at all. We have,
Speaker 1: you know, a very specific applied problem that we are
Speaker 1: trying to solve.
Speaker 2: Papers are a mean to an end.
Speaker 1: Papers are you know, we don't write papers for external consumption.
Speaker 1: We do lots of writing internally, and that's to make
Speaker 1: sure that that you know, we're keeping track of our
Speaker 1: own kind of scientific process.
Speaker 2: But you're fairly widely published in statistics and machine learning. Yes,
Speaker 2: what purpose does that serve other than a calling card
Speaker 2: for the fund as well as Hey, I have this
Speaker 2: idea and I want to see what the rest of
Speaker 2: my peers think of it. When when you put stuff
Speaker 2: out into the world, what sort of feedback or pushback
Speaker 2: do you get?
Speaker 1: I guess I would have to say, I really I
Speaker 1: do that as kind of a double life of non
Speaker 1: financial research. So it's just something that I really enjoy. Principally,
Speaker 1: what it means is that I get to work with
Speaker 1: PhD students, and you know, we have really outstanding PhD
Speaker 1: students at Berkeley in statistics, and so it's an opportunity
Speaker 1: for me to do a kind of intellectual work that namely,
Speaker 1: you know, writing a paper laying out an argument for
Speaker 1: public consumption, et cetera that is kind of closed off
Speaker 1: as far as so.
Speaker 2: Not adjacent to what you guys are doing at Volleyon generally. No, No,
Speaker 2: that's really interesting. So then I always assume that that
Speaker 2: was part of your process for developing new models to
Speaker 2: apply machine learning to new assets. Take us through the process.
Speaker 2: How do you go about saying, Hey, this is an
Speaker 2: asset class we don't have exposure to. Let's see how
Speaker 2: to apply what we already know to that specific area.
Speaker 1: Yeah, we have it's a great question. So we're trying
Speaker 1: as much as possible to get the problem for a
Speaker 1: new asset class into a familiar setup, into you know,
Speaker 1: as standard a setup as we can, and so we
Speaker 1: know what these systems look like in the world of equity.
Speaker 1: And so if you're trying to do the same kind,
Speaker 1: if you're trying to build the same kind of system
Speaker 1: for corporate bonds, and you start off by saying, well, okay,
Speaker 1: i'd like I need to know, you know, closing prices
Speaker 1: or inter day prices for all the bonds. Already, you
Speaker 1: have a very big problem in corporate bonds because there
Speaker 1: is no there is no live price feeds that's showing
Speaker 1: you a bit offer quote in the way that there
Speaker 1: is inequity. And so before you can even get started
Speaker 1: thinking about predicting how a price is going to change,
Speaker 1: it would be nice if you know what the price
Speaker 1: currently was, and that is already a problem you have
Speaker 1: to solve in corporate bonds as opposed to being just
Speaker 1: an input that you have access to.
Speaker 2: The old joke was trading by appointment only. Yeah, and
Speaker 2: that seems to be a bit of an issue. And
Speaker 2: there are so many more bond issues than there are equities. Absolutely,
Speaker 2: is this just a database challenge or how do you work?
Speaker 1: No, it's a statistics problem, but it's it's a different
Speaker 1: kind of statistics problem. We're not in this case. We're
Speaker 1: not trying to yet. We're not yet trying to predict
Speaker 1: the future of any quantity. We're trying to say, I
Speaker 1: wish I knew what the fair value of this of
Speaker 1: this CSIP was. I can't see that exactly because there's
Speaker 1: no live order book that with a bid and an
Speaker 1: offer that's got lots of liquidity that lets me figure
Speaker 1: out the fair value. But I do know what.
Speaker 2: At best, you have a recent price, maybe not even
Speaker 2: so recent.
Speaker 1: I have lots of related information. I know you know
Speaker 1: this bond. Maybe this bond didn't trade today, but it
Speaker 1: traded a few times yesterday. I get to say I
Speaker 1: know where it traded. I'm in touch with bond dealers,
Speaker 1: so I know where they've quoted this bond, maybe only
Speaker 1: on one side over the last few days. I have
Speaker 1: some information about the company that issued this bond, et cetera.
Speaker 1: So I have lots of stuff that's related to the
Speaker 1: number I know that I want to know. I just
Speaker 1: don't know that number, right, And so what I want
Speaker 1: to try to do is kind of fill in and
Speaker 1: do what's what in statistics or in control we would
Speaker 1: call a now casting problem, huh, And it's an analogy
Speaker 1: actually is too automatically controlling an airplane. So surprisingly, Oh,
Speaker 1: the main there there are there are when you're if
Speaker 1: you're trying if a software is trying to fly in
Speaker 1: air plane, there are six things that it absolutely has
Speaker 1: to know. Has to know the x y z of
Speaker 1: where the plane is and the x y z of
Speaker 1: its velocity where it's headed. Right, those are the six
Speaker 1: most important numbers. Now, nature does not just supply those
Speaker 1: numbers to you. You cannot know those numbers with perfect exactitude.
Speaker 1: But there's lots of instruments on the plane, and there's
Speaker 1: GPS and all sorts of information that is very closely
Speaker 1: related to the numbers You wish you knew, and you
Speaker 1: can use statistics to go from all that stuff that's
Speaker 1: adjacent to a guess and infill of the thing you
Speaker 1: wish you knew, And the same goes with the current
Speaker 1: price of a corporate bond.
Speaker 2: Huh. That's really kind of interesting. So I'm curious as
Speaker 2: to how often you start working your way into one
Speaker 2: particular asset or a particular strategy for that asset and
Speaker 2: just suddenly realize, oh, this is wildly different than we
Speaker 2: previously expected, and suddenly you're down a rabbit hole to
Speaker 2: just wildly unexpected areas. It sounds like that isn't all
Speaker 2: then uncommon.
Speaker 1: It is not uncommon at all.
Speaker 2: Huh.
Speaker 1: No, it's a nice you know, there's this kind of
Speaker 1: wishful thinking that all we have. You know, we figured
Speaker 1: it out in one asset class in the sense that
Speaker 1: we have a system that's kind of stable and performing
Speaker 1: reasonably well that we that we have a feel for,
Speaker 1: and now we want to take that system and somehow
Speaker 1: replicate it in a different situation. And while we're going
Speaker 1: to standardize the new situation to make it look like
Speaker 1: the old situation. That's the principle. That principle kind of
Speaker 1: quickly goes out the window when you when you start
Speaker 1: to make contact with the reality of how the new
Speaker 1: asset class actually behaves.
Speaker 2: So stocks are different than credit, are different than bonds,
Speaker 2: or different than commodities. They're all like starting fresh. Yeah,
Speaker 2: over what some of the more surprising things you've learned
Speaker 2: as you've applied machine learning to totally different asset classes.
Speaker 1: Well, I think, you know, corporate bonds provide a lot
Speaker 1: of examples of this. I mean, the fact that you
Speaker 1: don't actually really know a good live price or a
Speaker 1: good live bid offers it seems seems you know, it's surprising.
Speaker 1: I mean, this is this fact has started to change.
Speaker 1: Like over the years, there's been an accelerating electronification of
Speaker 1: corporate bond treading, and that's you know, that's that's been
Speaker 1: a big advantage for us actually because we were kind
Speaker 1: of first movers and so we've really benefited from that.
Speaker 1: So the problem is diminished relative to how it was,
Speaker 1: you know, six seven years ago when we started, but
Speaker 1: it's still relative equities, it's absolutely there.
Speaker 2: Yeah, So you get so when in other words, if
Speaker 2: I'm looking at a bond mutual fund or even a
Speaker 2: bondytf that's trading during the day. That price is somebody's
Speaker 2: best approximation of the value of all the bonds inside.
Speaker 2: But really you don't know the nav, do you.
Speaker 1: You're just kind of guessing, Barry, don't even get me
Speaker 1: started on bonditfs real because.
Speaker 2: That it seems like that would be the first place
Speaker 2: that would show up. Hey, bondytf's sound like throughout the
Speaker 2: day they're gonna be mispriced a little bit or wildly mispriced.
Speaker 1: Well, the bond ETF there's a sense if you're a
Speaker 1: if you're a market purist, in which they can't be
Speaker 1: mispriced because there's their price is set by supplying demand
Speaker 1: in the ETF market, and that's a super liquid market,
Speaker 1: and so there may be a difference between the market
Speaker 1: price of the ETF and the under the nave of
Speaker 1: the underlying portfolio, except in many cases with bond ETF
Speaker 1: there's not even a crisply defined underlying portfolio. It turns
Speaker 1: out that the authorized participants in those ETF markets can
Speaker 1: negotiate with the fund manager about exactly what the constituents
Speaker 1: are of the create redeem baskets, and so it's not
Speaker 1: even at all clear what you mean when you say
Speaker 1: that the nav is this or that relative to the
Speaker 1: price of the ETF.
Speaker 2: So when I asked about what's surprising when you work
Speaker 2: you in on a rabbit hole. Hey, we don't know
Speaker 2: what the hell's in this bond ETF. Trust us, it's
Speaker 2: all good. That's a pretty surprise. And I'm only exaggerating
Speaker 2: a little bit, But that seems like that's kind of shocking.
Speaker 1: It's it is surprising when you find out about it,
Speaker 1: but you quickly come to understand. If you trade single
Speaker 1: name bonds, as we do, you quickly come to understand
Speaker 1: why bond ETFs work that way.
Speaker 2: I recall a couple of years ago there was a
Speaker 2: big Wall Street Journal article on the g l d
Speaker 2: E t F, and from that article I learned that
Speaker 2: GLD was formed because gold dealers had just excess gold
Speaker 2: piling up in their warehouses and they needed a way
Speaker 2: to move it. So that was kind of shocking about
Speaker 2: that ETF any other space that that led to a
Speaker 2: sort of big surprise as you worked your way into it.
Speaker 1: Well, I think ETFs are a kind of a good
Speaker 1: source of these examples. So the volatility ETFs, the you know,
Speaker 1: the ETFs that are that are based on the VIX
Speaker 1: or that are short the vics. You may remember several years.
Speaker 2: Ago I was gonna say the ones that haven't blown up.
Speaker 1: Yeah right, there was this event called Valmageddon where.
Speaker 2: That was ETF notes, wasn't it the Yeah.
Speaker 1: Right, there are these essentially these investment products that were
Speaker 1: short VIX, and VIX went through a spike that caused
Speaker 1: them to have to liquidate, which was part I mean,
Speaker 1: the people who designed the sixteene traded note. They understood
Speaker 1: that this was a possibility, so they had a sort
Speaker 1: of uh descriptions in their in their contract for what
Speaker 1: it would mean. But yeah, always surprising to watch something
Speaker 1: suddenly go out of business.
Speaker 2: We seem to get a thousand year flood every couple
Speaker 2: of years. Maybe we shouldn't be calling these things thousand
Speaker 2: year flood. That's right, that's a that's a big misnomer.
Speaker 1: As statisticians, we tell people, you know, if you if
Speaker 1: you think that you've experienced a six sigma event, the
Speaker 1: problem is that you have underestimated sigma.
Speaker 2: That that's really interesting. So so, given the gap in
Speaker 2: the world between computer science and an investment management, how
Speaker 2: long is it going to be before that narrows and
Speaker 2: we start seeing a whole lot more of the sort
Speaker 2: of work you're doing applied across the board to to
Speaker 2: the world of investment.
Speaker 1: Well, I think it's happening. It's been happening for for
Speaker 1: quite a long time. I mean, for example, all of
Speaker 1: modern portfolio theory. Really, it kind of began in the
Speaker 1: fifties with you know, first of all, Markowitz and other
Speaker 1: people thinking about, you know, what it means to benefit
Speaker 1: from diversification, and the idea that you know, diversification is
Speaker 1: the only free lunch in finance. So I would I
Speaker 1: would say that, you know, the idea of thinking in
Speaker 1: a in a systematic and scientific way about how to
Speaker 1: to manage and grow wealth, not you know, not even
Speaker 1: just for institutions, but also for individuals. Has is an
Speaker 1: example of a way that these ideas have kind of
Speaker 1: had profound effects.
Speaker 2: I know, I only have you for a little while longer,
Speaker 2: So let's jump to our favorite questions that we ask
Speaker 2: all of our guests, starting with tell us what you're
Speaker 2: streaming these days? What are you either listening to or
Speaker 2: watching to keep yourself entertained.
Speaker 1: I A few things I've been watching recently. The Bear,
Speaker 1: I don't know if you've heard So Great, So great, right,
Speaker 1: and I'm in Chicago, as I know, we were just
Speaker 1: from Yeah, so.
Speaker 2: So and and there are parts of that show that
Speaker 2: are kind of a love letter to absolutely as you
Speaker 2: get deeper into the series, because it starts out kind
Speaker 2: of gritty and you're seeing the underside, and then as
Speaker 2: we progress, it really becomes like a lovely postcard. Such
Speaker 2: an amazing show.
Speaker 1: So really really love that show. Was I was late
Speaker 1: to better call Saul that I'm finishing up. I think
Speaker 1: as good as as Breaking Bad, So I maybe when
Speaker 1: you haven't heard of there's a show called Mister in Between.
Speaker 2: Which is mister Yeah.
Speaker 1: It's not Hulu, it's from it's from Australia. It's about
Speaker 1: a guy who's, you know, a doting father living his life.
Speaker 1: He's also essentially a muscle man and hitman for for
Speaker 1: local criminals in his part of Australia. But it's half
Speaker 1: hour dark comedy.
Speaker 2: Right, so not quite Barry and not quite Sopranos somewhere.
Speaker 1: Yeah, that's exactly.
Speaker 2: Yeah, sounds really interesting. Tell us about your early mentors
Speaker 2: who helped shape your career.
Speaker 1: Well, Berry, I'd been lucky to have a lot of
Speaker 1: people who were you know, both really smart and talented
Speaker 1: and willing to you know, take the time to help
Speaker 1: me learn and understand things. So actually, my co founder,
Speaker 1: Michael Caratanov, he was kind of my first mentor in finance.
Speaker 1: He he had been a d SHAW for several years
Speaker 1: when I got there, and he he really taught me
Speaker 1: kind of the ins and outs of of market micro structure.
Speaker 1: I worked with a couple of people who managed me
Speaker 1: at d SHAW yo see Friedman and Kapeel Mature, who
Speaker 1: have gone on to hugely successful careers in quantitative finance,
Speaker 1: and they taught me a lot to when I did
Speaker 1: my PhD. My advisor Mike Jordan, who's a kind of
Speaker 1: world famous machine learning researcher. You know, I learned enormously
Speaker 1: from him. And there's another professor of statistics who sadly
Speaker 1: passed away about fifteen years ago named David Friedman. He
Speaker 1: was really just an intellectual giant of the twentieth century
Speaker 1: and probability and statistics. He was both, you know, one
Speaker 1: of the most brilliant probabilists and also an applied statistician.
Speaker 1: And this is this is like a pink diamond kind
Speaker 1: of combination. It's that rare to find someone who has
Speaker 1: that kind of technical capability but also understands the pragmatics
Speaker 1: of actually doing data analysis. He spent a lot of
Speaker 1: time as an expert witness. He was the lead statistical
Speaker 1: consultant for the case on census adjustment that went to
Speaker 1: the Supreme Court. In fact, he told me, uh, what
Speaker 1: went that in the end? Uh, you know, the the
Speaker 1: people against adjustment they won in a unanimous Supreme Court decision.
Speaker 1: And David Freeman told me, he said, you know, all
Speaker 1: that work and we only convinced nine people.
Speaker 2: But not nine people that kind of matter, Yeah, exactly.
Speaker 1: So it was just it was a real it was
Speaker 1: kind of a once in a lifetime privilege to get
Speaker 1: to spend time with someone of that intellectual caliber. And
Speaker 1: there were others too. I mean, I've been I've been
Speaker 1: very fortunate that.
Speaker 2: That's quite a list to begin with. Let's talk about books.
Speaker 2: What are some of your favorites and what are you
Speaker 2: reading right now?
Speaker 1: Uh? Well, I'm a I'm a big book reader, so
Speaker 1: I had a long list.
Speaker 2: But probably by the way, this is everybody's favorite section
Speaker 2: of the podcast. People are always looking for good book recommendations,
Speaker 2: and if they like what you said earlier, they're gonna
Speaker 2: love love your book recommendations, so fire away.
Speaker 1: So I'm a big fan of kind of modernist dystopian fiction.
Speaker 1: So a couple of examples of that would be the
Speaker 1: book Infinite Jest by David Foster Wallace, wind Up Bird
Speaker 1: Chronicle by Hirouki Murakami. Those are two of my all
Speaker 1: time favorite books. There's a I think much less well
Speaker 1: known but beautiful novel. It's a kind of academic coming
Speaker 1: of age novel called Stoner by John Williams. A really moving,
Speaker 1: just a tremendous book. Sort of more dystopia would be
Speaker 1: White Noise to Lilo and kind of the classics that
Speaker 1: everybody knows nineteen eighty four and Brave New World. Those
Speaker 1: are two more of my favorite.
Speaker 2: Huh, it's funny when you mentioned The Bear. I'm in
Speaker 2: the middle of reading a book that I would swear
Speaker 2: the writers of The Bear leaned on called Unreasonable Hospitality
Speaker 2: by somebody who worked for the Danny Meyer's Hospitality Group.
Speaker 2: Eleven Madison Park in Ramsey Tavern and all these famous
Speaker 2: New York haunts, and the scene in The Bear where
Speaker 2: they overhear a couple say, oh, we visited Chicago when
Speaker 2: you never had deep dish, So they send the guy
Speaker 2: out to get deep dish. There's part of the book
Speaker 2: where at eleven Medicine Park this people actually showed up
Speaker 2: with suitcases. It was the last thing they would eat
Speaker 2: doing before they heading to the airport. And they said, oh,
Speaker 2: we ate all these great places in New York, but
Speaker 2: we never had a New York hot dog. And what
Speaker 2: do they do. They send them out to get someone
Speaker 2: out to get a hot dog. They played it and
Speaker 2: use all the condiments to make it very special, and
Speaker 2: it looks like it was ripped right out of the Bear,
Speaker 2: or vice versa. But if you're interested in just hey,
Speaker 2: how can we disrupt the restaurant business and make it
Speaker 2: not just about the celebrity chef in the kitchen but
Speaker 2: the whole experience. Fascinating kind of nonfiction book?
Speaker 1: That does sound really interesting.
Speaker 2: Yeah, really, you mentioned the Bear and it just popped
Speaker 2: into my head. Any of the books you want to
Speaker 2: mention that's that's a good list to start with.
Speaker 1: Yeah. My other kind of big interest is science fiction,
Speaker 1: speculative fiction. Unsurprisingly right, Sorry, sorry, but so there are
Speaker 1: some classics that I think everybody should read. Ursula LeGuin
Speaker 1: loves just amazing. So The Dispossessed and The Left Hand
Speaker 1: of Darkness, those are just two of the best books
Speaker 1: I've ever read period.
Speaker 2: Forget Left Handed Darkness stays with you for a long time.
Speaker 1: Yeah right, yeah, really really amazing books. I'm rereading right now,
Speaker 1: Cryptonomicon Neil Stevenson. And one other thing I try to
Speaker 1: do is I have very big gaps in my reading.
Speaker 1: For example, I've never read Updyke, so I started reading
Speaker 1: The Rabbit.
Speaker 2: Serious World of Corn. It's a garb and they're they're
Speaker 2: very much of an era.
Speaker 1: Yeah, that's right.
Speaker 2: What else give us more? Uh?
Speaker 1: Wow? Okay, let's see George so.
Speaker 2: He.
Speaker 1: Oh wow, I think I think you'd love him. So
Speaker 1: He's his real strength is short fiction. He had He's
Speaker 1: written great novels too, but tenth of December this is
Speaker 1: his best collection of of fiction. And that this is
Speaker 1: more kind of modern dystopian, kind of comic dystopian stuff.
Speaker 2: You keep coming back to dystopia, yeasinating.
Speaker 1: I find, you know, it's uh, it's very different from
Speaker 1: my my day to day reality. So I think it's
Speaker 1: a you know, it's a great change of pace for
Speaker 1: me to be able to read this stuff. So, uh,
Speaker 1: some some science writing, I can tell you. Probably the
Speaker 1: best science book I ever read is The Selfish Gene
Speaker 1: by Richard Dawkins, which kind of really you know, you
Speaker 1: have a kind of intuitive understanding of genetics and natural
Speaker 1: selection in Darwin, but the language that Dawkins uses really
Speaker 1: makes you appreciate just how how much the genes are
Speaker 1: in charge and how little we as the as the
Speaker 1: you know he calls he calls organisms survival machines that
Speaker 1: the genes have kind of built and and exist inside
Speaker 1: in order to ensure their propagation. And his whole the
Speaker 1: whole point of view in that book just gives you, Uh,
Speaker 1: it's really eye opening, makes you think about natural selection
Speaker 1: and evolution and genetics in a completely different way, even
Speaker 1: though it's all based on the same kind of facts that.
Speaker 2: You know, it's just framing.
Speaker 1: It's the framing and the perspective that are really that
Speaker 1: really kind of blow your mind. So it's a great
Speaker 1: it's a great book to read.
Speaker 2: Huh, that's a hell of a list. You've given people
Speaker 2: a lot of things to start with, and now down
Speaker 2: to our last two questions, HM, what advice would you
Speaker 2: give to a recent college grad who is interested in
Speaker 2: a career in either investment management or machine learning.
Speaker 1: Yeah? So, I mean I work in a very specialized
Speaker 1: sub domain of finance, So there are a lot of
Speaker 1: people who are going to be interested in investment in
Speaker 1: finance that I that I couldn't give any specific advice to.
Speaker 1: I have kind of general advice that I think is
Speaker 1: useful both for finance and even more broadly. This advice
Speaker 1: is really kind of top of Maslow's pyramid advice. If
Speaker 1: you know, if you're trying to kind of write your
Speaker 1: novel and pay the rent while you get it done,
Speaker 1: this is I can't really help you with that. But
Speaker 1: you know, if what you care about is building this career,
Speaker 1: then I would say number one piece of advice is
Speaker 1: work with incredible people. Like far and away, much more
Speaker 1: important than what the particular field is the details of
Speaker 1: what you're working on, is the caliber of the people
Speaker 1: that you do it with, both in terms of your
Speaker 1: own satisfaction and how much you learn and and and
Speaker 1: all of that. I think you know you'll learn, you'll
Speaker 1: benefit hugely on a personal level from working with incredible
Speaker 1: people and if you don't work with people that are
Speaker 1: like that, then you're probably going to have a lot
Speaker 1: of professional unhappiness. So it's kind of either or.
Speaker 2: That's a really intriguing answer. So final question, what do
Speaker 2: you know about the world of investing, machine learning, large
Speaker 2: language models, just the application of technology to the field
Speaker 2: of investing that you wish you knew twenty five years
Speaker 2: or so ago when you were really first ramping up.
Speaker 1: I think one of the most important lessons that I learned,
Speaker 1: had to learn the hard way kind of going through
Speaker 1: and running these systems, was that it's kind of comes
Speaker 1: back to the point you made earlier about the primacy
Speaker 1: of prediction rules. And it may be true that the
Speaker 1: most important thing is the prediction quality, but there are
Speaker 1: lots of other very necessary, mandatory ingredients, and I would
Speaker 1: put kind of risk management at the top of that list.
Speaker 1: So I think it's easy to to maybe neglect risk
Speaker 1: management to a certain extent and focus all of your
Speaker 1: attention on predictive accuracy. But I think it really does
Speaker 1: turn out that if you don't have high quality risk
Speaker 1: management to go along with that predictive accuracy, you won't succeed.
Speaker 1: And I guess I wish I had appreciated that in
Speaker 1: a really deep way twenty five years ago.
Speaker 2: John, This has been really, absolutely fascinating. I don't even
Speaker 2: know where to begin other than saying thank you for
Speaker 2: being so generous with your time and your expertise. We
Speaker 2: have been speaking with John mccauloff. He is the co
Speaker 2: founder and chief investment officer at the five billion dollar
Speaker 2: hedge fund Volleyon Group. If you enjoy this conversation, well,
Speaker 2: be sure and check out any of the previous five
Speaker 2: hundred we've done over the past nine years. You can
Speaker 2: find those at iTunes, Spotify, YouTube, or wherever you find
Speaker 2: your favorite podcast. Sign up for my daily reading list
Speaker 2: at rid Halts. Follow me on Twitter at Barry Underscore
Speaker 2: Rit Halts until I get my hacked account at rid
Speaker 2: Holt's back. I say that. I say that because the
Speaker 2: process of dealing with the seventeen people left at once
Speaker 2: Twitter now x is unbelievably frustrating and annoying. Follow all
Speaker 2: of the fine family of podcasts on Twitter at podcast
Speaker 2: I would be remiss if I did not thank the
Speaker 2: crack team that helps put these conversations together each week.
Speaker 2: Paris Woald is my producer. Attiko val Bron is my
Speaker 2: project manager. Sean Russo is my director of Research. I'm
Speaker 2: Barry rid Halts. You've been listening to Masters in Business
Speaker 2: on Bloomberg Radio at