Speaker 1: Hello, and welcome to another episode of the Odd Lots Podcast.
Speaker 1: I'm Joe Wisenthal and I'm Tracy Alloway. Tracy, I really
Speaker 1: liked last week's episode with Andrew Lowe talking about quant
Speaker 1: stuff and his sort of evolution of the efficient market
Speaker 1: hypothesis and where that might go. Yeah, I did too.
Speaker 1: I really like the ecosystem analogy, the idea that you
Speaker 1: have all these different players with different motivations and they're
Speaker 1: constantly evolving and adapting to the market. That point about
Speaker 1: it adapting, and of course that's the name of his book,
Speaker 1: Adaptive Market is the name of his hypothesis. Is really
Speaker 1: key because one of the main points that he made
Speaker 1: that I loved was this idea of hedge funds as
Speaker 1: sort of the R and D laboratory for all of
Speaker 1: the financial industry. Right, the funds are where innovative new
Speaker 1: techniques get to be sort of hashed out without doing
Speaker 1: usually too much damage, I guess, to the wider ecosyst
Speaker 1: hopefully right, not always. There's certainly examples of hedge funds
Speaker 1: actually having done major damage from time to time. But ideally,
Speaker 1: you know what the evolution seems to be that some
Speaker 1: new idea sort of starts in the hedge fund world
Speaker 1: and eventually makes its way to the broader world. I
Speaker 1: think the most obvious example of that that we could
Speaker 1: say these days is a lot of this sort of
Speaker 1: popular smart beta strategies e t f s that are
Speaker 1: built on things like momentum or value or other factors,
Speaker 1: sort of quantitative ideas that for many years were only
Speaker 1: available to uh, you know, researchers at hedge funds. Right. So,
Speaker 1: these sorts of quantitative investment or trading methods were usually
Speaker 1: the purview of sophisticated hedge funds who had the time
Speaker 1: and resources to develop them. And then you had a
Speaker 1: bunch of e t f s who kind of caught
Speaker 1: on and managed to replicate them. And now we can
Speaker 1: all trade lighthedge funds for zero percent fees, right, exactly right,
Speaker 1: And of course, once everyone can do it for very
Speaker 1: few fees, I think it's safe to say those strategies
Speaker 1: aren't going to produce the same returns, and hence the
Speaker 1: market is forced to adapt again. Right. Presumably the hedge
Speaker 1: funds are always trying to stay one step ahead as well, right,
Speaker 1: exactly so, which raises the idea of like what will
Speaker 1: be the next thing. If anyone can sort of invest
Speaker 1: in a crude momentum strategy for virtually no fees, then
Speaker 1: that requires the people on the cutting edge, the people
Speaker 1: doing the R and D of this industry, to uh,
Speaker 1: you know, figure out what the next big thing it's
Speaker 1: gonna be. Do you know what the next big thing
Speaker 1: is going to be? Joe? Can you share it with
Speaker 1: your fellow partner at Odd Thoughts LLC? Sadly and unfortunately
Speaker 1: to all of the odd Lots fan, I myself do
Speaker 1: not know what the next big thing in quantitative strategy
Speaker 1: or sort of advanced investing is going to be. But
Speaker 1: I'm hoping that our guests on today's episode might be
Speaker 1: able to shed some light. Who are there? Okay, so
Speaker 1: today we're going to be talking to John Elberg. He
Speaker 1: is the founder of Euclidean Technologies, a quant firm, as
Speaker 1: well as Zach Lipton, a professor at Carnegie Mellon University
Speaker 1: in the Business School and expert on machine learning. They
Speaker 1: recently published a paper titled Improving factor based Quantitative Investing
Speaker 1: by Forecasting company fundamentals. So what I think that means
Speaker 1: and we'll talk to them, is, you know, we talk
Speaker 1: all this stuff about price and computers and algorithms figuring
Speaker 1: out what signal we can get from price, But maybe
Speaker 1: the next generation can actually tell us something about the
Speaker 1: fundamental workings of the company itself, and maybe this could
Speaker 1: be sort of the next wave of where quant investing goes.
Speaker 1: And this sounds absolutely senating, Joe, let's bring them on.
Speaker 1: John and Zach, thank you very much for joining us.
Speaker 1: Was that a reasonable characterization of sort of where your
Speaker 1: paper and where your research is taking things? Yeah, I
Speaker 1: think it is so. So. First of all, machine learning
Speaker 1: has been kind of on a rocket ship of innovation
Speaker 1: for the last ten years or so, and with the
Speaker 1: advent of deep learning, you know, computers and machine learning
Speaker 1: have been able to do things that you know, historically
Speaker 1: have been very challenging, like image captioning and language translation.
Speaker 1: So we Zach and I, you know, a couple of
Speaker 1: years back, thought of the idea of collaborating to apply
Speaker 1: deep learning to the problem of long term investing. So
Speaker 1: how did you actually go about doing that and what
Speaker 1: exactly do you mean by deep learning? That's exactly what
Speaker 1: I wanted to know. To deep learning sort of the
Speaker 1: rebranding of neural networks research to say I say I
Speaker 1: had some data about a company, right like I had
Speaker 1: machine learning. We call a vector of features. But what
Speaker 1: we mean it's just like a list of attributes, each
Speaker 1: of which is somehow like be made into a numerical quantity,
Speaker 1: whether it's like their income, they're the number of assets whatever.
Speaker 1: One way of deciding how to predict what the say,
Speaker 1: what the price will be or something, as we say, well,
Speaker 1: we're going to have this long vector of features, and
Speaker 1: then we're going for every single company, uh, you know,
Speaker 1: at every single time while this vector of features corresponding
Speaker 1: to the state of the company at some period of time,
Speaker 1: and then we'll have some target that we want to predict.
Speaker 1: This could be a binary quantity like will the stock
Speaker 1: go up or down in the next you know, time
Speaker 1: unit of your choice, whether it's the next day or
Speaker 1: in the next month or in the next year. Or
Speaker 1: you could try to directly predict say the relative price,
Speaker 1: so like you know, the percent improvement or decrease based
Speaker 1: on sort of the available features. So one of the
Speaker 1: simplest ways you can make a model is you say, hey,
Speaker 1: I've got a bunch of features. I'm gonna do is
Speaker 1: I'm gonna take a weighted some of these features the
Speaker 1: way like you'd calculate a score to see, like what's
Speaker 1: your risk of a heart disease. Maybe you take you know, well,
Speaker 1: four times your cholesterol plus two times your age minus
Speaker 1: one times you know, your amount of good cholesterol is
Speaker 1: something like this if you come up with some formula
Speaker 1: that's expressed simply as a weighted sum, so that would
Speaker 1: be a linear model. Where deep learning make things different
Speaker 1: is that you have many different layers of computation that
Speaker 1: you basically are learning very complex patterns that maybe couldn't
Speaker 1: be expressed as a as a weighted sum. So maybe
Speaker 1: you're uncovering interactions between all of your features. Um. So,
Speaker 1: for example, if you want to learn to recognize a
Speaker 1: dog versus a cat in an image, there's no weighted
Speaker 1: sum of pixel values it's actually going to tell you
Speaker 1: this because it's just the patterns too complicated. So in
Speaker 1: that case you need some some more like heavy duty machinery.
Speaker 1: So what you do in deep learning essentially is that
Speaker 1: you learn multiple successive transformations of your data such that
Speaker 1: after applying many such transformations, you know, could be two,
Speaker 1: four or five, ten, whatever, you come out at the
Speaker 1: end of a representation of your data where you actually
Speaker 1: can learn a very simple model on top of that.
Speaker 1: So we sometimes call deep learning representation learning because it's
Speaker 1: what we're doing is we're both learning how to feature
Speaker 1: eye our data essentially, how to transform it and how
Speaker 1: to classify it at the same time. So one of
Speaker 1: the things in sort of traditional quantity, a lot of
Speaker 1: quantitative investing focuses a lot on price and sort of
Speaker 1: listening to your characterization. It seems like price and this
Speaker 1: is relatively speaking, of course, price is a fairly you know,
Speaker 1: it's sort of easy idea to capture. So you can
Speaker 1: come up with some definition of what momentum is and
Speaker 1: then sort of say, okay, these stocks are experiencing momentum
Speaker 1: right now, or these stocks aren't, and then his history
Speaker 1: tell us the stocks are going to do next if
Speaker 1: they sort of meet these characterizations. Your paper really looks
Speaker 1: at what can you do with this technology for sort
Speaker 1: of looking at future fundamental so looking at the sort
Speaker 1: of characteristics of the company and not just trying to
Speaker 1: see where prices going, but where those characteristics are going,
Speaker 1: so explain sort of what your research specifically attempts to uncover.
Speaker 1: So one thing that deep learning allows a researcher to
Speaker 1: do is look at kind of more raw features um.
Speaker 1: Like Zach explained in the image case, you're looking at
Speaker 1: raw pixels. Now, if you think about most quant funds
Speaker 1: and most quant models, they the features that go into
Speaker 1: the model are highly engineered, and they include things like
Speaker 1: price and maybe book value, price divided by book value,
Speaker 1: price divided by earnings, and then maybe some momentum features.
Speaker 1: The interesting thing about deep learning is it allows you
Speaker 1: to potentially let it uncover what the best features are.
Speaker 1: If you over engineer features, you may not find the
Speaker 1: ones that are best to predict what you're interested in predicting.
Speaker 1: So that, you know, allows you to potentially find features
Speaker 1: in the data that you wouldn't find through which traditional
Speaker 1: feature engineering process. Yeah, and you know, to directly address
Speaker 1: your question, your point is that the very most obvious
Speaker 1: thing you could say, now, if I have this, I
Speaker 1: have this learning machine, I have a bunch of features,
Speaker 1: and I have to choose what am I going to predict?
Speaker 1: The very most obvious thing to try to predict is
Speaker 1: the price, because if you can actually do that perfectly,
Speaker 1: then you're done, right. If if you actually know which
Speaker 1: way the price is going to move in the next year,
Speaker 1: then you can make the perfect choice. So the problem
Speaker 1: is that that's that's not so easy because the markets
Speaker 1: are quite capricious, right, Um, So one problem that we
Speaker 1: found is we actually did these models where we were
Speaker 1: trying to predict price directly. But among the other things
Speaker 1: that you have is that one, it's hard to learn
Speaker 1: models that do a good job of this that are
Speaker 1: sort of robust across different time periods. So you might
Speaker 1: have like, hey, I'm going to train on these like
Speaker 1: decades of data and I'm going to try to directly
Speaker 1: predict the price. But then I come into periods of
Speaker 1: time where the markets behaving a little bit differently, and
Speaker 1: we call this nonstationarity. Basically, like you're modeled, there's a
Speaker 1: great job of uncovering the pattern that's present in the
Speaker 1: data that you gave to the model, but that data
Speaker 1: is anchored to some period of time, and the future
Speaker 1: data that comes in, you know, the patterns changed a
Speaker 1: little bit, and so the kind of like function that
Speaker 1: you've learned no longer does a great job. So so
Speaker 1: what we do instead of directly trying to predict the price,
Speaker 1: the idea that we had was to think, well, this
Speaker 1: core idea behind a factor model, generally right, is to
Speaker 1: just say, hey, I'm going to sort all the stocks
Speaker 1: according to some reason idea, Hey, the price of the
Speaker 1: company should be tied to its income, any company, and
Speaker 1: somehow it is justified by like it's the long term
Speaker 1: discounted cash as well. Let's just say a factor strategy
Speaker 1: just something very simple. It says, well, let's just look
Speaker 1: at the current income divided by say the current price
Speaker 1: or current income divided by the current you know, market
Speaker 1: cap or enterprise value, some some notion of income and
Speaker 1: some notion notion of financial performance, and divided by some
Speaker 1: notion of company size and this, and then I'm going
Speaker 1: to sort the stocks according to this. The ones that
Speaker 1: come out highest are like most cheaply priced, so let's
Speaker 1: buy those. So the ideas to say, hey, well, what
Speaker 1: if I told you so. We actually know that this
Speaker 1: does pretty well in back testing whether or not the
Speaker 1: patterns will hold in the future. But you know, many
Speaker 1: people have made a lot of money for many years,
Speaker 1: so there's an idea of if you knew the income,
Speaker 1: this is a good thing, a reasonable thing to try
Speaker 1: to do. Our question that we asked, Unfortunately, um, John,
Speaker 1: because he's actually in finance and I'm not, has this
Speaker 1: really great set of like industry grade tools that unlike
Speaker 1: most academic papers that look at like one stock over
Speaker 1: a short period of time or something, we actually had,
Speaker 1: you know, forty plus years of financial data and can
Speaker 1: actually simulate like an applausible that guess what's going on.
Speaker 1: We said, well, what if you did a factor model,
Speaker 1: but someone gave you a crystal ball. So basically, instead
Speaker 1: of dividing the current income divided by the current enterprise value,
Speaker 1: someone gave you next year's income, and so you sorted
Speaker 1: the stocks according to next year's income divided by the
Speaker 1: current enterprise value something like this. So you're you're able
Speaker 1: to peek into the future. You know how the company
Speaker 1: will be performing next year, and you're saying, is how
Speaker 1: is its next year's performance? Is that based on next
Speaker 1: year's performance? Is its current price? Is it currently priced
Speaker 1: cheaply or not? So it's what we call like a
Speaker 1: clairvoyant factor model. Like you don't actually have such a
Speaker 1: crystal ball, but if you, you know, give us some
Speaker 1: license and you imagine that you did what would have
Speaker 1: happened if you went back in history and you had
Speaker 1: this crystal ball and you traded based on a clairvoyant
Speaker 1: factor model, and it turns out that the clairvoyant factor
Speaker 1: model just crushes it. So it does really, really well
Speaker 1: and and not surprisingly, the more clairvoyant the model is.
Speaker 1: So if it if it knows the performance of the
Speaker 1: company six months out versus now, or twelve months out
Speaker 1: versus six months out, it keeps getting better and better
Speaker 1: and better. So what we decided was, well, maybe trying
Speaker 1: to predict price directly as a bit you know, subject
Speaker 1: to you know, a kind of fickle market, but the
Speaker 1: patterns present in the fundamental reporting data itself is more stable.
Speaker 1: So in our method what we do is instead of
Speaker 1: just trying to predict a return, we try to predict
Speaker 1: actually the fundamental reporting data itself, just so we're given
Speaker 1: um these these features for like a trailing window of
Speaker 1: of time corresponding to the company's like financial reporting, and
Speaker 1: then we're trying to predict what they're going to report
Speaker 1: next year. And then based on what they're going we
Speaker 1: think they're going to report next year, we sort the
Speaker 1: companies according to a value factor. So in essence, you
Speaker 1: can pick out of that future prediction the components of
Speaker 1: the factor model. Let let it whether it's a few
Speaker 1: future predicted earnings, and you can take that out of
Speaker 1: the future predicted fundamentals, divide that by current enterprise value
Speaker 1: and and sort and then you have basically a factor
Speaker 1: model which you are using. Instead of trailing twelve months earnings,
Speaker 1: you're using the future predicted earnings by the deep learning
Speaker 1: the deep neural network. So, as I understand it, the
Speaker 1: deep learning or the neural networks are used primarily to
Speaker 1: forecast the future fundamentals based on historic performance. Is that right,
Speaker 1: historic fundamentals? Yeah, okay, So walk us through how you
Speaker 1: actually develop an application that's able to do that, Like
Speaker 1: what are those neural networks looking at and what sort
Speaker 1: of information are they drawing in other than you know,
Speaker 1: past predictive data to make those forecasts. There's two parts
Speaker 1: of that. One is the data that we use, and
Speaker 1: then two is the technology we use to build the
Speaker 1: deep you know, neural network models. So on the data side,
Speaker 1: what you use is historical fundamentals on all companies you
Speaker 1: know that have ever you know, been listed in the
Speaker 1: US for the past fifty years, and so what a
Speaker 1: historical fundamentals mean? What it means earnings, book value, anything
Speaker 1: you can find on an income statement and balance sheet
Speaker 1: going back in time. In addition to fundamentals, we also
Speaker 1: use as inputs to the to the model, you know,
Speaker 1: momentum over you know, one month, six months, twelve months.
Speaker 1: So then you know, if you think of it as
Speaker 1: like a big you know, spreadsheet table where each row
Speaker 1: is a point in time for a specific company, and
Speaker 1: then you can think of sequences going back through time.
Speaker 1: You know IBM in March of and then all of
Speaker 1: its fundamentals in one row, plus it's moment at them,
Speaker 1: and then that going back five years and time. So
Speaker 1: those sequences, both the fundamentals and the momentum are fed
Speaker 1: into a neural network and uh and and all of
Speaker 1: those sequences for all companies and all time are fed
Speaker 1: into a neural network and are trained to predict what
Speaker 1: the fundamentals will be, you know, one time step out
Speaker 1: in the future. So just to sort of summarize it
Speaker 1: all up, you know, it's like, if you have all
Speaker 1: these strategies, if you have all these funds chasing things
Speaker 1: like earnings, quality, earnings, growth, momentum, all kinds of stuff
Speaker 1: like that, your goal is to anticipate today when those
Speaker 1: funds are going to be buying tomorrow. Is that a
Speaker 1: fair way to characterize it. I think that's a fair
Speaker 1: way to characterize is it. I think what we're really
Speaker 1: just doing is trying to build a better a better
Speaker 1: factor model, A better factor model in the sense that
Speaker 1: you know, as Zach explained, if you had a clairvoyant
Speaker 1: model where you actually knew what future fundamentals were and
Speaker 1: could plug that into a factor model, you do substantially
Speaker 1: better than what you could achieve with a value factor model. Today,
Speaker 1: we're not like directly considering the psychology of the other
Speaker 1: players in the market in this particular approach, right, No, sure,
Speaker 1: but it's essentially saying, like, maybe the way to characterize
Speaker 1: it is, if you want to invest on some fundamental
Speaker 1: factor like earnings quality or earnings growth, bottom line, is
Speaker 1: better to look at future twelve month results rather than
Speaker 1: trailing twelve months. You look at the trailing, but you're
Speaker 1: trying to predict the future. Like, so those two components, right,
Speaker 1: you could say, like one is we have the component
Speaker 1: that is trying to predict the future fundamentals. You know,
Speaker 1: imagine that I came for the future, and I got
Speaker 1: out of my time machine, and I gave you the
Speaker 1: earnings reports from the future. Right, So so the first
Speaker 1: thing you need is how do I get an approximate
Speaker 1: time machine, right, which in our case is a predictive
Speaker 1: model that has a good guess about what the future
Speaker 1: will look like. The second thing is you still need
Speaker 1: a way of executing on the strategy ones I. You
Speaker 1: still need a way to decide which stocks to buy, right, So,
Speaker 1: based on based on this future information, Like, it's possible
Speaker 1: that if I if I come from the future and
Speaker 1: I give you the earnings report, and I tell you
Speaker 1: what the future income will be, what, it's possible that
Speaker 1: the income is going to go up with the stock
Speaker 1: price is going to go down, you know, like say
Speaker 1: it's an Apple and like they made a lot more money,
Speaker 1: but it was also like announced that they had a
Speaker 1: major plant failure in the iPhone fourteen or whatever they're
Speaker 1: up to is going to be delayed. So these two
Speaker 1: components are are a little bit modular, Like we could
Speaker 1: come up with m. John I think is more the
Speaker 1: domain expert, so I'm I'm more the machine learning guy.
Speaker 1: Like I'm sure John could come up with you know,
Speaker 1: a million other ways that you might imagine that someone
Speaker 1: would try to execute on this information. In our case,
Speaker 1: what we're doing is we've adapted a factor model to
Speaker 1: work with this kind of future guess. So one other example,
Speaker 1: so so again, in our case, what we're doing is
Speaker 1: taking the predicted future fundamentals and feeding that into a
Speaker 1: value factor model. But you could imagine using let's say
Speaker 1: the deep neural networks said, you know, a company is
Speaker 1: going to do a hundred million, but consensus estimates in
Speaker 1: in in earnings. Let's say, but consensus estimates said it's
Speaker 1: gonna do seventy five million in in earnings. Well, you
Speaker 1: know that might be you could you can imagine devising
Speaker 1: a strategy around that where you'd want to go, you know,
Speaker 1: bet on those guys and ones where consensus estimates are
Speaker 1: above what the deep neural network is predicting, you'd want
Speaker 1: to bet against. Right, assuming the current price is pricing
Speaker 1: and that that's a really you know, John, you shouldnt
Speaker 1: give away so goods, that that's a really good idea.
Speaker 1: So are these kinds of machine learning driven predictive models
Speaker 1: the future of investing? You think is that the way
Speaker 1: that we're heading. I think what this paper showed is
Speaker 1: that there's a lot of potential in using deep learning
Speaker 1: to long term investing. I think that there's been some
Speaker 1: debate about whether, you know, deep learning, which requires a
Speaker 1: lot of data um to to to build successful models,
Speaker 1: um whether in finance there's enough data, or whether you
Speaker 1: even need this these kinds of complex models in finance,
Speaker 1: I mean a lot of quant people feel, you know,
Speaker 1: linear simple factor models are the best route to go,
Speaker 1: And I think what we showed here is that if
Speaker 1: you're trying to predict price changes, that might be true.
Speaker 1: But if you decompose the problem into first trying to
Speaker 1: predict fundamentals and then later you know, through a factor
Speaker 1: model or some other method, trying to use those predicted
Speaker 1: fundamentals to predict price, deep learning has a lot of
Speaker 1: potential and does does substantially better at predicting future fundamentals
Speaker 1: than than what you could do with a linear model.
Speaker 1: There's a sort of a technical reason to recommend the
Speaker 1: way we've cast a problem also without going too far
Speaker 1: into the weeds. Basically, uh, you think really really powerful
Speaker 1: machine learning models, deep neural networks. The thing that you
Speaker 1: worried about is John was talking about how people people
Speaker 1: agonize over what can you bring us to bear on
Speaker 1: long term investing because you don't have as much data
Speaker 1: right as if you were looking at the you know,
Speaker 1: micro second kind of trade frequency, then you'd have, you know,
Speaker 1: trillions of trade examples or something you get on. But
Speaker 1: if you if you're looking at you know, your your
Speaker 1: time tick is I have a data point you know,
Speaker 1: once per month or once per year suddenly, and I
Speaker 1: only have thousands of stocks, not millions of stocks. You
Speaker 1: don't have such a huge amount of data. Um, So
Speaker 1: what you worry about is that a model given given
Speaker 1: a super powerful model, like a super overpowered model, and
Speaker 1: then not too much data, that there's a propensity for
Speaker 1: the models to do what we call overfitting, which is
Speaker 1: the model basically it does a really good job of
Speaker 1: memorizing the training data it's seen, but it learns kind
Speaker 1: of a spurious pattern that doesn't generalize to future data
Speaker 1: that it hasn't seen. So one cool thing about the
Speaker 1: way that we're casting the problem is that we're not
Speaker 1: just trying to predict the factor of interest. We're actually
Speaker 1: trying to predict all the factors in the future. And
Speaker 1: this means that the model has to simultaneously get the
Speaker 1: income right, and get the assets right, and get the
Speaker 1: debt right, and get all these different factors that are available.
Speaker 1: So John was a fifteen target factors that we have
Speaker 1: that we're trying to predict. So so in this case,
Speaker 1: this sort of like this is this is what we
Speaker 1: call multitask learning and the machine learning literature. And one
Speaker 1: nice effect of multitask learning as that has a generalization
Speaker 1: effect in that it's it's harder to fit a spurious
Speaker 1: hypothesis because you have to come up with a representation
Speaker 1: that is good for task one and also good for
Speaker 1: task two, and also good for tast three. And the
Speaker 1: probability that you come up with a pattern that's that's
Speaker 1: good for solving all of these tasks that is not
Speaker 1: the true pattern is much smaller than if you're only
Speaker 1: like trying to solve one task, where it's easier to
Speaker 1: just kind of memorize those data points. So we have
Speaker 1: like essentially sixteen times as much training data and in
Speaker 1: some relevant sense, so I have to ask in the
Speaker 1: abstract of your paper or in the intro, you say that, um,
Speaker 1: with this approach, you can improve your annual returns pretty
Speaker 1: substantially over a standard factor model. In a bad test.
Speaker 1: Seventeen point one percent versus a fourteen point four percent,
Speaker 1: just pretty big beat. But as we know, and as
Speaker 1: there's a lot of people pointed out, there's a lot
Speaker 1: of strategies that seem to work in academic papers and
Speaker 1: then when they're put into pract is, they don't seem
Speaker 1: the results don't seem to arrive as easily. John, in
Speaker 1: your firm, are you seeing the results of your research
Speaker 1: that on paper look very compelling actually play out in
Speaker 1: the market. So so this this paper, uh, we we
Speaker 1: have not put this model to test, so to speak,
Speaker 1: in in a fund yet, but you know, we're very
Speaker 1: interested in in in doing that. I will I will
Speaker 1: add though here that many of the back tests that
Speaker 1: are done in the industry are done where you just run,
Speaker 1: you know, thousands of back tests on a data set
Speaker 1: over some time period ten years, twenty years, thirty years,
Speaker 1: and there's no out of sample testing, meaning that they
Speaker 1: don't then take that and then apply it to a
Speaker 1: new data set. One thing that machine learning. One technique
Speaker 1: that is used in machine learning to prevent overfitting, and
Speaker 1: that we do here is we train them or we
Speaker 1: build the model on one data set and then test
Speaker 1: it at a sample on another on another data set
Speaker 1: during a different time period, and the results we present
Speaker 1: there are at a sample out of sample, always being
Speaker 1: sort of ahead right in the future, you could. So
Speaker 1: the model is the model at every given time. It
Speaker 1: is trained on the path data. So we're simulating like
Speaker 1: what if, you know, if you train the model back
Speaker 1: then based on that it was only available up to
Speaker 1: that point. I think more broadly, there's a good question
Speaker 1: there of um, it's hard to say which which patterns
Speaker 1: are just you know, especially I think with short term investing,
Speaker 1: it's very obvious that any any pattern that exists on
Speaker 1: a scale of seconds is something that could be sort
Speaker 1: of traded away. It's not as clear and I believe,
Speaker 1: I mean, John can speak more to it, right, But
Speaker 1: I believe part of the ethos of long term investing
Speaker 1: is very much that rather than interacting in a place
Speaker 1: where the price most price movements are due to the
Speaker 1: behavior of the high frequency traders, when you're in the
Speaker 1: long term space, the price movement is more tied to
Speaker 1: the actual fiscal performance of the company, and that's maybe
Speaker 1: a more durable pattern. So Joe and I were talking
Speaker 1: about financial players and how quickly they adapt to new
Speaker 1: markets and new situations. At the beginning of this episode,
Speaker 1: from your respective viewpoints, how fast are these sorts of
Speaker 1: technologies and models and applications being developed, And for how
Speaker 1: long would something like, you know, a clairvoyant factor predicting
Speaker 1: model actually give you an edge four until someone else,
Speaker 1: maybe an E t F came along and copied it.
Speaker 1: I think that's a hard question to answer, because again
Speaker 1: it gets back to how you would use this model. Right,
Speaker 1: So in the in the paper we give one very
Speaker 1: specific example. We use the deep learning normal network to
Speaker 1: predict fundamentals and then we plug that into one kind
Speaker 1: of factor model, right uh, in particular operating income predicted
Speaker 1: operating income over enterprise value. But as I suggested, you
Speaker 1: could use it to you know, figure you know, figure
Speaker 1: out whether consensus forecasts are good or bad. Um. So,
Speaker 1: you know, I think that just saying in general, deep
Speaker 1: learning applied to you know, investing is going to get
Speaker 1: used and then a year later is going to be
Speaker 1: arbitraged away. Miss is the point that, look, you know,
Speaker 1: you can use deep learning in a myriad of ways
Speaker 1: to attack the problem of long term investing and presumably
Speaker 1: trading as well. To address your question about how quickly
Speaker 1: is this kind of technology getting adopted? UM my sense
Speaker 1: and based a little bit on an outsider's view as
Speaker 1: an academic machine learning person, um talking to collegues who
Speaker 1: have either gone into fintech or who flirted with it
Speaker 1: or tried to recruit me into it. The sense that
Speaker 1: I get is that actually, obviously a lot of people
Speaker 1: aren't talking about what they're doing, right, But my sense
Speaker 1: is that there's a lot of people doing this kind
Speaker 1: of stuff in the high frequency space, not maybe on
Speaker 1: the scale of you know, fractions of seconds, but but
Speaker 1: on on a pretty short time scale. And the reason
Speaker 1: why is because, um, right, it's it's easy to collect
Speaker 1: a lot of data. If the patterns are very different, um,
Speaker 1: a year from now, Well, you just you have enough data.
Speaker 1: Like if you're trading like at the scale of months
Speaker 1: or years, then you have to look back twenty years, right,
Speaker 1: you have to look back thirty If you're trading at
Speaker 1: the scale seconds, then your whole universe could be formed
Speaker 1: by the previous four days. There's a very fast cycle
Speaker 1: of development. So if you're in it and you just
Speaker 1: wanna you don't you don't have any kind of strong
Speaker 1: beliefs about finance. You're just a machine learning person throwing
Speaker 1: your hammer at financing. Then going in the high frequency
Speaker 1: space gives you or are the comparatively high frequency space
Speaker 1: gives you like the sandbacked box to just really quickly
Speaker 1: test stuff validated, see if it works. My feeling and
Speaker 1: when I've talked to friends who are doing this kind
Speaker 1: of stuff, but what we're doing is that I think
Speaker 1: almost no one that I've talked to out of a
Speaker 1: lot of people doing this stuff with finance, is looking
Speaker 1: at the same kinds of time scales. And John might
Speaker 1: be able to to speak to that because he might
Speaker 1: actually be deeper and the I mean, he's definitely differ
Speaker 1: in the finance community than I am. But my sense
Speaker 1: is people doing deep learning for finance, and there are many,
Speaker 1: Um it's on the rise, but they're not necessarily looking
Speaker 1: at it in the same way, and certainly very few
Speaker 1: on as long a time scale. Yeah, I mean, I
Speaker 1: think if you look, you know, the a q r
Speaker 1: s and the d f as of the world, which
Speaker 1: are you know, these huge you know, quantitative shops. They do.
Speaker 1: They certainly do long term investing, but um, there's not
Speaker 1: a lot of evidence that there's a ton of machine
Speaker 1: learning deep learning going on there. But you know, I
Speaker 1: think if if if stuff is a successful, you know
Speaker 1: it's likely to be a opted, so probably won't be
Speaker 1: true forever. John Elberg and Zachary Lipton, that was a
Speaker 1: fascinating conversation, so much to think about and wrap our
Speaker 1: heads around. Really appreciate you both coming on. Thanks for
Speaker 1: having us. Thank you guys. Well, Tracy, we didn't really
Speaker 1: plan it that way, but I really do think that
Speaker 1: was sort of the perfect follow up to Andrew Lot left. No, Joe,
Speaker 1: you're supposed to pretend we did plan it that way
Speaker 1: so everyone will think we're really working so good. I mean,
Speaker 1: like we should continue this series on quantitative strategies and
Speaker 1: new ways to evolve to beat the market. Let's continue
Speaker 1: this continue. Yes, absolutely, Okay, in all seriousness, Yes, it
Speaker 1: was fascinating. I really like the idea of well, who
Speaker 1: doesn't like the idea of a clairvoyant robot who can
Speaker 1: predict how well a company is going to do in
Speaker 1: the future and then apply that to a factor based
Speaker 1: investment model. If someone comes back in time and they're
Speaker 1: like giving me hints on what the stock market is
Speaker 1: gonna do, it's like just give me the winning stocks.
Speaker 1: You know what I'm saying, Like, if you're time traveling,
Speaker 1: don't like be a tease, just give me the winning stocks. No,
Speaker 1: But in all seriousness, hey, I felt like several times
Speaker 1: in that conversation, it's just like the level that they're
Speaker 1: operating and thinking about the market on is so like
Speaker 1: high above anything that you and I like typically talk
Speaker 1: about it on a day. Like several times I felt
Speaker 1: like I had to catch my breath speak for yourself, Joe,
Speaker 1: because it's just like absorbing all of that, and you know,
Speaker 1: obviously there's probably lots that I didn't get. But then
Speaker 1: the other thing I really thought that last point was
Speaker 1: very interesting about time. So obviously going back to the
Speaker 1: adaptive framework for thinking about markets. You know, if there
Speaker 1: is a strategy that works over a day and you
Speaker 1: can get it, and you can you just have to
Speaker 1: back test four days or whatever, it's very easy to see, Okay,
Speaker 1: this works, this doesn't. Let's go with the thing that
Speaker 1: works and then everyone can sort of figure out the
Speaker 1: things that work and then it doesn't work anymore. But
Speaker 1: this idea that maybe a quantitative approach to long term
Speaker 1: fundamental investing, you don't get that sort of instant feedback
Speaker 1: on whether it's working as fast, and so maybe winning
Speaker 1: strategies might prove to be a bit more do Yeah,
Speaker 1: and presumably it's much more difficult to actually develop them
Speaker 1: and see them evolved. It's like, um, I guess it's
Speaker 1: like if you bred successive generations of like rabbits, right,
Speaker 1: Like it takes like a year two, well less than
Speaker 1: a year, you could breathe like a hundred generations in
Speaker 1: a year. Yeah, right, Or I guess, like, you know,
Speaker 1: it's like laboratories use mice and because they do, like
Speaker 1: they can get so many so fast. But if you
Speaker 1: had to sort of, you know, breed hippopotamus is you
Speaker 1: wouldn't know for a much longer period of time whether
Speaker 1: you're down the road you had sort of created the
Speaker 1: master hip hop. Does that make sense? Wow? This past? Yeah, Okay,
Speaker 1: let's leave it at master hippo. Okay, alright. This has
Speaker 1: been another episode of the Odd Thoughts Podcast. I'm Tracy Alloway.
Speaker 1: You can follow me on Twitter at Tracy Alloway and
Speaker 1: I'm Joe Why Isn't All? You can follow me on
Speaker 1: Twitter at the Stalwart, and you can follow our guests
Speaker 1: on Twitter John Elberg is at at John Elbert. There's
Speaker 1: just one L in that. Zachary Lipton is on Twitter
Speaker 1: at Zachary Lipton, and you can follow our producer Sarah
Speaker 1: Patterson on Twitter at Sarah pat with two teams. Thanks
Speaker 1: for listening,