Speaker 1: Hello, and welcome to another episode of the Odd Lots podcast.
Speaker 1: I'm Joe Wisn't All and I'm Tracy Halloway. Tracy, you
Speaker 1: know what the funny thing is is that even though
Speaker 1: it's been an incredible year in the stock market, I
Speaker 1: mean just extraordinary biole accounts as everyone knows, I feel
Speaker 1: like it's also probably been a frustrating one for a
Speaker 1: lot of investors. Oh yeah, for sure. I mean, first
Speaker 1: of all, markets didn't really do what a lot of people,
Speaker 1: I guess would would say they should do rationally in
Speaker 1: the face of the biggest economic crisis in decades. But
Speaker 1: I feel like a lot of people just sort of
Speaker 1: missed various turning points in the market as well, and
Speaker 1: are very very frustrated. Absolutely, I mean just super super high,
Speaker 1: super high levels of frustration. Also, even if you were
Speaker 1: along this market and sort of like generally bullish, the
Speaker 1: only way to have really won this year would be
Speaker 1: super concentration in tech stocks. And I feel like if
Speaker 1: you were under exposed to like a handful of tech stocks,
Speaker 1: which we could count down about two hands, then you're
Speaker 1: almost guaranteed to be sort of underperforming your benchmark this year,
Speaker 1: whatever it is Yeah, I think that's absolutely true. And
Speaker 1: of course we've been talking about for years and years
Speaker 1: and years that the big Tex stocks, saying whatever you
Speaker 1: want to call it, are potentially overvalued. So it's it's
Speaker 1: doubly ironic that this year you would have underperformed have
Speaker 1: you not invested in the stocks that people say it
Speaker 1: might be the most overvalued? Right, And of course that
Speaker 1: is a big frustration to investors who have been waiting
Speaker 1: a long time for other sort of factors to do well.
Speaker 1: So investors like to talk into factors and this sort
Speaker 1: of the growth factor has done phenomenally well, but historically
Speaker 1: the value factor, so called cheaper stocks, those have done well,
Speaker 1: and everyone keeps waiting for this turn or for other
Speaker 1: factors to immerge, whether it's value or low beta or
Speaker 1: something else. Uh, never seems to happen. And if anything,
Speaker 1: this year did not prove to be a turning point
Speaker 1: in the market, but really just sort of an accelerant
Speaker 1: of it. Yeah, I think that's right. I'm actually looking
Speaker 1: at a chart from Big of America Merrill Lynch right now,
Speaker 1: and they point out that values relative performance to growth
Speaker 1: was the worst this year since the dot com bubble,
Speaker 1: So um something to remember, But we're not. This isn't
Speaker 1: this podcast isn't about value versus growth? Is it? No,
Speaker 1: it's not. But I think that the frustration that people
Speaker 1: probably have this year does lead to um, you know,
Speaker 1: people looking for other approaches to investing, and of course
Speaker 1: in times like this, people wonder if, like maybe other
Speaker 1: sort of quantitative or algorithmic strategies more money should be
Speaker 1: poured into them as an alternative to this ride where
Speaker 1: you just sort of by the big tech s docks
Speaker 1: and I hope that you you know, avoid the turning point. Well,
Speaker 1: I guess another way of putting it is a lot
Speaker 1: of the a lot of the quant strategies are sort
Speaker 1: of momentum based, right, So if you can figure out
Speaker 1: where the money is flowing to, even if it's tex stocks, Uh,
Speaker 1: that might be a good way of investing in the
Speaker 1: current environment. If everything's about liquidity and following the flows,
Speaker 1: then quant investing or algorithmic trading, whatever you want to
Speaker 1: call it, might be a good way forward. Yeah. But
Speaker 1: you know, backing up, it's like we talk about quant
Speaker 1: investing and the word quant gets used all the time,
Speaker 1: and sometimes uh, it's used to describe these super technical funds,
Speaker 1: and sometimes it gets used to just describe sort of
Speaker 1: anything that has some statistical analysis of it. That that
Speaker 1: term feels extremely vague. Yeah, and potentially overused as well. Right,
Speaker 1: Like everyone wants to seem like they are quantitative in
Speaker 1: some way or another. No one wants to say that
Speaker 1: they're investing purely on emotion and gut feeling and that
Speaker 1: kind of stuff. So quant gets bandied about quite a bit.
Speaker 1: So today we are going to talk with an expert
Speaker 1: who is ah, knows a lot about quant investing studies
Speaker 1: that can help us define it and uh also hopefully
Speaker 1: sort of explain to us what it takes to win
Speaker 1: in this space, because again, everyone sort of wants to
Speaker 1: be in the space, even you know, traditional hedge funds
Speaker 1: over the years have allocated more and more money to quant,
Speaker 1: to hiring PhD s, to building up their computer systems,
Speaker 1: But what it really takes to win and can lots
Speaker 1: of players succeed is still uh kind of an open question. Yeah,
Speaker 1: I think that's exactly right. And as we're going to discuss,
Speaker 1: quant investing is probably one of the most expensive ventures
Speaker 1: that you can sort of embark on. Yes, Okay, So
Speaker 1: without further Ado. Let's bring in our guest. He is
Speaker 1: an expert in the field. He is ciamac Millmy. He
Speaker 1: is a professor of business. He's a professor at the
Speaker 1: Columbia Business School, done a lot of research in the
Speaker 1: area of quant investor. He's also a part time partner
Speaker 1: at a fund himself. Thank you very much for joining us.
Speaker 1: Thanks for having me. I'm delighted to be here. When
Speaker 1: I say quant investing or when people say quant investing,
Speaker 1: what does that mean to you? Like, how would you
Speaker 1: just define that term and so that it's a useful
Speaker 1: so that it's a useful term. Well, people have different definitions.
Speaker 1: I personally define it as having two key characteristics. Um.
Speaker 1: The first characteristic is that the investment process is entirely systematic.
Speaker 1: So there's many different times of types of investment strategies
Speaker 1: that the people implement that employ at some level quantitative methods.
Speaker 1: But I think the key to the quantitative methods that
Speaker 1: we're going to speak about today is that at the
Speaker 1: trade by trade level, there is no discretion. Right. Um,
Speaker 1: You've you've set up an algorithm, a particular system on
Speaker 1: a you know, second by second trade by trade basis,
Speaker 1: the everything is being automatically done. You know. That isn't
Speaker 1: to say that there isn't like a portfolio manager involved.
Speaker 1: But the job of the portfolio manager is not so
Speaker 1: much deciding on trades and sizing them and so on,
Speaker 1: but more setting up the computer algorithms in advance and
Speaker 1: tweaking them and improving them over time. So so that's
Speaker 1: really the first big component to be entirely systematic nondiscretionary.
Speaker 1: The second component of the ones that I focus on
Speaker 1: is that they're really active investment strategies in the sense
Speaker 1: that you're buying now because you think the asset will
Speaker 1: be worth more later it's it's mispriced in some level,
Speaker 1: or alternatively, you're selling short now because you think the
Speaker 1: value later will be will be lower. There are other
Speaker 1: flavors of quantitative strategies that are somewhat more passive, things
Speaker 1: like uh, you know, exotic beta, investing in factors and
Speaker 1: so on. Um, those are not so much a little
Speaker 1: bit less my area, and I have my own views,
Speaker 1: and then we can get into later perhaps, But the
Speaker 1: key things I'm thinking about here, you're using algorithms and
Speaker 1: data and machine learning and so on. You're taking an
Speaker 1: active view on what the current prices are relative to
Speaker 1: what you know the value might be later. So is
Speaker 1: quant investing proof that markets aren't efficient? I feel like
Speaker 1: this comes up a lot, but maybe it's worth asking
Speaker 1: this question early on. If the whole strategy is to
Speaker 1: automatically arbitrage price discrepancies in the short term versus the
Speaker 1: long term, does that mean that markets aren't doing their job? Well?
Speaker 1: I mean, I think if you want to sort of
Speaker 1: take the straw man that the markets are, you know,
Speaker 1: sort of a hundred percent efficient and prices are incorporating
Speaker 1: all potential information, I think that's clearly not true. And
Speaker 1: I think, um, the long term success uh and incredible
Speaker 1: performance of you know, quant investors like Renaissance is is
Speaker 1: sort of one piece of that um. But that doesn't
Speaker 1: mean that the markets are completely uh inefficient either. LASSA
Speaker 1: Peterson who's from from n y U and a q R.
Speaker 1: He has that he has a nice phrase called inefficiently efficient,
Speaker 1: or I should say efficiently inefficient, meaning that there are inefficiencies,
Speaker 1: but it's a competitive game and there are lots of
Speaker 1: smart people with you know, a lot of resources going
Speaker 1: after these inefficiencies, and when you identify them and trade
Speaker 1: on them, Um, they disappear, they're armed away. So you know,
Speaker 1: these these inefficiencies typically lie around the frontier of the
Speaker 1: transaction costs, of what it costs to trade. So, um, yes,
Speaker 1: there are inefficiencies, but they're they're hard to find, and
Speaker 1: you know, they disappear over time. So one common concept
Speaker 1: that quants talk about is is alpha decay. Like you
Speaker 1: you identify some some signal or some inefficiency and uh,
Speaker 1: you know, generates a certain amount of P and L
Speaker 1: and literally year over year you can see that decay away.
Speaker 1: And you know that's that's because that inefficiency eventually is
Speaker 1: identified by other people and as more and more people
Speaker 1: trade on it, you know again it disappears. So it's
Speaker 1: not that um, you set up an algorithm and it
Speaker 1: just you know, sort of prints money. Um uh you know,
Speaker 1: some sort of gross violation of the the efficient markets
Speaker 1: hypoth is That's that's not how it works. The people
Speaker 1: who are successful at this are constantly investing and deploying
Speaker 1: enormous resources, hiring large numbers of PhD s, and uhum,
Speaker 1: progressively innovating in order to have new models because because
Speaker 1: the old stuff will simply stop working. So it sounds
Speaker 1: like I mean I guess you just said it, but
Speaker 1: it sounds like the key to winning. And we'll get
Speaker 1: more granular in a second. Is that continuous process. It's
Speaker 1: not about identifying some flaw in the market or some
Speaker 1: inefficiency or some opportunity to make money. It's about having
Speaker 1: a team and a process to keep finding those over
Speaker 1: and over again. That's right again, because all the inefficiencies
Speaker 1: that I've ever seen are are short lived. So can
Speaker 1: you maybe um talk to us a little bit more
Speaker 1: than about how a quant strategy might be developed. So
Speaker 1: obviously you have the techno logical aspect of it, the
Speaker 1: need for computers that are able to trade very very quickly.
Speaker 1: You have the need for servers, many of them co
Speaker 1: located close to the exchanges. But then you also have
Speaker 1: proprietary data sets sometimes and then you have proprietary algorithms.
Speaker 1: So how does that all come together into one quant strategy?
Speaker 1: And which one of those is sort of the most
Speaker 1: or the biggest investment for a quant firm? Got it?
Speaker 1: So I think there's definitely a technological investment, may or
Speaker 1: may not involve things like co location near the exchanges,
Speaker 1: so at least anecdotally. For example, Renaissance, which is the
Speaker 1: most successful quantitative firm does not co locate. You know, again,
Speaker 1: I don't know, but that's that's that's what I've heard.
Speaker 1: Co location is quite important when you're trading UH and
Speaker 1: you require very low latency and and that's typically the
Speaker 1: high frequency trading domain and which again intersects with with
Speaker 1: quant in many ways. But if you're looking UM a
Speaker 1: little bit longer, if your horizons are a little bit longer,
Speaker 1: it becomes a little bit a little bit less important.
Speaker 1: Your broader point, I think is correct. Technology is important.
Speaker 1: I think I'm more important. It's kind of a research process.
Speaker 1: There there's a number of kind of high level pieces
Speaker 1: to a successful quantitative strategy. It's not like uh, UM
Speaker 1: there's just a black box and UM in gooes data
Speaker 1: outgoes trades. There's there's a number of pieces in there
Speaker 1: that UM sort of split the problem into to kind
Speaker 1: of make it manageable. UM at the front end UM.
Speaker 1: You know, going back to the heart of active investing,
Speaker 1: you've got to have a view on asset prices. Right,
Speaker 1: so you're trading some universes I don't know, US equities
Speaker 1: something like that, you've got to have a view stock
Speaker 1: by stock what's the price is going to be in
Speaker 1: a day? Um, uh, two weeks, a month, so on
Speaker 1: and so forth, right and so UM that front end
Speaker 1: is called signal generation or generating alpha's right, using data
Speaker 1: and machine learning techniques to come up with anomalies to
Speaker 1: that that you identify, and then you build models upon
Speaker 1: to sort of make a prediction of, um, what the
Speaker 1: what the what the price is going to be. So
Speaker 1: there's all sorts of types of data and uh algorithms
Speaker 1: that people use. Historically, much of quant investment has been
Speaker 1: building um what are called um quote unquote technical models,
Speaker 1: wherein basically you're using historical price and trade data to
Speaker 1: forecast future price movements. Right, so you might think of
Speaker 1: things like momentum or reversals or so on and so forth.
Speaker 1: That's you know, leveraging you know, kind of purely um
Speaker 1: technical data from the markets. UM. What we've seen emerged
Speaker 1: really over the past ten years is there's also been
Speaker 1: a shift to sort of quote unquote alternative data. Right,
Speaker 1: so you might look at things like you know, everybody's
Speaker 1: heard the famous story of satellite images of parking lots,
Speaker 1: right to try and assess you know, um, is you know,
Speaker 1: what's the occupancy at Walmart. This you're gonna they're gonna
Speaker 1: make their earnings. You know. Quantitative investment would take that
Speaker 1: kind of data and leverage it to a model which forecasts, Okay, um,
Speaker 1: what's the return going to be up for Walmart over
Speaker 1: the next week, the next month, next two months, and
Speaker 1: so on. Right, So the front end you have um
Speaker 1: uh this this identifying the data combined with the machine
Speaker 1: learning technology which is going to build build predictions. Now,
Speaker 1: oftentimes you're looking at um or I should say really
Speaker 1: always these days, you're looking at having many, many um anomalies.
Speaker 1: So you may have a technical model based on momentum
Speaker 1: and reversals. You may have bought a whole bunch of
Speaker 1: parking lot data, you have some some model for the
Speaker 1: retail sector based on that. You have some some credit
Speaker 1: card data, some social media data, maybe some some news data.
Speaker 1: You have all of these. And so the second part
Speaker 1: of the process is to kind of uh combine these
Speaker 1: different types of signals or views into sort of one
Speaker 1: compositive view because at the end of the day, um,
Speaker 1: all you care about is is net and that is
Speaker 1: this asset price is going to go up or go down,
Speaker 1: and and and that part is called alpha mixing or
Speaker 1: signal mixing. Right, Um, you have these these separate models
Speaker 1: that that that you've built, and you want to combine
Speaker 1: them to one kind of uh composite view. So um,
Speaker 1: that's that's kind of the front end again, having a
Speaker 1: view on what prices are going to be over the
Speaker 1: other relevant timeframes. Historically that is where the vast majority
Speaker 1: of U the energy was spent. The idea was that
Speaker 1: if you have a good signals, if you have good predictions,
Speaker 1: you can make money. If you don't have good signals,
Speaker 1: you're not going to make money, and the rest of
Speaker 1: it doesn't matter so much. So I believe if you
Speaker 1: don't have signals, you're not going to make money. That's
Speaker 1: certainly true. But these days the market has gotten competitive
Speaker 1: enough and there are enough kind of quant players that, um,
Speaker 1: what you do with the signals also matters how you
Speaker 1: try to to monetize them. So here the kind of
Speaker 1: the next step is that you have Now you know,
Speaker 1: you're waking up to open to the market. It's nine
Speaker 1: thirty in the morning, right, you have a prediction for
Speaker 1: you know, a universe of three thousand US equities. Now
Speaker 1: you have to kind of decide, um, what's the target
Speaker 1: portfolio you want to form? So that's kind of a
Speaker 1: called a portfolio construction case, right, And so the kind
Speaker 1: of things you're thinking about are balancing sort of risk
Speaker 1: versus return. You know, you don't want to be um
Speaker 1: um uh longer short, maybe you want to be market neutral.
Speaker 1: You don't want too much exposure in the individual set ters,
Speaker 1: you know, UM, so on and so forth. Right, UM,
Speaker 1: you're balancing that also with with with with transaction costs
Speaker 1: and so on, and you kind of decide like, um,
Speaker 1: you know again based on what my current view is
Speaker 1: of the world, UM, what's the target portfolio I want
Speaker 1: to hold? And this is something you periodically revisit. It
Speaker 1: used to be sort of um quants sort of you know,
Speaker 1: traded once a day and had a trade list the
Speaker 1: beginning of the day and you know, um generated trades
Speaker 1: and revisited the next day. Now it's much more of
Speaker 1: a continuous procedure because you know, as as the market
Speaker 1: evolves and as you get more data and news comes
Speaker 1: out and so on, those underlying views which you're driving
Speaker 1: the trades are changing. So so that's the kind of
Speaker 1: the middle piece, UM, figuring out what what portfolio to hold,
Speaker 1: and and then the final piece is actually um, sort
Speaker 1: of generating the trades. Sometimes quants do this themselves. I
Speaker 1: think more and more quants of doing this themselves. Um,
Speaker 1: you can farm this also to UH. Basically every major
Speaker 1: bank or prime broker that services UH quants has an
Speaker 1: agency algorithms desk that will do this for you. But
Speaker 1: here here the idea is, Okay, I decided I need
Speaker 1: to buy two million dollars of Google stock over the
Speaker 1: uh the next fifteen minutes. Um, how can I do that? Um?
Speaker 1: You know? Should I use exchanges? Should I use dark pools? Um?
Speaker 1: How should I uh spread that out over time? Um?
Speaker 1: You know, should I use limit orders? Market orders? Um?
Speaker 1: This kind of thing? And uh again, Historically, UM, you know,
Speaker 1: people focus a little bit less on that, but now
Speaker 1: as the market has gotten more more competitive, it's also
Speaker 1: being important. If if you're not doing those latter two phases,
Speaker 1: the portfolio construction and the trade optimization, well you're you're
Speaker 1: leaving money on the table in a way that almost
Speaker 1: may may not be may not be profitable. I think
Speaker 1: one thing that's that's not obvious that or I should say,
Speaker 1: it's quite different about plant trading versus uh, other types
Speaker 1: of hedge fund trading. If you look at a guy
Speaker 1: like um, you know, I don't know, just to sort
Speaker 1: of pick someone random like Bill Ackman, right, um, when
Speaker 1: when he goes in and buys a stock, he has like,
Speaker 1: you know, really kind of strong um conviction. He takes
Speaker 1: some massive positions, and he also he probably expects to
Speaker 1: make or you know, something like that. Again, I don't
Speaker 1: do that type of trading. I don't know, but he
Speaker 1: expects to make tens of percents, right, a quant in
Speaker 1: any individual position, you probably measure your expected profit in
Speaker 1: basis points, right, And it's all this and you know,
Speaker 1: you might expect to make three basis points on the
Speaker 1: transaction cost of two basis points, right, So you really
Speaker 1: like carefully controlling your costs and managing execution and so
Speaker 1: on is extremely important. Like you know, Bill Ackman, if
Speaker 1: he thinks he's going to make on a particular trade,
Speaker 1: it doesn't matter if he's paying uh, you know, two
Speaker 1: basis points for twenty basis points or even a hundred
Speaker 1: basis points, right, he's going to make so much more
Speaker 1: in his mind, it's irrelevant. Whereas for for for quants,
Speaker 1: you're really operating on a very thin margin. First of all,
Speaker 1: that was a sort of great explanation of the whole process,
Speaker 1: really nice overview, But I want to go back to
Speaker 1: just these sort of search for the original signals or
Speaker 1: search for this sort of the the initial inputs. And
Speaker 1: I'm thinking about large tech companies like Microsoft and Google
Speaker 1: and Facebook and how they have a lot of like
Speaker 1: researchers who are engaged in sort of pure tech research
Speaker 1: and you know, always out there filing patents, and there's
Speaker 1: probably a long sort of distance between anything that they
Speaker 1: discover and their own research budget um and then what
Speaker 1: ultimately might show up in a consumer product or a
Speaker 1: business product. And I'm curious if there is sort of
Speaker 1: an analogy in quant land where you have people who
Speaker 1: really are sort of at the frontier without a sort
Speaker 1: of crystal clear idea of okay, this is going to
Speaker 1: lead to something that will turn into a trade. But
Speaker 1: it's that process of sort of really exploring that frontier
Speaker 1: which eventually leads to concrete ideas that do lead to trades.
Speaker 1: And I'm curious if that's sort of like the analogy
Speaker 1: and how investors and how the portfolio managers think about
Speaker 1: where to explore and where those frontiers are, and where
Speaker 1: to invest expensive sort of time, energy and computing power
Speaker 1: in discovering these alpha generating signals. So I think quantity
Speaker 1: of investors operate quite differently than some of the research
Speaker 1: groups and in big tech places, like if you go
Speaker 1: to a place like Google Research or Microsoft Research, it's
Speaker 1: really not that different than an academic institution. Um, their
Speaker 1: their main output is really papers, right in journal papers,
Speaker 1: conference papers, so on and so forth. Uh, And it's
Speaker 1: really just a different way to do uh, almost academic research,
Speaker 1: kind of the classical Bell Labs model. And and maybe
Speaker 1: I mean they they do consult on internal projects and
Speaker 1: so forth. But I think in the in the in
Speaker 1: the quant world, it is much much much more applied.
Speaker 1: So I think typically the kind of thing would be like, um,
Speaker 1: you think, you know, maybe someone comes to you a vendor,
Speaker 1: or you identify a data set that you might that
Speaker 1: you think might have some relevance. You start looking at
Speaker 1: building various models of trying to predict prices or you know,
Speaker 1: things that are relatively of relevant to prices. You try
Speaker 1: and parent some different machine learning kind of techniques. But
Speaker 1: I think from the beginning it's really oriented around concrete
Speaker 1: things like let me build a price for let me
Speaker 1: build a model, sorry for for what the return of
Speaker 1: this asset is going to be over the next month, right,
Speaker 1: Or let me build a model for how I should
Speaker 1: efficiently trade large box of stock over the next fifteen minutes.
Speaker 1: Broadly speaking, it's much less of them sort of blue
Speaker 1: sky research that that isn't to say that some people
Speaker 1: don't do that. I think, uh, I think people do.
Speaker 1: But um, the the incentives aren't there because you know,
Speaker 1: for the most part speaking practitioners, um, there's no publishing, right,
Speaker 1: and um, I think people are extremely paranoid and sensitive
Speaker 1: because if if your IP leaks and other people do
Speaker 1: similar things, maybe what you do will stop working as well.
Speaker 1: And so there's, uh, there's not that much of an
Speaker 1: incentive to have to do that versus the very kind
Speaker 1: of visceral incentive of you know, making money, having you know,
Speaker 1: outperforming in the market in the in the short term.
Speaker 1: So so research in the quant world, for the most part,
Speaker 1: tends to be much more implied. I have a sort
Speaker 1: of related question, but why is why is quant investing
Speaker 1: or why are quants so um secretive about everything? Or
Speaker 1: I mean I don't want to call them weird, but
Speaker 1: there is this sort of like odd culture around quant investing.
Speaker 1: And you think of places like Renaissance and Citadel, they're
Speaker 1: all sort of shrouded in mystique. I once heard that
Speaker 1: Citadel had an original Enigma machine from World War Two
Speaker 1: in one of its offices. I don't know if that's true,
Speaker 1: but just the fact that people are saying this kind
Speaker 1: of thing tells you something about how they regard these big,
Speaker 1: storied quant companies. Why is there this very specific culture,
Speaker 1: mysterious secretive culture. So I think, broadly speaking, um, people
Speaker 1: in the by side teople in the Hedgeman industry are
Speaker 1: are generally secretive, but I think the with with regards
Speaker 1: to sort of their internal I P and and processes.
Speaker 1: But I think the nature of I P in the
Speaker 1: quant space creates incentives for people to be more secretive. Right.
Speaker 1: So again, just you know, pulling our hypothetical kind of
Speaker 1: Bill Ackman example, if he identifies some asset that's undervalued, UM,
Speaker 1: he's going to be sort of very quiet about it
Speaker 1: until he goes in and accumulates the position you watch,
Speaker 1: because he doesn't want other people to know and other
Speaker 1: people to front run him and to sort of take
Speaker 1: that opportunity away. Now, once he's a mass, that position
Speaker 1: perhaps will actually start even advertising it, right because now
Speaker 1: if sort of people sort of follow him, works to
Speaker 1: his benefit and it will push prices in the way
Speaker 1: that he wants. The quant space doesn't quite work like that.
Speaker 1: Like again, any individual trade is a very short rise
Speaker 1: and maybe a couple of weeks. Right. Trades are sort
Speaker 1: of very small and diffused across many many assets. But
Speaker 1: the idea of the trade, the data source coupled with
Speaker 1: whatever is generating the signal and the uilarity methodology and
Speaker 1: so on, that has lasting value that might you know,
Speaker 1: work for for the next six years. Again, year on year.
Speaker 1: It will the performance goes down as anomalies disappear, but uh,
Speaker 1: you know, it has multiple years of value. So uh.
Speaker 1: The general feeling is if people sort of figure out
Speaker 1: what you're doing, and um, where the opportunities are, and
Speaker 1: what data sets you're doing and so on, they will
Speaker 1: also do a similar kind of thing. That will they
Speaker 1: will copy you and then those anomalies will disappear faster,
Speaker 1: you know, at least in my experience, because of the
Speaker 1: longer time horizons over which this um, this IP decays
Speaker 1: people are more paranoid about being extremely secretive. And that's
Speaker 1: not only for for outsiders, but that's even within firms.
Speaker 1: So so many firms are siloed down to the level
Speaker 1: of individual quant researchers, where um you maybe um uh
Speaker 1: you know, you may have a team of a couple
Speaker 1: dozen people all um uh let's say under a single PM,
Speaker 1: all working on the same overall strategy, but you won't
Speaker 1: know what the guy next to you is working on, right,
Speaker 1: And if you pass data sets across maybe you um
Speaker 1: label them in sort of random ways and so on.
Speaker 1: So no, nobody, nobody, sort of maybe has the full
Speaker 1: picture except a handful of people um on the top.
Speaker 1: And again the idea there is that you know, over
Speaker 1: time people quit or leave or whatever, um you want,
Speaker 1: the firms would like them to have as little of
Speaker 1: the I P as possible in terms of uh, you know,
Speaker 1: not decaying the value of their own IP. Now, I
Speaker 1: think famously renaissance does not operate this way, So ret
Speaker 1: renaissance is um. One example I've heard where a firm
Speaker 1: which is uh I think, very very difficult to get
Speaker 1: into in terms of being hired. But but once you're
Speaker 1: in there. They're quite open in terms of what are
Speaker 1: the different things we've tried, Where are the things that
Speaker 1: are working now, where the things that haven't worked before,
Speaker 1: and you know, so on and so forth. And I
Speaker 1: think actually from the perspective of research, that works much
Speaker 1: better um quant researchers tend to, uh believe it or not,
Speaker 1: tend to be kind of social animals, and it's it's
Speaker 1: always more fun to work on things with other people
Speaker 1: rather than just sort of sit at your desk with
Speaker 1: with the with the blinders on and so on. You know.
Speaker 1: Interesting about Renaissance is how they've been able to manage
Speaker 1: it so that very very few people have have have
Speaker 1: left and it seems like, you know, they have not
Speaker 1: had the kind of ip loss that other people worry about.
Speaker 1: So Renaissance famously just puts up extraordinary numbers year after
Speaker 1: year after year. And the sort of the trick or
Speaker 1: one trick besides there being a bunch of mathematical geniuses,
Speaker 1: is a having this sort of open culture of collaboration
Speaker 1: and research and be somehow preventing a lot of exodus
Speaker 1: so that no one else has really been able to
Speaker 1: replicate their approaches. Uh In any way, how hard is this?
Speaker 1: So you think about like someone like I don't know,
Speaker 1: like you hear about other other managers, like you know,
Speaker 1: Steve Cohen is like, oh, I wanna allocate money to
Speaker 1: quant How hard is it? And this is sort of
Speaker 1: something I want to explore more now, is like, how
Speaker 1: hard is it to sort of anti up into that
Speaker 1: game and to sort of start being competitive if this
Speaker 1: if you're sort of starting from zero right now, Um,
Speaker 1: I think it's uh, it's a tough place. It's a
Speaker 1: it's a it's a competitive game. Maybe not so much anymore.
Speaker 1: But over the past five seven years, Um, my general
Speaker 1: perspective is that the buy side active managing sort of
Speaker 1: hedge funds have been shrinking overall. The one sector that
Speaker 1: has not been shrinking his quant and so I think
Speaker 1: there has been an entrance of uh kind of new
Speaker 1: players there. Um. Now, I'm Steve Cohen you specifically mentioned.
Speaker 1: He's actually been at it for a while. He's been
Speaker 1: in the quant space for the since the early two thousand's.
Speaker 1: He on the order of twenty to thirty percent of
Speaker 1: his assets are actually quant some some something like that,
Speaker 1: like a non you know, people mainly think of him
Speaker 1: as a long short uh kind of guy, and that's
Speaker 1: probably mainly what he is. But again, um, you know,
Speaker 1: maybe a third of his assets are in a quant
Speaker 1: space through after Cubist and so on. Now he operates
Speaker 1: very differently, um, he operates. His quant funds operate in
Speaker 1: uh kind of like traditional UM long short guys operate,
Speaker 1: wherein you hire individual pms, you watch them a very
Speaker 1: kind of carefully. They make money or they lose money
Speaker 1: if if they if they're not making money quickly enough,
Speaker 1: you fire them. And uh you sort of you kind
Speaker 1: of have a portfolio of these these individual managers who
Speaker 1: are who are doing their own things, who are tightly siloed,
Speaker 1: and uh you know, uh you try to manage that.
Speaker 1: And that's the way his quant operation manages. So there's
Speaker 1: you know, again, um, a whole bunch of small um
Speaker 1: let's call them pods or whatever, of you know, two
Speaker 1: or three people each kind of doing their own thing
Speaker 1: in an uncoordinated way. You know. That's again quite a
Speaker 1: different model than let's say, the Renaissance, which is uh
Speaker 1: um uh you know, one kind of open strategy. And
Speaker 1: I think the the advantage of the Steve Cohen model
Speaker 1: is that uh Um, you know, uh, it's it's easy
Speaker 1: to to hire people from the HR. Process is very easy.
Speaker 1: You don't have to care when people come and go
Speaker 1: on so on, because you're not really investing in any
Speaker 1: of their their individual I P. Right when when someone
Speaker 1: leaves or like let's you fire someone, it's because they
Speaker 1: didn't do well and whatever they have is maybe not
Speaker 1: worth um that much and they don't know anything else
Speaker 1: about what your other p are doing and so so
Speaker 1: that process is very easy. But I think the downside
Speaker 1: is that what we're sort of starting to see is
Speaker 1: throughout the quant space, like you know, the broader technology industry,
Speaker 1: we're starting to see that there are a lot of
Speaker 1: increasing returns to scale that as you get bigger and
Speaker 1: bigger firms are able to build advantages. And one kind
Speaker 1: of concrete source of this is around trading costs. Right, Um,
Speaker 1: when you're thinking about, like let's say, on an individual
Speaker 1: trade by trade basis, do I want to get into
Speaker 1: this trade? You have a prediction of how much you're
Speaker 1: going to make if your if your models are correct,
Speaker 1: but also there are these costs that you're paying, these
Speaker 1: transaction costs, and if your prediction doesn't exceed your costs.
Speaker 1: You shouldn't put on that trade because even in the
Speaker 1: best case, you're you're not going to make your money, right. So, um,
Speaker 1: what what's happened is that as more and more people
Speaker 1: have gotten into the quant space and more and more
Speaker 1: of these identically anomaly sorry are identified and markets get
Speaker 1: the more efficient, the signals have gotten weaker, right, and
Speaker 1: so um, just to sort of give a give a
Speaker 1: maybe a concrete example. One signal that's sort of um,
Speaker 1: quite well known throughout the klant industry and un academics
Speaker 1: of published papers and so on, its order book of balance. Right,
Speaker 1: if you go out and you look at an electronic
Speaker 1: order book and they're more buyers than there are sellers
Speaker 1: in terms of the resting limit orders. Um, it's it's
Speaker 1: more likely that the press will go up and go down. Right.
Speaker 1: You can. You can go out and try that that
Speaker 1: has a predictive value. Now, however, if that's all you know,
Speaker 1: you won't make money because you might think the price
Speaker 1: is going to go up you know, a tenth of
Speaker 1: a basis point just to throughout a number, but your
Speaker 1: transaction costs or two basis points, and you know you're
Speaker 1: just not. Um, you can't exceed your your costs. So, um,
Speaker 1: the transaction costs to a first approximation, they're they're kind
Speaker 1: of like on a trade by trade basis, a fixed
Speaker 1: costs that you have to exceed. Now, if you're in
Speaker 1: a world where you have many, many signals, maybe tens,
Speaker 1: maybe hundreds, maybe thousands, and you're adding them up and
Speaker 1: they're independent and you trade when they're all aligned, now
Speaker 1: you can have sort of, um, you know, signals that
Speaker 1: are weak individually, and nevertheless, when you you combine them,
Speaker 1: when you aggregate them, you are able to exceed transaction
Speaker 1: costs and monetize them. So that that order and balanced
Speaker 1: signal that I just sort of talked about. If you're
Speaker 1: sort of one guy in your basement and that's all
Speaker 1: you knew, you can't make money off that. But if
Speaker 1: you have twenty other signals and you're you know, you're
Speaker 1: going to put on a trade anyway, in some sense,
Speaker 1: the transaction costs become a sunk cost, and and that
Speaker 1: you know, point one basis point that you're going to
Speaker 1: get because of this well known signal, that becomes free money.
Speaker 1: So so as you get that kind of economies of
Speaker 1: scale because of fixed costs. I think it becomes harder
Speaker 1: and harder to have UM quant strategies where um, you
Speaker 1: don't have a lot of people UM you know, in
Speaker 1: a very kind of coordinated research process where you have
Speaker 1: people working essentially independently UM the kinds of UM you
Speaker 1: know places that are structured like like like let's say
Speaker 1: Renaissance again, where you might have like two plant researchers
Speaker 1: all working on different aspects of the thing, and then
Speaker 1: you know, these things combined to one sort of overall
Speaker 1: view of the market. I think that is able to
Speaker 1: sort of better monetize a lot of these signals in
Speaker 1: this kind of more competitive world. So on that note,
Speaker 1: if if you are running a lot of these strategies,
Speaker 1: getting a lot of these signals, and you're able to
Speaker 1: lower your transaction costs because of that scale, and at
Speaker 1: the same time, quant investing has these big barriers to
Speaker 1: entry because you have to have these technological outlays, you
Speaker 1: have to hire a bunch of PhDs and things like that.
Speaker 1: Does that mean that the industry is inevitably sort of
Speaker 1: trending towards a monopoly? Are are we going to get
Speaker 1: a situation where there is just one or maybe two
Speaker 1: or three really big quant investors because no one else
Speaker 1: can compete with them effectively. I think we're kind of there.
Speaker 1: I mean, I think there are only a handful of
Speaker 1: large quants. Most of them have been doing it for
Speaker 1: a long time. I mean, Renaissance, d SHAW, PDT, two Sigma.
Speaker 1: You know, there's there, there's a there's a handful of others.
Speaker 1: I think it's it's harder to see, you know, maybe
Speaker 1: there's some exceptions, you know, in terms of funds that
Speaker 1: have launched UHM more more recently, but it's difficult to
Speaker 1: see people of of that scale with with the similar
Speaker 1: track records. So I think we are seeing some degree
Speaker 1: of consolidation. I don't know what the altar I mean,
Speaker 1: I don't know if it's gonna um come down to
Speaker 1: one firm. I think, you know, probably not, just probably
Speaker 1: kind of more competition, but I think it will be
Speaker 1: harder to have uh sort of either either more independent
Speaker 1: managers or like UM kind of the siloed model of
Speaker 1: places like uh you know, UM S, A C and Millennium.
Speaker 1: If I want to start a quant fund, what are
Speaker 1: we talking about in terms of how much it's just
Speaker 1: gonna cost for computers and data just to give even
Speaker 1: get in the game. Don't do it, Joe. I feel
Speaker 1: like this whole conversation is how you shouldn't be doing that. No,
Speaker 1: I realized, I realized that it's a bad idea. But
Speaker 1: let's say I'm an idiot and I try anyway, Like,
Speaker 1: what are we talking about? So? Um, I think things
Speaker 1: have gotten over time much more expensive, things like data
Speaker 1: feeds and uh you know, so on the exchanges have
Speaker 1: constantly been ramping the prices on on these things. But
Speaker 1: um you know, these days what's become one of the
Speaker 1: biggest costs is actually just pure computation and and this
Speaker 1: is also a trend we see um uh, you know,
Speaker 1: more broadly in a technology, you know, if you look
Speaker 1: at kind of the state of the art models for
Speaker 1: things like um uh, computer vision, object recognition for um uh,
Speaker 1: you know, playing games like a chess and go and
Speaker 1: so on, these types of models leverage approaches and machine
Speaker 1: learning that are really based on having a lot of
Speaker 1: data and doing even more than that, doing a lot
Speaker 1: of computation and and and so the spirit there, you know,
Speaker 1: coming out of places like deep minded Google or be
Speaker 1: ai and stuff, um open ai, um of you know
Speaker 1: are artificial intelligence UM company that their main model is
Speaker 1: is literally like we're going to do simple things, but
Speaker 1: we're going to leverage it to massive scale computation, right,
Speaker 1: and so so I think you're starting to see that
Speaker 1: in finance as well, where you need to do things
Speaker 1: like let's say you need to um UM back test
Speaker 1: a trading strategy. UM, but you have some parameters, and
Speaker 1: you want to try tens of thousands of combinations of
Speaker 1: those trading parameters, and each one involves a simulation over
Speaker 1: you know, twenty years and so on and so forth.
Speaker 1: You need a lot of computers. So UM. Someone told
Speaker 1: me anecdotally that at a major quant shop, each quantitative
Speaker 1: researcher has given kind of a quote unquote budget of
Speaker 1: of of ten thousand CPUs, right, so I ain't given time,
Speaker 1: they can use up to ten thousand individual kind of
Speaker 1: processing units. And just to give you a sense of
Speaker 1: what that costs, UM, you know, if you're to go,
Speaker 1: you know, buy that on Amazon at AWS, that would
Speaker 1: be the order of magnitude maybe a million dollars a year. Right,
Speaker 1: And this is just for this is just for research.
Speaker 1: This is not to actually generate the trades or whatever.
Speaker 1: This is just a tune all the parameters and and
Speaker 1: and sort of really optimize your performance. That's really interesting.
Speaker 1: It kind of makes me wonder how how good I
Speaker 1: guess academic research is at gauging quant strategies if the
Speaker 1: outlays just to run a few experiments are so massive.
Speaker 1: But on a slightly different topic, I wanted to ask you,
Speaker 1: I guess this question is kind of inevitable. Whenever you
Speaker 1: talk about algorithmic trading or systematic trading, what value do
Speaker 1: you think quant investing actually creates for society? So, for instance,
Speaker 1: when we talk about traditional investing, that's supposed to channel
Speaker 1: capital in the most efficient way possible to good companies,
Speaker 1: and that should in theory benefit the entire economy. But
Speaker 1: quant investing, as we've discussed, isn't really about that. It's
Speaker 1: about arbitraging these small differences. So maybe it makes prices
Speaker 1: slightly more efficient, but is that worth the enormous infrastructure
Speaker 1: investment that we've been discussing being spent on it? So
Speaker 1: I think there is, um there are some benefits. You know,
Speaker 1: it varies based on the strategy and based on really
Speaker 1: the incident time. But I think a lot of uh,
Speaker 1: you know, to to a first approximation. If you see
Speaker 1: a price move in a direction that's unusual, UM, it
Speaker 1: could continue or it could revert. Right to the extent
Speaker 1: that you think it's going to revert, you're going to
Speaker 1: sort of bet against it. And what what that amounts
Speaker 1: to is basically supplying temporary liquidity to the market. Right, So,
Speaker 1: I think the positive aspect to UM quantitative investing is
Speaker 1: that UM I think a lot of it is supplying
Speaker 1: liquidity to the market on a horizon of uh, let's
Speaker 1: say days to two weeks right now. The flip side
Speaker 1: is if you're if you're really it's more of a
Speaker 1: momentum that you might be accelerating the trends UM you're
Speaker 1: taking away like wuity, you're competing for that liquidity, but
Speaker 1: as you said, maybe you're making prices more more efficient.
Speaker 1: So I think, on balance, I think that that probably
Speaker 1: there is some benefit. I think it's probably small. Admittedly,
Speaker 1: is it worth all these uh, you know, very smart
Speaker 1: people being drawn away from other fields and so on,
Speaker 1: I'm not sure, But you know, probably as much or
Speaker 1: more resources or spent at places like Facebook and Google
Speaker 1: getting people to click on ads. Right, I'm not sure
Speaker 1: that that's uh as positive the pressing. Think about all
Speaker 1: these people, um, you know, looking for signals to squeeze
Speaker 1: out three basis points in the market, because there could
Speaker 1: be some great innovations in squeezing more ads onto a
Speaker 1: mobile phone that they'd be working on. And there you go,
Speaker 1: kind of a sad allocation of resources. See you think
Speaker 1: Joe's joking, but he's he probably doesn't. So uh, here's
Speaker 1: one thing that also always tends to come up. It's
Speaker 1: this idea of um, this type of trading reaching the
Speaker 1: limits of available technology and pushing the strategies to sort
Speaker 1: of greater extremes. But those extremes eventually have limits. And
Speaker 1: so I guess I'm just wondering, is there a limit
Speaker 1: to quant investing? Is there a point at which quants
Speaker 1: sort of arbitrage everything out of the market and the
Speaker 1: signals are no longer useful or the algorithms themselves are
Speaker 1: impacting the market in some way? And on that note,
Speaker 1: what's what's the next big thing in quant investing? I
Speaker 1: guess yeah, So, I mean I think there's a constant balance.
Speaker 1: These are finishing of inefficiencies are being identified in arbitrage
Speaker 1: the way because there's money in it, right, and so
Speaker 1: as arbitraged in it, Um, the money sort of disappears
Speaker 1: and then you get sort of a fewer people kind
Speaker 1: of doing it. But um, so long as there's uh,
Speaker 1: you know, kind of traders out there, we're not paying
Speaker 1: attention to this stuff and uh you know, the Robin
Speaker 1: Hood traders or whatever and are kind of leaving money
Speaker 1: on the table, Um, there will be people um there
Speaker 1: who are trying to uh sweep up the crumbs in
Speaker 1: terms of where it's going. What the what the next
Speaker 1: big thing is. I think it's it's it's pretty hard
Speaker 1: to predict, but I think um, uh you know, broadly
Speaker 1: a shift towards uh things that are even more black box,
Speaker 1: even more computationally driven and uh um not so much.
Speaker 1: Uh you know, have like kind of nice structural explanations.
Speaker 1: Um again sort of following a lot of what's going
Speaker 1: on in in the tech world as we shift to
Speaker 1: ideas like deep neural networks and reinforcement learning and so
Speaker 1: on and so forth. Um. You know, you know, again,
Speaker 1: you have these these systems that work worked great for
Speaker 1: Let's say I'm playing go, but it's really hard to
Speaker 1: explain what's going on and I think we're starting to
Speaker 1: see that in the quant world as well, again leveraging
Speaker 1: a computation but really really ending up with with things
Speaker 1: that are you know, um, you know black boxes that
Speaker 1: you know just are completely not transparent. So in other words,
Speaker 1: you know, like you could look at something like satellite
Speaker 1: images and say, oh, there's a lot of cars parked
Speaker 1: of Walmart and then predict the Walmart stock is going
Speaker 1: to be up. But the next, um, the next generation
Speaker 1: of things to watch out for is this works, and
Speaker 1: it works consistently, but we as humans can't really articulate
Speaker 1: why exactly. That's super interesting. Well, on that note of
Speaker 1: humans not really even being being able to explain what
Speaker 1: they're doing, um, it seems like a perfect place to stop.
Speaker 1: Thank you so much for joining us. Thank you so much, Tracy.
Speaker 1: You know, uh, as a as a media person, I
Speaker 1: have my own experience with the sort of alpha decay
Speaker 1: that CMX was talking about. Do you know what it is? Um,
Speaker 1: did you build some sort of algorithm to take advantage
Speaker 1: of like Google Ads or something and then it stopped working? No,
Speaker 1: there's nothing so sophisticated. But back in the early days
Speaker 1: of like blogging and stuff, I remember this phenomenon where
Speaker 1: you would come up with some like headline construction. You'd
Speaker 1: like five things you need to know today. Remember, like
Speaker 1: the old upworthy headlines, they were like and you could
Speaker 1: and you can't guess what you know. And then those
Speaker 1: work and those generate like excess traffic, and they get
Speaker 1: shared on Facebook, and then everybody discovers that these headlines
Speaker 1: cliches work, and then everyone does them, and then people
Speaker 1: stopped clicking on them, and you need to like find
Speaker 1: I don't do clickbait anymore. But I always thought at
Speaker 1: the time like that was like a very similar process
Speaker 1: to uh, to this sort of quant approach to investing,
Speaker 1: the sort of search for alpha and alpha decay of
Speaker 1: a blog headline anymore was the key word in that
Speaker 1: sentence about clickbait. But I think it's a really good analogy.
Speaker 1: It is a good analogy because like the usefulness of
Speaker 1: those headline constructions decays over time, as you point out,
Speaker 1: because more people are copying them. But it also kind
Speaker 1: of gets to that point about the limits of this
Speaker 1: type of investing. There are only so many ways that
Speaker 1: you can construct a headline, and eventually people kind of
Speaker 1: catch on two different ones and they become not so enticing,
Speaker 1: and I kind of wonder if the same thing could
Speaker 1: eventually happen to quant investing. So obviously there are many
Speaker 1: many more possibilities in quant investing, and it's possible that
Speaker 1: markets are always changing and so opportunities for arbitrage and
Speaker 1: identifying these signals are always coming up. But it does
Speaker 1: make you wonder. It certainly does. And what he's talking
Speaker 1: about at the end, where maybe the signals of the
Speaker 1: future are just things that work it can't be articulated,
Speaker 1: is just like a super kind of fascinating phenomenon to
Speaker 1: just like wrap your head around. Yeah, I feel like
Speaker 1: that's a good microcosm for maybe the human experience in
Speaker 1: the future. Like we have the technology, we're not entirely
Speaker 1: sure how it works, but we're just going to sort
Speaker 1: of let it run and hope for the best. One
Speaker 1: of the things that sort of interested me is like
Speaker 1: a sort of thing to watch going forward, is okay,
Speaker 1: So we talked about a huge aspect of that was
Speaker 1: just the costs and how like you might be able
Speaker 1: to identify a profitable anomaly. But unless the cost of
Speaker 1: getting the data and executing the trade is lower than that,
Speaker 1: it's um, it's useless, but you know, you also have
Speaker 1: to wonder, like, okay, right now, like a certain handful
Speaker 1: of exchanges, say, control a lot of the trade data costs.
Speaker 1: In theory, that seems like an area where maybe new
Speaker 1: entities will come and find a way to provide data
Speaker 1: cheaper Amazon Web services. You know, presumably computation costs are
Speaker 1: going to keep coming down, and obviously that was a
Speaker 1: big breakthrough from probably the old days where you had
Speaker 1: some sort of main frame on premise services. You know,
Speaker 1: computation has gotten cheaper, so there's probably always going to
Speaker 1: be new opportunities to squeeze out even smaller profits because
Speaker 1: there are ways to shave costs in sort of your
Speaker 1: in your research, your work. Yeah. Maybe the other thing
Speaker 1: that was really interesting was the idea that quants um.
Speaker 1: I think CMAC described them as actually social animals, which
Speaker 1: kind of flies in the face and I think of
Speaker 1: a lot of stereotypes. But I'm also I'm really curious.
Speaker 1: I would love to be embedded in a firm like
Speaker 1: Citadel and just observe how they work together and what's
Speaker 1: considered a good out go, a good systematic strategy versus
Speaker 1: a bad systematic strategy. Obviously you wanted to make money,
Speaker 1: but are there certain things that are more valued over others?
Speaker 1: Maybe cheapness to execute or um, I don't know, risk management,
Speaker 1: something like that. I'd be so curious to see how
Speaker 1: that all works. Um, I'm sure if we just walked
Speaker 1: in there, just let us in the door and let
Speaker 1: we could just hang out there for a while. Yeah,
Speaker 1: I'm sure they wouldn't mind at all. No, let us
Speaker 1: see their white boards stuff like that. Citadel, if you're listening,
Speaker 1: we would like to come sit you. Okay, should we
Speaker 1: leave it there? Let's leave it there. This has been
Speaker 1: another episode of the ad Thoughts podcast. I'm Tracy Alloway.
Speaker 1: You can follow me on Twitter at Tracy Alloway and
Speaker 1: I'm Joe Wisntal. You can follow me on Twitter at
Speaker 1: a Stalwart. And you should follow our guest on Twitter
Speaker 1: cmx Millenmy he's at Cmax. Follow our producer on Twitter,
Speaker 1: Laura Carlson at Laura M. Carlton. Follow the Bloomberg head
Speaker 1: of podcast, Francesca Levi at Francesca Today, and check out
Speaker 1: all of our podcasts under the handle AD Podcasts. Thanks
Speaker 1: for listening to the year