WEBVTT - Bridgewater's Greg Jensen on AI, Inflation and What Markets Are Getting Wrong 0:00:10.240 --> 0:00:13.520 Hello, and welcome to another episode of the Odd Lots Podcast. 0:00:13.640 --> 0:00:15.080 I'm Tracy Alloway. 0:00:14.760 --> 0:00:15.960 And I'm Joe Wisenthal. 0:00:16.360 --> 0:00:19.759 Joe, I think it's fair to say there is a 0:00:19.760 --> 0:00:24.239 lot of excitement about investing in AI. There is also 0:00:24.280 --> 0:00:27.160 a lot of excitement about using AI to invest. 0:00:27.920 --> 0:00:31.400 Yes, I mean it's I think there's like a new 0:00:31.440 --> 0:00:34.720 like chat ETF I saw an ad for and there's like, oh, 0:00:34.720 --> 0:00:37.440 we're getting No. I think I saw another like project. 0:00:37.440 --> 0:00:39.600 It was like, we're gonna have chat GPT pick the 0:00:39.640 --> 0:00:42.639 stocks for us, And you know, I get it. It's 0:00:42.720 --> 0:00:45.000 kind of exciting and maybe there's some new way of 0:00:45.120 --> 0:00:48.839 like these super advanced digital brains that can beat the market, 0:00:48.880 --> 0:00:51.400 et cetera. But like, I don't totally get it. 0:00:51.880 --> 0:00:52.160 Well. 0:00:52.280 --> 0:00:55.360 I also feel like there's a tendency nowadays for people 0:00:55.400 --> 0:01:00.560 to talk about artificial intelligence in a sort of abstract manner. 0:01:00.680 --> 0:01:04.600 You hear people bring up AI almost as a synonym 0:01:04.800 --> 0:01:08.520 for just software at this point. I think you pointed 0:01:08.560 --> 0:01:13.360 out recently that the Kroger CEO mentioned AI like times 0:01:13.400 --> 0:01:16.520 on the earnings call. So a supermarket chain, right. 0:01:16.480 --> 0:01:21.000 Yeah, And you know it's like machine learning, tech, algebra, algorithms, 0:01:21.040 --> 0:01:24.440 it's all existed for a long time quantitative investing, but 0:01:24.480 --> 0:01:27.839 it feels like because of the excitement around a few 0:01:27.880 --> 0:01:31.480 specific consumer phasing products that have been unveiled over the 0:01:31.600 --> 0:01:34.520 last six months and the way they've captured people's attention, 0:01:35.000 --> 0:01:37.480 people like you know, suddenly there's a lot of interest 0:01:37.480 --> 0:01:41.279 in like, how are companies using this tech to do something? 0:01:41.480 --> 0:01:44.319 Yeah, well, I'm glad you mentioned that because today we 0:01:44.400 --> 0:01:47.200 really do have the perfect guest. This is someone we've 0:01:47.280 --> 0:01:52.480 actually spoken to about AI before last year, in fact, 0:01:52.840 --> 0:01:56.480 someone who is at a firm that has a lot 0:01:56.520 --> 0:02:01.720 of experience using machine learning and of different types, and 0:02:01.760 --> 0:02:05.160 we're going to get into the differences between all those technologies. 0:02:05.520 --> 0:02:07.680 I'm very pleased to say we're going to be speaking 0:02:07.760 --> 0:02:11.280 once again with Greg Jensen, the co chief investment officer 0:02:11.560 --> 0:02:14.560 at Bridgewater Associates. So, Greg, thank you so much for 0:02:14.600 --> 0:02:16.359 coming back on OD thoughts. 0:02:16.800 --> 0:02:18.799 Yeah, it's great to be here. Exciting topic. 0:02:19.280 --> 0:02:23.240 Yeah, So I actually revisited our conversation from last year, 0:02:23.520 --> 0:02:25.799 I think it was in May of twenty twenty two, 0:02:25.880 --> 0:02:29.920 and you said two things that stuck out in retrospect. 0:02:30.000 --> 0:02:32.480 So number one, you said that markets had further to fall, 0:02:32.560 --> 0:02:36.880 which turned out to be correct. And two you brought 0:02:36.960 --> 0:02:41.400 up artificial intelligence as a major point of interest for Bridgewater, 0:02:41.480 --> 0:02:44.520 And this was all before chat GPT really became a 0:02:44.560 --> 0:02:48.359 thing and everyone started talking about AI at every single 0:02:48.440 --> 0:02:52.000 conference and earnings call and so on. So I guess, 0:02:52.080 --> 0:02:54.400 just to begin with, maybe you could lay the scene 0:02:54.639 --> 0:02:58.480 and going back to Joe's point in the intro, we 0:02:58.520 --> 0:03:02.560 are used to hearing these terms. So Bridgewater does machine 0:03:02.639 --> 0:03:07.920 learning and systematic strategy strategies and quantitative trading strategies and 0:03:08.000 --> 0:03:11.840 AI and things like that. What's the difference between all 0:03:11.880 --> 0:03:14.600 of these things and how do they relate to each 0:03:14.600 --> 0:03:16.480 other at a firm like Bridgewater. 0:03:18.919 --> 0:03:21.400 Yeah, great question. So I think to answer that, let 0:03:21.440 --> 0:03:23.040 me take a step back for a second and give 0:03:23.040 --> 0:03:24.640 you a little bit of my background, because it all 0:03:25.280 --> 0:03:28.160 kind of comes together in a way. You can connect 0:03:28.200 --> 0:03:32.240 these different pieces. So you know, even as a kid 0:03:32.320 --> 0:03:36.000 or whatever, I was certainly interested in kind of translating 0:03:36.440 --> 0:03:42.000 and predicting things using some mix of my thinking and technology. 0:03:42.080 --> 0:03:45.200 So I can think back to in the late eighties 0:03:45.560 --> 0:03:49.080 using stratomatic baseball cards, know what they are, but programming 0:03:49.120 --> 0:03:51.160 them into computers to try to calculate the way to 0:03:51.200 --> 0:03:55.960 create the best baseball lineup and use that in fantasy 0:03:56.000 --> 0:04:00.880 baseball type situations and similar things wither and whatever, and 0:04:00.920 --> 0:04:03.800 try to learn how to kind of use technology to 0:04:03.800 --> 0:04:08.120 combine with human intuition to get at what was different 0:04:08.120 --> 0:04:10.920 ways to create edges. And then in college when I 0:04:10.920 --> 0:04:13.520 heard about Bridgewater Bridge Order. It was a tiny place 0:04:13.520 --> 0:04:17.080 at the time, but the basic idea that there was 0:04:17.120 --> 0:04:19.040 a place where we were trying to understand the world, 0:04:19.440 --> 0:04:22.799 trying to predict what was next, but doing that by 0:04:22.960 --> 0:04:27.239 taking human intuition and translating that into algorithms to predict 0:04:27.240 --> 0:04:30.080 what was next kind of mixed two things that I loved. 0:04:30.080 --> 0:04:31.960 I love to try to understand the world, and I 0:04:32.000 --> 0:04:34.560 love the idea of having the discipline to write down 0:04:34.600 --> 0:04:36.960 what you believed and stress test what you believed and 0:04:37.120 --> 0:04:39.240 utilize that. Right. So, if you go back, and this 0:04:39.279 --> 0:04:43.040 is now in the nineties kind of where artificial intelligence 0:04:43.160 --> 0:04:46.000 was at the time, most of the focus was still 0:04:46.040 --> 0:04:48.359 on expert systems, was still on the notion that you 0:04:48.400 --> 0:04:53.800 could take human intuition, you could translate that into algorithms, 0:04:53.839 --> 0:04:55.800 and if you did enough of that, if you kept 0:04:55.920 --> 0:05:00.840 kind of representing things in symbolic algorithm that you could 0:05:00.880 --> 0:05:04.440 build enough human knowledge to get kind of a superpowered human. 0:05:05.160 --> 0:05:08.880 And Bridgewater was a rare example of where that worked. 0:05:08.920 --> 0:05:10.960 Where given the focus of trying to predict what was 0:05:11.040 --> 0:05:14.400 next in markets, given the incredible investment that we made 0:05:14.920 --> 0:05:18.240 into creating the technology to take human intuition and translate 0:05:18.320 --> 0:05:22.360 that into algorithms and stress tests, that it's incredibly successful 0:05:22.920 --> 0:05:27.039 expert system essentially that was built over the years, I'd 0:05:27.040 --> 0:05:32.040 say probably the most profitable expert system out there. And 0:05:32.080 --> 0:05:34.960 that's really what Bridgewater has been about, which is building 0:05:34.960 --> 0:05:37.800 this great technology to help us take human intuition out 0:05:37.800 --> 0:05:40.920 of the brain, get it into technology where it's both 0:05:40.960 --> 0:05:46.000 then readable by let's say investment experts, but also runs 0:05:46.080 --> 0:05:49.280 on a technology basis. And that's kind of where algorithms, 0:05:49.360 --> 0:05:52.880 let's say, the mix of algorithms and human intuition it 0:05:52.960 --> 0:05:55.240 was really important. You know, if you go through the 0:05:55.279 --> 0:05:58.280 history of our competitors, they're littered by people that tried 0:05:58.279 --> 0:06:01.520 to do something more statistical, meaning that they would take 0:06:01.560 --> 0:06:04.920 the data, run regressions, and then after regressions, let's say 0:06:04.920 --> 0:06:07.880 basic machine learning techniques to predict the future. And the 0:06:07.920 --> 0:06:11.480 problem that always had is that there wasn't enough data like, 0:06:11.520 --> 0:06:15.960 the truth is that market data isn't like the data 0:06:16.000 --> 0:06:18.039 and the physical world in the sense that a you 0:06:18.080 --> 0:06:20.760 only have one run through human history, you don't have 0:06:20.880 --> 0:06:23.840 very many cycles, even cycles that debt cycles could take 0:06:23.839 --> 0:06:26.400 seventy years to play out. Economic cycles tend to plan 0:06:26.520 --> 0:06:29.440 around for seven years. There's just not enough data to 0:06:29.560 --> 0:06:35.120 represent the world. And secondly that the game changes as 0:06:35.160 --> 0:06:38.839 participants learned, So the existence of algorithms, as an example, 0:06:39.320 --> 0:06:42.400 change the nature of markets such that the history that 0:06:42.640 --> 0:06:45.320 preceded it was less and less relevant to the world 0:06:45.400 --> 0:06:49.279 you're living in. So those are big problems with let's 0:06:49.279 --> 0:06:53.520 say a more pure statistical technique to markets. So you 0:06:53.600 --> 0:06:57.120 had to get to a world where statistical techniques or 0:06:57.120 --> 0:07:02.160 machine learning could substitute for human intuition. And that's really 0:07:02.160 --> 0:07:06.480 where kind of the exciting leaps are. Now that you're 0:07:06.520 --> 0:07:09.280 getting closer. It's not totally there, but you're much closer 0:07:09.279 --> 0:07:13.400 than you've ever been, where large language models actually allow 0:07:13.440 --> 0:07:16.520 a path to something that at least mimics human intuition, 0:07:16.640 --> 0:07:21.880 if not is human intuition, and that you can then 0:07:21.920 --> 0:07:25.280 combine that with other techniques and suddenly you have a 0:07:25.400 --> 0:07:28.520 much more powerful set of tools that can deal at 0:07:28.600 --> 0:07:30.680 least in take a big leap forward on dealing with 0:07:30.720 --> 0:07:33.800 the problem of very small data sets and the fact 0:07:33.800 --> 0:07:36.640 that the world changes as people learn in a way 0:07:36.760 --> 0:07:39.640 that up until the big breakthroughs in large language models, 0:07:40.320 --> 0:07:44.000 I think we're much further away. So that's a huge 0:07:44.160 --> 0:07:49.040 change in the limits of ways that statistical machine learning 0:07:49.440 --> 0:07:53.480 could affect something with small amounts of data, something where 0:07:53.520 --> 0:07:57.680 the future varies from the past. All of those problems 0:07:57.680 --> 0:08:00.520 we're closer to having. At least way is to take 0:08:00.560 --> 0:08:03.320 on more and more of what humans have done at Bridgewater, 0:08:03.400 --> 0:08:06.320 what humans generally do in investment management firms, And that's 0:08:06.520 --> 0:08:09.280 that's a huge leap forward that's going on now. 0:08:09.640 --> 0:08:12.920 I have one very short quick question. I realized just 0:08:13.040 --> 0:08:16.480 know that not long after we talked to last year, 0:08:16.720 --> 0:08:19.360 last spring, like a month later, you won your first 0:08:19.600 --> 0:08:22.920 World Series of poker bracelets. So congratulations on that. I 0:08:22.920 --> 0:08:24.640 at least say that because you mentioned poker, did you 0:08:24.680 --> 0:08:26.080 play the World Series this year? 0:08:27.720 --> 0:08:30.240 I'm heading out actually after this. 0:08:30.160 --> 0:08:34.880 Because I know there are okay, congrats and good luck. 0:08:34.920 --> 0:08:36.680 Yeah, And it kind of connects to this because I 0:08:36.720 --> 0:08:38.600 never get to I don't get to play very much poker, 0:08:38.600 --> 0:08:44.800 but I really studied what machines were learning about poker. 0:08:45.080 --> 0:08:46.760 So much has been learned in the last five years, 0:08:46.760 --> 0:08:51.520 ten years, and and one of the you know, basically 0:08:51.520 --> 0:08:55.480 trying to translate that into intuitions that I could use, 0:08:55.600 --> 0:08:58.679 you know that basically can't actually replicate Peter Place. Poker 0:08:58.679 --> 0:09:02.640 are very complex way, but you can pull the concepts 0:09:02.679 --> 0:09:05.720 out right. And this actually mirrors to what part of 0:09:05.760 --> 0:09:08.640 what we're doing at Bridge Order, which is that as 0:09:08.679 --> 0:09:12.600 you get to computer generated theories that if you can 0:09:12.679 --> 0:09:15.960 pull the concepts out of these complex algorithms, you know, 0:09:16.040 --> 0:09:19.400 you can make more of an assessment human assessment of 0:09:19.480 --> 0:09:21.720 whether they make sense and what the problems might be. 0:09:22.200 --> 0:09:24.280 And that's really a big deal. So there's actually a 0:09:24.320 --> 0:09:29.240 link between what I'm doing in poker, imperfectly for sure, 0:09:29.360 --> 0:09:31.800 and many of the concepts that we're trying to apply 0:09:32.920 --> 0:09:36.640 at Bridge Order. And like you said, just we had 0:09:36.679 --> 0:09:40.640 talked kind of before the lms had really hit the 0:09:40.640 --> 0:09:42.599 public scene. But yeah, I mean, just to give you 0:09:42.600 --> 0:09:44.760 a little bit of background for me the you know, 0:09:44.960 --> 0:09:48.440 if you go back to twenty twelve, First off, we 0:09:48.559 --> 0:09:51.160 brought Dave Ferriucci, who had run the Watson project at 0:09:51.160 --> 0:09:54.440 IBM that had beat Jeopardy into Bridgewater, and that was 0:09:54.440 --> 0:09:56.920 that was a time when I was trying to experiment with, okay, 0:09:56.920 --> 0:09:59.480 what can we do with more machine learning techniques? And 0:09:59.600 --> 0:10:02.160 Dave was trying to take what he had done to 0:10:02.160 --> 0:10:04.160 win a Jeopardy but actually put in more of a 0:10:04.160 --> 0:10:07.920 reasoning engine, because while what happened to on Jeopardy was impressive, 0:10:07.920 --> 0:10:09.880 it was pure data. It had no idea why it 0:10:09.920 --> 0:10:13.600 was doing what it was doing, and therefore really a 0:10:13.600 --> 0:10:16.000 lot of the path with Watson or whatever was going 0:10:16.040 --> 0:10:21.000 to be very hard to move forward with because because 0:10:21.360 --> 0:10:23.680 at its end, it was just statistical and it didn't 0:10:23.720 --> 0:10:27.640 really have any reasoning capability. So Dave came to Bridge 0:10:27.720 --> 0:10:29.599 Order and later partnered with Bridge Order roll out of 0:10:29.640 --> 0:10:34.959 company Elemental Cognition that's focused on using large language models, 0:10:34.960 --> 0:10:39.680 et cetera, but overlaying a reasoning engine that essentially helps 0:10:39.679 --> 0:10:43.360 with things like hallucinations that out that large language models 0:10:43.400 --> 0:10:46.040 have and focus on how what is human reasoning and 0:10:46.040 --> 0:10:48.800 how does it work and how does that limit views 0:10:48.840 --> 0:10:52.240 that are unlikely to be true? So that's one thing, 0:10:52.240 --> 0:10:54.600 And then in twenty sixteen or seventeen, I was introduced 0:10:54.640 --> 0:10:57.520 to open ai and actually as they transition from a 0:10:57.720 --> 0:11:02.280 charity to a company. I was one of the in 0:11:02.320 --> 0:11:04.439 that first round, and it was like met a lot 0:11:04.440 --> 0:11:08.040 of the people and looked hard at their vision to 0:11:08.720 --> 0:11:12.200 using scale and technical scale to build general intelligence and 0:11:12.200 --> 0:11:16.280 build reasoning. So I both was working with Dave Rucci 0:11:16.360 --> 0:11:18.440 and sort of understood many of the people at open 0:11:18.480 --> 0:11:21.520 ai at the time and moving forward with those things. 0:11:21.520 --> 0:11:23.719 And then I was literally the first check for anthropic 0:11:23.840 --> 0:11:27.760 and other large language model kind of people that had 0:11:27.800 --> 0:11:30.880 been at open AI. And so I've been passionate about this, 0:11:31.040 --> 0:11:34.600 realized trying to take different paths to how will we 0:11:34.720 --> 0:11:39.880 build a reasoning engine to overlay on statistical things, and 0:11:40.080 --> 0:11:43.040 a couple of different approaches that were being applied at 0:11:43.040 --> 0:11:46.120 the time, and obviously different they panned out to a 0:11:46.120 --> 0:11:49.679 different degree, but many things are coming together now to say, Okay, 0:11:49.760 --> 0:11:53.680 you can actually in a way at a pace and 0:11:53.679 --> 0:11:57.200 a speed humans can never do, you could replicate human reasoning. 0:11:57.840 --> 0:12:00.440 And that's a huge deal. And if you could really 0:12:00.440 --> 0:12:02.680 break through that, you could start to apply it in 0:12:02.720 --> 0:12:05.720 so many ways in our industry, I believe, and obviously 0:12:05.760 --> 0:12:08.520 way beyond our industry. 0:12:23.679 --> 0:12:27.600 You talked about earlier generations trying to embed human knowledge. 0:12:27.600 --> 0:12:29.480 And I'm wondering, you know if an analogy is like 0:12:29.880 --> 0:12:31.840 I remember when Deep Blue came out and they had 0:12:31.880 --> 0:12:35.120 all the grand masters sort of work with IBM to 0:12:35.240 --> 0:12:37.280 like come up with this a great computer program that 0:12:37.400 --> 0:12:40.839 was basically as good or eventually better than Gary Kasparov. 0:12:41.520 --> 0:12:44.679 But then the next generation of a chess computers didn't 0:12:44.679 --> 0:12:46.960 even have the grand masters playing it. It just learned the 0:12:47.000 --> 0:12:52.199 game from ground up and crushed those crush those previous generation. 0:12:52.640 --> 0:12:54.920 Is that sort of the what we're talking about here 0:12:55.000 --> 0:12:58.839 with with the transition from earlier engines to the new 0:12:58.880 --> 0:13:01.480 sort of LLM folks, which is like the sort of 0:13:01.559 --> 0:13:04.679 reasoning come becomes comes out of the computer rather than 0:13:04.679 --> 0:13:07.199 having to be taught directly by the experts. 0:13:08.440 --> 0:13:10.920 Yeah, I think something like that is happening. Right. You 0:13:10.960 --> 0:13:13.920 got that in chess because once you had the ability 0:13:14.640 --> 0:13:17.240 you had enough data and enough compute, you were able 0:13:17.440 --> 0:13:21.679 to do enough sampling that the pure that you got 0:13:21.720 --> 0:13:24.839 to the point where the pure data process, with good 0:13:24.920 --> 0:13:27.160 human intuition on how to build that data process, but 0:13:27.960 --> 0:13:31.880 a data process, was able to beat that those rules 0:13:31.880 --> 0:13:36.000 based things. Now, chess, unlike markets, is you know a 0:13:36.000 --> 0:13:39.080 little bit more static in the sense that while while 0:13:39.080 --> 0:13:41.160 there are adversaries, and the adversaries they'll try to learn 0:13:41.200 --> 0:13:43.400 your weaknesses, it's more static in the rules of the 0:13:43.400 --> 0:13:45.880 game are steady and those types of things, so that 0:13:45.880 --> 0:13:48.800 that sampling could work right. Although it was interesting, I 0:13:48.840 --> 0:13:50.760 love the like because it is an analogy to some 0:13:50.840 --> 0:13:52.920 of the problems that pop up and will pop up 0:13:52.920 --> 0:13:55.520 if you take Alpha go right on. The Go game 0:13:56.360 --> 0:14:00.920 Go got you also after Chess, obviously, but the Google 0:14:01.040 --> 0:14:04.400 was able to create this game that was beating the 0:14:04.440 --> 0:14:07.920 pros and radically beating the pros, killing everybody and getting 0:14:07.920 --> 0:14:11.880 better and better and better, although you know, I don't 0:14:11.880 --> 0:14:13.160 know how up to the day you are. But then 0:14:13.200 --> 0:14:15.520 there was this loophole in it where that's that another 0:14:15.559 --> 0:14:20.080 person who was a mediocre Go player, but a computer 0:14:20.120 --> 0:14:23.120 scientiists who thought there might be a hole in this 0:14:23.240 --> 0:14:27.280 super AI used a little program to find the hole. 0:14:28.000 --> 0:14:30.400 And what it illustrated was the a I had no 0:14:30.440 --> 0:14:33.040 idea how to play the game, because what a six 0:14:33.080 --> 0:14:35.720 year old wouldn't The mistake the AI was prone to 0:14:35.880 --> 0:14:37.560 was a mistake of six year old playing Go would 0:14:37.600 --> 0:14:39.920 never make where if you made a large enough in circling, 0:14:40.240 --> 0:14:43.720 if you now go works, but if you encircle the 0:14:43.800 --> 0:14:49.520 other guy's pieces, right, you eliminate them all. And something 0:14:49.560 --> 0:14:51.240 that would never work in a human game is you 0:14:51.360 --> 0:14:54.320 make a really big circle. And because it never came 0:14:54.400 --> 0:14:57.960 up in human games, and because when they perturbed human 0:14:58.000 --> 0:15:02.520 games and started playing computer against computer, they basically started 0:15:02.720 --> 0:15:06.160 with a seed of human games, they never perturbed it 0:15:06.280 --> 0:15:09.040 enough to try this out, to try a massive circle, 0:15:09.680 --> 0:15:11.840 and a human would never let the massive circle have 0:15:11.920 --> 0:15:15.560 it. It's so easy to defend against. But actually the best 0:15:15.880 --> 0:15:19.040 Go algorithm in the world allowed it to happen, right, 0:15:19.080 --> 0:15:23.000 And now a mediocre Go player with a little bit 0:15:23.000 --> 0:15:26.400 of AI found a way to beat this incredible Go 0:15:26.480 --> 0:15:30.480 game again because the Go algorithm at that time had 0:15:30.560 --> 0:15:32.560 this tremendous amount of data, but the things that weren't 0:15:32.600 --> 0:15:34.840 in this data wasn't aware of and it wasn't in 0:15:34.880 --> 0:15:38.400 any deep sense understanding the principles of the game. So 0:15:38.560 --> 0:15:40.680 that's the type of you know, data problem you can 0:15:40.720 --> 0:15:42.720 have even with a massive amount of data played, you know, 0:15:43.160 --> 0:15:45.600 millions and millions of games, but to play every possible 0:15:45.640 --> 0:15:48.440 Go board, you'd have to there's more possible Go boards 0:15:48.480 --> 0:15:50.400 than there are atoms in the universe. So it was 0:15:50.440 --> 0:15:53.640 never going to calculate every possibility and it never got 0:15:53.680 --> 0:15:57.920 to reasoning, right, and that therefore that was a weakness, right. 0:15:57.960 --> 0:16:01.600 And on the other hand, you mix that blend that 0:16:01.680 --> 0:16:04.000 even with a basic reason error, that a language model 0:16:04.000 --> 0:16:06.040 could come up with understanding the rules of GO and 0:16:06.040 --> 0:16:08.800 being able to talk about it. There's an element of 0:16:09.080 --> 0:16:12.480 knowing those things that humans already know that's possible with 0:16:12.520 --> 0:16:16.120 a blend of let's say a statistical technique like alpha 0:16:16.160 --> 0:16:21.600 GO was using and a reasoner to prevent these types 0:16:21.600 --> 0:16:22.200 of mistakes. 0:16:22.960 --> 0:16:24.720 I like that story because it makes me think I 0:16:24.760 --> 0:16:29.800 have a chance against the super smart supercomputer. Okay, that's 0:16:30.000 --> 0:16:33.000 kind of comforting, But I definitely want to ask you 0:16:33.040 --> 0:16:38.160 more about weaknesses in AI and large language models, but 0:16:38.440 --> 0:16:41.400 maybe before we do, you know, just sort of setting 0:16:41.400 --> 0:16:44.720 the groundwork once again. But when we see headlines like 0:16:45.120 --> 0:16:50.880 Bridgewater restructures will put more focus on AI, what does 0:16:50.920 --> 0:16:54.000 that mean exactly? What does it mean for a firm, 0:16:54.040 --> 0:16:59.000 an investment firm like Bridgewater to build up resources in AI? 0:16:59.280 --> 0:17:02.800 And then secondly, could you walk us through a concrete 0:17:02.960 --> 0:17:08.080 example of how AI would be deployed in a particular 0:17:08.440 --> 0:17:11.720 trading strategy. I feel like the more concrete we can 0:17:11.760 --> 0:17:14.000 get with this, the more helpful it'll be. 0:17:15.000 --> 0:17:19.040 Yeah. Great. So I think as we restructured, one of 0:17:19.040 --> 0:17:21.879 the things that as we've made the transition at Bridgewater, 0:17:22.280 --> 0:17:25.680 you know, from Ray having the key ownership to ownership 0:17:25.680 --> 0:17:29.560 at a board level and that transition, we have done 0:17:29.560 --> 0:17:31.639 something we hadn't done in the past, which is essentially 0:17:32.200 --> 0:17:34.879 retain earnings in a very significant way, which allows us 0:17:34.880 --> 0:17:38.119 to invest in things that you know, are aren't going 0:17:38.160 --> 0:17:40.720 to be part profitable right right away, but are the 0:17:40.720 --> 0:17:44.960 big long term bats that we're making, and certainly recognizing 0:17:45.000 --> 0:17:47.280 that there's a way to reinvent a lot of what 0:17:47.359 --> 0:17:54.359 we do using AI machine learning techniques to improve what 0:17:54.359 --> 0:17:57.920 we're doing to understand the world, accelerate that, and specifically 0:17:58.240 --> 0:18:01.119 what we've done on the aimlside is we've set up 0:18:01.119 --> 0:18:04.440 this venture. Essentially they're seventeen of us with me leading it. 0:18:04.800 --> 0:18:06.960 You know, I'm still very much involved in core bridge Order, 0:18:06.960 --> 0:18:10.680 but the sixteen others are one hundred percent dedicated to 0:18:12.280 --> 0:18:15.359 kind of reinventing Bridge Order in a way with machine learning. 0:18:15.359 --> 0:18:17.760 We're going to have a fund specifically run by machine 0:18:17.800 --> 0:18:19.919 learning techniques which will take me into tracy what kind 0:18:19.920 --> 0:18:22.520 of strategies you can do, you know, that's what we're 0:18:22.520 --> 0:18:26.760 working on right now in that lab and pressing the 0:18:27.080 --> 0:18:29.760 edges of what AI is capable of now a like 0:18:29.840 --> 0:18:32.639 machine learning is capable of now right now. There are 0:18:32.680 --> 0:18:36.640 big problems right A. You take large language models and 0:18:37.200 --> 0:18:39.800 they have two types of problems. One thing is the 0:18:39.840 --> 0:18:43.480 basic problem is there they are trained on the structure 0:18:43.480 --> 0:18:46.399 of language, so they usually return something that looks like 0:18:47.160 --> 0:18:50.880 good structure of language. They don't always return accurate answers, 0:18:51.080 --> 0:18:53.520 so that's a problem. It hallucinates, It makes things up 0:18:53.720 --> 0:18:56.160 because it's more focused on the structure of what word 0:18:56.480 --> 0:18:59.440 or what concept would come next, then whether it's accurate 0:18:59.560 --> 0:19:00.520 in what comes. 0:19:00.760 --> 0:19:04.199 Can I just say when I hear AI hallucinations, it 0:19:04.320 --> 0:19:07.840 becomes so science fiction for me. It's very like robot 0:19:07.920 --> 0:19:11.600 stream of electric cheap kind of it's just so surreal. 0:19:13.320 --> 0:19:15.520 Yeah, well, I mean in this case, you can imagine 0:19:15.600 --> 0:19:17.720 what's happening, right because it's just what it's what it's 0:19:17.800 --> 0:19:22.560 trained on. Right. So if you're just if basically the 0:19:22.600 --> 0:19:25.639 basic concept is give it any stream of words and 0:19:25.720 --> 0:19:28.560 it'll predict based on having read everything that's ever been read. 0:19:28.880 --> 0:19:32.440 What comes next, right, and that if it's a little 0:19:32.480 --> 0:19:37.320 bit wrong in what comes next, it can misfire and 0:19:37.359 --> 0:19:40.120 give you something that sounds like something that could come next, 0:19:40.119 --> 0:19:42.480 but actually wrong, you know. And it's just what it's 0:19:42.480 --> 0:19:44.520 trained on, right, It's trained to predict the next word. 0:19:44.760 --> 0:19:48.360 Slight errors in that create those types of issues. Now, 0:19:48.359 --> 0:19:53.000 the algorithm is pretty remarkable, particularly like we as I said, 0:19:53.000 --> 0:19:57.040 I've been tracking in AI as an investor for a 0:19:57.040 --> 0:19:59.840 long time and looking at their technology for a long 0:19:59.880 --> 0:20:05.880 time time. And you know, up until there's GPT one, two, three, 0:20:05.920 --> 0:20:08.640 and many versions of between, and then at GPT three 0:20:08.720 --> 0:20:10.760 it started to have some use. GPT one and two 0:20:10.760 --> 0:20:14.280 were you know, barely coherent, GPT three was you know, 0:20:14.400 --> 0:20:17.240 somewhat usable for certain tasks. Three and a half, which 0:20:17.280 --> 0:20:20.359 is what CHAT GPT is, you know, got to a 0:20:20.359 --> 0:20:23.520 certain level, like on Bridgewater's internal tests, you suddenly got 0:20:23.520 --> 0:20:27.000 to the point where it was able to answer our 0:20:27.160 --> 0:20:29.960 investment associate tests at the level of a first year 0:20:30.400 --> 0:20:33.240 IA right around with chat GPT three point five and 0:20:34.359 --> 0:20:39.200 anthropics most recent quad and then GPT four was able 0:20:39.280 --> 0:20:42.199 to do significantly better. And these are you know, at 0:20:42.280 --> 0:20:45.320 least what we thought were conceptual tests significantly better than 0:20:45.359 --> 0:20:47.800 our average you know, first year investment associate that went 0:20:47.840 --> 0:20:52.200 through training. And similarly, it's able to take the LSAD 0:20:52.240 --> 0:20:55.320 and do well, et cetera. So it can be basically 0:20:55.680 --> 0:20:58.480 pretty smart. It is pretty smart in a wide variety 0:20:58.520 --> 0:21:00.880 of things with errors, but pretty smart on a wide 0:21:00.960 --> 0:21:05.240 variety of whether it's BMCAT or the LSAD or Bridgewaters 0:21:05.359 --> 0:21:07.960 internal tests or whatever, a whole wide variety of things. 0:21:08.359 --> 0:21:10.920 This is a big deal that it can achieve all 0:21:10.960 --> 0:21:15.200 of those kind of academic things, and yet it's still 0:21:15.200 --> 0:21:17.480 eightieth percentile kind of thing on a lot of those things, 0:21:17.520 --> 0:21:20.440 which is remarkable to be eightieth percentile on many many 0:21:20.440 --> 0:21:23.640 different things. But at the same time, it's eightieth percentile 0:21:23.680 --> 0:21:26.560 for a reason. There are flaws, meaning it's not one 0:21:26.640 --> 0:21:30.640 hundred percentile, and so that leads to like you need 0:21:30.680 --> 0:21:34.960 to find a way to work through those flaws, right, 0:21:35.000 --> 0:21:38.280 and that's really where you know. So if somebody's going 0:21:38.359 --> 0:21:41.000 to use large language models to pick stocks, I think 0:21:41.080 --> 0:21:44.760 that's hopeless. That is a hopeless path. But if you 0:21:44.880 --> 0:21:49.760 use large language models to create some theories which it 0:21:49.800 --> 0:21:53.919 can theorize about things, and you use other techniques to 0:21:54.160 --> 0:21:57.880 judge those theories and you iterate between them to create 0:21:57.920 --> 0:22:01.160 a sort of an artificial reasoner. Where language models are 0:22:01.160 --> 0:22:04.800 good at certainly generating theories any theories that already exist 0:22:04.880 --> 0:22:09.120 in human knowledge, and putting those things connect together, they're 0:22:09.160 --> 0:22:12.320 bad at determining whether they're true. But there are other 0:22:12.359 --> 0:22:16.560 ways to pair it with statistical models and other types 0:22:16.560 --> 0:22:19.359 of AI to combine those together. And that's really what 0:22:19.400 --> 0:22:22.360 we're focused on, which is combining large language models that 0:22:22.800 --> 0:22:26.400 are bad at precision with statistical models that are good 0:22:26.440 --> 0:22:29.560 at being precise about the past but terrible about the future, 0:22:30.119 --> 0:22:33.760 and combining those together you start to build an ecosystem 0:22:34.200 --> 0:22:38.760 that can achieve I believe can achieve the types of 0:22:38.840 --> 0:22:42.800 things that bridge order analysts combined with our stress testing 0:22:42.840 --> 0:22:46.320 process and compounding understanding process at Bridgeworker can do, but 0:22:46.400 --> 0:22:48.520 it can do it at so much more scale, because 0:22:48.520 --> 0:22:51.040 all of a sudden, if you have an eightieth percentile 0:22:51.119 --> 0:22:55.119 investment associate, technologically you have millions of them at once, 0:22:55.600 --> 0:23:00.760