WEBVTT - Josh Wolfe on Where Investors Will Make Money in AI 0:00:10.200 --> 0:00:14.200 Hello, and welcome to another episode of the Odd Lots Podcast. 0:00:14.240 --> 0:00:16.600 I'm Joe Wisenthal and I'm Tracy Alloway. 0:00:16.840 --> 0:00:19.960 Tracy, I mean, I think it goes without saying that 0:00:20.040 --> 0:00:24.439 the appetite and interest in anything related to AI and 0:00:24.520 --> 0:00:27.280 making money from AI continues unabated. 0:00:27.960 --> 0:00:30.920 Yes, I think that's accurate. Well, it's interesting because you 0:00:31.000 --> 0:00:33.760 kind of see it from two different sides at the moment. 0:00:33.840 --> 0:00:35.760 So there are a lot of companies that are talking 0:00:35.800 --> 0:00:40.440 about investing internally in AI technology, and then there are 0:00:40.440 --> 0:00:45.080 a lot of investors talking about investing in AI in 0:00:45.159 --> 0:00:46.040 one way or another. 0:00:46.560 --> 0:00:48.880 I mean needless to say. You know, you like throw 0:00:48.920 --> 0:00:51.440 a dart at nasdextoc some they're talking about, you know, 0:00:51.720 --> 0:00:54.760 the way AI incorporated. It's also funny because like you 0:00:54.880 --> 0:00:57.400 like read the uh the earnings transcript of like a 0:00:57.440 --> 0:01:01.280 grocery storagechep. Yes, and the seats we'll be talking about, 0:01:01.320 --> 0:01:04.200 you know, how AI is going to help them. 0:01:04.400 --> 0:01:05.280 Wasn't this Kroger? 0:01:05.520 --> 0:01:07.400 I think so. And to be fair, like I think 0:01:07.440 --> 0:01:10.560 they actually kind of legitimately have been investing. So it's 0:01:10.760 --> 0:01:15.039 totally but like they mentioned AI one company I don't 0:01:15.480 --> 0:01:17.240 like mention it like a dozen times. 0:01:17.000 --> 0:01:18.959 I'm pretty sure it was Kroger. Yeah, But I think 0:01:19.000 --> 0:01:21.319 I've said this on the podcast before. It does feel 0:01:21.440 --> 0:01:23.360 like there are some people out there who at this 0:01:23.440 --> 0:01:27.039 point are basically using AI as a synonym for any 0:01:27.080 --> 0:01:30.560 type of software, just like we use software, we're using AI. 0:01:31.040 --> 0:01:36.320 But it begs the question of how to smartly invest 0:01:36.400 --> 0:01:38.720 in a technology that clearly a lot of people are 0:01:38.840 --> 0:01:42.600 enthused about. But it's also kind of hard to disaggregate 0:01:42.680 --> 0:01:45.400 a lot of the marketing from the reality. 0:01:45.560 --> 0:01:47.480 Well, and the thing that gets me in that I'm 0:01:47.520 --> 0:01:50.720 still trying to wrap my head around too, is like, 0:01:51.240 --> 0:01:54.040 especially for the big tech incumbents, And I'm thinking of 0:01:54.120 --> 0:01:57.360 like Alphabet, Google or whatever, Like they have an amazing 0:01:57.400 --> 0:02:00.640 business model right now, right, like people search for something 0:02:00.760 --> 0:02:04.080 and then you're telling the machine exactly what you are 0:02:04.120 --> 0:02:06.360 looking for, and then the machine knows, okay, well, here 0:02:06.360 --> 0:02:08.920 are some ads that we can put And I know, 0:02:09.080 --> 0:02:12.120 like obviously Google is you know, a head very like 0:02:12.240 --> 0:02:15.000 front of the curve in terms of AI tech, and 0:02:15.040 --> 0:02:17.160 they have their own large language models and all of 0:02:17.160 --> 0:02:19.760 that stuff. But does anyone know that like this is 0:02:19.800 --> 0:02:22.120 going to turn into like a money making thing for them? 0:02:22.400 --> 0:02:22.480 Right? 0:02:22.680 --> 0:02:25.720 Will it be anywhere close to this amazing money printing 0:02:25.760 --> 0:02:29.200 machine that they built when they built the Google search box. 0:02:29.360 --> 0:02:31.600 Well, I think that's a really good question. And also, 0:02:31.760 --> 0:02:35.480 so far we have seen the incumbents come out as 0:02:35.560 --> 0:02:39.480 the big winners of a lot of this new technology, 0:02:39.880 --> 0:02:43.560 and I think a that's been unexpected. If you'd ask someone, 0:02:43.720 --> 0:02:45.359 you know, five years ago, who was going to be 0:02:45.400 --> 0:02:47.840 the big winner in AI, I don't think anyone would 0:02:47.880 --> 0:02:52.280 have said microsom right, right, And so that also raises 0:02:52.280 --> 0:02:54.440 a question of Okay, how did the giants monetize this? 0:02:54.639 --> 0:02:57.480 To your point? And then secondly, are there going to 0:02:57.520 --> 0:03:00.600 be new players who somehow come in and find a 0:03:00.600 --> 0:03:01.560 better way to do it right? 0:03:01.639 --> 0:03:03.440 Like how disruptive it will be? 0:03:03.760 --> 0:03:04.200 Yeah? 0:03:04.240 --> 0:03:07.040 Well I am excited because I believe we do have 0:03:08.120 --> 0:03:11.000 the perfect guest. We're going to be speaking with Josh Wolf, 0:03:11.280 --> 0:03:15.000 founding partner and managing director at Lux Capital, who has 0:03:15.080 --> 0:03:19.080 been investing in AI since long before it was cool, 0:03:19.120 --> 0:03:23.040 long before everyone started asking like chat gpt to like, 0:03:23.440 --> 0:03:26.040 you know, write a song about the FED and the 0:03:26.080 --> 0:03:29.000 style of Johnny Cash or whatever like long before No 0:03:29.160 --> 0:03:34.320 I feel seen, I feel I'm describing myself. I'm describing 0:03:34.360 --> 0:03:36.400 both of us long before we were all doing that, 0:03:36.840 --> 0:03:38.560 and so he's going to talk to us about how 0:03:38.600 --> 0:03:42.000 he thinks about making money in AI and where the 0:03:42.080 --> 0:03:45.160 value is going to crue in identifying investments. So, Josh, 0:03:45.200 --> 0:03:47.000 thank you so much for coming on Odd Laps. 0:03:47.320 --> 0:03:48.800 Joe Tracy, great to be with you. 0:03:48.960 --> 0:03:51.000 Can I ask you a question for real? Like we 0:03:51.120 --> 0:03:54.440 obviously have this boom, etc. But the tech's been around. 0:03:54.520 --> 0:03:57.360 Did people basically just get excited because like someone finally 0:03:57.360 --> 0:04:00.120 put a good ux on top of this technology for 0:04:00.120 --> 0:04:02.200 a while and suddenly like, oh my god, Like was 0:04:02.280 --> 0:04:07.280 that sort of what catalyzed this current stage of enthusiasm. 0:04:08.520 --> 0:04:12.200 I think the lay answer is what catalyzed this was 0:04:12.240 --> 0:04:15.560 a sense of conjuring magic. People felt like they were 0:04:15.640 --> 0:04:18.560 effectively casting spells, like you said, whether it was a 0:04:18.640 --> 0:04:20.840 Johnny Cash song conjured to you know, to talk about 0:04:20.839 --> 0:04:24.080 the FED, but it's the feeling that somebody had a superpower. 0:04:24.240 --> 0:04:26.200 So I think that's what catalyzed that. Where it was 0:04:26.240 --> 0:04:29.080 a feeling of positive surprise where people were like, oh 0:04:29.120 --> 0:04:32.080 my god, like I just created magic and that you know, 0:04:32.120 --> 0:04:36.599 classic cliche that any sufficiently advanced tech is indistinguishable for magic. 0:04:36.640 --> 0:04:40.440 This felt like magic now the roots of it go 0:04:40.560 --> 0:04:44.520 back over a decade, and that's what the public doesn't see. 0:04:44.560 --> 0:04:46.560 And part of that is the bones, and part of 0:04:46.600 --> 0:04:49.960 it is the brains the tech infrastructure. So you start 0:04:50.279 --> 0:04:54.039 with the GPUs. Now, we've known computing for decades and 0:04:54.120 --> 0:04:58.400 Intel was the dominant force. Intel made CPU central processing units, 0:04:58.600 --> 0:05:00.120 and there was this thing off on the side that 0:05:00.200 --> 0:05:02.599 was just doing graphic processing and it was for video games. 0:05:03.080 --> 0:05:06.440 I've got this mental model, this framework where a lot 0:05:06.480 --> 0:05:10.120 of people say that the most dangerous words in investing are, 0:05:10.400 --> 0:05:13.560 this time is different. I actually think that there's a 0:05:13.640 --> 0:05:15.960 secret that people can follow, which is that the most 0:05:16.040 --> 0:05:19.719 valuable words are whenever you hear a parent say it 0:05:19.760 --> 0:05:23.520 will rot your brain, that basically presages it predicts the 0:05:23.560 --> 0:05:26.400 next ten billion dollar industry. So think about this. I 0:05:26.400 --> 0:05:30.800 mean literally nineteen sixties those hip shaken, Johnny Cash, Elvis, 0:05:30.839 --> 0:05:32.919 you know, rock and Roll, It'll rot your brain. Boom 0:05:33.000 --> 0:05:37.280 ten billion dollar industry. You know, seventies, you know personal 0:05:37.320 --> 0:05:40.960 computers and chat rooms, and and you know nineties the 0:05:41.000 --> 0:05:45.520 Internet and these online chat rooms and then gaming. My god, 0:05:45.600 --> 0:05:47.880 you know, these kids are turning into couch potatoes. Get 0:05:47.920 --> 0:05:50.839 them off the video games, the video game players of 0:05:50.920 --> 0:05:54.080 yesterday are today's you know, robotic surgeons and drone pilots. 0:05:54.080 --> 0:05:56.600 But what's really important is the tech that was underlying 0:05:56.600 --> 0:06:00.720 that these massively multiplayer games and people demand and ever 0:06:00.839 --> 0:06:07.200 higher video resolution and PlayStation competing with Xbox, completing with Nintendo. 0:06:07.640 --> 0:06:10.640 It created these chips that took in video from a 0:06:10.800 --> 0:06:14.040 fifteen billion dollar market cap at the time when Intel 0:06:14.120 --> 0:06:16.120 was one hundred and fifty billion a few years ago, 0:06:16.480 --> 0:06:20.120 and today it's a trillion dollar business. We had invested 0:06:20.120 --> 0:06:24.039 in this company going back about eight years. It was 0:06:24.120 --> 0:06:27.960 four people off the Stanford campus, your classic garage, and 0:06:28.000 --> 0:06:30.120 they were literally in a garage at the slack the 0:06:30.120 --> 0:06:33.880 Stanford Linear Accelerator, sort of secret group, and they were 0:06:33.920 --> 0:06:36.640 trying to develop self driving cars. And we had put 0:06:36.680 --> 0:06:40.160 about twenty five million into this small team called Zookes, 0:06:40.640 --> 0:06:42.839 which was a zoo for robotics. It was a silly name. 0:06:43.400 --> 0:06:45.960 And we go in there and I see all these 0:06:45.960 --> 0:06:48.400 people playing video games and I start to get a 0:06:48.440 --> 0:06:50.359 little bit upset. I say, we just put a lot 0:06:50.400 --> 0:06:52.960 of money into this company. What are all these engineers doing. 0:06:53.440 --> 0:06:55.400 And the founder turned to me and said, no, no, 0:06:55.440 --> 0:06:58.640 you don't understand the cars that you see outside on 0:06:58.720 --> 0:07:01.720 these tracks that are running, they're ingesting information at the 0:07:01.800 --> 0:07:05.120 rate of one second per second, what we call reality, 0:07:05.480 --> 0:07:08.560 and they're taking light ar and radar and visual cues 0:07:08.640 --> 0:07:11.720 and thermal sensors and vibrational sensors and all that and 0:07:11.720 --> 0:07:14.920 they're processing it. But inside these rooms air conditioned, these 0:07:14.960 --> 0:07:17.440 people are not playing video games. This is not grand 0:07:17.480 --> 0:07:20.200 theft or this is not call of duty. The machines 0:07:20.240 --> 0:07:23.160 are actually running simulations and they are training the cars 0:07:23.280 --> 0:07:26.080 thousands of simulations a second, and the machines don't know 0:07:26.080 --> 0:07:28.880 the difference between reality and simulation. And I was like, 0:07:28.920 --> 0:07:30.920 oh my god, okay, this is pretty amazing. What are 0:07:30.960 --> 0:07:32.960 these things running on And they said, those two guys 0:07:33.000 --> 0:07:34.880 over there are from this company, in Vidia, and we 0:07:34.920 --> 0:07:37.160 have chips that aren't on the market yet and they 0:07:37.160 --> 0:07:39.960 are able to do processing like never before. And that 0:07:40.160 --> 0:07:43.920 sent us on this path as investors in AI. And 0:07:44.560 --> 0:07:48.080 from there we found this amazing team that was developing 0:07:48.120 --> 0:07:51.920 like the next gen GPUs and was a guy Navine Row. 0:07:52.120 --> 0:07:55.400 He had a company called Nirvana Systems. Intel buys them 0:07:55.440 --> 0:07:57.360 within a year of US investing for about three hundred 0:07:57.360 --> 0:08:01.240 and fifty million, becomes the core kernel of Intel's AI system. 0:08:01.680 --> 0:08:04.320 We going back that guy again, and just last week 0:08:04.560 --> 0:08:07.160 Data Bricks bought his new company called Mosaic for a 0:08:07.200 --> 0:08:10.920 billion three congratulations, thank you, wild Ride. But you just 0:08:10.960 --> 0:08:14.239 try to find these people that have this irreverent view 0:08:14.840 --> 0:08:16.800 and they sort of see the future and they invented, 0:08:16.840 --> 0:08:17.840 and you get behind them. 0:08:18.200 --> 0:08:21.600 So maybe if I could just step back for a second, 0:08:21.960 --> 0:08:24.840 and this will sort of maybe tell us a little 0:08:24.840 --> 0:08:26.920 bit more about what you're doing in the space. But 0:08:27.000 --> 0:08:32.160 how do you evaluate AI opportunities between the hardware? So 0:08:32.320 --> 0:08:35.000 the chips where they're so far seems to have been 0:08:35.200 --> 0:08:38.319 a lot of excitement and activity versus some of the 0:08:38.920 --> 0:08:41.200 software and the sort of underlying models. 0:08:41.520 --> 0:08:43.520 Today in video really has the lead. It's very hard 0:08:43.559 --> 0:08:47.240 for people to compete. Obviously, there's all kinds of considerations 0:08:47.280 --> 0:08:51.680 of geopolitical dependency and TSMC and ASML and who's helping 0:08:51.720 --> 0:08:55.200 to make these chips. But there's an entire very sophisticated 0:08:55.280 --> 0:09:00.960 stack of semicab equipment manufacturing, IP design so that people 0:09:00.960 --> 0:09:03.679 can make these chips and then the chips themselves. And 0:09:03.880 --> 0:09:06.320 these things are very expensive. I mean, these h one 0:09:06.400 --> 0:09:09.199 hundred chips from Nvidia one hundred thousand dollars. They are 0:09:09.200 --> 0:09:12.680 in scarce supply. One of the other really interesting things 0:09:12.720 --> 0:09:15.280 right now in this chip domain that people should watch for. 0:09:15.960 --> 0:09:17.400 And then I'll tell you where I think in Vidia 0:09:17.480 --> 0:09:20.400 is actually quite vulnerable and they're not just pure monopoly here. 0:09:21.400 --> 0:09:23.880 Anytime that there's hype in a sector, just like you 0:09:23.880 --> 0:09:26.480 were talking about, you know, Kroger's adding AI to their name. 0:09:26.640 --> 0:09:28.280 You saw this in the dot coms, you saw it 0:09:28.320 --> 0:09:32.800 in the Internet, you saw it in mobile everything exactly. 0:09:33.200 --> 0:09:35.680 And you know, look, they lower the cost of capital, 0:09:35.960 --> 0:09:40.000 they take advantage of people's irrationality, they capitalize. And what 0:09:40.120 --> 0:09:42.920 happens in every field, the hype gets high, the cost 0:09:42.960 --> 0:09:45.760 of capital gets low, Hundreds, if not thousands of new 0:09:45.760 --> 0:09:48.560 companies get funded, ninety nine percent of them fail, and 0:09:48.600 --> 0:09:51.440 from that detritus it becomes the comminatorial fodder for the 0:09:51.440 --> 0:09:56.000 next wave. So interestingly, with crypto, crypto people were like 0:09:56.200 --> 0:09:58.880 clamoring for these GPUs. They couldn't get enough of them 0:09:58.920 --> 0:10:01.920 so that they could do bitcoin mining. As that market went, 0:10:02.120 --> 0:10:05.439 you know, hyperbolic, and then crashed. Now you have all 0:10:05.440 --> 0:10:08.760 these xsgpus and you're starting to read headlines about the 0:10:08.760 --> 0:10:11.360 bitcoin miners that are selling their GPUs now to the 0:10:11.400 --> 0:10:13.680 AI researchers, and they're able to do it now, you know, 0:10:13.760 --> 0:10:16.080 cents on the dollar of what they paid before. So 0:10:16.240 --> 0:10:20.160 on the hardware side, that's very interesting. But our bed 0:10:20.280 --> 0:10:22.439 is that there's something different that's happening than betting on 0:10:22.480 --> 0:10:26.640 the next chip. Okay, there's More's law which everybody knows about. 0:10:26.720 --> 0:10:29.080 There's Rocks Law, which basically is that the cost of 0:10:29.120 --> 0:10:32.840 these semiconductor fabs increases exponentially even as our chips get cheaper. 0:10:32.840 --> 0:10:33.960 And sorry, which law? 0:10:34.559 --> 0:10:37.840 Which law Rocks named after Arthur Rock, who is one 0:10:37.840 --> 0:10:40.440 of the you know, first vcs, one of the first 0:10:40.679 --> 0:10:45.320 funders of Intel in East Coast og VC. So they 0:10:45.400 --> 0:10:48.040 called it Rocks law because basically, the cost to build 0:10:48.040 --> 0:10:50.960 these fabs, to make the chips keeps getting more expensive 0:10:51.000 --> 0:10:52.839 every year and a half or two years. It used 0:10:52.840 --> 0:10:54.720 to be you know, one hundred million, then it's a billion, 0:10:54.800 --> 0:10:56.840 and it's ten billion and so on. Okay, why is 0:10:56.840 --> 0:11:00.319 that relevant to make these foundation models that we're all 0:11:00.400 --> 0:11:03.920 using behind the scenes, GPT three and GPT four and 0:11:03.960 --> 0:11:06.720 what comes next. It used to be a few million 0:11:06.760 --> 0:11:10.120 dollars maybe ten million to train GPT three GPT four 0:11:10.720 --> 0:11:13.160 is estimated in the low hundreds of millions of dollars. 0:11:13.520 --> 0:11:16.160 And whatever comes next, people believe is going to cost 0:11:16.200 --> 0:11:18.480 about a billion dollars. Why because they have to buy 0:11:18.520 --> 0:11:21.119 all these in video chips. So in video is telling everybody, 0:11:21.280 --> 0:11:23.160 you've got to get these a one hundred chips. We 0:11:23.200 --> 0:11:26.280 make these h one hundred chips. We make. The reality 0:11:26.520 --> 0:11:29.200 is actually that there's this interesting vulnerability. This is where 0:11:29.240 --> 0:11:31.280 we make our speculative bets. Now we might be wrong, 0:11:31.320 --> 0:11:34.000 but this is what we're betting. We're betting that that's 0:11:34.040 --> 0:11:36.120 not going to be the case, that it's not going 0:11:36.160 --> 0:11:39.160 to be just the domain of open AI, that it's 0:11:39.200 --> 0:11:41.320 not going to be anthropic, that it's not going to 0:11:41.320 --> 0:11:43.040 be just the big giants. And we'll talk about how 0:11:43.040 --> 0:11:45.960 those guys are all intertwined, like you said, with the 0:11:46.000 --> 0:11:48.719 Microsofts and the Googles and the Metas, etc. Because that's 0:11:48.720 --> 0:11:52.160 an interesting dynamic. In Vidia has a language, a computer 0:11:52.280 --> 0:11:55.520 language that people program on and it's called Kuda Cuda, 0:11:56.320 --> 0:11:59.520 and this has been the dominant form, but it's really vulnerable, 0:11:59.720 --> 0:12:04.400 and it's vulnerable interestingly because of Facebook Meta. They came 0:12:04.480 --> 0:12:07.240 up with this language called Pi Torch and a lot 0:12:07.280 --> 0:12:10.200 of the developers are moving to pitorch. It's open source, 0:12:11.160 --> 0:12:14.280 it allows people to do a lot of the AI processing, 0:12:14.360 --> 0:12:17.120 but hardware agnostic, meaning you don't have to use an 0:12:17.120 --> 0:12:19.439 in video chip in video. If you use in video chip, 0:12:19.440 --> 0:12:22.480 you've got a program on Kuda. These guys are saying, 0:12:22.679 --> 0:12:25.199 we can use an AMD chip, we can use an 0:12:25.240 --> 0:12:29.880 ASK an application specific integrated circuit. They are saying, we're 0:12:29.920 --> 0:12:32.160 not going to be beholden to this. So there's two 0:12:32.200 --> 0:12:35.360 competing software languages that are merging sort of quietly. One 0:12:35.440 --> 0:12:38.840 is called PyTorch and one is called Triton. And Triton 0:12:38.880 --> 0:12:41.880 is from open Ai. People probably trust that one a 0:12:41.920 --> 0:12:44.480 little bit less, and Pietorch is totally open source but 0:12:45.080 --> 0:12:47.480 originated from Meta, which is really interesting. 0:13:03.800 --> 0:13:06.000 Man, I'm already learning a lot because I was not 0:13:06.120 --> 0:13:10.320 familiar with PyTorch. You know. So obviously, as you mentioned, 0:13:10.360 --> 0:13:14.920 and we talked about GPT three, GPT four, chat GPT, 0:13:15.360 --> 0:13:20.120 this magical search box that got that got everyone's attention. 0:13:20.360 --> 0:13:22.160 This sort of like what came out of open Ai. 0:13:22.280 --> 0:13:24.719 But there are others that are building chat pots. How 0:13:24.720 --> 0:13:27.080 many winners can there be? A like sort of like 0:13:27.360 --> 0:13:30.240 how do you think about winners? Either at the foundational 0:13:30.360 --> 0:13:34.640 model level, like if we started odd lots GPT, is 0:13:34.640 --> 0:13:38.319 that a worthwhile area or are you thinking like some 0:13:38.400 --> 0:13:40.720 of these problems are kind of solved and it makes 0:13:40.760 --> 0:13:44.040 more sense to focus on building something on top of 0:13:44.080 --> 0:13:47.720 one of these like core winners like GPT four and 0:13:47.720 --> 0:13:50.960 build some sort of a specific application for an industry 0:13:51.360 --> 0:13:54.360 that uses a foundational model that already exists. 0:13:54.600 --> 0:13:57.160 You're thinking exactly right, and you know you should be 0:13:57.160 --> 0:13:59.600 a VC because we're betting on the ladder that you're 0:13:59.640 --> 0:14:04.079 gonna start to get these generalized models which are wowing everybody, 0:14:04.200 --> 0:14:05.760 although you know, you start to look at some of 0:14:05.760 --> 0:14:08.680 the usage pattern classic thing, right, people get really excited 0:14:08.679 --> 0:14:10.560 about the thing, then it starts to die off. Maybe 0:14:10.559 --> 0:14:13.080 it's the summer, but maybe it's reached a little bit 0:14:13.120 --> 0:14:16.600 of a plateau of incremental interest. Okay, so let's break 0:14:16.600 --> 0:14:19.160 this down in terms of the models. You've got open AI, 0:14:19.360 --> 0:14:21.280 which will continue to invest a huge amount of money, 0:14:21.320 --> 0:14:24.640 continue to develop models, continue to wow people. Their next 0:14:24.640 --> 0:14:27.400 thing will be quote unquote multi modal instead of just 0:14:27.480 --> 0:14:30.880 text or voice transcription. You'll start to have all kinds 0:14:30.880 --> 0:14:33.280 of interesting things where like you said, you know, make 0:14:33.360 --> 0:14:36.800 me the Johnny Cash song, give me a full music video, 0:14:38.480 --> 0:14:40.560 print out, all kinds of crazy images. You know, it'll 0:14:40.640 --> 0:14:43.440 it'll do four different things, and that'll be really limited 0:14:43.440 --> 0:14:45.680 by people's creativity. But it's going to be general. Now. 0:14:45.720 --> 0:14:48.560 One of the problems with the general stuff GPT four 0:14:48.680 --> 0:14:51.640 was Plint was trained on the public Internet, and proud 0:14:51.680 --> 0:14:53.320 of the problem in the public Internet is that you 0:14:53.360 --> 0:14:54.880 got a lot of information, but you also have a 0:14:54.920 --> 0:14:57.840 lot of misinformation. It was trained on Reddit and Twitter 0:14:57.880 --> 0:15:00.640 and all kinds of repositories of public info, and so 0:15:00.760 --> 0:15:03.360 it's going to hallucinate. It's going to give you bs answers. 0:15:03.800 --> 0:15:07.320 So you have people saying, Okay, that's a problem, there's 0:15:07.360 --> 0:15:10.320 white space, let me solve it. And it's probably going 0:15:10.400 --> 0:15:13.800 to be financial and healthcare is our guests, where you 0:15:13.880 --> 0:15:18.240 get very specialized models that need to have high accuracy, 0:15:18.680 --> 0:15:20.960 and they're going to be smaller models. So instead of 0:15:21.000 --> 0:15:24.240 these giant models, they're going to be smaller, more bespoke, 0:15:24.720 --> 0:15:28.800 more industry verticalized. Even Bloomberg Bloomberg GPT people I think 0:15:28.840 --> 0:15:31.160 are really fascinated by what that's going to portend. Because 0:15:31.200 --> 0:15:34.400 you have a proprietary data set, you've got a locked 0:15:34.440 --> 0:15:36.320 and user base and sort of a social network and 0:15:36.360 --> 0:15:39.600 you have reliable, high quality data. So I think that's 0:15:39.640 --> 0:15:41.640 going to be the next wave. It's going to happen 0:15:41.680 --> 0:15:44.360 in financial data. My bet is on Bloomberg. It's going 0:15:44.440 --> 0:15:47.240 to be in healthcare with some of the major healthcare systems, 0:15:47.840 --> 0:15:51.360 and I think that that's sort of the next wave. Now, 0:15:51.800 --> 0:15:54.520 when you look at the big players today with the 0:15:54.520 --> 0:15:57.800 big foundation models, open ai, if you're really honest about it, 0:15:58.240 --> 0:16:02.160 they're captive to Microsoft. Microsoft did an incredibly clever deal. 0:16:02.280 --> 0:16:04.920 They knew that there's no way that the DOJA or 0:16:04.960 --> 0:16:07.840 the FTC would allow them to actually acquire open ai, 0:16:08.000 --> 0:16:09.600 so they structured a deal in a way that they 0:16:09.600 --> 0:16:14.040 effectively control it, but without doing an acquisition. You look 0:16:14.080 --> 0:16:17.840 at Google, they're closely tied up with Anthropic, and a 0:16:17.840 --> 0:16:20.120 lot of these deals are interesting because what happens is 0:16:20.160 --> 0:16:23.360 the company gets a giant equity investment. In this case, 0:16:23.360 --> 0:16:25.640 I think Anthropic got about three hundred million from Google, 0:16:25.840 --> 0:16:29.720 but that money sored around trips. Google gets equity, Anthropic 0:16:29.720 --> 0:16:32.640 gets cash. That cash then goes back to Google and 0:16:32.720 --> 0:16:35.040 is spent on compute, so they get to book it 0:16:35.040 --> 0:16:39.480 as revenue in Google Cloud. Now Meta is really interesting 0:16:39.520 --> 0:16:41.960 because just like you said before, Tracy, nobody would have 0:16:42.000 --> 0:16:44.600 thought Microsoft was the leader or would be a leader 0:16:44.720 --> 0:16:49.520 in AI. Meta, you know, has been under congressional scrutiny 0:16:49.600 --> 0:16:52.680 and has been the sort of evil villain of consumer 0:16:52.800 --> 0:16:56.080 and social media and disrupting and destroying our democracy and 0:16:56.120 --> 0:16:58.320 all this stuff. And then they made this bet on 0:16:58.360 --> 0:17:02.160 the metaverse, which nobody cares about, but this idea of 0:17:02.200 --> 0:17:06.080 this fetiverse that they're starting to talk about with threads. 0:17:06.480 --> 0:17:09.840 It's really interesting because they are embracing this idea of 0:17:09.880 --> 0:17:12.760 open source. Now. They're not doing it benevolently because they 0:17:12.760 --> 0:17:14.520 think it's a good thing. It's in their self interest. 0:17:14.880 --> 0:17:16.800 They want to be the sort of network that is 0:17:16.800 --> 0:17:19.440 connected to everything else. They don't want to be siloed, 0:17:19.680 --> 0:17:21.320 and they get to use it as a little bit 0:17:21.320 --> 0:17:24.800 of achine to stave off the regulatory scrutiny and the 0:17:24.840 --> 0:17:27.560 public criticism. But I think you've got to watch Meta 0:17:27.640 --> 0:17:31.119 really closely. In the coming weeks, new releases of open 0:17:31.160 --> 0:17:33.760 source models that are going to really compete with open AI, 0:17:34.359 --> 0:17:38.520 lots of partnerships with interesting companies knowing that they themselves 0:17:38.640 --> 0:17:40.520 couldn't possibly do an acquisition. 0:17:40.840 --> 0:17:43.520 You mentioned data just then, And this is something I've 0:17:43.520 --> 0:17:47.400 been thinking about. But when it comes to AI technology, 0:17:47.520 --> 0:17:53.280 what's the most important factor? Is it access to reliable 0:17:53.320 --> 0:17:56.720 data as you mentioned, maybe reliable and exclusive sources of 0:17:56.720 --> 0:17:59.719 big data, or is it the sort of like underlying 0:18:00.080 --> 0:18:04.240 modeling technology. And I guess another way of framing it 0:18:04.320 --> 0:18:06.560 is like, are the big winners going to just be 0:18:06.880 --> 0:18:11.200 companies like you know, banks, insurers that have huge data 0:18:11.200 --> 0:18:12.439 sets that they can do things with. 0:18:12.840 --> 0:18:16.359 I think so. I think that today it has been 0:18:16.480 --> 0:18:20.439 a little bit of ignorance arbitrage, meaning the people that 0:18:20.600 --> 0:18:22.720 really were in the know were the model makers, the 0:18:22.760 --> 0:18:25.640 people that could design the algorithms to do the predictive 0:18:25.680 --> 0:18:29.760 analysis and make the models. All those models are either 0:18:30.000 --> 0:18:32.960 held proprietary in the case of like open AI, or 0:18:33.000 --> 0:18:34.880 in the case of one of our companies, which has 0:18:35.040 --> 0:18:37.480 one of the most powerful repositories and one of the 0:18:37.520 --> 0:18:41.359 most ridiculous names, Hugging Face. One of my partner's amazing 0:18:41.359 --> 0:18:44.760 guy Brandon Reeves. He says, you know, there's these French 0:18:44.800 --> 0:18:48.400 PhD computer scientists and mathematicians. They're hanging out in Brooklyn 0:18:48.760 --> 0:18:50.880 and they've got this company called Hugging Face. And here's 0:18:50.880 --> 0:18:53.800 the irony they started out as a chatbot, you know, 0:18:53.840 --> 0:18:57.159 almost like Joaquin Phoenix and her, and then they became 0:18:57.560 --> 0:19:00.840 this open source repository for all all the models, and 0:19:00.880 --> 0:19:04.520 now hundreds of thousands of models, including corporate models that 0:19:04.560 --> 0:19:06.840 are hosted there and constantly improving, and it's all open source. 0:19:07.400 --> 0:19:11.119 And the irony is that OpenAI started as this open 0:19:11.200 --> 0:19:13.920 model company has become the world's greatest chatbot. So it's 0:19:13.920 --> 0:19:17.920 sort of inverse. So hugging Face is making these models, 0:19:18.040 --> 0:19:21.280 and they were the beneficiary very early on of everybody 0:19:21.320 --> 0:19:23.760 trying to deploy the models. They could run them on 0:19:23.840 --> 0:19:27.160 Hugging Face, they could use the cloud compute that they provide. 0:19:27.800 --> 0:19:30.640 And now, Tracy, to your point, you're starting to see 0:19:30.680 --> 0:19:33.199 people saying, Okay, the thing that we want to do 0:19:33.320 --> 0:19:35.879 is build on top of all of these models. What 0:19:36.000 --> 0:19:39.280 was expensive and scarce and rare before was the compute 0:19:39.680 --> 0:19:42.919 and the algorithms, and those are becoming increasingly abundant. So 0:19:43.000 --> 0:19:47.159 what is scarce today reliable data and proprietary data and 0:19:47.240 --> 0:19:49.480 the data sets, like you said, it could be big banks, 0:19:49.920 --> 0:19:54.080 could be consumer data, could be Amazon retail spending information 0:19:54.200 --> 0:19:57.960 could be Spotify with user's behavior, it could be healthcare 0:19:58.000 --> 0:20:02.359 systems appropriately anonymized and protected and compliant with HIPPA, but 0:20:02.600 --> 0:20:05.720 being able to collect all this information and have it 0:20:06.680 --> 0:20:10.040 do high quality inference and training. So you have to 0:20:10.080 --> 0:20:12.240 train the models on the data, and then you have 0:20:12.320 --> 0:20:14.600 to be able to do the predictability, which is the 0:20:14.640 --> 0:20:18.000 inference from somebody putting in a prompt. I will say 0:20:18.040 --> 0:20:21.160 the area that we're probably the most excited about, which 0:20:21.200 --> 0:20:23.760 is not something that the everydayly person is going to 0:20:23.760 --> 0:20:25.600 spend time doing. They're not going to be making those 0:20:25.640 --> 0:20:28.600 Johnny Cash songs or conjuring images on mid Journey and 0:20:28.720 --> 0:20:34.200 Dolly is biology. And the key breakthrough here is there's 0:20:34.240 --> 0:20:36.639 something in all these models called a context window, and 0:20:36.640 --> 0:20:38.679 all it means is basically how much information you can 0:20:38.720 --> 0:20:40.600 put it. And if you ever tried to take like 0:20:40.600 --> 0:20:43.160 a long transcript let's say, of odd lots and throw 0:20:43.200 --> 0:20:44.880 it into a context window, it might say, oh, it's 0:20:44.880 --> 0:20:50.080 too long, right. The context window for open AI has 0:20:50.080 --> 0:20:53.639 been about eight thousand, eight thousand tokens Anthropic now has 0:20:53.680 --> 0:20:56.600 one hundred thousand, and that's growing. What that means is 0:20:56.640 --> 0:20:58.400 the amount of data that you can put into a 0:20:58.440 --> 0:21:03.280 single prompt is growing exponentially. If you think about the 0:21:03.320 --> 0:21:05.639 human genome, if you think about genetic data, where you 0:21:05.640 --> 0:21:07.640