WEBVTT - How to Build the Ultimate GPU Cloud to Power AI 0:00:10.160 --> 0:00:14.400 Hello, and welcome to another episode of the Odd Lots Podcast. 0:00:14.480 --> 0:00:16.680 I'm Joe Wisenthal and I'm Tracy Alloway. 0:00:16.800 --> 0:00:19.360 Tracy, have you looked at in video stock chart lately? 0:00:19.440 --> 0:00:21.080 And by lately, I don't mean like over the last 0:00:21.120 --> 0:00:22.680 two years. I mean like just like over the last 0:00:22.720 --> 0:00:24.520 like two weeks or two months. 0:00:24.560 --> 0:00:26.400 I don't need to look at it because everyone keeps 0:00:26.440 --> 0:00:28.760 talking about it. So I know, I know what's happening. 0:00:28.960 --> 0:00:30.880 You know what, I'm pretty happy about. Could I just say, 0:00:31.160 --> 0:00:33.960 you know, we did that episode like two months ago, yes, 0:00:34.200 --> 0:00:37.519 with Stacy Raskin, and we were like, what's what's up 0:00:37.520 --> 0:00:39.239 a good video, like well, you know, I know it's 0:00:39.240 --> 0:00:42.920 at the center of the AI chips boom and whatever. 0:00:43.479 --> 0:00:45.199 And then like we did that episode and it came 0:00:45.200 --> 0:00:46.680 out and then a week later like they just like 0:00:46.800 --> 0:00:47.600 knocked it out of the park. 0:00:48.479 --> 0:00:49.639 Yeah, so you. 0:00:49.640 --> 0:00:51.000 Know, we were early. 0:00:51.880 --> 0:00:53.640 We were at least like, you know, a good like 0:00:53.680 --> 0:00:54.400 two weeks earlier. 0:00:55.160 --> 0:00:56.840 Hey, hey, two weeks I'll take it. 0:00:57.440 --> 0:00:58.360 I'll take it. 0:00:58.440 --> 0:01:01.800 So clearly something that you know, we and we talked 0:01:01.800 --> 0:01:03.760 about this with Stacy, like you know, something that in 0:01:03.840 --> 0:01:06.840 Nvidia has is like everyone's trying to buy it. Everyone's 0:01:06.840 --> 0:01:08.839 trying to get it, But then it raises the next 0:01:08.880 --> 0:01:11.240 question of like, okay, but what is that market? Like 0:01:11.520 --> 0:01:12.560 how do you buy a chip? 0:01:13.000 --> 0:01:14.959 Yeah? How do you buy a chip? And then I 0:01:14.959 --> 0:01:17.080 guess what do you actually do with it once you 0:01:17.200 --> 0:01:19.280 have it? Because my impression is that for a lot 0:01:19.319 --> 0:01:23.279 of these AI applications, the way you use the chips, 0:01:23.319 --> 0:01:26.119 the way you set up the data centers is very, 0:01:26.240 --> 0:01:29.399 very different to what we've seen in the past. And 0:01:29.440 --> 0:01:32.280 I think also what in Vidia is doing now is 0:01:32.360 --> 0:01:34.959 kind of different. But maybe we can get into this 0:01:35.040 --> 0:01:36.920 with our guests. My impression is they're trying to create 0:01:37.280 --> 0:01:40.960 a sort of like holistic approach for customers where they 0:01:41.000 --> 0:01:44.840 provide not just the hardware, but also some services to 0:01:44.880 --> 0:01:45.680 go along with it. 0:01:46.000 --> 0:01:48.720 Yes, right, and like all the software and Stacy talked 0:01:48.760 --> 0:01:50.880 about that with the Kuda ecosystemica that. 0:01:50.840 --> 0:01:53.160 Was it, how dominant that is? But right, like what 0:01:53.200 --> 0:01:55.360 do you do with it? Like how do you get one? If? 0:01:55.400 --> 0:01:56.800 Like what you know, what would we do. 0:01:56.760 --> 0:02:00.760 Tracy if a big palette of in Vidia chips wound 0:02:00.840 --> 0:02:01.840 up here? 0:02:01.480 --> 0:02:03.920 Do you want to know a secret? Yeah, my basement 0:02:03.960 --> 0:02:06.600 is filled with h one hundred chips. Just got a 0:02:06.640 --> 0:02:08.080 pile of them. It came with the house. 0:02:08.440 --> 0:02:10.880 It was on that ship that was stuck off the Chesapeake, 0:02:10.960 --> 0:02:12.960 and instead of getting your cowards, you got it. 0:02:13.080 --> 0:02:15.720 I just caught a palette of age four hundreds. 0:02:15.440 --> 0:02:19.680 That that well, we're manifesting that into reality. So anyway, 0:02:19.760 --> 0:02:22.600 I like how this world works so essentially, like the 0:02:22.800 --> 0:02:27.160 trading and dealing of these, like the hottest commodity in 0:02:27.200 --> 0:02:30.920 the world right which is these these advanced chips from AI, 0:02:31.000 --> 0:02:32.920 and how that works and who can get one? I 0:02:32.960 --> 0:02:35.520 still think is like a sort of mystery that we 0:02:35.639 --> 0:02:38.160 need to delve further into this question. 0:02:38.400 --> 0:02:40.359 I agree, And there is also there's a lot of 0:02:40.440 --> 0:02:43.600 excitement around it right now for the obvious reasons of 0:02:43.680 --> 0:02:48.239 everyone's really into generative AI and in video stock is exploding, 0:02:48.240 --> 0:02:50.640 as we already talked about, but we're also seeing a 0:02:50.680 --> 0:02:55.560 lot of previous I guess consumers of chips, like the 0:02:55.600 --> 0:02:59.079 crypto miners start to pivot into the space, and I'd 0:02:59.080 --> 0:03:01.720 be curious to see what they're doing in it as well, 0:03:01.800 --> 0:03:04.960 and how much of that is just you know, desperation 0:03:05.520 --> 0:03:08.959 versus versus a real business opportunity. 0:03:08.440 --> 0:03:09.440 In the video game market. 0:03:09.520 --> 0:03:11.079 Yeah, oh totally, I forgot about. 0:03:11.160 --> 0:03:13.320 Which was like the other thing. It's like, for years 0:03:13.360 --> 0:03:15.519 I thought of Nvidia is the video game company. Yeah, 0:03:15.560 --> 0:03:17.639 because they had their logo on xboxes. 0:03:17.720 --> 0:03:21.680 And how realistic is that pivot? What proportion of those 0:03:21.720 --> 0:03:23.840 types of chips can be used for AI? 0:03:24.000 --> 0:03:27.080 Now, well, I'm very excited. We do have I believe 0:03:27.240 --> 0:03:29.400 the perfect guest. We are going to be speaking with 0:03:29.440 --> 0:03:32.919 Brandon McBee. He is the chief strategy officer and co 0:03:33.000 --> 0:03:36.720 founder of core Weave, which is a specialized cloud services 0:03:36.800 --> 0:03:40.680 provider that's basically providing this sort of like high volume 0:03:40.800 --> 0:03:45.160 compute to AI type companies. They recently raised over four 0:03:45.240 --> 0:03:47.560 hundred million dollars. Have been in this space for a 0:03:47.600 --> 0:03:50.280 little while. So Brandon, thank you so much for coming 0:03:50.320 --> 0:03:51.160 on odd lots. 0:03:51.520 --> 0:03:53.640 Thanks for the opportunity. Guys, really excited to chat with 0:03:53.640 --> 0:03:54.160 you all today. 0:03:54.720 --> 0:03:57.240 So let's just let me sorry if Tracy and I, like, 0:03:57.360 --> 0:03:58.880 I don't know why they would do this, but if 0:03:58.920 --> 0:04:00.880 like some VC was like, you know, we want you 0:04:00.960 --> 0:04:03.680 to do on launch GPT. We want you to like 0:04:03.800 --> 0:04:07.880 do a pore base large language model off of all 0:04:07.920 --> 0:04:09.640 the work you've done. We want you to compete with 0:04:09.720 --> 0:04:12.240 open AI. And they gave us like I don't know 0:04:12.440 --> 0:04:15.400 some like, you know, one hundred million dollar rays, they said, 0:04:15.440 --> 0:04:18.640 go start, do your startup? Could I call in video 0:04:19.320 --> 0:04:21.560 and buy chips? Would I be able to get in 0:04:21.600 --> 0:04:22.120 the door there? 0:04:22.600 --> 0:04:25.440 Gosh? I mean you're I think you and everyone else 0:04:25.800 --> 0:04:27.960 is asking that question, and you're going to have a 0:04:28.240 --> 0:04:31.039 huge problem doing that. Right now, it's mostly just around 0:04:31.720 --> 0:04:35.520 how much in demand this infrastructure became, right I mean, 0:04:35.600 --> 0:04:38.599 you could argue it's one of the most critical pieces 0:04:38.600 --> 0:04:41.840 of information technology resources on the planet right now and 0:04:42.400 --> 0:04:45.279 suddenly everyone needs it, and you know, I like to 0:04:45.279 --> 0:04:49.800 contextualize it in that, you know, the piece of software 0:04:49.800 --> 0:04:53.080 adoption for AIS like one of the fastest adoption curves 0:04:53.080 --> 0:04:57.440 we've ever seen, right Like, you're you're hitting these milestones 0:04:57.480 --> 0:05:01.280 faster than any other software platform previously, and now all 0:05:01.320 --> 0:05:04.560 of a sudden, you're asking infrastructure build to keep up 0:05:04.600 --> 0:05:07.360 with that, right a space that traditionally takes more time, 0:05:07.480 --> 0:05:12.120 and it's created this massive supply demand and balance just 0:05:12.200 --> 0:05:16.280 on in place infrastructure today and not only infratructure is 0:05:16.279 --> 0:05:20.560 available to purchase, and it's an issue that is going 0:05:20.600 --> 0:05:22.480 to be ongoing for a bit as well, we think. 0:05:23.360 --> 0:05:26.479 So can I ask the basic question, which is core weave. 0:05:27.400 --> 0:05:30.880 What do you do exactly? Joe mentioned the capital raise, 0:05:30.920 --> 0:05:33.320 which I think has you valued at something like two 0:05:33.320 --> 0:05:37.719 billion dollars, So congrats, but what exactly are you doing here? 0:05:38.160 --> 0:05:41.400 Yeah? Thank you. So Corey is a specialized cloud service 0:05:41.400 --> 0:05:45.920 provider that is focused on highly parallelizable workloads. So we 0:05:46.320 --> 0:05:50.440 build and operate the world's most performant GPU infrastructure at 0:05:50.480 --> 0:05:54.760 scale and predominantly serve three sectors. That's the artificial intelligence sector, 0:05:54.920 --> 0:05:58.520 the media and entertainment sector, and the computational chemistry sector. 0:05:58.640 --> 0:06:04.320 So we build specialize in building this infrastructure at supercompute scale. 0:06:04.480 --> 0:06:07.560 It's like quite literally, you know, it's sixteen thousand GPU 0:06:07.760 --> 0:06:09.640 fabric and we can get into all the details and 0:06:09.680 --> 0:06:12.200 how complex that is. But we build that so that 0:06:12.320 --> 0:06:16.280 entities can come in and train these next generation foundation 0:06:16.440 --> 0:06:19.040 machine learning models on. And you know, we found ourselves 0:06:19.080 --> 0:06:20.760 in a spot where we can do that better than 0:06:20.880 --> 0:06:23.279 literally anyone else in the market and do it on 0:06:23.320 --> 0:06:27.159 a timeline that's faster or I think the only entity 0:06:27.200 --> 0:06:32.080 with H one hundred available to clients at scale globally today. 0:06:32.800 --> 0:06:35.120 So you have an actual basement full of H one 0:06:35.200 --> 0:06:38.960 hundred chips. Well, can you talk to us. You know, 0:06:39.040 --> 0:06:42.560 when you say infrastructure, we help clients build out the infrastructure, 0:06:42.640 --> 0:06:47.599 help us conceptualize this. What does what does the infrastructure 0:06:48.080 --> 0:06:51.880 for this type of AI actually look like? And how 0:06:51.880 --> 0:06:55.720 does it differ to infrastructure for other types of large 0:06:55.720 --> 0:06:57.440 scale technology projects. 0:06:57.920 --> 0:07:01.000 Yeah, totally, so, you know, I I think during the 0:07:01.080 --> 0:07:04.200 last in video quarterly earnings called Jensen put this a 0:07:04.200 --> 0:07:06.839 really great way in the Q and A section, he 0:07:06.960 --> 0:07:10.000 said that we are at the first year of a 0:07:10.120 --> 0:07:13.760 decade long modernization of the data center, or like making 0:07:13.760 --> 0:07:16.640 the data center intelligent. Right, you can kind of you 0:07:16.640 --> 0:07:19.840 could suggest that the last generation or the twenty tens 0:07:20.040 --> 0:07:23.280 data center was comprised of CPU, compute, storage and these 0:07:23.320 --> 0:07:26.840 things that didn't really work together that intelligently. And the 0:07:26.880 --> 0:07:29.880 way that in Nvidia has positioned itself is to make 0:07:29.880 --> 0:07:32.320 it a smart data center that's like smart routing of 0:07:32.480 --> 0:07:36.160 data packets of different pieces of infrastructure in there. That's 0:07:36.200 --> 0:07:39.680 all focused on how do you expand the throughput in 0:07:39.720 --> 0:07:46.440 communicability of and between pieces of infrastructure. Right, It's just 0:07:46.840 --> 0:07:50.840 an amazingly different approach to data center deployments. And so 0:07:51.680 --> 0:07:54.160 the way that we're building it and we're working with 0:07:54.480 --> 0:07:58.360 Nvidia infrastructure. We design everything to a DGX reference back 0:07:58.400 --> 0:08:01.160 in dgx's in videos like how do you draw the 0:08:01.200 --> 0:08:04.720 most performance out of Nvidia infrastructure is possible with all 0:08:04.720 --> 0:08:08.000 the anciliary components associated with it. So all this stuff 0:08:08.040 --> 0:08:10.960 is going into what's qualified as a Tier three or 0:08:11.000 --> 0:08:14.440 a Tier four data center. We collocate with within these things, 0:08:14.440 --> 0:08:17.520 so we're not quite building in a basement, even though 0:08:17.800 --> 0:08:21.400 like in our past history we certainly you know, had 0:08:21.840 --> 0:08:24.640 time doing that, but this is within you know, just 0:08:25.120 --> 0:08:28.640 amazing collocation sites that are operated by our partners such 0:08:28.640 --> 0:08:30.800 as switch right. So a Tier three a Tier four 0:08:30.920 --> 0:08:35.680 site is something that's qualified based on its ability to 0:08:35.720 --> 0:08:39.480 serve workloads with an extremely high uptime. So we're talking 0:08:39.520 --> 0:08:43.840 like ninety nine point nine nine percent uptime rate, and 0:08:43.880 --> 0:08:49.760 that's guaranteed by its power redundancy, it's Internet redundancy, and 0:08:49.800 --> 0:08:53.480 its security and then ultimately like it's connectivity to the 0:08:53.559 --> 0:08:56.959 Internet backbone. Right, So as it's like, as a first step, 0:08:57.360 --> 0:09:02.520 you're housed within these data centers that are just critical 0:09:02.600 --> 0:09:06.880 parts of the Internet infrastructure, and then from there you 0:09:06.920 --> 0:09:09.079 start building out the servers within there. And I can 0:09:09.240 --> 0:09:10.160 go into that detail. 0:09:10.600 --> 0:09:13.320 So you mentioned actually I want to just get sort 0:09:13.320 --> 0:09:15.480 of defined some terms. Can you just real quickly before 0:09:15.520 --> 0:09:17.240 we move on Tier three tier four? 0:09:17.240 --> 0:09:18.080 What do you mean by this? 0:09:18.880 --> 0:09:21.880 Yeah? So tier three, tier four. This all goes back 0:09:21.920 --> 0:09:24.360 to like the quality of the data center that you're 0:09:24.360 --> 0:09:26.760 in it. It's all about the reliability and up time 0:09:26.880 --> 0:09:28.640 that you should be able to achieve out of that 0:09:28.760 --> 0:09:32.600 data center. It's another way to qualify the services around it. 0:09:32.600 --> 0:09:35.960 It's like power. You get redundant power, right like multiple 0:09:35.960 --> 0:09:39.880 power services in case one goes offline, there's another one 0:09:40.080 --> 0:09:44.320 you get, you know, redundant cooling, you get redundant Internet connectivity. 0:09:44.360 --> 0:09:47.480 It's all these services that like have extra fiil safs 0:09:47.840 --> 0:09:50.760 that allow for you to operate at the highest up 0:09:50.800 --> 0:09:52.400 time and security level possible. 0:09:52.640 --> 0:09:54.760 Is higher tier better? Like tier three four? Is that 0:09:54.800 --> 0:09:56.120 better than Tier one and tier two? 0:09:57.040 --> 0:09:57.760 That's correct? 0:09:57.960 --> 0:10:01.400 Okay, so quick follow up question. Then you know we're 0:10:01.400 --> 0:10:03.280 interested in, like, okay, where the rubber hits the road. 0:10:03.280 --> 0:10:08.040 The scarcity is here. Let's say Tracy miraculously opens her 0:10:08.120 --> 0:10:10.400 basement and there really is like you know, all these 0:10:10.440 --> 0:10:14.480 palettes of these video chips, there is there capacity at 0:10:14.520 --> 0:10:16.440 the data centers right now, She's like, you know, what 0:10:16.440 --> 0:10:19.120 we want to co locate with you. You guys have great power, 0:10:19.679 --> 0:10:22.080 pretty well connected to the internet. You have like good 0:10:22.120 --> 0:10:24.920 security guards. So there's operated twenty four to seven. We 0:10:24.920 --> 0:10:27.000 want to set something up, like is there space there? 0:10:28.240 --> 0:10:30.839 Yeah, it's a fantastic question. It's a it's an issue 0:10:30.840 --> 0:10:33.480 that didn't really pop up until really in the last 0:10:33.559 --> 0:10:34.560 eight weeks or so. 0:10:34.800 --> 0:10:37.760 Oh, it's really happening that fast. 0:10:38.400 --> 0:10:39.760 It's happening that fast, Joe. 0:10:39.880 --> 0:10:40.880 And it's okay, So. 0:10:40.960 --> 0:10:43.440 That we said the two week lead time on in 0:10:43.520 --> 0:10:45.080 video was very important, Joe. 0:10:45.360 --> 0:10:48.400 Yeah, you're right, you're right. Is wow? Wait what happened? 0:10:48.600 --> 0:10:48.800 Wait? 0:10:49.240 --> 0:10:52.600 What happened sixteen described? Sixteen weeks ago? 0:10:52.720 --> 0:10:53.760 Verus eight weeks ago? 0:10:54.520 --> 0:10:57.720 Sure, it even last year? Right, So this is a space, 0:10:57.760 --> 0:11:02.240 the data centers space, collocation space that's been fairly chronically 0:11:02.360 --> 0:11:05.440 underinvested in because the Hyperscale has just built out their 0:11:05.480 --> 0:11:09.680 own data centers, right instead. But what's happened is the 0:11:09.760 --> 0:11:13.600 infrastructure changed. The type of compute that we're putting in 0:11:13.640 --> 0:11:17.240 these data centers, it's different than the last generation, right, 0:11:17.280 --> 0:11:20.800 so we're predominantly focused on GPU compute instead of CPU 0:11:20.840 --> 0:11:25.439 compute and GPU compute. It's about four times more power 0:11:25.559 --> 0:11:29.839 dance than CPU compute, and that throws the data center 0:11:29.960 --> 0:11:33.440 planning into chaos, right because ultimately, let's say you have 0:11:33.480 --> 0:11:36.319 a ten thousand square foot room in the data center, right, 0:11:36.360 --> 0:11:38.160 and you have a certain amount of power it's called 0:11:38.160 --> 0:11:39.960 one hundred units of power that go into that ten 0:11:40.000 --> 0:11:44.079 thousand square feet. Well, because I'm four times more power DNS, 0:11:44.760 --> 0:11:47.320 it means that now I take those hundred units of power, 0:11:47.400 --> 0:11:50.560 but I only require about twenty five percent of that 0:11:50.640 --> 0:11:53.439 data center footprint or in other words, twenty five hundred 0:11:53.440 --> 0:11:56.400 square feet within that ten thousand square foot footprint. So 0:11:56.800 --> 0:12:00.480 that then leads to like, not only is the space 0:12:00.760 --> 0:12:03.960 in the data center being used inefficiently now because you 0:12:04.240 --> 0:12:06.720 theoretically have to run more power into the data center 0:12:06.800 --> 0:12:08.800 to use that full ten thousand square feet due to 0:12:08.800 --> 0:12:12.360 the Poara density delta, but now you have cooling issues, 0:12:12.960 --> 0:12:15.720 right because you designed that footprint to be able to 0:12:15.720 --> 0:12:19.920 cool ten thousand square feet spread out across that entire area. 0:12:20.000 --> 0:12:21.040 But now you're dropping storry. 0:12:21.559 --> 0:12:23.640 Sorry, I just want to back up because this is 0:12:23.880 --> 0:12:26.040 extremely interesting, so I don't want to I just want 0:12:26.080 --> 0:12:27.079 to get this detail right. 0:12:27.640 --> 0:12:30.840 Just sorry, just to and then move on. 0:12:30.920 --> 0:12:34.439 But the let's given an x amount of power at 0:12:34.480 --> 0:12:37.720 one hundred units of power. What you're saying is that 0:12:37.800 --> 0:12:41.880 with this next generation of compute, it now only gets 0:12:42.120 --> 0:12:42.720 that's now. 0:12:42.600 --> 0:12:44.720 Only sufficient for a quarter of the data center. 0:12:44.760 --> 0:12:48.040 In other words, that to power that whole that space, 0:12:48.520 --> 0:12:50.800 and that to then power the whole space, you really 0:12:50.840 --> 0:12:52.360 would need like four x the power. 0:12:53.320 --> 0:12:57.160 That's accurate. Okay. The complication really arises out of the 0:12:57.160 --> 0:13:00.400 cooling that that's required from that, right, So if you 0:13:00.400 --> 0:13:03.160 imagine you can cool a ten thousand square foot space 0:13:03.160 --> 0:13:05.400 and you designed for that, that's one thing. But now 0:13:05.440 --> 0:13:08.080 if you have to cool in a much more dense area, 0:13:08.559 --> 0:13:12.400 that's a different type of cooling requirement. And so that's 0:13:12.480 --> 0:13:15.719 led to this issue where there's only a certain subset 0:13:15.960 --> 0:13:18.640 of Tier three and four data centers across the US 0:13:19.120 --> 0:13:23.440 that can are currently designed for or can quickly be 0:13:23.559 --> 0:13:27.880 designed and changed to be able to accommodate this new 0:13:27.960 --> 0:13:31.760 power density issue. So now not only like if you 0:13:31.840 --> 0:13:34.440 had all those eighth one hundreds in your basement, you 0:13:34.520 --> 0:13:36.480 might not have a place to plug them into. And 0:13:37.320 --> 0:13:40.160 that's become a pretty big problem for the industry very quickly, 0:13:40.160 --> 0:13:42.520 and truly has only arisen in the last eight weeks 0:13:42.640 --> 0:13:45.120 or so, and it's going to persist for a few quarters. 0:13:46.040 --> 0:13:50.040 So you were describing the difference between CPU and GPU. 0:13:50.600 --> 0:13:54.600 How do you actually connect these newer types or these 0:13:54.640 --> 0:13:58.600 different types of chips together, because I imagine, you know, 0:13:58.760 --> 0:14:01.160 old data centers you just have a bunch of like 0:14:01.240 --> 0:14:04.520 Ethernet cables or something like that. But for this type 0:14:04.559 --> 0:14:06.640 of processing power, do you need something different? 0:14:08.000 --> 0:14:12.160 That's exactly correct, Chracy. So what we so the legacy 0:14:12.280 --> 0:14:15.240 the generalized compute data centers are really what the hyperscalers 0:14:15.280 --> 0:14:19.560 look like. You know, Amazon, Google, Microsoft, Oracle. They predominantly 0:14:19.640 --> 0:14:23.240 use something that's called Ethernet to connect all the service together. 0:14:23.280 --> 0:14:25.600 And the reason you use that was, you know, you 0:14:25.600 --> 0:14:29.120 don't really need to have high data throughput to connect 0:14:29.240 --> 0:14:31.040 all these servers together, right, They just need to be 0:14:31.080 --> 0:14:33.040 able to send some messages back and forth. They talk 0:14:33.080 --> 0:14:35.960 to each other about what they're working on, but they're not, 0:14:36.480 --> 0:14:41.000 you know, necessarily doing highly collaborative tasks that require moving 0:14:41.080 --> 0:14:44.720 lots of data in between each other. That's changed. So 0:14:45.080 --> 0:14:48.160 so today what people are focused on and need to 0:14:48.200 --> 0:14:52.400 build are these effectively supercomputers. Right, and so we refer 0:14:52.520 --> 0:14:56.000 to the connectivity between them, the network between them as 0:14:56.120 --> 0:14:59.560 a fabric, right, it's called a network fabric. So if 0:14:59.600 --> 0:15:02.480 we're build holding something to help train like the next 0:15:02.520 --> 0:15:07.600 generation GPT model, typically clients are coming to us saying, hey, 0:15:07.640 --> 0:15:11.400 I need a sixteen thousand GPU fabric of H one hundred. 0:15:11.960 --> 0:15:15.800 So that's there's about eight GPUs that go into each server, 0:15:16.120 --> 0:15:18.960 and then you have to run this connectivity between each 0:15:19.080 --> 0:15:21.240 one of those servers. But it's now done in a 0:15:21.280 --> 0:15:25.480 different way to your point, So we're using a in 0:15:25.520 --> 0:15:30.560 Nvidio technology called InfiniBand which has the highest data throughput 0:15:30.680 --> 0:15:34.000 to connect each of these devices together. And you know, 0:15:34.080 --> 0:15:38.560 taking this sixteen thousand GPU cluster as an example, there's 0:15:38.600 --> 0:15:41.680 two crazy numbers in here. One is that there are 0:15:41.960 --> 0:15:47.200 forty eight thousand discrete connections that need to be made, 0:15:47.400 --> 0:15:50.280 right like plugging one thing in from one computer to 0:15:50.320 --> 0:15:54.680 another computer. But there's lots of switches and routers that 0:15:54.720 --> 0:15:57.200 are between there. But you need to that forty eight 0:15:57.240 --> 0:16:03.120 thousand times, and it takes over five hundred miles of 0:16:03.200 --> 0:16:07.200 fiber optic cabling to do that successfully across the sixteen 0:16:07.240 --> 0:16:09.840 thousand GPU cluster. And now again you're doing that within 0:16:09.880 --> 0:16:11.800 a small space with a ton of power density, with 0:16:11.840 --> 0:16:14.720 a ton of cooling, and it's just a completely different 0:16:14.760 --> 0:16:17.960 way to build this infrastructure. It's just because the requirements 0:16:17.960 --> 0:16:20.560 have changed, right, Like we've moved into this, like this 0:16:20.720 --> 0:16:25.040 area where we are designing next generation AI models and 0:16:25.120 --> 0:16:27.840 it requires a completely different type of compute, and it's 0:16:27.920 --> 0:16:31.320 just it's caught the whole sector by surprise so much 0:16:31.360 --> 0:16:34.880 so that you know, it's really challenging to go procure 0:16:34.880 --> 0:16:38.240 it at the hyperscalers today because they didn't specialize in 0:16:38.280 --> 0:16:40.560 building it. And that's you know where where core we've 0:16:40.560 --> 0:16:43.440 comes in is we only focus on building this type 0:16:43.480 --> 0:16:46.240 of compute for clients. It's our specialty. We hire all 0:16:46.240 --> 0:16:48.400 of our engineering around it, all of our research goes 0:16:48.440 --> 0:16:51.040 into it, and it's you know, it's been a fantastic 0:16:51.040 --> 0:16:53.640 spot to be but our goal at the end of 0:16:53.640 --> 0:16:54.760 the day is just to be able to get this 0:16:54.840 --> 0:16:57.480 infrastructure into the hands of end consumers so that they 0:16:57.480 --> 0:17:00.640 can build the amazing AI companies that have ones looking 0:17:00.680 --> 0:17:16.439 forward to using and incorporating to enterprises and software companies. 0:17:21.520 --> 0:17:25.560 You know, you mentioned these special or purpose built connections 0:17:25.680 --> 0:17:28.480 that Nvidia is making, and this kind of leads nicely 0:17:28.640 --> 0:17:31.760 into my next question, which is what exactly is your 0:17:31.880 --> 0:17:37.879 relationship with Nvidia and in order to provide this type 0:17:37.960 --> 0:17:42.280 of service, you know, vast amounts of processing power that 0:17:42.440 --> 0:17:46.080 is well suited to a particular type of technology in 0:17:46.119 --> 0:17:49.160 this case AI, do you have to have a really 0:17:49.200 --> 0:17:51.919 good relationship with Nvidia to make that work? Like do 0:17:52.000 --> 0:17:55.120 you have to have special access to H one, hundreds 0:17:55.160 --> 0:17:56.840 and other chips. 0:17:57.840 --> 0:18:00.119 It's a great question, and I'll try to offer or 0:18:00.680 --> 0:18:03.159 from Nvidia's perspective, and it goes a little bit back 0:18:03.200 --> 0:18:05.160 to the answer I just provided as well in that 0:18:06.200 --> 0:18:09.560 I would think from in Nvidia's seat, what's most important 0:18:09.720 --> 0:18:13.640 is empowering end users of their compute to be able 0:18:13.640 --> 0:18:18.200 to access their compute and the most performant variant possible 0:18:19.160 --> 0:18:21.440 at scale, and to be able to access it quickly, right, 0:18:21.480 --> 0:18:23.000 Like a new generation comes out, they want to be 0:18:23.000 --> 0:18:25.240 able to get their hands on it, right. And we've 0:18:25.320 --> 0:18:29.200 built Core. We've around hitting every single one of those checkboxes. Right. 0:18:29.200 --> 0:18:31.440 We build it at DGX reference back, we build it 0:18:31.560 --> 0:18:34.160 at scale, and we bring it online on a timeline 0:18:34.200 --> 0:18:37.720 that's you know, within months of a next generation chipset launch, 0:18:37.840 --> 0:18:41.680 as opposed to you know, the more traditional legacy hyperscalers 0:18:41.680 --> 0:18:45.679 that take quarters at a times, so US being in 0:18:45.720 --> 0:18:49.399 a position to do that has has enabled us fantastic 0:18:49.640 --> 0:18:53.600 access within Nvidia, and we have a history of consistently 0:18:53.640 --> 0:18:56.960 executing on exactly what we've what we say we'll do right, 0:18:57.000 --> 0:19:02.120 we under promise and over deliver as a business, and 0:19:02.520 --> 0:19:04.480 I think that's just put us in this place where 0:19:04.800 --> 0:19:08.720 Nvidia has the confidence in allocating infrastructure to us because 0:19:08.760 --> 0:19:10.680 they know it's going to come online, they know it's 0:19:10.680 --> 0:19:14.240 going to get to consumers faster than anyone else in 0:19:14.280 --> 0:19:15.760 the market, and they know it's going to be delivered 0:19:15.760 --> 0:19:18.520 in its most performance configuration that exists. 0:19:19.760 --> 0:19:22.359 You know, I was thinking as I listened to some 0:19:22.400 --> 0:19:25.240 of these answers, I keep having like these like imagines, 0:19:25.280 --> 0:19:29.520 like you know, there's probably like some random industrial company 0:19:29.600 --> 0:19:32.159 that's like traded like you know on the like S 0:19:32.200 --> 0:19:36.320 and P four hundred that makes some cooling fluid whose 0:19:36.359 --> 0:19:38.080 like sales are going to be up ten x. So 0:19:38.119 --> 0:19:40.040 I'm like googling while we're talking, like what is a 0:19:40.040 --> 0:19:42.000 company that makes kool aid fluid? Or like who is 0:19:42.040 --> 0:19:44.000 some company that's like really good at making these like 0:19:44.040 --> 0:19:48.360 infinite bands, Because it just likes. 0:19:46.840 --> 0:19:48.479 Right, yeah, like what are the anyway? 0:19:48.560 --> 0:19:48.760 Right? 0:19:48.920 --> 0:19:50.840 But like right, like you know there's going to be 0:19:50.880 --> 0:19:54.760 some yeah urchiery plate that are like thirty x up. 0:19:54.960 --> 0:19:56.760 But you know, I want to get a sense from 0:19:56.800 --> 0:20:00.879 you of so it's really changed a lot, and I 0:20:01.000 --> 0:20:02.600 kind of you know, in the last several months. 0:20:02.600 --> 0:20:04.080 Could we see it from in video results? 0:20:04.119 --> 0:20:08.040 What you're describing, like how big is the market getting 0:20:08.119 --> 0:20:09.520 and the way I think you know, I know, like 0:20:09.520 --> 0:20:13.040 with AI, there's training and they sort of build the 0:20:13.080 --> 0:20:15.199 model and then there's inference, and the inference is how 0:20:15.240 --> 0:20:17.600 they spit out the results. Can you talk a little 0:20:17.600 --> 0:20:20.920 bit about what you're seeing in terms of the growth 0:20:21.400 --> 0:20:24.760 of both of those aspects of AI, which is bigger 0:20:25.000 --> 0:20:27.240 and which is growing faster? And how do they compare 0:20:27.280 --> 0:20:29.720 to like the size of the installed compute base that 0:20:29.760 --> 0:20:30.440 already exists. 0:20:31.240 --> 0:20:34.760 Oh. Absolutely, So this is one of my favorite topics 0:20:34.800 --> 0:20:37.280 because it's just mind blowing the scale that's going to 0:20:37.280 --> 0:20:41.719 be needed to support AI and scale this infrastructure. So okay, 0:20:41.800 --> 0:20:45.600 so today most of the funding that's going into the 0:20:45.600 --> 0:20:49.800 AI space is too for funding to train next generation 0:20:50.600 --> 0:20:53.400 foundation models. Right, So when a company's raising a bunch 0:20:53.440 --> 0:20:55.119 of money at the end of the day, most of 0:20:55.119 --> 0:20:57.680