WEBVTT - Silicon States: US Data Center Expansion in the AI Era 0:00:00.160 --> 0:00:02.719 This is Tom Rowlands Reese and you're listening to Switched 0:00:02.759 --> 0:00:05.960 on the podcast brought to you by BNF. The rapid 0:00:06.040 --> 0:00:09.400 rise of energy intensive AI data centers is reshaping the 0:00:09.440 --> 0:00:13.000 near term outlook for US power markets. Outpacing the energy 0:00:13.039 --> 0:00:16.720 demand growth of EVS, hydrogen and all other demand classes 0:00:16.760 --> 0:00:19.599 out to twenty thirty, data centers will account for eight 0:00:19.640 --> 0:00:22.959 point six percent of US electricity demand by twenty thirty five. 0:00:23.120 --> 0:00:25.919 That's almost twice as much as today. Largely owned and 0:00:25.960 --> 0:00:29.720 operated by a few highly consolidated companies with very deep pockets, 0:00:29.920 --> 0:00:33.280 this concentration of capital allows for rapid expansion and a 0:00:33.360 --> 0:00:37.760 significant influence over future energy infrastructure investment. So what strategies 0:00:37.760 --> 0:00:40.839 are these companies employing to optimize their data center rollout? 0:00:41.080 --> 0:00:43.520 On today's show, I'm joined by BNF's head of US 0:00:43.560 --> 0:00:47.720 Power Helen Co and US Power Senior Associate Natalie Lemandebrata, 0:00:47.960 --> 0:00:50.640 and together we discuss findings from their note US data 0:00:50.640 --> 0:00:53.519 center market outlook the Age of AI, which B and 0:00:53.560 --> 0:00:55.800 EF clients can find at BNF go on the Bloomberg 0:00:55.880 --> 0:00:58.800 Terminal or on BNF dot com. All right, let's get 0:00:58.800 --> 0:01:01.640 to talking about the outlook AI data centers with Helen 0:01:01.720 --> 0:01:15.959 and Natalie. Helen, thank you for being here. Thanks Tom 0:01:16.240 --> 0:01:18.039 and Natalie thanks for being here as well. 0:01:18.200 --> 0:01:18.880 Thank you Tom. 0:01:19.600 --> 0:01:23.240 So Natalie reports up to Helen, and Helen reports up 0:01:23.280 --> 0:01:25.560 to me, and I'm not saying that to flex. I'm 0:01:25.560 --> 0:01:27.960 saying it for a couple of reasons. One is, I'm 0:01:28.000 --> 0:01:31.840 like super proud to have such smart people on my team, 0:01:32.160 --> 0:01:34.559 and I'm also particularly proud of the work they've done 0:01:34.640 --> 0:01:39.640 around data centers. But also this situation of this reporting 0:01:39.720 --> 0:01:41.720 line means that I get to have catch ups with 0:01:41.800 --> 0:01:44.880 them regularly, which has been pretty useful to me personally 0:01:45.120 --> 0:01:48.520 because this question around AI and data centers has had 0:01:48.560 --> 0:01:50.760 a lot of people talking, a lot of people have opinions, 0:01:51.000 --> 0:01:54.040 and so I have often found myself in situations where 0:01:54.200 --> 0:01:57.960 people are expressing their opinions, and in those situations, I've 0:01:58.040 --> 0:02:01.520 developed this tactic to different myself from the pack, which 0:02:01.600 --> 0:02:06.040 is that in the situation, I just regurgitate whatever Helen 0:02:06.040 --> 0:02:08.400 and Natalie last told me about data centers, and everyone 0:02:08.440 --> 0:02:11.200 thinks that I'm really smart. So first off, let's start 0:02:11.200 --> 0:02:13.480 just with the headline numbers. How much data center build 0:02:13.560 --> 0:02:16.680 are we expecting in the US According to the report 0:02:16.680 --> 0:02:17.919 that we just published, so. 0:02:17.919 --> 0:02:21.440 BNF's latest outlook has data center and demand more than 0:02:21.480 --> 0:02:25.080 doubling from thirty five gigatts today to close to eighty 0:02:25.120 --> 0:02:28.360 gigawatts in twenty thirty five. This would account for close 0:02:28.360 --> 0:02:31.600 to nine percent of total US electricity demand. 0:02:32.000 --> 0:02:34.480 Wow, so we're expecting let me just do the mass 0:02:34.600 --> 0:02:38.440 suddenly like forty five gigawatts ish, which for those of 0:02:38.480 --> 0:02:41.000 you who are you know, maybe new to the energy space, 0:02:41.080 --> 0:02:45.000 that's like twenty to thirty nuclear plants, and nuclear plants 0:02:45.000 --> 0:02:46.960 are really really big and take a long time to build. 0:02:46.960 --> 0:02:50.480 That's a lot of demand. So we are forecasting some 0:02:50.840 --> 0:02:53.680 astronomical amount of data center build. How do we compare 0:02:53.880 --> 0:02:56.680 to everyone else that has an opinion on this topic? 0:02:57.320 --> 0:03:03.760 We're relatively conservative and relatively conservative. Yes, yeah, our overall 0:03:04.000 --> 0:03:07.800 demand build is fairly low in terms of uptake relative 0:03:07.880 --> 0:03:09.160 to our third parties. 0:03:09.639 --> 0:03:13.840 Okay, so how come they are forecasting something so much 0:03:13.919 --> 0:03:16.120 more aggressive than us, or how come we are so 0:03:16.240 --> 0:03:17.720 much more conservative than them? 0:03:18.000 --> 0:03:22.239 Well, we don't really know what our third party counterparts 0:03:22.320 --> 0:03:24.560 do in terms of their forecast, but what we do 0:03:24.680 --> 0:03:27.960 know is like how we forecast data centers, and our 0:03:28.040 --> 0:03:31.359 focus was really to look at like how data centers 0:03:31.400 --> 0:03:34.280 move from one stage to the next. So in our 0:03:34.320 --> 0:03:36.280 project data base, what we know is that we can 0:03:36.320 --> 0:03:39.520 see data center stages. So we have like early stage, 0:03:39.800 --> 0:03:43.160 which is basically anything kind of just just got announced. 0:03:43.200 --> 0:03:45.840 We have projects that are committed, which is anything that 0:03:45.880 --> 0:03:49.000 has some type of like land or permitting agreement, things 0:03:49.000 --> 0:03:52.400 that are under construction and then live. And what we 0:03:52.480 --> 0:03:55.080 did was we tracked how these data centers moved from 0:03:55.080 --> 0:03:57.200 one stage to the next, and we looked at like 0:03:57.280 --> 0:03:59.920 the probability of how these data centers moved. 0:03:59.760 --> 0:04:02.520 From and stay to the next. So typically, how long 0:04:02.560 --> 0:04:05.000 does it take for a data center, you know, from 0:04:05.680 --> 0:04:08.960 early on in this pipeline to being commissioned, how long 0:04:08.960 --> 0:04:09.600 would that take? 0:04:10.040 --> 0:04:13.360 Yeah, So what we've found based on data between twenty 0:04:13.440 --> 0:04:15.920 twenty to twenty twenty four is that it takes seven 0:04:16.040 --> 0:04:18.719 years to build a data center, which is a really, 0:04:18.880 --> 0:04:19.680 really long time. 0:04:19.960 --> 0:04:23.240 Okay, So that's really interesting. So in a way, our 0:04:23.320 --> 0:04:25.760 forecast we're saying with a fair amount of confidence because 0:04:25.800 --> 0:04:27.440 you know, twenty thirty five is only just over a 0:04:27.480 --> 0:04:31.160 decade away, and we have data on everything that's getting built, 0:04:31.240 --> 0:04:33.400 and we know that it takes most of that decade 0:04:33.440 --> 0:04:37.440 for it to get built. So anyone who's forecasting something 0:04:37.440 --> 0:04:39.480 more aggressive than us either has a different view on 0:04:39.520 --> 0:04:41.479 how long it takes to build data centers, or they 0:04:41.560 --> 0:04:43.960 must have different data, or maybe they're using a completely 0:04:43.960 --> 0:04:46.680 different methodology. But it's good to know that we're the 0:04:46.680 --> 0:04:48.920 ones doing it right. So who's building all of these 0:04:49.000 --> 0:04:53.919 data centers this sort of colossal volume of new demand. 0:04:54.360 --> 0:04:57.720 Yeah, so the data center market is pretty concentrated. There's 0:04:57.760 --> 0:05:01.800 two main types of owners. There's colo data centers who 0:05:01.960 --> 0:05:05.719 have buildings with multiple tenants, and you have these self 0:05:05.760 --> 0:05:09.360 build companies, which are typically your large tech companies or 0:05:09.440 --> 0:05:14.440 hyperscalers is what they're usually referred to. And the hyperscalers 0:05:14.480 --> 0:05:19.640 of Google, Amazon, Microsoft, and Meta are close to fifty 0:05:19.640 --> 0:05:23.560 percent of total operating capacity today and this is only 0:05:23.600 --> 0:05:26.960 set to grow. They're building much larger campuses close to 0:05:27.040 --> 0:05:31.640 gigawatts scale. Amazon has multiple gigawat data center campuses in 0:05:31.720 --> 0:05:36.680 development in Virginia. Meta has another two gigawatts in Louisiana, 0:05:36.960 --> 0:05:40.039 and just as a point of reference, in the last decade, 0:05:40.120 --> 0:05:43.880 data centers have typically been in the tens of megawatts. 0:05:44.279 --> 0:05:46.680 So really, as we're pushing through these hundreds of megawatts 0:05:46.680 --> 0:05:50.560 and gigawat size, the rise of uptake will be much 0:05:50.720 --> 0:05:51.960 faster and larger. 0:05:52.240 --> 0:05:54.680 Okay, And so just just to make sure I've understood 0:05:54.800 --> 0:05:58.640 overall correctly, those four companies are fifty percent of the 0:05:58.760 --> 0:06:02.120 data center build today, but we think that there's going 0:06:02.160 --> 0:06:04.440 to be even more because they're the companies that are 0:06:04.440 --> 0:06:07.400 building these really big data centers that are so much 0:06:07.400 --> 0:06:10.360 of what we're expecting. So you've already alluded to these 0:06:10.440 --> 0:06:12.840 data centers are maybe bigger than the ones we've seen 0:06:12.880 --> 0:06:15.000 in the past, that's the trend. But in what other 0:06:15.040 --> 0:06:17.719 ways are the data centers that we're expecting different to 0:06:17.800 --> 0:06:19.960 the ones that we've seen in the past. 0:06:20.680 --> 0:06:23.279 Yeah, before we jump into that, I think it's important 0:06:23.279 --> 0:06:26.560 to understand how be any of categorized as data centers. 0:06:26.800 --> 0:06:30.760 So we categorize data centers in three different ways. Vers size, 0:06:30.800 --> 0:06:35.200 which Natalie had alluded to, first retail, wholesale and hyperscaler 0:06:35.320 --> 0:06:37.359 So that's just based on the project size of a 0:06:37.440 --> 0:06:40.640 data center. And then there's operator type, which is the 0:06:40.680 --> 0:06:45.280 ownership so either self build or co location, which Natalie 0:06:45.320 --> 0:06:50.120 had already explained. And then there's workload, So workload is 0:06:50.200 --> 0:06:54.080 based on just the computing process of a data center, 0:06:54.400 --> 0:06:58.040 and there are many different types of workload from cloud 0:06:58.279 --> 0:07:02.520 or enterprise, telecom, crypto mining, and that determines a lot 0:07:02.600 --> 0:07:06.960 about the data center's overall infrastructure and then also their 0:07:07.040 --> 0:07:09.240 overall load and power consumption. 0:07:09.760 --> 0:07:11.720 So then, I mean one of the other things that 0:07:11.760 --> 0:07:13.720 you've spoken to me about is when we're talking about 0:07:13.760 --> 0:07:16.840 AI data centers, that there's two main flavors, So can 0:07:16.880 --> 0:07:18.920 you just talk me through those as well. 0:07:19.560 --> 0:07:23.400 Within AI, we mainly branch it out in two main workloads, 0:07:23.440 --> 0:07:27.280 AI training and AI inference. AI training is processing a 0:07:27.360 --> 0:07:29.720 large amount of data in order to train these large 0:07:29.800 --> 0:07:33.600 language models, and AI inference is taking those already train 0:07:33.720 --> 0:07:36.520 models for real time applications in use, like when you're 0:07:36.600 --> 0:07:40.160 careering in chat GPT, And those two workloads have also 0:07:40.800 --> 0:07:46.400 different location and constraints of the data center itself. AI inference, 0:07:46.560 --> 0:07:50.840 since they're theoretically interacting with the end user in real time, 0:07:51.120 --> 0:07:54.600 they'll care more about latency and location parameters to that 0:07:54.760 --> 0:07:57.960 end user. AI training a lot of that processing happens 0:07:57.960 --> 0:08:01.040 on site, so theoretically they could be more flexible on 0:08:01.080 --> 0:08:04.520 where they locate, and they could follow where there's available 0:08:04.560 --> 0:08:06.480 power or other constraints. 0:08:06.840 --> 0:08:09.480 Okay, I mean you use the word theoretically there, which 0:08:09.480 --> 0:08:11.160 maybe is doing a lot of work. And we'll come 0:08:11.160 --> 0:08:14.560 back to where people are actually building data centers later on, 0:08:14.560 --> 0:08:17.160 because I do have a question about that. But roughly, 0:08:17.240 --> 0:08:20.200 what is do we have an idea of what proportion 0:08:20.320 --> 0:08:24.080 of the data center building the pipeline is for AI 0:08:24.240 --> 0:08:26.400 and what proportion of it is training and what proportion 0:08:26.520 --> 0:08:27.360 of it is inference. 0:08:27.960 --> 0:08:32.240 That's actually very tough to ascertain exactly what the split is. 0:08:32.280 --> 0:08:35.440 If we look at a large Gigawat campus, some of 0:08:35.480 --> 0:08:37.800 their buildings could be used for training today, but it 0:08:37.800 --> 0:08:40.200 could be used for inference in the future. Similarly, we 0:08:40.280 --> 0:08:44.400 talked about different owner types a co location building, they 0:08:44.440 --> 0:08:47.520 could have, you know, multiple tenants, and unless you know 0:08:47.559 --> 0:08:50.320 exactly who the tenant is and what type of workloads 0:08:50.320 --> 0:08:52.920 they're running, it's also difficult to know if it is 0:08:52.960 --> 0:08:55.839 for AI training at inference. But we can infer based 0:08:55.880 --> 0:08:57.280 on where they're getting built. 0:08:57.679 --> 0:09:00.679 But these distinctions, I mean, what I'm here here is 0:09:00.880 --> 0:09:03.640 that it's not just that nobody tells us which it 0:09:03.720 --> 0:09:05.760 is that makes it a little bit of a gray area. 0:09:05.800 --> 0:09:09.000 It's that actually even a data center itself might sometimes 0:09:09.200 --> 0:09:11.000 at one point in its life be doing one thing 0:09:11.040 --> 0:09:12.520 and at a different point in it's life be doing 0:09:12.559 --> 0:09:13.040 something else. 0:09:13.360 --> 0:09:15.880 I think for power folks, I often try to like 0:09:16.000 --> 0:09:18.720 frame it like a data center is very similar to 0:09:18.760 --> 0:09:21.840 a battery. Like batteries can do multiple different types of 0:09:21.960 --> 0:09:25.720 energy services or ancillary services, a data center can do 0:09:25.800 --> 0:09:29.000 multiple different types of workload. It can be doing AI 0:09:29.040 --> 0:09:32.760 training or cloud as long as the configuration is correct, 0:09:32.840 --> 0:09:34.560 or it could do those types of workload. 0:09:34.679 --> 0:09:37.000 Got it. So just because you've optimized one to one 0:09:37.000 --> 0:09:39.839 thing doesn't mean that it can't do the other thing. 0:09:40.120 --> 0:09:45.360 Yeah, And particularly in colocation data centers where basically these 0:09:45.360 --> 0:09:48.720 companies are renting out their RT servers to tenants, those 0:09:48.760 --> 0:09:51.920 tenants are probably doing different types of workload. There can 0:09:52.000 --> 0:09:54.360 be multiple applications in a data center. 0:09:54.640 --> 0:09:56.800 Yeah, a workload, which is often why you see in 0:09:56.840 --> 0:10:00.079 colocation that they're optimizing for everything because they don't know 0:10:00.120 --> 0:10:03.720 who their tenant is. I will add, though, in terms 0:10:03.720 --> 0:10:07.199 of AI data centers that they are quite different from 0:10:07.280 --> 0:10:10.000 data centers today. We already talked that these data centers 0:10:10.000 --> 0:10:12.280 are getting larger. A lot of that has to do 0:10:12.320 --> 0:10:15.360 with the GPUs in those servers being much more so. 0:10:15.400 --> 0:10:17.280 Can you just what a GPUs. 0:10:17.040 --> 0:10:22.520 Graphical processing units, which is they're similar to CPUs central 0:10:22.520 --> 0:10:26.640 processing units, but they're very specialized in they're cooler. 0:10:28.080 --> 0:10:30.719 I think that we've got to the level of understanding. 0:10:30.920 --> 0:10:32.839 I'm good with it. So it's like it's like a chip, 0:10:32.960 --> 0:10:34.040 but it's a different kind of chip. 0:10:34.120 --> 0:10:37.320 Yeah, it's very good at parallel processing, which is optimized 0:10:37.360 --> 0:10:40.440 for training large language models. I'm sure lots of people 0:10:40.440 --> 0:10:43.080 have heard of Navidia and their GPUs. A lot of 0:10:43.080 --> 0:10:46.439 tech companies are also building their AI accelerators, which are 0:10:46.520 --> 0:10:50.720 specialized chips for these models. So a lot of the 0:10:50.760 --> 0:10:54.520 design of the data center today in order to accommodate 0:10:54.559 --> 0:10:57.960 AI training workloads are different. A lot of that has 0:10:58.000 --> 0:11:00.520 to do with these GPUs and the rack that they're 0:11:00.520 --> 0:11:02.800 on are going to be much more power dense, which 0:11:02.800 --> 0:11:06.520 means they need a lot more sophisticated cooling technologies to 0:11:06.559 --> 0:11:09.640 accommodate those sort of rack densities that could be tenext 0:11:09.720 --> 0:11:13.440 to what typical data centers are today. We've seen, for example, 0:11:13.600 --> 0:11:16.960 Metas scrapping data centers that were not ail ready, so 0:11:17.000 --> 0:11:20.120 it's quite difficult to retrofit a data center from five 0:11:20.160 --> 0:11:22.040 years ago to an AI workload. 0:11:22.400 --> 0:11:23.360 So while a. 0:11:23.400 --> 0:11:26.079 Data center in the future could have multiple uses, it's 0:11:26.120 --> 0:11:28.920 also very specialized to what they're going to be running. 0:11:29.400 --> 0:11:31.560 Got it is it? Would it be right for me 0:11:31.640 --> 0:11:34.360 to say that, like, a data center designed for AI 0:11:34.760 --> 0:11:37.439 can be used for other things too, but a data 0:11:37.440 --> 0:11:40.560 center not designed for AI probably can't do AI? Is 0:11:40.559 --> 0:11:41.880 that a fair statement? 0:11:42.200 --> 0:11:45.040 I think that's a pretty fair statement. We're basically seeing 0:11:45.080 --> 0:11:48.199 new data centers being designed in a way that allows 0:11:48.280 --> 0:11:51.800 for AI training, and so there's an influence of like 0:11:51.880 --> 0:11:55.080 AI and data center design to be larger and more 0:11:55.200 --> 0:11:56.000 power tents. 0:11:56.440 --> 0:12:00.439 So in these training data centers that building these models. 0:12:00.679 --> 0:12:02.800 And one of the charts in the note that I 0:12:02.840 --> 0:12:06.240 really loved but haven't quite fully digested is one showing 0:12:06.480 --> 0:12:09.040 the amount of and you don't have to explain this 0:12:09.200 --> 0:12:15.120 unit amount of terror flops required too. Terrorflop that's just 0:12:15.160 --> 0:12:17.520 the unit of like computer work, isn't it. 0:12:17.600 --> 0:12:19.640 Yeah, it's just a basic unit of computation. 0:12:20.040 --> 0:12:24.360 Yeah, the amount of terror flops needed to design different 0:12:24.559 --> 0:12:28.320 AI models as they've become more and more in sophisticated 0:12:28.600 --> 0:12:31.840 and so obviously there's been an expectation that trend is 0:12:31.840 --> 0:12:34.680 going to continue, and everyone was you know, freaking out 0:12:34.960 --> 0:12:36.920 in both good ways and bad ways about all the 0:12:36.920 --> 0:12:38.920 power demand that this will intel. And then I remember 0:12:38.960 --> 0:12:41.320 deep seek came along and a lot of the people 0:12:41.440 --> 0:12:44.480 were like, oh, this changes everything. Can you just provide 0:12:44.520 --> 0:12:46.120 a bit of clarity on all of this. 0:12:46.720 --> 0:12:49.080 There was a couple of things I really tried to 0:12:49.200 --> 0:12:52.160 understand about all of this, just in terms of like 0:12:52.360 --> 0:12:55.040 power market fundamentals. I think when it comes to like 0:12:55.240 --> 0:12:59.480 forecasting for power demand, all things come down to some 0:13:00.160 --> 0:13:04.280 very basic constructs. It's usually like how much quantity of 0:13:04.280 --> 0:13:08.800 something and then the energy intensity of that something. And 0:13:08.960 --> 0:13:14.200 for data centers it's a very similar process. And when 0:13:14.200 --> 0:13:18.240 we think about like the energy efficiency of large language 0:13:18.280 --> 0:13:21.360 model AI training data centers, there's like a couple things 0:13:21.400 --> 0:13:24.760 to think through. First is around the energy intensity of 0:13:24.800 --> 0:13:28.920 power consumption from a chips perspective, and chip innovation is 0:13:28.960 --> 0:13:33.960 often like confused with like increasing energy efficiency. That's not 0:13:34.080 --> 0:13:37.840 necessarily the case. Typically when we think about chip innovation, 0:13:38.320 --> 0:13:42.400 it's often optimized for like operations per second, which means 0:13:42.440 --> 0:13:46.280 that like typically more like every generation of chips tend 0:13:46.320 --> 0:13:49.880 to have draw more power and therefore, like it increases 0:13:49.920 --> 0:13:53.920 the energy intensity of a data center. So there are 0:13:54.480 --> 0:13:57.800 new chips that are getting invented, like the Nvidia Blackwell 0:13:57.800 --> 0:14:01.400 that focuses on energy efficiency. But in general what we've 0:14:01.400 --> 0:14:04.400 seen is like the more advanced chips tend to draw 0:14:04.480 --> 0:14:08.240 on more power. The other thing that is really important 0:14:08.280 --> 0:14:11.640 for AI training is then like the number of parameters 0:14:12.000 --> 0:14:15.560 that an AI training model focuses on. So parameters is 0:14:15.640 --> 0:14:20.360 just like points of active information for a model to 0:14:20.440 --> 0:14:21.000 think through. 0:14:21.240 --> 0:14:23.360 So it's just the number of different things it thinks 0:14:23.400 --> 0:14:26.520 about sort of like so if I was thinking, like 0:14:27.000 --> 0:14:29.120 should I go to work today or should I walk 0:14:29.160 --> 0:14:32.440 to work today? Like, if I had one parameter, it 0:14:32.560 --> 0:14:34.760 might be what's the weather like outside? And if the 0:14:34.840 --> 0:14:37.720 two parameter might be what's the weather like outside and 0:14:37.760 --> 0:14:39.360 what day of the week is it? Yeah, you might 0:14:39.400 --> 0:14:42.240 determine whether or not I go to work. So that's 0:14:42.280 --> 0:14:44.520 like what each of those is a parameter. 0:14:44.280 --> 0:14:49.160 Exactly, exactly precisely and in general, like in the large 0:14:49.240 --> 0:14:51.760 language model community historically, or at least in the power 0:14:51.800 --> 0:14:55.440 industry historically, what people had thought through was that like 0:14:56.040 --> 0:15:01.600 more sophisticated models required more parameters. So with every generation 0:15:01.800 --> 0:15:05.600 of like chatchipt or Gemina or cloud. If you look 0:15:05.640 --> 0:15:08.960 at their like technical reports, what you'll see is that 0:15:09.000 --> 0:15:12.000 there's increasing amount of parameters that is used to like 0:15:12.080 --> 0:15:14.960 train these large language models, and so in general, like 0:15:15.240 --> 0:15:19.160 there was this overall consensus that the energy intensity of 0:15:19.200 --> 0:15:23.320 air training is an upward trend. You use more powerful chips, 0:15:23.360 --> 0:15:26.600 and you're using your training on more and more parameters, 0:15:26.840 --> 0:15:30.280 and so that's all driving more and more electricity consumption. 0:15:30.680 --> 0:15:34.320 Deep Seek came out in December twenty twenty four, and 0:15:34.400 --> 0:15:37.920 in their technical report, what was really cool was that 0:15:38.000 --> 0:15:41.800 they had a different training process. So it uses something 0:15:41.840 --> 0:15:45.800 called a mixture of experts training process, where instead of 0:15:45.880 --> 0:15:49.320 just like plugging in all of their parameters all at once, 0:15:49.720 --> 0:15:53.920 what they did was they pre categorized their parameters into experts. 0:15:54.240 --> 0:15:58.720 So like certain parameters that specialize in math or like colors, 0:15:58.960 --> 0:16:01.520 they would like pre categze them so that when you 0:16:01.560 --> 0:16:04.560 put in a query of a question to ask the 0:16:04.600 --> 0:16:08.080 model to train, it would know which specialization to pull 0:16:08.160 --> 0:16:12.040 the parameters from, which then drew on less power. So 0:16:12.400 --> 0:16:15.760 the other thing that the technical report published was that 0:16:15.880 --> 0:16:19.840 deep Seek performed at a very high level with chat 0:16:19.880 --> 0:16:23.000 GPT or like other types of large language models that 0:16:23.160 --> 0:16:26.320 used a lot more parameters, and so it kind of 0:16:26.320 --> 0:16:30.520 broke the assumption that you needed a whole bunch of 0:16:30.560 --> 0:16:33.760 parameters to make really sophisticated large language models. 0:16:34.280 --> 0:16:37.240 And so do you think that that will massively change 0:16:37.440 --> 0:16:39.480 the outlook for power demand from AI? 0:16:40.160 --> 0:16:42.280 So to answer your question from like a data center 0:16:42.320 --> 0:16:47.520 demand perspective, not necessarily in our near term forecast on 0:16:47.880 --> 0:16:50.920 in our data center outlook, we use like a project 0:16:50.920 --> 0:16:54.680 by project level forecasts, right. But in our new energy outlook, 0:16:54.920 --> 0:16:58.440 which focused more on like long term forecasts for data 0:16:58.440 --> 0:17:02.920 center demand, it focused on that fundamentals based way of forecasting, 0:17:02.960 --> 0:17:06.800 which it looks at long term in any given like market, 0:17:07.240 --> 0:17:11.240 what data generation and data usage looks like relative to 0:17:11.280 --> 0:17:14.320 the energy intensity of that data generation. 0:17:14.880 --> 0:17:15.040 Right. 0:17:15.200 --> 0:17:17.280 And there's a point you make in the report I 0:17:17.320 --> 0:17:19.359 recall and I don't think you were talking about deep 0:17:19.400 --> 0:17:22.160 seek here. I think you were talking about data center efficiency. 0:17:22.200 --> 0:17:24.760 But bring up Jevins paradox, which is this idea that 0:17:24.760 --> 0:17:27.320 if you make something more efficient, it doesn't mean necessarily 0:17:27.359 --> 0:17:30.440 that we save energy, it's that we just do more 0:17:30.480 --> 0:17:33.160 with what we were going to consume anyway, And could 0:17:33.200 --> 0:17:35.240 the same logic be applied for deep seek is if 0:17:35.280 --> 0:17:38.280 it is more efficient with parameters and therefore energy and 0:17:38.440 --> 0:17:41.320 also computer usage, then that just opens the door to 0:17:41.359 --> 0:17:44.360 do cooler things with AI than would have previously been possible, 0:17:44.680 --> 0:17:45.879 rather than to save energy. 0:17:46.280 --> 0:17:50.240 Yeah, the energy intensity curve of AI training data centers 0:17:50.280 --> 0:17:53.080 like instead of it being like an upward swing, it 0:17:53.800 --> 0:17:56.919 goes down. But we also then know it opens up 0:17:57.000 --> 0:17:59.480 a lot more opportunities for a lot of different types 0:17:59.480 --> 0:18:03.920 of business to maybe do AI training right, which means 0:18:03.920 --> 0:18:07.120 that you have more companies that may be doing this. 0:18:07.520 --> 0:18:10.080 So got it runs paradox. 0:18:09.920 --> 0:18:12.480 Very interesting and actually I think that brings some real 0:18:12.480 --> 0:18:15.840 clarity into what this whole deep seat thing means for 0:18:15.880 --> 0:18:20.920 power demand, which my main takeaway is like not that much. Ultimately, 0:18:21.240 --> 0:18:24.080 it's very difficult to say, but we shouldn't be saying, oh, 0:18:24.119 --> 0:18:26.159 this means all this data center demand growth isn't going 0:18:26.200 --> 0:18:29.560 to happen. Yes, So pulling out again, there's all of 0:18:29.600 --> 0:18:32.560 this data center build that's going to happen in the US, 0:18:32.800 --> 0:18:35.040 four companies are going to be behind a little bit 0:18:35.040 --> 0:18:36.960 more than half of it. Where are they going to 0:18:37.000 --> 0:18:39.640 be building all of this? And why are they going 0:18:39.680 --> 0:18:40.960 to be building in those places? 0:18:41.440 --> 0:18:44.800 Yeah? I'll also add on those four companies that are 0:18:45.119 --> 0:18:48.159 most of the data center market, they're also the companies 0:18:48.440 --> 0:18:51.560 that you know aren't building these AI training models and 0:18:51.640 --> 0:18:55.120 have the capacity to train large scale models because it's 0:18:55.160 --> 0:18:59.280 a very costly exercise that is only set to grow. 0:18:59.480 --> 0:19:02.080 They're also forty of the data center market, but they're 0:19:02.119 --> 0:19:05.560 also the ones training AI models because actually, not many 0:19:05.560 --> 0:19:08.040 companies can train A models, right, right. 0:19:07.880 --> 0:19:10.360 So a lot of the new big demand is becoming 0:19:10.440 --> 0:19:13.359 from these four companies. So a lot of the demand 0:19:13.440 --> 0:19:14.680 that we're talking about. 0:19:14.880 --> 0:19:17.000 Yeah, I guess today when we look at the data 0:19:17.000 --> 0:19:20.480 center fleet, they're mostly for cloud. But going forward, the 0:19:20.520 --> 0:19:24.040 companies that can actually train AI models is less than 0:19:24.080 --> 0:19:26.720 ten companies training like frontier models, and that those are 0:19:26.800 --> 0:19:28.240 going to be the big tech companies. 0:19:28.520 --> 0:19:31.760 And so then where is this happening and why in 0:19:31.800 --> 0:19:32.800 those locations. 0:19:33.119 --> 0:19:36.160 So in our forecast we see three main markets emerge 0:19:36.200 --> 0:19:39.280 through twenty thirty five. We break them down by power 0:19:39.320 --> 0:19:42.680 region for power forecasting purposes. 0:19:42.200 --> 0:19:45.960 And that's just because that's how we think. I don't 0:19:46.000 --> 0:19:47.560 see states, I see power region. 0:19:47.880 --> 0:19:51.280 Yeah, so we see PGM or Coat and Southeast, and 0:19:51.359 --> 0:19:57.720 within PGM, which spans fourteen states, we see Virginia continuing 0:19:57.760 --> 0:20:01.320 being one of the biggest markets so than Virginia has 0:20:01.400 --> 0:20:05.800 been data center capital for the last decade. If Virginia 0:20:05.840 --> 0:20:09.000 was a country, it would follow the US and China 0:20:09.040 --> 0:20:11.360 as having the largest data center market. 0:20:11.320 --> 0:20:12.480 In the world. 0:20:12.680 --> 0:20:14.680 Wow. Can I say that again? Wow? 0:20:14.840 --> 0:20:15.080 Yeah. 0:20:15.119 --> 0:20:19.119 So Northern Virginia has been kind of at the center 0:20:19.320 --> 0:20:21.760 of data center build out. A lot of this has 0:20:21.800 --> 0:20:24.399 to do with a bit of history. They had the 0:20:24.440 --> 0:20:28.760 first Internet exchange point in the nineties, which was kind 0:20:28.760 --> 0:20:31.480 of the beginning of small data center build out, and 0:20:31.720 --> 0:20:35.679 data centers typically continue to cluster an existing markets, so 0:20:36.000 --> 0:20:42.000 as data centers grow, they'll have supporting infrastructure like fiber optics, 0:20:42.600 --> 0:20:48.120 utility relationships, and workforce availability that allows more data centers 0:20:48.160 --> 0:20:51.240 to continue to grow. So over time, Northern Virginia just 0:20:51.280 --> 0:20:53.879 has gotten hotter and hotter, and we see in our 0:20:53.880 --> 0:20:57.320 project Pipeline a lot of that continuing to grow. We 0:20:57.440 --> 0:21:00.320 do know that a lot of this could be more 0:21:00.400 --> 0:21:03.920 AI inference rather than AI training, but still a lot 0:21:03.920 --> 0:21:07.840 of data center demand happening. There. Another state in PJM 0:21:07.960 --> 0:21:11.440 is Ohio, which is emerging as one of the main 0:21:11.680 --> 0:21:16.000 hubs in the Midwest. Google is building data centers in 0:21:16.080 --> 0:21:19.359 New Albany, which is just outside Columbus, one of the 0:21:19.440 --> 0:21:22.600 largest cities in Ohio, as well as other co location 0:21:22.960 --> 0:21:24.520 and hyperscular companies. 0:21:24.800 --> 0:21:27.719 You know, you've highlighted some a couple of major markets 0:21:27.720 --> 0:21:29.560