Speaker 1: This is Dana Perkins and you're listening to Switched on
Speaker 1: the BNAF podcast. Today we bring you a recording from
Speaker 1: our BNAF summit in San Francisco, which took place on
Speaker 1: the fourth and fifth of February. The panel was titled
Speaker 1: Data Dynamics. Now certainly a hot topic in energy circles
Speaker 1: has been the growth in data centers and how AI
Speaker 1: has led to a rise in demand for power. In
Speaker 1: order to meet this, big tech companies are looking for
Speaker 1: solutions and firms like Google are signing power purchase agreements
Speaker 1: in an array of technologies like nuclear, long duration energy storage,
Speaker 1: and even geothermal. While these data centers are undeniably energy intensive,
Speaker 1: efficiencies can be found when it comes to reducing load requirements.
Speaker 1: This is seen with the recent release of Deepseek's AI
Speaker 1: model in January twenty twenty five. On today's show, the
Speaker 1: panelists discuss the increasing capital expenditure on data centers, as
Speaker 1: well as what has been driving the decision making of
Speaker 1: big tech companies when it comes to this spend and
Speaker 1: how potential bottlenecks might slow them down. The panelists include
Speaker 1: Stephen Carlini, Chief Advocate Data Centers and AI at Schneider Electric,
Speaker 1: Will Conkling, head of Data Center Energy for the America's
Speaker 1: in Amia at Google, Kleiber Costa, chief commercial officer for
Speaker 1: AS Corporation, and Darwesh Singh, the founder and CEO of
Speaker 1: Bold Graphics. The panel was moderated by Mark Daily, BNF's
Speaker 1: head of Technology and Innovation. For more information about BNF
Speaker 1: Summit's taking place around the world, as well as our
Speaker 1: upcoming event in New York on the twenty ninth and
Speaker 1: thirtieth of April, an to view recordings from this and
Speaker 1: other previous events, head to about dot BNF dot com
Speaker 1: Forward Slash Summit. Right now, let's hear from our panel
Speaker 1: regarding power demand and data centers.
Speaker 2: Thank you everyone very much for joining us here today.
Speaker 2: We've heard an awful lot about the energy transition, like
Speaker 2: at all be enough events, and I imagine a lot
Speaker 2: of events that everyone here goes to. It's very energy
Speaker 2: focused group, transport focused group, but there's lots of big
Speaker 2: transitions happening in the global economy. We're here today to
Speaker 2: talk about the intersection of the energy transition with one
Speaker 2: of the other really big transformations in the economy, maybe
Speaker 2: the biggest one of our lifetime. If you believe some people.
Speaker 2: So we're here to talk about data centers, energy demand,
Speaker 2: artificial intelligence. I'm joined here today by Stephen Carlini, Chief
Speaker 2: Advocate Data Centers and AI at Schneider Electric, Will Conkling,
Speaker 2: head of Data Center Energy America's Animea at Google, Clever Costa,
Speaker 2: chief commercial Officer at the AEES Corporation, and Darbush Sing,
Speaker 2: Founder and CEO of Bolt Graphics. So the first thing
Speaker 2: that I want to talk about is really why has
Speaker 2: there been so much interest in data centers this year? Like,
Speaker 2: let's just start from the beginning. What's causing out go
Speaker 2: to you first, Star Resh, and if everyone can actually
Speaker 2: just introduce themselves and kind of their view on the
Speaker 2: industry in their first answer, that'd be great.
Speaker 3: Yeah. Thanks Mark.
Speaker 4: I'm dar Wesh Funer Ceobo Graphics. We are a semi
Speaker 4: conductory startup focusing on GPUs. My background is in building
Speaker 4: data centers, so I share a lot of the pains
Speaker 4: at the panel and in the industry in general. Mark,
Speaker 4: I think to your question, compute requirements have always been increasing.
Speaker 4: This is I think it's not new to anyone. We
Speaker 4: have phones now that are more powerful than PlayStation fours
Speaker 4: from twelve years ago. What is new, though, is the
Speaker 4: demand for ten years in the future computing power right now.
Speaker 4: And a lot of these AI companies, whether they're training models,
Speaker 4: whether they're building data centers to host these models, or
Speaker 4: whether they're making phones that can run inference on these things,
Speaker 4: they want it right now. And so I think that
Speaker 4: creates a lot of interest, a lot of hype. It
Speaker 4: also creates an opportunity for new players in the market
Speaker 4: to come in, whether they're DC builders that can build
Speaker 4: data centers in six months instead of four years, small
Speaker 4: modular nuclear reactor companies that are building these and seven
Speaker 4: years instead of twenty five years. So I think it's
Speaker 4: just like a timeline. Let's shift all this left. I
Speaker 4: want to do right now and what does that really enable?
Speaker 2: Okay, and so Stephen, could you give us a primer
Speaker 2: on how Schneider Electric relates to this conversation and when
Speaker 2: did your job change in this kind of this new
Speaker 2: hype cycle that we're going through.
Speaker 5: Yeah, Kinder Electric, if you're not familiar with this largest
Speaker 5: power and cooling solution provider for data centers in the world,
Speaker 5: and as you said in an earlier panel, you know
Speaker 5: AI is not new AI has been around for a while.
Speaker 5: We've been talking to a lot of the hyperscalers. We
Speaker 5: have staff of people signed to each hyperscaler, each large COLO,
Speaker 5: each large enterprise accounts. And we you know, we noticed
Speaker 5: five or six years ago, you know, these data centers
Speaker 5: can't be built overnight. And five or six years ago,
Speaker 5: a lot of the hyper scalers were coming to us,
Speaker 5: not only talking to us about higher densities, but higher
Speaker 5: densities and scales that we've never heard of. You know,
Speaker 5: twenty twenty megawat data centers. Back then it was a
Speaker 5: big data center. They're talking one hundred, one hundred and
Speaker 5: fifty three hundred megawat campuses back then, and we're like, wow,
Speaker 5: this is this is a big change. So the densities,
Speaker 5: you know, started to really change when Nvidia came out
Speaker 5: with you know, the A one hundreds and the.
Speaker 3: A one hundreds.
Speaker 5: It was about three years ago and and the A
Speaker 5: one hundreds were kind of the first you know, at
Speaker 5: scale deployments for a lot of the hyperscalers, and a
Speaker 5: lot of those were air cool they were twenty five
Speaker 5: kilowats parac Then you saw the grace hoppers, which were
Speaker 5: last year, which were thirty six kilowats are acting seventy
Speaker 5: two kilowats parak and now we have the Blackwells. The
Speaker 5: black Wells are one hundred and thirty two kilowats per rack,
Speaker 5: which is really pushing the limit of what we can
Speaker 5: get powered to these racks and cooling to these racks. Next,
Speaker 5: they're working on Ruben, which is the next generation. They're
Speaker 5: already we're already working on the designs for that at
Speaker 5: two hundred and forty kilowats per rack. So it's just exponentially.
Speaker 5: For years, it was just two socket x eighty six
Speaker 5: pizza box servers in these data centers, and we were
Speaker 5: doing ten kilo wats per rack. And in the last
Speaker 5: four or five years, it's just exponentially, you know, increased,
Speaker 5: and not just increased in small scale, but it's a
Speaker 5: large scale that we talked about.
Speaker 2: Okay, so will you're you're coming from Google, the company
Speaker 2: that started the all it was Google's R and D
Speaker 2: Live a transformer and Google's obviously been a huge energy
Speaker 2: procurement company for years and years though and when did
Speaker 2: when did this translate into changes in your job? The
Speaker 2: recent technology advances.
Speaker 6: Yeah, so the story for me is like I've been
Speaker 6: at Google just over ten years in some way to
Speaker 6: performed buying energy for data centers and buying renewal energy
Speaker 6: for data centers, and working with utilities and working energy supply.
Speaker 6: And you know, Google has been an AI first company
Speaker 6: since about like twenty sixteen, twenty seventeen. Our CEO has
Speaker 6: been saying that for a long time, right, And we've
Speaker 6: been infusing AI and machine learning into our products since
Speaker 6: then in various various ways that you know, we probably
Speaker 6: don't notice as users, but it's been there. And then
Speaker 6: in the last you know, eighteen months or so, it
Speaker 6: see the emergence of you know, sort of more consumer
Speaker 6: facing pure mL and AI products you know before so,
Speaker 6: like i'd say, mid twenty twenty three was kind of
Speaker 6: a watershed moment for us, but we saw the green
Speaker 6: suits of this before that, when you know, the grid
Speaker 6: started to see demand growth, you know, generally across a
Speaker 6: number of sectors and industries, right from more manufacturing and
Speaker 6: more batteries and manufacturing and more car manufacturing and more
Speaker 6: s reshoring of a lot of stuff back to the US,
Speaker 6: and that started to manifest itself, right, and utilities coming
Speaker 6: to us and saying, you know, they have to buil
Speaker 6: out their transmission system in ways that they hadn't anticipated
Speaker 6: in order to continue to serve load. And that's sort
Speaker 6: of you know, power for the course as business as
Speaker 6: usual for us. You couple that sort of like growth
Speaker 6: of going from twenty years ago or sorry for the
Speaker 6: last twenty years twenty twenty three of zero point five
Speaker 6: percent annual growth on the grid to to three four
Speaker 6: now four and a half five percent annual growth from
Speaker 6: the grid, and then you have an emergence of a
Speaker 6: new sort of large you know, data center demand with
Speaker 6: machine learning chips to what Steve has talked about with
Speaker 6: with nvidious chips and then our own internal tens of
Speaker 6: processing units. We saw in mid twenty twenty three the
Speaker 6: sort of crossover point from you know, being able to
Speaker 6: sort of generally source power where when we needed it
Speaker 6: on a timescal that made sense to us, to an
Speaker 6: acceleration of demand to what Darbush just said around you know,
Speaker 6: needing it sooner, and then a push out of lead
Speaker 6: times on some of the grid buildout and some of
Speaker 6: and serving that demand by utilities, and it's a it
Speaker 6: was kind of a collision of technology sort of product
Speaker 6: development curves and infrastructure timelines that don't always mesh well.
Speaker 6: And we've been kind of been in that that soup
Speaker 6: ever since. And it's I like to say, it's been
Speaker 6: one of the more dynamic times in my career. And
Speaker 6: I don't thpect I'll see anything like I would again, So.
Speaker 2: And clever, let's get the energy company perspective on this.
Speaker 2: When did you start noticing a big change from what
Speaker 2: was happening.
Speaker 7: Look, I think if you hear my my my colleagues
Speaker 7: here on the panel, it has been about a lot
Speaker 7: of changes over a very short period of time, a
Speaker 7: lot of volatility. I would say, you know, I think,
Speaker 7: I think you'll see, uh, these these massive growth over
Speaker 7: the past few years projections and then one day opened
Speaker 7: the papers and there's deep, deep seek there changing everything
Speaker 7: and you don't know what is true what is not.
Speaker 7: So everybody's trying to figure figure that out. Look the industry,
Speaker 7: I'm with AES corporation. I've been at AS for for
Speaker 7: about seven years there, but I've been in the energy
Speaker 7: business for about twenty five years. I started my career
Speaker 7: right when air Ron was the greatest thing on earth.
Speaker 7: Everybody wanted to work for and run not not after
Speaker 7: that and Ron, we all know what happened. And then UH,
Speaker 7: there's the boom and bust on the gas cycle. Hainesville
Speaker 7: Economics showed up with Shell Gas UH. A lot of
Speaker 7: companies also went bankrupt, and volatility in the in the
Speaker 7: energy sector disappeared for a long period of time, with
Speaker 7: very small growth over the past five years or maybe
Speaker 7: a little a little more, as Will said, is explosive demand.
Speaker 7: I think it's been one of the most dynamics periods
Speaker 7: of times of my career as well.
Speaker 3: But I guess the point here.
Speaker 7: Is that the energy the industry is not UH is
Speaker 7: very familiar with challenges and how to overcome those challenges.
Speaker 7: So I think my job really changed when when we
Speaker 7: moved from being providers of projects of or or or
Speaker 7: or or technology to partnering with some of these hyperscalers,
Speaker 7: some of these large data center companies and start putting
Speaker 7: together solutions with them. Not only at AS, not only
Speaker 7: we're one of the largest according to bn EF. Actually
Speaker 7: we're the largest provider of renewable energy to corporates over
Speaker 7: the past three years, so we know how that works.
Speaker 7: But we also own utility companies. We own utilities in
Speaker 7: Ohio and Indiana, and we work very closely with those
Speaker 7: hyper scalers and data center companies to meet their needs
Speaker 7: at the utility level as well.
Speaker 3: But again, I think the biggest takeaway.
Speaker 7: It feels very very uncertain in terms of demand projections.
Speaker 7: We're going to talk about that later. But whatever projections
Speaker 7: you look, however you slice and dice there. The growth
Speaker 7: is here, is real, and I think the industry is
Speaker 7: everybody said to meet those challenges and meet solutions for
Speaker 7: that growth.
Speaker 2: So this growth is real, we need to deal with it.
Speaker 2: What's been a big question that I've been asked actually
Speaker 2: a lot in my role is does this look different
Speaker 2: than the data center growth that came before? With the
Speaker 2: idea being data centers have been cited close to populations
Speaker 2: so that latency is low and your Netflix loads really quick.
Speaker 2: But that might not be the case for a new
Speaker 2: AI applications where actually training doesn't need to take place
Speaker 2: close to people. Maybe it'll take place in a far
Speaker 2: away place. Starsh what's your sense of this kind of
Speaker 2: regional dynamic.
Speaker 4: Yeah, definitely, AI is changing the requirements for where you
Speaker 4: build data centers, where you can source power and also
Speaker 4: the conversation. Like I think four years ago, if I
Speaker 4: had a conversation with someone about sourcing like hydrogen power
Speaker 4: for a data center, be laughed at.
Speaker 3: And now I'd be laughed at, but like a lot less.
Speaker 4: I think I'd be pointed towards other areas maybe in
Speaker 4: this direction.
Speaker 3: But definitely.
Speaker 4: The workload that that's running in the data center does
Speaker 4: impact really heavily what those requirements are where I can
Speaker 4: put that data center. And now with training and with
Speaker 4: let's say training is a batch workload, all right, HPEC
Speaker 4: supercomputing is also a batch workload that can also be
Speaker 4: an maybe choose Jensen's common It can be an Antarctica somewhere, right,
Speaker 4: it doesn't be very close to me. So yeah, that
Speaker 4: definitely does change, But I don't think it changes the
Speaker 4: economics that much because you still have to deliver a
Speaker 4: lot of power to it. You still need a really
Speaker 4: fat gigabit multi hunter gigabit network link, and these requirements
Speaker 4: like haven't really changed, I think, to be honest, Like,
Speaker 4: if you're going to build a lot of data centers,
Speaker 4: you're already going in the direction of, hey, I need
Speaker 4: to build these further and further away from large metropolitan
Speaker 4: areas because I can't get power anyway.
Speaker 3: So this is a problem like three four years ago,
Speaker 3: it's just worse now.
Speaker 2: Well, actually, interested in you, you're the energy part of
Speaker 2: the equation at Google. Can you give us a bit
Speaker 2: of information? How does that factor into decision making rom
Speaker 2: where to put data centers? Is it it's decided where
Speaker 2: it goes and then we need to find energy or
Speaker 2: is it part of the decision making process?
Speaker 3: Is that the first thing you decide on?
Speaker 6: Yes? And yes it you know historically so before you know,
Speaker 6: before the growth and the grid that I talked about
Speaker 6: kind of started. You know, it's not quite true, but
Speaker 6: you could almost like throw a dart on a map
Speaker 6: and you know, if you were somewhere within a reasonable
Speaker 6: distance of a metropolitan area, you could probably find power
Speaker 6: and a utility would at least build something for you
Speaker 6: in a couple of years, right, And and that was
Speaker 6: generally okay. You know, today, available grid and available generation
Speaker 6: on that grid are are more scarce, at least for
Speaker 6: the you know, short to medium term, right, And so
Speaker 6: you have to be sort of smarter and better at
Speaker 6: picking locations that have available power in the time scale
Speaker 6: you're looking for and and and sort of act quick
Speaker 6: to go and and reserve it and use it, right. So,
Speaker 6: so yet, yes, it's a it's a more important factor
Speaker 6: for us than maybe it was in the past. But
Speaker 6: it doesn't I think divers to your last point there.
Speaker 6: It's like if you go in the middle of nowhere,
Speaker 6: like there isn't power, there isn't fiber, there isn't there
Speaker 6: aren't people to build things, right, There's just these things
Speaker 6: require a certain amount of infrastructure and civilization around them
Speaker 6: to support them, right, and no one wants to live
Speaker 6: next to it. You have to have people that work
Speaker 6: there all the time, right like so so so there's
Speaker 6: there's that factor. And and then our products also like
Speaker 6: don't want to be in the middle of nowhere if
Speaker 6: they can help it. Because if you build a building
Speaker 6: for a data center, and and and so I'll back up.
Speaker 6: There's there's a few different ways that or its fuveent
Speaker 6: things that happen in an mL data center, right or
Speaker 6: a data center. One, you could be training at Google
Speaker 6: at least an internal model too. You could have a
Speaker 6: customer paying you to be able to train their model, right,
Speaker 6: or Three, you could be serving a model or serve
Speaker 6: doing inference and serving AI to customers, right. Only that
Speaker 6: first thing really is that flexible because our customers still
Speaker 6: want to be like within spitting distance of their of
Speaker 6: their footprint on the cloud. And then for inference, we
Speaker 6: still want to be having little latency service to our
Speaker 6: to our consumers, right. And so the internal training, yes,
Speaker 6: it's more flexible. But if you build a data in
Speaker 6: the middle of nowhere just for that and then you
Speaker 6: finish that job, you actually have a Strandard asset, right.
Speaker 6: And so if you think about efficiency of capital and
Speaker 6: how you want to be able to reuse your capital
Speaker 6: and recycle it, you actually don't necessarily always want to
Speaker 6: be just going far afield to to you know, the
Speaker 6: antarcticas or the deserts. We're might to be power, but
Speaker 6: no people or no no users or no customers. So
Speaker 6: so we tend to still have to follow where our
Speaker 6: products want to be. And and then within those you
Speaker 6: know regions or those uber regions we have to we
Speaker 6: have to go then you know, find power availability.
Speaker 2: So okay, great, and Stephen, you you have great insight
Speaker 2: into the entire kind of data center supply chain in
Speaker 2: your ola Schneider, I'm interested, do you have any kind
Speaker 2: of sense of how much of this new data build,
Speaker 2: data center build that we're seeing is specifically I related.
Speaker 2: Is that even something that makes sense? Is there an
Speaker 2: AI data center versus something else or is it just bits?
Speaker 5: Absolutely, the you know, the servers are completely different, and
Speaker 5: you know, the power and the cooling is different. And
Speaker 5: if you look at a data center, it's an AI
Speaker 5: data center today, say it's a ten megawaut data center.
Speaker 5: You know, five or six years ago, there'll be a
Speaker 5: thousand IT racks and the data hall would be huge.
Speaker 5: Now the data hall has seven d I t racks
Speaker 5: and you have all these chillers outside, you know, to
Speaker 5: support the cooling that so it's much different. It's not
Speaker 5: these Amazon warehouse type data centers.
Speaker 6: More.
Speaker 5: There's smaller, more confined in the IT rooms, a lot
Speaker 5: of power and cooling going through it. It's not a
Speaker 5: place that you know, it used to be walk around
Speaker 5: and you know your place, servers and everyone had a
Speaker 5: good time, but not anymore. It's a it's a business.
Speaker 5: But you know, as we were saying, you know, the
Speaker 5: you know, putting an asset where there's where there's power,
Speaker 5: and everybody in the world right now is saying that
Speaker 5: the data centers are going to go where.
Speaker 3: The power is.
Speaker 5: But but what we're what we're seeing and the hyper
Speaker 5: scales are all doing this as you just said, is
Speaker 5: they're they're building these training clusters that are close to
Speaker 5: where people are with the intention of using those to
Speaker 5: make money. There's not a lot of money being made,
Speaker 5: you know, training a model, and the question is how
Speaker 5: many more of these models are we going to need,
Speaker 5: so the money is going to be made and deploying
Speaker 5: them in the field. And we're seeing a big shift
Speaker 5: and this, in my opinion, is kind of the year
Speaker 5: of infernts and and we're starting to see you know
Speaker 5: a lot of these training clusters that were originally deployed
Speaker 5: just to train are now doing infrints, either full time
Speaker 5: or part time. And the other thing that we're seeing
Speaker 5: is you know, with infrints close to the users, optimized
Speaker 5: for different applications. We're not seeing those yet because AI
Speaker 5: is still developing. We're still at the beginning of this.
Speaker 5: We don't know, you know, what it's going to take
Speaker 5: to do an AI agent and a genta AI. You
Speaker 5: know what's that going to take. It's going to be
Speaker 5: multi modal how much processing, how much how much of
Speaker 5: the IT stack is going to be needed and where,
Speaker 5: and as we start inputting more and more video. You
Speaker 5: right now everything's text to text, you know, multimodal, it's
Speaker 5: going to be video, tech, text, image, it's going to
Speaker 5: be all these different things. So we can't optimize the
Speaker 5: data centers close to the users yet for inference, we're
Speaker 5: still going to have to do those in data centers.
Speaker 5: I think that's going to be the case for a
Speaker 5: few years.
Speaker 2: Okay, so things not really changing too dramatically. It's just
Speaker 2: build close to the users, like always.
Speaker 3: So clever.
Speaker 2: Actually, something I want to ask you about. I'm from
Speaker 2: Ireland and so when I hear data center, I think,
Speaker 2: oh my god, it's destroying the power system. There's like
Speaker 2: moratorium and new data centers there because it's such a
Speaker 2: large share of the power system. And there's a couple
Speaker 2: of reasons in Europe where something like this has happened,
Speaker 2: and now there's been conversations about this. This is going to
Speaker 2: happen in the United States. Kind of seems like that's
Speaker 2: calm down in the last few months. But interested to
Speaker 2: hear your thoughts on how big a challenge this will
Speaker 2: be for your business.
Speaker 7: It's hard to talk about that without politicizing things. But
Speaker 7: I think, look, there will be parts of the country
Speaker 7: where there's there would be some resistance uh to data
Speaker 7: center deployment. We are seeing some of that in the
Speaker 7: Southeast and other and other parts. I think I think
Speaker 7: it's the real answer to that is all going to
Speaker 7: depend on the solutions that we bring to to that
Speaker 7: data center load growth.
Speaker 3: When you say we, do you mean as or do
Speaker 3: you mean we.
Speaker 7: The providers of energy and together with the data center
Speaker 7: operators and the and the hyper skaters. I think a
Speaker 7: lot of all we talked about here is that there
Speaker 7: was a there was a time not long ago, there
Speaker 7: was this idea that not because of the l LAM
Speaker 7: training phase, a lot of data centers, large data centers
Speaker 7: will co locate with generation in places where there's as
Speaker 7: Will was saying, there's no load, that there's no infrastructure,
Speaker 7: there's no fiber offs so you have to make up
Speaker 7: for the lack of all that with low cost of power.
Speaker 7: But when you when you when when you look at
Speaker 7: the amount of capital that in you to deploy in
Speaker 7: those data centers, you don't want to run the risk
Speaker 7: of being stranded after the training phase of the of
Speaker 7: the large language model uh ends, so you want to
Speaker 7: use that for something else. So we talked a lot
Speaker 7: about this that here you end up going back to
Speaker 7: where the traditional data center markets are, and in some
Speaker 7: of those markets we are seeing local resistance.
Speaker 3: Uh.
Speaker 7: We're also seeing local resistance to the development of power
Speaker 7: plants to supply those those data centers. The famous NIM
Speaker 7: business not in my backyard. So I think that is
Speaker 7: a challenge that that the industry, both the energy industry
Speaker 7: and the data center industry, has to overcome. It is
Speaker 7: a real challenge, uh. And I think it's one that
Speaker 7: will be will be met with deployment of transmission, because
Speaker 7: I think that there's more flexibility where you can deploy
Speaker 7: data centers than there will be on where you deploy
Speaker 7: the generation, whether it's gas or or renewable. I hopefully
Speaker 7: we don't get to new coal to supply to supply
Speaker 7: this demand. And the bottomneck here is actually transmission to
Speaker 7: get to from from from the generational sources to the
Speaker 7: to the data centers.
Speaker 2: Okay, and we'll actually going to ask you about Google
Speaker 2: has been quite active on trying to develop new sources
Speaker 2: of clean energy before actually this whole AI drive became
Speaker 2: a thing, but you signed a couple of pretty first
Speaker 2: of a kind PPAs in the last couple of years.
Speaker 2: It'd be interesting to hear how progress in that is developing.
Speaker 3: Yeah.
Speaker 6: Sure, So the history of you know, energy consumption and
Speaker 6: energy generation Google is a long one. But the short
Speaker 6: story is this, since I've been there last you know,
Speaker 6: ten and a half yars or so, are our energy
Speaker 6: consumption globally has grown between twenty twenty five percent a year.
Speaker 6: We're now approaching thirty plus maybe more tearrawad hours of
Speaker 6: energy you know, consumed every year. That's doubling every you know,
Speaker 6: five years or so. And we've also signed you know,
Speaker 6: you know, twenty plus gigawatts of renewable energy generation you know,
Speaker 6: contracts to with folks like A Yes and others to
Speaker 6: to help supply energy to our facilities. We we maintain
Speaker 6: a strong climent to our to our clean energy goals,
Speaker 6: and we have an hourly carbon free energy goal a
Speaker 6: BYT twenty thirty that we continue to chase and uh
Speaker 6: chase very vigorously and and to that end, you know,
Speaker 6: we The story there is you can get to about
Speaker 6: seventy to eighty percent carbon free and energy supplies you know,
Speaker 6: through wind and solar and batteries and sort of like
Speaker 6: the mix of things there. But that last twenty to
Speaker 6: twenty five percent, that last mile is is harder because
Speaker 6: you have to start thinking about capacity and baseload and
Speaker 6: reliability and this sort of stuff. And so we spent
Speaker 6: the last few years thinking about and working on, you know,
Speaker 6: sort of what are those next gen technologies after wind
Speaker 6: and solar that are going to be carbon free and
Speaker 6: start to supply you know, up to that that nearly
Speaker 6: one hundred percent carbon free energy. And for US, it's
Speaker 6: things like nuclear power, it's things like launderation storage, it's
Speaker 6: potentially hydrogen, don't look too hard. It's potentially like carbon
Speaker 6: capture and storage and geothermal and we've done a couple
Speaker 6: of these in the last couple of years. We did
Speaker 6: a deal in Nevada between US and a company called Ferbo,
Speaker 6: who's a geothermal developer. They're using our cloud technology to
Speaker 6: optimize how they drill wells and operate their wells and
Speaker 6: operate their plants to then build advanced geothermal plants to
Speaker 6: sell that power to Novada Energy, the utility with whom
Speaker 6: We designed a tariff for a new rate that the
Speaker 6: regulator is approving that's gonna a sign or a lot
Speaker 6: allocate the costs of that geothermal above and beyond sort
Speaker 6: of business as usual to us, the customer right so
Speaker 6: that we don't so that we get the product we want,
Speaker 6: the grid gets a clean baseload generation source, and other
Speaker 6: right pairers don't don't pay the cost. So that's what
Speaker 6: we're really proud of. And that's a model, you know,
Speaker 6: that sort of rate structure and that you can slot
Speaker 6: different technologies into is a model we're working on with
Speaker 6: a lot of other utilities across the US. And then
Speaker 6: the other is a partnership we recently as signed with
Speaker 6: a small modular reactor technology provider called Chiros Power. Chiros
Speaker 6: is developing Gen four small monulor reactors. They have a
Speaker 6: pilot planned for twenty twenty nine for which we're going
Speaker 6: to be a customer in the Tennessee Valley. And in
Speaker 6: addition to being a customer for the pilot to help
Speaker 6: get that commercialized and off the ground, we committed to
Speaker 6: being a customer for their next five reactors. And what
Speaker 6: we get out of that is, you know, some confidence
Speaker 6: of having access to power, assuming that everything goes well
Speaker 6: with our technology, but also the ability to sort of
Speaker 6: partner with them to cite those next reactors in places
Speaker 6: that you know, hopefully make make sense for us and
Speaker 6: our loads and our data centers to then meet that
Speaker 6: hourly carbon free energy goal in places we have growing load.
Speaker 6: And so we expect in the twenty thirties to be
Speaker 6: deploying small modulor reactors onto the grid. It's not going
Speaker 6: to be like a you know, Antarctica small modul the
Speaker 6: actor of data center behind the meter microgrid thing. That's
Speaker 6: not the plan. It's really meant to be a grid
Speaker 6: participant and putting capacity on the grid to supply our needs.
Speaker 6: And so Bi you said about that as well.
Speaker 3: Yeah, okay, great.
Speaker 2: So something someone alluded to earlier was the topic of
Speaker 2: energy efficiency and the word deep seek, which hads.
Speaker 3: Up in the room.
Speaker 2: Does everyone know like what I'm referring to when I
Speaker 2: talk about deep seek? Yes, okay, great, darsh I'm gonna
Speaker 2: go to you here because we're having a conversation whether
Speaker 2: this this something that everyone saw coming. Not necessarily the
Speaker 2: idea that like deep seek was going to release a
Speaker 2: model and this would be the exact market reaction, but
Speaker 2: that there was going to be big gains in energy
Speaker 2: efficiency improvements. This has obviously been the history of data
Speaker 2: centers for ages that energy efficiency has improved. So like,
Speaker 2: why were people so surprised by this? And what's the
Speaker 2: future of energy efficiency in artificial intelligence? Simple question if.
Speaker 3: You could just do great question.
Speaker 4: I think technology comes in waves where like you make
Speaker 4: really good hardware and then you try to extract performance
Speaker 4: out of the hardware as much you can. When you
Speaker 4: reach the limit of what you can do in software space,
Speaker 4: you go back and make better hardware. And ideally that's
Speaker 4: like a very quick trend of like, hey, I spent
Speaker 4: six months, let's say one to two years making hardware,
Speaker 4: one to two years making software, and then I find
Speaker 4: the holes in the hardware and I make it better. Right,
Speaker 4: So I think, what I think, it's just more mostly
Speaker 4: timing like this happened now, I think, yeah, this is
Speaker 4: the expectation is that, hey, I can only buy a
Speaker 4: certain number of Vida GPUs. What can I do with this?
Speaker 3: Now?
Speaker 4: Let me hire highly specialized PTX programmers that don't write
Speaker 4: Kuda code. They right level below that because Kuda doesn't
Speaker 4: solve the problem that I needed to solve. It's too abstracted,
Speaker 4: it too, it's too power hungry, right, so I don't
Speaker 4: get I don't get enough control over the hardware, so
Speaker 4: I need to go to a lower level. You keep
Speaker 4: going down that stack. You get down to hardware, then
Speaker 4: you go down, you go to TSMC, right, then you
Speaker 4: go down you get to like minerals and things like that.
Speaker 3: So you can keep going down that stack.
Speaker 4: But definitely, I think what Deep Sea proved and if
Speaker 4: you guys with the pre the paper, the last page,
Speaker 4: there's suggestions on how to improve in Vida GPUs, which
Speaker 4: is really interesting because this is a redesigning them how
Speaker 4: to use them. Yeah, the micro architecture of the Vida
Speaker 4: GPUs is not optimal for what deep SEEQ wants basically,
Speaker 4: which is interesting, right because that's the that's the cycle
Speaker 4: we're going to go through and so not related. But
Speaker 4: you know, the gp that we're designing solves those problems.
Speaker 4: We did our own benchmarks three years ago and we
Speaker 4: found out some of the problems that Deep Sea found
Speaker 4: out as well. So there are ways to improve hardware honestly,
Speaker 4: like the vendor should do you know, benchmarking and research
Speaker 4: and make the GPUs better themselves. But yeah, it does
Speaker 4: require some interactivity with and end user that's like, hey,
Speaker 4: I want this to be better. Here's like four things
Speaker 4: you can do to make it better. And that will
Speaker 4: continue happening, right. People will make new GPUs, new accelerators,
Speaker 4: people will make co package optics, they'll do all sorts
Speaker 4: of fancy stuff and people will use it. But like,
Speaker 4: I'd actually don't like the way this is running. It's
Speaker 4: actually too slow for my use case. Can you fix
Speaker 4: this thing? So I think this is like normal, but
Speaker 4: I think there's so much focus on the volume of
Speaker 4: GPU ship that I think people forgot that there's still
Speaker 4: optimization room. There's a lot of headroom and optimizing software
Speaker 4: for that.
Speaker 2: Okay, how do you how does that kind of coordination
Speaker 2: work between the software developers and a video like is there?
Speaker 2: Do they have channels to work on this together? Doing
Speaker 2: video have their own internal research teams? Yeah, it's called
Speaker 2: hardware software code design. Some companies do it much better
Speaker 2: than others. We do it the best. Yeah, we do
Speaker 2: a good job.
Speaker 4: But yeah, no, they have teams internally that are building
Speaker 4: fundational models at video train them on supercomputers, AI clusters
Speaker 4: and then they're finding these things. But I think it's
Speaker 4: interest that, Like your next question is, well, why didn't
Speaker 4: they find this out, you know last year when they
Speaker 4: made the hardware, Well, it's the same thing. Why is
Speaker 4: black WU one hundred three to two kilowats per wack
Speaker 4: instead of one hundred and twenty five? So there are
Speaker 4: like there are fuzzy zones where you can miss things,
Speaker 4: and I'm sure I'll miss things and it happens. But
Speaker 4: I think the magnitude of that coming out so aggressively
Speaker 4: saying hey, we don't need this much computing power. And
Speaker 4: also there are things in the GPU at the micro
Speaker 4: architecture level in the silicon that I don't like that
Speaker 4: I want you to change.
Speaker 3: Is is a step shift because.
Speaker 4: Now you're expecting every customer to go down to that level.
Speaker 4: And I think that's the race now as to how
Speaker 4: optimized can you get with one water one hundred watts
Speaker 4: or one giga water or whatever?
Speaker 2: Do you have a kind of benchmark in your own
Speaker 2: mind internally of this is the energy standard that like
Speaker 2: a query and chat GPT was a year ago. How
Speaker 2: much more energy efficient can we get? That? Is it
Speaker 2: orders of magnitude or.
Speaker 3: Orders of magnitude orders of magnitude.
Speaker 4: Absolutely, yeah, I think this conversation is good because everyone
Speaker 4: in the panels like it. Will we're delivering power, that's great,
Speaker 4: that's a problem. But also like I'm a chip guy,
Speaker 4: like we should make better chips that consume less power.
Speaker 4: Perhaps maybe there's like a push and pulled balance there
Speaker 4: of you know, like keep using a good job. Right,
Speaker 4: that's orders of magnitude less, less power consumption, more efficient
Speaker 4: than and envita GPU. It is the main specific in
Speaker 4: that sense, but it does solve the problem and it's
Speaker 4: more efficient. So you will see like a chip startups,
Speaker 4: AI startups, quote package optic startups, all these all these
Speaker 4: companies competing and being able to deliver much orders a
Speaker 4: magnitude better efficiency.
Speaker 3: That's what we're doing.
Speaker 2: Okay, So for the three other panelists, I'm got to
Speaker 2: ask you the same question is did you see this coming? Well? Actually, sorry,
Speaker 2: The first question is how do you operate and like
Speaker 2: you need to make these big decisions about what to
Speaker 2: build and what to allocate resource to under this level
Speaker 2: of uncertainty around energy efficiency improvements. But then I guess
Speaker 2: the second part of the question is does everyone who
Speaker 2: works in the industry kind of assume there's gonna be
Speaker 2: these energy efficiency imrovements you're kind of making decisions around that.
Speaker 5: So ho to you first student, I think everyone expected
Speaker 5: more efficiencies to be to be gained in the transformers
Speaker 5: and the algorithms and and but you know, I think
Speaker 5: he had a panelist earlier that said, you know, all
Speaker 5: the all the models that have been trained have been
Speaker 5: trained on the public data, and there's all this other,
Speaker 5: you know, private data that's actually going to be you know,
Speaker 5: more beneficial. But all I can say about about that
Speaker 5: is that it was this is probably the most confusing topic,
Speaker 5: all the experts that are weighing in and saying completely
Speaker 5: different things. But we haven't seen any reaction, negative reaction
Speaker 5: from of our from our customers as far as you know.
Speaker 3: Orders or or forecasts. It's all. It's all, you know,
Speaker 3: business as usual. There's been like no effect at all.
Speaker 6: Yeah, well, yeah, I think you know, Google and look,
Speaker 6: training models and and and uh, software and hardware are
Speaker 6: by no means my own expertise, and so so i'm
Speaker 6: I'm I don't have a lot to offer here except
Speaker 6: for I think we always expect deficiency gains. I think
Speaker 6: we welcome them. I think we're seeing the same sort
Speaker 6: of efficiency gains and our own you know, model building
Speaker 6: and model training and uh and uh, you know that
Speaker 6: it hasn't to See's point, it hasn't really changed our
Speaker 6: look on on our on our business planning because training
Speaker 6: is only part of you know, the AI story and
Speaker 6: the machine learning story. The serving and the inferences is
Speaker 6: A is also a big part of it. And and
Speaker 6: you know, don't forget that Google does myriad other things
Speaker 6: and data centers right, and so data center, so mL
Speaker 6: and A I are are but a portion of our
Speaker 6: of our forecast and our outlook, and so so we
Speaker 6: remain sort of on a stalle path.
Speaker 7: Yeah, I think the same thing here. I haven't seen
Speaker 7: any real change perhaps with the with the small exception
Speaker 7: of investors over reaction to to headlines, but but the
Speaker 7: fundamentals of the business remain very strong. Let's not forget
Speaker 7: here that a lot of what we're trying to solve
Speaker 7: for in the power markets is also replacement of aging
Speaker 7: infrastructure in the US, especially no cold generation gas generation.
Speaker 7: So efficiency gains are definitely welcome because what that's going
Speaker 7: to do is to make sure that we don't overbuild.
Speaker 7: The worst thing that could happen here is if the
Speaker 7: market starts overbuild, overbuilding generation, transmission, distribution, and rate payers
Speaker 7: get left with massive bills and the demand doesn't show up, right,
Speaker 7: that would be a cycle of boom and bus.
Speaker 2: Right.
Speaker 7: So we believe in competitive markets. We believe in market efficiency.
Speaker 7: And I think if if the efficiency is on the on,
Speaker 7: the on, the on the computing power, also if that
Speaker 7: brings demand back to a more reasonable level. But no
Speaker 7: matter what the projections are, the numbers are staggering. The
Speaker 7: projections for demand for new generation and new and new
Speaker 7: infrastructure for transmission are staggering. So we just hope that
Speaker 7: that that we don't overbuild. I guess that's right.
Speaker 3: Okay.
Speaker 2: So I have one quick fire question which you're only
Speaker 2: allowed to answer one number two by twenty thirty. What
Speaker 2: percentage of US power demand do you think will come
Speaker 2: from data centers?
Speaker 3: Not no explanation, just a number.
Speaker 2: There's a networking session after seven and a half, well SIXI.
Speaker 7: Ish, yeah, I think I'm a little perhaps a little
Speaker 7: more more bullish than that. I think it would be
Speaker 7: like somewhere between eight and ten percent.
Speaker 2: Okay, especially pretty clustered around a certain range. But thank
Speaker 2: you very much. This is really informantive panel. I think
Speaker 2: we learned a lot about the idea that the data
Speaker 2: center industry is going to grow. We're very all, very
Speaker 2: confident on that, but actually maybe won't change quite as
Speaker 2: much in terms of its geographic structure or capital investment cycled.
Speaker 2: So thank you very much for joining me. Please join
Speaker 2: me and giving my Palels a round of applause.
Speaker 1: Today's episode of Switched On was produced by Cam Gray
Speaker 1: with production assistance from Kamala Shelling. Bloomberg NIF is a
Speaker 1: service provided by Bloomberg Finance LP and its affiliates. This
Speaker 1: recording does not constitute, nor should it be construed as
Speaker 1: investment in vice, investment recommendations, or a recommendation as to
Speaker 1: an investment or other strategy. Bloomberg ANIF should not be
Speaker 1: considered as information sufficient upon which to base an investment decision.
Speaker 1: Neither Bloomberg Finance LP nor any of its affiliates makes
Speaker 1: any representation or warranty as to the accuracy or completeness
Speaker 1: of the information contained in this recording, and any liability
Speaker 1: as a result of this recording is expressly disclaimed,