Speaker 1: Hello, and welcome to another episode of the Odd Lots Podcast.
Speaker 2: I'm Joe Wisenthal and I'm Tracy Alloway.
Speaker 1: Tracy, have you looked at in video stock chart lately?
Speaker 1: And by lately, I don't mean like over the last
Speaker 1: two years. I mean like just like over the last
Speaker 1: like two weeks or two months.
Speaker 2: I don't need to look at it because everyone keeps
Speaker 2: talking about it. So I know, I know what's happening.
Speaker 1: You know what, I'm pretty happy about. Could I just say,
Speaker 1: you know, we did that episode like two months ago, yes,
Speaker 1: with Stacy Raskin, and we were like, what's what's up
Speaker 1: a good video, like well, you know, I know it's
Speaker 1: at the center of the AI chips boom and whatever.
Speaker 1: And then like we did that episode and it came
Speaker 1: out and then a week later like they just like
Speaker 1: knocked it out of the park.
Speaker 3: Yeah, so you.
Speaker 2: Know, we were early.
Speaker 3: We were at least like, you know, a good like
Speaker 3: two weeks earlier.
Speaker 2: Hey, hey, two weeks I'll take it.
Speaker 3: I'll take it.
Speaker 1: So clearly something that you know, we and we talked
Speaker 1: about this with Stacy, like you know, something that in
Speaker 1: Nvidia has is like everyone's trying to buy it. Everyone's
Speaker 1: trying to get it, But then it raises the next
Speaker 1: question of like, okay, but what is that market? Like
Speaker 1: how do you buy a chip?
Speaker 2: Yeah? How do you buy a chip? And then I
Speaker 2: guess what do you actually do with it once you
Speaker 2: have it? Because my impression is that for a lot
Speaker 2: of these AI applications, the way you use the chips,
Speaker 2: the way you set up the data centers is very,
Speaker 2: very different to what we've seen in the past. And
Speaker 2: I think also what in Vidia is doing now is
Speaker 2: kind of different. But maybe we can get into this
Speaker 2: with our guests. My impression is they're trying to create
Speaker 2: a sort of like holistic approach for customers where they
Speaker 2: provide not just the hardware, but also some services to
Speaker 2: go along with it.
Speaker 1: Yes, right, and like all the software and Stacy talked
Speaker 1: about that with the Kuda ecosystemica that.
Speaker 3: Was it, how dominant that is? But right, like what
Speaker 3: do you do with it? Like how do you get one? If?
Speaker 3: Like what you know, what would we do.
Speaker 1: Tracy if a big palette of in Vidia chips wound
Speaker 1: up here?
Speaker 2: Do you want to know a secret? Yeah, my basement
Speaker 2: is filled with h one hundred chips. Just got a
Speaker 2: pile of them. It came with the house.
Speaker 1: It was on that ship that was stuck off the Chesapeake,
Speaker 1: and instead of getting your cowards, you got it.
Speaker 2: I just caught a palette of age four hundreds.
Speaker 1: That that well, we're manifesting that into reality. So anyway,
Speaker 1: I like how this world works so essentially, like the
Speaker 1: trading and dealing of these, like the hottest commodity in
Speaker 1: the world right which is these these advanced chips from AI,
Speaker 1: and how that works and who can get one? I
Speaker 1: still think is like a sort of mystery that we
Speaker 1: need to delve further into this question.
Speaker 2: I agree, And there is also there's a lot of
Speaker 2: excitement around it right now for the obvious reasons of
Speaker 2: everyone's really into generative AI and in video stock is exploding,
Speaker 2: as we already talked about, but we're also seeing a
Speaker 2: lot of previous I guess consumers of chips, like the
Speaker 2: crypto miners start to pivot into the space, and I'd
Speaker 2: be curious to see what they're doing in it as well,
Speaker 2: and how much of that is just you know, desperation
Speaker 2: versus versus a real business opportunity.
Speaker 3: In the video game market.
Speaker 2: Yeah, oh totally, I forgot about.
Speaker 1: Which was like the other thing. It's like, for years
Speaker 1: I thought of Nvidia is the video game company. Yeah,
Speaker 1: because they had their logo on xboxes.
Speaker 2: And how realistic is that pivot? What proportion of those
Speaker 2: types of chips can be used for AI?
Speaker 1: Now, well, I'm very excited. We do have I believe
Speaker 1: the perfect guest. We are going to be speaking with
Speaker 1: Brandon McBee. He is the chief strategy officer and co
Speaker 1: founder of core Weave, which is a specialized cloud services
Speaker 1: provider that's basically providing this sort of like high volume
Speaker 1: compute to AI type companies. They recently raised over four
Speaker 1: hundred million dollars. Have been in this space for a
Speaker 1: little while. So Brandon, thank you so much for coming
Speaker 1: on odd lots.
Speaker 4: Thanks for the opportunity. Guys, really excited to chat with
Speaker 4: you all today.
Speaker 1: So let's just let me sorry if Tracy and I, like,
Speaker 1: I don't know why they would do this, but if
Speaker 1: like some VC was like, you know, we want you
Speaker 1: to do on launch GPT. We want you to like
Speaker 1: do a pore base large language model off of all
Speaker 1: the work you've done. We want you to compete with
Speaker 1: open AI. And they gave us like I don't know
Speaker 1: some like, you know, one hundred million dollar rays, they said,
Speaker 1: go start, do your startup? Could I call in video
Speaker 1: and buy chips? Would I be able to get in
Speaker 1: the door there?
Speaker 4: Gosh? I mean you're I think you and everyone else
Speaker 4: is asking that question, and you're going to have a
Speaker 4: huge problem doing that. Right now, it's mostly just around
Speaker 4: how much in demand this infrastructure became, right I mean,
Speaker 4: you could argue it's one of the most critical pieces
Speaker 4: of information technology resources on the planet right now and
Speaker 4: suddenly everyone needs it, and you know, I like to
Speaker 4: contextualize it in that, you know, the piece of software
Speaker 4: adoption for AIS like one of the fastest adoption curves
Speaker 4: we've ever seen, right Like, you're you're hitting these milestones
Speaker 4: faster than any other software platform previously, and now all
Speaker 4: of a sudden, you're asking infrastructure build to keep up
Speaker 4: with that, right a space that traditionally takes more time,
Speaker 4: and it's created this massive supply demand and balance just
Speaker 4: on in place infrastructure today and not only infratructure is
Speaker 4: available to purchase, and it's an issue that is going
Speaker 4: to be ongoing for a bit as well, we think.
Speaker 2: So can I ask the basic question, which is core weave.
Speaker 2: What do you do exactly? Joe mentioned the capital raise,
Speaker 2: which I think has you valued at something like two
Speaker 2: billion dollars, So congrats, but what exactly are you doing here?
Speaker 4: Yeah? Thank you. So Corey is a specialized cloud service
Speaker 4: provider that is focused on highly parallelizable workloads. So we
Speaker 4: build and operate the world's most performant GPU infrastructure at
Speaker 4: scale and predominantly serve three sectors. That's the artificial intelligence sector,
Speaker 4: the media and entertainment sector, and the computational chemistry sector.
Speaker 4: So we build specialize in building this infrastructure at supercompute scale.
Speaker 4: It's like quite literally, you know, it's sixteen thousand GPU
Speaker 4: fabric and we can get into all the details and
Speaker 4: how complex that is. But we build that so that
Speaker 4: entities can come in and train these next generation foundation
Speaker 4: machine learning models on. And you know, we found ourselves
Speaker 4: in a spot where we can do that better than
Speaker 4: literally anyone else in the market and do it on
Speaker 4: a timeline that's faster or I think the only entity
Speaker 4: with H one hundred available to clients at scale globally today.
Speaker 2: So you have an actual basement full of H one
Speaker 2: hundred chips. Well, can you talk to us. You know,
Speaker 2: when you say infrastructure, we help clients build out the infrastructure,
Speaker 2: help us conceptualize this. What does what does the infrastructure
Speaker 2: for this type of AI actually look like? And how
Speaker 2: does it differ to infrastructure for other types of large
Speaker 2: scale technology projects.
Speaker 4: Yeah, totally, so, you know, I I think during the
Speaker 4: last in video quarterly earnings called Jensen put this a
Speaker 4: really great way in the Q and A section, he
Speaker 4: said that we are at the first year of a
Speaker 4: decade long modernization of the data center, or like making
Speaker 4: the data center intelligent. Right, you can kind of you
Speaker 4: could suggest that the last generation or the twenty tens
Speaker 4: data center was comprised of CPU, compute, storage and these
Speaker 4: things that didn't really work together that intelligently. And the
Speaker 4: way that in Nvidia has positioned itself is to make
Speaker 4: it a smart data center that's like smart routing of
Speaker 4: data packets of different pieces of infrastructure in there. That's
Speaker 4: all focused on how do you expand the throughput in
Speaker 4: communicability of and between pieces of infrastructure. Right, It's just
Speaker 4: an amazingly different approach to data center deployments. And so
Speaker 4: the way that we're building it and we're working with
Speaker 4: Nvidia infrastructure. We design everything to a DGX reference back
Speaker 4: in dgx's in videos like how do you draw the
Speaker 4: most performance out of Nvidia infrastructure is possible with all
Speaker 4: the anciliary components associated with it. So all this stuff
Speaker 4: is going into what's qualified as a Tier three or
Speaker 4: a Tier four data center. We collocate with within these things,
Speaker 4: so we're not quite building in a basement, even though
Speaker 4: like in our past history we certainly you know, had
Speaker 4: time doing that, but this is within you know, just
Speaker 4: amazing collocation sites that are operated by our partners such
Speaker 4: as switch right. So a Tier three a Tier four
Speaker 4: site is something that's qualified based on its ability to
Speaker 4: serve workloads with an extremely high uptime. So we're talking
Speaker 4: like ninety nine point nine nine percent uptime rate, and
Speaker 4: that's guaranteed by its power redundancy, it's Internet redundancy, and
Speaker 4: its security and then ultimately like it's connectivity to the
Speaker 4: Internet backbone. Right, So as it's like, as a first step,
Speaker 4: you're housed within these data centers that are just critical
Speaker 4: parts of the Internet infrastructure, and then from there you
Speaker 4: start building out the servers within there. And I can
Speaker 4: go into that detail.
Speaker 1: So you mentioned actually I want to just get sort
Speaker 1: of defined some terms. Can you just real quickly before
Speaker 1: we move on Tier three tier four?
Speaker 3: What do you mean by this?
Speaker 4: Yeah? So tier three, tier four. This all goes back
Speaker 4: to like the quality of the data center that you're
Speaker 4: in it. It's all about the reliability and up time
Speaker 4: that you should be able to achieve out of that
Speaker 4: data center. It's another way to qualify the services around it.
Speaker 4: It's like power. You get redundant power, right like multiple
Speaker 4: power services in case one goes offline, there's another one
Speaker 4: you get, you know, redundant cooling, you get redundant Internet connectivity.
Speaker 4: It's all these services that like have extra fiil safs
Speaker 4: that allow for you to operate at the highest up
Speaker 4: time and security level possible.
Speaker 1: Is higher tier better? Like tier three four? Is that
Speaker 1: better than Tier one and tier two?
Speaker 4: That's correct?
Speaker 1: Okay, so quick follow up question. Then you know we're
Speaker 1: interested in, like, okay, where the rubber hits the road.
Speaker 1: The scarcity is here. Let's say Tracy miraculously opens her
Speaker 1: basement and there really is like you know, all these
Speaker 1: palettes of these video chips, there is there capacity at
Speaker 1: the data centers right now, She's like, you know, what
Speaker 1: we want to co locate with you. You guys have great power,
Speaker 1: pretty well connected to the internet. You have like good
Speaker 1: security guards. So there's operated twenty four to seven. We
Speaker 1: want to set something up, like is there space there?
Speaker 4: Yeah, it's a fantastic question. It's a it's an issue
Speaker 4: that didn't really pop up until really in the last
Speaker 4: eight weeks or so.
Speaker 3: Oh, it's really happening that fast.
Speaker 4: It's happening that fast, Joe.
Speaker 3: And it's okay, So.
Speaker 2: That we said the two week lead time on in
Speaker 2: video was very important, Joe.
Speaker 3: Yeah, you're right, you're right. Is wow? Wait what happened?
Speaker 4: Wait?
Speaker 1: What happened sixteen described? Sixteen weeks ago?
Speaker 3: Verus eight weeks ago?
Speaker 4: Sure, it even last year? Right, So this is a space,
Speaker 4: the data centers space, collocation space that's been fairly chronically
Speaker 4: underinvested in because the Hyperscale has just built out their
Speaker 4: own data centers, right instead. But what's happened is the
Speaker 4: infrastructure changed. The type of compute that we're putting in
Speaker 4: these data centers, it's different than the last generation, right,
Speaker 4: so we're predominantly focused on GPU compute instead of CPU
Speaker 4: compute and GPU compute. It's about four times more power
Speaker 4: dance than CPU compute, and that throws the data center
Speaker 4: planning into chaos, right because ultimately, let's say you have
Speaker 4: a ten thousand square foot room in the data center, right,
Speaker 4: and you have a certain amount of power it's called
Speaker 4: one hundred units of power that go into that ten
Speaker 4: thousand square feet. Well, because I'm four times more power DNS,
Speaker 4: it means that now I take those hundred units of power,
Speaker 4: but I only require about twenty five percent of that
Speaker 4: data center footprint or in other words, twenty five hundred
Speaker 4: square feet within that ten thousand square foot footprint. So
Speaker 4: that then leads to like, not only is the space
Speaker 4: in the data center being used inefficiently now because you
Speaker 4: theoretically have to run more power into the data center
Speaker 4: to use that full ten thousand square feet due to
Speaker 4: the Poara density delta, but now you have cooling issues,
Speaker 4: right because you designed that footprint to be able to
Speaker 4: cool ten thousand square feet spread out across that entire area.
Speaker 4: But now you're dropping storry.
Speaker 1: Sorry, I just want to back up because this is
Speaker 1: extremely interesting, so I don't want to I just want
Speaker 1: to get this detail right.
Speaker 3: Just sorry, just to and then move on.
Speaker 1: But the let's given an x amount of power at
Speaker 1: one hundred units of power. What you're saying is that
Speaker 1: with this next generation of compute, it now only gets
Speaker 1: that's now.
Speaker 3: Only sufficient for a quarter of the data center.
Speaker 1: In other words, that to power that whole that space,
Speaker 1: and that to then power the whole space, you really
Speaker 1: would need like four x the power.
Speaker 4: That's accurate. Okay. The complication really arises out of the
Speaker 4: cooling that that's required from that, right, So if you
Speaker 4: imagine you can cool a ten thousand square foot space
Speaker 4: and you designed for that, that's one thing. But now
Speaker 4: if you have to cool in a much more dense area,
Speaker 4: that's a different type of cooling requirement. And so that's
Speaker 4: led to this issue where there's only a certain subset
Speaker 4: of Tier three and four data centers across the US
Speaker 4: that can are currently designed for or can quickly be
Speaker 4: designed and changed to be able to accommodate this new
Speaker 4: power density issue. So now not only like if you
Speaker 4: had all those eighth one hundreds in your basement, you
Speaker 4: might not have a place to plug them into. And
Speaker 4: that's become a pretty big problem for the industry very quickly,
Speaker 4: and truly has only arisen in the last eight weeks
Speaker 4: or so, and it's going to persist for a few quarters.
Speaker 2: So you were describing the difference between CPU and GPU.
Speaker 2: How do you actually connect these newer types or these
Speaker 2: different types of chips together, because I imagine, you know,
Speaker 2: old data centers you just have a bunch of like
Speaker 2: Ethernet cables or something like that. But for this type
Speaker 2: of processing power, do you need something different?
Speaker 4: That's exactly correct, Chracy. So what we so the legacy
Speaker 4: the generalized compute data centers are really what the hyperscalers
Speaker 4: look like. You know, Amazon, Google, Microsoft, Oracle. They predominantly
Speaker 4: use something that's called Ethernet to connect all the service together.
Speaker 4: And the reason you use that was, you know, you
Speaker 4: don't really need to have high data throughput to connect
Speaker 4: all these servers together, right, They just need to be
Speaker 4: able to send some messages back and forth. They talk
Speaker 4: to each other about what they're working on, but they're not,
Speaker 4: you know, necessarily doing highly collaborative tasks that require moving
Speaker 4: lots of data in between each other. That's changed. So
Speaker 4: so today what people are focused on and need to
Speaker 4: build are these effectively supercomputers. Right, and so we refer
Speaker 4: to the connectivity between them, the network between them as
Speaker 4: a fabric, right, it's called a network fabric. So if
Speaker 4: we're build holding something to help train like the next
Speaker 4: generation GPT model, typically clients are coming to us saying, hey,
Speaker 4: I need a sixteen thousand GPU fabric of H one hundred.
Speaker 4: So that's there's about eight GPUs that go into each server,
Speaker 4: and then you have to run this connectivity between each
Speaker 4: one of those servers. But it's now done in a
Speaker 4: different way to your point, So we're using a in
Speaker 4: Nvidio technology called InfiniBand which has the highest data throughput
Speaker 4: to connect each of these devices together. And you know,
Speaker 4: taking this sixteen thousand GPU cluster as an example, there's
Speaker 4: two crazy numbers in here. One is that there are
Speaker 4: forty eight thousand discrete connections that need to be made,
Speaker 4: right like plugging one thing in from one computer to
Speaker 4: another computer. But there's lots of switches and routers that
Speaker 4: are between there. But you need to that forty eight
Speaker 4: thousand times, and it takes over five hundred miles of
Speaker 4: fiber optic cabling to do that successfully across the sixteen
Speaker 4: thousand GPU cluster. And now again you're doing that within
Speaker 4: a small space with a ton of power density, with
Speaker 4: a ton of cooling, and it's just a completely different
Speaker 4: way to build this infrastructure. It's just because the requirements
Speaker 4: have changed, right, Like we've moved into this, like this
Speaker 4: area where we are designing next generation AI models and
Speaker 4: it requires a completely different type of compute, and it's
Speaker 4: just it's caught the whole sector by surprise so much
Speaker 4: so that you know, it's really challenging to go procure
Speaker 4: it at the hyperscalers today because they didn't specialize in
Speaker 4: building it. And that's you know where where core we've
Speaker 4: comes in is we only focus on building this type
Speaker 4: of compute for clients. It's our specialty. We hire all
Speaker 4: of our engineering around it, all of our research goes
Speaker 4: into it, and it's you know, it's been a fantastic
Speaker 4: spot to be but our goal at the end of
Speaker 4: the day is just to be able to get this
Speaker 4: infrastructure into the hands of end consumers so that they
Speaker 4: can build the amazing AI companies that have ones looking
Speaker 4: forward to using and incorporating to enterprises and software companies.
Speaker 2: You know, you mentioned these special or purpose built connections
Speaker 2: that Nvidia is making, and this kind of leads nicely
Speaker 2: into my next question, which is what exactly is your
Speaker 2: relationship with Nvidia and in order to provide this type
Speaker 2: of service, you know, vast amounts of processing power that
Speaker 2: is well suited to a particular type of technology in
Speaker 2: this case AI, do you have to have a really
Speaker 2: good relationship with Nvidia to make that work? Like do
Speaker 2: you have to have special access to H one, hundreds
Speaker 2: and other chips.
Speaker 4: It's a great question, and I'll try to offer or
Speaker 4: from Nvidia's perspective, and it goes a little bit back
Speaker 4: to the answer I just provided as well in that
Speaker 4: I would think from in Nvidia's seat, what's most important
Speaker 4: is empowering end users of their compute to be able
Speaker 4: to access their compute and the most performant variant possible
Speaker 4: at scale, and to be able to access it quickly, right,
Speaker 4: Like a new generation comes out, they want to be
Speaker 4: able to get their hands on it, right. And we've
Speaker 4: built Core. We've around hitting every single one of those checkboxes. Right.
Speaker 4: We build it at DGX reference back, we build it
Speaker 4: at scale, and we bring it online on a timeline
Speaker 4: that's you know, within months of a next generation chipset launch,
Speaker 4: as opposed to you know, the more traditional legacy hyperscalers
Speaker 4: that take quarters at a times, so US being in
Speaker 4: a position to do that has has enabled us fantastic
Speaker 4: access within Nvidia, and we have a history of consistently
Speaker 4: executing on exactly what we've what we say we'll do right,
Speaker 4: we under promise and over deliver as a business, and
Speaker 4: I think that's just put us in this place where
Speaker 4: Nvidia has the confidence in allocating infrastructure to us because
Speaker 4: they know it's going to come online, they know it's
Speaker 4: going to get to consumers faster than anyone else in
Speaker 4: the market, and they know it's going to be delivered
Speaker 4: in its most performance configuration that exists.
Speaker 1: You know, I was thinking as I listened to some
Speaker 1: of these answers, I keep having like these like imagines,
Speaker 1: like you know, there's probably like some random industrial company
Speaker 1: that's like traded like you know on the like S
Speaker 1: and P four hundred that makes some cooling fluid whose
Speaker 1: like sales are going to be up ten x. So
Speaker 1: I'm like googling while we're talking, like what is a
Speaker 1: company that makes kool aid fluid? Or like who is
Speaker 1: some company that's like really good at making these like
Speaker 1: infinite bands, Because it just likes.
Speaker 3: Right, yeah, like what are the anyway?
Speaker 4: Right?
Speaker 1: But like right, like you know there's going to be
Speaker 1: some yeah urchiery plate that are like thirty x up.
Speaker 1: But you know, I want to get a sense from
Speaker 1: you of so it's really changed a lot, and I
Speaker 1: kind of you know, in the last several months.
Speaker 3: Could we see it from in video results?
Speaker 1: What you're describing, like how big is the market getting
Speaker 1: and the way I think you know, I know, like
Speaker 1: with AI, there's training and they sort of build the
Speaker 1: model and then there's inference, and the inference is how
Speaker 1: they spit out the results. Can you talk a little
Speaker 1: bit about what you're seeing in terms of the growth
Speaker 1: of both of those aspects of AI, which is bigger
Speaker 1: and which is growing faster? And how do they compare
Speaker 1: to like the size of the installed compute base that
Speaker 1: already exists.
Speaker 4: Oh. Absolutely, So this is one of my favorite topics
Speaker 4: because it's just mind blowing the scale that's going to
Speaker 4: be needed to support AI and scale this infrastructure. So okay,
Speaker 4: so today most of the funding that's going into the
Speaker 4: AI space is too for funding to train next generation
Speaker 4: foundation models. Right, So when a company's raising a bunch
Speaker 4: of money at the end of the day, most of
Speaker 4: that money is going into cloud compute to go train
Speaker 4: this next generation found model to build that intellectual property.
Speaker 4: So they have this model and they can go bring
Speaker 4: it into the inference market. And what I would say is,
Speaker 4: we're having a supply demand issue like a chip access
Speaker 4: crunch in the training phase, where in reality, the scale
Speaker 4: of the inference market is where we're all the demand
Speaker 4: truly is going to sit. So what I'd offer to
Speaker 4: help contextualize that is, let's take you know, there's some
Speaker 4: well known models in the market today. Let's let's say
Speaker 4: there's a preach an in market trained model and it
Speaker 4: took about let's say ten thousand. A one hundred or
Speaker 4: so to train A one hundred is the last generation
Speaker 4: in chip, but you know it still applies in terms
Speaker 4: of relative scale here. So that company that used ten
Speaker 4: train their model, our understanding is they're going to need
Speaker 4: about a million GPUs within one to two years of
Speaker 4: launch to support the entire inference demand.
Speaker 1: So ten you could train the model on ten thousand
Speaker 1: of these chips, ten thousand of these sisters, whatever they're
Speaker 1: and then if they're actually going to be in the
Speaker 1: market and sell something or provide some service to make
Speaker 1: it worthwhile.
Speaker 3: They're going to need a million.
Speaker 4: A million, and I think that's just within first two
Speaker 4: years of launch. Show like we're we're talking about something
Speaker 4: that's going to continue growing afterwards. And so what does
Speaker 4: a million GPUs mean? Obviously? Right, so you know a
Speaker 4: couple I think it was like into last year, all
Speaker 4: the hyperscalers combined, right, Amazon, Google, Microsoft, Oracle. You can
Speaker 4: throw a COORWY from there, it was about, you know,
Speaker 4: five hundred thousand GPUs globally right available across those platforms.
Speaker 4: I'd see it end of this year, it'll be closer
Speaker 4: to a million or so. But that's suggesting then that
Speaker 4: one AI company with one model could consume the entire
Speaker 4: global footprint of GPUs. And and now you start to think, wait,
Speaker 4: aren't there a bunch of other companies training these models
Speaker 4: in market right now? And I leo'ld say, yes, there are.
Speaker 4: So it can imply that there are in the short term,
Speaker 4: the demand of several million GPUs just to support the
Speaker 4: inference market. And there's just there's just nowhere near enough
Speaker 4: globally of this infrastructure, and it's it's going to be
Speaker 4: a big challenge for the market as we exit this
Speaker 4: training phase and move into the productization or really just
Speaker 4: the commercialization of these models, like how do you generate
Speaker 4: revenue off them? And it's it's something that I don't
Speaker 4: think many people truly understand just the amount of scale
Speaker 4: and construction that needs to take place. And now you
Speaker 4: put that in the same framework of the data centers
Speaker 4: that we were talking about, right, So there's this lack
Speaker 4: of data center space, there's lack of chipset supply, like
Speaker 4: it's it's going to be an issue for years that
Speaker 4: we see.
Speaker 2: So when it comes to scale, you know, you keep
Speaker 2: mentioning the hyper scalers, which is a great term, but
Speaker 2: people like Amazon, Google, I guess, Microsoft, IBM, et cetera.
Speaker 2: How quickly or what is your impression of how quickly
Speaker 2: they are able to ramp up in this space? Like
Speaker 2: how fast could they react to some of the trends
Speaker 2: that you've been outlining.
Speaker 4: Yeah, so I can offer what I'm seeing today. You know,
Speaker 4: the h one hundreds started to be distributed globally to
Speaker 4: all of us, right, like all the entities that have
Speaker 4: these you know, kind of upper tier relationships with Nvidia
Speaker 4: back in March, right, so we started getting them the
Speaker 4: s infrastructure online in April, really scaling in May, and
Speaker 4: you know, we have builds going on at ten data
Speaker 4: centers across the US right now, and we're delivering it
Speaker 4: to clients. The guidance that we're seeing from the hyperscalers
Speaker 4: is that they're not going to begin delivering scale access
Speaker 4: to the H one hundred chipset until late Q three,
Speaker 4: maybe mid Q four, and some of them are even
Speaker 4: beginning to guide into Q one. And it's all driven
Speaker 4: by the fact that this is just a different type
Speaker 4: of compute that they're building relative to last generation. Right
Speaker 4: You're no longer just running Ethernet to your point, between
Speaker 4: all these devices, you're not just plugging in CPU blades.
Speaker 4: You're having to deal with like totally different data center
Speaker 4: power density and cooling requirements. You're having to build supercomputers instead,
Speaker 4: with five hundred miles of fiber and all these connections.
Speaker 4: It's just it's a completely different way to build the cloud,
Speaker 4: and it's taking them some time to catch up because
Speaker 4: you have to retrain entire organizations to do this. So
Speaker 4: you know, as of now, i'd say the direct answer
Speaker 4: is three quarters after a chip set launch, But it's
Speaker 4: seeming it might take longer, And I think that's all
Speaker 4: going to contribute to this just kind of slower ability
Speaker 4: to scale infrastructure than what's being dictated by the adoption
Speaker 4: rate of AI software, And it's going to lead to
Speaker 4: this supply demand imbalance that will just last for a while.
Speaker 2: You know, you keep mentioning or we both keep mentioning
Speaker 2: the H one hundred for obvious reasons, But do you
Speaker 2: look at other chips or what would happen to you know,
Speaker 2: your own business if, for instance, a new chip was
Speaker 2: developed that could do the same thing or better than
Speaker 2: an Nvidia H one hundred. Like, for instance, I hear
Speaker 2: a lot of excitement about some of the stuff that
Speaker 2: AMD is developing. And I'm not a chips expert, except
Speaker 2: maybe when it comes to freedo's or layers. But like,
Speaker 2: how how big a difference would that make to you
Speaker 2: if we suddenly got a different chip manufacturer gain prominence
Speaker 2: in AI.
Speaker 4: Sure? So I'd offer kind of two broad responses. One, typically,
Speaker 4: when you train a model you're going to use, you're
Speaker 4: going to use the same chips for inference on that
Speaker 4: model as well. Right, So two PT four, For example,
Speaker 4: I was trained on a one hundreds, they're predominantly going
Speaker 4: to use a one hundreds going for or you might
Speaker 4: fit in some kind of newer generation hyper efficient chips
Speaker 4: into there, but it's not like you need a quote
Speaker 4: a GP with more vram on it, right, Like you're
Speaker 4: in a need your forty gig or your your eighty
Speaker 4: grid gig ram chip because that's the size of the
Speaker 4: model that that you trained, Right, You're not gonna need
Speaker 4: like next multiple generations. You're not going to like really
Speaker 4: be able to adopt them to change the efficiency of
Speaker 4: serving that model. So what we view is that a
Speaker 4: chip's lifespan is like this first two to three years
Speaker 4: is spent training models, and then it's next four to
Speaker 4: five years is spent doing inference for those models that
Speaker 4: it trained. And then within there as well, you do
Speaker 4: this thing called fine tuning, which is updating the model
Speaker 4: with new information, right, Like how do you keep a
Speaker 4: model like up to date with what's happened on a
Speaker 4: Twitter or what's happened on in the media. Right, you
Speaker 4: have to keep retraining it, right, and you'll use those
Speaker 4: same chips to do that. But so your question on
Speaker 4: other chip sets, and this is something that we have
Speaker 4: a particularly interesting view into because we have like you know,
Speaker 4: call it six hundred and fifty AI clients right, and
Speaker 4: we're having conversations with them daily to ensure that we're
Speaker 4: meeting their scaling demands. So it gives us a look
Speaker 4: into six to twelve months into the future what type
Speaker 4: of infrastructure they expect to need, and it's it's overwhelmingly
Speaker 4: people still want access to Nvidia chips and the reason
Speaker 4: for this is something that dates back, I think it's
Speaker 4: nearly fifteen years when Nvidia and Jensen made the decision
Speaker 4: to open source Kuda and to make this software set
Speaker 4: accessible to the machine learning community. And you know today
Speaker 4: if you go to GitHub and you search a machine
Speaker 4: learning project, they's all reference Kuda drivers. And he's established
Speaker 4: this utter dominance of ecosystem around his compute within the
Speaker 4: mL space really similar to like the x eighty six
Speaker 4: instruction set for CP you versus ARM, right, like X
Speaker 4: eighty six is is used prodominantly. ARM has been trying
Speaker 4: to find its way into the space for a while
Speaker 4: now and it's just really struggled because all the engineers
Speaker 4: and developers are used to x eighty six. Similar to
Speaker 4: how all the engineers and developers in the AI space
Speaker 4: are used to using KUDA, so it's something that like
Speaker 4: obviously AMD is highly incentivized to find a way into
Speaker 4: the sector, but they just don't have the ecosystem and
Speaker 4: it's a huge moat to deal with. Then, you know,
Speaker 4: kudos to Nvidia for establishing themselves and having the patients
Speaker 4: to stick with it and to continue to support that
Speaker 4: community over the last fifteen years, and it's it's really
Speaker 4: paying off for them in spades today. You know, if
Speaker 4: the demand comes for that infrastructure at some point, it's
Speaker 4: you know, we can run other pieces of infrastructure within
Speaker 4: our data center. But I also find that Nvidia has
Speaker 4: such an advantage on the competition with not only its GPUs,
Speaker 4: but all its components that support the gp USE, like
Speaker 4: the infinite band fabric, that it's it's gonna be a
Speaker 4: really difficult company to displace from the market in terms
Speaker 4: of the best standard for AI infrastructure.
Speaker 1: Can I ask a question, and I'm gonna I want
Speaker 1: to ask this politely because it's not intended to be
Speaker 1: accusatory or anything like that, So I don't want you
Speaker 1: to you know here it is like but like when
Speaker 1: you're like talking about like hyperscalers and you're like you know, Amazon, Google, Microsoft,
Speaker 1: and you know kind of core Weave, and it's like, okay,
Speaker 1: those are trillion dollar companies and you're a two billion
Speaker 1: dollar company. Like why like I don't still don't think
Speaker 1: I like wrap my head around, like and I know,
Speaker 1: like they're all like in they're all talking about.
Speaker 3: AI et cetera.
Speaker 1: Like can you still just like explain to me a
Speaker 1: little bit, like why aren't they just gonna frankly like
Speaker 1: steamroll you or be able to let's put it this way,
Speaker 1: be able to Okay, maybe it'll take a few quarters
Speaker 1: to re evaluate things, but like you know, eventually this
Speaker 1: just becomes this sort of de facto offering from these
Speaker 1: big is that have these huge cloud budgets that must
Speaker 1: be orders of magnitude large larger than yours.
Speaker 4: Yeah. Yeah, I would really love to be able to
Speaker 4: have access to their their cost of capital, that's pretty sure.
Speaker 4: So the way, look, it's the way I had to talk
Speaker 4: about this is we don't have a silver bullet necessarily, right.
Speaker 4: I can't point to like a super secret piece of
Speaker 4: technology that we put inside of our servers or anything
Speaker 4: along those lines. But the way I like to broadly contextualize,
Speaker 4: it is referencing another sector, and it's that like Ford
Speaker 4: should be able to produce a model, why right, Like
Speaker 4: they have the budget, they have the people, they have
Speaker 4: the decades of expertise. But in order to ask them
Speaker 4: to produce a model, why you would have to ask
Speaker 4: them to foundationally change the way that they produce a vehicle,
Speaker 4: all the way from research to servicing and that entire mechanism.
Speaker 4: Like it's a giant organization. Now you have to go
Speaker 4: ask that huge organization and people to change the way
Speaker 4: that they go about producing things.
Speaker 1: And I get that, but just to push back a
Speaker 1: little bit, and I get And this is like a
Speaker 1: theme that we that comes up in various flavors on
Speaker 1: odd lots a lot, which is that like companies have
Speaker 1: internal it's really hard to replicate sort of like tacit
Speaker 1: knowledge within a corporation. And we see that with companies
Speaker 1: that make semiconductor equipment. We see that with companies that
Speaker 1: make airplanes. We see that with real estate developers that
Speaker 1: know how to turn an office building into a condo.
Speaker 1: And so I think this is like a deep point,
Speaker 1: but you know they are offering AI stuff like I
Speaker 1: can look at Google right now, like there's Cloud AI
Speaker 1: like and there's Asia AI, and they all have their announcements.
Speaker 1: So I'm still trying to understand, like what is it
Speaker 1: that you're offering that all the hyper scalers they all have,
Speaker 1: they all say they have AI offering, So what is
Speaker 1: the difference between sort of like what you have and
Speaker 1: what they say is like their you know, AI compute platforms.
Speaker 4: Absolutely, and this will really depend on how much technical
Speaker 4: detail you like me to get into. But broadly, through
Speaker 4: structure differentiation like literally using different components to build our cloud,
Speaker 4: and through software differentiation we use different pieces of software
Speaker 4: to operate and optimize our cloud, we're able to deliver
Speaker 4: a product that's about forty to sixty percent more efficient
Speaker 4: on a workload adjusted basis than what you find across
Speaker 4: any of the hyperscalers. So, in other words, if you
Speaker 4: were to take the same workload or like go do
Speaker 4: the same process at a hyperscaler on the exact same
Speaker 4: GPU compute versus core weave, we're going to be forty
Speaker 4: to sixty percent more efficient at doing that because of
Speaker 4: the way that we've configured everything relative to the hyperscalers
Speaker 4: and it comes back to this analogy between like why
Speaker 4: Forward can't produce the model? Why again like they can.
Speaker 4: These are trillion dollar companies we're talking about. To your point,
Speaker 4: they have the budget, they have the personnel, and they
Speaker 4: certainly have the motivation to do so. But you know,
Speaker 4: it's not just one singular thing they have to change.
Speaker 4: It's a completely different way to building their their business
Speaker 4: that they would have to orchestrate. And it's what's the analogies.
Speaker 4: However many miles it takes to turn an aircraft carrier, right,
Speaker 4: like it's it's going to take them a while to
Speaker 4: do that. And I think if they do get there
Speaker 4: at some point, which you know, I don't disagree with you,
Speaker 4: they're certainly motivated to, it's it's going to have taken
Speaker 4: them some time, literally years to get there, and they're
Speaker 4: going to look really similar to us. And meanwhile, I've
Speaker 4: dominated mortgage share and I've really established my product and
Speaker 4: market and I continue when I'll continue to differentiate myself
Speaker 4: on the software asign business as well.
Speaker 2: Since we're on the topic of adaptation, can I ask about,
Speaker 2: you know, your own evolution as a company, because I
Speaker 2: think I've read that you started out in ethereum mining,
Speaker 2: and at one point I'm pretty sure crypto mining was
Speaker 2: a substantial, if not the biggest, portion of your business.
Speaker 2: But you have clearly adapted or pivoted into this AI space.
Speaker 2: So what has that been like and can you maybe
Speaker 2: describe some of the trends that you've seen over your history.
Speaker 4: Yes, absolutely, and you're right. We did start within the
Speaker 4: cryptocurrency space back in twenty seventeen or so, and that
Speaker 4: was spawned out of just frankly curiosity from a group
Speaker 4: of former commodity traders. So myself, my two co founders,
Speaker 4: we ran hedge funds, we ran family offices, so we
Speaker 4: traded in these energy markets. So we were always attracted
Speaker 4: to supply demand mechanics. But what attracted us but then
Speaker 4: cryptocurrency was there's this arbitrage opportunity that was a permissionless
Speaker 4: revenue stream, right, Like I knew the cost of power,
Speaker 4: I knew what the hardware could generate in terms of
Speaker 4: revenue with using a power input. Thus it's effectively an
Speaker 4: arbitrage right, So we explored that we had some of
Speaker 4: the infrastructure operating literally in our basements. As you said,
Speaker 4: then that like quickly turned into scaling across warehouses, and
Speaker 4: at some point in twenty I think it was twenty eighteen,
Speaker 4: maybe late twenty eighteen, we were the largest ethereum miner
Speaker 4: in North America. We were operating over fifty thousand GPUs,
Speaker 4: we represented over one percent of the ethereum network. But
Speaker 4: during that whole time, we just kept coming back to
Speaker 4: the idea that there's no moat, there's no advantage that
Speaker 4: we could create for ourselves relative to our competitors, right, Like, sure,
Speaker 4: you could maybe focus on power price and just kind
Speaker 4: of chase the cheapest power, but that just felt like
Speaker 4: chasing to the bottom of the bucket, right. You know,
Speaker 4: I think an area we could have gone into is
Speaker 4: producing your own chips, right because if you produce the
Speaker 4: own chips and you run the mining equipment before anyone
Speaker 4: else has access to it, then you have an advantage
Speaker 4: for that period. But you know, we weren't going to
Speaker 4: go design and fabric own chips. So what we kept
Speaker 4: coming back to was this GPU compute, Man, what if
Speaker 4: we could do other things, right, like what if they
Speaker 4: were what if we could develop uncorrelated optionality into multiple
Speaker 4: high growth markets right in those markets or where we
Speaker 4: predominantly sit today with an artificial intelligence, media and entertainment,
Speaker 4: and computational chemistry. And the original thesis was, well, whenever
Speaker 4: our compute isn't being allocated into those sectors, we'll just
Speaker 4: have it mining cryptocurrency and we'll build out this fantastic
Speaker 4: company that has one hundred percent utilization rate across the
Speaker 4: infrastructure because it could switch immediately from being released from
Speaker 4: an AI workload into going back into the Ethereum network.
Speaker 4: And we did get a brief glimpse of being able
Speaker 4: to operate that way in twenty twenty one, as we
Speaker 4: had our cloud live and we had AI clients in place,
Speaker 4: but Ethereum mining effectively ended during the merge in Q
Speaker 4: three of twenty twenty two. But I'd say the other
Speaker 4: thing that we never appreciated was the utter complexity of
Speaker 4: running a CSP forgetting about the software side of the business,
Speaker 4: which in and of itself, you know, we spent about
Speaker 4: four years developing the software to build a modern cloud
Speaker 4: to do infrastructure orchestration and actually be a cloud service provider.
Speaker 4: The components themselves that the sector broadly used for crypto
Speaker 4: mining were these retail grade GPUs, right, the kind of
Speaker 4: things that you plug in your desktop to go play video.
Speaker 3: They were like selling them on stock X. Yes, yes
Speaker 3: it was.
Speaker 4: It was crazy during that period to get your hands
Speaker 4: on that infrastructure for crypto.
Speaker 1: Mining, and all the video gamers hated the crypto people,
Speaker 1: right because they're like, I want to like play this
Speaker 1: game and they would like line up what is it?
Speaker 1: Game stop and like the geek wire shop and all
Speaker 1: that or whatever it is, and they like couldn't get
Speaker 1: it because you got it, not you.
Speaker 3: But yeah, they're orbating.
Speaker 1: Access to the chips first and getting more value out
Speaker 1: of them so that you could bit them up.
Speaker 4: We were certainly part of the problem, and that's absolutely correct.
Speaker 4: But you know what we found ultimately is like those chips,
Speaker 4: that's not what you run enterprise grade workloads on. That's
Speaker 4: not what's supporting you know, the largest AI companies in
Speaker 4: the world, And starting in twenty nineteen, we stopped buying
Speaker 4: any of those chips and only focused on purchasing enterprise
Speaker 4: grade GPU chip sets that you know, Nvidia has a
Speaker 4: probably about twelve different SKUs that they offer, including a
Speaker 4: one hundred and h one hundred chips, and really oriented
Speaker 4: our business around it. So it's a. I don't expect
Speaker 4: to see much repurposing of this kind of older retail
Speaker 4: grade GPU equipment that was used for crypto mining, because
Speaker 4: in crypto mining, you want to buy the cheapest chip
Speaker 4: that can do the thing for it, right, that can
Speaker 4: participate in crypto mining. But there's a huge difference in
Speaker 4: price between a retail plugin into your computer so you
Speaker 4: can play video games chip and an enterprise grade you
Speaker 4: can run it twenty four to seven. There's not going
Speaker 4: to be downtime, You're going to have a low failure rate.
Speaker 4: Like there's a large technology difference and there's a large
Speaker 4: pricing difference between those and the crypto miners. You only
Speaker 4: needed the retail grade ship because you know, if it
Speaker 4: went down for two percent five percent of the time
Speaker 4: for a failure rate, that's not a big deal. But
Speaker 4: the tolerance, the uptime tolerance for these enterprise grade workloads
Speaker 4: is measured on the thousandths of a percent, and it's
Speaker 4: a different type of infrastructure, so we don't expect to
Speaker 4: see the components really being reused, if at all. And
Speaker 4: then the other variable, going back to the very beginning
Speaker 4: of our conversation are the data centers in which these
Speaker 4: are housed. So Joe, to your point earlier, you know,
Speaker 4: we sit within tier three tier four data centers, and
Speaker 4: that's the basically the broad industry standard for being able
Speaker 4: to serve these kind of workloads. The crypto miners sat
Speaker 4: within tier zero tier one data centers, and these things
Speaker 4: are like highly interruptible. They do like really interesting things
Speaker 4: like helping load balance the power markets in places like
Speaker 4: ir Cot right, Like they'll shut down when power prices
Speaker 4: go too high and it load balances the grid. But
Speaker 4: enterprise AI workloads don't have a tolerance for that. Their
Speaker 4: tolerance again is measured on the thousands of a percentage
Speaker 4: in terms of uptime. So not only does the infrastructure
Speaker 4: not work from cryptomning, but the data centers that they
Speaker 4: built within don't work either the way that they're currently configured. Now,
Speaker 4: they could potentially convert their sites into tier three and
Speaker 4: tier four data centers. I'll tell you that in and
Speaker 4: of itself, that is an extremely challenging task and it
Speaker 4: takes a lot of proprietary knowledge and industry expertise to
Speaker 4: do so. It's not just throwing a few fans in
Speaker 4: a room and a few air conditioning units. It's a
Speaker 4: it's it. Honestly, it feels like walking to a spaceship.
Speaker 3: Tracy, this is this is an episode.
Speaker 1: I don't know about you, Tracy, there's like six another
Speaker 1: like six follow on episodes. No, seriously, like the whole
Speaker 1: like data center market and the coolant and all, you know,
Speaker 1: the electricity, Like there's so many different rabbit holes you
Speaker 1: could go down, just like with the infrastructure you're.
Speaker 2: Talking about, for sure, And I think the estimates that
Speaker 2: I've seen on repurposing crypto GPUs, I think I've seen
Speaker 2: like five to fifteen percent, so to Brannon's point, but
Speaker 2: I'm sure, I'm sure there will be people out there
Speaker 2: who try.
Speaker 4: You got to try, right, because what if it works right,
Speaker 4: If you can make that work, that's amazing. But we're
Speaker 4: just you know, coming as an entity that was an
Speaker 4: extremely large operator of that infrastructure and has built, you know,
Speaker 4: one of the largest cloud service providers for AI workloads.
Speaker 4: I can tell you it's it's gonna be really really
Speaker 4: hard to do it because we've had exposure in both
Speaker 4: those places and at the end of the day, they're
Speaker 4: just very very different businesses, both from the type of
Speaker 4: nearing and developers that you employed to the infrastructure to
Speaker 4: the data centers that you sit within.
Speaker 1: So can I just go back you know, yeah, just
Speaker 1: sort of like big picture, and I guess it sort
Speaker 1: of goes back to like who gets access to what?
Speaker 3: Who gets access to chips?
Speaker 1: And I imagine that you know, not only do you
Speaker 1: need a lot of money to like build a relationship
Speaker 1: with like Nvidio, you also probably need like a you know,
Speaker 1: expectation you're going to be back the next year, back
Speaker 1: the next year, back the next year, and that you
Speaker 1: actually like have relationship and so forth. But I have
Speaker 1: to imagine like planning is really tough, and when like
Speaker 1: you know, you have this sort of like AI Machine
Speaker 1: language whatever like industry, and then something like chet GPT
Speaker 1: comes out and like suddenly everyone like, oh I need
Speaker 1: to like have AI access. Talk to us about like
Speaker 1: the sort of like challenge of just sort of like
Speaker 1: planning to build when it can move that fast, and
Speaker 1: like everyone is just sort of guessing how big this
Speaker 1: market is going to be in two to three years.
Speaker 4: Oh my gosh, it's it's it's been utterly insane right
Speaker 4: like the it you know, back to last year, you know,
Speaker 4: the supply chain and the ability to get your hands
Speaker 4: on components. You know, you would call your your OEM.
Speaker 4: The OEM is the original equipment manufacturer, Like those are
Speaker 4: the super micros, the gigabytes of the world, who actually
Speaker 4: you know, build the nodes, build the servers, and you're
Speaker 4: you're buying through them, and then they buy the GPUs
Speaker 4: from Nvidia and build all the components together. Right, So
Speaker 4: if you called them and said, hey, I need this
Speaker 4: many nodes to be delivered, they'll say, great, we'll start assembling.
Speaker 4: Takes us, you know, a week to two weeks to
Speaker 4: get the parts in assembling, and then it's another week
Speaker 4: for them to ship them to you, and then it
Speaker 4: takes us two to three weeks to plug them in
Speaker 4: and put them online get them going. Right now, that's
Speaker 4: completely changed, as you know, like all the supply chain
Speaker 4: has gotten thrown off so much so that you know,
Speaker 4: in Vidia is fully allocated, like they've fully sold out
Speaker 4: their infrastructure through the end of the year. Right, you
Speaker 4: can't call them, You can't call the OEM and just
Speaker 4: say you need more compute chips like that, that's not possible.
Speaker 4: So much so that you know, when clients are coming
Speaker 4: to us today and they're asking for like a four
Speaker 4: thousand GPU cluster to be built for them. We're telling
Speaker 4: them Q one, and increasingly it's moving towards Q two
Speaker 4: at this point because Q one is starting to get
Speaker 4: booked up right now, So it's something that a lot
Speaker 4: of time has been added to it. And then there's
Speaker 4: other supply chain variables within there as well. You know,
Speaker 4: we had a client earlier this year that we were
Speaker 4: in negotiations with them on the contract and you know,
Speaker 4: we really wanted to perform well on timing for it.
Speaker 4: So we knew because of our orientation within the supply
Speaker 4: chain that there were some critical components that needed to
Speaker 4: be ordered ahead of time so that it would reduce
Speaker 4: our time to bring in the infrastructure online. And at
Speaker 4: that point it was the power supply units and the
Speaker 4: fans for the nodes that the Oliams were putting together,
Speaker 4: and if we hadn't have done that, it would have
Speaker 4: been another i think eight weeks on top of the
Speaker 4: build process, just because not all the components would have
Speaker 4: been there at the same time. So you're navigating this,
Speaker 4: you know, within other kind of global supply chain disruptions
Speaker 4: and inflation and all these other things that are going
Speaker 4: on right now and it's just an insanely complex task
Speaker 4: that I think, you know, the generation of software developers
Speaker 4: and founders that we're working with today were used to
Speaker 4: being able to go to a cloud service provider and
Speaker 4: just getting whatever infrastructure they needed. Right. You go to
Speaker 4: your hyperscalers and say all right, any of this and
Speaker 4: it was just there and available. And that just doesn't
Speaker 4: exist today because the pace of demand growth that we've
Speaker 4: been on and just the lack of this infrastructure's availability,
Speaker 4: and it's just caught everyone by surprise. Again. You're you're
Speaker 4: asking infrastructure to keep pace with the fastest adoption of
Speaker 4: a new piece of software that's ever occurred.
Speaker 1: Brandon McBee, core Leave, Thank you so much. That was
Speaker 1: a great conversation. Like I said, I always sort of
Speaker 1: measure the quality of a conversation of like do I
Speaker 1: get seven.
Speaker 2: How many additional episode like that is.
Speaker 1: A pretty good proxy for a good conversation. Do you
Speaker 1: get like eight ideas for future episodes? We got a
Speaker 1: bunch there, So thank you so much for coming.
Speaker 3: On the podcast.
Speaker 4: Always happy to chat with you guys, and thank you
Speaker 4: for the invitation.
Speaker 3: Tracy.
Speaker 1: I want to find that company that makes the coolant
Speaker 1: for the data No, seriously, for the data centers. That
Speaker 1: allows them to pack more compute and more energy into
Speaker 1: this space, because it's like it feels like they're probably
Speaker 1: going to make a fortune of the.
Speaker 2: Next Joe, I think you just want to talk to
Speaker 2: an HVACT contractor that's like installing out.
Speaker 3: Can we talk to it?
Speaker 1: Just some random like I love the Maybe it was
Speaker 1: such a funny thought like these like really advanced data centers,
Speaker 1: like oh, do we have like a local air conditioning
Speaker 1: guy who can like.
Speaker 2: But I imagine actually that would have been a good
Speaker 2: question for Brandon, wouldn't it. Like the labor constraints in
Speaker 2: building and adapting those data centers. But there was so
Speaker 2: much in there. One of the things, one of the
Speaker 2: things that I was thinking about was the point about how, well, okay,
Speaker 2: if you train a model on one type of chip,
Speaker 2: you're going to keep using that type of chip. And
Speaker 2: I guess, I guess it's kind of obvious, but it
Speaker 2: does suggest that there's some stickiness there, Like if you
Speaker 2: start out using an Nvidia each one hundred, you're going
Speaker 2: to keep using them, and in fact, you're going to
Speaker 2: consume even more because the processing power required the compute
Speaker 2: required for the inference is higher than for the actual
Speaker 2: initial training.
Speaker 1: Which I knew that that was the case because Stacy
Speaker 1: said so as well, but I did not realize quite
Speaker 1: the scale of like how much more Like okay, like
Speaker 1: if you train a model and then we try to
Speaker 1: take it to market product tize it.
Speaker 3: As a business person.
Speaker 1: I'd say, if we try to productize, like how much
Speaker 1: more computing power or we would need for the inferant aspect?
Speaker 1: And meanwhile we have to keep training it all the
Speaker 1: time to keep it up with fresh data and.
Speaker 3: Stuff like that.
Speaker 2: Yeah, totally. And the other thing that I was thinking about,
Speaker 2: and again Stacy mentioned this in our discussion with him
Speaker 2: as well, but this idea of Nvidia building a kind
Speaker 2: of large ecosystem around the hardware. So you have the
Speaker 2: open source software Kudo, which we talked about a little bit,
Speaker 2: and then you have these sort of high touch partnerships
Speaker 2: with companies like core Weave where they're trying to make
Speaker 2: it as easy as possible for you to use their
Speaker 2: chips and set them up in a way that works
Speaker 2: for you. It feels like maybe it feels almost like
Speaker 2: what bitmin used to do.
Speaker 3: Do you remember that, uh, no.
Speaker 2: Maybe they're still doing it anyway, but it does feel
Speaker 2: like they're trying to build this like ecosystem mote around
Speaker 2: the chip technology.
Speaker 4: Yeah.
Speaker 3: No, absolutely true.
Speaker 1: And you know, I really do take that point that
Speaker 1: Brandon made about like every company has a sort of
Speaker 1: like knowledge that cannot be written down on a piece
Speaker 1: of paper. Yeah, which is a Dan Wong point that
Speaker 1: we've been talking about for years. And so it's like
Speaker 1: to your point, you know, like you have to like
Speaker 1: use different types of connectors and different types of power
Speaker 1: and all these stuff like the ease with which any
Speaker 1: sort of traditional cloud provider or data center provider can
Speaker 1: you know, sort of switch to it's like a you know,
Speaker 1: it's not trivial even with lots of no.
Speaker 2: But coming away, I'm coming away from that conversation thinking,
Speaker 2: like the big question here is how quickly can those
Speaker 2: other hyperscalers adapt and like how big a moat can
Speaker 2: Nvidia build around this business?
Speaker 1: And then I mean the other question I have is
Speaker 1: like what if none of these companies make any money
Speaker 1: building AI models, Like I still don't think like that's
Speaker 1: been proven and so you can have this like huge
Speaker 1: boom and like, hey, we got to build any by
Speaker 1: a model is what we're going to build, like you know,
Speaker 1: outlaws GPT for like data stuff and whatever. But it
Speaker 1: all is somewhat predicated on these companies being successful and
Speaker 1: making a lot of money. And if they're not, and
Speaker 1: if it turns out that like the monetization of AI
Speaker 1: products is trickier than expected, then that also raises this
Speaker 1: question about like how long.
Speaker 2: This Like I'm sorry, Joe, so you're saying that tech
Speaker 2: companies should make money? Is that it? Are you sure?
Speaker 3: That's right? That's it's real post zerp thinking of it?
Speaker 2: I know, all right? Shall we leave it there?
Speaker 3: Let's leave it there.
Speaker 2: This has been another episode of the All Thoughts podcast.
Speaker 2: I'm Tracy Alloway. You can follow me on Twitter at
Speaker 2: Tracy Alloway.
Speaker 1: And I'm Joe Wisenthal. You can follow me on Twitter
Speaker 1: at the Stalwart. Follow our guest Brannon McBee.
Speaker 3: He's at Brannon McBee.
Speaker 1: Follow our producers Carmen Rodriguez at Carmen Arman and dash
Speaker 1: Ol Bennett at dashbot. And check out all of the
Speaker 1: Bloomberg podcasts under the handle at podcasts, and for more
Speaker 1: Odd Lots content, go to Bloomberg dot com slash odd
Speaker 1: lots where we have transcripts, a blog, and a newsletter
Speaker 1: that comes out each Friday. And check out our Discord.
Speaker 1: We have an AI channel and a semiconductor channel in
Speaker 1: there so people talk about these topics twenty four to seven.
Speaker 3: Maybe they'll be.
Speaker 1: Talking about them in both of those rooms when this
Speaker 1: comes out. Discord dot gg, slash.
Speaker 2: Outline and if you enjoy all thoughts, if you appreciate
Speaker 2: conversations like the one we just had with Brandon McBee,
Speaker 2: then please leave us a positive review on your favorite
Speaker 2: podcast platform. Thanks for listening.