Speaker 1: Bloomberg Audio Studios, podcasts, radio news.
Speaker 2: It is a joy to be joined by Matt. Thank you,
Speaker 2: remain Matt. Welcome from Las Vegas, Reinvent upon You Chip,
Speaker 2: Ultra Clusters, Ultra Service Chip Development LLM, and we've got
Speaker 2: talk about developer tools. There is a lot to get through.
Speaker 2: Let's start with the ultra cluster. Can you just talk
Speaker 2: to us a little bit about this more than one
Speaker 2: hundred thousand chips that will be put in one single
Speaker 2: era for AI hardware. Why do it? What will it offer?
Speaker 1: Yeah, this is something that we're building together with our
Speaker 1: partner's anthropic and it's actually going to be several hundred
Speaker 1: thousand chips, and they're focused on it's our new Trainium
Speaker 1: two chips, and so we expect that this should deliver
Speaker 1: them five times more compute than they've used in their
Speaker 1: last model training, and so we're really excited about what
Speaker 1: they're able to accomplish, and they're expecting to build much
Speaker 1: bigger and much more capable AI models from that large
Speaker 1: compute clubs.
Speaker 2: Do we have a US location? Do we have an
Speaker 2: exact timeframe other than early twenty twenty five?
Speaker 1: No, No, no location to share with.
Speaker 2: Matt what then, of the technical challenges paint the picture
Speaker 2: for our audience how difficult this is in terms of cooling,
Speaker 2: in terms of energy use. What has been the things
Speaker 2: that you have to overcome with this?
Speaker 1: Yeah, Well, the first core innovation is that we built
Speaker 1: our own chip actually, and so we've built our own chip.
Speaker 1: It's called Trainium two, and we're quite excited about the
Speaker 1: performance that we get out of that. And that's combined
Speaker 1: all in these really large ultra clusters that combine sixty
Speaker 1: four tranium chips all to deliver eighty three petaflops from
Speaker 1: a single node. And so that is the first innovation
Speaker 1: is our Amazon designed custom silicon that gives us really
Speaker 1: unparalleled performance for generative AIA capabilities. And then we build
Speaker 1: these together with high performance networking that we also build
Speaker 1: in house, and then of course they'll be cooling and
Speaker 1: heating and power that we need to go to go
Speaker 1: build the data centers. But it really starts with that
Speaker 1: silicon down at the level, and we innovate on the
Speaker 1: entire stack of AI to make sure that we can
Speaker 1: control everything that goes into those clusters.
Speaker 2: Can you talk to us a little bit compare and
Speaker 2: contrast here this is about less dependency on Nvidio GPUs
Speaker 2: too many ways, doing an alternative for your clients. What
Speaker 2: sort of cost saving will clients get? What sort of
Speaker 2: energy and efficiency will they get?
Speaker 1: Yeah? Well, first of all, I think we like to
Speaker 1: think of it as a supplement to in video GPUs
Speaker 1: and Vidia has a fantastic product that team has done
Speaker 1: an outstanding job executing, and we think that the vast
Speaker 1: majority of workloads are going to continue to run in
Speaker 1: video processors for a long time. But customers want choice,
Speaker 1: and they want choice that can give them some lower
Speaker 1: cost options. And we think that for certain workloads, for
Speaker 1: many workloads, Trainium two can give customers thirty to forty
Speaker 1: percent cost performance benefits over today's GPU powered instances. And
Speaker 1: so we think that that's a huge win for customers,
Speaker 1: particularly as they're looking to lower the cost of Generator
Speaker 1: I workloads. But we'll be great partners with Nvidia and
Speaker 1: continue to lean in on building great technologies together with
Speaker 1: them for a long time.
Speaker 2: Train two, Trainium two available, TRAININGUM three in the works.
Speaker 2: And all of this is as Nvidia at the moment
Speaker 2: is estimated has about ninety five percent market shared. Do
Speaker 2: you agree with that sort of number they have and
Speaker 2: what sort of area do you think that that will
Speaker 2: come down to.
Speaker 1: Yeah, I mean I think it's probably higher than that.
Speaker 1: I think that the vast majority of workloads in generative
Speaker 1: AI today run on in Nvidia technology and they've absolutely
Speaker 1: been the leaders in that space. But we do think
Speaker 1: we hear from customers that they want choice, and just
Speaker 1: with our processors where we're type partners with Intel and
Speaker 1: with AMD, but we decided to go build a general
Speaker 1: purpose processor are called Graviton, and it's been hugely successful
Speaker 1: with our customers. But we also provide lots of Intel
Speaker 1: and lots of AMD processors in our cloud today and
Speaker 1: those businesses continue to grow. So I expect that our
Speaker 1: usage of Nvidia will continue to grow from our customers
Speaker 1: and that choice is really going to be powerful. And
Speaker 1: as you see this explosion of generative AI usage, I
Speaker 1: think there's going to be plenty of business for multiple
Speaker 1: different people to be successful.
Speaker 2: I keep partners Intel. How concerned have you been about
Speaker 2: the change at the top.
Speaker 1: Oh, it's okay. I wish pat the best. You know,
Speaker 1: he's I know Patent, he's been a good partner of ours,
Speaker 1: but we've been partners with Intel for a long time.
Speaker 1: It's been eighteen years since we launched our first Intel
Speaker 1: instance when AWS and EC two first launched, and will
Speaker 1: continue to be great partners with Intel, and they have
Speaker 1: a great technology team there, and we look forward to
Speaker 1: to continuing to roll out the latest technologies from Intel
Speaker 1: for our customers to be able to use, and.
Speaker 2: He've been continuing to roll out, as you say in
Speaker 2: video offerings last time you're on, we're talking about Blackwell.
Speaker 2: Of course, many have been frustrated, perhaps by some slowness
Speaker 2: to market there. When do you anticipate that Blackwell will
Speaker 2: be unfolding? And how difficult has it been to ensure
Speaker 2: the supply side.
Speaker 1: Is there.
Speaker 2: Well.
Speaker 1: They've obviously had some manufacturing things that they're going through,
Speaker 1: but we're very excited about it. I think the early
Speaker 1: returns and the early looks at Blackwell look fantastic. We
Speaker 1: expect almost a two and a half time's gain in
Speaker 1: the compute power that you get from Blackwell that we
Speaker 1: saw from H one hundreds, and so I think it'll
Speaker 1: be a really material jump for customers once we get
Speaker 1: those out, and you know, I think those will be
Speaker 1: early next year, and we're excited about putting them into
Speaker 1: customer's hands, and we'll get them out there as soon
Speaker 1: as they're available.
Speaker 2: Investors, though, they find it wild, this whole frenemy existence
Speaker 2: that's going on. Do you truly think that investors here think, oh,
Speaker 2: we want to see in video dependency as well as
Speaker 2: AWS having its own offerings. Is that something you think
Speaker 2: everyone can swallow? Or do you think ultimately there will
Speaker 2: be a broadening out other than just in video winning all.
Speaker 1: Look, I do think it's a partner if you think
Speaker 1: about AWS. We started from the very beginning thinking about
Speaker 1: this partnership mindset, and we built the entire business around
Speaker 1: AWS thinking about how AWS would have services and our
Speaker 1: partners would have services, and that there's plenty of space
Speaker 1: for all of us to really grow and build our businesses.
Speaker 1: And that is true for software providers, true for service providers,
Speaker 1: and it's true for technology providers. And so I think
Speaker 1: that we've proven time and time again over the last
Speaker 1: eighteen years that AWS can have products and our partners
Speaker 1: can have products, and then as we make them all available,
Speaker 1: that the whole, the whole pie gets bigger. And so
Speaker 1: I think there's plenty of opportunity for both and so
Speaker 1: it really isn't. I think it makes for a fun
Speaker 1: narrative that it's either or. But we're great partners with
Speaker 1: n Video. We will continue to be, And this is
Speaker 1: all about making the pie get bigger.
Speaker 2: We love fun narratives. US journalists talk about that pie.
Speaker 2: When it's to do with large language models. You are
Speaker 2: unveiling Nova, you're really saying that the Nova offering compares
Speaker 2: really well to other offerings out there when it comes
Speaker 2: to multimodi multimodal, but also from a cost perspective and
Speaker 2: efficiency perspective. Why invest so much in LLM provisions when
Speaker 2: you already offer the rest?
Speaker 1: Yeah, and again, this is all about giving customers more choice.
Speaker 1: I think we as Amazon were investing in these models
Speaker 1: because we couldn't find the exact right mix of capabilities
Speaker 1: and customizations and things that we needed internally, so we
Speaker 1: started building them. And as we started building these, we
Speaker 1: saw that these models were actually getting quite good. The
Speaker 1: benchmarks were really good, and we were seeing some good
Speaker 1: capabilities come out of there. And our models are quite
Speaker 1: good at some particular areas. They're very good at executing
Speaker 1: agentic workflows. They're really good at pulling knowledge out of
Speaker 1: a RAG index there and they're very very low latency
Speaker 1: and low cost, and so we think that that capability
Speaker 1: is going to be really compelling for a lot of
Speaker 1: use cases. We found use cases inside of Amazon, but
Speaker 1: even inside of Amazon, we use a mix of different models.
Speaker 1: We use models from Anthropic, we use models from Meta
Speaker 1: and Lama, we use models from a bunch of different sources,
Speaker 1: and so I think that hopefully customers see a lot
Speaker 1: of value from these new Nova models, but we also
Speaker 1: expect that customers are going to combine lots of different
Speaker 1: models in lots of different ways.
Speaker 2: You haven't yet got the access to open ai, and
Speaker 2: for obvious reasons, with their relationship with Microsoft, would you
Speaker 2: think that that will at some point unfold.
Speaker 1: Look, I think my view is anytime customers demand something
Speaker 1: that eventually that sorts its way out. And I think
Speaker 1: open Ai obviously has a great set of models as well,
Speaker 1: and I'm sure many of our customers would love to
Speaker 1: have those available in Bedrock as well, and we'd love
Speaker 1: to support those there. And you know, I think I'd
Speaker 1: look at and we have a long term view on this,
Speaker 1: and over the long term, we'd like to make every
Speaker 1: technology available inside of Amazon and inside of AWS to use,
Speaker 1: and that means all the models that are out there,
Speaker 1: we'd love to offer them at Bedrock. All the software
Speaker 1: that's out there, we'd love to offer and make sure
Speaker 1: that it's available inside of AWS. Every service that everyone
Speaker 1: else in third parties are providing we want to make available.
Speaker 1: And that all goes back to that choice story. And
Speaker 1: so in the fullness of time, I believe that we
Speaker 1: would love open AI models inside of AWS. We absolutely
Speaker 1: it's you know, were we listen to our customers and
Speaker 1: if that's something the customers want, we're absolutely game. Obviously,
Speaker 1: with a partnership, it takes multiple sources and there's probably
Speaker 1: some complexities in there, as you allude to, but I
Speaker 1: think in the fullness of time it likely is the case.
Speaker 2: In the here and now. You spelled out on the
Speaker 2: previous that AI is already a multi billion dollar business
Speaker 2: growing at more than one hundred percent. Can you give
Speaker 2: us any more detailed numbers as to how large AAAI
Speaker 2: offering is for AWS?
Speaker 1: Now, You're right, it is a multi billion dollar business
Speaker 1: for us, and it's really growing rapidly. Take Bedrock, which
Speaker 1: is our platform that lots of customers are building all
Speaker 1: of their production AI applications on top of bedrock has
Speaker 1: grown the number of users five x in the last
Speaker 1: year alone, so it is really a rocket ship growth
Speaker 1: of people building this. I think what's really exciting is
Speaker 1: that people are using bedrock not just to do proof
Speaker 1: of concepts, but they're using it to really deeply integrate
Speaker 1: with their own enterprise data and to launcher production applications,
Speaker 1: which I think is a particularly telling sign of where
Speaker 1: the technology is going, where.
Speaker 2: It's evolved to and clients willing to spend it a
Speaker 2: time of a lot of uncertainty that we've got a
Speaker 2: new administration coming in, do you see any concerns in
Speaker 2: terms of well geopolitics, your road supply chain relationship with TSMC,
Speaker 2: for example.
Speaker 1: No, I mean, I think, look, we've we've worked with
Speaker 1: various administrations all throughout the last eighteen years that AWS
Speaker 1: has been around, and we're really excited to work with
Speaker 1: this administration. You know. I think that everybody is interested
Speaker 1: in ensuring that we have technology continuity and and there's
Speaker 1: a ton of opportunity here like that this is how
Speaker 1: we drive the economy, and I think all administrations are
Speaker 1: interested in continuing to make sure that we're able to
Speaker 1: drive the economy forward, and and AWS is a big
Speaker 1: enabler of that, and so we're excited to work with
Speaker 1: the new administration. I think we're we're obviously anxiously watched
Speaker 1: all of the geopolitical events all around the world to
Speaker 1: make sure that we're trying to insulate our customers as
Speaker 1: much as possible from any eventuality. But but you know,
Speaker 1: I think we're we're cautiously optimistic that we're in a
Speaker 1: good spot.
Speaker 2: You might have been cautiously watching what was happening in
Speaker 2: South Korea today amid your own event any exposure there.
Speaker 1: Matt, Sorry I missed that I was on stage, so
Speaker 1: just I'm from the stage right here to talk to you.
Speaker 2: So okay, I understood, But that in terms of the
Speaker 2: South Korean political destabilization, nothing that was a worry there.
Speaker 1: No, I mean, look, we have a we have a
Speaker 1: region in South Korea, and we have many of our
Speaker 1: customers that operate in South Korea and and will continue
Speaker 1: to support them and and operate with with with any
Speaker 1: with ever whichever government is in operation.
Speaker 2: We thank you so much for your time rushing off
Speaker 2: a stage A w S ce O Matt Garman