Speaker 1: Bloomberg Audio Studios, podcasts, radio news.
Speaker 2: Welcome to our Bloomberg radio and television audiences worldwide. We
Speaker 2: go right now to a conversation with Matt Garman, AWS CEO. Matt,
Speaker 2: it's good to catch up. It has been basically one
Speaker 2: year that you've been in the role as AWS CEO.
Speaker 2: Is a place to start what has been the biggest
Speaker 2: achievement in that time for AWS.
Speaker 3: Yeah, thanks for having me on. It's nice to be
Speaker 3: here again. Yeah, it's been a fantastic year of innovation.
Speaker 3: It's really been incredible and as I look out there,
Speaker 3: one of the things that I've been most excited about
Speaker 3: is how fast our customers are innovating and ten adopting
Speaker 3: many of the new technologies that we have. And as
Speaker 3: you think about customers that are on this cloud mication journey,
Speaker 3: many of them have been doing that for over the
Speaker 3: last several years, but this year in particular, that we've
Speaker 3: really seen an explosion of AI technologies, of agentic technologies,
Speaker 3: and increasingly we're seeing more and more customers move their
Speaker 3: entire estates into the cloud and AWS.
Speaker 4: So it's been really fun to see.
Speaker 3: It's been an incredible pace of technology and it's been
Speaker 3: a really fun first year.
Speaker 2: The moment that investors kind of sat up and paid
Speaker 2: attention was when Amazon said that it's AI business was
Speaker 2: at a multi billion dollar run rate in terms of sales.
Speaker 2: What we don't understand as well is what proportion of
Speaker 2: that is AWS infrastructure?
Speaker 3: Yeah, that is AWS, right, And so the key is
Speaker 3: that's a mix of customers running their own models. Some
Speaker 3: of that is on Amazon Bedrock, which is our own
Speaker 3: hosted models, where we have first party models like Amazon Nova,
Speaker 3: as well as many of the third party models like
Speaker 3: Anthropics models, and some of those are applications things like
Speaker 3: on q which helps people do automated software development, as
Speaker 3: well as a host of other capabilities, and so there's
Speaker 3: a mix of that, and I think part of the
Speaker 3: most interesting thing about being at a multi billion dollar
Speaker 3: run rate is we're at the very earliest stages of
Speaker 3: how AI is going to completely transform every single customer
Speaker 3: out there.
Speaker 4: And we talk to customers and we look at where the.
Speaker 3: Technology landscape is, and we firmly believe that every single business,
Speaker 3: every single industry, and really every single job is going
Speaker 3: to be fundamentally transformed by AI. And I think we're
Speaker 3: starting to see the early start the stages of that.
Speaker 3: But again we're just at the very earliest stages that
Speaker 3: I think what's going to be possible, and so that
Speaker 3: multi billion dollar business that we have today is really
Speaker 3: just the start.
Speaker 2: Can you give me a generative AI revenue number.
Speaker 4: For the world or for awls?
Speaker 2: Are you guys for AWS? Maybe Amazon as a whole.
Speaker 3: Yeah, Like I said, we are in multiple billions of dollars,
Speaker 3: and that's for customers using AWS. We also use lots
Speaker 3: of generative AI inside of Amazon for a wide range
Speaker 3: of things. We use it to optimize our fulfillment centers.
Speaker 3: We use it when you go to the retail site
Speaker 3: to summarize reviews, or to help customers find products in
Speaker 3: a faster and more interesting way. We use AI in
Speaker 3: Alexa in our new Alexa Plus offering, where we conversationally
Speaker 3: talk to customers through the Alexa interface and help them
Speaker 3: accomplish things through voice that they were never able to
Speaker 3: do before. So every single aspect of what Amazon does
Speaker 3: leverages AI, and our customers are exactly the same. Customers
Speaker 3: are looking to AWS to completely change, whether it's their
Speaker 3: contact centers through something like Amazon Connect where it shows
Speaker 3: AI capabilities so that you don't have to go program
Speaker 3: it all the way down to our custom chips or
Speaker 3: Nvidia processors or anything where customers at the metal are
Speaker 3: building their own models. We have the whole range of
Speaker 3: people that are building AI on top of AWS as
Speaker 3: well as Amazon themselves.
Speaker 2: We always credit AWS is being number one hyperscala. But
Speaker 2: just what you said there about what the client's using
Speaker 2: the silicon level through to capacity, it would really help
Speaker 2: if you could proportionately tell me what percentage of workloads
Speaker 2: are being run for training and which proportion of workloads
Speaker 2: being run for inference.
Speaker 4: Sure, yeah, and that changes over time. I think.
Speaker 3: Look as we progress over time, more and more of
Speaker 3: the AI workloads are being inference. I'd say in the
Speaker 3: early stages of AI and general of AI, a lot
Speaker 3: of that usage was dominated by training as people were
Speaker 3: building these very large models with small amounts of usage.
Speaker 3: Now the models are getting bigger and bigger, but the
Speaker 3: usage is exploding at a rapid rate, and so I
Speaker 3: expect that over the fullness of time, eighty percent, ninety percent,
Speaker 3: the vast majority of usage is going to be in
Speaker 3: inference out there, and really and just for all those
Speaker 3: out there, inference It really is how AI is embedded
Speaker 3: in the applications that everybody uses. And so as we
Speaker 3: think about our customers building, you know, there's a small
Speaker 3: number of people who are going to be building these models,
Speaker 3: but everyone out there is going to use inference as
Speaker 3: a core building block in everything they do. And every
Speaker 3: application is going to have inference, and already is starting
Speaker 3: to see inference built in to every application. And we
Speaker 3: think about it as just the new building block. It's
Speaker 3: just like compute, it's just like storage, it's just like
Speaker 3: a database.
Speaker 4: Inference is a core building block.
Speaker 3: And so as you talk to people who are building
Speaker 3: new applications, they don't think about it as AI is
Speaker 3: over here and my application is over here. They really
Speaker 3: think about AI is embedded in the experience. And so
Speaker 3: it's increasingly I think it's going to be difficult for
Speaker 3: people to say what part of your revenue is going
Speaker 3: to be driven by AI. It's just part of the
Speaker 3: application that you're building, and it's going to be a
Speaker 3: core part of that experience, and it's going to deliver
Speaker 3: lots of benefits from efficiency, from capabilities, and from user
Speaker 3: experience for all sorts of applications and industries.
Speaker 2: But present day, it's fair to say majority is still training.
Speaker 3: No, I think that at this point more definitely more
Speaker 3: usage as inference than training.
Speaker 2: We want to welcome our radio and television audiences around
Speaker 2: the world. We're speaking to AWS CEO Matt Garman, who
Speaker 2: officially next week celebrates one year in that role leading AWS.
Speaker 2: A new metric that has been discussed, particularly this earning season.
Speaker 2: We discussed it with Nvidia CEO Jensen one this week
Speaker 2: is token growth and tokenization. Has AWS got a metric
Speaker 2: to share on that front?
Speaker 3: I don't have any metrics to share on that front,
Speaker 3: but I think it's one of the measures that we
Speaker 3: can look at as the numbers of tokens that are
Speaker 3: being served out there, but it's not the only one,
Speaker 3: and I increasingly think that people are going to be
Speaker 3: thinking about these things differently. Tokens are a particularly interesting
Speaker 3: thing to look at when you're thinking about text generation,
Speaker 3: but not all things are created equal.
Speaker 4: I think, particularly as you think about.
Speaker 3: AI reasoning models, the input and output tokens don't necessarily
Speaker 3: talk about the work that's being done, and increasingly you're
Speaker 3: seeing models can do work for a really long period
Speaker 3: of time before they output tokens, and so you're having
Speaker 3: these models that can sometimes think for hours at a time. Right,
Speaker 3: you ask these things to go and actually do research
Speaker 3: on your behalf. They can go out to the internet,
Speaker 3: they can pull information back, they can synthesize, they can
Speaker 3: redo things. If you think about coding and que developer,
Speaker 3: we're seeing lots of coding where it goes and actually
Speaker 3: reasons and does iterations and iterations and improves on itself,
Speaker 3: looks at what it's done, and then eventually outputs the
Speaker 3: end result. And so at some point kind of the
Speaker 3: final output token is not really the best measure of
Speaker 3: how much work is being done. If you think about images,
Speaker 3: if you think about videos, there's a lot of content
Speaker 3: that's being created and a lot of thought that's being done.
Speaker 4: And so tokens are one aspect of it.
Speaker 3: And it's an interesting measure, but I don't think it's
Speaker 3: the only measure to look at. Although they are rapidly increasing.
Speaker 2: Project RAY near Massive Custom Server Design project. Yeah, what
Speaker 2: is the operational statu and latest on project right now?
Speaker 3: Yeah, So we're incredibly excited about so project right here
Speaker 3: is a collaboration that we have with our partners at
Speaker 3: Anthropic to build the largest compute cluster that they'll use
Speaker 3: to train their next generation of their claud models, and
Speaker 3: Anthropic has the very best models out there today. Claude
Speaker 3: four just launched, I think it was last week, and
Speaker 3: it's been getting incredible adoption out there from our customer base.
Speaker 3: Anthropic is going to be training their next version of
Speaker 3: their model on top of Trainium two, which is Amazon's
Speaker 3: custom built accelerator processors purpose built for AI workloads, and
Speaker 3: we're building one of the largest clusters ever released. It's
Speaker 3: an enormous cluster, more than five times the size of
Speaker 3: the cluster compared to the last one that they trained on,
Speaker 3: which again is the world's leading model.
Speaker 4: So we're super excited about that.
Speaker 3: We're landing Trainium to servers now and they're already in
Speaker 3: operation and Nthropic has already is our using parts of
Speaker 3: that cluster, and.
Speaker 4: So super excited about that.
Speaker 3: And the performance that we're seeing out of Trainium too
Speaker 3: continues to be very impressive and really pushes the envelope
Speaker 3: I think on what's possible both from an absolute performance
Speaker 3: basis as well as a cost, performance and scale basis.
Speaker 3: I think some of those are equally going to be
Speaker 3: really important as we move forward in this world, because
Speaker 3: today much of the feedback you get is that AI
Speaker 3: is still too expensive. But costs are coming down pretty aggressively,
Speaker 3: and it's still too expensive, and so we think there's
Speaker 3: a number of things that need to happen there. Innovation
Speaker 3: on the silicon level is one of those things that
Speaker 3: needs to help bring the cost down, as well as
Speaker 3: innovation on the software side and algorithmic side so that
Speaker 3: you have to use less compute per unit of inference
Speaker 3: or training. So all of those are important to bring
Speaker 3: that cost down to make it more and more possible
Speaker 3: for ADI to be used in all of the places
Speaker 3: that we think that it will be over time Matt.
Speaker 2: On Wednesday, Nvidia CEO Jensen Wang summarized inference demand for me.
Speaker 2: I just wanted to play you that SoundBite.
Speaker 4: Sure, well, we.
Speaker 5: Got a whole bunch of engines firing right now. The
Speaker 5: biggest one, of course, is the reasoning AI inference. The
Speaker 5: demand is just off the charts. You see the popularity
Speaker 5: of all these AI services.
Speaker 2: Now your pitch for trainium too. And as you know,
Speaker 2: I've kind of taken a part the serve a design
Speaker 2: and looked at it is the efficiency and cost efficiency
Speaker 2: relative to Nvidia tech. Are you seeing that same demand
Speaker 2: Jensen outlined for Trainium two outside of the relationship with Amthropic.
Speaker 3: Yeah, Look, we're seeing it across a number of different places,
Speaker 3: but it's not really Trainingum two versus in Nvidia, and
Speaker 3: I think that's not really the right way to think
Speaker 3: about it. I think there's plenty of room. The opportunity
Speaker 3: in this space is massive. It's not one versus the other.
Speaker 3: We think that there's plenty of room for both these
Speaker 3: and Jensen and I speak about this all the time
Speaker 3: that in Vidia is an incredibly fantastic platform. They've built
Speaker 3: a really strong platform that's useful and is the leading
Speaker 3: platform for many many applications out there, and so we
Speaker 3: are incredible design partners with them. We make sure that
Speaker 3: we have the latest in video technology for everyone, and
Speaker 3: we continue to push the envelope on what's possible with
Speaker 3: all of the latest in Vidia capabilities. And we think
Speaker 3: there's room for Trainium and other technologies as well, and
Speaker 3: we're really excited about that, and so we have many
Speaker 3: of the leading AI labs are incredibly excited about using
Speaker 3: Trainium too, and really leaning into the benefits that you
Speaker 3: get there, But for the law for a long time,
Speaker 3: these things are going to be living in concert together,
Speaker 3: and I think there's plenty of room, and customers want choice.
Speaker 3: At the end of the day, Customers don't want to
Speaker 3: be forced into using one platform or the other. They'd
Speaker 3: love to have choice in Our job at AWS is
Speaker 3: to give customers as much choice as possible.
Speaker 2: What is general availability of Nvidia GB two hundred for AWS?
Speaker 2: And have you, I guess, launched Grace Blackwell backed instances yet?
Speaker 3: Yes, yep, so we've launched our they would call them
Speaker 3: P six instances, And so those are available in AWS
Speaker 3: today and customers are using them and liking them and
Speaker 3: the performance is fantastic. So those are available today. We're
Speaker 3: continuing to ramp capacity. We work very closely with the
Speaker 3: Nvidia team to aggressively ramp capacity and demand as strong
Speaker 3: for those P six instances. But customers are able to
Speaker 3: go and test those out today, and like I said,
Speaker 3: we're ramping capacity incredibly fast all around the world and
Speaker 3: in our various different regions.
Speaker 2: Now, what is your attitude to Claude Anthropics model being
Speaker 2: available elsewhere on Azure Foundry for example?
Speaker 4: Great I mean that's okay too.
Speaker 3: I think many of our customers make their applications available
Speaker 3: in different places, and we understand that various different customers
Speaker 3: want to use capabilities in different areas and different clouds.
Speaker 3: Our job is to make AWS and this is what
Speaker 3: we do, is to make AWS the best place to
Speaker 3: run every type of workload, and that includes anthropic claud models, but.
Speaker 4: It includes a wide range of things.
Speaker 3: And frankly, that's why we see big customers migrating over
Speaker 3: to AWS. Take somebody like a Mondali's who's really gone
Speaker 3: all in with AWS and moved some of their workloads
Speaker 3: to there. One of the reasons is that they see
Speaker 3: that we have capabilities sometimes using AI by the way,
Speaker 3: in order to really help them optimize their costs and
Speaker 3: have the most available, most secure platform in monthlies. This case,
Speaker 3: they're taking many of their legacy Windows platforms and transforming
Speaker 3: them into Linux applications and saving.
Speaker 4: All of that licensing costs.
Speaker 3: But we have many customers who are doing that, and
Speaker 3: so our job is to make AWS by far the
Speaker 3: most technically capable platform that has the most and widest
Speaker 3: set of services, and that's.
Speaker 4: What we do.
Speaker 3: But I'm perfectly happy for other people to use, Like,
Speaker 3: it's great that Claud's making their services available elsewhere and
Speaker 3: we see the vast majority of that usage happening in AWS.
Speaker 4: Though.
Speaker 2: Will we see open AI models on AWS this year?
Speaker 3: Well, just like you know, we encourage all of our
Speaker 3: partners to be able to be available elsewhere. I'd love
Speaker 3: for others to take that same tack.
Speaker 2: Let's end it with this a question from the audience actually,
Speaker 2: which is where you're going to grow data center capacity
Speaker 2: around the world. I got a lot of questions from
Speaker 2: Latin America and Europe in particular where Jensen flies to
Speaker 2: next week?
Speaker 4: Yeah. Great.
Speaker 3: So in Latin America we're continuing to span expand our
Speaker 3: capacity pretty aggressively. Actually, earlier this year we launched our
Speaker 3: Mexico region, which has been really well received by customers,
Speaker 3: and we've announced a new region in Chile. And we
Speaker 3: already have and for many years have had a region
Speaker 3: in Brazil which is quite popular and has many of
Speaker 3: the largest financial institutions in South America running there. So
Speaker 3: across Central and South America, we are continuing to rapidly expand.
Speaker 3: In Europe we're expanding as well. We have many regions
Speaker 3: already in Europe. One of the things I'm most excited
Speaker 3: about actually is at the end of this year we're
Speaker 3: going to be launching the European Sovereign Cloud, which is
Speaker 3: a unique capability that no one has, which is completely
Speaker 3: designed for critical EU focused sovereign workloads, and we think
Speaker 3: given some of the concerns that folks have around data sovereignty,
Speaker 3: particularly for government workloads as well as regulated workloads, we
Speaker 3: think that's going to be an incredibly op popular opportunity
Speaker 3: for everybody.
Speaker 2: Matt Garman AWSCO, thank you very much.
Speaker 4: Thank you for having me