Speaker 1: Bloomberg Audio Studios, podcasts, radio news.
Speaker 2: AWS CEO Matt Garman joins us now on Bloomberg Television
Speaker 2: and Bloomberg Radio. Matt, it's great to have you back
Speaker 2: on the show. You know, this is the fastest growth
Speaker 2: for AWS in nearly five years. But to kind of
Speaker 2: capture a moment in time, could we start by talking
Speaker 2: about how much of that was driven by the frontier labs,
Speaker 2: the big ones Open AI and Thropic, or how much
Speaker 2: of it was something broader across the enterprise, something bigger
Speaker 2: in AI.
Speaker 1: Yeah, it's actually across the board this growth that we're
Speaker 1: seeing right now, and much of that is some of
Speaker 1: the AI labs that are building their models on top
Speaker 1: of AWS. But actually a lot of that growth is
Speaker 1: spread across all of our startup and enterprise customers, where
Speaker 1: AI is really impacting almost every single industry. And so
Speaker 1: whether it's financial services companies or healthcare companies, or retail
Speaker 1: companies or media companies. Really the growth is coming across
Speaker 1: the board as companies of all different sizes in all
Speaker 1: different industries are looking to use AI to grow their business.
Speaker 1: And so we're seeing a lot of growth from the
Speaker 1: top frontier labs, but for AWS, We're not like some others.
Speaker 1: Maybe it's concentrated on just one or two large customers,
Speaker 1: but it's actually growth from a really broad set of customers,
Speaker 1: which is nice to see.
Speaker 3: Matt.
Speaker 2: When AWS says the AI business has a revenue run
Speaker 2: rate of twenty five billion dollars, what does that mean?
Speaker 3: What does the figuring compass?
Speaker 1: Yeah, that includes both training from very large companies like
Speaker 1: Anthropic and Open Ai and other large tech companies, as
Speaker 1: well as many startups. But it also and then a
Speaker 1: big chunk of that is really inferenced that that broad
Speaker 1: swath of companies are doing. And so as companies think
Speaker 1: about how they take models in something on Amazon Bedrock
Speaker 1: and run those models to get value out of their business,
Speaker 1: some of them are automating processes. Many of them are
Speaker 1: running agentic workloads that they build on AWS to further
Speaker 1: drive efficiencies in their business or deliver new customer experiences.
Speaker 1: We consider all of that work, whether it's agent growth,
Speaker 1: whether it's inference, and some of that is training new models,
Speaker 1: all of that encompasses the AI business for us.
Speaker 2: I think I've asked you this question at various points
Speaker 2: in time since you became AWS CEO. But is there
Speaker 2: a percentage split right now training versus inference that you
Speaker 2: can give me.
Speaker 1: Yeah, you do, and it actually keeps shifting, I would
Speaker 1: say more and more. It keeps shifting more towards inference.
Speaker 1: I think we still see large training clusters being used
Speaker 1: by a number of companies, but as these as these
Speaker 1: models get really popular and really powerful, more and more
Speaker 1: companies are integrating that inference into their workloads. So, you know,
Speaker 1: I don't know the exact percentage today, but it keeps
Speaker 1: shifting more and more towards inference, and we expect that
Speaker 1: to continue as the economics makes sense where you really
Speaker 1: want most of that cost and spend being where value
Speaker 1: is being created for end customers.
Speaker 2: Amazon's overall CAPEX number for this year is very big,
Speaker 2: two hundred and twenty billion dollars, So it's up twenty billion,
Speaker 2: and Andy Jesse explained that's mostly AI, but it also
Speaker 2: accounts for higher memory pricing. Right, capex higher because of
Speaker 2: the cost environment's higher. But from aws's perspective, what's the
Speaker 2: trajectory for next year?
Speaker 3: You expect that capex.
Speaker 2: Will be biggest still because I think one of the
Speaker 2: things that Amazon is quite clear about is that even
Speaker 2: at two hundred and twenty billion dollars, it might not
Speaker 2: be enough to meet current demand.
Speaker 1: Yeah, one of the things that we're quite excited about
Speaker 1: is just the potential business for us is just massive,
Speaker 1: and we see this as a huge opportunity for us
Speaker 1: to really invest and help customers take advantage of the
Speaker 1: AI opportunity, and so we will keep investing. We think
Speaker 1: that there's a big opportunity for us and for customers,
Speaker 1: And as Andy kind of called out last week, we
Speaker 1: have really great insight into what that demand is going
Speaker 1: to be from customers, and so when we invest and
Speaker 1: we will keep investing in CAPEX next year as well,
Speaker 1: we have great insight into when that revenue is going
Speaker 1: to land. And so it's pretty well known for us,
Speaker 1: And as Andy mentioned, much of our capacity has already
Speaker 1: spoken through through the end of twenty seven and even
Speaker 1: through much of twenty eight, and so as we're investing,
Speaker 1: we're getting five year commitments from customers. We're getting these
Speaker 1: long term commitments, and so we're out there making investments
Speaker 1: to keep being able to grow the business and try
Speaker 1: to meet customer demand. But as you say, today, demands
Speaker 1: still significantly outstripped supply. And we're trying to build and
Speaker 1: invest to keep up with what customers are asking for.
Speaker 2: And so we expect the CAPEX number will be bigger
Speaker 2: next year than it is this year.
Speaker 1: I expect us to keep investing as we see opportunities,
Speaker 1: and some of that we'll see how the market continues
Speaker 1: to grow. But right now we're excited about our investments
Speaker 1: and we'll continue to invest.
Speaker 2: The chip business has also got a lot of momentum.
Speaker 2: You know, Trainium for some time has been a big
Speaker 2: part of the strategy, and we've discussed that. I've been
Speaker 2: able to go to Austin spend a lot of time
Speaker 2: in Anapurna Labs to look at the server design. Let's
Speaker 2: start by asking what when you say the chip business
Speaker 2: as a revenue run rate twenty five billion dollars, what
Speaker 2: does that mean? That is the business of selling the
Speaker 2: chips to third parties or renting out capacity based on
Speaker 2: the chips.
Speaker 1: Yeah, renting out capacity based on those chips. So today
Speaker 1: we run all of our own chips inside of the
Speaker 1: AWS cloud, and that includes both Trainingum chips as well
Speaker 1: as Graviton chips or which our general purpose processors. Graviton
Speaker 1: is incredibly popular We've been building Graviton for many years now,
Speaker 1: and in fact, almost all of our large customers use
Speaker 1: Graviton as some part of their deployment, and it's a
Speaker 1: key part of how help customers deliver extra value by
Speaker 1: lowering their costs. Trainium is wildly popular, and as we've
Speaker 1: mentioned a couple of times now, we're largely sold out
Speaker 1: through the end of next year for Trainium capacity and
Speaker 1: it's really really popular. We're starting to see both large
Speaker 1: customers as well as startups really love the benefits of
Speaker 1: running on Trainium and having that choice in a cloud
Speaker 1: where you can run on purpose built processors like Trainium
Speaker 1: that give you really great AI performance, together with in
Speaker 1: Nvidia chips for when you need some of that processing
Speaker 1: for GPUs as well, gives customers the right mix that
Speaker 1: they're oftentimes looking for, and so we find that combination
Speaker 1: to be quite powerful and our Trainium three chips are
Speaker 1: really really popular with customers.
Speaker 3: Right now, we're live.
Speaker 2: On Bloomberg Television and Bloomberg Radio. This is Bloomberg Tech.
Speaker 2: In conversation with Matt garm and the COO of AWS
Speaker 2: explain the difference in economics between capacity based on Trainium
Speaker 2: and capacity based on Blackwell, for example, what is the
Speaker 2: selling point of going with Amazon Silicon.
Speaker 3: Yeah, we think that.
Speaker 1: Look, we think customers want choice, and so we offer
Speaker 1: great offerings for both products. Some customers really like to
Speaker 1: build on in Nvidia GPUs and those are fantastic, and
Speaker 1: we're one of Nvidia's absolute largest customers in the world.
Speaker 1: We've been offering in Vidia GPUs in the cloud for
Speaker 1: over a decade now, and AWS is the most scalable
Speaker 1: and secure and stable place to run in Vidia servers anywhere,
Speaker 1: and so we're very excited about that business. But when
Speaker 1: you add trainingum, we're able to lower the costs for
Speaker 1: many workloads, and so customers like to have that opportunity
Speaker 1: where they can take these really massively scaled AI workloads
Speaker 1: or inference art workloads where we can tune them because
Speaker 1: we kind of control that whole stack, control the training chips,
Speaker 1: the data centers, and the inference stack, and can really
Speaker 1: tune that whole stack. And that helps customers lower their
Speaker 1: costs and improve performance. And we're really starting to see
Speaker 1: that flywheel go where model providers of all types and
Speaker 1: customers love that they get that value and great performance.
Speaker 1: And we think that that's a real flywheel that we
Speaker 1: can keep turning and are quite excited about continuing to
Speaker 1: grow that business.
Speaker 3: Could you make it tangible for me? What is the cost?
Speaker 1: Say, it's workload by workloads, so there's not like a
Speaker 1: specific cost savings, but for workloads where we see this
Speaker 1: optimization happen, customers can oftentimes save twenty thirty percent off
Speaker 1: of their inference costs when they run that on trainium interesting.
Speaker 2: Would you consider selling the chips outright as opposed to
Speaker 2: renting capacity based on them.
Speaker 1: Yeah, we've mentioned that we might consider that in the future.
Speaker 1: Right now, we have so much demand inside of the
Speaker 1: AWS cloud that we really love that business and so
Speaker 1: that's the place we're focusing right now. But it's an
Speaker 1: interesting opportunity and it's something that we would definitely consider
Speaker 1: in the future.
Speaker 2: The other big development was AWS signing the Open Weights letter.
Speaker 2: You yourself, you know, communicate up that. Why was it
Speaker 2: important to you, Matt to sign that and participate in that?
Speaker 3: Yeah?
Speaker 1: I think that Again, a lot of this boils down
Speaker 1: to customers wanting choice, and the big frontier labs have
Speaker 1: awesome models today, whether it's open AI running on bedrock,
Speaker 1: whether it's anthropy running on bed Rock. Customers really love
Speaker 1: using those models, but they also like being able to
Speaker 1: customize models, and so having a broad ecosystem of open
Speaker 1: weights models, I think is incredibly important for innovation. It's
Speaker 1: important for our customers, and so you think about things
Speaker 1: like the Neematron models from Nvideo, or you think about
Speaker 1: Kimmi three or some of the Chinese open weights models.
Speaker 1: Those really help customers be able to customize to their
Speaker 1: own data and really be able to innovate. And so
Speaker 1: we think that it's important to not overlegislate there and
Speaker 1: give that flexibility for customers to use whichever models they
Speaker 1: find to be the best fit. And that's actually why
Speaker 1: we really focus on having all of those available inside
Speaker 1: of Bedrock, so the customers can choose which ones they
Speaker 1: want to use.
Speaker 2: Not to over legislate. I mean, I had the conversation
Speaker 2: with Jensen Wong the day the letter was posted about
Speaker 2: why then, what was the rationale? It does seem like
Speaker 2: the concern is that the US government overly regulates open models.
Speaker 2: Was that sort of a motivation for you?
Speaker 1: Yeah, I think that's the concern, and I think you
Speaker 1: just want to make sure that we kind of have
Speaker 1: an even playing field, and it doesn't mean that there's none.
Speaker 1: By the way, I think there should probably be the
Speaker 1: same level of oversight of both frontier models as well
Speaker 1: as open weights models, and I think we should have
Speaker 1: a consistent framework, but kind of making sure that folks
Speaker 1: realize how important that is to what customers in our
Speaker 1: industry is out there building on and that those open
Speaker 1: weights are a key building block of that, and we
Speaker 1: want to make sure that, however legislation lands, it lands
Speaker 1: evenly across all of those.
Speaker 2: Matt a question from our Bloomback Tech audience for you
Speaker 2: is aws's attitude towards Kimmi K three. The open weight
Speaker 2: was released July twenty seventh. You know, given the platform's
Speaker 2: AWS offers, what is the plan there.
Speaker 1: Yeah, it's a great model. The team there has done
Speaker 1: a really nice job and we see really great performance
Speaker 1: there and we'll continue to support in Bedrock offering the
Speaker 1: Kimmi models. I think the interesting thing is also as
Speaker 1: you see many of these open weight companies, they're starting
Speaker 1: to think about how they make money in these models
Speaker 1: as well, and so they're starting to introduce licensing around
Speaker 1: some of these open weights models as when customers want
Speaker 1: to run them in a cloud environment, and so I
Speaker 1: anticipate that there's going to be a blending of some
Speaker 1: of these models too, where these companies won't continue to
Speaker 1: spend lots of money and then just offer their IP
Speaker 1: up to the world. They're going to be licensing models.
Speaker 1: They're going to have ways of making money on those
Speaker 1: as well. And you're starting to see that with some
Speaker 1: of the open weights models actually having some licensing around them.
Speaker 1: And so we'll keep supporting these in bedrock and working
Speaker 1: with those companies in order to offer these in the
Speaker 1: best possible way.
Speaker 2: That's why I'd like to end the conversation the business
Speaker 2: case and economic opportunity for open models from aws's perspective.
Speaker 2: So if the model makers themselves would like to make money,
Speaker 2: you know, how does AWS see that going in your
Speaker 2: favor to make money from wide use of open models?
Speaker 1: Yeah, yeah, I mean we basically that is exactly what AWS.
Speaker 1: It's a great platform for company needs to come and
Speaker 1: offer their IP and their capabilities to the world. And
Speaker 1: so everybody, whether they're startups, whether they're governments, whether they're
Speaker 1: large enterprises, everybody can build on top of AWS. And
Speaker 1: if they have choice and they have access to these
Speaker 1: different models. It allows companies to be able to come
Speaker 1: and monetize them and sell when they have value to customers,
Speaker 1: to be able to offer value when they think that
Speaker 1: they can go at lower prices or better performance. And
Speaker 1: so having that open platform allows everybody to have that
Speaker 1: competitive chance. And AWS is a fantastic partner for all
Speaker 1: these model providers to get access to the broad set
Speaker 1: of customers. And so it's a great ecosystem where model
Speaker 1: providers benefit and customers benefit by having all of these
Speaker 1: in the same place.
Speaker 2: Matt Gorman, COOFAWS Amazon Web Services back on blom bug Tech.
Speaker 3: Thank you very much.