Speaker 1: Bloomberg Audio Studios, podcasts, radio news.
Speaker 2: Now, let's narrow our focus from the broader markets to
Speaker 2: one single stock. Amazon, the tech giant, hosting its annual
Speaker 2: Amazon Web Services Reinvent Conference down in Las Vegas this week.
Speaker 2: The Cloud focused confab draws developers, engineers, and other thought
Speaker 2: leaders in tech to explore the latest cloud and AI
Speaker 2: projects happening under Amazon's roof, including a new AI chip.
Speaker 2: Let's go live now, we're Bloomberg Tech co host ed
Speaker 2: Ludlow is joined by a special guest ed take it away. Yeah.
Speaker 3: Three pieces of news move markets this morning. A new
Speaker 3: generation of Frontier model from AWS new agentic tools, and
Speaker 3: then a very quickly released, installed and now ramping generation
Speaker 3: of in house custom accelerator which is Trainium three. All
Speaker 3: points of discussion for Matt Garman, AWSCO. You know the
Speaker 3: base point with Trainium three and you've moved quickly to
Speaker 3: bring it to the real world is cost performance efficiency
Speaker 3: over the prior generation, but also over in Vidia GPUs,
Speaker 3: over Google TPUs. I think what people are trying to
Speaker 3: understand is that ramp part I was talking about when
Speaker 3: real world customers use it beyond this anchor customer of Enenthropic,
Speaker 3: which relies on it currently.
Speaker 1: Yeah. Well, look, we're quite excited about Trainium and Trainium
Speaker 1: three in particular, as you mentioned, excited to get it
Speaker 1: into customer's hands. And part of where we have a
Speaker 1: benefit that we can bring to bear is, as you mentioned,
Speaker 1: getting it into market quickly, and it's because we control
Speaker 1: that full stack, We control the silicon development, we control
Speaker 1: the data centers that all land in. We know that
Speaker 1: full environment, and we can land that in very large
Speaker 1: clusters for people to take advantage of that, and the
Speaker 1: performance that we're seeing out of it is quite incredible,
Speaker 1: and so we're anxious and excited to get more and
Speaker 1: more people using it.
Speaker 3: I've been able to go inside out a Perna Labs
Speaker 3: and look at the engineering work between the first generation
Speaker 3: of Trainium and second It wasn't just the accelerator at
Speaker 3: the server level as well.
Speaker 1: That's right.
Speaker 3: But a part of the surprise of today is this,
Speaker 3: you appear to be committing to an annual cadence a
Speaker 3: new generation of Trainium. How do you keep that up?
Speaker 1: Well, the key thing that we're focused on is making
Speaker 1: sure that we can iterate on the technology as fast
Speaker 1: as possible. The desire and the hunger out there for
Speaker 1: more power and more compute is almost insatiable. And so
Speaker 1: the more we can take an existing power footprint, an
Speaker 1: existing set of capabilities and bring more and more compute
Speaker 1: into that for customers to build cool applications and cool
Speaker 1: environments and to get value from that, that's we're focused on. Then,
Speaker 1: so we're going to be pushing that envelope as fast
Speaker 1: as we as we possibly can to get those new
Speaker 1: and new capabilities out to customers.
Speaker 3: The pitch for Trainium in both the training and inference
Speaker 3: use case is that it's a great deal, you know,
Speaker 3: cost effective performance. At the same time, you went on
Speaker 3: stage and said AWS is quote by far the best
Speaker 3: place to run in Vidia GPS how above if possible.
Speaker 1: Well, I mean that both both are possible because that
Speaker 1: is a great environment to run accelerators and compute in.
Speaker 1: And so we've been working for fifteen plus years with
Speaker 1: the in Nvidia team and Jensen and team to deliver
Speaker 1: outstanding capabilities for our customers and for when you're running
Speaker 1: a large cluster of Nvidia GPUs, people will tell you
Speaker 1: AIGHTWS is the best place you get the best performance,
Speaker 1: the most stable cluster the best capabilities out there and
Speaker 1: broad scale, and it's why folks like OpenAI and others
Speaker 1: are running in AWS and we have that choice. And
Speaker 1: so for others that want to be able to take
Speaker 1: advantage of Trainium, and there's some use cases that are
Speaker 1: best for Trainum, there's other use cases where in vidio
Speaker 1: GPUs are going to be your best option. We want
Speaker 1: to have all of those available, and so we think
Speaker 1: that if we can continue to push the envelope on
Speaker 1: what Trainium can deliver for customers and make sure that
Speaker 1: we are supporting the latest and greatest from everything that
Speaker 1: the awesome team in Nvidia is delivering, that's going to
Speaker 1: be the best outcome for our customers.
Speaker 3: The plan for AWS is to basically double capacity by
Speaker 3: the end of twenty twenty seven to round eight gigawatts,
Speaker 3: so you have a sense of how you apportion that
Speaker 3: capacity in how silicon and server designs to traineum versus
Speaker 3: and video gp is.
Speaker 1: We're just going to keep pushing as fast as we
Speaker 1: can and we'll see where customer demands drives us as
Speaker 1: we go. And as you said, we're massively adding capacity.
Speaker 1: In the last year alone, we've added three point eight
Speaker 1: gigawatts of capacity, and we'll continue to add more and
Speaker 1: more as over the next couple of years, and we'll
Speaker 1: let customer demands drive us a little bit on what
Speaker 1: they're looking for and what they want, and that's what
Speaker 1: we always listen to and that's what we'll continue to
Speaker 1: listen to.
Speaker 3: The focus with Trainium in the time I've been able
Speaker 3: to interact with you and talk about not again not
Speaker 3: just the accelerator, but at the server design level, there's
Speaker 3: a lot of benefits the customer. When does that benefit
Speaker 3: start accruing to AWS in terms of profitability, Like if
Speaker 3: it's such a good financial proposition, you must be able
Speaker 3: soon to say when making a lot of money on this.
Speaker 1: Yeah, Well, you're already seeing some of the benefits of
Speaker 1: crew You see things like bedrock growing really really rapidly,
Speaker 1: and you see trainingum powering that under the covers, and
Speaker 1: we announced today that more than half of all tokens
Speaker 1: and inference done in bedrock are done on TRAININGUM two
Speaker 1: servers under the covers, and so you're already seeing that
Speaker 1: benefit come. You see the models that we're building in
Speaker 1: Nova and Nova two start to get better and better
Speaker 1: over time and be accelerated by Trainum, and so we
Speaker 1: really think that there's a whole bunch of dimensions on
Speaker 1: which both our customers, our partners, and our own products
Speaker 1: are going to get accelerated all from Trainium.
Speaker 3: Every time you come onto the program, I always offer
Speaker 3: the audience opportunity to pose a question to you. There's
Speaker 3: a lot of interests in AWS right. Many of your
Speaker 3: customers span global technology. Actually most of the questions were
Speaker 3: about anthropic. That wasn't much said on stage. I think
Speaker 3: people are trying to understand what is the benefit and
Speaker 3: advantage AWS Office to anthropic while they are ramping Trainium
Speaker 3: through Project Raineer, but also ramping their tea allocations as well.
Speaker 1: Well. Look, our partners are an anthropic. Our partnership with
Speaker 1: them is incredibly strong and it's never been stronger, and
Speaker 1: we do a ton of collaboration with them, and as
Speaker 1: I mentioned through Project Right here, it's a huge collaboration
Speaker 1: there to go build their current generation models and all
Speaker 1: their models run today and launch on day one on
Speaker 1: top of Trainingum and on top of AWS which we're
Speaker 1: incredibly excited about it, and we'll continue that partnership for
Speaker 1: a long time. I think from them, they have a
Speaker 1: huge demand for compute, and so they'll go to other
Speaker 1: places where it makes sense to round out their compute
Speaker 1: needs because they just have such massive needs for compute,
Speaker 1: and they have customers in other clouds as well, But
Speaker 1: we're definitely their their primary cloud provider and closest partner.
Speaker 1: For sure.
Speaker 3: Supply constraints so am Fropic is supply constrained, they can't
Speaker 3: get the compute they need. We've talked about the rampont
Speaker 3: and video GPU and in house silicon. Is there a
Speaker 3: supply constraint element with AWS so you able to get
Speaker 3: the chips that you need.
Speaker 1: Yeah, I think there's always anytime you see an industry
Speaker 1: that's growing as fast as this is right now, when
Speaker 1: you think about AI and model development and chips, there
Speaker 1: are going to be constraints. No matter what. There is
Speaker 1: more demand than there is supply. Sometimes it's in chips,
Speaker 1: Sometimes it's in power and data centers. Sometimes it's in
Speaker 1: you know, different parts of that. At some points it's
Speaker 1: you know, networking equipment. At some point it's transistors, you know,
Speaker 1: resistors or whatever it is, and you look at the
Speaker 1: entire supply chain that is needed to ramp up at
Speaker 1: such a massive rate right Never before has the technology
Speaker 1: industry ramped at the rate that we are right now,
Speaker 1: and so there are always constraints, and so it's not
Speaker 1: necessarily that there is necessarily one constraint where it's like, wow,
Speaker 1: I can't get in vidio chips. We can get in
Speaker 1: video chips. And actually Jensen team have been incredibly supportive
Speaker 1: and great partners and helping us get capacity there. It's
Speaker 1: not that you can't get power. We're getting power all
Speaker 1: over the place. But it's just we're ramping all of
Speaker 1: these places in such rapid rates that always there's a
Speaker 1: constraint in that system, and it'll change every month you
Speaker 1: ask me of what the current one is.
Speaker 3: Throughout the day, we was just speaking with your team
Speaker 3: about the idea we're moving from AI assistance to AI
Speaker 3: co workers. You know, particular focus on the agentic offering
Speaker 3: that you've done. You're in the camp of people, if
Speaker 3: you don't mind me saying that sees basically ninety percent
Speaker 3: of the value in enterprise coming from agentic technology. Do
Speaker 3: you have any data or evidence to support that all
Speaker 3: of your customers are ready for that?
Speaker 1: Yeah, I don't think all of our customers are. You
Speaker 1: get ready for that, but they're excited about it, So,
Speaker 1: you know, I think it'd definitely be an overstatement to
Speaker 1: say everybody's ready for it. And part of that is
Speaker 1: because it is going to take change. Right, people are
Speaker 1: going to have to change how they think about work.
Speaker 1: They're going to have to change their process flows, they're
Speaker 1: going to have to change some of the things about how
Speaker 1: they get work done. It's not just going to be
Speaker 1: a magic one that's going to come in and magically
Speaker 1: get them to get value. But almost everyone that I
Speaker 1: talked to you definitely sees that that's the path. These
Speaker 1: the agentic power of the power of agents is what
Speaker 1: allows customers to actually get that work done. And when
Speaker 1: they see that efficiency gain, they see them able to
Speaker 1: accomplish things they weren't able to do before. That is
Speaker 1: when it's worth it to go make these changes. And
Speaker 1: so there's going to be work for people, and it's
Speaker 1: going to take some time, right, It's taken We're twenty
Speaker 1: years into the cloud journey and still only a fraction
Speaker 1: of workloads have moved to the cloud. So it's going
Speaker 1: to take time. It's not like the people are going
Speaker 1: to magically switch. And I think it's going to be really.
Speaker 3: Fair that we just have sixty seconds twenty years into
Speaker 3: the cloud journey. When I touched down in Vegas. Everyone
Speaker 3: accepts AWS number one in terms of scale infrastructure. They question,
Speaker 3: is AWS number one in AI? Just in the thirty
Speaker 3: seconds we have left? Yeah, I think I'll give it.
Speaker 1: It's a question that we got a lot two years
Speaker 1: ago and not that much a year ago, and today
Speaker 1: I don't think we get that nearly as much. It's
Speaker 1: just people that are kind of playing the same tapes.
Speaker 1: We have a huge choice of models. We see when
Speaker 1: customers are actually moving their workloads to production, they want
Speaker 1: to run those AI workloads on AWS, and that to
Speaker 1: me is the biggest signal. When we see our customers
Speaker 1: they say, I ran proof of concepts in a lot
Speaker 1: of places. When I want to move to production, I
Speaker 1: want to run on AWS. And that's the thing that
Speaker 1: we hear over and over again, which makes me think
Speaker 1: we're actually in a great position.
Speaker 3: Matt Garman, AWS, CEO with the Full Stack AI Company,
Speaker 3: pitch here Vegas at reinvent