Bloomberg Audio Studios, podcasts, radio news.
It is a joy to be joined by Matt. Thank you, remain Matt. Welcome from Las Vegas, Reinvent upon You Chip, Ultra Clusters, Ultra Service Chip Development LLM, and we've got talk about developer tools. There is a lot to get through. Let's start with the ultra cluster. Can you just talk to us a little bit about this more than one hundred thousand chips that will be put in one single era for AI hardware. Why do it? What will it offer?
Yeah, this is something that we're building together with our partner's anthropic and it's actually going to be several hundred thousand chips, and they're focused on it's our new Trainium two chips, and so we expect that this should deliver them five times more compute than they've used in their last model training, and so we're really excited about what they're able to accomplish, and they're expecting to build much bigger and much more capable AI models from that large compute clubs.
Do we have a US location? Do we have an exact timeframe other than early twenty twenty five?
No, No, no location to share with.
Matt what then, of the technical challenges paint the picture for our audience how difficult this is in terms of cooling, in terms of energy use. What has been the things that you have to overcome with this?
Yeah, Well, the first core innovation is that we built our own chip actually, and so we've built our own chip. It's called Trainium two, and we're quite excited about the performance that we get out of that. And that's combined all in these really large ultra clusters that combine sixty four tranium chips all to deliver eighty three petaflops from a single node. And so that is the first innovation is our Amazon designed custom silicon that gives us really
unparalleled performance for generative AIA capabilities. And then we build these together with high performance networking that we also build in house, and then of course they'll be cooling and heating and power that we need to go to go build the data centers. But it really starts with that silicon down at the level, and we innovate on the entire stack of AI to make sure that we can control everything that goes into those clusters.
Can you talk to us a little bit compare and contrast here this is about less dependency on Nvidio GPUs too many ways, doing an alternative for your clients. What sort of cost saving will clients get? What sort of energy and efficiency will they get?
Yeah? Well, first of all, I think we like to think of it as a supplement to in video GPUs and Vidia has a fantastic product that team has done an outstanding job executing, and we think that the vast majority of workloads are going to continue to run in video processors for a long time. But customers want choice, and they want choice that can give them some lower cost options. And we think that for certain workloads, for many workloads, Trainium two can give customers thirty to forty
percent cost performance benefits over today's GPU powered instances. And so we think that that's a huge win for customers, particularly as they're looking to lower the cost of Generator I workloads. But we'll be great partners with Nvidia and continue to lean in on building great technologies together with them for a long time.
Train two, Trainium two available, TRAININGUM three in the works. And all of this is as Nvidia at the moment is estimated has about ninety five percent market shared. Do you agree with that sort of number they have and what sort of area do you think that that will come down to.
Yeah, I mean I think it's probably higher than that. I think that the vast majority of workloads in generative AI today run on in Nvidia technology and they've absolutely been the leaders in that space. But we do think we hear from customers that they want choice, and just with our processors where we're type partners with Intel and with AMD, but we decided to go build a general purpose processor are called Graviton, and it's been hugely successful
with our customers. But we also provide lots of Intel and lots of AMD processors in our cloud today and those businesses continue to grow. So I expect that our usage of Nvidia will continue to grow from our customers and that choice is really going to be powerful. And as you see this explosion of generative AI usage, I think there's going to be plenty of business for multiple different people to be successful.
I keep partners Intel. How concerned have you been about the change at the top.
Oh, it's okay. I wish pat the best. You know, he's I know Patent, he's been a good partner of ours, but we've been partners with Intel for a long time. It's been eighteen years since we launched our first Intel instance when AWS and EC two first launched, and will continue to be great partners with Intel, and they have a great technology team there, and we look forward to to continuing to roll out the latest technologies from Intel for our customers to be able to use, and.
He've been continuing to roll out, as you say in video offerings last time you're on, we're talking about Blackwell. Of course, many have been frustrated, perhaps by some slowness to market there. When do you anticipate that Blackwell will be unfolding? And how difficult has it been to ensure the supply side.
Is there.
Well.
They've obviously had some manufacturing things that they're going through, but we're very excited about it. I think the early
returns and the early looks at Blackwell look fantastic. We expect almost a two and a half time's gain in the compute power that you get from Blackwell that we saw from H one hundreds, and so I think it'll be a really material jump for customers once we get those out, and you know, I think those will be early next year, and we're excited about putting them into customer's hands, and we'll get them out there as soon as they're available.
Investors, though, they find it wild, this whole frenemy existence that's going on. Do you truly think that investors here think, oh, we want to see in video dependency as well as AWS having its own offerings. Is that something you think everyone can swallow? Or do you think ultimately there will be a broadening out other than just in video winning all.
Look, I do think it's a partner if you think about AWS. We started from the very beginning thinking about this partnership mindset, and we built the entire business around AWS thinking about how AWS would have services and our partners would have services, and that there's plenty of space for all of us to really grow and build our businesses. And that is true for software providers, true for service providers,
and it's true for technology providers. And so I think that we've proven time and time again over the last eighteen years that AWS can have products and our partners can have products, and then as we make them all available, that the whole, the whole pie gets bigger. And so I think there's plenty of opportunity for both and so it really isn't. I think it makes for a fun narrative that it's either or. But we're great partners with
n Video. We will continue to be, And this is all about making the pie get bigger.
We love fun narratives. US journalists talk about that pie. When it's to do with large language models. You are unveiling Nova, you're really saying that the Nova offering compares really well to other offerings out there when it comes to multimodi multimodal, but also from a cost perspective and efficiency perspective. Why invest so much in LLM provisions when you already offer the rest?
Yeah, and again, this is all about giving customers more choice. I think we as Amazon were investing in these models because we couldn't find the exact right mix of capabilities and customizations and things that we needed internally, so we started building them. And as we started building these, we saw that these models were actually getting quite good. The benchmarks were really good, and we were seeing some good capabilities come out of there. And our models are quite
good at some particular areas. They're very good at executing agentic workflows. They're really good at pulling knowledge out of a RAG index there and they're very very low latency and low cost, and so we think that that capability is going to be really compelling for a lot of use cases. We found use cases inside of Amazon, but even inside of Amazon, we use a mix of different models.
We use models from Anthropic, we use models from Meta and Lama, we use models from a bunch of different sources, and so I think that hopefully customers see a lot of value from these new Nova models, but we also expect that customers are going to combine lots of different models in lots of different ways.
You haven't yet got the access to open ai, and for obvious reasons, with their relationship with Microsoft, would you think that that will at some point unfold.
Look, I think my view is anytime customers demand something that eventually that sorts its way out. And I think open Ai obviously has a great set of models as well, and I'm sure many of our customers would love to have those available in Bedrock as well, and we'd love
to support those there. And you know, I think I'd look at and we have a long term view on this, and over the long term, we'd like to make every technology available inside of Amazon and inside of AWS to use, and that means all the models that are out there, we'd love to offer them at Bedrock. All the software that's out there, we'd love to offer and make sure that it's available inside of AWS. Every service that everyone else in third parties are providing we want to make available.
And that all goes back to that choice story. And so in the fullness of time, I believe that we would love open AI models inside of AWS. We absolutely it's you know, were we listen to our customers and if that's something the customers want, we're absolutely game. Obviously, with a partnership, it takes multiple sources and there's probably some complexities in there, as you allude to, but I think in the fullness of time it likely is the case.
In the here and now. You spelled out on the previous that AI is already a multi billion dollar business growing at more than one hundred percent. Can you give us any more detailed numbers as to how large AAAI offering is for AWS?
Now, You're right, it is a multi billion dollar business for us, and it's really growing rapidly. Take Bedrock, which is our platform that lots of customers are building all of their production AI applications on top of bedrock has grown the number of users five x in the last year alone, so it is really a rocket ship growth
of people building this. I think what's really exciting is that people are using bedrock not just to do proof of concepts, but they're using it to really deeply integrate with their own enterprise data and to launcher production applications, which I think is a particularly telling sign of where the technology is going, where.
It's evolved to and clients willing to spend it a time of a lot of uncertainty that we've got a new administration coming in, do you see any concerns in terms of well geopolitics, your road supply chain relationship with TSMC, for example.
No, I mean, I think, look, we've we've worked with various administrations all throughout the last eighteen years that AWS has been around, and we're really excited to work with
this administration. You know. I think that everybody is interested in ensuring that we have technology continuity and and there's a ton of opportunity here like that this is how we drive the economy, and I think all administrations are interested in continuing to make sure that we're able to drive the economy forward, and and AWS is a big enabler of that, and so we're excited to work with
the new administration. I think we're we're obviously anxiously watched all of the geopolitical events all around the world to make sure that we're trying to insulate our customers as much as possible from any eventuality. But but you know, I think we're we're cautiously optimistic that we're in a good spot.
You might have been cautiously watching what was happening in South Korea today amid your own event any exposure there.
Matt, Sorry I missed that I was on stage, so just I'm from the stage right here to talk to you.
So okay, I understood, But that in terms of the South Korean political destabilization, nothing that was a worry there.
No, I mean, look, we have a we have a region in South Korea, and we have many of our customers that operate in South Korea and and will continue to support them and and operate with with with any with ever whichever government is in operation.
We thank you so much for your time rushing off a stage A w S ce O Matt Garman