NVIDIA: At the Heart of the AI Boom

Pushkin.

Just a quick note at the start of the show. We are very eager to hear what you think of the show and how we can make it better, how we can make a show that you like more than the show we're making.

Now.

Tell us what we should do, more of what we should do, less of what we should do, wildly differently. Email us at problem at pushkin dot fm. Again problem at pushkin dot fm. I'm looking forward to reading every email. Artificial intelligence feels very abstract, feels very ephemeral, feels more like an idea than a thing. But there is in fact a thing there. AI is rooted in the physical world. The thing or things are these little pieces of silicon

In the world.

I'm Jacob Goldstein, and this is what's your problem. My guest today is take him. He's a staff writer at Baron's and the author of a new book about in Nvidia. The books called The Nvidia Way. The book, of course, points up being a lot about Jensen Wong, who co founded the company back in nineteen ninety three and who is still the CEO and who is really a wild

The key moment was with Mark Harris, where he was a researcher at the University of North Carolina, and him and a lot a bunch of other academics realized there's all this computing power that you can hack the algorithm to use that computing power to model thermodynamics of fluids inside clouds, which was his thesis.

And he realized that worked better than the CPU than just sort of doing it the traditional way through the computes.

Yeah, and a GPU has all these different cores that run in parallel, and the CPU only typically has four to eight cores, and GPUs have hundreds of thousands of cores.

What's a CPU better at than a GPU? Like, why would you have four to eight when you could have thousands?

Well, it could run more complicated pieces of instructions and software, and GPUs typically break down into much simpler tasks. But the difference is you could run all those tasks across a thousands So for certain workloads like AI, it's so much faster than a CPU, which has to do things kind of one after another serially. And Mark Harris and all these academics started hacking into the graphics algorithms and using that power to do all this scientific high performance computing.

Yeah, that's like your dream if you're a company. Right, It's like, oh, there's all these other things you could do with this thing we're already.

Making exactly so, and Jensen kind of had the foresight to see this, Wow, this is a really big deal. We should invest in this. So they wind up hiring Mark Harris and look at all the things that all these researchers are doing. And this is the beginning of development Kuda, which is a programming platform for general purpose GPU computing.

And just to be clear, Kuda is like, it's a programming language, but in this case they're programming what used to be just the thing for graphics, but it turns out to be good for these other things. And that language it's in video's own language. Right, This is I feel like, going to be important in the sort of business story for why and video is so dominant today, right, Like they write this language, they own the language, and it's written just to work with their chips, is that right?

Yes, no one else can use this. It's extensions on a programming language that makes parallel computing easier for programmers to make. So Jensen just invests in this, and he actually thinks longer term than the average CEO. So most CEOs looking out maybe the next quarter, maybe a year or two. Jensen thinks in five, ten, fifteen year increments. So he's always looking out, what's the next big computing shift,

And when you say wasting all this die space, like de space is like space on the chip, right, Like he's making a physical thing and to commit to Kuda to have this dedicated programming language, you actually have to give up space on the chip. So that's it's costly, right, Their margins go down. They're not optimizing for profits at that moment.

It's literal circuits called Kuda cores. Their hardware circuits are optimized to run the Kuda language. So even internally executives are like, why are we spending all this money allocating dye space for something that people really aren't using that much, that isn't generating revenue. But Jensen saw this as a future. I actually kind of bring up the analogy of read

Till chat GPT. Basically twenty twenty two is chatchipt That's when in video goes totally bonker.

And that was the power of the large language model of AI and all that stuff. And he's investing throughout this whole thing for ten fifteen years.

Yeah, so there is a moment. There's a moment in the middle there. I do obviously want to get to the chat GPT moment and the present, but there is one more moment I feel like in the in the middle, that is where in video is in the center of it. And that's twenty twelve, right, So you have early you know, two thousand and one, people realize, oh, you can pack the GPU to do other things. Andybody's like, oh, that's interesting,

So alex Nett was a program that was created by two researchers at the University of Toronto and they competed in this competition called image net, which basically fed images into the model, and they were able to recognize and category rise image much more effectively than any other model in the past. And the breakthrough was they used GPUs for the first time and.

It was like just a few, right, Like they were grad students and they got their hands on like a few in video GPUs and had them like running on servers in the hallway or something.

Yes, right, and these are video game GPUs. To be clear, this isn't some enterprise complicated. They literally went to a store and bought a bunch of video game GPUs and it turned out to be very effective where they could do the power and effectiveness of two thousand CPUs on only twelve GPUs.

And to be clear or that kind of vision AI, it's machine learning. It's the same basic technology as is used in large language models. Right, it's it's modern AI. And so that moment is this moment when it's like, oh, holy shit, GPUs are one hundred x better than CPUs for AI. Yes, that's interesting, that's useful to know.

So it was a combination of GPUs data and algorithms that kind of combusted at that perfect moment. And then Jensen saw this and it's like this is a big deal and someday AI is going to be huge and we need to invest big in this. So on top of KUDA, he invests in all these AI libraries that effectively managed the GPUs to do AI workloads the most

And so that's that's more of the like it's not just the chip, right, it's like the chip that is optimized to be not only efficient on the hardware level. But like there's all this software. So if you are if you are building AI, if you're if you're writing code for AI, like they make it really easy for you to do it on in video GPUs.

So it's all the math libraries, it's all the AI frameworks work the best on in video GPUs. And they added these things called tensor cores similar to krudercres. They actually added hardware circuitry inside the GPUs that are optimized to run AI training and workloads and all the all the software that you're running. And this is over again, this is over another nine eight years where they're building all the software and hardware circuitry to run deep learning AI the best.

Okay, it's time. It's twot twenty two. Twenty twenty two, chat GPT comes out. What's that mean for in video?

So actually check chip takes off and it's it takes about two quarters for the whole world to realize this is a big deal. This, this model of natural language processing where the computer actually understands what you're asking, and that ability to draw insights and be effective with all that natural language stuff just just blows people away, and companies and startups realize we have a new AI boom here, because it really unleashes a wave of capabilities that wasn't

In like a day. Yes, right, there was one day, and it was spring of twenty twenty three. Way talking about and.

This is three, this is three weeks after I had the meeting with the book publisher who asked me if I could do a book on video. So literally I started.

Also good timing for you, good timing for a video, Good timing for you, And and right, no, I remember that day. And and there was a question then of like, oh my god, is this a one off? Are they going to keep growing this way?

Right?

Basically what they said was we sold way way more GPUs than we thought we were gonna. Right, that's that's what That's the basic thing they reported in their earnings report and the subtext is this is the world and in particular, like big tech companies with lots of money realizing, oh, we've got to get into this AI game more than we have been and to do that, we got to buy a lot of really expensive GPUs from video.

And Jensen is really smart at selling this stuff. Like, he's very smart because every company, every company CEO, every startup, they saw this risk as the extential for them because if your competitor comes out with the AI featured offering that's much better than what you have today, that that isn't advantaged by the AI models, Yeah, that competitor could could drive you out of business. So Jensen was very smart as at selling to people that you need to get on board.

At making everybody scared of losing. Yes, well, and in particular, right, I mean there is a small number of very large, very rich tech companies that are a very big part of in videos revenue. Right, it's like whatever the ones you quld think of, Google, Meta, Microsoft, Amazon, Yes, I mean, I guess open ai sort of is kind of connected with Like those companies are buying a huge share of these GPUs, right, They account for a big, big chunk

But that's being said, they're reselling that GPU power to companies and startups, right, so startups are buying that GPU competing capacity from these.

GNT tech companies. Yeah, that's a good point, right. So they're in a way not the end customer. They're the sort of intermediate kind of service providers.

But they also use that for their own internal systems too, like Meta uses a ton of GP power to make their advertising algorithms more effective and to pick the videos like tech talk does.

And obviously Google Search is now becoming more and more AI driven, and there's Gemini, which is right, I mean, there's there's more direct use of AI by all of the big tech companies as well. So there's another piece of this which is interesting. I mean, because there's one universe wherever it's like, oh, we got to get it on AI, and they all buy the GPUs and then that's kind of it. It's like a step function where like there's this momentary rush where everybody buys the GPUs

So the scaling hypothesis, similar to what I said before, is the combination of computing power. The more computing power you add, the more data you increase, and the better ways you figure out software algorithms for each of these three buckets. If you increase it, the AI model becomes more capable and more effective.

And like, just to be clear, like even if the algorithms aren't getting that much better, right, Like my understanding is, and this is kind of a new idea. It's like, oh, if we can just have more data and more computing power will get better results.

Yes, so it's great if you have all three doing you know, basically showing up. But for this time period, the last few years, the scaling law has really taken off and companies are literally ten xing their compute power, and these AI clusters going from sixteen thousand GPUs now to one hundred thousand gps and now people are saying they're going to build one million GPU clusters in the next two three years.

And in videos selling most of those GPUs.

Yes, So if you go from sixteen thousand to one hundred thousand GPUs in a couple of.

Years, that's a good business to do.

If you're increasing the scale of the hardware by ten ten x every couple of years. And the exciting thing for Nvidia is that, like what I argue is that this thing is going to this trend is going to continue in the next few years because you have the scaling laws. Then you have this thing called multimodal where they're using this GPUs not just for texts like in chat TPT, you're using it for video and images generating those things. And now there's these two other two other

Right, you could be like book me a ticket to San Diego some weekend in June whenever it's the best deal, yes, and then it does that.

So that's going to happen in the next twelve months. Yes, these AI agents will eventually, in the next year or two be able to do all that tedious work automatically and probably with less errors than an actual human being. So AI agents multimodal. And now there's this thing called test time compute, where the AI models, instead of just spinning back an instant apply, can actually think about what's the best way to respond to your question and spend more time thinking about it and then give you a

We'll be back in a minute and we will talk, among other things, about Jensen Wong, one of the most successful entrepreneurs of the twenty first century and a man who once told the colleague that he wakes up every morning, looks in the mirror and says to himself, you suck. I want to talk about Jensen a little bit. I mean your book. You know, he is obviously the main character in the history of Nvidio is the main character

He was born in Taiwan, and they moved around a little bit to Thailand, and his father came to training in New York and fell in love with America. This is like the great American dream story. And the mother and father started teaching Jensen's brother English ten words a day, and they sent him to his aunt and uncle when he was about age eight or nine, and the aunt

He just thought it was a boarding school, yes.

But it turned out to be a school for troubled.

Kids, huh, which Jensen was not. He was just like a smart kid with ambitious for him parents.

Yes, But he talks about how his time at Oneita Baptist and student Kentucky was formational for him.

What was it like for him?

It was hard at the beginning, but he started befriending people. He started playing chess with the janitor, and he just learned how to deal with other people much better. And he talks about how he learned his street fighter.

Mentality because he was literally fighting.

Yes, I mean there were bullies and all that stuff, but he just learned how to deal with people and the rough and tumble of kids back then. Eventually his parents came from abroad and they settled in Oregon, and he learned his work ethic from working at Denny's. He talks about how he washed more dishes and cleaned more bathrooms than any CEO in the history of CEOs. He says he Denny's helped them in terms of social skills and dealing with time pressure and dealing with customers. But

And I feel like at the beginning that's common, right, Like that is the classic startup story. But the classic startup stories you do that for five years and then either you get giant and you hire a grown up CEO and you you know, go start your blimp company or whatever, or you sell or whatever. He's still doing that, right He's sixty years old and like wildly rich and and he's still working that much.

He's working all weekend. He's working Saturday Sunday. When he talks about when he goes to a movie, he doesn't remember the movie because he's thinking about work. He finds work relaxing and fund This is what drives him. He loves working.

There's a moment, you know, you don't you don't write about his personal life at all. Once he sort of grows up and starts in video. You know, reasonably because the book's about in video, and so I just assumed he just worked all the time and didn't have a family. And the I think the only time in the book that his family comes up is there's this scene where he's on vacation and he's talking to some senior manager at a video on the phone and they're like, what

So both executives told me they hate it when Jensen so called goes on vacation because he winds up giving them more directives and orders and more stuff to do when he's not on vacation because he's emailing in them do this, do that, and they they actually yell at him, So play with your kids. He's like, no, I could get real work done when I'm on vacation by doing his emails. I mean, so it shows you his obsessions. He's constantly thinking, he's constantly worried about what's happening at video.

I mean it's interesting, right, Like he's not a balanced person, like, which is why to some extent he has built the thing that he has built.

Right, He's completely obsessed with winning and he's extremely competitive.

Yeah, there's a few other specific moments that really stood out to me. There's one where I think this, I think a salesperson told you this where they had just had a great quarter they sold a ton of GPUs, and the sales guy's talking to Jensen about how great they're doing, and Jensen says to the sales guy about Jensen, about himself. Jensen says, I look in the mirror every morning and say you suck.

So he he's almost like a self psychologist. He knows if you start thinking that you're you're the best and your your hot stuff, you might get complacent, you start resting on your laurels, you might not work as hard enough. So he he sees that, oh, we just had a blowout quarter. What's the risk here? The risk is me gain complacent. So I'm gonna look myself in the mirror and say you suck and psych myself out.

I mean that's that's that's one reading of it. That's the like ten dimensional chest reading of it. I mean there's another reading of it, which is like he's messed up in a way that makes him super driven and successful.

But it works.

He actually definitely works. We can agree that it works.

Like he does the opposite too, So when things are really intimidating and he feels like, oh my gosh, how am I going to do this, he tells himself, how hard can it be? How hard can it be? So he does it on both both ends in the spectrum where he cites himself out when he feels intimidated, and when he feels like he's on top of the world, he tries to bring himself down back the earth.

So the one other big piece of the Jensen Wong experience that we haven't talked about is the way he treats the people who work for him. Right, I want to talk about a particular scene because I think it puts a finer point on it. A scene from the book where it's like a company wide meeting and it's on like zoom or whatever, and he's yelling at a guy in the meeting. He's yelling at him, and then on top of that, he keeps telling the person filming the meeting to zoom in on the guy he's yelling at.

I had multiple sources tell me that it was the most humiliating thing they've ever seen.

I mean, it's that seems like bullying, like why zoom in on the guy? Like what lesson is added by that?

So he keeps on saying to mister Rayfield, you got to get this chip back on track. The chip that heels in charge was behind schedule, and he kept us pointing to him, zooming in his face and telling him, You've got to get this. This is not how you run a business. You need to get this chip back on track. And this kind of like high standard demanding

Maybe I don't want the world to be that way, but it is that way, you know what I mean, Like, maybe I want to believe that, like there's a kinder world where people could do equally good work. But maybe I'm wrong.

And Steve Jobs was the same way. Yeah, I mean it wasn't all sure, It wasn't all sunshine and rainbows.

No, no, I mean the Isaacson book was really clear, right, the Isaacson biography of Steve Jobs. He did not seem like a good person in that book. He seemed like really good at building amazing products, but like not like a good human being by the sort of standard way we think of what makes a good person.

But I think the key point that we also have to think about is people at Nvidia stay.

Yeah, that's really interesting.

The turnover is like one of the lowest in the industry at only three percent compared to the average of thirteen to fifteen percent.

Yeah, certainly in the last few years. People stay because you get rich if you stay, and you lose your equity if you leave. But is that number true if you go back to when the stock was flat.

I don't have the numbers, but a lot of the senior executives have been there more so than any other company. Yeah, fifteen twenty twenty five years.

Yeah, that's it.

You don't really hear other than one instance of people leaving to become a CEO of a different company. Uh huh. And I think part of it is people realize that Nvidia or the Gensen way of doing things is super effective, so people want to be on the winning team. So in one sense, he draws people hard, he dresses them down. He's very blunt and direct. But he compensates people. No one ever, hardly anyone complains about compensation. If you're effective and you do a good job, he will like double

Yeah. Yeah, So you interviewed Jenson as as you were working on the book. Was it scary to interview him?

He was intimidating in the meeting, he he at he didn't yell at me, but a couple of times he's saying, you don't get nvidio in this line of questioning, and I it takes a while to get used to it, but you appreciate it because you see where the other person is. Like when someone is blunt and direct like Jensen, you know where that where he is at all times, and then you could work at improving yourself. So he

You need if it's a person's.

Yeah, Jensen's like, why is Bob the only one that gets a learning here? If I am blunt and direct and show Bob what he's doing incorrectly, like that person at that meeting, everyone in the room can learn. All the employees up and down the ladder can learn. So that's his philosophy is everyone should learn from their mistakes and everyone should know where they are at all times.

So I mean, so you write in the book that in video really is like an extension of Jensen, right. I think he used the metaphor of like the formula one card that's like optimized for him as the driver. So and he's in his sixties, which is not old old but definitely not young. Like, is he gonna run in video for another twenty years? What's going to happen? Like there's no obvious successor, Like where does that go?

I think there's there's no one else at in video. I think that can run in video as effectively as Jensen. And he loves the company so much. He loves what he's doing. They're having enormous impact with this AI wave, and he's so excited about the potential for curing cancer and digital biology, the potential for robots, the potential for AI to kind of disrupt education and help kids learn better. That I don't see him leaving anytime soon. He loves his job so much and he can't argue that he's

I mean people also talk about limits to scaling right in AI. We were talking earlier about how part of in video's wild growth over the last couple of years has come from this fact that that you can just add more GPUs basically and get better results. More GPUs, more data, and you know, people talk about a running out of data because like they're basically, as I understand it, training on the whole Internet right now, which is like, Okay, it's a lot of data. I mean, is that it

I think it's an open question. But like I said, there's the multimodal stuff, there's AI agents, and then there's proprietary data inside companies.

Uh huh.

So all these corporations have data going back decades. Companies are going to use the power of these AI computing systems to go through all their data internally, all their proprietary data, and have all that knowledge at employees fingertips. So an employee can ask what's the best way to do this, and the AI computer is going to be able to go back thirty years of data and figure out the best piece of insight to help that employee. And that's not really being done today.

So Nvidia designs their chips their gips, but they don't actually manufacture them, right Is it right that they're made in Taiwan, where most cutting its chips are made now?

Yes, So TESMC actually makes and manufacturers in videos chips ever since the late nineteen nineties, and they're the best at doing this all. A lot of the fabulist chip designers in California use TSMC, and Video uses them too.

And so I mean it is really interesting to me to a lot of people that Taiwan is this incredibly fraught geopolitical place. Right, China says Taiwan is part of China. Taiwan says no, we are not part of China, and that Taiwan is the only place in the world that makes like the most important physical thing in the world of technology today. Right, it makes like that's wild, Like what do you make of that? And how does that fit with the in video story?

I think eventually Nvidia chips will be made in the US TSMC. I mean, that's all part of this Chips Act that the Biden administration has posed.

So this this plant that TSMC is building in Arizona, Like, is that advanced enough to make like frontier in Nvidia chips?

So the CSMC factories in the US are always going to be one or two generations behind the factories in Taiwan. But that doesn't mean Nvidia can't use the factories in the US if they're one or two generations behind for their older chips. So I think that will eventually happen.

It seems wild to me, Like, I mean, it seems very possible that China will try and make Taiwan be part of China, right, and and like we have all these export controls to try and keep China from getting cutting edge chips. Like it's a really interesting, complicated dynamic.

I think people make a big issue at this, But I think if it happens, like in Vidio, chips are not going to be the main issue. It will cost a global calamity where you know, we won't be able to upgrade our cars, at laptops, nearly every computing device, every appliance will not work if we don't have access to Taiwan chips.

I mean, presumably we could do some like sub Manhattan Project scale Manhattan project to get a state of the art TSMC factory somewhere that is not Taiwan, right, Like.

We're doing some of that now, but it's not going to be able to it's going to be ten twenty of the capacity we need.

Yeah, the capacity, right, it takes a long time, Like the fabs are wildly complicated to build, and they cost billions of dollars and they take years, so you couldn't just do it.

They cost ten to twenty billion dollars to build and three to four years over three to four years, So it's not something that can happen overnight. And if something happens, like it's just gonna be so terrible and it'll be like a depression. So I just hopefully it doesn't happen, and you know it's not gonna The question isn't gonna be about in videotips. The question is gonna be about the global economy of that.

So what do you think is the biggest risk for video in the like five year timeframe?

It's really tough to say.

Now.

I mean I see the next few years with all the different AI innovations and all the progress. But again, just like every big computing shift, what's the next big thing? Could it be quantum computing? It can be like who knows, five, ten, fifteen years from now? And every from the pcach with Microsoft and Intel it's called Wintel. And then it went to smartphones where Apple dominated with the iPhone, and now

We'll be back in a minute with the Lightning round. I want to finish with the Lightning round, which is going to be considerably more random and digressive than the conversation to this point. I want to talk about video games and your experience with video games. What was the first video game you ever loved?

I think it's the first VAM I've ever tried, which was Combat for the Datar twenty six hundred.

I love that.

I mean it's literally I remember coming home and my dad buying the Autari twenty six hundred connected to a black and white TV and being to use that joystick to control the little tank going around the screen. That was an amazing moment. It was an incredible moment that this was possible. At the home, I had.

An Atari twenty six hundred. I remember the joystick. I remember Demon Attack, so kind of second tier game, but I got really into it. What's the game you spent the most hours on?

Probably this game called Lemmings. I don't know if you it's it's a puzzle strategy game where you control lemmings and you guide them across a maze to get.

The little creatures that are famed for jumping off cliffs, which maybe they don't actually do.

Yes, if you guide them the wrong way, they'll fall off a cliff and to their death, and they make this cute sound like oh no, so so. I played so many hours of.

That game, one thousand hours.

Probably hundreds of hours. I want to say a thousand.

Yeah. What's your most proud video game accomplishment?

I still remember playing the Legend of Zelda, the first one for the Nintendo Entertainment System and beating that game and just like pumping my fists in the air. I was over young man. Yeah.

What is the most exciting video game innovation that's going to happen in the next few years?

Well, this is a little technical, but I think DLSS from the video is going to get even more powerful, which is this upscaling technology the gents actually invented in a meeting where they fill in the details using AI, so the frame rates are able to go much faster and better.

So it's basically like interpolation. It's sort of doing what AI AI always does, yes, fort.

But that's really effective. People can't tell the difference between the real thing and the interpolated AI graphics, so that will allow as this technology get better, that will allow the graphics to be more for the realistic, and the physics and the rate tracing better than ever before.

What game are you most excited to play in twenty twenty five?

I mean Grand Theft Auto six if it comes out, but it might get delayed.

Next here, Okay, let's do a few non video game questions. What's your favorite tech book besides the one you wrote.

I guess it's still a tech book, The Innovator's Dilemma, which is actually one of Jens's here books. It talks about how companies get disrupted by startups and people underneath them. It really goes into in depth about how the distrived industry. Every successive generation that distribes there was a new market leader because they couldn't disrupt themselves to the new or smaller format.

I mean, to me, the really interesting insight of that book is the innovator is actually making a crappier product, right, Like that's the surprise. It's not exactly like they come along and do something better. They go to the crappy end of the market and they make a cheap product and it's not better than the thing the market leaders and the market lader' is like, oh that, who cares about that? That's just some low market, some low margin thing,

And they scale the volumes, and once you have the volume, it pretty much becomes game over for the incumbent because they can't match that scale and the economies of scale that comes with that.

Defend pineapple on pizza.

It just tastes good. I love it. I don't know if you saw my tweets on it. I actually like pineapple on pizza.

Yes, I think it was Instagram. I think it was Instagram. What's the best deal you ever got at Costco?

It's still the hot dog and you can't beat the dollar.

Fishbee Take Him is the author of the Nvidia Way. Today's show was produced by Gabriel Hunter Chang. It was edited by Lyddy Jean Kott and engineered by Sarah Bruguer. You can email us at problem at Pushkin dot FM. I'm Jacob Goldstein and we'll be back next week with another episode of What's Your Problem.