Speaker 1: Pushkin.
Speaker 2: Just a quick note at the start of the show.
Speaker 2: We are very eager to hear what you think of
Speaker 2: the show and how we can make it better, how
Speaker 2: we can make a show that you like more than
Speaker 2: the show we're making.
Speaker 1: Now.
Speaker 2: Tell us what we should do, more of what we
Speaker 2: should do, less of what we should do, wildly differently.
Speaker 2: Email us at problem at pushkin dot fm. Again problem
Speaker 2: at pushkin dot fm. I'm looking forward to reading every email.
Speaker 2: Artificial intelligence feels very abstract, feels very ephemeral, feels more
Speaker 2: like an idea than a thing. But there is in
Speaker 2: fact a thing there. AI is rooted in the physical world.
Speaker 2: The thing or things are these little pieces of silicon
Speaker 2: and metal called graphics processing units GPUs. These GPUs are expensive.
Speaker 2: They cost thousands of dollars each. If you want to
Speaker 2: build a state of the art AI model, you have
Speaker 2: to buy tens of thousands of these GPUs, and most
Speaker 2: of the GPUs come from a single company, in Vidia,
Speaker 2: which is why in just the past few years Nvidia
Speaker 2: has become one of the most valuable, most important companies.
Speaker 1: In the world.
Speaker 2: I'm Jacob Goldstein, and this is what's your problem. My
Speaker 2: guest today is take him. He's a staff writer at
Speaker 2: Baron's and the author of a new book about in Nvidia.
Speaker 2: The books called The Nvidia Way. The book, of course,
Speaker 2: points up being a lot about Jensen Wong, who co
Speaker 2: founded the company back in nineteen ninety three and who
Speaker 2: is still the CEO and who is really a wild
Speaker 2: kind of terrifying, brilliant figure. And Tay and I talk
Speaker 2: a lot about Jensen later in the interview, but we
Speaker 2: started with this moment in two thousand and one that
Speaker 2: led from Nvidia being a company that made graphics cards
Speaker 2: that let people play video games on computers to becoming
Speaker 2: this key company at the center of the AI revolution today.
Speaker 2: So you write about this moment in two thousand and
Speaker 2: one when a researcher, right, an academic at the University
Speaker 2: of North Carolina, realizes he can basically like hack these
Speaker 2: graphics cards, right, this hardware that's just made to make
Speaker 2: pretty graphics on the computer to help in his research,
Speaker 2: which is like not at all about graphics, right, It's
Speaker 2: like modeling the weather or something. And it seems like
Speaker 2: this moment is a big winds up being a big
Speaker 2: turning point in the history of technology. Really tell me
Speaker 2: about that moment.
Speaker 1: The key moment was with Mark Harris, where he was
Speaker 1: a researcher at the University of North Carolina, and him
Speaker 1: and a lot a bunch of other academics realized there's
Speaker 1: all this computing power that you can hack the algorithm
Speaker 1: to use that computing power to model thermodynamics of fluids
Speaker 1: inside clouds, which was his thesis.
Speaker 2: And he realized that worked better than the CPU than
Speaker 2: just sort of doing it the traditional way through the computes.
Speaker 1: Yeah, and a GPU has all these different cores that
Speaker 1: run in parallel, and the CPU only typically has four
Speaker 1: to eight cores, and GPUs have hundreds of thousands of cores.
Speaker 2: What's a CPU better at than a GPU? Like, why
Speaker 2: would you have four to eight when you could have thousands?
Speaker 1: Well, it could run more complicated pieces of instructions and software,
Speaker 1: and GPUs typically break down into much simpler tasks. But
Speaker 1: the difference is you could run all those tasks across
Speaker 1: a thousands So for certain workloads like AI, it's so
Speaker 1: much faster than a CPU, which has to do things
Speaker 1: kind of one after another serially. And Mark Harris and
Speaker 1: all these academics started hacking into the graphics algorithms and
Speaker 1: using that power to do all this scientific high performance computing.
Speaker 1: And Mark Harris started a website and congregated all these
Speaker 1: researchers and everyone's sharing the knowledge, sharing the tricks that
Speaker 1: actually hack into this. So Nvidia sees this, they're seeing
Speaker 1: people use this to do model stock options, the model
Speaker 1: weather and saying, wow, this is actually really interesting.
Speaker 2: Yeah, that's like your dream if you're a company. Right,
Speaker 2: It's like, oh, there's all these other things you could
Speaker 2: do with this thing we're already.
Speaker 1: Making exactly so, and Jensen kind of had the foresight
Speaker 1: to see this, Wow, this is a really big deal.
Speaker 1: We should invest in this. So they wind up hiring
Speaker 1: Mark Harris and look at all the things that all
Speaker 1: these researchers are doing. And this is the beginning of
Speaker 1: development Kuda, which is a programming platform for general purpose
Speaker 1: GPU computing.
Speaker 2: And just to be clear, Kuda is like, it's a
Speaker 2: programming language, but in this case they're programming what used
Speaker 2: to be just the thing for graphics, but it turns
Speaker 2: out to be good for these other things. And that
Speaker 2: language it's in video's own language. Right, This is I
Speaker 2: feel like, going to be important in the sort of
Speaker 2: business story for why and video is so dominant today, right,
Speaker 2: Like they write this language, they own the language, and
Speaker 2: it's written just to work with their chips, is that right?
Speaker 1: Yes, no one else can use this. It's extensions on
Speaker 1: a programming language that makes parallel computing easier for programmers
Speaker 1: to make. So Jensen just invests in this, and he
Speaker 1: actually thinks longer term than the average CEO. So most
Speaker 1: CEOs looking out maybe the next quarter, maybe a year
Speaker 1: or two. Jensen thinks in five, ten, fifteen year increments.
Speaker 1: So he's always looking out, what's the next big computing shift,
Speaker 1: what's the next big technology phase. So he saw KUDA
Speaker 1: as the main thing where eventually his GPUs, you have
Speaker 1: all this latent, enormous computing power will be able to
Speaker 1: be used for science research, all these other simulation things.
Speaker 1: And he didn't give up even when Wall Street wents
Speaker 1: down on his next day, Why you wasting all this
Speaker 1: die space is crushing your gross profit margins? He said, no,
Speaker 1: this is the future computing. I'm going to invest in
Speaker 1: it and it will work out someday.
Speaker 2: And when you say wasting all this die space, like
Speaker 2: de space is like space on the chip, right, Like
Speaker 2: he's making a physical thing and to commit to Kuda
Speaker 2: to have this dedicated programming language, you actually have to
Speaker 2: give up space on the chip. So that's it's costly, right,
Speaker 2: Their margins go down. They're not optimizing for profits at
Speaker 2: that moment.
Speaker 1: It's literal circuits called Kuda cores. Their hardware circuits are
Speaker 1: optimized to run the Kuda language. So even internally executives
Speaker 1: are like, why are we spending all this money allocating
Speaker 1: dye space for something that people really aren't using that much,
Speaker 1: that isn't generating revenue. But Jensen saw this as a future.
Speaker 1: I actually kind of bring up the analogy of read
Speaker 1: Hastings and Netflix. So when he started Netflix, you know,
Speaker 1: the technology wasn't ready, consumers didn't really have broadband, but
Speaker 1: he had this intuitive sense that someday video will be
Speaker 1: streamed over the Internet and that was the future, and
Speaker 1: it makes it's so obvious now, but back then it wasn't.
Speaker 1: So Hastings positioned Netflix. He made money with DVD rentals
Speaker 1: for a while and just stayed on top of the technology,
Speaker 1: kept on investing in investing, and when the technology was
Speaker 1: good enough, that's when he really pivoted Netflix to dominate
Speaker 1: Internet stream Jensen did this multiple times with three D graphics,
Speaker 1: video game graphics. He knew that someday PC video games
Speaker 1: would be a big market programmable GPUs, Kuda and later
Speaker 1: with AI on these full stack data center AI servers.
Speaker 1: He just sees the future and is willing to keep
Speaker 1: investing even if it's five ten years out. And Kuda
Speaker 1: started in two thousand and six. I mean, things got
Speaker 1: incrementally better for the next ten years, but it didn't
Speaker 1: really take off till twenty twenty two.
Speaker 2: Till chat GPT. Basically twenty twenty two is chatchipt That's
Speaker 2: when in video goes totally bonker.
Speaker 1: And that was the power of the large language model
Speaker 1: of AI and all that stuff. And he's investing throughout
Speaker 1: this whole thing for ten fifteen years.
Speaker 2: Yeah, so there is a moment. There's a moment in
Speaker 2: the middle there. I do obviously want to get to
Speaker 2: the chat GPT moment and the present, but there is
Speaker 2: one more moment I feel like in the in the middle,
Speaker 2: that is where in video is in the center of it.
Speaker 2: And that's twenty twelve, right, So you have early you know,
Speaker 2: two thousand and one, people realize, oh, you can pack
Speaker 2: the GPU to do other things. Andybody's like, oh, that's interesting,
Speaker 2: let's build a whole programming language. So people didn't do that,
Speaker 2: And then in twenty twelve, you have this moment that
Speaker 2: really seems like the birth of modern AI. Right, this
Speaker 2: moment when when this AI model called alex net sort
Speaker 2: of emerges into the world and everybody in the AI
Speaker 2: world is like, oh my god, AI is here, right,
Speaker 2: tell me about that moment.
Speaker 1: So alex Nett was a program that was created by
Speaker 1: two researchers at the University of Toronto and they competed
Speaker 1: in this competition called image net, which basically fed images
Speaker 1: into the model, and they were able to recognize and
Speaker 1: category rise image much more effectively than any other model
Speaker 1: in the past. And the breakthrough was they used GPUs
Speaker 1: for the first time and.
Speaker 2: It was like just a few, right, Like they were
Speaker 2: grad students and they got their hands on like a
Speaker 2: few in video GPUs and had them like running on
Speaker 2: servers in the hallway or something.
Speaker 1: Yes, right, and these are video game GPUs. To be clear,
Speaker 1: this isn't some enterprise complicated. They literally went to a
Speaker 1: store and bought a bunch of video game GPUs and
Speaker 1: it turned out to be very effective where they could
Speaker 1: do the power and effectiveness of two thousand CPUs on
Speaker 1: only twelve GPUs.
Speaker 2: And to be clear or that kind of vision AI,
Speaker 2: it's machine learning. It's the same basic technology as is
Speaker 2: used in large language models. Right, it's it's modern AI.
Speaker 2: And so that moment is this moment when it's like, oh,
Speaker 2: holy shit, GPUs are one hundred x better than CPUs
Speaker 2: for AI. Yes, that's interesting, that's useful to know.
Speaker 1: So it was a combination of GPUs data and algorithms
Speaker 1: that kind of combusted at that perfect moment. And then
Speaker 1: Jensen saw this and it's like this is a big
Speaker 1: deal and someday AI is going to be huge and
Speaker 1: we need to invest big in this. So on top
Speaker 1: of KUDA, he invests in all these AI libraries that
Speaker 1: effectively managed the GPUs to do AI workloads the most
Speaker 1: in the most effective manner. And he invested in libraries,
Speaker 1: invested in software, he invested in researchers, and allocated a
Speaker 1: lot of employees to this project starting in around twenty thirteen.
Speaker 2: And so that's that's more of the like it's not
Speaker 2: just the chip, right, it's like the chip that is
Speaker 2: optimized to be not only efficient on the hardware level.
Speaker 2: But like there's all this software. So if you are
Speaker 2: if you are building AI, if you're if you're writing
Speaker 2: code for AI, like they make it really easy for
Speaker 2: you to do it on in video GPUs.
Speaker 1: So it's all the math libraries, it's all the AI
Speaker 1: frameworks work the best on in video GPUs. And they
Speaker 1: added these things called tensor cores similar to krudercres. They
Speaker 1: actually added hardware circuitry inside the GPUs that are optimized
Speaker 1: to run AI training and workloads and all the all
Speaker 1: the software that you're running. And this is over again,
Speaker 1: this is over another nine eight years where they're building
Speaker 1: all the software and hardware circuitry to run deep learning
Speaker 1: AI the best.
Speaker 2: Okay, it's time. It's twot twenty two. Twenty twenty two,
Speaker 2: chat GPT comes out. What's that mean for in video?
Speaker 1: So actually check chip takes off and it's it takes
Speaker 1: about two quarters for the whole world to realize this
Speaker 1: is a big deal. This, this model of natural language
Speaker 1: processing where the computer actually understands what you're asking, and
Speaker 1: that ability to draw insights and be effective with all
Speaker 1: that natural language stuff just just blows people away, and
Speaker 1: companies and startups realize we have a new AI boom here,
Speaker 1: because it really unleashes a wave of capabilities that wasn't
Speaker 1: wasn't doable before. So about six months later, that's when
Speaker 1: the big bang I call it for Nvidia happens when
Speaker 1: they say, oh my gosh, we're gonna we're gonna beat
Speaker 1: our numbers that the street expects by four billion up.
Speaker 1: And the stock went up like one hundred and seventy
Speaker 1: billion dollars in value because the world realized.
Speaker 2: In like a day. Yes, right, there was one day,
Speaker 2: and it was spring of twenty twenty three. Way talking
Speaker 2: about and.
Speaker 1: This is three, this is three weeks after I had
Speaker 1: the meeting with the book publisher who asked me if
Speaker 1: I could do a book on video. So literally I started.
Speaker 2: Also good timing for you, good timing for a video,
Speaker 2: Good timing for you, And and right, no, I remember
Speaker 2: that day. And and there was a question then of like,
Speaker 2: oh my god, is this a one off? Are they
Speaker 2: going to keep growing this way?
Speaker 1: Right?
Speaker 2: Basically what they said was we sold way way more
Speaker 2: GPUs than we thought we were gonna. Right, that's that's
Speaker 2: what That's the basic thing they reported in their earnings
Speaker 2: report and the subtext is this is the world and
Speaker 2: in particular, like big tech companies with lots of money realizing, oh,
Speaker 2: we've got to get into this AI game more than
Speaker 2: we have been and to do that, we got to
Speaker 2: buy a lot of really expensive GPUs from video.
Speaker 1: And Jensen is really smart at selling this stuff. Like,
Speaker 1: he's very smart because every company, every company CEO, every startup,
Speaker 1: they saw this risk as the extential for them because
Speaker 1: if your competitor comes out with the AI featured offering
Speaker 1: that's much better than what you have today, that that
Speaker 1: isn't advantaged by the AI models, Yeah, that competitor could
Speaker 1: could drive you out of business. So Jensen was very
Speaker 1: smart as at selling to people that you need to
Speaker 1: get on board.
Speaker 2: At making everybody scared of losing. Yes, well, and in particular, right,
Speaker 2: I mean there is a small number of very large,
Speaker 2: very rich tech companies that are a very big part
Speaker 2: of in videos revenue. Right, it's like whatever the ones
Speaker 2: you quld think of, Google, Meta, Microsoft, Amazon, Yes, I mean,
Speaker 2: I guess open ai sort of is kind of connected
Speaker 2: with Like those companies are buying a huge share of
Speaker 2: these GPUs, right, They account for a big, big chunk
Speaker 2: of in videos revenue. Yes, and those are good companies
Speaker 2: they have as customers because they're super rich, right, they
Speaker 2: can afford to pay you tens of billions of dollars
Speaker 2: for your GPUs.
Speaker 1: But that's being said, they're reselling that GPU power to
Speaker 1: companies and startups, right, so startups are buying that GPU
Speaker 1: competing capacity from these.
Speaker 2: GNT tech companies. Yeah, that's a good point, right. So
Speaker 2: they're in a way not the end customer. They're the
Speaker 2: sort of intermediate kind of service providers.
Speaker 1: But they also use that for their own internal systems too,
Speaker 1: like Meta uses a ton of GP power to make
Speaker 1: their advertising algorithms more effective and to pick the videos
Speaker 1: like tech talk does.
Speaker 2: And obviously Google Search is now becoming more and more
Speaker 2: AI driven, and there's Gemini, which is right, I mean,
Speaker 2: there's there's more direct use of AI by all of
Speaker 2: the big tech companies as well. So there's another piece
Speaker 2: of this which is interesting. I mean, because there's one
Speaker 2: universe wherever it's like, oh, we got to get it
Speaker 2: on AI, and they all buy the GPUs and then
Speaker 2: that's kind of it. It's like a step function where
Speaker 2: like there's this momentary rush where everybody buys the GPUs
Speaker 2: and then whatever. They just upgrade every couple of years,
Speaker 2: and it's more like a regular tech hardware business, which
Speaker 2: is like good but not amazing. But there's another piece
Speaker 2: of it that has been a huge deal for end video,
Speaker 2: and that's the scaling law or the scaling hypothesis. Let's
Speaker 2: talk about that. What's the what's the scaling hypothesis.
Speaker 1: So the scaling hypothesis, similar to what I said before,
Speaker 1: is the combination of computing power. The more computing power
Speaker 1: you add, the more data you increase, and the better
Speaker 1: ways you figure out software algorithms for each of these
Speaker 1: three buckets. If you increase it, the AI model becomes
Speaker 1: more capable and more effective.
Speaker 2: And like, just to be clear, like even if the
Speaker 2: algorithms aren't getting that much better, right, Like my understanding is,
Speaker 2: and this is kind of a new idea. It's like, oh,
Speaker 2: if we can just have more data and more computing
Speaker 2: power will get better results.
Speaker 1: Yes, so it's great if you have all three doing
Speaker 1: you know, basically showing up. But for this time period,
Speaker 1: the last few years, the scaling law has really taken
Speaker 1: off and companies are literally ten xing their compute power,
Speaker 1: and these AI clusters going from sixteen thousand GPUs now
Speaker 1: to one hundred thousand gps and now people are saying
Speaker 1: they're going to build one million GPU clusters in the
Speaker 1: next two three years.
Speaker 2: And in videos selling most of those GPUs.
Speaker 1: Yes, So if you go from sixteen thousand to one
Speaker 1: hundred thousand GPUs in a couple of.
Speaker 2: Years, that's a good business to do.
Speaker 1: If you're increasing the scale of the hardware by ten
Speaker 1: ten x every couple of years. And the exciting thing
Speaker 1: for Nvidia is that, like what I argue is that
Speaker 1: this thing is going to this trend is going to
Speaker 1: continue in the next few years because you have the
Speaker 1: scaling laws. Then you have this thing called multimodal where
Speaker 1: they're using this GPUs not just for texts like in
Speaker 1: chat TPT, you're using it for video and images generating
Speaker 1: those things. And now there's these two other two other
Speaker 1: waves of demand that are happening. There's this thing called
Speaker 1: AI agents where these AI models can do multi step
Speaker 1: tasks for you.
Speaker 2: Right, you could be like book me a ticket to
Speaker 2: San Diego some weekend in June whenever it's the best deal, yes,
Speaker 2: and then it does that.
Speaker 1: So that's going to happen in the next twelve months. Yes,
Speaker 1: these AI agents will eventually, in the next year or
Speaker 1: two be able to do all that tedious work automatically
Speaker 1: and probably with less errors than an actual human being.
Speaker 1: So AI agents multimodal. And now there's this thing called
Speaker 1: test time compute, where the AI models, instead of just
Speaker 1: spinning back an instant apply, can actually think about what's
Speaker 1: the best way to respond to your question and spend
Speaker 1: more time thinking about it and then give you a
Speaker 1: higher quality answer. So there's all these things that all
Speaker 1: of these test time compute, AI agents, multi modal. These
Speaker 1: are all things that need more computing power, they need
Speaker 1: more GPUs, and these are all things that are going
Speaker 1: to drive in video's revenue in the next couple of years.
Speaker 2: We'll be back in a minute and we will talk,
Speaker 2: among other things, about Jensen Wong, one of the most
Speaker 2: successful entrepreneurs of the twenty first century and a man
Speaker 2: who once told the colleague that he wakes up every morning,
Speaker 2: looks in the mirror and says to himself, you suck.
Speaker 2: I want to talk about Jensen a little bit. I
Speaker 2: mean your book. You know, he is obviously the main
Speaker 2: character in the history of Nvidio is the main character
Speaker 2: in your book. He's been the CEO of n Video
Speaker 2: for more than thirty years, which is like longer than
Speaker 2: Bill Gates with the CEO of Microsoft, longer than any
Speaker 2: you know, almost anybody in the s and p. Five
Speaker 2: hundred has been a CEO at this point. His childhood
Speaker 2: is quite interesting, right, Like, tell me just a little
Speaker 2: bit about his childhood.
Speaker 1: He was born in Taiwan, and they moved around a
Speaker 1: little bit to Thailand, and his father came to training
Speaker 1: in New York and fell in love with America. This
Speaker 1: is like the great American dream story. And the mother
Speaker 1: and father started teaching Jensen's brother English ten words a day,
Speaker 1: and they sent him to his aunt and uncle when
Speaker 1: he was about age eight or nine, and the aunt
Speaker 1: and uncle and the families. The funny thing is they
Speaker 1: sent him to a reform school in Kentucky by mistake,
Speaker 1: thinking that he will get a great education at this
Speaker 1: boarding school in Kentucky, which.
Speaker 2: He just thought it was a boarding school, yes.
Speaker 1: But it turned out to be a school for troubled.
Speaker 2: Kids, huh, which Jensen was not. He was just like
Speaker 2: a smart kid with ambitious for him parents.
Speaker 1: Yes, But he talks about how his time at Oneita
Speaker 1: Baptist and student Kentucky was formational for him.
Speaker 2: What was it like for him?
Speaker 1: It was hard at the beginning, but he started befriending people.
Speaker 1: He started playing chess with the janitor, and he just
Speaker 1: learned how to deal with other people much better. And
Speaker 1: he talks about how he learned his street fighter.
Speaker 2: Mentality because he was literally fighting.
Speaker 1: Yes, I mean there were bullies and all that stuff,
Speaker 1: but he just learned how to deal with people and
Speaker 1: the rough and tumble of kids back then. Eventually his
Speaker 1: parents came from abroad and they settled in Oregon, and
Speaker 1: he learned his work ethic from working at Denny's. He
Speaker 1: talks about how he washed more dishes and cleaned more
Speaker 1: bathrooms than any CEO in the history of CEOs. He
Speaker 1: says he Denny's helped them in terms of social skills
Speaker 1: and dealing with time pressure and dealing with customers. But
Speaker 1: it's the work ethic that really sets him apart. So
Speaker 1: even at the beginning, he was working from nine am
Speaker 1: to midnight, and he just set a culture where Nvidia
Speaker 1: employees work really hard.
Speaker 2: And I feel like at the beginning that's common, right,
Speaker 2: Like that is the classic startup story. But the classic
Speaker 2: startup stories you do that for five years and then
Speaker 2: either you get giant and you hire a grown up
Speaker 2: CEO and you you know, go start your blimp company
Speaker 2: or whatever, or you sell or whatever. He's still doing that,
Speaker 2: right He's sixty years old and like wildly rich and
Speaker 2: and he's still working that much.
Speaker 1: He's working all weekend. He's working Saturday Sunday. When he
Speaker 1: talks about when he goes to a movie, he doesn't
Speaker 1: remember the movie because he's thinking about work. He finds
Speaker 1: work relaxing and fund This is what drives him. He
Speaker 1: loves working.
Speaker 2: There's a moment, you know, you don't you don't write
Speaker 2: about his personal life at all. Once he sort of
Speaker 2: grows up and starts in video. You know, reasonably because
Speaker 2: the book's about in video, and so I just assumed
Speaker 2: he just worked all the time and didn't have a family.
Speaker 2: And the I think the only time in the book
Speaker 2: that his family comes up is there's this scene where
Speaker 2: he's on vacation and he's talking to some senior manager
Speaker 2: at a video on the phone and they're like, what
Speaker 2: are you doing. He's like, I'm sitting here on the
Speaker 2: balcony watching my kids play in the sand and writing emails.
Speaker 2: And it's like, like, if it's a movie, the kids
Speaker 2: are never on stage, right, you just hear them off stage.
Speaker 2: At that moment, You're like, oh, he has kids, Like
Speaker 2: that was that was a weird moment to me reading
Speaker 2: that line.
Speaker 1: So both executives told me they hate it when Jensen
Speaker 1: so called goes on vacation because he winds up giving
Speaker 1: them more directives and orders and more stuff to do
Speaker 1: when he's not on vacation because he's emailing in them
Speaker 1: do this, do that, and they they actually yell at him,
Speaker 1: So play with your kids. He's like, no, I could
Speaker 1: get real work done when I'm on vacation by doing
Speaker 1: his emails. I mean, so it shows you his obsessions.
Speaker 1: He's constantly thinking, he's constantly worried about what's happening at video.
Speaker 2: I mean it's interesting, right, Like he's not a balanced person, like,
Speaker 2: which is why to some extent he has built the
Speaker 2: thing that he has built.
Speaker 1: Right, He's completely obsessed with winning and he's extremely competitive.
Speaker 2: Yeah, there's a few other specific moments that really stood
Speaker 2: out to me. There's one where I think this, I
Speaker 2: think a salesperson told you this where they had just
Speaker 2: had a great quarter they sold a ton of GPUs,
Speaker 2: and the sales guy's talking to Jensen about how great
Speaker 2: they're doing, and Jensen says to the sales guy about Jensen,
Speaker 2: about himself. Jensen says, I look in the mirror every
Speaker 2: morning and say you suck.
Speaker 1: So he he's almost like a self psychologist. He knows
Speaker 1: if you start thinking that you're you're the best and
Speaker 1: your your hot stuff, you might get complacent, you start
Speaker 1: resting on your laurels, you might not work as hard enough.
Speaker 1: So he he sees that, oh, we just had a
Speaker 1: blowout quarter. What's the risk here? The risk is me
Speaker 1: gain complacent. So I'm gonna look myself in the mirror
Speaker 1: and say you suck and psych myself out.
Speaker 2: I mean that's that's that's one reading of it. That's
Speaker 2: the like ten dimensional chest reading of it. I mean
Speaker 2: there's another reading of it, which is like he's messed
Speaker 2: up in a way that makes him super driven and successful.
Speaker 1: But it works.
Speaker 2: He actually definitely works. We can agree that it works.
Speaker 1: Like he does the opposite too, So when things are
Speaker 1: really intimidating and he feels like, oh my gosh, how
Speaker 1: am I going to do this, he tells himself, how
Speaker 1: hard can it be? How hard can it be? So
Speaker 1: he does it on both both ends in the spectrum
Speaker 1: where he cites himself out when he feels intimidated, and
Speaker 1: when he feels like he's on top of the world,
Speaker 1: he tries to bring himself down back the earth.
Speaker 2: So the one other big piece of the Jensen Wong
Speaker 2: experience that we haven't talked about is the way he
Speaker 2: treats the people who work for him. Right, I want
Speaker 2: to talk about a particular scene because I think it
Speaker 2: puts a finer point on it. A scene from the
Speaker 2: book where it's like a company wide meeting and it's
Speaker 2: on like zoom or whatever, and he's yelling at a
Speaker 2: guy in the meeting. He's yelling at him, and then
Speaker 2: on top of that, he keeps telling the person filming
Speaker 2: the meeting to zoom in on the guy he's yelling at.
Speaker 1: I had multiple sources tell me that it was the
Speaker 1: most humiliating thing they've ever seen.
Speaker 2: I mean, it's that seems like bullying, like why zoom
Speaker 2: in on the guy? Like what lesson is added by that?
Speaker 1: So he keeps on saying to mister Rayfield, you got
Speaker 1: to get this chip back on track. The chip that
Speaker 1: heels in charge was behind schedule, and he kept us
Speaker 1: pointing to him, zooming in his face and telling him,
Speaker 1: You've got to get this. This is not how you
Speaker 1: run a business. You need to get this chip back
Speaker 1: on track. And this kind of like high standard demanding
Speaker 1: attitude I think is effective. It drives people. If you
Speaker 1: get dressed down by Jensen, the next time, you're going
Speaker 1: to work ten times more to make sure that you
Speaker 1: do a better job.
Speaker 2: Maybe I don't want the world to be that way,
Speaker 2: but it is that way, you know what I mean, Like,
Speaker 2: maybe I want to believe that, like there's a kinder
Speaker 2: world where people could do equally good work. But maybe
Speaker 2: I'm wrong.
Speaker 1: And Steve Jobs was the same way. Yeah, I mean
Speaker 1: it wasn't all sure, It wasn't all sunshine and rainbows.
Speaker 2: No, no, I mean the Isaacson book was really clear, right,
Speaker 2: the Isaacson biography of Steve Jobs. He did not seem
Speaker 2: like a good person in that book. He seemed like
Speaker 2: really good at building amazing products, but like not like
Speaker 2: a good human being by the sort of standard way
Speaker 2: we think of what makes a good person.
Speaker 1: But I think the key point that we also have
Speaker 1: to think about is people at Nvidia stay.
Speaker 2: Yeah, that's really interesting.
Speaker 1: The turnover is like one of the lowest in the
Speaker 1: industry at only three percent compared to the average of
Speaker 1: thirteen to fifteen percent.
Speaker 2: Yeah, certainly in the last few years. People stay because
Speaker 2: you get rich if you stay, and you lose your
Speaker 2: equity if you leave. But is that number true if
Speaker 2: you go back to when the stock was flat.
Speaker 1: I don't have the numbers, but a lot of the
Speaker 1: senior executives have been there more so than any other company. Yeah,
Speaker 1: fifteen twenty twenty five years.
Speaker 2: Yeah, that's it.
Speaker 1: You don't really hear other than one instance of people
Speaker 1: leaving to become a CEO of a different company. Uh huh.
Speaker 1: And I think part of it is people realize that
Speaker 1: Nvidia or the Gensen way of doing things is super effective,
Speaker 1: so people want to be on the winning team. So
Speaker 1: in one sense, he draws people hard, he dresses them down.
Speaker 1: He's very blunt and direct. But he compensates people. No
Speaker 1: one ever, hardly anyone complains about compensation. If you're effective
Speaker 1: and you do a good job, he will like double
Speaker 1: your stock compensation on the spot. So there's this meritocracy
Speaker 1: that people really adore. And if you see an effective
Speaker 1: leadership a culture that's based on meritocracy, if you're a
Speaker 1: top engineer, you stay at that company.
Speaker 2: Yeah. Yeah, So you interviewed Jenson as as you were
Speaker 2: working on the book. Was it scary to interview him?
Speaker 1: He was intimidating in the meeting, he he at he
Speaker 1: didn't yell at me, but a couple of times he's saying,
Speaker 1: you don't get nvidio in this line of questioning, and
Speaker 1: I it takes a while to get used to it,
Speaker 1: but you appreciate it because you see where the other
Speaker 1: person is. Like when someone is blunt and direct like Jensen,
Speaker 1: you know where that where he is at all times,
Speaker 1: and then you could work at improving yourself. So he
Speaker 1: talks about how what are you optimizing for? Are you
Speaker 1: optimizing for a person's feelings or optimizing for what's good
Speaker 1: for the company. So that's why he doesn't do one
Speaker 1: on one meetings or career coaching. Typically at a large company,
Speaker 1: when someone's doing poorly, the CEO will take them aside
Speaker 1: and say, Bob, you need to do this.
Speaker 2: You need if it's a person's.
Speaker 1: Yeah, Jensen's like, why is Bob the only one that
Speaker 1: gets a learning here? If I am blunt and direct
Speaker 1: and show Bob what he's doing incorrectly, like that person
Speaker 1: at that meeting, everyone in the room can learn. All
Speaker 1: the employees up and down the ladder can learn. So
Speaker 1: that's his philosophy is everyone should learn from their mistakes
Speaker 1: and everyone should know where they are at all times.
Speaker 2: So I mean, so you write in the book that
Speaker 2: in video really is like an extension of Jensen, right.
Speaker 2: I think he used the metaphor of like the formula
Speaker 2: one card that's like optimized for him as the driver.
Speaker 2: So and he's in his sixties, which is not old
Speaker 2: old but definitely not young. Like, is he gonna run
Speaker 2: in video for another twenty years? What's going to happen?
Speaker 2: Like there's no obvious successor, Like where does that go?
Speaker 1: I think there's there's no one else at in video.
Speaker 1: I think that can run in video as effectively as Jensen.
Speaker 1: And he loves the company so much. He loves what
Speaker 1: he's doing. They're having enormous impact with this AI wave,
Speaker 1: and he's so excited about the potential for curing cancer
Speaker 1: and digital biology, the potential for robots, the potential for
Speaker 1: AI to kind of disrupt education and help kids learn better.
Speaker 1: That I don't see him leaving anytime soon. He loves
Speaker 1: his job so much and he can't argue that he's
Speaker 1: being ineffective. So I don't think for the next few
Speaker 1: years there's anyone that's going to take over for him.
Speaker 1: Someday Nvidia is going to have to come up with
Speaker 1: a new CEO or a successor to Jensen, and that's
Speaker 1: going to be a big question mark. It's that next
Speaker 1: person going to be as effective as Jensen in terms
Speaker 1: of being able to have the technical skill and the
Speaker 1: compency to steer in video in the right direction. Well,
Speaker 1: they have his business genius of coming up with all
Speaker 1: these new strategies that he does time and time again.
Speaker 1: That's going to be a huge question Marke.
Speaker 2: I mean people also talk about limits to scaling right
Speaker 2: in AI. We were talking earlier about how part of
Speaker 2: in video's wild growth over the last couple of years
Speaker 2: has come from this fact that that you can just
Speaker 2: add more GPUs basically and get better results. More GPUs,
Speaker 2: more data, and you know, people talk about a running
Speaker 2: out of data because like they're basically, as I understand it,
Speaker 2: training on the whole Internet right now, which is like, Okay,
Speaker 2: it's a lot of data. I mean, is that it
Speaker 2: seems like nobody knows, right. It seems like they are
Speaker 2: smart people who say, no, no, we'll have synthetic data
Speaker 2: blah blah blah, we won't hit a sort of scaling wall.
Speaker 2: And then there are people who make the other argument.
Speaker 2: I certainly don't know, but does that seem right, Like
Speaker 2: is that an open question right now? And is that
Speaker 2: a meaningful question for nvideo?
Speaker 1: I think it's an open question. But like I said,
Speaker 1: there's the multimodal stuff, there's AI agents, and then there's
Speaker 1: proprietary data inside companies.
Speaker 2: Uh huh.
Speaker 1: So all these corporations have data going back decades. Companies
Speaker 1: are going to use the power of these AI computing
Speaker 1: systems to go through all their data internally, all their
Speaker 1: proprietary data, and have all that knowledge at employees fingertips.
Speaker 1: So an employee can ask what's the best way to
Speaker 1: do this, and the AI computer is going to be
Speaker 1: able to go back thirty years of data and figure
Speaker 1: out the best piece of insight to help that employee.
Speaker 1: And that's not really being done today.
Speaker 2: So Nvidia designs their chips their gips, but they don't
Speaker 2: actually manufacture them, right Is it right that they're made
Speaker 2: in Taiwan, where most cutting its chips are made now?
Speaker 1: Yes, So TESMC actually makes and manufacturers in videos chips
Speaker 1: ever since the late nineteen nineties, and they're the best
Speaker 1: at doing this all. A lot of the fabulist chip
Speaker 1: designers in California use TSMC, and Video uses them too.
Speaker 2: And so I mean it is really interesting to me
Speaker 2: to a lot of people that Taiwan is this incredibly
Speaker 2: fraught geopolitical place. Right, China says Taiwan is part of China.
Speaker 2: Taiwan says no, we are not part of China, and
Speaker 2: that Taiwan is the only place in the world that
Speaker 2: makes like the most important physical thing in the world
Speaker 2: of technology today. Right, it makes like that's wild, Like
Speaker 2: what do you make of that? And how does that
Speaker 2: fit with the in video story?
Speaker 1: I think eventually Nvidia chips will be made in the
Speaker 1: US TSMC. I mean, that's all part of this Chips
Speaker 1: Act that the Biden administration has posed.
Speaker 2: So this this plant that TSMC is building in Arizona, Like,
Speaker 2: is that advanced enough to make like frontier in Nvidia chips?
Speaker 1: So the CSMC factories in the US are always going
Speaker 1: to be one or two generations behind the factories in Taiwan.
Speaker 1: But that doesn't mean Nvidia can't use the factories in
Speaker 1: the US if they're one or two generations behind for
Speaker 1: their older chips. So I think that will eventually happen.
Speaker 2: It seems wild to me, Like, I mean, it seems
Speaker 2: very possible that China will try and make Taiwan be
Speaker 2: part of China, right, and and like we have all
Speaker 2: these export controls to try and keep China from getting
Speaker 2: cutting edge chips. Like it's a really interesting, complicated dynamic.
Speaker 1: I think people make a big issue at this, But
Speaker 1: I think if it happens, like in Vidio, chips are
Speaker 1: not going to be the main issue. It will cost
Speaker 1: a global calamity where you know, we won't be able
Speaker 1: to upgrade our cars, at laptops, nearly every computing device,
Speaker 1: every appliance will not work if we don't have access
Speaker 1: to Taiwan chips.
Speaker 2: I mean, presumably we could do some like sub Manhattan
Speaker 2: Project scale Manhattan project to get a state of the
Speaker 2: art TSMC factory somewhere that is not Taiwan, right, Like.
Speaker 1: We're doing some of that now, but it's not going
Speaker 1: to be able to it's going to be ten twenty
Speaker 1: of the capacity we need.
Speaker 2: Yeah, the capacity, right, it takes a long time, Like
Speaker 2: the fabs are wildly complicated to build, and they cost
Speaker 2: billions of dollars and they take years, so you couldn't
Speaker 2: just do it.
Speaker 1: They cost ten to twenty billion dollars to build and
Speaker 1: three to four years over three to four years, So
Speaker 1: it's not something that can happen overnight. And if something happens,
Speaker 1: like it's just gonna be so terrible and it'll be
Speaker 1: like a depression. So I just hopefully it doesn't happen,
Speaker 1: and you know it's not gonna The question isn't gonna
Speaker 1: be about in videotips. The question is gonna be about
Speaker 1: the global economy of that.
Speaker 2: So what do you think is the biggest risk for
Speaker 2: video in the like five year timeframe?
Speaker 1: It's really tough to say.
Speaker 2: Now.
Speaker 1: I mean I see the next few years with all
Speaker 1: the different AI innovations and all the progress. But again,
Speaker 1: just like every big computing shift, what's the next big thing?
Speaker 1: Could it be quantum computing? It can be like who knows, five, ten,
Speaker 1: fifteen years from now? And every from the pcach with
Speaker 1: Microsoft and Intel it's called Wintel. And then it went
Speaker 1: to smartphones where Apple dominated with the iPhone, and now
Speaker 1: in Vidia is dominating. In terms of the AI computing shift,
Speaker 1: There's gonna be another shift, and right now we're at
Speaker 1: the early stages of the AI computing movement. In five
Speaker 1: ten years, there might be the next big thing that
Speaker 1: I can't can't even foresee right now. And is Nvidia
Speaker 1: gonna be able to see that coming? If Jensen's around,
Speaker 1: I think he will. But if Jensen's not there, you know,
Speaker 1: then just like every other computer, major computer company in history,
Speaker 1: say IBM, it's very easy to get disrupted in the
Speaker 1: technology industry.
Speaker 2: We'll be back in a minute with the Lightning round.
Speaker 2: I want to finish with the Lightning round, which is
Speaker 2: going to be considerably more random and digressive than the
Speaker 2: conversation to this point. I want to talk about video
Speaker 2: games and your experience with video games. What was the
Speaker 2: first video game you ever loved?
Speaker 1: I think it's the first VAM I've ever tried, which
Speaker 1: was Combat for the Datar twenty six hundred.
Speaker 2: I love that.
Speaker 1: I mean it's literally I remember coming home and my
Speaker 1: dad buying the Autari twenty six hundred connected to a
Speaker 1: black and white TV and being to use that joystick
Speaker 1: to control the little tank going around the screen. That
Speaker 1: was an amazing moment. It was an incredible moment that
Speaker 1: this was possible. At the home, I had.
Speaker 2: An Atari twenty six hundred. I remember the joystick. I
Speaker 2: remember Demon Attack, so kind of second tier game, but
Speaker 2: I got really into it. What's the game you spent
Speaker 2: the most hours on?
Speaker 1: Probably this game called Lemmings. I don't know if you
Speaker 1: it's it's a puzzle strategy game where you control lemmings
Speaker 1: and you guide them across a maze to get.
Speaker 2: The little creatures that are famed for jumping off cliffs,
Speaker 2: which maybe they don't actually do.
Speaker 1: Yes, if you guide them the wrong way, they'll fall
Speaker 1: off a cliff and to their death, and they make
Speaker 1: this cute sound like oh no, so so. I played
Speaker 1: so many hours of.
Speaker 2: That game, one thousand hours.
Speaker 1: Probably hundreds of hours. I want to say a thousand.
Speaker 2: Yeah. What's your most proud video game accomplishment?
Speaker 1: I still remember playing the Legend of Zelda, the first
Speaker 1: one for the Nintendo Entertainment System and beating that game
Speaker 1: and just like pumping my fists in the air. I
Speaker 1: was over young man. Yeah.
Speaker 2: What is the most exciting video game innovation that's going
Speaker 2: to happen in the next few years?
Speaker 1: Well, this is a little technical, but I think DLSS
Speaker 1: from the video is going to get even more powerful,
Speaker 1: which is this upscaling technology the gents actually invented in
Speaker 1: a meeting where they fill in the details using AI,
Speaker 1: so the frame rates are able to go much faster
Speaker 1: and better.
Speaker 2: So it's basically like interpolation. It's sort of doing what
Speaker 2: AI AI always does, yes, fort.
Speaker 1: But that's really effective. People can't tell the difference between
Speaker 1: the real thing and the interpolated AI graphics, so that
Speaker 1: will allow as this technology get better, that will allow
Speaker 1: the graphics to be more for the realistic, and the
Speaker 1: physics and the rate tracing better than ever before.
Speaker 2: What game are you most excited to play in twenty
Speaker 2: twenty five?
Speaker 1: I mean Grand Theft Auto six if it comes out,
Speaker 1: but it might get delayed.
Speaker 2: Next here, Okay, let's do a few non video game questions.
Speaker 2: What's your favorite tech book besides the one you wrote.
Speaker 1: I guess it's still a tech book, The Innovator's Dilemma,
Speaker 1: which is actually one of Jens's here books. It talks
Speaker 1: about how companies get disrupted by startups and people underneath them.
Speaker 1: It really goes into in depth about how the distrived industry.
Speaker 1: Every successive generation that distribes there was a new market
Speaker 1: leader because they couldn't disrupt themselves to the new or
Speaker 1: smaller format.
Speaker 2: I mean, to me, the really interesting insight of that
Speaker 2: book is the innovator is actually making a crappier product, right,
Speaker 2: Like that's the surprise. It's not exactly like they come
Speaker 2: along and do something better. They go to the crappy
Speaker 2: end of the market and they make a cheap product
Speaker 2: and it's not better than the thing the market leaders
Speaker 2: and the market lader' is like, oh that, who cares
Speaker 2: about that? That's just some low market, some low margin thing,
Speaker 2: and the innovator comes up from below. Like that to
Speaker 2: me is the really key insight of that book.
Speaker 1: And they scale the volumes, and once you have the volume,
Speaker 1: it pretty much becomes game over for the incumbent because
Speaker 1: they can't match that scale and the economies of scale
Speaker 1: that comes with that.
Speaker 2: Defend pineapple on pizza.
Speaker 1: It just tastes good. I love it. I don't know
Speaker 1: if you saw my tweets on it. I actually like
Speaker 1: pineapple on pizza.
Speaker 2: Yes, I think it was Instagram. I think it was Instagram.
Speaker 2: What's the best deal you ever got at Costco?
Speaker 1: It's still the hot dog and you can't beat the dollar.
Speaker 2: Fishbee Take Him is the author of the Nvidia Way.
Speaker 2: Today's show was produced by Gabriel Hunter Chang. It was
Speaker 2: edited by Lyddy Jean Kott and engineered by Sarah Bruguer.
Speaker 2: You can email us at problem at Pushkin dot FM.
Speaker 2: I'm Jacob Goldstein and we'll be back next week with
Speaker 2: another episode of What's Your Problem.