Speaker 1: Today, we're going to start on a drive in Hawaii.
Speaker 2: We're on north Shore, going deeper into the jungle on
Speaker 2: the north shore, so we're passing twin falls right now.
Speaker 1: I'm driving through the lush green forests of Maui. Annie Altman,
Speaker 1: Sam Altman's little sister is sitting in the passenger seat.
Speaker 1: You heard from her briefly in the first episode.
Speaker 2: And I love this because you wouldn't turn here if
Speaker 2: you didn't know you could turn in.
Speaker 1: I certainly would not be driving down this if I
Speaker 1: didn't tell you a tiny little round. We're taking a
Speaker 1: tour of the different places Annie has moved around in
Speaker 1: the last couple of years, driving down dirt roads to
Speaker 1: look at cabins and houses hidden behind enormous tropical plants.
Speaker 2: We got a huge monster.
Speaker 3: I look at that.
Speaker 2: It's so it's so goofy. Then I'm driving this, I
Speaker 2: know because then you see it in someone's office plant
Speaker 2: and you're like, is that the same thing.
Speaker 1: For much of the past two years, Annie hasn't been
Speaker 1: able to afford a stable place to live.
Speaker 4: The place you just passed is one of the places
Speaker 4: I stayed at longer term.
Speaker 2: In all of the houselessness two months on a newly
Speaker 2: built no running water or no electricity house at the
Speaker 2: far and back of the property.
Speaker 1: And I think she's an important part of Sam's story.
Speaker 5: And at the time I had nowhere to stay and
Speaker 5: no rent money, certainly no deposit money, and barely enough room,
Speaker 5: barely enough money for rent.
Speaker 1: Recently, over the course of just a year, she moved
Speaker 1: twenty two times, and that's on average about twice a month.
Speaker 1: Sometimes she has stayed places for a week at a time,
Speaker 1: or even just a night or two. Some of them
Speaker 1: have been illegal rentals without running water. She says she's
Speaker 1: slept on floors and friends' houses. She stayed with strangers
Speaker 1: when she didn't have another option.
Speaker 2: The man who lived in the front house messaged me
Speaker 2: on Instagram, and I stayed in his kids.
Speaker 4: Room the week that they weren't there, and then slept
Speaker 4: on the floor in the common room the week that
Speaker 4: the kids were there.
Speaker 2: I was houseless. I didn't have somewhere to go. I
Speaker 2: stayed in this cabin with destly anti roof right there
Speaker 2: for three months.
Speaker 1: How many different places have you lived in that didn't
Speaker 1: have running water?
Speaker 2: Maybe five ish five or sus six, I don't know.
Speaker 1: Meanwhile, thousands of miles away in San Francisco, her brother
Speaker 1: Sam was having a spectacular year. In twenty twenty three,
Speaker 1: the success of chat GBT had launched open Ai into
Speaker 1: the stratosphere. Sam was named CEO of the Year by
Speaker 1: Time magazine. He spent months flying around the globe talking
Speaker 1: to world leaders about AI.
Speaker 6: So what outforted you?
Speaker 7: Let's give a big round of applause.
Speaker 8: There a special guests.
Speaker 1: Sam Altman.
Speaker 9: It is with great pleasure that I invite on stage
Speaker 9: Sam Altman.
Speaker 10: The individual transcends introduction, mister Sam Altman, the CEO pop
Speaker 10: open AI.
Speaker 1: Sure, Sam had been positioning himself as a public intellectual
Speaker 1: for some time, but for a long while he had
Speaker 1: merely been Silicon Valley famous. This was a new level
Speaker 1: of fame. Suddenly it seemed like Sam Altman was a
Speaker 1: spokesman for this entire AI boom. On stage, on podcasts
Speaker 1: and interviews, people kept turning to Sam for answers. They
Speaker 1: were asking him what our AI future would hold. In
Speaker 1: May of that year, he confidently suggested a future where
Speaker 1: no one is poor. It's an idea he's talked about
Speaker 1: for years, and the remarks show that his tune hasn't
Speaker 1: changed despite growing renown and wealth.
Speaker 8: One thing I think we all could agree on is
Speaker 8: that we just shouldn't have poverty in the world.
Speaker 1: So he's saying something like poverty, which is as deeply
Speaker 1: entrenched as civilization itself, could be fixed. In the age
Speaker 1: of AI. Everyone will have what they need to live,
Speaker 1: including a home.
Speaker 8: I think we are not that far away from being
Speaker 8: able to eliminate poverty asthectically worldwide, certainly developed countries, and
Speaker 8: I think there are things that are fifty years from
Speaker 8: now will be basic human rights, like healthcare and enough
Speaker 8: food to eat and a place to live. I'm confident
Speaker 8: we're going to get that done.
Speaker 1: He sounds so sure of himself. How will these changes happen?
Speaker 1: Sam says that they could come from AI. He's basically
Speaker 1: saying the technology his company is working on can eliminate poverty.
Speaker 1: Here he is at a Bloomberg conference in June twenty
Speaker 1: twenty three.
Speaker 8: I think it'd be good to ten poverty. Maybe you
Speaker 8: think we should stop a technology they can do that,
Speaker 8: I personally don't.
Speaker 1: He's even scolding people who say they want to stop
Speaker 1: or slow down the development of AI by saying that
Speaker 1: they want to keep people poor. Solving poverty seems to
Speaker 1: become part of his personal brand. There's almost a stump
Speaker 1: speech of sorts, and there's one word he likes to
Speaker 1: use to encapsulate our bountiful future.
Speaker 8: I think it's how we get to this world of abundance.
Speaker 8: In a world with the level of abundance AI and
Speaker 8: the abundance that comes with that, If we can have
Speaker 8: abundant and cheap energy and intelligence that will transform people's lives.
Speaker 1: It sounds wonderful, almost utopian. But Sam was saying on
Speaker 1: stage that everyone should have enough money, enough food, everyone
Speaker 1: should have a place to live, while his own sister
Speaker 1: was struggling with homelessness. I want to believe Sam's promises
Speaker 1: about abundance, but Annie's story complicates a lot of the
Speaker 1: things Sam has projected about the future. You're listening to Foundering.
Speaker 1: I'm your host, Ellen Hewitt. By the time open ai
Speaker 1: had been around for a few years, its employees began
Speaker 1: sensing that they were working on something profound. AI was
Speaker 1: starting to do things even its creators didn't quite understand,
Speaker 1: and the response divided people. Some people at open ai,
Speaker 1: including Sam, argued that an all powerful computer intelligence would
Speaker 1: be good for the world. Some thought it could be very,
Speaker 1: very bad. These visions were extreme, polarizing, as different as
Speaker 1: heaven and hell. This period ultimately led up to the
Speaker 1: release of chat GPT, which was a defining moment for
Speaker 1: the company and for the entire AI complex. The two
Speaker 1: predictions people talked about the most during this time were
Speaker 1: that AI would either wipe out the human race or
Speaker 1: completely change the definition of work. The first one is
Speaker 1: hugely controversial, the second one is less controversial. People will
Speaker 1: lose jobs and the economy will have to adapt. I'm
Speaker 1: going to tell this story in two parts. First, the
Speaker 1: disputes within open Ai, the reckoning with what they'd built,
Speaker 1: and the elevation of Sam as some kind of AI hero.
Speaker 1: In the second part, we'll hear more from Sam's sister
Speaker 1: and explore the debate around poverty and what an AI
Speaker 1: economy might look like. And you'll see exactly how far
Speaker 1: away Annie's life is from Sam's vision of AI utopia.
Speaker 8: We'll be right back.
Speaker 1: Let's pick the story up in twenty twenty one. It's
Speaker 1: about a year and a half before open Ai launches
Speaker 1: chat GPT, and open AI's technology had been rapidly gaining
Speaker 1: new capabilities.
Speaker 11: This was around the time that OpenAI was building more
Speaker 11: and more powerful models such as GPT two and GBT
Speaker 11: three that could ingest text and spit out ever more
Speaker 11: human sounding text.
Speaker 1: That's my colleague Rachel Metz, who covers AI.
Speaker 11: They were focused on building increasingly powerful AI models, doing
Speaker 11: it very rapidly, moving towards this goal that they have
Speaker 11: long stated of creating artificial general intelligence.
Speaker 1: This rapid progress was good. It was great for Sam,
Speaker 1: great for the company, but it made some open ai
Speaker 1: employees nervous, even ones very high up. Here's Dario Amidae,
Speaker 1: who at the time was the vice president of research
Speaker 1: at open Ai. He's talking on the Logan Bartlett podcast
Speaker 1: and he sounds genuinely agitated.
Speaker 12: So, you know, I really freaked out about this stuff.
Speaker 12: In twenty eighteen or twenty nineteen or so. The first
Speaker 12: time I looked at GBT too, I was like, oh my,
Speaker 12: this is like this is crazy. This is you know,
Speaker 12: there's there's nothing like, there's nothing like this in the world,
Speaker 12: Like it's crazy that this is possible.
Speaker 1: He looked at what he was helping build at open ai,
Speaker 1: and his overwhelming emotion was fear.
Speaker 12: Why am I scared.
Speaker 6: Okay, this thing's going to be powerful. It could destroy us,
Speaker 6: and like all the ones built so far, like you know,
Speaker 6: are at pretty decent risk of doing some random shit.
Speaker 6: We don't understand if such a model wanted to recavoc
Speaker 6: and destroy humanity or whatever. I think we have basically
Speaker 6: no ability to stop it.
Speaker 1: That's from a tech show called the Door Kesh podcast.
Speaker 1: You can hear Dario's uneasiness, which spread to some of
Speaker 1: his peers at work. In twenty twenty one, he and
Speaker 1: six of his open ai colleagues all left at once
Speaker 1: and they started a new rival company focused on building
Speaker 1: AI that was safe.
Speaker 12: We need to make these models safe in a certain way,
Speaker 12: and you know, we need to do them within an
Speaker 12: organization where we can really believe that these principles are
Speaker 12: incorporated top to bottom.
Speaker 1: While his tone sounds innocuous, if you read between the lines,
Speaker 1: he's implying that open ai was not that concerned with
Speaker 1: safety and not following those principles top to bottom. He
Speaker 1: and his co founders felt they needed to create a
Speaker 1: whole new company focused on preventing AI from potentially wiping
Speaker 1: us all out. This idea was baked into their name anthropic,
Speaker 1: which refers to the existence of humanity on Earth. Anthropics
Speaker 1: split from open ai was a big deal. The group
Speaker 1: of employees who walked away included executives and key people
Speaker 1: who had worked there for a long time. It cast
Speaker 1: this suspicious pall over open Ai. Their exit suggested that
Speaker 1: they didn't trust Sam to do the right thing. So
Speaker 1: fast forward to late twenty twenty two, open Ai released
Speaker 1: chat GBT. They weren't expecting a huge response, but it
Speaker 1: drew people in gradually. Then suddenly everyone was trying it out.
Speaker 1: Here's Rachel again.
Speaker 11: I kept an eye on it for a few days
Speaker 11: and I started noticing people were talking about it. But
Speaker 11: it started getting more and more traction, and I remember
Speaker 11: reaching out to my editors and saying, hey, we should
Speaker 11: write about this.
Speaker 1: Chat GBT wasn't new technology. It was basically powered by
Speaker 1: GPT three point five, pre existing model, but it was
Speaker 1: a new way of presenting that technology to the world.
Speaker 1: It was a free, easy to use chat based AI tool.
Speaker 1: It felt a little like texting a friend.
Speaker 11: This was much more fluid feeling. It gave you a
Speaker 11: sense that you were communicating with a person, a person
Speaker 11: that would make a lot of things up right, who's
Speaker 11: the answers? You would have to constantly.
Speaker 1: Check and people were hooked. Chat gpt reached one hundred
Speaker 1: million users in just two months, the fastest growth ever
Speaker 1: at the time. For reference, it took TikTok nine months
Speaker 1: and Instagram two and a half years to reach the
Speaker 1: same popularity. Chat gpt felt intelligent even when it got
Speaker 1: things wrong. It prompted a lot of people to ask themselves,
Speaker 1: how is AI going to change my life? The technology
Speaker 1: had some immediate practical uses. Started using chat gpt to
Speaker 1: write code more quickly, to translate documents more fluidly, and
Speaker 1: to draft emails. Students used it for homework help good
Speaker 1: and to cheat on papers not so good. Needless to say,
Speaker 1: all this excitement was a huge boost for open Ai.
Speaker 11: With chat gbt, OpenAI basically skyrocketed to brand awareness and success,
Speaker 11: and people immediately started thinking of it as the hot
Speaker 11: AI company, a leader in AI.
Speaker 1: It also skyrocketed Sam's public profile. Now Ai was the
Speaker 1: story of twenty twenty three, and Sam was the main character.
Speaker 1: He became a household name and across the tech industry.
Speaker 1: Investors were desperate to throw money at AI. Companies pivoted
Speaker 1: their focus to AI, and highway billboards touted new AI startups,
Speaker 1: and this chat gbt mania did something else too.
Speaker 11: In addition to making all kinds of people interested in
Speaker 11: and aware of cutting edge AI, the release of chat
Speaker 11: gbt inflamed this small subsection of people that have long
Speaker 11: been interested in this idea that AI is going to
Speaker 11: become more and more powerful, and that it will inevitably
Speaker 11: go rogue and threaten us, and we have to figure
Speaker 11: out how to save humanity from it.
Speaker 1: Fear of AI destroying the world. It's the fear that
Speaker 1: Dario described feeling before he quit Open AI to found Anthropic.
Speaker 1: It might sound a little ridiculous, but to many people
Speaker 1: it's dead serious. This Silicon Valley subculture that believes AI
Speaker 1: might destroy us soon has steadily been growing more influential
Speaker 1: and powerful. These ideas are a major motivating force in
Speaker 1: AI right now. Many tech workers in AI feel that
Speaker 1: they are working on the most consequential thing possible for
Speaker 1: the human race, a matter of life and death for
Speaker 1: the whole planet. This belief influences where billions of dollars
Speaker 1: are directed, and which problems get worked on and which
Speaker 1: ones get ignored. Chat GBT brought these fears into the
Speaker 1: public consciousness big time. One survey in the UK shows
Speaker 1: that in the course of one year, right after chat
Speaker 1: gbt's release, the percentage of people who believed that AI
Speaker 1: was a top possible cause of human extinction more than doubled.
Speaker 1: To be clear, this claim is hotly disputed. A lot
Speaker 1: of people, even in Silicon Valley, think AI numerism is
Speaker 1: overblown or a quasi religious ideology, but many take it seriously.
Speaker 1: We'll be right back. I want to talk about how
Speaker 1: it feels to believe AI might destroy humanity. Here's Chao
Speaker 1: Chu Yun. He's just one of many people in tech
Speaker 1: who held this belief. His introduction to this world was
Speaker 1: bizarrely through Harry Potter fan fiction he found online that
Speaker 1: was written by a very influential AI doomer named Eliezer Yudkowski.
Speaker 1: That discovery led him to lots more of Eliezer's writing
Speaker 1: about AI. He eventually became convinced that the world was
Speaker 1: probably going to be swallowed by super intelligent AI.
Speaker 10: Soon I was like, Okay, it's happening, We're getting there.
Speaker 10: It's going to happen in my lifetime. It's going to
Speaker 10: happen like before I retire. I made a conscious decision
Speaker 10: around this time to not open a retirement account, which
Speaker 10: some people might think was silly, but I was like,
Speaker 10: I really do not think money is going to matter
Speaker 10: by the time I hit retirement age. I think by
Speaker 10: the time I hit retirement age, the world is going
Speaker 10: to be unrecognizably strange. I just don't think money is
Speaker 10: going to be a thing. Either we're all going to
Speaker 10: be dead, or we're going to be into some crazy
Speaker 10: post Singularity environment where it's just I just don't think
Speaker 10: it's going to matter.
Speaker 1: The Singularity is the point when robots become smarter than humans.
Speaker 1: For him, it went beyond skipping out on a retirement account.
Speaker 1: Once chou Chu really started believing this, other things in
Speaker 1: his life just didn't seem to matter. He dropped out
Speaker 1: of his math PhD program five years in. He drifted
Speaker 1: away from old friends. He thought they didn't understand the
Speaker 1: magnitude of the situation.
Speaker 10: It was like this black hole in my kind of
Speaker 10: sense of in which things were important, and like sort
Speaker 10: of the closer I got to it, like the less
Speaker 10: important everything else seemed that I was like, oh, well,
Speaker 10: there's this other stuff that doesn't really matter. There's this
Speaker 10: other stuff that doesn't really matter, and this is like
Speaker 10: maybe the only thing that matters.
Speaker 1: I know this sounds scary, extreme and maybe ridiculous, but
Speaker 1: if you spend enough time talking to people in AI
Speaker 1: and around Silicon Valley, you will hear this attitude. Sometimes
Speaker 1: it's whispered, sometimes it's said loudly. Sometimes they'll use terms
Speaker 1: like AI safety, AI alignment, or AI existential risk, but
Speaker 1: they're all talking about the same thing. There are significant
Speaker 1: numbers of influential people in AI who believe that our
Speaker 1: world in twenty years will be utterly unrecognizable from the
Speaker 1: world of today. Some people think it'll be way better,
Speaker 1: but a lot think it might be way worse, and
Speaker 1: they feel that they should devote their lives to preventing
Speaker 1: AI doomsday. In the last five or ten years, a
Speaker 1: lot of famous, rich tech people have thrown money at
Speaker 1: this new cause. Facebook co founder Dustin Moskovitz and crypto
Speaker 1: fraudster Sam Bankman Freed each pledged hundreds of millions of
Speaker 1: dollars toward AI safety projects. Tech leaders have signed public
Speaker 1: statements about how AI could put our human civilization at risk.
Speaker 1: Chau Chu and his fellow adherents felt it was deeply
Speaker 1: important to get other people to accept this threat. They
Speaker 1: became evangelists. At events. They would trap people in long
Speaker 1: conversations about AI apocalypse.
Speaker 10: And they started off skeptical and then they ended up
Speaker 10: somewhere between like either like really convinced or like scared
Speaker 10: of just like oh, oh this is terrifying. This is
Speaker 10: like really like I'm like kind of sicked up about this.
Speaker 10: This was a very frightening idea. It's like, oh, hey,
Speaker 10: what if everyone dies in ten years? That's a scary idea,
Speaker 10: and like it's it's it's sort of safer psychologically to
Speaker 10: just dismiss it as like that's ridiculous, that could never happen.
Speaker 10: But if someone really like get gets to you about that,
Speaker 10: like as someone you know spends four hours talking to
Speaker 10: him very carefully convincing you, like what if it's possible? Though,
Speaker 10: Like that's really that's very scary. That's like oh that,
Speaker 10: how do I?
Speaker 5: Oh?
Speaker 10: Like that's like terriff Like what do I do about that?
Speaker 10: There's a world The difference between like reflexively dismissing that
Speaker 10: idea and like really considering it seriously as a possibility.
Speaker 10: On an emotional level.
Speaker 1: Chou Chu took jobs at AI safety organizations. He volunteered
Speaker 1: at rationality workshops to try to get more young minds
Speaker 1: working on AI safety. In this world, when people talk
Speaker 1: about what's at stake, it's on a galactic scale. Okay,
Speaker 1: So that's one point of view that the most important
Speaker 1: thing right now is to make sure AI is safe
Speaker 1: in the future. But there are some really smart people,
Speaker 1: academics and researchers who find this AI doomsday frenzy very
Speaker 1: frustrating and harmful. Like Emily Bender, an academic who specializes
Speaker 1: computational linguistics.
Speaker 13: It's a distraction because there's all kinds of harms that
Speaker 13: are happening right now in terms of labor exploitation, in
Speaker 13: terms of data theft, in terms of discriminative outcomes, in
Speaker 13: terms of representational harms. And the more we focus on
Speaker 13: these very sort of exciting in the way that action
Speaker 13: movies are exciting fantasy scenarios at exisential risks, the less
Speaker 13: time and effort goes in to actually dealing with the
Speaker 13: real harms that are happening right now.
Speaker 1: Emily and lots of other experts argue that all of
Speaker 1: this talk of AI doomsday is an enormous, dangerous distraction,
Speaker 1: like a big flashing alarm screaming We're all going to die.
Speaker 1: It steals all the air in the room. It makes
Speaker 1: it easy to ignore issues happening today, like racial bias
Speaker 1: in AI systems used in criminal justice, stealing copyrighted work
Speaker 1: of artists to train models, and so many more.
Speaker 3: You've got the surveillance applications of these technologies that are
Speaker 3: being used to over police communities, or performance of facial
Speaker 3: recognition technology, especially for women with darker skin in particular
Speaker 3: Google and putting identity terms up for sale to search
Speaker 3: for black girls, you get a whole.
Speaker 13: Bunch of porn. There's enormous risks around synthetic media, all
Speaker 13: of our ability to find trustworthy information and then trust
Speaker 13: it when we find it is threatened helblic health, democracy.
Speaker 1: The list goes on and on. Emily reminds us that
Speaker 1: these are urgent problems, ways that people are being harmed
Speaker 1: by AI right.
Speaker 13: Now, and yet we are wasting our time talking about
Speaker 13: these fantasy scenarios because the people with all the money
Speaker 13: decided to get worried about it. In the meantime, there's
Speaker 13: the doomer stuff, the existential stuff is also another kind
Speaker 13: of AI hype, because if these systems are so powerful
Speaker 13: they might destroy the world, then these systems are really powerful.
Speaker 1: That hype is a key part of this, because a
Speaker 1: super powerful AI system that's exciting to investors and to employees.
Speaker 1: Emily thinks AI apocalypse beliefs are harmful and misleading, but
Speaker 1: she also thinks many of the people concerned with AI
Speaker 1: safety are coming from a sincere place.
Speaker 13: But it does seem to be a genuine belief of
Speaker 13: some of these folks. The people who get deep into
Speaker 13: this set of beliefs about super intelligent AI taking over
Speaker 13: the world see themselves as the heroes and their stories
Speaker 13: who are going to work to stop that. I think
Speaker 13: it's genuine. I think it's misplaced, but I think it's genuine.
Speaker 1: I think Emily is hitting on something real here, that
Speaker 1: this urge to be a hero becomes a motivating force
Speaker 1: in the AI industry. Here's chaw choo again.
Speaker 10: There is this kind of very boyish desire to be
Speaker 10: a hero, Like most people don't get a chance to
Speaker 10: do that in any meaningful sense. Not only just what
Speaker 10: if I could be a hero, but like, oh what
Speaker 10: if I could like smart my way to being a hero.
Speaker 10: I think it'll be difficult for some people to admit
Speaker 10: this because it's like kind of it feels immature or something.
Speaker 10: But I really do think that is part of the
Speaker 10: of the appeal of the pitch.
Speaker 1: Being a hero by using your brain. Chauchu is saying
Speaker 1: the quiet part out loud that many people in this
Speaker 1: field are motivated by this pride. They want to feel
Speaker 1: like they're important and that their work is cosmically significant.
Speaker 10: People want to feel like they made a difference. Like
Speaker 10: people want to feel like their lives matter, and that's
Speaker 10: that's a huge part of the hook is like what
Speaker 10: if this is like the most significant era of human
Speaker 10: history that there ever has been, and like the choices
Speaker 10: we make now are going to like reverberate into the future.
Speaker 10: You know, like people on Twitter who started like getting
Speaker 10: really into this stuff will uses like very hyper block clamers.
Speaker 10: Oh yeah, we're going to like conquer the stars, you know,
Speaker 10: We're going to like go create a galactic civilization. Like
Speaker 10: I think some people think that those guys are exaggerating
Speaker 10: and joking, Like I think they should be taking exactly
Speaker 10: at face value, Like literally, that's what people think the sticks.
Speaker 1: Are believing that this is the most significant era in
Speaker 1: human history because of what we're building with AI. You
Speaker 1: know who that sounds like Sam? In speech after speech
Speaker 1: over the years, he has said that the AI we're
Speaker 1: working on now is going to be historically significant. Here
Speaker 1: he is with my colleague Emily Chang.
Speaker 8: I think we have an opportunity that comes along only
Speaker 8: every couple of centuries to redo the socioeconomic contract. And
Speaker 8: how we include everybody in that, make everybody a winner,
Speaker 8: and how we don't destroy ourselves in the process is
Speaker 8: a huge question. You know, what does it meaning to
Speaker 8: build something that is more capable than ourselves, Like, what
Speaker 8: does that say about our humanity? What's that world going
Speaker 8: to look like? What's in place in that world? How
Speaker 8: is that going to be equitably shared? How do we
Speaker 8: make sure that it's not like a handful of people
Speaker 8: in San Francisco making the decisions and reaping all the benefits.
Speaker 1: He says that the AI being built now will reshape
Speaker 1: the world, our humanity, our social contract. His words are
Speaker 1: brimming with the heroism that Emily and Chow Chew are
Speaker 1: pointing out. It's pretty common for people in tech to
Speaker 1: think they are uniquely smart enough to fix big hairy
Speaker 1: problems like the tech billionaires who are creating a new
Speaker 1: city near San Francisco, or the entire crypto industry, which
Speaker 1: thinks it has invented a superior financial system. Maybe Sam
Speaker 1: sees himself as a hero and he understands the motivating
Speaker 1: power of a good story. In twenty nineteen, he even
Speaker 1: hired a fiction writer on contract to write for open
Speaker 1: ai for a few months.
Speaker 9: I ended up basically writing a novella full of short stories,
Speaker 9: science fiction short stories.
Speaker 1: That's Patrick House. He's a neuroscientist and an author. He
Speaker 1: says he has no idea whether OpenAI still uses his
Speaker 1: novella in any way, but they saw value in commissioning it.
Speaker 9: Sam Maltman is influenced by certain kinds of fiction. A
Speaker 9: lot of startups are kind of motivated by story, and
Speaker 9: that this story often comes from science fiction.
Speaker 1: A powerful story can make your employees work really hard.
Speaker 9: How do you motivate people, like, how do you motivate people,
Speaker 9: especially in the like mostly secular San Francisco. Maybe you
Speaker 9: give him a foundational document in an apocalypse myth and
Speaker 9: you know, tell them they're averting, averting the end times,
Speaker 9: and that's a tried and true, historically known way to
Speaker 9: motivate people.
Speaker 1: All of this made me wonder, does Sam believe that
Speaker 1: AI might destroy humanity? Honestly, it's hard to tell. His
Speaker 1: answers have changed over the years. In twenty fifteen, he
Speaker 1: very clearly wrote that he believes advanced AI is quote
Speaker 1: probably the greatest threat to the continued existence of humanity.
Speaker 1: We heard him say in the past with a kind
Speaker 1: of funny tone.
Speaker 8: I think AI will probably like most likely sort of
Speaker 8: lead to the end of the world.
Speaker 1: And around that same time, in a New Yorker profile,
Speaker 1: Sam basically said he was a doomsday prepper. He told
Speaker 1: a reporter that he had stockpiled guns, gold, ammonium, antibiotics,
Speaker 1: and gas masks from the Israeli Defense Force. He said
Speaker 1: he has a big patch of land in Big Sur
Speaker 1: that he can fly to in case the world crumbles.
Speaker 1: This was really funny to me and probably to anyone
Speaker 1: who has visited Big Sur, because the land out there
Speaker 1: is literally crumbling right now, leading to rock slides, road closures,
Speaker 1: and difficulty getting food and supplies in and out. Seems
Speaker 1: like a precarious place for your apocalypse bunker. When another
Speaker 1: reporter asked him follow up questions. Sam tried to blow
Speaker 1: it off. He said that he does this stuff for
Speaker 1: fun because it quote appeals to little boys survival fantasies.
Speaker 1: In the years then, Sam has notably avoided mentioning the
Speaker 1: big sur property or his stockpile of guns. Annie, his sister,
Speaker 1: says she never saw the property, but that it's in line.
Speaker 8: With what she knows about him.
Speaker 7: He's big into that or was uh safety like guns gold,
Speaker 7: how to stock up on the worst case scenario apocalyptic
Speaker 7: movie scene events. My guess would be that he has
Speaker 7: it and stop talking about it.
Speaker 1: Sam is a savvy guy. As his profile has gotten
Speaker 1: bigger after he helped build the world's leading AI company,
Speaker 1: he has stopped saying things like AI will kill us all. Instead,
Speaker 1: he talks about how society will be profoundly changed, but
Speaker 1: overall it will be for the better. Since his newfound
Speaker 1: chat GPT fame, he has shifted toward presenting himself and
Speaker 1: by extension, open AI, as more middle of the road.
Speaker 1: Sam is allowed to change his views, but people have
Speaker 1: also so complain to me in private that Sam has
Speaker 1: a tendency to talk out of both sides of his mouth.
Speaker 1: He's good at telling people what they want to hear
Speaker 1: in that moment, so it's not surprising that if it's
Speaker 1: advantageous for him to seem more moderate, that he would
Speaker 1: start to sound that way. In any case, he definitely
Speaker 1: knows how powerful an explosive apocalypse story is. We trigger
Speaker 1: more easily on dramatic fears than boring ones. Sam says
Speaker 1: this himself here he is speaking at an Airbnb conference
Speaker 1: in twenty fifteen. He's talking about nuclear power. But it's
Speaker 1: an analogy that maps very clearly onto AI existential risk,
Speaker 1: and you can hear again Sam's undertone of arrogance, pointing
Speaker 1: out how he's aware of something rational that other people
Speaker 1: are blind to.
Speaker 8: People are much more sensitive to sort of like theatrical
Speaker 8: extreme risk than they are to sort of like boring,
Speaker 8: slow plotting risk. Like nuclear energy said, unbelievable safety record.
Speaker 8: You know, it's like a thousand or ten thousand times
Speaker 8: safer than coal, But most people would they rather live
Speaker 8: next to a coal plant or a nucleanertry plant. They
Speaker 8: picked the coal plant all day long, and when they
Speaker 8: die in thirty years of lung cancer, it doesn't feel
Speaker 8: as sort of dramatic is dying in a nuclear meltdown,
Speaker 8: and no, it's true, like this is like the human
Speaker 8: risk miscalculation that always happens.
Speaker 5: Right.
Speaker 8: People always underweight the boring slow stuff and overweight the
Speaker 8: quick dramatic stuff.
Speaker 1: People underweight the boring slow stuff like misinformation, racial bias,
Speaker 1: and they overweight the big dramatic stuff. That may be
Speaker 1: why the AI industry has been telling us disaster stories.
Speaker 1: You've just listened to part one of Heaven and Hell.
Speaker 1: Listen to part two for the rest of this story.
Speaker 1: Foundering is hosted by me Ellen Hewitt. Sean Wen is
Speaker 1: our executive producer. Rachel Metz contributed reporting to this episode.
Speaker 1: Molly Nugent is our associate producer. Blake Maples is our
Speaker 1: audio engineer, Mark Million, Anne Vandermay, Seth Fiegerman, Tom Giles,
Speaker 1: and Molly Schutz are our story editors. We had production
Speaker 1: help from Jessica Nix and Antonia mufferetch. Thanks for listening.
Speaker 1: If you like our show, leave a review, and most importantly,
Speaker 1: tell your friends. See you next time.