WEBVTT - OpenAI Part 3: Heaven and Hell, Part 1 0:00:00.080 --> 0:00:03.040 Today, we're going to start on a drive in Hawaii. 0:00:05.600 --> 0:00:09.399 We're on north Shore, going deeper into the jungle on 0:00:09.440 --> 0:00:12.360 the north shore, so we're passing twin falls right now. 0:00:12.440 --> 0:00:16.160 I'm driving through the lush green forests of Maui. Annie Altman, 0:00:16.360 --> 0:00:19.119 Sam Altman's little sister is sitting in the passenger seat. 0:00:19.640 --> 0:00:21.760 You heard from her briefly in the first episode. 0:00:22.079 --> 0:00:24.160 And I love this because you wouldn't turn here if 0:00:24.160 --> 0:00:25.200 you didn't know you could turn in. 0:00:25.760 --> 0:00:27.800 I certainly would not be driving down this if I 0:00:27.840 --> 0:00:30.720 didn't tell you a tiny little round. We're taking a 0:00:30.760 --> 0:00:33.360 tour of the different places Annie has moved around in 0:00:33.400 --> 0:00:36.680 the last couple of years, driving down dirt roads to 0:00:36.720 --> 0:00:40.600 look at cabins and houses hidden behind enormous tropical plants. 0:00:40.880 --> 0:00:41.880 We got a huge monster. 0:00:42.000 --> 0:00:42.519 I look at that. 0:00:43.520 --> 0:00:45.600 It's so it's so goofy. Then I'm driving this, I 0:00:45.680 --> 0:00:47.880 know because then you see it in someone's office plant 0:00:47.880 --> 0:00:49.040 and you're like, is that the same thing. 0:00:49.159 --> 0:00:51.919 For much of the past two years, Annie hasn't been 0:00:51.960 --> 0:00:53.920 able to afford a stable place to live. 0:00:54.480 --> 0:00:58.080 The place you just passed is one of the places 0:00:58.120 --> 0:00:59.200 I stayed at longer term. 0:00:59.240 --> 0:01:04.479 In all of the houselessness two months on a newly 0:01:04.520 --> 0:01:08.480 built no running water or no electricity house at the 0:01:08.600 --> 0:01:09.920 far and back of the property. 0:01:10.240 --> 0:01:13.000 And I think she's an important part of Sam's story. 0:01:13.200 --> 0:01:17.800 And at the time I had nowhere to stay and 0:01:17.840 --> 0:01:22.840 no rent money, certainly no deposit money, and barely enough room, 0:01:22.920 --> 0:01:24.120 barely enough money for rent. 0:01:24.680 --> 0:01:27.520 Recently, over the course of just a year, she moved 0:01:27.640 --> 0:01:31.080 twenty two times, and that's on average about twice a month. 0:01:31.959 --> 0:01:34.479 Sometimes she has stayed places for a week at a time, 0:01:34.760 --> 0:01:37.200 or even just a night or two. Some of them 0:01:37.240 --> 0:01:40.720 have been illegal rentals without running water. She says she's 0:01:40.720 --> 0:01:44.000 slept on floors and friends' houses. She stayed with strangers 0:01:44.040 --> 0:01:45.199 when she didn't have another option. 0:01:45.640 --> 0:01:49.760 The man who lived in the front house messaged me 0:01:49.840 --> 0:01:54.640 on Instagram, and I stayed in his kids. 0:01:54.560 --> 0:01:57.000 Room the week that they weren't there, and then slept 0:01:57.040 --> 0:01:59.320 on the floor in the common room the week that 0:01:59.360 --> 0:02:00.160 the kids were there. 0:02:00.400 --> 0:02:02.440 I was houseless. I didn't have somewhere to go. I 0:02:02.520 --> 0:02:05.720 stayed in this cabin with destly anti roof right there 0:02:05.960 --> 0:02:07.320 for three months. 0:02:08.160 --> 0:02:10.200 How many different places have you lived in that didn't 0:02:10.240 --> 0:02:11.920 have running water? 0:02:13.720 --> 0:02:16.639 Maybe five ish five or sus six, I don't know. 0:02:16.800 --> 0:02:20.880 Meanwhile, thousands of miles away in San Francisco, her brother 0:02:21.000 --> 0:02:25.000 Sam was having a spectacular year. In twenty twenty three, 0:02:25.600 --> 0:02:29.079 the success of chat GBT had launched open Ai into 0:02:29.120 --> 0:02:33.079 the stratosphere. Sam was named CEO of the Year by 0:02:33.120 --> 0:02:37.280 Time magazine. He spent months flying around the globe talking 0:02:37.280 --> 0:02:38.840 to world leaders about AI. 0:02:39.560 --> 0:02:40.480 So what outforted you? 0:02:40.560 --> 0:02:42.359 Let's give a big round of applause. 0:02:42.400 --> 0:02:43.520 There a special guests. 0:02:43.360 --> 0:02:43.959 Sam Altman. 0:02:44.120 --> 0:02:47.040 It is with great pleasure that I invite on stage 0:02:47.400 --> 0:02:48.160 Sam Altman. 0:02:48.440 --> 0:02:53.640 The individual transcends introduction, mister Sam Altman, the CEO pop 0:02:53.680 --> 0:02:54.200 open AI. 0:02:56.160 --> 0:02:59.839 Sure, Sam had been positioning himself as a public intellectual 0:03:00.040 --> 0:03:03.120 for some time, but for a long while he had 0:03:03.200 --> 0:03:07.880 merely been Silicon Valley famous. This was a new level 0:03:07.919 --> 0:03:11.600 of fame. Suddenly it seemed like Sam Altman was a 0:03:11.639 --> 0:03:17.600 spokesman for this entire AI boom. On stage, on podcasts 0:03:17.600 --> 0:03:22.120 and interviews, people kept turning to Sam for answers. They 0:03:22.120 --> 0:03:26.240 were asking him what our AI future would hold. In 0:03:26.280 --> 0:03:29.880 May of that year, he confidently suggested a future where 0:03:29.919 --> 0:03:33.119 no one is poor. It's an idea he's talked about 0:03:33.160 --> 0:03:36.480 for years, and the remarks show that his tune hasn't 0:03:36.600 --> 0:03:39.000 changed despite growing renown and wealth. 0:03:39.360 --> 0:03:41.600 One thing I think we all could agree on is 0:03:41.640 --> 0:03:44.320 that we just shouldn't have poverty in the world. 0:03:44.720 --> 0:03:48.080 So he's saying something like poverty, which is as deeply 0:03:48.280 --> 0:03:52.160 entrenched as civilization itself, could be fixed. In the age 0:03:52.200 --> 0:03:55.520 of AI. Everyone will have what they need to live, 0:03:56.160 --> 0:03:57.000 including a home. 0:03:57.240 --> 0:03:59.080 I think we are not that far away from being 0:03:59.080 --> 0:04:05.240 able to eliminate poverty asthectically worldwide, certainly developed countries, and 0:04:05.320 --> 0:04:08.600 I think there are things that are fifty years from 0:04:08.600 --> 0:04:12.560 now will be basic human rights, like healthcare and enough 0:04:12.560 --> 0:04:15.120 food to eat and a place to live. I'm confident 0:04:15.200 --> 0:04:16.400 we're going to get that done. 0:04:16.560 --> 0:04:20.680 He sounds so sure of himself. How will these changes happen? 0:04:21.880 --> 0:04:26.200 Sam says that they could come from AI. He's basically 0:04:26.200 --> 0:04:30.960 saying the technology his company is working on can eliminate poverty. 0:04:31.120 --> 0:04:33.640 Here he is at a Bloomberg conference in June twenty 0:04:33.640 --> 0:04:34.159 twenty three. 0:04:34.520 --> 0:04:36.280 I think it'd be good to ten poverty. Maybe you 0:04:36.320 --> 0:04:37.960 think we should stop a technology they can do that, 0:04:38.000 --> 0:04:38.799 I personally don't. 0:04:39.320 --> 0:04:42.040 He's even scolding people who say they want to stop 0:04:42.160 --> 0:04:45.599 or slow down the development of AI by saying that 0:04:45.680 --> 0:04:49.400 they want to keep people poor. Solving poverty seems to 0:04:49.400 --> 0:04:52.719 become part of his personal brand. There's almost a stump 0:04:52.800 --> 0:04:55.960 speech of sorts, and there's one word he likes to 0:04:56.080 --> 0:04:58.719 use to encapsulate our bountiful future. 0:04:59.040 --> 0:05:01.039 I think it's how we get to this world of abundance. 0:05:01.080 --> 0:05:03.320 In a world with the level of abundance AI and 0:05:03.360 --> 0:05:04.920 the abundance that comes with that, If we can have 0:05:04.960 --> 0:05:09.080 abundant and cheap energy and intelligence that will transform people's lives. 0:05:09.720 --> 0:05:14.400 It sounds wonderful, almost utopian. But Sam was saying on 0:05:14.600 --> 0:05:18.800 stage that everyone should have enough money, enough food, everyone 0:05:18.800 --> 0:05:21.800 should have a place to live, while his own sister 0:05:22.080 --> 0:05:26.680 was struggling with homelessness. I want to believe Sam's promises 0:05:26.760 --> 0:05:30.680 about abundance, but Annie's story complicates a lot of the 0:05:30.720 --> 0:05:38.240 things Sam has projected about the future. You're listening to Foundering. 0:05:38.680 --> 0:05:42.600 I'm your host, Ellen Hewitt. By the time open ai 0:05:42.720 --> 0:05:45.880 had been around for a few years, its employees began 0:05:46.040 --> 0:05:50.400 sensing that they were working on something profound. AI was 0:05:50.440 --> 0:05:53.760 starting to do things even its creators didn't quite understand, 0:05:54.520 --> 0:05:59.120 and the response divided people. Some people at open ai, 0:05:59.400 --> 0:06:04.080 including Sam, argued that an all powerful computer intelligence would 0:06:04.080 --> 0:06:07.440 be good for the world. Some thought it could be very, 0:06:07.560 --> 0:06:13.160 very bad. These visions were extreme, polarizing, as different as 0:06:13.240 --> 0:06:17.120 heaven and hell. This period ultimately led up to the 0:06:17.160 --> 0:06:20.680 release of chat GPT, which was a defining moment for 0:06:20.760 --> 0:06:25.560 the company and for the entire AI complex. The two 0:06:25.680 --> 0:06:29.159 predictions people talked about the most during this time were 0:06:29.200 --> 0:06:32.560 that AI would either wipe out the human race or 0:06:32.760 --> 0:06:36.599 completely change the definition of work. The first one is 0:06:36.720 --> 0:06:41.919 hugely controversial, the second one is less controversial. People will 0:06:41.960 --> 0:06:46.080 lose jobs and the economy will have to adapt. I'm 0:06:46.120 --> 0:06:49.919 going to tell this story in two parts. First, the 0:06:49.960 --> 0:06:53.720 disputes within open Ai, the reckoning with what they'd built, 0:06:54.360 --> 0:06:57.440 and the elevation of Sam as some kind of AI hero. 0:06:58.960 --> 0:07:01.760 In the second part, we'll hear more from Sam's sister 0:07:02.440 --> 0:07:05.799 and explore the debate around poverty and what an AI 0:07:05.920 --> 0:07:10.080 economy might look like. And you'll see exactly how far 0:07:10.160 --> 0:07:14.880 away Annie's life is from Sam's vision of AI utopia. 0:07:16.000 --> 0:07:16.800 We'll be right back. 0:07:21.440 --> 0:07:25.360 Let's pick the story up in twenty twenty one. It's 0:07:25.400 --> 0:07:28.200 about a year and a half before open Ai launches 0:07:28.320 --> 0:07:32.840 chat GPT, and open AI's technology had been rapidly gaining 0:07:33.040 --> 0:07:34.120 new capabilities. 0:07:34.720 --> 0:07:38.840 This was around the time that OpenAI was building more 0:07:38.880 --> 0:07:42.280 and more powerful models such as GPT two and GBT 0:07:42.400 --> 0:07:47.000 three that could ingest text and spit out ever more 0:07:47.080 --> 0:07:48.200 human sounding text. 0:07:48.600 --> 0:07:51.080 That's my colleague Rachel Metz, who covers AI. 0:07:51.680 --> 0:07:56.440 They were focused on building increasingly powerful AI models, doing 0:07:56.480 --> 0:07:59.640 it very rapidly, moving towards this goal that they have 0:07:59.720 --> 0:08:03.680 long stated of creating artificial general intelligence. 0:08:04.480 --> 0:08:07.640 This rapid progress was good. It was great for Sam, 0:08:07.880 --> 0:08:11.000 great for the company, but it made some open ai 0:08:11.120 --> 0:08:16.440 employees nervous, even ones very high up. Here's Dario Amidae, 0:08:17.040 --> 0:08:19.600 who at the time was the vice president of research 0:08:19.640 --> 0:08:23.280 at open Ai. He's talking on the Logan Bartlett podcast 0:08:23.680 --> 0:08:25.760 and he sounds genuinely agitated. 0:08:26.320 --> 0:08:29.040 So, you know, I really freaked out about this stuff. 0:08:29.160 --> 0:08:32.320 In twenty eighteen or twenty nineteen or so. The first 0:08:32.360 --> 0:08:34.520 time I looked at GBT too, I was like, oh my, 0:08:34.679 --> 0:08:36.680 this is like this is crazy. This is you know, 0:08:36.720 --> 0:08:39.240 there's there's nothing like, there's nothing like this in the world, 0:08:39.320 --> 0:08:40.800 Like it's crazy that this is possible. 0:08:41.120 --> 0:08:43.720 He looked at what he was helping build at open ai, 0:08:44.240 --> 0:08:46.520 and his overwhelming emotion was fear. 0:08:47.120 --> 0:08:48.000 Why am I scared. 0:08:48.240 --> 0:08:51.080 Okay, this thing's going to be powerful. It could destroy us, 0:08:51.280 --> 0:08:54.360 and like all the ones built so far, like you know, 0:08:54.760 --> 0:08:57.120 are at pretty decent risk of doing some random shit. 0:08:57.160 --> 0:09:01.520 We don't understand if such a model wanted to recavoc 0:09:01.559 --> 0:09:05.000 and destroy humanity or whatever. I think we have basically 0:09:05.040 --> 0:09:06.080 no ability to stop it. 0:09:06.440 --> 0:09:08.960 That's from a tech show called the Door Kesh podcast. 0:09:09.720 --> 0:09:13.040 You can hear Dario's uneasiness, which spread to some of 0:09:13.080 --> 0:09:16.400 his peers at work. In twenty twenty one, he and 0:09:16.480 --> 0:09:19.079 six of his open ai colleagues all left at once 0:09:19.720 --> 0:09:23.360 and they started a new rival company focused on building 0:09:23.400 --> 0:09:24.720 AI that was safe. 0:09:24.880 --> 0:09:27.920 We need to make these models safe in a certain way, 0:09:28.440 --> 0:09:30.560 and you know, we need to do them within an 0:09:30.640 --> 0:09:33.839 organization where we can really believe that these principles are 0:09:34.280 --> 0:09:35.760 incorporated top to bottom. 0:09:36.280 --> 0:09:40.120 While his tone sounds innocuous, if you read between the lines, 0:09:40.520 --> 0:09:43.520 he's implying that open ai was not that concerned with 0:09:43.559 --> 0:09:47.840 safety and not following those principles top to bottom. He 0:09:47.960 --> 0:09:50.280 and his co founders felt they needed to create a 0:09:50.320 --> 0:09:54.200 whole new company focused on preventing AI from potentially wiping 0:09:54.280 --> 0:09:58.559 us all out. This idea was baked into their name anthropic, 0:09:59.040 --> 0:10:03.400 which refers to the existence of humanity on Earth. Anthropics 0:10:03.440 --> 0:10:06.959 split from open ai was a big deal. The group 0:10:07.000 --> 0:10:10.760 of employees who walked away included executives and key people 0:10:10.840 --> 0:10:13.920 who had worked there for a long time. It cast 0:10:14.040 --> 0:10:19.000 this suspicious pall over open Ai. Their exit suggested that 0:10:19.080 --> 0:10:24.400 they didn't trust Sam to do the right thing. So 0:10:24.679 --> 0:10:28.160 fast forward to late twenty twenty two, open Ai released 0:10:28.320 --> 0:10:32.679 chat GBT. They weren't expecting a huge response, but it 0:10:32.760 --> 0:10:37.840 drew people in gradually. Then suddenly everyone was trying it out. 0:10:38.480 --> 0:10:39.439 Here's Rachel again. 0:10:40.000 --> 0:10:41.960 I kept an eye on it for a few days 0:10:42.320 --> 0:10:45.320 and I started noticing people were talking about it. But 0:10:45.360 --> 0:10:49.240 it started getting more and more traction, and I remember 0:10:49.400 --> 0:10:52.679 reaching out to my editors and saying, hey, we should 0:10:52.679 --> 0:10:53.480 write about this. 0:10:54.200 --> 0:10:58.520 Chat GBT wasn't new technology. It was basically powered by 0:10:58.559 --> 0:11:02.360 GPT three point five, pre existing model, but it was 0:11:02.480 --> 0:11:05.320 a new way of presenting that technology to the world. 0:11:06.200 --> 0:11:09.720 It was a free, easy to use chat based AI tool. 0:11:10.480 --> 0:11:12.360 It felt a little like texting a friend. 0:11:13.120 --> 0:11:17.160 This was much more fluid feeling. It gave you a 0:11:17.240 --> 0:11:21.200 sense that you were communicating with a person, a person 0:11:21.880 --> 0:11:24.360 that would make a lot of things up right, who's 0:11:24.360 --> 0:11:26.200 the answers? You would have to constantly. 0:11:25.840 --> 0:11:31.440 Check and people were hooked. Chat gpt reached one hundred 0:11:31.559 --> 0:11:35.559 million users in just two months, the fastest growth ever 0:11:35.679 --> 0:11:39.760 at the time. For reference, it took TikTok nine months 0:11:39.960 --> 0:11:42.959 and Instagram two and a half years to reach the 0:11:43.040 --> 0:11:48.600 same popularity. Chat gpt felt intelligent even when it got 0:11:48.600 --> 0:11:52.400 things wrong. It prompted a lot of people to ask themselves, 0:11:53.000 --> 0:11:57.040 how is AI going to change my life? The technology 0:11:57.080 --> 0:12:01.840 had some immediate practical uses. Started using chat gpt to 0:12:01.960 --> 0:12:06.200 write code more quickly, to translate documents more fluidly, and 0:12:06.280 --> 0:12:10.480 to draft emails. Students used it for homework help good 0:12:11.160 --> 0:12:15.600 and to cheat on papers not so good. Needless to say, 0:12:15.920 --> 0:12:18.880 all this excitement was a huge boost for open Ai. 0:12:19.800 --> 0:12:25.240 With chat gbt, OpenAI basically skyrocketed to brand awareness and success, 0:12:25.760 --> 0:12:31.320 and people immediately started thinking of it as the hot 0:12:31.360 --> 0:12:34.320 AI company, a leader in AI. 0:12:35.880 --> 0:12:40.600 It also skyrocketed Sam's public profile. Now Ai was the 0:12:40.760 --> 0:12:44.520 story of twenty twenty three, and Sam was the main character. 0:12:45.280 --> 0:12:49.520 He became a household name and across the tech industry. 0:12:49.800 --> 0:12:54.559 Investors were desperate to throw money at AI. Companies pivoted 0:12:54.600 --> 0:12:59.360 their focus to AI, and highway billboards touted new AI startups, 0:13:00.360 --> 0:13:03.760 and this chat gbt mania did something else too. 0:13:04.480 --> 0:13:07.839 In addition to making all kinds of people interested in 0:13:08.200 --> 0:13:11.440 and aware of cutting edge AI, the release of chat 0:13:11.520 --> 0:13:18.640 gbt inflamed this small subsection of people that have long 0:13:18.720 --> 0:13:23.160 been interested in this idea that AI is going to 0:13:23.200 --> 0:13:26.800 become more and more powerful, and that it will inevitably 0:13:27.320 --> 0:13:31.120 go rogue and threaten us, and we have to figure 0:13:31.160 --> 0:13:32.839 out how to save humanity from it. 0:13:33.440 --> 0:13:37.440 Fear of AI destroying the world. It's the fear that 0:13:37.559 --> 0:13:42.200 Dario described feeling before he quit Open AI to found Anthropic. 0:13:43.320 --> 0:13:46.520 It might sound a little ridiculous, but to many people 0:13:46.800 --> 0:13:52.600 it's dead serious. This Silicon Valley subculture that believes AI 0:13:52.720 --> 0:13:56.959 might destroy us soon has steadily been growing more influential 0:13:57.040 --> 0:14:01.760 and powerful. These ideas are a major motivating force in 0:14:01.880 --> 0:14:06.440 AI right now. Many tech workers in AI feel that 0:14:06.600 --> 0:14:10.160 they are working on the most consequential thing possible for 0:14:10.240 --> 0:14:13.880 the human race, a matter of life and death for 0:14:13.960 --> 0:14:18.680 the whole planet. This belief influences where billions of dollars 0:14:18.720 --> 0:14:22.320 are directed, and which problems get worked on and which 0:14:22.360 --> 0:14:27.320 ones get ignored. Chat GBT brought these fears into the 0:14:27.360 --> 0:14:32.400 public consciousness big time. One survey in the UK shows 0:14:32.440 --> 0:14:35.600 that in the course of one year, right after chat 0:14:35.600 --> 0:14:39.720 gbt's release, the percentage of people who believed that AI 0:14:40.120 --> 0:14:44.280 was a top possible cause of human extinction more than doubled. 0:14:45.520 --> 0:14:48.880 To be clear, this claim is hotly disputed. A lot 0:14:48.960 --> 0:14:52.760 of people, even in Silicon Valley, think AI numerism is 0:14:52.920 --> 0:14:59.000 overblown or a quasi religious ideology, but many take it seriously. 0:15:00.120 --> 0:15:03.800 We'll be right back. I want to talk about how 0:15:03.800 --> 0:15:09.040 it feels to believe AI might destroy humanity. Here's Chao 0:15:09.120 --> 0:15:12.720 Chu Yun. He's just one of many people in tech 0:15:12.800 --> 0:15:17.040 who held this belief. His introduction to this world was 0:15:17.640 --> 0:15:22.600 bizarrely through Harry Potter fan fiction he found online that 0:15:22.720 --> 0:15:27.560 was written by a very influential AI doomer named Eliezer Yudkowski. 0:15:28.720 --> 0:15:32.200 That discovery led him to lots more of Eliezer's writing 0:15:32.320 --> 0:15:36.400 about AI. He eventually became convinced that the world was 0:15:36.560 --> 0:15:39.640 probably going to be swallowed by super intelligent AI. 0:15:40.280 --> 0:15:44.120 Soon I was like, Okay, it's happening, We're getting there. 0:15:44.160 --> 0:15:46.520 It's going to happen in my lifetime. It's going to 0:15:46.520 --> 0:15:51.160 happen like before I retire. I made a conscious decision 0:15:51.160 --> 0:15:54.000 around this time to not open a retirement account, which 0:15:54.120 --> 0:15:56.040 some people might think was silly, but I was like, 0:15:56.120 --> 0:15:59.840 I really do not think money is going to matter 0:16:00.240 --> 0:16:03.600 by the time I hit retirement age. I think by 0:16:03.600 --> 0:16:06.280 the time I hit retirement age, the world is going 0:16:06.360 --> 0:16:08.800 to be unrecognizably strange. I just don't think money is 0:16:08.840 --> 0:16:12.000 going to be a thing. Either we're all going to 0:16:12.000 --> 0:16:14.120 be dead, or we're going to be into some crazy 0:16:14.200 --> 0:16:17.200 post Singularity environment where it's just I just don't think 0:16:17.200 --> 0:16:17.960 it's going to matter. 0:16:18.360 --> 0:16:22.200 The Singularity is the point when robots become smarter than humans. 0:16:23.560 --> 0:16:27.000 For him, it went beyond skipping out on a retirement account. 0:16:27.760 --> 0:16:31.400 Once chou Chu really started believing this, other things in 0:16:31.400 --> 0:16:35.440 his life just didn't seem to matter. He dropped out 0:16:35.480 --> 0:16:39.400 of his math PhD program five years in. He drifted 0:16:39.440 --> 0:16:43.400 away from old friends. He thought they didn't understand the 0:16:43.480 --> 0:16:44.840 magnitude of the situation. 0:16:45.480 --> 0:16:49.240 It was like this black hole in my kind of 0:16:49.440 --> 0:16:52.640 sense of in which things were important, and like sort 0:16:52.680 --> 0:16:54.840 of the closer I got to it, like the less 0:16:54.880 --> 0:16:56.560 important everything else seemed that I was like, oh, well, 0:16:56.600 --> 0:16:58.360 there's this other stuff that doesn't really matter. There's this 0:16:58.360 --> 0:17:00.680 other stuff that doesn't really matter, and this is like 0:17:00.720 --> 0:17:01.800 maybe the only thing that matters. 0:17:03.720 --> 0:17:09.199 I know this sounds scary, extreme and maybe ridiculous, but 0:17:09.359 --> 0:17:12.480 if you spend enough time talking to people in AI 0:17:12.680 --> 0:17:17.680 and around Silicon Valley, you will hear this attitude. Sometimes 0:17:17.680 --> 0:17:22.320 it's whispered, sometimes it's said loudly. Sometimes they'll use terms 0:17:22.359 --> 0:17:27.520 like AI safety, AI alignment, or AI existential risk, but 0:17:27.560 --> 0:17:31.679 they're all talking about the same thing. There are significant 0:17:31.800 --> 0:17:35.600 numbers of influential people in AI who believe that our 0:17:35.720 --> 0:17:39.280 world in twenty years will be utterly unrecognizable from the 0:17:39.280 --> 0:17:43.159 world of today. Some people think it'll be way better, 0:17:43.720 --> 0:17:47.199 but a lot think it might be way worse, and 0:17:47.240 --> 0:17:50.280 they feel that they should devote their lives to preventing 0:17:50.320 --> 0:17:54.720 AI doomsday. In the last five or ten years, a 0:17:54.760 --> 0:17:58.400 lot of famous, rich tech people have thrown money at 0:17:58.400 --> 0:18:03.840 this new cause. Facebook co founder Dustin Moskovitz and crypto 0:18:03.960 --> 0:18:08.439 fraudster Sam Bankman Freed each pledged hundreds of millions of 0:18:08.440 --> 0:18:13.040 dollars toward AI safety projects. Tech leaders have signed public 0:18:13.040 --> 0:18:17.280 statements about how AI could put our human civilization at risk. 0:18:19.240 --> 0:18:22.480 Chau Chu and his fellow adherents felt it was deeply 0:18:22.520 --> 0:18:26.520 important to get other people to accept this threat. They 0:18:26.560 --> 0:18:30.720 became evangelists. At events. They would trap people in long 0:18:30.840 --> 0:18:34.000 conversations about AI apocalypse. 0:18:33.640 --> 0:18:35.840 And they started off skeptical and then they ended up 0:18:35.920 --> 0:18:38.720 somewhere between like either like really convinced or like scared 0:18:38.840 --> 0:18:41.280 of just like oh, oh this is terrifying. This is 0:18:41.280 --> 0:18:43.680 like really like I'm like kind of sicked up about this. 0:18:43.680 --> 0:18:45.520 This was a very frightening idea. It's like, oh, hey, 0:18:45.640 --> 0:18:48.639 what if everyone dies in ten years? That's a scary idea, 0:18:49.200 --> 0:18:54.000 and like it's it's it's sort of safer psychologically to 0:18:54.080 --> 0:18:56.280 just dismiss it as like that's ridiculous, that could never happen. 0:18:56.680 --> 0:18:59.919 But if someone really like get gets to you about that, 0:19:00.400 --> 0:19:02.680 like as someone you know spends four hours talking to 0:19:02.800 --> 0:19:06.399 him very carefully convincing you, like what if it's possible? Though, 0:19:06.440 --> 0:19:10.480 Like that's really that's very scary. That's like oh that, 0:19:11.040 --> 0:19:11.560 how do I? 0:19:11.680 --> 0:19:11.880 Oh? 0:19:12.119 --> 0:19:14.520 Like that's like terriff Like what do I do about that? 0:19:14.880 --> 0:19:17.720 There's a world The difference between like reflexively dismissing that 0:19:17.800 --> 0:19:21.200 idea and like really considering it seriously as a possibility. 0:19:21.520 --> 0:19:22.680 On an emotional level. 0:19:23.400 --> 0:19:27.840 Chou Chu took jobs at AI safety organizations. He volunteered 0:19:27.840 --> 0:19:30.800 at rationality workshops to try to get more young minds 0:19:30.880 --> 0:19:34.680 working on AI safety. In this world, when people talk 0:19:34.720 --> 0:19:39.879 about what's at stake, it's on a galactic scale. Okay, 0:19:39.960 --> 0:19:42.600 So that's one point of view that the most important 0:19:42.640 --> 0:19:45.199 thing right now is to make sure AI is safe 0:19:45.240 --> 0:19:49.199 in the future. But there are some really smart people, 0:19:49.560 --> 0:19:54.520 academics and researchers who find this AI doomsday frenzy very 0:19:54.560 --> 0:19:59.760 frustrating and harmful. Like Emily Bender, an academic who specializes 0:20:00.080 --> 0:20:01.480 computational linguistics. 0:20:02.000 --> 0:20:05.399 It's a distraction because there's all kinds of harms that 0:20:05.440 --> 0:20:07.920 are happening right now in terms of labor exploitation, in 0:20:08.000 --> 0:20:11.080 terms of data theft, in terms of discriminative outcomes, in 0:20:11.160 --> 0:20:16.199 terms of representational harms. And the more we focus on 0:20:16.440 --> 0:20:21.880 these very sort of exciting in the way that action 0:20:22.000 --> 0:20:25.640 movies are exciting fantasy scenarios at exisential risks, the less 0:20:25.680 --> 0:20:27.919 time and effort goes in to actually dealing with the 0:20:27.960 --> 0:20:29.440 real harms that are happening right now. 0:20:31.040 --> 0:20:34.080 Emily and lots of other experts argue that all of 0:20:34.119 --> 0:20:38.440 this talk of AI doomsday is an enormous, dangerous distraction, 0:20:39.280 --> 0:20:42.520 like a big flashing alarm screaming We're all going to die. 0:20:43.119 --> 0:20:46.320 It steals all the air in the room. It makes 0:20:46.359 --> 0:20:50.240 it easy to ignore issues happening today, like racial bias 0:20:50.400 --> 0:20:54.560 in AI systems used in criminal justice, stealing copyrighted work 0:20:54.600 --> 0:20:58.000 of artists to train models, and so many more. 0:20:58.960 --> 0:21:02.600 You've got the surveillance applications of these technologies that are 0:21:02.640 --> 0:21:07.119 being used to over police communities, or performance of facial 0:21:07.160 --> 0:21:11.080 recognition technology, especially for women with darker skin in particular 0:21:11.119 --> 0:21:13.720 Google and putting identity terms up for sale to search 0:21:13.760 --> 0:21:15.240 for black girls, you get a whole. 0:21:15.119 --> 0:21:17.920 Bunch of porn. There's enormous risks around synthetic media, all 0:21:17.960 --> 0:21:21.800 of our ability to find trustworthy information and then trust 0:21:21.800 --> 0:21:24.920 it when we find it is threatened helblic health, democracy. 0:21:25.960 --> 0:21:30.000 The list goes on and on. Emily reminds us that 0:21:30.040 --> 0:21:33.560 these are urgent problems, ways that people are being harmed 0:21:33.560 --> 0:21:34.760 by AI right. 0:21:34.600 --> 0:21:39.320 Now, and yet we are wasting our time talking about 0:21:39.359 --> 0:21:43.399 these fantasy scenarios because the people with all the money 0:21:43.760 --> 0:21:47.560 decided to get worried about it. In the meantime, there's 0:21:47.840 --> 0:21:52.200 the doomer stuff, the existential stuff is also another kind 0:21:52.200 --> 0:21:55.960 of AI hype, because if these systems are so powerful 0:21:56.000 --> 0:21:59.080 they might destroy the world, then these systems are really powerful. 0:22:00.200 --> 0:22:03.320 That hype is a key part of this, because a 0:22:03.480 --> 0:22:08.760 super powerful AI system that's exciting to investors and to employees. 0:22:10.119 --> 0:22:14.960 Emily thinks AI apocalypse beliefs are harmful and misleading, but 0:22:15.040 --> 0:22:17.639 she also thinks many of the people concerned with AI 0:22:17.720 --> 0:22:20.200 safety are coming from a sincere place. 0:22:20.760 --> 0:22:24.399 But it does seem to be a genuine belief of 0:22:24.440 --> 0:22:27.399 some of these folks. The people who get deep into 0:22:27.440 --> 0:22:31.960 this set of beliefs about super intelligent AI taking over 0:22:32.000 --> 0:22:35.800 the world see themselves as the heroes and their stories 0:22:35.840 --> 0:22:37.680 who are going to work to stop that. I think 0:22:37.720 --> 0:22:40.639 it's genuine. I think it's misplaced, but I think it's genuine. 0:22:41.200 --> 0:22:44.919 I think Emily is hitting on something real here, that 0:22:45.000 --> 0:22:48.320 this urge to be a hero becomes a motivating force 0:22:48.359 --> 0:22:51.560 in the AI industry. Here's chaw choo again. 0:22:52.000 --> 0:22:55.679 There is this kind of very boyish desire to be 0:22:55.720 --> 0:22:57.520 a hero, Like most people don't get a chance to 0:22:57.600 --> 0:23:00.679 do that in any meaningful sense. Not only just what 0:23:00.680 --> 0:23:02.240 if I could be a hero, but like, oh what 0:23:02.280 --> 0:23:04.920 if I could like smart my way to being a hero. 0:23:05.200 --> 0:23:07.560 I think it'll be difficult for some people to admit 0:23:07.560 --> 0:23:10.720 this because it's like kind of it feels immature or something. 0:23:10.760 --> 0:23:12.439 But I really do think that is part of the 0:23:12.640 --> 0:23:14.160 of the appeal of the pitch. 0:23:14.520 --> 0:23:18.760 Being a hero by using your brain. Chauchu is saying 0:23:18.760 --> 0:23:21.879 the quiet part out loud that many people in this 0:23:21.920 --> 0:23:25.359 field are motivated by this pride. They want to feel 0:23:25.359 --> 0:23:29.000 like they're important and that their work is cosmically significant. 0:23:29.320 --> 0:23:31.200 People want to feel like they made a difference. Like 0:23:31.200 --> 0:23:33.720 people want to feel like their lives matter, and that's 0:23:33.840 --> 0:23:35.720 that's a huge part of the hook is like what 0:23:35.760 --> 0:23:39.119 if this is like the most significant era of human 0:23:39.200 --> 0:23:41.320