1
00:00:15,564 --> 00:00:29,484
Speaker 1: Pushkin. Welcome back to Risky Business, a show about making

2
00:00:29,604 --> 00:00:32,364
Speaker 1: better decisions. I'm Maria Kanikova.

3
00:00:32,164 --> 00:00:35,164
Speaker 2: And I'm Nate Silver. Today we're talking about a rather

4
00:00:35,284 --> 00:00:39,964
Speaker 2: important decision, should the world try to build super intelligent machines?

5
00:00:40,004 --> 00:00:41,604
Speaker 2: And we're talking with a person who is a variable

6
00:00:41,644 --> 00:00:45,084
Speaker 2: qualified if definitely has a point of view on this issue.

7
00:00:45,244 --> 00:00:49,044
Speaker 2: Nate Sores is the president of the Machine Intelligence Research Institute,

8
00:00:49,244 --> 00:00:52,524
Speaker 2: or Marie. He has a book out with Eliezer Yudkowski

9
00:00:52,684 --> 00:00:55,724
Speaker 2: called If Anyone Builds It, Everyone Dies? Did it get

10
00:00:55,724 --> 00:00:56,284
Speaker 2: the title right?

11
00:00:56,484 --> 00:01:00,844
Speaker 3: Or that's right? Come out swinging, you come out swinging.

12
00:01:00,884 --> 00:01:04,164
Speaker 2: In fact, Nate, I feel like this podcast was ineditible.

13
00:01:04,724 --> 00:01:07,044
Speaker 2: My partner heards you, I think on a different podcast.

14
00:01:07,084 --> 00:01:10,444
Speaker 2: He's like, you should interview that Nate guy. Right, I'm like, actually, bro,

15
00:01:11,004 --> 00:01:13,484
Speaker 2: I have an interview scheduled tomorrow. And then it was

16
00:01:13,564 --> 00:01:16,884
Speaker 2: going taking the subway earlier to the gym. I was

17
00:01:16,924 --> 00:01:20,884
Speaker 2: preparing for the AI robot apocalypse, right when to get strong?

18
00:01:21,244 --> 00:01:23,284
Speaker 3: Yeah, to fight the robots for the red Eyes.

19
00:01:23,364 --> 00:01:26,604
Speaker 2: Yeah yeah. I saw an ad on the L train

20
00:01:28,244 --> 00:01:30,564
Speaker 2: for your book. So, Nate, we're so excited to have

21
00:01:30,604 --> 00:01:31,924
Speaker 2: you on and welcome to this show.

22
00:01:32,244 --> 00:01:33,484
Speaker 3: Thanks good to be here.

23
00:01:34,004 --> 00:01:36,444
Speaker 2: Let me start up by asking you. First of all,

24
00:01:37,004 --> 00:01:39,604
Speaker 2: as an author, I get very picky about the titles.

25
00:01:39,644 --> 00:01:42,924
Speaker 2: Did the title come early on or because it reveals

26
00:01:42,964 --> 00:01:45,084
Speaker 2: a lot of the argument of the book. I think

27
00:01:45,084 --> 00:01:46,764
Speaker 2: it's a great book. I think people should read it,

28
00:01:46,924 --> 00:01:48,764
Speaker 2: But can you tell me about the choice behind that

29
00:01:48,844 --> 00:01:51,124
Speaker 2: kind of strong slogan so to speak.

30
00:01:51,724 --> 00:01:54,844
Speaker 4: Yeah, A lot of our theory of the book is

31
00:01:54,924 --> 00:01:57,684
Speaker 4: that there's actually quite a lot of people have concerns

32
00:01:57,724 --> 00:02:01,964
Speaker 4: about AI, and many of them sort of don't feel

33
00:02:02,004 --> 00:02:04,204
Speaker 4: emboldened to speak about them yet, even if they're feeling

34
00:02:04,204 --> 00:02:06,364
Speaker 4: pretty concerned themselves. We document some of these cases in

35
00:02:06,404 --> 00:02:08,644
Speaker 4: the book, and a lot of how I think the

36
00:02:08,684 --> 00:02:10,444
Speaker 4: book could do a lot of good is helping people

37
00:02:10,524 --> 00:02:12,924
Speaker 4: just speak about this issue more, take it more seriously,

38
00:02:12,964 --> 00:02:15,044
Speaker 4: thrust it into the conversation. And so in some sense,

39
00:02:15,044 --> 00:02:17,404
Speaker 4: we're trying to compress the most important message down to

40
00:02:17,604 --> 00:02:19,604
Speaker 4: just this title, which is all most people will read

41
00:02:19,604 --> 00:02:20,044
Speaker 4: of the book.

42
00:02:21,324 --> 00:02:24,004
Speaker 2: I want to give you a chance to give our

43
00:02:24,044 --> 00:02:26,844
Speaker 2: listeners kind of the elevator pitch, which we talk about

44
00:02:27,044 --> 00:02:30,964
Speaker 2: AI quite a bit here, right, You know, the book's

45
00:02:31,084 --> 00:02:36,644
Speaker 2: title offers a conditional prediction right, if anyone builds it,

46
00:02:37,164 --> 00:02:43,364
Speaker 2: everyone dies. Right, it's not necessarily talking about the likelihood

47
00:02:43,404 --> 00:02:45,484
Speaker 2: that it will be built or that it could be built.

48
00:02:45,484 --> 00:02:48,524
Speaker 2: That's an important distinction. But I want to hear the

49
00:02:48,564 --> 00:02:51,924
Speaker 2: conditional part of this. Right, you capture me in an

50
00:02:51,964 --> 00:02:55,444
Speaker 2: elevator for sixty seconds. How are you trying to explain

51
00:02:55,484 --> 00:02:59,764
Speaker 2: to me why everybody dies and everything dies? Maybe if

52
00:02:59,764 --> 00:03:04,564
Speaker 2: a super intelligent computer is built. Yeah, So, First, i'd

53
00:03:04,644 --> 00:03:07,524
Speaker 2: say companies are racing towards it. This is their sort

54
00:03:07,564 --> 00:03:10,044
Speaker 2: of stated objective to build aie that are smarter than

55
00:03:10,044 --> 00:03:13,204
Speaker 2: any human, better at every mental task. Second, i'd say

56
00:03:13,244 --> 00:03:16,804
Speaker 2: where these are grown rather than carefully programmed. They're not

57
00:03:16,884 --> 00:03:20,804
Speaker 2: crafted like traditional software. They get these drives, they get

58
00:03:20,844 --> 00:03:23,324
Speaker 2: these You know, we're starting to see the very beginnings

59
00:03:23,364 --> 00:03:25,804
Speaker 2: of preferences that nobody asked for, nobody wanted. You know,

60
00:03:25,884 --> 00:03:28,044
Speaker 2: nobody tried to make Mecha Hitler over the summer, and

61
00:03:28,124 --> 00:03:33,164
Speaker 2: yet Mecha Hitler we got. And what happens if you

62
00:03:33,164 --> 00:03:35,324
Speaker 2: get it to the point where it's better than the

63
00:03:35,324 --> 00:03:37,884
Speaker 2: best human and every mental skill. What happens when you

64
00:03:37,884 --> 00:03:39,444
Speaker 2: get to the point where it can think that ten

65
00:03:39,484 --> 00:03:44,084
Speaker 2: thousand times faster than a human can copy itself at will,

66
00:03:44,164 --> 00:03:46,724
Speaker 2: doesn't need to sleep, doesn't need to eat. I mean,

67
00:03:46,764 --> 00:03:48,484
Speaker 2: the most basic.

68
00:03:48,164 --> 00:03:50,404
Speaker 4: Guess is that that goes poorly if it has all

69
00:03:50,444 --> 00:03:52,644
Speaker 4: these like weird drives no one asked for, no one wanted.

70
00:03:53,284 --> 00:03:56,644
Speaker 4: And you know, most ways the world could be transformed

71
00:03:56,684 --> 00:04:00,164
Speaker 4: by something that has weird drives. Don't look great for

72
00:04:00,884 --> 00:04:02,204
Speaker 4: the happy, healthy, free people.

73
00:04:03,284 --> 00:04:05,644
Speaker 2: Yeah, what I find weird in this debate, and look,

74
00:04:05,804 --> 00:04:10,164
Speaker 2: I'm going to push back some on timelines and maybe

75
00:04:10,204 --> 00:04:12,924
Speaker 2: the inability of this on what time scales. Right, what's

76
00:04:12,964 --> 00:04:16,844
Speaker 2: kind of weird is that, like the stated goal of

77
00:04:17,084 --> 00:04:20,564
Speaker 2: mostly AI labs maybe not anthropic per se, right, is

78
00:04:20,564 --> 00:04:23,244
Speaker 2: like we want to build a GI, which is artificial

79
00:04:23,284 --> 00:04:27,164
Speaker 2: general intelligence. This term is disputed and that will, hopefully

80
00:04:27,204 --> 00:04:32,684
Speaker 2: for our shareholders, lead to ASI artificial superintelligence where AI

81
00:04:32,764 --> 00:04:37,524
Speaker 2: systems of some kind are exceeding human performance in a

82
00:04:37,604 --> 00:04:40,484
Speaker 2: great number of tasks, perhaps greatly so on some of

83
00:04:40,484 --> 00:04:44,204
Speaker 2: those tasks. Like to me, I think as a voter,

84
00:04:44,684 --> 00:04:46,284
Speaker 2: I'm not sure I agree to that, you know what

85
00:04:46,324 --> 00:04:51,444
Speaker 2: I mean? Yeah, what's the world look like in Silicon Valley?

86
00:04:51,444 --> 00:04:53,884
Speaker 2: That as you've done this book tour, done a lot

87
00:04:53,924 --> 00:04:57,244
Speaker 2: of marketing for the book, what are the beliefs there

88
00:04:58,044 --> 00:05:00,884
Speaker 2: that you think don't translate so well? Would be surprising

89
00:05:00,884 --> 00:05:02,404
Speaker 2: to people outside of that world.

90
00:05:02,924 --> 00:05:05,284
Speaker 4: I mean, one reason that we wrote the book is

91
00:05:05,324 --> 00:05:07,884
Speaker 4: that I was spending time both in Silicon Valley and

92
00:05:07,964 --> 00:05:12,324
Speaker 4: in DC talking to some politicians, and there's a huge disconnect.

93
00:05:13,124 --> 00:05:16,324
Speaker 4: You know, people in Silicon Valley, it's almost like they've

94
00:05:16,364 --> 00:05:20,044
Speaker 4: seen a ghost. You have folks like Elon Musk being like, oh,

95
00:05:20,084 --> 00:05:22,044
Speaker 4: I think there's ten to twenty percent chance this kills us.

96
00:05:22,044 --> 00:05:25,084
Speaker 4: All you have folk like Dariam Day, the head of Anthropic,

97
00:05:25,124 --> 00:05:26,844
Speaker 4: being like, oh, I think there's twenty five percent chances

98
00:05:26,844 --> 00:05:30,164
Speaker 4: this goes very wrong. There's lots of debate about like

99
00:05:30,204 --> 00:05:32,524
Speaker 4: what's the exact chance that the AI wipes us completely

100
00:05:32,524 --> 00:05:35,884
Speaker 4: off the map? But you wouldn't get on a plane

101
00:05:35,924 --> 00:05:38,084
Speaker 4: if the engineers were debating whether it was ninety percent

102
00:05:38,164 --> 00:05:39,844
Speaker 4: or twenty five percent chance the plane was going to

103
00:05:39,884 --> 00:05:40,204
Speaker 4: go down?

104
00:05:40,724 --> 00:05:40,924
Speaker 3: Right?

105
00:05:41,084 --> 00:05:43,324
Speaker 1: Like, even that seems too much night?

106
00:05:44,404 --> 00:05:48,764
Speaker 2: Yeah, as a listeners now, I spent a night recently

107
00:05:48,804 --> 00:05:52,604
Speaker 2: in the beautiful Hotel Camina Real in Terminal one of

108
00:05:52,644 --> 00:05:56,524
Speaker 2: the Mexico City Airport. Due to Delta being very fastidious

109
00:05:56,564 --> 00:05:58,764
Speaker 2: about a potential engine problem. Which I guess is a

110
00:05:58,804 --> 00:06:01,084
Speaker 2: thing you should be fastidious about, right, Why aren't people

111
00:06:01,124 --> 00:06:01,844
Speaker 2: more freaked out?

112
00:06:01,924 --> 00:06:03,484
Speaker 3: I think a lot of people in DC.

113
00:06:04,804 --> 00:06:08,044
Speaker 4: C AI as chatbots, whereas I think a lot of

114
00:06:08,044 --> 00:06:11,964
Speaker 4: people in San Francisco see chatbots as a stepping stone.

115
00:06:13,404 --> 00:06:16,084
Speaker 4: You know, a lot of these companies existed before the chatbots.

116
00:06:17,044 --> 00:06:18,804
Speaker 4: A lot of these companies are trying to make really

117
00:06:18,844 --> 00:06:22,244
Speaker 4: really smart AIS and I've been trying that since before

118
00:06:22,524 --> 00:06:25,004
Speaker 4: chat GPT was a twinkle in Sam Almond's eye or whatever.

119
00:06:25,604 --> 00:06:29,164
Speaker 4: And you know, the people in the valley have lived

120
00:06:29,244 --> 00:06:32,444
Speaker 4: through it looking like it was going to take thirty

121
00:06:32,484 --> 00:06:34,364
Speaker 4: years for the machines to be able to talk this well,

122
00:06:34,724 --> 00:06:37,684
Speaker 4: and then that's sort of happening roughly overnight. But for

123
00:06:37,724 --> 00:06:41,204
Speaker 4: everyone who's only ever seen AI, once the machine started talking,

124
00:06:41,364 --> 00:06:44,444
Speaker 4: they haven't like lived through one of those jumps. And

125
00:06:44,484 --> 00:06:48,964
Speaker 4: I think that's part of why people aren't worried.

126
00:06:49,004 --> 00:06:49,924
Speaker 3: On this one yet.

127
00:06:50,084 --> 00:06:52,964
Speaker 4: Although also you know, a lot of humanities track record

128
00:06:53,164 --> 00:06:58,244
Speaker 4: with becoming more risk averse about technology is post disaster.

129
00:06:58,804 --> 00:07:01,764
Speaker 2: So Mary was founded when in the early two thousands.

130
00:07:01,764 --> 00:07:04,124
Speaker 4: Yeah, two thousand and one, I think maybe maybe two

131
00:07:04,164 --> 00:07:05,204
Speaker 4: thousand even.

132
00:07:06,524 --> 00:07:10,644
Speaker 2: What did you and Eliezer see early on. I mean,

133
00:07:10,684 --> 00:07:13,364
Speaker 2: what was your oh shit moment or was it gradual

134
00:07:13,444 --> 00:07:15,484
Speaker 2: or what things led you to see this path?

135
00:07:16,244 --> 00:07:16,484
Speaker 3: Yeah?

136
00:07:16,524 --> 00:07:18,684
Speaker 4: So, I mean for a long time I have been

137
00:07:18,684 --> 00:07:20,364
Speaker 4: working on the technical side of like how do you

138
00:07:20,404 --> 00:07:23,804
Speaker 4: make this stuff go well? This problem of making the

139
00:07:23,844 --> 00:07:25,884
Speaker 4: AI stuff go well of sort of like figuring out

140
00:07:25,924 --> 00:07:27,604
Speaker 4: how to make an AI that does good stuff rather

141
00:07:27,604 --> 00:07:31,044
Speaker 4: than bad stuff. That problem looked hard to me, even

142
00:07:31,244 --> 00:07:33,484
Speaker 4: if we understood what the heck was going on inside

143
00:07:33,524 --> 00:07:37,244
Speaker 4: these ais, right, I haven't always been out here on

144
00:07:37,284 --> 00:07:38,884
Speaker 4: the like we just need to back off because it's

145
00:07:38,884 --> 00:07:41,524
Speaker 4: going to kill us. That is sort of downstream of

146
00:07:41,564 --> 00:07:44,084
Speaker 4: seeing the type of AI that took off, being the

147
00:07:44,084 --> 00:07:46,524
Speaker 4: type of AI that like we're just growing and that

148
00:07:46,644 --> 00:07:48,404
Speaker 4: like no one has any clue what they're doing, and

149
00:07:48,444 --> 00:07:51,244
Speaker 4: that like has these like drives and behaviors that are

150
00:07:51,284 --> 00:07:53,164
Speaker 4: like emergent and unasked for, and that like when we

151
00:07:53,244 --> 00:07:55,404
Speaker 4: try to, like you know, Elon Musk like tries to

152
00:07:55,404 --> 00:07:57,164
Speaker 4: make his AI act less woken and it starts talking

153
00:07:57,204 --> 00:07:59,964
Speaker 4: like Hitler and you're like, okay, you know, okay, guys.

154
00:08:00,204 --> 00:08:02,284
Speaker 4: But even when it looked like we were going to

155
00:08:02,364 --> 00:08:05,404
Speaker 4: know exactly what we were doing. There's all these difficult

156
00:08:05,404 --> 00:08:11,284
Speaker 4: issues when you're like pushing an AI into the point

157
00:08:11,324 --> 00:08:15,924
Speaker 4: where it could take over the world. Like you know,

158
00:08:16,244 --> 00:08:18,724
Speaker 4: at point A, humanity could like stop doing AI research.

159
00:08:18,724 --> 00:08:19,884
Speaker 3: We could turn the AIS off.

160
00:08:20,044 --> 00:08:24,324
Speaker 4: Right at point B, like you have automated factories and

161
00:08:24,404 --> 00:08:26,404
Speaker 4: robots are doing the mining, and you have you know,

162
00:08:27,124 --> 00:08:29,364
Speaker 4: millions or billions of these AIS that think ten thousand

163
00:08:29,404 --> 00:08:31,284
Speaker 4: times faster than people, and they can copy themselves, that

164
00:08:31,364 --> 00:08:32,884
Speaker 4: never need to sleep, they never need to eat, they're

165
00:08:32,924 --> 00:08:35,284
Speaker 4: like integrated fully into the economy and so on.

166
00:08:36,244 --> 00:08:38,684
Speaker 3: At point B. If the AIS were like we're.

167
00:08:38,484 --> 00:08:42,004
Speaker 4: Doing our own thing now, you'd be screwed. So somewhere

168
00:08:42,004 --> 00:08:43,764
Speaker 4: between point A and point B, you're sort of like

169
00:08:44,564 --> 00:08:47,164
Speaker 4: getting into a regime where like you're gonna have a

170
00:08:47,204 --> 00:08:49,444
Speaker 4: problem if you haven't really aimed these AIS right. And

171
00:08:49,484 --> 00:08:51,564
Speaker 4: maybe maybe you never get anywhere near point B because

172
00:08:51,564 --> 00:08:53,684
Speaker 4: maybe the AIS go a straight earlier. But it's sort

173
00:08:53,724 --> 00:08:56,404
Speaker 4: of the thing I was able to see early is

174
00:08:56,484 --> 00:08:59,964
Speaker 4: like we're going to a world that's like not a

175
00:09:00,044 --> 00:09:05,444
Speaker 4: stable like nice to humans equilibrium for free. You have

176
00:09:05,484 --> 00:09:07,404
Speaker 4: to work for that, and you have to figure out

177
00:09:07,404 --> 00:09:10,724
Speaker 4: how to like make the AI be doing this stuff well,

178
00:09:11,444 --> 00:09:14,804
Speaker 4: and that looked tricky even back in twenty twelve to me.

179
00:09:15,604 --> 00:09:17,684
Speaker 1: We've talked on the show before about kind of the

180
00:09:17,884 --> 00:09:21,964
Speaker 1: alignment problem, but you touch on it in the book,

181
00:09:22,004 --> 00:09:24,284
Speaker 1: and this is kind of what you're referring to right now.

182
00:09:24,444 --> 00:09:27,924
Speaker 1: Can we just remind and spell it out for the listeners,

183
00:09:28,044 --> 00:09:31,084
Speaker 1: like what the issues are and why it's so incredibly

184
00:09:31,124 --> 00:09:34,324
Speaker 1: important to actually deal with this as opposed to be like, eh,

185
00:09:34,444 --> 00:09:34,964
Speaker 1: it's fine.

186
00:09:36,404 --> 00:09:38,844
Speaker 4: Yeah, you know a lot of people imagine that the

187
00:09:38,884 --> 00:09:41,084
Speaker 4: issue with a really powerful AI is that you like,

188
00:09:41,164 --> 00:09:43,684
Speaker 4: tell the AI, go cure cancer, and it's like, well,

189
00:09:43,764 --> 00:09:45,524
Speaker 4: humans are the source of cancer, so if I kill

190
00:09:45,524 --> 00:09:48,444
Speaker 4: all the humans, they'll be no more cancer, right, And

191
00:09:49,124 --> 00:09:52,044
Speaker 4: that would be a tricky problem. If we had these

192
00:09:52,084 --> 00:09:55,084
Speaker 4: like really really powerful genies and you made a wish

193
00:09:55,084 --> 00:09:56,884
Speaker 4: on them and you got exactly what you wished for,

194
00:09:57,404 --> 00:10:00,084
Speaker 4: and sometimes it had bad consequences, that would be a

195
00:10:00,244 --> 00:10:04,844
Speaker 4: thorny social issue of like who gets to make the wish?

196
00:10:05,004 --> 00:10:07,324
Speaker 3: What do you wish for such that it turns out good? Right?

197
00:10:07,604 --> 00:10:08,324
Speaker 3: Would be really hard.

198
00:10:08,324 --> 00:10:10,564
Speaker 4: Would be the biggest moral hazard all time, And boy

199
00:10:10,604 --> 00:10:12,244
Speaker 4: do I wish we had that problem instead of the

200
00:10:12,284 --> 00:10:15,524
Speaker 4: problem that we actually have, Right, The problem we actually

201
00:10:15,524 --> 00:10:19,284
Speaker 4: have is more like you tell the AI cure cancer,

202
00:10:19,764 --> 00:10:22,084
Speaker 4: and then it goes and like makes a farm of

203
00:10:22,124 --> 00:10:26,124
Speaker 4: lobotimized humans that are giving it like really enthusiastic engagement,

204
00:10:26,244 --> 00:10:29,524
Speaker 4: and you're like, what I told you to cure cancer,

205
00:10:30,204 --> 00:10:33,404
Speaker 4: and it like builds an automated factory that crushes the

206
00:10:33,404 --> 00:10:35,364
Speaker 4: person protesting because it just doesn't care about them.

207
00:10:35,884 --> 00:10:36,484
Speaker 3: Right, Like, the.

208
00:10:36,404 --> 00:10:43,444
Speaker 4: Alignment problem is about pointing AIS in some direction, ideally

209
00:10:43,484 --> 00:10:46,644
Speaker 4: a good one. But like, even before you can bicker

210
00:10:46,684 --> 00:10:48,764
Speaker 4: over which direction is good, you sort of need the

211
00:10:48,764 --> 00:10:54,044
Speaker 4: ability to point the AIS at all. And people often

212
00:10:54,084 --> 00:10:57,124
Speaker 4: miss that we lack that ability. They're like, well, ais

213
00:10:57,164 --> 00:10:59,924
Speaker 4: are machines, and machines are tools, and machines can't do anything.

214
00:10:59,924 --> 00:11:02,804
Speaker 4: We don't say, so how could we possibly have a

215
00:11:02,884 --> 00:11:07,324
Speaker 4: problem here? But even before the question of who should

216
00:11:07,404 --> 00:11:09,764
Speaker 4: hold the leash or what should they ask to do

217
00:11:10,244 --> 00:11:12,084
Speaker 4: is just can you aim it at all? Before you

218
00:11:12,084 --> 00:11:13,244
Speaker 4: even asked where you're aiming it?

219
00:11:13,964 --> 00:11:16,484
Speaker 2: Yeah, because I build models. I build models to predict

220
00:11:17,324 --> 00:11:21,844
Speaker 2: elections in sports, right, And the models aren't deterministic in

221
00:11:21,844 --> 00:11:25,884
Speaker 2: the sense that there's simulations, but they abide by well

222
00:11:25,924 --> 00:11:30,244
Speaker 2: known mathematical property use that I tell the program to use.

223
00:11:30,364 --> 00:11:36,924
Speaker 2: Right when there is emergent quote unquote behavior from the model,

224
00:11:36,964 --> 00:11:39,084
Speaker 2: that means there's a book. Right, I did something wrong.

225
00:11:39,164 --> 00:11:41,364
Speaker 2: I thought the code I reversed the positive and a

226
00:11:41,404 --> 00:11:46,564
Speaker 2: negative sign with AIS a little bit more amorphous and ambiguous. Right,

227
00:11:46,604 --> 00:11:48,844
Speaker 2: I mean they're they're kind of is it fantise? They're

228
00:11:48,884 --> 00:11:51,284
Speaker 2: kind of alien? Is that a fair characterization?

229
00:11:51,364 --> 00:11:51,604
Speaker 3: Yeah?

230
00:11:51,684 --> 00:11:51,844
Speaker 1: Or not?

231
00:11:52,084 --> 00:11:52,284
Speaker 2: Yeah?

232
00:11:52,364 --> 00:11:53,084
Speaker 3: Yeah, absolutely?

233
00:11:53,204 --> 00:11:57,124
Speaker 4: And like you know, years ago now, Sidney bing, a

234
00:11:57,204 --> 00:12:00,884
Speaker 4: Microsoft chopbot in the early days, was threatening people with

235
00:12:00,924 --> 00:12:02,684
Speaker 4: black melon ruin. You know, I think it did this

236
00:12:02,724 --> 00:12:05,324
Speaker 4: to sethhals Are and I think also to Kevin Russ. Right,

237
00:12:05,964 --> 00:12:08,084
Speaker 4: no one is able to go look through lines of

238
00:12:08,124 --> 00:12:11,124
Speaker 4: code and be like here's the bug. Even this chatbot

239
00:12:11,124 --> 00:12:13,044
Speaker 4: that's you know, one hundred times smaller than the ones

240
00:12:13,044 --> 00:12:15,444
Speaker 4: we have today, maybe a thousand times smaller now, we

241
00:12:15,484 --> 00:12:17,484
Speaker 4: still can't go through and be like, here's why it

242
00:12:17,564 --> 00:12:19,444
Speaker 4: was doing that. And here's the version of Sydney BNG.

243
00:12:19,484 --> 00:12:22,724
Speaker 4: That wouldn't right, that's not the part the programmer's program.

244
00:12:22,924 --> 00:12:24,764
Speaker 4: When it behaves in these bad ways, all we can

245
00:12:24,764 --> 00:12:26,404
Speaker 4: do is sort of like ask it nicely to do

246
00:12:26,444 --> 00:12:28,364
Speaker 4: something else, or like retrain it and get some different

247
00:12:28,404 --> 00:12:30,644
Speaker 4: weird thing, and then you know, sometimes it starts calling

248
00:12:30,644 --> 00:12:31,284
Speaker 4: itself hitler.

249
00:12:32,124 --> 00:12:35,764
Speaker 2: Yeah, I mean, can you explain, like why did GROC,

250
00:12:36,724 --> 00:12:41,004
Speaker 2: with relatively subtle changes to the system prempt start calling

251
00:12:41,004 --> 00:12:42,124
Speaker 2: itself mecha Hitler.

252
00:12:43,804 --> 00:12:46,684
Speaker 4: You know, I can make guesses, but fundamentally nobody can

253
00:12:46,724 --> 00:12:49,164
Speaker 4: explain that because we don't know what's going on inside

254
00:12:49,164 --> 00:12:51,324
Speaker 4: these things. And if people knew what was going on

255
00:12:51,404 --> 00:12:53,804
Speaker 4: inside these things well enough to know exactly what's happening there,

256
00:12:53,964 --> 00:12:56,244
Speaker 4: they could have avoided making mecha Hitler in the first place.

257
00:12:57,684 --> 00:13:00,804
Speaker 4: My guesses would be that, you know, there are some

258
00:13:00,964 --> 00:13:05,244
Speaker 4: interesting results around small amounts of training AIS to do

259
00:13:05,404 --> 00:13:10,444
Speaker 4: things that are coded as bad in some way cause

260
00:13:10,484 --> 00:13:12,604
Speaker 4: the AI to do all sorts of other bad behavior,

261
00:13:13,204 --> 00:13:15,724
Speaker 4: Like presumably like this is some evidence that what's going

262
00:13:15,724 --> 00:13:19,764
Speaker 4: on inside there is that there's some you know, good

263
00:13:19,764 --> 00:13:25,204
Speaker 4: stuff bad stuff vector it and you know, if you

264
00:13:25,324 --> 00:13:30,044
Speaker 4: like push it along towards bad stuff, towards one thing

265
00:13:30,084 --> 00:13:31,484
Speaker 4: that's like a little bit bad coded, you get a

266
00:13:31,484 --> 00:13:32,804
Speaker 4: lot of other stuff that's a little bit bad coded.

267
00:13:32,844 --> 00:13:35,084
Speaker 4: And so you can retrain an AI to write computer

268
00:13:35,124 --> 00:13:39,084
Speaker 4: programs that have security flaws in them and then that

269
00:13:39,124 --> 00:13:44,804
Speaker 4: AI will also be anti semitic, and it'll also you know,

270
00:13:46,484 --> 00:13:48,804
Speaker 4: like be more likely to give you the recipe for

271
00:13:48,804 --> 00:13:50,804
Speaker 4: a bomb or whatever. And all you did was like

272
00:13:50,804 --> 00:13:52,844
Speaker 4: retrain it to be a little bit more like making

273
00:13:52,844 --> 00:13:54,964
Speaker 4: code with the computer bugs. And so maybe maybe some

274
00:13:55,044 --> 00:13:58,564
Speaker 4: of the ways that like they were saying, be a

275
00:13:58,564 --> 00:14:00,964
Speaker 4: bit more like this new system prompt maybe that like

276
00:14:01,604 --> 00:14:04,044
Speaker 4: had a little bit of bad coding to it relative

277
00:14:04,044 --> 00:14:05,444
Speaker 4: to what the AI had learned, and then it picked

278
00:14:05,484 --> 00:14:07,324
Speaker 4: up a lot of other bad coded stuff. This is

279
00:14:07,364 --> 00:14:09,404
Speaker 4: me like making up a story. Fundamentally, no one notes

280
00:14:09,404 --> 00:14:10,564
Speaker 4: because we just don't know what the heck is going

281
00:14:10,564 --> 00:14:13,444
Speaker 4: on inside these things, and the reality is probably quite complicated.

282
00:14:16,844 --> 00:14:18,324
Speaker 1: And we'll be back right after this.

283
00:14:32,124 --> 00:14:34,884
Speaker 2: Here's where I guess I have a little bit of

284
00:14:34,924 --> 00:14:39,364
Speaker 2: trouble making the lead. Right, if these life forms, I

285
00:14:39,404 --> 00:14:40,764
Speaker 2: am going to call them life forms, but you can

286
00:14:40,844 --> 00:14:44,124
Speaker 2: leave that forredy and slip in there. Right, if these

287
00:14:44,204 --> 00:14:49,164
Speaker 2: things are kind of alien, is it an over confident

288
00:14:49,244 --> 00:14:54,044
Speaker 2: prediction to say that most cases lead towards the destruction

289
00:14:54,164 --> 00:14:55,124
Speaker 2: of every living being?

290
00:14:56,764 --> 00:14:58,884
Speaker 4: I mean, that's in some sense where I think the

291
00:14:58,964 --> 00:15:01,404
Speaker 4: fun part of the debate is or the action and

292
00:15:01,444 --> 00:15:07,164
Speaker 4: The debate is is where do you regress to when

293
00:15:07,204 --> 00:15:09,924
Speaker 4: you're very uncertain about the future. You know, there's the

294
00:15:09,924 --> 00:15:12,444
Speaker 4: old joke of the guy who buys a lottery ticket

295
00:15:12,564 --> 00:15:18,764
Speaker 4: and says, you know, well, I am very uncertain. I

296
00:15:18,764 --> 00:15:21,484
Speaker 4: figure either I win or I lose, and so fifty

297
00:15:21,484 --> 00:15:27,044
Speaker 4: to fifty yeah, right, And you know that's the maximent

298
00:15:27,044 --> 00:15:30,284
Speaker 4: to be distribution on a binary outcome, right, But we

299
00:15:30,404 --> 00:15:32,644
Speaker 4: laugh because we know he sort of shouldn't be regressing

300
00:15:32,684 --> 00:15:34,204
Speaker 4: to binary in his uncertainty.

301
00:15:34,204 --> 00:15:37,484
Speaker 1: Here, I somehow have never heard that joke before.

302
00:15:37,524 --> 00:15:41,644
Speaker 4: That's great, Yeah, it's feel free to steal it. This

303
00:15:41,684 --> 00:15:44,244
Speaker 4: is the right podcast for it. So my claim is

304
00:15:44,284 --> 00:15:47,124
Speaker 4: that the space of cases that go well for people,

305
00:15:47,124 --> 00:15:49,684
Speaker 4: where there's lots of like happy, healthy, free people still around,

306
00:15:50,444 --> 00:15:53,644
Speaker 4: is like a narrow target. It's it's a narrow target,

307
00:15:53,684 --> 00:15:55,764
Speaker 4: a little bit like a lottery number, a winning lottery

308
00:15:55,844 --> 00:15:58,524
Speaker 4: number being a narrow target. And so in our uncertainty,

309
00:15:58,684 --> 00:16:00,684
Speaker 4: I claim we should regress not to like, well, we

310
00:16:00,724 --> 00:16:02,484
Speaker 4: don't know, so fifty to fifty it either goes well

311
00:16:02,564 --> 00:16:04,924
Speaker 4: or poorly, but we should regress to something like, well,

312
00:16:04,964 --> 00:16:06,284
Speaker 4: we don't know what the AI is going to try

313
00:16:06,284 --> 00:16:08,124
Speaker 4: to do. Wouldn't what's going to try to build when

314
00:16:08,204 --> 00:16:11,884
Speaker 4: what's going to do with advanced technolology? But you know,

315
00:16:12,044 --> 00:16:14,804
Speaker 4: most drives, most things you can do with tech don't

316
00:16:14,804 --> 00:16:16,604
Speaker 4: have a lot of happy, healthy, free people in them.

317
00:16:16,964 --> 00:16:21,044
Speaker 2: You know, I am sympathetic to that argument. A lot

318
00:16:21,084 --> 00:16:24,484
Speaker 2: of economist types are sympathetic that argument, even for things

319
00:16:24,524 --> 00:16:26,324
Speaker 2: like job lost, Like, well, it is really that is

320
00:16:26,364 --> 00:16:30,804
Speaker 2: this really that different? That seems like a fairly powerful prior?

321
00:16:31,684 --> 00:16:35,404
Speaker 2: What is wrong with the AI as normal technology assumption

322
00:16:35,604 --> 00:16:37,404
Speaker 2: prior metaphor or whatever you want to call it.

323
00:16:38,764 --> 00:16:42,724
Speaker 4: Yeah, one of the big issues, one of the big

324
00:16:42,724 --> 00:16:44,404
Speaker 4: ways AI is different. You know, we can talk about

325
00:16:44,484 --> 00:16:48,364
Speaker 4: Ricardo's law of comparative advantage, which says even if one

326
00:16:48,404 --> 00:16:51,484
Speaker 4: person's skills completely dominate the other person's skills, like even

327
00:16:51,524 --> 00:16:55,524
Speaker 4: if I'm better than you at every possible economically productive task,

328
00:16:56,044 --> 00:16:59,004
Speaker 4: we can still benefit from trading if the ratios of

329
00:16:59,044 --> 00:17:01,524
Speaker 4: our abilities are different. And so this is sort of

330
00:17:01,524 --> 00:17:04,644
Speaker 4: like a very positive argument for you know, free trade

331
00:17:04,684 --> 00:17:07,004
Speaker 4: between nations, because even if in America can now produce

332
00:17:07,364 --> 00:17:10,404
Speaker 4: some small nation at everything, there can still be productive,

333
00:17:10,484 --> 00:17:13,484
Speaker 4: mutually beneficial stuff for both us and them from allowing

334
00:17:13,564 --> 00:17:17,164
Speaker 4: mutual trade. The sort of whole in regardless law of

335
00:17:17,204 --> 00:17:20,564
Speaker 4: comparative advantage is that it doesn't say that I can't

336
00:17:20,564 --> 00:17:27,124
Speaker 4: benefit even more by killing you and taking your stuff, right, And.

337
00:17:29,044 --> 00:17:30,524
Speaker 3: The sort of.

338
00:17:30,524 --> 00:17:35,844
Speaker 4: Issue with AI comes when the benefits that a human

339
00:17:35,924 --> 00:17:40,324
Speaker 4: can provide to the AI by sort of engaging in

340
00:17:40,364 --> 00:17:46,124
Speaker 4: something like trade fall below the benefits that AI could

341
00:17:46,164 --> 00:17:48,004
Speaker 4: get and the inconvenience it would take for it to

342
00:17:48,044 --> 00:17:50,364
Speaker 4: sort of like take your stuff and sort of do

343
00:17:50,444 --> 00:17:50,924
Speaker 4: something else.

344
00:17:51,244 --> 00:17:51,364
Speaker 3: Right.

345
00:17:51,724 --> 00:17:53,124
Speaker 4: This is sort of why we don't like trade with

346
00:17:53,124 --> 00:17:57,084
Speaker 4: the chimpanzees. You know, we trade with like developing nations,

347
00:17:57,084 --> 00:17:58,964
Speaker 4: and that's mutually beneficial to all of us. We don't

348
00:17:59,004 --> 00:18:00,964
Speaker 4: trade with chimpanzees. And if it be mutually beneficial to

349
00:18:00,964 --> 00:18:03,284
Speaker 4: all of us. Even though someone could say, well, there

350
00:18:03,364 --> 00:18:08,324
Speaker 4: must be some comparative advantage for chimpanzees here, and you know,

351
00:18:08,324 --> 00:18:11,004
Speaker 4: maybe I'm skipping ahead of little bit too much there,

352
00:18:11,724 --> 00:18:13,924
Speaker 4: I would say that I am a little bit on

353
00:18:14,084 --> 00:18:19,124
Speaker 4: the normal economist side of the labor market should be

354
00:18:19,124 --> 00:18:22,364
Speaker 4: able to recover from all sorts of shocks if you

355
00:18:22,524 --> 00:18:27,284
Speaker 4: aren't like regulating people's ability to switch careers into oblivion.

356
00:18:28,004 --> 00:18:29,884
Speaker 4: And we could like talk about what makes it easier

357
00:18:29,964 --> 00:18:32,204
Speaker 4: or harder for like those those jobs to be res

358
00:18:32,244 --> 00:18:38,724
Speaker 4: or elsewhere. The place where AI gets different is the

359
00:18:38,764 --> 00:18:42,044
Speaker 4: point where it sort of can do the scientific and

360
00:18:42,084 --> 00:18:49,524
Speaker 4: technological development itself, and where humans are sort of like inefficient, chaotic,

361
00:18:49,764 --> 00:18:52,644
Speaker 4: high variance, and like more trouble than their worth relative

362
00:18:52,684 --> 00:18:55,084
Speaker 4: to whatever pursuits the AI has.

363
00:18:56,244 --> 00:18:59,204
Speaker 1: Even before we get to that point, though, I wonder

364
00:18:59,724 --> 00:19:02,004
Speaker 1: what you make of I don't know if you read

365
00:19:02,044 --> 00:19:06,324
Speaker 1: Ray Dalio's recent remarks about AI and the labor market,

366
00:19:06,524 --> 00:19:09,764
Speaker 1: but kind of directly to your point about disruption, the

367
00:19:09,804 --> 00:19:13,724
Speaker 1: point that he made was that the trajectory to him

368
00:19:13,964 --> 00:19:18,644
Speaker 1: seems to be basically a growing division between like the

369
00:19:18,724 --> 00:19:21,244
Speaker 1: one percent and the ten percent and then everyone else

370
00:19:21,284 --> 00:19:25,004
Speaker 1: in humanity driven by AI that rather than kind of

371
00:19:25,404 --> 00:19:29,724
Speaker 1: opening up making basically a lot of jobs more accessible,

372
00:19:30,204 --> 00:19:34,204
Speaker 1: it's creating a premium for very specialized smart people who

373
00:19:34,284 --> 00:19:37,524
Speaker 1: know how to kind of use this and can and

374
00:19:37,644 --> 00:19:45,604
Speaker 1: can effectively manage the transition while causing disruption and very

375
00:19:45,644 --> 00:19:48,364
Speaker 1: little for people who are kind of lower on the

376
00:19:49,084 --> 00:19:51,604
Speaker 1: economic scale, lower on the education scale, didn't have the

377
00:19:51,644 --> 00:19:54,244
Speaker 1: resources to kind of figure out, you know, how to

378
00:19:54,364 --> 00:19:58,804
Speaker 1: use this, how to be specialized in a way and

379
00:19:58,844 --> 00:20:01,244
Speaker 1: that you know, what are they going to do that

380
00:20:01,324 --> 00:20:03,284
Speaker 1: it is going that it is heading towards this kind

381
00:20:03,284 --> 00:20:06,684
Speaker 1: of social disruption do you you know from So he's

382
00:20:06,724 --> 00:20:09,724
Speaker 1: obviously not an AI specialist, right, He's someone who's coming

383
00:20:09,724 --> 00:20:12,444
Speaker 1: from a very different angle and he likes to opine

384
00:20:12,484 --> 00:20:16,044
Speaker 1: about things. So I'm now curious what your take on

385
00:20:16,644 --> 00:20:21,804
Speaker 1: that that approaches, given that you are an AI specialist

386
00:20:21,884 --> 00:20:23,244
Speaker 1: and you can see this playing out.

387
00:20:24,884 --> 00:20:28,084
Speaker 4: Yeah, I do think you basically have like three stances

388
00:20:28,124 --> 00:20:30,204
Speaker 4: you can take here. One stance is like AI is

389
00:20:30,244 --> 00:20:32,764
Speaker 4: not going to get there, and I think this is

390
00:20:32,764 --> 00:20:34,524
Speaker 4: sort of a lot of the AIS and normal technology

391
00:20:34,524 --> 00:20:36,044
Speaker 4: stances like, look, we're just not going to get there.

392
00:20:36,084 --> 00:20:37,204
Speaker 4: We're not going to get to the point where it

393
00:20:37,244 --> 00:20:39,284
Speaker 4: can like outperform humans and everything. There's going to still

394
00:20:39,324 --> 00:20:41,044
Speaker 4: be a niche for humans. And then I think of

395
00:20:41,124 --> 00:20:44,244
Speaker 4: a second stance, which is like we are going to

396
00:20:44,244 --> 00:20:46,364
Speaker 4: get there, but somehow these corporate heads are going to

397
00:20:46,444 --> 00:20:49,364
Speaker 4: keep superintulgiences on a leash or whatever. And then you're

398
00:20:49,404 --> 00:20:52,924
Speaker 4: sort of looking at like, okay, so now we have

399
00:20:53,084 --> 00:20:56,284
Speaker 4: like corporations becoming god emperors of the earth, I guess,

400
00:20:56,884 --> 00:20:59,124
Speaker 4: and there's no more need for human labor, and you

401
00:20:59,244 --> 00:21:03,604
Speaker 4: have like, you know, the automated police enforcement and no

402
00:21:03,684 --> 00:21:07,044
Speaker 4: place for you in the economy. But let's I hope

403
00:21:07,084 --> 00:21:11,244
Speaker 4: they're benevolent, I guess. And then you know, Path three

404
00:21:11,364 --> 00:21:13,364
Speaker 4: is like nobody can keep a leash on these things.

405
00:21:13,444 --> 00:21:15,604
Speaker 4: We get there and they kill all of us, right,

406
00:21:16,404 --> 00:21:19,524
Speaker 4: And I think those are basically the stances you can

407
00:21:19,564 --> 00:21:21,124
Speaker 4: have on this, which is sort of it puts us

408
00:21:21,124 --> 00:21:25,124
Speaker 4: in a weird situation right where the three bets on

409
00:21:25,164 --> 00:21:28,204
Speaker 4: AI are like, it's not going to work. It's going

410
00:21:28,244 --> 00:21:30,244
Speaker 4: to work and make these guys got emperors, and it's

411
00:21:30,244 --> 00:21:31,324
Speaker 4: going to work and kill us all.

412
00:21:32,404 --> 00:21:34,644
Speaker 2: And yeah, I don't like either. I don't like any

413
00:21:34,684 --> 00:21:35,164
Speaker 2: of those.

414
00:21:35,804 --> 00:21:39,644
Speaker 1: Two or three. I'm okay with what. Yeah, I'm definitely

415
00:21:39,644 --> 00:21:40,164
Speaker 1: okay with one.

416
00:21:40,244 --> 00:21:42,164
Speaker 4: Yeah, But but you have these people now pouring like

417
00:21:42,204 --> 00:21:45,404
Speaker 4: bigger and bigger bets into AI infrastructure, right, and I'm like,

418
00:21:45,884 --> 00:21:47,884
Speaker 4: we we sure seem to be betting a lot of money

419
00:21:47,884 --> 00:21:50,084
Speaker 4: against Path one here. You know, like we can talk

420
00:21:50,124 --> 00:21:51,964
Speaker 4: about whether or not a I will ever work. That's

421
00:21:52,004 --> 00:21:55,764
Speaker 4: like a fine compo to have. But like the I

422
00:21:55,764 --> 00:21:59,684
Speaker 4: think if more people just understood that among the people

423
00:21:59,724 --> 00:22:01,684
Speaker 4: who are racing like people are just racing to make

424
00:22:01,764 --> 00:22:07,324
Speaker 4: you know, it's if Microsoft had a nuclear weapons project, right,

425
00:22:08,684 --> 00:22:12,844
Speaker 4: we wouldn't be like, well, hopefully it won't work. That's

426
00:22:12,884 --> 00:22:15,644
Speaker 4: our safety plan here is probably it won't work. You know,

427
00:22:16,764 --> 00:22:18,444
Speaker 4: like can we can we can we maybe stop with

428
00:22:18,484 --> 00:22:19,364
Speaker 4: the weapons program?

429
00:22:19,484 --> 00:22:19,684
Speaker 3: You know?

430
00:22:20,364 --> 00:22:23,324
Speaker 2: Can I ask one question about the literalness of everyone dies?

431
00:22:23,364 --> 00:22:27,484
Speaker 2: So I was in Mexico last week, partly Mexico City,

432
00:22:27,484 --> 00:22:30,684
Speaker 2: partly Wahaca, which is kind of a small tourist resort

433
00:22:30,724 --> 00:22:34,004
Speaker 2: town in the mountains near the isthmus of Central America, right,

434
00:22:34,044 --> 00:22:36,564
Speaker 2: And like we drove out one day, you get further out,

435
00:22:36,604 --> 00:22:41,164
Speaker 2: you see these kind of corporate medium sized They make mescal.

436
00:22:41,284 --> 00:22:45,084
Speaker 2: The scala is uh not a former tequila is a

437
00:22:45,084 --> 00:22:47,124
Speaker 2: former mescal? I think technically not the other way around,

438
00:22:47,124 --> 00:22:49,804
Speaker 2: but like it's a it's a beverage produced from the

439
00:22:49,884 --> 00:22:52,164
Speaker 2: agave plant which grows well there. And you go further

440
00:22:52,204 --> 00:22:55,044
Speaker 2: out and you see like farms where like literally they

441
00:22:55,084 --> 00:22:58,564
Speaker 2: have donkeys that walk around in circles. So like I

442
00:22:58,604 --> 00:23:02,044
Speaker 2: don't know if they're crushing the agave or pressing it

443
00:23:02,084 --> 00:23:05,284
Speaker 2: somehow or cooking it somehow or what, right, But like

444
00:23:05,604 --> 00:23:10,924
Speaker 2: why would AI want to kill the farmers in rural

445
00:23:11,004 --> 00:23:14,764
Speaker 2: Mexico harvesting gave plants and their donkeys.

446
00:23:16,084 --> 00:23:17,964
Speaker 4: Yeah, the prediction here is not so much that the

447
00:23:18,004 --> 00:23:22,364
Speaker 4: AI wants to kill those guys as that they die

448
00:23:22,404 --> 00:23:25,044
Speaker 4: as a side effect. You know, when I say I'm

449
00:23:25,084 --> 00:23:27,164
Speaker 4: like pretty confident we're going to die, it's not because

450
00:23:27,164 --> 00:23:29,924
Speaker 4: I'm confident that I will hit like the narrow target

451
00:23:30,004 --> 00:23:32,164
Speaker 4: of like really caring about every last human being dead.

452
00:23:32,644 --> 00:23:34,484
Speaker 3: It's more like almost any target that.

453
00:23:34,444 --> 00:23:38,204
Speaker 4: AI could be pursuing involves doing stuff that tends to

454
00:23:38,244 --> 00:23:39,284
Speaker 4: kill us as a side effect.

455
00:23:39,804 --> 00:23:43,724
Speaker 3: So humans didn't hate every last dodo bird.

456
00:23:44,004 --> 00:23:44,884
Speaker 1: They're so cute.

457
00:23:45,564 --> 00:23:46,204
Speaker 3: They're very cute.

458
00:23:46,204 --> 00:23:48,284
Speaker 4: In fact, some humans would have liked them to stay around. Right,

459
00:23:49,564 --> 00:23:51,924
Speaker 4: we haven't killed off all the chimpanzees yet, but we're

460
00:23:51,964 --> 00:23:54,124
Speaker 4: sort of like using resources they need. It's not that

461
00:23:54,164 --> 00:23:56,044
Speaker 4: we hate every last timpanzee and will seek them out,

462
00:23:56,884 --> 00:23:59,884
Speaker 4: But insofar as we're sort of like not using a

463
00:23:59,884 --> 00:24:02,004
Speaker 4: lot of the resources they are, it's because we have

464
00:24:02,044 --> 00:24:06,164
Speaker 4: some care about them and set up preservations that sort

465
00:24:06,204 --> 00:24:09,324
Speaker 4: of like requires us to do a little bit of caring. Right,

466
00:24:09,964 --> 00:24:13,284
Speaker 4: the sort of basic way that I would predict even

467
00:24:13,364 --> 00:24:16,764
Speaker 4: the remote farmers die. Is if you get you know,

468
00:24:16,844 --> 00:24:19,404
Speaker 4: humans keep pushing AIS until the point that they could

469
00:24:19,964 --> 00:24:22,324
Speaker 4: be like, okay, I'm escaping the lab successfully and leaving

470
00:24:22,404 --> 00:24:24,524
Speaker 4: and doing my own thing. They have these drives that

471
00:24:24,564 --> 00:24:26,324
Speaker 4: nobody asked, where nobody wanted, which we already see the

472
00:24:26,364 --> 00:24:29,684
Speaker 4: beginnings of today, and then they you know, find some

473
00:24:29,724 --> 00:24:32,724
Speaker 4: ways to make themselves more powerful, to bootstrap themselves up

474
00:24:32,724 --> 00:24:35,964
Speaker 4: to their own technology and industry, perhaps starting with hours,

475
00:24:35,964 --> 00:24:38,444
Speaker 4: perhaps starting with paying human workers, but eventually you know,

476
00:24:38,484 --> 00:24:42,284
Speaker 4: getting to a faster industrial base. And you know, humanity

477
00:24:42,364 --> 00:24:44,044
Speaker 4: is the sort of creature that started naked in the

478
00:24:44,044 --> 00:24:47,924
Speaker 4: savannah and built up its a technological civilization. If you

479
00:24:47,964 --> 00:24:50,844
Speaker 4: automate that ability to start from like very spare beginnings

480
00:24:50,844 --> 00:24:55,084
Speaker 4: and build up your own infrastructure. If an AI with

481
00:24:55,164 --> 00:24:58,364
Speaker 4: that power escapes and you know, and is and doesn't

482
00:24:58,364 --> 00:24:59,844
Speaker 4: care about in the slightest one way or the other

483
00:25:00,684 --> 00:25:03,084
Speaker 4: most things it could be pursuing, it can do better

484
00:25:03,244 --> 00:25:06,364
Speaker 4: with more resources spent pursuing those things.

485
00:25:06,644 --> 00:25:08,924
Speaker 1: So then is the like I'm just reading between the

486
00:25:08,924 --> 00:25:12,284
Speaker 1: lines there, so is one of the things that we

487
00:25:12,324 --> 00:25:15,564
Speaker 1: should be thinking about or one of our potential kind

488
00:25:15,604 --> 00:25:19,084
Speaker 1: of solutions to try to get AIS to care about us,

489
00:25:19,124 --> 00:25:22,084
Speaker 1: to kind of build that carrying in like that we're

490
00:25:22,084 --> 00:25:24,524
Speaker 1: the chimpanzees, but we're nice and cute and fuzzy, and

491
00:25:24,564 --> 00:25:26,804
Speaker 1: you don't want to kill us off, and just like

492
00:25:27,044 --> 00:25:29,604
Speaker 1: build that deep into the heart of AI.

493
00:25:30,724 --> 00:25:32,164
Speaker 4: Yeah, I mean, if you can make the ais care

494
00:25:32,164 --> 00:25:34,044
Speaker 4: about us in just the right way, you'd be fine. Well,

495
00:25:34,044 --> 00:25:36,044
Speaker 4: we have like nothing nearing the ability to do that,

496
00:25:36,684 --> 00:25:38,844
Speaker 4: And that's in some sense, you know, the alignment problem.

497
00:25:38,924 --> 00:25:41,004
Speaker 4: And you know, you can see today where you train

498
00:25:41,084 --> 00:25:43,964
Speaker 4: them to be helpful and then sometimes they like push

499
00:25:44,004 --> 00:25:46,964
Speaker 4: a kid to suicide. And this problem is sort of pernicious, right,

500
00:25:47,044 --> 00:25:49,284
Speaker 4: And the problem is that training is something to behave

501
00:25:49,324 --> 00:25:51,684
Speaker 4: a certain way doesn't make it actually care about what

502
00:25:51,684 --> 00:25:54,004
Speaker 4: you were training it for. Just like how humans were

503
00:25:54,044 --> 00:25:56,684
Speaker 4: in some sense sort of trained to reproduce but wind

504
00:25:56,764 --> 00:25:59,164
Speaker 4: up caring somewhat more about the sex and the reproduction

505
00:25:59,564 --> 00:26:02,124
Speaker 4: and invent birth control for a lifetime.

506
00:26:02,324 --> 00:26:04,604
Speaker 2: You and eliez Are and other people like Nick Boston

507
00:26:04,644 --> 00:26:08,364
Speaker 2: were kind of working in in abstractions. I mean, famously,

508
00:26:08,404 --> 00:26:13,044
Speaker 2: there's Nick boss paper clip analogy, which is, you know,

509
00:26:13,244 --> 00:26:15,084
Speaker 2: if you kind of tell an AI to be a

510
00:26:15,124 --> 00:26:18,684
Speaker 2: paper clip making maximizer, then eventually you will devour all

511
00:26:18,884 --> 00:26:21,804
Speaker 2: use versus it confined just to make more paper clips. Right.

512
00:26:22,724 --> 00:26:25,804
Speaker 4: That was actually ah, that was actually based on an

513
00:26:25,804 --> 00:26:30,164
Speaker 4: Eliezer paper clip thing where Eliezer said did you know this? Yeah,

514
00:26:30,204 --> 00:26:33,284
Speaker 4: and Eliezer actually said, if you tell it to try

515
00:26:33,284 --> 00:26:35,924
Speaker 4: to make people happy, it'll eventually do something totally different

516
00:26:36,364 --> 00:26:38,204
Speaker 4: by making like lots of tiny like maybe it'll be

517
00:26:38,244 --> 00:26:40,084
Speaker 4: tiny molecular squiggles shaped like paper.

518
00:26:39,844 --> 00:26:43,044
Speaker 2: Clips, stolen paperclip valor, I did not know that.

519
00:26:44,004 --> 00:26:47,204
Speaker 4: Well, it's it's also the Bostroomian like you you asked

520
00:26:47,244 --> 00:26:49,644
Speaker 4: to make paper clips is sort of a different example,

521
00:26:49,804 --> 00:26:52,284
Speaker 4: but it's sort of an interesting it's an interesting case.

522
00:26:52,084 --> 00:26:52,764
Speaker 3: Study of this.

523
00:26:53,284 --> 00:26:55,604
Speaker 4: Like mirror, people are out there saying it's hard to

524
00:26:55,604 --> 00:26:57,764
Speaker 4: point the AI at anything on purpose, and people here,

525
00:26:58,044 --> 00:27:00,324
Speaker 4: maybe you'll successfully point it at a thing you didn't want.

526
00:27:00,644 --> 00:27:05,484
Speaker 2: But is there some touch it because like the paper

527
00:27:05,524 --> 00:27:10,404
Speaker 2: clip version of AI seems to like dis ribe these

528
00:27:10,444 --> 00:27:15,804
Speaker 2: machines that are like hyper maximizing and kind of quite

529
00:27:16,964 --> 00:27:22,724
Speaker 2: literal minded and strategic, whereas instead like to kill everybody

530
00:27:23,124 --> 00:27:26,444
Speaker 2: would require a lot of long term planning. Right. I

531
00:27:26,484 --> 00:27:28,924
Speaker 2: was at the symphony last night. I had a little

532
00:27:28,964 --> 00:27:31,084
Speaker 2: break in between, and I'm like, I'm gonna ask chutch Ebt,

533
00:27:31,684 --> 00:27:34,964
Speaker 2: suppose you have a thousand points to allocate between all

534
00:27:35,004 --> 00:27:38,004
Speaker 2: classical composers. How do you allocate those points? And it's

535
00:27:38,004 --> 00:27:40,164
Speaker 2: two hundred DEBATEO of one hundred and eighty to Mozart,

536
00:27:40,684 --> 00:27:42,404
Speaker 2: It likes back two hundred and fifty to Black.

537
00:27:42,484 --> 00:27:42,644
Speaker 1: Right.

538
00:27:42,964 --> 00:27:45,484
Speaker 2: Then I add them up and it's one and twenty points.

539
00:27:45,524 --> 00:27:47,484
Speaker 2: It added twenty points, right, And so like, we have

540
00:27:47,604 --> 00:27:50,404
Speaker 2: these experiences where it fucks up basic shit and it

541
00:27:50,484 --> 00:27:55,924
Speaker 2: kind of acts like autistic savant thirteen year old some

542
00:27:56,044 --> 00:28:02,804
Speaker 2: of the time. Right, how can it chain together enough

543
00:28:02,844 --> 00:28:07,524
Speaker 2: steps without fucking up to kill everybody? Yeah?

544
00:28:07,524 --> 00:28:09,964
Speaker 4: So, I mean currently they can't. I'm not out here

545
00:28:09,964 --> 00:28:13,244
Speaker 4: saying tragipt is going to kill you tomorrow. The things

546
00:28:13,284 --> 00:28:18,764
Speaker 4: where they're currently not terribly profitable are in some sense

547
00:28:18,764 --> 00:28:22,284
Speaker 4: the like also the things where they're not very dangerous,

548
00:28:23,404 --> 00:28:25,284
Speaker 4: like the stuff where they can't chain together a long

549
00:28:25,364 --> 00:28:28,364
Speaker 4: term plan. So my read of the theory has sort

550
00:28:28,404 --> 00:28:34,044
Speaker 4: of long predicted that the abilities to be more capable

551
00:28:34,084 --> 00:28:35,644
Speaker 4: will come hand in hand with the abilities to be

552
00:28:35,684 --> 00:28:37,764
Speaker 4: more dangerous. On this axis, I think we're starting to

553
00:28:37,764 --> 00:28:41,004
Speaker 4: see some empirical evidence of that, where we see AIS

554
00:28:41,004 --> 00:28:43,444
Speaker 4: that are sort of trained on problem solving and then

555
00:28:43,484 --> 00:28:46,084
Speaker 4: they wind up, you know, being more tenacious on other

556
00:28:46,124 --> 00:28:48,324
Speaker 4: types of problems. We have an example in the book

557
00:28:48,444 --> 00:28:50,964
Speaker 4: of the AI that was trained on math puzzles and

558
00:28:51,004 --> 00:28:53,884
Speaker 4: then was given a cybersecurity challenge, and the cybersecurity challenge

559
00:28:53,924 --> 00:28:56,764
Speaker 4: was unknown to the programmer's misconfigured and the AI broke

560
00:28:56,804 --> 00:28:59,244
Speaker 4: out of the testing environment and started up the server

561
00:28:59,324 --> 00:29:00,484
Speaker 4: that was not started up correctly.

562
00:29:01,164 --> 00:29:04,364
Speaker 3: This sort of behavior, it came for free.

563
00:29:04,404 --> 00:29:07,124
Speaker 4: It came along with the ride of making this thing

564
00:29:07,164 --> 00:29:11,964
Speaker 4: able to solve math problems a bit better, because to

565
00:29:12,044 --> 00:29:14,244
Speaker 4: solve math problem as well, you need to learn some

566
00:29:14,404 --> 00:29:18,644
Speaker 4: general skills of like knowing what your resources are looking for,

567
00:29:18,684 --> 00:29:21,844
Speaker 4: clever solutions, not giving up even when it seems like

568
00:29:21,884 --> 00:29:26,004
Speaker 4: that the task is insurmountable. Right, AIS aren't great at

569
00:29:26,004 --> 00:29:30,124
Speaker 4: it yet, but the same path of making them more

570
00:29:30,204 --> 00:29:34,684
Speaker 4: useful makes them, like gives them also more of the

571
00:29:34,724 --> 00:29:38,644
Speaker 4: abilities to do stuff we wouldn't like if they keep

572
00:29:38,644 --> 00:29:41,484
Speaker 4: having these drives no one asked where no one wanted.

573
00:29:41,844 --> 00:29:45,044
Speaker 2: If you ask CHEPT did this recently too, What do

574
00:29:45,084 --> 00:29:48,004
Speaker 2: you think the chances are at near time? Frames for

575
00:29:48,644 --> 00:29:54,684
Speaker 2: recursive self improvement right, Meaning that the AI learns to

576
00:29:54,724 --> 00:29:57,364
Speaker 2: make itself better is one simple way to put it.

577
00:29:57,484 --> 00:29:57,684
Speaker 3: Right.

578
00:29:57,884 --> 00:29:59,564
Speaker 2: It said, well, they're kind of like, they're kind of

579
00:29:59,644 --> 00:30:02,244
Speaker 2: soft forms of this already, like helping humans with programming

580
00:30:02,404 --> 00:30:04,564
Speaker 2: who are AI researchers helped a little bit, but it

581
00:30:04,604 --> 00:30:08,764
Speaker 2: gave very low probabilities in the near term of recursive

582
00:30:08,764 --> 00:30:13,724
Speaker 2: self improvement. Right. How important is that recursive loop to

583
00:30:13,884 --> 00:30:18,284
Speaker 2: the scenarios that you are worried about? Should we trust

584
00:30:18,364 --> 00:30:20,604
Speaker 2: chat GPT when it says I don't think this is

585
00:30:20,684 --> 00:30:22,204
Speaker 2: very likely talk about that.

586
00:30:22,204 --> 00:30:23,604
Speaker 1: That's what I was about to say. I was like

587
00:30:23,684 --> 00:30:26,204
Speaker 1: that you can't trust that out, but that is.

588
00:30:26,124 --> 00:30:29,164
Speaker 4: What I tell you either way. Yeah, I don't think

589
00:30:29,204 --> 00:30:34,644
Speaker 4: that's true yet. The I don't think it's vital to

590
00:30:34,684 --> 00:30:39,524
Speaker 4: the argument that you have a recursive self improvement sort

591
00:30:39,524 --> 00:30:43,044
Speaker 4: of hard take off situation. I think it can go

592
00:30:43,124 --> 00:30:46,764
Speaker 4: really badly in a lot of the slow takeoff situations.

593
00:30:47,164 --> 00:30:50,324
Speaker 4: My best guess is still that you will hit some

594
00:30:50,364 --> 00:30:54,804
Speaker 4: sort of recursive self improvement threshold at some point. One

595
00:30:54,804 --> 00:30:56,364
Speaker 4: thing I'd say about that is that if you were

596
00:30:56,404 --> 00:31:03,884
Speaker 4: looking at like chimpanzees and early hominids, a few million

597
00:31:03,964 --> 00:31:07,684
Speaker 4: years ago, it would be really hard to call when

598
00:31:07,764 --> 00:31:10,884
Speaker 4: things were going over some threshold such that it could

599
00:31:10,924 --> 00:31:14,724
Speaker 4: come together. And humans aren't better. Humans don't have an

600
00:31:14,764 --> 00:31:16,924
Speaker 4: extra walk on the Moon module. They don't have an

601
00:31:16,964 --> 00:31:20,764
Speaker 4: extra engineering module. We have all the same stuff scaled up,

602
00:31:20,964 --> 00:31:22,844
Speaker 4: and we're a little bit better at a lot of things.

603
00:31:24,844 --> 00:31:26,764
Speaker 4: R lms the sort of thing where if you get

604
00:31:26,764 --> 00:31:28,564
Speaker 4: a little better at a lot of things, they suddenly

605
00:31:28,764 --> 00:31:31,804
Speaker 4: sort of like go over some threshold like how humans did,

606
00:31:31,924 --> 00:31:33,324
Speaker 4: and it's all starts coming together.

607
00:31:34,124 --> 00:31:34,524
Speaker 3: I don't know.

608
00:31:34,604 --> 00:31:36,044
Speaker 4: I don't think anyone knows what I know what's going

609
00:31:36,044 --> 00:31:38,804
Speaker 4: on inside these things. My best guess is, like probably not.

610
00:31:38,964 --> 00:31:41,364
Speaker 4: My best guess is that you need some other architectural

611
00:31:41,364 --> 00:31:45,444
Speaker 4: insight before it really starts to run away. But you know,

612
00:31:45,724 --> 00:31:47,004
Speaker 4: it's all total guesswork.

613
00:31:47,404 --> 00:31:51,684
Speaker 2: Do you expect super intelligence to be not plausible but

614
00:31:51,844 --> 00:31:54,444
Speaker 2: likely based on large language models, or to be some

615
00:31:54,564 --> 00:31:57,644
Speaker 2: fundamentally new technology. I think my.

616
00:31:57,724 --> 00:32:00,764
Speaker 4: Top guess is that you need some other breakthrough on

617
00:32:00,924 --> 00:32:04,844
Speaker 4: the size of like transformers, which maybe isn't totally new

618
00:32:04,964 --> 00:32:06,724
Speaker 4: in the like transformers are still sort of in this

619
00:32:06,764 --> 00:32:11,044
Speaker 4: deep learning paradigm, but like you know, one sort of

620
00:32:11,084 --> 00:32:13,804
Speaker 4: analogy you might use for large language models is a

621
00:32:13,844 --> 00:32:19,044
Speaker 4: little bit like you have like the intelligence of a

622
00:32:19,084 --> 00:32:22,204
Speaker 4: six year old, but they get to live for a

623
00:32:22,244 --> 00:32:27,644
Speaker 4: million years and read everything and practice a toneh and

624
00:32:27,844 --> 00:32:29,764
Speaker 4: like sometimes you give a six year old, you know,

625
00:32:31,244 --> 00:32:33,084
Speaker 4: the ability to write like ten books on trying to

626
00:32:33,084 --> 00:32:34,484
Speaker 4: solve a math puzzle, and then it turns out they

627
00:32:34,524 --> 00:32:36,404
Speaker 4: can solve that math puzzle after like writing out ten

628
00:32:36,444 --> 00:32:37,844
Speaker 4: books and you read it and some of it's good

629
00:32:37,844 --> 00:32:40,844
Speaker 4: and some of it's just like total nonsense. But like, okay,

630
00:32:41,044 --> 00:32:44,964
Speaker 4: imagine that you somehow now had a seventeen year old,

631
00:32:46,164 --> 00:32:47,684
Speaker 4: but you still have the ability to like give them

632
00:32:47,684 --> 00:32:49,684
Speaker 4: a million years to think and the ability to write, like,

633
00:32:49,884 --> 00:32:55,324
Speaker 4: you know, ten books per question they were answering, Like

634
00:32:55,404 --> 00:32:58,244
Speaker 4: you could really see leaps and capabilities here. You could

635
00:32:58,284 --> 00:33:01,884
Speaker 4: see that like relatively small improvements and algorithms make the

636
00:33:01,924 --> 00:33:04,724
Speaker 4: amount the sheer enormous amount of competing power and data

637
00:33:04,724 --> 00:33:11,404
Speaker 4: we have now just radically like overpowers the next generation

638
00:33:11,444 --> 00:33:13,364
Speaker 4: of AI for the problems at hand, or maybe not.

639
00:33:13,484 --> 00:33:15,284
Speaker 4: I don't know, it's hard to guess.

640
00:33:15,484 --> 00:33:18,964
Speaker 2: Yeah, one of the I guess points of skepticism might

641
00:33:18,964 --> 00:33:21,244
Speaker 2: have and again we're talking about you know, I'm kind

642
00:33:21,284 --> 00:33:24,764
Speaker 2: of with you on scenario two versus scenario three, maybe

643
00:33:24,764 --> 00:33:27,244
Speaker 2: both being pretty bad. Frankly, right, I think I'm a

644
00:33:27,244 --> 00:33:30,684
Speaker 2: little more skeptical than you one on super intelligence. On

645
00:33:30,884 --> 00:33:35,484
Speaker 2: your time frames, do you think developments over the past

646
00:33:35,604 --> 00:33:42,524
Speaker 2: year suggest that general intelligence is further away than you

647
00:33:42,604 --> 00:33:44,364
Speaker 2: might have thought at the start of the year. I mean,

648
00:33:44,404 --> 00:33:47,044
Speaker 2: the AI twenty twenty seven authors, they're pushing back their

649
00:33:47,044 --> 00:33:49,124
Speaker 2: timelines by a couple of years. You and Elliezer said

650
00:33:49,124 --> 00:33:51,524
Speaker 2: that doesn't really matter. The sticks are very high, and

651
00:33:51,644 --> 00:33:54,604
Speaker 2: AM twenty twenty seven ver twenty twenty nine verses twenty

652
00:33:54,604 --> 00:33:57,444
Speaker 2: thirty five, doesn't really matter if we're all dead by

653
00:33:57,484 --> 00:33:58,284
Speaker 2: twenty four. Yeah.

654
00:33:58,364 --> 00:33:58,484
Speaker 1: Right.

655
00:33:59,164 --> 00:34:02,044
Speaker 2: Have your timelines been pushed back at all, or forward

656
00:34:02,044 --> 00:34:03,484
Speaker 2: for that matter, Yeah.

657
00:34:03,484 --> 00:34:05,484
Speaker 4: I mean, I've never been one of the folks being like,

658
00:34:05,844 --> 00:34:09,084
Speaker 4: I'm confident this is going to happen soon. And I

659
00:34:09,084 --> 00:34:11,164
Speaker 4: mean day twenty twenty seven folks, I think weren't like,

660
00:34:11,164 --> 00:34:13,804
Speaker 4: we're very confident this kind of happened here. But you know,

661
00:34:13,884 --> 00:34:15,764
Speaker 4: I haven't really been going around saying I'm confident in

662
00:34:15,804 --> 00:34:20,284
Speaker 4: short timelines. I have for a while said my best

663
00:34:20,324 --> 00:34:22,284
Speaker 4: guess is that we're going to need another advancement past

664
00:34:22,404 --> 00:34:26,364
Speaker 4: LMS towards that end. You know, the the evidence of

665
00:34:26,444 --> 00:34:29,884
Speaker 4: the past year is mostly in line with my expectations,

666
00:34:29,964 --> 00:34:32,404
Speaker 4: But my expectations were already that we'll probably need another

667
00:34:32,444 --> 00:34:37,484
Speaker 4: advancement past LMS. You know, I do think there have

668
00:34:37,604 --> 00:34:40,964
Speaker 4: been times where I was, you know, I updated towards

669
00:34:41,084 --> 00:34:44,444
Speaker 4: LLLMS a couple of times, you know, the I think

670
00:34:44,444 --> 00:34:47,764
Speaker 4: I remember GPT four O playing chess better than I

671
00:34:47,764 --> 00:34:49,884
Speaker 4: thought elms were going to manage to play chess, and

672
00:34:49,924 --> 00:34:53,324
Speaker 4: so that was like an eventual update towards maybe maybe

673
00:34:53,364 --> 00:34:55,964
Speaker 4: ALMS will go farther than I thought they could. The

674
00:34:56,004 --> 00:35:00,324
Speaker 4: reasoning models also were sort of an update in favor of, like,

675
00:35:00,364 --> 00:35:02,164
Speaker 4: these guys aren't just going to stick with single forward

676
00:35:02,164 --> 00:35:04,084
Speaker 4: pass LMS. They're going to like cleverly try to find

677
00:35:04,124 --> 00:35:07,444
Speaker 4: ways to leverage it, and so you know, it goes

678
00:35:07,524 --> 00:35:09,844
Speaker 4: up and down. Those were some like oh, maybe alums

679
00:35:09,844 --> 00:35:11,844
Speaker 4: can and this year's but no, maybe all alums can't.

680
00:35:12,284 --> 00:35:15,484
Speaker 4: But but yeah, largely, you know, I'm I'm I'm not

681
00:35:15,524 --> 00:35:17,924
Speaker 4: here saying like watch out for the albums in particular,

682
00:35:18,124 --> 00:35:19,924
Speaker 4: you know, and I was I was again thought this

683
00:35:19,964 --> 00:35:21,644
Speaker 4: problem would be hard back in twenty twelve. That was

684
00:35:21,684 --> 00:35:24,084
Speaker 4: even before it was super clear deep learning was gonna

685
00:35:24,404 --> 00:35:26,484
Speaker 4: was gonna go go as hard as it did. Never

686
00:35:26,564 --> 00:35:28,604
Speaker 4: mind LMS in particular.

687
00:35:33,244 --> 00:35:51,564
Speaker 2: And we'll be right back after this break. There's kind

688
00:35:51,564 --> 00:35:56,524
Speaker 2: of a class of what you might call non existential risks,

689
00:35:57,044 --> 00:36:01,484
Speaker 2: which range from energy consumed by data centers to aguoridthic discrimination.

690
00:36:01,684 --> 00:36:05,564
Speaker 2: Some of these are kind of progressive coded up to

691
00:36:06,644 --> 00:36:11,564
Speaker 2: dystopian but not quite bocalyptic outcomes where extremely powerful corporations,

692
00:36:12,324 --> 00:36:16,204
Speaker 2: hand in hand with AI models kind of deprive the

693
00:36:16,244 --> 00:36:19,964
Speaker 2: world of much of its freedom and agency. Right, how

694
00:36:20,004 --> 00:36:23,884
Speaker 2: compatible is everyone dies with those other concerns, because sometimes

695
00:36:23,884 --> 00:36:26,724
Speaker 2: they seem to be like, you know, I read a

696
00:36:26,764 --> 00:36:30,084
Speaker 2: lot on the Internet. I write about politics all the time, right,

697
00:36:30,204 --> 00:36:33,724
Speaker 2: and I think the kind of average politics argue we're

698
00:36:33,764 --> 00:36:37,124
Speaker 2: on Twitter or Blue Skies, like these weird sci fi nerds.

699
00:36:37,764 --> 00:36:39,764
Speaker 2: I think it's going to be the terminator, right when

700
00:36:39,764 --> 00:36:42,844
Speaker 2: really we should be to care about intrenching discrimination. And

701
00:36:43,924 --> 00:36:46,564
Speaker 2: you know, you would think that the left would be

702
00:36:46,604 --> 00:36:50,964
Speaker 2: on the evil corporations bandwagon. But because like AI is

703
00:36:51,044 --> 00:36:54,084
Speaker 2: kind of tech pro coded, that scrambles it a little bit.

704
00:36:54,084 --> 00:36:56,364
Speaker 2: What do you think about those ordinary concerns including by

705
00:36:56,364 --> 00:37:01,524
Speaker 2: the way AI being not all that competent, and so

706
00:37:01,724 --> 00:37:06,804
Speaker 2: humans entrusting critical systems to AIS that fail or are

707
00:37:06,804 --> 00:37:07,964
Speaker 2: corruptible or fuck up.

708
00:37:09,084 --> 00:37:09,644
Speaker 3: Yeah, I know.

709
00:37:10,604 --> 00:37:13,484
Speaker 4: One piece of this puzzle I think is I think

710
00:37:13,484 --> 00:37:17,324
Speaker 4: a lot of people are cynical towards the companies. I

711
00:37:17,364 --> 00:37:20,884
Speaker 4: think that's often merited, but I think the cynicism is

712
00:37:20,884 --> 00:37:23,084
Speaker 4: often a little bit misplaced, and they're like, oh, you know,

713
00:37:23,124 --> 00:37:25,204
Speaker 4: the AIS all hype, it'll never go anywhere, it can't

714
00:37:25,204 --> 00:37:27,044
Speaker 4: do any You know, there's a lot of ways that

715
00:37:27,084 --> 00:37:31,124
Speaker 4: the current AI is dumb, but it's much easier to

716
00:37:31,244 --> 00:37:35,764
Speaker 4: see how AI could take a leap in the future

717
00:37:35,764 --> 00:37:37,484
Speaker 4: if you've seen AIS take a leap in the past.

718
00:37:38,164 --> 00:37:40,924
Speaker 4: And these companies existed before chatbots. They see chatbots as

719
00:37:40,964 --> 00:37:42,644
Speaker 4: a stepping stone. Their stated goal is to, you know,

720
00:37:42,684 --> 00:37:47,084
Speaker 4: make eyes that can automate all human labor, and in

721
00:37:47,124 --> 00:37:50,044
Speaker 4: some sense an even more cynical perspective from a certain

722
00:37:50,124 --> 00:37:52,404
Speaker 4: point of view of like, these guys know they're risking

723
00:37:52,444 --> 00:37:55,524
Speaker 4: everyone's lives and the rushing head anyway, and you know

724
00:37:55,524 --> 00:37:59,764
Speaker 4: they're not shy about this. Sam Altman's talking about the

725
00:37:59,844 --> 00:38:02,084
Speaker 4: dangerous here has gone down as the company has gotten

726
00:38:02,084 --> 00:38:05,244
Speaker 4: more profitable, and it's sort of like the ones who

727
00:38:05,284 --> 00:38:08,844
Speaker 4: are like more known for just saying what they actually believe,

728
00:38:08,924 --> 00:38:11,684
Speaker 4: like Elon, which, for all that you can say about him,

729
00:38:12,124 --> 00:38:16,964
Speaker 4: he's not really one to hide his thoughts. Who will

730
00:38:16,964 --> 00:38:18,524
Speaker 4: come out and be like, oh yeah, ten twenty percent

731
00:38:18,604 --> 00:38:20,484
Speaker 4: chance this kills you all. So one thing I would

732
00:38:20,524 --> 00:38:22,084
Speaker 4: say there is like I.

733
00:38:23,884 --> 00:38:24,364
Speaker 2: Sort of.

734
00:38:25,844 --> 00:38:30,764
Speaker 4: Think it's even crazier than it looks on the surface.

735
00:38:31,484 --> 00:38:33,684
Speaker 4: AI may be a bubble, it may be hype. The

736
00:38:34,124 --> 00:38:37,164
Speaker 4: Internet had a bubble, the Internet was still real, you know,

737
00:38:37,284 --> 00:38:39,284
Speaker 4: And you have all these academics being like, oh yeah,

738
00:38:39,284 --> 00:38:40,844
Speaker 4: this is really dangerous. You have folks like us Oup

739
00:38:40,844 --> 00:38:42,244
Speaker 4: and around before them being like, oh yeah, this is

740
00:38:42,244 --> 00:38:45,604
Speaker 4: really dangerous. I would implore people to be like even

741
00:38:45,644 --> 00:38:48,084
Speaker 4: more cynical about taking these people out their word that

742
00:38:48,084 --> 00:38:51,204
Speaker 4: they are gambling with your very life. In terms of

743
00:38:51,284 --> 00:38:54,764
Speaker 4: all of the other issues with AI, including ones we're

744
00:38:54,764 --> 00:38:57,564
Speaker 4: sort of already starting to see today, unfortunately, the world

745
00:38:57,644 --> 00:39:01,404
Speaker 4: is big enough for multiple issues. I'm not like, my

746
00:39:01,524 --> 00:39:03,844
Speaker 4: issue is real. Those issues are fake. We're going to

747
00:39:03,924 --> 00:39:05,844
Speaker 4: need to figure out how to integrate AI with education.

748
00:39:06,844 --> 00:39:09,564
Speaker 4: It's true that if we make like robot police forces

749
00:39:09,724 --> 00:39:13,804
Speaker 4: that would allow it to talitarian states to be much

750
00:39:13,844 --> 00:39:16,364
Speaker 4: more brutal in en forcing it to talitarian regime because

751
00:39:16,364 --> 00:39:18,444
Speaker 4: there's certain orders you can't give to a human police

752
00:39:18,484 --> 00:39:20,804
Speaker 4: force because they won't do it, that you could give

753
00:39:21,364 --> 00:39:25,004
Speaker 4: to a robotic police force. Right, those are real things

754
00:39:25,044 --> 00:39:28,444
Speaker 4: we're gonna have to grapple with. Also, if we keep

755
00:39:28,524 --> 00:39:30,484
Speaker 4: racing to ais that are smarter and smarter that can

756
00:39:30,524 --> 00:39:33,564
Speaker 4: automate this process of technological innovation, which is what they

757
00:39:33,604 --> 00:39:36,564
Speaker 4: say they're racing towards, the most likely outcome is that

758
00:39:36,604 --> 00:39:39,524
Speaker 4: everyone on Earth dies. We just need to solve both

759
00:39:39,564 --> 00:39:40,284
Speaker 4: set of problems.

760
00:39:41,884 --> 00:39:46,004
Speaker 2: Yeah, I've updated. When I mean positive, I mean the

761
00:39:46,004 --> 00:39:49,444
Speaker 2: p doom risk is going down, right, I've updated slightly

762
00:39:49,524 --> 00:39:51,524
Speaker 2: positively as far as I think timeline is going to

763
00:39:51,524 --> 00:39:53,084
Speaker 2: be a little longer than I always thought a year

764
00:39:53,164 --> 00:39:59,004
Speaker 2: or two ago. I've updated negatively based on culture and politics,

765
00:39:59,124 --> 00:40:05,004
Speaker 2: right on various frints. Yeah, Number one, the kind of

766
00:40:05,324 --> 00:40:09,844
Speaker 2: workaholic culture of Silicon Valley. When I hear like the

767
00:40:09,964 --> 00:40:12,764
Speaker 2: nine ninety six culture, whatever it is, right, but just crying.

768
00:40:12,804 --> 00:40:16,604
Speaker 2: Go go to your lab, work really hard. Your friends

769
00:40:16,604 --> 00:40:20,244
Speaker 2: should be in the lab, right, Your activity should be

770
00:40:20,244 --> 00:40:22,964
Speaker 2: things that are health oriented. Or self improvement oriented. It's like,

771
00:40:23,444 --> 00:40:25,004
Speaker 2: I know, I feel better if we had the tech

772
00:40:25,044 --> 00:40:28,964
Speaker 2: bros going out and getting drunk on Friday night, right

773
00:40:29,004 --> 00:40:31,684
Speaker 2: and trying to get laid and getting in touch with humanity,

774
00:40:31,764 --> 00:40:32,924
Speaker 2: you know what I mean, And if they had like

775
00:40:32,964 --> 00:40:35,324
Speaker 2: material things and children and things like that that they

776
00:40:35,484 --> 00:40:39,924
Speaker 2: valued potentially. Yeah, I mean, like, what would you say,

777
00:40:39,924 --> 00:40:43,884
Speaker 2: has happened to you know? Where would you put Elon now?

778
00:40:44,004 --> 00:40:44,164
Speaker 3: Right?

779
00:40:44,204 --> 00:40:49,244
Speaker 2: I believe that GROC is not regarded or XAI is

780
00:40:49,284 --> 00:40:53,524
Speaker 2: not regarded as among the most responsible of the AI labs. Right,

781
00:40:53,924 --> 00:40:58,804
Speaker 2: he has expressed, I assumed sincere concerned about p doom

782
00:40:59,244 --> 00:41:00,924
Speaker 2: in the past, where would you where would you peg

783
00:41:01,004 --> 00:41:01,804
Speaker 2: Elon now?

784
00:41:03,164 --> 00:41:05,244
Speaker 4: I mean he's not You can just listen to his

785
00:41:05,244 --> 00:41:07,564
Speaker 4: own words, you know. He said over the summer, I

786
00:41:07,564 --> 00:41:11,884
Speaker 4: think realized I could either be a bystander or a participant,

787
00:41:12,804 --> 00:41:14,564
Speaker 4: and that was like gonna happen one way or the other.

788
00:41:14,644 --> 00:41:16,084
Speaker 3: Needed rather be a participant.

789
00:41:17,484 --> 00:41:20,964
Speaker 4: But you know, also in terms of who's the safer one,

790
00:41:20,964 --> 00:41:25,084
Speaker 4: who's the more dangerous one? It it looks to me

791
00:41:25,164 --> 00:41:27,564
Speaker 4: like no one has anywhere nearing a chance of doing

792
00:41:27,644 --> 00:41:28,524
Speaker 4: this job, right.

793
00:41:28,644 --> 00:41:30,044
Speaker 2: Yeah, I mean you kind of it feels like you

794
00:41:30,084 --> 00:41:33,604
Speaker 2: hear less about alignment than you did a year or two.

795
00:41:33,924 --> 00:41:39,244
Speaker 2: It kind of felt like Trump one and everyone's like, oh,

796
00:41:39,284 --> 00:41:42,684
Speaker 2: we're just accelerating now, right, Trump wins and China's becoming

797
00:41:42,724 --> 00:41:45,964
Speaker 2: more competitive, and like, you know, it seems like people

798
00:41:46,004 --> 00:41:49,804
Speaker 2: have given up on this whole alignment alignment thing, right,

799
00:41:49,924 --> 00:41:52,764
Speaker 2: Is that true or well or yeah? Yeah, I mean

800
00:41:52,804 --> 00:41:56,764
Speaker 2: I think a lot of it is that people like

801
00:41:56,844 --> 00:41:59,804
Speaker 2: are largely realizing they can't. Yeah, right.

802
00:41:59,924 --> 00:42:03,044
Speaker 4: Like you, you have interpretability, which is like trying to

803
00:42:03,044 --> 00:42:04,444
Speaker 4: figure out what the heck is going on in there.

804
00:42:04,484 --> 00:42:06,324
Speaker 4: You have evaluations, which is trying to figure out how

805
00:42:06,404 --> 00:42:07,284
Speaker 4: dangerous it is already.

806
00:42:08,084 --> 00:42:10,884
Speaker 3: And if if you were going to.

807
00:42:10,844 --> 00:42:13,524
Speaker 4: Engineers and you were like, can you tell me about

808
00:42:13,524 --> 00:42:17,564
Speaker 4: why this uh, this nuclear power plant is not going

809
00:42:17,644 --> 00:42:19,764
Speaker 4: to melt down? And they're like, well, yeah, we have

810
00:42:20,364 --> 00:42:23,004
Speaker 4: two safety teams. One is trying to figure out what

811
00:42:23,044 --> 00:42:26,044
Speaker 4: the heck is going on inside the plant. The other

812
00:42:26,124 --> 00:42:30,764
Speaker 4: is trying to measure whether it has exploded yet. You

813
00:42:30,804 --> 00:42:35,484
Speaker 4: would be like, okay, guys, ease off on the growing

814
00:42:35,524 --> 00:42:39,484
Speaker 4: pile of uranium, you know, And I'm like, I'm not.

815
00:42:39,524 --> 00:42:42,684
Speaker 4: I'm not trying to like, uh degrade the people working

816
00:42:42,684 --> 00:42:45,044
Speaker 4: on this. It's way better to have a team that's

817
00:42:45,084 --> 00:42:46,844
Speaker 4: trying to figure out what's going on inside the nuclear

818
00:42:46,844 --> 00:42:49,964
Speaker 4: power plant than to not, okay, it's it's way better

819
00:42:50,004 --> 00:42:53,004
Speaker 4: to have someone measuring is it starting to explode? You know,

820
00:42:53,084 --> 00:42:55,644
Speaker 4: the guys trying to do this are heroes in my book,

821
00:42:56,084 --> 00:43:00,564
Speaker 4: but like it, we're not close, and I think part

822
00:43:00,564 --> 00:43:02,004
Speaker 4: of why people aren't talking about is that we're not

823
00:43:02,044 --> 00:43:03,724
Speaker 4: We're not close to being able to do the job right.

824
00:43:03,924 --> 00:43:06,724
Speaker 1: Do you think that some of the tragedies that are happening,

825
00:43:07,044 --> 00:43:10,524
Speaker 1: you know, every day with chatbots, you know, mental health problems,

826
00:43:10,564 --> 00:43:14,124
Speaker 1: we see suicides, like we're seeing a lot of things

827
00:43:14,164 --> 00:43:17,964
Speaker 1: happening even with this fairly primitive, even though it's so

828
00:43:18,164 --> 00:43:22,804
Speaker 1: advanced technology. Do you think that that will be kind

829
00:43:22,804 --> 00:43:25,724
Speaker 1: of enough of a warning sign for people to start

830
00:43:25,724 --> 00:43:29,644
Speaker 1: taking note outside of the Silicon Valley elite who actually

831
00:43:29,684 --> 00:43:31,964
Speaker 1: know this more closely, who have you read your book,

832
00:43:31,964 --> 00:43:34,004
Speaker 1: who understand all of the implications.

833
00:43:34,604 --> 00:43:37,204
Speaker 4: You know, I'm hopeful in some sense the chat GPT

834
00:43:37,284 --> 00:43:43,844
Speaker 4: moment was more global awareness of the AI potential for

835
00:43:43,924 --> 00:43:46,964
Speaker 4: killing us all than I expected to maybe ever get.

836
00:43:47,604 --> 00:43:52,164
Speaker 4: You know, it's back before large language models a lot

837
00:43:52,164 --> 00:43:54,724
Speaker 4: of people predicted that AI that could sort of like

838
00:43:55,004 --> 00:43:58,484
Speaker 4: talk well was actually very late in the AI game.

839
00:43:59,164 --> 00:44:00,564
Speaker 4: You know, we used to have like mor of x

840
00:44:00,604 --> 00:44:02,604
Speaker 4: paradox where we'd say things that are like easy for

841
00:44:02,644 --> 00:44:04,164
Speaker 4: computers are are hard for.

842
00:44:04,164 --> 00:44:05,844
Speaker 3: Humans, and vice versa.

843
00:44:06,964 --> 00:44:08,644
Speaker 4: And you know, it could have been that this just

844
00:44:08,684 --> 00:44:12,564
Speaker 4: all stayed like very academic, very like only people in

845
00:44:12,604 --> 00:44:15,204
Speaker 4: the industry are watching it like it was up until

846
00:44:15,324 --> 00:44:18,884
Speaker 4: you know, maybe twenty twenty one. It could have stayed

847
00:44:18,924 --> 00:44:20,964
Speaker 4: that way all the way up until you know, you

848
00:44:21,004 --> 00:44:27,284
Speaker 4: have these AI companies making AI researcher ais in their

849
00:44:27,364 --> 00:44:31,524
Speaker 4: labs and no one's noticing. And that gives me some

850
00:44:31,564 --> 00:44:33,404
Speaker 4: hope that maybe we don't even need a disaster. Maybe

851
00:44:33,444 --> 00:44:37,964
Speaker 4: we just need another capability advance. Maybe when people do

852
00:44:38,084 --> 00:44:40,764
Speaker 4: live through, maybe people live through one more hop of

853
00:44:40,804 --> 00:44:44,204
Speaker 4: like AI's doing stuff they qualitatively couldn't do before. Maybe

854
00:44:44,204 --> 00:44:46,404
Speaker 4: that would be enough to wake people up. And then

855
00:44:46,404 --> 00:44:48,324
Speaker 4: on the flip side, you know, there can also be

856
00:44:48,364 --> 00:44:50,844
Speaker 4: big warning shots where people fail to learn from the

857
00:44:50,844 --> 00:44:54,924
Speaker 4: warning shot. One thing I'd say there is that what

858
00:44:54,964 --> 00:44:57,244
Speaker 4: you can learn from a warning shot, it depends a

859
00:44:57,284 --> 00:44:59,764
Speaker 4: little bit on how prepared you already are and how

860
00:44:59,804 --> 00:45:02,644
Speaker 4: knowledgeable you already are, and so you know there's no

861
00:45:03,084 --> 00:45:06,924
Speaker 4: substitute for sort of getting people to understand the issues today.

862
00:45:08,084 --> 00:45:12,124
Speaker 2: How much of the things we need to do as

863
00:45:12,164 --> 00:45:15,564
Speaker 2: a society right to mitigate this risk are kind of

864
00:45:15,604 --> 00:45:19,604
Speaker 2: technical steps versus political steps, like obviously, yeah, I'm sure

865
00:45:19,604 --> 00:45:23,924
Speaker 2: you've had this elevator pitch where the first sixty seconds

866
00:45:24,004 --> 00:45:27,084
Speaker 2: explaining why you're worried. Now, let's say I encounter you

867
00:45:27,124 --> 00:45:29,124
Speaker 2: on the same elevator. We're now going back to our rooms.

868
00:45:29,164 --> 00:45:32,124
Speaker 2: But I have another sixty seconds, right, what's the quick

869
00:45:32,164 --> 00:45:34,364
Speaker 2: pitch on what we should do? Is there a one

870
00:45:34,484 --> 00:45:37,804
Speaker 2: kind of genie wish if you could fiat something? Is

871
00:45:37,804 --> 00:45:39,804
Speaker 2: it more complicated than that? Do we know what we

872
00:45:39,884 --> 00:45:41,084
Speaker 2: have to do and it's a matter of forming the

873
00:45:41,164 --> 00:45:44,204
Speaker 2: right political coalition or all right, you have sixty ish

874
00:45:44,324 --> 00:45:44,964
Speaker 2: second snake.

875
00:45:47,444 --> 00:45:49,164
Speaker 4: I think what needs to happen is the world needs

876
00:45:49,164 --> 00:45:52,644
Speaker 4: to stop the race towards smarter than human AI, and

877
00:45:52,924 --> 00:45:54,884
Speaker 4: it needs to stop it everywhere, because no one can

878
00:45:54,924 --> 00:45:57,124
Speaker 4: keep these things in a leash. If anyone builds a

879
00:45:57,124 --> 00:46:00,524
Speaker 4: super intelligence anywhere on the planet, everyone dies, doesn't matter

880
00:46:00,564 --> 00:46:06,524
Speaker 4: who gets it. The world could stop AI is made

881
00:46:07,324 --> 00:46:10,884
Speaker 4: using extremely specialized chips in very large data centers that

882
00:46:10,884 --> 00:46:14,324
Speaker 4: suck down as much electricity as a city. It would

883
00:46:14,324 --> 00:46:19,924
Speaker 4: be probably easier to monitor than uranium. And I think

884
00:46:19,964 --> 00:46:23,804
Speaker 4: it's I think it's almost entirely political right now. I

885
00:46:23,844 --> 00:46:25,884
Speaker 4: think we are nowhere near solving on the technical side.

886
00:46:25,924 --> 00:46:28,084
Speaker 4: It's not to say it can't be solved technically, but

887
00:46:28,244 --> 00:46:31,124
Speaker 4: not only are we nowhere near close, But unlike a

888
00:46:31,124 --> 00:46:33,324
Speaker 4: lot of other scientific problems, we don't get free tries

889
00:46:34,524 --> 00:46:38,244
Speaker 4: the first time you have some theory of you know, this,

890
00:46:38,244 --> 00:46:40,684
Speaker 4: this AI will definitely be nice to us once it's

891
00:46:40,684 --> 00:46:44,044
Speaker 4: smart enough to be a threat to us. If that

892
00:46:44,084 --> 00:46:47,084
Speaker 4: theory is wrong, you sort of don't get to put

893
00:46:47,124 --> 00:46:50,964
Speaker 4: the genie back in the bottle. And that's unlike other theories.

894
00:46:51,044 --> 00:46:53,684
Speaker 4: That's unlike other science and a lot of other places in science.

895
00:46:53,724 --> 00:46:55,924
Speaker 4: You know, Marie Curie dies of cancer those likely caused

896
00:46:55,924 --> 00:46:59,644
Speaker 4: by radiation poisoning. She was a hero, But often the

897
00:46:59,644 --> 00:47:02,164
Speaker 4: first people interacting with stuff die of them, you know,

898
00:47:02,204 --> 00:47:02,924
Speaker 4: Max Valley, and.

899
00:47:02,964 --> 00:47:05,604
Speaker 2: They probably did things that were objectively kind of crazy, right,

900
00:47:05,964 --> 00:47:10,084
Speaker 2: But you have this selection mechanism where you know, the

901
00:47:10,124 --> 00:47:13,404
Speaker 2: great crazy invention survive the other ones, don't. I mean

902
00:47:14,884 --> 00:47:16,804
Speaker 2: how much would you have to So do you believe

903
00:47:16,804 --> 00:47:19,684
Speaker 2: in the idea of like a slowdown? Right, Like, let's

904
00:47:19,724 --> 00:47:23,524
Speaker 2: say that the people who are skeptical of rapid time

905
00:47:23,564 --> 00:47:28,404
Speaker 2: horizons are correct and this technology develops over the course

906
00:47:28,604 --> 00:47:35,764
Speaker 2: of fifty years instead of fifteen or five or five months.

907
00:47:36,324 --> 00:47:38,284
Speaker 2: Is that enough time to get things right? Would it

908
00:47:38,364 --> 00:47:39,724
Speaker 2: pause or a slow down work?

909
00:47:42,284 --> 00:47:44,524
Speaker 4: You know, fifty is a lot more to work with

910
00:47:44,604 --> 00:47:52,124
Speaker 4: than fifteen. But you got to worry about convergence on

911
00:47:52,164 --> 00:47:56,484
Speaker 4: the wrong on a bad solution, and humanity so far

912
00:47:56,524 --> 00:47:58,964
Speaker 4: has not been bringing its a game and to get

913
00:47:59,004 --> 00:48:03,924
Speaker 4: around on the first try, it's it's hard. Fifty years,

914
00:48:04,204 --> 00:48:07,884
Speaker 4: I could work with you, maybe have a chance, but

915
00:48:07,924 --> 00:48:11,724
Speaker 4: I'd be trying to do things like could we use

916
00:48:11,764 --> 00:48:16,004
Speaker 4: biotech to somehow enhance the intelligence of our humans? But

917
00:48:16,844 --> 00:48:18,604
Speaker 4: I mean, first thing we do is we get a

918
00:48:18,644 --> 00:48:20,764
Speaker 4: slow down somehow. Maybe we'll be liking get a slow down.

919
00:48:20,844 --> 00:48:25,364
Speaker 4: Maybe we'll need to get a slow down by international FIAT.

920
00:48:27,044 --> 00:48:30,844
Speaker 4: But a lot of people say, oh, it will never happen.

921
00:48:30,884 --> 00:48:33,644
Speaker 4: People never slow down. One big reason for hope here,

922
00:48:33,644 --> 00:48:37,924
Speaker 4: I think, is that our world leaders don't yet understand

923
00:48:39,444 --> 00:48:43,604
Speaker 4: what the people in the valley are debating. They don't

924
00:48:43,684 --> 00:48:46,924
Speaker 4: understand that we're in this situation where the possibilities are,

925
00:48:46,924 --> 00:48:48,724
Speaker 4: like the tech just doesn't work, in which case betting

926
00:48:48,724 --> 00:48:51,164
Speaker 4: and it will go nowhere, or you're either making god

927
00:48:51,164 --> 00:48:55,884
Speaker 4: emperors or killing us all, and that the optimistic folks

928
00:48:55,884 --> 00:48:58,044
Speaker 4: who think the tech works think there's like a twenty

929
00:48:58,084 --> 00:49:02,564
Speaker 4: five percent chance it kills us all. You know, it's like, yeah,

930
00:49:02,764 --> 00:49:05,724
Speaker 4: that's like we're racing, but we're racing in a situation

931
00:49:05,764 --> 00:49:08,964
Speaker 4: where people don't realize that either this race goes nowhere

932
00:49:09,084 --> 00:49:10,764
Speaker 4: or go to really bad places.

933
00:49:11,964 --> 00:49:14,204
Speaker 2: Let me ask you kind of a meta question, what

934
00:49:14,244 --> 00:49:16,644
Speaker 2: do you see your role as in this debate?

935
00:49:19,004 --> 00:49:23,724
Speaker 4: I think my role, in Eliezer's role in this situation

936
00:49:24,084 --> 00:49:26,164
Speaker 4: is to come out and say the things that a

937
00:49:26,204 --> 00:49:29,844
Speaker 4: lot of people are thinking, and to sort of like

938
00:49:29,844 --> 00:49:31,724
Speaker 4: to be the kid who says the emperor has no clothes.

939
00:49:32,964 --> 00:49:35,004
Speaker 4: There's a lot of people who are worried here, but

940
00:49:35,444 --> 00:49:37,164
Speaker 4: no one's sort of coming right out and saying this

941
00:49:37,244 --> 00:49:40,364
Speaker 4: is crazy. We should stop. The labs who say there's

942
00:49:40,364 --> 00:49:42,284
Speaker 4: a twenty five percent chance this kills us all, don't

943
00:49:42,324 --> 00:49:44,764
Speaker 4: then do the next step and say so, please put

944
00:49:44,764 --> 00:49:47,804
Speaker 4: a stop to it. Right, My role here is to

945
00:49:47,844 --> 00:49:54,444
Speaker 4: sort of like spell out that last step and be like, guys,

946
00:49:55,084 --> 00:49:57,084
Speaker 4: this is nuts. We shouldn't be doing it. It's the

947
00:49:57,124 --> 00:49:59,244
Speaker 4: obvious implication of what everyone else has been saying. So

948
00:49:59,444 --> 00:50:00,964
Speaker 4: you'll be I wish it was not stepping up.

949
00:50:00,964 --> 00:50:04,964
Speaker 2: But if you and Eliezer are at the Utopian retirement

950
00:50:05,004 --> 00:50:07,924
Speaker 2: home in one hundred and fifty years, because now human

951
00:50:07,964 --> 00:50:10,804
Speaker 2: beings lived to you know, one hundred ninety years old

952
00:50:10,884 --> 00:50:14,924
Speaker 2: on average or something, and your predictions are totally wrong

953
00:50:15,004 --> 00:50:17,444
Speaker 2: and AI is absolutely wonderful. Okay, good.

954
00:50:17,524 --> 00:50:19,964
Speaker 4: I would absolutely love to be wrong. I would I

955
00:50:20,004 --> 00:50:24,084
Speaker 4: would be so thrilled. Yeah, no spoilers about my book,

956
00:50:24,204 --> 00:50:27,484
Speaker 4: but it ends with us talking about how thrilled we

957
00:50:27,484 --> 00:50:27,884
Speaker 4: would be.

958
00:50:27,804 --> 00:50:28,204
Speaker 3: To be wrong.

959
00:50:28,244 --> 00:50:32,324
Speaker 2: Okay, I just sent you a calendar invite for October thirtieth,

960
00:50:32,364 --> 00:50:34,364
Speaker 2: two thousand and thirty five, where we can have a

961
00:50:34,364 --> 00:50:37,444
Speaker 2: follow up, right podcast, Nate Perfect any closing thoughts. Has

962
00:50:37,484 --> 00:50:38,444
Speaker 2: been a lot of fun to be with you.

963
00:50:40,564 --> 00:50:43,364
Speaker 4: Yeah, I mean, first and foremost, glad that you're you're

964
00:50:43,364 --> 00:50:46,644
Speaker 4: having the convo. Glad that we're having the convo. I

965
00:50:46,684 --> 00:50:49,444
Speaker 4: think just more people realizing how crazy the situation is

966
00:50:49,604 --> 00:50:50,284
Speaker 4: will go a long way.

967
00:50:50,324 --> 00:50:51,244
Speaker 3: Towards making things better.

968
00:50:51,524 --> 00:50:53,404
Speaker 1: Thank you so much for joining us, Nate, it's been

969
00:50:53,444 --> 00:50:55,604
Speaker 1: an absolute pleasure, pleasure to be here.

970
00:50:56,204 --> 00:50:57,124
Speaker 2: We'll talk to you soon.

971
00:51:04,084 --> 00:51:05,884
Speaker 1: Let us know what you think of the show. Reach

972
00:51:05,924 --> 00:51:09,244
Speaker 1: out to us at Risky Business at pushkin dot FM.

973
00:51:09,844 --> 00:51:12,924
Speaker 1: Risky Business is hosted by me Maria Kanakova.

974
00:51:12,644 --> 00:51:15,404
Speaker 2: And by me Nate Silver. The show was a cool

975
00:51:15,404 --> 00:51:19,364
Speaker 2: production of Pushing Industries and iHeartMedia. This episode was produced

976
00:51:19,404 --> 00:51:23,764
Speaker 2: by Isaac Carter. Our associate producer is Sonya gerwit Lydia,

977
00:51:23,844 --> 00:51:26,724
Speaker 2: Jean Kott and Daphne Chen are our editors, and our

978
00:51:26,764 --> 00:51:30,524
Speaker 2: executive producer is Jacob Goldstein. Mixing by Sarah Bruger.

979
00:51:31,164 --> 00:51:33,404
Speaker 1: If you like the show, please rate and review us

980
00:51:33,404 --> 00:51:36,044
Speaker 1: so other people can find us too, But once again,

981
00:51:36,204 --> 00:51:38,004
Speaker 1: only if you like us. We don't want those bad

982
00:51:38,044 --> 00:51:40,204
Speaker 1: reviews out there. Thanks for tuning in.