1
00:00:15,356 --> 00:00:15,796
Speaker 1: Pushkin.

2
00:00:20,236 --> 00:00:21,156
Speaker 2: Hey, it's Jacob.

3
00:00:21,236 --> 00:00:25,316
Speaker 1: There's another Pushkin show called Risky Business. It's a show

4
00:00:25,356 --> 00:00:29,276
Speaker 1: about decision making and strategic thinking. It's hosted by Nate

5
00:00:29,356 --> 00:00:32,196
Speaker 1: Silver and Riya Kanakova, and they just had a really

6
00:00:32,276 --> 00:00:36,356
Speaker 1: interesting episode about the future of AI. So we're going

7
00:00:36,436 --> 00:00:39,516
Speaker 1: to play that episode for you right now. We'll be

8
00:00:39,596 --> 00:00:42,156
Speaker 1: back later this week with the regular episode of What's

9
00:00:42,156 --> 00:00:48,836
Speaker 1: Your Problem.

10
00:00:48,956 --> 00:00:52,316
Speaker 3: Welcome back to Risky Business, a show about making better decisions.

11
00:00:52,476 --> 00:00:54,156
Speaker 3: I'm Maria Kanakova.

12
00:00:53,716 --> 00:00:54,516
Speaker 2: And I'm Nate Silver.

13
00:00:54,836 --> 00:00:56,556
Speaker 3: Hey. Today on the show is going to be a

14
00:00:56,596 --> 00:00:57,836
Speaker 3: little bit doomtastic.

15
00:00:58,556 --> 00:01:00,996
Speaker 2: Yeah, I mean, I don't know if it's like worse

16
00:01:01,156 --> 00:01:05,156
Speaker 2: and thinking about like the global economy going into a

17
00:01:05,196 --> 00:01:09,676
Speaker 2: recession because of dumb fuck tariff policies about how we're

18
00:01:09,676 --> 00:01:11,436
Speaker 2: all going to die in seven years and staid no,

19
00:01:11,516 --> 00:01:14,716
Speaker 2: I'm just kidding. This is a very very intelligent and

20
00:01:14,756 --> 00:01:18,196
Speaker 2: well written and thoughtful report called AI twenty twenty seven

21
00:01:18,276 --> 00:01:20,556
Speaker 2: that we're to spend the whole show on I because

22
00:01:20,556 --> 00:01:23,196
Speaker 2: think it's such an interesting to talk about, but that

23
00:01:23,316 --> 00:01:29,116
Speaker 2: you know, includes some dystopian possibilities. I would say it does.

24
00:01:28,956 --> 00:01:32,036
Speaker 3: Indeed, So let's get into it and hope that you

25
00:01:32,076 --> 00:01:34,196
Speaker 3: guys are all still here to listen to us in

26
00:01:34,276 --> 00:01:38,316
Speaker 3: seven years.

27
00:01:38,476 --> 00:01:43,516
Speaker 2: The contrast is interesting between like all the chaos we're

28
00:01:43,556 --> 00:01:47,356
Speaker 2: seeing with tariff policy in terms of starting a trade

29
00:01:47,356 --> 00:01:51,316
Speaker 2: war with China and then other types of chaos. It's

30
00:01:51,316 --> 00:01:53,476
Speaker 2: interesting to kind of look at this. I mean, I

31
00:01:53,476 --> 00:01:58,396
Speaker 2: wouldn't call it a more optimistic future exactly, like but

32
00:01:58,476 --> 00:02:00,476
Speaker 2: like on a different trajectory of like a future that's

33
00:02:00,476 --> 00:02:04,556
Speaker 2: going to change very fast, according to these authors, with

34
00:02:04,716 --> 00:02:11,036
Speaker 2: profound implications for you know, everything, the humans species. These

35
00:02:11,196 --> 00:02:14,036
Speaker 2: researchers and authors are saying that everything is going to

36
00:02:14,156 --> 00:02:16,076
Speaker 2: change profoundly.

37
00:02:15,756 --> 00:02:16,476
Speaker 3: And.

38
00:02:18,156 --> 00:02:22,116
Speaker 2: Even though there is some hedging hair, this is kind

39
00:02:22,156 --> 00:02:26,076
Speaker 2: of their base case scenario. And like you know, base

40
00:02:26,156 --> 00:02:28,836
Speaker 2: case number one and base case number two differ. There's

41
00:02:28,876 --> 00:02:30,476
Speaker 2: like kind of a choose your own adventure at some

42
00:02:30,556 --> 00:02:33,396
Speaker 2: point in this report, but they're both very different than

43
00:02:33,396 --> 00:02:36,236
Speaker 2: the status quo, right, and the notionately hear from that,

44
00:02:36,396 --> 00:02:42,196
Speaker 2: like everything becomes different If ais become substantially more intelligent

45
00:02:42,236 --> 00:02:44,196
Speaker 2: than human beings. People can debate and we will debate

46
00:02:44,236 --> 00:02:46,156
Speaker 2: on this program what that means. But yeah, do you

47
00:02:46,156 --> 00:02:48,036
Speaker 2: want to do you want to contextualize this for me?

48
00:02:48,116 --> 00:02:49,476
Speaker 2: Do you want to tell people too. Yeah, there's are

49
00:02:49,516 --> 00:02:50,396
Speaker 2: of this report.

50
00:02:50,396 --> 00:02:56,196
Speaker 3: Absolutely absolutely. So the report is authored officially by five people,

51
00:02:56,396 --> 00:02:59,516
Speaker 3: but I think unofficially there's also a sixth. We've got

52
00:03:00,476 --> 00:03:05,556
Speaker 3: Eli Leiflund, who's a super forecaster and he was ranked

53
00:03:05,636 --> 00:03:09,916
Speaker 3: first on Rand's forecasting initiative, so he is someone who

54
00:03:10,036 --> 00:03:13,116
Speaker 3: is very good at kind of looking at the future

55
00:03:13,156 --> 00:03:16,556
Speaker 3: trying to predict what's going to happen. You have Jonas Walmer,

56
00:03:16,636 --> 00:03:21,436
Speaker 3: who's a VC at Macroscopic Ventures. Thomas Larson was the

57
00:03:21,516 --> 00:03:26,476
Speaker 3: former executive director of the Center for AI Policy, so

58
00:03:26,596 --> 00:03:30,396
Speaker 3: that is a center that advises both sides of the

59
00:03:30,436 --> 00:03:34,756
Speaker 3: aisle on how AI is going to go. And Romeo Dean,

60
00:03:34,876 --> 00:03:39,676
Speaker 3: who is part of Harvard's AI Safety Student Team, so

61
00:03:39,796 --> 00:03:42,876
Speaker 3: someone who is still a student, still learning, but kind

62
00:03:42,876 --> 00:03:48,196
Speaker 3: of the next generation of people looking at AI. And

63
00:03:48,396 --> 00:03:53,276
Speaker 3: finally we have Daniel Cochtaylo, who basically had written a

64
00:03:53,356 --> 00:03:57,316
Speaker 3: report back in twenty twenty one. He was a AI

65
00:03:57,396 --> 00:04:01,596
Speaker 3: researcher and he looked at predictions for AI for twenty

66
00:04:01,636 --> 00:04:04,916
Speaker 3: twenty six and it turns out that his predictions were

67
00:04:04,956 --> 00:04:09,076
Speaker 3: pretty spot on, and so Open AI actually hired Daniel

68
00:04:09,316 --> 00:04:10,476
Speaker 3: as a result of this report.

69
00:04:10,476 --> 00:04:12,516
Speaker 2: Obviously he left, yes, and.

70
00:04:12,436 --> 00:04:14,596
Speaker 3: Then and then he left exactly.

71
00:04:14,236 --> 00:04:17,996
Speaker 2: And importantly, So there's also Scott Alexander exactly.

72
00:04:17,996 --> 00:04:19,916
Speaker 3: So I was saying, he's the he's the person who

73
00:04:20,356 --> 00:04:23,276
Speaker 3: kind of is in the background, and he's you guys

74
00:04:23,356 --> 00:04:26,556
Speaker 3: might know him as the author behind astral Codex.

75
00:04:27,396 --> 00:04:29,036
Speaker 2: And I know Scott. He's one of the kind of

76
00:04:29,076 --> 00:04:36,236
Speaker 2: fathers of what you might call rationalism. I think Scott,

77
00:04:36,236 --> 00:04:38,516
Speaker 2: when I interviewed him from a book, was happy enough

78
00:04:38,516 --> 00:04:40,916
Speaker 2: with that term and accused me or co opted me

79
00:04:40,956 --> 00:04:45,916
Speaker 2: into also being a rationalist. These people are somewhat adjacent

80
00:04:45,956 --> 00:04:48,516
Speaker 2: to the effective altruist, but not quite right. They're just

81
00:04:48,516 --> 00:04:53,596
Speaker 2: trying to apply a sort of thoughtful, rigorous, quantitative lens

82
00:04:53,796 --> 00:04:57,876
Speaker 2: to big picture problems, including existential risk, of which most

83
00:04:57,916 --> 00:05:00,756
Speaker 2: people in this community believe that AI is both an

84
00:05:00,756 --> 00:05:04,596
Speaker 2: existential risk and also kind of an existential opportunity, right,

85
00:05:04,596 --> 00:05:06,676
Speaker 2: that it could transform things. You talk to Sam Alp

86
00:05:06,676 --> 00:05:09,076
Speaker 2: and he'll say, we're going to cure cancer and elimina

87
00:05:09,196 --> 00:05:13,676
Speaker 2: poverty and whatever else. Right. And Scott's also an excellent writer.

88
00:05:13,916 --> 00:05:17,116
Speaker 2: And so let me disclose something which is slightly important here.

89
00:05:17,196 --> 00:05:20,116
Speaker 2: So I actually was approached by some of the authors

90
00:05:20,196 --> 00:05:22,996
Speaker 2: of this report a couple of months ago, I guess

91
00:05:22,996 --> 00:05:26,276
Speaker 2: it was in February ish, just to give feedback and

92
00:05:26,356 --> 00:05:29,796
Speaker 2: chat with them. So I'm working off the draft version right,

93
00:05:29,836 --> 00:05:32,356
Speaker 2: which I do not leave. They changed very much, so

94
00:05:32,396 --> 00:05:34,636
Speaker 2: my notes pertained to an earlier draft. I did not

95
00:05:34,676 --> 00:05:35,836
Speaker 2: have time this morning to go back and re.

96
00:05:35,836 --> 00:05:39,636
Speaker 3: Read it, so I was not on the inside loops.

97
00:05:39,716 --> 00:05:42,396
Speaker 3: So I did not get an earlier draft. And I've

98
00:05:42,436 --> 00:05:46,276
Speaker 3: read this draft and basically just to kind of big

99
00:05:46,316 --> 00:05:51,876
Speaker 3: picture sum it up, it outlines two scenarios, right, two

100
00:05:51,916 --> 00:05:56,276
Speaker 3: major scenarios for how AI might change the world as

101
00:05:56,556 --> 00:06:01,916
Speaker 3: soon as twenty thirty. Now important note like that that

102
00:06:02,196 --> 00:06:04,996
Speaker 3: date is could have hedged. It might be sooner, it

103
00:06:05,116 --> 00:06:08,476
Speaker 3: might be later. There's kind of a there's a confidence

104
00:06:08,516 --> 00:06:13,556
Speaker 3: interval there. But the two different scenarios. One basically we

105
00:06:13,716 --> 00:06:18,156
Speaker 3: do twenty thirty, humanity disappears and is taken over by AI.

106
00:06:18,716 --> 00:06:22,836
Speaker 3: The positive report is in twenty thirty basically we get

107
00:06:23,636 --> 00:06:26,716
Speaker 3: AIS that are aligned to our interests, and we get

108
00:06:26,796 --> 00:06:30,676
Speaker 3: kind of this AI utopia where AIS actually help make

109
00:06:30,756 --> 00:06:33,556
Speaker 3: life much better for everyone and make the standard of

110
00:06:33,556 --> 00:06:36,436
Speaker 3: living much higher. But the crucial turning point is before

111
00:06:36,516 --> 00:06:39,556
Speaker 3: twenty thirty. And the crucial kind of question at the

112
00:06:39,596 --> 00:06:44,716
Speaker 3: center of this is will we be able to design

113
00:06:45,076 --> 00:06:50,116
Speaker 3: AIS that are truly aligned to human interests rather than

114
00:06:50,196 --> 00:06:52,956
Speaker 3: just appear to be aligned and kind of lying to

115
00:06:53,036 --> 00:06:57,436
Speaker 3: us while actually following their own agenda. And how we

116
00:06:57,556 --> 00:07:01,396
Speaker 3: handle that is kind of the lynchpin. And it's actually interesting,

117
00:07:01,516 --> 00:07:03,876
Speaker 3: Nate that you started out with China, because a lot

118
00:07:03,916 --> 00:07:06,156
Speaker 3: of the policy choices and a lot of what they

119
00:07:06,236 --> 00:07:09,916
Speaker 3: see as kind of the decision points will affect the

120
00:07:09,956 --> 00:07:15,076
Speaker 3: future of humanity actually hinge on the US China dynamic,

121
00:07:15,476 --> 00:07:19,756
Speaker 3: how they compete with each other, and how that sometimes

122
00:07:19,876 --> 00:07:24,596
Speaker 3: might basically clash against safety concerns because no one wants

123
00:07:24,596 --> 00:07:27,876
Speaker 3: to be left behind. Can we manage that effectively and

124
00:07:27,956 --> 00:07:31,436
Speaker 3: can kind of that transition work in our favor as

125
00:07:31,436 --> 00:07:33,636
Speaker 3: opposed to against us. I think that this is kind

126
00:07:33,636 --> 00:07:35,356
Speaker 3: of one of the big questions here, And so it's

127
00:07:35,396 --> 00:07:37,556
Speaker 3: funny that we're seeing all of this trade war right

128
00:07:37,596 --> 00:07:40,156
Speaker 3: now as the sideport is coming out.

129
00:07:41,036 --> 00:07:46,396
Speaker 2: Yeah. Look, I think this exercise is partly just a

130
00:07:46,516 --> 00:07:49,676
Speaker 2: forecasting exercise, right, I mean, obviously it's just kind of

131
00:07:49,716 --> 00:07:53,836
Speaker 2: like fork at the bottom where we learn to have

132
00:07:53,916 --> 00:07:56,996
Speaker 2: an AI slow down or we kind of are pressing

133
00:07:57,036 --> 00:07:59,676
Speaker 2: fully on the accelerator, right, Like, in some ways, scenarios

134
00:07:59,676 --> 00:08:05,676
Speaker 2: are like not that different, right, either one assumes remarkable

135
00:08:05,756 --> 00:08:09,276
Speaker 2: rates of technological growth that I think even AI I'm

136
00:08:09,276 --> 00:08:11,956
Speaker 2: never quite sure who to call an optimist or a pessimist, right,

137
00:08:12,036 --> 00:08:15,916
Speaker 2: even AI believers you know, might think is a little

138
00:08:15,956 --> 00:08:17,796
Speaker 2: bit aggressive. Right, But what they want to do is

139
00:08:17,796 --> 00:08:22,476
Speaker 2: they want to have like a specific, fleshed out scenario

140
00:08:22,596 --> 00:08:24,436
Speaker 2: for how the world would look like. It's kind of

141
00:08:24,476 --> 00:08:28,236
Speaker 2: like a modal scenario. And like, I think they'd say that, like,

142
00:08:29,796 --> 00:08:32,156
Speaker 2: we're not totally sure about either of these necessarily, right,

143
00:08:32,156 --> 00:08:34,516
Speaker 2: And I don't think they'd be as like pedantic as

144
00:08:34,516 --> 00:08:36,756
Speaker 2: to say, if you do X, Y and Z, then

145
00:08:37,476 --> 00:08:39,636
Speaker 2: we'll save the world and have utopia, and if you don't,

146
00:08:39,716 --> 00:08:42,516
Speaker 2: then we'll all die, Right. I think they'd probably say

147
00:08:42,596 --> 00:08:45,516
Speaker 2: it's unclear and there's kind of like risk either way.

148
00:08:45,636 --> 00:08:47,636
Speaker 2: We wanted to go through the scenario of like fleshing

149
00:08:47,636 --> 00:08:51,076
Speaker 2: out like what the world might look like, right. I

150
00:08:51,116 --> 00:08:53,436
Speaker 2: do think one thing that's important is that whatever decisions

151
00:08:54,036 --> 00:08:58,156
Speaker 2: are made now could get locked in, right that you

152
00:08:58,436 --> 00:09:03,236
Speaker 2: pass certain points in overturn and it becomes very hard

153
00:09:03,276 --> 00:09:06,196
Speaker 2: to decelerate like an arms race. This is you know

154
00:09:06,236 --> 00:09:09,236
Speaker 2: what we found during the Cold War for example. I

155
00:09:09,236 --> 00:09:10,636
Speaker 2: mean one of the big things I look at is like,

156
00:09:11,236 --> 00:09:15,116
Speaker 2: do we force the AI to be transparent in its

157
00:09:15,156 --> 00:09:18,156
Speaker 2: thinking with humans? Right? Like, now there's been a movement

158
00:09:18,196 --> 00:09:22,116
Speaker 2: toward the ai'll actually explicate it's thinking more. I'll ask

159
00:09:22,116 --> 00:09:25,556
Speaker 2: it a query open AI. The Chinese models to this too, right,

160
00:09:25,556 --> 00:09:27,836
Speaker 2: and I'll say, I am thinking about X, Y and Z,

161
00:09:27,956 --> 00:09:30,116
Speaker 2: and I'm looking up PD and Q and now I'm

162
00:09:30,116 --> 00:09:33,276
Speaker 2: reconsidering this. It's actually has this chain of thought process, right,

163
00:09:33,676 --> 00:09:36,876
Speaker 2: which is explicated in English. You know. One concern is

164
00:09:36,876 --> 00:09:39,236
Speaker 2: that what if the AI just kind of communicates to

165
00:09:39,276 --> 00:09:44,756
Speaker 2: another in these implicit vectors that's inferring from all the

166
00:09:44,796 --> 00:09:48,236
Speaker 2: texts it has. It's kind of unintelligible to human beings, right,

167
00:09:48,276 --> 00:09:50,196
Speaker 2: and maybe kind of quote unquote thinking in that way

168
00:09:50,796 --> 00:09:52,676
Speaker 2: in the first place, and then does us the favor

169
00:09:52,756 --> 00:09:55,356
Speaker 2: of like translating back so it it goes from English

170
00:09:55,916 --> 00:09:58,676
Speaker 2: to kind of this big bag of numbers is one

171
00:09:58,716 --> 00:10:02,156
Speaker 2: a researcher called it, right, and then it translates it

172
00:10:02,236 --> 00:10:04,396
Speaker 2: back into English or whatever land which you want. Really,

173
00:10:04,876 --> 00:10:07,956
Speaker 2: in the end, what if it just skip cuts out

174
00:10:07,956 --> 00:10:10,276
Speaker 2: that last step, right, then we can't kind of like

175
00:10:11,036 --> 00:10:13,996
Speaker 2: check what AI is doing. Then it can behave deceptively

176
00:10:14,596 --> 00:10:17,596
Speaker 2: more easily. So you know, so that part seems to

177
00:10:17,596 --> 00:10:21,636
Speaker 2: be important. I want to hear your your first impressions

178
00:10:21,676 --> 00:10:23,756
Speaker 2: before I kind of poison the well too much.

179
00:10:24,916 --> 00:10:28,796
Speaker 3: Well, my first impressions is that the alignment problem is

180
00:10:28,836 --> 00:10:32,436
Speaker 3: a very real one and an incredibly important one to solve.

181
00:10:32,756 --> 00:10:35,276
Speaker 3: And what I got from this is that actually, the

182
00:10:35,356 --> 00:10:39,476
Speaker 3: problem that I've had with like these initial AI lms

183
00:10:39,996 --> 00:10:42,596
Speaker 3: is the kernel of what they're seeing there. Right, So

184
00:10:42,716 --> 00:10:44,476
Speaker 3: you and I have talked about this on the show

185
00:10:44,556 --> 00:10:47,596
Speaker 3: in the past, and I've said, well, my problem is

186
00:10:47,596 --> 00:10:50,516
Speaker 3: that when I'm a domain expert, right, I start seeing

187
00:10:50,596 --> 00:10:54,316
Speaker 3: some inaccuracies, and I start seeing like places where like

188
00:10:54,356 --> 00:10:57,876
Speaker 3: it either just didn't do well or made shit up

189
00:10:57,956 --> 00:11:00,956
Speaker 3: or whatever it is. Now, I think it's very clear

190
00:11:00,996 --> 00:11:03,516
Speaker 3: that those problems aren't going to go away, right, that

191
00:11:03,516 --> 00:11:07,036
Speaker 3: that is going to get much much better. However, the

192
00:11:07,196 --> 00:11:10,436
Speaker 3: kernel of it's showing me some thing, but that might

193
00:11:10,516 --> 00:11:13,076
Speaker 3: just be you know, I have no way of verifying

194
00:11:13,116 --> 00:11:15,996
Speaker 3: if that's what's going on, what it's reading, like, how

195
00:11:16,036 --> 00:11:18,396
Speaker 3: it's I don't want to say thinking about it, even

196
00:11:18,436 --> 00:11:23,276
Speaker 3: though in the report they do use thinking, but it's normal.

197
00:11:22,316 --> 00:11:24,636
Speaker 2: Vizing too much, I think is correct. I think it's.

198
00:11:24,756 --> 00:11:28,316
Speaker 3: Okay, we'll stick to that language. Yeah, okay, So, so

199
00:11:28,476 --> 00:11:33,276
Speaker 3: how it's thinking about it that those little problems and

200
00:11:33,316 --> 00:11:36,836
Speaker 3: like the little the glitches and the things that it

201
00:11:36,916 --> 00:11:40,036
Speaker 3: might be doing where it starts actually glitching on purpose

202
00:11:40,436 --> 00:11:43,516
Speaker 3: are not going to be visible to the human eye.

203
00:11:43,716 --> 00:11:45,516
Speaker 3: And so one of the main things that they say

204
00:11:45,556 --> 00:11:50,036
Speaker 3: here is that as AI internal R and D gets

205
00:11:50,316 --> 00:11:55,516
Speaker 3: rapidly faster, so that means basically AI's researching AIS, right,

206
00:11:55,796 --> 00:11:59,316
Speaker 3: and so internally they start developing new models, and as

207
00:11:59,356 --> 00:12:03,516
Speaker 3: they kind of surpass human ability to monitor it, it

208
00:12:03,596 --> 00:12:07,396
Speaker 3: becomes progressively more difficult to figure out, Okay, is the

209
00:12:07,436 --> 00:12:10,636
Speaker 3: AI actually doing what I want to do? Is the

210
00:12:10,676 --> 00:12:15,076
Speaker 3: output that it's giving me its actual thought process, and

211
00:12:15,356 --> 00:12:18,196
Speaker 3: is it accurate or is it like trying to deceive me?

212
00:12:18,276 --> 00:12:21,356
Speaker 3: But it's actually kind of inserting certain things on purpose

213
00:12:21,516 --> 00:12:24,036
Speaker 3: because it has different goals, right, because it is actually

214
00:12:24,076 --> 00:12:27,436
Speaker 3: secretly misaligned, but it's very good at persuading me that

215
00:12:27,476 --> 00:12:29,956
Speaker 3: it's aligned. Because one of the things that actually came

216
00:12:29,996 --> 00:12:31,716
Speaker 3: out of this report and I was like, huh, you

217
00:12:31,756 --> 00:12:37,356
Speaker 3: know this is interesting is if we get this remarkable

218
00:12:37,636 --> 00:12:42,556
Speaker 3: improvement in AI, it will also remarkably improve at persuading us, right,

219
00:12:42,636 --> 00:12:46,876
Speaker 3: so as part of making yeah, so this is this

220
00:12:46,916 --> 00:12:50,116
Speaker 3: is but I've never even thought about that. I was like, okay, fine,

221
00:12:50,276 --> 00:12:52,476
Speaker 3: But one of the things that I do buy is

222
00:12:52,516 --> 00:12:54,996
Speaker 3: that it's going to be very difficult for us to

223
00:12:55,036 --> 00:12:57,476
Speaker 3: monitor it and to figure out like is it truly

224
00:12:57,516 --> 00:13:01,276
Speaker 3: aligned with human wants, with human desires, with human goals,

225
00:13:01,636 --> 00:13:04,676
Speaker 3: And the experts who are capable of doing that, I

226
00:13:04,676 --> 00:13:07,796
Speaker 3: think are actually going to dwindle right as AI starts

227
00:13:07,796 --> 00:13:11,036
Speaker 3: proliferating in society. And so to me, that is something

228
00:13:11,036 --> 00:13:13,196
Speaker 3: that is actually quite worrisome, and that is something that

229
00:13:13,236 --> 00:13:15,916
Speaker 3: we really need to be paying attention to now. Just

230
00:13:15,956 --> 00:13:19,596
Speaker 3: to fast forward a little bit, in their doomsday scenario

231
00:13:20,236 --> 00:13:24,036
Speaker 3: in twenty thirty one, AI takes over it basically like

232
00:13:24,476 --> 00:13:28,476
Speaker 3: suddenly releases some chemical agents, right, and humanity dies and

233
00:13:28,516 --> 00:13:30,996
Speaker 3: the rest of the stragglers are taken care of by drones,

234
00:13:30,996 --> 00:13:34,476
Speaker 3: et cetera. I don't even like it's it doesn't even.

235
00:13:34,396 --> 00:13:35,516
Speaker 2: Quick and painless death.

236
00:13:35,596 --> 00:13:38,876
Speaker 3: I will say, let's hope we don't know what the

237
00:13:38,916 --> 00:13:42,196
Speaker 3: chemical agents are might not be quick and painless. Some

238
00:13:42,276 --> 00:13:45,316
Speaker 3: chemical agents are actually a very painful death thing. So

239
00:13:45,716 --> 00:13:49,116
Speaker 3: let's hope let's hope it's quick and painless. Quick, quick, yes,

240
00:13:49,116 --> 00:13:53,756
Speaker 3: stuff quick? Okay, hopefully some chemical agents are not quick.

241
00:13:54,756 --> 00:13:58,636
Speaker 3: Let's hope it's it's quick and painless. But if they're

242
00:13:58,676 --> 00:14:01,796
Speaker 3: actually capable of deception at that high level, then you

243
00:14:02,036 --> 00:14:05,196
Speaker 3: technically don't even need them to do it. If we're

244
00:14:05,196 --> 00:14:08,196
Speaker 3: trusting medicine and all sorts of things to the AIS,

245
00:14:08,236 --> 00:14:11,556
Speaker 3: it's pretty easy for it to actually manipulate something and

246
00:14:11,716 --> 00:14:16,836
Speaker 3: actually insert something into codes, et cetera that will fuck

247
00:14:16,916 --> 00:14:19,636
Speaker 3: up humanity in a way that we can't that we

248
00:14:19,676 --> 00:14:22,876
Speaker 3: can't actually figure out at the moment, right, Like the

249
00:14:22,916 --> 00:14:24,556
Speaker 3: way I think of it, And this is not from

250
00:14:24,716 --> 00:14:26,556
Speaker 3: the this is not from the paper, but this is

251
00:14:26,636 --> 00:14:29,276
Speaker 3: just the way that like my mind processed. It is

252
00:14:29,316 --> 00:14:33,876
Speaker 3: like think about DNA, right, Like you have these remarkably complex,

253
00:14:34,436 --> 00:14:38,796
Speaker 3: huge strands of data, and as we've found out, but

254
00:14:38,876 --> 00:14:43,516
Speaker 3: it's taken forever, one tiny mutation can actually be fatal, right,

255
00:14:43,556 --> 00:14:47,156
Speaker 3: but you can't spot that mutation. Sometimes that mutation isn't

256
00:14:47,196 --> 00:14:50,156
Speaker 3: fatal immediately but will only manifest at a certain point

257
00:14:50,196 --> 00:14:52,396
Speaker 3: in time. That's the way that my mind tried to

258
00:14:52,556 --> 00:14:56,836
Speaker 3: kind of try to conceptualize what this actually means. And

259
00:14:56,876 --> 00:14:59,396
Speaker 3: so I think that, you know, that would be easy

260
00:14:59,476 --> 00:15:03,556
Speaker 3: for a deceptive AI to do and to me like that,

261
00:15:03,556 --> 00:15:06,076
Speaker 3: that's kind of the big takeaway from this report is

262
00:15:06,116 --> 00:15:08,996
Speaker 3: that we need to make sure that we are building

263
00:15:09,316 --> 00:15:15,196
Speaker 3: AIS that will not deceive right that they're capabilities, they

264
00:15:15,556 --> 00:15:18,756
Speaker 3: explain them in an honest way, and that honesty and

265
00:15:18,796 --> 00:15:22,196
Speaker 3: trust is actually prioritized over other things, even though it

266
00:15:22,276 --> 00:15:24,916
Speaker 3: might slow down research, it might slow down other things,

267
00:15:24,956 --> 00:15:29,236
Speaker 3: but that that kind of alignment step is absolutely crucial

268
00:15:29,396 --> 00:15:32,956
Speaker 3: at the beginning, because otherwise humans are human, right, they're

269
00:15:32,956 --> 00:15:37,956
Speaker 3: easily manipulated, And we often trust that computers are quote

270
00:15:38,036 --> 00:15:41,116
Speaker 3: unquote rational because they're computers, but they're not. They have

271
00:15:41,196 --> 00:15:43,276
Speaker 3: their own inputs, they have their own weights, they have

272
00:15:43,316 --> 00:15:47,516
Speaker 3: their own values, and that could just lead us down

273
00:15:47,756 --> 00:15:48,476
Speaker 3: a dark path.

274
00:15:48,556 --> 00:15:51,876
Speaker 2: Yeah, so let me follow up with this pushback. I

275
00:15:51,876 --> 00:15:54,636
Speaker 2: guess right, Like, first of all, I don't know that

276
00:15:54,716 --> 00:15:58,556
Speaker 2: humans are so easily persuaded. This is my big critique

277
00:15:58,636 --> 00:16:03,636
Speaker 2: with like all the misinformation people who say, well, misinformation

278
00:16:03,716 --> 00:16:06,196
Speaker 2: is the biggest problem that society basis. It's like people

279
00:16:06,236 --> 00:16:10,476
Speaker 2: are actually pretty stubborn and they're kind of sound pretentious.

280
00:16:10,916 --> 00:16:13,676
Speaker 2: They're kind of basian and how they formulate their beliefs, right,

281
00:16:13,716 --> 00:16:15,796
Speaker 2: they have some notion of reality. They're looking at the

282
00:16:15,836 --> 00:16:20,356
Speaker 2: credibility of the person who is telling them these remarks.

283
00:16:20,676 --> 00:16:22,956
Speaker 2: If it's an unpersuasive source, it might make them less

284
00:16:22,996 --> 00:16:25,516
Speaker 2: likely to believe they're balancing what other information with their

285
00:16:25,556 --> 00:16:29,716
Speaker 2: lived experience so called Right. You know, part of the

286
00:16:29,756 --> 00:16:33,116
Speaker 2: reason that like I am skeptical of AIS being super

287
00:16:33,116 --> 00:16:36,316
Speaker 2: persuasive is like you know that it's an AI. You

288
00:16:36,316 --> 00:16:37,796
Speaker 2: know it's trying to persuade you. You know what I mean.

289
00:16:38,036 --> 00:16:40,156
Speaker 2: Like if you go and play poker against like a

290
00:16:40,196 --> 00:16:42,636
Speaker 2: really chatty player, if Phil Hemmi with or Scott Sieber

291
00:16:42,956 --> 00:16:45,396
Speaker 2: or someone like that, Right, you know, on some level

292
00:16:45,476 --> 00:16:49,636
Speaker 2: of the best play is just to totally ignore it. Right,

293
00:16:49,756 --> 00:16:51,716
Speaker 2: You know that they are trying to sweet talk you

294
00:16:51,916 --> 00:16:55,436
Speaker 2: into doing exactly what they want you to do, and

295
00:16:55,516 --> 00:16:59,196
Speaker 2: so the best play is to disengage or literally you

296
00:16:59,196 --> 00:17:01,036
Speaker 2: can randomize your movis videos no notion of what the

297
00:17:01,076 --> 00:17:06,236
Speaker 2: game theoretical optimal play might be, right, or salesmen or

298
00:17:06,396 --> 00:17:09,836
Speaker 2: politicians have reputations for being Oh he's a little too smooth.

299
00:17:09,876 --> 00:17:12,916
Speaker 2: Gavin newsome a little too fucking smooth. Right, I don't

300
00:17:12,956 --> 00:17:15,076
Speaker 2: find Gavin news some persuasive at all. Right, use a

301
00:17:15,076 --> 00:17:19,076
Speaker 2: little too from the hair gel to the constantly shifting vibes.

302
00:17:19,116 --> 00:17:20,956
Speaker 2: I mean, I don't really find Gavin news some persuasive

303
00:17:20,956 --> 00:17:22,596
Speaker 2: at all, even though like an AI, I'd say, boy,

304
00:17:22,596 --> 00:17:25,796
Speaker 2: Gavin new some good looking guys look gravelly throated. But

305
00:17:25,956 --> 00:17:28,276
Speaker 2: you know whatever, I mean. Look, the big critique I

306
00:17:28,276 --> 00:17:31,636
Speaker 2: have of this project, and by the way, I think

307
00:17:31,636 --> 00:17:35,396
Speaker 2: this is an amazing project in addition to like wonderful

308
00:17:35,436 --> 00:17:37,956
Speaker 2: writing if you viewed on the web not your phone.

309
00:17:38,036 --> 00:17:41,516
Speaker 2: So these very cool, like little infographic that updates everything

310
00:17:41,556 --> 00:17:43,516
Speaker 2: from like the market value. If they don't call it

311
00:17:43,516 --> 00:17:45,596
Speaker 2: open aye, they call it open brain. I guess is

312
00:17:45,596 --> 00:17:46,796
Speaker 2: what they settle on for a substitute.

313
00:17:47,036 --> 00:17:50,836
Speaker 3: Yeah, they call everything something else, just to make sure

314
00:17:50,876 --> 00:17:53,476
Speaker 3: that they're not stepping on any toes. So they have

315
00:17:53,636 --> 00:17:57,476
Speaker 3: open Brain, and then they have Deep Scent from China.

316
00:17:57,756 --> 00:18:02,036
Speaker 2: I wonder which one that could be. I wonder. But

317
00:18:02,116 --> 00:18:06,196
Speaker 2: it's beautifully presented and written, and like I appreciate they're

318
00:18:06,196 --> 00:18:09,436
Speaker 2: going out on a limb here, you know, I mean,

319
00:18:09,476 --> 00:18:11,676
Speaker 2: I think they have. It's been fairly well received. They've

320
00:18:11,676 --> 00:18:15,156
Speaker 2: gotten some pushback both from inside and I think outside

321
00:18:15,236 --> 00:18:18,276
Speaker 2: the AI safety community. Right, but they're putting their necks

322
00:18:18,316 --> 00:18:21,076
Speaker 2: in the line, hear. They will look if things look

323
00:18:21,076 --> 00:18:23,716
Speaker 2: pretty normal, if looks pretty normal in twenty thirty two

324
00:18:23,956 --> 00:18:27,676
Speaker 2: or whatever, right, then they will look dumb for having

325
00:18:27,676 --> 00:18:28,876
Speaker 2: published this.

326
00:18:29,196 --> 00:18:32,676
Speaker 3: Well, and they actually have that right as a scenario

327
00:18:32,796 --> 00:18:35,516
Speaker 3: that you end up looking stupid if everything goes well.

328
00:18:35,796 --> 00:18:38,516
Speaker 3: But that's okay. Now, can I push back on the

329
00:18:38,556 --> 00:18:42,676
Speaker 3: persuasion thing a little bit, just on two things. So

330
00:18:42,836 --> 00:18:45,676
Speaker 3: first of all, the poker example is not actually a

331
00:18:45,676 --> 00:18:49,956
Speaker 3: particularly applicable one here, because you know that you're playing

332
00:18:50,036 --> 00:18:52,196
Speaker 3: poker and you know that someone is trying to get

333
00:18:52,196 --> 00:18:55,796
Speaker 3: information and deceive you the tricky thing. So this is

334
00:18:55,876 --> 00:18:57,996
Speaker 3: kind of when I spend time with con artists. The

335
00:18:57,996 --> 00:19:01,356
Speaker 3: best con artists aren't Gavin Newsome, Like, they're not car salesmen.

336
00:19:01,676 --> 00:19:04,036
Speaker 3: You have no idea they're trying to persuade you to

337
00:19:04,076 --> 00:19:07,316
Speaker 3: do something. They are just like nice, affable people who

338
00:19:07,396 --> 00:19:10,756
Speaker 3: are incredibly charismatic, and even in the poker community, by

339
00:19:10,756 --> 00:19:12,796
Speaker 3: the way, like some of the biggest grifters who like

340
00:19:12,836 --> 00:19:16,276
Speaker 3: it comes out later on, we're just like stealing money

341
00:19:16,276 --> 00:19:19,076
Speaker 3: and doing all of these things. Are charming, right, They're

342
00:19:19,116 --> 00:19:22,236
Speaker 3: not sleazy looking like they have no signs of, oh,

343
00:19:22,276 --> 00:19:24,516
Speaker 3: I'm a salesman. I'm trying to sell you something. The

344
00:19:24,556 --> 00:19:28,116
Speaker 3: people who are actually good at persuasion you do not

345
00:19:28,316 --> 00:19:31,316
Speaker 3: realize you are being persuaded. And I think people are

346
00:19:31,356 --> 00:19:35,556
Speaker 3: incredibly easy to kind of to subtly lead in a

347
00:19:35,596 --> 00:19:38,756
Speaker 3: certain direction if you know how to do it, and

348
00:19:38,836 --> 00:19:42,236
Speaker 3: I think ais could do that, and they might persuade

349
00:19:42,236 --> 00:19:44,116
Speaker 3: you when you don't even think they're trying to persuade you.

350
00:19:44,116 --> 00:19:46,436
Speaker 3: You might just ask like, can you please summarize this

351
00:19:46,516 --> 00:19:49,196
Speaker 3: research report and the way that it frames it right,

352
00:19:49,236 --> 00:19:53,116
Speaker 3: the way that it summarizes it just subtly changes the

353
00:19:53,156 --> 00:19:55,516
Speaker 3: way that you think of this issue. We see that

354
00:19:55,556 --> 00:19:57,596
Speaker 3: in psych studies all the time, by the way, where

355
00:19:57,636 --> 00:20:00,956
Speaker 3: you have different articles presented in slightly different orders, slightly

356
00:20:00,956 --> 00:20:04,316
Speaker 3: different ways, and people from the same political beliefs, you know,

357
00:20:04,356 --> 00:20:09,436
Speaker 3: same starting point, come away with different impressions of what

358
00:20:09,556 --> 00:20:11,716
Speaker 3: kind of the right course of action is or what

359
00:20:11,756 --> 00:20:13,956
Speaker 3: this is actually trying to tell you. Because the way

360
00:20:13,996 --> 00:20:18,236
Speaker 3: the information is presented actually influences how you think about it.

361
00:20:18,236 --> 00:20:22,156
Speaker 3: It's very very easy to do subtle manipulations like that.

362
00:20:22,516 --> 00:20:25,236
Speaker 3: And if we're relying on AI on a large scale

363
00:20:25,276 --> 00:20:27,676
Speaker 3: for a lot of our lives, I think that if

364
00:20:27,676 --> 00:20:30,396
Speaker 3: it has like a quote unquote master plan, you know,

365
00:20:30,476 --> 00:20:33,236
Speaker 3: the way that they present in this report, then persuasion

366
00:20:33,276 --> 00:20:34,756
Speaker 3: in that sense is actually going to be.

367
00:20:35,356 --> 00:20:37,796
Speaker 2: You'll know you're being manipulated, right, that's the issue.

368
00:20:37,876 --> 00:20:39,076
Speaker 3: No, you don't know. That's the thing.

369
00:20:39,196 --> 00:20:42,036
Speaker 2: People will be manipulated because it's AI and that I

370
00:20:42,076 --> 00:20:43,836
Speaker 2: don't but I don't know.

371
00:20:43,956 --> 00:20:47,796
Speaker 3: I honestly, like Nate, I applaud your belief in human's

372
00:20:47,796 --> 00:20:50,916
Speaker 3: ability to adjust to this, but I don't know that

373
00:20:50,956 --> 00:20:53,916
Speaker 3: they will because I've just seen enough people who are

374
00:20:53,916 --> 00:21:00,116
Speaker 3: incredibly intelligent fall for cons and then be very unpersuadable

375
00:21:00,276 --> 00:21:02,996
Speaker 3: that they have been conned, right, instead doubling down and

376
00:21:03,036 --> 00:21:06,196
Speaker 3: saying no, I have not. So humans are stubborn, but

377
00:21:06,196 --> 00:21:08,676
Speaker 3: they're also stubborn and saying I have not been deceived,

378
00:21:08,676 --> 00:21:10,996
Speaker 3: I have not been manipulated, when in fact, they have

379
00:21:11,436 --> 00:21:14,036
Speaker 3: to protect their ego and to protect their view of

380
00:21:14,076 --> 00:21:17,076
Speaker 3: themselves as people who are not capable of being manipulated

381
00:21:17,196 --> 00:21:20,156
Speaker 3: or deceived, and I think that that is incredibly powerful,

382
00:21:20,476 --> 00:21:23,596
Speaker 3: and I think that that's going to push against your optimism.

383
00:21:23,636 --> 00:21:25,836
Speaker 3: I hope you're right, but from what I know, I

384
00:21:25,836 --> 00:21:26,516
Speaker 3: don't think you are.

385
00:21:27,436 --> 00:21:29,436
Speaker 2: I'm not quite sure you call it optimism some I

386
00:21:29,436 --> 00:21:31,476
Speaker 2: guess maybe we do like different views of human nature.

387
00:21:31,516 --> 00:21:33,876
Speaker 2: But like there's not yet a substantial market for like

388
00:21:33,996 --> 00:21:38,836
Speaker 2: AI driven art we're writing, and I'm sure there will

389
00:21:38,876 --> 00:21:44,236
Speaker 2: be one eventually, right, But like people understand that context matters, right,

390
00:21:44,236 --> 00:21:45,756
Speaker 2: that you could heavy I create a rip off the

391
00:21:45,796 --> 00:21:47,316
Speaker 2: Mona Lisa, but you can also buy rip off the

392
00:21:47,356 --> 00:21:49,436
Speaker 2: Mona Lisa and Canal Street for five bucks, right, and

393
00:21:49,516 --> 00:21:53,316
Speaker 2: like it. You know, So it's it's the intentionality of

394
00:21:53,916 --> 00:21:57,556
Speaker 2: the act and the context of the speaker. Now, sound

395
00:21:57,636 --> 00:22:01,836
Speaker 2: like super woke, I guess right where you're coming from,

396
00:22:01,836 --> 00:22:05,556
Speaker 2: A I think that actually is how humans communicate. Like

397
00:22:06,076 --> 00:22:08,596
Speaker 2: art that might be pointless drivel coming from somebody can

398
00:22:08,596 --> 00:22:10,876
Speaker 2: be something to coming from a Jackson Ballock or whatever.

399
00:22:10,916 --> 00:22:11,076
Speaker 1: You know.

400
00:22:11,156 --> 00:22:13,436
Speaker 3: Absolutely, I think that that's a really important point. By

401
00:22:13,476 --> 00:22:14,956
Speaker 3: the way, I think it's a different point, but I

402
00:22:14,956 --> 00:22:16,516
Speaker 3: think that that is a very important point. I think

403
00:22:16,516 --> 00:22:17,436
Speaker 3: context does matter.

404
00:22:22,556 --> 00:22:37,476
Speaker 2: We'll be right back after this message. I was buttering

405
00:22:37,556 --> 00:22:41,396
Speaker 2: up this report before. My big critique of it is like,

406
00:22:41,876 --> 00:22:44,956
Speaker 2: where are the human beings in this? Or put another way,

407
00:22:44,996 --> 00:22:49,356
Speaker 2: kind of like where is the politics? Right? They're trying

408
00:22:49,436 --> 00:22:55,556
Speaker 2: not to use any remotely controversial real names, right, so

409
00:22:55,636 --> 00:22:58,516
Speaker 2: you have open brain for example, So where's where is

410
00:22:59,276 --> 00:23:01,516
Speaker 2: President Trump? Let me steal a quick search to make

411
00:23:01,556 --> 00:23:03,636
Speaker 2: sure their named Trump does not appear to know the name.

412
00:23:03,756 --> 00:23:06,556
Speaker 3: So they do. Actually, I don't know if this existed.

413
00:23:07,036 --> 00:23:10,596
Speaker 3: Maybe they took your criticism in this, but they do

414
00:23:10,676 --> 00:23:13,996
Speaker 3: have like the vice president and the president like they

415
00:23:13,996 --> 00:23:16,316
Speaker 3: do put politicians in this version of the report. They

416
00:23:16,316 --> 00:23:18,636
Speaker 3: don't have names, but they say the vice president in

417
00:23:18,676 --> 00:23:21,236
Speaker 3: one of these scenarios, you know, handily wins the election

418
00:23:21,476 --> 00:23:23,396
Speaker 3: of twenty twenty eight. We have one vice president.

419
00:23:23,556 --> 00:23:25,716
Speaker 2: They have general secretary. I think I'm not sure if

420
00:23:25,716 --> 00:23:31,276
Speaker 2: they I mean general secretary. General secretary resembles she yep,

421
00:23:31,916 --> 00:23:36,396
Speaker 2: and the vice president kind of resembles jd Vance, Right,

422
00:23:36,996 --> 00:23:39,956
Speaker 2: I don't think the president resembles Trump at all.

423
00:23:40,116 --> 00:23:42,956
Speaker 3: Right, it's kind of the no, they didn't character like, yeah,

424
00:23:42,956 --> 00:23:46,036
Speaker 3: they tried to side stuff that if.

425
00:23:45,876 --> 00:23:48,436
Speaker 2: We think it's all happening in the next four years then,

426
00:23:49,956 --> 00:23:53,356
Speaker 2: you know, presidential politics matter quite a bit. I mean,

427
00:23:53,396 --> 00:23:55,876
Speaker 2: I know, I h this is such a fucking you know,

428
00:23:55,916 --> 00:23:59,196
Speaker 2: I was jogging earlier on the East Side and I

429
00:23:59,276 --> 00:24:02,876
Speaker 2: was listening to the Sraclient interview with Thomas Friedman. It's

430
00:24:02,876 --> 00:24:06,756
Speaker 2: such a fucking yuppie fucking thing, right, It's.

431
00:24:06,596 --> 00:24:08,556
Speaker 3: Okay, It's okay, You're allowed to be a.

432
00:24:08,476 --> 00:24:12,356
Speaker 2: YOUNGESTI even not a huge fan of necessarily, but like,

433
00:24:12,436 --> 00:24:14,716
Speaker 2: you know, as well versed some like geopolitics and like

434
00:24:14,796 --> 00:24:17,236
Speaker 2: China issues, is like, yeah, China, You've just been back

435
00:24:17,236 --> 00:24:20,196
Speaker 2: from Chinese. Like, yeah, China's kind of winning, you know

436
00:24:20,236 --> 00:24:23,676
Speaker 2: what I mean. And like, I'm not sure how how

437
00:24:23,716 --> 00:24:29,556
Speaker 2: Trump's hawkishness on China, but like kind of imbecilically executed

438
00:24:30,876 --> 00:24:33,996
Speaker 2: hawkishness on China, Like I'm not sure how that figure's

439
00:24:33,996 --> 00:24:37,556
Speaker 2: in to this. Right, If we're reducing US China trade,

440
00:24:38,636 --> 00:24:42,996
Speaker 2: that probably does produce an AI slow down, maybe more

441
00:24:42,996 --> 00:24:44,916
Speaker 2: for US if we're like if they're not exporting their

442
00:24:44,916 --> 00:24:48,636
Speaker 2: like raw earth materials and and so forth, but we're

443
00:24:48,636 --> 00:24:50,236
Speaker 2: making it hard for them to get in video chips,

444
00:24:50,236 --> 00:24:52,796
Speaker 2: so they probably have like lots of workarounds and things

445
00:24:52,836 --> 00:24:55,836
Speaker 2: like that. Maybe maybe Trump tariffs are good. I would

446
00:24:55,876 --> 00:24:57,596
Speaker 2: like to ask the authors this report, because it means

447
00:24:57,596 --> 00:25:00,276
Speaker 2: that we're gonna like have slower AI progress. And I'm

448
00:25:00,276 --> 00:25:05,116
Speaker 2: not joking, right, that increases the hostility between the US

449
00:25:05,596 --> 00:25:07,836
Speaker 2: and China in the long run, right, I mean that's

450
00:25:08,036 --> 00:25:09,636
Speaker 2: if we were send it all the terrafs. I think

451
00:25:09,636 --> 00:25:12,516
Speaker 2: we still permanently or let's not say permanent, let's say

452
00:25:12,516 --> 00:25:16,396
Speaker 2: at least for a decade or so, have injured us

453
00:25:16,436 --> 00:25:19,876
Speaker 2: standing in the world. And so I don't know how

454
00:25:19,876 --> 00:25:22,036
Speaker 2: that figures in. And I'm like, I'm also not sure

455
00:25:22,116 --> 00:25:26,516
Speaker 2: like kind of quote what the rational response might be.

456
00:25:26,556 --> 00:25:28,156
Speaker 2: But one thing they tried to let me make sure

457
00:25:28,276 --> 00:25:31,636
Speaker 2: that they kept us into their report, right, So they

458
00:25:31,676 --> 00:25:40,036
Speaker 2: actually have their implied approval rating for how people feel

459
00:25:40,076 --> 00:25:45,156
Speaker 2: about open brain, which is there not very self open AI.

460
00:25:45,556 --> 00:25:48,196
Speaker 2: I think this actually is some feedback who that they

461
00:25:48,196 --> 00:25:51,356
Speaker 2: took into account. Right. They originally had it slightly less negative,

462
00:25:51,356 --> 00:25:54,476
Speaker 2: but they have this being persistently negative and then getting

463
00:25:54,516 --> 00:25:56,876
Speaker 2: more negative over time. It was a little softer in

464
00:25:56,916 --> 00:26:00,156
Speaker 2: the in the previous version that that I saw, so

465
00:26:00,196 --> 00:26:03,356
Speaker 2: they did change that one thing on at some stage.

466
00:26:03,836 --> 00:26:07,196
Speaker 2: But like the fact that like AI scares people, it

467
00:26:07,276 --> 00:26:12,396
Speaker 2: scares people for both good and bad reasons, but I

468
00:26:12,396 --> 00:26:16,036
Speaker 2: think mostly for valid reasons, right, that the fear is

469
00:26:16,076 --> 00:26:19,916
Speaker 2: fairly bipartisan. That the biggest AI accelerators are now these

470
00:26:19,996 --> 00:26:24,196
Speaker 2: kind of Republican techno optimists who are not looking particularly

471
00:26:24,636 --> 00:26:28,156
Speaker 2: wise given how it's going with the first ninety days

472
00:26:28,196 --> 00:26:32,396
Speaker 2: whatever we are, the Trump administration, and the likelihood of

473
00:26:32,476 --> 00:26:37,316
Speaker 2: like a substantial political backlash, right, which could lead to

474
00:26:37,436 --> 00:26:39,356
Speaker 2: dumb types of regulations. But like, you know, part of

475
00:26:39,396 --> 00:26:43,236
Speaker 2: it too is like okay, AI they're saying can do

476
00:26:43,596 --> 00:26:47,356
Speaker 2: not just computer disk jobs, but like all types of things, right,

477
00:26:47,436 --> 00:26:51,676
Speaker 2: And like humans kind of play this role initially as supervisors,

478
00:26:51,716 --> 00:26:54,596
Speaker 2: and then literally within a couple of years people start

479
00:26:54,596 --> 00:26:57,916
Speaker 2: to say, you know what, am I really adding much

480
00:26:57,996 --> 00:27:02,236
Speaker 2: value here? Right? You kind of have like these legacy jobs,

481
00:27:02,236 --> 00:27:03,956
Speaker 2: and there is a lot of money. I think most

482
00:27:03,996 --> 00:27:11,196
Speaker 2: these scenarios imagine very fast economic grow although maybe very lumpy,

483
00:27:11,316 --> 00:27:14,116
Speaker 2: right for some parts of the world and not others.

484
00:27:14,916 --> 00:27:17,316
Speaker 2: We're kind of just sitting around with a lot of

485
00:27:17,316 --> 00:27:20,396
Speaker 2: idle time. It might be good for live poker, Maria. Right,

486
00:27:20,796 --> 00:27:24,036
Speaker 2: all of a sudden, all these smart people, their open earth,

487
00:27:24,036 --> 00:27:26,716
Speaker 2: their open brain, excuse me, stock is now worth billions

488
00:27:26,756 --> 00:27:29,556
Speaker 2: of dollars, right, and like nothing to do because the

489
00:27:29,596 --> 00:27:31,356
Speaker 2: AI is doing all their work, right, they have a

490
00:27:31,356 --> 00:27:33,156
Speaker 2: lot of fucking time to play some fucking texas hold

491
00:27:33,156 --> 00:27:33,596
Speaker 2: them right.

492
00:27:36,716 --> 00:27:39,356
Speaker 3: That is, that is the one way of thinking about it.

493
00:27:40,796 --> 00:27:44,716
Speaker 3: Let's go back to your earlier point, which I actually

494
00:27:44,756 --> 00:27:48,356
Speaker 3: think is an important one, because obviously they were trying

495
00:27:48,396 --> 00:27:51,956
Speaker 3: to do as all you know, super forecasting tries to

496
00:27:51,956 --> 00:27:55,876
Speaker 3: do is you try to create a report that will

497
00:27:55,916 --> 00:27:57,276
Speaker 3: work in multiple scenarios.

498
00:27:57,356 --> 00:27:57,516
Speaker 2: Right.

499
00:27:57,756 --> 00:28:00,556
Speaker 3: You can't tie it too much to like the present moment,

500
00:28:00,596 --> 00:28:04,436
Speaker 3: otherwise your forecasts are going to be quite biased. However,

501
00:28:05,316 --> 00:28:07,916
Speaker 3: I do think that what you raise kind of our

502
00:28:07,956 --> 00:28:12,396
Speaker 3: current situation with et cetera, has very real implications. Given

503
00:28:12,556 --> 00:28:14,756
Speaker 3: that this is kind of the central dynamic of this

504
00:28:14,836 --> 00:28:17,716
Speaker 3: report that their predictions are based on, I think that

505
00:28:17,716 --> 00:28:21,836
Speaker 3: it's incredibly valid to actually speculate, you know, how will

506
00:28:22,196 --> 00:28:25,596
Speaker 3: if at all, this effect the timeline of the predictions,

507
00:28:25,796 --> 00:28:28,716
Speaker 3: the possible the likelihood of the two scenarios. And I

508
00:28:28,756 --> 00:28:30,676
Speaker 3: will also say that one of the things in the

509
00:28:30,716 --> 00:28:33,436
Speaker 3: report is that all of these negotiations on like will

510
00:28:33,476 --> 00:28:36,036
Speaker 3: we slow down, will we not? How aligned is it?

511
00:28:36,116 --> 00:28:38,276
Speaker 3: This all takes place in secret, right, Like we don't

512
00:28:38,356 --> 00:28:41,836
Speaker 3: know the humans don't know that it's going on. We

513
00:28:41,916 --> 00:28:45,076
Speaker 3: don't know what's happening behind the scenes, and we don't

514
00:28:45,076 --> 00:28:47,676
Speaker 3: know what the decision makers are kind of thinking. And

515
00:28:48,116 --> 00:28:50,676
Speaker 3: so for all we know, you know, President Trump is

516
00:28:50,716 --> 00:28:55,196
Speaker 3: meeting with Sam Altman and trying to trying to kind

517
00:28:55,196 --> 00:28:58,316
Speaker 3: of do some of these things. And it's funny because

518
00:28:58,356 --> 00:29:01,196
Speaker 3: we were kind of pushing for transparency in one way,

519
00:29:01,196 --> 00:29:03,196
Speaker 3: but there's a lot of things here that are very

520
00:29:03,276 --> 00:29:04,236
Speaker 3: much not transparent.

521
00:29:04,356 --> 00:29:07,196
Speaker 2: Yeah, it's kind of the deep state, right, but also

522
00:29:08,396 --> 00:29:12,716
Speaker 2: also a lot than negotiations are now AI versus AI, right,

523
00:29:13,316 --> 00:29:16,476
Speaker 2: And look, I'm not sure that AIS will have that

524
00:29:16,516 --> 00:29:20,156
Speaker 2: trust both with the external actor and internally. I'm skeptical

525
00:29:20,196 --> 00:29:22,956
Speaker 2: of that. Right. If that does happen, they kind of

526
00:29:22,956 --> 00:29:25,236
Speaker 2: think this might be good because the AIS will probably

527
00:29:25,316 --> 00:29:30,556
Speaker 2: behave in like it literally game theory optimal way, right

528
00:29:30,716 --> 00:29:34,556
Speaker 2: and understand these things and make I guess, like fewer

529
00:29:34,596 --> 00:29:38,076
Speaker 2: mistakes than humans might say, if they're properly aligned.

530
00:29:38,156 --> 00:29:40,916
Speaker 3: Like that's a crucial thing because in the doomsday scenario,

531
00:29:40,996 --> 00:29:45,636
Speaker 3: AI negotiates with AI, but they conspire to destroy humanity. Right,

532
00:29:45,676 --> 00:29:48,676
Speaker 3: So there are two scenarios. One, it's actually properly aligned,

533
00:29:48,676 --> 00:29:53,716
Speaker 3: so AI negotiates with AI. Game theory works out, and

534
00:29:54,276 --> 00:29:57,676
Speaker 3: we end up with you know, democracy and wonderful things.

535
00:29:57,956 --> 00:30:01,116
Speaker 3: But in the other one, where they're misaligned, AI negotiates

536
00:30:01,116 --> 00:30:06,196
Speaker 3: with AI to create a new AI basically and destroy humanity.

537
00:30:06,476 --> 00:30:08,756
Speaker 3: So it can go one way or the other depending

538
00:30:08,796 --> 00:30:09,916
Speaker 3: on that alignment step.

539
00:30:09,996 --> 00:30:12,596
Speaker 2: First of all, I mean, the utopian didn't seem that

540
00:30:12,676 --> 00:30:14,916
Speaker 2: utopian to me, right, I'm not sure that no, it

541
00:30:14,956 --> 00:30:15,436
Speaker 2: did it.

542
00:30:15,436 --> 00:30:18,156
Speaker 3: It actually seemed quite distolling to me, Like it seems

543
00:30:18,196 --> 00:30:20,556
Speaker 3: incredibly disturbed. It's kind of like, you know, I look,

544
00:30:20,596 --> 00:30:23,916
Speaker 3: but at least we're still alive, we'll have chures.

545
00:30:23,636 --> 00:30:26,556
Speaker 2: Will probably live longer. And again, lots of lots of

546
00:30:26,636 --> 00:30:28,996
Speaker 2: lots of poker the ail you're writing Silver Bulletin and

547
00:30:29,196 --> 00:30:33,276
Speaker 2: and hosting our podcast. Right, let me let me back

548
00:30:33,356 --> 00:30:34,836
Speaker 2: up a little bit, because I think we maybe take

549
00:30:34,916 --> 00:30:38,076
Speaker 2: for granted that some of these premises are are kind

550
00:30:38,076 --> 00:30:42,796
Speaker 2: of controversial, right, so they have a breakpoint. I think

551
00:30:42,876 --> 00:30:47,516
Speaker 2: in twenty twenty six, well says Art, why aren't certainly

552
00:30:47,516 --> 00:30:49,596
Speaker 2: increased potentially be on twenty twenty six, right, So that's

553
00:30:49,636 --> 00:30:51,596
Speaker 2: kind of the breakpoint. Is like twenty seven is this

554
00:30:51,676 --> 00:30:54,236
Speaker 2: inflection point? I think I'm using that term correctly in

555
00:30:54,236 --> 00:30:57,436
Speaker 2: this context. You know. So I'm reading this report and

556
00:30:58,956 --> 00:31:01,156
Speaker 2: up to twenty twenty six and like, thumbs up, yia,

557
00:31:01,196 --> 00:31:05,276
Speaker 2: this seems like very smart and detailed about, like, you know,

558
00:31:05,316 --> 00:31:07,756
Speaker 2: how the economy is reacting and how politics are reacting

559
00:31:07,756 --> 00:31:10,236
Speaker 2: in the in the race dynamic with China. Maybe there

560
00:31:10,236 --> 00:31:11,436
Speaker 2: needs to be a little bit more Trump in there.

561
00:31:11,476 --> 00:31:13,356
Speaker 2: I understand why politically they didn't want to get into

562
00:31:13,396 --> 00:31:18,156
Speaker 2: that mess, right, But like, so there's kind of three

563
00:31:18,156 --> 00:31:20,836
Speaker 2: different things here, right. One is a notion of what's

564
00:31:20,876 --> 00:31:25,676
Speaker 2: sometimes called AGI, or artificial general intelligence. And if you

565
00:31:25,756 --> 00:31:28,436
Speaker 2: ask a hundred different researcher you get a hundred different

566
00:31:28,436 --> 00:31:30,756
Speaker 2: definitions of what AGI is. But you know, I think

567
00:31:30,756 --> 00:31:33,076
Speaker 2: it is based like being able to do a large

568
00:31:33,196 --> 00:31:37,676
Speaker 2: majority of things that a human being could do competently,

569
00:31:38,396 --> 00:31:41,636
Speaker 2: something we're limiting it to kind of like desk job

570
00:31:42,036 --> 00:31:45,036
Speaker 2: type tasks. Right, anything that can be done remotely is

571
00:31:45,036 --> 00:31:48,476
Speaker 2: sometimes definition that is used or through remote work, right,

572
00:31:48,516 --> 00:31:52,596
Speaker 2: because clearly AIS are inferior to humans and like sorting

573
00:31:52,596 --> 00:31:55,596
Speaker 2: and folding laundryth or things like that, right that require

574
00:31:55,636 --> 00:31:57,676
Speaker 2: certain type of intelligence. Right, if you use the kind

575
00:31:57,716 --> 00:32:01,876
Speaker 2: of desk job definition, then like AI is already pretty

576
00:32:01,916 --> 00:32:05,996
Speaker 2: close to AGI. Right. I use large language models all

577
00:32:06,036 --> 00:32:08,676
Speaker 2: the freaking time, and they're not perfect for everything. I

578
00:32:08,676 --> 00:32:10,836
Speaker 2: felt like, you know, in terms of like being able

579
00:32:10,876 --> 00:32:15,796
Speaker 2: to do the large majority of desk work at levels

580
00:32:15,876 --> 00:32:20,316
Speaker 2: ranging from competent intern to super genius, Like on average,

581
00:32:20,516 --> 00:32:24,596
Speaker 2: it's probably pretty close to being generally intelligent by that definition, right,

582
00:32:24,836 --> 00:32:25,396
Speaker 2: if you're the.

583
00:32:25,356 --> 00:32:27,636
Speaker 3: One using it. I just want to like once again

584
00:32:27,676 --> 00:32:29,476
Speaker 3: point that out because one of the things that they

585
00:32:29,476 --> 00:32:31,556
Speaker 3: say in the report is that as it gets more

586
00:32:31,636 --> 00:32:34,716
Speaker 3: and more involved what we're asking AI to do, it's

587
00:32:34,716 --> 00:32:38,076
Speaker 3: like the human process to evaluate whether it's accurate and

588
00:32:38,116 --> 00:32:40,796
Speaker 3: whether it's making mistakes will get longer and longer. And

589
00:32:40,836 --> 00:32:43,476
Speaker 3: I think they say, like for every like basically one

590
00:32:43,596 --> 00:32:48,116
Speaker 3: day of work, it'll take several it's like a two

591
00:32:48,116 --> 00:32:50,076
Speaker 3: to one ratio at the beginning for how long it

592
00:32:50,116 --> 00:32:53,236
Speaker 3: will take humans to verify the output. Right, So you think, like,

593
00:32:53,356 --> 00:32:55,316
Speaker 3: you think you save time by having AI do this,

594
00:32:55,676 --> 00:32:58,356
Speaker 3: but if you want it to actually develop correctly, then

595
00:32:58,396 --> 00:33:00,036
Speaker 3: you need a team and it takes them twice as

596
00:33:00,076 --> 00:33:02,956
Speaker 3: long to verify that. What the AI did is actually

597
00:33:02,996 --> 00:33:06,796
Speaker 3: true and actually valid and actually aligned, et cetera, et cetera.

598
00:33:06,956 --> 00:33:09,116
Speaker 3: Now you're not asking it to do things that require

599
00:33:09,276 --> 00:33:11,316
Speaker 3: that amount of time, but there do need to be

600
00:33:11,396 --> 00:33:14,396
Speaker 3: little caveats to how we how we think about their

601
00:33:14,476 --> 00:33:18,316
Speaker 3: usefulness and how you are able to evaluate the output

602
00:33:18,716 --> 00:33:20,276
Speaker 3: versus Southern scenarios.

603
00:33:20,356 --> 00:33:23,796
Speaker 2: When I use AI the things it's best with our

604
00:33:23,836 --> 00:33:28,676
Speaker 2: things that like save any time. Right where I feeded

605
00:33:28,716 --> 00:33:30,996
Speaker 2: a bunch of different names for different college basketball teams

606
00:33:30,996 --> 00:33:33,396
Speaker 2: were working in our NCAA model, I'm like, take these

607
00:33:33,436 --> 00:33:37,796
Speaker 2: seven different naming conventions that are all different and create

608
00:33:37,836 --> 00:33:40,156
Speaker 2: a cross reference table of these, which is kind of

609
00:33:40,156 --> 00:33:41,436
Speaker 2: like a hard task. You need to have a little

610
00:33:41,436 --> 00:33:45,876
Speaker 2: context about basketball. And it did that very well. Right,

611
00:33:45,876 --> 00:33:47,596
Speaker 2: That's something I could have done. It might have taken

612
00:33:48,196 --> 00:33:51,516
Speaker 2: an hour or two, but you know, instead, instead it

613
00:33:51,516 --> 00:33:53,396
Speaker 2: could do it. It's in a few minutes, and it gets faster.

614
00:33:53,436 --> 00:33:56,356
Speaker 2: It's like, oh, I've learned from this from you before, Nate,

615
00:33:56,556 --> 00:33:58,276
Speaker 2: so now I can be faster doing this type of

616
00:33:58,276 --> 00:34:00,956
Speaker 2: task in the future. I was at the poker tournament

617
00:34:00,996 --> 00:34:04,876
Speaker 2: down in Florida last week and like, uh, you know,

618
00:34:04,956 --> 00:34:09,036
Speaker 2: I ask open research. Excuse me, God, is that like.

619
00:34:09,996 --> 00:34:15,236
Speaker 3: See exactly right. It all because deep research, Deep research,

620
00:34:15,356 --> 00:34:18,476
Speaker 3: Deep research, I asked, deep reach, eating too many of

621
00:34:18,516 --> 00:34:21,396
Speaker 3: these twenty twenty seven reports, brand gets.

622
00:34:21,236 --> 00:34:23,676
Speaker 2: To pull a bunch of stock market data for me.

623
00:34:23,916 --> 00:34:25,836
Speaker 2: And then I'm playing at poker hand and like and

624
00:34:25,956 --> 00:34:31,076
Speaker 2: make a really thin sexy value bet with like fourth pair.

625
00:34:31,236 --> 00:34:33,716
Speaker 2: No one knows that means right, I bet a very weekend.

626
00:34:33,836 --> 00:34:34,916
Speaker 2: I thought the other guy would call it with an

627
00:34:34,916 --> 00:34:36,316
Speaker 2: even weaker hand, and I was right. And I feel

628
00:34:36,356 --> 00:34:39,076
Speaker 2: like I'm such a fucking stud here valuating fourth pair,

629
00:34:39,156 --> 00:34:41,036
Speaker 2: And well AI does the work for me, and then

630
00:34:41,076 --> 00:34:43,156
Speaker 2: of course I like bust out of the tournament an

631
00:34:43,156 --> 00:34:46,356
Speaker 2: hour later, and meanwhile, you know, deep research bungles this

632
00:34:46,396 --> 00:34:48,876
Speaker 2: particular task. But in general AI has been very reliable.

633
00:34:48,916 --> 00:34:51,356
Speaker 2: But the point is that like there's like a a

634
00:34:51,636 --> 00:34:54,316
Speaker 2: inflection point where like I'm asking you to do things

635
00:34:54,316 --> 00:34:57,076
Speaker 2: that like are just a faster version of what I

636
00:34:57,076 --> 00:35:00,876
Speaker 2: could do myself. I would at the moment I ask, aid, be, like,

637
00:35:00,916 --> 00:35:03,636
Speaker 2: I want you design a new NCA model for me

638
00:35:03,716 --> 00:35:06,996
Speaker 2: with these parameters, because like I wouldn't know how to

639
00:35:07,116 --> 00:35:09,796
Speaker 2: test it. But anyway, I mean long wanted to hear,

640
00:35:09,916 --> 00:35:11,996
Speaker 2: so AGI, we're going to get AGI, or at least

641
00:35:11,996 --> 00:35:15,516
Speaker 2: we're going to get someone calling something AGI soon. Right,

642
00:35:17,276 --> 00:35:21,476
Speaker 2: artificial super intelligence where it's doing things much better than

643
00:35:21,516 --> 00:35:25,356
Speaker 2: human beings. I think this report takes for granted or

644
00:35:25,436 --> 00:35:28,956
Speaker 2: not takes for granted. It has lots of documentation about

645
00:35:28,956 --> 00:35:32,036
Speaker 2: its assumptions, but it's saying, Okay, this trajectory has been

646
00:35:32,116 --> 00:35:34,396
Speaker 2: very robust so far, and people make all types of

647
00:35:34,396 --> 00:35:36,836
Speaker 2: bullshit predictions. Of the fact that these guys in particular

648
00:35:36,876 --> 00:35:40,476
Speaker 2: have made accurate predictions in the past is certainly worth something,

649
00:35:40,476 --> 00:35:42,316
Speaker 2: I think, Right, But they're like, Okay, you kind of

650
00:35:42,356 --> 00:35:48,556
Speaker 2: follow the scaling law, and before too much longer, you know,

651
00:35:48,676 --> 00:35:51,876
Speaker 2: AI starts to be more intelligent than human beings. You

652
00:35:51,876 --> 00:35:53,716
Speaker 2: can debate what intelligent means if you want, but do

653
00:35:53,756 --> 00:35:57,276
Speaker 2: superhuman types of things or and or do them very fast,

654
00:35:57,356 --> 00:35:58,916
Speaker 2: you thing might be different, right, And and I can

655
00:35:58,956 --> 00:36:02,356
Speaker 2: do things very fast. Is maybe it's certainly a component

656
00:36:02,356 --> 00:36:04,676
Speaker 2: of intelligence, right, But like, but I don't take for

657
00:36:04,716 --> 00:36:11,276
Speaker 2: granted that like quote unquote AI I can reliably extrapolate

658
00:36:12,116 --> 00:36:15,956
Speaker 2: beyond the data set. I just think that, like, it's

659
00:36:15,956 --> 00:36:19,116
Speaker 2: not an absurdly pavologic It may even be like the

660
00:36:19,236 --> 00:36:21,436
Speaker 2: base case are close to the base case, but like

661
00:36:21,556 --> 00:36:26,276
Speaker 2: that's not assumable from first principles. I don't think we've

662
00:36:26,316 --> 00:36:29,036
Speaker 2: all seen lots of trend charts of the you know,

663
00:36:29,076 --> 00:36:33,276
Speaker 2: if you look at a chart of Japan's GDP in

664
00:36:33,276 --> 00:36:35,836
Speaker 2: the nineteen eighties, you might have said, Okay, well, Japan's

665
00:36:35,876 --> 00:36:38,516
Speaker 2: gonna take over the world, and being people bought this

666
00:36:39,916 --> 00:36:42,316
Speaker 2: and now it's kind of hasn't grown for like forty

667
00:36:42,396 --> 00:36:44,516
Speaker 2: years basically, right, And so like we've all seen lots

668
00:36:44,556 --> 00:36:46,036
Speaker 2: of curves that go up and then it's actually an

669
00:36:46,156 --> 00:36:47,556
Speaker 2: estra where the fuck you call it? Where like it

670
00:36:47,636 --> 00:36:49,676
Speaker 2: begins to bend the other way at some point and

671
00:36:49,676 --> 00:36:52,396
Speaker 2: and we can't tell until later. Right. The other thing

672
00:36:52,436 --> 00:36:57,836
Speaker 2: is like the abillity of AI to plan and manipulate

673
00:36:58,436 --> 00:37:01,316
Speaker 2: the physical world. I mean some of these things where

674
00:37:01,316 --> 00:37:05,036
Speaker 2: they're talking about, like you know, brain uploading and dice

675
00:37:05,116 --> 00:37:08,836
Speaker 2: and swarms and nanobots like you know there, I would

676
00:37:09,196 --> 00:37:14,516
Speaker 2: literally wager money against this happening on the time scales

677
00:37:14,556 --> 00:37:18,236
Speaker 2: that they're talking about. Right, they double the timescale, Okay,

678
00:37:18,276 --> 00:37:20,036
Speaker 2: then I might start to give some more probability. And look,

679
00:37:20,036 --> 00:37:21,356
Speaker 2: I'm willing to be wrong about that. I guess we'll

680
00:37:21,356 --> 00:37:24,716
Speaker 2: I'll be dead anyway, fifty percent likely in this scenario,

681
00:37:24,716 --> 00:37:25,956
Speaker 2: but like.

682
00:37:25,556 --> 00:37:28,116
Speaker 3: This scenario fifty percent to do like AA.

683
00:37:28,316 --> 00:37:32,956
Speaker 2: You know, the physical world requires sensory input and lots

684
00:37:32,956 --> 00:37:35,276
Speaker 2: of parts of our brain that AI is not as

685
00:37:35,276 --> 00:37:39,316
Speaker 2: effective at manipulating. It also requires like being given or

686
00:37:39,356 --> 00:37:42,476
Speaker 2: commandeering resources somehow. By the way, this is like a

687
00:37:42,516 --> 00:37:47,036
Speaker 2: little bit of a problem for the United States. I mean,

688
00:37:47,116 --> 00:37:52,036
Speaker 2: we are behind China by like quite a bit in

689
00:37:52,876 --> 00:37:55,716
Speaker 2: robotics and related things, right, So, like, I don't know

690
00:37:55,716 --> 00:38:00,516
Speaker 2: what happens if like we have the brainier, smarter AIS,

691
00:38:00,716 --> 00:38:02,956
Speaker 2: but like they're very good at manufacturing machinery. So like,

692
00:38:02,956 --> 00:38:04,276
Speaker 2: what if we have the brains and they have the

693
00:38:05,196 --> 00:38:09,156
Speaker 2: brawn so to speak, right, and they have maybe a

694
00:38:09,356 --> 00:38:14,636
Speaker 2: more authoritarian but functional infrastructure. So I don't know what

695
00:38:14,676 --> 00:38:17,516
Speaker 2: happens then, right, Like, but the ability of AIS to

696
00:38:17,556 --> 00:38:20,396
Speaker 2: comment to your resources to control the physical world, to

697
00:38:20,556 --> 00:38:24,276
Speaker 2: me seems far fetched on these timelines, in part because

698
00:38:24,316 --> 00:38:27,156
Speaker 2: of politics, right, I mean, the fact that it takes

699
00:38:27,596 --> 00:38:30,076
Speaker 2: so long to build a new building in the US

700
00:38:30,596 --> 00:38:34,516
Speaker 2: or a new subway station or a new highway, and

701
00:38:34,556 --> 00:38:37,956
Speaker 2: the fact that our politics is kind of sclerotic. Right,

702
00:38:38,116 --> 00:38:39,796
Speaker 2: And look, I mean I don't want to sound like

703
00:38:41,116 --> 00:38:43,636
Speaker 2: too pessimistic, but if you read the book I recommended

704
00:38:43,956 --> 00:38:46,276
Speaker 2: last week, fight, I mean, we basically did not have

705
00:38:46,276 --> 00:38:48,676
Speaker 2: a fully competent president for the last two years, and

706
00:38:48,716 --> 00:38:50,236
Speaker 2: I would argue that we don't have one for the

707
00:38:50,236 --> 00:38:52,996
Speaker 2: next four years. Right, So like all these kind of

708
00:38:53,076 --> 00:38:56,276
Speaker 2: things that we have to plan for, like who's doing

709
00:38:56,356 --> 00:39:00,476
Speaker 2: that fucking planning? Our government's kind of dysfunctional, and you know,

710
00:39:00,756 --> 00:39:02,916
Speaker 2: maybe that means we just lose to China, right, Maybe

711
00:39:02,916 --> 00:39:04,356
Speaker 2: that means we lose to China, at least we'll have

712
00:39:04,436 --> 00:39:06,436
Speaker 2: like nice cars. I guess.

713
00:39:10,076 --> 00:39:24,556
Speaker 3: We'll be back right after this. I think that your

714
00:39:24,996 --> 00:39:30,436
Speaker 3: point about the growth trajectories not necessarily being reliable is

715
00:39:30,476 --> 00:39:33,276
Speaker 3: a very valid one. The Japanic example is great. You know,

716
00:39:33,436 --> 00:39:39,236
Speaker 3: malfusion population growth is another is another big one. Right,

717
00:39:39,316 --> 00:39:42,836
Speaker 3: we thought that population would explode and instead we're actually

718
00:39:42,836 --> 00:39:47,116
Speaker 3: seeing population decline, So you know, the world does change.

719
00:39:47,356 --> 00:39:50,316
Speaker 3: The thing that I think they rely on is that

720
00:39:50,356 --> 00:39:55,236
Speaker 3: the AIS are capable of designing just this incredible technology

721
00:39:55,356 --> 00:39:57,916
Speaker 3: much more quickly, so that our building process and all

722
00:39:57,956 --> 00:40:01,276
Speaker 3: of that gets sped up one hundredfold from what it

723
00:40:01,356 --> 00:40:03,676
Speaker 3: is right now. But it's still at least at this

724
00:40:03,796 --> 00:40:06,436
Speaker 3: point needs humans to implement it right, and needs all

725
00:40:06,476 --> 00:40:09,036
Speaker 3: of these different workers. And so yeah, I think there's

726
00:40:09,156 --> 00:40:12,916
Speaker 3: some assumptions built into here that I hope, like I

727
00:40:12,956 --> 00:40:16,036
Speaker 3: hope that that timeline isn't feasible, and I do think

728
00:40:16,076 --> 00:40:19,116
Speaker 3: that there are things that are holding us back all

729
00:40:19,156 --> 00:40:21,276
Speaker 3: the same. I think it's really I think it's interesting.

730
00:40:21,316 --> 00:40:23,716
Speaker 3: One of the reasons I like this report is that

731
00:40:23,836 --> 00:40:26,156
Speaker 3: it forces you to think about these things right and

732
00:40:26,276 --> 00:40:29,116
Speaker 3: try to game out some of these worst case scenarios

733
00:40:29,156 --> 00:40:32,516
Speaker 3: to try to prevent them, which I think is always

734
00:40:32,516 --> 00:40:34,996
Speaker 3: an important thought exercise. I do want to go back

735
00:40:35,036 --> 00:40:38,516
Speaker 3: to kind of their good scenario, which just so bad

736
00:40:38,556 --> 00:40:42,676
Speaker 3: scenario is, you know, we're all wiped out by a

737
00:40:42,756 --> 00:40:46,916
Speaker 3: chemical warfare that the AI is release on us. Good

738
00:40:46,956 --> 00:40:50,716
Speaker 3: scenario is that you know, everyone gets a universal basic

739
00:40:50,756 --> 00:40:53,076
Speaker 3: income and AI does everything and no one has to

740
00:40:53,076 --> 00:40:56,596
Speaker 3: do anything and we can just live, you know, happily

741
00:40:56,596 --> 00:40:59,916
Speaker 3: with Maria at play poker. Yeah, and that just as

742
00:41:00,596 --> 00:41:03,076
Speaker 3: you suggest it, that seems actually like a very dystopian

743
00:41:03,436 --> 00:41:08,596
Speaker 3: scenario where people can become much more easy to brainwash, control,

744
00:41:08,636 --> 00:41:11,596
Speaker 3: et cetera, et cetera. It's like a dumbing town, right

745
00:41:12,036 --> 00:41:16,436
Speaker 3: where where we're not challenged to produce good art to

746
00:41:16,596 --> 00:41:19,556
Speaker 3: advance in any sort of way. Just to me, it

747
00:41:19,556 --> 00:41:21,956
Speaker 3: does not seem like a very meaningful The.

748
00:41:22,036 --> 00:41:23,996
Speaker 2: Question is there's not way another if you read AI

749
00:41:24,036 --> 00:41:26,436
Speaker 2: twenty twenty seven, which I highly recommend that you read it.

750
00:41:26,556 --> 00:41:31,876
Speaker 2: There's also another post by a pseudonymous poster called L.

751
00:41:32,116 --> 00:41:35,596
Speaker 2: Rudolph L who wrote something called A History of the

752
00:41:35,596 --> 00:41:41,476
Speaker 2: Future twenty twenty five to twenty forty, which is very

753
00:41:41,516 --> 00:41:43,676
Speaker 2: detailed but goes through kind of like what this looks

754
00:41:43,756 --> 00:41:48,676
Speaker 2: like at like more of a human level, how society evolves,

755
00:41:48,876 --> 00:41:53,436
Speaker 2: how the economy evolves, how work evolves, right, and like

756
00:41:54,236 --> 00:41:56,996
Speaker 2: very detailed, just like AI twenty twenty seven. Is that

757
00:41:57,156 --> 00:41:58,676
Speaker 2: kind of focused on the parts of A twenty seven,

758
00:41:58,676 --> 00:42:01,756
Speaker 2: then I think it kind of deliberately ignores maybe can

759
00:42:01,756 --> 00:42:04,356
Speaker 2: call them mild blind spots or whatever, right, But like,

760
00:42:04,516 --> 00:42:05,996
Speaker 2: but that's interesting because that kind of thinks about, like

761
00:42:05,996 --> 00:42:07,276
Speaker 2: what types of jobs are there in the future. There

762
00:42:07,276 --> 00:42:10,116
Speaker 2: a probably lots of lawyers actually, right, because you know,

763
00:42:10,356 --> 00:42:13,556
Speaker 2: the law is very sluggish to change, especially in a

764
00:42:13,596 --> 00:42:17,036
Speaker 2: constitutional system where there are lots of vitail points. Right,

765
00:42:17,156 --> 00:42:19,836
Speaker 2: probably high end service sector. You know, you go to

766
00:42:20,076 --> 00:42:21,796
Speaker 2: a restaurant because everyone's for a lot of people are

767
00:42:21,876 --> 00:42:24,996
Speaker 2: rich now, right, and you're flattered by the attractive young

768
00:42:25,796 --> 00:42:28,596
Speaker 2: server and things like that is kind of highly kind

769
00:42:28,636 --> 00:42:32,636
Speaker 2: of like catered and curated experiences. I guess I have

770
00:42:34,356 --> 00:42:39,236
Speaker 2: some faith in humanities abilieved to fight back quote unquote

771
00:42:39,276 --> 00:42:42,396
Speaker 2: against like two scenarios that it might not really like

772
00:42:42,436 --> 00:42:44,836
Speaker 2: either one, you know what I mean. And like the

773
00:42:44,876 --> 00:42:48,836
Speaker 2: scenario where like AI is producing like ten percent GDP

774
00:42:48,996 --> 00:42:51,156
Speaker 2: growth or or whatever. Right, man, it's great a few

775
00:42:51,236 --> 00:42:54,596
Speaker 2: own stocks that are exposed to AI and tech companies probably, right,

776
00:42:54,876 --> 00:42:57,156
Speaker 2: but it's also making that money on the backs of

777
00:42:57,516 --> 00:43:01,356
Speaker 2: mass job displacement, and like, you know, it kind of

778
00:43:01,396 --> 00:43:03,676
Speaker 2: a cert of confident in the long run that human

779
00:43:03,716 --> 00:43:07,076
Speaker 2: beings fine productive things to do. And you know, massive

780
00:43:07,116 --> 00:43:09,876
Speaker 2: employment has been predicted many times and never really occurred, right,

781
00:43:09,916 --> 00:43:12,636
Speaker 2: But like, but it's not occurring this fast where they

782
00:43:12,636 --> 00:43:14,916
Speaker 2: think the world ends in six years or whether they're

783
00:43:14,916 --> 00:43:17,116
Speaker 2: predicting or we have utopia in six years, and like

784
00:43:17,476 --> 00:43:20,076
Speaker 2: just the ability of like human society to like deal

785
00:43:20,116 --> 00:43:23,036
Speaker 2: with that change at these times scales leads to like

786
00:43:23,516 --> 00:43:26,036
Speaker 2: more chaos than I think. They're more predicting. But I

787
00:43:26,116 --> 00:43:28,756
Speaker 2: think also and I told them this too, right, I

788
00:43:28,756 --> 00:43:33,956
Speaker 2: think also leads to more constraints. Right that you hit bottlenecks.

789
00:43:33,996 --> 00:43:37,836
Speaker 2: If you have five things you have to do, right

790
00:43:38,956 --> 00:43:42,036
Speaker 2: and you have the world's fastest computer, et cetera, et cetera.

791
00:43:42,076 --> 00:43:45,236
Speaker 2: But there's like a power outage in your neighborhood, right,

792
00:43:45,276 --> 00:43:47,556
Speaker 2: and that's a that's a bottle deck. Right, Maybe there

793
00:43:47,556 --> 00:43:50,236
Speaker 2: are ways around it. Youre like go to home depot

794
00:43:50,476 --> 00:43:52,956
Speaker 2: and buy a generator or you know what I mean.

795
00:43:52,996 --> 00:43:54,676
Speaker 2: But like, but the point is that like you're often

796
00:43:54,676 --> 00:43:57,636
Speaker 2: defined by like the slowest link, and politics are sometimes

797
00:43:57,636 --> 00:43:59,876
Speaker 2: the slowest link, but also by like Also, you know,

798
00:43:59,956 --> 00:44:04,516
Speaker 2: I think the report maybe under states, and I think

799
00:44:04,596 --> 00:44:07,476
Speaker 2: kind of in general the a saify community like maybe

800
00:44:07,556 --> 00:44:10,836
Speaker 2: understates the ability of like human beings to cause harm

801
00:44:11,396 --> 00:44:14,796
Speaker 2: to other human beings with AI, Right, that concern kind

802
00:44:14,836 --> 00:44:18,636
Speaker 2: of gets brushed off, is like to pedestrian or like

803
00:44:18,956 --> 00:44:21,116
Speaker 2: say to pedestri And I was saying, yeah.

804
00:44:20,716 --> 00:44:22,876
Speaker 3: The exact word I was thinking of. I think that's

805
00:44:22,876 --> 00:44:24,796
Speaker 3: a good I mean, I think that's a great place

806
00:44:24,876 --> 00:44:28,036
Speaker 3: to end it, because yes, we do need to be

807
00:44:28,316 --> 00:44:31,556
Speaker 3: concerned about all of these things about AI. But like

808
00:44:31,636 --> 00:44:34,916
Speaker 3: that phrase I think is very crucial, like do not

809
00:44:35,076 --> 00:44:38,876
Speaker 3: underestimate the ability of humans to cause harm to other humans.

810
00:44:39,196 --> 00:44:41,676
Speaker 3: And I think that that's you know, it's not a

811
00:44:41,796 --> 00:44:44,676
Speaker 3: very optim it's not a very pleasant place to end,

812
00:44:44,676 --> 00:44:47,116
Speaker 3: but I think it's a really important place to end.

813
00:44:47,156 --> 00:44:49,636
Speaker 3: And I think that that's a very valid kind of

814
00:44:49,676 --> 00:44:51,676
Speaker 3: way of reflecting on this name.

815
00:44:51,596 --> 00:44:55,476
Speaker 2: Or to trust AIS too much, right, I generally think

816
00:44:55,476 --> 00:45:00,036
Speaker 2: that concern is like somewhat misplays but like if we're

817
00:45:00,036 --> 00:45:04,836
Speaker 2: handing over critical systems to AI, right, it can cause

818
00:45:04,876 --> 00:45:07,556
Speaker 2: problems if it's very smart and deceives us and doesn't

819
00:45:07,596 --> 00:45:10,156
Speaker 2: like us very much. It also cause problems if it

820
00:45:10,196 --> 00:45:15,716
Speaker 2: has hallucinations, bugs in critical areas where it isn't as

821
00:45:15,796 --> 00:45:19,356
Speaker 2: robust and hasn't really been tested yet that are outside

822
00:45:19,396 --> 00:45:24,276
Speaker 2: of its domain yep, or there could be espionage. Anyway,

823
00:45:24,476 --> 00:45:28,396
Speaker 2: we will have plenty of time, although maybe only seven

824
00:45:28,436 --> 00:45:34,516
Speaker 2: more years actually to like explore expore these scenarios.

825
00:45:35,756 --> 00:45:38,356
Speaker 3: Yes, and in seven years we'll be like, welcome back

826
00:45:38,396 --> 00:45:41,116
Speaker 3: to the final episode of Risky Business, because the prediction

827
00:45:41,316 --> 00:45:44,316
Speaker 3: is we're all going to be done tomorrow. But yeah,

828
00:45:44,356 --> 00:45:48,636
Speaker 3: this was an interesting exercise, and I think my my

829
00:45:48,756 --> 00:45:51,436
Speaker 3: p doom has slightly gone up as a result of

830
00:45:51,476 --> 00:45:55,276
Speaker 3: reading this, but I also remain optimistic that humans can

831
00:45:55,796 --> 00:45:57,436
Speaker 3: can do good as well as harm.

832
00:45:58,836 --> 00:46:01,876
Speaker 2: Yeah, my interest in learning Chinese has increased as a

833
00:46:01,876 --> 00:46:05,796
Speaker 2: result of recent developments I know about my pe doome.

834
00:46:05,676 --> 00:46:09,236
Speaker 3: All right, Yeah, let's let's do some language emmergis and

835
00:46:09,676 --> 00:46:17,556
Speaker 3: I'm with you. That's it for today. If you're a

836
00:46:17,596 --> 00:46:20,556
Speaker 3: premium subscriber, we will be answering a question about whether

837
00:46:20,796 --> 00:46:24,436
Speaker 3: mface can ever be plus ev right after the credits.

838
00:46:24,796 --> 00:46:27,036
Speaker 3: And if you're not a subscriber, it's not too late.

839
00:46:27,356 --> 00:46:30,316
Speaker 3: For six ninety nine a month the price of a midber,

840
00:46:30,676 --> 00:46:33,956
Speaker 3: you get access to all these conversations and all premium

841
00:46:33,996 --> 00:46:38,716
Speaker 3: content across the Pushkin Network. Risky Business is hosted by

842
00:46:38,716 --> 00:46:40,396
Speaker 3: me Maria Kannakova.

843
00:46:40,036 --> 00:46:42,396
Speaker 2: And by me Nate Silver. The show is a co

844
00:46:42,476 --> 00:46:46,596
Speaker 2: production of Pushkin Industries and iHeartMedia. This episode was produced

845
00:46:46,596 --> 00:46:50,036
Speaker 2: by Isabel Carter. Our associate producer is Sonia Gerwit. Sally

846
00:46:50,116 --> 00:46:53,116
Speaker 2: helm is our editor, and our executive producer is Jacob Goldstein.

847
00:46:53,636 --> 00:46:56,796
Speaker 2: Mixing by Sarah Bruger. If you like the show, please

848
00:46:56,876 --> 00:46:59,316
Speaker 2: rate and review us. You know we like we take

849
00:46:59,316 --> 00:47:01,836
Speaker 2: a four or five. We take the five. Rate and

850
00:47:01,876 --> 00:47:03,636
Speaker 2: review us. To other people, Thank you for listening.