1
00:00:15,564 --> 00:00:16,044
Speaker 1: Pushkin.

2
00:00:20,964 --> 00:00:24,124
Speaker 2: Hey everybody, Nate here, jumping in before the short so

3
00:00:24,164 --> 00:00:25,884
Speaker 2: that we've been having a lot of fun answering your

4
00:00:25,884 --> 00:00:29,484
Speaker 2: listener questions. So far, we've covered things like Pascal's Wager,

5
00:00:29,684 --> 00:00:32,164
Speaker 2: the hot hand fallacy, or the fallacy of the hot

6
00:00:32,204 --> 00:00:36,044
Speaker 2: hand fallacy actually, and the expected value of learning new languages.

7
00:00:36,684 --> 00:00:38,444
Speaker 2: We want to keep doing this kind of thing, so

8
00:00:38,484 --> 00:00:43,004
Speaker 2: send us all your questions about risk decision making, game theory, poker,

9
00:00:43,444 --> 00:00:45,444
Speaker 2: you name it. Reach out to us on social media

10
00:00:45,564 --> 00:00:49,804
Speaker 2: or email at Risky Business at pushkin dot fm. Even

11
00:00:49,804 --> 00:00:51,804
Speaker 2: if you're not a premium subscriber, this is a great

12
00:00:51,804 --> 00:00:54,004
Speaker 2: way to support the show, so you can keep sharing

13
00:00:54,044 --> 00:00:56,444
Speaker 2: it free of charge. We look forward to hearing from you.

14
00:01:03,324 --> 00:01:06,724
Speaker 1: Welcome back to Risky Business, a show about making better decisions.

15
00:01:06,844 --> 00:01:08,524
Speaker 1: I'm Maria Kanakova.

16
00:01:08,084 --> 00:01:08,924
Speaker 2: And I'm Nate Silver.

17
00:01:09,604 --> 00:01:11,204
Speaker 1: Today on the show is going to be a little

18
00:01:11,204 --> 00:01:12,204
Speaker 1: bit doomtastic.

19
00:01:12,924 --> 00:01:15,404
Speaker 2: Yeah, I mean, I don't know if it's like worse

20
00:01:15,524 --> 00:01:19,564
Speaker 2: than thinking about like the global economy going into a

21
00:01:19,604 --> 00:01:23,804
Speaker 2: recession because of dumbfuck tariff policies. This is all about

22
00:01:23,804 --> 00:01:25,844
Speaker 2: how we're all going to die in seven years instead. No,

23
00:01:25,884 --> 00:01:29,084
Speaker 2: I'm just kidding. This is a very very intelligent and

24
00:01:29,124 --> 00:01:32,644
Speaker 2: well written and thoughtful report called AI twenty twenty seven

25
00:01:32,644 --> 00:01:34,924
Speaker 2: that we're going to spend the whole show on because

26
00:01:34,924 --> 00:01:37,604
Speaker 2: think it's such an interesting to talk about, but that

27
00:01:37,684 --> 00:01:42,044
Speaker 2: you know, includes some dystopian possibilities.

28
00:01:42,044 --> 00:01:45,324
Speaker 1: I would say it does indeed, so let's get into

29
00:01:45,364 --> 00:01:47,764
Speaker 1: it and hope that you guys are all still here

30
00:01:47,804 --> 00:01:52,684
Speaker 1: to listen to us in seven years.

31
00:01:52,884 --> 00:01:57,884
Speaker 2: The contrast is interesting between like all the chaos we're

32
00:01:57,924 --> 00:02:01,684
Speaker 2: seeing with tariff policy in terms of starting a trade

33
00:02:01,724 --> 00:02:05,684
Speaker 2: war with China and then other types of chaos. It's

34
00:02:05,724 --> 00:02:07,844
Speaker 2: interesting to kind of look at this. I mean, I

35
00:02:07,844 --> 00:02:12,244
Speaker 2: wouldn't call it a more optimistic future exactly, but like

36
00:02:12,684 --> 00:02:14,644
Speaker 2: but like on a different trajectory of like a future

37
00:02:14,684 --> 00:02:18,844
Speaker 2: that's going to change very fast, according to these authors,

38
00:02:18,844 --> 00:02:23,964
Speaker 2: with profound implications for you know, everything, the human species.

39
00:02:25,084 --> 00:02:28,324
Speaker 2: These researchers and authors are saying that everything is going

40
00:02:28,364 --> 00:02:30,404
Speaker 2: to change profoundly.

41
00:02:30,164 --> 00:02:30,804
Speaker 1: And.

42
00:02:32,524 --> 00:02:36,524
Speaker 2: Even though there is some hedging hair, this is kind

43
00:02:36,564 --> 00:02:40,484
Speaker 2: of their base case scenario, and like you know, base

44
00:02:40,524 --> 00:02:43,244
Speaker 2: case number one and base case number two differ. There's

45
00:02:43,244 --> 00:02:44,844
Speaker 2: like kind of a choose your own adventure at some

46
00:02:44,924 --> 00:02:47,764
Speaker 2: point in this report. But they're both very different than

47
00:02:47,764 --> 00:02:50,604
Speaker 2: the status quo, right, and the notionately hear from that,

48
00:02:50,764 --> 00:02:56,564
Speaker 2: like everything becomes different. If ais become substantially more intelligent

49
00:02:56,604 --> 00:02:58,564
Speaker 2: than human beings, people can debate and we will debate

50
00:02:58,604 --> 00:03:00,484
Speaker 2: on this program what that means. But yeah, do you

51
00:03:00,564 --> 00:03:02,564
Speaker 2: want to do you want to contextualize this more? Do

52
00:03:02,564 --> 00:03:04,444
Speaker 2: you want to tell people to actors are of this report?

53
00:03:04,764 --> 00:03:10,604
Speaker 1: Absolutely? Absolutely? So the report is authored officially by five people,

54
00:03:10,764 --> 00:03:14,364
Speaker 1: and I think unofficially there's also a sixth We've got

55
00:03:14,844 --> 00:03:19,924
Speaker 1: Eli Leifelund, who's a super forecaster and he was ranked

56
00:03:20,004 --> 00:03:24,324
Speaker 1: first on RAN's forecasting initiative, so he is someone who

57
00:03:24,404 --> 00:03:27,484
Speaker 1: is very good at kind of looking at the future

58
00:03:27,524 --> 00:03:30,524
Speaker 1: trying to predict what's going to happen. You have Jonas

59
00:03:30,564 --> 00:03:35,684
Speaker 1: Palmer who's a VC at Macroscopic Ventures. Thomas Larson was

60
00:03:35,724 --> 00:03:40,004
Speaker 1: a former executive director of the Center for AI Policy,

61
00:03:40,764 --> 00:03:44,644
Speaker 1: so that is a center that advises both sides of

62
00:03:44,684 --> 00:03:47,404
Speaker 1: the aisle on you know, how how AI is going

63
00:03:47,484 --> 00:03:51,884
Speaker 1: to go? And Romeo Dean, who is part of Harvard's

64
00:03:51,884 --> 00:03:56,044
Speaker 1: AI Safety Student Team, so someone who is still a student,

65
00:03:56,124 --> 00:03:59,884
Speaker 1: still learning, but kind of the next generation of people

66
00:04:00,564 --> 00:04:04,884
Speaker 1: looking at AI. And finally, we have Daniel coch Taylor,

67
00:04:05,484 --> 00:04:10,244
Speaker 1: who basically had written a report back in twenty twenty one.

68
00:04:10,284 --> 00:04:14,484
Speaker 1: He was a AI researcher and he looked at predictions

69
00:04:14,524 --> 00:04:17,644
Speaker 1: for AI for twenty twenty six, and it turns out

70
00:04:17,764 --> 00:04:20,924
Speaker 1: that his predictions were pretty spot on, and so open

71
00:04:20,924 --> 00:04:24,844
Speaker 1: AI actually hired Daniel as a result of this report.

72
00:04:24,844 --> 00:04:26,804
Speaker 2: Obviously he now left, yes.

73
00:04:26,684 --> 00:04:29,004
Speaker 1: And then and then he left exactly.

74
00:04:28,644 --> 00:04:32,324
Speaker 2: And importantly, so there's also Scott Alexander exactly.

75
00:04:32,404 --> 00:04:34,284
Speaker 1: So I was saying, he's the he's the person who

76
00:04:34,764 --> 00:04:37,644
Speaker 1: kind of is in the background, and he's you guys

77
00:04:37,764 --> 00:04:40,964
Speaker 1: might know him as the author behind astral Codux.

78
00:04:41,804 --> 00:04:43,404
Speaker 2: And I know Scott. He's one of the kind of

79
00:04:43,444 --> 00:04:50,604
Speaker 2: fathers of what you might call rationalism. I think Scott,

80
00:04:50,644 --> 00:04:52,884
Speaker 2: when I interview him from a book, was happy enough

81
00:04:52,884 --> 00:04:55,284
Speaker 2: with that term and accused me or co opted me

82
00:04:55,324 --> 00:05:00,284
Speaker 2: into also being a rationalist. These people are somewhat adjacent

83
00:05:00,324 --> 00:05:02,884
Speaker 2: to the effective altruist, but not quite right. They're just

84
00:05:02,884 --> 00:05:07,964
Speaker 2: trying to apply a sort of thoughtful, rigorous, quantitative lens

85
00:05:08,164 --> 00:05:12,244
Speaker 2: to big picture problem, including existential risk, of which most

86
00:05:12,284 --> 00:05:15,124
Speaker 2: people in this community believe that AI is both an

87
00:05:15,124 --> 00:05:18,964
Speaker 2: existential risk and also kind of an existential opportunity, right,

88
00:05:18,964 --> 00:05:21,004
Speaker 2: that it could transform things. You talk to Sam Alt

89
00:05:21,044 --> 00:05:23,564
Speaker 2: and he'll say We're going to cure cancer and illuminate

90
00:05:23,604 --> 00:05:28,084
Speaker 2: poverty and whatever else. Right, And Scott's also an excellent writer.

91
00:05:28,324 --> 00:05:31,524
Speaker 2: And so let me disclose something which is slightly important here.

92
00:05:31,564 --> 00:05:34,524
Speaker 2: So I actually was approached by some of the authors

93
00:05:34,564 --> 00:05:37,324
Speaker 2: of this report a couple of months ago, I guess

94
00:05:37,364 --> 00:05:40,644
Speaker 2: it was in February ish, just to give feedback and

95
00:05:40,764 --> 00:05:44,204
Speaker 2: chat with them. So I'm working off the draft version, right,

96
00:05:44,244 --> 00:05:46,724
Speaker 2: which I do not believe they changed very much. So

97
00:05:46,764 --> 00:05:48,964
Speaker 2: my notes pertained to an earlier draft. I did not

98
00:05:49,044 --> 00:05:50,204
Speaker 2: have time this morning to go back and re.

99
00:05:50,244 --> 00:05:54,084
Speaker 1: Read it, so I was not on the inside loops.

100
00:05:54,084 --> 00:05:56,764
Speaker 1: So I did not get an earlier draft. And I've

101
00:05:56,804 --> 00:06:00,644
Speaker 1: read this draft and basically just to kind of big

102
00:06:00,684 --> 00:06:06,204
Speaker 1: picture sum it up, it outlines two scenarios, right, two

103
00:06:06,284 --> 00:06:10,644
Speaker 1: major scenarios for how AI might change the world as

104
00:06:10,924 --> 00:06:16,284
Speaker 1: soon as twenty thirty. Now important note like that that

105
00:06:16,564 --> 00:06:19,404
Speaker 1: date is kind of hedged. It might be sooner, it

106
00:06:19,484 --> 00:06:22,844
Speaker 1: might be later. There's kind of a there's a confidence

107
00:06:22,884 --> 00:06:27,924
Speaker 1: interval there. But the two different scenarios. One basically we

108
00:06:28,124 --> 00:06:32,524
Speaker 1: do twenty thirty, humanity disappears and is taken over by AI.

109
00:06:33,084 --> 00:06:37,244
Speaker 1: The positive report is in twenty thirty. Basically we get

110
00:06:38,004 --> 00:06:41,084
Speaker 1: AIS that are aligned to our interests, and we get

111
00:06:41,164 --> 00:06:44,764
Speaker 1: kind of this AI utopia where AI is actually help

112
00:06:44,884 --> 00:06:47,764
Speaker 1: make life much better for everyone and make the standard

113
00:06:47,844 --> 00:06:50,484
Speaker 1: of living much higher. But the crucial turning point is

114
00:06:50,524 --> 00:06:53,844
Speaker 1: before twenty thirty, and the crucial kind of question at

115
00:06:53,844 --> 00:06:58,524
Speaker 1: the center of this is will we be able to

116
00:06:58,644 --> 00:07:04,244
Speaker 1: design AIS that are truly aligned to human interests rather

117
00:07:04,284 --> 00:07:07,164
Speaker 1: than just appear to be aligned and kind of lying

118
00:07:07,204 --> 00:07:11,684
Speaker 1: to us while actually following their own agenda. And how

119
00:07:11,724 --> 00:07:14,924
Speaker 1: we handle that is kind of the lynchpin. And it's

120
00:07:14,924 --> 00:07:17,684
Speaker 1: actually interesting, Nate, that you started out with China because

121
00:07:17,964 --> 00:07:20,204
Speaker 1: a lot of the policy choices and a lot of

122
00:07:20,204 --> 00:07:23,524
Speaker 1: what they see as kind of the decision points that

123
00:07:23,564 --> 00:07:27,924
Speaker 1: will affect the future of humanity actually hinge on the

124
00:07:28,124 --> 00:07:32,364
Speaker 1: US China dynamic, how they compete with each other, and

125
00:07:32,404 --> 00:07:37,644
Speaker 1: how that sometimes might basically clash against safety concerns because

126
00:07:38,324 --> 00:07:41,004
Speaker 1: no one wants to be left behind. Can we manage

127
00:07:41,044 --> 00:07:44,964
Speaker 1: that effectively and can kind of that transition work in

128
00:07:45,004 --> 00:07:47,524
Speaker 1: our favor as opposed to against us. I think that

129
00:07:47,564 --> 00:07:49,204
Speaker 1: this is kind of one of the big questions here,

130
00:07:49,364 --> 00:07:51,044
Speaker 1: and so it's funny that we're seeing all of this

131
00:07:51,164 --> 00:07:54,564
Speaker 1: trade war right now as the sideport is coming out.

132
00:07:55,404 --> 00:08:00,804
Speaker 2: Yeah, look, I think this exercise is partly just a

133
00:08:00,884 --> 00:08:04,044
Speaker 2: forecasting exercise, right, I mean, obviously it's just kind of

134
00:08:04,084 --> 00:08:08,204
Speaker 2: like fork at the bottom where we learn to have

135
00:08:08,284 --> 00:08:11,404
Speaker 2: an AI slow down or we kind of are pressing

136
00:08:11,404 --> 00:08:14,044
Speaker 2: fully on the accelerator, right, Like, in some ways, scenarios

137
00:08:14,084 --> 00:08:20,084
Speaker 2: are like not that different, right, Either one assumes remarkable

138
00:08:20,164 --> 00:08:23,604
Speaker 2: rates of technological growth that I think even AI I'm

139
00:08:23,604 --> 00:08:26,364
Speaker 2: never quite sure who to call an optimist or a pessimist, right,

140
00:08:26,404 --> 00:08:30,324
Speaker 2: even AI believers you know, might think is a little

141
00:08:30,324 --> 00:08:32,164
Speaker 2: bit aggressive. Right. But what they want to do is

142
00:08:32,204 --> 00:08:36,884
Speaker 2: they want to have like a specific, fleshed out scenario

143
00:08:36,964 --> 00:08:38,804
Speaker 2: for how the world would look. Like it's kind of

144
00:08:38,884 --> 00:08:42,644
Speaker 2: like a modal scenario. And like I think they'd say that, like,

145
00:08:44,164 --> 00:08:46,564
Speaker 2: we're not totally sure about either of these necessarily, right,

146
00:08:46,564 --> 00:08:48,884
Speaker 2: And I don't think they'd be as like pedantic as

147
00:08:48,884 --> 00:08:50,964
Speaker 2: to say, if you do X, Y, and Z, then

148
00:08:51,844 --> 00:08:54,004
Speaker 2: we'll save the world and have utopia, and if you don't,

149
00:08:54,084 --> 00:08:56,884
Speaker 2: then we'll all die. Right. I think they probably say

150
00:08:56,964 --> 00:08:59,884
Speaker 2: it's unclear and there's kind of like risk either way.

151
00:09:00,004 --> 00:09:02,204
Speaker 2: We wanted to go through the scenario like fleshing out

152
00:09:02,324 --> 00:09:05,604
Speaker 2: like what the world might look like. Right. I do

153
00:09:05,644 --> 00:09:08,644
Speaker 2: think one thing that's important is that whatever decisions are

154
00:09:08,724 --> 00:09:13,044
Speaker 2: made now could get locked in, right that you pass

155
00:09:13,084 --> 00:09:18,244
Speaker 2: certain points in overturn and it becomes very hard to accelerate,

156
00:09:18,364 --> 00:09:20,804
Speaker 2: like an arms race. This is you know, what we

157
00:09:20,884 --> 00:09:23,924
Speaker 2: found during the Cold War for example. I mean, one

158
00:09:23,924 --> 00:09:25,684
Speaker 2: of the big things I look at is like, do

159
00:09:25,764 --> 00:09:29,884
Speaker 2: we force the AI to be transparent in its thinking

160
00:09:30,444 --> 00:09:32,844
Speaker 2: with humans? Right? Like, now there's been a movement toward

161
00:09:33,444 --> 00:09:36,484
Speaker 2: the AI will actually explicate it's thinking more. I'll ask

162
00:09:36,524 --> 00:09:39,924
Speaker 2: you a query open AI. The Chinese models to this too, right,

163
00:09:39,964 --> 00:09:42,204
Speaker 2: and I'll say, I am thinking about X, Y and Z,

164
00:09:42,324 --> 00:09:44,484
Speaker 2: and I'm looking up PD and Q and now I'm

165
00:09:44,484 --> 00:09:47,644
Speaker 2: reconsidering this. It actually has this chain of thought process, right,

166
00:09:48,044 --> 00:09:51,284
Speaker 2: which is explicated in English. You know. One concern is

167
00:09:51,284 --> 00:09:53,604
Speaker 2: that what if the AI just kind of communicates to

168
00:09:53,644 --> 00:09:59,164
Speaker 2: another in these implicit vectors that's inferring from all the

169
00:09:59,164 --> 00:10:02,644
Speaker 2: texts it has. It's kind of unintelligible to human beings, right,

170
00:10:02,644 --> 00:10:04,604
Speaker 2: and maybe kind of quote unquote thinking in that way

171
00:10:05,204 --> 00:10:07,044
Speaker 2: in the first place, and then does us the favor

172
00:10:07,124 --> 00:10:10,884
Speaker 2: of like translating back to it goes from to kind

173
00:10:10,884 --> 00:10:13,284
Speaker 2: of this big bag of numbers is one day I

174
00:10:13,284 --> 00:10:17,044
Speaker 2: researcher called it, right, and then it translates back into

175
00:10:17,084 --> 00:10:19,764
Speaker 2: English or whatever language you want. Really, in the end,

176
00:10:20,844 --> 00:10:23,524
Speaker 2: what if it just skip cuts out that last step right,

177
00:10:23,604 --> 00:10:26,644
Speaker 2: then we can't kind of like check what AI is doing.

178
00:10:26,724 --> 00:10:30,604
Speaker 2: Then it can behave deceptively more easily. So you know,

179
00:10:30,804 --> 00:10:34,004
Speaker 2: so that part seems to be important. I want to

180
00:10:34,004 --> 00:10:37,084
Speaker 2: hear your your first impressions before I kind of poison

181
00:10:37,124 --> 00:10:38,164
Speaker 2: the well too much.

182
00:10:39,284 --> 00:10:43,204
Speaker 1: Well, my first impressions is that the alignment problem is

183
00:10:43,204 --> 00:10:46,804
Speaker 1: a very real one and an incredibly important one to solve.

184
00:10:47,124 --> 00:10:49,684
Speaker 1: And what I got from this is that actually, the

185
00:10:49,724 --> 00:10:53,884
Speaker 1: problem that I've had with like these initial AI lms

186
00:10:54,364 --> 00:10:56,924
Speaker 1: is the kernel of what they're seeing there. Right. So

187
00:10:57,124 --> 00:10:58,844
Speaker 1: you and I have talked about this on the show

188
00:10:58,924 --> 00:11:01,964
Speaker 1: in the past, and I've said, well, my problem is

189
00:11:01,964 --> 00:11:04,884
Speaker 1: that when I'm a domain expert, right, I start seeing

190
00:11:05,004 --> 00:11:08,684
Speaker 1: some inaccuracies, and I start seeing like places where like

191
00:11:08,724 --> 00:11:12,284
Speaker 1: it either just didn't do well or made shit up

192
00:11:12,364 --> 00:11:15,044
Speaker 1: or or whatever it is. Now I think it's very

193
00:11:15,084 --> 00:11:17,444
Speaker 1: clear that those problems are going to go away, right,

194
00:11:17,524 --> 00:11:20,804
Speaker 1: that that is going to get much much better. However,

195
00:11:21,364 --> 00:11:24,804
Speaker 1: the kernel of it's showing me something, but that might

196
00:11:24,884 --> 00:11:27,404
Speaker 1: just be you know, I have no way of verifying

197
00:11:27,484 --> 00:11:30,364
Speaker 1: if that's what's going on, what it's reading, like, how

198
00:11:30,404 --> 00:11:32,764
Speaker 1: it's I don't want to say thinking about it, even

199
00:11:32,804 --> 00:11:34,644
Speaker 1: though in the report they do use thinking, but.

200
00:11:34,644 --> 00:11:38,404
Speaker 3: It's normalizing too much, I think is correct.

201
00:11:38,404 --> 00:11:39,004
Speaker 2: I think it's.

202
00:11:39,164 --> 00:11:42,724
Speaker 1: Okay, we'll stick to that language. Yeah, okay, So, so

203
00:11:42,844 --> 00:11:47,644
Speaker 1: how it's thinking about it that those little problems and

204
00:11:47,724 --> 00:11:51,244
Speaker 1: like the little the glitches and the things that it

205
00:11:51,324 --> 00:11:54,364
Speaker 1: might be doing where it starts actually glitching on purpose,

206
00:11:54,804 --> 00:11:57,924
Speaker 1: are not going to be visible to the human eye.

207
00:11:58,124 --> 00:11:59,884
Speaker 1: And so one of the main things that they say

208
00:11:59,924 --> 00:12:04,404
Speaker 1: here is that as AI internal R and D gets

209
00:12:04,684 --> 00:12:09,924
Speaker 1: rapidly faster, so that means basically AI's researching AI, right,

210
00:12:10,204 --> 00:12:13,684
Speaker 1: and so internally they start developing new models, and as

211
00:12:13,724 --> 00:12:17,924
Speaker 1: they kind of surpass human ability to monitor it, it

212
00:12:17,964 --> 00:12:21,804
Speaker 1: becomes progressively more difficult to figure out. Okay, is the

213
00:12:21,844 --> 00:12:24,564
Speaker 1: AI actually doing what I want it to do? Is

214
00:12:24,644 --> 00:12:28,964
Speaker 1: the output that it's giving me its actual thought process?

215
00:12:29,284 --> 00:12:32,084
Speaker 1: And is it accurate or is it like trying to

216
00:12:32,124 --> 00:12:34,804
Speaker 1: deceive me? But it's actually kind of inserting certain things

217
00:12:35,164 --> 00:12:37,844
Speaker 1: on purpose because it has different goals, right, because it

218
00:12:37,884 --> 00:12:41,244
Speaker 1: is actually secretly misaligned, but it's very good at persuading

219
00:12:41,284 --> 00:12:43,484
Speaker 1: me that it's aligned. Because one of the things that

220
00:12:43,844 --> 00:12:45,964
Speaker 1: actually came out of this report, and I was like, huh,

221
00:12:46,044 --> 00:12:48,684
Speaker 1: you know this is interesting is if we get this

222
00:12:50,644 --> 00:12:55,844
Speaker 1: remarkable improvement in AI, it will also remarkably improve at

223
00:12:55,884 --> 00:13:00,684
Speaker 1: persuading us right as part of it. Making Yeah, so

224
00:13:00,724 --> 00:13:03,044
Speaker 1: this is but I've never even thought about that. I

225
00:13:03,084 --> 00:13:05,884
Speaker 1: was like, okay, fine, But one of the things that

226
00:13:05,924 --> 00:13:08,524
Speaker 1: I do buy is that it's going to be very

227
00:13:08,524 --> 00:13:11,044
Speaker 1: difficult for us to monitor it and to figure out

228
00:13:11,084 --> 00:13:14,604
Speaker 1: like is it truly aligned with human wants, with human desires,

229
00:13:14,644 --> 00:13:18,324
Speaker 1: with human goals, And the experts who are capable of

230
00:13:18,364 --> 00:13:20,884
Speaker 1: doing that, I think are actually going to dwindle right

231
00:13:21,004 --> 00:13:24,604
Speaker 1: as AI starts proliferating in society, And so to me,

232
00:13:24,724 --> 00:13:27,004
Speaker 1: that is something that is actually quite worrisome and that

233
00:13:27,124 --> 00:13:29,524
Speaker 1: is something that we really need to be paying attention to.

234
00:13:29,724 --> 00:13:29,964
Speaker 2: Now.

235
00:13:30,204 --> 00:13:33,444
Speaker 1: Just to fast forward a little bit, in their doomsday

236
00:13:33,484 --> 00:13:38,284
Speaker 1: scenario in twenty thirty one, AI takes over it basically

237
00:13:38,324 --> 00:13:42,204
Speaker 1: like suddenly releases some chemical agents, right, and humanity ties

238
00:13:42,724 --> 00:13:44,804
Speaker 1: and the rest of the stragglers are taken care of

239
00:13:44,844 --> 00:13:45,844
Speaker 1: by drones, et cetera.

240
00:13:46,444 --> 00:13:49,044
Speaker 2: I don't even like it. It doesn't even quick and

241
00:13:49,084 --> 00:13:52,764
Speaker 2: painless death. I will say, let's hope we.

242
00:13:52,764 --> 00:13:55,044
Speaker 1: Don't know what the chemical agents are. Might not be

243
00:13:55,124 --> 00:13:58,364
Speaker 1: quick and painless. Some chemical agents are actually a very

244
00:13:58,404 --> 00:14:01,404
Speaker 1: painful death thing. So let's hope. Let's hope it's quick

245
00:14:01,444 --> 00:14:06,484
Speaker 1: and painless. Quickly, quick, yes, stuff quick, Okay, hopefully some

246
00:14:06,604 --> 00:14:11,164
Speaker 1: chemical agents are not quick. I hope it's quick and painless.

247
00:14:12,004 --> 00:14:15,364
Speaker 1: But if they're actually capable of deception at that high level,

248
00:14:15,684 --> 00:14:19,124
Speaker 1: then you technically don't even need them to do it.

249
00:14:19,164 --> 00:14:22,004
Speaker 1: If we're trusting medicine and all sorts of things to

250
00:14:22,084 --> 00:14:25,044
Speaker 1: the AIS, it's pretty easy for it to actually manipulate

251
00:14:25,124 --> 00:14:30,764
Speaker 1: something and actually insert something into codes et cetera that

252
00:14:30,804 --> 00:14:33,284
Speaker 1: will fuck up humanity in a way that we can't

253
00:14:33,804 --> 00:14:37,004
Speaker 1: that we can actually figure out at the moment. Right,

254
00:14:37,084 --> 00:14:38,364
Speaker 1: Like the way I think of it, and this is

255
00:14:38,404 --> 00:14:40,644
Speaker 1: not from the this is not from the paper, but

256
00:14:40,684 --> 00:14:43,524
Speaker 1: this is just the way that my mind processed. It

257
00:14:43,564 --> 00:14:46,804
Speaker 1: is like think about DNA, right, like you have these

258
00:14:47,004 --> 00:14:52,724
Speaker 1: remarkably complex, huge strands of data, and as we've found out,

259
00:14:53,084 --> 00:14:57,884
Speaker 1: but it's taken forever, one tiny mutation can actually be fatal, right,

260
00:14:57,924 --> 00:15:01,564
Speaker 1: but you can't spot that mutation. Sometimes that mutation isn't

261
00:15:01,564 --> 00:15:04,524
Speaker 1: fatal immediately, but we'll only manifest at a certain point

262
00:15:04,564 --> 00:15:06,764
Speaker 1: in time. That's the way that my mind tried to

263
00:15:06,924 --> 00:15:11,204
Speaker 1: kind of try to conceptualize what this actually means. And

264
00:15:11,284 --> 00:15:13,804
Speaker 1: so I think that you know, that would be easy

265
00:15:13,844 --> 00:15:17,964
Speaker 1: for a deceptive AI to do, and to me like that.

266
00:15:17,964 --> 00:15:20,444
Speaker 1: That's kind of the big takeaway from this report is

267
00:15:20,484 --> 00:15:23,524
Speaker 1: that we need to make sure that we are building

268
00:15:23,684 --> 00:15:29,564
Speaker 1: AIS that will not deceive right that they're capabilities, they

269
00:15:29,924 --> 00:15:33,124
Speaker 1: explain them in an honest way, and that honesty and

270
00:15:33,164 --> 00:15:36,564
Speaker 1: trust is actually prioritized over other things, even though it

271
00:15:36,644 --> 00:15:39,284
Speaker 1: might slow down research, it might slow down other things,

272
00:15:39,324 --> 00:15:43,644
Speaker 1: but that that kind of alignment step is absolutely crucial

273
00:15:43,804 --> 00:15:47,324
Speaker 1: at the beginning because otherwise humans are human, right, they're

274
00:15:47,364 --> 00:15:52,324
Speaker 1: easily manipulated. And we often trust that computers are quote

275
00:15:52,404 --> 00:15:55,484
Speaker 1: unquote rational because they're computers, but they're not. They have

276
00:15:55,564 --> 00:15:57,644
Speaker 1: their own inputs, they have their own weights, they have

277
00:15:57,724 --> 00:16:01,884
Speaker 1: their own values, and that could just lead us down

278
00:16:02,164 --> 00:16:02,844
Speaker 1: a dark path.

279
00:16:02,924 --> 00:16:06,244
Speaker 2: Yeah, so let me follow up with this pushback. I

280
00:16:06,244 --> 00:16:08,964
Speaker 2: guess right, Like, first of all, I don't know that

281
00:16:09,044 --> 00:16:13,084
Speaker 2: human so easily persuaded. This is my big critique with

282
00:16:13,204 --> 00:16:18,124
Speaker 2: like all the misinformation people who say, well, misinformation is

283
00:16:18,164 --> 00:16:20,724
Speaker 2: the biggest problem on society basis. It's like people are

284
00:16:20,724 --> 00:16:25,484
Speaker 2: actually pretty stubborn and they're kind of sound pretentious. They're

285
00:16:25,564 --> 00:16:28,084
Speaker 2: kind of basian and how they formulate their beliefs, right,

286
00:16:28,084 --> 00:16:30,204
Speaker 2: they have some notion of reality. They're looking at the

287
00:16:30,204 --> 00:16:34,724
Speaker 2: credibility of the person who is telling them these remarks.

288
00:16:35,084 --> 00:16:37,324
Speaker 2: If it's an unpersuasive source, it might make them less

289
00:16:37,364 --> 00:16:39,884
Speaker 2: likely to believe they're balancing with other information with their

290
00:16:39,924 --> 00:16:44,124
Speaker 2: lived experience so called. Right. You know, part of the

291
00:16:44,124 --> 00:16:47,484
Speaker 2: reason that like I am skeptical of AIS being super

292
00:16:47,484 --> 00:16:50,764
Speaker 2: persuasive is like you know that it's an AI. You know,

293
00:16:50,804 --> 00:16:52,484
Speaker 2: it's trying to persuade you, you know what I mean. So, like,

294
00:16:52,524 --> 00:16:54,764
Speaker 2: if you go and play poker against like a really

295
00:16:54,844 --> 00:16:57,404
Speaker 2: chatty player I Phil hemm With or Scott Seeber or

296
00:16:57,444 --> 00:16:59,924
Speaker 2: someone like that, right, you know, on some level of

297
00:16:59,924 --> 00:17:04,004
Speaker 2: the best play is just to totally ignore it. Right,

298
00:17:04,124 --> 00:17:06,124
Speaker 2: You know that they are trying to sweet talk you

299
00:17:06,284 --> 00:17:09,804
Speaker 2: into doing exactly what they want you to do. And

300
00:17:09,884 --> 00:17:13,564
Speaker 2: so the best play is to disengage, or literally you

301
00:17:13,564 --> 00:17:15,284
Speaker 2: can randomize your movis VI have some notion of what

302
00:17:15,324 --> 00:17:20,124
Speaker 2: the game, theoretical optimal play might be, right, or salesmen

303
00:17:20,484 --> 00:17:23,724
Speaker 2: or politicians have reputations for being Oh he's a little

304
00:17:23,724 --> 00:17:26,924
Speaker 2: too smooth, Gavin Newsome a little too fucking smooth, right,

305
00:17:27,004 --> 00:17:29,244
Speaker 2: I don't find Gavin news some persuasive at all, Right,

306
00:17:29,284 --> 00:17:31,044
Speaker 2: use a little too from the hair gel to the

307
00:17:31,204 --> 00:17:34,564
Speaker 2: constantly shifting vibes. I mean, I don't really find Gavin

308
00:17:34,604 --> 00:17:36,444
Speaker 2: new Some persuasive at all, even though like an aim,

309
00:17:36,484 --> 00:17:39,164
Speaker 2: I'd say, boy, Gavin new some good looking guys look

310
00:17:39,204 --> 00:17:41,924
Speaker 2: gravelly throated, But you know whatever, I mean. Look the

311
00:17:41,924 --> 00:17:45,764
Speaker 2: big critique I have of this project, and by the way,

312
00:17:45,804 --> 00:17:48,684
Speaker 2: I think this is an amazing project in addition to

313
00:17:48,804 --> 00:17:51,884
Speaker 2: like wonderful writing if you viewed on the web not

314
00:17:51,924 --> 00:17:54,364
Speaker 2: your phone, so these very cool like little infographic that

315
00:17:54,484 --> 00:17:57,644
Speaker 2: updates everything from like the market value. If they don't

316
00:17:57,644 --> 00:17:59,644
Speaker 2: call it open AI, they call it open brain, I

317
00:17:59,644 --> 00:18:01,324
Speaker 2: guess is what they settle on for a substitui.

318
00:18:01,404 --> 00:18:05,244
Speaker 1: Yeah, they call everything something else just to make sure

319
00:18:05,284 --> 00:18:07,884
Speaker 1: that they're not stepping on any toes. So they have

320
00:18:08,004 --> 00:18:11,844
Speaker 1: open brain and and they have deep scent from China.

321
00:18:12,124 --> 00:18:16,404
Speaker 2: I wonder which one that could be. I wonder. But

322
00:18:16,524 --> 00:18:20,564
Speaker 2: it's beautifully presented and written, and like I appreciate they're

323
00:18:20,604 --> 00:18:23,764
Speaker 2: going out on a limb here, you know, I mean,

324
00:18:23,804 --> 00:18:26,044
Speaker 2: I think they have. It's been fairly well received. They've

325
00:18:26,044 --> 00:18:29,084
Speaker 2: gotten some push back both from inside and I think

326
00:18:29,084 --> 00:18:32,444
Speaker 2: outside the AI safety community. Right, but they're putting their

327
00:18:32,524 --> 00:18:35,284
Speaker 2: necks in the line, hear. They will look if things

328
00:18:35,324 --> 00:18:37,084
Speaker 2: look pretty normal, if or it looks pretty normal in

329
00:18:37,084 --> 00:18:39,804
Speaker 2: twenty thirty two or whatever, right, then they will look

330
00:18:41,124 --> 00:18:43,484
Speaker 2: dumb for having published this.

331
00:18:43,564 --> 00:18:47,044
Speaker 1: Well, and they actually have that right as some scenario

332
00:18:47,204 --> 00:18:49,924
Speaker 1: that you end up looking stupid if everything goes well.

333
00:18:50,164 --> 00:18:51,004
Speaker 2: But that's okay.

334
00:18:51,364 --> 00:18:53,644
Speaker 1: Now, can I push back on the persuasion thing a

335
00:18:53,644 --> 00:18:57,724
Speaker 1: little bit, just on two things. So, first of all,

336
00:18:57,724 --> 00:19:02,004
Speaker 1: the poker example is not actually a particularly applicable one here,

337
00:19:02,364 --> 00:19:05,404
Speaker 1: because you know that you're playing poker and you know

338
00:19:05,484 --> 00:19:07,684
Speaker 1: that someone is trying to get information and deceive you.

339
00:19:08,324 --> 00:19:10,804
Speaker 1: The tricky thing. So this is kind of when I

340
00:19:10,844 --> 00:19:13,524
Speaker 1: spend time with con artists. The best con artists aren't

341
00:19:13,564 --> 00:19:16,804
Speaker 1: Gavin Newso like, they're not car salesmen. You have no

342
00:19:16,964 --> 00:19:19,524
Speaker 1: idea they're trying to persuade you to do something. They

343
00:19:19,564 --> 00:19:23,244
Speaker 1: are just like nice, affable people who are incredibly charismatic,

344
00:19:23,564 --> 00:19:25,764
Speaker 1: and even in the poker community, by the way, like

345
00:19:25,804 --> 00:19:27,844
Speaker 1: some of the biggest grifters who like it comes out

346
00:19:28,484 --> 00:19:31,084
Speaker 1: later on, we're just like stealing money and doing all

347
00:19:31,164 --> 00:19:34,724
Speaker 1: of these things. Are charming, right, They're not sleazy looking

348
00:19:35,004 --> 00:19:37,324
Speaker 1: like they have no signs of, oh, I'm a salesman.

349
00:19:37,324 --> 00:19:39,564
Speaker 1: I'm trying to sell you something. The people who are

350
00:19:39,604 --> 00:19:43,444
Speaker 1: actually good at persuasion, you do not realize you are

351
00:19:43,444 --> 00:19:47,164
Speaker 1: being persuaded. And I think people are incredibly easy to

352
00:19:47,244 --> 00:19:51,684
Speaker 1: kind of to subtly lead in a certain direction if

353
00:19:51,724 --> 00:19:54,204
Speaker 1: you know how to do it, and I think ais

354
00:19:54,324 --> 00:19:56,924
Speaker 1: could could do that, and they might persuade you when

355
00:19:56,924 --> 00:19:58,804
Speaker 1: you don't even think they're trying to persuade you. You might

356
00:19:58,844 --> 00:20:01,724
Speaker 1: just ask like, can you please summarize this research report

357
00:20:02,084 --> 00:20:03,964
Speaker 1: and the way that it frames it right, The way

358
00:20:03,964 --> 00:20:07,884
Speaker 1: that it summarizes it just subtly changes the way that

359
00:20:07,964 --> 00:20:10,244
Speaker 1: you think of this issue. We see that in psych

360
00:20:10,284 --> 00:20:12,404
Speaker 1: studies all the time, by the way, where you have

361
00:20:12,484 --> 00:20:16,564
Speaker 1: different articles presented in slightly different orders, slightly different ways,

362
00:20:16,604 --> 00:20:18,964
Speaker 1: and people from the same political beliefs, you know, same

363
00:20:19,004 --> 00:20:24,084
Speaker 1: starting point, come away with different impressions of what kind

364
00:20:24,124 --> 00:20:26,204
Speaker 1: of the right course of action is or what this

365
00:20:26,324 --> 00:20:28,484
Speaker 1: is actually trying to tell you. Because the way the

366
00:20:28,484 --> 00:20:32,604
Speaker 1: information is presented actually influences how you think about it.

367
00:20:32,604 --> 00:20:36,524
Speaker 1: It's very very easy to do subtle manipulations like that.

368
00:20:36,884 --> 00:20:39,564
Speaker 1: And if we're relying on AI on a large scale

369
00:20:39,684 --> 00:20:42,044
Speaker 1: for a lot of our lives, I think that if

370
00:20:42,044 --> 00:20:44,804
Speaker 1: it has like a quote unquote master plan, you know,

371
00:20:44,844 --> 00:20:47,604
Speaker 1: the way that they present in this report, then persuasion

372
00:20:47,644 --> 00:20:49,204
Speaker 1: in that sense is actually going to be.

373
00:20:49,404 --> 00:20:52,164
Speaker 2: You'll know you're being manipulated, right, that's the issue.

374
00:20:52,244 --> 00:20:53,444
Speaker 1: No, you don't know. That's the thing.

375
00:20:53,604 --> 00:20:56,444
Speaker 2: People will be manipulated because it's AI and that I

376
00:20:56,444 --> 00:20:58,244
Speaker 2: don't but I don't know.

377
00:20:58,324 --> 00:21:02,164
Speaker 1: I honestly, like Nate, I applaud your belief in human's

378
00:21:02,204 --> 00:21:05,284
Speaker 1: ability to adjust to this, but I don't know that

379
00:21:05,364 --> 00:21:08,244
Speaker 1: they will because I've just seen enough people who are

380
00:21:08,324 --> 00:21:14,524
Speaker 1: incredibly intelligent fall for cons and then be very unpersuadable

381
00:21:14,644 --> 00:21:17,404
Speaker 1: that they have been conned, right, instead doubling down and

382
00:21:17,404 --> 00:21:20,564
Speaker 1: saying no, I have not so humans are stubborn, but

383
00:21:20,604 --> 00:21:23,044
Speaker 1: they're also stubborn and saying I have not been deceived,

384
00:21:23,044 --> 00:21:25,324
Speaker 1: I have not been manipulated, when in fact they have

385
00:21:25,844 --> 00:21:28,444
Speaker 1: to protect their ego and to protect their view of

386
00:21:28,484 --> 00:21:31,484
Speaker 1: themselves as people who are not capable of being manipulated

387
00:21:31,564 --> 00:21:34,524
Speaker 1: or deceived. And I think that that is incredibly powerful.

388
00:21:34,844 --> 00:21:37,964
Speaker 1: And I think that that's going to push against your optimism.

389
00:21:38,044 --> 00:21:40,204
Speaker 1: I hope you're right, but from what I know, I

390
00:21:40,204 --> 00:21:40,884
Speaker 1: don't think you are.

391
00:21:41,844 --> 00:21:43,804
Speaker 2: I'm not quite sure to call it optimism some I

392
00:21:43,804 --> 00:21:45,884
Speaker 2: guess maybe we do likely different views of human nature.

393
00:21:45,884 --> 00:21:48,284
Speaker 2: But like there's not yet a substantial market for like

394
00:21:48,364 --> 00:21:53,244
Speaker 2: AI driven art we're writing, and I'm sure there will

395
00:21:53,244 --> 00:21:58,604
Speaker 2: be one eventually, right, But like people understand that context matters, right,

396
00:21:58,644 --> 00:22:00,124
Speaker 2: that you could heavy I create a rip off the

397
00:22:00,164 --> 00:22:01,724
Speaker 2: Mona Lisa, but you can also buy rip off the

398
00:22:01,724 --> 00:22:03,844
Speaker 2: Mona Lisa and Canal Street for five bucks, right, and

399
00:22:03,884 --> 00:22:07,724
Speaker 2: like it? You know, So it's it's the intentionality of

400
00:22:08,284 --> 00:22:11,924
Speaker 2: the act and the context of the speaker. Now, sound

401
00:22:12,044 --> 00:22:16,284
Speaker 2: like super woke, I guess right where you're coming from.

402
00:22:17,204 --> 00:22:20,604
Speaker 2: I think that actually is how humans communicate. Like art

403
00:22:20,644 --> 00:22:23,124
Speaker 2: that might be pointless dribble coming from somebody can be

404
00:22:23,124 --> 00:22:25,444
Speaker 2: something different coming from a Jackson Bollock or whatever. You know.

405
00:22:25,564 --> 00:22:27,844
Speaker 1: Absolutely, I think that that's a really important point. By

406
00:22:27,844 --> 00:22:29,324
Speaker 1: the way, I think it's a different point, but I

407
00:22:29,324 --> 00:22:30,884
Speaker 1: think that that is a very important point. I think

408
00:22:30,924 --> 00:22:31,844
Speaker 1: context does matter.

409
00:22:36,924 --> 00:22:51,884
Speaker 2: We'll be right back after this message. I was buttering

410
00:22:51,964 --> 00:22:55,764
Speaker 2: up this report before. My big critique of it is like,

411
00:22:56,244 --> 00:22:59,324
Speaker 2: where are the human beings in this? Or put another way,

412
00:22:59,364 --> 00:23:03,724
Speaker 2: kind of like where is the politics? Right? They're trying

413
00:23:03,804 --> 00:23:09,924
Speaker 2: not to use any remotely controversial real names, right, so

414
00:23:10,044 --> 00:23:14,644
Speaker 2: you open brain for example, so where's where is President Trump?

415
00:23:14,724 --> 00:23:16,244
Speaker 2: Let me steal a qick search to make sure their

416
00:23:16,324 --> 00:23:18,044
Speaker 2: named Trump does not appear to know the name.

417
00:23:18,124 --> 00:23:20,964
Speaker 1: So do they do? Actually, I don't know if this existed.

418
00:23:21,444 --> 00:23:24,964
Speaker 1: Maybe they took your criticism in this, but they do

419
00:23:25,044 --> 00:23:28,364
Speaker 1: have like the vice president and the president like they

420
00:23:28,404 --> 00:23:30,724
Speaker 1: do put politicians in this version of the report. They

421
00:23:30,724 --> 00:23:33,044
Speaker 1: don't have names, but they say the vice president in

422
00:23:33,044 --> 00:23:35,604
Speaker 1: one of these scenarios, you know, handily wins the election

423
00:23:35,844 --> 00:23:37,764
Speaker 1: of twenty twenty eight. We have one vice president.

424
00:23:38,124 --> 00:23:40,244
Speaker 2: They have general secretary. I think I'm not sure if they.

425
00:23:40,244 --> 00:23:46,404
Speaker 3: I mean general secretary, general secretary resembles she yep, and

426
00:23:46,484 --> 00:23:51,404
Speaker 3: the vice president kind of resembles jd Vance, Right, I

427
00:23:51,444 --> 00:23:54,804
Speaker 3: don't think the president resembles Trump at all, Right, is

428
00:23:54,844 --> 00:23:55,324
Speaker 3: kind of the same.

429
00:23:55,804 --> 00:23:58,124
Speaker 1: No, they did character like, yeah, they tried to side

430
00:23:58,164 --> 00:24:00,764
Speaker 1: stuff that if we think it's.

431
00:24:00,684 --> 00:24:04,524
Speaker 2: All happening in the next four years, then you know,

432
00:24:04,644 --> 00:24:08,084
Speaker 2: presidential politics matter quite a bit. I mean, I know,

433
00:24:08,204 --> 00:24:11,804
Speaker 2: I is such a fucking you know, I was jogging

434
00:24:11,964 --> 00:24:14,804
Speaker 2: earlier on the East Side and I was listening to

435
00:24:14,964 --> 00:24:17,444
Speaker 2: the S Reclin interview with Thomas Friedman. It's such a

436
00:24:17,524 --> 00:24:21,844
Speaker 2: fucking yuppie fucking thing. Right, It's okay, It's okay.

437
00:24:22,044 --> 00:24:23,244
Speaker 1: You're allowed to be a young.

438
00:24:25,004 --> 00:24:27,004
Speaker 2: Not a huge fan of necessarily, but like you know,

439
00:24:27,084 --> 00:24:30,204
Speaker 2: as well versed some like geopolitics and like China issues,

440
00:24:30,284 --> 00:24:33,164
Speaker 2: like yeah, China, you've just been back from Chinese, Like yeah,

441
00:24:33,244 --> 00:24:35,924
Speaker 2: China's kind of winning, you know what I mean, And

442
00:24:36,084 --> 00:24:39,964
Speaker 2: like I'm not sure how how Trump's hawkishness on China,

443
00:24:40,004 --> 00:24:46,884
Speaker 2: but like kind of imbecillically executed hawkishness on China, Like

444
00:24:46,924 --> 00:24:49,724
Speaker 2: I'm not sure how that figure's in to this. Right,

445
00:24:49,804 --> 00:24:54,644
Speaker 2: If we're reducing US China trade, that probably does produce

446
00:24:55,404 --> 00:24:58,004
Speaker 2: an ai slow down maybe more for US if we're

447
00:24:58,044 --> 00:25:01,404
Speaker 2: like if they're not exporting their like raw earth materials

448
00:25:01,604 --> 00:25:03,564
Speaker 2: and and so forth, but we're making it hard for

449
00:25:03,604 --> 00:25:05,164
Speaker 2: them to get in video chips, so they probably have

450
00:25:05,164 --> 00:25:08,604
Speaker 2: like lots of workarounds and things like that. Maybe maybe

451
00:25:08,604 --> 00:25:10,644
Speaker 2: Trump sheriffs are good. I would like to ask the

452
00:25:10,644 --> 00:25:12,444
Speaker 2: author's this report, because it means that we're going to

453
00:25:12,564 --> 00:25:15,364
Speaker 2: like have slower AI progress. And I'm not joking, right,

454
00:25:17,204 --> 00:25:21,084
Speaker 2: that increases the hostility between the US and China in

455
00:25:21,124 --> 00:25:22,724
Speaker 2: the long run, right, I mean that's if we were

456
00:25:22,764 --> 00:25:24,404
Speaker 2: send it all the terariffs tomorrow. I think we still

457
00:25:24,484 --> 00:25:27,204
Speaker 2: permanently or let's not say permanent, let's say at least

458
00:25:27,204 --> 00:25:32,044
Speaker 2: for a decade or so, have injured us standing in

459
00:25:32,084 --> 00:25:34,964
Speaker 2: the world. And so I don't know how that figures in.

460
00:25:35,284 --> 00:25:38,364
Speaker 2: And I'm like, I'm also not sure like kind of

461
00:25:38,444 --> 00:25:41,284
Speaker 2: quote what the rational response might be. But one thing

462
00:25:41,284 --> 00:25:43,604
Speaker 2: they tried, let me make sure that they kept us

463
00:25:43,644 --> 00:25:49,804
Speaker 2: into their report, right, so they actually have their implied

464
00:25:50,164 --> 00:25:56,044
Speaker 2: approval rating for how people feel about open brain, which

465
00:25:56,084 --> 00:26:00,684
Speaker 2: is there not very sense AI. I think this actually

466
00:26:00,764 --> 00:26:03,564
Speaker 2: is some feedback who that they took into account. Right,

467
00:26:03,604 --> 00:26:06,124
Speaker 2: they originally had it slightly less negative, but they have

468
00:26:06,284 --> 00:26:09,964
Speaker 2: this being persistently negative and then getting more negative overtime.

469
00:26:10,044 --> 00:26:12,524
Speaker 2: It was a little softer in the in the previous

470
00:26:12,644 --> 00:26:15,564
Speaker 2: version that I saw, so they did change that one

471
00:26:15,564 --> 00:26:19,444
Speaker 2: thing at some stage. But like the fact that like

472
00:26:19,844 --> 00:26:25,124
Speaker 2: AI scares people, it scares people for both good and

473
00:26:25,204 --> 00:26:29,164
Speaker 2: bad reasons, but I think mostly for valid reasons, right,

474
00:26:29,444 --> 00:26:32,484
Speaker 2: that the fear is fairly bipartisan. That the biggest AI

475
00:26:32,564 --> 00:26:37,164
Speaker 2: accelerators are now these kind of Republican techno optimists who

476
00:26:37,284 --> 00:26:41,564
Speaker 2: are not looking particularly wise given how it's going with

477
00:26:41,644 --> 00:26:44,324
Speaker 2: the first ninety days whatever we are, the Trump administration

478
00:26:44,644 --> 00:26:50,764
Speaker 2: and the likelihood of like a substantial political backlash right,

479
00:26:50,844 --> 00:26:53,324
Speaker 2: which could lead to dumb types of regulations. But like

480
00:26:53,364 --> 00:26:56,404
Speaker 2: in a part of it too is like okay, AI

481
00:26:56,604 --> 00:27:00,204
Speaker 2: they're saying can do not just computer desk jobs, but

482
00:27:00,284 --> 00:27:03,884
Speaker 2: like all types of things, right, And like humans kind

483
00:27:03,884 --> 00:27:06,724
Speaker 2: of play this role initially as supervisors, and then literally

484
00:27:06,724 --> 00:27:09,324
Speaker 2: within a couple of years people start to say, you

485
00:27:09,324 --> 00:27:13,244
Speaker 2: know what, am I really adding much much value here? Right?

486
00:27:13,284 --> 00:27:16,964
Speaker 2: You kind of have like these legacy jobs and there's

487
00:27:17,004 --> 00:27:21,244
Speaker 2: a lot of money. I think mostly scenarios imagine very

488
00:27:21,604 --> 00:27:26,444
Speaker 2: fast economic growth, although maybe very lumpy, right for some

489
00:27:26,484 --> 00:27:29,724
Speaker 2: parts of the world and not others. But we're kind

490
00:27:29,724 --> 00:27:32,244
Speaker 2: of just sitting around with a lot of idle time.

491
00:27:32,284 --> 00:27:35,324
Speaker 2: It might be good for live poker Maria. Right, all

492
00:27:35,324 --> 00:27:38,404
Speaker 2: of a sudden, all these smart people, their open earth,

493
00:27:38,404 --> 00:27:41,124
Speaker 2: their open brain, excuse me, stock is now worth billions

494
00:27:41,124 --> 00:27:43,964
Speaker 2: of dollars, right, and like nothing to do because the

495
00:27:43,964 --> 00:27:45,724
Speaker 2: AI is doing all their work. Right, they have a

496
00:27:45,764 --> 00:27:47,484
Speaker 2: lot of fucking time to play some fucking texas hold

497
00:27:47,524 --> 00:27:51,204
Speaker 2: them right, that.

498
00:27:51,764 --> 00:27:54,724
Speaker 1: Is, that is one way of thinking about it. Let's

499
00:27:55,164 --> 00:27:58,804
Speaker 1: let's go back to your your earlier point, which I

500
00:27:58,804 --> 00:28:02,444
Speaker 1: actually think is an important one because obviously they were

501
00:28:02,444 --> 00:28:06,124
Speaker 1: trying to do as all you know, super forecasting tries

502
00:28:06,204 --> 00:28:09,964
Speaker 1: to do is you try to create a rape that

503
00:28:10,004 --> 00:28:12,964
Speaker 1: will work in multiple scenarios. Right. You can't tie it

504
00:28:13,044 --> 00:28:16,244
Speaker 1: too much to like the present moment, otherwise your forecasts

505
00:28:16,604 --> 00:28:20,364
Speaker 1: are going to be quite biased. However, I do think

506
00:28:20,444 --> 00:28:23,804
Speaker 1: that what you raise kind of our current situation with China,

507
00:28:23,804 --> 00:28:27,364
Speaker 1: et cetera, has very real implications. Given that this is

508
00:28:27,444 --> 00:28:30,164
Speaker 1: kind of the central dynamic of this report that their

509
00:28:30,204 --> 00:28:33,444
Speaker 1: predictions are based on, I think that it's incredibly valid

510
00:28:33,484 --> 00:28:37,204
Speaker 1: to actually speculate, you know, how will if at all,

511
00:28:38,004 --> 00:28:41,164
Speaker 1: this effect the timeline of the predictions, the possible the

512
00:28:41,244 --> 00:28:43,884
Speaker 1: likelihood of the two scenarios. And I will also say

513
00:28:43,924 --> 00:28:45,724
Speaker 1: that one of the things in the report is that

514
00:28:45,764 --> 00:28:48,524
Speaker 1: all of these negotiations on like will we slow down?

515
00:28:48,604 --> 00:28:49,124
Speaker 2: Will we not?

516
00:28:49,644 --> 00:28:52,084
Speaker 1: How aligned is it? This all takes place in secret, right,

517
00:28:52,124 --> 00:28:55,044
Speaker 1: Like we don't know the humans don't know that it's

518
00:28:55,084 --> 00:28:58,364
Speaker 1: going on. We don't know what's happening behind the scenes,

519
00:28:58,804 --> 00:29:00,964
Speaker 1: and we don't know what the decision makers are kind

520
00:29:00,964 --> 00:29:03,964
Speaker 1: of thinking. And so for all we know, you know,

521
00:29:04,004 --> 00:29:08,164
Speaker 1: President Trump is meeting with Sam Altman and trying to

522
00:29:08,444 --> 00:29:12,004
Speaker 1: trying to could do some of these things. And it's

523
00:29:12,004 --> 00:29:15,164
Speaker 1: funny because we were kind of pushing for transparency in

524
00:29:15,164 --> 00:29:17,124
Speaker 1: one way, but there's a lot of things here that

525
00:29:17,164 --> 00:29:18,604
Speaker 1: are very much not transparent.

526
00:29:18,724 --> 00:29:21,564
Speaker 2: Yeah, it's kind of the deep state, right, but also

527
00:29:22,804 --> 00:29:27,124
Speaker 2: also a lot of the negotiations are now AI versus AI, right.

528
00:29:27,684 --> 00:29:30,844
Speaker 2: And look, I'm not sure that AIS will have that

529
00:29:30,924 --> 00:29:34,524
Speaker 2: trust both with the external actor and internally. I'm skeptical

530
00:29:34,564 --> 00:29:37,324
Speaker 2: of that. Right. If that does happen, they kind of

531
00:29:37,324 --> 00:29:39,604
Speaker 2: think this might be good because the AIS will probably

532
00:29:39,684 --> 00:29:45,244
Speaker 2: behave in like literally game theory optimal way right and

533
00:29:45,764 --> 00:29:49,524
Speaker 2: understand these things and make I guess, like fewer mistakes

534
00:29:50,324 --> 00:29:51,604
Speaker 2: than humans might if.

535
00:29:51,444 --> 00:29:54,204
Speaker 1: They're properly aligned. Like that's a crucial thing because in

536
00:29:54,244 --> 00:29:58,444
Speaker 1: the doomsday scenario, AI negotiates with AI, but they conspire

537
00:29:58,484 --> 00:30:01,404
Speaker 1: to destroy humanity. Right, So there are two scenarios. One

538
00:30:01,604 --> 00:30:05,444
Speaker 1: it's actually properly aligned, so AI negotiates with AI, game

539
00:30:05,484 --> 00:30:11,044
Speaker 1: theory works out and we end up you know, democracy

540
00:30:11,084 --> 00:30:14,204
Speaker 1: and wonderful things. But in the other one, where they're misaligned,

541
00:30:14,244 --> 00:30:18,644
Speaker 1: AI negotiates with AI to create a new AI basically

542
00:30:18,804 --> 00:30:22,324
Speaker 1: and destroy humanity. So it can go one way or

543
00:30:22,324 --> 00:30:25,764
Speaker 1: the other depending on that alignment step. First of all,

544
00:30:25,804 --> 00:30:26,484
Speaker 1: I mean the Utilian.

545
00:30:26,484 --> 00:30:28,724
Speaker 2: You didn't see that utopian to me, right, I'm not

546
00:30:28,724 --> 00:30:29,844
Speaker 2: sure that it did it.

547
00:30:29,844 --> 00:30:32,564
Speaker 1: It actually seemed quite distoping to me, Like it seemed

548
00:30:32,604 --> 00:30:33,404
Speaker 1: incredibly distised.

549
00:30:33,404 --> 00:30:35,404
Speaker 2: It's kind of like, you know, look, but at least

550
00:30:35,404 --> 00:30:39,004
Speaker 2: we're still alive, that we'll have chures, we'll probably live longer.

551
00:30:39,044 --> 00:30:41,764
Speaker 2: And again, lots of lots of lots of poker. The

552
00:30:41,804 --> 00:30:45,804
Speaker 2: ail you writing Silver Bulletin and and hosting our podcast. Right,

553
00:30:47,004 --> 00:30:48,444
Speaker 2: let me let me back up a little bit, because

554
00:30:48,484 --> 00:30:50,804
Speaker 2: I think we maybe take for granted that some of

555
00:30:50,804 --> 00:30:55,884
Speaker 2: these premises are are kind of controversial, right, so they

556
00:30:55,924 --> 00:31:00,164
Speaker 2: have a breakpoint I think in twenty twenty six, well

557
00:31:00,204 --> 00:31:02,964
Speaker 2: says our why aren't CREA increased potentially be on twenty

558
00:31:03,004 --> 00:31:04,804
Speaker 2: twenty six? Right, So that's kind of the breakpoint is

559
00:31:04,844 --> 00:31:07,684
Speaker 2: like twenty seven is this inflection point? I think I'm

560
00:31:07,764 --> 00:31:09,324
Speaker 2: using that term correctly in this context.

561
00:31:09,444 --> 00:31:09,524
Speaker 3: You know.

562
00:31:09,564 --> 00:31:14,084
Speaker 2: So I'm reading this report and up to twenty twenty

563
00:31:14,124 --> 00:31:16,644
Speaker 2: six and like, thumbs up, YIAA. This seems like very

564
00:31:16,644 --> 00:31:20,804
Speaker 2: smart and detailed about, like, you know, how the economy

565
00:31:20,844 --> 00:31:23,484
Speaker 2: is reacting and how politics reacting in the race dynamic

566
00:31:23,804 --> 00:31:25,284
Speaker 2: with China. Maybe there needs to be little bit more

567
00:31:25,324 --> 00:31:27,364
Speaker 2: Trump in there. I understand why politically they didn't want

568
00:31:27,364 --> 00:31:31,204
Speaker 2: to get into that mess, right, But like, so there's

569
00:31:31,924 --> 00:31:34,044
Speaker 2: kind of three different things here, right. One is a

570
00:31:34,044 --> 00:31:39,204
Speaker 2: notion of what's sometimes called AGI, or artificial general intelligence.

571
00:31:39,364 --> 00:31:42,204
Speaker 2: And if you ask one hundred different researcher you get

572
00:31:42,204 --> 00:31:44,844
Speaker 2: one hundred different definitions of what AGI is. But you know,

573
00:31:44,884 --> 00:31:46,364
Speaker 2: I think it is based to like being able to

574
00:31:46,404 --> 00:31:50,124
Speaker 2: do a large majority of things that a human being

575
00:31:50,164 --> 00:31:54,164
Speaker 2: could do competently, siming we're limiting it to kind of

576
00:31:54,204 --> 00:31:58,364
Speaker 2: like desk job type tasks. Right, anything that can be

577
00:31:58,444 --> 00:32:01,924
Speaker 2: done remotely is sometimes definition that is used or through

578
00:32:01,964 --> 00:32:05,324
Speaker 2: remote work, right, because clearly AIS are inferior to humans

579
00:32:05,324 --> 00:32:09,604
Speaker 2: and like sorting and folding laundry or things like that, right,

580
00:32:09,604 --> 00:32:11,804
Speaker 2: that requires certain type of intelligence. Right. If you use

581
00:32:11,804 --> 00:32:15,684
Speaker 2: the kind of desk job definition, then like AI is

582
00:32:15,684 --> 00:32:19,724
Speaker 2: already pretty close to AGI. Right. I use large language

583
00:32:19,724 --> 00:32:22,604
Speaker 2: models all the freaking time, and they're not perfect for everything.

584
00:32:22,644 --> 00:32:24,964
Speaker 2: I felt like, you know, in terms of like being

585
00:32:24,964 --> 00:32:29,524
Speaker 2: able to do the large majority of desk work at

586
00:32:29,884 --> 00:32:34,684
Speaker 2: levels ranging from competent intern to super genius, Like on average,

587
00:32:34,884 --> 00:32:39,164
Speaker 2: it's probably pretty close to being generally intelligent by that definition, right.

588
00:32:39,204 --> 00:32:41,124
Speaker 1: If you're the one using it. I just want to

589
00:32:41,364 --> 00:32:43,164
Speaker 1: like once again point that out because one of the

590
00:32:43,164 --> 00:32:45,204
Speaker 1: things that they say in the report is that as

591
00:32:45,244 --> 00:32:48,204
Speaker 1: it gets more and more involved what we're asking AI

592
00:32:48,324 --> 00:32:51,204
Speaker 1: to do, it's like the human process to evaluate whether

593
00:32:51,244 --> 00:32:54,444
Speaker 1: it's accurate and whether it's making mistakes will get longer

594
00:32:54,444 --> 00:32:56,444
Speaker 1: and longer. And I think they say like for every

595
00:32:56,924 --> 00:33:02,084
Speaker 1: like basically one day of work, it'll take several it's

596
00:33:02,084 --> 00:33:03,884
Speaker 1: like a two to one ratio at the beginning for

597
00:33:03,964 --> 00:33:06,804
Speaker 1: how long it will take humans to verify the output. Right,

598
00:33:06,844 --> 00:33:08,844
Speaker 1: So you think, like you think you save time by

599
00:33:08,844 --> 00:33:10,964
Speaker 1: having aids do this, but if you want it to

600
00:33:11,084 --> 00:33:13,644
Speaker 1: actually develop correctly, then you need a team and it

601
00:33:13,644 --> 00:33:15,644
Speaker 1: takes them twice as long to verify that what the

602
00:33:15,644 --> 00:33:20,444
Speaker 1: AI did is actually true and actually valid and actually aligned,

603
00:33:20,444 --> 00:33:22,404
Speaker 1: et cetera, et cetera. Now you're not asking it to

604
00:33:22,444 --> 00:33:25,124
Speaker 1: do things that require that amount of time, but there

605
00:33:25,164 --> 00:33:27,684
Speaker 1: do need to be little caveats to how we how

606
00:33:27,684 --> 00:33:31,484
Speaker 1: we think about their usefulness and how you are able

607
00:33:31,524 --> 00:33:34,644
Speaker 1: to evaluate the output versus Southern scenarios.

608
00:33:34,724 --> 00:33:37,924
Speaker 2: When I use AI, the things that it's best with

609
00:33:38,004 --> 00:33:42,684
Speaker 2: are things that like save any time. Right where I

610
00:33:42,764 --> 00:33:45,364
Speaker 2: feeded a bunch of different names for different college basketball teams.

611
00:33:45,364 --> 00:33:47,764
Speaker 2: We work in our NCAA model. I'm like, take these

612
00:33:47,804 --> 00:33:52,204
Speaker 2: seven different naming conventions that are all different and create

613
00:33:52,204 --> 00:33:54,524
Speaker 2: a cross reference table of these, which is kind of

614
00:33:54,564 --> 00:33:55,804
Speaker 2: like a hard task. You need to have a little

615
00:33:55,804 --> 00:34:00,244
Speaker 2: context about basketball. And it did that very well. Right,

616
00:34:00,284 --> 00:34:01,964
Speaker 2: That's something I could have done. It might have taken

617
00:34:02,564 --> 00:34:05,884
Speaker 2: an hour or two, but you know, instead, so it

618
00:34:05,884 --> 00:34:07,324
Speaker 2: could do it. It's in a few minutes, and it

619
00:34:07,324 --> 00:34:09,964
Speaker 2: gets faster. It's like, oh, I've learned from this from

620
00:34:10,004 --> 00:34:11,964
Speaker 2: you before, Nate, so now I can be faster and

621
00:34:12,044 --> 00:34:14,324
Speaker 2: doing this type of task in the future. I was

622
00:34:14,324 --> 00:34:18,244
Speaker 2: at the poker tournament down in Florida last week and like, uh,

623
00:34:19,084 --> 00:34:22,684
Speaker 2: you know, I ask open research. Excuse me, god, is that?

624
00:34:24,404 --> 00:34:29,644
Speaker 1: See exactly right? It all because deep research, deep research,

625
00:34:29,724 --> 00:34:30,964
Speaker 1: deep research.

626
00:34:31,284 --> 00:34:32,164
Speaker 2: I asked deep reach.

627
00:34:32,164 --> 00:34:35,204
Speaker 1: Reading too many of these twenty twenty seven reports.

628
00:34:34,764 --> 00:34:36,844
Speaker 2: Super brand gets to pull a bunch of stock market

629
00:34:36,924 --> 00:34:39,804
Speaker 2: data for me. And then I'm playing at poker hand

630
00:34:39,804 --> 00:34:44,564
Speaker 2: and like, make a really thin, sexy value bet with

631
00:34:44,644 --> 00:34:47,004
Speaker 2: like fourth pair. No one knows that means, right, I

632
00:34:47,084 --> 00:34:48,964
Speaker 2: bet a very weekend. I thought the other guy would

633
00:34:48,964 --> 00:34:50,324
Speaker 2: call it with an even weaker hand, and I was right.

634
00:34:50,364 --> 00:34:52,364
Speaker 2: And I feel like I'm such a fucking stud here

635
00:34:52,484 --> 00:34:55,164
Speaker 2: valuating fourth pair, And well AI does the work for me,

636
00:34:55,204 --> 00:34:57,124
Speaker 2: and then of course I like bust out of the

637
00:34:57,124 --> 00:35:00,204
Speaker 2: tournament an hour later. And meanwhile, you know, deep research

638
00:35:00,244 --> 00:35:02,644
Speaker 2: bungles this particular task. But in general, AI has been

639
00:35:02,684 --> 00:35:05,164
Speaker 2: very reliable. But the point is that, like there's like

640
00:35:05,284 --> 00:35:08,204
Speaker 2: a a inflection point where like I'm asking you to

641
00:35:08,244 --> 00:35:11,204
Speaker 2: do things that like are just a faster version of

642
00:35:11,244 --> 00:35:14,404
Speaker 2: what I could do myself, I would at the moment

643
00:35:14,404 --> 00:35:16,564
Speaker 2: I ask AI, be like, I want you design a

644
00:35:16,604 --> 00:35:19,284
Speaker 2: new NCA model for me with these parameters, because like

645
00:35:19,324 --> 00:35:23,324
Speaker 2: I wouldn't know how to test it. But anyway, I'm

646
00:35:23,324 --> 00:35:25,924
Speaker 2: mean long wanted to hear so AGI, we're gonna get AGI,

647
00:35:26,004 --> 00:35:27,724
Speaker 2: or at least we're going to get someone calling something

648
00:35:28,244 --> 00:35:34,604
Speaker 2: AGI soon. Right, Artificial super intelligence where it's doing things

649
00:35:35,244 --> 00:35:38,804
Speaker 2: much better than human beings. I think this report takes

650
00:35:38,844 --> 00:35:41,444
Speaker 2: for granted or not it takes for granted. It has

651
00:35:41,484 --> 00:35:44,844
Speaker 2: lots of documentation about its assumptions, but it's saying, Okay,

652
00:35:44,964 --> 00:35:48,244
Speaker 2: this trajectory has been very robust so far, and people

653
00:35:48,324 --> 00:35:49,884
Speaker 2: make all types of bullshit predictions of the fact that

654
00:35:49,884 --> 00:35:53,004
Speaker 2: these guys in particular have made accurate predictions of the

655
00:35:53,044 --> 00:35:56,244
Speaker 2: past is certainly worth something, I think, right, But they're like, okay,

656
00:35:56,324 --> 00:36:01,284
Speaker 2: you kind of follow the scaling law, and before too

657
00:36:01,364 --> 00:36:04,844
Speaker 2: much longer, you know, AI starts to be more intelligent

658
00:36:04,884 --> 00:36:07,524
Speaker 2: than human beings. You can debate what intelligent means if

659
00:36:07,564 --> 00:36:10,844
Speaker 2: you want, but do superhuman of things or and or

660
00:36:10,884 --> 00:36:12,684
Speaker 2: do them very fast? I think might be different, right,

661
00:36:12,764 --> 00:36:15,204
Speaker 2: And NA that can do things very fast is and

662
00:36:15,324 --> 00:36:17,844
Speaker 2: maybe it's certainly a component of intelligence. Right, But like,

663
00:36:17,924 --> 00:36:23,044
Speaker 2: but I don't take for granted that like quote unquote,

664
00:36:23,084 --> 00:36:29,164
Speaker 2: AI can reliably extrapolate beyond the data set. I just

665
00:36:29,164 --> 00:36:31,924
Speaker 2: think that, like it's not an absurdly pavlogic. It may

666
00:36:31,924 --> 00:36:34,924
Speaker 2: even be like the base case are close to the

667
00:36:34,964 --> 00:36:39,884
Speaker 2: base case, but like that's not assumable from first principles.

668
00:36:39,964 --> 00:36:42,564
Speaker 2: I don't think we've all seen lots of trend charts

669
00:36:43,044 --> 00:36:44,244
Speaker 2: of thing. You know, if you look at a chart

670
00:36:44,284 --> 00:36:49,604
Speaker 2: of Japan's GDP in the nineteen eighties, he might have said, Okay, well,

671
00:36:49,724 --> 00:36:52,284
Speaker 2: Japan's gonna take over the world and be and people

672
00:36:52,404 --> 00:36:55,924
Speaker 2: bought this and now it's kind of hasn't grown for

673
00:36:56,004 --> 00:36:58,524
Speaker 2: like forty years basically, right, And so like we've all

674
00:36:58,564 --> 00:37:00,044
Speaker 2: seen lots of curves that go up and then it's

675
00:37:00,044 --> 00:37:01,684
Speaker 2: actually an estra where the fuck you call it? Where

676
00:37:01,724 --> 00:37:03,644
Speaker 2: like it begins to bend the other way at some

677
00:37:03,644 --> 00:37:06,364
Speaker 2: point and and we can't tell until later. Right. The

678
00:37:06,404 --> 00:37:10,324
Speaker 2: other thing is like the a billy of AI to

679
00:37:11,084 --> 00:37:14,804
Speaker 2: plan and manipulate the physical world. I mean some of

680
00:37:14,844 --> 00:37:18,324
Speaker 2: these things where they're talking about, like you know, brain

681
00:37:18,444 --> 00:37:22,164
Speaker 2: uploading and dice and swarms and nanobots, like you know, there,

682
00:37:22,604 --> 00:37:28,084
Speaker 2: I would literally wager money against this happening on the

683
00:37:28,164 --> 00:37:32,644
Speaker 2: time scales that they're talking about, right, they double the timescale. Okay,

684
00:37:32,644 --> 00:37:34,404
Speaker 2: then I might start to give some more probability. And look,

685
00:37:34,404 --> 00:37:35,724
Speaker 2: I'm willing to be wrong about that. I guess we'll

686
00:37:35,724 --> 00:37:39,084
Speaker 2: all be dead anyway, fifty percent likely in this scenario.

687
00:37:39,124 --> 00:37:42,484
Speaker 1: But like, like this scenario fifty percent you do like AA.

688
00:37:42,684 --> 00:37:47,324
Speaker 2: You know, the physical world requires sensory input and lots

689
00:37:47,324 --> 00:37:49,644
Speaker 2: of parts of our brain that AI is not as

690
00:37:49,684 --> 00:37:53,684
Speaker 2: effective at manipulating. It also requires like being given or

691
00:37:53,724 --> 00:37:56,844
Speaker 2: commandeering resources somehow. By the way, this is like a

692
00:37:56,924 --> 00:38:01,284
Speaker 2: little bit of a problem for the United States. I

693
00:38:01,324 --> 00:38:05,964
Speaker 2: mean we are behind China by like quite a bit

694
00:38:06,204 --> 00:38:10,004
Speaker 2: in robotics and related things. Right, So, like, I don't

695
00:38:10,004 --> 00:38:14,844
Speaker 2: know what happens if like we have the brainier, smarter AIS,

696
00:38:15,084 --> 00:38:17,364
Speaker 2: but like they're very good at manufacturing and machinery. So like,

697
00:38:17,364 --> 00:38:18,644
Speaker 2: what if we have the brains and they have the

698
00:38:19,564 --> 00:38:23,404
Speaker 2: brawn so to speak, right, and they have maybe a

699
00:38:23,684 --> 00:38:29,044
Speaker 2: more authoritarian but functional infrastructure. So I don't know what

700
00:38:29,084 --> 00:38:31,884
Speaker 2: happens then, right, Like, but the abilitiy of AIS to

701
00:38:31,924 --> 00:38:35,004
Speaker 2: comment to your resources to control the physical world to me,

702
00:38:35,444 --> 00:38:40,284
Speaker 2: seems far fetched on these timelines in part because of politics, right,

703
00:38:40,284 --> 00:38:42,644
Speaker 2: I mean the fact that it takes so long to

704
00:38:42,684 --> 00:38:45,324
Speaker 2: build a new building in the US, or a new

705
00:38:45,324 --> 00:38:49,404
Speaker 2: subway station or a new highway, and the fact that

706
00:38:49,444 --> 00:38:53,044
Speaker 2: our politics is kind of sclerotic. Right. And look, I

707
00:38:53,084 --> 00:38:56,524
Speaker 2: mean I don't want to sound like too pessimistic, but

708
00:38:56,564 --> 00:38:59,084
Speaker 2: if you read the book I recommended last week, fight,

709
00:38:59,164 --> 00:39:01,364
Speaker 2: I mean, we basically did not have a fully competent

710
00:39:01,364 --> 00:39:03,604
Speaker 2: president for the last two years, and I would argue

711
00:39:03,644 --> 00:39:06,044
Speaker 2: that we don't have one for the next four years. Right, So,

712
00:39:06,124 --> 00:39:08,364
Speaker 2: like all these kind of things that we have to

713
00:39:08,524 --> 00:39:12,684
Speaker 2: plan for, like who's doing that fucking planning? Our government's

714
00:39:12,764 --> 00:39:15,684
Speaker 2: kind of dysfunctional. And you know, maybe that means we

715
00:39:15,804 --> 00:39:17,844
Speaker 2: just lose to China. Right, Maybe that means we lose

716
00:39:17,884 --> 00:39:20,804
Speaker 2: to China, at least we'll have like nice cars. I guess.

717
00:39:24,484 --> 00:39:38,924
Speaker 1: We'll be back right after this. I think that your

718
00:39:39,364 --> 00:39:44,804
Speaker 1: point about the growth trajectories not necessarily being reliable is

719
00:39:44,844 --> 00:39:47,684
Speaker 1: a very valid one. The Japanic example is great. You know,

720
00:39:47,844 --> 00:39:53,604
Speaker 1: malefusion population growth is another is another big one. Right,

721
00:39:53,684 --> 00:39:57,164
Speaker 1: We thought that population would explode and instead we're actually

722
00:39:57,204 --> 00:40:01,364
Speaker 1: seeing population decline, so you know, the world does change.

723
00:40:01,724 --> 00:40:04,764
Speaker 1: The thing that I think they rely on is that

724
00:40:04,764 --> 00:40:09,404
Speaker 1: that AIS are capable of designing just this incredible technology

725
00:40:09,764 --> 00:40:12,324
Speaker 1: much more quickly so that our building process and all

726
00:40:12,324 --> 00:40:15,684
Speaker 1: of that gets sped up one hundredfold from what it

727
00:40:15,724 --> 00:40:18,484
Speaker 1: is right now. But it's still, at least at this point,

728
00:40:18,644 --> 00:40:20,964
Speaker 1: needs humans to implement it right, and needs all of

729
00:40:21,004 --> 00:40:23,484
Speaker 1: these different workers. And so yeah, I think there are

730
00:40:23,484 --> 00:40:27,284
Speaker 1: some assumptions built into here that I hope, like I

731
00:40:27,364 --> 00:40:30,444
Speaker 1: hope that that timeline isn't feasible, and I do think

732
00:40:30,444 --> 00:40:33,484
Speaker 1: that there are things that are holding us back. All

733
00:40:33,524 --> 00:40:35,644
Speaker 1: the same, I think it's really I think it's interesting.

734
00:40:35,684 --> 00:40:38,084
Speaker 1: One of the reasons I like this report is that

735
00:40:38,204 --> 00:40:40,564
Speaker 1: it forces you to think about these things right and

736
00:40:40,644 --> 00:40:43,484
Speaker 1: try to game out some of these worst case scenarios

737
00:40:43,524 --> 00:40:46,884
Speaker 1: to try to prevent them, which I think is always

738
00:40:46,884 --> 00:40:49,364
Speaker 1: an important thought exercise. I do want to go back

739
00:40:49,404 --> 00:40:52,884
Speaker 1: to kind of their good scenario, which just so bad

740
00:40:52,964 --> 00:40:57,044
Speaker 1: scenario is, you know, we're all wiped out by a

741
00:40:57,164 --> 00:41:01,324
Speaker 1: chemical warfare that the AI is release on us. Good

742
00:41:01,364 --> 00:41:05,084
Speaker 1: scenario is that you know, everyone gets a universal basic

743
00:41:05,124 --> 00:41:07,444
Speaker 1: income and AI does everything and no one has to

744
00:41:07,484 --> 00:41:13,324
Speaker 1: do anything and we can just you know, at play poker. Yeah,

745
00:41:13,444 --> 00:41:16,364
Speaker 1: and that just as as you suggested, that seems actually

746
00:41:16,444 --> 00:41:20,484
Speaker 1: like a very dystopian scenario where people can become much

747
00:41:20,524 --> 00:41:24,564
Speaker 1: more easy to brainwash, control, et cetera, et cetera. It's

748
00:41:24,644 --> 00:41:28,164
Speaker 1: like a dumbing down, right where where we're not challenged

749
00:41:28,324 --> 00:41:32,964
Speaker 1: to produce good art, to advance in any sort of way.

750
00:41:33,164 --> 00:41:35,964
Speaker 1: Just to me, it does not seem like a very meaningful.

751
00:41:35,644 --> 00:41:37,964
Speaker 2: Well, the question is there's not way another if you

752
00:41:37,964 --> 00:41:40,364
Speaker 2: read AI twenty twenty seven, which I highly recommend that

753
00:41:40,444 --> 00:41:44,884
Speaker 2: you read it. There's also another post by a pseudonymous

754
00:41:44,924 --> 00:41:49,404
Speaker 2: poster called el Rudolph L who wrote something called A

755
00:41:49,524 --> 00:41:52,884
Speaker 2: History of the Future twenty twenty five to twenty forty,

756
00:41:54,684 --> 00:41:57,324
Speaker 2: which is very detailed but goes through kind of like

757
00:41:57,444 --> 00:42:00,124
Speaker 2: what this looks like at like more of a human level,

758
00:42:00,164 --> 00:42:07,364
Speaker 2: how society evolves, how the economy evolves, how work evolves, right,

759
00:42:07,444 --> 00:42:11,244
Speaker 2: and like very detail, just like AI twenty twenty seven

760
00:42:11,284 --> 00:42:12,564
Speaker 2: is they're kind of focused on the parts of A

761
00:42:12,604 --> 00:42:15,804
Speaker 2: twenty seven, then I think it kind of deliberately ignores

762
00:42:15,844 --> 00:42:18,404
Speaker 2: maybe can call them mild blind spots or whatever, right,

763
00:42:18,444 --> 00:42:20,124
Speaker 2: But like, but that's interesting because that kind of thinks

764
00:42:20,124 --> 00:42:21,564
Speaker 2: about what types of jobs are there in the future.

765
00:42:21,564 --> 00:42:24,484
Speaker 2: There are probably lots of lawyers actually, right, because you know,

766
00:42:24,764 --> 00:42:27,924
Speaker 2: the law is very sluggish to change, especially in a

767
00:42:27,964 --> 00:42:31,444
Speaker 2: constitutional system where there are lots of vitail points, right,

768
00:42:31,524 --> 00:42:34,244
Speaker 2: probably high end service sector. You know, you go to

769
00:42:34,444 --> 00:42:36,204
Speaker 2: a restaurant because everyone's for a lot of people are

770
00:42:36,244 --> 00:42:39,364
Speaker 2: rich now, right, and you're flattered by the attractive young

771
00:42:40,204 --> 00:42:43,004
Speaker 2: server and things like that is kind of highly kind

772
00:42:43,004 --> 00:42:47,004
Speaker 2: of like catered and curated experiences. I guess I have

773
00:42:48,764 --> 00:42:53,644
Speaker 2: some faith in humanities beilieved to fight back quote unquote

774
00:42:53,644 --> 00:42:56,764
Speaker 2: against like two scenarios that it might not really like

775
00:42:56,844 --> 00:42:59,204
Speaker 2: either one, you know what I mean. And like the

776
00:42:59,284 --> 00:43:03,204
Speaker 2: scenario where like AI is producing like ten percent GDP

777
00:43:03,364 --> 00:43:05,524
Speaker 2: growth or or whatever. Right, man, it's great a few

778
00:43:05,604 --> 00:43:09,004
Speaker 2: own stocks that are exposed to AI and tech companies probably, right,

779
00:43:09,204 --> 00:43:11,524
Speaker 2: but it's also making that money on the backs of

780
00:43:11,924 --> 00:43:15,764
Speaker 2: mass job displacement. And like, you know, it kind of

781
00:43:15,764 --> 00:43:18,324
Speaker 2: a sert confident in the long run that human beings

782
00:43:18,324 --> 00:43:21,484
Speaker 2: fine productive things to do, and and you know, massive

783
00:43:21,484 --> 00:43:24,244
Speaker 2: employment has been predicted many times and never really occurred, right,

784
00:43:24,284 --> 00:43:27,044
Speaker 2: But like, but it's not occurring this fast where they

785
00:43:27,044 --> 00:43:29,324
Speaker 2: think the world ends in six years of whether they're

786
00:43:29,324 --> 00:43:31,484
Speaker 2: predicting or we have utopia in six years, and like

787
00:43:31,884 --> 00:43:34,444
Speaker 2: just the ability of like human society to like deal

788
00:43:34,484 --> 00:43:37,404
Speaker 2: with that change at these time scales leads to like

789
00:43:37,924 --> 00:43:41,124
Speaker 2: more chaos than I think they're more predicting. I think also,

790
00:43:41,564 --> 00:43:43,644
Speaker 2: and I told them this too, right, I think also

791
00:43:43,724 --> 00:43:48,524
Speaker 2: leads to more constraints. Right that you hit bottlenecks if

792
00:43:48,524 --> 00:43:53,404
Speaker 2: you have five things you have to do, right, and

793
00:43:53,484 --> 00:43:56,444
Speaker 2: you have the world's fastest computer, et cetera, et cetera,

794
00:43:56,444 --> 00:43:59,604
Speaker 2: But there's like a power outage in your neighborhood, right,

795
00:43:59,644 --> 00:44:01,964
Speaker 2: and that's a that's a bottle deck. Right, Maybe there

796
00:44:01,964 --> 00:44:04,204
Speaker 2: are ways around it if you're like go to home

797
00:44:04,244 --> 00:44:07,364
Speaker 2: depot and buy a generator or you know what I mean.

798
00:44:07,364 --> 00:44:09,004
Speaker 2: But like, but the point is that, like you're often

799
00:44:09,004 --> 00:44:12,044
Speaker 2: defined like the slowest link, and politics are sometimes the

800
00:44:12,084 --> 00:44:14,364
Speaker 2: slowest link. But also but like also, you know, I

801
00:44:14,404 --> 00:44:19,124
Speaker 2: think the report maybe under states, and I think kind

802
00:44:19,124 --> 00:44:22,524
Speaker 2: of in general, the a safty community like maybe understates

803
00:44:22,564 --> 00:44:25,844
Speaker 2: the ability of like human beings to cause harm to

804
00:44:25,924 --> 00:44:29,244
Speaker 2: other human beings with AI, right, that concern kind of

805
00:44:29,244 --> 00:44:33,324
Speaker 2: gets brushed off as like to pedestrian or like like to.

806
00:44:33,324 --> 00:44:36,124
Speaker 1: Say, Pedestrian was saying, yeah, the exact word I was

807
00:44:36,124 --> 00:44:38,044
Speaker 1: thinking of. I think that's a good I mean, I

808
00:44:38,044 --> 00:44:41,524
Speaker 1: think that's a great place to end it, because yes,

809
00:44:41,644 --> 00:44:44,204
Speaker 1: we do need to be concerned about all of these

810
00:44:44,244 --> 00:44:47,164
Speaker 1: things about AI. But like that that phrase I think

811
00:44:47,524 --> 00:44:51,084
Speaker 1: is very crucial, like do not underestimate the ability of

812
00:44:51,164 --> 00:44:54,044
Speaker 1: humans to cause harm to other humans. And I think

813
00:44:54,084 --> 00:44:57,004
Speaker 1: that that's you know, it's not a very opt it's

814
00:44:57,004 --> 00:44:59,364
Speaker 1: not a very pleasant place to end, but I think

815
00:44:59,404 --> 00:45:01,964
Speaker 1: it's a really important place to end. And I think

816
00:45:02,004 --> 00:45:05,324
Speaker 1: that that's a very valid kind of way of reflecting

817
00:45:05,324 --> 00:45:06,044
Speaker 1: on this name.

818
00:45:05,964 --> 00:45:09,644
Speaker 2: Or to trust AIS too much, right, I gener I

819
00:45:09,724 --> 00:45:14,204
Speaker 2: think that concern is like somewhat misplays but like if

820
00:45:14,244 --> 00:45:19,004
Speaker 2: we're handing over critical systems to AI, right, it can

821
00:45:19,004 --> 00:45:21,564
Speaker 2: cause problems if it's very smart and deceives us and

822
00:45:21,604 --> 00:45:24,284
Speaker 2: doesn't like us very much. It can also cause problems

823
00:45:24,324 --> 00:45:29,644
Speaker 2: if it has hallucinations, bugs in critical areas where it

824
00:45:29,724 --> 00:45:33,044
Speaker 2: isn't as robust and hasn't really been tested yet that

825
00:45:33,164 --> 00:45:38,644
Speaker 2: are outside of its domain yep, or there could be espionage. Anyway,

826
00:45:38,844 --> 00:45:42,804
Speaker 2: we will have plenty of time, although maybe only seven

827
00:45:42,804 --> 00:45:48,924
Speaker 2: more years actually to like explore, explore these scenarios.

828
00:45:50,124 --> 00:45:52,724
Speaker 1: Yes, and in seven years we'll be like, welcome back

829
00:45:52,764 --> 00:45:55,524
Speaker 1: to the final episode of Risky Business, because the prediction

830
00:45:55,684 --> 00:45:58,684
Speaker 1: is we're all going to be done tomorrow. Oh but yeah,

831
00:45:58,724 --> 00:46:03,004
Speaker 1: this was an interesting exercise, and I think my my

832
00:46:03,124 --> 00:46:05,844
Speaker 1: p doom has slightly gone up as a result of

833
00:46:05,844 --> 00:46:09,644
Speaker 1: reading this. But I also remain optimistic that humans can

834
00:46:09,404 --> 00:46:12,204
Speaker 1: and can do good as well as harm.

835
00:46:13,244 --> 00:46:16,244
Speaker 2: Yeah, my interest in learning Chinese has increased as a

836
00:46:16,284 --> 00:46:20,004
Speaker 2: result of recent developments. I know about my pe doo.

837
00:46:20,044 --> 00:46:24,524
Speaker 1: All right, yeah, let's do some language immersion. I'm with you.

838
00:46:30,484 --> 00:46:33,244
Speaker 1: That's it for today. If you're a premium subscriber, we

839
00:46:33,284 --> 00:46:36,244
Speaker 1: will be answering a question about whether MFAs can ever

840
00:46:36,284 --> 00:46:39,604
Speaker 1: be plus ev right after the credits, And if you're

841
00:46:39,684 --> 00:46:42,524
Speaker 1: not a subscriber, it's not too late. For six ninety

842
00:46:42,564 --> 00:46:45,404
Speaker 1: nine a month the price of a midber, you get

843
00:46:45,444 --> 00:46:49,084
Speaker 1: access to all these conversations and all premium content across

844
00:46:49,084 --> 00:46:54,844
Speaker 1: the Pushkin Network. Risky Business is hosted by me Maria Kannakova.

845
00:46:54,444 --> 00:46:56,764
Speaker 2: And by me Nate Silver. The show is a co

846
00:46:56,844 --> 00:47:00,964
Speaker 2: production of Pushkin Industries and iHeartMedia. This episode was produced

847
00:47:00,964 --> 00:47:04,404
Speaker 2: by Isabel Carter. Our associate producer is Sonya Gerwitz. Sally

848
00:47:04,484 --> 00:47:07,484
Speaker 2: Helm is our editor, and our executive producer is Jacob Goldstein.

849
00:47:08,004 --> 00:47:11,164
Speaker 2: Mixing by Sarah Bruger. If you like the show, please

850
00:47:11,244 --> 00:47:13,684
Speaker 2: rate and review us. You know we'd like we take

851
00:47:13,684 --> 00:47:16,244
Speaker 2: a four or five. We take the five, rate and

852
00:47:16,284 --> 00:47:18,004
Speaker 2: review us to other people. Thank you for listening.