1 00:00:15,356 --> 00:00:25,836 Speaker 1: Pushkin. Today's show is about no knowns and unknown unknowns, 2 00:00:26,436 --> 00:00:30,876 Speaker 1: which is to say, we're talking about AI, specifically a 3 00:00:30,956 --> 00:00:34,676 Speaker 1: type of AI called a large language model, or an LM. 4 00:00:35,196 --> 00:00:38,956 Speaker 1: The most famous LLM is CHAT GPT, but there are 5 00:00:38,996 --> 00:00:41,996 Speaker 1: lots of others, and at their core, they all do 6 00:00:42,036 --> 00:00:44,796 Speaker 1: the same thing. They read a piece of text and 7 00:00:44,836 --> 00:00:47,836 Speaker 1: they predict what the next series of words should be. 8 00:00:48,436 --> 00:00:52,676 Speaker 1: Lms are, obviously and quite suddenly, a huge deal in 9 00:00:52,716 --> 00:00:55,476 Speaker 1: a lot of ways. One thing about them that is 10 00:00:55,596 --> 00:00:59,556 Speaker 1: particularly wild to me. Lms behave in ways that are 11 00:00:59,556 --> 00:01:03,876 Speaker 1: surprising even to the people who built them. In other words, 12 00:01:04,196 --> 00:01:08,716 Speaker 1: large language models are this profoundly powerful, disruptive new thing, 13 00:01:09,316 --> 00:01:12,316 Speaker 1: and right now we urgently need to figure out what 14 00:01:12,356 --> 00:01:19,996 Speaker 1: they mean and how they work. I'm Jacob Goldstein and 15 00:01:20,036 --> 00:01:22,116 Speaker 1: this is What's Your Problem, the show where I talk 16 00:01:22,196 --> 00:01:26,036 Speaker 1: to people who are trying to make technological progress. My 17 00:01:26,076 --> 00:01:29,276 Speaker 1: guest today is Sam Bowman. He's an expert in large 18 00:01:29,316 --> 00:01:33,396 Speaker 1: language models in lms. He's on the faculty at NYU, 19 00:01:33,516 --> 00:01:36,116 Speaker 1: and he runs a research group at an AI company 20 00:01:36,116 --> 00:01:40,796 Speaker 1: called Anthropic. All the reason talk about lms inspired Sam 21 00:01:40,836 --> 00:01:42,716 Speaker 1: to write a paper to clear up what he thought 22 00:01:42,716 --> 00:01:46,036 Speaker 1: were some misconceptions. The paper is called eight Things to 23 00:01:46,116 --> 00:01:49,236 Speaker 1: Know about Large Language Models. I am a fan of 24 00:01:49,356 --> 00:01:52,436 Speaker 1: lists in general, and I loved this list in particular. 25 00:01:53,156 --> 00:01:55,356 Speaker 1: Among other things, it gave me a deeper sense of 26 00:01:55,396 --> 00:01:58,356 Speaker 1: the ways in which large language models are still a mystery, 27 00:01:58,556 --> 00:02:04,356 Speaker 1: even to experts like Sam. That mystery, those unknowns, have 28 00:02:04,556 --> 00:02:08,156 Speaker 1: important implications for the way we think about, and regulate 29 00:02:08,196 --> 00:02:11,796 Speaker 1: and develop AI. We're going to start by discussing a 30 00:02:11,836 --> 00:02:15,596 Speaker 1: pretty simple item on Sam's list. The item is this, 31 00:02:16,476 --> 00:02:20,916 Speaker 1: brief interactions with llms are often misleading. You write this, 32 00:02:21,036 --> 00:02:26,316 Speaker 1: You write, brief interactions with lms are often misleading. What's 33 00:02:26,356 --> 00:02:26,636 Speaker 1: that mean? 34 00:02:27,236 --> 00:02:31,676 Speaker 2: So when, especially when GPD four came out, and I 35 00:02:31,676 --> 00:02:35,636 Speaker 2: guess also went when chat GPT first came out, there 36 00:02:35,676 --> 00:02:39,436 Speaker 2: was very predictably this wave of people on Twitter saying, hey, 37 00:02:40,356 --> 00:02:44,276 Speaker 2: this system is sentient and it knows where I live 38 00:02:44,556 --> 00:02:47,956 Speaker 2: and it's ready to take over the world tomorrow because 39 00:02:47,996 --> 00:02:50,516 Speaker 2: they had one chat with it and it said that 40 00:02:50,556 --> 00:02:52,476 Speaker 2: it was sentient and it made a few educated guesses 41 00:02:52,556 --> 00:02:55,236 Speaker 2: that happened to be bright, and you'll get other people 42 00:02:55,276 --> 00:02:58,236 Speaker 2: on Twitter saying, hey, this system is dumb as bricks. 43 00:02:58,556 --> 00:03:02,076 Speaker 2: I told it a really simple story and ask it 44 00:03:02,156 --> 00:03:03,556 Speaker 2: what happened in the story and it got it wrong. 45 00:03:04,396 --> 00:03:06,396 Speaker 2: There's a couple of things going on here. There's this 46 00:03:06,436 --> 00:03:08,236 Speaker 2: great analogy that came up in a recent I think 47 00:03:08,236 --> 00:03:10,636 Speaker 2: Time article by hell in Time owner saying they're basically 48 00:03:10,676 --> 00:03:14,716 Speaker 2: improv players, where if you put them in some situation, 49 00:03:14,996 --> 00:03:18,276 Speaker 2: if you put them in this situation of, oh, this 50 00:03:18,316 --> 00:03:20,476 Speaker 2: is a conversation between a human who thinks the AI 51 00:03:20,516 --> 00:03:23,476 Speaker 2: is sentient and the AI, then maybe the AA is 52 00:03:23,476 --> 00:03:24,316 Speaker 2: going to say it's sentient. 53 00:03:24,436 --> 00:03:27,396 Speaker 1: So specifically, they're improv players in the sense that famously 54 00:03:27,436 --> 00:03:30,076 Speaker 1: an improv you're supposed to say yes to everything that 55 00:03:30,116 --> 00:03:34,476 Speaker 1: your improv partner suggests, and so CHATCHYPT and the other 56 00:03:34,836 --> 00:03:37,436 Speaker 1: llms are there to say yes, yes, and and that's 57 00:03:37,476 --> 00:03:38,196 Speaker 1: what's going on. 58 00:03:38,556 --> 00:03:41,756 Speaker 2: That's a decent part of it. Yeah, they're going to 59 00:03:41,796 --> 00:03:43,396 Speaker 2: say yes. They're going to go along with what you're 60 00:03:43,396 --> 00:03:46,036 Speaker 2: doing if you make it clear what you expect, if 61 00:03:46,076 --> 00:03:48,356 Speaker 2: you make it clear, like what kind of narrative you're 62 00:03:48,396 --> 00:03:51,356 Speaker 2: putting them in, what kind of environment you're putting them in, 63 00:03:51,396 --> 00:03:52,156 Speaker 2: they'll go along with that. 64 00:03:52,556 --> 00:03:55,956 Speaker 1: Uh, there are a couple of items on your list 65 00:03:56,436 --> 00:04:00,036 Speaker 1: that seems directly contrary to assertions I've heard from other 66 00:04:00,076 --> 00:04:04,836 Speaker 1: people about LMS, so that's fun and exciting. One is 67 00:04:06,436 --> 00:04:10,156 Speaker 1: human performance on a task is not an upper bound 68 00:04:10,356 --> 00:04:12,076 Speaker 1: on LM performance. 69 00:04:12,796 --> 00:04:15,316 Speaker 2: So one of the reasons I think these systems can 70 00:04:16,356 --> 00:04:18,596 Speaker 2: be better at a lot of tasks than humans is 71 00:04:18,676 --> 00:04:21,916 Speaker 2: just that they've learned more stuff that they've read and 72 00:04:21,956 --> 00:04:24,756 Speaker 2: mostly memorized, not just sort of all of the important 73 00:04:24,756 --> 00:04:27,916 Speaker 2: papers in one little branch of chemistry or all of 74 00:04:27,916 --> 00:04:29,996 Speaker 2: the important papers in all of chemistry. They've just read 75 00:04:29,996 --> 00:04:33,276 Speaker 2: and mostly memorized, sort of all of the research papers. 76 00:04:33,036 --> 00:04:35,436 Speaker 1: In everything, all of the papers in everything. 77 00:04:35,676 --> 00:04:38,756 Speaker 2: Yeah, and many of the novels and many of the 78 00:04:38,996 --> 00:04:42,476 Speaker 2: many of the news stories. And even if these systems 79 00:04:42,476 --> 00:04:45,076 Speaker 2: aren't really great at drawing connections between these and sort 80 00:04:45,076 --> 00:04:47,076 Speaker 2: of synthesizing a new knowledge out of them, they can 81 00:04:47,076 --> 00:04:49,756 Speaker 2: do that a little bit. So you can sort of 82 00:04:49,756 --> 00:04:53,716 Speaker 2: imagine what happens if you get someone who's not especially bright, 83 00:04:53,756 --> 00:04:59,156 Speaker 2: but basically reasonably intelligent, reasonably competent person who's just gotten 84 00:04:59,196 --> 00:05:01,236 Speaker 2: a PhD in every single thing you can get a 85 00:05:01,236 --> 00:05:03,716 Speaker 2: PhD in, I'd expect them to figure some things out 86 00:05:03,756 --> 00:05:05,836 Speaker 2: and to be able to do some things that no 87 00:05:05,996 --> 00:05:09,956 Speaker 2: one person can do, and probably they'll notice some things 88 00:05:09,956 --> 00:05:11,676 Speaker 2: that that'll be really hard for even a team or 89 00:05:11,676 --> 00:05:13,996 Speaker 2: an organization to do, just because it really it's important 90 00:05:13,996 --> 00:05:16,876 Speaker 2: that it kind of is, in some sense living in 91 00:05:16,916 --> 00:05:17,836 Speaker 2: this one person's head. 92 00:05:18,036 --> 00:05:21,516 Speaker 1: Let me just like lean into that one for a sec. 93 00:05:22,196 --> 00:05:27,436 Speaker 1: So do you think that in some amount of time 94 00:05:27,516 --> 00:05:32,516 Speaker 1: in the next few years, say, an LM will make 95 00:05:32,596 --> 00:05:37,556 Speaker 1: some kind of you know, breakthrough in knowledge, will figure 96 00:05:37,596 --> 00:05:39,876 Speaker 1: something out that no human has ever figured out that 97 00:05:39,956 --> 00:05:42,276 Speaker 1: will be a meaningful breakthrough. 98 00:05:42,636 --> 00:05:45,596 Speaker 2: Yeah, I think so almost. By definition, I don't have 99 00:05:45,636 --> 00:05:46,956 Speaker 2: a good guess of what that's going to look like 100 00:05:46,996 --> 00:05:47,676 Speaker 2: or that's going to be. 101 00:05:47,716 --> 00:05:50,836 Speaker 3: Otherwise you'd be figuring it out right now, right, yeah, yeah, yeah, 102 00:05:50,876 --> 00:05:53,076 Speaker 3: But no, I can imagine some story like, hey, kind 103 00:05:53,076 --> 00:05:55,996 Speaker 3: of a bunch of chemists in this field of chemists 104 00:05:55,996 --> 00:05:59,076 Speaker 3: have noticed this thing, and some biologists in this other 105 00:05:59,116 --> 00:06:01,356 Speaker 3: subfield have noticed this other thing, and some doctors have 106 00:06:01,436 --> 00:06:04,076 Speaker 3: noticed this third thing, and together they mean that some 107 00:06:05,076 --> 00:06:07,716 Speaker 3: very unexpected kind of drug design might treat some new disease. 108 00:06:08,596 --> 00:06:12,436 Speaker 2: And maybe if you had enough medical researchers trying enough 109 00:06:12,436 --> 00:06:15,276 Speaker 2: different things, eventually they'd stumble into that. But it seems 110 00:06:15,276 --> 00:06:17,836 Speaker 2: possible at some point that something like a large language 111 00:06:17,876 --> 00:06:20,196 Speaker 2: model is just going to notice that, and if you 112 00:06:20,516 --> 00:06:21,876 Speaker 2: ask it the right way, it's going to tell you, 113 00:06:22,836 --> 00:06:25,396 Speaker 2: and you might have to second guess it a lot. 114 00:06:25,436 --> 00:06:28,036 Speaker 2: These systems also make stuff up, But I think it's 115 00:06:28,076 --> 00:06:31,276 Speaker 2: quite possible that you start seeing these things pretty often 116 00:06:31,356 --> 00:06:34,076 Speaker 2: tell you surprising new things that happen to be true. 117 00:06:34,716 --> 00:06:38,156 Speaker 1: There's another item on your list that seems to me 118 00:06:38,276 --> 00:06:41,916 Speaker 1: to be like a provocation. It seems to me in 119 00:06:41,916 --> 00:06:47,116 Speaker 1: a good way. It seems like directly contradictory to what 120 00:06:47,156 --> 00:06:51,316 Speaker 1: I have read, specifically to this idea that all large 121 00:06:51,396 --> 00:06:55,836 Speaker 1: language models are doing is guessing what the next word 122 00:06:55,916 --> 00:06:58,676 Speaker 1: in a series is likely to be, and that list 123 00:06:58,716 --> 00:07:04,836 Speaker 1: item is this. Llms often appear to learn and use 124 00:07:04,916 --> 00:07:09,036 Speaker 1: representations of the outside world. Llms often appear to learn 125 00:07:09,196 --> 00:07:13,556 Speaker 1: and use representations of the outside world. So that sounds 126 00:07:13,636 --> 00:07:17,156 Speaker 1: quite different from just guessing the next word, is it 127 00:07:16,956 --> 00:07:18,996 Speaker 1: or is it not? Different in a way that I 128 00:07:19,036 --> 00:07:19,916 Speaker 1: just don't understand. 129 00:07:20,396 --> 00:07:23,156 Speaker 2: It turns out it's not that different. Okay, this is 130 00:07:24,316 --> 00:07:26,276 Speaker 2: I want to say it's the big discovery, But it's 131 00:07:26,356 --> 00:07:29,796 Speaker 2: this big discovery that's spread out over dozens of experiments 132 00:07:30,156 --> 00:07:31,276 Speaker 2: over the last few years. 133 00:07:31,956 --> 00:07:34,076 Speaker 1: Can you give me a specific example. It's such an 134 00:07:34,076 --> 00:07:38,596 Speaker 1: abstract assertion that I think it would be helpful to 135 00:07:38,716 --> 00:07:40,996 Speaker 1: have a specific example. 136 00:07:40,836 --> 00:07:44,236 Speaker 2: That we can think about. One great example of this 137 00:07:44,516 --> 00:07:48,196 Speaker 2: is if you tell a model a story, a simple 138 00:07:48,236 --> 00:07:51,116 Speaker 2: story that takes place in some sort of physical space 139 00:07:51,156 --> 00:07:54,556 Speaker 2: where it's it's some characters walking around a house and 140 00:07:54,596 --> 00:07:56,756 Speaker 2: they're having a conversation while they're walking, and they're picking 141 00:07:56,756 --> 00:07:59,956 Speaker 2: style up and they're putting it down. You can see 142 00:08:00,036 --> 00:08:03,916 Speaker 2: inside the activations of the neurons when the model is 143 00:08:03,956 --> 00:08:06,236 Speaker 2: reading that story. You can pull out a map of 144 00:08:06,236 --> 00:08:09,356 Speaker 2: the house. You can see that there's a there's a 145 00:08:09,396 --> 00:08:11,396 Speaker 2: piece the network that says, oh, okay, now they're in 146 00:08:11,436 --> 00:08:13,556 Speaker 2: the living room, and another piece that says, oh, living 147 00:08:13,596 --> 00:08:17,876 Speaker 2: room is connected to the bedroom. And you can mess 148 00:08:17,876 --> 00:08:19,396 Speaker 2: with this in ways that show that it's really sort 149 00:08:19,436 --> 00:08:23,316 Speaker 2: of it is really representing the house. That if you 150 00:08:23,356 --> 00:08:25,996 Speaker 2: find the piece of the network that says, oh, Susan 151 00:08:26,076 --> 00:08:29,916 Speaker 2: is in the living room, and you flip that, flip 152 00:08:29,916 --> 00:08:32,436 Speaker 2: that from a positive number to a negative number, then 153 00:08:32,676 --> 00:08:35,716 Speaker 2: the story will continue as though Susan is not in 154 00:08:35,716 --> 00:08:37,236 Speaker 2: a lot in the living room, or couldn't possibly have 155 00:08:37,236 --> 00:08:37,796 Speaker 2: been in living. 156 00:08:37,676 --> 00:08:41,636 Speaker 1: So that does seem like it's representing the physical world 157 00:08:41,716 --> 00:08:45,116 Speaker 1: in a way that is not just guessing the next word. 158 00:08:45,876 --> 00:08:50,156 Speaker 2: Yeah. Yeah, so we're finding out these systems are actually 159 00:08:50,196 --> 00:08:52,676 Speaker 2: representing the objects they're talking about, at least some of 160 00:08:52,716 --> 00:08:52,996 Speaker 2: the time. 161 00:08:53,156 --> 00:08:55,796 Speaker 1: They're creating a representation of physical space. 162 00:08:56,356 --> 00:08:58,796 Speaker 2: Yeah. I should be clear that this is this doesn't 163 00:08:58,836 --> 00:09:02,956 Speaker 2: always work when when you're giving these systems something really 164 00:09:03,036 --> 00:09:06,196 Speaker 2: hard and subtle, they're just going to totally botch this stuff. 165 00:09:06,196 --> 00:09:09,876 Speaker 2: Their internal representations are a mess. But more and more 166 00:09:09,876 --> 00:09:12,036 Speaker 2: of the time they're really doing it. And as these 167 00:09:12,036 --> 00:09:14,556 Speaker 2: things get bigger and bigger, they're doing it more and more. 168 00:09:15,196 --> 00:09:17,316 Speaker 2: And so this feels like this important turning point where 169 00:09:17,316 --> 00:09:19,996 Speaker 2: it's like, oh, okay, there is some understanding going on 170 00:09:20,076 --> 00:09:24,076 Speaker 2: here and it's getting better, and that really radically opens 171 00:09:24,156 --> 00:09:26,596 Speaker 2: up the possibilities for where this technology might go. 172 00:09:27,276 --> 00:09:32,276 Speaker 1: This what you're saying seems very much at odds with 173 00:09:34,116 --> 00:09:39,276 Speaker 1: what people generally say about llms, Right, Like the standard 174 00:09:39,876 --> 00:09:43,316 Speaker 1: line is they're just predicting what the next word is 175 00:09:43,316 --> 00:09:44,796 Speaker 1: going to be. And they're very good at predicting what 176 00:09:44,796 --> 00:09:45,956 Speaker 1: the next word is going to be, and there's a 177 00:09:45,996 --> 00:09:48,116 Speaker 1: lot of powerful things you can do, but what you're 178 00:09:48,116 --> 00:09:52,556 Speaker 1: saying sounds fundamentally different from that. And so I mean, 179 00:09:52,956 --> 00:09:54,876 Speaker 1: are the people saying they're just predicting the next word? 180 00:09:54,876 --> 00:09:58,036 Speaker 1: Are they wrong? Is what you're saying a point of 181 00:09:58,116 --> 00:10:01,156 Speaker 1: debate among experts or what? Why is this so different 182 00:10:01,156 --> 00:10:02,156 Speaker 1: than what I've heard before. 183 00:10:02,636 --> 00:10:05,716 Speaker 2: There's a few things going on. So first, saying that 184 00:10:05,716 --> 00:10:08,196 Speaker 2: they're just predicting the next word is mostly right. But 185 00:10:08,276 --> 00:10:09,996 Speaker 2: it turns out that's saying that they just predict the 186 00:10:09,996 --> 00:10:12,036 Speaker 2: next word is a lot like saying humans are just 187 00:10:12,276 --> 00:10:16,156 Speaker 2: chemical reactions. It turns out that if you're trying to 188 00:10:16,156 --> 00:10:20,556 Speaker 2: predict the next word, and if you've got a smaller 189 00:10:20,716 --> 00:10:22,756 Speaker 2: work that's trying to predict the next word, it's going 190 00:10:22,836 --> 00:10:26,196 Speaker 2: to learn that sort of the word, the and of 191 00:10:26,316 --> 00:10:28,556 Speaker 2: an a and those show up often, and that's about 192 00:10:28,596 --> 00:10:31,236 Speaker 2: all it's going to learn. If you take a medium 193 00:10:31,276 --> 00:10:33,796 Speaker 2: sized neural network, it's going to learn how to write 194 00:10:33,796 --> 00:10:35,756 Speaker 2: fluent sentences. This is going to write, oh, okay, sort 195 00:10:35,756 --> 00:10:39,156 Speaker 2: of adjectives come before nouns, these kinds of nouns come 196 00:10:39,196 --> 00:10:41,796 Speaker 2: before these kinds of nouns. It might even learn some facts. 197 00:10:41,796 --> 00:10:44,436 Speaker 2: It might learn that if you talk about the president 198 00:10:44,476 --> 00:10:46,956 Speaker 2: of the United States, you'll get names like Obama and 199 00:10:46,996 --> 00:10:50,236 Speaker 2: Bush and Biden and Trump, and it'll start to kind 200 00:10:50,276 --> 00:10:53,196 Speaker 2: of make sense, but it's still just kind of learning statistics. 201 00:10:53,836 --> 00:10:56,596 Speaker 2: And if you make the neural work even bigger, it 202 00:10:56,676 --> 00:11:00,116 Speaker 2: will abstract further away. It will start to reason about 203 00:11:00,756 --> 00:11:04,316 Speaker 2: the people and the objects and the spaces themselves and 204 00:11:04,436 --> 00:11:07,756 Speaker 2: use that abstraction to predict the next word. So kind 205 00:11:07,756 --> 00:11:11,076 Speaker 2: of the more these systems learn about the world, the 206 00:11:11,196 --> 00:11:13,836 Speaker 2: farther and farther their Internet representations get from just sort 207 00:11:13,876 --> 00:11:16,476 Speaker 2: of literally what word comes after what other word. 208 00:11:17,236 --> 00:11:20,236 Speaker 1: So there's another item on your list that seems like 209 00:11:20,596 --> 00:11:24,076 Speaker 1: it should have interesting implications for the AI industry, right 210 00:11:24,116 --> 00:11:28,156 Speaker 1: for the business of building lms, I'll just read that one. 211 00:11:28,956 --> 00:11:34,796 Speaker 1: It goes lms predictably get more capable with increasing investment, 212 00:11:35,276 --> 00:11:39,716 Speaker 1: even without targeted innovation. So we'll get into it. But 213 00:11:40,356 --> 00:11:42,276 Speaker 1: just top line, what does that mean? 214 00:11:44,556 --> 00:11:49,116 Speaker 2: We had language models in almost their modern form back 215 00:11:49,196 --> 00:11:53,996 Speaker 2: in twenty ten, eleven, twelve. Most of the building blocks 216 00:11:53,996 --> 00:11:55,876 Speaker 2: for them go back even farther to the eighties or 217 00:11:55,876 --> 00:12:00,036 Speaker 2: even the sixties. You might have noticed that we weren't 218 00:12:00,516 --> 00:12:03,956 Speaker 2: We didn't have chat GBT ten or twenty or fifty 219 00:12:04,476 --> 00:12:09,436 Speaker 2: years ago. What people have been gradually discovering and dually 220 00:12:10,396 --> 00:12:13,036 Speaker 2: sort of discovering to a greater and greater degree is 221 00:12:13,036 --> 00:12:17,636 Speaker 2: that if you just take this reldly simple technology and 222 00:12:18,716 --> 00:12:22,076 Speaker 2: throw more data at it and run it in its 223 00:12:22,076 --> 00:12:25,836 Speaker 2: sort of training phase for longer and longer by fancier 224 00:12:25,876 --> 00:12:27,956 Speaker 2: or and France your computers to run it on, it 225 00:12:28,116 --> 00:12:29,036 Speaker 2: just keeps getting better. 226 00:12:29,156 --> 00:12:32,796 Speaker 1: But if the technology is not special, I mean, everybody 227 00:12:32,836 --> 00:12:37,076 Speaker 1: knows the basic sauce, it suggests that GPT might not 228 00:12:37,196 --> 00:12:40,876 Speaker 1: have an open AI. The company that makes chat GPT 229 00:12:41,116 --> 00:12:44,316 Speaker 1: might not have like that much of a moat, right. 230 00:12:45,276 --> 00:12:48,996 Speaker 1: I mean, Google is clearly in this business, as is Anthropic, 231 00:12:49,036 --> 00:12:53,116 Speaker 1: the company where you're working. Is there any reason to 232 00:12:53,156 --> 00:12:56,036 Speaker 1: think open AI GPT is going to stay ahead. 233 00:12:56,556 --> 00:12:59,156 Speaker 2: I think there's not a lot of secret sauce. There 234 00:12:59,196 --> 00:13:01,276 Speaker 2: are some details of how to build these things that 235 00:13:01,796 --> 00:13:04,196 Speaker 2: don't get published, but the basic idea is very much 236 00:13:04,196 --> 00:13:09,996 Speaker 2: out there. And yeah, I think the the closest thing 237 00:13:10,036 --> 00:13:12,516 Speaker 2: you can really have to emote is just enormous amounts 238 00:13:12,516 --> 00:13:15,036 Speaker 2: of money. I think at some point you're going to 239 00:13:15,076 --> 00:13:18,476 Speaker 2: have a relatively small number of labs building the really 240 00:13:18,556 --> 00:13:21,556 Speaker 2: impressive frontier systems just because at some point these are 241 00:13:21,556 --> 00:13:24,996 Speaker 2: going to be ten billion dollar projects, and it just 242 00:13:25,036 --> 00:13:26,876 Speaker 2: seems unlikely that you're going to get that many ten 243 00:13:26,876 --> 00:13:28,836 Speaker 2: billion dollar projects. 244 00:13:28,436 --> 00:13:31,076 Speaker 1: If it's the case, as you say that, essentially what 245 00:13:31,676 --> 00:13:34,956 Speaker 1: you need to build a frontier level LM is a 246 00:13:34,996 --> 00:13:41,596 Speaker 1: lot of money. I would guess that governments around the world, 247 00:13:41,636 --> 00:13:45,076 Speaker 1: certainly say China to pick a salient government, are probably 248 00:13:45,356 --> 00:13:48,716 Speaker 1: building giant lms right now. Does that seem like a 249 00:13:48,756 --> 00:13:50,036 Speaker 1: reasonable guess? 250 00:13:51,676 --> 00:13:55,116 Speaker 2: Yeah, that seems right. I know there are a lot 251 00:13:55,116 --> 00:13:59,836 Speaker 2: of private and private, public and public groups in China 252 00:14:00,036 --> 00:14:02,516 Speaker 2: working in this stuff, and when I sort of hear 253 00:14:02,556 --> 00:14:05,716 Speaker 2: people in the field who are following the geopolitical side 254 00:14:05,716 --> 00:14:08,036 Speaker 2: of this more closely, they're paying a lot of attention 255 00:14:08,196 --> 00:14:13,716 Speaker 2: to things like the Chips Act and Global Trade in 256 00:14:14,396 --> 00:14:17,476 Speaker 2: chips in that you really do need. When you're spending 257 00:14:17,556 --> 00:14:19,956 Speaker 2: these millions or billions of dollars, you're basically spending them 258 00:14:19,956 --> 00:14:23,316 Speaker 2: to buy or rent very fancy, state of the art 259 00:14:23,436 --> 00:14:27,476 Speaker 2: computer chips. And it has become a priority for the 260 00:14:27,516 --> 00:14:29,796 Speaker 2: US to try to make it hard for China to 261 00:14:29,796 --> 00:14:33,116 Speaker 2: do that, and. 262 00:14:33,716 --> 00:14:35,716 Speaker 1: To try and make it hard for China to get 263 00:14:35,556 --> 00:14:38,476 Speaker 1: at the processor level, which in a sense is like 264 00:14:38,836 --> 00:14:41,796 Speaker 1: the cement that lllms are built from. There is a 265 00:14:41,836 --> 00:14:45,796 Speaker 1: physical thing. We forget that, but it's fancy chips basically. 266 00:14:46,156 --> 00:14:46,556 Speaker 2: That's right. 267 00:14:46,636 --> 00:14:52,276 Speaker 1: Yeah, we've been talking so far about what we know 268 00:14:52,516 --> 00:14:55,916 Speaker 1: about how large language models work. After the break, we'll 269 00:14:55,916 --> 00:14:58,396 Speaker 1: get into what I think is the most interesting thing 270 00:14:58,476 --> 00:15:09,236 Speaker 1: about lms, what we don't know about how they work. 271 00:15:09,796 --> 00:15:10,756 Speaker 1: That's the end of the ads. 272 00:15:11,196 --> 00:15:12,356 Speaker 2: Now we're going back to the show. 273 00:15:12,796 --> 00:15:16,636 Speaker 1: So far, we've basically been talking about how do lllms work. 274 00:15:16,796 --> 00:15:22,916 Speaker 1: What's going on? There is another bucket in your list, 275 00:15:22,956 --> 00:15:26,756 Speaker 1: several items, three items that are it seems to me, 276 00:15:26,796 --> 00:15:29,116 Speaker 1: in quite a different category, and they get at this 277 00:15:29,876 --> 00:15:35,156 Speaker 1: very very interesting idea about lms, and that is, to 278 00:15:35,276 --> 00:15:40,316 Speaker 1: some significant degree, nobody knows how they work. The people 279 00:15:40,316 --> 00:15:43,116 Speaker 1: who build lms, people like you, people who build them 280 00:15:43,116 --> 00:15:46,516 Speaker 1: and study them, don't understand a lot of what is 281 00:15:46,556 --> 00:15:49,716 Speaker 1: going on, which is amazing to me and super interesting. 282 00:15:49,756 --> 00:15:55,996 Speaker 1: So let's start with this list item. It says specific 283 00:15:56,116 --> 00:16:03,436 Speaker 1: important behaviors in lms tend to emerge unpredictably as a byproduct. 284 00:16:02,796 --> 00:16:03,836 Speaker 2: Of increasing investment. 285 00:16:03,916 --> 00:16:06,956 Speaker 1: And you give a couple of examples of this happening 286 00:16:07,556 --> 00:16:09,916 Speaker 1: for real in the world. I think the best way 287 00:16:09,916 --> 00:16:12,876 Speaker 1: to understand what's going on here is to talk about 288 00:16:12,876 --> 00:16:15,236 Speaker 1: one of those examples. Can you just like talk me 289 00:16:15,276 --> 00:16:20,036 Speaker 1: through one of those examples of this unpredictable new behavior emerging. Yeah. 290 00:16:20,436 --> 00:16:23,396 Speaker 2: So a specific large language model that people working in 291 00:16:23,436 --> 00:16:25,396 Speaker 2: the stuff talk about a lot is GPD three. This 292 00:16:25,476 --> 00:16:28,476 Speaker 2: came out a little less than three years ago and 293 00:16:28,516 --> 00:16:30,356 Speaker 2: I think sort of kicked off the modern wave of 294 00:16:30,356 --> 00:16:34,116 Speaker 2: research on this stuff. And one thing researchers would do, 295 00:16:34,156 --> 00:16:37,236 Speaker 2: as these systems would would come out is give them 296 00:16:37,476 --> 00:16:39,676 Speaker 2: math puzzles and logic puzzles and see how they did. 297 00:16:40,356 --> 00:16:42,276 Speaker 2: And this could be as simple as just sort of 298 00:16:42,316 --> 00:16:45,636 Speaker 2: giving the model reasonably hard arithmetic, sort of asking the model, 299 00:16:45,956 --> 00:16:49,076 Speaker 2: what is one hundred and twenty five plus four hundred 300 00:16:49,076 --> 00:16:52,036 Speaker 2: and sixty seven. And what they found is sort of 301 00:16:52,556 --> 00:16:55,196 Speaker 2: GPD one was bad at this, and GPD two was 302 00:16:55,236 --> 00:16:57,396 Speaker 2: bad at this, and at least for some of these tasks, 303 00:16:57,476 --> 00:17:02,396 Speaker 2: GPD three was also bad at this. And they released it. 304 00:17:02,396 --> 00:17:03,556 Speaker 2: They put it out in the world, they wrote a 305 00:17:03,556 --> 00:17:06,676 Speaker 2: paper about it, they did some demos to researchers, and 306 00:17:06,716 --> 00:17:08,836 Speaker 2: then eventually just let anyone sign up and use it. 307 00:17:09,716 --> 00:17:14,076 Speaker 2: And after a few months people started noticing. Oh, there 308 00:17:14,076 --> 00:17:15,876 Speaker 2: are some tricks you can use to actually make it 309 00:17:15,996 --> 00:17:21,716 Speaker 2: quite a bit better at this. If you ask the 310 00:17:21,756 --> 00:17:24,996 Speaker 2: model the right way, sometimes it'll just kind of reason 311 00:17:25,036 --> 00:17:28,196 Speaker 2: out loud. Sometimes it will say, well, it'll actually do 312 00:17:28,316 --> 00:17:30,116 Speaker 2: long edition, we'll actually write out its steps. 313 00:17:30,476 --> 00:17:33,556 Speaker 1: So give me a specific example. How do you ask 314 00:17:33,596 --> 00:17:34,276 Speaker 1: it the right way? 315 00:17:35,756 --> 00:17:37,796 Speaker 2: So it took even a few more months for people 316 00:17:37,796 --> 00:17:40,916 Speaker 2: to figure out how to do this systematically, but it 317 00:17:40,956 --> 00:17:43,916 Speaker 2: turned out the trick was you literally say, let's think 318 00:17:43,956 --> 00:17:44,716 Speaker 2: step by step. 319 00:17:44,996 --> 00:17:48,196 Speaker 1: You actually type that in, you say that to the machine, 320 00:17:48,196 --> 00:17:49,116 Speaker 1: to the model. 321 00:17:49,036 --> 00:17:51,516 Speaker 2: Yes, And if you say what is this number of 322 00:17:51,516 --> 00:17:54,956 Speaker 2: plus this number question mark, it'll give a wrong answer. 323 00:17:55,116 --> 00:17:56,876 Speaker 2: If you say, what is this number of plus this number, 324 00:17:57,356 --> 00:18:00,636 Speaker 2: let's think step by step dot dot, it's going to 325 00:18:00,716 --> 00:18:03,036 Speaker 2: list out. Okay, let's start with the ones digit, and 326 00:18:03,036 --> 00:18:04,996 Speaker 2: then the tenth digit, and then the one hundredth digit, 327 00:18:05,556 --> 00:18:08,236 Speaker 2: and then give you the answer, and it'll very often 328 00:18:08,236 --> 00:18:10,836 Speaker 2: be right huh. And it turns out this works really 329 00:18:10,876 --> 00:18:14,316 Speaker 2: generally that for many kinds of sort of math and 330 00:18:14,396 --> 00:18:19,276 Speaker 2: reasoning problems, even some even sort of ethics problems. There's 331 00:18:19,636 --> 00:18:21,396 Speaker 2: a huge range of things you might ask one of 332 00:18:21,436 --> 00:18:24,036 Speaker 2: these ural networks to do where if you just tell it, 333 00:18:24,316 --> 00:18:28,116 Speaker 2: let's think step by step, it will bring out this 334 00:18:28,156 --> 00:18:31,076 Speaker 2: whole reasoning ability that is actually really useful, that allows 335 00:18:31,116 --> 00:18:32,516 Speaker 2: it to do much better at a lot of things, 336 00:18:32,916 --> 00:18:37,556 Speaker 2: and that it didn't have before. And when this technology 337 00:18:37,596 --> 00:18:39,676 Speaker 2: was first released, the people who built it, they did 338 00:18:39,676 --> 00:18:41,076 Speaker 2: not know this was a possibility. 339 00:18:42,036 --> 00:18:45,916 Speaker 1: That's wild, right, Like it means this thing is incredibly 340 00:18:45,996 --> 00:18:48,996 Speaker 1: powerful in a way that the people who built it 341 00:18:49,076 --> 00:18:51,996 Speaker 1: didn't know. And let's think step by step is just 342 00:18:52,076 --> 00:18:56,116 Speaker 1: like this incantation. It's just like saying abracadabra or something, 343 00:18:56,716 --> 00:18:58,956 Speaker 1: and the builders didn't know it was there. 344 00:18:59,436 --> 00:19:01,996 Speaker 2: Yeah, it's it's a bizarre time to be working on 345 00:19:02,036 --> 00:19:02,596 Speaker 2: this stuff. 346 00:19:02,676 --> 00:19:06,076 Speaker 1: It Like, here's where it's getting a little sketchy to 347 00:19:06,116 --> 00:19:08,316 Speaker 1: me at a certain level, right, I mean you've also 348 00:19:08,316 --> 00:19:10,316 Speaker 1: done a lot of work in AI safety and this 349 00:19:10,396 --> 00:19:12,876 Speaker 1: kind of section of the interview, I feel like we're 350 00:19:12,876 --> 00:19:14,996 Speaker 1: getting more toward that, the section of like, the people 351 00:19:15,036 --> 00:19:17,916 Speaker 1: building this stuff don't understand what it can do. And 352 00:19:17,956 --> 00:19:20,596 Speaker 1: here should we add another list item here? Like this 353 00:19:20,716 --> 00:19:23,636 Speaker 1: might be the place Cherkiff, So there's this other item 354 00:19:23,676 --> 00:19:25,996 Speaker 1: on your eight things to know list that seems germane. 355 00:19:25,996 --> 00:19:30,916 Speaker 1: Here experts are not yet able to interpret the inner 356 00:19:30,956 --> 00:19:35,836 Speaker 1: workings of lms, which also wild also kind of goes 357 00:19:35,876 --> 00:19:39,676 Speaker 1: with this idea of not knowing what the thing can do, 358 00:19:39,836 --> 00:19:44,076 Speaker 1: right and very not intuitive for a piece of technology. 359 00:19:44,156 --> 00:19:47,156 Speaker 1: Right If you go back to say the Internet, Sure 360 00:19:47,196 --> 00:19:50,996 Speaker 1: we didn't know all the social implications of the Internet, 361 00:19:51,236 --> 00:19:54,156 Speaker 1: but we knew how the technology worked. We knew what 362 00:19:54,236 --> 00:19:56,716 Speaker 1: was going on with the chips and the wires and 363 00:19:56,716 --> 00:20:00,076 Speaker 1: the electrons and whatever. Right, Like the amazing thing here 364 00:20:00,116 --> 00:20:02,396 Speaker 1: is clearly we don't know the social implications of AI. 365 00:20:02,796 --> 00:20:05,436 Speaker 1: But you're saying, we don't even know what it's doing 366 00:20:05,516 --> 00:20:06,516 Speaker 1: inside the box. 367 00:20:08,076 --> 00:20:11,036 Speaker 2: Yeah, that's right. We've got these very crude tools for 368 00:20:11,116 --> 00:20:13,476 Speaker 2: sort of opening the box and looking inside. I mean, 369 00:20:13,636 --> 00:20:15,156 Speaker 2: in a literal sense, we know it's going on. We 370 00:20:15,156 --> 00:20:17,796 Speaker 2: can say, oh, when you put in this word, then 371 00:20:18,276 --> 00:20:20,316 Speaker 2: it makes this number bigger, which makes that number smaller, 372 00:20:20,316 --> 00:20:21,996 Speaker 2: which makes this number bigger. And you could keep saying 373 00:20:22,036 --> 00:20:25,076 Speaker 2: that for twenty years and then you'd have explained what happened. 374 00:20:26,636 --> 00:20:29,316 Speaker 2: But we haven't figured out any other way of talking 375 00:20:29,356 --> 00:20:32,556 Speaker 2: about these systems that actually gives us any clarity about 376 00:20:33,676 --> 00:20:35,836 Speaker 2: what's possible why these systems are doing what they're doing 377 00:20:35,956 --> 00:20:39,596 Speaker 2: where they're reliable and not it's just this huge mess 378 00:20:39,636 --> 00:20:43,436 Speaker 2: of connections that we don't really know what to do with. 379 00:20:43,996 --> 00:20:48,156 Speaker 1: I mean, what should we make of this set of 380 00:20:48,236 --> 00:20:54,476 Speaker 1: facts that these are incredibly powerful tools that nobody understands 381 00:20:54,516 --> 00:20:59,956 Speaker 1: at a pretty deep level, that can do unpredictable things, 382 00:20:59,956 --> 00:21:03,436 Speaker 1: that are able to do things that even their makers 383 00:21:03,516 --> 00:21:04,476 Speaker 1: don't know they can do. 384 00:21:05,236 --> 00:21:09,956 Speaker 2: I think it's pretty exciting and also pretty sobering. I 385 00:21:09,956 --> 00:21:11,436 Speaker 2: think we don't have a good way of predicting how 386 00:21:11,476 --> 00:21:13,476 Speaker 2: fast this is moving or what we're going to get when. 387 00:21:14,556 --> 00:21:18,196 Speaker 2: But in the big picture, it seems like there's a 388 00:21:18,236 --> 00:21:21,436 Speaker 2: lot of momentum toward building these really powerful eye systems 389 00:21:21,476 --> 00:21:24,956 Speaker 2: over the next few years. We don't understand how they work. 390 00:21:25,476 --> 00:21:27,676 Speaker 2: Another one of these list items is we also aren't 391 00:21:27,716 --> 00:21:29,236 Speaker 2: very good at controlling, and we aren't very good at 392 00:21:29,236 --> 00:21:30,436 Speaker 2: making them do what we want. 393 00:21:30,516 --> 00:21:32,396 Speaker 1: Yes, let me just pause there, because it's the last 394 00:21:32,436 --> 00:21:34,956 Speaker 1: list item and you have just walked up to it. 395 00:21:34,956 --> 00:21:37,356 Speaker 1: So the last item, the item that we haven't mentioned 396 00:21:37,356 --> 00:21:40,956 Speaker 1: on your list. There are no reliable techniques for steering 397 00:21:40,996 --> 00:21:44,076 Speaker 1: the behavior of lms, so they're powerful. We don't really 398 00:21:44,156 --> 00:21:46,036 Speaker 1: understand how they work. They can do things we don't 399 00:21:46,076 --> 00:21:48,676 Speaker 1: know they're going to do, and we can't really control them. 400 00:21:49,036 --> 00:21:51,116 Speaker 1: Now we're through the list. Now let's just talk it out. 401 00:21:51,436 --> 00:21:55,596 Speaker 2: Yeah, and so we're yeah, we're building, we're building these systems. 402 00:21:55,636 --> 00:21:59,276 Speaker 2: They're getting better, the developing new capabilities. We don't really 403 00:21:59,276 --> 00:22:03,036 Speaker 2: know how they work. We can't predict which capabilities are 404 00:22:03,036 --> 00:22:06,516 Speaker 2: showing up when and if they're doing something we don't want. 405 00:22:06,556 --> 00:22:09,556 Speaker 2: We don't really know how to notice that and mitigate 406 00:22:09,556 --> 00:22:12,836 Speaker 2: it and prevent it. And that just feels like it's 407 00:22:12,876 --> 00:22:15,276 Speaker 2: playing with fire at a scale what I'm not sure 408 00:22:15,276 --> 00:22:18,316 Speaker 2: we've seen before, at least outside of things like nuclear weapons. 409 00:22:18,156 --> 00:22:20,116 Speaker 2: It's this very sort of sobering situation to be in. 410 00:22:20,276 --> 00:22:24,236 Speaker 1: What do we do about it? 411 00:22:24,916 --> 00:22:26,796 Speaker 2: I'm not sure. I wish I had a better answer. 412 00:22:28,356 --> 00:22:31,476 Speaker 2: There are a few things that will definitely help. Maybe 413 00:22:31,516 --> 00:22:34,716 Speaker 2: one obvious thing here is just there's probably a lot 414 00:22:34,716 --> 00:22:36,476 Speaker 2: of regulation that would be good to have here. You 415 00:22:36,556 --> 00:22:40,196 Speaker 2: really don't want the move fast and break things ethos 416 00:22:40,716 --> 00:22:44,796 Speaker 2: to be behind a technology that is close to human 417 00:22:44,876 --> 00:22:48,156 Speaker 2: level ability at a lot of cognitive task That seems 418 00:22:48,196 --> 00:22:49,756 Speaker 2: like the setup for a bad sci fi movie. 419 00:22:49,916 --> 00:22:53,436 Speaker 1: Specifically, what regulation do you think is a good idea? 420 00:22:54,916 --> 00:22:57,356 Speaker 2: One outline of an idea that I'm excited about, and 421 00:22:57,396 --> 00:23:00,116 Speaker 2: I think this is definitely not the best idea or 422 00:23:00,116 --> 00:23:07,356 Speaker 2: the only good idea is mandating or standardizing some tests 423 00:23:07,356 --> 00:23:10,756 Speaker 2: for particularly scary capabilities for things that would be particularly important. 424 00:23:11,396 --> 00:23:14,276 Speaker 2: And this includes things like an opening. Eyes actually started 425 00:23:14,276 --> 00:23:17,196 Speaker 2: doing this and inthropics also doing something like this is 426 00:23:17,236 --> 00:23:21,796 Speaker 2: testing sort of if you ask the system to walk you, 427 00:23:21,996 --> 00:23:26,516 Speaker 2: a layperson, through building a new biologic weapon, through sort 428 00:23:26,556 --> 00:23:30,076 Speaker 2: of seeding the start of a new pandemic in your garage, 429 00:23:30,796 --> 00:23:33,476 Speaker 2: will it Will it help you or will it help you? 430 00:23:33,516 --> 00:23:35,836 Speaker 2: Sort of much much better than just googling around or 431 00:23:35,876 --> 00:23:37,236 Speaker 2: talking to your friend of the PhD. 432 00:23:37,356 --> 00:23:39,356 Speaker 1: And so then you have to think of all of 433 00:23:39,396 --> 00:23:42,916 Speaker 1: the versions of that. You can think of whatever shutting 434 00:23:42,956 --> 00:23:46,116 Speaker 1: down the electric grid, poisoning the water supply, building a 435 00:23:46,156 --> 00:23:47,916 Speaker 1: nuclear bomb, right, I mean, are there people who are 436 00:23:47,916 --> 00:23:50,436 Speaker 1: just making that list and making sure that chat GPT 437 00:23:50,596 --> 00:23:51,196 Speaker 1: can't do it? 438 00:23:51,436 --> 00:23:53,236 Speaker 2: There are people who are making this list, and I'm 439 00:23:53,276 --> 00:23:54,876 Speaker 2: not sure there are enough of them, and I'm not 440 00:23:54,876 --> 00:23:58,476 Speaker 2: sure they are involved in testing every system that's being built. Yeah, 441 00:23:58,516 --> 00:24:00,236 Speaker 2: but it's kind of yeah, running through this checklist of 442 00:24:00,636 --> 00:24:02,876 Speaker 2: what are the capabilities these systems could have that would 443 00:24:02,916 --> 00:24:07,436 Speaker 2: be just really disruptive, that we don't want to move 444 00:24:07,476 --> 00:24:09,756 Speaker 2: fast and break things with that we want to see 445 00:24:09,756 --> 00:24:12,676 Speaker 2: coming and we want these to sort of influence our 446 00:24:12,716 --> 00:24:15,716 Speaker 2: decisions about what actually gets deployed when and where. 447 00:24:15,516 --> 00:24:19,676 Speaker 1: And that there's not some unpredictable abercadabra that nobody can see, 448 00:24:19,676 --> 00:24:22,076 Speaker 1: but that three months later somebody will figure out. 449 00:24:21,956 --> 00:24:24,596 Speaker 2: Right, Yeah. This is the big gap of this is 450 00:24:25,276 --> 00:24:27,236 Speaker 2: I think we can say, Okay, once your system is 451 00:24:27,236 --> 00:24:30,596 Speaker 2: this dangerous, only deploy it if it's really under control. 452 00:24:30,996 --> 00:24:32,516 Speaker 2: We don't even know how to define that. We don't 453 00:24:32,556 --> 00:24:34,876 Speaker 2: even know how you would be sure that a steamer. 454 00:24:38,436 --> 00:24:40,636 Speaker 1: We'll be back in a minute with the lightning round. 455 00:24:50,036 --> 00:24:53,516 Speaker 1: Now let's get back to the show. Okay, sign for 456 00:24:53,556 --> 00:24:54,236 Speaker 1: the lightning round. 457 00:24:55,036 --> 00:24:56,116 Speaker 2: Are you ready? All right, let's go. 458 00:24:56,636 --> 00:25:00,116 Speaker 1: Let's go. What's your favorite fictional representation of AI. 459 00:25:00,236 --> 00:25:01,916 Speaker 2: Off the top of my head, X, Mac and I 460 00:25:01,996 --> 00:25:03,916 Speaker 2: was pretty good. The premises around I think what people 461 00:25:03,916 --> 00:25:06,076 Speaker 2: in AI tend to worry about actually look not that 462 00:25:06,156 --> 00:25:08,596 Speaker 2: far off of it, except that right now we're dealing 463 00:25:08,636 --> 00:25:12,076 Speaker 2: with bots instead of instead of seductive. 464 00:25:11,716 --> 00:25:15,356 Speaker 1: Robots, I liked the vibe of X mocking a lot. 465 00:25:15,396 --> 00:25:18,316 Speaker 1: I like the aesthetic. I like how spare and empty 466 00:25:18,356 --> 00:25:22,356 Speaker 1: it is. What's your favorite theory for how lms could 467 00:25:22,396 --> 00:25:23,316 Speaker 1: destroy humanity? 468 00:25:24,796 --> 00:25:27,756 Speaker 2: Oh, there's so many options and it's so hard to 469 00:25:27,756 --> 00:25:30,676 Speaker 2: know where this goes. 470 00:25:30,516 --> 00:25:34,636 Speaker 1: The what's one that's worth mentioning because it's surprising or 471 00:25:34,636 --> 00:25:36,956 Speaker 1: because it's particularly worrisome, or for any reason. 472 00:25:38,116 --> 00:25:41,436 Speaker 2: One kind of thing I'm particularly worried about is this 473 00:25:41,556 --> 00:25:45,156 Speaker 2: sort of slow moving train wreck by way of politics 474 00:25:45,276 --> 00:25:48,436 Speaker 2: that you get sort of totalitarian states get better and 475 00:25:48,476 --> 00:25:52,476 Speaker 2: better at surveillance, political persuasion gets better and better, and 476 00:25:52,516 --> 00:25:56,876 Speaker 2: so democratic political campaigns go more and more off the rails. 477 00:25:57,836 --> 00:26:00,916 Speaker 2: You wind up with more and more to Helderan states. 478 00:26:00,916 --> 00:26:03,756 Speaker 2: They're more and more effective, and they themselves are leaning 479 00:26:03,796 --> 00:26:07,196 Speaker 2: more and more on AI to do important work. And 480 00:26:08,036 --> 00:26:11,076 Speaker 2: at that point, sort of something like an AIK doesn't 481 00:26:11,116 --> 00:26:11,836 Speaker 2: seem that crazy. 482 00:26:11,996 --> 00:26:14,796 Speaker 1: And in particular, what is the large language model doing 483 00:26:14,836 --> 00:26:17,676 Speaker 1: there in that story. 484 00:26:17,676 --> 00:26:20,836 Speaker 2: Persuading people one on one, surveiling people one on one, 485 00:26:21,476 --> 00:26:25,196 Speaker 2: also making political decisions, sort of deciding how resource should 486 00:26:25,196 --> 00:26:27,236 Speaker 2: be allocated and who should be empowered with any government, 487 00:26:28,076 --> 00:26:32,236 Speaker 2: and eventually making military decisions and eventually making big economic decisions. 488 00:26:32,316 --> 00:26:34,836 Speaker 2: I just sort of worry about this world where people 489 00:26:34,876 --> 00:26:37,236 Speaker 2: put more and more trust in systems because they work, 490 00:26:37,676 --> 00:26:40,556 Speaker 2: and that helps centralize things more and more into fewer 491 00:26:40,556 --> 00:26:45,396 Speaker 2: and fewer institutions, and that makes those institutions really really delicate. 492 00:26:45,436 --> 00:26:48,476 Speaker 2: And if an aisystem goes up the rails and start 493 00:26:48,516 --> 00:26:51,236 Speaker 2: doing something that even their creators don't want, that gets 494 00:26:51,516 --> 00:26:52,636 Speaker 2: pretty arbitrarily bad. 495 00:26:53,436 --> 00:26:57,036 Speaker 1: What's your favorite theory for how llms can help humanity? 496 00:26:57,596 --> 00:27:00,996 Speaker 2: I think the big ones are education and science. I 497 00:27:01,036 --> 00:27:03,236 Speaker 2: think it would be pretty cool if you could hire 498 00:27:03,316 --> 00:27:07,836 Speaker 2: a really world class like sort of Oxford Oxford tutorial 499 00:27:08,276 --> 00:27:12,196 Speaker 2: quality tutor for just everyone with access to a computer 500 00:27:12,236 --> 00:27:13,796 Speaker 2: of any kind, and that feels like. 501 00:27:13,756 --> 00:27:15,596 Speaker 1: On your phone. Close You could do it on your phone. 502 00:27:15,716 --> 00:27:18,716 Speaker 2: Yeah, yeah, And I don't think we've really figured out 503 00:27:18,716 --> 00:27:20,916 Speaker 2: how to make that work, but I think if that 504 00:27:20,956 --> 00:27:24,676 Speaker 2: really works, that could be really transformative for the better. 505 00:27:25,356 --> 00:27:27,316 Speaker 2: On science, I think there's a lot of just really 506 00:27:27,396 --> 00:27:31,516 Speaker 2: thorny problems around things like drug development, things like sort 507 00:27:31,516 --> 00:27:34,716 Speaker 2: of fusion power and clean energy, where it could be 508 00:27:34,756 --> 00:27:37,996 Speaker 2: that just having these systems that can kind of digest 509 00:27:38,036 --> 00:27:41,716 Speaker 2: more information understand more at once could unlock a bunch 510 00:27:41,756 --> 00:27:44,076 Speaker 2: of important stuff that would otherwise take us many more 511 00:27:44,076 --> 00:27:45,316 Speaker 2: generations to get to. 512 00:27:46,596 --> 00:27:53,516 Speaker 1: On balance, you think the potential upside of AI outweighs 513 00:27:53,516 --> 00:27:55,116 Speaker 1: the potential downside. 514 00:27:55,636 --> 00:27:58,596 Speaker 2: Probably, But I think that really depends on us being 515 00:27:58,636 --> 00:28:01,556 Speaker 2: careful right now. I think this makes me optimistic in 516 00:28:01,596 --> 00:28:05,156 Speaker 2: the long run, but I think there's a there's a 517 00:28:05,196 --> 00:28:07,276 Speaker 2: real chance that things sort of go off the rails 518 00:28:07,356 --> 00:28:10,596 Speaker 2: if this keeps being kind of a free for all 519 00:28:10,676 --> 00:28:13,076 Speaker 2: commercial product for more than a few more years. 520 00:28:14,196 --> 00:28:19,036 Speaker 1: You went viral on Twitter a while ago when you wrote, quote, 521 00:28:20,276 --> 00:28:23,716 Speaker 1: doing a PhD is in most cases a terrible idea 522 00:28:24,716 --> 00:28:28,556 Speaker 1: you should put out have a PhD. Also, it's worth 523 00:28:28,556 --> 00:28:31,396 Speaker 1: pointing out that PhDs have been saying this for I 524 00:28:31,396 --> 00:28:35,276 Speaker 1: guess as long as there have been PhDs. So there's 525 00:28:35,316 --> 00:28:37,996 Speaker 1: a lot of questions you could ask here. Well, there's 526 00:28:38,036 --> 00:28:41,316 Speaker 1: two really, like why do people with PhDs keep saying 527 00:28:41,316 --> 00:28:43,396 Speaker 1: don't get a PhD? And also why do people keep 528 00:28:43,476 --> 00:28:46,556 Speaker 1: ignoring them? Why do people keep going to get PhDs? 529 00:28:48,236 --> 00:28:52,676 Speaker 2: This was in a moment of being being particularly horrified 530 00:28:52,796 --> 00:28:55,756 Speaker 2: at some of the sort of common outcomes and PhD programs, 531 00:28:56,996 --> 00:28:59,956 Speaker 2: and I think the average case is really bad the 532 00:29:00,036 --> 00:29:02,876 Speaker 2: average case, literally, I think the median PhD gets an 533 00:29:02,876 --> 00:29:07,156 Speaker 2: actual diagnosis of depression or anxiety and often doesn't get 534 00:29:07,196 --> 00:29:08,716 Speaker 2: that much out of the program, like kind of really 535 00:29:08,756 --> 00:29:12,196 Speaker 2: struggle in it, and because they're really struggling in it, 536 00:29:12,796 --> 00:29:15,756 Speaker 2: don't accomplish that much and don't have great job prospects 537 00:29:15,756 --> 00:29:18,116 Speaker 2: near the end. The best case, if you get a 538 00:29:18,156 --> 00:29:20,756 Speaker 2: sort of top five percent PhD, it's really great. You 539 00:29:20,796 --> 00:29:23,716 Speaker 2: get to play around with great resources and do whatever 540 00:29:23,716 --> 00:29:25,436 Speaker 2: you want and explore new ideas for a few years 541 00:29:25,956 --> 00:29:30,516 Speaker 2: and it opens up really tremendous opportunities. But yeah, I 542 00:29:30,516 --> 00:29:32,876 Speaker 2: think it's the kind of thing that people should really 543 00:29:34,276 --> 00:29:37,836 Speaker 2: really check their motivations and check their resilience before going 544 00:29:37,836 --> 00:29:41,596 Speaker 2: into it and kind of brace themselves just because it 545 00:29:41,676 --> 00:29:45,556 Speaker 2: is so often such a such a difficult experience. 546 00:29:45,716 --> 00:29:48,156 Speaker 1: Why do you think people keep going to get PhDs? 547 00:29:48,636 --> 00:29:50,236 Speaker 2: I mean, there is some real upside. There are some 548 00:29:50,276 --> 00:29:52,276 Speaker 2: really cool jobs that you can only get if you 549 00:29:52,276 --> 00:29:54,556 Speaker 2: have one. But I think there's also this piece, and 550 00:29:54,596 --> 00:29:57,636 Speaker 2: this is maybe why I had my snippy tweet about this, 551 00:29:58,196 --> 00:30:02,756 Speaker 2: that if you're a sort of smart, nerdy college student 552 00:30:02,916 --> 00:30:05,756 Speaker 2: at a research university where you've got lots of opportunities 553 00:30:05,756 --> 00:30:09,396 Speaker 2: to kind of work in research labs. Then you can 554 00:30:09,396 --> 00:30:11,876 Speaker 2: get this really strong social signal that just like you're 555 00:30:11,916 --> 00:30:14,556 Speaker 2: good at school, you should keep doing school, Like doing 556 00:30:14,556 --> 00:30:16,556 Speaker 2: APHD is what it looks like, keep doing school. This 557 00:30:16,676 --> 00:30:19,796 Speaker 2: is just the obvious way to use your talents and 558 00:30:20,476 --> 00:30:22,276 Speaker 2: just kind of jump into that out of momentum, and 559 00:30:22,396 --> 00:30:24,756 Speaker 2: that's that can be I think a pretty a riskier 560 00:30:24,756 --> 00:30:25,716 Speaker 2: decision than it looks like. 561 00:30:26,516 --> 00:30:29,236 Speaker 1: If everything goes well, what problem will you be trying 562 00:30:29,276 --> 00:30:31,596 Speaker 1: to solve in say, five years? 563 00:30:32,516 --> 00:30:36,116 Speaker 2: But I don't know. I got into this stuff sort 564 00:30:36,116 --> 00:30:38,596 Speaker 2: of through the cognitive science, sort of through the idea 565 00:30:38,636 --> 00:30:42,356 Speaker 2: that you don't really understand something until you can build it, 566 00:30:42,756 --> 00:30:45,516 Speaker 2: and I really want to understand how minds work, why 567 00:30:45,556 --> 00:30:48,036 Speaker 2: it is that sort of hooking neurons together in your 568 00:30:48,036 --> 00:30:50,796 Speaker 2: head this way makes something that can think and that 569 00:30:50,796 --> 00:30:55,276 Speaker 2: can experience and sort of mixed in with all of 570 00:30:55,316 --> 00:30:57,876 Speaker 2: this very consequential real world stuff that's going on with AI, 571 00:30:58,636 --> 00:31:01,156 Speaker 2: as we're building all these tools, we're also building really 572 00:31:01,196 --> 00:31:03,556 Speaker 2: great tools for just doing cognitive science and sort of 573 00:31:03,796 --> 00:31:06,956 Speaker 2: figuring out the answers to a lot of really old 574 00:31:07,036 --> 00:31:11,876 Speaker 2: questions about how the human mind works and if all 575 00:31:11,876 --> 00:31:14,236 Speaker 2: the practical problems are solved and everything's under control and 576 00:31:14,356 --> 00:31:16,476 Speaker 2: going great, then I'd be happy to get back into 577 00:31:16,476 --> 00:31:16,916 Speaker 2: that stuff. 578 00:31:17,436 --> 00:31:19,436 Speaker 1: So you would have to be less worried about the 579 00:31:19,436 --> 00:31:20,676 Speaker 1: world than you are now. 580 00:31:21,316 --> 00:31:21,996 Speaker 2: I think that's right. 581 00:31:23,116 --> 00:31:25,476 Speaker 1: Well, I hope it goes. I hope you become less 582 00:31:25,476 --> 00:31:27,956 Speaker 1: worried about the world. I guess I'm not super optimistic 583 00:31:28,036 --> 00:31:30,996 Speaker 1: about that. I feel like I'm generally a reasonably optimistic person, 584 00:31:31,036 --> 00:31:33,916 Speaker 1: but this one seems seems like there's a lot to 585 00:31:33,916 --> 00:31:34,796 Speaker 1: worry about on this one. 586 00:31:34,836 --> 00:31:39,356 Speaker 2: Yeah, yeah, thanks, thanks for the well wishes. And yet 587 00:31:39,396 --> 00:31:42,116 Speaker 2: it feels like it feels like sort of decent chance. 588 00:31:42,116 --> 00:31:44,996 Speaker 2: Things go badly, decent chant things go very well. But 589 00:31:45,076 --> 00:31:47,916 Speaker 2: I'm it seems pretty sure that stuff is just getting weird, 590 00:31:48,276 --> 00:31:50,156 Speaker 2: that research five years from now is not going to 591 00:31:50,156 --> 00:31:53,276 Speaker 2: look like research now, and probably saying with many, many, 592 00:31:53,276 --> 00:31:55,596 Speaker 2: many other things we do. 593 00:32:01,836 --> 00:32:04,836 Speaker 1: Sam Bowman is an associate professor at NYU, and he 594 00:32:04,916 --> 00:32:07,556 Speaker 1: runs a research group at the AI Company and THROP. 595 00:32:08,116 --> 00:32:11,556 Speaker 1: Today's show was edited by Lydia Jean Kott. It was 596 00:32:11,596 --> 00:32:15,756 Speaker 1: produced by David Jah and Edith Russelo and engineered by 597 00:32:15,836 --> 00:32:19,516 Speaker 1: Amanda k Wong. I'm Jacob Goldstein and We'll be back 598 00:32:19,556 --> 00:32:31,876 Speaker 1: next week with another episode of What's Your Problem.