Speaker 1: Pushkin. Today's show is about no knowns and unknown unknowns,
Speaker 1: which is to say, we're talking about AI, specifically a
Speaker 1: type of AI called a large language model, or an LM.
Speaker 1: The most famous LLM is CHAT GPT, but there are
Speaker 1: lots of others, and at their core, they all do
Speaker 1: the same thing. They read a piece of text and
Speaker 1: they predict what the next series of words should be.
Speaker 1: Lms are, obviously and quite suddenly, a huge deal in
Speaker 1: a lot of ways. One thing about them that is
Speaker 1: particularly wild to me. Lms behave in ways that are
Speaker 1: surprising even to the people who built them. In other words,
Speaker 1: large language models are this profoundly powerful, disruptive new thing,
Speaker 1: and right now we urgently need to figure out what
Speaker 1: they mean and how they work. I'm Jacob Goldstein and
Speaker 1: this is What's Your Problem, the show where I talk
Speaker 1: to people who are trying to make technological progress. My
Speaker 1: guest today is Sam Bowman. He's an expert in large
Speaker 1: language models in lms. He's on the faculty at NYU,
Speaker 1: and he runs a research group at an AI company
Speaker 1: called Anthropic. All the reason talk about lms inspired Sam
Speaker 1: to write a paper to clear up what he thought
Speaker 1: were some misconceptions. The paper is called eight Things to
Speaker 1: Know about Large Language Models. I am a fan of
Speaker 1: lists in general, and I loved this list in particular.
Speaker 1: Among other things, it gave me a deeper sense of
Speaker 1: the ways in which large language models are still a mystery,
Speaker 1: even to experts like Sam. That mystery, those unknowns, have
Speaker 1: important implications for the way we think about, and regulate
Speaker 1: and develop AI. We're going to start by discussing a
Speaker 1: pretty simple item on Sam's list. The item is this,
Speaker 1: brief interactions with llms are often misleading. You write this,
Speaker 1: You write, brief interactions with lms are often misleading. What's
Speaker 1: that mean?
Speaker 2: So when, especially when GPD four came out, and I
Speaker 2: guess also went when chat GPT first came out, there
Speaker 2: was very predictably this wave of people on Twitter saying, hey,
Speaker 2: this system is sentient and it knows where I live
Speaker 2: and it's ready to take over the world tomorrow because
Speaker 2: they had one chat with it and it said that
Speaker 2: it was sentient and it made a few educated guesses
Speaker 2: that happened to be bright, and you'll get other people
Speaker 2: on Twitter saying, hey, this system is dumb as bricks.
Speaker 2: I told it a really simple story and ask it
Speaker 2: what happened in the story and it got it wrong.
Speaker 2: There's a couple of things going on here. There's this
Speaker 2: great analogy that came up in a recent I think
Speaker 2: Time article by hell in Time owner saying they're basically
Speaker 2: improv players, where if you put them in some situation,
Speaker 2: if you put them in this situation of, oh, this
Speaker 2: is a conversation between a human who thinks the AI
Speaker 2: is sentient and the AI, then maybe the AA is
Speaker 2: going to say it's sentient.
Speaker 1: So specifically, they're improv players in the sense that famously
Speaker 1: an improv you're supposed to say yes to everything that
Speaker 1: your improv partner suggests, and so CHATCHYPT and the other
Speaker 1: llms are there to say yes, yes, and and that's
Speaker 1: what's going on.
Speaker 2: That's a decent part of it. Yeah, they're going to
Speaker 2: say yes. They're going to go along with what you're
Speaker 2: doing if you make it clear what you expect, if
Speaker 2: you make it clear, like what kind of narrative you're
Speaker 2: putting them in, what kind of environment you're putting them in,
Speaker 2: they'll go along with that.
Speaker 1: Uh, there are a couple of items on your list
Speaker 1: that seems directly contrary to assertions I've heard from other
Speaker 1: people about LMS, so that's fun and exciting. One is
Speaker 1: human performance on a task is not an upper bound
Speaker 1: on LM performance.
Speaker 2: So one of the reasons I think these systems can
Speaker 2: be better at a lot of tasks than humans is
Speaker 2: just that they've learned more stuff that they've read and
Speaker 2: mostly memorized, not just sort of all of the important
Speaker 2: papers in one little branch of chemistry or all of
Speaker 2: the important papers in all of chemistry. They've just read
Speaker 2: and mostly memorized, sort of all of the research papers.
Speaker 1: In everything, all of the papers in everything.
Speaker 2: Yeah, and many of the novels and many of the
Speaker 2: many of the news stories. And even if these systems
Speaker 2: aren't really great at drawing connections between these and sort
Speaker 2: of synthesizing a new knowledge out of them, they can
Speaker 2: do that a little bit. So you can sort of
Speaker 2: imagine what happens if you get someone who's not especially bright,
Speaker 2: but basically reasonably intelligent, reasonably competent person who's just gotten
Speaker 2: a PhD in every single thing you can get a
Speaker 2: PhD in, I'd expect them to figure some things out
Speaker 2: and to be able to do some things that no
Speaker 2: one person can do, and probably they'll notice some things
Speaker 2: that that'll be really hard for even a team or
Speaker 2: an organization to do, just because it really it's important
Speaker 2: that it kind of is, in some sense living in
Speaker 2: this one person's head.
Speaker 1: Let me just like lean into that one for a sec.
Speaker 1: So do you think that in some amount of time
Speaker 1: in the next few years, say, an LM will make
Speaker 1: some kind of you know, breakthrough in knowledge, will figure
Speaker 1: something out that no human has ever figured out that
Speaker 1: will be a meaningful breakthrough.
Speaker 2: Yeah, I think so almost. By definition, I don't have
Speaker 2: a good guess of what that's going to look like
Speaker 2: or that's going to be.
Speaker 3: Otherwise you'd be figuring it out right now, right, yeah, yeah, yeah,
Speaker 3: But no, I can imagine some story like, hey, kind
Speaker 3: of a bunch of chemists in this field of chemists
Speaker 3: have noticed this thing, and some biologists in this other
Speaker 3: subfield have noticed this other thing, and some doctors have
Speaker 3: noticed this third thing, and together they mean that some
Speaker 3: very unexpected kind of drug design might treat some new disease.
Speaker 2: And maybe if you had enough medical researchers trying enough
Speaker 2: different things, eventually they'd stumble into that. But it seems
Speaker 2: possible at some point that something like a large language
Speaker 2: model is just going to notice that, and if you
Speaker 2: ask it the right way, it's going to tell you,
Speaker 2: and you might have to second guess it a lot.
Speaker 2: These systems also make stuff up, But I think it's
Speaker 2: quite possible that you start seeing these things pretty often
Speaker 2: tell you surprising new things that happen to be true.
Speaker 1: There's another item on your list that seems to me
Speaker 1: to be like a provocation. It seems to me in
Speaker 1: a good way. It seems like directly contradictory to what
Speaker 1: I have read, specifically to this idea that all large
Speaker 1: language models are doing is guessing what the next word
Speaker 1: in a series is likely to be, and that list
Speaker 1: item is this. Llms often appear to learn and use
Speaker 1: representations of the outside world. Llms often appear to learn
Speaker 1: and use representations of the outside world. So that sounds
Speaker 1: quite different from just guessing the next word, is it
Speaker 1: or is it not? Different in a way that I
Speaker 1: just don't understand.
Speaker 2: It turns out it's not that different. Okay, this is
Speaker 2: I want to say it's the big discovery, But it's
Speaker 2: this big discovery that's spread out over dozens of experiments
Speaker 2: over the last few years.
Speaker 1: Can you give me a specific example. It's such an
Speaker 1: abstract assertion that I think it would be helpful to
Speaker 1: have a specific example.
Speaker 2: That we can think about. One great example of this
Speaker 2: is if you tell a model a story, a simple
Speaker 2: story that takes place in some sort of physical space
Speaker 2: where it's it's some characters walking around a house and
Speaker 2: they're having a conversation while they're walking, and they're picking
Speaker 2: style up and they're putting it down. You can see
Speaker 2: inside the activations of the neurons when the model is
Speaker 2: reading that story. You can pull out a map of
Speaker 2: the house. You can see that there's a there's a
Speaker 2: piece the network that says, oh, okay, now they're in
Speaker 2: the living room, and another piece that says, oh, living
Speaker 2: room is connected to the bedroom. And you can mess
Speaker 2: with this in ways that show that it's really sort
Speaker 2: of it is really representing the house. That if you
Speaker 2: find the piece of the network that says, oh, Susan
Speaker 2: is in the living room, and you flip that, flip
Speaker 2: that from a positive number to a negative number, then
Speaker 2: the story will continue as though Susan is not in
Speaker 2: a lot in the living room, or couldn't possibly have
Speaker 2: been in living.
Speaker 1: So that does seem like it's representing the physical world
Speaker 1: in a way that is not just guessing the next word.
Speaker 2: Yeah. Yeah, so we're finding out these systems are actually
Speaker 2: representing the objects they're talking about, at least some of
Speaker 2: the time.
Speaker 1: They're creating a representation of physical space.
Speaker 2: Yeah. I should be clear that this is this doesn't
Speaker 2: always work when when you're giving these systems something really
Speaker 2: hard and subtle, they're just going to totally botch this stuff.
Speaker 2: Their internal representations are a mess. But more and more
Speaker 2: of the time they're really doing it. And as these
Speaker 2: things get bigger and bigger, they're doing it more and more.
Speaker 2: And so this feels like this important turning point where
Speaker 2: it's like, oh, okay, there is some understanding going on
Speaker 2: here and it's getting better, and that really radically opens
Speaker 2: up the possibilities for where this technology might go.
Speaker 1: This what you're saying seems very much at odds with
Speaker 1: what people generally say about llms, Right, Like the standard
Speaker 1: line is they're just predicting what the next word is
Speaker 1: going to be. And they're very good at predicting what
Speaker 1: the next word is going to be, and there's a
Speaker 1: lot of powerful things you can do, but what you're
Speaker 1: saying sounds fundamentally different from that. And so I mean,
Speaker 1: are the people saying they're just predicting the next word?
Speaker 1: Are they wrong? Is what you're saying a point of
Speaker 1: debate among experts or what? Why is this so different
Speaker 1: than what I've heard before.
Speaker 2: There's a few things going on. So first, saying that
Speaker 2: they're just predicting the next word is mostly right. But
Speaker 2: it turns out that's saying that they just predict the
Speaker 2: next word is a lot like saying humans are just
Speaker 2: chemical reactions. It turns out that if you're trying to
Speaker 2: predict the next word, and if you've got a smaller
Speaker 2: work that's trying to predict the next word, it's going
Speaker 2: to learn that sort of the word, the and of
Speaker 2: an a and those show up often, and that's about
Speaker 2: all it's going to learn. If you take a medium
Speaker 2: sized neural network, it's going to learn how to write
Speaker 2: fluent sentences. This is going to write, oh, okay, sort
Speaker 2: of adjectives come before nouns, these kinds of nouns come
Speaker 2: before these kinds of nouns. It might even learn some facts.
Speaker 2: It might learn that if you talk about the president
Speaker 2: of the United States, you'll get names like Obama and
Speaker 2: Bush and Biden and Trump, and it'll start to kind
Speaker 2: of make sense, but it's still just kind of learning statistics.
Speaker 2: And if you make the neural work even bigger, it
Speaker 2: will abstract further away. It will start to reason about
Speaker 2: the people and the objects and the spaces themselves and
Speaker 2: use that abstraction to predict the next word. So kind
Speaker 2: of the more these systems learn about the world, the
Speaker 2: farther and farther their Internet representations get from just sort
Speaker 2: of literally what word comes after what other word.
Speaker 1: So there's another item on your list that seems like
Speaker 1: it should have interesting implications for the AI industry, right
Speaker 1: for the business of building lms, I'll just read that one.
Speaker 1: It goes lms predictably get more capable with increasing investment,
Speaker 1: even without targeted innovation. So we'll get into it. But
Speaker 1: just top line, what does that mean?
Speaker 2: We had language models in almost their modern form back
Speaker 2: in twenty ten, eleven, twelve. Most of the building blocks
Speaker 2: for them go back even farther to the eighties or
Speaker 2: even the sixties. You might have noticed that we weren't
Speaker 2: We didn't have chat GBT ten or twenty or fifty
Speaker 2: years ago. What people have been gradually discovering and dually
Speaker 2: sort of discovering to a greater and greater degree is
Speaker 2: that if you just take this reldly simple technology and
Speaker 2: throw more data at it and run it in its
Speaker 2: sort of training phase for longer and longer by fancier
Speaker 2: or and France your computers to run it on, it
Speaker 2: just keeps getting better.
Speaker 1: But if the technology is not special, I mean, everybody
Speaker 1: knows the basic sauce, it suggests that GPT might not
Speaker 1: have an open AI. The company that makes chat GPT
Speaker 1: might not have like that much of a moat, right.
Speaker 1: I mean, Google is clearly in this business, as is Anthropic,
Speaker 1: the company where you're working. Is there any reason to
Speaker 1: think open AI GPT is going to stay ahead.
Speaker 2: I think there's not a lot of secret sauce. There
Speaker 2: are some details of how to build these things that
Speaker 2: don't get published, but the basic idea is very much
Speaker 2: out there. And yeah, I think the the closest thing
Speaker 2: you can really have to emote is just enormous amounts
Speaker 2: of money. I think at some point you're going to
Speaker 2: have a relatively small number of labs building the really
Speaker 2: impressive frontier systems just because at some point these are
Speaker 2: going to be ten billion dollar projects, and it just
Speaker 2: seems unlikely that you're going to get that many ten
Speaker 2: billion dollar projects.
Speaker 1: If it's the case, as you say that, essentially what
Speaker 1: you need to build a frontier level LM is a
Speaker 1: lot of money. I would guess that governments around the world,
Speaker 1: certainly say China to pick a salient government, are probably
Speaker 1: building giant lms right now. Does that seem like a
Speaker 1: reasonable guess?
Speaker 2: Yeah, that seems right. I know there are a lot
Speaker 2: of private and private, public and public groups in China
Speaker 2: working in this stuff, and when I sort of hear
Speaker 2: people in the field who are following the geopolitical side
Speaker 2: of this more closely, they're paying a lot of attention
Speaker 2: to things like the Chips Act and Global Trade in
Speaker 2: chips in that you really do need. When you're spending
Speaker 2: these millions or billions of dollars, you're basically spending them
Speaker 2: to buy or rent very fancy, state of the art
Speaker 2: computer chips. And it has become a priority for the
Speaker 2: US to try to make it hard for China to
Speaker 2: do that, and.
Speaker 1: To try and make it hard for China to get
Speaker 1: at the processor level, which in a sense is like
Speaker 1: the cement that lllms are built from. There is a
Speaker 1: physical thing. We forget that, but it's fancy chips basically.
Speaker 2: That's right.
Speaker 1: Yeah, we've been talking so far about what we know
Speaker 1: about how large language models work. After the break, we'll
Speaker 1: get into what I think is the most interesting thing
Speaker 1: about lms, what we don't know about how they work.
Speaker 1: That's the end of the ads.
Speaker 2: Now we're going back to the show.
Speaker 1: So far, we've basically been talking about how do lllms work.
Speaker 1: What's going on? There is another bucket in your list,
Speaker 1: several items, three items that are it seems to me,
Speaker 1: in quite a different category, and they get at this
Speaker 1: very very interesting idea about lms, and that is, to
Speaker 1: some significant degree, nobody knows how they work. The people
Speaker 1: who build lms, people like you, people who build them
Speaker 1: and study them, don't understand a lot of what is
Speaker 1: going on, which is amazing to me and super interesting.
Speaker 1: So let's start with this list item. It says specific
Speaker 1: important behaviors in lms tend to emerge unpredictably as a byproduct.
Speaker 2: Of increasing investment.
Speaker 1: And you give a couple of examples of this happening
Speaker 1: for real in the world. I think the best way
Speaker 1: to understand what's going on here is to talk about
Speaker 1: one of those examples. Can you just like talk me
Speaker 1: through one of those examples of this unpredictable new behavior emerging. Yeah.
Speaker 2: So a specific large language model that people working in
Speaker 2: the stuff talk about a lot is GPD three. This
Speaker 2: came out a little less than three years ago and
Speaker 2: I think sort of kicked off the modern wave of
Speaker 2: research on this stuff. And one thing researchers would do,
Speaker 2: as these systems would would come out is give them
Speaker 2: math puzzles and logic puzzles and see how they did.
Speaker 2: And this could be as simple as just sort of
Speaker 2: giving the model reasonably hard arithmetic, sort of asking the model,
Speaker 2: what is one hundred and twenty five plus four hundred
Speaker 2: and sixty seven. And what they found is sort of
Speaker 2: GPD one was bad at this, and GPD two was
Speaker 2: bad at this, and at least for some of these tasks,
Speaker 2: GPD three was also bad at this. And they released it.
Speaker 2: They put it out in the world, they wrote a
Speaker 2: paper about it, they did some demos to researchers, and
Speaker 2: then eventually just let anyone sign up and use it.
Speaker 2: And after a few months people started noticing. Oh, there
Speaker 2: are some tricks you can use to actually make it
Speaker 2: quite a bit better at this. If you ask the
Speaker 2: model the right way, sometimes it'll just kind of reason
Speaker 2: out loud. Sometimes it will say, well, it'll actually do
Speaker 2: long edition, we'll actually write out its steps.
Speaker 1: So give me a specific example. How do you ask
Speaker 1: it the right way?
Speaker 2: So it took even a few more months for people
Speaker 2: to figure out how to do this systematically, but it
Speaker 2: turned out the trick was you literally say, let's think
Speaker 2: step by step.
Speaker 1: You actually type that in, you say that to the machine,
Speaker 1: to the model.
Speaker 2: Yes, And if you say what is this number of
Speaker 2: plus this number question mark, it'll give a wrong answer.
Speaker 2: If you say, what is this number of plus this number,
Speaker 2: let's think step by step dot dot, it's going to
Speaker 2: list out. Okay, let's start with the ones digit, and
Speaker 2: then the tenth digit, and then the one hundredth digit,
Speaker 2: and then give you the answer, and it'll very often
Speaker 2: be right huh. And it turns out this works really
Speaker 2: generally that for many kinds of sort of math and
Speaker 2: reasoning problems, even some even sort of ethics problems. There's
Speaker 2: a huge range of things you might ask one of
Speaker 2: these ural networks to do where if you just tell it,
Speaker 2: let's think step by step, it will bring out this
Speaker 2: whole reasoning ability that is actually really useful, that allows
Speaker 2: it to do much better at a lot of things,
Speaker 2: and that it didn't have before. And when this technology
Speaker 2: was first released, the people who built it, they did
Speaker 2: not know this was a possibility.
Speaker 1: That's wild, right, Like it means this thing is incredibly
Speaker 1: powerful in a way that the people who built it
Speaker 1: didn't know. And let's think step by step is just
Speaker 1: like this incantation. It's just like saying abracadabra or something,
Speaker 1: and the builders didn't know it was there.
Speaker 2: Yeah, it's it's a bizarre time to be working on
Speaker 2: this stuff.
Speaker 1: It Like, here's where it's getting a little sketchy to
Speaker 1: me at a certain level, right, I mean you've also
Speaker 1: done a lot of work in AI safety and this
Speaker 1: kind of section of the interview, I feel like we're
Speaker 1: getting more toward that, the section of like, the people
Speaker 1: building this stuff don't understand what it can do. And
Speaker 1: here should we add another list item here? Like this
Speaker 1: might be the place Cherkiff, So there's this other item
Speaker 1: on your eight things to know list that seems germane.
Speaker 1: Here experts are not yet able to interpret the inner
Speaker 1: workings of lms, which also wild also kind of goes
Speaker 1: with this idea of not knowing what the thing can do,
Speaker 1: right and very not intuitive for a piece of technology.
Speaker 1: Right If you go back to say the Internet, Sure
Speaker 1: we didn't know all the social implications of the Internet,
Speaker 1: but we knew how the technology worked. We knew what
Speaker 1: was going on with the chips and the wires and
Speaker 1: the electrons and whatever. Right, Like the amazing thing here
Speaker 1: is clearly we don't know the social implications of AI.
Speaker 1: But you're saying, we don't even know what it's doing
Speaker 1: inside the box.
Speaker 2: Yeah, that's right. We've got these very crude tools for
Speaker 2: sort of opening the box and looking inside. I mean,
Speaker 2: in a literal sense, we know it's going on. We
Speaker 2: can say, oh, when you put in this word, then
Speaker 2: it makes this number bigger, which makes that number smaller,
Speaker 2: which makes this number bigger. And you could keep saying
Speaker 2: that for twenty years and then you'd have explained what happened.
Speaker 2: But we haven't figured out any other way of talking
Speaker 2: about these systems that actually gives us any clarity about
Speaker 2: what's possible why these systems are doing what they're doing
Speaker 2: where they're reliable and not it's just this huge mess
Speaker 2: of connections that we don't really know what to do with.
Speaker 1: I mean, what should we make of this set of
Speaker 1: facts that these are incredibly powerful tools that nobody understands
Speaker 1: at a pretty deep level, that can do unpredictable things,
Speaker 1: that are able to do things that even their makers
Speaker 1: don't know they can do.
Speaker 2: I think it's pretty exciting and also pretty sobering. I
Speaker 2: think we don't have a good way of predicting how
Speaker 2: fast this is moving or what we're going to get when.
Speaker 2: But in the big picture, it seems like there's a
Speaker 2: lot of momentum toward building these really powerful eye systems
Speaker 2: over the next few years. We don't understand how they work.
Speaker 2: Another one of these list items is we also aren't
Speaker 2: very good at controlling, and we aren't very good at
Speaker 2: making them do what we want.
Speaker 1: Yes, let me just pause there, because it's the last
Speaker 1: list item and you have just walked up to it.
Speaker 1: So the last item, the item that we haven't mentioned
Speaker 1: on your list. There are no reliable techniques for steering
Speaker 1: the behavior of lms, so they're powerful. We don't really
Speaker 1: understand how they work. They can do things we don't
Speaker 1: know they're going to do, and we can't really control them.
Speaker 1: Now we're through the list. Now let's just talk it out.
Speaker 2: Yeah, and so we're yeah, we're building, we're building these systems.
Speaker 2: They're getting better, the developing new capabilities. We don't really
Speaker 2: know how they work. We can't predict which capabilities are
Speaker 2: showing up when and if they're doing something we don't want.
Speaker 2: We don't really know how to notice that and mitigate
Speaker 2: it and prevent it. And that just feels like it's
Speaker 2: playing with fire at a scale what I'm not sure
Speaker 2: we've seen before, at least outside of things like nuclear weapons.
Speaker 2: It's this very sort of sobering situation to be in.
Speaker 1: What do we do about it?
Speaker 2: I'm not sure. I wish I had a better answer.
Speaker 2: There are a few things that will definitely help. Maybe
Speaker 2: one obvious thing here is just there's probably a lot
Speaker 2: of regulation that would be good to have here. You
Speaker 2: really don't want the move fast and break things ethos
Speaker 2: to be behind a technology that is close to human
Speaker 2: level ability at a lot of cognitive task That seems
Speaker 2: like the setup for a bad sci fi movie.
Speaker 1: Specifically, what regulation do you think is a good idea?
Speaker 2: One outline of an idea that I'm excited about, and
Speaker 2: I think this is definitely not the best idea or
Speaker 2: the only good idea is mandating or standardizing some tests
Speaker 2: for particularly scary capabilities for things that would be particularly important.
Speaker 2: And this includes things like an opening. Eyes actually started
Speaker 2: doing this and inthropics also doing something like this is
Speaker 2: testing sort of if you ask the system to walk you,
Speaker 2: a layperson, through building a new biologic weapon, through sort
Speaker 2: of seeding the start of a new pandemic in your garage,
Speaker 2: will it Will it help you or will it help you?
Speaker 2: Sort of much much better than just googling around or
Speaker 2: talking to your friend of the PhD.
Speaker 1: And so then you have to think of all of
Speaker 1: the versions of that. You can think of whatever shutting
Speaker 1: down the electric grid, poisoning the water supply, building a
Speaker 1: nuclear bomb, right, I mean, are there people who are
Speaker 1: just making that list and making sure that chat GPT
Speaker 1: can't do it?
Speaker 2: There are people who are making this list, and I'm
Speaker 2: not sure there are enough of them, and I'm not
Speaker 2: sure they are involved in testing every system that's being built. Yeah,
Speaker 2: but it's kind of yeah, running through this checklist of
Speaker 2: what are the capabilities these systems could have that would
Speaker 2: be just really disruptive, that we don't want to move
Speaker 2: fast and break things with that we want to see
Speaker 2: coming and we want these to sort of influence our
Speaker 2: decisions about what actually gets deployed when and where.
Speaker 1: And that there's not some unpredictable abercadabra that nobody can see,
Speaker 1: but that three months later somebody will figure out.
Speaker 2: Right, Yeah. This is the big gap of this is
Speaker 2: I think we can say, Okay, once your system is
Speaker 2: this dangerous, only deploy it if it's really under control.
Speaker 2: We don't even know how to define that. We don't
Speaker 2: even know how you would be sure that a steamer.
Speaker 1: We'll be back in a minute with the lightning round.
Speaker 1: Now let's get back to the show. Okay, sign for
Speaker 1: the lightning round.
Speaker 2: Are you ready? All right, let's go.
Speaker 1: Let's go. What's your favorite fictional representation of AI.
Speaker 2: Off the top of my head, X, Mac and I
Speaker 2: was pretty good. The premises around I think what people
Speaker 2: in AI tend to worry about actually look not that
Speaker 2: far off of it, except that right now we're dealing
Speaker 2: with bots instead of instead of seductive.
Speaker 1: Robots, I liked the vibe of X mocking a lot.
Speaker 1: I like the aesthetic. I like how spare and empty
Speaker 1: it is. What's your favorite theory for how lms could
Speaker 1: destroy humanity?
Speaker 2: Oh, there's so many options and it's so hard to
Speaker 2: know where this goes.
Speaker 1: The what's one that's worth mentioning because it's surprising or
Speaker 1: because it's particularly worrisome, or for any reason.
Speaker 2: One kind of thing I'm particularly worried about is this
Speaker 2: sort of slow moving train wreck by way of politics
Speaker 2: that you get sort of totalitarian states get better and
Speaker 2: better at surveillance, political persuasion gets better and better, and
Speaker 2: so democratic political campaigns go more and more off the rails.
Speaker 2: You wind up with more and more to Helderan states.
Speaker 2: They're more and more effective, and they themselves are leaning
Speaker 2: more and more on AI to do important work. And
Speaker 2: at that point, sort of something like an AIK doesn't
Speaker 2: seem that crazy.
Speaker 1: And in particular, what is the large language model doing
Speaker 1: there in that story.
Speaker 2: Persuading people one on one, surveiling people one on one,
Speaker 2: also making political decisions, sort of deciding how resource should
Speaker 2: be allocated and who should be empowered with any government,
Speaker 2: and eventually making military decisions and eventually making big economic decisions.
Speaker 2: I just sort of worry about this world where people
Speaker 2: put more and more trust in systems because they work,
Speaker 2: and that helps centralize things more and more into fewer
Speaker 2: and fewer institutions, and that makes those institutions really really delicate.
Speaker 2: And if an aisystem goes up the rails and start
Speaker 2: doing something that even their creators don't want, that gets
Speaker 2: pretty arbitrarily bad.
Speaker 1: What's your favorite theory for how llms can help humanity?
Speaker 2: I think the big ones are education and science. I
Speaker 2: think it would be pretty cool if you could hire
Speaker 2: a really world class like sort of Oxford Oxford tutorial
Speaker 2: quality tutor for just everyone with access to a computer
Speaker 2: of any kind, and that feels like.
Speaker 1: On your phone. Close You could do it on your phone.
Speaker 2: Yeah, yeah, And I don't think we've really figured out
Speaker 2: how to make that work, but I think if that
Speaker 2: really works, that could be really transformative for the better.
Speaker 2: On science, I think there's a lot of just really
Speaker 2: thorny problems around things like drug development, things like sort
Speaker 2: of fusion power and clean energy, where it could be
Speaker 2: that just having these systems that can kind of digest
Speaker 2: more information understand more at once could unlock a bunch
Speaker 2: of important stuff that would otherwise take us many more
Speaker 2: generations to get to.
Speaker 1: On balance, you think the potential upside of AI outweighs
Speaker 1: the potential downside.
Speaker 2: Probably, But I think that really depends on us being
Speaker 2: careful right now. I think this makes me optimistic in
Speaker 2: the long run, but I think there's a there's a
Speaker 2: real chance that things sort of go off the rails
Speaker 2: if this keeps being kind of a free for all
Speaker 2: commercial product for more than a few more years.
Speaker 1: You went viral on Twitter a while ago when you wrote, quote,
Speaker 1: doing a PhD is in most cases a terrible idea
Speaker 1: you should put out have a PhD. Also, it's worth
Speaker 1: pointing out that PhDs have been saying this for I
Speaker 1: guess as long as there have been PhDs. So there's
Speaker 1: a lot of questions you could ask here. Well, there's
Speaker 1: two really, like why do people with PhDs keep saying
Speaker 1: don't get a PhD? And also why do people keep
Speaker 1: ignoring them? Why do people keep going to get PhDs?
Speaker 2: This was in a moment of being being particularly horrified
Speaker 2: at some of the sort of common outcomes and PhD programs,
Speaker 2: and I think the average case is really bad the
Speaker 2: average case, literally, I think the median PhD gets an
Speaker 2: actual diagnosis of depression or anxiety and often doesn't get
Speaker 2: that much out of the program, like kind of really
Speaker 2: struggle in it, and because they're really struggling in it,
Speaker 2: don't accomplish that much and don't have great job prospects
Speaker 2: near the end. The best case, if you get a
Speaker 2: sort of top five percent PhD, it's really great. You
Speaker 2: get to play around with great resources and do whatever
Speaker 2: you want and explore new ideas for a few years
Speaker 2: and it opens up really tremendous opportunities. But yeah, I
Speaker 2: think it's the kind of thing that people should really
Speaker 2: really check their motivations and check their resilience before going
Speaker 2: into it and kind of brace themselves just because it
Speaker 2: is so often such a such a difficult experience.
Speaker 1: Why do you think people keep going to get PhDs?
Speaker 2: I mean, there is some real upside. There are some
Speaker 2: really cool jobs that you can only get if you
Speaker 2: have one. But I think there's also this piece, and
Speaker 2: this is maybe why I had my snippy tweet about this,
Speaker 2: that if you're a sort of smart, nerdy college student
Speaker 2: at a research university where you've got lots of opportunities
Speaker 2: to kind of work in research labs. Then you can
Speaker 2: get this really strong social signal that just like you're
Speaker 2: good at school, you should keep doing school, Like doing
Speaker 2: APHD is what it looks like, keep doing school. This
Speaker 2: is just the obvious way to use your talents and
Speaker 2: just kind of jump into that out of momentum, and
Speaker 2: that's that can be I think a pretty a riskier
Speaker 2: decision than it looks like.
Speaker 1: If everything goes well, what problem will you be trying
Speaker 1: to solve in say, five years?
Speaker 2: But I don't know. I got into this stuff sort
Speaker 2: of through the cognitive science, sort of through the idea
Speaker 2: that you don't really understand something until you can build it,
Speaker 2: and I really want to understand how minds work, why
Speaker 2: it is that sort of hooking neurons together in your
Speaker 2: head this way makes something that can think and that
Speaker 2: can experience and sort of mixed in with all of
Speaker 2: this very consequential real world stuff that's going on with AI,
Speaker 2: as we're building all these tools, we're also building really
Speaker 2: great tools for just doing cognitive science and sort of
Speaker 2: figuring out the answers to a lot of really old
Speaker 2: questions about how the human mind works and if all
Speaker 2: the practical problems are solved and everything's under control and
Speaker 2: going great, then I'd be happy to get back into
Speaker 2: that stuff.
Speaker 1: So you would have to be less worried about the
Speaker 1: world than you are now.
Speaker 2: I think that's right.
Speaker 1: Well, I hope it goes. I hope you become less
Speaker 1: worried about the world. I guess I'm not super optimistic
Speaker 1: about that. I feel like I'm generally a reasonably optimistic person,
Speaker 1: but this one seems seems like there's a lot to
Speaker 1: worry about on this one.
Speaker 2: Yeah, yeah, thanks, thanks for the well wishes. And yet
Speaker 2: it feels like it feels like sort of decent chance.
Speaker 2: Things go badly, decent chant things go very well. But
Speaker 2: I'm it seems pretty sure that stuff is just getting weird,
Speaker 2: that research five years from now is not going to
Speaker 2: look like research now, and probably saying with many, many,
Speaker 2: many other things we do.
Speaker 1: Sam Bowman is an associate professor at NYU, and he
Speaker 1: runs a research group at the AI Company and THROP.
Speaker 1: Today's show was edited by Lydia Jean Kott. It was
Speaker 1: produced by David Jah and Edith Russelo and engineered by
Speaker 1: Amanda k Wong. I'm Jacob Goldstein and We'll be back
Speaker 1: next week with another episode of What's Your Problem.