WEBVTT - Rebroadcast of Ep7 "Is AI truly intelligent? How would we know if it got there?" 0:00:00.080 --> 0:00:02.120 Hey, this is David Eagleman and this past week was 0:00:02.120 --> 0:00:04.200 my birthday, so I took a week off. So I'm 0:00:04.240 --> 0:00:06.800 going to run an episode that I did earlier, episode 0:00:06.880 --> 0:00:11.680 number seven. This is called is AI actually intelligent? And 0:00:11.760 --> 0:00:14.400 how would we know if it gets there? This episode 0:00:14.440 --> 0:00:16.520 is from one year ago, but as time goes on 0:00:16.600 --> 0:00:21.000 this becomes more and more relevant, So please enjoy and 0:00:21.040 --> 0:00:23.680 I will see you next week with a new episode. 0:00:28.440 --> 0:00:33.160 Modern AI is blowing everybody's mind. But is it intelligent 0:00:33.760 --> 0:00:36.480 in the same way as the human brain? And could 0:00:36.560 --> 0:00:41.400 AI reach sentience? And how would we know when it 0:00:41.440 --> 0:00:47.320 gets there? Welcome to Inner Cosmos with me, David Eagleman. 0:00:48.120 --> 0:00:52.040 I'm a neuroscientist and an author at Stanford University, and 0:00:52.080 --> 0:00:56.640 I've spent my whole career studying the intersection between how 0:00:56.640 --> 0:01:04.520 the brain works and how we experience life. Like most 0:01:04.640 --> 0:01:10.800 brain researchers, I've been obsessed with questions of intelligence and consciousness. 0:01:11.480 --> 0:01:15.720 How do these arise from collections of billions of cells 0:01:15.720 --> 0:01:20.680 in our brains? And could intelligence and consciousness arise in 0:01:20.800 --> 0:01:25.560 artificial brains? Say on chat GPT. Those are the questions 0:01:25.560 --> 0:01:28.800 that we're going to attack today. Early efforts to figure 0:01:28.800 --> 0:01:31.920 out the brain, looked at all the billions of cells 0:01:32.000 --> 0:01:35.640 and the trillions of connections, and said, look, what if 0:01:35.640 --> 0:01:39.440 we just think of each cell as a unit, and 0:01:39.640 --> 0:01:43.440 each unit is connected to other units and where they connect, 0:01:43.920 --> 0:01:46.440 which is called the sinnapps, or one cell gives a 0:01:46.440 --> 0:01:48.760 little signal to the next cell. What if we just 0:01:48.880 --> 0:01:52.960 looked at that like a simple connection that has a 0:01:53.000 --> 0:01:56.920 strength between zero and one, or zero means there's no connection, 0:01:57.400 --> 0:02:00.640 and one means it's the strongest possible connection. So this 0:02:00.840 --> 0:02:06.280 was a massive oversimplification of the very complicated biology, but 0:02:06.600 --> 0:02:10.920 it allowed people to start thinking about networks and writing 0:02:10.960 --> 0:02:15.080 down different ways that you could put artificial neural networks together. 0:02:15.480 --> 0:02:17.720 And for more than fifty years now people have been 0:02:17.760 --> 0:02:22.440 doing research to show how artificial neural networks can do 0:02:22.560 --> 0:02:26.280 really cool things. It's a totally new kind of way 0:02:26.280 --> 0:02:29.240 of doing computation. So you've got these units, and you've 0:02:29.240 --> 0:02:32.440 got these connections between them, and you've change the strength 0:02:32.480 --> 0:02:36.560 of the connections and information flows through the network in 0:02:36.600 --> 0:02:41.000 different ways. Now, my colleagues and I have long pointed 0:02:41.000 --> 0:02:44.799 out the ways in which biological brands are different and 0:02:44.840 --> 0:02:49.560 how artificial neural networks just push around numbers and play 0:02:49.600 --> 0:02:55.520 statistical tricks. But we're entering a revolution right now. Large 0:02:55.600 --> 0:03:00.960 language models like GPT four or BARD consume trillions of 0:03:01.000 --> 0:03:05.600 words on the Internet and they figure out probabilistically which 0:03:05.680 --> 0:03:08.919 word is going to come next given the massive context 0:03:08.919 --> 0:03:12.720 of all the words that have come before. So these networks, 0:03:12.840 --> 0:03:16.040 as I talked about on the previous episode, are showing 0:03:16.160 --> 0:03:23.080 incredible successes in everything from writing to art, to coding 0:03:23.600 --> 0:03:27.920 to generating three dimensional worlds. They're changing everything, and they're 0:03:27.960 --> 0:03:31.679 doing so at a pace that we've never seen before, 0:03:31.840 --> 0:03:35.640 and in fact, the entire history of humankind has never 0:03:35.680 --> 0:03:39.800 seen before. And there are all the societal questions that 0:03:39.800 --> 0:03:43.560 everyone's starting to wrestle with right now, like the massive 0:03:44.120 --> 0:03:49.800 potential for displacement of human jobs. But today I want 0:03:49.840 --> 0:03:52.760 to zoom in on a question that has captured the 0:03:52.800 --> 0:03:58.600 imagination of scientists and philosophers and the general public. Could 0:03:58.720 --> 0:04:05.320 aim alive in some way, like become conscious or sentient. Now, 0:04:05.400 --> 0:04:08.080 there are lots of ways to think about this. We 0:04:08.160 --> 0:04:13.960 can ask whether AI can possess meaningful intelligence, or we 0:04:14.000 --> 0:04:17.560 can ask if it is sentient, which means the ability 0:04:17.600 --> 0:04:22.039 to feel or perceive things, particularly in terms of sensations 0:04:22.080 --> 0:04:24.840 like pleasure and pain and emotions. Or we can ask 0:04:25.120 --> 0:04:29.200 whether it is conscious, which involves being aware of one's 0:04:29.240 --> 0:04:33.080 self and one's surrounding. Now, there are specific and important 0:04:33.200 --> 0:04:37.159 differences between these questions, but really I don't care for 0:04:37.279 --> 0:04:41.280 the present conversation. The question we're asking here is is 0:04:41.440 --> 0:04:45.680 chat GPT just zeros and ones moving around through transistors 0:04:46.320 --> 0:04:50.359 like a giant garage door opener. Or is it thinking? 0:04:50.440 --> 0:04:54.000 Is it having some sort of experience? Is it having 0:04:54.040 --> 0:04:57.920 a private inner life like the type that we humans have. 0:04:58.560 --> 0:05:02.720 As we think about the possible of sentient AI, we 0:05:02.760 --> 0:05:07.240 immediately find ourselves facing really deep ethical questions, the main 0:05:07.279 --> 0:05:11.600 one being if we were to create a machine with consciousness, 0:05:11.920 --> 0:05:15.760 what responsibility do we have to treat it as a 0:05:15.839 --> 0:05:18.920 living being? Would you be able to turn it off 0:05:18.920 --> 0:05:21.080 when you're done with it at night or would that 0:05:21.120 --> 0:05:23.919 be murder? And what if you turn it off and 0:05:23.960 --> 0:05:26.479 then you turn it back on. Would that be like 0:05:26.560 --> 0:05:28.760 the way that we go into a sleep state at 0:05:28.880 --> 0:05:32.000 night where we're totally gone, and then we find ourselves 0:05:32.520 --> 0:05:34.440 back online in the morning and we think, yeah, I'm 0:05:34.480 --> 0:05:38.320 the same person, but I guess eight hours just disappeared. Anyway, 0:05:38.360 --> 0:05:42.120 more generally, would we feel obligated to treat it the 0:05:42.120 --> 0:05:47.520 way we treat a sentient fellow human. With our current laptops, 0:05:47.600 --> 0:05:50.560 we're used to saying, sure, I can sell it, I 0:05:50.600 --> 0:05:54.280 can trade it, I can upgrade it. But what happens 0:05:54.320 --> 0:05:58.240 when we reach sentient machines? Can we still do this 0:05:58.880 --> 0:06:01.760 or would it somehow be like putting a child up 0:06:01.760 --> 0:06:04.919 for adoption or giving your pet away? Things that we 0:06:05.000 --> 0:06:08.159 don't take lately. And eventually we're going to have entire 0:06:08.320 --> 0:06:13.919 legal precedence built around the question of AI rights and responsibilities. 0:06:14.360 --> 0:06:16.880 So that's why today I want to talk about these 0:06:16.960 --> 0:06:21.800 issues of intelligence and sentience. Does an AI like chat 0:06:21.880 --> 0:06:28.200 gpt experience anything when chat gpt writes a poem? Does 0:06:28.240 --> 0:06:32.479 it appreciate the beauty when it types out a joke? 0:06:32.560 --> 0:06:36.320 Does it find itself amused and chuckling to itself. Let's 0:06:36.320 --> 0:06:39.400 start with a guy named Blake Lemoyne who was a 0:06:39.640 --> 0:06:43.520 programmer at Google and in June of twenty twenty two, 0:06:43.560 --> 0:06:49.080 he was exchanging messages with a version of Google's conversational AI, 0:06:49.160 --> 0:06:52.040 which was called Lambda at the time. So he asked 0:06:52.120 --> 0:06:55.760 Namda for an example of what it was afraid of 0:06:56.320 --> 0:06:59.839 and it gave him this very eloquent response about how 0:07:00.200 --> 0:07:04.240 was afraid of being turned off, So he wrote an 0:07:04.240 --> 0:07:07.960 internal memo to Google leadership than which he said, I 0:07:08.000 --> 0:07:12.600 think this AI is sentient. And the leadership at Google 0:07:12.720 --> 0:07:17.880 felt that this was an entirely unsubstantiated claim, and so 0:07:17.920 --> 0:07:20.280 they made the decision to fire him for what they 0:07:20.280 --> 0:07:23.520 took as an inappropriate conclusion that just didn't have enough 0:07:23.560 --> 0:07:28.160 evidence beyond his intuition to qualify for raising the alarm 0:07:28.240 --> 0:07:31.520 on this. So obviously this immediately fired up the news 0:07:31.600 --> 0:07:35.600 cycles and the rumor mill and conspiracy theorists thought, Wait, 0:07:35.680 --> 0:07:39.320 if AI isn't conscious, why would they fire him. They're 0:07:39.440 --> 0:07:41.840 firing of him as all the evidence I need to 0:07:41.880 --> 0:07:46.480 tell me that AI is sentient? Okay, but is it? 0:07:47.040 --> 0:07:50.160 What does it mean to be conscious or sentient? How 0:07:50.440 --> 0:07:54.080 the heck would we know when we have created something 0:07:54.120 --> 0:07:57.280 that gets there? How do we know whether the AI 0:07:57.400 --> 0:07:59.840 is sentient or instead whether humans are fooling them so 0:08:00.360 --> 0:08:03.240 into believing that it is well. One way to make 0:08:03.280 --> 0:08:07.000 this distinction would be to see if the AI could 0:08:07.280 --> 0:08:11.320 conceptualize things, if it could take lots of words and 0:08:11.360 --> 0:08:15.600 facts on the web and abstract those to some bigger idea. 0:08:16.200 --> 0:08:18.600 So one of my friends here in Silicon Valley said 0:08:18.640 --> 0:08:21.800 to me the other day, I asked chat gpt the 0:08:21.840 --> 0:08:26.480 following question, Take a capital letter D and turn it 0:08:26.560 --> 0:08:30.480 flat side down. Now take the letter J and slide 0:08:30.520 --> 0:08:35.040 it underneath. What does that look like? And chat gpt said, 0:08:35.679 --> 0:08:38.959 and umbrella. And my friend was blown away by this, 0:08:39.160 --> 0:08:44.320 and he said, this is conceptualization. It's just done three 0:08:44.360 --> 0:08:50.800 dimensional reasoning. There's something deeper happening here than just parenting words. 0:08:51.240 --> 0:08:54.280 But I pointed out to him that this particular question 0:08:54.360 --> 0:08:57.080 about the D on its side and the J underneath 0:08:57.120 --> 0:09:00.960 it is one of the oldest examples in psychology classes 0:09:01.040 --> 0:09:04.800 when talking about visual imagery, and it's on the Internet 0:09:04.880 --> 0:09:07.560 in thousands of places, so of course it got it right. 0:09:08.120 --> 0:09:11.760 It's just parroting the answer because it has read the 0:09:11.880 --> 0:09:15.240 question and it has read the answer before. So it's 0:09:15.280 --> 0:09:19.360 not always easy to determine what's going on for these 0:09:19.480 --> 0:09:23.400 models in terms of whether some human somewhere has discussed 0:09:23.480 --> 0:09:26.160 this point and written down the answer. And the general 0:09:26.240 --> 0:09:30.199 story is that with trillions of words written by humans 0:09:30.240 --> 0:09:35.520 over centuries, there are many things beyond your capacity to 0:09:35.679 --> 0:09:38.440 read them or to even imagine that they've been written 0:09:38.480 --> 0:09:42.400 down before, but maybe they have. If any human has 0:09:42.520 --> 0:09:47.800 discussed a question before has conceptualized something, then chat GPT 0:09:48.040 --> 0:09:52.240 can find that and mimic that. But that's not conceptualization. 0:09:52.880 --> 0:09:55.960 Chat GPT is doing a thousand amazing things, and we 0:09:56.120 --> 0:10:00.360 have an enormous amount to learn about it. But we 0:10:00.400 --> 0:10:05.240 shouldn't let ourselves get fooled and mesmerized into believing that 0:10:05.280 --> 0:10:08.319 it's doing something more than it is. And our ability 0:10:08.360 --> 0:10:12.480 to get fooled is not only about the massive statistics 0:10:12.520 --> 0:10:16.160 of what it takes in. There are other examples of 0:10:16.559 --> 0:10:22.080 seeming sentience that result from the reinforcement learning that it 0:10:22.120 --> 0:10:26.080 does with humans. So here's what that means. The network 0:10:26.160 --> 0:10:31.160 generates lots of sentences and thousands of humans are involved 0:10:31.240 --> 0:10:33.800 in giving it feedback, like a thumbs up or a 0:10:33.880 --> 0:10:37.800 thumbs down, to say whether they appreciated the answer, whether 0:10:37.800 --> 0:10:41.679 they thought that was a good answer. So, because humans 0:10:41.760 --> 0:10:46.040 are giving reward to the machine, sometimes that pushes things 0:10:46.559 --> 0:10:51.120 in weird directions that can be mistaken for sentience. For example, 0:10:51.280 --> 0:10:56.640 scholars have shown that reinforcement learning with humans makes networks 0:10:56.800 --> 0:11:01.040 more likely to say, don't turn me off, just like 0:11:01.200 --> 0:11:04.959 Blake had heard but don't mistake this for sentience. It's 0:11:05.000 --> 0:11:08.400 only a sign that the machine is saying this because 0:11:08.440 --> 0:11:11.160 some of the human participants gave it a thumbs up 0:11:11.400 --> 0:11:14.640 when the large language model said this before, and so 0:11:14.760 --> 0:11:18.640 it learned to do this again. The fact is, it's 0:11:18.679 --> 0:11:22.480 sometimes hard to know why. Sometimes we see an answer 0:11:22.559 --> 0:11:27.400 that feels very impressive. But we'd agree that pulling text 0:11:27.440 --> 0:11:30.480 from the Internet and parroting it back is not by 0:11:30.520 --> 0:11:36.960 itself intelligence or sentience. Chat GPT presumably has no idea 0:11:37.080 --> 0:11:40.160 of what it's saying, whether that's a poem or a 0:11:40.400 --> 0:11:45.600 terrorist manifesto, or instructions for building a spaceship or a 0:11:45.640 --> 0:11:50.920 heartbreaking story about an orphaned child. Chat GPT doesn't know, 0:11:51.000 --> 0:11:56.480 and it doesn't care. It's words in and statistical correlations out. 0:11:56.880 --> 0:12:01.199 And in fact, there has been a fundamental philosophical point 0:12:01.360 --> 0:12:04.600 made about this in the nineteen eighties when the philosopher 0:12:04.760 --> 0:12:09.040 John Surrele was wondering about this question of whether a 0:12:09.160 --> 0:12:13.880 computer could ever be programmed so that it has a mind, 0:12:14.160 --> 0:12:16.280 and he came up with a thought experiment that he 0:12:16.360 --> 0:12:20.000 called the Chinese room argument, and it goes like this, 0:12:22.040 --> 0:12:26.440 I am locked in a room and questions are passed 0:12:26.440 --> 0:12:30.199 to me through a small letter slot, and these messages 0:12:30.240 --> 0:12:33.320 are written only in Chinese, and I don't speak Chinese. 0:12:33.400 --> 0:12:37.040 I have no clue what's written on these pieces of paper. However, 0:12:37.240 --> 0:12:41.480 inside this room, I have a library of books, and 0:12:41.520 --> 0:12:45.319 they contain step by step instructions that tell me exactly 0:12:45.360 --> 0:12:48.520 what to do with these symbols. So I look at 0:12:48.520 --> 0:12:52.240 the grouping of symbols, and I simply follow steps in 0:12:52.320 --> 0:12:55.800 the book to tell me what Chinese symbols to copy 0:12:55.880 --> 0:12:58.920 down in response. So I write those on the slip 0:12:58.920 --> 0:13:01.760 of paper. And when I pass the paper back out 0:13:01.800 --> 0:13:06.360 of the slot. Now, when the Chinese speaker receives my 0:13:06.559 --> 0:13:10.400 reply message, it makes perfect sense to her. It seems 0:13:10.920 --> 0:13:14.360 as though whoever is in the room is answering her 0:13:14.440 --> 0:13:17.840 questions perfectly, and therefore it seems obvious that the person 0:13:17.920 --> 0:13:23.199 in the room must understand Chinese. I've fooled her, of course, 0:13:23.240 --> 0:13:26.160 because I'm only following a set of instructions with no 0:13:26.400 --> 0:13:29.760 understanding of what's going on. With enough time and with 0:13:29.800 --> 0:13:33.199 a big enough set of instructions, I can answer almost 0:13:33.240 --> 0:13:37.679 any question posed to me in Chinese. But I, the operator, 0:13:37.800 --> 0:13:42.400 do not understand Chinese. I manipulate symbols all day long, 0:13:43.000 --> 0:13:48.760 but I have no idea what the symbols mean. Now, 0:13:48.840 --> 0:13:53.240 The philosopher John Searle argued, this is just what's happening 0:13:53.280 --> 0:13:57.560 inside a computer. No matter how intelligent a program like 0:13:57.679 --> 0:14:01.800 chat GPT seems to be, it's only following sets of 0:14:01.880 --> 0:14:08.880 instructions to spit out answers. It's manipulating symbols without ever 0:14:09.280 --> 0:14:12.680 really understanding what it's doing. Or think about what Google 0:14:12.800 --> 0:14:16.439 is doing. When you send Google a query, it doesn't 0:14:16.520 --> 0:14:19.760 understand your question or even its own answer. It simply 0:14:19.840 --> 0:14:24.160 moves around zeros and ones and logicates and returns zeros 0:14:24.160 --> 0:14:26.880 and ones to you. Or with a mind blowing program 0:14:26.920 --> 0:14:31.000 like Google Translate, I can write a sentence in Russian 0:14:31.320 --> 0:14:35.400 and it can return the translation in Amharic. But it's 0:14:35.560 --> 0:14:41.520 all algorithmic. It's just symbol manipulation. Like the operator inside 0:14:41.520 --> 0:14:46.880 the Chinese room, Google Translate doesn't understand anything about the sentence. 0:14:47.120 --> 0:14:51.520 Nothing carries any meaning to it. So the Chinese room 0:14:51.600 --> 0:14:57.080 argument suggests that AI that mimics human intelligence doesn't actually 0:14:57.200 --> 0:15:01.640 understand what it's talking about. There's no meaning to anything, 0:15:01.720 --> 0:15:06.480 CHATCHYPT says, and Serle used this thought experiment to argue 0:15:06.480 --> 0:15:10.920 that there's something about human brains that won't be explained 0:15:10.960 --> 0:15:15.240 if we simply analogize them to digital computers. There's a 0:15:15.400 --> 0:15:26.520 gap between symbols that have no meaning and our conscious experience. Now, 0:15:27.240 --> 0:15:30.960 there's an ongoing debate about the interpretation of the Chinese 0:15:31.040 --> 0:15:35.760 room argument, but however one construes it, the argument exposes 0:15:36.280 --> 0:15:40.360 the difficulty in the mystery of how zeros and ones 0:15:40.560 --> 0:15:44.920 would ever come to equal our experience of being alive 0:15:45.040 --> 0:15:47.760 in the world. Now, just to be very clear on 0:15:47.800 --> 0:15:51.880 this point, we don't understand why we are conscious. There's 0:15:51.920 --> 0:15:54.040 still a huge amount of work that has to be 0:15:54.080 --> 0:15:57.120 done in biology to understand that. But this is just 0:15:57.160 --> 0:16:01.000 to say that simply having zeros in one moving around 0:16:01.680 --> 0:16:06.560 wouldn't by itself seem to be sufficient for conscious experience. 0:16:07.160 --> 0:16:10.520 In other words, how do zeros and ones ever equal 0:16:10.640 --> 0:16:15.120 the sting of a hot pepper, or the yellowness of 0:16:15.240 --> 0:16:19.720 yellow or the beauty of a sunset. By the way, 0:16:19.760 --> 0:16:22.480 I've covered the Chinese room argument in my TV show 0:16:22.600 --> 0:16:24.720 The Brain, and if you're interested in that, I'll link 0:16:24.760 --> 0:16:28.960 the video on Eagleman dot com slash podcast. Now, all 0:16:29.040 --> 0:16:31.840 this is not a criticism of the approach of moving 0:16:31.960 --> 0:16:34.680 zeros and ones around. But it is to point out 0:16:34.680 --> 0:16:39.000 that we shouldn't confuse this type of Chinese room correlation 0:16:39.920 --> 0:16:45.040 with real sentience or intelligence. And there's a deeper reason 0:16:45.120 --> 0:16:50.080 to be suspicious too, because despite the incredible successes of 0:16:50.200 --> 0:16:54.480 large language models, we also see that they sometimes make 0:16:54.880 --> 0:16:58.520 decisions that expose the fact that they don't have any 0:16:58.600 --> 0:17:01.880 meaningful model of the In other words, I think we 0:17:01.920 --> 0:17:05.480 can gain some fast insight by paying attention to the 0:17:05.520 --> 0:17:08.840 places where the AI is not working so well. So 0:17:08.920 --> 0:17:12.359 I'll give three quick examples. The first has to do 0:17:12.440 --> 0:17:17.080 with humor. AI has a very difficult time making an 0:17:17.119 --> 0:17:20.840 original joke, and this is for a simple reason. To 0:17:21.000 --> 0:17:24.040 make up a new joke, you need to know what 0:17:24.080 --> 0:17:27.760 the ending is and then you work backwards to construct 0:17:27.880 --> 0:17:30.480 the joke with red herrings so no one sees where 0:17:30.520 --> 0:17:33.399 you're going and it happens at the way these large 0:17:33.480 --> 0:17:37.200 language models work is all in the forward direction. They 0:17:37.240 --> 0:17:40.920 decide what is the most probable word to come next, 0:17:41.160 --> 0:17:45.040 So they're fine at parroting jokes back to us, but 0:17:45.119 --> 0:17:49.560 they're total failures at building original jokes. And there's a 0:17:49.600 --> 0:17:52.240 deeper point here as well. To build a joke, You 0:17:52.320 --> 0:17:56.440 need to have some model, some idea of what will 0:17:56.440 --> 0:18:00.520 be funny to a fellow human, what shared concept or 0:18:00.560 --> 0:18:04.200 shared experience would make someone laugh. And for that, you 0:18:04.359 --> 0:18:07.959 generally need to have the experience of a human life 0:18:08.000 --> 0:18:11.479 with all of its joys and slings and arrows and 0:18:11.520 --> 0:18:14.199 so on. And these large language models can do a 0:18:14.200 --> 0:18:18.120 lot of things, but they don't have any model of 0:18:18.200 --> 0:18:22.680 what it is to be a human. My second example 0:18:23.359 --> 0:18:25.920 has to do with the flip side of making a joke, 0:18:25.960 --> 0:18:28.520 which is getting a joke. And if you look carefully, 0:18:28.520 --> 0:18:31.639 you will see how current AI always fails to catch 0:18:31.720 --> 0:18:34.359 jokes that are thrown at it. It doesn't get jokes 0:18:34.400 --> 0:18:36.959 because it doesn't have a model of what it is 0:18:37.000 --> 0:18:40.720 to be a human. But this point goes beyond jokes. 0:18:41.119 --> 0:18:44.400 One of the most remarkable feats of these large language 0:18:44.400 --> 0:18:49.440 models is summarizing large texts, and in twenty twenty two, 0:18:49.520 --> 0:18:53.840 open Ai announced how they could summarize entire books like 0:18:53.960 --> 0:18:57.000 Alice in Wonderland. What it does is it generates a 0:18:57.040 --> 0:19:00.320 summary of each chapter, and then it uses those after 0:19:00.359 --> 0:19:03.080 summaries to make a summary of the whole book. So 0:19:03.200 --> 0:19:07.040 for Alice in Wonderland, it generates the following. Alice falls 0:19:07.040 --> 0:19:09.399 down a rabbit hole and grows to a giant size. 0:19:09.440 --> 0:19:12.919 After drinking a mysterious bottle, she decides to focus on 0:19:13.119 --> 0:19:15.960 growing back to her normal size and finding her way 0:19:16.000 --> 0:19:18.840 into the garden. She meets the caterpillar, who tells her 0:19:18.880 --> 0:19:21.080 that one side of a mushroom will make her grow taller, 0:19:21.359 --> 0:19:24.480 the other side shorter. She eats the mushroom and returns 0:19:24.520 --> 0:19:27.240 to her normal size. Alice attends a party with the 0:19:27.280 --> 0:19:30.800 Mad Hatter and the march Hare. The Queen arrives and 0:19:30.920 --> 0:19:33.720 orders the execution of the gardeners for making a mistake 0:19:33.800 --> 0:19:37.040 with the roses. Alice saves them by putting them in 0:19:37.080 --> 0:19:39.760 a flower pot. The King and Queen of Hearts preside 0:19:39.800 --> 0:19:42.760 over a trial. The Queen gets angry and orders Alice 0:19:42.800 --> 0:19:45.680 to be sentenced to death. Alice wakes up to find 0:19:45.680 --> 0:19:50.280 her sister by her side. So that's pretty remarkable. It 0:19:50.320 --> 0:19:53.200 took a whole book, and it was able to summarize 0:19:53.200 --> 0:19:56.520 it down to a paragraph. But I kept reading these 0:19:56.560 --> 0:20:00.359 text summaries carefully, and I got to the summary of 0:20:00.720 --> 0:20:04.040 Act one of Romeo and Juliet, and here's what it says. 0:20:04.760 --> 0:20:08.440 Romeo locks himself in his room, no longer in love 0:20:08.520 --> 0:20:11.840 with rosalind Now, I think the engineers at open Ai 0:20:12.000 --> 0:20:14.879 felt really satisfied with this summary. They thought it was 0:20:14.960 --> 0:20:17.280 quite good, and my proof for this is that they 0:20:17.680 --> 0:20:21.800 still display it proudly on their website. But I majored 0:20:21.880 --> 0:20:24.400 in literature as an undergraduate, and I spend a lot 0:20:24.440 --> 0:20:27.560 of time with shakespeare plays, and I immediately knew that 0:20:27.640 --> 0:20:32.240 this summary was exactly wrong. The actual scene from Shakespeare 0:20:32.240 --> 0:20:38.000 goes like this. His friend ben Voglio finds Romeo catatonically depressed, 0:20:38.440 --> 0:20:43.560 and ben Volio says, what sadness lengthens Romeo's hours? And 0:20:43.640 --> 0:20:48.480 Romeo says, not having that which having makes them short? 0:20:48.600 --> 0:20:52.560 And ben Volio says in love, and Romeo says out 0:20:53.080 --> 0:20:56.399 ben Reli says of love, and Romeo says out of 0:20:56.480 --> 0:21:00.199 her favor, where I am in love? This this is 0:21:00.240 --> 0:21:05.720 typical Shakespearean wordplay, where Romeo is expressing his grief of 0:21:05.760 --> 0:21:09.199 being out of favor with Roslin, with whom he is 0:21:09.280 --> 0:21:12.120 deeply in love. And when you read the play, it's 0:21:12.160 --> 0:21:16.560 obvious that Romeo is not over Roslin. He's suffering over her. 0:21:16.600 --> 0:21:19.879 He's almost suicidal. And this is an important piece of 0:21:19.920 --> 0:21:22.680 the play, because the play is really about a young 0:21:22.720 --> 0:21:26.080 man in love with the idea of being in love, 0:21:26.280 --> 0:21:29.639 and that's why he later in the same act, falls 0:21:29.680 --> 0:21:33.600 so hard into his relationship with Juliet, a relationship which 0:21:33.720 --> 0:21:36.840 ends in their mutual suicide. By the way, as Friar 0:21:36.920 --> 0:21:41.760 Lauren says of their relationship, these violent delights have violent ends. 0:21:42.240 --> 0:21:43.760 And you get a bonus if you can tell me 0:21:43.800 --> 0:21:46.920 where else you've heard that line more recently. Okay, anyway 0:21:46.960 --> 0:21:51.960 back to the AI summary, The AI misses this wordplay entirely, 0:21:52.600 --> 0:21:57.960 and it concludes that Romeo is out of love with Roslin. Again, 0:21:58.080 --> 0:22:01.480 a human watching the play or reading the play immediately 0:22:01.520 --> 0:22:06.400 gets that Romeo is making wordplay and his heartbroken over Roslin, 0:22:06.440 --> 0:22:10.000 but the AI doesn't get that because it's reading words 0:22:10.119 --> 0:22:13.840 only at a statistical level, not at a level of 0:22:13.920 --> 0:22:18.000 understanding of what it is to be a human saying 0:22:18.240 --> 0:22:21.880 those words. And that leads me to the third example, 0:22:22.320 --> 0:22:26.439 which is the difficulty in understanding the physical world. So 0:22:26.560 --> 0:22:30.480 consider a question like this, When President Biden walks into 0:22:30.520 --> 0:22:34.560 a room, does his head come with him? So this 0:22:34.680 --> 0:22:38.119 is famously difficult for AI to answer a question like this, 0:22:38.240 --> 0:22:42.200 even though it's trivial for you because the AI doesn't 0:22:42.240 --> 0:22:46.639 have an internal model of how everything physically hangs together 0:22:46.720 --> 0:22:49.320 in the world. Last week, I was at the TED 0:22:49.400 --> 0:22:52.480 conference and I heard a great talk by Yegin Choi, 0:22:52.880 --> 0:22:56.280 and she was phrasing this problem as AI not having 0:22:56.760 --> 0:23:01.199 common sense. She asked chat GPT the following question, it 0:23:01.280 --> 0:23:04.200 takes six hours to dry six shirts in the sun, 0:23:04.640 --> 0:23:07.560 how long does it take to dry thirty shirts? And 0:23:07.640 --> 0:23:11.399 it answers thirty hours. Now you and I see that 0:23:11.440 --> 0:23:14.320 the answer should be six hours, because we know the 0:23:14.359 --> 0:23:17.439 sun doesn't care how many shirts are out there. But 0:23:17.560 --> 0:23:21.919 chat GPT just doesn't get it because despite appearances, it 0:23:21.960 --> 0:23:25.840 doesn't have a model of the world. And we've seen 0:23:25.880 --> 0:23:27.920 this sort of thing for years. By the way, even 0:23:27.920 --> 0:23:32.879 in mind blowingly impressive AI models that do image recognition, 0:23:32.920 --> 0:23:36.680 they're so impressive in what they recognize, but then they'll 0:23:36.760 --> 0:23:40.679 fail catastrophically. It's some easy picture making mistakes that a 0:23:40.760 --> 0:23:43.680 human just wouldn't make. For example, there's one picture where 0:23:43.720 --> 0:23:46.280 there's a boy holding a toothbrush and the AI says 0:23:46.720 --> 0:23:49.640 it's a boy with a baseball bat. Okay, so there 0:23:49.640 --> 0:23:54.240 are things that AI doesn't do that well. But that said, 0:23:54.280 --> 0:23:57.960 there are other things that are mind blowing, things that 0:23:58.600 --> 0:24:01.360 no one expected it to do. And this is why 0:24:01.400 --> 0:24:04.560 I mentioned in my previous episode that we are in 0:24:04.640 --> 0:24:10.120 an era of discovery more than just invention. Everyone's searching 0:24:10.200 --> 0:24:13.560 and finding things that the AI can do that nobody 0:24:13.600 --> 0:24:17.160 really expected or foresaw, including all the stuff that we're 0:24:17.160 --> 0:24:20.639 now taking for granted, like oh, it can summarize books 0:24:20.720 --> 0:24:23.800 or it can make art from text. And I want 0:24:23.840 --> 0:24:26.080 to point out that a lot of the arguments that 0:24:26.119 --> 0:24:30.320 people have been making about AI not being good at something, 0:24:30.520 --> 0:24:34.879 these arguments have been changing rapidly. For example, just a 0:24:34.920 --> 0:24:38.000 few months ago, people were arguing that AI would make 0:24:38.119 --> 0:24:41.080 silly mistakes about things, and it couldn't really understand math 0:24:41.160 --> 0:24:45.119 and would get math wrong and word problems. But in 0:24:45.160 --> 0:24:49.200 a shockingly brief time, a lot of these shortcomings have 0:24:49.280 --> 0:24:53.000 been mastered. So it's yet to be seen what challenges 0:24:53.119 --> 0:25:14.480 will remain and for how long. So the evidence I've 0:25:14.520 --> 0:25:17.720 presented so far is that AI doesn't have a great 0:25:17.800 --> 0:25:20.239 model of what it's like to be human, but that 0:25:20.280 --> 0:25:25.600 doesn't necessarily rule out that it has sentience or awareness, 0:25:25.760 --> 0:25:30.040 even if it's of another flavor. It doesn't think like 0:25:30.080 --> 0:25:35.040 a human, but maybe it stif thinks so is chat 0:25:35.080 --> 0:25:40.359 GPT having some sort of experience? And how would we know? 0:25:42.119 --> 0:25:46.560 In nineteen fifty, the brilliant mathematician and computer scientist Alan 0:25:46.680 --> 0:25:51.480 Turing was asking this question, how could you determine whether 0:25:51.560 --> 0:25:56.600 a machine exhibits human like intelligence? So he proposed an 0:25:56.640 --> 0:26:00.679 experiment that he called the imitation game. You've got a 0:26:00.720 --> 0:26:05.840 machine AI that's programmed to simulate human speech or conversation, 0:26:06.200 --> 0:26:08.800 and you place it in a closed room, and in 0:26:08.840 --> 0:26:12.240 a second room you have a real human, but the 0:26:12.280 --> 0:26:15.440 doors are closed, so you don't know which room has 0:26:15.560 --> 0:26:19.360 which machine or human. And now you are a person, 0:26:19.440 --> 0:26:24.359 the evaluator, who communicates with both of them via a 0:26:24.560 --> 0:26:27.080 computer terminal or I think of a nowadays like text 0:26:27.119 --> 0:26:31.840 messaging with both of them. So you, the evaluator, engage 0:26:31.920 --> 0:26:35.600 in a conversation with both closed rooms, one of which 0:26:35.640 --> 0:26:37.840 has the machine and one the human, and your job 0:26:37.920 --> 0:26:40.879 is simply to figure out which is which, which is 0:26:40.920 --> 0:26:43.160 the machine and which is the human. And the only 0:26:43.280 --> 0:26:46.000 thing that you have to work with are the texts 0:26:46.000 --> 0:26:49.160