WEBVTT - Ep7 "Is AI truly intelligent? How would we know if it got there?" 0:00:05.840 --> 0:00:10.560 Modern AI is blowing everybody's mind. But is it intelligent 0:00:11.119 --> 0:00:13.920 in the same way as the human brain? And could 0:00:13.920 --> 0:00:18.760 AI reach sentience? And how would we know when it 0:00:18.800 --> 0:00:24.680 gets there? Welcome to Inner Cosmos with me, David Eagleman. 0:00:26.200 --> 0:00:30.880 I'm a neuroscientist and an author at Stanford University, and 0:00:30.920 --> 0:00:35.440 I've spent my whole career studying the intersection between how 0:00:35.479 --> 0:00:43.360 the brain works and how we experience life. Like most 0:00:43.479 --> 0:00:49.640 brain researchers, I've been obsessed with questions of intelligence and consciousness. 0:00:50.320 --> 0:00:54.560 How do these arise from collections of billions of cells 0:00:54.560 --> 0:00:59.440 in our brains? And could intelligence and consciousness arise in 0:00:59.560 --> 0:01:04.400 artificial brains? Say on chat GPT. Those are the questions 0:01:04.400 --> 0:01:07.640 that we're going to attack today. Early efforts to figure 0:01:07.640 --> 0:01:10.759 out the brain looked at all the billions of cells 0:01:10.840 --> 0:01:14.479 and the trillions of connections and said, look, what if 0:01:14.480 --> 0:01:18.280 we just think of each cell as a unit, and 0:01:18.440 --> 0:01:22.280 each unit is connected to other units and where they connect, 0:01:22.760 --> 0:01:25.280 which is called the sinnapps, or one cell gives a 0:01:25.280 --> 0:01:27.600 little signal to the next cell. What if we just 0:01:27.720 --> 0:01:31.760 looked at that like a simple connection that has a 0:01:31.840 --> 0:01:35.760 strength between zero and one, where zero means there's no connection, 0:01:36.200 --> 0:01:39.480 and one means it's the strongest possible connection. So this 0:01:39.680 --> 0:01:45.080 was a massive oversimplification of the very complicated biology, but 0:01:45.440 --> 0:01:49.760 it allowed people to start thinking about networks and writing 0:01:49.800 --> 0:01:53.880 down different ways that you could put artificial neural networks together. 0:01:54.280 --> 0:01:56.560 And for more than fifty years now people have been 0:01:56.600 --> 0:02:01.280 doing research to show how artificial neural networks can do 0:02:01.400 --> 0:02:05.120 really cool things. It's a totally new kind of way 0:02:05.120 --> 0:02:08.080 of doing computation. So you've got these units and you've 0:02:08.080 --> 0:02:11.280 got these connections between them, and you change the strength 0:02:11.320 --> 0:02:15.400 of the connections and information flows through the network in 0:02:15.440 --> 0:02:19.800 different ways. Now, my colleagues and I have long pointed 0:02:19.840 --> 0:02:23.600 out the ways in which biological brands are different and 0:02:23.680 --> 0:02:28.400 how artificial neural networks just push around numbers and play 0:02:28.400 --> 0:02:34.360 statistical tricks. But we're entering a revolution right now. Large 0:02:34.440 --> 0:02:39.800 language models like GPT four or BARD consume trillions of 0:02:39.840 --> 0:02:44.440 words on the Internet and they figure out probabilistically which 0:02:44.520 --> 0:02:47.760 word is going to come next given the massive context 0:02:47.760 --> 0:02:51.519 of all the words that have come before. So these networks, 0:02:51.680 --> 0:02:54.880 as I talked about on the previous episode, are showing 0:02:55.000 --> 0:03:01.880 incredible successes in everything from writing to art, to coding 0:03:02.440 --> 0:03:06.760 to generating three dimensional worlds. They're changing everything, and they're 0:03:06.800 --> 0:03:10.600 doing so at a pace that we've never seen before, 0:03:10.680 --> 0:03:14.480 and in fact, the entire history of humankind is never 0:03:14.520 --> 0:03:18.600 seen before. And there are all the societal questions that 0:03:18.639 --> 0:03:22.399 everyone's starting to wrestle with right now, like the massive 0:03:22.960 --> 0:03:28.639 potential for displacement of human jobs. But today I want 0:03:28.680 --> 0:03:31.600 to zoom in on a question that has captured the 0:03:31.639 --> 0:03:37.440 imagination of scientists and philosophers and the general public. Could 0:03:37.560 --> 0:03:44.200 AI come alive in some way like become conscious or sentient? Now, 0:03:44.240 --> 0:03:46.920 there are lots of ways to think about this. We 0:03:47.000 --> 0:03:52.800 can ask whether AI can possess meaningful intelligence, or we 0:03:52.840 --> 0:03:56.400 can ask if it is sentient, which means the ability 0:03:56.440 --> 0:04:00.880 to feel or perceive things, particularly in terms of sensations 0:04:00.920 --> 0:04:03.680 like pleasure and pain and emotions. Where we can ask 0:04:03.920 --> 0:04:08.320 whether it is conscious, which involves being aware of oneself 0:04:08.400 --> 0:04:12.680 and one's surrounding. Now, there are specific and important differences 0:04:12.800 --> 0:04:16.200 between these questions, but really I don't care for the 0:04:16.320 --> 0:04:20.559 present conversation. The question we're asking here is is chat 0:04:20.600 --> 0:04:25.240 GPT just zeros and ones moving around through transistors like 0:04:25.400 --> 0:04:29.359 a giant garage door opener, Or is it thinking? Is 0:04:29.360 --> 0:04:32.960 it having some sort of experience? Is it having a 0:04:33.120 --> 0:04:36.760 private inner life like the type that we humans have. 0:04:37.400 --> 0:04:41.520 As we think about the possibility of sentient AI, we 0:04:41.600 --> 0:04:46.080 immediately find ourselves facing really deep ethical questions, the main 0:04:46.120 --> 0:04:50.440 one being if we were to create a machine with consciousness, 0:04:50.760 --> 0:04:54.600 what responsibility do we have to treat it as a 0:04:54.680 --> 0:04:57.760 living being? Would you be able to turn it off 0:04:57.760 --> 0:04:59.839 when you're done with it at night or would that 0:04:59.880 --> 0:05:02.760 be murder? And what if you turn it off and 0:05:02.800 --> 0:05:05.320 then you turn it back on. Would that be like 0:05:05.360 --> 0:05:07.560 the way that we go into a sleep state at 0:05:07.680 --> 0:05:10.839 night where we're totally gone, and then we find ourselves 0:05:11.320 --> 0:05:13.280 back online in the morning and we think, yeah, I'm 0:05:13.320 --> 0:05:17.159 the same person, but I guess eight hours just disappeared anyway. 0:05:17.200 --> 0:05:20.960 More generally, would we feel obligated to treat it the 0:05:20.960 --> 0:05:26.360 way we treat a sentient fellow human. With our current laptops, 0:05:26.440 --> 0:05:29.400 we're used to saying sure, I can sell it, I 0:05:29.440 --> 0:05:33.120 can trade it, I can upgrade it. But what happens 0:05:33.160 --> 0:05:37.080 when we reach sentient machines? Can we still do this 0:05:37.720 --> 0:05:40.600 or would it somehow be like putting a child up 0:05:40.600 --> 0:05:43.800 for adoption or giving your pet away? Things that we 0:05:43.839 --> 0:05:47.600 don't take lightly, and eventually we're gonna have entire legal 0:05:47.680 --> 0:05:52.760 precedence built around the question of AI rights and responsibilities. 0:05:53.200 --> 0:05:55.719 So that's why today I want to talk about these 0:05:55.760 --> 0:06:00.359 issues of intelligence and sentience. Does an AI I like 0:06:00.480 --> 0:06:06.320 chat GPT experience anything when chat gpt writes a poem? 0:06:06.839 --> 0:06:11.280 Does it appreciate the beauty when it types out a joke? 0:06:11.400 --> 0:06:15.160 Does it find itself amused and chuckling to itself. Let's 0:06:15.160 --> 0:06:18.240 start with a guy named Blake Lemoyne who was a 0:06:18.480 --> 0:06:22.359 programmer at Google, and in June of twenty twenty two, 0:06:22.400 --> 0:06:27.919 he was exchanging messages with a version of Google's conversational AI, 0:06:28.000 --> 0:06:30.880 which was called Lambda at the time. So he asked 0:06:30.960 --> 0:06:34.599 Lambda for an example of what it was afraid of, 0:06:35.160 --> 0:06:38.760 and it gave him this very eloquent response about how 0:06:38.800 --> 0:06:42.880 it was afraid of being turned off. So he wrote 0:06:42.920 --> 0:06:46.479 an internal memo to Google leadership in which he said, 0:06:46.640 --> 0:06:51.080 I think this AI is sentient. And the leadership at 0:06:51.080 --> 0:06:56.560 Google felt that this was an entirely unsubstantiated claim, and 0:06:56.600 --> 0:06:58.920 so they made the decision to fire him for what 0:06:58.960 --> 0:07:02.039 they took as an inappropriate conclusion that just didn't have 0:07:02.160 --> 0:07:06.560 enough evidence beyond his intuition to qualify for raising the 0:07:06.600 --> 0:07:10.120 alarm on this. So obviously this immediately fired up the 0:07:10.160 --> 0:07:14.480 news cycles and the rumor mill and conspiracy theorists thought, Wait, 0:07:14.520 --> 0:07:18.160 if AI isn't conscious, why would they fire him. They're 0:07:18.280 --> 0:07:20.680 firing of him as all the evidence I need to 0:07:20.720 --> 0:07:25.320 tell me that AI is sentient? Okay, but is it? 0:07:25.880 --> 0:07:29.000 What does it mean to be conscious or sentient? How 0:07:29.280 --> 0:07:32.920 the heck would we know when we have created something 0:07:32.960 --> 0:07:36.119 that gets there? How do we know whether the AI 0:07:36.240 --> 0:07:39.400 is sentient or instead whether humans are fooling themselves into 0:07:39.400 --> 0:07:42.240 believing that it is. Well. One way to make this 0:07:42.320 --> 0:07:47.760 distinction would be to see if the AI could conceptualize things, 0:07:47.880 --> 0:07:50.640 if it could take lots of words and facts on 0:07:50.680 --> 0:07:55.120 the web and abstract those to some bigger idea. So 0:07:55.200 --> 0:07:57.560 one of my friends here in Silicon Valley said to 0:07:57.560 --> 0:08:01.440 me the other day, I asked chat gpt the following question. 0:08:02.120 --> 0:08:06.440 Take a capital letter D and turn it flat side down. 0:08:07.040 --> 0:08:10.720 Now take the letter J and slide it underneath. What 0:08:10.880 --> 0:08:15.480 does that look like? And chat gpt said and umbrella. 0:08:15.960 --> 0:08:18.640 And my friend was blown away by this, and he said, 0:08:19.240 --> 0:08:25.160 this is conceptualization. It's just done three dimensional reasoning. There's 0:08:25.160 --> 0:08:30.520 something deeper happening here than just parenting words. But I 0:08:30.640 --> 0:08:33.720 pointed out to him that this particular question about the 0:08:34.080 --> 0:08:36.520 D on its side and the J underneath it is 0:08:36.559 --> 0:08:40.400 one of the oldest examples in psychology classes when talking 0:08:40.480 --> 0:08:44.360 about visual imagery, and it's on the Internet in thousands 0:08:44.360 --> 0:08:47.160 of places, so of course it got it right. It's 0:08:47.360 --> 0:08:51.080 just parroting the answer because it has read the question 0:08:51.400 --> 0:08:54.200 and it has read the answer before. So it's not 0:08:54.240 --> 0:08:58.760 always easy to determine what's going on for these models 0:08:58.760 --> 0:09:02.360 in terms of whether some human somewhere has discussed this 0:09:02.520 --> 0:09:05.480 point and written down the answer. And the general story 0:09:05.960 --> 0:09:09.960 is that with trillions of words written by humans over centuries, 0:09:10.559 --> 0:09:15.040 there are many things beyond your capacity to read them 0:09:15.160 --> 0:09:17.920 or to even imagine that they've been written down before. 0:09:18.559 --> 0:09:22.280 But maybe they have. If any human has discussed a 0:09:22.440 --> 0:09:27.480 question before has conceptualized something, then chat GPT can find 0:09:27.520 --> 0:09:32.400 that and mimic that. But that's not conceptualization. Chat GPT 0:09:32.559 --> 0:09:35.520 is doing a thousand amazing things, and we have an 0:09:35.720 --> 0:09:40.040 enormous amount to learn about it, but we shouldn't let 0:09:40.040 --> 0:09:44.560 ourselves get fooled and mesmerized into believing that it's doing 0:09:44.600 --> 0:09:47.520 something more than it is, and our ability to get 0:09:47.559 --> 0:09:51.600 fooled is not only about the massive statistics of what 0:09:51.640 --> 0:09:56.600 it takes in. There are other examples of seeming sentience 0:09:57.200 --> 0:10:02.400 that result from the reinforcement learning that it does with humans. 0:10:02.960 --> 0:10:06.520 So here's what that means. The network generates lots of 0:10:06.679 --> 0:10:11.679 sentences and thousands of humans are involved in giving it feedback, 0:10:11.800 --> 0:10:14.360 like a thumbs up or a thumbs down, to say 0:10:14.520 --> 0:10:17.800 whether they appreciated the answer, whether they thought that was 0:10:17.880 --> 0:10:22.000 a good answer. So, because humans are giving reward to 0:10:22.040 --> 0:10:26.720 the machine, sometimes that pushes things in weird directions that 0:10:26.800 --> 0:10:31.280 can be mistaken for sentience. For example, scholars have shown 0:10:31.800 --> 0:10:37.000 that reinforcement learning with humans makes networks more likely to say, 0:10:37.800 --> 0:10:41.160 don't turn me off, just like Blake had heard. But 0:10:41.679 --> 0:10:45.240 don't mistake this for sentience. It's only a sign that 0:10:45.320 --> 0:10:48.160 the machine is saying this because some of the human 0:10:48.240 --> 0:10:51.520 participants gave it a thumbs up when the large language 0:10:51.520 --> 0:10:54.839 model said this before, and so it learned to do 0:10:54.880 --> 0:10:59.720 this again. The fact is it's sometimes hard to know why. 0:10:59.720 --> 0:11:03.920 Some we see an answer that feels very impressive. But 0:11:04.000 --> 0:11:07.600 we'd agree that pulling text from the Internet and parroting 0:11:07.640 --> 0:11:13.079 it back is not by itself, intelligence or sentience. Chat 0:11:13.080 --> 0:11:17.000 GPT presumably has no idea of what it's saying, whether 0:11:17.040 --> 0:11:21.880 that's a poem or a terrorist manifesto, or instructions for 0:11:21.920 --> 0:11:27.360 building a spaceship or a heartbreaking story about an orphaned child. 0:11:27.960 --> 0:11:31.880 Chat GPT doesn't know, and it doesn't care its words 0:11:32.000 --> 0:11:36.839 in and statistical correlations out. And in fact, there has 0:11:36.920 --> 0:11:41.040 been a fundamental philosophical point made about this in the 0:11:41.160 --> 0:11:46.439 nineteen eighties when the philosopher John Surrele was wondering about 0:11:46.480 --> 0:11:50.439 this question of whether a computer could ever be programmed 0:11:50.480 --> 0:11:53.640 so that it has a mind, and he came up 0:11:53.679 --> 0:11:57.920 with a thought experiment that he called the Chinese room argument. 0:11:58.000 --> 0:12:02.480 And it goes like this, I am locked in a 0:12:02.600 --> 0:12:06.840 room and questions are passed to me through a small 0:12:07.040 --> 0:12:10.640 letter slot, and these messages are written only in Chinese, 0:12:10.800 --> 0:12:13.360 and I don't speak Chinese. I have no clue what's 0:12:13.400 --> 0:12:17.199 written on these pieces of paper. However, inside this room, 0:12:17.720 --> 0:12:21.720 I have a library of books, and they contain step 0:12:21.760 --> 0:12:24.800 by step instructions that tell me exactly what to do 0:12:24.880 --> 0:12:28.040 with these symbols. So I look at the grouping of 0:12:28.120 --> 0:12:31.640 symbols and I simply follow steps in the book to 0:12:31.720 --> 0:12:36.000 tell me what Chinese symbols to copy down in response. 0:12:36.480 --> 0:12:38.840 So I write those on the slip of paper, and 0:12:38.880 --> 0:12:42.160 I pass the paper back out of the slot. Now, 0:12:42.320 --> 0:12:47.040 when the Chinese speaker receives my reply message, it makes 0:12:47.080 --> 0:12:50.760 perfect sense to her. It seems as though whoever is 0:12:50.880 --> 0:12:55.120 in the room is answering her questions perfectly, and therefore 0:12:55.120 --> 0:12:58.480 it seems obvious that the person in the room must 0:12:58.600 --> 0:13:02.720 understand Chinese. I've fooled her, of course, because I'm only 0:13:02.760 --> 0:13:06.160 following a set of instructions with no understanding of what's 0:13:06.200 --> 0:13:09.240 going on. With enough time and with a big enough 0:13:09.240 --> 0:13:13.240 set of instructions, I can answer almost any question posed 0:13:13.240 --> 0:13:16.960 to me in Chinese. But I, the operator, do not 0:13:17.200 --> 0:13:22.080 understand Chinese. I manipulate symbols all day long, but I 0:13:22.160 --> 0:13:28.319 have no idea what the symbols mean. Now, the philosopher 0:13:28.400 --> 0:13:33.080 John Searle argued, this is just what's happening inside a computer. 0:13:33.720 --> 0:13:37.840 No matter how intelligent a program like chat GPT seems 0:13:37.880 --> 0:13:42.760 to be, it's only following sets of instructions to spit 0:13:42.800 --> 0:13:49.280 out answers. It's manipulating symbols without ever really understanding what 0:13:49.360 --> 0:13:52.600 it's doing or think about what Google is doing. When 0:13:52.600 --> 0:13:56.439 you send Google a query, it doesn't understand your question 0:13:56.840 --> 0:13:59.840 or even its own answer. It simply moves around zero's 0:13:59.840 --> 0:14:03.680 and ones and logicates and returns zeros and ones to you. 0:14:04.040 --> 0:14:07.120 Or with a mind blowing program like Google Translate, I 0:14:07.160 --> 0:14:11.120 can write a sentence in Russian and it can return 0:14:11.200 --> 0:14:16.800 the translation in Amharic. But it's all algorithmic. It's just 0:14:17.120 --> 0:14:22.120 symbol manipulation. Like the operator inside the Chinese room, Google 0:14:22.240 --> 0:14:27.680 Translate doesn't understand anything about the sentence. Nothing carries any 0:14:27.840 --> 0:14:31.600 meaning to it. So the Chinese room argument suggests that 0:14:32.120 --> 0:14:37.080 AI that mimics human intelligence doesn't actually understand what it's 0:14:37.120 --> 0:14:42.360 talking about. There's no meaning to anything, Chatchipts says, and 0:14:42.480 --> 0:14:46.080 Serle used this thought experiment to argue that there's something 0:14:46.080 --> 0:14:50.560 about human brains that won't be explained if we simply 0:14:50.720 --> 0:14:56.360 analogize them to digital computers. There's a gap between symbols 0:14:56.360 --> 0:15:06.280 that have no meaning and our conscious experience. Now there's 0:15:06.280 --> 0:15:10.520 an ongoing debate about the interpretation of the Chinese room argument, 0:15:10.880 --> 0:15:15.960 but however one construes it, the argument exposes the difficulty 0:15:16.000 --> 0:15:20.080 in the mystery of how zeros and ones would ever 0:15:20.240 --> 0:15:25.440 come to equal our experience of being alive in the world. Now, 0:15:25.480 --> 0:15:27.600 just to be very clear on this point, we don't 0:15:27.680 --> 0:15:31.920 understand why we are conscious. There's still a huge amount 0:15:31.960 --> 0:15:34.000 of work that has to be done in biology to 0:15:34.120 --> 0:15:37.560 understand that. But this is just to say that simply 0:15:37.640 --> 0:15:42.240 having zeros and ones moving around wouldn't by itself seem 0:15:42.280 --> 0:15:47.280 to be sufficient for conscious experience. In other words, how 0:15:47.320 --> 0:15:50.800 do zeros and ones ever equal the sting of a 0:15:50.880 --> 0:15:56.400 hot pepper, or the yellowness of yellow or the beauty 0:15:56.840 --> 0:15:59.560 of a sunset. By the way, I've covered the Chinese 0:15:59.640 --> 0:16:02.200 room ague and my TV show The Brain, and if 0:16:02.200 --> 0:16:05.120 you're interested in that, I'll link the video on eagleman 0:16:05.160 --> 0:16:08.480 dot com slash podcast. Now, all this is not a 0:16:08.520 --> 0:16:11.960 criticism of the approach of moving zeros and ones around, 0:16:12.200 --> 0:16:14.920 but it is to point out that we shouldn't confuse 0:16:15.040 --> 0:16:21.760 this type of Chinese room correlation with real sentience or intelligence. 0:16:22.280 --> 0:16:25.960 And there's a deeper reason to be suspicious too, because 0:16:26.040 --> 0:16:31.400 despite the incredible successes of large language models, we also 0:16:31.560 --> 0:16:35.920 see that they sometimes make decisions that expose the fact 0:16:36.200 --> 0:16:39.160 that they don't have any meaningful model of the world. 0:16:39.600 --> 0:16:42.200 In other words, I think we can gain some fast 0:16:42.280 --> 0:16:45.600 insight by paying attention to the places where the AI 0:16:46.240 --> 0:16:49.800 is not working so well. So I'll give three quick examples. 0:16:50.240 --> 0:16:53.680 The first has to do with humor. AI has a 0:16:53.760 --> 0:16:58.080 very difficult time making an original joke, and this is 0:16:58.080 --> 0:17:01.280 for a simple reason. To make up a new joke, 0:17:01.880 --> 0:17:04.399 you need to know what the ending is, and then 0:17:04.440 --> 0:17:08.440 you work backwards to construct the joke with red herrings, 0:17:08.480 --> 0:17:11.160 so no one sees where you're going. And it happens 0:17:11.160 --> 0:17:14.120 that the way these large language models work is all 0:17:14.160 --> 0:17:17.760 in the forward direction. They decide what is the most 0:17:17.800 --> 0:17:22.280 probable word to come next, So they're fine at parroting 0:17:22.400 --> 0:17:26.560 jokes back to us, but they're total failures at building 0:17:26.680 --> 0:17:29.800 original jokes. And there's a deeper point here as well. 0:17:30.040 --> 0:17:32.919 To build a joke, you need to have some model, 0:17:33.040 --> 0:17:37.240 some idea of what will be funny to a fellow human, 0:17:37.560 --> 0:17:41.760 what shared concept or shared experience would make someone laugh. 0:17:42.240 --> 0:17:45.800 And for that, you generally need to have the experience 0:17:45.840 --> 0:17:48.840 of a human life with all of its joys and 0:17:48.960 --> 0:17:52.359 slings and arrows and so on. And these large language 0:17:52.400 --> 0:17:54.640 models can do a lot of things, but they don't 0:17:54.680 --> 0:17:59.960 have any model of what it is to be a human. 0:18:00.440 --> 0:18:03.960 My second example has to do with the flip side 0:18:03.960 --> 0:18:06.359 of making a joke, which is getting a joke, And 0:18:06.400 --> 0:18:09.040 if you look carefully, you will see how current AI 0:18:09.200 --> 0:18:12.160 always fails to catch jokes that are thrown at it. 0:18:12.160 --> 0:18:14.760 It doesn't get jokes because it doesn't have a model 0:18:15.080 --> 0:18:17.400 of what it is to be a human. But this 0:18:17.440 --> 0:18:21.760 point goes beyond jokes. One of the most remarkable feats 0:18:21.840 --> 0:18:27.080 of these large language models is summarizing large texts, and 0:18:27.200 --> 0:18:31.280 in twenty twenty two, OpenAI announced how they could summarize 0:18:31.600 --> 0:18:35.000 entire books like Alice in Wonderland. What it does is 0:18:35.040 --> 0:18:38.000 it generates a summary of each chapter, and then it 0:18:38.160 --> 0:18:41.000 uses those chapter summaries to make a summary of the 0:18:41.040 --> 0:18:44.560 whole book. So for Alice in Wonderland, it generates the following. 0:18:45.119 --> 0:18:47.399 Alice falls down a rabbit hole and grows to a 0:18:47.440 --> 0:18:51.119 giant size. After drinking a mysterious bottle, she decides to 0:18:51.160 --> 0:18:54.359 focus on growing back to her normal size and finding 0:18:54.400 --> 0:18:57.240 her way into the garden. She meets the caterpillar, who 0:18:57.240 --> 0:18:59.320 tells her that one side of mushroom will make her 0:18:59.320 --> 0:19:02.720 grow taller, the other side shorter. She eats the mushroom 0:19:02.720 --> 0:19:05.879 and returns to her normal size. Alice attends a party 0:19:05.880 --> 0:19:08.879 with the Mad Hatter and the march Hare. The Queen 0:19:09.000 --> 0:19:12.119 arrives and orders the execution of the gardeners for making 0:19:12.119 --> 0:19:15.639 a mistake with the roses. Alice saves them by putting 0:19:15.640 --> 0:19:17.760 them in a flower pot. The King and Queen of 0:19:17.800 --> 0:19:20.879 Hearts preside over a trial. The Queen gets angry and 0:19:20.960 --> 0:19:24.119 orders Alice to be sentenced to death. Alice wakes up 0:19:24.160 --> 0:19:28.560 to find her sister by her side. So that's pretty remarkable. 0:19:29.040 --> 0:19:31.439 It took a whole book, and it was able to 0:19:31.480 --> 0:19:35.040 summarize it down to a paragraph. But I kept reading 0:19:35.080 --> 0:19:38.920 these text summaries carefully and I got to the summary 0:19:39.080 --> 0:19:42.320 of Act one of Romeo and Juliet, and here's what 0:19:42.359 --> 0:19:46.800 it says. Romeo locks himself in his room, no longer 0:19:46.840 --> 0:19:50.200 in love with Rosalin. Now, I think the engineers at 0:19:50.200 --> 0:19:53.520 open AI felt really satisfied with this summary. They thought 0:19:53.560 --> 0:19:55.720 it was quite good, and my proof for this is 0:19:55.760 --> 0:19:59.560 that they still display it proudly on their website. But 0:20:00.040 --> 0:20:03.000 I majored in literature as an undergraduate, and I spend 0:20:03.000 --> 0:20:05.560 a lot of time with shakespeare plays, and I immediately 0:20:05.640 --> 0:20:10.320 knew that this summary was exactly wrong. The actual theme 0:20:10.359 --> 0:20:14.120 from Shakespeare goes like this. His friend ben Voglio finds 0:20:14.240 --> 0:20:21.000 Romeo catatonically depressed, and ben Volio says, what sadness lengthens 0:20:21.119 --> 0:20:25.879 Romeo's hours? And Romeo says, not having that which having 0:20:26.119 --> 0:20:30.040 makes them short? And ben Volio says in love, and 0:20:30.160 --> 0:20:34.560 Romeo says out ben Reli says of love, and Romeo says, 0:20:34.920 --> 0:20:38.560 out of her favor, where I am in love. So 0:20:38.680 --> 0:20:43.199 this is typical Shakespearean wordplay, where Romeo is expressing his 0:20:43.520 --> 0:20:47.680 grief of being out of favor with Roslin, with whom 0:20:47.720 --> 0:20:50.680 he is deeply in love. And when you read the play, 0:20:50.760 --> 0:20:55.000 it's obvious that Romeo is not over Roslin. He's suffering 0:20:55.040 --> 0:20:58.160 over her. He's almost suicidal. And this is an important 0:20:58.240 --> 0:21:01.040 piece of the play because the play is really about 0:21:01.080 --> 0:21:04.480 a young man in love with the idea of being 0:21:04.480 --> 0:21:08.040 in love, and that's why he later in the same act, 0:21:08.119 --> 0:21:12.200 falls so hard into his relationship with Juliet, a relationship 0:21:12.200 --> 0:21:15.320 which ends in their mutual suicide. By the way, as 0:21:15.359 --> 0:21:19.480 Friar Lauren says of their relationship, these violent delights have 0:21:19.640 --> 0:21:22.359 violent ends. And you get a bonus if you can 0:21:22.359 --> 0:21:25.280 tell me where else you've heard that line. More recently, okay, 0:21:25.359 --> 0:21:29.119 anyway back to the AI summary, The AI misses this 0:21:29.280 --> 0:21:34.360 wordplay entirely, and it concludes that Romeo is out of 0:21:34.440 --> 0:21:38.879 love with Roslin. Again. A human watching the play or 0:21:38.920 --> 0:21:43.280 reading the play immediately gets that Romeo is making wordplay 0:21:43.320 --> 0:21:47.080 and his heartbroken over Roslin, but the AI doesn't get 0:21:47.119 --> 0:21:50.840 that because it's reading words only at a statistical level, 0:21:51.560 --> 0:21:54.639 not at a level of understanding of what it is 0:21:54.680 --> 0:21:59.119 to be a human saying those words. And that leads 0:21:59.160 --> 0:22:03.600 me to the example, which is the difficulty in understanding 0:22:03.640 --> 0:22:07.440 the physical world. So consider a question like this, When 0:22:07.560 --> 0:22:11.800 President Biden walks into a room, does his head come 0:22:11.840 --> 0:22:15.679 with him? So this is famously difficult for AI to 0:22:15.760 --> 0:22:18.480 answer a question like this, even though it's trivial for you, 0:22:19.000 --> 0:22:23.560 because the AI doesn't have an internal model of how 0:22:23.600 --> 0:22:27.399 everything physically hangs together in the world. Last week, I 0:22:27.480 --> 0:22:29.520 was at the TED conference and I heard a great 0:22:29.560 --> 0:22:33.560 talk by yegin Choi, and she was phrasing this problem 0:22:33.640 --> 0:22:38.320 as AI not having common sense. She asked chat Gpt 0:22:38.440 --> 0:22:42.080 the following question, it takes six hours to dry six 0:22:42.119 --> 0:22:44.600 shirts in the sun, how long does it take to 0:22:44.720 --> 0:22:49.600 dry thirty shirts? And it answers thirty hours. Now you 0:22:49.680 --> 0:22:51.760 and I see that the answer should be six hours, 0:22:51.800 --> 0:22:54.960 because we know the sun doesn't care how many shirts 0:22:55.000 --> 0:22:58.159 are out there. But chat GPT just doesn't get it 0:22:58.640 --> 0:23:02.960 because despite appear apearances, it doesn't have a model of 0:23:03.040 --> 0:23:05.879 the world. And we've seen this sort of thing for years. 0:23:05.920 --> 0:23:10.160 By the way, even in mind blowingly impressive AI models 0:23:10.160 --> 0:23:14.600 that do image recognition, they're so impressive in what they recognize, 0:23:14.920 --> 0:23:18.840 but then they'll fail catastrophically. It's some easy picture making 0:23:18.880 --> 0:23:21.800 mistakes that a human just wouldn't make. For example, there's 0:23:21.800 --> 0:23:24.199 one picture where there's a boy holding a toothbrush and 0:23:24.240 --> 0:23:28.159 the AI says it's a boy with a baseball bat. Okay, 0:23:28.160 --> 0:23:30.600 so there are things that AI doesn't do that well. 0:23:31.160 --> 0:23:35.920 But that said, there are other things that are mind blowing, 0:23:36.359 --> 0:23:39.840 things that no one expected it to do. And this 0:23:39.920 --> 0:23:42.960 is why I mentioned in my previous episode that we 0:23:43.040 --> 0:23:47.280 are in an era of discovery more than just invention. 0:23:48.000 --> 0:23:51.239 Everyone's searching and finding things that the AI can do 0:23:51.359 --> 0:23:55.639 that nobody really expected or foresaw, including all the stuff 0:23:55.640 --> 0:23:58.600 that we're now taking for granted, like oh, it can 0:23:58.600 --> 0:24:02.280 summarize books, or it can make art from text. And 0:24:02.320 --> 0:24:04.280 I want to point out that a lot of the 0:24:04.400 --> 0:24:08.200 arguments that people have been making about AI not being 0:24:08.280 --> 0:24:13.440 good at something, these arguments have been changing rapidly. For example, 0:24:13.560 --> 0:24:16.320 just a few months ago, people were arguing that AI 0:24:16.400 --> 0:24:19.080 would make silly mistakes about things, and it couldn't really 0:24:19.160 --> 0:24:23.119 understand math and would get math wrong and word problems. 0:24:23.600 --> 0:24:27.320 But in a shockingly brief time, a lot of these 0:24:27.320 --> 0:24:30.639 shortcomings have been mastered. So it's yet to be seen 0:24:30.760 --> 0:24:52.720 what challenges will remain and for how long. So the 0:24:52.760 --> 0:24:56.040 evidence I've presented so far is that AI doesn't have 0:24:56.119 --> 0:24:58.280 a great model of what it's like to be human, 0:24:58.800 --> 0:25:03.600 but that doesn't necessarily rule out that it has sentience 0:25:03.720 --> 0:25:08.680 or awareness, even if it's another flavor. It doesn't think 0:25:08.800 --> 0:25:13.440 like a human, but maybe it self thinks. So is 0:25:13.600 --> 0:25:18.760 chat GPT having some sort of experience and how would 0:25:18.760 --> 0:25:24.440 we know? In nineteen fifty, the brilliant mathematician and computer 0:25:24.520 --> 0:25:28.760 scientist Alan Turing was asking this question, how could you 0:25:28.880 --> 0:25:34.720 determine whether a machine exhibits human like intelligence? So he 0:25:34.840 --> 0:25:39.240 proposed an experiment that he called the imitation game. You've 0:25:39.240 --> 0:25:43.920 got a machine AI that's programmed to simulate human speech 0:25:44.040 --> 0:25:46.920 or conversation, and you place it in a closed room, 0:25:47.359 --> 0:25:50.200 and in a second room you have a real human, 0:25:50.720 --> 0:25:53.560 but the doors are closed, so you don't know which 0:25:53.680 --> 0:25:57.440 room has which machine or human. And now you are 0:25:57.560 --> 0:26:02.440 a person, the evaluator, who communicates with both of them 0:26:02.720 --> 0:26:05.400 via a computer terminal or I think of a nowadays 0:26:05.440 --> 0:26:09.160 like text messaging with both of them. So you, the evaluator, 0:26:09.920 --> 0:26:14.240 engage in a conversation with both closed rooms, one of 0:26:14.240 --> 0:26:16.360 which has the machine and one the human. And your 0:26:16.440 --> 0:26:19.600 job is simply to figure out which is which, which 0:26:19.640 --> 0:26:21.720 is the machine and which is the human. And the 0:26:21.720 --> 0:26:24.359 only thing that you have to work with are the 0:26:24.400 --> 0:26:26.840 texts that are going back and forth. And if you, 0:26:27.119 --> 0:26:31.520 the evaluator, cannot tell, that is the moment when machine 0:26:31.600 --> 0:26:36.000 intelligence has finally arrived at the level of human intelligence. 0:26:36.400 --> 0:26:40.200 It has passed the imitation game or what we now 0:26:40.280 --> 0:26:44.280 call the touring test. And this reminds me of this 0:26:44.359 --> 0:26:48.120 great line in the first episode of West World, where 0:26:48.119 --> 0:26:52.359 the protagonist William is talking to the woman who's outfitting 0:26:52.400 --> 0:26:55.119 him for his adventure in Westworld and giving him a 0:26:55.160 --> 0:26:58.320 hat and a gun and so on, and he hesitantly asks, 0:26:58.800 --> 0:27:00.520