WEBVTT - Testing With Turing 0:00:00.160 --> 0:00:07.200 Brought to you by Toyota. Let's go places. Welcome to 0:00:07.400 --> 0:00:14.720 Forward Thinking. Hey the everyone, and welcome to Forward Thinking, 0:00:14.800 --> 0:00:17.040 the podcast that looks at the future and says Marvin, 0:00:17.360 --> 0:00:21.520 I love you. I'm Jonathan Strickland and I'm Joe McCormick. Joe, 0:00:21.520 --> 0:00:23.960 I got a question for you, Okay, ask me a question. 0:00:24.560 --> 0:00:27.520 What did you have for lunch today? I don't want 0:00:27.560 --> 0:00:32.360 to talk about that. That's personal, Okay, fine, all right, Well, 0:00:33.440 --> 0:00:38.400 what was the last movie you saw? Movies are weird 0:00:38.479 --> 0:00:44.280 these days, you know. I'm going to ask one more question, Joe. 0:00:44.440 --> 0:00:47.599 Who's your favorite person to work with in the office? 0:00:48.080 --> 0:00:52.720 Obviously you Joe? One one? I lied one last question? 0:00:52.760 --> 0:00:57.120 Are you? Are you human? Why would you ask? Of course, 0:00:58.320 --> 0:01:03.480 your responses have left the question somewhat vague. I'm not 0:01:03.600 --> 0:01:07.440 really sure. So it could be that you are, in 0:01:07.480 --> 0:01:10.360 fact some sort of simulation that is drawing upon a 0:01:10.400 --> 0:01:15.720 massive data bank of pre determined answers to questions, and 0:01:15.720 --> 0:01:17.679 and you're just drawing from that, and you're not actually 0:01:17.760 --> 0:01:21.120 genuinely coming up with responses. Why would you say that? 0:01:21.200 --> 0:01:24.560 Tell me more about your feelings? Are doctor space Zo 0:01:25.480 --> 0:01:31.200 or Eliza? We wanted to talk today? About artificial intelligence 0:01:31.240 --> 0:01:33.440 and the Touring test and whether or not the Touring 0:01:33.440 --> 0:01:37.119 test is in fact a good test for artificial intelligence 0:01:37.160 --> 0:01:40.120 and what exactly is it and and it does it 0:01:40.160 --> 0:01:42.880 doesn't really do what we think it does, because there 0:01:42.959 --> 0:01:45.319 is that huge news story a couple of weeks back, 0:01:45.400 --> 0:01:50.080 a couple of weeks back, being in June of where 0:01:50.120 --> 0:01:54.720 in uh, Eugene Goostman, who is a chat butt, not 0:01:54.840 --> 0:01:57.720 a real human, not a real human, um it is 0:01:57.760 --> 0:02:01.000 a chat butt, I suppose it's more technically correct, was 0:02:01.040 --> 0:02:03.800 announced to have passed the Turing test for the first 0:02:03.840 --> 0:02:07.280 time kind of sort of, well, not for the first 0:02:07.280 --> 0:02:10.080 time for this program. A lot of the media coverage 0:02:10.120 --> 0:02:14.560 suggested like, for the first time a computer program has 0:02:14.680 --> 0:02:17.880 quote passed the Turing test or quote beaten the Turing test, 0:02:18.680 --> 0:02:22.399 which there are multiple issues that we need to address 0:02:22.560 --> 0:02:25.440 to to actually talk about that, But first let's let's 0:02:25.560 --> 0:02:31.480 go on and explain what what Goostman did. Well, what 0:02:31.520 --> 0:02:35.000 it definitely did was full ten of thirty judges, which 0:02:35.040 --> 0:02:38.040 is thirty three for anyone following along doing the math 0:02:38.080 --> 0:02:42.040 at home that we're participating in a celebration of turning 0:02:42.240 --> 0:02:45.640 on the sixtieth anniversary of his death. It was a 0:02:45.680 --> 0:02:48.840 celebrity guest judge panel. It was pretty fancy. There was 0:02:48.880 --> 0:02:51.200 one of the actors from Red Dwarf on there and 0:02:51.240 --> 0:02:54.040 by fool amument that the computer program tricked them into 0:02:54.080 --> 0:02:57.760 thinking it was a human, right right. This program has 0:02:57.760 --> 0:03:00.000 been around since about two thousand one, when a team 0:03:00.040 --> 0:03:03.560 out of the Ukraine began designing it, and it poses 0:03:03.600 --> 0:03:06.480 as a thirteen year old boy from Odessa, Ukraine. It's 0:03:06.480 --> 0:03:08.760 taken part in a few different competitions like this over 0:03:08.800 --> 0:03:10.520 the years, and you can even find a version online 0:03:10.560 --> 0:03:13.079 if you want to play around with it. But the 0:03:13.080 --> 0:03:15.120 news that this news that really brought it into the 0:03:15.160 --> 0:03:19.240 public eye was a press release again yes, claiming that 0:03:19.280 --> 0:03:21.560 it was the first chatbot to pass the Turing test 0:03:21.600 --> 0:03:24.920 in an open ended event. And open ended is kind 0:03:24.919 --> 0:03:26.920 of one of those keywords here, because a lot of 0:03:26.919 --> 0:03:31.160 competitions in which chatbots are put up against the Turing 0:03:31.280 --> 0:03:35.720 Test involve a closed series of questions that is predetermined. 0:03:35.760 --> 0:03:38.480 It's a specific set of parameters. Either you are only 0:03:38.520 --> 0:03:42.640 allowed to ask certain questions, or the whole interview took 0:03:42.640 --> 0:03:45.760 place before the event and you are reviewing a transcript 0:03:46.160 --> 0:03:49.920 and then have to determine whether or not the participant 0:03:50.240 --> 0:03:54.720 was a human or a machine. So those are generally speaking, 0:03:54.720 --> 0:03:57.119 the different variations on this. We'll we'll get more into 0:03:57.160 --> 0:03:59.800 the actual touring test in a second, but that's sort 0:03:59.800 --> 0:04:03.000 of how they are performed these days, right, and so 0:04:03.040 --> 0:04:06.640 this achievement is pretty cool. But it did get way 0:04:06.680 --> 0:04:09.440 crazy over hyped in the media, I think largely because 0:04:09.480 --> 0:04:12.080 the press release, which was mostly a quotation from one 0:04:12.160 --> 0:04:16.880 Dr Kevin Warwick, who was an engineer slash futurist, slash provocateur, 0:04:17.600 --> 0:04:23.560 slash cyborg right form, he designed it to be kind 0:04:23.600 --> 0:04:26.400 of overhyped, which means that it did drum up public 0:04:26.440 --> 0:04:30.000 interest in touring and chat bots and artificial intelligence and 0:04:30.000 --> 0:04:34.240 technology at large, which is totally awesome. But the fact 0:04:34.279 --> 0:04:37.039 that the news media wasn't quite sure what the turing 0:04:37.120 --> 0:04:41.400 test was, or how this chat bought was interacting with it, 0:04:41.520 --> 0:04:43.520 or the fact that this was a chat bought and not, 0:04:43.600 --> 0:04:46.600 for example, a supercomputer, which is a ludicrous concept. Lots 0:04:46.600 --> 0:04:49.960 of people supercomputer defeat touring test. No, it was a 0:04:50.120 --> 0:04:52.440 it was software, Yes it was. It was a program, 0:04:52.560 --> 0:04:56.559 was independent, independent of hardware, It could run on lots 0:04:56.560 --> 0:04:59.320 of different types of hardware. Probably your smartphone, for example. 0:05:00.240 --> 0:05:03.480 So so we might need a new Turing test, right, 0:05:03.520 --> 0:05:06.520 So before we talk about new Turing tests, let's talk 0:05:06.520 --> 0:05:09.600 about the old one. Shall we maybe make a distinction 0:05:09.720 --> 0:05:15.360 between what idea actually came from the early computer scientists, 0:05:15.360 --> 0:05:18.719 Alan Touring, and how that has evolved into what we 0:05:18.800 --> 0:05:22.200 popularly think of as the Turing test today. Was just 0:05:22.240 --> 0:05:25.320 talking about. Yeah, yeah, so Touring test in general is 0:05:25.440 --> 0:05:28.880 kind of a misnomer because Touring He wrote a paper 0:05:28.880 --> 0:05:32.000 in nineteen fifty in which he was discussing the idea 0:05:32.040 --> 0:05:36.600 of machine intelligence and whether machines could actually possess intelligence, 0:05:37.240 --> 0:05:42.280 And his philosophy was that machines could demonstrate behaviors that 0:05:42.360 --> 0:05:46.839 we associate we humans associate with human intelligence, right, and 0:05:46.880 --> 0:05:52.040 that if a machine could produce such behavior reliably, then 0:05:52.160 --> 0:05:54.760 it stands to reason that we call that machine intelligent, 0:05:55.000 --> 0:06:00.240 because it, to all appearances is intelligent. Therefore we might 0:06:00.240 --> 0:06:02.719 as well go ahead and extended the courtesy. So that 0:06:02.800 --> 0:06:04.840 was generally what he was saying, And he said that 0:06:04.880 --> 0:06:08.279 he believed that by a fifty years time, which would 0:06:08.279 --> 0:06:12.920 be the year two thousand, a typical computer of sufficient 0:06:12.920 --> 0:06:16.719 power and sophistication would be able to fool people thirty 0:06:16.760 --> 0:06:19.200 pc of the time that it was in fact a 0:06:19.279 --> 0:06:21.880 human and not a machine, that that was the level 0:06:21.920 --> 0:06:26.200 of sophistication he expected. And this was more of a prediction, 0:06:26.320 --> 0:06:29.520 not a test. People have since looked at it as 0:06:29.560 --> 0:06:32.920 being a kind of test to apply to computers to 0:06:32.960 --> 0:06:37.200 see whether or not they pass this bar for machine intelligence, 0:06:37.520 --> 0:06:41.120 which is sort of a reverse engineering. Look at what 0:06:41.200 --> 0:06:44.120 he was talking about now. He used a specific example 0:06:44.680 --> 0:06:48.880 in his in his uh explanation of fooling someone thinking 0:06:48.880 --> 0:06:53.320 it was human, and that was a variation on something 0:06:53.320 --> 0:06:58.120 he called the imitation game. And the imitation game is 0:06:59.040 --> 0:07:01.160 a game. It's a part the game. Yeah. The original 0:07:01.279 --> 0:07:04.200 idea would be that you have three players. You've got 0:07:04.200 --> 0:07:06.880 an interrogator who could be a man or a woman, 0:07:07.279 --> 0:07:09.760 and then you have a male player and a female 0:07:09.800 --> 0:07:13.240 player who will be interrogated. All three people are separated 0:07:13.240 --> 0:07:15.880 from one another. They are not able to see and 0:07:16.720 --> 0:07:19.920 preferably not be able to hear one another, right, communicating 0:07:19.960 --> 0:07:25.600 through right and preferably communicating through teletype or some other 0:07:25.720 --> 0:07:29.600 form of non handwriting communication, just in case handwriting would 0:07:29.600 --> 0:07:32.440 give away some form of gender identity. Right, because it's 0:07:32.440 --> 0:07:35.240 the interviewer's job to figure out which of the participants 0:07:35.320 --> 0:07:37.160 is the man and which is the woman, right, and 0:07:37.200 --> 0:07:41.200 it's the man player's job to try and trick the interrogator. 0:07:41.240 --> 0:07:43.840 It's the female player's job to try and get the 0:07:43.880 --> 0:07:47.760 interrogator to get the answer correct. So the man will 0:07:48.320 --> 0:07:52.040 attempt to throw doubt into the situation, and the interrogator 0:07:52.080 --> 0:07:54.960 has to figure out without any other like visual or 0:07:55.000 --> 0:07:59.440 audible cues, who is who, And Touring said, well, let's 0:07:59.560 --> 0:08:03.760 replay this with saying it's a human interrogator, and then 0:08:03.800 --> 0:08:08.080 you have a human respond it and a machine responded 0:08:08.280 --> 0:08:10.920 as the interrogator's job to figure out whether or not 0:08:11.040 --> 0:08:14.240 the entity he or she is talking to is in 0:08:14.240 --> 0:08:18.480 fact human or machine, and suggested that by the year 0:08:18.520 --> 0:08:21.800 two thousand, computers would be at a level sophisticate enough 0:08:21.840 --> 0:08:23.600 where thirty percent of the time you would not be 0:08:23.680 --> 0:08:26.040 able to tell you would you know, or thirty if 0:08:26.080 --> 0:08:29.680 you prefer, of the people who would interview said machine 0:08:29.840 --> 0:08:31.880 would be unable to tell that it was a machine. 0:08:32.240 --> 0:08:35.559 So Turing didn't really lay out a path to how 0:08:35.600 --> 0:08:38.280 this would be achieved, did he? More or less, he 0:08:38.280 --> 0:08:41.920 he talked about the principles of computer science and that 0:08:41.960 --> 0:08:45.040 they would become more and more sophisticated to the point 0:08:45.040 --> 0:08:47.680 where this would become possible, but did not, in fact 0:08:48.200 --> 0:08:52.600 layout like, here's my fifty year plan for making computers intelligence. 0:08:52.720 --> 0:08:56.160 I guess he probably he wasn't talking about natural language 0:08:56.160 --> 0:08:59.719 processing algorithms and such stuff. No, no, no, it was 0:09:00.000 --> 0:09:02.480 our general approach was talking about the idea of a 0:09:02.559 --> 0:09:06.000 machine possessing the general intelligence that would allow it to 0:09:06.040 --> 0:09:07.800 do something right. This is this was more of a 0:09:07.840 --> 0:09:12.800 philosophical perspective than anything else. And uh. On top of that, 0:09:12.800 --> 0:09:17.160 there were a lot of people who disagreed with Tourings philosophy. 0:09:17.240 --> 0:09:20.360 They said, you know, you're saying that because you're able 0:09:20.400 --> 0:09:23.480 to mimic intelligence that that means you have some form 0:09:23.559 --> 0:09:26.920 of possession of intelligence. You've asserted this without providing any 0:09:27.000 --> 0:09:32.840 actual proof. One of the criticisms I read specifically stated 0:09:32.880 --> 0:09:35.000 that one of the things I had said earlier was 0:09:35.040 --> 0:09:38.640 how we say, if a machine appears to be intelligent, 0:09:38.720 --> 0:09:41.560 we might as well say it is intelligent. And uh. 0:09:41.559 --> 0:09:45.560 And one interpretation of that is that touring would say, well, 0:09:46.080 --> 0:09:48.600 let's say two people are talking to one another. They 0:09:48.600 --> 0:09:53.480 each assumed the other person possesses intelligence, partly because of 0:09:53.520 --> 0:09:56.600 the conversation that's going on. Therefore, if a machine were 0:09:56.600 --> 0:09:59.440 able to hold a conversation with you, wouldn't it be 0:09:59.600 --> 0:10:01.719 just party is to go ahead? And a shouldn't it 0:10:01.760 --> 0:10:05.880 be entitled to write? Whereas the critic I read said, well, 0:10:06.280 --> 0:10:10.400 to be fair, humans pretty much assume that other humans 0:10:10.480 --> 0:10:13.400 possess intelligence. It has nothing to do with the conversation. 0:10:13.400 --> 0:10:15.600 It has everything to do with the human being a human, 0:10:15.800 --> 0:10:18.559 Which doesn't necessarily mean it's based upon any kind of 0:10:18.600 --> 0:10:23.800 science or any kind of rational approach, but that is 0:10:23.800 --> 0:10:26.240 how humans are. I mean humans are also a little 0:10:26.280 --> 0:10:29.520 bit on the irrational side. I mean we we kind 0:10:29.520 --> 0:10:33.640 of also extend that courtesy to like our pets, which 0:10:33.800 --> 0:10:36.040 you know, yeah, we we at least we at least 0:10:36.160 --> 0:10:39.840 in in ways that we write about and and verbalize, 0:10:40.280 --> 0:10:43.960 will anthropomorphize lots of stuff, sure, like like our toaster. 0:10:44.240 --> 0:10:47.640 Given enough capacity, we go like, that's a good smart toaster, 0:10:47.840 --> 0:10:50.439 or we'll say some things like this toaster hates me. 0:10:50.920 --> 0:10:54.160 It burns it either either just warms the bread or 0:10:54.160 --> 0:10:56.000 it burns it to a crisp. It just hates me. 0:10:56.320 --> 0:10:59.920 And that's when we're projecting things upon these objects that 0:11:00.080 --> 0:11:04.240 they don't inherently possess. But part of this philosophical argument 0:11:04.440 --> 0:11:07.400 is that we're really only assuming that other humans have 0:11:07.480 --> 0:11:10.000 this intelligence, right, I mean it's tied into sort of 0:11:10.040 --> 0:11:14.400 the philosophical problem of solipsism. It's a thing nobody wants 0:11:14.440 --> 0:11:17.080 to assume that they are the only mind in the universe, 0:11:17.160 --> 0:11:19.400 or some people might want to assume that, but hopefully 0:11:19.480 --> 0:11:24.199 nobody wants to don't want to assume it, but proved 0:11:24.280 --> 0:11:26.280 me wrong, is what I'm saying. It's one of those 0:11:26.280 --> 0:11:29.520 where you just operate day to day under the assumption 0:11:29.559 --> 0:11:31.400 that that is not the case, but you have no 0:11:31.440 --> 0:11:35.480 way to prove it's not the case. But philosophy aside, 0:11:35.840 --> 0:11:38.119 the turn test is being used as a very practical 0:11:38.280 --> 0:11:41.640 means of evaluating AI these days, right, at least as 0:11:41.640 --> 0:11:44.920 far as being able to mimic a humans ability to 0:11:45.040 --> 0:11:47.920 hold a conversation. Right. So, Yeah, it's sort of moved 0:11:48.000 --> 0:11:51.040 beyond the bounds of this philosophical discussion in which it 0:11:51.120 --> 0:11:54.800 was originally proposed, and now it has become a thing 0:11:55.040 --> 0:11:58.880 people try to do. Yeah, it's become something that has 0:11:58.960 --> 0:12:02.400 been uh a goal in of itself, which again has 0:12:02.440 --> 0:12:05.760 become part of a growing conversation around AI and whether 0:12:05.840 --> 0:12:09.960 or not that's a legitimate goal sure, which is partially 0:12:10.040 --> 0:12:13.400 cool because it's encouraging competition and a good kind of 0:12:13.400 --> 0:12:16.040 sense of go get them fun amongst a lot of 0:12:16.040 --> 0:12:20.800 of programmers and computer scientists and thinkers, whereas other people say, yeah, 0:12:20.880 --> 0:12:24.319 but it's kind of diverting attention away from real problems 0:12:24.320 --> 0:12:28.240 and machine intelligence that need to have more work put 0:12:28.240 --> 0:12:31.600 into them. But everyone's worried about making chat bought two 0:12:31.640 --> 0:12:35.800 thousand full of people that it's actually you know, Bob 0:12:35.880 --> 0:12:38.640 from Minnesota. Yeah. Whether or not those people were actors 0:12:38.640 --> 0:12:42.720 on Red Dwarf, yeah, it doesn't matter. Okay. So in 0:12:42.760 --> 0:12:46.240 the in the sort of popular thinking today, the popular 0:12:46.320 --> 0:12:49.959 tech thinking, what is the Turing test? Now? What? What 0:12:50.280 --> 0:12:52.679 does what does it mean when someone says I've beaten 0:12:52.720 --> 0:12:55.720 the Turing tests? The Turing test today is based again 0:12:55.760 --> 0:12:59.760 on that prediction that by two thousand a computer would 0:12:59.760 --> 0:13:02.360 be able a full thirty percent of the time, or 0:13:02.760 --> 0:13:04.319 in thirty percent of the cases, the computer would be 0:13:04.320 --> 0:13:06.560 able to fool someone that it itself was a human being. 0:13:07.200 --> 0:13:09.800 So now we've taken that on and turned it on 0:13:09.840 --> 0:13:13.480 its head and said, the Touring test is a test 0:13:13.520 --> 0:13:17.160 where if a chatbot is able to convince a human 0:13:17.200 --> 0:13:20.680 interviewer that it in fact is a human or more 0:13:20.920 --> 0:13:23.800 of the time it passes the touring test and it is, 0:13:24.559 --> 0:13:28.360 I guess, showing some form of intelligence. I mean, that's 0:13:28.360 --> 0:13:30.439 the that's the real problem where it all breaks down 0:13:30.559 --> 0:13:34.160 right as the definition of intelligence. And part of it 0:13:34.200 --> 0:13:36.680 was just let's see if we can create a program 0:13:36.720 --> 0:13:40.080 sophisticated enough so that it can respond to questions in 0:13:40.120 --> 0:13:44.079 a way that is natural and will fool folks into thinking, oh, 0:13:44.120 --> 0:13:46.560 it's another human being on the other end of this terminal. 0:13:46.720 --> 0:13:48.600 And as we've talked about before on this show, the 0:13:48.679 --> 0:13:52.280 problem of computers understanding natural language is a huge one. 0:13:52.360 --> 0:13:55.319 So it's not a terrible benchmark as benchmarks go. No, 0:13:55.800 --> 0:13:59.160 it can in fact improve lots of different areas and 0:13:59.280 --> 0:14:01.719 artificial and elligience. It's just a question of how do 0:14:01.760 --> 0:14:03.880 you implement it. If you're implementing it in a very 0:14:03.920 --> 0:14:07.640 superficial way, you might be able to get some some 0:14:07.800 --> 0:14:12.000 fairly interesting results from an actual quote unquote touring test 0:14:12.040 --> 0:14:14.839 where you submit this to judges and the judges believe 0:14:14.920 --> 0:14:16.760 in fact that it is a human. But you might 0:14:16.800 --> 0:14:20.360 be doing so using essentially smoke and mirrors away where 0:14:20.600 --> 0:14:24.440 the computer does not have any real natural language processing 0:14:24.480 --> 0:14:27.160 ability to it. It just responds in a way that 0:14:27.280 --> 0:14:30.080 convinces the judges that it, in fact is a human. 0:14:30.440 --> 0:14:33.600 It's got some clever tricks, right. And in fact, most 0:14:33.680 --> 0:14:36.440 of the chatbots that have passed the Turing test, and 0:14:37.360 --> 0:14:39.320 this has gone back to the nineteen seventies, is it's 0:14:39.360 --> 0:14:42.360 not a new thing that has just happened. Whoa you 0:14:42.400 --> 0:14:45.440 said past the Turing test? I mean, I thought based 0:14:45.480 --> 0:14:48.600 on the media reporting that the thirteen year old Ukrainian boy, 0:14:48.680 --> 0:14:51.800 the fake thirteen year old Ukrainian boy, Eugene Goostman was 0:14:51.840 --> 0:14:55.040 the first to pass. No, we we previously to that 0:14:55.120 --> 0:14:59.960 had a fake paranoid schizophrenic yeah, Perry r r y 0:15:00.240 --> 0:15:05.160 and a fake psychologist Eliza Eliza. Yeah, so Eliza was 0:15:05.440 --> 0:15:07.840 a program. In fact, Eliza is probably the first really 0:15:07.880 --> 0:15:11.600 well known chatbot that kind of I didn't necessary to 0:15:12.240 --> 0:15:15.760 probably heard me doing a little bit of Aliza earlier. 0:15:15.760 --> 0:15:20.360 It's a Rogerian psychiatrist personality. Actually, Eliza had several different 0:15:20.400 --> 0:15:23.520 personalities she could assume, but the doctor one is the 0:15:23.560 --> 0:15:27.440 most popular of the most well known of all the variations. 0:15:27.840 --> 0:15:31.720 And then you have Perry who mimic the behavior of 0:15:31.760 --> 0:15:36.120 a paranoid schizophrenic, including a backstory. The the programmer for 0:15:36.160 --> 0:15:41.480 Perry created basic facts about Perry that Perry would share, 0:15:42.080 --> 0:15:46.360 sometimes spontaneously or or if you just got quote unquote 0:15:46.360 --> 0:15:50.520 frustrated they were. The way this would work is that 0:15:50.560 --> 0:15:53.320 the program would analyze what was being said just by 0:15:53.320 --> 0:15:55.920 looking for certain keywords and then trying to create us. 0:15:55.920 --> 0:16:00.760 And it's based on those keywords. Not really a sophisticate analysis. 0:16:00.840 --> 0:16:04.200 It was just very basic, and Perry would occasionally throw 0:16:04.240 --> 0:16:08.120 in complete non sequadors, but in a paranoid schizophrenic kind 0:16:08.120 --> 0:16:10.880 of way, which was convincing enough to at least full 0:16:11.040 --> 0:16:15.080 some psychiatrists who were reading transcripts of interviews with Perry 0:16:15.120 --> 0:16:17.880 and saying, no, this actually sounds like it could be 0:16:17.920 --> 0:16:21.040 a human being. Okay, so it sounds like in all 0:16:21.160 --> 0:16:24.920 these cases we're dealing with a chat about that is 0:16:24.960 --> 0:16:29.760 this assuming the personality of sort of a specialized interlocutor, like, uh, 0:16:30.120 --> 0:16:33.320 you have a doctor who has a very particular way 0:16:33.360 --> 0:16:35.800 of talking, meaning not really responding to what you're saying 0:16:35.840 --> 0:16:37.680 and just saying how does that make you feel? Right? 0:16:37.920 --> 0:16:41.280 Or you have a paranoid schizophrenic who's sort of range 0:16:41.320 --> 0:16:45.160 of response is naturally very limited, especially a skewed a 0:16:45.240 --> 0:16:49.000 skewed view of reality, or or you have Eugene Goostman, 0:16:49.080 --> 0:16:52.040 even the more recent one, who has probably a limited 0:16:52.120 --> 0:16:55.800 command of English and a limited understanding of the popular 0:16:55.880 --> 0:16:59.600 culture outside of his own community. Just all the world 0:16:59.640 --> 0:17:01.800 works in general. If you're talking about a thirteen year old, 0:17:01.840 --> 0:17:04.680 you're not going to have the expectation of maturity and 0:17:04.720 --> 0:17:08.879 sophistication at the world. So so that's a big criticism 0:17:09.920 --> 0:17:13.520 year olds listening who are probably fabulous humans. You're probably you, 0:17:13.520 --> 0:17:17.680 You're You're the exception, okay, but all those other year olds, 0:17:17.840 --> 0:17:21.280 you're the future. And you know what Cheers was about. 0:17:22.280 --> 0:17:27.480 So we'll get into that. So the the the point 0:17:27.560 --> 0:17:31.320 here is that we have not seen really an example 0:17:31.359 --> 0:17:34.320 of a chat bot that is able to assume the 0:17:34.440 --> 0:17:38.160 role of a typical however you wanted to find that, 0:17:38.359 --> 0:17:41.920 you know, average human being who communicates and whatever the 0:17:42.000 --> 0:17:45.760 language of choice happens to be, usually as English. But 0:17:45.840 --> 0:17:49.600 let's say that we had a the ideal goal here 0:17:49.640 --> 0:17:54.520 would be have having a chat bot that communicates fluently 0:17:54.800 --> 0:17:58.800 in whatever language was picked for the competition and can 0:17:58.840 --> 0:18:04.359 answer questions that any person the age of said chatbot 0:18:04.440 --> 0:18:07.600 would be able to answer and uh and be able 0:18:07.640 --> 0:18:09.760 to to roll with any of those kind of of 0:18:10.000 --> 0:18:16.640 um conversational venues. Right. Basically, we're looking for Lieutenant Data. Yeah, yeah, 0:18:17.080 --> 0:18:20.640 Mr Data, Okay, I want to move us along here, 0:18:20.760 --> 0:18:25.120 please to ask the question. So, given that h this 0:18:25.240 --> 0:18:29.560 test is not actually necessarily a test, and that it 0:18:29.600 --> 0:18:33.040 has it has, depending on your interpretation, been bested in 0:18:33.119 --> 0:18:36.480 kind of limited ways throughout the years, why is it 0:18:36.560 --> 0:18:40.959 still popularly considered the bar for artificial intelligence? Well? I 0:18:41.000 --> 0:18:45.399 think mainly it's because, first of all, it mimics a 0:18:45.480 --> 0:18:48.199 part of intelligence that we all recognize, right, So that 0:18:48.280 --> 0:18:51.200 makes it easy for it's especially easy to sell it 0:18:51.320 --> 0:18:53.439 to a general public who may or may not be 0:18:53.560 --> 0:18:57.520 very savvy as far as what machine intelligence entails. So 0:18:57.840 --> 0:19:00.359 it's an easy story. So that has If you can 0:19:00.440 --> 0:19:02.760 have a conversation with the thing, it's clearly smart. Yeah, 0:19:02.840 --> 0:19:06.400 So that has that going for it. Uh. Secondly, this 0:19:06.480 --> 0:19:10.000 is a non trivial problem. I mean, creating some form 0:19:10.160 --> 0:19:14.119 of computer or machine intelligence that can respond to human 0:19:14.200 --> 0:19:17.679 language in a way that is natural, whether it is 0:19:17.720 --> 0:19:19.800 doing so in a way that makes you think it's 0:19:19.880 --> 0:19:23.280 human or not. That's a non trivial problem. I mean 0:19:23.280 --> 0:19:26.320 you're talking about humans have the ability to put the 0:19:26.400 --> 0:19:32.200 same kind of information into vastly different wording so that 0:19:32.480 --> 0:19:34.840 they're conveying the same meaning, but they could do it 0:19:34.880 --> 0:19:38.000 in multiple ways, and you have to be able to 0:19:38.040 --> 0:19:41.720 design a program and a machine that can roll with that, 0:19:41.720 --> 0:19:45.400 that can understand or at least respond in all these 0:19:45.400 --> 0:19:49.480 different variations with the appropriate information. So whether or not 0:19:49.600 --> 0:19:51.919 you're talking about a chatbot or you're talking about some 0:19:51.960 --> 0:19:55.080 form of interface. Let's say it's a call center where 0:19:55.119 --> 0:19:57.560 you're trying to get help for a problem, there could 0:19:57.600 --> 0:20:00.199 be a hundred different ways for you to phrase what 0:20:00.400 --> 0:20:04.080 has gone wrong with whatever the issue is, and the 0:20:04.119 --> 0:20:06.639 machine intelligence has to be able to interpret all one 0:20:06.720 --> 0:20:09.280 hundred of those ways and come back with the appropriate 0:20:09.320 --> 0:20:12.440 response that is the same for all one hundred different 0:20:12.480 --> 0:20:16.359 ways that you you could phrase it. These are really 0:20:16.400 --> 0:20:19.800 tough challenges. We've seen it uh demonstrated in lots of 0:20:19.800 --> 0:20:22.560 different types of technology, the IBM S Watson being a 0:20:22.560 --> 0:20:25.280 big one that we can point to, where it took 0:20:25.320 --> 0:20:29.440 the lots of different subtle things about the English language 0:20:29.840 --> 0:20:33.120 and was able to interpret them and respond in the 0:20:33.200 --> 0:20:36.800 format of a game show and do it successfully. And 0:20:37.160 --> 0:20:41.080 that was pretty tricky because it involved not just analyzing 0:20:41.080 --> 0:20:43.640 all right, well, this, this word here in this sentence 0:20:43.840 --> 0:20:46.440 is the object, this one is the verb, this one's 0:20:46.480 --> 0:20:49.720 the subject, and not just consulting Google and coming up 0:20:49.720 --> 0:20:54.160 with answers to questions, but understanding puns for example. Yeah, 0:20:54.160 --> 0:20:57.399 it had to be able to uh, to respond to 0:20:57.480 --> 0:21:01.320 word play things that that, so that could interpret what 0:21:01.359 --> 0:21:05.480 was actually being asked of it. That was a significant achievement. 0:21:05.560 --> 0:21:07.880 So it's something that you can point to and say, 0:21:08.160 --> 0:21:10.320 look how hard this problem is. It's something that's very 0:21:10.359 --> 0:21:13.000 easy for us humans. We we learn language, we even 0:21:13.080 --> 0:21:16.320 learn to uh you know how, it's flexible, We even 0:21:16.560 --> 0:21:19.600 change it ourselves. I mean there are words that exist 0:21:19.680 --> 0:21:22.159 now that didn't exist when I was a kid, that, 0:21:22.280 --> 0:21:25.160 you know, things that we entered the just the popular 0:21:25.800 --> 0:21:30.480 parlance and then became actual words. So it's something that 0:21:30.640 --> 0:21:34.440 is difficult to to program into a machine. It's probably 0:21:34.440 --> 0:21:38.160 one of the real reasons why this bar is seen 0:21:38.280 --> 0:21:41.320 as like the defining element of our artificial intelligence, because 0:21:41.320 --> 0:21:45.359 it's not an easy problem. Law zs there you go. 0:21:47.800 --> 0:21:51.160 But so okay, so so there there are good reasons 0:21:51.200 --> 0:21:54.560 for this to be the bar, and there are good 0:21:54.600 --> 0:21:56.680 reasons for it to no longer be the bar. Let's 0:21:56.720 --> 0:21:59.080 let's talk about a few of those, yes, please, Okay, Well, 0:21:59.119 --> 0:22:02.119 I would like to wing out that when we're making 0:22:02.160 --> 0:22:05.080 these sort of what you might read as criticisms of 0:22:05.119 --> 0:22:09.439 the Turing test, we're not criticizing Alan Turing and his 0:22:09.760 --> 0:22:11.680 vision for this. You know, he's one of the best 0:22:11.760 --> 0:22:15.919 human people qualitatively. Yeah, well, he certainly. He certainly was 0:22:16.840 --> 0:22:20.439 like the founding father of computer science. Like the reason 0:22:20.480 --> 0:22:23.159 computer science is the way it is is largely in 0:22:23.280 --> 0:22:26.680 part due to his work. So there's that. No, Yeah, 0:22:26.680 --> 0:22:29.160 I just more I think we should focus on sort 0:22:29.160 --> 0:22:32.840 of the natural limitations of this concept as it is 0:22:33.400 --> 0:22:37.879 most often envisioned, and why it doesn't necessarily apply to 0:22:38.320 --> 0:22:41.240 artificial intelligence in general. One of them is that, in 0:22:41.240 --> 0:22:44.000 the way people use this test, it's not designed to 0:22:44.119 --> 0:22:48.600 talk about artificial intelligence in general, because artificial intelligence spans 0:22:48.600 --> 0:22:52.520 a lot more than natural language processing. I mean, it's 0:22:52.520 --> 0:22:59.040 specifically focused on mimicking human behavior in a text based conversation. Now, 0:22:59.160 --> 0:23:02.120 that would have to be a feature of what people 0:23:02.160 --> 0:23:06.159 would call say, an artificial general intelligence, something that that 0:23:06.200 --> 0:23:09.240 a general intelligence should be capable of doing, but not 0:23:09.359 --> 0:23:12.600 the only thing it should be capable of. But AI 0:23:12.720 --> 0:23:15.119 research focuses on all kinds of other things. You have 0:23:15.200 --> 0:23:20.040 AI algorithms that recognize objects by sight, that's artificial intelligence. 0:23:20.280 --> 0:23:23.840 You can have AI algorithms that control physical movement that 0:23:23.880 --> 0:23:26.840 can be artificial intelligence. You might even say that you 0:23:26.880 --> 0:23:29.680 know the room BA that's going around on your floors. 0:23:29.720 --> 0:23:32.719 There there is a type of artificial intelligence that governs 0:23:32.760 --> 0:23:35.040 its movements. Well, let let's put it to you this way, Joe. 0:23:35.200 --> 0:23:37.640