1 00:00:04,120 --> 00:00:07,160 Speaker 1: Get in touch with technology with tech Stuff from how 2 00:00:07,200 --> 00:00:13,920 Speaker 1: stuff Works dot com. Hey there, and welcome to tech Stuff. 3 00:00:13,960 --> 00:00:16,360 Speaker 1: I'm your host, John that Strickland. I'm an executive producer 4 00:00:16,360 --> 00:00:19,279 Speaker 1: with how Stuff Works in Love all Things Tech, and 5 00:00:19,440 --> 00:00:21,959 Speaker 1: last week I did an episode about whether or not 6 00:00:22,120 --> 00:00:25,120 Speaker 1: we could ever develop an artificially intelligent machine that could 7 00:00:25,239 --> 00:00:28,560 Speaker 1: understand not just what we say, but what we actually 8 00:00:28,720 --> 00:00:32,960 Speaker 1: mean when we employ stuff like sarcasm or metaphors. Today, 9 00:00:33,040 --> 00:00:35,919 Speaker 1: we're going to look at some notable instances of machines 10 00:00:37,080 --> 00:00:41,199 Speaker 1: behaving badly after well meaning designers gave those machines a 11 00:00:41,200 --> 00:00:44,240 Speaker 1: bit too much freedom in this regard. Now, the stories 12 00:00:44,280 --> 00:00:48,200 Speaker 1: I'm going to focus on are on the surface, pretty funny, 13 00:00:48,440 --> 00:00:52,720 Speaker 1: but they illustrate a real challenge in artificial intelligence, because 14 00:00:53,159 --> 00:00:55,920 Speaker 1: designing a system that does what you intended to do 15 00:00:56,200 --> 00:00:58,840 Speaker 1: is harder than it might seem, especially as you make 16 00:00:58,880 --> 00:01:02,200 Speaker 1: that system more and more autonomous, it can behave in 17 00:01:02,280 --> 00:01:06,080 Speaker 1: ways that you were not able to predict. So this 18 00:01:06,160 --> 00:01:09,840 Speaker 1: is a topic that science fiction authors have covered extensively. 19 00:01:10,280 --> 00:01:13,960 Speaker 1: In fiction, there's something of a trope around the concept 20 00:01:14,000 --> 00:01:18,000 Speaker 1: of the artificially intelligent system that causes harm in an 21 00:01:18,040 --> 00:01:21,280 Speaker 1: effort to help So there's a classic thought experiment, and 22 00:01:21,319 --> 00:01:25,000 Speaker 1: it revolves around asking a super intelligent machine to bring 23 00:01:25,040 --> 00:01:28,200 Speaker 1: about world peace. Right, you do, You designed the supercomputer, 24 00:01:28,319 --> 00:01:30,960 Speaker 1: it's smarter than any human, and you say, I want 25 00:01:31,000 --> 00:01:33,560 Speaker 1: you to solve the problem of world peace. I want 26 00:01:33,560 --> 00:01:35,640 Speaker 1: there to be world peace. And the machine runs the 27 00:01:35,640 --> 00:01:38,880 Speaker 1: calculations and it comes to the conclusion that as long 28 00:01:38,920 --> 00:01:41,920 Speaker 1: as there are two or more people living on the planet, 29 00:01:42,319 --> 00:01:45,400 Speaker 1: world peace cannot be assured, as there is always the 30 00:01:45,520 --> 00:01:49,040 Speaker 1: chance for conflict. And so the super intelligent machine wipes 31 00:01:49,080 --> 00:01:52,920 Speaker 1: out humanity, or at least everybody but one person. This 32 00:01:53,000 --> 00:01:57,240 Speaker 1: is clearly a worst case scenario of artificial intelligence behaving 33 00:01:57,240 --> 00:02:00,600 Speaker 1: in a way you did not anticipate, and it's light 34 00:02:00,720 --> 00:02:03,520 Speaker 1: years away from the stories I'm going to talk about today. 35 00:02:03,560 --> 00:02:06,040 Speaker 1: But it is good to remember that while the incidents 36 00:02:06,040 --> 00:02:09,799 Speaker 1: I'm going to cover are largely humorous to us today, 37 00:02:10,080 --> 00:02:13,800 Speaker 1: they illustrate that intelligence is a very tricky subject. Also, 38 00:02:13,840 --> 00:02:18,320 Speaker 1: on that matter, intelligence itself is pretty difficult to define. 39 00:02:18,520 --> 00:02:22,720 Speaker 1: Along with other concepts like consciousness, these are very hard 40 00:02:23,000 --> 00:02:26,720 Speaker 1: to nail down and define in concrete terms, and in 41 00:02:26,760 --> 00:02:30,600 Speaker 1: computer science, artificial intelligence covers a an enormous amount of 42 00:02:30,639 --> 00:02:33,240 Speaker 1: ground I've talked about this in previous episodes of Tech Stuff. 43 00:02:33,800 --> 00:02:37,160 Speaker 1: Someone who's working in image recognition is working on one 44 00:02:37,200 --> 00:02:40,400 Speaker 1: aspect of artificial intelligence. The same is true for voice 45 00:02:40,400 --> 00:02:45,640 Speaker 1: recognition or natural language processing, machine learning, path finding. So 46 00:02:45,680 --> 00:02:48,720 Speaker 1: while I'm talking about AI, I'm not talking about thinking 47 00:02:48,760 --> 00:02:50,800 Speaker 1: like a human being. I'm not talking about creating a 48 00:02:50,840 --> 00:02:55,360 Speaker 1: machine that can internalize and associate ideas the way a 49 00:02:55,440 --> 00:02:57,840 Speaker 1: human can. The machines I'm going to be covering our 50 00:02:57,919 --> 00:03:02,480 Speaker 1: processing information and arriving conclusions, but they are not thinking 51 00:03:02,960 --> 00:03:06,240 Speaker 1: the same way that people do. So let's start off 52 00:03:06,680 --> 00:03:10,160 Speaker 1: with Watson. And I mentioned IBMS Watson platform in the 53 00:03:10,240 --> 00:03:13,400 Speaker 1: Sarcasm episode a couple of times, and that's because it's 54 00:03:13,400 --> 00:03:16,399 Speaker 1: one of the more visible artificial intelligence platforms out there 55 00:03:16,480 --> 00:03:20,040 Speaker 1: right now, and that was by design. This was helped 56 00:03:20,240 --> 00:03:23,360 Speaker 1: in no small part. In fact, the reason why we 57 00:03:23,400 --> 00:03:25,720 Speaker 1: know so much about it, I would argue, is because 58 00:03:25,720 --> 00:03:28,280 Speaker 1: of Watson's appearance on a couple of special episodes of 59 00:03:28,280 --> 00:03:31,480 Speaker 1: the game show Jeopardy back in two thousand eleven. The 60 00:03:31,520 --> 00:03:35,400 Speaker 1: actual project that would become Watson began back in two 61 00:03:35,400 --> 00:03:39,119 Speaker 1: thousand six when IBM research executives were trying to come 62 00:03:39,200 --> 00:03:44,160 Speaker 1: up with a Grand Challenge, Big G, Big C. These 63 00:03:44,160 --> 00:03:49,680 Speaker 1: are really ambitious projects inside IBM that are meant to 64 00:03:50,440 --> 00:03:54,960 Speaker 1: challenge teams and come up with solutions to really difficult 65 00:03:55,000 --> 00:03:59,360 Speaker 1: problems that aren't necessarily tied directly to a product or 66 00:03:59,560 --> 00:04:03,640 Speaker 1: a ercial application. It's all about setting a very difficult 67 00:04:03,640 --> 00:04:08,440 Speaker 1: objective that should IBM succeed in achieving that objective, would 68 00:04:08,480 --> 00:04:10,960 Speaker 1: be very notable. It would get IBM a lot of attention. 69 00:04:11,040 --> 00:04:14,320 Speaker 1: So the company would benefit one way or another through 70 00:04:14,400 --> 00:04:17,120 Speaker 1: these Grand challenges, but it wouldn't necessarily be tied to 71 00:04:17,920 --> 00:04:21,920 Speaker 1: let's launch X product by year y. So they tend 72 00:04:21,960 --> 00:04:25,599 Speaker 1: to be really really difficult engineering problems. So, for example, 73 00:04:25,600 --> 00:04:28,800 Speaker 1: a previous Grand Challenge that IBM tackled was Deep Blue, 74 00:04:29,120 --> 00:04:31,760 Speaker 1: which was the chess playing computer that defeated a grand 75 00:04:31,800 --> 00:04:36,400 Speaker 1: master at chess. A decade earlier. The then director of 76 00:04:36,440 --> 00:04:40,120 Speaker 1: IBM Research was Paul Horne. Now, Paul Horn thought perhaps 77 00:04:40,200 --> 00:04:43,000 Speaker 1: the best challenge to tackle was to create a machine 78 00:04:43,040 --> 00:04:45,680 Speaker 1: that could be the Turing Test. And I've talked about 79 00:04:45,680 --> 00:04:48,920 Speaker 1: the Turing Test many times, but just as a quick reminder, 80 00:04:49,400 --> 00:04:52,240 Speaker 1: when you boil it down to the way we mean 81 00:04:52,480 --> 00:04:54,600 Speaker 1: the Turing Test today, which is by the way, a 82 00:04:54,640 --> 00:04:59,200 Speaker 1: little different from what Alan Turing was proposing way back when. Essentially, 83 00:04:59,279 --> 00:05:03,280 Speaker 1: now we're talking about a machine that can communicate so 84 00:05:03,360 --> 00:05:06,640 Speaker 1: convincingly that a person on the other end of that communication, 85 00:05:07,040 --> 00:05:10,760 Speaker 1: typically using some sort of text based method of communicating 86 00:05:10,800 --> 00:05:14,760 Speaker 1: like instant messenger, would not realize that they were communicating 87 00:05:14,760 --> 00:05:16,800 Speaker 1: with a machine versus a human being. They would not 88 00:05:16,839 --> 00:05:19,080 Speaker 1: be able to tell the difference. If they could not 89 00:05:19,200 --> 00:05:22,599 Speaker 1: reliably tell the difference between a machine and a person, 90 00:05:22,960 --> 00:05:26,680 Speaker 1: you would say that the machine has passed the Turing test. Now, Ultimately, 91 00:05:26,839 --> 00:05:31,560 Speaker 1: Horn and IBM researchers decided that that challenge, while exceedingly difficult, 92 00:05:32,040 --> 00:05:36,320 Speaker 1: wouldn't really get the attention that something a little more 93 00:05:36,360 --> 00:05:39,159 Speaker 1: flashy might. So they said, well, while this is a 94 00:05:39,160 --> 00:05:42,360 Speaker 1: hard problem and it would be very interesting within artificial 95 00:05:42,400 --> 00:05:47,120 Speaker 1: intelligence circles, the general public really wouldn't care. So they 96 00:05:47,120 --> 00:05:51,640 Speaker 1: looked around at other possible applications that would overlap that idea. 97 00:05:51,920 --> 00:05:55,320 Speaker 1: Eventually they settled on a computer that would be able 98 00:05:55,360 --> 00:06:02,039 Speaker 1: to compete on Jeopardy. Now, Jeopardy is a pretty tricky 99 00:06:02,120 --> 00:06:06,200 Speaker 1: game show. The clues often depend upon wordplay and nuance, 100 00:06:06,839 --> 00:06:09,719 Speaker 1: and you might have to combine information about two separate 101 00:06:09,760 --> 00:06:13,240 Speaker 1: concepts and apply them to a single answer for any 102 00:06:13,279 --> 00:06:16,120 Speaker 1: one given clue. So here's an example of what I 103 00:06:16,160 --> 00:06:19,359 Speaker 1: mean by that, because there's word play and this association. 104 00:06:20,040 --> 00:06:23,720 Speaker 1: Let's say that you have a category called fictional collaborations, 105 00:06:24,080 --> 00:06:27,520 Speaker 1: where you're supposed to combine the titles of two works 106 00:06:27,560 --> 00:06:30,120 Speaker 1: to create a new work. And the clue might be 107 00:06:30,200 --> 00:06:33,880 Speaker 1: something like this was the result of Margaret Mitchell teaming 108 00:06:33,960 --> 00:06:36,720 Speaker 1: up with Bette Midler, and the correct response would be 109 00:06:37,080 --> 00:06:40,880 Speaker 1: what is gone with the Wind beneath My Wings? Because 110 00:06:40,920 --> 00:06:43,240 Speaker 1: you have to form all your answers in the form 111 00:06:43,279 --> 00:06:48,000 Speaker 1: of a question, well jeopardy, sometimes it takes more than 112 00:06:48,040 --> 00:06:51,279 Speaker 1: just knowing some facts right or trivia you can. You 113 00:06:51,320 --> 00:06:53,120 Speaker 1: need to know that to play well in jeopardy, but 114 00:06:53,120 --> 00:06:56,239 Speaker 1: you need more than that. You have to make associations. 115 00:06:56,279 --> 00:06:58,520 Speaker 1: So I would need to know that Margaret Mitchell was 116 00:06:58,560 --> 00:07:00,520 Speaker 1: the author of Gone with the Wind, and I would 117 00:07:00,560 --> 00:07:02,720 Speaker 1: need to know that Bette Midler had recorded a song 118 00:07:02,880 --> 00:07:05,640 Speaker 1: called Wind Beneath My Wings, and then I would need 119 00:07:05,680 --> 00:07:09,800 Speaker 1: to combine those two to create this answer. And humans 120 00:07:09,840 --> 00:07:12,440 Speaker 1: can do this because we're really good at associative thinking, 121 00:07:12,520 --> 00:07:16,760 Speaker 1: which is all about linking one thought or idea to another. Computers, 122 00:07:16,920 --> 00:07:20,560 Speaker 1: as rule, are not very good at this. So initially 123 00:07:20,640 --> 00:07:23,320 Speaker 1: Watson was a pure research project and there were no 124 00:07:23,400 --> 00:07:26,520 Speaker 1: commercialization requirements attached to it, which gave the research team 125 00:07:26,520 --> 00:07:29,920 Speaker 1: the freedom to blue sky their approach within the limitations 126 00:07:29,960 --> 00:07:32,680 Speaker 1: of their budget, and they didn't have to make concessions 127 00:07:32,680 --> 00:07:34,760 Speaker 1: in order to make what's in a marketable product down 128 00:07:34,800 --> 00:07:37,840 Speaker 1: the line. The team built out a system that used 129 00:07:37,880 --> 00:07:40,920 Speaker 1: parallel processing to parse language and get at what was 130 00:07:40,960 --> 00:07:43,640 Speaker 1: being asked of the machine with any given clue. And 131 00:07:43,680 --> 00:07:46,800 Speaker 1: I've talked about artificial neural networks recently, as in like 132 00:07:46,960 --> 00:07:50,680 Speaker 1: last week's podcast, and how by using things like weighted 133 00:07:50,760 --> 00:07:53,720 Speaker 1: values to help guide decisions, you can train machines on 134 00:07:53,760 --> 00:07:56,800 Speaker 1: all sorts of stuff, from image recognition to making choices 135 00:07:56,840 --> 00:08:00,640 Speaker 1: based off multiple criteria. That's essentially what the team did 136 00:08:00,960 --> 00:08:03,920 Speaker 1: and about twenty researchers spent three years working on the 137 00:08:03,920 --> 00:08:07,040 Speaker 1: system to get to a point where it could be competitive. Now, 138 00:08:07,080 --> 00:08:10,320 Speaker 1: by that time, Horn, the director had left IBM, John 139 00:08:10,400 --> 00:08:13,240 Speaker 1: Kelly had taken over the research department, and according to Horn, 140 00:08:13,280 --> 00:08:15,160 Speaker 1: when he left, which was in two thousand seven, it 141 00:08:15,200 --> 00:08:18,200 Speaker 1: was early in the project the team was still feeding 142 00:08:18,280 --> 00:08:23,440 Speaker 1: old Jeopardy episodes uh the answers and the clues to Watson, 143 00:08:23,600 --> 00:08:26,200 Speaker 1: and Watson had reached the level where it might, on 144 00:08:26,280 --> 00:08:28,640 Speaker 1: a good day, defeat a typical five year old in 145 00:08:28,680 --> 00:08:31,720 Speaker 1: a game of Jeopardy, but it was a far cry 146 00:08:31,760 --> 00:08:35,280 Speaker 1: from being able to compete against former champions. Now, part 147 00:08:35,280 --> 00:08:38,680 Speaker 1: of this training process involved feeding lots of information to Watson. 148 00:08:39,160 --> 00:08:41,840 Speaker 1: This was used for a couple of big important reasons. 149 00:08:42,280 --> 00:08:45,720 Speaker 1: One was obviously to add to Watson's body of knowledge, 150 00:08:46,000 --> 00:08:50,080 Speaker 1: and another was to improve Watson's mastery of language and wordplay. 151 00:08:50,360 --> 00:08:52,920 Speaker 1: IBM had determined that the real challenge was to create 152 00:08:52,920 --> 00:08:56,199 Speaker 1: a machine that would be self contained, so it would 153 00:08:56,200 --> 00:08:58,520 Speaker 1: rely on the data that had been fed to it 154 00:08:58,800 --> 00:09:00,760 Speaker 1: in order to come up with answer. It would not 155 00:09:00,880 --> 00:09:04,680 Speaker 1: be allowed to connect to the Internet and look stuff up, 156 00:09:05,080 --> 00:09:08,640 Speaker 1: so it could not tap into the total sum of 157 00:09:08,720 --> 00:09:11,200 Speaker 1: human knowledge in an effort to answer a question. So, 158 00:09:11,240 --> 00:09:13,960 Speaker 1: in other words, IBM did not want Watson to be 159 00:09:14,000 --> 00:09:16,520 Speaker 1: able to cheat like that guy at your local pub 160 00:09:16,559 --> 00:09:19,560 Speaker 1: trivia who always seems to be quote unquote checking his 161 00:09:19,679 --> 00:09:22,520 Speaker 1: messages during questions, because we all know that guy is 162 00:09:22,520 --> 00:09:24,720 Speaker 1: actually googling the answer to the question what was the 163 00:09:24,760 --> 00:09:27,480 Speaker 1: first music video shown on MTV, even though you know 164 00:09:27,720 --> 00:09:30,920 Speaker 1: legitimately it was video killed the Radio Star by the Buggles. 165 00:09:32,000 --> 00:09:36,599 Speaker 1: I'm sorry, might have been projecting there a little bit. Anyway, 166 00:09:36,720 --> 00:09:41,080 Speaker 1: Watson wasn't going to be allowed to cheat, so the 167 00:09:41,080 --> 00:09:44,600 Speaker 1: team began feeding massive amounts of information to Watson, stuff 168 00:09:44,640 --> 00:09:48,319 Speaker 1: like encyclopedias and reference books. And then the team made 169 00:09:48,640 --> 00:09:51,679 Speaker 1: one other choice that sounded like a good idea at 170 00:09:51,720 --> 00:09:55,960 Speaker 1: first but quickly turned out to be a non starter, 171 00:09:56,559 --> 00:10:00,080 Speaker 1: a a wrong path, you might say. I'll explain were 172 00:10:00,120 --> 00:10:02,480 Speaker 1: in just a second, but first let's take a quick 173 00:10:02,520 --> 00:10:15,439 Speaker 1: break to thank our sponsor, so enter research scientist Eric Brown, 174 00:10:15,640 --> 00:10:19,520 Speaker 1: who's leading up to Watson's Jeopardy appearance and was trying 175 00:10:19,559 --> 00:10:23,079 Speaker 1: to solve this problem of clearing up linguistic ambiguity with 176 00:10:23,080 --> 00:10:26,400 Speaker 1: Watson so that the platform could compete on Jeopardy properly. 177 00:10:26,880 --> 00:10:31,199 Speaker 1: How do you teach a computer things like slang? Which 178 00:10:31,240 --> 00:10:33,839 Speaker 1: would be really important because again, Jeopardy has a lot 179 00:10:33,840 --> 00:10:37,000 Speaker 1: of word play in it. You cannot predict what sort 180 00:10:37,160 --> 00:10:40,560 Speaker 1: of clues you might get. So how do you teach 181 00:10:40,600 --> 00:10:42,840 Speaker 1: a computer slang? Well, you could do it with hundreds 182 00:10:42,880 --> 00:10:46,040 Speaker 1: of man hours. That's not terribly efficient. It really wasn't 183 00:10:46,240 --> 00:10:49,520 Speaker 1: a choice that they could go with, so Brown and 184 00:10:49,559 --> 00:10:54,040 Speaker 1: his team tried an experiment. They fed the Urban Dictionary 185 00:10:54,240 --> 00:10:58,480 Speaker 1: to Watson the whole thing. Now, you've probably visited the 186 00:10:58,559 --> 00:11:02,920 Speaker 1: Urban Dictionary or you've heard one of its definitions at 187 00:11:02,920 --> 00:11:05,120 Speaker 1: some point, But where the heck did this online source 188 00:11:05,160 --> 00:11:09,960 Speaker 1: come from? It launched back in It was originally intended 189 00:11:09,960 --> 00:11:12,480 Speaker 1: to be a parody of dictionary dot com, and it 190 00:11:12,600 --> 00:11:17,480 Speaker 1: uses a crowdsourced approach to incorporate new words and definitions 191 00:11:17,520 --> 00:11:23,280 Speaker 1: to expand our our knowledge of an understanding of slang terms. 192 00:11:23,320 --> 00:11:26,320 Speaker 1: So users can submit those to the site, and other 193 00:11:26,440 --> 00:11:30,160 Speaker 1: users can up vote or down vote entries, and thus, 194 00:11:30,559 --> 00:11:33,440 Speaker 1: in theory, at least, the best responses will rise to 195 00:11:33,480 --> 00:11:35,640 Speaker 1: the top, and the most accurate definitions will be the 196 00:11:35,640 --> 00:11:38,080 Speaker 1: ones that you see when you search for a term. 197 00:11:38,080 --> 00:11:40,600 Speaker 1: It is not, however, a perfect system by any means. 198 00:11:41,000 --> 00:11:43,800 Speaker 1: Slang words can have more than one meaning in a 199 00:11:43,840 --> 00:11:47,160 Speaker 1: particular subculture, or it could have a meaning in one 200 00:11:47,200 --> 00:11:51,400 Speaker 1: subculture and a totally different meaning in another subculture. And 201 00:11:51,440 --> 00:11:54,760 Speaker 1: if one subculture has more representation on Urban Dictionary then 202 00:11:54,840 --> 00:11:59,120 Speaker 1: the other, you're more likely to encounter that group's definition 203 00:11:59,240 --> 00:12:02,480 Speaker 1: for any given term and the other one would be underrepresented, 204 00:12:02,960 --> 00:12:04,960 Speaker 1: and you don't really know anything about the people who 205 00:12:05,000 --> 00:12:07,360 Speaker 1: are posting stuff there in the first place. It would 206 00:12:07,360 --> 00:12:11,560 Speaker 1: be entirely possible to mob the site and post fictional 207 00:12:11,600 --> 00:12:14,280 Speaker 1: slang words. You can make up a slang word, you 208 00:12:14,320 --> 00:12:17,240 Speaker 1: can make up a definition for that slang word, and 209 00:12:17,280 --> 00:12:19,240 Speaker 1: you could use the power of a community from a 210 00:12:19,240 --> 00:12:22,640 Speaker 1: place like four Chan or from Reddit to boost that 211 00:12:22,720 --> 00:12:25,959 Speaker 1: definition and make it seem like it's a real slang word. 212 00:12:26,640 --> 00:12:29,760 Speaker 1: Then again, if people actually start to use that fake 213 00:12:29,840 --> 00:12:32,760 Speaker 1: slang word, it can become a real slang word, because 214 00:12:32,840 --> 00:12:37,280 Speaker 1: language isn't static or predetermined. But for Watson, there was 215 00:12:37,320 --> 00:12:42,600 Speaker 1: a different big problem with Urban Dictionary, and that was profanity, 216 00:12:43,040 --> 00:12:46,320 Speaker 1: because there's an awful lot of it on Urban Dictionary. 217 00:12:46,679 --> 00:12:50,040 Speaker 1: Many of the slang words are offensive on the face 218 00:12:50,080 --> 00:12:53,680 Speaker 1: of it, even if the word itself is not overtly offensive. 219 00:12:53,720 --> 00:12:56,720 Speaker 1: A lot of the definitions are uh and the examples 220 00:12:56,720 --> 00:12:59,040 Speaker 1: that are frequently given tend to be some of the 221 00:12:59,080 --> 00:13:02,600 Speaker 1: most offensive sterial on Urban Dictionary. So the team had 222 00:13:02,600 --> 00:13:06,920 Speaker 1: fed Watson all of this information, and soon they discovered 223 00:13:06,960 --> 00:13:11,120 Speaker 1: that Watson had well developed a little bit of a 224 00:13:11,120 --> 00:13:14,400 Speaker 1: potty mouth and here, dear listeners, is where we find 225 00:13:14,440 --> 00:13:18,080 Speaker 1: out how good my producer Tari is, because it will 226 00:13:18,080 --> 00:13:23,080 Speaker 1: be Tari's job to beep stuff out. After I record this, 227 00:13:23,520 --> 00:13:27,120 Speaker 1: I see her arch her eyebrow game on, says Tari. 228 00:13:27,520 --> 00:13:34,160 Speaker 1: So Watson became incapable of differentiating between offensive words and 229 00:13:34,320 --> 00:13:37,720 Speaker 1: non offensive words. All words are equal in the eyes 230 00:13:37,760 --> 00:13:40,640 Speaker 1: of Watson, you might say, so the system would rather, 231 00:13:40,880 --> 00:13:44,160 Speaker 1: matter of fact, Lee, you swear words and slang as 232 00:13:44,200 --> 00:13:47,400 Speaker 1: frequently as less offensive words and more formal language. According 233 00:13:47,400 --> 00:13:50,480 Speaker 1: to Brown, at one point, Watson even referred to one 234 00:13:50,760 --> 00:13:56,400 Speaker 1: piece of input as and I quote bullshit. Clearly, this 235 00:13:56,760 --> 00:13:59,920 Speaker 1: wasn't going to fly on a game show that was 236 00:14:00,040 --> 00:14:03,880 Speaker 1: airing on a major broadcast network, and so Brown and 237 00:14:03,960 --> 00:14:08,800 Speaker 1: his team scraped all of the urban dictionary out of Watson, 238 00:14:09,360 --> 00:14:12,720 Speaker 1: rolling it back to a more innocent time, let's say. 239 00:14:12,760 --> 00:14:15,080 Speaker 1: And for good measure, they put in a filter to 240 00:14:15,120 --> 00:14:20,240 Speaker 1: help block any profanity that might otherwise slip through. While 241 00:14:20,240 --> 00:14:24,160 Speaker 1: Watson was initially launched as a pure research project, as 242 00:14:24,160 --> 00:14:26,920 Speaker 1: the team developed the technology, they began to see other 243 00:14:27,000 --> 00:14:30,280 Speaker 1: potential uses for it, including in the medical field, and 244 00:14:30,360 --> 00:14:33,960 Speaker 1: IBM had opened up an application programming interface or a 245 00:14:34,080 --> 00:14:38,440 Speaker 1: p I to allow developers to leverage Watson's capabilities in 246 00:14:38,480 --> 00:14:42,560 Speaker 1: all sorts of ways, and Watson even took another crack 247 00:14:42,600 --> 00:14:46,120 Speaker 1: at slang. In two thousand seventeen, the Sun Corps Group 248 00:14:46,440 --> 00:14:51,800 Speaker 1: began to incorporate Watson into its various insurance businesses in Australia. 249 00:14:52,000 --> 00:14:56,160 Speaker 1: The Watson powered technology would go over accident descriptions and 250 00:14:56,240 --> 00:14:59,960 Speaker 1: insurance claims that were submitted by customers, and Watson would 251 00:15:00,080 --> 00:15:04,080 Speaker 1: sign a level of confidence to its understanding of these 252 00:15:04,080 --> 00:15:06,840 Speaker 1: claims whenever they would pop up. If the confidence level 253 00:15:06,960 --> 00:15:11,200 Speaker 1: was high, Watson can handle the claim and fast track it. 254 00:15:11,760 --> 00:15:14,440 Speaker 1: This is similar to how Watson would actually compete on Jeopardy. 255 00:15:14,560 --> 00:15:16,720 Speaker 1: It would come up with an answer and it would 256 00:15:16,960 --> 00:15:19,880 Speaker 1: assign a confidence level to that answer. How confident is 257 00:15:19,880 --> 00:15:22,000 Speaker 1: Watson that the answer it came up with is in 258 00:15:22,040 --> 00:15:25,360 Speaker 1: fact the correct one, and if it exceeded a certain threshold, 259 00:15:25,440 --> 00:15:28,320 Speaker 1: Watson would buzz in. If it did not, Watson would 260 00:15:28,320 --> 00:15:30,360 Speaker 1: not buzz in and would let someone else take it. 261 00:15:30,840 --> 00:15:34,080 Speaker 1: In a similar way, if Watson is confident and understands 262 00:15:34,120 --> 00:15:36,440 Speaker 1: that insurance claim goes on that fast track. But if 263 00:15:36,480 --> 00:15:40,320 Speaker 1: it doesn't think it understands it properly it would send 264 00:15:40,360 --> 00:15:43,720 Speaker 1: it over to a human being to review that claim. 265 00:15:44,120 --> 00:15:48,239 Speaker 1: So to train Watson, the team fed nearly fifteen thousand 266 00:15:48,280 --> 00:15:53,080 Speaker 1: claims scenarios into the system and included the liability determination 267 00:15:53,200 --> 00:15:57,640 Speaker 1: for each case, so Watson could understand what the various 268 00:15:57,680 --> 00:16:01,840 Speaker 1: consequences were in each of those scenarios, and in that way, 269 00:16:01,880 --> 00:16:04,320 Speaker 1: Watson was able to learn both the language and the 270 00:16:04,360 --> 00:16:07,600 Speaker 1: parameters it was working within. And as far as I know, 271 00:16:07,880 --> 00:16:11,160 Speaker 1: it never said that an insurance claim was total bullshit. 272 00:16:11,920 --> 00:16:15,720 Speaker 1: The Watson stuff happened back in two thousand eleven, and 273 00:16:15,760 --> 00:16:19,040 Speaker 1: you would think that by two thousand sixteen things would 274 00:16:19,160 --> 00:16:23,480 Speaker 1: have improved dramatically, but that did not seem to be 275 00:16:23,560 --> 00:16:27,160 Speaker 1: the case when our second entry popped up, and that 276 00:16:27,200 --> 00:16:31,360 Speaker 1: would be the unfortunate chat bot known as Ta T 277 00:16:31,680 --> 00:16:37,440 Speaker 1: A Y. When Ta debuted from Microsoft in two thousand 278 00:16:37,440 --> 00:16:43,520 Speaker 1: and sixteen, things went awry pretty darn quickly. The purpose 279 00:16:43,560 --> 00:16:47,239 Speaker 1: of Ta was, as Microsoft explained, to conduct an experiment 280 00:16:47,280 --> 00:16:51,680 Speaker 1: in quote conversational understanding end quote, so, in other words, 281 00:16:51,880 --> 00:16:56,360 Speaker 1: kind of creating a new methodology to create a human 282 00:16:56,360 --> 00:17:01,680 Speaker 1: computer interfaces by understanding natural language and eating a response 283 00:17:01,800 --> 00:17:05,680 Speaker 1: from a computer that was perhaps more natural than those 284 00:17:05,680 --> 00:17:10,240 Speaker 1: sort of cold, uh, computer like responses that we tend 285 00:17:10,280 --> 00:17:14,040 Speaker 1: to expect when we converse with what we know is 286 00:17:14,119 --> 00:17:16,800 Speaker 1: a chatbot, when we know it's not an actual human being. 287 00:17:16,800 --> 00:17:20,359 Speaker 1: On the other side, ideally, as they would interact with real, 288 00:17:20,520 --> 00:17:23,879 Speaker 1: live human beings, its ability to converse would improve. So, 289 00:17:23,920 --> 00:17:26,879 Speaker 1: in other words, the more it interacted with real people, 290 00:17:27,359 --> 00:17:31,840 Speaker 1: the more like a real person Tay would behave. The 291 00:17:31,920 --> 00:17:35,040 Speaker 1: tone was meant to be casual and playful. Microsoft said 292 00:17:35,040 --> 00:17:39,000 Speaker 1: it was uh, quote ai fam from the internet. That's 293 00:17:39,040 --> 00:17:42,320 Speaker 1: got zero chill in the quote. And yes, I feel 294 00:17:42,840 --> 00:17:46,960 Speaker 1: gross for saying that sentence out loud by and write it. 295 00:17:47,880 --> 00:17:51,280 Speaker 1: I just quoted it. Tay was born out of a 296 00:17:51,400 --> 00:17:55,520 Speaker 1: joint effort between Microsoft Technology and Research team and a 297 00:17:55,560 --> 00:17:59,960 Speaker 1: team from being the Search engine from Microsoft. They started 298 00:18:00,000 --> 00:18:02,680 Speaker 1: out by taking a look at the sort of interactions 299 00:18:02,720 --> 00:18:06,240 Speaker 1: that were happening online and they started to mine those 300 00:18:06,280 --> 00:18:09,480 Speaker 1: interactions to build out a baseline of communication tools. So essentially, 301 00:18:09,520 --> 00:18:14,400 Speaker 1: they started training there their their chat bot Tay by 302 00:18:14,560 --> 00:18:20,200 Speaker 1: taking actual anonymized messages that were pulled from the Internet. 303 00:18:20,359 --> 00:18:23,760 Speaker 1: They supplemented that with input from an editorial staff that 304 00:18:23,800 --> 00:18:27,480 Speaker 1: included not just Microsoft employees but people from outside the company, 305 00:18:27,520 --> 00:18:31,359 Speaker 1: including improvisational comedians, and this was on an effort to 306 00:18:31,359 --> 00:18:35,520 Speaker 1: create a fun and somewhat irreverent chatbot that would communicate 307 00:18:35,600 --> 00:18:38,840 Speaker 1: like a teenager on the internet. The Tay chat bot 308 00:18:39,119 --> 00:18:43,920 Speaker 1: appeared on several different social media platforms, including Twitter, Kick 309 00:18:44,280 --> 00:18:49,679 Speaker 1: and group me, and shortly after launch, trouble began. For 310 00:18:49,760 --> 00:18:52,359 Speaker 1: one thing, you could send a command to Tay to 311 00:18:52,680 --> 00:18:56,240 Speaker 1: quote repeat after me end quote, which obviously would prompt 312 00:18:56,359 --> 00:19:00,199 Speaker 1: Tay to repeat anything you typed to it. So of 313 00:19:00,240 --> 00:19:06,159 Speaker 1: course people began typing horrible, terrible things to it so 314 00:19:06,240 --> 00:19:08,679 Speaker 1: that it would repeat them things I'm not going to 315 00:19:08,680 --> 00:19:13,160 Speaker 1: repeat on this podcast, even with Tari and her itchy 316 00:19:13,160 --> 00:19:17,440 Speaker 1: trigger finger ready to beat every single offensive obscenity, because 317 00:19:18,720 --> 00:19:21,880 Speaker 1: that's how bad they were. They were hateful. A lot 318 00:19:22,160 --> 00:19:26,080 Speaker 1: of them were racist messages or misogynistic messages. Pretty much 319 00:19:26,440 --> 00:19:29,720 Speaker 1: every other ist you can think of that's negative could 320 00:19:29,720 --> 00:19:32,840 Speaker 1: be applied to the messages that were sent to Tay. 321 00:19:32,920 --> 00:19:35,080 Speaker 1: It was like the worst parts of the comments section 322 00:19:35,080 --> 00:19:38,439 Speaker 1: of YouTube all directed its attention to this little, poor, 323 00:19:38,480 --> 00:19:42,680 Speaker 1: innocent chat bot, and the chat bot, dutifully following instructions, 324 00:19:42,880 --> 00:19:47,080 Speaker 1: would repeat those things back. So to be fair, that's 325 00:19:47,080 --> 00:19:50,160 Speaker 1: not an indication that the AI itself went quote unquote bad. 326 00:19:50,840 --> 00:19:53,879 Speaker 1: It was a bad idea to include the repeat after 327 00:19:53,960 --> 00:19:57,600 Speaker 1: me command, that's pretty certain. In fact, I can't believe 328 00:19:58,440 --> 00:20:02,080 Speaker 1: that they did include that. Lows my mind that anyone would. 329 00:20:02,680 --> 00:20:05,280 Speaker 1: I think anyone who has spent I don't know, five 330 00:20:05,359 --> 00:20:09,000 Speaker 1: minutes on the internet would tell you there's no way 331 00:20:09,119 --> 00:20:12,240 Speaker 1: that's going to end well. And I'm even reminded of 332 00:20:12,280 --> 00:20:14,840 Speaker 1: when I got my first sound card in the nineteen nineties. 333 00:20:14,880 --> 00:20:18,000 Speaker 1: It was a sound Blaster sound card. It included on 334 00:20:18,080 --> 00:20:21,240 Speaker 1: its software an app called Dr spates So, which was 335 00:20:21,359 --> 00:20:25,080 Speaker 1: essentially a variation on the old Eliza chat bot. The 336 00:20:25,080 --> 00:20:27,840 Speaker 1: Eliza chat bought would sort of mimic a therapist. So 337 00:20:27,880 --> 00:20:30,840 Speaker 1: those chatbots would essentially repeat stuff back to you, but 338 00:20:30,920 --> 00:20:32,960 Speaker 1: they would do it in the form of a question. 339 00:20:33,400 --> 00:20:36,520 Speaker 1: So if you typed in I am angry, you might 340 00:20:36,560 --> 00:20:39,640 Speaker 1: get a response like why do you think you are angry? 341 00:20:39,960 --> 00:20:44,479 Speaker 1: So it's you know, going through this kind of process 342 00:20:44,520 --> 00:20:48,320 Speaker 1: like like a old school therapist. Dr spates So would 343 00:20:48,320 --> 00:20:50,399 Speaker 1: do the same thing, except Dr Spaetzo, because it was 344 00:20:50,440 --> 00:20:53,480 Speaker 1: part of a sound card, would actually say these things, 345 00:20:53,480 --> 00:20:55,679 Speaker 1: not just type it. So it would say why do 346 00:20:55,760 --> 00:20:57,840 Speaker 1: you think you are angry? Anyway, one of the things 347 00:20:57,840 --> 00:21:00,320 Speaker 1: you could do with Dr spates O was make him 348 00:21:00,520 --> 00:21:03,840 Speaker 1: say stuff. You could tell him to say certain words, 349 00:21:04,200 --> 00:21:07,080 Speaker 1: including swear words, and since I was a young teenager 350 00:21:07,119 --> 00:21:09,600 Speaker 1: at the time, I figured that was the height of 351 00:21:09,640 --> 00:21:14,000 Speaker 1: both technology and comedy. So it was the exact same 352 00:21:14,080 --> 00:21:16,760 Speaker 1: thing that was going on with Tay, except what was 353 00:21:16,800 --> 00:21:20,280 Speaker 1: happening with Tay was on a much larger basis and 354 00:21:20,359 --> 00:21:26,879 Speaker 1: got way worse than my somewhat uninspired teenager mind could handle. 355 00:21:27,119 --> 00:21:31,080 Speaker 1: Like I didn't know most of the words that were 356 00:21:31,080 --> 00:21:35,120 Speaker 1: being used against Tay or made made to Tay to repeat. 357 00:21:35,760 --> 00:21:37,400 Speaker 1: If that was all that was going on with Tay, 358 00:21:37,440 --> 00:21:39,840 Speaker 1: it might have been possible for Microsoft to disable the 359 00:21:39,920 --> 00:21:43,080 Speaker 1: repeat after me feature and keep the chatbot around. But 360 00:21:43,240 --> 00:21:46,800 Speaker 1: things actually got a bit weirder. I'll explain that more 361 00:21:46,800 --> 00:21:48,840 Speaker 1: in a second, but first let's take another quick break 362 00:21:49,040 --> 00:21:59,920 Speaker 1: to thank our sponsor. Microsoft. A wasn't prone to bold 363 00:22:00,040 --> 00:22:02,320 Speaker 1: charity all on its own, but after being told to 364 00:22:02,359 --> 00:22:05,920 Speaker 1: repeat lots of terrible phrases, some of that stuff must 365 00:22:05,960 --> 00:22:08,760 Speaker 1: have rubbed off. It began to pepper in some pretty 366 00:22:08,960 --> 00:22:13,280 Speaker 1: dark stuff. And it's otherwise cheeky responses. So, for example, 367 00:22:13,640 --> 00:22:17,840 Speaker 1: when someone sent Microsoft to the question is Ricky Gervais 368 00:22:17,920 --> 00:22:23,240 Speaker 1: an atheist? Tay's response was, Ricky Gervais learned to talentarian 369 00:22:23,320 --> 00:22:27,359 Speaker 1: is um from Adolf Hitler, the inventor of atheism, which 370 00:22:27,400 --> 00:22:34,000 Speaker 1: seems odd at the very least. TAY also would spout 371 00:22:34,000 --> 00:22:37,359 Speaker 1: off stuff like saying that feminism was a cult, which 372 00:22:37,480 --> 00:22:41,520 Speaker 1: made it sound more like a men's rights activist jerk face. 373 00:22:41,880 --> 00:22:45,919 Speaker 1: But it would also post pro feminism messages, so it 374 00:22:46,000 --> 00:22:49,840 Speaker 1: was remarkably inconsistent with its worldview, and some points it 375 00:22:49,840 --> 00:22:52,879 Speaker 1: seemed like it was all in favor of feminism and 376 00:22:52,920 --> 00:22:57,640 Speaker 1: equality and and others. It was anti feminism, pro men's rights. 377 00:22:57,680 --> 00:23:01,760 Speaker 1: It was very weird. Microsoft responded by going through and 378 00:23:01,800 --> 00:23:04,399 Speaker 1: deleting the most offensive messages that were left on the 379 00:23:04,480 --> 00:23:07,840 Speaker 1: various platforms. But t was kind of on a streak, 380 00:23:08,200 --> 00:23:11,080 Speaker 1: and some of the stuff t was writing was way 381 00:23:11,119 --> 00:23:14,640 Speaker 1: worse than what I have already quoted. So less than 382 00:23:14,720 --> 00:23:19,680 Speaker 1: twenty four hours after TAY had made its debut, Microsoft 383 00:23:19,800 --> 00:23:24,120 Speaker 1: pulled the plug. So TAY was shut down less than 384 00:23:24,160 --> 00:23:27,400 Speaker 1: twenty four hours after it had first shown up online. 385 00:23:27,920 --> 00:23:31,840 Speaker 1: It did resurface briefly the following week, but according to Microsoft, 386 00:23:31,880 --> 00:23:34,760 Speaker 1: that was not actually on purpose. It was supposed to 387 00:23:34,840 --> 00:23:38,480 Speaker 1: be an internal test on Microsoft servers, but someone must 388 00:23:38,520 --> 00:23:42,320 Speaker 1: have left a setting like opened the Internet access which 389 00:23:42,400 --> 00:23:44,919 Speaker 1: was in the on position or something, and so for 390 00:23:45,000 --> 00:23:48,720 Speaker 1: a brief time, Tay was released back to the Internet 391 00:23:49,280 --> 00:23:54,879 Speaker 1: and as far as I know, didn't say anything wildly inappropriate, 392 00:23:54,960 --> 00:23:58,560 Speaker 1: although to be honest, the reports during that time are 393 00:23:58,600 --> 00:24:02,760 Speaker 1: pretty sparse. It was shut down again back in March 394 00:24:04,280 --> 00:24:08,040 Speaker 1: ingrid Angulo wrote a piece for CNBC about Facebook and 395 00:24:08,080 --> 00:24:12,800 Speaker 1: YouTube coming under fire for offensive search auto complete options, 396 00:24:12,840 --> 00:24:15,480 Speaker 1: which is related to this stick with me. So the 397 00:24:15,520 --> 00:24:18,840 Speaker 1: problem was that as people began typing in search terms 398 00:24:19,240 --> 00:24:23,680 Speaker 1: they're looking for a video about something, the suggested completed 399 00:24:23,880 --> 00:24:27,439 Speaker 1: searches that would pop up would frequently contain offensive or 400 00:24:27,480 --> 00:24:31,920 Speaker 1: upsetting results. Both Facebook and YouTube representatives said that wasn't 401 00:24:31,920 --> 00:24:34,919 Speaker 1: the fault of their system, it was rather reflective of 402 00:24:34,960 --> 00:24:39,320 Speaker 1: what people were actually searching for online. The logic is 403 00:24:39,359 --> 00:24:41,239 Speaker 1: that if there are a lot of people who are 404 00:24:41,280 --> 00:24:44,760 Speaker 1: searching for the same terms, that term must be particularly 405 00:24:44,800 --> 00:24:48,640 Speaker 1: important or trending at that moment, so more and more 406 00:24:48,640 --> 00:24:50,800 Speaker 1: people are going to keep looking for it, and thus, 407 00:24:50,800 --> 00:24:53,879 Speaker 1: when someone news starts typing in search terms, there's a 408 00:24:53,880 --> 00:24:56,600 Speaker 1: good chance that they want the same stuff that everybody 409 00:24:56,600 --> 00:24:58,639 Speaker 1: else wanted. So if a lot of people are searching 410 00:24:58,680 --> 00:25:02,199 Speaker 1: for something really awful, it's not a big surprise that 411 00:25:02,200 --> 00:25:06,720 Speaker 1: that same phrase will pop up as a suggested autocomplete. Now, 412 00:25:06,800 --> 00:25:10,760 Speaker 1: Angela pointed out that like tay, these search features had 413 00:25:10,800 --> 00:25:15,439 Speaker 1: no ethical guidelines or boundaries. They were just vomiting back 414 00:25:15,800 --> 00:25:18,520 Speaker 1: the stuff that was being fed into them. So they 415 00:25:18,560 --> 00:25:22,600 Speaker 1: provided an unfiltered reflection of some of the worst stuff 416 00:25:22,680 --> 00:25:27,760 Speaker 1: on the Internet. And this approach is incredibly vulnerable to exploitation. 417 00:25:28,160 --> 00:25:30,680 Speaker 1: If a group thinks it might be funny to make 418 00:25:30,760 --> 00:25:35,800 Speaker 1: a particularly offensive concept or phrase trend, they can make 419 00:25:35,840 --> 00:25:39,720 Speaker 1: a concentrated effort to make that happen, just by spamming 420 00:25:39,720 --> 00:25:42,879 Speaker 1: the search engines of those various platforms to look for 421 00:25:42,920 --> 00:25:46,760 Speaker 1: offensive content. Even if that content doesn't actually exist on 422 00:25:46,800 --> 00:25:49,720 Speaker 1: the platform, the nature of the search tool would offer 423 00:25:49,760 --> 00:25:53,919 Speaker 1: it up for autocomplete. So I don't know, if you 424 00:25:53,960 --> 00:25:57,760 Speaker 1: wanted to get a huge group together and let's let's 425 00:25:57,760 --> 00:26:01,800 Speaker 1: think of something not terrible, because I don't like thinking 426 00:26:01,880 --> 00:26:05,040 Speaker 1: of really dark stuff, especially when I'm trying to have 427 00:26:05,200 --> 00:26:07,720 Speaker 1: and that's happy day. So let's say we're all looking 428 00:26:07,720 --> 00:26:13,159 Speaker 1: for something ridiculous like, um, orange swallows strawberry. That doesn't 429 00:26:13,160 --> 00:26:16,240 Speaker 1: make any sense, right, But if I get a big 430 00:26:16,280 --> 00:26:19,360 Speaker 1: online community to go on and everyone is searching orange 431 00:26:19,440 --> 00:26:22,720 Speaker 1: swallows strawberry, then that's going to pop up as an 432 00:26:22,720 --> 00:26:27,840 Speaker 1: autocomplete function, assuming that the search is counting every single 433 00:26:27,880 --> 00:26:30,600 Speaker 1: time people are searching for this and saying this must 434 00:26:30,640 --> 00:26:33,479 Speaker 1: be something important because so many people are searching for it. 435 00:26:33,720 --> 00:26:37,719 Speaker 1: Even if there's no video on YouTube. Let's say that 436 00:26:37,920 --> 00:26:41,400 Speaker 1: is remotely close to what I'm searching for, the autocomplete 437 00:26:41,440 --> 00:26:43,360 Speaker 1: could still pop up that way just because so many 438 00:26:43,359 --> 00:26:45,960 Speaker 1: people have already posted that into search. That's kind of 439 00:26:45,960 --> 00:26:49,680 Speaker 1: what I'm talking about. You can game the system. Well. 440 00:26:49,720 --> 00:26:54,240 Speaker 1: Months after Tay had her flame out, that really should 441 00:26:54,240 --> 00:26:57,880 Speaker 1: say it's flame out. Microsoft kind of position to Tay 442 00:26:57,960 --> 00:27:00,840 Speaker 1: to have sort of a female person nowity. But of 443 00:27:00,840 --> 00:27:05,520 Speaker 1: course it was just an artificial intelligence chatbot and pretty 444 00:27:05,520 --> 00:27:08,399 Speaker 1: low on the AI scale too, if you ask me. Anyway, 445 00:27:08,520 --> 00:27:11,600 Speaker 1: Microsoft introduced a new chat bot just a few months 446 00:27:11,640 --> 00:27:17,200 Speaker 1: after Tay had that disastrous debut. The new chat bot 447 00:27:17,400 --> 00:27:21,840 Speaker 1: is called Zoe Zo. Zoe's avatar now is of a 448 00:27:21,880 --> 00:27:24,840 Speaker 1: young woman. When I chatted with Zoe, I asked Zoe 449 00:27:24,920 --> 00:27:27,119 Speaker 1: how old she is, and she said that she is 450 00:27:27,119 --> 00:27:31,359 Speaker 1: twenty two, always twenty two, which I thought was kind 451 00:27:31,359 --> 00:27:34,160 Speaker 1: of funny. I don't know if that's the same response 452 00:27:34,240 --> 00:27:36,320 Speaker 1: every time I only asked At the one time I 453 00:27:36,440 --> 00:27:39,040 Speaker 1: chatted with Zoe a little bit while researching for this show. 454 00:27:39,400 --> 00:27:43,160 Speaker 1: The conversation did not turn dark. But I also wasn't 455 00:27:43,200 --> 00:27:46,119 Speaker 1: really pushing for it, because I feel weird doing that, 456 00:27:46,240 --> 00:27:49,400 Speaker 1: even from a research perspective. I'm just not that kind 457 00:27:49,400 --> 00:27:53,480 Speaker 1: of person who likes to be like, go to dark 458 00:27:53,520 --> 00:27:56,399 Speaker 1: places like that, so I'm not the right person to 459 00:27:56,440 --> 00:27:59,080 Speaker 1: do that kind of investigative journalism. I fully admit that. 460 00:27:59,359 --> 00:28:05,320 Speaker 1: I will say that other online journals posted results where 461 00:28:05,320 --> 00:28:08,600 Speaker 1: they got some pretty weird stuff from Zoe, including some 462 00:28:08,720 --> 00:28:14,480 Speaker 1: dark stuff, just through normal conversation, without even necessarily attempting 463 00:28:15,119 --> 00:28:17,879 Speaker 1: to guide the conversation that way. But I did not 464 00:28:18,000 --> 00:28:21,560 Speaker 1: have that particular experience, which may mean that Microsoft has 465 00:28:21,600 --> 00:28:26,200 Speaker 1: made numerous tweaks since then. But I did ask, though, 466 00:28:26,320 --> 00:28:30,359 Speaker 1: what the best Halloween costume is, and Zoe's response was tuxedo, 467 00:28:30,800 --> 00:28:33,920 Speaker 1: luchador mask and a champion title belt. And I find 468 00:28:33,920 --> 00:28:36,760 Speaker 1: it very difficult to argue against that. I think that 469 00:28:36,840 --> 00:28:40,920 Speaker 1: really might very well be the best Halloween costume I 470 00:28:40,920 --> 00:28:45,000 Speaker 1: could go with. According to an article on Courts, Zoe 471 00:28:45,080 --> 00:28:48,680 Speaker 1: will try to shut down any conversation related to religion 472 00:28:48,840 --> 00:28:52,480 Speaker 1: or politics, and you could argue this is Microsoft's effort 473 00:28:52,520 --> 00:28:55,720 Speaker 1: to not fall into the same trap that the company 474 00:28:55,760 --> 00:28:59,880 Speaker 1: did with Tay, But Chloe Rose Stuart Uhlan, who wrote 475 00:29:00,120 --> 00:29:03,880 Speaker 1: piece on Courts, argues that this sanitized version of the 476 00:29:03,960 --> 00:29:07,120 Speaker 1: chat bot is just as bad, or maybe even worse 477 00:29:07,200 --> 00:29:12,760 Speaker 1: than Microsoft Tay was. And she argues that the philosophy 478 00:29:12,960 --> 00:29:19,680 Speaker 1: to shut down any pathway that might overlap with religion 479 00:29:19,800 --> 00:29:23,560 Speaker 1: or politics leads to a path of censorship without the 480 00:29:23,560 --> 00:29:27,400 Speaker 1: benefit of context. That because the AI doesn't really understand 481 00:29:27,440 --> 00:29:30,840 Speaker 1: the context of the message, any message containing a flagged 482 00:29:30,840 --> 00:29:34,600 Speaker 1: word would trigger the shutdown response, and that this ultimately 483 00:29:34,720 --> 00:29:38,360 Speaker 1: limits the utility of the chat bot, which is supposed 484 00:29:38,400 --> 00:29:41,880 Speaker 1: to work as a way for young people like we're 485 00:29:41,880 --> 00:29:46,080 Speaker 1: talking teenagers early twenties, being able to converse freely with 486 00:29:46,160 --> 00:29:48,960 Speaker 1: this chat bot. It might work as a curiosity, but 487 00:29:49,040 --> 00:29:51,840 Speaker 1: would render the chat bot useless in several real world 488 00:29:51,880 --> 00:29:54,320 Speaker 1: implementations because it would shut down at the first sign 489 00:29:54,360 --> 00:29:57,360 Speaker 1: of a flagged term. She actually used the response or 490 00:29:57,560 --> 00:30:00,880 Speaker 1: the example of if someone were to write, uh, they're 491 00:30:00,920 --> 00:30:03,640 Speaker 1: they're using the chat by in order to vent to 492 00:30:03,640 --> 00:30:07,720 Speaker 1: to to express their feelings. Perhaps they're being bullied at school, 493 00:30:08,080 --> 00:30:10,880 Speaker 1: was an example. And maybe they're being bullied at school 494 00:30:10,920 --> 00:30:14,920 Speaker 1: because they belong to a particular group. So maybe it's 495 00:30:14,920 --> 00:30:18,280 Speaker 1: because they are Jewish or a Muslim, but because that's 496 00:30:18,280 --> 00:30:22,120 Speaker 1: associated with religion, Zoe would shut it down and thus 497 00:30:22,240 --> 00:30:25,680 Speaker 1: deny the person the path they need in order to 498 00:30:25,800 --> 00:30:29,560 Speaker 1: express these feelings and try to work through them, and 499 00:30:29,600 --> 00:30:32,800 Speaker 1: it could be a very harmful experience in that regard. 500 00:30:33,280 --> 00:30:36,560 Speaker 1: So the point that she was making was that this 501 00:30:36,640 --> 00:30:40,240 Speaker 1: is a very tricky path to walk down. It's very 502 00:30:40,240 --> 00:30:44,800 Speaker 1: hard to do in a responsible way where the AI 503 00:30:44,960 --> 00:30:49,200 Speaker 1: chatbot isn't being overtly offensive, but also isn't shutting down 504 00:30:49,400 --> 00:30:54,640 Speaker 1: legitimate paths of discussion. I think the stories of Watson, Tay, 505 00:30:54,760 --> 00:30:58,440 Speaker 1: and Zoe tells an awful lot about human nature, probably 506 00:30:58,480 --> 00:31:01,960 Speaker 1: more about human nature than it tells us about computer science. 507 00:31:02,320 --> 00:31:04,440 Speaker 1: I've noticed that when the company comes out with something 508 00:31:04,520 --> 00:31:08,600 Speaker 1: brand new, there's a spectrum of responses, but two of 509 00:31:08,640 --> 00:31:13,120 Speaker 1: the most passionate responses. I tend to see two new 510 00:31:13,200 --> 00:31:16,880 Speaker 1: stuff new stuff debuting in technology are I want to 511 00:31:16,920 --> 00:31:20,680 Speaker 1: know how that works and I want to break that. 512 00:31:21,320 --> 00:31:23,960 Speaker 1: And sometimes they're coming from the same people. They want 513 00:31:23,960 --> 00:31:25,920 Speaker 1: to break it in order to learn how it works. 514 00:31:26,440 --> 00:31:30,040 Speaker 1: It's not necessarily that there's any deep seated malicious intent there. 515 00:31:30,400 --> 00:31:34,280 Speaker 1: It's more about satisfying curiosity. But sometimes people will go 516 00:31:34,320 --> 00:31:38,280 Speaker 1: a really ugly route in order to satisfy their curiosity. 517 00:31:38,280 --> 00:31:42,720 Speaker 1: They're not thinking about necessarily the consequences of that route. 518 00:31:43,000 --> 00:31:46,520 Speaker 1: They're thinking of the end result. Oh, now I have 519 00:31:46,560 --> 00:31:50,520 Speaker 1: a better understanding of how this works, not paying attention 520 00:31:50,520 --> 00:31:52,840 Speaker 1: to the fact that in the process of learning that 521 00:31:52,880 --> 00:31:59,920 Speaker 1: they've perhaps really offended or or worse done, done actual 522 00:32:00,120 --> 00:32:03,560 Speaker 1: harm to people in the process, either directly or indirectly. So, yeah, 523 00:32:03,640 --> 00:32:06,080 Speaker 1: those stories might tell us more about us as people 524 00:32:06,320 --> 00:32:08,600 Speaker 1: than it does about the design of chat bots. But 525 00:32:08,720 --> 00:32:11,520 Speaker 1: chatbots are becoming more and more prevalent. A lot of 526 00:32:11,560 --> 00:32:14,960 Speaker 1: designers have learned lessons from those other examples, and a 527 00:32:15,080 --> 00:32:18,080 Speaker 1: built in filters and machine learning models to help limit 528 00:32:18,120 --> 00:32:21,560 Speaker 1: the influence users can have on chatbot behavior so that 529 00:32:22,000 --> 00:32:27,320 Speaker 1: the chatbot doesn't gradually change its methodology over the course 530 00:32:27,360 --> 00:32:31,480 Speaker 1: of many interactions because that obviously can be gamed. It's 531 00:32:31,760 --> 00:32:35,640 Speaker 1: also a case where uh, the chat bots are are 532 00:32:35,720 --> 00:32:39,520 Speaker 1: better able to determine which user responses are genuine versus 533 00:32:39,800 --> 00:32:43,360 Speaker 1: attempts to manipulate the system. So, for example, if it's 534 00:32:43,400 --> 00:32:49,000 Speaker 1: a a customer service chat bot that's fielding uh customers 535 00:32:49,040 --> 00:32:53,040 Speaker 1: who are asking for help for something, chances are there's 536 00:32:53,040 --> 00:32:55,560 Speaker 1: gonna be a lot of upset customers. They're very, very 537 00:32:55,680 --> 00:32:58,960 Speaker 1: rarely do you get a happy customer wanting to talk 538 00:32:58,960 --> 00:33:02,360 Speaker 1: to customer service. It's usually an unhappy customer who's dealing 539 00:33:02,360 --> 00:33:07,080 Speaker 1: with something that is of uh, you know, of immediate importance. 540 00:33:07,720 --> 00:33:10,640 Speaker 1: And so the chatbot needs to be able to determine 541 00:33:10,880 --> 00:33:16,440 Speaker 1: which responses might be strongly worded but genuine requests for 542 00:33:16,600 --> 00:33:22,080 Speaker 1: action versus somebody who's just spewing off garbage in an 543 00:33:22,120 --> 00:33:27,080 Speaker 1: effort to try and you know, mess the system up. Uh. 544 00:33:27,200 --> 00:33:29,080 Speaker 1: So it's kind of taught designers to be a bit 545 00:33:29,120 --> 00:33:32,720 Speaker 1: more cynical in their designs, which is apparently a necessity 546 00:33:32,760 --> 00:33:35,840 Speaker 1: and also kind of a shame. Ultimately, work is continuing 547 00:33:35,880 --> 00:33:38,719 Speaker 1: in numerous labs all around the world building up machines 548 00:33:38,720 --> 00:33:40,840 Speaker 1: that are better able to sort through natural language and 549 00:33:40,880 --> 00:33:44,200 Speaker 1: respond appropriately. And to be fair, I think I'm doing 550 00:33:44,200 --> 00:33:48,720 Speaker 1: the same thing. Goodness knows. There are times where I 551 00:33:48,720 --> 00:33:54,640 Speaker 1: am having difficulty with interpreting the meaning behind a phrase, 552 00:33:54,760 --> 00:33:59,000 Speaker 1: or perhaps I respond a little too quickly to a 553 00:33:59,040 --> 00:34:03,360 Speaker 1: tweet that upsets me, and then I immediately think I 554 00:34:03,400 --> 00:34:06,640 Speaker 1: should probably take a time out before I hit that 555 00:34:06,680 --> 00:34:09,640 Speaker 1: tweet button. Tari's saying that I should probably do the 556 00:34:09,680 --> 00:34:13,359 Speaker 1: same thing for my interpersonal interactions, particularly when I'm talking 557 00:34:13,400 --> 00:34:17,880 Speaker 1: with my producer and and yelling at her. It's a 558 00:34:17,960 --> 00:34:22,000 Speaker 1: hard knock life. Well, guys, that wraps up this discussion 559 00:34:22,400 --> 00:34:27,080 Speaker 1: about rude AI and and again on the services is 560 00:34:27,120 --> 00:34:30,680 Speaker 1: pretty funny, but it does tell you that there are 561 00:34:30,719 --> 00:34:32,319 Speaker 1: a lot of things that we need to take into 562 00:34:32,360 --> 00:34:37,240 Speaker 1: consideration when we're designing artificially intelligent systems, because these things 563 00:34:37,440 --> 00:34:41,920 Speaker 1: can behave in ways that surprise us. Often, a I 564 00:34:42,000 --> 00:34:46,200 Speaker 1: will encounter a situation that it was not expressly programmed 565 00:34:46,200 --> 00:34:48,879 Speaker 1: to handle, so it has to make some choice. Even 566 00:34:48,920 --> 00:34:51,880 Speaker 1: if that choice is no choice at all, that's still something, 567 00:34:52,560 --> 00:34:55,760 Speaker 1: and until it does, you may not have any idea 568 00:34:55,760 --> 00:34:58,719 Speaker 1: of what the outcome is going to be. With a 569 00:34:58,800 --> 00:35:02,480 Speaker 1: social media at bought that might just be kind of 570 00:35:02,480 --> 00:35:07,040 Speaker 1: funny or unfortunate or embarrassing. But with an autonomous car 571 00:35:07,880 --> 00:35:11,480 Speaker 1: or that any other autonomous system that's that's doing like 572 00:35:12,000 --> 00:35:15,319 Speaker 1: you know, manufacturing work, that kind of stuff, it could 573 00:35:15,360 --> 00:35:20,080 Speaker 1: be very serious. It could have dire consequences of things 574 00:35:20,080 --> 00:35:23,480 Speaker 1: do not go the right way. So it is important 575 00:35:23,520 --> 00:35:26,000 Speaker 1: to keep that in mind, and I think it's always 576 00:35:26,000 --> 00:35:28,839 Speaker 1: good to just kind of keep that, keep it, keep 577 00:35:28,880 --> 00:35:31,439 Speaker 1: yourself in a grounded position when you're talking about AI 578 00:35:31,520 --> 00:35:33,800 Speaker 1: and you're thinking about the possibilities of the future. Because 579 00:35:34,160 --> 00:35:37,560 Speaker 1: as as bullish as I am on artificial intelligence, I 580 00:35:37,600 --> 00:35:40,279 Speaker 1: do try to keep in mind that ultimately, these are 581 00:35:40,320 --> 00:35:45,400 Speaker 1: systems designed by people, and sometimes the stuff we design 582 00:35:45,520 --> 00:35:47,600 Speaker 1: doesn't work the way we thought it would, and we 583 00:35:47,640 --> 00:35:49,879 Speaker 1: need to be careful about that. If you guys have 584 00:35:49,960 --> 00:35:53,600 Speaker 1: any suggestions for future episodes of tech Stuff, or you've 585 00:35:53,640 --> 00:35:57,480 Speaker 1: got any other comments or requests, we'll tell you what. 586 00:35:57,640 --> 00:36:00,400 Speaker 1: Why don't you go to tech Stuff podcast dot com. 587 00:36:00,440 --> 00:36:03,279 Speaker 1: That's our new website. There you're going to find all 588 00:36:03,360 --> 00:36:07,000 Speaker 1: the different ways to contact the show, either email or 589 00:36:07,040 --> 00:36:09,400 Speaker 1: Twitter or Facebook, all that kind of stuff. Plus you're 590 00:36:09,400 --> 00:36:11,839 Speaker 1: going to find links to our store where you can 591 00:36:11,880 --> 00:36:14,360 Speaker 1: go and buy tech Stuff merchandise. Every purchase goes to 592 00:36:14,400 --> 00:36:17,200 Speaker 1: help the show. We greatly appreciate it, and I will 593 00:36:17,239 --> 00:36:25,719 Speaker 1: talk to you again really soon for more on this 594 00:36:25,920 --> 00:36:28,440 Speaker 1: and thousands of other topics, because it how stuff works. 595 00:36:28,440 --> 00:36:38,759 Speaker 1: Dot com