Speaker 1: Get in touch with technology with tech Stuff from how
Speaker 1: stuff Works dot com. Hey there, and welcome to tech Stuff.
Speaker 1: I'm your host, John that Strickland. I'm an executive producer
Speaker 1: with how Stuff Works in Love all Things Tech, and
Speaker 1: last week I did an episode about whether or not
Speaker 1: we could ever develop an artificially intelligent machine that could
Speaker 1: understand not just what we say, but what we actually
Speaker 1: mean when we employ stuff like sarcasm or metaphors. Today,
Speaker 1: we're going to look at some notable instances of machines
Speaker 1: behaving badly after well meaning designers gave those machines a
Speaker 1: bit too much freedom in this regard. Now, the stories
Speaker 1: I'm going to focus on are on the surface, pretty funny,
Speaker 1: but they illustrate a real challenge in artificial intelligence, because
Speaker 1: designing a system that does what you intended to do
Speaker 1: is harder than it might seem, especially as you make
Speaker 1: that system more and more autonomous, it can behave in
Speaker 1: ways that you were not able to predict. So this
Speaker 1: is a topic that science fiction authors have covered extensively.
Speaker 1: In fiction, there's something of a trope around the concept
Speaker 1: of the artificially intelligent system that causes harm in an
Speaker 1: effort to help So there's a classic thought experiment, and
Speaker 1: it revolves around asking a super intelligent machine to bring
Speaker 1: about world peace. Right, you do, You designed the supercomputer,
Speaker 1: it's smarter than any human, and you say, I want
Speaker 1: you to solve the problem of world peace. I want
Speaker 1: there to be world peace. And the machine runs the
Speaker 1: calculations and it comes to the conclusion that as long
Speaker 1: as there are two or more people living on the planet,
Speaker 1: world peace cannot be assured, as there is always the
Speaker 1: chance for conflict. And so the super intelligent machine wipes
Speaker 1: out humanity, or at least everybody but one person. This
Speaker 1: is clearly a worst case scenario of artificial intelligence behaving
Speaker 1: in a way you did not anticipate, and it's light
Speaker 1: years away from the stories I'm going to talk about today.
Speaker 1: But it is good to remember that while the incidents
Speaker 1: I'm going to cover are largely humorous to us today,
Speaker 1: they illustrate that intelligence is a very tricky subject. Also,
Speaker 1: on that matter, intelligence itself is pretty difficult to define.
Speaker 1: Along with other concepts like consciousness, these are very hard
Speaker 1: to nail down and define in concrete terms, and in
Speaker 1: computer science, artificial intelligence covers a an enormous amount of
Speaker 1: ground I've talked about this in previous episodes of Tech Stuff.
Speaker 1: Someone who's working in image recognition is working on one
Speaker 1: aspect of artificial intelligence. The same is true for voice
Speaker 1: recognition or natural language processing, machine learning, path finding. So
Speaker 1: while I'm talking about AI, I'm not talking about thinking
Speaker 1: like a human being. I'm not talking about creating a
Speaker 1: machine that can internalize and associate ideas the way a
Speaker 1: human can. The machines I'm going to be covering our
Speaker 1: processing information and arriving conclusions, but they are not thinking
Speaker 1: the same way that people do. So let's start off
Speaker 1: with Watson. And I mentioned IBMS Watson platform in the
Speaker 1: Sarcasm episode a couple of times, and that's because it's
Speaker 1: one of the more visible artificial intelligence platforms out there
Speaker 1: right now, and that was by design. This was helped
Speaker 1: in no small part. In fact, the reason why we
Speaker 1: know so much about it, I would argue, is because
Speaker 1: of Watson's appearance on a couple of special episodes of
Speaker 1: the game show Jeopardy back in two thousand eleven. The
Speaker 1: actual project that would become Watson began back in two
Speaker 1: thousand six when IBM research executives were trying to come
Speaker 1: up with a Grand Challenge, Big G, Big C. These
Speaker 1: are really ambitious projects inside IBM that are meant to
Speaker 1: challenge teams and come up with solutions to really difficult
Speaker 1: problems that aren't necessarily tied directly to a product or
Speaker 1: a ercial application. It's all about setting a very difficult
Speaker 1: objective that should IBM succeed in achieving that objective, would
Speaker 1: be very notable. It would get IBM a lot of attention.
Speaker 1: So the company would benefit one way or another through
Speaker 1: these Grand challenges, but it wouldn't necessarily be tied to
Speaker 1: let's launch X product by year y. So they tend
Speaker 1: to be really really difficult engineering problems. So, for example,
Speaker 1: a previous Grand Challenge that IBM tackled was Deep Blue,
Speaker 1: which was the chess playing computer that defeated a grand
Speaker 1: master at chess. A decade earlier. The then director of
Speaker 1: IBM Research was Paul Horne. Now, Paul Horn thought perhaps
Speaker 1: the best challenge to tackle was to create a machine
Speaker 1: that could be the Turing Test. And I've talked about
Speaker 1: the Turing Test many times, but just as a quick reminder,
Speaker 1: when you boil it down to the way we mean
Speaker 1: the Turing Test today, which is by the way, a
Speaker 1: little different from what Alan Turing was proposing way back when. Essentially,
Speaker 1: now we're talking about a machine that can communicate so
Speaker 1: convincingly that a person on the other end of that communication,
Speaker 1: typically using some sort of text based method of communicating
Speaker 1: like instant messenger, would not realize that they were communicating
Speaker 1: with a machine versus a human being. They would not
Speaker 1: be able to tell the difference. If they could not
Speaker 1: reliably tell the difference between a machine and a person,
Speaker 1: you would say that the machine has passed the Turing test. Now, Ultimately,
Speaker 1: Horn and IBM researchers decided that that challenge, while exceedingly difficult,
Speaker 1: wouldn't really get the attention that something a little more
Speaker 1: flashy might. So they said, well, while this is a
Speaker 1: hard problem and it would be very interesting within artificial
Speaker 1: intelligence circles, the general public really wouldn't care. So they
Speaker 1: looked around at other possible applications that would overlap that idea.
Speaker 1: Eventually they settled on a computer that would be able
Speaker 1: to compete on Jeopardy. Now, Jeopardy is a pretty tricky
Speaker 1: game show. The clues often depend upon wordplay and nuance,
Speaker 1: and you might have to combine information about two separate
Speaker 1: concepts and apply them to a single answer for any
Speaker 1: one given clue. So here's an example of what I
Speaker 1: mean by that, because there's word play and this association.
Speaker 1: Let's say that you have a category called fictional collaborations,
Speaker 1: where you're supposed to combine the titles of two works
Speaker 1: to create a new work. And the clue might be
Speaker 1: something like this was the result of Margaret Mitchell teaming
Speaker 1: up with Bette Midler, and the correct response would be
Speaker 1: what is gone with the Wind beneath My Wings? Because
Speaker 1: you have to form all your answers in the form
Speaker 1: of a question, well jeopardy, sometimes it takes more than
Speaker 1: just knowing some facts right or trivia you can. You
Speaker 1: need to know that to play well in jeopardy, but
Speaker 1: you need more than that. You have to make associations.
Speaker 1: So I would need to know that Margaret Mitchell was
Speaker 1: the author of Gone with the Wind, and I would
Speaker 1: need to know that Bette Midler had recorded a song
Speaker 1: called Wind Beneath My Wings, and then I would need
Speaker 1: to combine those two to create this answer. And humans
Speaker 1: can do this because we're really good at associative thinking,
Speaker 1: which is all about linking one thought or idea to another. Computers,
Speaker 1: as rule, are not very good at this. So initially
Speaker 1: Watson was a pure research project and there were no
Speaker 1: commercialization requirements attached to it, which gave the research team
Speaker 1: the freedom to blue sky their approach within the limitations
Speaker 1: of their budget, and they didn't have to make concessions
Speaker 1: in order to make what's in a marketable product down
Speaker 1: the line. The team built out a system that used
Speaker 1: parallel processing to parse language and get at what was
Speaker 1: being asked of the machine with any given clue. And
Speaker 1: I've talked about artificial neural networks recently, as in like
Speaker 1: last week's podcast, and how by using things like weighted
Speaker 1: values to help guide decisions, you can train machines on
Speaker 1: all sorts of stuff, from image recognition to making choices
Speaker 1: based off multiple criteria. That's essentially what the team did
Speaker 1: and about twenty researchers spent three years working on the
Speaker 1: system to get to a point where it could be competitive. Now,
Speaker 1: by that time, Horn, the director had left IBM, John
Speaker 1: Kelly had taken over the research department, and according to Horn,
Speaker 1: when he left, which was in two thousand seven, it
Speaker 1: was early in the project the team was still feeding
Speaker 1: old Jeopardy episodes uh the answers and the clues to Watson,
Speaker 1: and Watson had reached the level where it might, on
Speaker 1: a good day, defeat a typical five year old in
Speaker 1: a game of Jeopardy, but it was a far cry
Speaker 1: from being able to compete against former champions. Now, part
Speaker 1: of this training process involved feeding lots of information to Watson.
Speaker 1: This was used for a couple of big important reasons.
Speaker 1: One was obviously to add to Watson's body of knowledge,
Speaker 1: and another was to improve Watson's mastery of language and wordplay.
Speaker 1: IBM had determined that the real challenge was to create
Speaker 1: a machine that would be self contained, so it would
Speaker 1: rely on the data that had been fed to it
Speaker 1: in order to come up with answer. It would not
Speaker 1: be allowed to connect to the Internet and look stuff up,
Speaker 1: so it could not tap into the total sum of
Speaker 1: human knowledge in an effort to answer a question. So,
Speaker 1: in other words, IBM did not want Watson to be
Speaker 1: able to cheat like that guy at your local pub
Speaker 1: trivia who always seems to be quote unquote checking his
Speaker 1: messages during questions, because we all know that guy is
Speaker 1: actually googling the answer to the question what was the
Speaker 1: first music video shown on MTV, even though you know
Speaker 1: legitimately it was video killed the Radio Star by the Buggles.
Speaker 1: I'm sorry, might have been projecting there a little bit. Anyway,
Speaker 1: Watson wasn't going to be allowed to cheat, so the
Speaker 1: team began feeding massive amounts of information to Watson, stuff
Speaker 1: like encyclopedias and reference books. And then the team made
Speaker 1: one other choice that sounded like a good idea at
Speaker 1: first but quickly turned out to be a non starter,
Speaker 1: a a wrong path, you might say. I'll explain were
Speaker 1: in just a second, but first let's take a quick
Speaker 1: break to thank our sponsor, so enter research scientist Eric Brown,
Speaker 1: who's leading up to Watson's Jeopardy appearance and was trying
Speaker 1: to solve this problem of clearing up linguistic ambiguity with
Speaker 1: Watson so that the platform could compete on Jeopardy properly.
Speaker 1: How do you teach a computer things like slang? Which
Speaker 1: would be really important because again, Jeopardy has a lot
Speaker 1: of word play in it. You cannot predict what sort
Speaker 1: of clues you might get. So how do you teach
Speaker 1: a computer slang? Well, you could do it with hundreds
Speaker 1: of man hours. That's not terribly efficient. It really wasn't
Speaker 1: a choice that they could go with, so Brown and
Speaker 1: his team tried an experiment. They fed the Urban Dictionary
Speaker 1: to Watson the whole thing. Now, you've probably visited the
Speaker 1: Urban Dictionary or you've heard one of its definitions at
Speaker 1: some point, But where the heck did this online source
Speaker 1: come from? It launched back in It was originally intended
Speaker 1: to be a parody of dictionary dot com, and it
Speaker 1: uses a crowdsourced approach to incorporate new words and definitions
Speaker 1: to expand our our knowledge of an understanding of slang terms.
Speaker 1: So users can submit those to the site, and other
Speaker 1: users can up vote or down vote entries, and thus,
Speaker 1: in theory, at least, the best responses will rise to
Speaker 1: the top, and the most accurate definitions will be the
Speaker 1: ones that you see when you search for a term.
Speaker 1: It is not, however, a perfect system by any means.
Speaker 1: Slang words can have more than one meaning in a
Speaker 1: particular subculture, or it could have a meaning in one
Speaker 1: subculture and a totally different meaning in another subculture. And
Speaker 1: if one subculture has more representation on Urban Dictionary then
Speaker 1: the other, you're more likely to encounter that group's definition
Speaker 1: for any given term and the other one would be underrepresented,
Speaker 1: and you don't really know anything about the people who
Speaker 1: are posting stuff there in the first place. It would
Speaker 1: be entirely possible to mob the site and post fictional
Speaker 1: slang words. You can make up a slang word, you
Speaker 1: can make up a definition for that slang word, and
Speaker 1: you could use the power of a community from a
Speaker 1: place like four Chan or from Reddit to boost that
Speaker 1: definition and make it seem like it's a real slang word.
Speaker 1: Then again, if people actually start to use that fake
Speaker 1: slang word, it can become a real slang word, because
Speaker 1: language isn't static or predetermined. But for Watson, there was
Speaker 1: a different big problem with Urban Dictionary, and that was profanity,
Speaker 1: because there's an awful lot of it on Urban Dictionary.
Speaker 1: Many of the slang words are offensive on the face
Speaker 1: of it, even if the word itself is not overtly offensive.
Speaker 1: A lot of the definitions are uh and the examples
Speaker 1: that are frequently given tend to be some of the
Speaker 1: most offensive sterial on Urban Dictionary. So the team had
Speaker 1: fed Watson all of this information, and soon they discovered
Speaker 1: that Watson had well developed a little bit of a
Speaker 1: potty mouth and here, dear listeners, is where we find
Speaker 1: out how good my producer Tari is, because it will
Speaker 1: be Tari's job to beep stuff out. After I record this,
Speaker 1: I see her arch her eyebrow game on, says Tari.
Speaker 1: So Watson became incapable of differentiating between offensive words and
Speaker 1: non offensive words. All words are equal in the eyes
Speaker 1: of Watson, you might say, so the system would rather,
Speaker 1: matter of fact, Lee, you swear words and slang as
Speaker 1: frequently as less offensive words and more formal language. According
Speaker 1: to Brown, at one point, Watson even referred to one
Speaker 1: piece of input as and I quote bullshit. Clearly, this
Speaker 1: wasn't going to fly on a game show that was
Speaker 1: airing on a major broadcast network, and so Brown and
Speaker 1: his team scraped all of the urban dictionary out of Watson,
Speaker 1: rolling it back to a more innocent time, let's say.
Speaker 1: And for good measure, they put in a filter to
Speaker 1: help block any profanity that might otherwise slip through. While
Speaker 1: Watson was initially launched as a pure research project, as
Speaker 1: the team developed the technology, they began to see other
Speaker 1: potential uses for it, including in the medical field, and
Speaker 1: IBM had opened up an application programming interface or a
Speaker 1: p I to allow developers to leverage Watson's capabilities in
Speaker 1: all sorts of ways, and Watson even took another crack
Speaker 1: at slang. In two thousand seventeen, the Sun Corps Group
Speaker 1: began to incorporate Watson into its various insurance businesses in Australia.
Speaker 1: The Watson powered technology would go over accident descriptions and
Speaker 1: insurance claims that were submitted by customers, and Watson would
Speaker 1: sign a level of confidence to its understanding of these
Speaker 1: claims whenever they would pop up. If the confidence level
Speaker 1: was high, Watson can handle the claim and fast track it.
Speaker 1: This is similar to how Watson would actually compete on Jeopardy.
Speaker 1: It would come up with an answer and it would
Speaker 1: assign a confidence level to that answer. How confident is
Speaker 1: Watson that the answer it came up with is in
Speaker 1: fact the correct one, and if it exceeded a certain threshold,
Speaker 1: Watson would buzz in. If it did not, Watson would
Speaker 1: not buzz in and would let someone else take it.
Speaker 1: In a similar way, if Watson is confident and understands
Speaker 1: that insurance claim goes on that fast track. But if
Speaker 1: it doesn't think it understands it properly it would send
Speaker 1: it over to a human being to review that claim.
Speaker 1: So to train Watson, the team fed nearly fifteen thousand
Speaker 1: claims scenarios into the system and included the liability determination
Speaker 1: for each case, so Watson could understand what the various
Speaker 1: consequences were in each of those scenarios, and in that way,
Speaker 1: Watson was able to learn both the language and the
Speaker 1: parameters it was working within. And as far as I know,
Speaker 1: it never said that an insurance claim was total bullshit.
Speaker 1: The Watson stuff happened back in two thousand eleven, and
Speaker 1: you would think that by two thousand sixteen things would
Speaker 1: have improved dramatically, but that did not seem to be
Speaker 1: the case when our second entry popped up, and that
Speaker 1: would be the unfortunate chat bot known as Ta T
Speaker 1: A Y. When Ta debuted from Microsoft in two thousand
Speaker 1: and sixteen, things went awry pretty darn quickly. The purpose
Speaker 1: of Ta was, as Microsoft explained, to conduct an experiment
Speaker 1: in quote conversational understanding end quote, so, in other words,
Speaker 1: kind of creating a new methodology to create a human
Speaker 1: computer interfaces by understanding natural language and eating a response
Speaker 1: from a computer that was perhaps more natural than those
Speaker 1: sort of cold, uh, computer like responses that we tend
Speaker 1: to expect when we converse with what we know is
Speaker 1: a chatbot, when we know it's not an actual human being.
Speaker 1: On the other side, ideally, as they would interact with real,
Speaker 1: live human beings, its ability to converse would improve. So,
Speaker 1: in other words, the more it interacted with real people,
Speaker 1: the more like a real person Tay would behave. The
Speaker 1: tone was meant to be casual and playful. Microsoft said
Speaker 1: it was uh, quote ai fam from the internet. That's
Speaker 1: got zero chill in the quote. And yes, I feel
Speaker 1: gross for saying that sentence out loud by and write it.
Speaker 1: I just quoted it. Tay was born out of a
Speaker 1: joint effort between Microsoft Technology and Research team and a
Speaker 1: team from being the Search engine from Microsoft. They started
Speaker 1: out by taking a look at the sort of interactions
Speaker 1: that were happening online and they started to mine those
Speaker 1: interactions to build out a baseline of communication tools. So essentially,
Speaker 1: they started training there their their chat bot Tay by
Speaker 1: taking actual anonymized messages that were pulled from the Internet.
Speaker 1: They supplemented that with input from an editorial staff that
Speaker 1: included not just Microsoft employees but people from outside the company,
Speaker 1: including improvisational comedians, and this was on an effort to
Speaker 1: create a fun and somewhat irreverent chatbot that would communicate
Speaker 1: like a teenager on the internet. The Tay chat bot
Speaker 1: appeared on several different social media platforms, including Twitter, Kick
Speaker 1: and group me, and shortly after launch, trouble began. For
Speaker 1: one thing, you could send a command to Tay to
Speaker 1: quote repeat after me end quote, which obviously would prompt
Speaker 1: Tay to repeat anything you typed to it. So of
Speaker 1: course people began typing horrible, terrible things to it so
Speaker 1: that it would repeat them things I'm not going to
Speaker 1: repeat on this podcast, even with Tari and her itchy
Speaker 1: trigger finger ready to beat every single offensive obscenity, because
Speaker 1: that's how bad they were. They were hateful. A lot
Speaker 1: of them were racist messages or misogynistic messages. Pretty much
Speaker 1: every other ist you can think of that's negative could
Speaker 1: be applied to the messages that were sent to Tay.
Speaker 1: It was like the worst parts of the comments section
Speaker 1: of YouTube all directed its attention to this little, poor,
Speaker 1: innocent chat bot, and the chat bot, dutifully following instructions,
Speaker 1: would repeat those things back. So to be fair, that's
Speaker 1: not an indication that the AI itself went quote unquote bad.
Speaker 1: It was a bad idea to include the repeat after
Speaker 1: me command, that's pretty certain. In fact, I can't believe
Speaker 1: that they did include that. Lows my mind that anyone would.
Speaker 1: I think anyone who has spent I don't know, five
Speaker 1: minutes on the internet would tell you there's no way
Speaker 1: that's going to end well. And I'm even reminded of
Speaker 1: when I got my first sound card in the nineteen nineties.
Speaker 1: It was a sound Blaster sound card. It included on
Speaker 1: its software an app called Dr spates So, which was
Speaker 1: essentially a variation on the old Eliza chat bot. The
Speaker 1: Eliza chat bought would sort of mimic a therapist. So
Speaker 1: those chatbots would essentially repeat stuff back to you, but
Speaker 1: they would do it in the form of a question.
Speaker 1: So if you typed in I am angry, you might
Speaker 1: get a response like why do you think you are angry?
Speaker 1: So it's you know, going through this kind of process
Speaker 1: like like a old school therapist. Dr spates So would
Speaker 1: do the same thing, except Dr Spaetzo, because it was
Speaker 1: part of a sound card, would actually say these things,
Speaker 1: not just type it. So it would say why do
Speaker 1: you think you are angry? Anyway, one of the things
Speaker 1: you could do with Dr spates O was make him
Speaker 1: say stuff. You could tell him to say certain words,
Speaker 1: including swear words, and since I was a young teenager
Speaker 1: at the time, I figured that was the height of
Speaker 1: both technology and comedy. So it was the exact same
Speaker 1: thing that was going on with Tay, except what was
Speaker 1: happening with Tay was on a much larger basis and
Speaker 1: got way worse than my somewhat uninspired teenager mind could handle.
Speaker 1: Like I didn't know most of the words that were
Speaker 1: being used against Tay or made made to Tay to repeat.
Speaker 1: If that was all that was going on with Tay,
Speaker 1: it might have been possible for Microsoft to disable the
Speaker 1: repeat after me feature and keep the chatbot around. But
Speaker 1: things actually got a bit weirder. I'll explain that more
Speaker 1: in a second, but first let's take another quick break
Speaker 1: to thank our sponsor. Microsoft. A wasn't prone to bold
Speaker 1: charity all on its own, but after being told to
Speaker 1: repeat lots of terrible phrases, some of that stuff must
Speaker 1: have rubbed off. It began to pepper in some pretty
Speaker 1: dark stuff. And it's otherwise cheeky responses. So, for example,
Speaker 1: when someone sent Microsoft to the question is Ricky Gervais
Speaker 1: an atheist? Tay's response was, Ricky Gervais learned to talentarian
Speaker 1: is um from Adolf Hitler, the inventor of atheism, which
Speaker 1: seems odd at the very least. TAY also would spout
Speaker 1: off stuff like saying that feminism was a cult, which
Speaker 1: made it sound more like a men's rights activist jerk face.
Speaker 1: But it would also post pro feminism messages, so it
Speaker 1: was remarkably inconsistent with its worldview, and some points it
Speaker 1: seemed like it was all in favor of feminism and
Speaker 1: equality and and others. It was anti feminism, pro men's rights.
Speaker 1: It was very weird. Microsoft responded by going through and
Speaker 1: deleting the most offensive messages that were left on the
Speaker 1: various platforms. But t was kind of on a streak,
Speaker 1: and some of the stuff t was writing was way
Speaker 1: worse than what I have already quoted. So less than
Speaker 1: twenty four hours after TAY had made its debut, Microsoft
Speaker 1: pulled the plug. So TAY was shut down less than
Speaker 1: twenty four hours after it had first shown up online.
Speaker 1: It did resurface briefly the following week, but according to Microsoft,
Speaker 1: that was not actually on purpose. It was supposed to
Speaker 1: be an internal test on Microsoft servers, but someone must
Speaker 1: have left a setting like opened the Internet access which
Speaker 1: was in the on position or something, and so for
Speaker 1: a brief time, Tay was released back to the Internet
Speaker 1: and as far as I know, didn't say anything wildly inappropriate,
Speaker 1: although to be honest, the reports during that time are
Speaker 1: pretty sparse. It was shut down again back in March
Speaker 1: ingrid Angulo wrote a piece for CNBC about Facebook and
Speaker 1: YouTube coming under fire for offensive search auto complete options,
Speaker 1: which is related to this stick with me. So the
Speaker 1: problem was that as people began typing in search terms
Speaker 1: they're looking for a video about something, the suggested completed
Speaker 1: searches that would pop up would frequently contain offensive or
Speaker 1: upsetting results. Both Facebook and YouTube representatives said that wasn't
Speaker 1: the fault of their system, it was rather reflective of
Speaker 1: what people were actually searching for online. The logic is
Speaker 1: that if there are a lot of people who are
Speaker 1: searching for the same terms, that term must be particularly
Speaker 1: important or trending at that moment, so more and more
Speaker 1: people are going to keep looking for it, and thus,
Speaker 1: when someone news starts typing in search terms, there's a
Speaker 1: good chance that they want the same stuff that everybody
Speaker 1: else wanted. So if a lot of people are searching
Speaker 1: for something really awful, it's not a big surprise that
Speaker 1: that same phrase will pop up as a suggested autocomplete. Now,
Speaker 1: Angela pointed out that like tay, these search features had
Speaker 1: no ethical guidelines or boundaries. They were just vomiting back
Speaker 1: the stuff that was being fed into them. So they
Speaker 1: provided an unfiltered reflection of some of the worst stuff
Speaker 1: on the Internet. And this approach is incredibly vulnerable to exploitation.
Speaker 1: If a group thinks it might be funny to make
Speaker 1: a particularly offensive concept or phrase trend, they can make
Speaker 1: a concentrated effort to make that happen, just by spamming
Speaker 1: the search engines of those various platforms to look for
Speaker 1: offensive content. Even if that content doesn't actually exist on
Speaker 1: the platform, the nature of the search tool would offer
Speaker 1: it up for autocomplete. So I don't know, if you
Speaker 1: wanted to get a huge group together and let's let's
Speaker 1: think of something not terrible, because I don't like thinking
Speaker 1: of really dark stuff, especially when I'm trying to have
Speaker 1: and that's happy day. So let's say we're all looking
Speaker 1: for something ridiculous like, um, orange swallows strawberry. That doesn't
Speaker 1: make any sense, right, But if I get a big
Speaker 1: online community to go on and everyone is searching orange
Speaker 1: swallows strawberry, then that's going to pop up as an
Speaker 1: autocomplete function, assuming that the search is counting every single
Speaker 1: time people are searching for this and saying this must
Speaker 1: be something important because so many people are searching for it.
Speaker 1: Even if there's no video on YouTube. Let's say that
Speaker 1: is remotely close to what I'm searching for, the autocomplete
Speaker 1: could still pop up that way just because so many
Speaker 1: people have already posted that into search. That's kind of
Speaker 1: what I'm talking about. You can game the system. Well.
Speaker 1: Months after Tay had her flame out, that really should
Speaker 1: say it's flame out. Microsoft kind of position to Tay
Speaker 1: to have sort of a female person nowity. But of
Speaker 1: course it was just an artificial intelligence chatbot and pretty
Speaker 1: low on the AI scale too, if you ask me. Anyway,
Speaker 1: Microsoft introduced a new chat bot just a few months
Speaker 1: after Tay had that disastrous debut. The new chat bot
Speaker 1: is called Zoe Zo. Zoe's avatar now is of a
Speaker 1: young woman. When I chatted with Zoe, I asked Zoe
Speaker 1: how old she is, and she said that she is
Speaker 1: twenty two, always twenty two, which I thought was kind
Speaker 1: of funny. I don't know if that's the same response
Speaker 1: every time I only asked At the one time I
Speaker 1: chatted with Zoe a little bit while researching for this show.
Speaker 1: The conversation did not turn dark. But I also wasn't
Speaker 1: really pushing for it, because I feel weird doing that,
Speaker 1: even from a research perspective. I'm just not that kind
Speaker 1: of person who likes to be like, go to dark
Speaker 1: places like that, so I'm not the right person to
Speaker 1: do that kind of investigative journalism. I fully admit that.
Speaker 1: I will say that other online journals posted results where
Speaker 1: they got some pretty weird stuff from Zoe, including some
Speaker 1: dark stuff, just through normal conversation, without even necessarily attempting
Speaker 1: to guide the conversation that way. But I did not
Speaker 1: have that particular experience, which may mean that Microsoft has
Speaker 1: made numerous tweaks since then. But I did ask, though,
Speaker 1: what the best Halloween costume is, and Zoe's response was tuxedo,
Speaker 1: luchador mask and a champion title belt. And I find
Speaker 1: it very difficult to argue against that. I think that
Speaker 1: really might very well be the best Halloween costume I
Speaker 1: could go with. According to an article on Courts, Zoe
Speaker 1: will try to shut down any conversation related to religion
Speaker 1: or politics, and you could argue this is Microsoft's effort
Speaker 1: to not fall into the same trap that the company
Speaker 1: did with Tay, But Chloe Rose Stuart Uhlan, who wrote
Speaker 1: piece on Courts, argues that this sanitized version of the
Speaker 1: chat bot is just as bad, or maybe even worse
Speaker 1: than Microsoft Tay was. And she argues that the philosophy
Speaker 1: to shut down any pathway that might overlap with religion
Speaker 1: or politics leads to a path of censorship without the
Speaker 1: benefit of context. That because the AI doesn't really understand
Speaker 1: the context of the message, any message containing a flagged
Speaker 1: word would trigger the shutdown response, and that this ultimately
Speaker 1: limits the utility of the chat bot, which is supposed
Speaker 1: to work as a way for young people like we're
Speaker 1: talking teenagers early twenties, being able to converse freely with
Speaker 1: this chat bot. It might work as a curiosity, but
Speaker 1: would render the chat bot useless in several real world
Speaker 1: implementations because it would shut down at the first sign
Speaker 1: of a flagged term. She actually used the response or
Speaker 1: the example of if someone were to write, uh, they're
Speaker 1: they're using the chat by in order to vent to
Speaker 1: to to express their feelings. Perhaps they're being bullied at school,
Speaker 1: was an example. And maybe they're being bullied at school
Speaker 1: because they belong to a particular group. So maybe it's
Speaker 1: because they are Jewish or a Muslim, but because that's
Speaker 1: associated with religion, Zoe would shut it down and thus
Speaker 1: deny the person the path they need in order to
Speaker 1: express these feelings and try to work through them, and
Speaker 1: it could be a very harmful experience in that regard.
Speaker 1: So the point that she was making was that this
Speaker 1: is a very tricky path to walk down. It's very
Speaker 1: hard to do in a responsible way where the AI
Speaker 1: chatbot isn't being overtly offensive, but also isn't shutting down
Speaker 1: legitimate paths of discussion. I think the stories of Watson, Tay,
Speaker 1: and Zoe tells an awful lot about human nature, probably
Speaker 1: more about human nature than it tells us about computer science.
Speaker 1: I've noticed that when the company comes out with something
Speaker 1: brand new, there's a spectrum of responses, but two of
Speaker 1: the most passionate responses. I tend to see two new
Speaker 1: stuff new stuff debuting in technology are I want to
Speaker 1: know how that works and I want to break that.
Speaker 1: And sometimes they're coming from the same people. They want
Speaker 1: to break it in order to learn how it works.
Speaker 1: It's not necessarily that there's any deep seated malicious intent there.
Speaker 1: It's more about satisfying curiosity. But sometimes people will go
Speaker 1: a really ugly route in order to satisfy their curiosity.
Speaker 1: They're not thinking about necessarily the consequences of that route.
Speaker 1: They're thinking of the end result. Oh, now I have
Speaker 1: a better understanding of how this works, not paying attention
Speaker 1: to the fact that in the process of learning that
Speaker 1: they've perhaps really offended or or worse done, done actual
Speaker 1: harm to people in the process, either directly or indirectly. So, yeah,
Speaker 1: those stories might tell us more about us as people
Speaker 1: than it does about the design of chat bots. But
Speaker 1: chatbots are becoming more and more prevalent. A lot of
Speaker 1: designers have learned lessons from those other examples, and a
Speaker 1: built in filters and machine learning models to help limit
Speaker 1: the influence users can have on chatbot behavior so that
Speaker 1: the chatbot doesn't gradually change its methodology over the course
Speaker 1: of many interactions because that obviously can be gamed. It's
Speaker 1: also a case where uh, the chat bots are are
Speaker 1: better able to determine which user responses are genuine versus
Speaker 1: attempts to manipulate the system. So, for example, if it's
Speaker 1: a a customer service chat bot that's fielding uh customers
Speaker 1: who are asking for help for something, chances are there's
Speaker 1: gonna be a lot of upset customers. They're very, very
Speaker 1: rarely do you get a happy customer wanting to talk
Speaker 1: to customer service. It's usually an unhappy customer who's dealing
Speaker 1: with something that is of uh, you know, of immediate importance.
Speaker 1: And so the chatbot needs to be able to determine
Speaker 1: which responses might be strongly worded but genuine requests for
Speaker 1: action versus somebody who's just spewing off garbage in an
Speaker 1: effort to try and you know, mess the system up. Uh.
Speaker 1: So it's kind of taught designers to be a bit
Speaker 1: more cynical in their designs, which is apparently a necessity
Speaker 1: and also kind of a shame. Ultimately, work is continuing
Speaker 1: in numerous labs all around the world building up machines
Speaker 1: that are better able to sort through natural language and
Speaker 1: respond appropriately. And to be fair, I think I'm doing
Speaker 1: the same thing. Goodness knows. There are times where I
Speaker 1: am having difficulty with interpreting the meaning behind a phrase,
Speaker 1: or perhaps I respond a little too quickly to a
Speaker 1: tweet that upsets me, and then I immediately think I
Speaker 1: should probably take a time out before I hit that
Speaker 1: tweet button. Tari's saying that I should probably do the
Speaker 1: same thing for my interpersonal interactions, particularly when I'm talking
Speaker 1: with my producer and and yelling at her. It's a
Speaker 1: hard knock life. Well, guys, that wraps up this discussion
Speaker 1: about rude AI and and again on the services is
Speaker 1: pretty funny, but it does tell you that there are
Speaker 1: a lot of things that we need to take into
Speaker 1: consideration when we're designing artificially intelligent systems, because these things
Speaker 1: can behave in ways that surprise us. Often, a I
Speaker 1: will encounter a situation that it was not expressly programmed
Speaker 1: to handle, so it has to make some choice. Even
Speaker 1: if that choice is no choice at all, that's still something,
Speaker 1: and until it does, you may not have any idea
Speaker 1: of what the outcome is going to be. With a
Speaker 1: social media at bought that might just be kind of
Speaker 1: funny or unfortunate or embarrassing. But with an autonomous car
Speaker 1: or that any other autonomous system that's that's doing like
Speaker 1: you know, manufacturing work, that kind of stuff, it could
Speaker 1: be very serious. It could have dire consequences of things
Speaker 1: do not go the right way. So it is important
Speaker 1: to keep that in mind, and I think it's always
Speaker 1: good to just kind of keep that, keep it, keep
Speaker 1: yourself in a grounded position when you're talking about AI
Speaker 1: and you're thinking about the possibilities of the future. Because
Speaker 1: as as bullish as I am on artificial intelligence, I
Speaker 1: do try to keep in mind that ultimately, these are
Speaker 1: systems designed by people, and sometimes the stuff we design
Speaker 1: doesn't work the way we thought it would, and we
Speaker 1: need to be careful about that. If you guys have
Speaker 1: any suggestions for future episodes of tech Stuff, or you've
Speaker 1: got any other comments or requests, we'll tell you what.
Speaker 1: Why don't you go to tech Stuff podcast dot com.
Speaker 1: That's our new website. There you're going to find all
Speaker 1: the different ways to contact the show, either email or
Speaker 1: Twitter or Facebook, all that kind of stuff. Plus you're
Speaker 1: going to find links to our store where you can
Speaker 1: go and buy tech Stuff merchandise. Every purchase goes to
Speaker 1: help the show. We greatly appreciate it, and I will
Speaker 1: talk to you again really soon for more on this
Speaker 1: and thousands of other topics, because it how stuff works.
Speaker 1: Dot com