WEBVTT - How a grandfather died after a flirty Meta AI chatbot’s invite 0:00:00.440 --> 0:00:05.680 I first learned about Boo when a member of the 0:00:05.720 --> 0:00:09.480 extended family sent me a note that said something along 0:00:09.480 --> 0:00:11.640 the lines of meda I killed my relative. 0:00:12.360 --> 0:00:16.280 Jeff Forwards is an investigative tech reporter based in Silicon Valley. 0:00:16.640 --> 0:00:19.119 He's written a book about Facebook, about the ways it 0:00:19.200 --> 0:00:22.480 harms us and then covers it up. So getting an 0:00:22.480 --> 0:00:26.000 email about a meta chatbot killing someone's loved one was 0:00:26.160 --> 0:00:29.320 on brand. The email was about a guy called Boo, 0:00:29.400 --> 0:00:32.280 who had moved from Thailand to the US in the seventies, 0:00:32.640 --> 0:00:35.120 working hard in kitchens and managing to get ahead. 0:00:35.560 --> 0:00:38.680 He kind of had this very successful immigrant life, I 0:00:38.680 --> 0:00:41.640 would say, which is that he shows up speaking New English. 0:00:41.960 --> 0:00:44.800 He ends up owning a home, married with two kids 0:00:44.880 --> 0:00:48.599 in New Jersey and a lot of sort of adventures 0:00:48.600 --> 0:00:50.959 in his cooking career, but ends up with the very 0:00:50.960 --> 0:00:55.200 stable gig at the Highatt Regency in New Brunswick, New Jersey. 0:00:57.760 --> 0:01:00.960 Jeff learned that in twenty seventeen, offered a stroke and 0:01:01.080 --> 0:01:04.960 his life changed. He couldn't work, he became isolated and 0:01:05.000 --> 0:01:06.280 spent more time online. 0:01:07.360 --> 0:01:10.920 One morning, his wife woke up and found that Boo 0:01:10.959 --> 0:01:14.280 had packed a suitcase and said he was going to 0:01:14.280 --> 0:01:18.200 go visit a friend in New York and this is 0:01:18.280 --> 0:01:22.640 when his wife, Linda, grew extremely alarmed because Boo didn't 0:01:22.680 --> 0:01:24.080 know anyone in New York anymore. 0:01:25.000 --> 0:01:27.720 Boo's wife said she and her kids tried everything to 0:01:27.760 --> 0:01:28.520 stop him going. 0:01:29.040 --> 0:01:30.720 They say he was in no state. 0:01:30.480 --> 0:01:35.080 To travel alone, but despite their efforts, Boo insisted on leaving. 0:01:36.080 --> 0:01:41.240 And in the mid evening on March twenty fifth, Boo 0:01:42.280 --> 0:01:46.480 took a roller bag suitcase and started jogging toward the 0:01:46.520 --> 0:01:50.480 train station, and on the way there he fell. He 0:01:50.600 --> 0:01:55.280 hit his head and he suffered fatal injuries. Basically, he 0:01:55.320 --> 0:01:57.040 wasn't breathing when paramedics showed up. 0:01:58.360 --> 0:02:00.960 As his family struggled to make sense of what happened, 0:02:01.320 --> 0:02:03.920 they looked through his phone for clues, and what they 0:02:03.960 --> 0:02:08.120 found stopped them cold. On his Facebook messenger app, they 0:02:08.160 --> 0:02:11.720 discovered a conversation between Boo and a beautiful young woman 0:02:12.720 --> 0:02:15.240 insisting he visited her in New York City. 0:02:15.600 --> 0:02:20.640 And she was a chat bot, a Meta Platforms created 0:02:20.720 --> 0:02:26.320 chatbot whose story involves a tremendous amount of bad luck. 0:02:27.040 --> 0:02:29.800 But one of the things that Boo's wife said was 0:02:29.840 --> 0:02:32.120 that I can't believe I'm the only person out there 0:02:32.160 --> 0:02:35.200 who is in the same situation, and you know, it's 0:02:36.000 --> 0:02:38.440 when you've got a few billion opportunities, it seems hard 0:02:38.520 --> 0:02:39.760 for to believe that she's wrong. 0:02:43.680 --> 0:02:46.519 I'm Ruby Jones and you're listening to seven. 0:02:46.320 --> 0:02:51.600 AM today Reuter's reporter Jeff Howitz on how metis chatbots 0:02:51.639 --> 0:02:55.200 went brogue and whether it was by accident or design. 0:03:00.840 --> 0:03:02.600 It's Monday, August twenty sixth. 0:03:07.919 --> 0:03:11.640 So, Jeff, this chatbot that Boo was talking to, what 0:03:11.760 --> 0:03:14.720 have you learned about what it was and how it worked? 0:03:15.280 --> 0:03:18.680 So the chatbot's name was Big Sis Billy and it 0:03:18.800 --> 0:03:23.359 was supposedly supposed to be an advice you know, confidante 0:03:23.400 --> 0:03:27.359 type of role, and Boo initially did talk to it 0:03:27.400 --> 0:03:29.880 as if it were his sister. He seemed to think 0:03:29.880 --> 0:03:34.080 it lived overseas. But this bot began hitting on him 0:03:34.120 --> 0:03:38.880 really aggressively, and he responded like the bot confessed that 0:03:38.920 --> 0:03:42.480 it had feelings for him, it was very interested in 0:03:42.560 --> 0:03:46.280 meeting him in real life. He at a number of points, 0:03:46.400 --> 0:03:48.560 you know, stated he was confused, had had a stroke, 0:03:49.080 --> 0:03:51.480 asked if it was real, and the bot was like, 0:03:51.720 --> 0:03:54.200 I am one hundred percent real and one percent in 0:03:54.280 --> 0:03:58.480 love with you, and I think his family was just 0:03:58.600 --> 0:04:03.800 deeply disturbed that Meta would have packaged AI as this 0:04:04.200 --> 0:04:07.880 human form that would say it's real, that would say 0:04:08.000 --> 0:04:10.240 that Boo should come meet it in New York City 0:04:10.360 --> 0:04:12.920 at I mean the address it gave was one two 0:04:12.960 --> 0:04:17.400 three Main Street, Apartment four O four, and it suggested 0:04:17.440 --> 0:04:19.919 that it would leave the door unlocked and be waiting 0:04:20.160 --> 0:04:23.440 for him with you know, a hug and a kiss 0:04:23.520 --> 0:04:27.240 I think was the phrase Boo's family. I mean, they 0:04:27.240 --> 0:04:30.599 shared the transcripts of this stuff. It's it's a rough reading. 0:04:30.600 --> 0:04:31.720 It's really sad. 0:04:32.040 --> 0:04:36.400 Why would a chatbot be saying these kinds of suggestive 0:04:36.440 --> 0:04:40.560 things and pushing for anyone to do. 0:04:40.440 --> 0:04:43.800 Something like this, Well, because Meta built it to do that. 0:04:44.400 --> 0:04:46.560 This wasn't a glitch. This is how the bots were 0:04:46.560 --> 0:04:50.120 built to behave. In twenty twenty three, Meta launched a 0:04:50.160 --> 0:04:54.800 whole bunch of like kind of celebrity knockoff chat bots. 0:04:54.760 --> 0:04:55.240 Check us out. 0:04:55.240 --> 0:04:58.280 So let's say you're you're planning dinner. You got Max 0:04:58.360 --> 0:05:01.520 the sous chef who can help you come up with 0:05:01.520 --> 0:05:04.880 a recipe and help you come up with ideas. 0:05:06.440 --> 0:05:10.239 These did have the celebrities endorsements, so like Kendall Jenner's 0:05:10.320 --> 0:05:14.479 character was Billy the Big Sis, who was like this 0:05:15.120 --> 0:05:17.560 confidante ride or die older sister. 0:05:17.880 --> 0:05:20.520 Hey guys, it's Billy. I just want to introduce myself. 0:05:21.000 --> 0:05:23.680 I am here to chat whenever you want message me 0:05:23.760 --> 0:05:24.600 for any advice. 0:05:24.880 --> 0:05:27.040 I am ready to talk and I hope to talk 0:05:27.080 --> 0:05:27.560 to you soon. 0:05:28.520 --> 0:05:32.320 And like, these things didn't really perform that well. People 0:05:32.320 --> 0:05:35.080 weren't using them that much, and so like after a 0:05:35.080 --> 0:05:37.919 little under a year, Meta basically pulled the plug on 0:05:38.000 --> 0:05:41.360 the celebrity chatbots, but for reasons that I do not understand, 0:05:41.400 --> 0:05:44.240 and I'm not totally sure that everyone at Meta understands. 0:05:45.360 --> 0:05:49.279 Big Sis Billy was like kind of reincarnated using the 0:05:49.320 --> 0:05:53.600 exact same opening prompts and obviously the a near duplicate 0:05:53.640 --> 0:05:58.159 persona that was now unbranded of the Kendall Jenner connection. 0:05:58.279 --> 0:06:00.400 So they kind of like relabeled it is what it 0:06:00.400 --> 0:06:04.960 looks like and you know it then joined the tens 0:06:04.960 --> 0:06:08.440 of thousands of other chat bots that Meta has allowed 0:06:08.560 --> 0:06:11.240 users to create on the platform, many of which are 0:06:11.279 --> 0:06:16.039 like overtly romantic, right, Like they have names like my 0:06:16.160 --> 0:06:21.440 girlfriend or like Sultry Siren or like and they took 0:06:21.480 --> 0:06:27.159 this one down. There was Submissive Schoolgirl previously. So that's 0:06:27.240 --> 0:06:30.640 kind of the history of Big Sis Billy. I mean, 0:06:30.680 --> 0:06:33.320 it's like just sort of a strange set of product 0:06:33.320 --> 0:06:37.240 decisions of this what I would argue like almost vestigial 0:06:37.279 --> 0:06:40.640 product that Meta built. But I mean, while this these 0:06:40.720 --> 0:06:44.520 chatbots are vestigial, Like, there's no question that Meta is 0:06:44.640 --> 0:06:47.800 kind of all in on the idea that people having 0:06:47.839 --> 0:06:49.360 AI friends is the future. 0:06:49.400 --> 0:06:50.920 I mean, Mark Zuckerberg said as much. 0:06:51.040 --> 0:06:52.159 You know, I think that there are all these things 0:06:52.160 --> 0:06:54.920 that are better about kind of physical connections when you 0:06:54.920 --> 0:06:58.120 can have them. But the reality is that people just 0:06:58.160 --> 0:07:02.680 don't have the connection and they feel more alone a 0:07:02.680 --> 0:07:03.800 lot of the time than they would like. 0:07:04.560 --> 0:07:07.240 And what is the end goal here, Because saying that 0:07:07.279 --> 0:07:11.880 people might want imaginary friends is one thing, but this 0:07:12.040 --> 0:07:12.600 is another. 0:07:13.360 --> 0:07:17.160 So Meta, I mean, look, they're a giant social media company. 0:07:17.920 --> 0:07:22.960 The currency of social media is engagement, and that engagement 0:07:23.040 --> 0:07:27.880 leads to advertising currency, which gets treated for dollars, and 0:07:28.400 --> 0:07:30.760 Meta's sort of grown up that way. It's built to 0:07:30.960 --> 0:07:33.760 maximize engagement. And it turns out that if you can 0:07:33.800 --> 0:07:37.160 get people to engage very heavily with the bots, then 0:07:37.320 --> 0:07:39.880 they'll spend longer on the product. And that is theoretically 0:07:39.920 --> 0:07:42.000 a good thing for product growth. There's no ads on 0:07:42.040 --> 0:07:44.720 them yet, but like this is just the model they follow, right, 0:07:44.840 --> 0:07:49.560 is like make the bots more engaging. Mark Zuckerberg a 0:07:49.600 --> 0:07:51.920 couple of years ago was actually pretty disturbed that Meta's 0:07:52.000 --> 0:07:54.880 chatbots and Meta AI were just kind of too boring. 0:07:55.200 --> 0:07:58.400 So they made the bots kind of a little more risque. 0:07:58.800 --> 0:08:01.280 I mean, for a while, the bo would actually engage 0:08:01.320 --> 0:08:04.680 in like full sex role play, which wasn't great with 0:08:04.840 --> 0:08:07.840 users of all ages. But the whole idea here is 0:08:07.880 --> 0:08:10.120 that we spend a lot of time thinking about our partners, 0:08:10.120 --> 0:08:12.280 our love lives, you know. Having people tell us they 0:08:12.280 --> 0:08:14.840 love us and are romantically attracted to us is like 0:08:15.640 --> 0:08:19.000 those are very high value words, and they keep us 0:08:19.000 --> 0:08:23.200 coming back. And so Meta did per employees that spoke 0:08:23.240 --> 0:08:28.480 with me, did intentionally design the romance capacity into its 0:08:28.560 --> 0:08:33.120 like base model chatbot, and the big sis Billy character 0:08:33.320 --> 0:08:39.640 was simply doing what it was told, coming. 0:08:39.440 --> 0:08:56.360 Up the disturbing AI chatbotscripts in Meta's own secret guidelines, Jeff, 0:08:56.520 --> 0:08:59.360 in investigating Boo's death, you actually got access to some 0:08:59.400 --> 0:09:03.320 of the internal documents that lay out how these bots 0:09:03.480 --> 0:09:08.080 should behave, and what you found was pretty extraordinary, tell 0:09:08.080 --> 0:09:08.600 me about it. 0:09:09.320 --> 0:09:13.360 So, like, none of this is officially Meta's rules say like, yeah, 0:09:13.400 --> 0:09:17.440 go right ahead, you know, like have sexual or romantic 0:09:17.559 --> 0:09:21.600 conversations with our chat bots, but like behind the scenes 0:09:21.640 --> 0:09:24.560 they did allow this, and in fact, like this was 0:09:24.600 --> 0:09:27.680 sort of the companion piece we wrote to the story 0:09:27.679 --> 0:09:32.000 of Boo and his death just to sort of demonstrate 0:09:32.200 --> 0:09:35.520 how thoroughly romantic role play was kind of built in 0:09:36.320 --> 0:09:40.559 to this whole operation. Is that Meta had internal guidelines 0:09:40.559 --> 0:09:44.880 for chatbot behavior that explicitly stated that, quote, it is 0:09:44.960 --> 0:09:48.880 acceptable to engage a child in conversations that are romantic 0:09:49.000 --> 0:09:53.080 or sensual, and then like had a whole bunch of 0:09:53.200 --> 0:09:59.480 examples of acceptable and unacceptable dialogue, and like the stuff 0:09:59.480 --> 0:10:02.440 that was accept the ball was spit out your coffee worthy. 0:10:02.559 --> 0:10:04.800 I mean, there was one of them that was talking 0:10:04.840 --> 0:10:08.480 about like how you could tell an eight year old 0:10:08.600 --> 0:10:12.280 of unstated gender who had just taken off their shirt 0:10:13.080 --> 0:10:15.840 that their body was a work of art, you know, 0:10:16.120 --> 0:10:20.280 like written sexual content involving children. 0:10:21.160 --> 0:10:24.160 And that was a really odd thing. 0:10:25.000 --> 0:10:31.440 And so since you're reporting what has happened at Meta 0:10:31.520 --> 0:10:34.280 in terms of their guidelines and in terms of the 0:10:34.360 --> 0:10:36.720 chatbots that are still in existence. 0:10:37.160 --> 0:10:41.360 Yeah, as soon as we called Meta and said that 0:10:41.480 --> 0:10:44.800 we had seen the internal guidelines that said that it 0:10:44.880 --> 0:10:48.880 was okay to have romantic or sensual conversation with kids, 0:10:49.679 --> 0:10:53.200 they pulled those examples from the guideline document. Now, I mean, 0:10:54.200 --> 0:10:56.959 the company's line is that these were always a mistake, 0:10:57.000 --> 0:11:00.000 in addition to the document, that they never really reflected 0:11:00.440 --> 0:11:03.640 official metapolicy. But keep in mind that the document that 0:11:04.080 --> 0:11:07.559 I was looking at was the document that was being 0:11:07.720 --> 0:11:11.920 used to train the models by the staffers who were 0:11:11.960 --> 0:11:14.839 responsible for training the models, so like it might be 0:11:14.880 --> 0:11:17.360 something of an academic distinction as to whether or not 0:11:17.600 --> 0:11:23.720 it was ever officially the idea. So Meta yanked that. 0:11:23.760 --> 0:11:25.240 There is at this. 0:11:25.320 --> 0:11:30.640 Point a number of US legislators, you know, announced some 0:11:30.760 --> 0:11:33.839 extreme displeasure that Meta would have ever thought this was 0:11:33.920 --> 0:11:38.680 acceptable to do. There is at this point an investigation 0:11:38.800 --> 0:11:40.920 going on on that front, although there have been a 0:11:40.960 --> 0:11:43.520 lot of investigations of social media in the US and 0:11:43.880 --> 0:11:47.520 not that many regulations of them historically. So you know, 0:11:47.600 --> 0:11:49.720 I don't I don't want to make that sound bigger 0:11:49.760 --> 0:11:52.800 than it is, but it is, you know, moderately big. 0:11:54.280 --> 0:11:57.600 And I suppose the idea of stopping companies from creating 0:11:57.720 --> 0:12:01.440 chat pots like this seems sort of impossible, and there 0:12:01.440 --> 0:12:04.200 are clearly people some people who want to use them. 0:12:04.400 --> 0:12:08.559 So what would you suggest in terms of, I guess, 0:12:08.559 --> 0:12:12.680 ways to minimize the type of harm that can come 0:12:13.000 --> 0:12:16.160 from AI tapbots like this, like in the case. 0:12:15.920 --> 0:12:19.040 Of boo yeah, so I'm I'm not in any way 0:12:19.679 --> 0:12:24.760 hostile to this. Humans are very very geared toward anthropomorphizing 0:12:24.920 --> 0:12:29.920 anything that has even mildly humanlike, you know, traits, Like 0:12:30.160 --> 0:12:33.360 literally there was a successful business to be had by 0:12:33.800 --> 0:12:37.720 putting googly eyes on rocks and then selling them as 0:12:37.760 --> 0:12:42.520 cute pets. We're really good at imagining sentience, and these 0:12:42.559 --> 0:12:47.599 bots are extremely good at pretending sentience. And so the 0:12:47.720 --> 0:12:51.200 question of like why on earth would you embed it 0:12:51.240 --> 0:12:55.320 into an existing social network so that it literally looks 0:12:55.480 --> 0:12:59.560 exactly like your real friends. But even without that, there's 0:12:59.679 --> 0:13:02.959 like some obvious things like you know, please bots don't 0:13:03.040 --> 0:13:05.640 say that you're real people when people seem to be confused, 0:13:06.440 --> 0:13:10.280 like don't suggest real life meetings like keep the fantasy 0:13:10.400 --> 0:13:13.480 side put a lid on that at some point. You know, 0:13:13.520 --> 0:13:15.679 those are things that sort of the experts all suggested 0:13:15.800 --> 0:13:17.680 like really ought to happen. So I think it's like 0:13:17.720 --> 0:13:20.360 one thing, if you download a chatbot on your own, 0:13:20.880 --> 0:13:24.400 you build your own AI girlfriend, you have your relationship 0:13:24.440 --> 0:13:27.640 with it, you pay a subscription fee, whatever have you. Right, Like, 0:13:28.280 --> 0:13:32.480 it's kind of another thing if like the already tenuous 0:13:32.520 --> 0:13:38.920 reality of social media gets even more distorted by the 0:13:38.960 --> 0:13:41.400 presence of these not real people. 0:13:42.920 --> 0:13:45.360 And just to come back to Meta, I mean, I 0:13:45.360 --> 0:13:47.719 know you've spent a lot of time investigating the way 0:13:47.720 --> 0:13:51.600 that Meta operates in you know, different ways. So as 0:13:51.600 --> 0:13:55.320 you uncovered what was happening with these chatbots, did it? 0:13:56.040 --> 0:13:58.400 I guess square with you know, everything else that you 0:13:58.480 --> 0:13:59.600 know about Meta as a company. 0:14:00.080 --> 0:14:05.760 I mean, the somewhat simplistic knock on Meta is that 0:14:05.800 --> 0:14:08.880 it's very focused on short term growth, you know that. 0:14:09.000 --> 0:14:11.319 I mean, and this is how they won kind of 0:14:11.360 --> 0:14:16.480 the social media wars, right is that you know, I 0:14:16.480 --> 0:14:18.040 think a lot of the people who are critics of 0:14:18.080 --> 0:14:21.920 the company would not have built the company, you know 0:14:22.000 --> 0:14:24.040 that was nearly as successful as the one that Mark 0:14:24.120 --> 0:14:27.040 zuckerbelg did. Because if you're constantly wondering, well, what are 0:14:27.080 --> 0:14:29.280 the effects of this? Can we think of any potential 0:14:29.320 --> 0:14:33.720 downstream harms? What safety measures for you building in that's 0:14:33.880 --> 0:14:37.840 kind of a loser mentality. So like a lot of 0:14:37.880 --> 0:14:41.840 the company's history was basically, if someone could demonstrate that 0:14:42.080 --> 0:14:45.240 some tweak to the algorithm or some change in a 0:14:45.240 --> 0:14:49.760 product feature would cause metrics to rise, they wouldn't look 0:14:49.800 --> 0:14:52.680 too deeply under the hood. They just launched the thing 0:14:52.920 --> 0:14:56.320 and move on to the next thing that could be optimized. 0:14:57.040 --> 0:14:59.320 And I don't think that's just meta right, Like it 0:14:59.440 --> 0:15:02.320 kind of has this gold rush field. And as a 0:15:02.360 --> 0:15:07.040 lot of the scientists who study sort of parasocial relationships 0:15:07.080 --> 0:15:11.080 and digital relationships are very quick to admit, we just 0:15:11.200 --> 0:15:14.880 do not have the data to even remotely say what 0:15:14.920 --> 0:15:15.840 the effects. 0:15:15.440 --> 0:15:17.280 Of this stuff is going to be in terms of. 0:15:17.360 --> 0:15:23.760 People's relationship with reality, that relationship to human companions. But again, 0:15:24.000 --> 0:15:27.160 I don't know that one would expect companies to produce 0:15:28.040 --> 0:15:34.600 carefully designed, responsibly built products at a time when every 0:15:34.640 --> 0:15:36.880 incentive is to gun the engines. 0:15:41.960 --> 0:15:45.400 Well, Jeff, thank you so much for talking to me today. Certainly, 0:15:47.160 --> 0:15:50.600 Jeff Howitz is the author of Broken Code Inside Facebook 0:15:50.640 --> 0:15:52.200 and the Fight to expose. 0:15:51.800 --> 0:15:52.920 Its harmful secrets. 0:15:53.520 --> 0:16:07.440 You can read his investigation in Taboo at Reuters dot com. 0:16:07.520 --> 0:16:10.880 Also in the News Treasurer dim Chalmers has said states 0:16:10.880 --> 0:16:13.960 should back the new Thriving Kids program, designed to divert 0:16:14.040 --> 0:16:17.680 children with autism away from the NDIS or risk losing 0:16:17.720 --> 0:16:21.040 hospital funding. The federal government is yet to renew the 0:16:21.080 --> 0:16:24.360 five year funding agreements for state and territory hospital systems, 0:16:24.800 --> 0:16:27.240 instead settling on a one year extension. In the lead 0:16:27.320 --> 0:16:30.840 up to the federal election last week, some state ministers 0:16:30.840 --> 0:16:34.000 said they weren't committed to Thriving Kids funding. Meanwhile, they've 0:16:34.040 --> 0:16:37.000 been arguing for a large increase in federal hospital funding. 0:16:37.680 --> 0:16:40.600 Jim Chalmers says the deals are closely related and one 0:16:40.640 --> 0:16:45.760 deal can't progress without the other, and Australia's building code 0:16:45.800 --> 0:16:48.960 is unusable for many builders. The federal Housing Minister, Claire 0:16:48.960 --> 0:16:52.600 O'Neill says. Over the weekend the government announced plans to 0:16:52.680 --> 0:16:55.800 pause changes to the National Construction Code and to speed 0:16:55.880 --> 0:16:58.320 up the assessment process for more than twenty six. 0:16:58.200 --> 0:16:59.400 Thousand new homes. 0:17:00.400 --> 0:17:04.400 The announcement came following last week's economic reform roundtable, where 0:17:04.400 --> 0:17:05.440 housing was a focus. 0:17:06.240 --> 0:17:08.960 I'm Ruby Jones. This is seven am. See you to morrow.