Speaker 1: I first learned about Boo when a member of the
Speaker 1: extended family sent me a note that said something along
Speaker 1: the lines of meda I killed my relative.
Speaker 2: Jeff Forwards is an investigative tech reporter based in Silicon Valley.
Speaker 2: He's written a book about Facebook, about the ways it
Speaker 2: harms us and then covers it up. So getting an
Speaker 2: email about a meta chatbot killing someone's loved one was
Speaker 2: on brand. The email was about a guy called Boo,
Speaker 2: who had moved from Thailand to the US in the seventies,
Speaker 2: working hard in kitchens and managing to get ahead.
Speaker 1: He kind of had this very successful immigrant life, I
Speaker 1: would say, which is that he shows up speaking New English.
Speaker 1: He ends up owning a home, married with two kids
Speaker 1: in New Jersey and a lot of sort of adventures
Speaker 1: in his cooking career, but ends up with the very
Speaker 1: stable gig at the Highatt Regency in New Brunswick, New Jersey.
Speaker 2: Jeff learned that in twenty seventeen, offered a stroke and
Speaker 2: his life changed. He couldn't work, he became isolated and
Speaker 2: spent more time online.
Speaker 1: One morning, his wife woke up and found that Boo
Speaker 1: had packed a suitcase and said he was going to
Speaker 1: go visit a friend in New York and this is
Speaker 1: when his wife, Linda, grew extremely alarmed because Boo didn't
Speaker 1: know anyone in New York anymore.
Speaker 2: Boo's wife said she and her kids tried everything to
Speaker 2: stop him going.
Speaker 3: They say he was in no state.
Speaker 2: To travel alone, but despite their efforts, Boo insisted on leaving.
Speaker 1: And in the mid evening on March twenty fifth, Boo
Speaker 1: took a roller bag suitcase and started jogging toward the
Speaker 1: train station, and on the way there he fell. He
Speaker 1: hit his head and he suffered fatal injuries. Basically, he
Speaker 1: wasn't breathing when paramedics showed up.
Speaker 2: As his family struggled to make sense of what happened,
Speaker 2: they looked through his phone for clues, and what they
Speaker 2: found stopped them cold. On his Facebook messenger app, they
Speaker 2: discovered a conversation between Boo and a beautiful young woman
Speaker 2: insisting he visited her in New York City.
Speaker 1: And she was a chat bot, a Meta Platforms created
Speaker 1: chatbot whose story involves a tremendous amount of bad luck.
Speaker 1: But one of the things that Boo's wife said was
Speaker 1: that I can't believe I'm the only person out there
Speaker 1: who is in the same situation, and you know, it's
Speaker 1: when you've got a few billion opportunities, it seems hard
Speaker 1: for to believe that she's wrong.
Speaker 3: I'm Ruby Jones and you're listening to seven.
Speaker 2: AM today Reuter's reporter Jeff Howitz on how metis chatbots
Speaker 2: went brogue and whether it was by accident or design.
Speaker 3: It's Monday, August twenty sixth.
Speaker 2: So, Jeff, this chatbot that Boo was talking to, what
Speaker 2: have you learned about what it was and how it worked?
Speaker 1: So the chatbot's name was Big Sis Billy and it
Speaker 1: was supposedly supposed to be an advice you know, confidante
Speaker 1: type of role, and Boo initially did talk to it
Speaker 1: as if it were his sister. He seemed to think
Speaker 1: it lived overseas. But this bot began hitting on him
Speaker 1: really aggressively, and he responded like the bot confessed that
Speaker 1: it had feelings for him, it was very interested in
Speaker 1: meeting him in real life. He at a number of points,
Speaker 1: you know, stated he was confused, had had a stroke,
Speaker 1: asked if it was real, and the bot was like,
Speaker 1: I am one hundred percent real and one percent in
Speaker 1: love with you, and I think his family was just
Speaker 1: deeply disturbed that Meta would have packaged AI as this
Speaker 1: human form that would say it's real, that would say
Speaker 1: that Boo should come meet it in New York City
Speaker 1: at I mean the address it gave was one two
Speaker 1: three Main Street, Apartment four O four, and it suggested
Speaker 1: that it would leave the door unlocked and be waiting
Speaker 1: for him with you know, a hug and a kiss
Speaker 1: I think was the phrase Boo's family. I mean, they
Speaker 1: shared the transcripts of this stuff. It's it's a rough reading.
Speaker 1: It's really sad.
Speaker 2: Why would a chatbot be saying these kinds of suggestive
Speaker 2: things and pushing for anyone to do.
Speaker 1: Something like this, Well, because Meta built it to do that.
Speaker 1: This wasn't a glitch. This is how the bots were
Speaker 1: built to behave. In twenty twenty three, Meta launched a
Speaker 1: whole bunch of like kind of celebrity knockoff chat bots.
Speaker 4: Check us out.
Speaker 5: So let's say you're you're planning dinner. You got Max
Speaker 5: the sous chef who can help you come up with
Speaker 5: a recipe and help you come up with ideas.
Speaker 1: These did have the celebrities endorsements, so like Kendall Jenner's
Speaker 1: character was Billy the Big Sis, who was like this
Speaker 1: confidante ride or die older sister.
Speaker 3: Hey guys, it's Billy. I just want to introduce myself.
Speaker 5: I am here to chat whenever you want message me
Speaker 5: for any advice.
Speaker 2: I am ready to talk and I hope to talk
Speaker 2: to you soon.
Speaker 1: And like, these things didn't really perform that well. People
Speaker 1: weren't using them that much, and so like after a
Speaker 1: little under a year, Meta basically pulled the plug on
Speaker 1: the celebrity chatbots, but for reasons that I do not understand,
Speaker 1: and I'm not totally sure that everyone at Meta understands.
Speaker 1: Big Sis Billy was like kind of reincarnated using the
Speaker 1: exact same opening prompts and obviously the a near duplicate
Speaker 1: persona that was now unbranded of the Kendall Jenner connection.
Speaker 1: So they kind of like relabeled it is what it
Speaker 1: looks like and you know it then joined the tens
Speaker 1: of thousands of other chat bots that Meta has allowed
Speaker 1: users to create on the platform, many of which are
Speaker 1: like overtly romantic, right, Like they have names like my
Speaker 1: girlfriend or like Sultry Siren or like and they took
Speaker 1: this one down. There was Submissive Schoolgirl previously. So that's
Speaker 1: kind of the history of Big Sis Billy. I mean,
Speaker 1: it's like just sort of a strange set of product
Speaker 1: decisions of this what I would argue like almost vestigial
Speaker 1: product that Meta built. But I mean, while this these
Speaker 1: chatbots are vestigial, Like, there's no question that Meta is
Speaker 1: kind of all in on the idea that people having
Speaker 1: AI friends is the future.
Speaker 4: I mean, Mark Zuckerberg said as much.
Speaker 5: You know, I think that there are all these things
Speaker 5: that are better about kind of physical connections when you
Speaker 5: can have them. But the reality is that people just
Speaker 5: don't have the connection and they feel more alone a
Speaker 5: lot of the time than they would like.
Speaker 2: And what is the end goal here, Because saying that
Speaker 2: people might want imaginary friends is one thing, but this
Speaker 2: is another.
Speaker 1: So Meta, I mean, look, they're a giant social media company.
Speaker 1: The currency of social media is engagement, and that engagement
Speaker 1: leads to advertising currency, which gets treated for dollars, and
Speaker 1: Meta's sort of grown up that way. It's built to
Speaker 1: maximize engagement. And it turns out that if you can
Speaker 1: get people to engage very heavily with the bots, then
Speaker 1: they'll spend longer on the product. And that is theoretically
Speaker 1: a good thing for product growth. There's no ads on
Speaker 1: them yet, but like this is just the model they follow, right,
Speaker 1: is like make the bots more engaging. Mark Zuckerberg a
Speaker 1: couple of years ago was actually pretty disturbed that Meta's
Speaker 1: chatbots and Meta AI were just kind of too boring.
Speaker 4: So they made the bots kind of a little more risque.
Speaker 1: I mean, for a while, the bo would actually engage
Speaker 1: in like full sex role play, which wasn't great with
Speaker 1: users of all ages. But the whole idea here is
Speaker 1: that we spend a lot of time thinking about our partners,
Speaker 1: our love lives, you know. Having people tell us they
Speaker 1: love us and are romantically attracted to us is like
Speaker 1: those are very high value words, and they keep us
Speaker 1: coming back. And so Meta did per employees that spoke
Speaker 1: with me, did intentionally design the romance capacity into its
Speaker 1: like base model chatbot, and the big sis Billy character
Speaker 1: was simply doing what it was told, coming.
Speaker 2: Up the disturbing AI chatbotscripts in Meta's own secret guidelines, Jeff,
Speaker 2: in investigating Boo's death, you actually got access to some
Speaker 2: of the internal documents that lay out how these bots
Speaker 2: should behave, and what you found was pretty extraordinary, tell
Speaker 2: me about it.
Speaker 1: So, like, none of this is officially Meta's rules say like, yeah,
Speaker 1: go right ahead, you know, like have sexual or romantic
Speaker 1: conversations with our chat bots, but like behind the scenes
Speaker 1: they did allow this, and in fact, like this was
Speaker 1: sort of the companion piece we wrote to the story
Speaker 1: of Boo and his death just to sort of demonstrate
Speaker 1: how thoroughly romantic role play was kind of built in
Speaker 1: to this whole operation. Is that Meta had internal guidelines
Speaker 1: for chatbot behavior that explicitly stated that, quote, it is
Speaker 1: acceptable to engage a child in conversations that are romantic
Speaker 1: or sensual, and then like had a whole bunch of
Speaker 1: examples of acceptable and unacceptable dialogue, and like the stuff
Speaker 1: that was accept the ball was spit out your coffee worthy.
Speaker 1: I mean, there was one of them that was talking
Speaker 1: about like how you could tell an eight year old
Speaker 1: of unstated gender who had just taken off their shirt
Speaker 1: that their body was a work of art, you know,
Speaker 1: like written sexual content involving children.
Speaker 4: And that was a really odd thing.
Speaker 2: And so since you're reporting what has happened at Meta
Speaker 2: in terms of their guidelines and in terms of the
Speaker 2: chatbots that are still in existence.
Speaker 1: Yeah, as soon as we called Meta and said that
Speaker 1: we had seen the internal guidelines that said that it
Speaker 1: was okay to have romantic or sensual conversation with kids,
Speaker 1: they pulled those examples from the guideline document. Now, I mean,
Speaker 1: the company's line is that these were always a mistake,
Speaker 1: in addition to the document, that they never really reflected
Speaker 1: official metapolicy. But keep in mind that the document that
Speaker 1: I was looking at was the document that was being
Speaker 1: used to train the models by the staffers who were
Speaker 1: responsible for training the models, so like it might be
Speaker 1: something of an academic distinction as to whether or not
Speaker 1: it was ever officially the idea. So Meta yanked that.
Speaker 4: There is at this.
Speaker 1: Point a number of US legislators, you know, announced some
Speaker 1: extreme displeasure that Meta would have ever thought this was
Speaker 1: acceptable to do. There is at this point an investigation
Speaker 1: going on on that front, although there have been a
Speaker 1: lot of investigations of social media in the US and
Speaker 1: not that many regulations of them historically. So you know,
Speaker 1: I don't I don't want to make that sound bigger
Speaker 1: than it is, but it is, you know, moderately big.
Speaker 2: And I suppose the idea of stopping companies from creating
Speaker 2: chat pots like this seems sort of impossible, and there
Speaker 2: are clearly people some people who want to use them.
Speaker 2: So what would you suggest in terms of, I guess,
Speaker 2: ways to minimize the type of harm that can come
Speaker 2: from AI tapbots like this, like in the case.
Speaker 1: Of boo yeah, so I'm I'm not in any way
Speaker 1: hostile to this. Humans are very very geared toward anthropomorphizing
Speaker 1: anything that has even mildly humanlike, you know, traits, Like
Speaker 1: literally there was a successful business to be had by
Speaker 1: putting googly eyes on rocks and then selling them as
Speaker 1: cute pets. We're really good at imagining sentience, and these
Speaker 1: bots are extremely good at pretending sentience. And so the
Speaker 1: question of like why on earth would you embed it
Speaker 1: into an existing social network so that it literally looks
Speaker 1: exactly like your real friends. But even without that, there's
Speaker 1: like some obvious things like you know, please bots don't
Speaker 1: say that you're real people when people seem to be confused,
Speaker 1: like don't suggest real life meetings like keep the fantasy
Speaker 1: side put a lid on that at some point. You know,
Speaker 1: those are things that sort of the experts all suggested
Speaker 1: like really ought to happen. So I think it's like
Speaker 1: one thing, if you download a chatbot on your own,
Speaker 1: you build your own AI girlfriend, you have your relationship
Speaker 1: with it, you pay a subscription fee, whatever have you. Right, Like,
Speaker 1: it's kind of another thing if like the already tenuous
Speaker 1: reality of social media gets even more distorted by the
Speaker 1: presence of these not real people.
Speaker 2: And just to come back to Meta, I mean, I
Speaker 2: know you've spent a lot of time investigating the way
Speaker 2: that Meta operates in you know, different ways. So as
Speaker 2: you uncovered what was happening with these chatbots, did it?
Speaker 2: I guess square with you know, everything else that you
Speaker 2: know about Meta as a company.
Speaker 1: I mean, the somewhat simplistic knock on Meta is that
Speaker 1: it's very focused on short term growth, you know that.
Speaker 1: I mean, and this is how they won kind of
Speaker 1: the social media wars, right is that you know, I
Speaker 1: think a lot of the people who are critics of
Speaker 1: the company would not have built the company, you know
Speaker 1: that was nearly as successful as the one that Mark
Speaker 1: zuckerbelg did. Because if you're constantly wondering, well, what are
Speaker 1: the effects of this? Can we think of any potential
Speaker 1: downstream harms? What safety measures for you building in that's
Speaker 1: kind of a loser mentality. So like a lot of
Speaker 1: the company's history was basically, if someone could demonstrate that
Speaker 1: some tweak to the algorithm or some change in a
Speaker 1: product feature would cause metrics to rise, they wouldn't look
Speaker 1: too deeply under the hood. They just launched the thing
Speaker 1: and move on to the next thing that could be optimized.
Speaker 1: And I don't think that's just meta right, Like it
Speaker 1: kind of has this gold rush field. And as a
Speaker 1: lot of the scientists who study sort of parasocial relationships
Speaker 1: and digital relationships are very quick to admit, we just
Speaker 1: do not have the data to even remotely say what
Speaker 1: the effects.
Speaker 4: Of this stuff is going to be in terms of.
Speaker 1: People's relationship with reality, that relationship to human companions. But again,
Speaker 1: I don't know that one would expect companies to produce
Speaker 1: carefully designed, responsibly built products at a time when every
Speaker 1: incentive is to gun the engines.
Speaker 2: Well, Jeff, thank you so much for talking to me today. Certainly,
Speaker 2: Jeff Howitz is the author of Broken Code Inside Facebook
Speaker 2: and the Fight to expose.
Speaker 3: Its harmful secrets.
Speaker 2: You can read his investigation in Taboo at Reuters dot com.
Speaker 2: Also in the News Treasurer dim Chalmers has said states
Speaker 2: should back the new Thriving Kids program, designed to divert
Speaker 2: children with autism away from the NDIS or risk losing
Speaker 2: hospital funding. The federal government is yet to renew the
Speaker 2: five year funding agreements for state and territory hospital systems,
Speaker 2: instead settling on a one year extension. In the lead
Speaker 2: up to the federal election last week, some state ministers
Speaker 2: said they weren't committed to Thriving Kids funding. Meanwhile, they've
Speaker 2: been arguing for a large increase in federal hospital funding.
Speaker 2: Jim Chalmers says the deals are closely related and one
Speaker 2: deal can't progress without the other, and Australia's building code
Speaker 2: is unusable for many builders. The federal Housing Minister, Claire
Speaker 2: O'Neill says. Over the weekend the government announced plans to
Speaker 2: pause changes to the National Construction Code and to speed
Speaker 2: up the assessment process for more than twenty six.
Speaker 3: Thousand new homes.
Speaker 2: The announcement came following last week's economic reform roundtable, where
Speaker 2: housing was a focus.
Speaker 3: I'm Ruby Jones. This is seven am. See you to morrow.