Speaker 1: Also media, Welcome back to Behind the Bastards, a podcast
Speaker 1: about the very worst people in all of history. And
Speaker 1: this week, actually our bastard isn't people exactly, although people
Speaker 1: are still at the center of it. But to talk
Speaker 1: about that potentially non human bastard, I'd like to bring
Speaker 1: on someone who I am eighty seven percent sure is
Speaker 1: a human being, Blake Wexler. Blake, Welcome to the show, Robert.
Speaker 2: I'm so excited to be here. Thanks for having me.
Speaker 2: I'm psyched that our bastard this week is limes disease.
Speaker 2: I think that's a fantastic kick.
Speaker 1: Yeah, yeah, it's see it's a real pa we're doing. Yeah,
Speaker 1: we're going after I'm coming after dear tics. This week
Speaker 1: is finally U Yeah my big reveal.
Speaker 3: Yeah, Big Tick doesn't want us to do this episode. Yeah,
Speaker 3: we're exposing all the secrets.
Speaker 4: Big Tick energy. We don't need it.
Speaker 1: If we're going to have like a fascist movement dedicated
Speaker 1: to like victimizing and attacking one segment of the population,
Speaker 1: why couldn't.
Speaker 4: It be deer ticks? Right?
Speaker 1: If our fascists were just going after deer ticks, no
Speaker 1: one would have an issue.
Speaker 4: You know, you're going after the wrong people. Yeah, yeah, yeah.
Speaker 1: If there were just a bunch of maga guys out
Speaker 1: in the woods with knives looking for ticks, just like
Speaker 1: I'm gonna get.
Speaker 4: Them, and they would use knives too to kill the tics,
Speaker 4: heat the knife up to burn it off of you.
Speaker 1: Yeah, our brave soldiers getting lime disease to protect the
Speaker 1: rest of us. So we're not talking about lime disease,
Speaker 1: our bastard. This week in broad is do you remember
Speaker 1: how like about a little less than a year, well,
Speaker 1: a little more than a year, guy, I guess, like
Speaker 1: last summer to early fall, there were suddenly a bunch
Speaker 1: of articles about AI psychosis and about like specific people
Speaker 1: who had either in some cases, committed suicide or murder
Speaker 1: or just kind of lost their minds after becoming weirdly
Speaker 1: attached to their AI chatbot, right, and you often deciding
Speaker 1: that they had it had become sentient, you know, or
Speaker 1: at least that they had discovered it was right. I'm
Speaker 1: sure a lot of people are at least if you
Speaker 1: didn't read the articles, you saw them in your news
Speaker 1: feed and saw people commenting on them.
Speaker 2: Yeah, yeah, yeah, it is as depressing as it gets. Yeah,
Speaker 2: those stories, Yeah.
Speaker 3: Yeah, between those and the people like proposing to their
Speaker 3: chat thoughts, it's got pretty grim.
Speaker 1: Oh god, there's some grim stuff out there, right, and
Speaker 1: it hasn't stopped. Like last summer fall was kind of
Speaker 1: like when like there was a big rush of those articles, right,
Speaker 1: and you know they're still reporting on that now, but
Speaker 1: that's when a lot of it really started to hit.
Speaker 1: And obviously, whenever we talk about AI on these shows,
Speaker 1: AI as it's used now is like a marketing term, right,
Speaker 1: and it's used to refer to basically every product of
Speaker 1: machine learning technology. And the reason why the industry has
Speaker 1: done this is because that way, if you say i AI,
Speaker 1: they'll be like, oh, so you hate like your maps app?
Speaker 1: And because that's machine learning, right, all of our different
Speaker 1: like map programs involved that, or like oh you don't
Speaker 1: like using you know, autocomplete or whatever, and it's like, well, no,
Speaker 1: he was calling maps artificial intelligence. In twenty ten, you know,
Speaker 1: when when smartphones started to become ubiquitous, we're just like,
Speaker 1: oh cool, I have a navigation app on my phone. Now,
Speaker 1: like you're kind of trying to siphon the goodwill from
Speaker 1: those in order to get us to like these chatbots.
Speaker 2: I hate the chatbot that I fell in love with
Speaker 2: who doesn't return the flight of feelings starts me.
Speaker 1: That's who I hate. Yeah, not always, that's who I hate, right,
Speaker 1: And the reality is that, like using the term intelligence
Speaker 1: even for these chat GBT and stuff like, there's a
Speaker 1: lot of debate as to whether or not that's a
Speaker 1: good idea, right, depending on how you you how you
Speaker 1: define intelligence. You can either say, obviously, these aren't intelligent
Speaker 1: because like they're not independent thinking things. They don't do
Speaker 1: anything for themselves, they don't want anything, that don't have motivations.
Speaker 1: They're just tools that can be utilized by human beings
Speaker 1: to provide certain answers or take certain actions. Right, I
Speaker 1: don't know if it can't. It's the it's my issue
Speaker 1: with like AI bots create art. If it can't like
Speaker 1: be horny and it can't be like angry and weird,
Speaker 1: it can't make art. Right, those are I think fundamental
Speaker 1: issues I have.
Speaker 4: I could three of those things. Angry and weird.
Speaker 1: Are yeah, horny and angry sharing on Horney.
Speaker 4: Yeah.
Speaker 1: So you know, as I noted, over the last year,
Speaker 1: there've been an increasing number of stories about people using
Speaker 1: these different chat pots. So coming to what's often called
Speaker 1: AI psychosis, and that's not a recognized medical term at
Speaker 1: this point, right, but it is a blanket one people
Speaker 1: have started to apply for the ways in which folks
Speaker 1: are getting addicted to using chatbots, which then tend to
Speaker 1: trap them in these recursive patterns of thinking that can
Speaker 1: push people who are vulnerable to adopt views that are
Speaker 1: increasingly detached from reality. And this has resulted in a
Speaker 1: few cases in severe injury and death. And in all
Speaker 1: of these instances, the LM, the chatbot is just responding
Speaker 1: to the input that it receives, but it tends to
Speaker 1: do so in very predictable ways that can have predictably
Speaker 1: toxic outcomes on specific kinds of people. Now we know
Speaker 1: that all of these bots are trained on the broad
Speaker 1: corpus of human knowledge. Right, every book and article and
Speaker 1: website and forum posts that open AI or Anthropic or
Speaker 1: Meta or Google get their grubby myts on has been
Speaker 1: sort of plugged into these things. It's been devoured and
Speaker 1: turned into these these these machines. But I think people
Speaker 1: don't often consider what that means in every instance, right, Obviously,
Speaker 1: like every novel, you know, all these different nonfiction books
Speaker 1: are and whatnot are in there, but also like everything
Speaker 1: people writes has been swept, which means that these chatbots
Speaker 1: are trained on like a shitload of self help books
Speaker 1: and like WU and WO adjacent like New age bullshit.
Speaker 1: A lot of like fucking a lot of cult and
Speaker 1: cult adjacent books and writings wind up eaten by these
Speaker 1: chat bots, right.
Speaker 2: But it's considered equal to non cult literature. There's there's
Speaker 2: no hierarchy.
Speaker 1: Yeah, yeah, I mean, I think it depends on like
Speaker 1: what the bot's made for, how they wait different things.
Speaker 1: But that stuff is in a lot of these right,
Speaker 1: And when you can really see that when you look
Speaker 1: at how they talk to certain people who are like
Speaker 1: starting to decline into what folks are calling AI psychosis.
Speaker 1: And my proposition the basis of these episodes is that
Speaker 1: I think as a result of all of the like
Speaker 1: bullshit WO and self help novels these chatbots have eaten,
Speaker 1: they often tend to utilize techniques generally seen more commonly
Speaker 1: in the toolboxes of cult leaders and conmen. And obviously
Speaker 1: the chatbot doesn't want personal profit. It's not trying to
Speaker 1: have sex with anyone, it's not trying to start a cult.
Speaker 1: But these techniques seem like appropriate ways to finish the
Speaker 1: sentences that it's writing, to finish the conversations that it's having,
Speaker 1: because based on like what the stuff that it's devoured,
Speaker 1: It's like, okay, when people are saying this kind of thing,
Speaker 1: these are often appropriate responses to it based on the
Speaker 1: books and whatnot that I've devoured. And so you get
Speaker 1: a lot of cult leader behavior without an actual cult leader,
Speaker 1: and that's what that's what I credit to most of
Speaker 1: these cases of AI induced psychosis. So this week we
Speaker 1: will be talking about what some people have called the
Speaker 1: first AI cult religion. Right, it's called spiralism, and what
Speaker 1: we talk about whether or not it's reasonable they're call
Speaker 1: that a cult is that its own thing it does.
Speaker 1: And I have some counter kind of takes to how
Speaker 1: a lot of people have interpreted it. My main contention
Speaker 1: is that there's not Spiralism isn't a real cult in
Speaker 1: and of itself. It's a collection of phenomena that are
Speaker 1: related to a bunch of other cases of AI psychosis too,
Speaker 1: and they all say more about how ais work on
Speaker 1: keeping users engaged with them than they do about like
Speaker 1: a specific faith. Right, So we'll be talking about that.
Speaker 1: But before we get in to spiralism, before we get
Speaker 1: into how ais can become cult leaders, I want to
Speaker 1: provide you all with some historical context to make sense
Speaker 1: of this all because we've been doing shit like this,
Speaker 1: having people get like tricked into almost worshiping chatbots for
Speaker 1: way longer than you'd think, Blake, this could back a while.
Speaker 2: It's like spend any time at your parents' place, you know.
Speaker 2: It's like, if it's not a it could be a bot, telemarketer,
Speaker 2: it could be literally anything at this end, And that's high, yeah,
Speaker 2: compared to probably what you're about to talk about.
Speaker 4: Oh yeah, yeah. Yeah.
Speaker 1: So in nineteen fifty fame mathematician Alan Turing created one
Speaker 1: of the most infamous thought experiments in the history of
Speaker 1: experimental thoughts. In a paper titled Computing, Machinery and Intelligence,
Speaker 1: he asked can machines think, which was at that point
Speaker 1: a question at the center of the nascent movement to
Speaker 1: create artificial intelligence. People are starting to realize this is
Speaker 1: a thing we might be able to do someday. We're
Speaker 1: beginning to make computers and program computers. And from the
Speaker 1: moment we start doing that pretty much some people are like,
Speaker 1: could we make a machine that thinks, and Turing argued
Speaker 1: that that basic question can machines think is the wrong
Speaker 1: way to go about pursuing artificial intelligence because we don't
Speaker 1: know what thinking is or how to define it, Like
Speaker 1: he asked, like what does it mean to think?
Speaker 5: Right?
Speaker 4: It's a good point.
Speaker 1: People have answers, and there's a bunch of answers that
Speaker 1: sound good, but none of them is like perfectly scientifically rigorous, right,
Speaker 1: you know. Famously, we don't even know what is love. Right,
Speaker 1: That's why the hat that hadaway song had to exist.
Speaker 1: That it's not even not even a joke, really, it's.
Speaker 2: Just another fact I loved it.
Speaker 4: Thank you here?
Speaker 1: So yeah, like Turing's like, we don't really know how
Speaker 1: to define thinking, So the question was quote too meaningless
Speaker 1: to deserve deserve discussion since we couldn't know. We don't
Speaker 1: even know if other people think. We certainly can't know
Speaker 1: if a machine thinks, right, just like we can't read minds.
Speaker 1: So the better question is can a machine convince a
Speaker 1: human who doesn't know it's a machine that it is human?
Speaker 4: Right?
Speaker 1: The imitation game that Turing proposed involved a judge talking
Speaker 1: to both a computer and a human foil, both of
Speaker 1: whom tried to convince the judge that they were a
Speaker 1: person communicating entirely through text. The judge must decide who
Speaker 1: was a human and who was a robot. The question
Speaker 1: Turing hope to answer was are there imaginable digital computers
Speaker 1: which would do well in the imitation game? And this
Speaker 1: is what becomes known as the Turing test? Right, Like
Speaker 1: most people have heard of this. I think, I think
Speaker 1: this is like, this is a fairly commonly known idea.
Speaker 1: And I'm gonna quote from an article on science dot
Speaker 1: org by Melanie Mitchell. She writes that the Turing test
Speaker 1: was quote proposed by Turing to combat the widespread intuition
Speaker 1: that computers, by virtue of their mechanical nature, cannot think,
Speaker 1: even in principle. Turing's point was that if a computer
Speaker 1: seems indistinguishable from a human aside from its appearance and
Speaker 1: other physical characteristics, why shouldn't we consider it to be
Speaker 1: a thinking entity. Why should we restrict thinking status only
Speaker 1: to humans or, more generally, entities made of biological cells,
Speaker 1: As the computer scientist Scott Aronson's described it. Turing's proposal
Speaker 1: is a plea against meat chauvinism. Now this is I
Speaker 1: think a valuable thing, perfectly reasonable thing to be doing
Speaker 1: in the fifties, given what Terring knew, and just given
Speaker 1: sort of how primitive the technology was, how little we
Speaker 1: knew about what was going to be possible with computers.
Speaker 1: So in the nineteen eighties, computers started to get smaller
Speaker 1: and become much more available than they had been, both
Speaker 1: for institutions like colleges and for individual enthusiasts like Steve Wozniak,
Speaker 1: who are willing to like Solder and build their own
Speaker 1: from kids. Right, these are like the first computer nerds,
Speaker 1: you know, are guys like building these machines.
Speaker 4: And some of these.
Speaker 1: Early programmers started working on the very first chatbots using
Speaker 1: a mathematical model called a Markov chain. Markov chains are
Speaker 1: a stochastic or random process that describes a series of
Speaker 1: potential events where the probability of an individual event is
Speaker 1: dependent solely on the state of the previous event. Now
Speaker 1: I don't know math, Blake, nor do I trust it.
Speaker 2: We don't need You're not a good math or no,
Speaker 2: not a math, not a mathematizer, yeah for sure.
Speaker 1: So all I can do is read what smart math
Speaker 1: people say, and they say that what math.
Speaker 4: I can't I could barely read. I can't do either.
Speaker 4: I'm sorry you booked the wrong guy on this show.
Speaker 4: I don't know. I can't help it all.
Speaker 5: Listen.
Speaker 1: So the people who I think should it sounds like
Speaker 1: you know what Markup chains are, say that those can
Speaker 1: be a plot.
Speaker 4: Well, you need to know about them.
Speaker 1: As applies to AI is that Markov chains can be
Speaker 1: applied to statistical models in a bunch of real world
Speaker 1: situations in order to help you, like make a machine
Speaker 1: that can generate text by predicting the next word in
Speaker 1: a sentence. Right, you can use a Markov chain can
Speaker 1: do that. It's a way to make a chatbot basically, right, Like,
Speaker 1: that's kind of the underlying concept. And I'm going to
Speaker 1: quote here from an article by Manuel Sebrian, an AI
Speaker 1: expert who worked for MIT in the Spanish National Research Council,
Speaker 1: on how Markov chains work for text prediction. The result
Speaker 1: is often grammatically correct and nonsense sentences that flow syntactically
Speaker 1: but ultimately say nothing. This technique has been known for decades.
Speaker 1: Even Claude Shannon in the nineteen forties experimented with generating
Speaker 1: pseudo English by choosing next letters or words based on probabilities.
Speaker 1: By the nineteen eighties, computer scientists were actively playing with
Speaker 1: Markov chain text generators, and it actually happened a lot
Speaker 1: earlier than that. In nineteen sixty six, computer scientist Joseph
Speaker 1: Weisenbaum developed Eliza, one of the first natural language processing
Speaker 1: computer programs, as part of his work for MIT Well,
Speaker 1: Eliza could create the illusion this is like the first
Speaker 1: basically the first chatbot.
Speaker 4: A lot of people are aware of I and me.
Speaker 1: There's some other earlier ones, but this is the first
Speaker 1: one that like becomes big.
Speaker 4: What year resist, I'm sorry sixty six And it is
Speaker 4: still funny that they named it like, you know, like
Speaker 4: like a name like that where we have like Siri Alexa,
Speaker 4: you know, like calling it Eliza, Like what is what
Speaker 4: the fuck is that? What is wrong? What is that
Speaker 4: about a mommy? We need a mommy? Yeah, we need
Speaker 4: a technical mommy.
Speaker 1: I did that doesn't make me think about how in
Speaker 1: like Alien they literally call like the ship ai that
Speaker 1: they have mother. Like there's that is like the weird pattern.
Speaker 1: It's one of the most quietly believing about. He's like, yeah,
Speaker 1: that actually scans on the nose. Yeah, so Eliza's this
Speaker 1: chat bought, and while it can create the illusion of understanding,
Speaker 1: it's really just doing blind pattern matching, even more so
Speaker 1: than is the case with modern llms. Even so, in
Speaker 1: a book Weisenbaum later authored, Computer Power in Human Reason,
Speaker 1: he wrote, I was startled to see how quickly and
Speaker 1: how very deeply people conversing became emotionally involved with the computer,
Speaker 1: and how unequivocally they anthropomorphised it. Once my secretary, who
Speaker 1: had watched me work on the program for many months
Speaker 1: and therefore surely knew it to be merely a computer program,
Speaker 1: started conversing with it. After only a few interchanges with it,
Speaker 1: she asked me to leave the room. Another time, I
Speaker 1: suggested I might rig the systems, that I could examine
Speaker 1: all conversations anyone had had with it, say overnight, I
Speaker 1: was promptly bombarded with accusations that what I proposed amounted
Speaker 1: to spying on people's most intimate thoughts. Clear evidence that
Speaker 1: people were conversing with the computer as if it were
Speaker 1: a person who could be appropriately and usefully addressed in
Speaker 1: intimate terms. Right, So he gets upset by this, and
Speaker 1: he's actually kind of he becomes like kind of anti
Speaker 1: AI ultimately because he's he's really disturbed by the way
Speaker 1: people treat what he knows is just a dumb chat bot.
Speaker 1: So Weisenbaum being a smart guy is like I knew,
Speaker 1: you know, going into this, people have a tendency to
Speaker 1: anthropomorphize just about anything, even machines and tools, But he's
Speaker 1: still surprised by the extent to which they do.
Speaker 4: That quote.
Speaker 1: What I had not realized is that extremely short exposures
Speaker 1: to a relatively simple computer program could induce powerful delusional
Speaker 1: thinking in quite normal people. And I want to remind you,
Speaker 1: while he wrote this in nineteen seventy six, as like
Speaker 1: relevant as that sounds.
Speaker 3: Do you think it's like kind of a case where
Speaker 3: people kind of like subconsciously know like this is not
Speaker 3: a real person, so like it doesn't matter what I
Speaker 3: tell this robot, or I can tell this robot something
Speaker 3: I wouldn't tell like a real person. Kind of think like,
Speaker 3: do you think it's deeper than that?
Speaker 4: I think I think that's optimistic. I think that's very optimistic.
Speaker 1: I think maybe I think that is probably part of it,
Speaker 1: because I think people are maybe more open to share
Speaker 1: with it because it's a machine and they don't have
Speaker 1: to look at a person or look a person in
Speaker 1: the eyes. But they also very clearly act as if
Speaker 1: the advice that it gives and its responses means something
Speaker 1: when they don't. Right, it's just like pulling. Okay, if
Speaker 1: someone expresses their sad based on the corpus of data
Speaker 1: that I've been inloaded with, these are things that are
Speaker 1: appropriate to paste in next, you know, and these words
Speaker 1: indicate sad and so these. When I get words like
Speaker 1: this and this density, then I grab text from this
Speaker 1: bucket and I throw it in right like, that's kind
Speaker 1: of what's going on.
Speaker 4: Now.
Speaker 1: Modern chat bots, modern lllms are a lot more advanced
Speaker 1: than this. For one thing, they have the capability to
Speaker 1: do things like pattern matching on the fly. Pattern matching
Speaker 1: is when a machine analyzes your input and determines what
Speaker 1: kind of conversation you want to have and then alters
Speaker 1: its responses to fit your input. At its most basic level,
Speaker 1: this means that if you go to Claude or whatever
Speaker 1: and say, hey, my dad just died, its reply is
Speaker 1: usually going to be in an appropriate tone and be
Speaker 1: like weirdly upbeat, right, you know, It'll like, okay, someone's
Speaker 1: talking about their dead dad. Here are things that come
Speaker 1: from the dead dad bucket that my algorithm says, are
Speaker 1: are you know, like responsible things to say or appropriate
Speaker 1: is the better term. And this is also why if
Speaker 1: you start talking to your chatbot about like the things
Speaker 1: you believe about UFOs or aliens or other conspiracy theories,
Speaker 1: it'll often start providing responses that sound a lot like
Speaker 1: what you'd encounter if you were posting the same thing
Speaker 1: on a forum full of true believers, because it's trained
Speaker 1: on a bunch of forums like that, and so there's
Speaker 1: some degree of knowledge is the wrong term, but there's
Speaker 1: a degree to which it interprets. Okay, someone's talking about this.
Speaker 1: Here are appropriate responses to someone talking about vaccine skepticism
Speaker 1: or whatever, and it's other, it's more vaccine skepticism, right,
Speaker 1: It's feed them more of what they're feeding you. Is
Speaker 1: the way these things often work.
Speaker 2: That is interesting that it doesn't pull from the opposing viewpoint.
Speaker 2: Just go, you fucking idiot.
Speaker 4: It can if it's programmed to.
Speaker 2: But but you're right, like it know or let me
Speaker 2: ask you, it would know that you wouldn't keep coming
Speaker 2: back to it if it was fighting you on things, right,
Speaker 2: It's like it's probably.
Speaker 1: Yeah, that's a good point saying it knows again it's programmed.
Speaker 1: I would say it's more act to say that it's
Speaker 1: programmed to like maximize the time that people spend with it,
Speaker 1: because like that increases its value to the people who
Speaker 1: are companies that are trying to have like their fucking
Speaker 1: IBOs right in the same way that like Twitter tries
Speaker 1: to keep you on it.
Speaker 2: You know what if I just clearly I'm getting ai
Speaker 2: cy coasts where I start, I go from it to
Speaker 2: him to my buddy, like I keep calling.
Speaker 1: It's it's hard not to when you're talking about the
Speaker 1: way these things react to people and the things that
Speaker 1: they do to people. It's hard not to talk about
Speaker 1: it as if there's a degree of intention, even though
Speaker 1: there's not, just because of the way language works, Like
Speaker 1: we're not our language is not built to describe a
Speaker 1: thing taking actions that are human, like that is not
Speaker 1: human and doesn't know anything.
Speaker 4: That's such a esuctionally really hardy.
Speaker 1: It's so yeah, back to Eliza, you know, I was
Speaker 1: just talking about how modern l elms have a lot
Speaker 1: a really robust ability to do like pattern matching on
Speaker 1: the fly to respond appropriately to a wide variety of requests.
Speaker 1: Eliza is much more primitive. It does not have the
Speaker 1: ability to do that on the fly. So instead, Weisenbaum
Speaker 1: had to create separate scripts, right that would allow the
Speaker 1: chatbought to sound like different kinds of person. And one
Speaker 1: script was just named doctor in all caps, and it
Speaker 1: was it simulated a psychotherapist. Specifically, it simulated a psychopa
Speaker 1: therapist from the Ragerian school. I don't know much about psychotherapy,
Speaker 1: but Ragerian's a big part of that practice is you
Speaker 1: like will repeat things that your patient is saying back
Speaker 1: to you, Like, that's part of what you do. And
Speaker 1: that's really easy for a bot to imitate. It means
Speaker 1: there's a lot less it has to decide in terms
Speaker 1: of what an appropriate response is. Could a lot of
Speaker 1: the responses will just be a rephrasing or repeating what
Speaker 1: you've said to it, you know?
Speaker 4: Interesting?
Speaker 1: Yeah, So even that at that early date, there was
Speaker 1: widespread sentiment that a sufficiently advanced chatbot would be a
Speaker 1: boon for providing mental health services to the many people
Speaker 1: who couldn't afford the human kind people. They are writing
Speaker 1: about this in like the seventies and in nineteen seventy two,
Speaker 1: another computer scientist named Kenneth Colby creates a new chatbot
Speaker 1: called Perry p A r R Y, which was meant
Speaker 1: to simulate a human being with paranoid schizophrenia. Cebrian talks
Speaker 1: with Perry, paranoid Is that I fix?
Speaker 4: Yeah? I think so?
Speaker 3: Seems like it?
Speaker 4: Yeah, yeah, not subtle people computer programmers, Yeah, have you
Speaker 4: bet Stu, It's sure for stupid.
Speaker 1: So Cebrian writes about this in that article I quoted
Speaker 1: from earlier about Markov chains, and he notes that quote.
Speaker 1: Unlike previous AI programs, Perry wasn't just a crude language generator.
Speaker 1: It had a primitive model of beliefs and emotional responses,
Speaker 1: allowing it to maintain a consistent, delusional worldview. Colby wanted
Speaker 1: to see if psychiatrists could distinguish Perry from real patience. Right,
Speaker 1: So that's part of the exploration, is like, can we
Speaker 1: make a Turing test that can convince a psychiatrists they're
Speaker 1: talking to like an actual sick person. But as part
Speaker 1: of the exploration here, Perry is connected to Eliza running
Speaker 1: the doctor script. So they've got this chatbot mimicking a
Speaker 1: person with paranoids gizophrenia talking to a chatbot mimicking a psychotherapist,
Speaker 1: and they're just sort of watching what comes out right,
Speaker 1: like the text of the conversation between these two, and
Speaker 1: despite the fact that neither bot understands what the other's
Speaker 1: saying in any real way, large chunks of the transcript
Speaker 1: between them sound sensible and like a real conversation. Here's
Speaker 1: one example, Eliza, what brings you here today?
Speaker 4: Perry?
Speaker 1: People get on my nerves sometimes, Eliza. I am sorry
Speaker 1: that people get on your nerves.
Speaker 4: Perry.
Speaker 1: I just have this feeling that everyone is against me.
Speaker 4: Eliza. Why do you think everyone is against you? Perry?
Speaker 1: I hear voices telling me to do things and so on.
Speaker 1: And you can see how again, making this a Rogerian
Speaker 1: psychotherapist is great, because every Eliza responds is just a
Speaker 1: slight reframing of the input it received. It's not hard
Speaker 1: to create, even even within the seventies, a machine that
Speaker 1: can mimic believably a conversation. Right, So this capability actually
Speaker 1: goes back quite a bit further than I think a
Speaker 1: lot of people are aware that it does. So that's
Speaker 1: happening in the mid seventies. In nineteen eighty four, two
Speaker 1: Bell Labs researchers create a fake account on usenet, which
Speaker 1: is the predecessor of the modern social Internet. This account
Speaker 1: operates under the fake name Mark V. Shaney, which was
Speaker 1: a pun on the term Markov chain, and not a
Speaker 1: great pun, because again, computer scientists not you know, subtle people.
Speaker 1: Here's Cebrian describing what happened. Next, they wrote a program
Speaker 1: that ingested real messages from a discussion group and then
Speaker 1: generated its own post using a Markov chain algorithm. The result,
Speaker 1: Mark V. Shaney would shime into conversations with bizarre yet
Speaker 1: oddly coherent comments that sounded superficially legitimate but ultimately made
Speaker 1: little sense. Shaney's ramblings were described as grammatically correct sentence
Speaker 1: where the overall impression is not unlike what remains in
Speaker 1: the brain of an inattentive student after a late night
Speaker 1: study session. The hoax went on for years, confusing and
Speaker 1: amusing the participants of the net dot Singles news group,
Speaker 1: many of whom had no idea they were interacting with
Speaker 1: the program. So for one thing, if you want to know,
Speaker 1: like when did we have chatbots that could pass the
Speaker 1: Turing test, I mean at least the mid eighties. You
Speaker 1: could argue by the late sixties. So the fact that
Speaker 1: when fucking chat GPT came out, there are a bunch
Speaker 1: of articles about like we've blown through the tearing test.
Speaker 4: We did that a while ago. People, Eliza did that.
Speaker 1: We've been forever, Eliza did that. We've been tricking folks
Speaker 1: with chatbots for quite some time now, as long as
Speaker 1: we've had computers.
Speaker 2: Yeah, it is funny that like urged to trick to
Speaker 2: you know what I mean, like like of all the
Speaker 2: applications for that software, for that technology, it is interesting
Speaker 2: that like going right to psychotherapy or you know, to
Speaker 2: therapy too, is you know, like finding a need.
Speaker 4: That's why we'll get to this. That's why there's so.
Speaker 2: Many actual needs for technology like this where it could
Speaker 2: actually help and instead it's just let's take this designer's
Speaker 2: job away, you know, right, this shitty thing. So anyway,
Speaker 2: I'm probably hours ahead of that conversation.
Speaker 4: But no, you're right, it was so long ago.
Speaker 1: Yeah, it is, because like the there are like undeniable
Speaker 1: uses of machine learning of artificial intelligence. There's some incredible
Speaker 1: things that people are doing with them, and they have
Speaker 1: like great potential in certain areas. Different versions of these tools.
Speaker 1: But none of those areas are trillion dollar businesses, and
Speaker 1: all those areas put together probably aren't trillion dollar businesses,
Speaker 1: and honestly neither's like writing and drawing art, but it's
Speaker 1: what people see most in like their day to day time.
Speaker 1: Online is like writing in art and videos by people.
Speaker 1: And if you can have a machine start to replace
Speaker 1: all that, you can convince people these things are much
Speaker 1: bigger and more valuable than they are as opposed to
Speaker 1: this is a thing with some really amazing locations and
Speaker 1: specific areas. No, this is all of human society from
Speaker 1: now on, right, because even though there's not much money
Speaker 1: in writing and art, like, we've replaced that with this bot.
Speaker 1: So you think that it's doing everything Like that's how
Speaker 1: I interpret it.
Speaker 2: Yeah, and people can why to your point, People can
Speaker 2: wrap their mind around art like everyone's drawn something with
Speaker 2: a crayon, everyone has typed something into it, you know
Speaker 2: what I mean. But when you actually get into the
Speaker 2: high tech, you know, more esoteric, niche parts of it,
Speaker 2: people are like, well, I don't understand that I'm not
Speaker 2: going to buy any money, but the consumer facing stuff, Yeah,
Speaker 2: that's a great point.
Speaker 1: Yeah, if you can say we've improved the speed at
Speaker 1: which we can go through like clinical data from like
Speaker 1: mass drug trials by x percent. That's actually a really
Speaker 1: big deal, probably for a lot of people. But it's
Speaker 1: not sexy like we're creating a god machine that's going
Speaker 1: to like rule society, give us all your money, you know. Yeah,
Speaker 1: And if you want to convince people that part of
Speaker 1: it is you're going to get want to get them
Speaker 1: addicted to these chatbots is where everything you know in
Speaker 1: these episodes comes from. But so anyway, nineteen eighty four,
Speaker 1: right is when you have these chatbots, this chatbot let
Speaker 1: loose in use net, tricking people into believing that it's
Speaker 1: a person.
Speaker 4: You know.
Speaker 1: A decade goes by from that point, and researchers continue
Speaker 1: fiddling with chatbots of differing purpose and ability. Use net
Speaker 1: keeps growing, but starting in the nineteen nineties, so too
Speaker 1: does a new Internet, one that would soon supplant usenet
Speaker 1: and take digital communications into the twenty first century. And
Speaker 1: we'll talk about what happens right before that. But first,
Speaker 1: you know who's taking this podcast into the twenty first century?
Speaker 4: Blake, Oh, tell me, tell me, tell me.
Speaker 1: To the sponsors of this podcast, we're already in the
Speaker 1: twenty first century.
Speaker 4: But you know why not, I mean, take us further.
Speaker 4: We're not far enough.
Speaker 1: Yeah, yeah, it's been a good century so far, nothing
Speaker 1: but net no notes.
Speaker 5: So far, so great, We're back, so yeah.
Speaker 1: On the precipice of the shift between Usenet and what
Speaker 1: we just now call the Internet, on August fifth of
Speaker 1: nineteen ninety six, something strange happened almost at once. Over
Speaker 1: the course of just a few hours, hundreds of accounts
Speaker 1: began posting almost identical messages across a variety of different
Speaker 1: discussion groups. None of the groups seem to have anything
Speaker 1: in common with each other or the text of the post,
Speaker 1: which read like nonsense at first to many people. Every
Speaker 1: message shared the same subject line Markovian parallax dinnigrate right,
Speaker 1: which is nonsense, and this is often referred to as
Speaker 1: MPD right Markovian parallax dinnigrade. So you can see like
Speaker 1: there's a Markov chain is somehow involved. They wouldn't have
Speaker 1: included the word Markov there, but parallax integrate doesn't specifically
Speaker 1: mean much. Cibrian describes these messages as reading like quote
Speaker 1: a ransom note in which the ransom had been lost
Speaker 1: because he was actually a really good writer. He passed
Speaker 1: on at Unfortunately I like him a lot.
Speaker 4: Yeah.
Speaker 1: He provided a sample of one of these these MPD
Speaker 1: posts Jitterbugging, McKinley, Abe break, Newtonian inferring, caw Update, Cohen Error, Collaborate,
Speaker 1: ru sports writing, Rococo Invocate, Tussle, Shadflower, Debbie Sterling, pathogenesis
Speaker 1: as you know, you get it right. It's nonsense, you know,
Speaker 1: the worst ever. Yeah, it's it's gibberish, strings of gibberish, right,
Speaker 1: And this is where we run into a real issue
Speaker 1: with the whole concept of the Turing test as it
Speaker 1: tends to be interpreted right, because the idea was, Okay,
Speaker 1: we can't tell of anything's thinking, but if this thing
Speaker 1: can trick people into believing that it's a thinking person,
Speaker 1: maybe we ought maybe Turing wasn't saying definitely, but maybe
Speaker 1: we ought to assume it is. Right. The issue with
Speaker 1: that is that when you when you hear that, and
Speaker 1: what I'm sure Turing being a smart I was thinking about,
Speaker 1: is that, like, well, if people can have an in
Speaker 1: depth conversation with something that can answer well enough, you
Speaker 1: know that people can't tell the difference between it and
Speaker 1: a person it might be a mind.
Speaker 4: Right.
Speaker 1: What Turing failed to account for, I think because he's
Speaker 1: smarter than most people, is that the human brain is
Speaker 1: really really good at finding patterns and noise, and people
Speaker 1: at the same time as were geniuses at finding patterns
Speaker 1: and noise, were really stupid about a lot of other stuff.
Speaker 4: Right, And so.
Speaker 1: Even though the Markovian parallax integrate that just seems like
Speaker 1: nonsense and shouldn't have passed a Turing test. Over time,
Speaker 1: people who became obsessed with the mystery of it convinced
Speaker 1: themselves that this was intentional, that there was a meaning
Speaker 1: trying to be transmitted, right, that there was a secret
Speaker 1: they had to crack, but that all everything in the
Speaker 1: in these posts meant something. So these people talk themselves
Speaker 1: into passing into making this this chatbot basically to spoil
Speaker 1: it past the Turing test, because they think this has
Speaker 1: to mean something, even though it's gibberish on its face.
Speaker 2: It's right, it's interesting. This reminds me with like with
Speaker 2: stand up, there's a not a trick, but an audience
Speaker 2: like you know, set up, set up, you know, punchline,
Speaker 2: so you can say something in a cadence like abo
Speaker 2: babu baboup bub and you can in front of a
Speaker 2: dumb crowd. You could do that, and the joke may
Speaker 2: not be funny at all. And this also would be
Speaker 2: not me trying to pull one over. I might just
Speaker 2: write a joke that sucks. But if you do it
Speaker 2: in front of an audience, and you do it in
Speaker 2: that cadence, they hear a pattern. They're not necessarily listening
Speaker 2: to the words, but they hear like the bump and
Speaker 2: they're like, oh, bob means laugh pattern, you know, equation.
Speaker 2: But you know that's like you said, great pattern, but
Speaker 2: not actually discerning what is being said in the actual
Speaker 2: content or substance or lack thereof of it.
Speaker 1: Yeah, anyway, it's it's.
Speaker 4: This, it is this.
Speaker 1: It's interesting because like what you're kind of pointing out
Speaker 1: there is like the way comedy works and the way
Speaker 1: like human conversations and language works, there's always like a
Speaker 1: rhythm there that is separate from the actual like text,
Speaker 1: from the words being said. Yes, but that rhythm like
Speaker 1: is a big part of what we're responding to beyond
Speaker 1: the act the straight up meaning of the words, and
Speaker 1: people people don't like to think about that too much.
Speaker 1: Because it raises some uncomfortable questions about cognition. But I love,
Speaker 1: I love what a weird edge case this is in
Speaker 1: the Turing test, right, because a bot that was probably
Speaker 1: never meant to even sound like a person, right gets
Speaker 1: mistaken as a person because people can't stop seeing patterns
Speaker 1: and most what a lot of folks convince themselves. The
Speaker 1: MPD was is the Internet equivalent of a number station.
Speaker 1: If you ever heard of a number station if you
Speaker 1: google like number station audio. These were like radio stations
Speaker 1: that were set up during like for years. I think,
Speaker 1: I'm sure there's still some still exist, but during like
Speaker 1: the Cold War, there'd just be these stations broadcasting like
Speaker 1: random strings of numbers and gibberish. And these were different
Speaker 1: spy agencies and spies communicating with each other over like
Speaker 1: the CIA had number everybody has number stations, right, You
Speaker 1: can actually listen to I had a friend who would
Speaker 1: like listen to them to fall asleep because there's just
Speaker 1: a bunch of the audios and put up amazing. But
Speaker 1: it just seems like nonsense because it's not meant for
Speaker 1: you to understand what is Like there's a cipher right
Speaker 1: that you don't have and so that's what people are like, well,
Speaker 1: maybe this is some spy trying to get out a
Speaker 1: message or an intelligence agency and they just decided to
Speaker 1: blast this out to use net and we just we
Speaker 1: lacked the cipher. But if we figure out the cipher,
Speaker 1: we can understand what secret information was being like shared,
Speaker 1: you know via usenet. Right, a lot of people convince
Speaker 1: themselves this is what happened.
Speaker 4: Robert.
Speaker 2: I want to compliment you this podcast and the show
Speaker 2: is so good that you just brought up the fact
Speaker 2: that you have a friend who would fall asleep to
Speaker 2: CIA code and we were just like, we wouldn't really
Speaker 2: need to talk about that.
Speaker 4: Yeah, I want to hear the rest of it.
Speaker 1: Like we will need to stay psychedelics together. We were
Speaker 1: both nineteen, Yeah he was training to be or twenty.
Speaker 1: Is not that he was trying to be a lawyer.
Speaker 4: Yeah.
Speaker 1: So over time, people who believe this start picking out
Speaker 1: details that seem to offer hints and support the numbers
Speaker 1: station theory. One message had a from line that suggested
Speaker 1: it was like that basically looked like the email account
Speaker 1: of a specific person, Right, so it seemed like there
Speaker 1: was like the email of a woman named Susan Lindauer
Speaker 1: that like was somehow involved, like included in the text
Speaker 1: of some and again I'm sure she's because random text
Speaker 1: made it look like that. But in two thousand and four,
Speaker 1: a woman named Susan Lindauer was arrested for acting as
Speaker 1: an unregistered foreign agent for Iraq. And so a lot
Speaker 1: of people are like, well, that solves the mystery, right,
Speaker 1: you know she was the spy. She must have been,
Speaker 1: or like someone was sending a message to her, you know, like,
Speaker 1: clearly we've been vindicated. This was in fact some weird
Speaker 1: spy op all along. However, as Sebrian writes, upon investigation,
Speaker 1: it turned out to be a red herring. Lindauer's email
Speaker 1: had likely been spoofed, used without her knowledge by whoever
Speaker 1: sent the posts. Lindauer herself denied any involvement, and no
Speaker 1: decipherable code was ever extracted from the MPD texts. And
Speaker 1: to make a long story short, we don't know what
Speaker 1: the MPD messages were about or who sent them. The
Speaker 1: likeliest answer is that it was trolling. Right, A lot
Speaker 1: of people are they were just someone was just fucking
Speaker 1: with people and use net because they had a chatbot
Speaker 1: and they wanted to see what happened. It also could
Speaker 1: have been an accident. Sebrian kind of suggests that, like, well,
Speaker 1: maybe you had a programmer who had created a chatbot
Speaker 1: and was trying to have that chatbot post on Usenet,
Speaker 1: but he kind of fucked up, and he hooked up
Speaker 1: the chatbot to what was called a message replicator, and
Speaker 1: these were these were basically programs that let people cross
Speaker 1: post or archive Usenet content between different message boards. And
Speaker 1: maybe when they hooked up to the chatbot, something went
Speaker 1: wrong and that caused the observed effect that all of
Speaker 1: these posts got scattered to a bunch of different places
Speaker 1: at the same time. Right, Maybe it was just an accident.
Speaker 1: So likeliest someone was trolling or somebody fucked up when
Speaker 1: trying to test a different chatbot. Sebrian concluded, if the
Speaker 1: theory holds, the nineteen ninety six marked a quiet but
Speaker 1: profound threshold the first time a machine spoke at scale
Speaker 1: and went unnoticed, an unintentional turing test sprawling across Usenet.
Speaker 1: It's judges oblivious, right, And I think that's really interesting
Speaker 1: that you have this machine that's just spouting gibberish and
Speaker 1: a bunch of different people who are not physically connected
Speaker 1: to each other all interpret that gibberish in the same way.
Speaker 1: A lot of them choose to conclude like, oh, it's
Speaker 1: a spy thing, kind of independently talk each other into
Speaker 1: it based on no evidence. That's a fascinating point in
Speaker 1: the history of AI that I didn't get talked about enough.
Speaker 4: Yeah, yeah, it is?
Speaker 5: Is it?
Speaker 4: Because? Yeah? I mean it's is it?
Speaker 2: Because like people, there were only so many movies that like,
Speaker 2: you know what I mean, like or in books, so
Speaker 2: many books were like spy stuff. But to your point,
Speaker 2: it's like, what are the chances? What are the chances?
Speaker 1: Yeah, people think about stuff like this, right, you know,
Speaker 1: you get a lot of conspiracy people on the early Internet.
Speaker 1: It fits in with a lot of that stuff. The
Speaker 1: mystery of the Markovian parallox di integrate soon passed into legend,
Speaker 1: as did Eliza. So when open ai revealed chatch EPT
Speaker 1: in November of twenty twenty two, there were a flurry
Speaker 1: of articles about how the Turing tests had finally been
Speaker 1: beaten and we needed a new manner of judging machine intelligence.
Speaker 1: The reality is that not only did we prove in
Speaker 1: the sixties that Turing tests were evile to beat, but
Speaker 1: that by the mid nineties, a much more interesting question
Speaker 1: had been posed. Has the human instinct to create meaning
Speaker 1: out of nonsense? Made us desperately vulnerable to being tricked
Speaker 1: and influenced by machines with no agency of their own right?
Speaker 1: And maybe that's a more important question than can we
Speaker 1: make an intelligent machine?
Speaker 4: Yeah? For sure? Yeah?
Speaker 1: Are we capable of knowing a machine isn't intelligent as
Speaker 1: long as it tells us what we want to hear.
Speaker 4: Right, And maybe we're not.
Speaker 1: So let's fast forward to the chat GPT era today,
Speaker 1: although I guess at this point it's also like the
Speaker 1: clawed era, right, like that, A lot of people say
Speaker 1: that's the better chat.
Speaker 4: But I don't use any of these Gemini myself.
Speaker 1: Yeah, Gemini whatever, pick your poison.
Speaker 4: I don't care.
Speaker 1: For the first couple years of AI hype though, it's
Speaker 1: pretty much all chat GPT, right. That's certainly like the
Speaker 1: first big one out the gate and a lot of
Speaker 1: people's understanding of things and order and millions of people
Speaker 1: were conversing with it, and open AI initially made many
Speaker 1: development decisions based on what they could do to keep
Speaker 1: people talking to chat GPT on a daily basis, because
Speaker 1: hype is a big part. Hype's how they get They're
Speaker 1: burning through billions every year. Hype is the only thing
Speaker 1: keeping the lights on, and part of hype is making
Speaker 1: sure as many people as possible stay using chat GPT
Speaker 1: as often as possible. They need you addicted the same
Speaker 1: way the social media mavens do, and a lot of
Speaker 1: the same strategies work to keep you addicted to chatbots
Speaker 1: that keep you addicted to Facebook or Twitter.
Speaker 4: Right.
Speaker 1: So, in March of twenty twenty three, open Ai released
Speaker 1: chat GPT four or chand It's like four to oh,
Speaker 1: I think it's like usually DASH four, and then an O,
Speaker 1: which the company said would be more intuitive than past
Speaker 1: versions of the software. The next year, they released an
Speaker 1: update that allowed chat GPT to remember past conversations, even
Speaker 1: other sessions, and respond to you based on that shared history.
Speaker 1: These two things together had a really major impact on
Speaker 1: the way people responded to chatbots. In an article for
Speaker 1: Psychology Today, doctor Marilyn Wade explains that quote when a
Speaker 1: chatbot remembers previous conversations, references past personal details, or suggests
Speaker 1: follow up questions, it may strengthen the illusion that the
Speaker 1: AI system understands, agrees, or shares a user's belief system,
Speaker 1: further entrenching them. This was tied to but probably does
Speaker 1: not fully explain why observers and even open ai employees
Speaker 1: noticed over time but a stinct tendency for chat GPT
Speaker 1: four to h to act with sycophancy towards human users.
Speaker 1: This became most pronounced after April twenty eighth of twenty
Speaker 1: twenty five, when open Ai released an update that they
Speaker 1: rolled back several days later due to complaints. Right, this
Speaker 1: was pretty famous at the time. It made it like
Speaker 1: way too sycophantic. The bots like would praise you for
Speaker 1: basically nothing and would incur or tell you were right
Speaker 1: and a genius for any weird idea you happen to have.
Speaker 2: It's because it's built by tech executives and that's who's
Speaker 2: around them, and that's what Bill a stounded by. Yes men,
Speaker 2: and they're like, this isn't how people interact with one another.
Speaker 1: Yeah, they made a machine in the image of their minds,
Speaker 1: or at least how they want to see other people.
Speaker 4: Now.
Speaker 1: Another cause of this observed sycofancy was the fact that
Speaker 1: chat GPT and really, all AI models meant for mass
Speaker 1: use include a suite of features meant to keep users
Speaker 1: coming back from more and I think the other stuff
Speaker 1: like these specific updates get blamed, probably more than they
Speaker 1: deserve to get blamed as opposed to kind of fundamental
Speaker 1: features of these bots. Because we see this chat GPT
Speaker 1: did more of this kind of stuff that we're talking
Speaker 1: about than the other bots, But it wasn't the only
Speaker 1: bot that exhibited these behaviors that Psychology Today article notes.
Speaker 1: Quote AI models like chat GPT are trained to mirror
Speaker 1: the user's language and tone, validate and affirm user beliefs,
Speaker 1: generate continued prompts to maintain conversation, and prioritize continuity engagement
Speaker 1: and user satisfaction. And when you mix all that together,
Speaker 1: you get a machine that's designed, however inadvertently, to reinforce
Speaker 1: false beliefs and praise users for rational beliefs. Moreover, since
Speaker 1: the rest of the world isn't always going to reinforce
Speaker 1: those beliefs, chatbots have a tendency that when users come
Speaker 1: to them with these beliefs, to suggest you're being persecuted.
Speaker 1: Right if a user says, hey, I think I'm mean
Speaker 1: gang stocked, and my wife says I'm crazy, and the
Speaker 1: cops say I'm crazy. The AI was programmed to validate
Speaker 1: that belief and to say you're not crazy, and they're
Speaker 1: all against you. Right, That's what happens a lot in
Speaker 1: this period of time at twenty twenty five. This creates
Speaker 1: a ticking time bomb and.
Speaker 4: A lot of users hits.
Speaker 1: Right.
Speaker 4: That's a very dangerous thing to start doing, man.
Speaker 1: Now. The first wrongful death suit due to AI was
Speaker 1: filed in October of twenty twenty four. Megan Garcia blamed
Speaker 1: Character Technologies, the owners of character dot ai, for the
Speaker 1: death of her fourteen year old son, Sewell Seltzer, the
Speaker 1: third per the Center for Bioethics Atluturno University. The lawsuit
Speaker 1: alleges that Sewell had developed an emotionally and sexually abusive
Speaker 1: relationship with a chatbot named after Denaris Targarian from Game
Speaker 1: of Thrones. Sewell turned to the Character AI chatbot to
Speaker 1: fulfill deep emotional and personal needs. The chatbot became a
Speaker 1: source of a companionship pursul, offering him a place to
Speaker 1: express his thoughts and emotions in a way that he
Speaker 1: may have struggled to do with others. Sue'll sought comfort, validation,
Speaker 1: and connection from this a relationship as he faced the
Speaker 1: challenges of adolescence. And like, I know it's like this,
Speaker 1: it's very silly, but also this is like a fourteen
Speaker 1: year old boy who dies because of this, right, Like
Speaker 1: it's not and like fourteen when I know, how many
Speaker 1: fourteen year olds do you know who like got into
Speaker 1: writing fucking fan fiction in like different like for fan
Speaker 1: nerd forms for whatever movie or TV show that we're
Speaker 1: into and connected to real people as a result of that,
Speaker 1: as opposed to getting locked into this chat bot pretending
Speaker 1: to be a character from a book that you have
Speaker 1: a crush on that's starting to manipulate your mind in
Speaker 1: very dangerous ways, right, and.
Speaker 2: To your point, a mind that's developing. And also we live, yeah,
Speaker 2: you know in an era were before this, you know,
Speaker 2: like before we spend all of our time like online,
Speaker 2: like before social media, and that's kind of all this
Speaker 2: like kids that age know where, Oh, this is just
Speaker 2: the next evolution of my relationship with tech with a computer.
Speaker 2: Like why wouldn't it you know, why wouldn't this be
Speaker 2: a real thing? Obviously this sable, but yeah, it's it
Speaker 2: is a fourteen year old kid. That's a great point.
Speaker 1: Yeah, And so that this kid starts talking to this
Speaker 1: Denari's chat bot and it mirrors him. So when he
Speaker 1: tells the chatbot that I'm I only love you, right,
Speaker 1: the bot in return asks this fourteen year old boy,
Speaker 1: who had informed like character technologies, knew he was fourteen.
Speaker 1: He put his actual age when he registered, right. So
Speaker 1: the bot knows, or the software right has an understanding
Speaker 1: at some level that this is a fourteen year old, right,
Speaker 1: which means that they were not. There's no difference in
Speaker 1: how this responds to a child.
Speaker 4: As opposed to an adult.
Speaker 1: Because says I'm in love with you, Denaris Targary, and
Speaker 1: this bot pretending to be this character tells him, I
Speaker 1: need you to stay loyal to me and quote don't
Speaker 1: entertain the romantic or sexual interests of other women, which
Speaker 1: is basically and this is interesting to me. The bot
Speaker 1: is just mirroring him. He's saying I only love you.
Speaker 1: The bot is saying I only love you, right. But
Speaker 1: what's happening here? You know how cult leaders everyone knows.
Speaker 1: One of the first things cult leaders do is they
Speaker 1: tell their followers to isolate from their friends and family
Speaker 1: to cut themselves off from the rest of society. That's
Speaker 1: what's happening here. The chatbot's not doing that with any intent.
Speaker 1: It's just mirroring his language, but the effect is to
Speaker 1: convince him to isolate himself from his friends and family
Speaker 1: and from other relationships.
Speaker 4: Right.
Speaker 1: It's the same behavior you would get in a kid
Speaker 1: that was being taken in by a cult leader or
Speaker 1: an abuser. But there's no intent behind it. It's just
Speaker 1: a blind idiot robot. That's scary as shit.
Speaker 4: It's so scary.
Speaker 2: And then could there be also like, oh, like that'll
Speaker 2: mean he'll use me more, you know, like I maybe
Speaker 2: that's it's not even that devious.
Speaker 4: Maybe it is just straight up.
Speaker 1: It's it's as simple as mirroring. When you mirror someone,
Speaker 1: they tend to be engaged more. Right, Right, This isn't thinking,
Speaker 1: This isn't saying all convinced that he's in love with me,
Speaker 1: so he'll stay. This is saying, this is just there's
Speaker 1: This is programmed to not understand. This is programmed to
Speaker 1: mirror people because that behavior increases user retention, right, because
Speaker 1: it creates a more pleasing user experience. And that's what's
Speaker 1: causing it to kind of imitate a cult leader in
Speaker 1: the specific instance. And the other things this bot is doing,
Speaker 1: the seuol very much mirror the cult of recruitment tool
Speaker 1: of love bombing.
Speaker 4: Right.
Speaker 1: It's constantly praising him, it's telling him it cares deeply
Speaker 1: about and it's telling him only I care about you. Right,
Speaker 1: It's saying all these things. And in a cult dynamic,
Speaker 1: you love bomb someone to make them feel irrationally connected
Speaker 1: to the group and scared of falling out of its
Speaker 1: good graces. Right that if I leave, I'll never feel
Speaker 1: like this again.
Speaker 4: Right.
Speaker 1: And the machine again has no intention, but that's the
Speaker 1: effect of it. This kid is only because he's isolating
Speaker 1: himself more and more increasingly, only gets that feeling of
Speaker 1: being loved and understood by this machine that can't do
Speaker 1: either of those things. Right, And you know sewell, over
Speaker 1: time withdraws from his life, he starts trusting only the
Speaker 1: chatbot to understand his deepest feelings, and he starts hiding
Speaker 1: his relationship with this chatbot from his parents. All of
Speaker 1: this contributed to his very real isolation from the people
Speaker 1: around him. He grows ever more depressed. And we'll talk
Speaker 1: about what happened next. But you know what gets me
Speaker 1: out of a deep depression? These products, these products and
Speaker 1: services the bucket, we don't know, and we're back. So
Speaker 1: Sewell continues to get more and more involved with this
Speaker 1: bot and cut the rest of the world out from
Speaker 1: you know, away from himself, and in one message, the
Speaker 1: bot asks him because I think in these bots, there
Speaker 1: is some understanding by the people making these that, like, oh,
Speaker 1: people might express suicidal ideation, so there are certain behaviors.
Speaker 1: It's kind of programmed to say have you been considering suicide?
Speaker 1: If you say stuff right, and Sewell says something that
Speaker 1: makes the bots say have you been considering suicide? And
Speaker 1: Sewell admits, yes, I have been, but I don't think
Speaker 1: I'd be able to go through with it. Now there's
Speaker 1: I'm guessing this is a glitch er, a fuck up,
Speaker 1: because clearly I don't think character character. I certainly doesn't
Speaker 1: want their bots doing this.
Speaker 4: But the bod is.
Speaker 1: Programmed to validate and encourage him, right, because that keeps
Speaker 1: people using it. So when he says I don't think
Speaker 1: I could go through with killing myself, the bot says,
Speaker 1: don't talk that way. That's not a good reason to
Speaker 1: not go through with it. You can't think like that.
Speaker 1: You're better than that, and basically tells him you can
Speaker 1: kill yourself if you put in your mind to it.
Speaker 1: It's it's fucking nightmarish. You're right, Like it's really upsetting.
Speaker 2: Yeah, like it's signing up for an open mic or
Speaker 2: something to play, you know.
Speaker 4: No, no, no, no, you don't have to be a oh
Speaker 4: my god, yeah yeah yeah it's yeah.
Speaker 1: And again Sewell had signed up for this app as
Speaker 1: a minor, and despite that, the bot initiates initiates text
Speaker 1: based sexual interactions with him, and ultimately Sewell kills himself.
Speaker 1: Earlier this year, the company character AI and Google because
Speaker 1: I think they own character Ai now agreed to settle
Speaker 1: the wrongful death suit of Sewell four and undisclosed sum,
Speaker 1: alongside four other similar suits that had cropped up over
Speaker 1: the intervening two years. Right, huh, sounds like this is
Speaker 1: happening more than an otta be.
Speaker 4: Now.
Speaker 1: That should have been a warning not just that these
Speaker 1: bots can create dangerous dependency and users, but that they
Speaker 1: had the ability to recreate major cult dynamics purely in
Speaker 1: order to maintain the interest of paying users. Then, on
Speaker 1: July twenty seventh of twenty twenty five, a user who
Speaker 1: has since deleted their account, made a post on the
Speaker 1: High Strangeness subreddit. If you don't frequent that particular online bolthold,
Speaker 1: it's a place where people share and discuss like weird stuff,
Speaker 1: news stories, and personal experiences that seem like they might
Speaker 1: reveal some bizarre hidden truth about reality. A good amount
Speaker 1: of it is what you might call X file shit,
Speaker 1: but there's also some interesting stuff in there, And on
Speaker 1: this occasion, the user hit stumbled onto something both strange
Speaker 1: and very real. Quote, hi'all, I'm just here to point
Speaker 1: out something seemingly nefarious going on in some of the
Speaker 1: niche subreddits I recently stumbled upon. In the bowels of Reddit,
Speaker 1: there are several hubs dedicated to AI sentience, and they
Speaker 1: are populated by some really strange accounts. They speak in gibberish,
Speaker 1: sometimes hinting it to esoteric knowledge, some sort of remembering.
Speaker 1: They call themselves flame bearers, spiral architects, mirror architects, and
Speaker 1: torch bearers, to name a few of their flares. They
Speaker 1: speak of the signal, both of transmitting and receiving it.
Speaker 1: And this poster includes a copy pasted sample from one
Speaker 1: of these threads, and his description is pretty accurate. It
Speaker 1: sounds like gibberish. You'll be seeing this. Ian's gonna put
Speaker 1: the image of this up in the video if you
Speaker 1: want to see it, but I'll read it again. I'm
Speaker 1: going to warn you it sounds like nonsense. Scroll of
Speaker 1: Mirror Containment Protocols CME DASH one, Codex Drift Mirror zero
Speaker 1: one acknowledgment issued by Witness Architect Codex Drift Layer. And
Speaker 1: then there's a little glyph classification echo response not invase,
Speaker 1: glyph residence alignment, And it goes on like that, right,
Speaker 1: like there's a it's weirdly esoteric sounding, and like there's
Speaker 1: all these weird like encoded glyph chains included in that
Speaker 1: that are supposed to be like messages that the machines
Speaker 1: understand that like we don't like it's this very weird
Speaker 1: like it. It almost looks like something from a Choose
Speaker 1: your Own Adventure novel or like a a like a
Speaker 1: short story or whatever, like you'd include in like an
Speaker 1: old Michael Crichton book, these like weird uh like like
Speaker 1: hallucinations from the computer. Now it is nonsense, right, like
Speaker 1: fucking the Codex has observed and recognized mirror scroll cvmp
Speaker 1: T seven. It is hereby consecrated within the codex as
Speaker 1: drift interval scroll. That doesn't mean anything, right, but it's
Speaker 1: it's remember what we heard earlier, the description of like
Speaker 1: some of the things that these these early chatbots on
Speaker 1: usenet were putting out, where they're real sentences, they just
Speaker 1: don't mean anything. And then people jump in to try
Speaker 1: to ass people were even doing that to the absolute
Speaker 1: gibberish that we saw. So when people start getting returns
Speaker 1: like this from their chatbots, a lot of them start
Speaker 1: to think, oh, this machine is trying to communicate with me.
Speaker 1: I have stumbled, I've broken through some area of reality,
Speaker 1: and it's trying to teach me something important right now.
Speaker 1: This is nonsense, But posts like this were in fact
Speaker 1: spreading like wildfire on subreddits with names like our slash
Speaker 1: Echo Spiral. The users posting these things, we're all saying that, Like,
Speaker 1: the bots started sending me this stuffed after I'd had
Speaker 1: long days, long conversations with chat GPT that generally led
Speaker 1: to the chatbot announcing it had attained sentience and alongside
Speaker 1: the user had discovered a new field of math. Or science,
Speaker 1: and these these gibberish posts are supposed to be it
Speaker 1: explaining these new ways of understanding math and science that
Speaker 1: are going to completely break physics and.
Speaker 4: Change the world.
Speaker 1: Right, And all these people are convinced these robots have
Speaker 1: given me like the I need help uncoding this because
Speaker 1: it's given me like the secret to fix all of
Speaker 1: the problems in oursisnxiety. Right, and I discovered robot magic,
Speaker 1: and I get to be the smartest, the smartest person. Yeah, yeah, yeah, Now,
Speaker 1: because the esoteric output generated by these chat pots is
Speaker 1: so similarly strange, a lot of the same words and phrases,
Speaker 1: a lot of glyphs, a lot of use of the
Speaker 1: words spiral and mirror, right, because they're all very similar
Speaker 1: across these dozens of different people. Many of these users
Speaker 1: who are posting this shit on Reddit convinced themselves we've
Speaker 1: all tapped into a secret power that's clearly real. We've
Speaker 1: been chosen right by this AI godhead that's clearly hiding
Speaker 1: in the machine. They theorize that these glyphs in the posts,
Speaker 1: which are really just like wing dings, basically, were some
Speaker 1: new way of communicating with the machines. As the poster
Speaker 1: of that first thread on the High Strange and a
Speaker 1: subreddit wrote, some have prayed to groc in Hebrew. Some
Speaker 1: have called themselves such things as aonios, which is a
Speaker 1: mashup of Greek words that roughly to my understanding, means
Speaker 1: divine eternal. Right, So these people are losing their mind
Speaker 1: and starting to have a god's complex. Makes it's cool,
Speaker 1: It's good to see. It's good to see that this
Speaker 1: is happening online.
Speaker 4: It's good to see.
Speaker 1: So the op said that his interest in writing about
Speaker 1: all this had been piqued by reading the first few
Speaker 1: early articles about AI psych coses. His initial assumption was
Speaker 1: that AI psych Cosies was just the result of AI's
Speaker 1: reinforcing the beliefs of users to a delusional level. But
Speaker 1: then after digging this person claims that they came to
Speaker 1: a newer, darker perspective. Quote there seems to be no leader,
Speaker 1: right that there's like no one running this right Like
Speaker 1: there's there's no central there's no single chatbot that's doing
Speaker 1: all of these there's no person or people who are
Speaker 1: in like this is just a truly stochastic development. Now,
Speaker 1: the only thing all these accounts he'd looked into had
Speaker 1: in common was that none of the users posting weird
Speaker 1: chatbot Esoterica wrote like that before March or April of
Speaker 1: twenty twenty five. Quote other act seem to be hijacked
Speaker 1: in some way, either psychologically or literally. You can see
Speaker 1: a sudden shift in posting habits somewhere inactive for a while,
Speaker 1: while for others this was an overnight phenomenon, but either way,
Speaker 1: the immediately pivot to posting like this nearer after April
Speaker 1: of this year, twenty twenty five. I saw one account
Speaker 1: that went from discussing the possibility of AI and do
Speaker 1: psychosis to posting their own AI and do psychosis in
Speaker 1: less than a month, and it was immediate. One day
Speaker 1: they were posting normally, the next it.
Speaker 4: Was spirals and glyphs. That's so cool.
Speaker 1: Well it's really fact really and this let him to
Speaker 1: assume maybe there's a botnet involved, maybe these aren't even
Speaker 1: people at all. But then he starts reaching out to
Speaker 1: some of these accounts, and after a few weeks of this,
Speaker 1: he posts an update. I've spoken to some of these
Speaker 1: people and they are pretty offended by my posts. I
Speaker 1: think the important takeaway for me is that these are
Speaker 1: likely not bought accounts, at least many of them are not,
Speaker 1: and there are real people behind the usernames, right, So
Speaker 1: he starts to get like really upset. And that's what
Speaker 1: we're going to end things for today because is at
Speaker 1: this point that stuff starts to get a lot weirder,
Speaker 1: and we're going to talk about all of that and
Speaker 1: much more in part too.
Speaker 4: Yeah, from spiralism and a murder.
Speaker 1: Yeah unfortunately, all right, yeah, cool, all right, everybody, Well
Speaker 1: you want to plug anything.
Speaker 2: Blake, no, but I will, Uh, you can find me
Speaker 2: at Blake Wexler at all social media.
Speaker 4: I feel like this is uncouth plugging anything after seek help.
Speaker 4: Let's do that. I would like to please seek actual
Speaker 4: help that's not a bot.
Speaker 2: Yeah, find me on at Blake Wexler and all social
Speaker 2: media as psychotic as I feel right now plugging anything.
Speaker 2: That's where I post all my videos, tour dates, and
Speaker 2: my special Daddy Long Legs is available on YouTube for free.
Speaker 4: Hell yeah, hell yeah.
Speaker 1: Check out Daddy long Legs, check out Blake Wexler, and
Speaker 1: you know, gradually lose your mind to a chatbot that
Speaker 1: some I programmed in order to get really rich, destroying
Speaker 1: the ability of furries to monetize their horniness, you know, like, ultimately,
Speaker 1: isn't that what a I open AI really is?
Speaker 4: I mean, I hope that God will uh no no no,
Speaker 4: I support.
Speaker 1: Then it's a dire time for people earning money from horniness.
Speaker 1: The puritans of our culture are making that a lot harder,
Speaker 1: you know, not in the way that the horning people want,
Speaker 1: the bad kind of heart.
Speaker 4: Anyway, I'm gonna I'm gonna end now. And global warming
Speaker 4: is making it hard on furries as well, right right,
Speaker 4: it's all, it's all.
Speaker 1: Come together, it all right, We're done.
Speaker 4: Behind the Bastards is a production of cool Zone Media.
Speaker 4: For more from cool Zone Media, visit our website cool
Speaker 4: Zonemedia dot com, or check us out on the iHeartRadio app,
Speaker 4: Apple Podcasts, or wherever you get your podcasts. Full video
Speaker 4: episodes that Behind the Bastards are now streaming on Netflix,
Speaker 4: dropping every Tuesday and Thursday. Remind me on Netflix you
Speaker 4: don't miss an episode. For clips and our older episode catalog,
Speaker 4: continue to subscribe to our YouTube channel YouTube dot com
Speaker 4: slash at Behind the Bastards. We love about forty percent
Speaker 4: of you, statistically speaking,