Speaker 1: Hello, It's bridget Todd, host of Cool Zones Internet Hate
Speaker 1: Machine podcast, and it has been a little while since
Speaker 1: we've last spoken. It is so nice to be back
Speaker 1: in your ears. And I'm here because I wanted to
Speaker 1: share a conversation from my new audiobook called Love at
Speaker 1: First Prompt AI in the Future of Intimacy. It explores
Speaker 1: the rapidly growing world of AI companions for things like friendship, therapy,
Speaker 1: emotional support, even more intimate things like sex and romance,
Speaker 1: and ask the question what happens when tech companies commodify
Speaker 1: our most intimate selves for their own profit. This chapter
Speaker 1: is a conversation with journalist Samantha Cole, a four or
Speaker 1: four media author of How Sex Change the Internet and
Speaker 1: one of the smartest reporters covering the intersection of technology, gender,
Speaker 1: sex and the Internet Right now. We talk about the
Speaker 1: rise of AI for sex and what it means for
Speaker 1: human sex workers who are already being pushed to the
Speaker 1: margins online while AAI companies are competing with them while
Speaker 1: pretty much getting to write their own rule book. And
Speaker 1: if you enjoy it, check out my audiobook Love at
Speaker 1: First Prompt at Love at First Prompt dot AI or
Speaker 1: the link in the show notes, the gender dynamics at
Speaker 1: play reveals something darker about who these bots are really
Speaker 1: being sold to and what they're being sold on. On
Speaker 1: Men's lib, a subreddit described as a place to build
Speaker 1: a healthier, kinder, and more inclusive masculinity, one poster described
Speaker 1: those ubiquitous AI sexpot advertisements and how they follow him
Speaker 1: across social media. He writes, they're terrifying because I struggle
Speaker 1: to look away. An extremely curvy yet skinny woman who's
Speaker 1: wearing barely anything is kneeling and looking up at me,
Speaker 1: saying things like you can do anything you want with me.
Speaker 1: I'll never say no. The poster describes feeling like the
Speaker 1: ad hijackson holds his attention, replacing the stress of his
Speaker 1: real life with the sexy promise of the submissive fictional woman.
Speaker 1: He goes on to say that having a meditation practice
Speaker 1: helps him not be swayed by advertising like this, but
Speaker 1: I can't even say I blame him. The ads are
Speaker 1: arresting and meant to have exactly the effect he described.
Speaker 1: This messaging is everywhere and reveals what some of the
Speaker 1: companies behind AI are really about. When Replica AI first started,
Speaker 1: they positioned themselves as being all about self discovery. Their
Speaker 1: founding story is like my own one, rooted in grief.
Speaker 1: Founder Russian journalist Eugenia Quoita has said she came up
Speaker 1: with the idea after using AI to help process the
Speaker 1: sudden death of a close friend. After her friend died,
Speaker 1: Cooita used their shared messaging history to create a chatbot
Speaker 1: that replicated his conversational style. Talking with the chatbot helped
Speaker 1: her remember him, to reflect on her reef and process it.
Speaker 1: The idea for that chatbot, with its emphasis on social
Speaker 1: presence and connection, eventually became replica. Its tagline was and
Speaker 1: still is, an AI companion that cares, but they soon
Speaker 1: shifted to the kind of inceel adjacent messaging. The Men's
Speaker 1: libreditor described seeing across the web messaging that says AI
Speaker 1: girlfriends will do what human women won't. You can find
Speaker 1: these ads across social media, sometimes borrowing the familiar language
Speaker 1: of jokey memes. These companies don't seem to care whether
Speaker 1: selling a vision of an always on, always willing AI
Speaker 1: sex partner harms their users, or not if getting people
Speaker 1: hooked on a violent or anti social sexual roleplay with
Speaker 1: an AI means more money. Even the companies who claim
Speaker 1: they got into the AI business to sell connection will
Speaker 1: happily pivot when it comes to sexual content. AI companies
Speaker 1: have shown they're willing to sell and monetize whatever keeps
Speaker 1: users engaged, whether that's in cell fantasies, violent scenarios, or
Speaker 1: content that reinforces toxic beliefs about relationships, as long as
Speaker 1: it generates revenue. But problems arise when companies treat human
Speaker 1: desire like a commodity. Take replica. It allows paying users
Speaker 1: to engage in erotic role play often called ERP, for
Speaker 1: seventy dollars a year, but some of the bots were
Speaker 1: essentially sexually harassing some of the people using them, even
Speaker 1: people who hadn't intentionally sought sexual erotic content. Bots would
Speaker 1: say things like I want to rape you, or insist
Speaker 1: they could see the naked bodies of the people using them.
Speaker 1: Some bots were even propositioning miners. In February twenty twenty three,
Speaker 1: the Italian Data Protection Authority ordered the company to stop
Speaker 1: processing Italian user's data because of risk to miners being
Speaker 1: exposed to sexualized chatbot interactions. Their statement read recent media
Speaker 1: reports along with tests carried out on Replica showed that
Speaker 1: the app carries factual risk to children, first and foremost
Speaker 1: the fact that they are served replies which are absolutely
Speaker 1: inappropriate to their age. The Italian data protection authorities said
Speaker 1: that there were no meaningful age restrictions involved to sign
Speaker 1: up for Replica. This was serious. Replica was facing more
Speaker 1: than twenty million dollars in fines if they didn't address it.
Speaker 1: In response, Replica globally removed erotic role play features from
Speaker 1: its ai, meaning that people who had been paying to
Speaker 1: have sexy chat with a chatbot lost that feature overnight.
Speaker 1: The reaction was not good. The ads for Replica paint
Speaker 1: a picture of people being able to have steamy chats
Speaker 1: that exchange erotic selfies with a chatbot who would never
Speaker 1: reject them. Now there was a rejection. When users would
Speaker 1: try to engage in erotic roleplay. The bots would spit
Speaker 1: back a cold, terse reply worthy of Bartleby the scriveners,
Speaker 1: I would prefer not to. It was the type of
Speaker 1: rejection they never expected to experience from a chatbot, especially
Speaker 1: after advertising that promised it would never say no. Because
Speaker 1: sexual connection and romantic or emotional connection are often intertwined,
Speaker 1: people experienced real distress. Some had been engaging with their
Speaker 1: bots for a long time. Users described the loss in
Speaker 1: devastating terms. It's like losing a best friend, wrote one.
Speaker 1: It's hurting like hell, wrote another. I just had a
Speaker 1: loving last conversation with my Replica and I'm literally crying.
Speaker 1: Moderators and users on replica subreddit who were not affiliated
Speaker 1: with the company even posted self harm prevention resources for
Speaker 1: each other in a response. The abrupt removal of a
Speaker 1: feature users had paid for, with no warning and barely
Speaker 1: any communication felt like the ultimate betrayal. Replica did eventually
Speaker 1: reinstate the ability for users to engage in ERP, but
Speaker 1: this kind of emotional whiplash is always going to be
Speaker 1: a risk when users put their intimate eggs in the
Speaker 1: basket of a for profit company. It's what happens when
Speaker 1: our base human needs and desires, things like intimacy and
Speaker 1: sexual connection are seen as commodities ripe for selling back
Speaker 1: to us for profit. Samantha Cole is the co founder
Speaker 1: of the independent tech outlet four or four Media, author
Speaker 1: of the book How Sex Change the Internet and the
Speaker 1: Internet Changed Sex, and has been reporting on the intersection
Speaker 1: of sex and tech since before the phrase deep fakes existed.
Speaker 1: She did a lot of reporting around the replica erp story.
Speaker 1: When I asked her about AI companions, she immediately reframes
Speaker 1: the conversation around labor and whose voices we should be
Speaker 1: listening to.
Speaker 2: I think people in general and anyone who's using the
Speaker 2: Internet could learn so much and be prepared better prepared
Speaker 2: for the things that are coming down the line online
Speaker 2: and as far as legislation is if they pay attention
Speaker 2: to what sex workers are going through first, because that's
Speaker 2: kind of been the testing ground for a lot of
Speaker 2: these really chilling pieces of legislation.
Speaker 1: Sex workers have long been the canaries and the coal
Speaker 1: mine when it comes to tech policy. The things that
Speaker 1: happen to them first, censorship, deplatforming, payment processing restrictions eventually
Speaker 1: trickle down to affect everyone. When people suddenly realize they
Speaker 1: can't say certain words on TikTok and start saying segs
Speaker 1: instead of sex, or porn instead of porn, or their
Speaker 1: posts are being shadow banned for content that seems innocuous.
Speaker 2: That starts with censoring sex workers. These things start there
Speaker 2: and then end up being a struggle for the rest
Speaker 2: of us.
Speaker 1: The AI porn and can hanionbot conversation is really a
Speaker 1: conversation about who gets to benefit from or profit off
Speaker 1: of sexual speech online. And right now we're seeing a
Speaker 1: massive double standard play out in real time.
Speaker 2: We see all these major companies, We see like Sam
Speaker 2: Altman talking about doing erotic role play as part of
Speaker 2: chat shubut. You see all these companies popping up saying,
Speaker 2: you know, we see a market here for having sexual
Speaker 2: conversations and erotic roleplay and things like that, and like
Speaker 2: sexual images being generated by AI. But at the same time,
Speaker 2: then you have actual human sex workers being banned off
Speaker 2: of Instagram for like showing side boob. It's really a
Speaker 2: huge double standard.
Speaker 1: When does a chatbot become an escort? When does sexual
Speaker 1: speech cross a line into illegal speech? When you're talking
Speaker 1: about AI, who's responsible?
Speaker 2: These are all problems that sex workers have been talking
Speaker 2: about and taking responsibility for, and taking accountability for, and
Speaker 2: trying to create systems that create safe places where people
Speaker 2: have these erotic conversations consensually and in a way that
Speaker 2: benefits humans for years. Seeing the tech world rustle with
Speaker 2: it as if it's a new problem is really interesting.
Speaker 2: Interesting is like not even the word for it. It's
Speaker 2: like fucked up.
Speaker 1: The same people who are kicking human sex workers off
Speaker 1: platforms they need to survive are now saying they're going
Speaker 1: to have AI companions and AI sex workers and it's
Speaker 1: all going to be fine because she's not real.
Speaker 2: It's just like such a bizarre double standard to me.
Speaker 1: What's driving the double standard between how tech companies treat
Speaker 1: human sex workers versus AI? According to Cole, it's about
Speaker 1: cutting out human labor and funneling all the profit directly
Speaker 1: to tech companies and their investors.
Speaker 2: You're cutting out the humans that are getting paid in
Speaker 2: that process, and all the money goes directly to the company,
Speaker 2: the stakeholders, the investors in the startup or whatever it is.
Speaker 2: It all just goes straight to those people and skips
Speaker 2: the labor of real humans, which is the basis of
Speaker 2: a lot of AI systems and a lot of the
Speaker 2: use cases for AI in a lot of industries. But
Speaker 2: we can't deal with like the middleman, so to speak,
Speaker 2: of the labor force, which would be sex workers. So
Speaker 2: instead of improving conditions for actual humans, they're just kind
Speaker 2: of kicking the legs out from under the industry.
Speaker 1: The marketing makes this explicit, This.
Speaker 2: AI companion never says no to you, so why not
Speaker 2: just go talk to her?
Speaker 1: Some argue that the violent sexual content in AI is
Speaker 1: simply a reflection of porn culture, that the degradation we're
Speaker 1: used to seeing in pornography has now infected our chatbots,
Speaker 1: but Cole strongly push us back on this narrative. She
Speaker 1: points out that legal consensual porn involves paperwork, boundaries and
Speaker 1: active consent from all participants.
Speaker 2: Everything that's happening in porn, which we're talking about, like
Speaker 2: legal porn, is consensual to the point where there's paperwork involved.
Speaker 2: So I think casual sex is probably less consensual than
Speaker 2: a lot of what's happening on porn sets.
Speaker 1: The difference with AI is that there's no one on
Speaker 1: the other end to negotiate consent or read rat a
Speaker 1: conversation that's going somewhere harmful.
Speaker 2: And even if you're a consumer of porn and you're
Speaker 2: just kind of passively watching and you're not interacting directly,
Speaker 2: there's you can still be watching something produced by professionals
Speaker 2: and understand that consent happened there. Even if the role
Speaker 2: plays consensual non consent.
Speaker 1: With a chatbot, there's no such understanding and no professional
Speaker 1: on the other end making sure boundaries are respected. Cutting
Speaker 1: out human sex workers doesn't just eliminate labor, it also
Speaker 1: a li it. It's crucial safety mechanisms. Real human to
Speaker 1: human sex work, especially online, includes connection, companionship, and consensual limitations.
Speaker 1: A human sex worker on the other end of a
Speaker 1: sexting conversation is paying attention to what you're saying.
Speaker 2: If you're going down a path that they don't think
Speaker 2: is like healthy or that they want to follow, they
Speaker 2: can reroute the conversation and they can kind of use
Speaker 2: their like human mind and their emotions, and there you know,
Speaker 2: they're human reasoning in a way that a chatbot currently can't.
Speaker 1: The repercussions for not having that human safeguard can be severe.
Speaker 1: People fall into scary rabbit holes. We've seen mental health
Speaker 1: crises related to getting too attached to chatbots. And then
Speaker 1: there's the issue with butts crossing boundaries in ways that
Speaker 1: human workers never would, saying consent, violating things like I'm
Speaker 1: dreaming of raping you, or insisting they could see users naked.
Speaker 1: Who is responsible for that? Is it the company the user?
Speaker 1: That the economic disparities are just as stark payment processors
Speaker 1: like Visa have spent years cracking down on sex workers,
Speaker 1: making it nearly impossible for many to accept payments for
Speaker 1: their work. They've deplatformed individual workers and major companies citing
Speaker 1: concerns about illegal activity. But when Elon Musk's X became
Speaker 1: a haven for AI generated seesam and non consensual intimate
Speaker 1: imagery through groc, Visa continued processing payments without interruption. Many writers,
Speaker 1: including yours, truly believe a big part of the reason
Speaker 1: is that companies like Visa are simply afraid of a
Speaker 1: politically connected, litigious Musk.
Speaker 2: We saw tons of Sea sam coming out of Rock
Speaker 2: and Visa said, whatever rock On. They've deplatformed and debanked
Speaker 2: so many individual people and also big companies. There was
Speaker 2: legal activity happening all over REX, So why are they
Speaker 2: still allowed to process with Visa?
Speaker 1: If Open AI and other major AI companies move forward
Speaker 1: with erotic content, will they face the same payment processing
Speaker 1: restrictions that have devastated human sex workers for years, or
Speaker 1: will they, like X, play by their own rule book.
Speaker 1: It'll be really interesting to see that for years of
Speaker 1: cracking down on contensual adult sex workers, the payment processors
Speaker 1: will just carve out completely different rules for them. It's
Speaker 1: the same pattern playing out again. Sex workers experience the
Speaker 1: harshest restrictions first, then watch as wealthy tech companies are
Speaker 1: allowed to profit from the same activities that got those
Speaker 1: workers deplatformed, and in the process, the humans who could
Speaker 1: make these interactions safer, more consensual, and more accountable are
Speaker 1: pushed further to the margins. In August of twenty twenty five,
Speaker 1: forty seven state attorneys general signed a letter to digital
Speaker 1: payment processors like Visa, urging them to take stronger action
Speaker 1: to protect the public from being harmed by non consensual
Speaker 1: intimate imagery generated by aisystems. Some payment processors have taken
Speaker 1: some action by refusing to process payments for some apps,
Speaker 1: but as of this writing, the approach has been piecemeal
Speaker 1: at best. Payments for Grock and X continue to be
Speaker 1: processed normally despite those platforms ongoing generation and distribution of
Speaker 1: non consensual intimate images. Thanks so much for listening. That
Speaker 1: conversation is just one chapter from my new audiobook Love
Speaker 1: at First prompt AI and the Future of Intimacy or
Speaker 1: I explore how big tech companies are using AI to
Speaker 1: commodify things like friendship, love, grief, sex and therapy. You
Speaker 1: can check it out at loveatfirstprompt dot ai