Speaker 1: Hi, Mostly Human listeners, It's Lori Siegel. Today, I have
Speaker 1: a treat for you.
Speaker 2: I'm sharing an episode of Kara Swisher's podcast On with
Speaker 2: Kara Swisher. In it, Kara hosts a panel of deep
Speaker 2: fake experts, including me, to discuss the threat, the current scope,
Speaker 2: and the future of sexually explicit defake abuse. Now, if
Speaker 2: you've been listening to our four part series Searching for
Speaker 2: Mister deep Fakes, this is a perfect cap. If you're
Speaker 2: new to the subject, it's a fantastic primmer.
Speaker 1: Hope you enjoy.
Speaker 3: Hey, folks, award of warning before we begin. Today's episode
Speaker 3: discusses sexual violence, including rape, graphic sexual content, and suicide,
Speaker 3: so you might want to keep this in mind when
Speaker 3: you decide if and where to listen. Hi, everyone from
Speaker 3: New York Magazine and the Vox Media podcast Now Work.
Speaker 3: This is on with Kara Swisher and I'm Kara Swisher.
Speaker 3: My guests today are attorney Kerry Goldberg, tech journalists and
Speaker 3: CEO of Mostly Human Media, Lori Siegel, and VS. Supermanian,
Speaker 3: the Walter P. Murphy, Professor of Computer Science at Northwestern University.
Speaker 3: We're talking about non consensual deep fake pornography or AI
Speaker 3: generated explicit videos and images that use real people's likenesses
Speaker 3: without their consent. Celebrities were often the target of early
Speaker 3: pornographic deep fakes. Now, according to one survey, one in
Speaker 3: eighteen's personally knows someone who has been targeted. Nearly all
Speaker 3: deep fake porn depicts women. Despite laws and moderation efforts,
Speaker 3: pornographic deep fake websites and so called newdify apps continue
Speaker 3: to multiply, and a recent analysis from Wired found that
Speaker 3: Elon musks rock chatbot is still being used to generate
Speaker 3: and host non consensual deep fakes. I am not surprised
Speaker 3: in the least. Carrie Goldberg is a victim's rights attorney.
Speaker 3: Her law firm specializes in fighting Internet abuse and revenge. Porn.
Speaker 3: Journalist Lori Siegel recently released Searching for Mister Deep Fakes,
Speaker 3: an investigative docuseries created for TikTok and partnership with Paris Hilton.
Speaker 3: It follows an investigation to unmask the anonymous operator behind
Speaker 3: one of the Internet's most notorious deep fake porn platforms.
Speaker 3: VS Subramanian is a top computer scientist and deep fake expert.
Speaker 3: He led the team that developed the Global online deep
Speaker 3: fake Detection system a free platform to help journalists detect
Speaker 3: deep fakes. I think it's critically important to talk about
Speaker 3: this issue. When I started my career, pornography was at
Speaker 3: this heart of the problems with the early Internet and
Speaker 3: the uses of it and the abuse of it was
Speaker 3: built into the system at the start. A lot of
Speaker 3: people were aware of it then and have had a
Speaker 3: very hard time dealing with it even as technology has
Speaker 3: gotten better and better at being more and more abusive.
Speaker 3: So it's critical that we keep talking about it, we
Speaker 3: criminalize this, and we start to hold platforms, big platforms
Speaker 3: responsible for the stuff that they distribute. Our expert question
Speaker 3: today comes from Wall Street Journal tech reporter Georgia Wells.
Speaker 3: This is an important conversation and one we will keep having,
Speaker 3: so stick around and one more thing before we get started.
Speaker 3: If you're going to be in Washington, DC on July sixteenth,
Speaker 3: please join us live at the Johns Hopkins University Bloomberg
Speaker 3: Center for a taping up on I'll be talking to
Speaker 3: Gina Rimando, Commerce Secretary under President Biden and former Rhode
Speaker 3: Island governor about what AI means for the American workforce,
Speaker 3: and as a special bonus before that, conversation. I'll be
Speaker 3: speaking with Johns Hopkins University President Ron Daniels and the
Speaker 3: University of Notre Dame President Father Robert Dowd about how
Speaker 3: universities are approaching AI and workforce issues. You can get
Speaker 3: tickets and learn more at boxmedia dot com. Slash Kara
Speaker 3: Swisher Live, Carrie, Laurie and VS. Thanks for coming on on.
Speaker 3: Could you be here?
Speaker 4: Thrilled to be here?
Speaker 3: Hello, Hello, So. I talked to experts back in January
Speaker 3: about the proliferation of sexualized non consensual AI deep fikes,
Speaker 3: including images of children on Elon muss x that were
Speaker 3: generated by its groc chatbot. Obviously, AI created deep fake
Speaker 3: porn hasn't gone away on Groc or elsewhere, and the
Speaker 3: technology continues to improve. And like each of you to
Speaker 3: explain the current scope of the problem and how pervasive
Speaker 3: it is and who it's affecting, and how technically advanced
Speaker 3: it is. We're going to get more into detail, So
Speaker 3: go high level here, Carrie, We'll start with you, than Laurie,
Speaker 3: than VS.
Speaker 2: So.
Speaker 5: I'm a plaintiff's attorney that sues tech companies, and generally
Speaker 5: I see problems at the very early stages. So you know,
Speaker 5: back probably seven years ago, we started getting situations where
Speaker 5: celebrities and really high level content creators were deep faked.
Speaker 5: And then over the last just i'd say year or so,
Speaker 5: it started trickling down to middle schoolers getting deep faked.
Speaker 5: But all the apps involved were these pretty obscure things.
Speaker 5: But then everything totally changed when Elon started sending around,
Speaker 5: you know, pictures of toasters in bikinis, and it just
Speaker 5: unleashed this never before seen sort of situation where everyone
Speaker 5: was able to create deep fakes and then immediately publish
Speaker 5: them on the X platform.
Speaker 3: So that's where you saw it, Laurie.
Speaker 2: Yeah, when we started looking at this years ago, it
Speaker 2: was very much focused on celebrities, oftentimes women in power politicians.
Speaker 1: It seems like the first line of defense.
Speaker 2: And I remember thinking at the time, oh, no, you know,
Speaker 2: you might not care about these people. We should care
Speaker 2: about these people. But what this means is this is
Speaker 2: going to happen in our high schools, and unfortunately that's
Speaker 2: what we're seeing happening. So I would just say it's rampant,
Speaker 2: and it's happening in our communities. It's happening to women,
Speaker 2: to all sorts of folks, and unfortunately it's something that
Speaker 2: teenagers are learning and it's becoming commonplace.
Speaker 4: Go ahead, yes, Kara, I'm a technologist, and so I've
Speaker 4: spent a lot more time thinking about the technology over
Speaker 4: the last you know, five six years ago, the ability
Speaker 4: to create deep fakes was in its infancy, but over
Speaker 4: the last five years, and in particular over the last
Speaker 4: year or two, it's really gotten to a stage where
Speaker 4: even experts like me on the detection of deep fakes
Speaker 4: have difficulty eyeballing something, listening to something, and telling if
Speaker 4: it's real or fake. So what we're seeing is something
Speaker 4: where you really need humans and AI to come together
Speaker 4: to work as some kind of partnership to try and
Speaker 4: figure this out. And you know, until we can get
Speaker 4: that right, I don't see an end in sight.
Speaker 3: And how has the technology improved in the last year
Speaker 3: Just ease of use.
Speaker 4: Ease of use has always been there. The improvement has
Speaker 4: been in the quality of the generator deep fakes. So today,
Speaker 4: for example, we're seeing things like face swapping, so I
Speaker 4: could record something of myself saying something thoroughly objectionable and
Speaker 4: then put, you know, replace my face with that of
Speaker 4: a celebrity in the past that was difficult to do.
Speaker 4: There were things around the edges of the face that
Speaker 4: would be noticeable. Today, much of that has been worked
Speaker 4: out and.
Speaker 3: Ease of use for normal people to do so correct.
Speaker 4: Indeed, absolutely yeah.
Speaker 3: So mister d Fakes launched in twenty eighteen and was
Speaker 3: once again the largest deep fake porn site online, getting
Speaker 3: seventeen million visits a month at its peak. Laurie, talk
Speaker 3: about your investigation into mister deep Fakes and how it began.
Speaker 2: I began a bit with an obsession, to be quite
Speaker 2: honest with you. I remember it was twenty twenty two,
Speaker 2: chatch Ept had launched. Everybody was talking about how AI
Speaker 2: was going to just do these incredible things. It's going
Speaker 2: to cure cancer, it's going to to you know, give
Speaker 2: us all this free time.
Speaker 1: And of course, I mean, car, you know this better
Speaker 1: than anyone.
Speaker 2: We're all kind of looking at this and saying, okay,
Speaker 2: there are a lot of unintended consequences, and this is
Speaker 2: moving very quickly without the correct guardrails. And someone told
Speaker 2: me about mister deep Fakes. I will never forget going
Speaker 2: to this site and opening it up and seeing thousands,
Speaker 2: thousands of highly sexually graphic images, deep fakes of women
Speaker 2: who did not consent, and they were violent in nature.
Speaker 2: It was a website where a creator could say, you know,
Speaker 2: could go and make money, and someone could go and
Speaker 2: say I want this woman doing this, and they could
Speaker 2: pay for it.
Speaker 1: So it was beyond just a website. It was a platform.
Speaker 2: And then what was even more concerning as we were
Speaker 2: digging into it was, you know, there was everything from
Speaker 2: training data of women's faces to create hyper realistic deep
Speaker 2: fake pornography, and then there were forums where people would
Speaker 2: exchange notes on here's the latest deep fake tools, here's
Speaker 2: how you can do this. You can pay a premium
Speaker 2: and get a deep fake training manual. And so my
Speaker 2: initial thought was this isn't going to just be for
Speaker 2: these high profile women and high profile I would say,
Speaker 2: let's take it with a grain of salt. And I
Speaker 2: remember thinking at the time, well, it's not going to
Speaker 2: stop here. You know, this is just the beginning of it.
Speaker 2: And by the way, you're looking at these forums and
Speaker 2: you're seeing people say, is it wrong that I want
Speaker 2: to do this to my sister in law? And oh,
Speaker 2: this is so easy to do, and so it's almost
Speaker 2: creating entertainment around deep fake abuse. And so as I
Speaker 2: was looking at this, I thought, well, surely there's something
Speaker 2: these women can do to help themselves, and the reality
Speaker 2: was absolutely not. There was nothing they could do because
Speaker 2: the man behind it was anonymous, and he'd been anonymous
Speaker 2: for six years. And even so, even if he wasn't anonymous,
Speaker 2: the laws hadn't caught up. And even when I went
Speaker 2: out and started talking about this, and by the way,
Speaker 2: you never want to be the person talking about deep
Speaker 2: fake porn at the dinner table, but turns out I was.
Speaker 2: People were like, well, the images aren't real that and
Speaker 2: so it doesn't matter, and I'm sure everybody on here
Speaker 2: is kind of just shaking their head. So we also
Speaker 2: had a cultural problem. We needed people to realize that
Speaker 2: these are so hyper realistic.
Speaker 1: Actually this causes.
Speaker 2: Deep, deep psychological abuse, and it can be used to
Speaker 2: try to digitally abuse a woman, I say, a woman anyone,
Speaker 2: And so that was what was alarming, and so we
Speaker 2: decided we had to go find him and raise awareness.
Speaker 1: So that was the beginning of the investigation.
Speaker 3: Let's hear from Joanne Chu. She's an artist and actor.
Speaker 3: One of the women featured in Laurie's investigation. She's talking
Speaker 3: about the moment when she realized her image had been
Speaker 3: used in porn on mister deep Fakes.
Speaker 6: I was just bored one evening and I'm just gonna
Speaker 6: google Google my name and see what comes up. I
Speaker 6: kept scrolling down, and then I said.
Speaker 4: What what what? What?
Speaker 6: What?
Speaker 4: What is this?
Speaker 6: It's like my face on very uh disturbing and graphic
Speaker 6: graphic content. It's you know, Joanne cho or Joanne Cheo.
Speaker 6: In the beginning, it was just a few listings mister
Speaker 6: deep fakes came up social media girls e Porner, and
Speaker 6: then when I typed it in against it multiplied. It's
Speaker 6: very much akin to physical assault. I have like less
Speaker 6: than three thousand followers, but it doesn't matter if you're
Speaker 6: famous or you're not. Just looking at mister dee fakes.
Speaker 6: It's like they have a whole alphabetized database. At one point,
Speaker 6: when I typed in my name, my stuff wasn't coming
Speaker 6: up first anymore. It was all of this content.
Speaker 3: So she compares to being physically assaulted. Carrie, does this
Speaker 3: reflect what you've heard from your clients, because it sounds
Speaker 3: like perpetrators are telling themselves that it isn't real harm.
Speaker 5: Yeah, I remember this when I first started my firm
Speaker 5: back in twenty fourteen, and it was everyone's attitude about
Speaker 5: revenge porn and how you know, this is just an
Speaker 5: online problem. And back then there were all these revenge
Speaker 5: porn websites and people were searchable through Google, which would
Speaker 5: lead people to the search engine results and it would
Speaker 5: just be pages and pages of nude images. Like all
Speaker 5: the harm was really really underestimated back then, and it
Speaker 5: took a huge, huge amount of advocacy to change the
Speaker 5: public opinion about the shame around being you know, nude
Speaker 5: online and we're just we're seeing something really really similar.
Speaker 5: The difference though, is that back you know, ten years ago,
Speaker 5: women were being blamed for having shared an intimate image
Speaker 5: in the first place, and with deep fakes, I don't
Speaker 5: see that same kind of victim blaming because here no
Speaker 5: one consent it to the image in the first place.
Speaker 5: But there is still a lot of I think marginalization
Speaker 5: and minimizing of the harm.
Speaker 3: That it's not physical assault, that it's.
Speaker 5: Yeah, I feel it is conceptually different than a physical assault,
Speaker 5: but that doesn't mean that it's not still really really
Speaker 5: harmful and humiliating for victims, especially like I see so
Speaker 5: many young people who are like victims, and it's other
Speaker 5: young people's doing it to them, you know, like where
Speaker 5: it's on peer in junior high school and high school settings,
Speaker 5: and a lot of times the offenders just think that
Speaker 5: they're doing a prank or if you know, and always
Speaker 5: the defense is, well, I barely shared it with anybody,
Speaker 5: I didn't post it online. But these things find a
Speaker 5: way to get published.
Speaker 3: So deepig technology first to merge around twenty fifteen. VS
Speaker 3: talk about the online communities that exist for sharing information
Speaker 3: making so called newdified tools accessible and how has it
Speaker 3: evolved over the past ten years in terms of how
Speaker 3: it's used.
Speaker 4: So Kara, there are a number of neutification AI tools.
Speaker 4: So these are tools that are of various varieties. One,
Speaker 4: as I mentioned earlier, is face swapping, where you've got
Speaker 4: an individual who's nude performing some sex act and their
Speaker 4: face has been replaced with the face of somebody like
Speaker 4: miss Chew, who you alluded to a couple of minutes back.
Speaker 4: So that's one class of techniques. Another is to use
Speaker 4: massive databases of training data of existing nude individuals to
Speaker 4: create and generate new nudes of people who are hyper
Speaker 4: realistic who are performing acts which the real individual portrayed
Speaker 4: never performed. So that's another class of techniques, and I
Speaker 4: don't want to go into the gory details of how
Speaker 4: that's done, but there are a number of techniques. These
Speaker 4: techniques have evolved better and better every single year, and
Speaker 4: so as a consequence, what we see is really a
Speaker 4: huge number of websites that can disseminate such stuff, not
Speaker 4: just the produced content, but also the tools that are
Speaker 4: used to create and disseminate this stuff. And that's typically
Speaker 4: done on the dark web through some kind of distribution
Speaker 4: network on the dark web.
Speaker 3: And what changed when Elon made it easy to do
Speaker 3: so because for all eyes in the dark web, you
Speaker 3: have to actually make an attempt to go get it right.
Speaker 3: It's like go into a porn store or go going
Speaker 3: to a porn movie or something back in the old days.
Speaker 3: Talk a little bit about how that changed.
Speaker 4: Well, I can't speak specifically to Elon and what he did,
Speaker 4: but what I can say is that the more these
Speaker 4: tools are available through normal channels, the more the audience
Speaker 4: that gets access to them. And as soon as you
Speaker 4: have a big audience, you have a bigger audience of
Speaker 4: bad guys or as Carrie mentioned, just teenagers who don't
Speaker 4: know better and haven't been taught better, possibly because their
Speaker 4: teachers and their parents, however decent they may be, were
Speaker 4: not aware of the threat, and by the time they
Speaker 4: figured out it was too late.
Speaker 3: Deep fakes and newifi apps are also a big problem
Speaker 3: in schools. As we mentioned earlier, recent survey of roughly
Speaker 3: five hundred and fifty US teens found that over half
Speaker 3: had created at least one image using neuification tools, while
Speaker 3: a third of the teens reported having their image created
Speaker 3: and shared non consentually. I'm sure the numbers are higher. Actually,
Speaker 3: Sites like mister deep Figs and apps like groc as
Speaker 3: we know to have normalized creating nons consensual in some
Speaker 3: cases violent sexual imagery, often of women. Laurie talk about
Speaker 3: the consequences for young people.
Speaker 2: Yeah, you know, I'll start. I always like to say,
Speaker 2: like to try to make it a little bit personal.
Speaker 2: I'll never forget, and I'm going to take it here
Speaker 2: and I promise I'll get to young folks, but I'll
Speaker 2: never forget. I think it was twenty twenty three when
Speaker 2: a group of technologists approached me and say, can we
Speaker 2: can we have AI attack you and see what it
Speaker 2: could find. And because I am a crazy person, I said, sure,
Speaker 2: let's see. And what they did was they had AI
Speaker 2: come up with an attack against me. And it took
Speaker 2: some true things, right, like I've interviewed Mark Zuckerberg before,
Speaker 2: or I'm a technology journalist.
Speaker 1: And then it ended up in front of an audience.
Speaker 2: It ended up creating the created sexually explicit deep fix
Speaker 2: of me. And I'll never forget because this I had consented.
Speaker 2: I didn't realize they were going to do that, but
Speaker 2: I had consented to this demo, and I'll never forget
Speaker 2: looking at the audience look at these sexually explicit deep
Speaker 2: fakes of me, and I could, as a person who
Speaker 2: generally formed sentences for a living, I stopped being able
Speaker 2: to form sentences.
Speaker 1: I felt shame, I felt humiliation.
Speaker 2: I felt like the world had seen me naked, to
Speaker 2: be quite honest with you, and then I thought to myself, well,
Speaker 2: if that's how I feel, and I know what this
Speaker 2: technology is, imagine what that's going to be like for
Speaker 2: an eleven year old girl, you know. And I think
Speaker 2: that's what we started hearing. So I've spoken to a
Speaker 2: lot of parents, a lot of teenagers, who've had this
Speaker 2: happen to them. I spoke to one woman who said,
Speaker 2: and you know, she's in her early twenties. She said
Speaker 2: she after this was happening, and she went out and
Speaker 2: she tried to get this stopped, and people said, there's
Speaker 2: nothing you can do. They blamed her, very similar to
Speaker 2: what Carrie is talking about. Back when we were talking
Speaker 2: about non consensual pornography, people didn't quite understand it. She
Speaker 2: told me she walked to the roof of her building
Speaker 2: and she considered jumping off. And you talk to a
Speaker 2: politician who describes this as digital rape. And so for me,
Speaker 2: it was really confusing because I was looking at this
Speaker 2: website that for me, this website represented this dystopian era
Speaker 2: where we lose consent, in an ai era where you know,
Speaker 2: our most intimate qualities can be weaponized against us. And
Speaker 2: then externally we were hearing but it's not real. It
Speaker 2: doesn't matter when it really impacts the way a young person,
Speaker 2: any person walks through the world. We spoke to a
Speaker 2: woman named Mollie who was a mother who had this
Speaker 2: her her husband's best friend. Deep faked the whole neighborhood,
Speaker 2: eighty women and the community and created it's sexually explicit
Speaker 2: deep fakes, and she said it impacted how she would
Speaker 2: go on the bus and look at people and wonder
Speaker 2: if they'd seen this, and she had to go on
Speaker 2: a she said, a porn hunt for her own face
Speaker 2: on the web. And I think it really impacts from
Speaker 2: a human perspective, how you walk through the world and
Speaker 2: how you hold.
Speaker 1: Yourself, and it can have reputational harm.
Speaker 3: Right, So, Carrie, your law firm in your or ensues
Speaker 3: quote psychos pervs, trolls and toxic tech. In January, you're
Speaker 3: from file the lawsuit against say On behalf of Ashley
Speaker 3: Saint Clair, a conservative influencer who had a child with
Speaker 3: Elon Musk. She alleges that his company's Chapbot was used
Speaker 3: to create and disseminate sexually explicit, non consensual images of her.
Speaker 3: Talk about the lawsuit and how is it representative of
Speaker 3: bigger legal issues around deep big porn.
Speaker 5: When Grock suddenly had this new capability of being able
Speaker 5: to create neutification images, this was the first time in
Speaker 5: history that a newdifying tool was integrated into a widely
Speaker 5: used social media platform, which combined the ability to just
Speaker 5: generate an image with the widespread dissemination network, and so
Speaker 5: an image could just be viewable by millions and millions
Speaker 5: of people. But also, I mean, you know, x's for
Speaker 5: people thirteen and up, and so children were also then
Speaker 5: consumers of the images, which is illegal. And we're also
Speaker 5: there were children that were the victims of it. And
Speaker 5: the images posted were not just people in bikinis, but
Speaker 5: GROC would pose them in sexual positions. It would respond
Speaker 5: to prompts telling them to drench their body and blood
Speaker 5: and seamen like fluids, whold suggestive props, and just really
Speaker 5: vulgar and explicit images, and it seemed very clear that
Speaker 5: this was an intended rollout a product that was known
Speaker 5: to be harmful. They were like something like one point
Speaker 5: eight million posts of women and children being deep faked,
Speaker 5: which accounted for like forty one percent of all the
Speaker 5: images posted during this nine day period. So Ashley Saint
Speaker 5: Clair is somebody who is really demonized on the X
Speaker 5: platform because of some of the hostile things that Elon
Speaker 5: Musk has said about her, was a huge target of
Speaker 5: the deep fakes, and there were just so many images
Speaker 5: that GROC generated and published on her own page, and
Speaker 5: so we sued them. We first got you attempted to
Speaker 5: get a temporary restraining order to stop Grock from deepfaking her,
Speaker 5: and then we sued them for defective design and other
Speaker 5: sort of product liability types of lawsuits, negligence, and the
Speaker 5: claim that I like the best is that we sue
Speaker 5: them for public nuisance.
Speaker 3: There's just no reasonable.
Speaker 5: Safe use of this product but to harass people publicly.
Speaker 3: So that's where it stands now.
Speaker 5: And yeah, well, we had to give XAI statutory notice
Speaker 5: that we were filing for a temporary restraining order, and
Speaker 5: they immediately sued Ashley in Texas, claiming that she had
Speaker 5: breached the XAI terms of service by even threatening to
Speaker 5: sue them in her home state of New York. So
Speaker 5: now we are litigating the issue in two different venues.
Speaker 5: They're also trying to use the x terms of service
Speaker 5: to get the XAI case transfer to New York, and
Speaker 5: the judge initially last week said that XAI can rely
Speaker 5: on the x terms of service to transfer.
Speaker 3: The case to Texas. You mean to Texas.
Speaker 5: Yeah, so even though x is not a defendant in
Speaker 5: our case, only XAI is, the judge still let them
Speaker 5: borrow from a different Elon company, which I think is
Speaker 5: really scary.
Speaker 3: So is that where it's going.
Speaker 5: Well, we filed an emergency petition for Mandamus to try
Speaker 5: to get the stay held up, but I don't know,
Speaker 5: we don't have the outcome on that.
Speaker 7: We'll be back in a minute.
Speaker 3: So let's get to the tech infrastructure that supports the
Speaker 3: creation dissemination of deep figs. Analysis from Media Outlet Indicator
Speaker 3: found that newdify apps often use infrastructure from tech companies vs.
Speaker 3: Explain the relationship between deep fake porn sites, tech companies,
Speaker 3: and the social media platforms and to what extent do
Speaker 3: these sites and newify apps rely on big tech infrastructure.
Speaker 3: This was an issue when I was interviewing people who
Speaker 3: were creating all manner of nonsense around the insurrection on
Speaker 3: January sixth, and companies move quickly to take some of
Speaker 3: it down, but they were clearly the way people got
Speaker 3: to these different sites.
Speaker 4: That's a broad question, Kara, so let me try and
Speaker 4: break it down into pieces. The first is depending if
Speaker 4: you're generating deep fakes at scale. So think the kind
Speaker 4: of thing Laurie was talking about with mister deep fake.
Speaker 4: He's trying to generate thousands, millions of deep fakes, and
Speaker 4: so that requires computational hardware what are called graphical computing
Speaker 4: units or GPUs, which today are pretty scarce and expensive.
Speaker 4: So and that scarcity and price doesn't look like it's
Speaker 4: coming down very much much anytime in the near future.
Speaker 4: Those resources are heavily supported by cloud infrastructure providers. Again,
Speaker 4: I'm not naming any specific companies, but there are a
Speaker 4: number of companies that offer cloud services, so part of
Speaker 4: the onus is placed on them if somebody is trying
Speaker 4: to use their service to generate this kind of stuff
Speaker 4: at scale. If somebody is trying to generate just a
Speaker 4: handful of deep fakes, they're probably not going to figure
Speaker 4: it out. But if somebody is generating millions and millions
Speaker 4: of them, or thousands and thousands of them, they may
Speaker 4: have some idea of what's going on. But that's not
Speaker 4: easy either, because you know, there are many challenges for
Speaker 4: some of these firms in figuring out what's a deep
Speaker 4: fake and what's not, what's consensual, what's not and so
Speaker 4: as the first issue, you know, the bad guys are
Speaker 4: continuously evolving, so that me means that the actors in
Speaker 4: question are changing. The technology used to generate this stuff
Speaker 4: is changing, and it turns out, having studied deep fake
Speaker 4: detectors quite a lot and created some on my own
Speaker 4: with my lab, what we know is that small changes
Speaker 4: can make a big difference. So to give you an example,
Speaker 4: if I'm drinking this cup of coffee and there's a
Speaker 4: deep fake of me with part of my face obscure
Speaker 4: as I'm drinking that cup of coffee, it's harder in
Speaker 4: many cases for many detectors to figure out that video
Speaker 4: clip of me saying something which I probably never said
Speaker 4: is not true. Likewise, the definition carry I'm not speaking
Speaker 4: from a legal perspective, but for a normal person, what
Speaker 4: constitutes an act that sexual in nature can vary a lot.
Speaker 4: A sexualized act versus an actual act of you know,
Speaker 4: some kind of sex very different. Also, the relative positions
Speaker 4: in which a sexual act is carried out, where certain
Speaker 4: body parts may be obscured but other things may be clear.
Speaker 4: All that causes machine learning algorithms to detect deep fakes
Speaker 4: to have a lot of problems. So I think those
Speaker 4: are some of the challenges from a technical perspective in
Speaker 4: figuring these things out.
Speaker 3: So one analyst has said that some deep fake sites
Speaker 3: now offer APIs to people creating non consensual images and
Speaker 3: video generators explain what that means.
Speaker 4: So, when you go to a website, you're typing stuff
Speaker 4: in a prompt or a query, or clicking on some
Speaker 4: buttons and you get a response. An API is an
Speaker 4: application program interface, and it replaces you with a piece
Speaker 4: of computer code, and that piece of computer code is
Speaker 4: interacting in a different kind of language with the server
Speaker 4: behind the website you're going to. And so if somebody
Speaker 4: is using an API to generate deep fakes, that suggests
Speaker 4: that they're generating it at scale. That suggests that they
Speaker 4: have a piece of code that is shipping requests automatically
Speaker 4: through that piece of code to the server, and that
Speaker 4: server is sending back whatever they requested, in this case
Speaker 4: some kind of sexualized image. That suggests somebody who's trying
Speaker 4: to do it at scale, who is, you know, who
Speaker 4: probably has his or her own databases of people they
Speaker 4: want to propose in these situations that they will never
Speaker 4: in to begin with.
Speaker 3: Indicators Analysis also estimates that the new toifier economy may
Speaker 3: be worth up to thirty six million dollars a year. Laurie,
Speaker 3: and you're reporting on mister deep fakes. You noted the
Speaker 3: financial incentives for the community members posting in the forum
Speaker 3: tell us about the incentives and talk about the money
Speaker 3: making Laurie here, and for the platforms that host this
Speaker 3: content too, although for people don't realize. Goo Goal says
Speaker 3: it doesn't allow apps to contain sexual content. Apple says
Speaker 3: the company's apps to or prohibits overtly sexual content. Both
Speaker 3: companies disabled searches for new defying their app stores. We'll
Speaker 3: get to the workarounds in a second, but talk a
Speaker 3: little bit aout the incentives, the financial incentives.
Speaker 2: It's interesting because when we were looking at this, this
Speaker 2: site that wasn't a site that was a platform. It
Speaker 2: was a whole ecosystem that enabled the site to exist,
Speaker 2: and that included platform liability from a lot of these
Speaker 2: different companies. So for example, you know, the incentive was
Speaker 2: you can go on not only can you view you know,
Speaker 2: thousands of videos of sexually explicit deep fakes. If your
Speaker 2: favorite celebrity, you can make your own. And this is
Speaker 2: why we believed mister deep Fakes was so dangerous. He
Speaker 2: was dangerous because he was creating an army of mister
Speaker 2: deep fakes. Because now there are all these creators on there,
Speaker 2: which is kind of pay to play, and there's a
Speaker 2: freemium model, and then there's this premium model where you
Speaker 2: can pay to have specialized deep fakes of public figures
Speaker 2: who didn't consent and made and so people were probably
Speaker 2: making more money than the guy behind mister deep fakes
Speaker 2: off of this platform. It was creating a whole ecosystem.
Speaker 2: But what else enabled that ecosystem? First, I would go
Speaker 2: from a regulatory standpoint, the laws just hadn't caught up,
Speaker 2: and we saw this with non consentual pornography. Oftentimes tech
Speaker 2: moves quickly when there are the correct guardrails. Women and
Speaker 2: young folks are the first ones to feel that pain.
Speaker 2: But also there were payment providers that were accepting payments
Speaker 2: on this platform, and so it was all of these things,
Speaker 2: and we talk about these small steps that actually helped
Speaker 2: enable this ecosystem to exist. And one of the coolest
Speaker 2: things we saw because I think so many folks would say, well,
Speaker 2: it's a game of whack a mole. You go after
Speaker 2: this guy and then there are all these other ones
Speaker 2: that are going to pop up, But actually what ended
Speaker 2: up happening is we did go after this guy. We
Speaker 2: found out who he was, and I think maybe most
Speaker 2: alarming was he wasn't a technical guy. This is a
Speaker 2: guy who was, i would say, in his thirties, worked
Speaker 2: at a pharmacist at a hospital, helping people, was newly married,
Speaker 2: had a young child, and he had been able to
Speaker 2: create a website that enabled a whole new wave.
Speaker 1: Of deep fake abuse because.
Speaker 2: He had an interest in technology and deep fakes, and
Speaker 2: he saw an opening because when Reddit banned sexually explicit
Speaker 2: deep fakes, we were able to track where his username
Speaker 2: was talking about seeing an opportunity that could be profitable
Speaker 2: and getting other people to build on this. And so
Speaker 2: all of these things together created i would say, an
Speaker 2: ecosystem that allowed it. And all of these things also
Speaker 2: once people started chipping away at this, once the laws
Speaker 2: started slowly but surely catching up, once Google deranked the platform,
Speaker 2: there were all sorts of small steps that actually created
Speaker 2: a friction that made it harder to exist.
Speaker 3: So let's shift to the current laws and regulations surrounding
Speaker 3: deep fake porn, including the Take It Down Act, the
Speaker 3: law that makes it a federal crime to publish non
Speaker 3: consensual explicit deep fakes. The federal government began enforcing it
Speaker 3: in May, Carrie, in your pursuit of tech platform accountability.
Speaker 3: You've cut up against section two thirty. Have many the
Speaker 3: law that protects platforms for being held legally liable for
Speaker 3: content that users post. You mentioned product liability law as
Speaker 3: a way to stop deep fake porn and get justice
Speaker 3: for victims. So talk about using product liability law.
Speaker 5: Section two thirty was intended to basically just immunize platforms
Speaker 5: for content that third parties created and posted. But when
Speaker 5: it comes to deep fakes, especially like in the situation
Speaker 5: of Groc, we're saying that Grock was the one who
Speaker 5: created this material and also published it, and therefore section
Speaker 5: two thirty shouldn't apply at all. But even if it does,
Speaker 5: by suing them under product liability, we're suing them not
Speaker 5: for publishing the content, but for you know, basically harming
Speaker 5: people through its defective design, defective manufacturing.
Speaker 3: These were all foreseeable uses.
Speaker 5: There's no reasonably safe use for new toifying product, and
Speaker 5: so that's just like a second way to overcome section
Speaker 5: two thirty. Now, XAI, they haven't fully exposed what their
Speaker 5: defenses are going to be in our case, but when
Speaker 5: they were opposing the temporary restraining order, they were claiming
Speaker 5: that they wanted. I mean, so far, this would be
Speaker 5: the only AI product that is saying that Section two
Speaker 5: thirty should apply to it and that they're not responsible
Speaker 5: for what people type in as prompts. But even more
Speaker 5: alarming is that they're claiming that they should have the
Speaker 5: First Amendment protections and that rock should basically be protected
Speaker 5: under free speech, which is a pretty alarming idea. You
Speaker 5: know that AI and a chatbot would have free speech rights.
Speaker 3: They don't.
Speaker 5: You know, so far, we're not at a point where
Speaker 5: we're giving constitutional rights to a chatbot. But again, the
Speaker 5: arguments haven't yet hit this the stage where there's been
Speaker 5: fully brief yet.
Speaker 3: So vas talk about workarounds because a lot of these
Speaker 3: these companies are very clever in terms of workarounds. Porn
Speaker 3: people have always been very clever. How easy is it
Speaker 3: to do workarounds here from a technological point of view?
Speaker 4: You know, I was thinking about this when Lori was
Speaker 4: speaking earlier Kara. You know, she talked about a website
Speaker 4: where somebody goes in and says, I want to create
Speaker 4: a deep fake of such and such person doing such
Speaker 4: and such thing. And so that's a classic example off
Speaker 4: a place where that website is using the APIs that
Speaker 4: you asked about earlier to access a cloud provider where
Speaker 4: this code running degenerate the newtification of the sort designed
Speaker 4: by whoever asked the query. But now think about it
Speaker 4: this way. Let's say the platform has the mechanism. Let's
Speaker 4: say they have a perfect mechanism that doesn't exis yes today,
Speaker 4: but suppose they did to detect the deep fake video
Speaker 4: of some form of sexual activity. Then the workarounds for
Speaker 4: the porn guys that are some simple ones. So one
Speaker 4: of the classic methods to generate deep fakes, regardless of
Speaker 4: whether it's for good or bad, is something called stable diffusion.
Speaker 4: So in stable diffusion, what you do is you, let's say,
Speaker 4: throw in an image. I'll use an image rather than
Speaker 4: a video as my example, but you throw in an
Speaker 4: image of somebody and that image gets converted to some
Speaker 4: very coarse representation. You know, think of this as something
Speaker 4: that's a very skeletal version of that image, and then
Speaker 4: what ends up happening is that skeletal version goes through
Speaker 4: the neuification process, but at the skeletal level, so that's
Speaker 4: skeletalized neutified image doesn't look anything like what a real
Speaker 4: nude might look like, and so you can think of
Speaker 4: it as some kind of computationally weird representation of, you know,
Speaker 4: a person who's about to be sexually abused via a
Speaker 4: deep fake. And then they could do all this on platform,
Speaker 4: on a cloud, and then without ever passing the source
Speaker 4: image or the final image, they bring back this course
Speaker 4: representation of the new deified image and perform the last
Speaker 4: step back on their own servers, where it generates the
Speaker 4: final nude. So that's an example of one way in
Speaker 4: which somebody with you know, just limited computer science knowledge,
Speaker 4: who's familiar with the code, who has access to all
Speaker 4: this code, can generate something while avoiding the guardrails of
Speaker 4: some of the platforms, even if those guardrails are close
Speaker 4: to perfect.
Speaker 3: We'll be back in a minute. The Defiance Act, which
Speaker 3: is aimed at allowing victims of non consensual deep figs
Speaker 3: to suit for damages, passed in the Senate, and it's
Speaker 3: currently stalled in the House. Even though some states have
Speaker 3: criminalized deep figs, enforcement remains low. Laurie, What extent are
Speaker 3: these laws effective when it comes to stopping or at
Speaker 3: least curbing the spread. It's happened before with credit card
Speaker 3: companies and certain businesses. You know, things tend to slow
Speaker 3: things down. But in this case, how important is it
Speaker 3: to have the Take It Down and the Defiance Act?
Speaker 2: I'll answer, and I'm curious for Carry's response too, but
Speaker 2: I think it's incredibly important. You know, the Take It
Speaker 2: Down Act, you know, force tech companies to pay attention
Speaker 2: and quicker. It gives them forty eight hours to take
Speaker 2: down non consensual abuse. But oftentimes when it comes to victims,
Speaker 2: it's really difficult to find recourse for this, you know,
Speaker 2: And so the Defiance Act would create civil penalties that
Speaker 2: would enable them to actually have the ability to go
Speaker 2: after the folks who have done this and have civil recourse,
Speaker 2: which I think makes a big difference. And when we
Speaker 2: started out on our search for mister Deep Fakes, which was,
Speaker 2: let's just be honest, really a search for asking why
Speaker 2: is this allowed to happen? You know, there weren't a
Speaker 2: lot of state laws, and I will say a lot
Speaker 2: of the state laws and a lot of the victims
Speaker 2: and survivors we spoke to. Unfortunately, the onus in many
Speaker 2: ways becomes on these survivors to speak out about it
Speaker 2: and help change laws, and they did in the state level,
Speaker 2: which you know, the state laws changing all of them,
Speaker 2: varying in scope. Civil criminal penalties actually were where a
Speaker 2: lot of these different victims initially had recourse. Which is
Speaker 2: also scary to watch as folks go after regulation at
Speaker 2: the state level because oftentimes the federal government is just
Speaker 2: slow to move. I would say, I hope that the
Speaker 2: Defiance Act passes, it can get through. I think this
Speaker 2: is a bipartisan issue because it's really a child safety
Speaker 2: issue as well.
Speaker 3: Right, which is how they're saying it. And so, by
Speaker 3: the way, Wired investigation found that GROC is still being
Speaker 3: used for non consensual explicit content, this despite regulatory scrutiny
Speaker 3: and multiple lawsuits, and after Musk's XAI said it would
Speaker 3: add restrictions in response to an NBC investigation, the x
Speaker 3: safety account post, do we strictly prohibit users from generating
Speaker 3: non consensual explicit deep fakes and from using our tools
Speaker 3: to undress real people? Explain how some of these legal
Speaker 3: dynamics apply to XAI in that regard. Since you're in
Speaker 3: a lawsuit with them, when it keeps showing up that
Speaker 3: they're doing the same thing, you would accuse them of Well.
Speaker 5: I mean that statement that they say that it's against
Speaker 5: their regulations and they're basically passing the blame to users.
Speaker 5: There's nothing in that statement that suggests that they are
Speaker 5: modifying their product so that deep fakes are no longer
Speaker 5: easy to create. And the problem with things like to
Speaker 5: take it down act it's great, you know, suddenly now
Speaker 5: there's a forty eight hour window of time that tech
Speaker 5: platforms are supposed to remove illegal content, But it has
Speaker 5: no teeth because victims can't enforce it. Victims can't soothe
Speaker 5: the platform and say, hey, you know, it's now been
Speaker 5: forty eight days and that content is still up. So
Speaker 5: the problem with all regulations and all these options is
Speaker 5: that the victims cannot enforce them. And so when we're
Speaker 5: talking about who's the bad guy? VS has talked about
Speaker 5: the bad guys in these situations, in my view, the
Speaker 5: worst guys are the platforms. And so until we have
Speaker 5: regulations that put the victims in the driver's seats to
Speaker 5: not just you know, get criminal justice against the individuals,
Speaker 5: but to actually get justice against the platforms, you know,
Speaker 5: I think the only recourse we have is to use
Speaker 5: our courts and make these companies pay.
Speaker 3: So every episode we get a question from an outside expert.
Speaker 3: Here's yours, and you're all going to answer it, So
Speaker 3: let's listen to it first. Hi.
Speaker 8: I'm Georgia Wells, a tech reporter for The Wall Street Journal.
Speaker 8: I just finished a deep dive into the issue of
Speaker 8: teens using deep fake nudes to harass each other, and
Speaker 8: so the big question I would ask is that, now
Speaker 8: that the government has just started enforcing the Take It
Speaker 8: Down Act, how might this change the harassment landscape for
Speaker 8: young people today?
Speaker 3: Thank you, Laura, you go first, then Carrie then VS. Yeah.
Speaker 2: I do agree a little bit with Carrie on saying,
Speaker 2: you know, there's only so much you can actually kind
Speaker 2: of enforce when the onus is on the victim. I
Speaker 2: do think from a cultural standpoint, it is very different
Speaker 2: than it was a couple of years ago of federal
Speaker 2: law Take It Down Act. Again, state laws are at
Speaker 2: least sending this message that the stakes will be high
Speaker 2: and that young boys, young teenagers can get into real
Speaker 2: trouble for doing this and can get charged. I mean,
Speaker 2: I think that at least from a perspective of schools
Speaker 2: being able to speak to students, parents being able to
Speaker 2: speak to their children about this, and having some structure
Speaker 2: will be helpful again, though I'm not sure the Take It.
Speaker 1: Down Act is going to be the thing that's going
Speaker 1: to actually change this.
Speaker 2: I also agree with Kerrie on platform liability in creating
Speaker 2: high stakes for the tech companies and the whole ecosystem
Speaker 2: that enables this to happen.
Speaker 5: Kerrie, So I think they'll get the memo that this
Speaker 5: is illegal. I mean, having criminal laws was really really
Speaker 5: effective when we were dealing with the scourge of image
Speaker 5: asexual abuse. Over the last decade. We went from having
Speaker 5: three states to having fifty states with those laws, and
Speaker 5: my firm saw such an enormous and quick plummet of
Speaker 5: offenders creating images like that. So that's great, But we
Speaker 5: still don't have an apparatus that targets the platforms themselves
Speaker 5: and prevents them from basically monetizing this kind of humiliation.
Speaker 5: And until we have laws that victims can enforce themselves, like,
Speaker 5: it's not really harming they who I consider the true offenders.
Speaker 3: Right, which are the platforms themselves, all right? Vs.
Speaker 4: You know, I have very mixed feelings about criminalizing a
Speaker 4: thirteen year old, all right, But at the same time,
Speaker 4: we need to protect perhaps other thirteen year olds or
Speaker 4: fourteen year olds or ten year olds from the consequences
Speaker 4: that accrue to them because of some thirteen year old
Speaker 4: putting out deep porn of that kid. So I think,
Speaker 4: you know, a first step has got to be something
Speaker 4: around education. A lot of our teachers in schools are
Speaker 4: simply not teaching their kids about deep fakes. And it's
Speaker 4: one thing if kids are using deep fakes or generative
Speaker 4: AI to you know, create reports for their class projects.
Speaker 4: I mean that brings up a whole bunch of other
Speaker 4: ethical issues. But it's another thing if they use deep
Speaker 4: fakes to explicitly cause damage to a kid and one
Speaker 4: of their classmates. So I think the ethics around us
Speaker 4: has got to be clearly or take in class and
Speaker 4: by schools. And some of us schools may have done
Speaker 4: a good job of this, others may not. And you
Speaker 4: know that education and the consequences of those actions in
Speaker 4: a school context are clear. I think it'll be challenging.
Speaker 2: You know, viaz just to say to respond if it's helpful,
Speaker 2: just to respond to that. I think the frustration is
Speaker 2: so many times, I agree, how do we expect thirteen
Speaker 2: year old boys to know that this isn't okay when
Speaker 2: they're getting served up neuification apps on social media.
Speaker 1: You know, I think it's it is.
Speaker 2: It's an end like two things can be true, and
Speaker 2: so there's a huge gap between education and being able
Speaker 2: to educate folks on this and what the platforms are
Speaker 2: allowing and what they say they're allowing and what's coming through.
Speaker 1: It creates a really difficult environment.
Speaker 4: You know, I worry generally about the broad statement or
Speaker 4: the broad platforms Moniker. I think there's a wide variety
Speaker 4: in what different platforms are doing. Some platforms are going
Speaker 4: to a great amount of trouble, spending a tremendous amount
Speaker 4: of money to try and perform the best innovation possible
Speaker 4: so they can put some of these problems to bed,
Speaker 4: and others are doing much much less. So there's like
Speaker 4: a huge range of what platforms are doing, some doing
Speaker 4: relatively little, some making every effort to try and get there.
Speaker 4: Now I want to add that, you know, again, even
Speaker 4: if platforms who are doing their very best, there's a
Speaker 4: famous saying, due to a statistician at the University of Wisconsin,
Speaker 4: all models are wrong, some are useful, all right, And
Speaker 4: so whatever models are being used by platforms to find
Speaker 4: these deep fakes. They're never going to be perfect, and
Speaker 4: so the question is are they making the effort? Some
Speaker 4: are making what I would consider a very good faith effort.
Speaker 3: Some who is doing a good job.
Speaker 4: I'd prefer to stay out of that, but I don't
Speaker 4: want to talk about specific platforms.
Speaker 3: But yeah, so let's end by looking at what individuals
Speaker 3: can do as deep fake technology continues to evolve. Carrying
Speaker 3: now that the Take It Down Act forces tech companies
Speaker 3: to remove deep fake nudes, how do you request a
Speaker 3: takedown if you're targeted? Well, part of what.
Speaker 5: The Takedown Act required was that there be a flow
Speaker 5: for people to easily request content removal, and so that
Speaker 5: kind of infrastructure I really appreciated that being incorporated into
Speaker 5: the laws. So ideally, you know, a victim within the
Speaker 5: platform itself can now report it, you know, whether it's
Speaker 5: through a URL or if it's like a Instagram or Facebook,
Speaker 5: then the image itself would ideally have a method to
Speaker 5: request removal.
Speaker 3: When digital forensics professor and deep fake expert Hani Farred
Speaker 3: was on the podcast in January, he expressed concerns that
Speaker 3: agentic AI will supercharge the creation of deep figs in
Speaker 3: the coming months vs. How do we make sure the
Speaker 3: new laws policies don't block positive uses in these tools.
Speaker 4: So I think, again, if we don't want to block
Speaker 4: the technology, we want the technology to evolve. Because again,
Speaker 4: as Laurie said much earlier in the podcast today, you know,
Speaker 4: deep fakes do have the capacity to help do things
Speaker 4: like better understand disease. I'm not going to call this
Speaker 4: a deep fake, but the generative AI is being used
Speaker 4: to create new kinds of proteins that will help cure diseases.
Speaker 4: Generative AI is being used to create materials that will
Speaker 4: degrade and perform in ways that we would like, that
Speaker 4: would protect the environment. And more so, I think what
Speaker 4: we have to do is to look at intent. You know,
Speaker 4: what is the intent of the individual who is creating
Speaker 4: a deep fake. And in the case of the thirteen
Speaker 4: year old boy or fourteen year old boy who's creating
Speaker 4: deep fakes of one of his classmates because he's pissed
Speaker 4: off about something, I think they're there's intent. Whereas if
Speaker 4: somebody you know, there was this guy who created a
Speaker 4: deep fake of the Pope but Francis in puffy robes
Speaker 4: and a very dapper looking version of him. You know
Speaker 4: that I think is something that's laughable, and I find
Speaker 4: that at least I find that somewhat entertaining. I doubt
Speaker 4: very much of the Pope was, you know, horribly offended.
Speaker 4: And so, you know, I think we want to allow
Speaker 4: artistic freedom and expression scientific inquiry at the same time
Speaker 4: regulate so that it's the uses that are banned or
Speaker 4: that lead.
Speaker 3: To the individual intent one. Yeah, Laurie, In the end,
Speaker 3: you track down mister deep fakes, what ultimately brought down
Speaker 3: the site? And where is he now?
Speaker 1: Great question? Where is he now? I would love to know.
Speaker 2: So eventually we did track him down with the help
Speaker 2: of you know, investigative journalists the internet and some of
Speaker 2: our cybersecurity friends who did some pro bono work for
Speaker 2: us because they have kids and they viewed this as
Speaker 2: a threat. And we were able to track him down,
Speaker 2: and you know, Carol, like I'll never forget confronting him
Speaker 2: on his way into the office. And I do remember
Speaker 2: a Carrie who I've known because I've covered nonconsensual pornography before.
Speaker 2: I called her before going to his parents' home to
Speaker 2: make sure that we did everything correct, just in case,
Speaker 2: you know, we didn't want to create any legal liabilities.
Speaker 2: But I will never forget him looking at me. He
Speaker 2: knew who I was with so much disdain. And I
Speaker 2: know this is a delusional part of me, but there
Speaker 2: was a part of me that thought maybe we would
Speaker 2: have tracked this guy down and we could show up
Speaker 2: and say, do you understand the real harm this site
Speaker 2: is causing? You're a father, And I remember he walked
Speaker 2: right into the hospital. He wouldn't say a word to me,
Speaker 2: and the doors closed, and I remember thinking at that point, well,
Speaker 2: we're gonna have to go talk about this to everyone
Speaker 2: who will listen to us, and that has to change
Speaker 2: because we have to change culture as a part of this.
Speaker 1: This can't be where the story ends.
Speaker 2: He can't just take himself off social media, which he
Speaker 2: did and continue running this site. And so a lot
Speaker 2: of things happened, which I think, think we're really exciting.
Speaker 2: I mean we I think were one of the first
Speaker 2: to show up and threaten his anonymity, so you cannot
Speaker 2: do this type of abuse and remain anonymous.
Speaker 1: So that was friction. That exposure was friction.
Speaker 2: A laws started changing when in the UK they started
Speaker 2: talking about different types of laws that would make it
Speaker 2: harder for folks to access mister deep Fix. They caught
Speaker 2: off access to folks in the UK. That created friction.
Speaker 2: A very valuable server went down for them to take
Speaker 2: it down. At past the CBC published their investigation as well.
Speaker 2: Mister deep Fix finally went offline for good. And I
Speaker 2: think it was such a great lesson in all of
Speaker 2: these players that actually impact how we build out a
Speaker 2: better future. And the headline is friction. It just created
Speaker 2: more and more friction. Now what's happened to him, I
Speaker 2: think is actually is just as important of a headline,
Speaker 2: which is nothing really. I mean, he lost his job
Speaker 2: at the pharmacy. But we've talked to many folks law enforcement.
Speaker 2: I actually spoke to a couple different folks in Canada.
Speaker 2: From the law enforcement perspective, there's no open investigations into
Speaker 2: David Doe. That's his name, and there's thoughts that maybe
Speaker 2: he's left the country. But that's it and I think
Speaker 2: that in and of itself tells its own story absolutely.
Speaker 3: So the last question dve Forne obviously isn't going away.
Speaker 3: So what's the best possible outcome each of you, what
Speaker 3: is your most optimistic case for how it pans out
Speaker 3: in the next five years and how we get there. Carrie,
Speaker 3: you go first, and then VS and Laurie you get
Speaker 3: the last word. Carrie.
Speaker 5: Okay, Well it should probably come as no shock too
Speaker 5: that my best case outcome is that the companies that
Speaker 5: unleashed these harms and profited significantly from them should have
Speaker 5: to pay and there should be disgorgement and compensation for
Speaker 5: all the victims, and it should be financially penalizing for
Speaker 5: these companies.
Speaker 3: So that's would be your best And is that a
Speaker 3: possible outcome? Do you feel like that's a possible outcome.
Speaker 5: I mean, we're on that road with litigation. We'll see
Speaker 5: how it works. I also, you know, I do condemn
Speaker 5: the platforms like the App Store and Google Play who
Speaker 5: for many many years were offering and selling newdifying apps
Speaker 5: and they just kind of were ostriches with their head
Speaker 5: in the sand, pretending like they didn't think these things
Speaker 5: were harmful. I'm really sick and tired of platforms not
Speaker 5: doing anything until something becomes actually criminal. You know, they
Speaker 5: deny that it's harmful until there's an actual criminal law VS.
Speaker 4: You know, that's what I would like in place, some
Speaker 4: kind of criminal law that says, well, look, here are
Speaker 4: the kinds of behaviors, and even intended behaviors are intended
Speaker 4: outcomes that should be criminalized. Here are the penalties that
Speaker 4: you will face if you engage in these acts. And
Speaker 4: this needs to be something that's at least in the
Speaker 4: US nationwide. But more importantly, many of these actors are
Speaker 4: going to be overseas. And so what we need is
Speaker 4: not just US law, but law in various countries and
Speaker 4: some broad international agreements, perhaps enforced by an organization like Interpol,
Speaker 4: which polices things like child grafficking already, they should be
Speaker 4: authorized to do and investigate things like this.
Speaker 3: That's a very good point. That's exactly where it goes
Speaker 3: as broad is what happens is immediately it goes abroad.
Speaker 2: Laurie, last word, Look, I agree with what everyone here
Speaker 2: has said, So I will end with saying, you know,
Speaker 2: my gripe with Silicon Valley, and I have a lot
Speaker 2: of gripes with Silicon Valley, is they're sitting there talking
Speaker 2: about AGI and what happens when AI becomes smarter than us,
Speaker 2: and it doesn't matter, It really doesn't matter, because here's
Speaker 2: what's happening.
Speaker 1: Here on Earth.
Speaker 2: Over here in reality, we have children who are ending
Speaker 2: their lives because of deep fake sex stortion, and people
Speaker 2: are sleeping on it. My hope is that I don't
Speaker 2: have to shout into the ether for years and say
Speaker 2: this is real harm, this is real home. That we
Speaker 2: can get better education, that we can change culture to
Speaker 2: understand that when new technology comes quickly without the correct guardrails,
Speaker 2: the folks who are impacted oftentimes are the ones that
Speaker 2: don't have the voice, and we can make some changes
Speaker 2: from there. No, it doesn't take, to Carrie's point, Oftentimes
Speaker 2: it takes criminal law if I'm putting sitting in my seat.
Speaker 2: Oftentimes it takes public pressure and investigations. I'd love to
Speaker 2: get to a world where Silicon Valley maybe has some
Speaker 2: different folks in it. And this isn't the last thought
Speaker 2: as people are having dinner and talking about AGI after
Speaker 2: taking a cold plunge.
Speaker 1: That's it.
Speaker 3: That's a very good point. But you're going to wait
Speaker 3: a long time for that. Just thank you, But I
Speaker 3: got manifested, go ahead, all right? Sure? Why not? Why not? Anyway?
Speaker 3: I really appreciate it, all three of you. It's a
Speaker 3: really important talk, and we're going to keep coming back
Speaker 3: to it again and again and hopefully see something happen
Speaker 3: by raising awareness of it. For sure. Thank you so much, all.
Speaker 4: Three of you, Thank you, Ker pleasure to be here.
Speaker 3: Thank you so much. Thanks. We've reached out to XAI
Speaker 3: and x for comment. They did not respond. Today's show
Speaker 3: was produced by Michelle Alloy, Katherine Millsop, Madeline LaPlante, Doobie,
Speaker 3: and Kaylin Lynch. Special thanks to Lissa Soap, Anika Robbins
Speaker 3: and Julia Sharp Levine. Our engineers are Fernando Aruda and
Speaker 3: Rick Kwan, and our theme music is by Trackademics. Nishat
Speaker 3: Kerwa is Vox Media's executive producer of podcasts. Go Wherever
Speaker 3: you listen to podcasts, search for On with Kara Swisher
Speaker 3: and hit follow. Thanks for listening to On with Kara
Speaker 3: Swisher from Podium Media, New York Magazine, the vox Media
Speaker 3: Podcast Network and US. We'll be back on Monday with
Speaker 3: more