Speaker 1: Pushkin. Earlier this year, an employee working in Hong Kong
Speaker 1: for an international company got a weird message from one
Speaker 1: of his colleagues. He was supposed to make a secret
Speaker 1: transfer of millions of dollars. It seems sketchy. It obviously
Speaker 1: seems sketchy, so he got on a video call with
Speaker 1: a bunch of people, including the company's CFO, the chief
Speaker 1: financial officer. The CFO said the request was legit, so
Speaker 1: the employee did what he was told. He transferred roughly
Speaker 1: twenty five million dollars to several bank accounts. As it
Speaker 1: turned out, the CFO on the video call was not
Speaker 1: really the CFO. It was a deep fake, an AI
Speaker 1: generated twin created from publicly available audio and video of
Speaker 1: the real CFO. By the time the company figured out
Speaker 1: what was going on, it was too late, the money
Speaker 1: was gone. I'm Jacob Goldstein and this is What's Your Problem,
Speaker 1: the show where I talk to people who are trying
Speaker 1: to make technological progress. My guest today is Ali Shahieri.
Speaker 1: He's the co founder and chief technology officer at the
Speaker 1: audaciously named Reality Defender. Ali's problem is this, how can
Speaker 1: you use AI to protect the world from AI? More specifically,
Speaker 1: how do you build a set of models to spot
Speaker 1: the difference between reality and AI generated deep fakes. How'd
Speaker 1: you get into the defending reality business?
Speaker 2: Yeah, so when I started, it was around actually generating
Speaker 2: videos and deep fikes.
Speaker 1: So you were attacking reality before you were defending it.
Speaker 2: I wouldn't said we were attacking anything, but we were
Speaker 2: definitely into looking into this technology. And it is way
Speaker 2: back before all this stuff kind of went crazy. This
Speaker 2: is back in like twenty nineteen around that time, So
Speaker 2: we were building digital twins and we're looking at how
Speaker 2: do you make it so that it looks realistic? Is
Speaker 2: it a cartoon looking thing? Is it like a unity
Speaker 2: three D thing? And then that's when we started to
Speaker 2: see like these early research papers where they were taking
Speaker 2: like someone's face and putting it on a video and
Speaker 2: blending it in and it looked really good, and we
Speaker 2: were like, oh, maybe we can do the digital twins
Speaker 2: that way. And while we were like in that business,
Speaker 2: we were like, you know, probably in a few years
Speaker 2: someone can download an app and just make anything very easily.
Speaker 2: And that's kind of the origins of how how we started.
Speaker 2: We're very mission driven. What we're trying to do here
Speaker 2: is really protect the world and people from the dangers
Speaker 2: of AI, but in a way where you know, we
Speaker 2: want people not to abuse the technology. We're very we
Speaker 2: love AI, we just don't want it to be abused.
Speaker 1: So let's talk about this sort of deep fake detection
Speaker 1: kind of you know, jen AI detection market more generally,
Speaker 1: like who's like, who's selling deep fake detection right now,
Speaker 1: and who's buying what's the what's the sort of market
Speaker 1: landscape look like.
Speaker 2: The type of clients that we have right now are banks.
Speaker 2: For example, we are currently live with one of the
Speaker 2: largest banks in the world. When you call that bank,
Speaker 2: the audio goes through our defake detection models and we're
Speaker 2: able to tell the call center this person might be
Speaker 2: a deep fake. And part of that is that's actually happened.
Speaker 2: Someone's called the bank and they've transferred money out and
Speaker 2: actually this this goes back to twenty nineteen, so the
Speaker 2: first incident of defake fraud actually and back in.
Speaker 1: Two in nineteen that we're aware of. Right, you're right exactly,
Speaker 1: So what happened in twenty nineteen.
Speaker 2: Yeah, so this is back where this is early and
Speaker 2: nobody really knew about this, and there was a CEO
Speaker 2: that called a smaller company that THEO was a parent
Speaker 2: company calling the child company. The CEO calling the other
Speaker 2: CEO and he wanted to transfer some money out and
Speaker 2: it sounded like him and the guy transferred I think
Speaker 2: it was in UK, about two hundred and three hundred
Speaker 2: thousand dollars out And that was like the first one
Speaker 2: of the first ones that we.
Speaker 1: Know of, and they got away with it, I believe.
Speaker 2: So. Yeah.
Speaker 1: And there was an instance earlier this year right where
Speaker 1: I think it was in Hong Kong and some employee
Speaker 1: was on a zoom call with the company's CFO and
Speaker 1: the CFO was like, you know, why are twenty five
Speaker 1: million dollars or something to some bank account? And then
Speaker 1: the employee did it and it turned out the CFO
Speaker 1: on the call was a deep fake, right.
Speaker 2: Yeah, So fast were they your client? They were not
Speaker 2: in our clients unfortunately. But this shows the how quickly
Speaker 2: the technology is evolving. You know, twenty nineteen audio fast
Speaker 2: forward a few years now, You've got a zoom call.
Speaker 2: There'd a bunch of people on it and they all
Speaker 2: look like people, you know, I know, they're all de fis.
Speaker 1: So you were starting to mention. Banks are some of
Speaker 1: your main clients. Who are some of your other main.
Speaker 2: Clients, media companies, I think think of some of the
Speaker 2: big ones, there is our product this year, especially with
Speaker 2: the election. You know, back twenty twenty, we thought it
Speaker 2: would be a problem. It wasn't. This year we think
Speaker 2: is a big problem. For sure. I think we were early,
Speaker 2: but it's already this is happening everywhere even this year.
Speaker 2: This year is the largest election year in the world.
Speaker 2: More than fifty percent of the people are voting, and
Speaker 2: we already have documented cases of election issues with the fix.
Speaker 1: Okay, media companies, banks, any other kind of big categories
Speaker 1: of clients.
Speaker 2: Yeah, so other ones are government agencies. But in the end,
Speaker 2: everyone we think, we believe everyone needs this Product's not
Speaker 2: It shouldn't be up to the people to decide or
Speaker 2: figure out if something's a deepic. If you're on the
Speaker 2: social media platform, you shouldn't have to figure out, hey,
Speaker 2: is this person real or not. It should just be
Speaker 2: built in and anyone should be able to use it.
Speaker 1: Well. Our social media companies either buying or building deep
Speaker 1: fake detection tools or do they want to like stay
Speaker 1: out of that business and be like no, we don't
Speaker 1: want to be in the business of saying yes, this
Speaker 1: is real, no, this isn't real.
Speaker 2: I can tell you we've been in contact and have
Speaker 2: talked to some social media platforms. I think one issue
Speaker 2: is they don't have to flag these things. It's up
Speaker 2: to them, right, there's not a lot of regulation, so
Speaker 2: I know they're thinking about it. We've chatted with some,
Speaker 2: but that's the extent of it.
Speaker 1: So okay, So let's talk about how it works. And
Speaker 1: there's two ways that I want to talk about how
Speaker 1: it works. So one is from the point of view
Speaker 1: of the user, whoever that may be, and then the
Speaker 1: other is sort of what's going on under the hood. Right,
Speaker 1: So let's start with the point of view of the user.
Speaker 1: If I'm a whatever, a bank, university, a media company
Speaker 1: who is paying for your service, how does it work
Speaker 1: for me?
Speaker 2: Depends on exactly the user and the use case. If
Speaker 2: let's say it's a media company, Uh, they're looking at
Speaker 2: maybe filtering through a lot of content, so content moderation.
Speaker 2: Actually that would be like a social media company. They're
Speaker 2: looking at content moderation. Maybe they want they're looking at
Speaker 2: millions of assets and they want to quickly flag those
Speaker 2: things if they were in that business. Uh, the bank
Speaker 2: there For the example I gave the issue, someone could
Speaker 2: call and biometrics fail. By the way, if you call
Speaker 2: a bank, some banks say repeat after me, your my
Speaker 2: voice is my passport? That actually fails. Now what do
Speaker 2: you think? So a bank wants to make sure the
Speaker 2: person calling in is actually that person. This is more
Speaker 2: relevant to more to private banking, where there's actually a
Speaker 2: one on one relationship between the client and the bank.
Speaker 1: And so in that case, So let's take that case.
Speaker 1: So in that case, someone calls in and talks to
Speaker 1: their banker. They're a rich person who has a private banker.
Speaker 1: Basically it's what you're talking about, right, So this rich
Speaker 1: person calls in and talks to their private banker, and
Speaker 1: it is the system just always running in the background
Speaker 1: in that case, And like, how does it work from
Speaker 1: the point of view of the of the private banker.
Speaker 2: Sure, and I have to be careful what I say here,
Speaker 2: But the high level is the models are listening and
Speaker 2: if they detect a potential deep fake, they will the
Speaker 2: call center. That person will get a notification so is
Speaker 2: integrated into their existing workflow. They'll get a notification that says, hey, this.
Speaker 1: Person get like a text or a slack or something
Speaker 1: they're using. You're talking to a deep fake.
Speaker 2: No, they're using software for the bank they're using they're
Speaker 2: still using a software and there's a dashboard. In that scenario,
Speaker 2: they do they ascalate, so they might say, let me
Speaker 2: ask you some more questions or let me call you back.
Speaker 1: Huh. Let me call you back is a super safe one,
Speaker 1: right because if they have a relationship, probably they know
Speaker 1: the number. They just call them back. Yeah, absolutely, okay,
Speaker 1: And then how does it work? How does it work
Speaker 1: for like when you say, like I presume by the
Speaker 1: way that you can't name your clients. You said a
Speaker 1: media company and a bank. It's it's secret that they're.
Speaker 2: Yeah, we're not allowed to okay.
Speaker 1: So let's say a media company. How's it work for
Speaker 1: a media company?
Speaker 2: Their their use case is slightly different, especially right now,
Speaker 2: as I mentioned around the election, So there there might
Speaker 2: be something that that's starting to go viral in the
Speaker 2: news and they want to check, hey, is this a
Speaker 2: real or not? I would like to say like something
Speaker 2: like this is usually when something goes viral, the damage
Speaker 2: is already ton.
Speaker 1: Yes, although if you're if you're whatever, the New York
Speaker 1: Times of the Wall Street Journal. You don't want to
Speaker 1: repeat the viral lie. Part of your business model is
Speaker 1: people are paying to subscribe to you because you are
Speaker 1: more reliable.
Speaker 2: Right exactly. So that's why they come to us. They
Speaker 2: upload the assets and are our web app returns the
Speaker 2: results I see.
Speaker 1: So it's just like you just go to whatever Real
Speaker 1: Defender dot whatever and you upload the viral video and
Speaker 1: your machine says it's a fake.
Speaker 2: Yeah, So we give results and probabilities that we don't
Speaker 2: have the ground truth, so we give a probability. There's
Speaker 2: several different models running, so we use an ensemble of models.
Speaker 2: We have different models looking at different things, and we
Speaker 2: give an overall score averaging those. In the case of
Speaker 2: a video, we actually highlight the areas of a defake.
Speaker 2: If the person is speaking and they're a fake, there'll
Speaker 2: be a red box around them. If there is a
Speaker 2: real they'll be a green box around it.
Speaker 1: And well, that latter part sounds more binary as opposed
Speaker 1: to probabilistic.
Speaker 2: We give both. So yeah, there's there was a probably
Speaker 2: score and there's just the visual.
Speaker 1: And so the probabilistic score is basically according to our model,
Speaker 1: there's a seventy percent chance that this is fake something
Speaker 1: of that nature.
Speaker 2: According to our ensemble of models.
Speaker 1: Yes, yeah, our model of models, our fund of funds
Speaker 1: of models exactly. So so okay, so you're actually looking
Speaker 1: us toward what's under the hood, right, I'm interested in
Speaker 1: discussing this on a few levels. Right, there is the
Speaker 1: sort of broad beyond reality defender. You know, what are
Speaker 1: the basic ways that the technology works, Like how does
Speaker 1: deepfake detection gen AI detection work? In a broad way?
Speaker 1: Like can you talk me through that?
Speaker 2: Absolutely? Yeah. There's currently two ways people are looking at
Speaker 2: this problem. Number one is prominence. For example, you water
Speaker 2: mark a media that you create, maybe you water market
Speaker 2: or you digitally sign it, maybe you put on a
Speaker 2: blockchain somewhere or something like that. But basically there's a
Speaker 2: source of true that this video is real. Yeah, and
Speaker 2: there's a water mark. That's number one.
Speaker 1: But we're concerned. We're concerned with instances where that is
Speaker 1: not the case. Right. Our world is full of videos
Speaker 1: today that are not clearly watermarked, blockchain whatever for prominence.
Speaker 1: So we have this problem. What are the ways people
Speaker 1: are solving it?
Speaker 2: Yeah? The second way is how we're solving it, which
Speaker 2: is basically we use AI to detect AI, so we
Speaker 2: which we call inference. So we train AI models, as
Speaker 2: I mentioned, a bunch of them to look at various
Speaker 2: various aspects of plus say video.
Speaker 1: So like, is it a sort of generative adversarial network
Speaker 1: the right term? I mean, it seems like you It
Speaker 1: seems like if I were making up how to do this,
Speaker 1: I'd be like, well, I'm gonna have one model that's
Speaker 1: like cranking out really good deep fikes, but I'll know
Speaker 1: which ones are the deep fis, and then I'm gonna
Speaker 1: feed the deep fis and the real ones to my
Speaker 1: other model, and I'll score it on how well it does,
Speaker 1: and it'll get really good at figuring out the difference.
Speaker 2: Yeah, that's actually exactly how a lot of these work.
Speaker 2: For if you go to there's a website you can
Speaker 2: go where it just generates a person every time you
Speaker 2: go to it a right, and that's actually using again
Speaker 2: to generate that person. So the way we detect and
Speaker 2: I can I can give a little bit more detail here.
Speaker 2: So for example, one of our models which we actually removed,
Speaker 2: was looking at blood flow. So yeah, so imagine actually
Speaker 2: in this video lighting and conditions are right, we can
Speaker 2: actually detect the heartbeat and the blood flow and the
Speaker 2: veins the way we're looking at each other.
Speaker 1: As I'm looking at my weirdly today, maybe because it's
Speaker 1: hot or because the light hair, I can actually see
Speaker 1: a vein bulging on my forehead. So, like you're saying,
Speaker 1: an AI could like measure my pulse from that or something.
Speaker 2: In the right conditions. Now, that model has a lot
Speaker 2: of limitations, and you need to have the right It's
Speaker 2: basically it has a lot of bias. Right, So we
Speaker 2: tossed that.
Speaker 1: Wait, you're saying it didn't work. You're saying it didn't work.
Speaker 2: It worked in the right conditions and the right skin tone,
Speaker 2: so yeah, so otherwise it was biased. So we this
Speaker 2: was experimental and we tossed it.
Speaker 1: A lot of things. It didn't work. So you tried
Speaker 1: it and in a broad way it didn't work. It
Speaker 1: worked in narrow conditions, but you need things that work
Speaker 1: more broadly. What's another thing you tried that didn't work?
Speaker 2: Well, I can tell you every month we may be
Speaker 2: throwing away models.
Speaker 1: Well, presumably there's things that work for a while and
Speaker 1: then they don't. Right, It's kind of like antibiotics versus bacteria, right,
Speaker 1: like your adversaries are getting better every day.
Speaker 2: Basically, what we use, what we like to use is
Speaker 2: we like to say we're like an anti virus company.
Speaker 2: So every time every month there's a new genera of technique,
Speaker 2: maybe we should go detective. But maybe it's something we
Speaker 2: don't anticipate and we don't detect, and so we have
Speaker 2: to make sure we quickly update our models. So and
Speaker 2: then a model that worked last year, it's completely irrelevant now.
Speaker 1: So what else, like, what else is happening technologically on
Speaker 1: the reality defense side, on the detection side.
Speaker 2: Okay, so the way, we have a few different products.
Speaker 2: One is, as I mentioned, real time audio like scanning
Speaker 2: and listening for telephone calls. The other one is a
Speaker 2: place where a journalist or any user can go and
Speaker 2: upload not just videos, but we also detect images. We
Speaker 2: also detect audio, We also detect texts like chat GPT,
Speaker 2: and these tools also explain to a user why something
Speaker 2: is a deep fake. We don't just give a score.
Speaker 2: Or for an image, we might put a heat map
Speaker 2: and see these are the areas that set the model off.
Speaker 2: For text, we might highlight areas and see these other
Speaker 2: areas that appear to be generated.
Speaker 1: There's a case study you have about a university that
Speaker 1: is a client of yours that, among other things, uses
Speaker 1: uses your service to tell when students are turning in
Speaker 1: papers written by chat GIPT. Basically as I read it, right, like,
Speaker 1: I just assume that, like everybody writes papers with chat
Speaker 1: GPT now and there's nothing anybody can do about it.
Speaker 1: But is that not true? Like if I like have
Speaker 1: GPT write my paper and then I like change a
Speaker 1: few words, does that sort of help get let me
Speaker 1: sail past your defense?
Speaker 2: It depends depends how much you change, Yeah, or if
Speaker 2: you change like over fifty percent, maybe maybe would So
Speaker 2: it depends.
Speaker 1: Over fifty percent is more than a few words. And
Speaker 1: so can you talk? I mean, I know you can't
Speaker 1: name the university, but in practice you know how they
Speaker 1: use it. So you know, somefess runs the student's papers
Speaker 1: through your software and it says of when student there's
Speaker 1: a whatever sixty percent chance that this was created using
Speaker 1: a large language model. I mean, do you know in practice?
Speaker 1: Obviously the professor could do whatever they want or the
Speaker 1: university could have whatever policy, but do you know in practice,
Speaker 1: what do they do with this information like that's that's
Speaker 1: in a way a harder one to figure out than
Speaker 1: the like banker who's like, oh, it might be a
Speaker 1: deep fake on the phone. I'll call you right back
Speaker 1: for security. Like if my I don't have a banker,
Speaker 1: but if I had a banker and they did that,
Speaker 1: I'd be like, oh, that's cool. I'm glad my bank
Speaker 1: is doing this thing. Whereas with like the professor and
Speaker 1: the student, that's a much more sort of fraud situation, right,
Speaker 1: and harder to think of how to deal with again
Speaker 1: the probabilistic nature of the output of the model.
Speaker 2: Yes, I think a couple more things here. First of all,
Speaker 2: I think even universities are trying to figure out this problem.
Speaker 2: How to you solve it? You know. But the second
Speaker 2: thing to note, most of our users are not interested
Speaker 2: in a text detector. That seems to be a much
Speaker 2: smaller market. The biggest one is actually audio. It's becoming
Speaker 2: imagine you get a call from a loved one and
Speaker 2: send me money, and you send money if you realize
Speaker 2: is not who it was a defate, right, That's actually
Speaker 2: a much widely used system.
Speaker 1: That's the big one in terms of the business it's interesting.
Speaker 1: I mean, I wonder if that's partly like relative we
Speaker 1: think about the video more, but is it partly because
Speaker 1: deep fake audio is now quite good and there are
Speaker 1: lots of instances where people will transfer lots of money
Speaker 1: based solely on audio.
Speaker 2: De fake audio is the best and it's getting better,
Speaker 2: right interested. I used to go to make your voice,
Speaker 2: maybe I need a minute. Now I need just a
Speaker 2: few seconds and I can make your voice. It's getting
Speaker 2: exponentially better. All of them are, but audio is definitely
Speaker 2: top of the list right now.
Speaker 1: Huh And how are you keeping up?
Speaker 2: Yeah? I mean, so when we detect audio, it's tricky.
Speaker 2: There's a lot of factors to think about a person's accent,
Speaker 2: right what is model biased? Does it not understand or
Speaker 2: is there an issue where it detects It detects one
Speaker 2: person with a certain type accent always as a d thing.
Speaker 2: There's also issues of like noise when when when there's
Speaker 2: a lot of background noise, the model could be impacted.
Speaker 2: When there's cosstop, multiple people speaking at the same time,
Speaker 2: that could impact the model. So there's a variety of factors.
Speaker 2: And the other thing you think about is our models
Speaker 2: are more they support multiple languages, so we don't just
Speaker 2: do English, and so all of these kind of make
Speaker 2: it very complicated. So when we detect something it's called
Speaker 2: pre processing, there's a whole bunch of steps to the
Speaker 2: audio before it actually goes to our AI models where
Speaker 2: we have to clean up the audio, do certain types
Speaker 2: of transformations before we push it to the models.
Speaker 1: And is that happening in real time with these companies?
Speaker 1: Huh huh? And and are you like, what is the
Speaker 1: frontier of preprocessing? Like is it is it an efficiency
Speaker 1: and speed problem because you're trying to do it in
Speaker 1: real time and so you're just trying to kind of
Speaker 1: make the sort of algorithmic part of it as fast
Speaker 1: and efficient as possible.
Speaker 2: Yeah, I mean this is a challenge. There's a lot
Speaker 2: to be done. So that's an ongoing research. How do
Speaker 2: we continue to speed up not just a preprocessing, but
Speaker 2: the inference. And there's a variety of one thing that's
Speaker 2: called a foundation model. I'm not sure if you heard
Speaker 2: what those are, but these are extremely large pre train
Speaker 2: model GPT is a foundation model is a pre train model.
Speaker 2: And so these models can be useful in some parts
Speaker 2: of the preprocessing where they can quickly extract certain features
Speaker 2: for us, and then we can use those two down
Speaker 2: the pipeline.
Speaker 1: Still to come on the show. The problems that Ali
Speaker 1: is trying to solve. Now, how good are you at
Speaker 1: detecting de fikes? Can you quantify how good you are?
Speaker 2: So the way they usually do this is they look
Speaker 2: at benchmarks. Right, there's public data sets which we can
Speaker 2: take and run and we're in the nineties and then
Speaker 2: but you know that's not the real world.
Speaker 1: When you say you're in the nineties, you mean you
Speaker 1: in a binary sense, you guess correctly ninety percent of
Speaker 1: the time.
Speaker 2: Yeah, So on a public benchmark, we're in the nineties.
Speaker 2: There's accuracy, precision and recall. Accuracy is how accurate are
Speaker 2: we Let's say there is one hundred sample set is
Speaker 2: one hundred, maybe fifty is fake, fifty is real? Right.
Speaker 2: The accuracy is you take, okay, how many of those
Speaker 2: did you get right? How many of the real I'm fake? Divided? Right,
Speaker 2: that's the that's the accuracy. The problem with that is
Speaker 2: like unbalanced data set, maybe maybe only two is fake
Speaker 2: and then the other ninety eight are real. So in
Speaker 2: that case, the accuracy. See if we had said that Okay,
Speaker 2: everything is real, we would be ninety eight percent. Right,
Speaker 2: that's not very useful because you missed the defix. So
Speaker 2: that's why precision and recall coming. They look specifically at
Speaker 2: how did you do on that specific like the fakes
Speaker 2: or the reals, So there's more than just accuracy. There's
Speaker 2: also other factors to look at.
Speaker 1: So there's it's kind of like the sort of false
Speaker 1: positive false negative challenge with medical tests, right you want
Speaker 1: to test that both says you have the thing, says
Speaker 1: you have the disease when you have the disease, and
Speaker 1: also says you don't have the disease when you don't
Speaker 1: have the disease, And that actually ends up being a
Speaker 1: really complicated problem given the nature of baselines, right like
Speaker 1: in your universe, certainly in the universe of people calling
Speaker 1: their banker. Almost everybody calling their banker is a real person, right,
Speaker 1: but there are these very high stakes, presumably very rare
Speaker 1: cases where it is a defake, and so that's like
Speaker 1: a complicated problem.
Speaker 2: It actually is, It absolutely is, and it's something as
Speaker 2: we work with each customer, we have to tweak those.
Speaker 2: Someonet higher false positives, someone higher false negatives. It depends
Speaker 2: on each use case, in the case of a bank,
Speaker 2: they want to be a bit more cautious. But that
Speaker 2: also causes a lot of It could cause a lot
Speaker 2: of pain depending on the volume.
Speaker 1: Right, because if every client it's like, oh sorry, I
Speaker 1: got to call you back to make sure you're not
Speaker 1: a deep fake, Like that's not great.
Speaker 2: Yeah, And if you have thousands of calls a day
Speaker 2: and even one percent is a false positive or negative,
Speaker 2: that that creates a lot of work, Yeah, because it
Speaker 2: adds up.
Speaker 1: How do you solve that? What do you do about that?
Speaker 2: So the way it works is all about adjusting. You
Speaker 2: can think of thresholds, right, We can adjust variety of
Speaker 2: parameters as the output for a model, not just the
Speaker 2: model itself, but the for example, in an audio as
Speaker 2: we speak, you know, we could look at okay, how
Speaker 2: long do you want to listen before you give an answer?
Speaker 2: You know, maybe maybe? And the longer you listen, the
Speaker 2: more the more confident, the more that's smart.
Speaker 1: That makes sense, right, because it's essentially more data for
Speaker 1: the model exactly. Yeah, what are you trying to figure
Speaker 1: out now? Like what is the frontier?
Speaker 2: What's really the latest now? And it's just amazing how quickly.
Speaker 2: It's going as videos. So the videos that we detect
Speaker 2: are like a face swap, Like you're sitting there speaking
Speaker 2: and another person's face is on there. That's a face swap.
Speaker 2: But now you can generate an entire video completely from scratch,
Speaker 2: and you just type in the description and the video
Speaker 2: comes out. You can take some you can I can
Speaker 2: take your voice, a few seconds of your voice. I
Speaker 2: can then have you say anything I want, which you
Speaker 2: can clearly see. The bad, bad person can misuse these tools.
Speaker 2: So the latest is these things are getting really good and.
Speaker 1: Over time, like with those videos, is your how is
Speaker 1: your reliability and accuracy changing? You're getting better or worse
Speaker 1: or staying the same as the technology to create the
Speaker 1: deep fix improves.
Speaker 2: So what's interesting is it has slowed down in terms
Speaker 2: of like the signatures, Like we don't need as much
Speaker 2: data as we used to. So of course there's still
Speaker 2: a lot of work and we're never going to stop,
Speaker 2: but it is stabilizing a little bit.
Speaker 1: When you say it, what is stabilizing a little bit,
Speaker 1: So like the.
Speaker 2: Defied signatures are stabilizing the way.
Speaker 1: The signatures, meaning the giveaways, the things that I can't see,
Speaker 1: but that your models can see that AI exactly.
Speaker 2: So our models going back and give them more detail.
Speaker 2: They're looking at different attributes of a piece of media,
Speaker 2: and they pull out those attributes and then they send
Speaker 2: those to our and house neural networks that steady those attributes.
Speaker 1: Like one that you have mentioned, that the company has
Speaker 1: mentioned publicly is the the sync of audio and video. Right, Yes,
Speaker 1: maybe that's one where it's gotten better and it doesn't
Speaker 1: matter anymore, but like it, from what I understand, from
Speaker 1: what I've read, there was at least a time when
Speaker 1: the sink of the audio and video tended to be
Speaker 1: off in deep fake videos. Right? Is that an example
Speaker 1: of a signature.
Speaker 2: So the way that works is we train the model.
Speaker 2: We say, hey, here's a bunch of people speaking, here's
Speaker 2: what they look like. Look at the sink. Here's a
Speaker 2: bunch of people like that or defikes, and look at
Speaker 2: the sink, and we tune the model so we can
Speaker 2: tell the difference. That's also happening to a video. By
Speaker 2: the way, if you look at Sora and some of
Speaker 2: these new models where someone's are walking, for example, their
Speaker 2: legs are not like you know, they're not really smooth,
Speaker 2: or they don't look right, So you can look at
Speaker 2: that as well. That's the temporal dynamics we call that.
Speaker 1: Uh Like temporal dynamics is basically are things proceeding in
Speaker 1: time in a natural.
Speaker 2: Way exactly how things change over time.
Speaker 1: So yeah, all of these seem like things that you
Speaker 1: can just that are going to be fleeting, right. Like
Speaker 1: my baseline assumption is it'll all get solved. Do you
Speaker 1: how long do you think you'll be able to defend
Speaker 1: reality for?
Speaker 2: You know, this question comes up all the time where
Speaker 2: there is always a giveaway or there is always a
Speaker 2: new way to look at the problem. We're not just
Speaker 2: looking always at the raw pixels, right, We could look
Speaker 2: at different aspects. We could look at the frequency. For example,
Speaker 2: if you look at an image, you can actually break
Speaker 2: it down into frequencies.
Speaker 1: When you say frequency, what do you mean when you
Speaker 1: say you can look at the frequency? What does that mean?
Speaker 2: So? For example, okay, so let's go with audio. You
Speaker 2: know you can use some of call four yer transformers
Speaker 2: to actually break up an audio into individual wavelengths science
Speaker 2: and co science that does a look. You can do
Speaker 2: the same with for an image, for example, you can
Speaker 2: break that.
Speaker 1: Up like like the analogy of a wave form of audio.
Speaker 2: Yeah, it can. It can be translated into a bunch
Speaker 2: of waves. So so there's multiples that we look at.
Speaker 2: There's and the AI there's always a giveaway, uh and
Speaker 2: and and again we're also thinking outside the box, right,
Speaker 2: like the blood flow for example. Right, But there's other
Speaker 2: kind of similar things we could think about.
Speaker 1: I mean, presumably you know there, you know, renaissance Renaissance Capital.
Speaker 1: The James Simons is one of the first quant hedge funds,
Speaker 1: and they made tons of money for a long time.
Speaker 1: They wildly outperformed the market. Clearly they had a technological advantage.
Speaker 1: And the thing Simon said, the founder of this math
Speaker 1: guy about about that company. One of the things he
Speaker 1: said was like, we actually don't want to hire like
Speaker 1: finance people who have some story about why a stock
Speaker 1: is going to outperform, because if there's a story about
Speaker 1: it that then then somebody else is going to know
Speaker 1: it already. Right. Their thing was just like, we just
Speaker 1: give the model all the data and let the model
Speaker 1: find these weird ass patterns that no human even understands.
Speaker 1: But they work more often than they don't work, and
Speaker 1: we make tons of money. And I would think that
Speaker 1: would be the case for you to some extent that
Speaker 1: if you could think of a thing like monitoring blood
Speaker 1: flow or whatever, then the bad guys or whatever, the
Speaker 1: people who want to make realistic Jenny I would also
Speaker 1: think of it. And the real kind of secret sauce
Speaker 1: would be in weird correlations that the model finds that
Speaker 1: we wouldn't even understand.
Speaker 2: Exactly. I mean, that is oftentimes what the model is
Speaker 2: training on, and the way it determines of something that
Speaker 2: you think looking at certain features, it is something that
Speaker 2: we don't even tell it, right, Yeah, it determines on
Speaker 2: its own.
Speaker 1: Like that's the beauty of this kind of new era
Speaker 1: of whatever, neural networks, machine learning. Right, it's just you
Speaker 1: throw everything at it and let the machine figure it out.
Speaker 2: We like to say we throw the kitchen sink at sometimes.
Speaker 1: Yes, yes, I mean, And so when you were talking
Speaker 1: before about explainability, right about sort of saying in your output,
Speaker 1: here's why we think it's fake. I feel like that
Speaker 1: kind of throw everything at it and let the machine
Speaker 1: figure it out makes it hard to like sometimes you
Speaker 1: don't know, right, it's just like, well, the machine is
Speaker 1: very smart in it says this is probably fake, Like yes,
Speaker 1: that is that intention that can happen.
Speaker 2: So you'll look at it. We'll show you an image
Speaker 2: and it'll say the model was looking at certain areas.
Speaker 2: And by the way, this also helps us with debugging
Speaker 2: him bias. Right, maybe it was like for some reason
Speaker 2: looking at an area of the face that we wouldn't tell.
Speaker 2: Why would that set off the model? And so in
Speaker 2: those scenario as we also investigate like why was this
Speaker 2: area flag? And it could be one hundred percent correct,
Speaker 2: it's just we do we do have to examine it further.
Speaker 1: Could you create a deep fake that would fool your
Speaker 1: deep fake detector? Yes, haha, Well if you could do it,
Speaker 1: somebody else could do it. Don't you think I could
Speaker 1: do it?
Speaker 2: Because I have access to a lot more knowledge, right, Like,
Speaker 2: you know I could if I was running an anti
Speaker 2: virus company, I could probably write a virus if I
Speaker 2: knew exactly what we're constantly actually trying to do that.
Speaker 1: By the way, yeah, I mean in a sense, that's
Speaker 1: the whole adversarial network thing, right, Like I guess you
Speaker 1: have to do that for your detection models or your
Speaker 1: suite of models to get better, right.
Speaker 2: Yeah, So we have what's called red teeming both black
Speaker 2: box and understanding of the codes. So we're trying to
Speaker 2: break the models. That's part of the what we do.
Speaker 1: Uh huh. And so are there like evil geniuses at
Speaker 1: your company who can make killer deep fakes?
Speaker 2: We definitely have geniuses one hundred percent, but we're in
Speaker 2: the business of detection, right, we don't. We don't try
Speaker 2: to generate too much other than just for training the models.
Speaker 1: I mean I have to think like, there are many
Speaker 1: people in the world world who want to make a
Speaker 1: deep fakes for many reasons, and they're at different levels
Speaker 1: of technological sophistication. Naively not knowing much about this, I
Speaker 1: would think you can catch most of them. But if
Speaker 1: you have people who can beat your models, I would
Speaker 1: imagine that, say, state actors, countries throwing billions of dollars
Speaker 1: at this probably also have people who could defeat your models.
Speaker 2: Yeah, I mean that's always a case with any cybersecurity company.
Speaker 2: We are a cybersecurity company. Every cyber security company does
Speaker 2: its best to defend right, but we did not promise
Speaker 2: one hundred percent. Our models are always a probability.
Speaker 1: Who's who's the best at making deep fikes that you're
Speaker 1: aware of.
Speaker 2: There's a few, right, there's like Sora from OpenAI. There's Runway,
Speaker 2: there's Synthesia, there's you.
Speaker 1: Better be able to catch right, anything I've heard of.
Speaker 1: You better be really good at the technic. Presumably it's
Speaker 1: like some like you know, Russian Genius Squad or I
Speaker 1: don't know, the North Koreans are some things. I would
Speaker 1: imagine it is some state funded actor, but.
Speaker 2: I would I would actually say, you know, we're in
Speaker 2: a place where this is a problem is getting bigger.
Speaker 2: But we're in a place where a lot of the
Speaker 2: defects coming out are actually fron entertainment and they're not
Speaker 2: like Youth for Evil. You know, you've seen the famous
Speaker 2: Tom kruzwe or or other actors running around and do things,
Speaker 2: and those are defakes, right, those are actually pretty good.
Speaker 2: We detect them, but they're actually very good.
Speaker 1: What are you thinking about in the context of the
Speaker 1: of the election in the US this year and do
Speaker 1: you have particular clients who are especially focused on election
Speaker 1: related deep fakes.
Speaker 2: Yeah, the media companies are the main ones, and we're ready.
Speaker 2: We detect the best, the best defakes, right, everything that's
Speaker 2: coming out we detect, So we're ready and we want
Speaker 2: to make sure we're there as one avenue of people
Speaker 2: verifying content. I believe late last year there was an
Speaker 2: election in Slovenia where there was an audio of one
Speaker 2: of the candidates saying he's gonna double the price of beer. Yeah,
Speaker 2: and that actually was a defake. It was caught, but
Speaker 2: it kind of costed some damage. So it's starting to
Speaker 2: happen now.
Speaker 1: It's an awesomely stupid deep fake. I mean, to me,
Speaker 1: the real risk of deep fakes is not people believing
Speaker 1: something that's false. It's people ceasing to believe anything, right.
Speaker 1: It's just saying, oh, that's probably just a deep fake,
Speaker 1: right like that. Actually, to me seems like the bigger
Speaker 1: risk is nothing is true anymore. Nobody cares about the
Speaker 1: truth anymore.
Speaker 2: That's definitely a problem as well. Now we're seeing people saying, oh,
Speaker 2: this is a defake. That's actually happened. There's a few.
Speaker 2: I believe it was a Cape Milton video if I'm correct,
Speaker 2: that was earlier this year, or everyone thought that was
Speaker 2: a defake and it wasn't. So this kind of problem
Speaker 2: is happening.
Speaker 1: Like that's because people people want to believe things that
Speaker 1: are consistent with their prior beliefs, and they don't want
Speaker 1: to believe things that call their prior beliefs into question, right,
Speaker 1: and so deep fakes in a way are an easy
Speaker 1: out where if you see something you like, you assume
Speaker 1: it's true. If you see something you don't like, you
Speaker 1: assume it's not true, or you assume everything's just kind
Speaker 1: of bullshit like that to me seems like a big
Speaker 1: quind of societal level risk of deep fakes.
Speaker 2: We'll never fix that. That's something that will never solve. Yeah,
Speaker 2: people have their own beliefs. You can show them anything,
Speaker 2: the facts, math, that's not going to fix it all. Yeah.
Speaker 1: No, I guess that's a human nature problem, if not
Speaker 1: an AI problem. We'll be back in a minute with
Speaker 1: the lighting round. Okay, let's close with a lightning round.
Speaker 2: Okay.
Speaker 1: How often do people applying to work at Reality Defender
Speaker 1: use generative AI to write cover letters?
Speaker 2: Oh, that's a good one. Not a lot of, but
Speaker 2: we've seen it for sure. I would say maybe about
Speaker 2: three percent.
Speaker 1: Okay. If I want to use generative AI to write
Speaker 1: a cover letter to apply to work at Reality Defender,
Speaker 1: but I don't want to get caught, what should I do.
Speaker 2: Change about seventy five percent.
Speaker 1: Of the words Okay, who is Gabe Reagan?
Speaker 2: Gabe was? I think it was our VP of public
Speaker 2: relations or something like that. Here's a dfake. We we
Speaker 2: created him as a as kind of a as kind
Speaker 2: of a fun joke. But obviously we tell everyone.
Speaker 1: Tell me, tell me a little bit more about that.
Speaker 2: If you go on certain websites where you put your
Speaker 2: photo and maybe your job experience, there's quite a large
Speaker 2: number of deficke profiles on these websites like LinkedIn.
Speaker 1: Yes, huh, why why people be doing that?
Speaker 2: Sorry scammers?
Speaker 1: I'm trying to think, how do you get money out
Speaker 1: of people by having a fake LinkedIn account?
Speaker 2: Oh? I can tell you. Let's say you start the
Speaker 2: most popular ones that I'm aware of, is like cryptocurrency.
Speaker 2: Maybe you create a coin and you're like, here's a
Speaker 2: CEO and here's this person and they have these great
Speaker 2: LinkedIn profiles. Here's their photo and they're not real, but
Speaker 2: it sells a story. Right.
Speaker 1: Is it right that you founded a clothing company?
Speaker 2: I did?
Speaker 1: Yes, what's one thing you learned about fashion from doing that?
Speaker 2: It's much different than software development.
Speaker 1: Sure, I don't think you needed to start a company
Speaker 1: to learn that. I mean, the marginal cost is not
Speaker 1: zero for one thing.
Speaker 2: Yeah, the software is easy, you write some It's not
Speaker 2: easy at all. But what I mean is you're writing
Speaker 2: some code and you ship it. Versus in fashion, you
Speaker 2: have to have like you got to source the fabric.
Speaker 2: You gotta you gotta design it, you gotta make the patterns,
Speaker 2: you gotta cut it, sew it, make sure it fits.
Speaker 2: It's a lot more work.
Speaker 1: What are the chances that we exist in a simulation?
Speaker 2: You know, I used to think this is kind of
Speaker 2: a joke, but I don't know. I'm seeing every every
Speaker 2: month it seems to get higher. From my perspective.
Speaker 1: Why do you say that.
Speaker 2: I'm seeing what's happening with tech and what we're building,
Speaker 2: and there's you can see there's there was one paper
Speaker 2: where they took a bunch of agents and they gave
Speaker 2: them all a job and they start to do it
Speaker 2: and they just started to like create their own kind
Speaker 2: of like work cloths. Right, I don't know, it shuld
Speaker 2: be getting there.
Speaker 1: So so it's like, well, if we can create a
Speaker 1: simulation that seems like reality, maybe someone created a simulation
Speaker 1: that is our reality exactly. Yeah, what do you wish
Speaker 1: more people understood about AI.
Speaker 2: I mean, it's a tool, and I don't think people
Speaker 2: should be afraid of it. They should embrace it. And
Speaker 2: you know there's people are just running away from it.
Speaker 2: It's fantastic, it's great. Embrace it. Just be careful. One
Speaker 2: thing I'd like to tell, like my friends and family,
Speaker 2: especially with the e fake audio, have a safe word.
Speaker 2: As somebody calls you and you're like that's weird, you know,
Speaker 2: call him back or ask for a safe word.
Speaker 1: What do you wish more people understood about reality reality?
Speaker 2: I would say, just be aware that you exist. And
Speaker 2: every day's a gift, So you should be excited that
Speaker 2: you hear. Like the chances of you existing it's like
Speaker 2: you've won the lottery a million times, So every day's
Speaker 2: a gift.
Speaker 1: Ali Shakiyari is the co founder and CTO at Reality Defender.
Speaker 1: Today's show was produced by Gabriel Hunter Chang. It was
Speaker 1: edited by Lyddy Jean Kott and engineered by Sarah Bruguer.
Speaker 1: You can email us at problem at Pushkin dot fm.
Speaker 1: I'm Jacob Goldstein and we'll be back next week with
Speaker 1: another episode of What's Your Problem