WEBVTT - Detecting Deepfakes With AI 0:00:15.356 --> 0:00:23.116 Pushkin. Earlier this year, an employee working in Hong Kong 0:00:23.196 --> 0:00:26.516 for an international company got a weird message from one 0:00:26.516 --> 0:00:29.516 of his colleagues. He was supposed to make a secret 0:00:29.556 --> 0:00:34.236 transfer of millions of dollars. It seems sketchy. It obviously 0:00:34.316 --> 0:00:36.956 seems sketchy, so he got on a video call with 0:00:36.996 --> 0:00:40.516 a bunch of people, including the company's CFO, the chief 0:00:40.516 --> 0:00:44.516 financial officer. The CFO said the request was legit, so 0:00:44.716 --> 0:00:47.716 the employee did what he was told. He transferred roughly 0:00:47.916 --> 0:00:52.076 twenty five million dollars to several bank accounts. As it 0:00:52.116 --> 0:00:54.396 turned out, the CFO on the video call was not 0:00:54.876 --> 0:00:58.796 really the CFO. It was a deep fake, an AI 0:00:58.956 --> 0:01:03.116 generated twin created from publicly available audio and video of 0:01:03.156 --> 0:01:06.756 the real CFO. By the time the company figured out 0:01:06.756 --> 0:01:09.516 what was going on, it was too late, the money 0:01:09.756 --> 0:01:18.316 was gone. I'm Jacob Goldstein and this is What's Your Problem, 0:01:18.476 --> 0:01:20.236 the show where I talk to people who are trying 0:01:20.276 --> 0:01:24.676 to make technological progress. My guest today is Ali Shahieri. 0:01:24.916 --> 0:01:28.156 He's the co founder and chief technology officer at the 0:01:28.236 --> 0:01:34.076 audaciously named Reality Defender. Ali's problem is this, how can 0:01:34.116 --> 0:01:39.516 you use AI to protect the world from AI? More specifically, 0:01:39.956 --> 0:01:41.996 how do you build a set of models to spot 0:01:41.996 --> 0:01:48.076 the difference between reality and AI generated deep fakes. How'd 0:01:48.076 --> 0:01:50.476 you get into the defending reality business? 0:01:51.796 --> 0:01:57.556 Yeah, so when I started, it was around actually generating 0:01:57.756 --> 0:02:00.236 videos and deep fikes. 0:02:00.396 --> 0:02:03.036 So you were attacking reality before you were defending it. 0:02:04.556 --> 0:02:06.796 I wouldn't said we were attacking anything, but we were 0:02:06.796 --> 0:02:10.756 definitely into looking into this technology. And it is way 0:02:10.796 --> 0:02:14.036 back before all this stuff kind of went crazy. This 0:02:14.076 --> 0:02:17.036 is back in like twenty nineteen around that time, So 0:02:17.076 --> 0:02:21.236 we were building digital twins and we're looking at how 0:02:21.236 --> 0:02:23.476 do you make it so that it looks realistic? Is 0:02:23.516 --> 0:02:27.116 it a cartoon looking thing? Is it like a unity 0:02:27.116 --> 0:02:29.596 three D thing? And then that's when we started to 0:02:29.636 --> 0:02:32.796 see like these early research papers where they were taking 0:02:32.836 --> 0:02:36.236 like someone's face and putting it on a video and 0:02:36.316 --> 0:02:39.436 blending it in and it looked really good, and we 0:02:39.436 --> 0:02:41.556 were like, oh, maybe we can do the digital twins 0:02:41.916 --> 0:02:46.276 that way. And while we were like in that business, 0:02:46.516 --> 0:02:49.116 we were like, you know, probably in a few years 0:02:49.116 --> 0:02:53.836 someone can download an app and just make anything very easily. 0:02:53.876 --> 0:02:57.356 And that's kind of the origins of how how we started. 0:02:57.996 --> 0:03:00.116 We're very mission driven. What we're trying to do here 0:03:00.236 --> 0:03:05.356 is really protect the world and people from the dangers 0:03:05.836 --> 0:03:08.716 of AI, but in a way where you know, we 0:03:08.756 --> 0:03:11.996 want people not to abuse the technology. We're very we 0:03:12.036 --> 0:03:15.116 love AI, we just don't want it to be abused. 0:03:16.036 --> 0:03:19.876 So let's talk about this sort of deep fake detection 0:03:20.516 --> 0:03:25.476 kind of you know, jen AI detection market more generally, 0:03:25.716 --> 0:03:30.796 like who's like, who's selling deep fake detection right now, 0:03:30.796 --> 0:03:32.676 and who's buying what's the what's the sort of market 0:03:32.796 --> 0:03:33.676 landscape look like. 0:03:34.636 --> 0:03:38.116 The type of clients that we have right now are banks. 0:03:39.036 --> 0:03:42.316 For example, we are currently live with one of the 0:03:42.436 --> 0:03:45.436 largest banks in the world. When you call that bank, 0:03:46.436 --> 0:03:50.316 the audio goes through our defake detection models and we're 0:03:50.356 --> 0:03:53.356 able to tell the call center this person might be 0:03:53.396 --> 0:03:57.036 a deep fake. And part of that is that's actually happened. 0:03:57.076 --> 0:04:01.676 Someone's called the bank and they've transferred money out and 0:04:01.956 --> 0:04:05.316 actually this this goes back to twenty nineteen, so the 0:04:05.396 --> 0:04:09.596 first incident of defake fraud actually and back in. 0:04:09.596 --> 0:04:13.316 Two in nineteen that we're aware of. Right, you're right exactly, 0:04:13.676 --> 0:04:15.556 So what happened in twenty nineteen. 0:04:16.156 --> 0:04:18.836 Yeah, so this is back where this is early and 0:04:18.876 --> 0:04:22.596 nobody really knew about this, and there was a CEO 0:04:22.756 --> 0:04:26.756 that called a smaller company that THEO was a parent 0:04:26.796 --> 0:04:29.756 company calling the child company. The CEO calling the other 0:04:29.796 --> 0:04:32.676 CEO and he wanted to transfer some money out and 0:04:33.196 --> 0:04:36.196 it sounded like him and the guy transferred I think 0:04:36.196 --> 0:04:37.916 it was in UK, about two hundred and three hundred 0:04:37.916 --> 0:04:39.636 thousand dollars out And that was like the first one 0:04:39.636 --> 0:04:40.796 of the first ones that we. 0:04:40.836 --> 0:04:44.036 Know of, and they got away with it, I believe. 0:04:44.076 --> 0:04:44.676 So. Yeah. 0:04:44.716 --> 0:04:46.876 And there was an instance earlier this year right where 0:04:47.156 --> 0:04:49.636 I think it was in Hong Kong and some employee 0:04:49.676 --> 0:04:51.836 was on a zoom call with the company's CFO and 0:04:51.876 --> 0:04:54.356 the CFO was like, you know, why are twenty five 0:04:54.356 --> 0:04:57.036 million dollars or something to some bank account? And then 0:04:57.076 --> 0:04:59.236 the employee did it and it turned out the CFO 0:04:59.276 --> 0:05:00.836 on the call was a deep fake, right. 0:05:01.076 --> 0:05:05.676 Yeah, So fast were they your client? They were not 0:05:05.916 --> 0:05:09.516 in our clients unfortunately. But this shows the how quickly 0:05:09.556 --> 0:05:13.756 the technology is evolving. You know, twenty nineteen audio fast 0:05:13.756 --> 0:05:15.676 forward a few years now, You've got a zoom call. 0:05:15.996 --> 0:05:17.956 There'd a bunch of people on it and they all 0:05:17.996 --> 0:05:20.476 look like people, you know, I know, they're all de fis. 0:05:21.156 --> 0:05:23.276 So you were starting to mention. Banks are some of 0:05:23.276 --> 0:05:25.196 your main clients. Who are some of your other main. 0:05:25.076 --> 0:05:28.756 Clients, media companies, I think think of some of the 0:05:28.756 --> 0:05:31.876 big ones, there is our product this year, especially with 0:05:31.996 --> 0:05:35.076 the election. You know, back twenty twenty, we thought it 0:05:35.076 --> 0:05:37.836 would be a problem. It wasn't. This year we think 0:05:37.916 --> 0:05:40.116 is a big problem. For sure. I think we were early, 0:05:40.636 --> 0:05:45.156 but it's already this is happening everywhere even this year. 0:05:45.756 --> 0:05:47.756 This year is the largest election year in the world. 0:05:47.876 --> 0:05:49.996 More than fifty percent of the people are voting, and 0:05:50.076 --> 0:05:54.756 we already have documented cases of election issues with the fix. 0:05:55.556 --> 0:05:59.436 Okay, media companies, banks, any other kind of big categories 0:05:59.436 --> 0:05:59.956 of clients. 0:06:00.636 --> 0:06:06.916 Yeah, so other ones are government agencies. But in the end, 0:06:07.276 --> 0:06:11.996 everyone we think, we believe everyone needs this Product's not 0:06:12.076 --> 0:06:14.356 It shouldn't be up to the people to decide or 0:06:14.356 --> 0:06:16.676 figure out if something's a deepic. If you're on the 0:06:16.756 --> 0:06:19.876 social media platform, you shouldn't have to figure out, hey, 0:06:19.916 --> 0:06:21.556 is this person real or not. It should just be 0:06:21.596 --> 0:06:23.596 built in and anyone should be able to use it. 0:06:24.156 --> 0:06:29.156 Well. Our social media companies either buying or building deep 0:06:29.196 --> 0:06:31.836 fake detection tools or do they want to like stay 0:06:31.876 --> 0:06:33.436 out of that business and be like no, we don't 0:06:33.436 --> 0:06:35.396 want to be in the business of saying yes, this 0:06:35.516 --> 0:06:36.596 is real, no, this isn't real. 0:06:37.356 --> 0:06:39.556 I can tell you we've been in contact and have 0:06:39.676 --> 0:06:44.636 talked to some social media platforms. I think one issue 0:06:44.676 --> 0:06:49.116 is they don't have to flag these things. It's up 0:06:49.156 --> 0:06:53.076 to them, right, there's not a lot of regulation, so 0:06:53.516 --> 0:06:55.436 I know they're thinking about it. We've chatted with some, 0:06:56.076 --> 0:06:57.596 but that's the extent of it. 0:06:58.316 --> 0:07:00.236 So okay, So let's talk about how it works. And 0:07:00.276 --> 0:07:02.036 there's two ways that I want to talk about how 0:07:02.036 --> 0:07:03.396 it works. So one is from the point of view 0:07:03.436 --> 0:07:06.356 of the user, whoever that may be, and then the 0:07:06.396 --> 0:07:08.476 other is sort of what's going on under the hood. Right, 0:07:08.756 --> 0:07:11.316 So let's start with the point of view of the user. 0:07:11.596 --> 0:07:15.156 If I'm a whatever, a bank, university, a media company 0:07:15.196 --> 0:07:18.196 who is paying for your service, how does it work 0:07:18.236 --> 0:07:18.436 for me? 0:07:19.436 --> 0:07:22.436 Depends on exactly the user and the use case. If 0:07:22.476 --> 0:07:26.076 let's say it's a media company, Uh, they're looking at 0:07:26.196 --> 0:07:30.596 maybe filtering through a lot of content, so content moderation. 0:07:30.916 --> 0:07:32.916 Actually that would be like a social media company. They're 0:07:32.956 --> 0:07:36.516 looking at content moderation. Maybe they want they're looking at 0:07:36.556 --> 0:07:39.716 millions of assets and they want to quickly flag those 0:07:39.716 --> 0:07:42.396 things if they were in that business. Uh, the bank 0:07:42.796 --> 0:07:46.196 there For the example I gave the issue, someone could 0:07:46.236 --> 0:07:48.836 call and biometrics fail. By the way, if you call 0:07:48.876 --> 0:07:52.116 a bank, some banks say repeat after me, your my 0:07:52.196 --> 0:07:54.196 voice is my passport? That actually fails. Now what do 0:07:54.316 --> 0:07:57.436 you think? So a bank wants to make sure the 0:07:57.476 --> 0:08:00.276 person calling in is actually that person. This is more 0:08:00.476 --> 0:08:03.876 relevant to more to private banking, where there's actually a 0:08:03.916 --> 0:08:06.996 one on one relationship between the client and the bank. 0:08:07.236 --> 0:08:09.436 And so in that case, So let's take that case. 0:08:09.476 --> 0:08:12.876 So in that case, someone calls in and talks to 0:08:12.996 --> 0:08:15.716 their banker. They're a rich person who has a private banker. 0:08:15.756 --> 0:08:18.316 Basically it's what you're talking about, right, So this rich 0:08:18.356 --> 0:08:21.996 person calls in and talks to their private banker, and 0:08:22.356 --> 0:08:25.436 it is the system just always running in the background 0:08:25.556 --> 0:08:28.156 in that case, And like, how does it work from 0:08:28.156 --> 0:08:30.276 the point of view of the of the private banker. 0:08:30.916 --> 0:08:33.556 Sure, and I have to be careful what I say here, 0:08:33.836 --> 0:08:38.076 But the high level is the models are listening and 0:08:38.116 --> 0:08:41.476 if they detect a potential deep fake, they will the 0:08:41.636 --> 0:08:44.476 call center. That person will get a notification so is 0:08:44.836 --> 0:08:48.716 integrated into their existing workflow. They'll get a notification that says, hey, this. 0:08:48.596 --> 0:08:51.116 Person get like a text or a slack or something 0:08:51.316 --> 0:08:53.796 they're using. You're talking to a deep fake. 0:08:54.596 --> 0:08:56.876 No, they're using software for the bank they're using they're 0:08:56.916 --> 0:09:00.516 still using a software and there's a dashboard. In that scenario, 0:09:00.636 --> 0:09:03.436 they do they ascalate, so they might say, let me 0:09:03.436 --> 0:09:05.436 ask you some more questions or let me call you back. 0:09:05.876 --> 0:09:07.756 Huh. Let me call you back is a super safe one, 0:09:07.796 --> 0:09:09.676 right because if they have a relationship, probably they know 0:09:09.716 --> 0:09:13.276 the number. They just call them back. Yeah, absolutely, okay, 0:09:13.596 --> 0:09:16.236 And then how does it work? How does it work 0:09:16.276 --> 0:09:18.796 for like when you say, like I presume by the 0:09:18.836 --> 0:09:21.276 way that you can't name your clients. You said a 0:09:21.316 --> 0:09:24.396 media company and a bank. It's it's secret that they're. 0:09:24.476 --> 0:09:26.036 Yeah, we're not allowed to okay. 0:09:26.036 --> 0:09:28.196 So let's say a media company. How's it work for 0:09:28.236 --> 0:09:30.036 a media company? 0:09:29.916 --> 0:09:33.196 Their their use case is slightly different, especially right now, 0:09:33.236 --> 0:09:35.676 as I mentioned around the election, So there there might 0:09:35.716 --> 0:09:38.076 be something that that's starting to go viral in the 0:09:38.156 --> 0:09:40.876 news and they want to check, hey, is this a 0:09:40.996 --> 0:09:43.916 real or not? I would like to say like something 0:09:44.036 --> 0:09:47.556 like this is usually when something goes viral, the damage 0:09:47.556 --> 0:09:48.276 is already ton. 0:09:48.996 --> 0:09:51.156 Yes, although if you're if you're whatever, the New York 0:09:51.196 --> 0:09:52.956 Times of the Wall Street Journal. You don't want to 0:09:52.996 --> 0:09:55.796 repeat the viral lie. Part of your business model is 0:09:55.876 --> 0:09:59.156 people are paying to subscribe to you because you are 0:09:59.636 --> 0:10:00.916 more reliable. 0:10:00.516 --> 0:10:02.716 Right exactly. So that's why they come to us. They 0:10:02.796 --> 0:10:05.796 upload the assets and are our web app returns the 0:10:05.876 --> 0:10:06.636 results I see. 0:10:06.676 --> 0:10:09.036 So it's just like you just go to whatever Real 0:10:09.676 --> 0:10:13.156 Defender dot whatever and you upload the viral video and 0:10:13.396 --> 0:10:15.956 your machine says it's a fake. 0:10:16.676 --> 0:10:19.516 Yeah, So we give results and probabilities that we don't 0:10:19.516 --> 0:10:22.756 have the ground truth, so we give a probability. There's 0:10:22.796 --> 0:10:25.876 several different models running, so we use an ensemble of models. 0:10:25.876 --> 0:10:29.596 We have different models looking at different things, and we 0:10:29.676 --> 0:10:32.676 give an overall score averaging those. In the case of 0:10:32.716 --> 0:10:35.636 a video, we actually highlight the areas of a defake. 0:10:36.116 --> 0:10:38.036 If the person is speaking and they're a fake, there'll 0:10:38.036 --> 0:10:39.836 be a red box around them. If there is a 0:10:39.916 --> 0:10:41.196 real they'll be a green box around it. 0:10:41.516 --> 0:10:46.236 And well, that latter part sounds more binary as opposed 0:10:46.276 --> 0:10:47.276 to probabilistic. 0:10:47.476 --> 0:10:50.076 We give both. So yeah, there's there was a probably 0:10:50.236 --> 0:10:52.436 score and there's just the visual. 0:10:52.276 --> 0:10:55.356 And so the probabilistic score is basically according to our model, 0:10:55.396 --> 0:10:58.596 there's a seventy percent chance that this is fake something 0:10:58.756 --> 0:10:59.836 of that nature. 0:10:59.676 --> 0:11:01.796 According to our ensemble of models. 0:11:01.916 --> 0:11:04.836 Yes, yeah, our model of models, our fund of funds 0:11:04.836 --> 0:11:09.316 of models exactly. So so okay, so you're actually looking 0:11:09.436 --> 0:11:12.876 us toward what's under the hood, right, I'm interested in 0:11:12.916 --> 0:11:15.796 discussing this on a few levels. Right, there is the 0:11:15.836 --> 0:11:20.036 sort of broad beyond reality defender. You know, what are 0:11:20.076 --> 0:11:22.636 the basic ways that the technology works, Like how does 0:11:23.036 --> 0:11:26.596 deepfake detection gen AI detection work? In a broad way? 0:11:26.636 --> 0:11:27.636 Like can you talk me through that? 0:11:27.676 --> 0:11:31.196 Absolutely? Yeah. There's currently two ways people are looking at 0:11:31.236 --> 0:11:35.836 this problem. Number one is prominence. For example, you water 0:11:35.996 --> 0:11:39.636 mark a media that you create, maybe you water market 0:11:40.036 --> 0:11:42.156 or you digitally sign it, maybe you put on a 0:11:42.156 --> 0:11:44.436 blockchain somewhere or something like that. But basically there's a 0:11:44.476 --> 0:11:47.036 source of true that this video is real. Yeah, and 0:11:47.076 --> 0:11:48.556 there's a water mark. That's number one. 0:11:50.156 --> 0:11:52.916 But we're concerned. We're concerned with instances where that is 0:11:52.916 --> 0:11:54.916 not the case. Right. Our world is full of videos 0:11:54.956 --> 0:12:00.436 today that are not clearly watermarked, blockchain whatever for prominence. 0:12:00.476 --> 0:12:02.676 So we have this problem. What are the ways people 0:12:02.676 --> 0:12:03.236 are solving it? 0:12:03.516 --> 0:12:05.676 Yeah? The second way is how we're solving it, which 0:12:05.716 --> 0:12:08.556 is basically we use AI to detect AI, so we 0:12:09.596 --> 0:12:13.196 which we call inference. So we train AI models, as 0:12:13.236 --> 0:12:16.116 I mentioned, a bunch of them to look at various 0:12:17.636 --> 0:12:20.036 various aspects of plus say video. 0:12:20.476 --> 0:12:24.836 So like, is it a sort of generative adversarial network 0:12:25.636 --> 0:12:27.436 the right term? I mean, it seems like you It 0:12:27.476 --> 0:12:29.276 seems like if I were making up how to do this, 0:12:29.316 --> 0:12:32.156 I'd be like, well, I'm gonna have one model that's 0:12:32.236 --> 0:12:35.196 like cranking out really good deep fikes, but I'll know 0:12:35.236 --> 0:12:36.876 which ones are the deep fis, and then I'm gonna 0:12:36.876 --> 0:12:38.476 feed the deep fis and the real ones to my 0:12:38.516 --> 0:12:41.076 other model, and I'll score it on how well it does, 0:12:41.116 --> 0:12:43.356 and it'll get really good at figuring out the difference. 0:12:43.796 --> 0:12:46.596 Yeah, that's actually exactly how a lot of these work. 0:12:46.676 --> 0:12:48.956 For if you go to there's a website you can 0:12:48.956 --> 0:12:51.316 go where it just generates a person every time you 0:12:51.396 --> 0:12:53.396 go to it a right, and that's actually using again 0:12:53.596 --> 0:12:56.916 to generate that person. So the way we detect and 0:12:56.956 --> 0:12:58.516 I can I can give a little bit more detail here. 0:12:58.556 --> 0:13:02.276 So for example, one of our models which we actually removed, 0:13:02.636 --> 0:13:07.636 was looking at blood flow. So yeah, so imagine actually 0:13:07.676 --> 0:13:11.716 in this video lighting and conditions are right, we can 0:13:11.796 --> 0:13:14.476 actually detect the heartbeat and the blood flow and the 0:13:14.556 --> 0:13:16.476 veins the way we're looking at each other. 0:13:16.916 --> 0:13:19.556 As I'm looking at my weirdly today, maybe because it's 0:13:19.556 --> 0:13:21.436 hot or because the light hair, I can actually see 0:13:21.436 --> 0:13:24.156 a vein bulging on my forehead. So, like you're saying, 0:13:24.156 --> 0:13:28.036 an AI could like measure my pulse from that or something. 0:13:27.996 --> 0:13:30.956 In the right conditions. Now, that model has a lot 0:13:30.996 --> 0:13:36.116 of limitations, and you need to have the right It's 0:13:36.156 --> 0:13:39.276 basically it has a lot of bias. Right, So we 0:13:39.356 --> 0:13:39.796 tossed that. 0:13:40.156 --> 0:13:42.396 Wait, you're saying it didn't work. You're saying it didn't work. 0:13:42.636 --> 0:13:45.876 It worked in the right conditions and the right skin tone, 0:13:46.316 --> 0:13:49.516 so yeah, so otherwise it was biased. So we this 0:13:49.676 --> 0:13:52.476 was experimental and we tossed it. 0:13:52.396 --> 0:13:54.596 A lot of things. It didn't work. So you tried 0:13:54.596 --> 0:13:56.356 it and in a broad way it didn't work. It 0:13:56.396 --> 0:13:58.436 worked in narrow conditions, but you need things that work 0:13:58.476 --> 0:14:01.196 more broadly. What's another thing you tried that didn't work? 0:14:02.356 --> 0:14:05.396 Well, I can tell you every month we may be 0:14:05.516 --> 0:14:06.636 throwing away models. 0:14:06.836 --> 0:14:09.196 Well, presumably there's things that work for a while and 0:14:09.236 --> 0:14:13.236 then they don't. Right, It's kind of like antibiotics versus bacteria, right, 0:14:13.356 --> 0:14:16.236 like your adversaries are getting better every day. 0:14:16.596 --> 0:14:19.116 Basically, what we use, what we like to use is 0:14:19.156 --> 0:14:21.196 we like to say we're like an anti virus company. 0:14:21.476 --> 0:14:25.316 So every time every month there's a new genera of technique, 0:14:25.516 --> 0:14:28.156 maybe we should go detective. But maybe it's something we 0:14:28.196 --> 0:14:30.516 don't anticipate and we don't detect, and so we have 0:14:30.556 --> 0:14:33.436 to make sure we quickly update our models. So and 0:14:33.436 --> 0:14:36.676 then a model that worked last year, it's completely irrelevant now. 0:14:37.156 --> 0:14:40.316 So what else, like, what else is happening technologically on 0:14:40.436 --> 0:14:42.956 the reality defense side, on the detection side. 0:14:43.556 --> 0:14:46.476 Okay, so the way, we have a few different products. 0:14:46.556 --> 0:14:50.436 One is, as I mentioned, real time audio like scanning 0:14:50.436 --> 0:14:53.236 and listening for telephone calls. The other one is a 0:14:53.276 --> 0:14:55.836 place where a journalist or any user can go and 0:14:55.996 --> 0:14:59.516 upload not just videos, but we also detect images. We 0:14:59.556 --> 0:15:03.036 also detect audio, We also detect texts like chat GPT, 0:15:03.596 --> 0:15:06.956 and these tools also explain to a user why something 0:15:07.196 --> 0:15:09.036 is a deep fake. We don't just give a score. 0:15:09.236 --> 0:15:11.476 Or for an image, we might put a heat map 0:15:11.476 --> 0:15:13.876 and see these are the areas that set the model off. 0:15:14.956 --> 0:15:17.796 For text, we might highlight areas and see these other 0:15:17.876 --> 0:15:19.996 areas that appear to be generated. 0:15:20.196 --> 0:15:22.756 There's a case study you have about a university that 0:15:22.876 --> 0:15:27.316 is a client of yours that, among other things, uses 0:15:28.116 --> 0:15:32.636 uses your service to tell when students are turning in 0:15:32.676 --> 0:15:36.396 papers written by chat GIPT. Basically as I read it, right, like, 0:15:36.556 --> 0:15:39.236 I just assume that, like everybody writes papers with chat 0:15:39.316 --> 0:15:41.516 GPT now and there's nothing anybody can do about it. 0:15:41.516 --> 0:15:43.716 But is that not true? Like if I like have 0:15:43.836 --> 0:15:46.196 GPT write my paper and then I like change a 0:15:46.196 --> 0:15:49.876 few words, does that sort of help get let me 0:15:50.116 --> 0:15:51.796 sail past your defense? 0:15:52.316 --> 0:15:55.236 It depends depends how much you change, Yeah, or if 0:15:55.276 --> 0:15:58.756 you change like over fifty percent, maybe maybe would So 0:15:59.036 --> 0:16:00.756 it depends. 0:16:00.476 --> 0:16:02.796 Over fifty percent is more than a few words. And 0:16:02.876 --> 0:16:04.996 so can you talk? I mean, I know you can't 0:16:05.036 --> 0:16:07.196 name the university, but in practice you know how they 0:16:07.276 --> 0:16:11.356 use it. So you know, somefess runs the student's papers 0:16:11.396 --> 0:16:14.316 through your software and it says of when student there's 0:16:14.316 --> 0:16:18.596 a whatever sixty percent chance that this was created using 0:16:18.596 --> 0:16:22.236 a large language model. I mean, do you know in practice? 0:16:22.276 --> 0:16:24.316 Obviously the professor could do whatever they want or the 0:16:24.436 --> 0:16:26.636 university could have whatever policy, but do you know in practice, 0:16:26.756 --> 0:16:30.156 what do they do with this information like that's that's 0:16:30.196 --> 0:16:31.876 in a way a harder one to figure out than 0:16:31.916 --> 0:16:34.436 the like banker who's like, oh, it might be a 0:16:34.436 --> 0:16:36.076 deep fake on the phone. I'll call you right back 0:16:36.116 --> 0:16:38.756 for security. Like if my I don't have a banker, 0:16:38.876 --> 0:16:40.636 but if I had a banker and they did that, 0:16:40.676 --> 0:16:43.076 I'd be like, oh, that's cool. I'm glad my bank 0:16:43.196 --> 0:16:45.796 is doing this thing. Whereas with like the professor and 0:16:45.836 --> 0:16:51.436 the student, that's a much more sort of fraud situation, right, 0:16:52.276 --> 0:16:55.036 and harder to think of how to deal with again 0:16:55.116 --> 0:16:59.156 the probabilistic nature of the output of the model. 0:16:59.476 --> 0:17:01.556 Yes, I think a couple more things here. First of all, 0:17:01.836 --> 0:17:05.436 I think even universities are trying to figure out this problem. 0:17:05.476 --> 0:17:08.076 How to you solve it? You know. But the second 0:17:08.236 --> 0:17:13.156 thing to note, most of our users are not interested 0:17:13.236 --> 0:17:15.636 in a text detector. That seems to be a much 0:17:15.676 --> 0:17:20.076 smaller market. The biggest one is actually audio. It's becoming 0:17:20.916 --> 0:17:22.516 imagine you get a call from a loved one and 0:17:22.836 --> 0:17:24.876 send me money, and you send money if you realize 0:17:24.956 --> 0:17:27.516 is not who it was a defate, right, That's actually 0:17:27.556 --> 0:17:30.636 a much widely used system. 0:17:31.116 --> 0:17:34.436 That's the big one in terms of the business it's interesting. 0:17:34.476 --> 0:17:36.556 I mean, I wonder if that's partly like relative we 0:17:36.596 --> 0:17:38.796 think about the video more, but is it partly because 0:17:39.116 --> 0:17:41.916 deep fake audio is now quite good and there are 0:17:41.956 --> 0:17:44.876 lots of instances where people will transfer lots of money 0:17:44.916 --> 0:17:46.476 based solely on audio. 0:17:47.116 --> 0:17:49.196 De fake audio is the best and it's getting better, 0:17:49.276 --> 0:17:51.996 right interested. I used to go to make your voice, 0:17:51.996 --> 0:17:54.196 maybe I need a minute. Now I need just a 0:17:54.196 --> 0:17:56.996 few seconds and I can make your voice. It's getting 0:17:57.436 --> 0:18:00.396 exponentially better. All of them are, but audio is definitely 0:18:00.676 --> 0:18:01.596 top of the list right now. 0:18:01.716 --> 0:18:05.316 Huh And how are you keeping up? 0:18:06.236 --> 0:18:09.516 Yeah? I mean, so when we detect audio, it's tricky. 0:18:09.516 --> 0:18:13.116 There's a lot of factors to think about a person's accent, 0:18:13.636 --> 0:18:17.076 right what is model biased? Does it not understand or 0:18:17.196 --> 0:18:20.076 is there an issue where it detects It detects one 0:18:20.076 --> 0:18:22.236 person with a certain type accent always as a d thing. 0:18:22.756 --> 0:18:26.116 There's also issues of like noise when when when there's 0:18:26.116 --> 0:18:28.436 a lot of background noise, the model could be impacted. 0:18:28.556 --> 0:18:31.236 When there's cosstop, multiple people speaking at the same time, 0:18:31.556 --> 0:18:34.956 that could impact the model. So there's a variety of factors. 0:18:35.116 --> 0:18:37.036 And the other thing you think about is our models 0:18:37.036 --> 0:18:40.956 are more they support multiple languages, so we don't just 0:18:40.956 --> 0:18:43.636 do English, and so all of these kind of make 0:18:43.676 --> 0:18:46.956 it very complicated. So when we detect something it's called 0:18:46.996 --> 0:18:50.036 pre processing, there's a whole bunch of steps to the 0:18:50.116 --> 0:18:52.796 audio before it actually goes to our AI models where 0:18:52.796 --> 0:18:56.076 we have to clean up the audio, do certain types 0:18:56.076 --> 0:18:58.276 of transformations before we push it to the models. 0:18:58.316 --> 0:19:01.076 And is that happening in real time with these companies? 0:19:01.156 --> 0:19:07.036 Huh huh? And and are you like, what is the 0:19:07.076 --> 0:19:10.236 frontier of preprocessing? Like is it is it an efficiency 0:19:10.236 --> 0:19:12.356 and speed problem because you're trying to do it in 0:19:12.356 --> 0:19:14.876 real time and so you're just trying to kind of 0:19:15.156 --> 0:19:18.036 make the sort of algorithmic part of it as fast 0:19:18.036 --> 0:19:19.156 and efficient as possible. 0:19:19.876 --> 0:19:22.636 Yeah, I mean this is a challenge. There's a lot 0:19:22.676 --> 0:19:25.716 to be done. So that's an ongoing research. How do 0:19:25.796 --> 0:19:28.836 we continue to speed up not just a preprocessing, but 0:19:28.876 --> 0:19:33.156 the inference. And there's a variety of one thing that's 0:19:33.156 --> 0:19:35.076 called a foundation model. I'm not sure if you heard 0:19:35.236 --> 0:19:37.556 what those are, but these are extremely large pre train 0:19:37.636 --> 0:19:40.196 model GPT is a foundation model is a pre train model. 0:19:40.596 --> 0:19:43.556 And so these models can be useful in some parts 0:19:43.556 --> 0:19:47.956 of the preprocessing where they can quickly extract certain features 0:19:47.956 --> 0:19:50.436 for us, and then we can use those two down 0:19:50.436 --> 0:19:55.516 the pipeline. 0:19:54.036 --> 0:19:56.436 Still to come on the show. The problems that Ali 0:19:56.636 --> 0:20:09.996 is trying to solve. Now, how good are you at 0:20:09.996 --> 0:20:12.996 detecting de fikes? Can you quantify how good you are? 0:20:13.956 --> 0:20:16.236 So the way they usually do this is they look 0:20:16.276 --> 0:20:19.876 at benchmarks. Right, there's public data sets which we can 0:20:19.916 --> 0:20:23.396 take and run and we're in the nineties and then 0:20:23.556 --> 0:20:25.476 but you know that's not the real world. 0:20:25.516 --> 0:20:27.556 When you say you're in the nineties, you mean you 0:20:29.156 --> 0:20:33.836 in a binary sense, you guess correctly ninety percent of 0:20:33.876 --> 0:20:34.316 the time. 0:20:35.036 --> 0:20:37.636 Yeah, So on a public benchmark, we're in the nineties. 0:20:37.636 --> 0:20:41.956 There's accuracy, precision and recall. Accuracy is how accurate are 0:20:41.996 --> 0:20:45.436 we Let's say there is one hundred sample set is 0:20:45.436 --> 0:20:50.076 one hundred, maybe fifty is fake, fifty is real? Right. 0:20:50.116 --> 0:20:52.396 The accuracy is you take, okay, how many of those 0:20:52.396 --> 0:20:55.276 did you get right? How many of the real I'm fake? Divided? Right, 0:20:55.836 --> 0:20:58.676