WEBVTT - could you be arrested by a computer? 0:00:13.360 --> 0:00:17.680 December of twenty twenty, during the pandemic cold winter morning, 0:00:18.079 --> 0:00:21.560 the security guard on a train platform and the outskirts 0:00:21.560 --> 0:00:27.200 of Saint Louis was working his job and suddenly saw 0:00:27.320 --> 0:00:31.040 a man in a security booth who wasn't supposed to 0:00:31.080 --> 0:00:33.960 be there. So he approached this man and told him 0:00:34.000 --> 0:00:39.080 to leave, and then the verbal altercation ensued and another 0:00:39.159 --> 0:00:41.919 man walked up behind him, so he was surrounded by 0:00:41.920 --> 0:00:44.480 these two guys who were getting more and more agitated 0:00:44.560 --> 0:00:47.800 with him. The next thing he knew, he was assaulted 0:00:47.880 --> 0:00:51.159 by these two guys, struck in the head, and when 0:00:51.159 --> 0:00:53.680 he was on the ground, they continued to beat him 0:00:53.880 --> 0:00:56.040 and hit him in the head and hit ananimous chest 0:00:56.080 --> 0:01:00.360 and his ribs. The guys ran off and Michael and 0:01:00.400 --> 0:01:04.880 the victim suffered concussion that day and he couldn't remember anything. 0:01:05.440 --> 0:01:08.760 So what the police were really left with to solve 0:01:08.760 --> 0:01:11.840 this case is the same thing that they're left with 0:01:12.120 --> 0:01:15.080 in a lot of cases where there's no witnesses. 0:01:15.280 --> 0:01:19.759 A random crime, a victim with no memory, and nobody 0:01:19.760 --> 0:01:21.040 seems to have seen anything. 0:01:22.040 --> 0:01:24.959 A video camera was the only witness to this crime. 0:01:25.480 --> 0:01:30.560 This kind of fuzzy, blurry surveillance video captured a still 0:01:31.160 --> 0:01:35.000 of these perpetrators, and that was basically the only thing 0:01:35.040 --> 0:01:36.280 that they had to go on. 0:01:36.880 --> 0:01:40.600 Douglas McMillan is a reporter for the Washington Post, So Doug, 0:01:40.920 --> 0:01:43.280 first off, for real, thanks for being down to talk 0:01:43.319 --> 0:01:46.199 with me about all this. Of course, Doug doesn't usually 0:01:46.240 --> 0:01:49.320 cover cops or the justice system. His usual beat at 0:01:49.320 --> 0:01:53.000 the Post is corporate accountability and technology. But there was 0:01:53.000 --> 0:01:55.760 something about this case in Saint Louis that caught us attention. 0:01:56.520 --> 0:02:00.480 There's a lot of crimes, hundreds or thousands of solved 0:02:00.800 --> 0:02:05.480 cases out there where the only evidence was a camera, 0:02:05.640 --> 0:02:08.839 The only evidence was a photo of this in perpetrator. 0:02:09.280 --> 0:02:14.200 And the potential of this technology is we have this 0:02:14.240 --> 0:02:18.200 seemingly magical tool to help solve those crimes. 0:02:18.600 --> 0:02:21.920 The technology he's talking about here is AI facial recognition. 0:02:22.600 --> 0:02:25.880 Police around the country are increasingly using this on cases 0:02:25.960 --> 0:02:28.720 that they can't find a witness for and this was 0:02:28.760 --> 0:02:32.600 one of those cases. The investigation into who had assaulted 0:02:32.600 --> 0:02:35.079 Michael Feldman had pretty much gone cold. 0:02:35.520 --> 0:02:41.840 Until about eight months after the incident. We believe the 0:02:41.880 --> 0:02:45.360 police officers they decided, oh hey, we have these grainy 0:02:45.400 --> 0:02:50.200 surveillance images, let's try to run them through the facial 0:02:50.200 --> 0:02:52.680 recognition system and see if that will give us any matches. 0:02:53.160 --> 0:02:57.320 And so they did that and they got some matches back. 0:02:57.560 --> 0:03:00.200 We believe it was between five and ten maybe of 0:03:00.200 --> 0:03:04.919 different possible matches. And the one they chose, the one 0:03:04.919 --> 0:03:08.560 that they thought looked the most like the perpetrator of 0:03:08.560 --> 0:03:10.520 the crime, or one of the two perpetrators of the crime, 0:03:11.200 --> 0:03:14.359 was this gentleman named Christopher Gatlin. And then they kind 0:03:14.360 --> 0:03:16.200 of were off and running. In this case, they had 0:03:16.200 --> 0:03:21.240 a suspect. Then this was the only thing that police 0:03:21.320 --> 0:03:26.240 had to arrest this man, take away his freedom, you know, 0:03:26.480 --> 0:03:29.160 came down to this AI program. 0:03:29.520 --> 0:03:32.400 I mean, because the thing here that of course we're 0:03:32.440 --> 0:03:35.800 sort of circling around, is that that's not the guy, 0:03:36.080 --> 0:03:37.280 Christopher Gatlin's innocence. 0:03:46.280 --> 0:03:47.240 I'm afraid. 0:03:51.000 --> 0:03:57.400 From Kaleidoscope in iHeart Podcasts. This is kill Switch. I'm 0:03:57.440 --> 0:03:58.120 Dexter Thomas. 0:03:58.920 --> 0:04:36.200 I'm sorry, I'm sorry. Good bye. 0:04:40.080 --> 0:04:45.080 So they went out investigating Christopher Gatlin and the first 0:04:45.120 --> 0:04:49.600 thing they did was print out a picture of his mugshot, 0:04:50.120 --> 0:04:53.880 and they took the picture of Chris Gatlin to the 0:04:54.279 --> 0:04:56.960 apartment building where Michael Feldman lives and they conducted a 0:04:56.960 --> 0:05:01.440 photo lineup with him, and they basically sat Michael Feldman 0:05:01.520 --> 0:05:04.200 down and gave him the photos, put them on the 0:05:04.240 --> 0:05:06.600 table in front of him, and instructed him to kind 0:05:06.600 --> 0:05:08.919 of spend time looking at each one and going through 0:05:09.360 --> 0:05:12.799 these photos, take. 0:05:12.600 --> 0:05:15.839 A look through all of them and see if anything. 0:05:15.600 --> 0:05:16.159 Rings a bell. 0:05:17.160 --> 0:05:20.320 Eight months after the assault, the cops visited the victim, 0:05:20.440 --> 0:05:23.760 Michael Feldman's house, and they brought some pictures with them. 0:05:24.200 --> 0:05:27.400 One of those pictures was of Chris Gatlin. And we 0:05:27.520 --> 0:05:31.040 know exactly what happened next because it's all on record. 0:05:31.839 --> 0:05:35.159 So we got the body camera a video of this 0:05:35.760 --> 0:05:40.520 photo lineup, and the officers who conducted this photo lineup 0:05:40.760 --> 0:05:44.400 did not follow the proper procedures for basically getting an 0:05:44.520 --> 0:05:48.480 affair and impartial identification of this witness. And you can 0:05:48.520 --> 0:05:52.839 see throughout the process when Michael Feldman's going through these photos, 0:05:53.200 --> 0:05:55.839 these police officers, who are really supposed to be just 0:05:55.839 --> 0:05:58.560 standing off to the side and not saying anything, are 0:05:58.680 --> 0:06:02.040 kind of coaching him and giving him kind of little 0:06:02.120 --> 0:06:06.000 clues throughout the process on which guy they want him 0:06:06.040 --> 0:06:06.400 to pick. 0:06:06.600 --> 0:06:10.760 Okay, let's think about before the incident, he went up 0:06:10.760 --> 0:06:14.640 and he talked to him a little bit right, Let's 0:06:14.640 --> 0:06:17.000 start to focus on things before you actually got any 0:06:17.080 --> 0:06:20.560 altercation of it, maybe even before you actually initially went. 0:06:20.520 --> 0:06:22.880 To talk to him. Just take a minute kind. 0:06:22.720 --> 0:06:23.479 Of ponder, think. 0:06:24.360 --> 0:06:26.480 And at one point he actually picks up the picture 0:06:26.480 --> 0:06:29.760 of Chris Gatlan and puts it aside, and he keeps 0:06:29.760 --> 0:06:34.799 on looking through these photos, and at one point he says, 0:06:34.839 --> 0:06:38.600 I want to say it's him pointing to a different guy. 0:06:38.720 --> 0:06:43.080 I want to say, jan I remember my memory is 0:06:44.160 --> 0:06:47.360 I want to say to him. 0:06:47.400 --> 0:06:50.400 When he does that, one of the detectives steps forward 0:06:50.400 --> 0:06:53.200 and says, okay. Instead of, you know, accepting that as 0:06:53.200 --> 0:06:57.279 an identification, the detective encourages him to continue going and 0:06:57.320 --> 0:07:01.039 continue thinking about the interaction. Think about, you know, was 0:07:01.040 --> 0:07:03.520 a guy wearing anything, Was he wearing a hat? Did 0:07:03.560 --> 0:07:07.480 you recognize anything about his face, his eyes, their clothing 0:07:07.560 --> 0:07:08.480 he had. 0:07:10.200 --> 0:07:11.200 I don't remember really. 0:07:11.240 --> 0:07:13.800 I thought he had like a hat on or something, or. 0:07:14.200 --> 0:07:17.720 Stocking something on. So let's picture the picture of these 0:07:17.760 --> 0:07:20.400 two guys wearing that, you know, a stocking cap or something. 0:07:20.400 --> 0:07:22.000 If you need to use your hands, if you've got 0:07:22.120 --> 0:07:23.920 to put hands on the papers. 0:07:23.560 --> 0:07:24.080 That's okay. 0:07:24.440 --> 0:07:26.320 And so you know, he kind of like, you know, 0:07:26.520 --> 0:07:30.320 prods Michael into thinking that he didn't pick the right 0:07:30.360 --> 0:07:33.360 person and then going back and kind of changing his mind, 0:07:33.360 --> 0:07:35.720 And eventually he does go back and he pulls the 0:07:35.760 --> 0:07:37.920 picture of Chris Gatland out and he points to that 0:07:37.960 --> 0:07:41.440 and he says it was him, kid. 0:07:42.560 --> 0:07:42.960 All right. 0:07:43.200 --> 0:07:45.559 I just remember he getting really angry quick. 0:07:46.200 --> 0:07:52.000 Just the mannerism I'm just trying to and the eyes probably. 0:07:53.440 --> 0:07:54.960 How pissed off he got. 0:07:55.920 --> 0:07:56.960 As not to be in that. 0:07:57.200 --> 0:07:58.360 You couldn't stand there. 0:07:59.520 --> 0:07:59.920 I think. 0:08:01.600 --> 0:08:01.960 Okay. 0:08:03.480 --> 0:08:06.240 So police take that identification and they're able to get 0:08:06.280 --> 0:08:09.560 a warrant to arrest Chris Gatlin. They have no other 0:08:09.640 --> 0:08:13.120 evidence other than that an algorithm picked him out. He 0:08:13.240 --> 0:08:18.440 spent seventeen months in jail. So, Doug, I've read about 0:08:18.480 --> 0:08:21.960 Chris Gatland's case, but you've met him in person. What's 0:08:21.960 --> 0:08:22.280 he like. 0:08:22.680 --> 0:08:26.000 He's a very sweet, charming guy, father of four kids. 0:08:26.520 --> 0:08:29.880 He did not have a criminal record, so the reason 0:08:29.920 --> 0:08:34.679 that he was in this system. So the fish recognition 0:08:35.240 --> 0:08:38.640 database in Saint Louis has over two hundred and fifty 0:08:38.679 --> 0:08:44.800 thousand mug shots of people, including people who were pulled 0:08:44.800 --> 0:08:48.240 over for traffic stops and arrested for speeding or arrested 0:08:48.240 --> 0:08:51.840 for very minor things. Saint Louis has a long history 0:08:52.040 --> 0:08:57.800 of over policing, especially black minority, low income populations, handing 0:08:57.800 --> 0:09:00.240 out tickets to people just so that police can meet 0:09:00.280 --> 0:09:04.000 their quotas. So Chris, he had had a few different 0:09:04.160 --> 0:09:08.040 traffic infractions, and he had one burglary charge from a 0:09:08.040 --> 0:09:12.120 few years earlier. It was ultimately dropped right prosecutors. But 0:09:12.160 --> 0:09:13.559 that's the reason that he came out, and that's the 0:09:13.559 --> 0:09:14.800 reason he was in this system. 0:09:15.120 --> 0:09:19.160 It's incredible that you could be in a system that 0:09:19.240 --> 0:09:21.720 would lead you to get arrested based off of a 0:09:21.760 --> 0:09:25.559 traffic ticket. I have a completely clean record, with the 0:09:25.600 --> 0:09:30.679 one exception that a literal decade ago, I was speeding 0:09:30.760 --> 0:09:32.880 to work on the freeway. I was late for work. 0:09:32.920 --> 0:09:35.520 I got pulled over. Now, I didn't get arrested, but 0:09:35.600 --> 0:09:37.880 had somebody decide to go a little harder on me? Sure, 0:09:37.920 --> 0:09:42.680 I suppose they could have. And I'm just imagining myself 0:09:42.800 --> 0:09:46.480 being at home one day and police coming to my 0:09:46.559 --> 0:09:51.839 door and saying, Hey, we're arresting you because we think 0:09:51.880 --> 0:09:53.520 you did something that I've never heard of in a 0:09:53.520 --> 0:09:54.480 place that I don't. 0:09:54.240 --> 0:09:56.000 Go, because you were in this database. 0:09:56.200 --> 0:09:58.760 Yeah, because I was five miles over the speed limit 0:09:58.800 --> 0:10:03.240 one day going for work. Yeah, so maybe this is 0:10:03.240 --> 0:10:06.480 a good time to say that all facial recognition technology 0:10:06.640 --> 0:10:10.040 isn't the same, or maybe more importantly, that it all 0:10:10.080 --> 0:10:13.000 doesn't come from the same place. The program that the 0:10:13.040 --> 0:10:16.160 Saint Louis Cops use searches the internal databases of the 0:10:16.160 --> 0:10:20.440 police department. But there's another very popular tool. It's able 0:10:20.480 --> 0:10:24.440 to pull data straight from the Internet. It's called clearview AI. 0:10:25.720 --> 0:10:33.440 Clearview scrapes together images from Facebook, from LinkedIn, from Venmo accounts, 0:10:33.480 --> 0:10:36.720 from news articles, from all of the public web sources 0:10:37.040 --> 0:10:39.320 that it can find images with people's face. 0:10:39.800 --> 0:10:44.480 Clearview is wild, and they've got billions of photos in there. 0:10:44.840 --> 0:10:47.439 Yeah, they say they have billions of images. And there's 0:10:47.480 --> 0:10:50.320 a question about whether it's legal for them to scrape 0:10:50.480 --> 0:10:52.679 these images and put them in a database, because they 0:10:52.679 --> 0:10:55.600 do not have they never really got permission from anybody 0:10:56.000 --> 0:10:57.080 to get these images. 0:10:57.559 --> 0:11:01.920 Clearview AI's website boats that they've scraped over fifty billion images. 0:11:02.320 --> 0:11:04.920 And again this is just from the general Internet. So 0:11:05.040 --> 0:11:07.680 when the police run a search, you could be in there, 0:11:08.040 --> 0:11:10.440 even if you've never had a run in with law enforcement. 0:11:12.400 --> 0:11:15.320 Do we even know how many police departments are using this. 0:11:16.000 --> 0:11:19.800 No, it's very hard to tell because they're not required 0:11:20.360 --> 0:11:23.080 to disclose this anyway. In most places, they're not required 0:11:23.679 --> 0:11:26.760 to disclose that they're using these tools. So the best 0:11:26.800 --> 0:11:31.079 proxy for that number is what the companies who make 0:11:31.120 --> 0:11:35.280 this software have announced. Clearview AI has said that it 0:11:35.360 --> 0:11:38.600 has thousands. I think they have said over three thousand 0:11:38.920 --> 0:11:43.760 police customers. I did a pretty exhaustive public record search 0:11:44.120 --> 0:11:47.800 for police departments that are using this tools. We only 0:11:47.840 --> 0:11:50.960 came up with a list of about seventy that we 0:11:50.960 --> 0:11:53.960 could verify that are using it in some way, and 0:11:54.080 --> 0:11:58.400 of those, we sought public records on individual cases, and 0:11:58.440 --> 0:12:02.880 we could only get individual case on something like thirty 0:12:02.960 --> 0:12:05.160 or forty of them. Getting to the bottom of you know, 0:12:05.160 --> 0:12:07.760 which police departments are using this, how they're using it 0:12:07.800 --> 0:12:10.080 is very difficult because for the most part, there's no 0:12:10.360 --> 0:12:13.720 requirement for them to make public how and when they're 0:12:13.800 --> 0:12:18.320 using the tools, and that is allowing police to mostly 0:12:18.400 --> 0:12:19.840 use these tools in secrets. 0:12:20.920 --> 0:12:24.840 But sometimes it's not the machines making the mistakes, it's 0:12:24.880 --> 0:12:39.960 the cops. More on that after the break, it's not 0:12:40.000 --> 0:12:42.480 too hard to understand why a police department would be 0:12:42.520 --> 0:12:46.560 interested in facial recognition tools. It seems pretty logical. I mean, 0:12:46.600 --> 0:12:48.800 you're having a hard time crack in a case and 0:12:48.840 --> 0:12:51.400 you either don't have any eyewitnesses, or you can't rely 0:12:51.440 --> 0:12:54.600 on the witnesses you have, or maybe, like with Michael Feldman, 0:12:54.840 --> 0:12:57.520 the victims suffered an injury that affected their memory of 0:12:57.559 --> 0:13:00.200 the event, or maybe too much time has passed for 0:13:00.280 --> 0:13:04.080 somebody to really remember the details. Eyewitness accounts are full 0:13:04.120 --> 0:13:07.840 of problems for law enforcement because human memory is unreliable 0:13:08.040 --> 0:13:11.480 and it can be influenced by people's biases. So there's 0:13:11.520 --> 0:13:15.080 this potential magical solution. You in put a face into 0:13:15.080 --> 0:13:18.120 the system, It searches a database and spits out a match. 0:13:18.520 --> 0:13:21.440 All you gotta do is go find that person and boom. 0:13:21.640 --> 0:13:22.319 Case closed. 0:13:23.040 --> 0:13:27.240 So we know that there are at least eight Americans 0:13:27.760 --> 0:13:32.160 who have been wrongly arrested due to police misuse of 0:13:32.240 --> 0:13:33.160 facial recognition. 0:13:34.240 --> 0:13:37.400 And just to be clear here, when he says wrongly arrested, 0:13:37.559 --> 0:13:40.080 that's not just his opinion. What he means is that 0:13:40.120 --> 0:13:43.400 the police admitted afterwards that they got the wrong person. 0:13:43.880 --> 0:13:47.080 Interestingly, one of those cases this man named Jason Vernaw 0:13:47.200 --> 0:13:50.120 who is based in Miami. In that case, the computer 0:13:50.920 --> 0:13:54.280 got the right person, but the police had fed the 0:13:54.360 --> 0:13:57.800 computer an image of the wrong person. So they fed 0:13:57.840 --> 0:14:00.000 it an image who they thought this was the perpetrator 0:14:00.240 --> 0:14:02.880 of somebody in a bank committing fraud. But it turns 0:14:02.920 --> 0:14:06.080 out they were mistaken and the surveillance image that they 0:14:06.160 --> 0:14:08.360 put in the system was just the wrong person to 0:14:08.400 --> 0:14:08.880 begin with. 0:14:09.040 --> 0:14:11.160 So that was the one time they got it right. 0:14:11.640 --> 0:14:15.120 Well, yeah, so the computer actually picked the right person, 0:14:15.520 --> 0:14:18.160 but the police were relying on this image they got 0:14:18.200 --> 0:14:20.600 from the bank and they just pulled an image from 0:14:20.600 --> 0:14:21.280 the wrong time. 0:14:22.240 --> 0:14:25.160 So the bank pulled an image from their security cameras 0:14:25.400 --> 0:14:28.000 and sent it to the police, and this image was 0:14:28.120 --> 0:14:31.400 Jason Vernow, but it was from a completely different time 0:14:31.400 --> 0:14:36.080 of the day. The facial recognition software had correctly identified Jason, 0:14:36.600 --> 0:14:39.600 but the police failed to make some obvious checks. 0:14:40.120 --> 0:14:44.200 What this case shows was that the police put the 0:14:44.240 --> 0:14:47.360 image in their system and it popped up Jason Vernow 0:14:47.520 --> 0:14:49.920 and they went out and arrested the guy, rather than 0:14:50.520 --> 0:14:53.400 taking a second and saying, okay, is we have a suspect. 0:14:53.760 --> 0:14:56.440 Let's see if, like, can we prove that Jason ver 0:14:56.480 --> 0:14:58.880 Now was tied to this crime. And in the case 0:14:58.920 --> 0:15:02.320 of a financial there is a lot of potential evidence there. 0:15:02.360 --> 0:15:06.040 There's a check that he supposedly cashed, a faulty check. Well, 0:15:06.200 --> 0:15:09.160 this check of his signature matches. There are witnesses, there's 0:15:09.160 --> 0:15:10.960 a bank teller who took let's see if the bank 0:15:11.000 --> 0:15:13.400 teller can confirm that was him. They didn't do any 0:15:13.440 --> 0:15:17.000 of that. They apparently didn't do any real investigative steps. 0:15:17.320 --> 0:15:19.080 They just took the word of the machine and went 0:15:19.120 --> 0:15:24.440 not arrested him. It speaks to kind of this phenomenon 0:15:24.960 --> 0:15:27.400 that seems to be happening with police as they are 0:15:27.600 --> 0:15:32.120 trusting this kind of they're kind of imbuing this software 0:15:32.200 --> 0:15:35.160 with this magical ability to lead them to the right suspect. 0:15:36.200 --> 0:15:40.720 Some researchers call this automation bias, that when a computer 0:15:40.840 --> 0:15:44.200 is telling you an answer, you're more likely to believe 0:15:44.200 --> 0:15:47.040 that that is the correct answer, even at the expense 0:15:47.240 --> 0:15:51.320 of your kind of typical due diligence, your typical you know, 0:15:51.520 --> 0:15:54.680 just rational brain and just normal steps that you should 0:15:54.720 --> 0:15:57.200 think through and follow before you go out and arrest 0:15:57.200 --> 0:15:59.320 somebody take away somebody's freedom. 0:16:00.080 --> 0:16:02.960 Automation bias is just one kind of bias here, though, 0:16:04.080 --> 0:16:07.640 I mean, there was something unique about this particular person though, right, 0:16:07.640 --> 0:16:11.600 compared to the other cases. Yeah, he was white, and 0:16:11.640 --> 0:16:14.400 that's the one that they got the computer got right. Yeah. 0:16:14.680 --> 0:16:15.000 Yeah. 0:16:15.920 --> 0:16:23.400 The facial recognition has been shown to often get certain 0:16:23.400 --> 0:16:24.480 people wrong. Right. 0:16:24.600 --> 0:16:27.600 Yeah. There's two things going on here. One, facial recognition 0:16:27.880 --> 0:16:32.280 has struggled with darker complexions, and part of this is 0:16:32.600 --> 0:16:36.760 due to how these programs are trained. So when the 0:16:36.760 --> 0:16:39.720 first face rerecation algorithms were trained, they were trained on 0:16:39.840 --> 0:16:44.680 lighter skin tones. There's a federal research lab it's called 0:16:44.760 --> 0:16:49.720 NIST NIST that does testing into how well facial recognition 0:16:49.760 --> 0:16:53.720 performs in laboratory settings, and they found in twenty nineteen 0:16:53.800 --> 0:16:59.040 that certain demographic groups, including African Americans, and certain facial 0:16:59.040 --> 0:17:02.160 rection algorithms could be up to one hundred times more 0:17:02.240 --> 0:17:05.919 likely to be mismatched than lighter skin tones. That was 0:17:05.920 --> 0:17:09.480 about six years ago, and the industry says, and clear 0:17:09.560 --> 0:17:12.240 View in other companies says that their algorithms have improved 0:17:12.320 --> 0:17:14.959 dramatically over that time, and there's some evidence that they 0:17:15.000 --> 0:17:20.240 have gotten better. However, we ultimately don't know how accurate 0:17:20.280 --> 0:17:23.560 and how reliable this software is in the way that 0:17:23.840 --> 0:17:27.720 law enforcement use it because it's never been tested in 0:17:27.760 --> 0:17:29.880 the way that law enforcement use it, the way that 0:17:29.880 --> 0:17:34.320 that federal lab tests things. It's looking at basically perfect lighting, 0:17:34.600 --> 0:17:38.760 a perfectly framed profile photo in perfect conditions. How does 0:17:38.760 --> 0:17:42.360 that algorithm do in those settings? Unlike how police use 0:17:42.400 --> 0:17:45.280 these tools, which is, we're going to use this grainy 0:17:45.320 --> 0:17:49.520 surveillance image shot at a weird angle from above, usually 0:17:50.080 --> 0:17:53.359 usually if very poor lighting, and usually the face is 0:17:53.400 --> 0:17:56.040 partly obscure or it could be. In the case in 0:17:56.040 --> 0:17:58.639 Saint Louis, the guy was actually wearing a COVID mask 0:17:58.720 --> 0:18:00.840 that was covering part of his and he was wearing 0:18:00.880 --> 0:18:03.159 a hood that covered another part of his face. So 0:18:03.480 --> 0:18:06.120 this is the typical setting where police are using these tools. 0:18:06.880 --> 0:18:11.280 There's a case where a cartoon came up in a 0:18:11.320 --> 0:18:13.240 match at one point, right. 0:18:13.600 --> 0:18:16.880 Yeah, So because clear VII just has these I guess 0:18:16.920 --> 0:18:19.679 web crawlers that are just scraping the Internet, and so 0:18:19.920 --> 0:18:22.760 the things that come up and the Clearview results sometimes 0:18:23.000 --> 0:18:26.640 are bizarre. So in this one case in Ohio, two 0:18:26.680 --> 0:18:28.399 of the search results that I think we're in the 0:18:28.400 --> 0:18:32.320 top ten of the search results were Michael Jordan just 0:18:32.960 --> 0:18:35.800 a picture of Michael Jordan and I by the way, 0:18:35.800 --> 0:18:38.680 I reached out to Michael Jordan's rep when I published 0:18:38.720 --> 0:18:41.040 this in a story, and she didn't have any comment 0:18:41.080 --> 0:18:44.160 on whether he might have been implicated into crime in Ohio. 0:18:45.920 --> 0:18:48.560 And then the other one was yeah, just like it 0:18:48.640 --> 0:18:51.439 illustrated a cartoon image of a black man. So it 0:18:51.520 --> 0:18:54.119 just makes you like, kind of makes it you scratch 0:18:54.160 --> 0:18:56.000 your head as to you know, what is this tool 0:18:56.040 --> 0:18:59.320 that police are relying on if it's feeding them, you know, 0:18:59.400 --> 0:19:01.760 garbage as results. 0:19:01.600 --> 0:19:04.280 And giving them Michael Jordan committed this crime in the 0:19:04.320 --> 0:19:04.919 top ten. 0:19:04.840 --> 0:19:06.280 Right, which right, Yeah. 0:19:06.040 --> 0:19:09.960 Again a big allegedly here, but I feel pretty sure 0:19:09.960 --> 0:19:12.560 they'd have a tough time connecting him to the scene 0:19:12.560 --> 0:19:14.840 of whatever that that particular crime was. 0:19:15.200 --> 0:19:18.400 Yeah, And if this is such an advanced algorithm, why 0:19:18.440 --> 0:19:20.720 can't their computer figure out that's Michael Jordan and just 0:19:20.960 --> 0:19:23.160 take him out? And why can't a computer figure out 0:19:23.200 --> 0:19:25.919 that that's a cartoon black man and not a real person. 0:19:26.320 --> 0:19:28.600 I just ca't like a lot of the people who 0:19:28.640 --> 0:19:34.520 are who put forward facial recognition as this science, they 0:19:34.560 --> 0:19:36.679 have sort of cast it in the kind of the 0:19:36.720 --> 0:19:40.119 cloak of this is a scientific tool. The police are 0:19:40.160 --> 0:19:42.840 now using a lot of those arguments kind of break 0:19:42.880 --> 0:19:45.000 down when you see stuff like this, when you see, oh, well, 0:19:45.280 --> 0:19:49.199 your scientific, highly advanced tool, you know, brought back a 0:19:49.200 --> 0:19:51.240 picture of you know, cartoon black Man. How do the 0:19:51.320 --> 0:19:52.520 science arrive at that? 0:19:56.000 --> 0:20:00.280 Facial recognition as a technology is probably a whole other episod. 0:20:00.720 --> 0:20:03.480 But if you've ever used face ID to unlock your iPhone, 0:20:03.560 --> 0:20:06.800 you've already used the version of facial recognition on yourself. 0:20:07.400 --> 0:20:09.760 In some ways, the technology isn't all that different from 0:20:09.800 --> 0:20:12.600 how it started in the nineteen sixties, when researchers were 0:20:12.640 --> 0:20:15.399 trying to figure out how to identify facial features like 0:20:15.480 --> 0:20:17.639 the bridge of the nose, the edge of the nose, 0:20:17.880 --> 0:20:21.520 or the eyes and measure the distance between them. They'd 0:20:21.520 --> 0:20:24.679 store all those measurements as data for a single face. 0:20:25.359 --> 0:20:28.399 Back in the sixties, this stuff was pretty rudimentary, but 0:20:28.720 --> 0:20:31.159 AI's made it more accurate because you can train the 0:20:31.200 --> 0:20:35.120 systems on more faces, which means theoretically it's more likely 0:20:35.160 --> 0:20:38.720 to tell your face from someone else's. But you've probably 0:20:38.720 --> 0:20:41.400 had your face ID fail on you before, and that's 0:20:41.480 --> 0:20:44.480 under good conditions. It's just trying to match your face 0:20:44.640 --> 0:20:48.000 with well your face. If you add a few million 0:20:48.080 --> 0:20:51.720 other faces into the mix, the potential for mistakes multiplies 0:20:52.119 --> 0:20:56.080 by a lot. But even with those issues, police are 0:20:56.119 --> 0:21:01.080 increasingly leaning on this technology. It seems like police are 0:21:01.280 --> 0:21:06.480 presenting the fact that facial recognition software has returned a match, 0:21:07.000 --> 0:21:11.879 whether it's true or false, as evidence. But that seems 0:21:12.119 --> 0:21:13.480 it seems backwards, right. 0:21:13.880 --> 0:21:17.200 Yeah, Well, not only that, but it's on the one hand, 0:21:17.320 --> 0:21:21.600 they actually acknowledge that facial recognition is not enough to 0:21:21.640 --> 0:21:25.400 make a case. They say this in their police rules 0:21:25.600 --> 0:21:28.160 and their public statements over and over and over. If 0:21:28.160 --> 0:21:30.800 you hear police talk about facial recognition, they say to 0:21:30.840 --> 0:21:34.040 the public, this is only an investigative tool. We are 0:21:34.080 --> 0:21:36.199 only going to use this to find investigative leads that 0:21:36.240 --> 0:21:38.960 we then go out and corroborate. And in some places 0:21:39.040 --> 0:21:42.560 that's actually the law. There's six states where actually police 0:21:42.600 --> 0:21:45.880 are required to corroborate any leads that they given facial recgnition. 0:21:46.280 --> 0:21:48.840 What we found in looking at cases all around the 0:21:48.840 --> 0:21:51.480 country was that the police were not doing that. Sometimes 0:21:51.480 --> 0:21:54.919 they do, but often they will just take it at 0:21:54.960 --> 0:21:58.080 face value that this software hit is enough, and then 0:21:58.119 --> 0:21:59.800 they will use that to go out and arrest the 0:21:59.800 --> 0:22:03.520 per I encountered this really surprising thing, which is I 0:22:03.560 --> 0:22:06.960 talked to a number of prosecutors and police who told 0:22:07.000 --> 0:22:10.919 me that, well, yeah, we did corroboration. As soon as 0:22:10.920 --> 0:22:13.560 we got the name from the software, we went and 0:22:13.600 --> 0:22:16.880 we looked at that person's other photos and it visually 0:22:16.960 --> 0:22:19.000 looked like the same person. And they said that that 0:22:19.119 --> 0:22:21.920 is corroboration, visual corroboration of a match. 0:22:22.640 --> 0:22:27.359 That sounds like you're corroborating evidence on your own, like 0:22:27.400 --> 0:22:28.920 you're becoming a witness at that point. 0:22:28.960 --> 0:22:31.639 As a police officer, this thing comes up over and 0:22:31.640 --> 0:22:36.240 over is that many people look alike, and humans are 0:22:36.480 --> 0:22:41.200 very bad at distinguishing between two people who are similar. 0:22:42.960 --> 0:22:45.560 You would think that this is where technology could help, 0:22:46.000 --> 0:22:49.800 but AI isn't much better at telling people apart. The 0:22:49.840 --> 0:22:52.399 first known case of a wrongful arrest made by AI 0:22:52.880 --> 0:22:56.439 was Robert Williams in Detroit in twenty twenty. Doug was 0:22:56.480 --> 0:23:01.959 able to get the police interrogation video. William's case is 0:23:02.040 --> 0:23:05.800 really interesting because we have the interrogation video of this 0:23:05.880 --> 0:23:10.119 and I was just watching this and it's incredible. So 0:23:10.520 --> 0:23:12.520 this is you I know you've seen this video. So 0:23:12.520 --> 0:23:16.359 this is where they're you know, these two detectives are 0:23:16.760 --> 0:23:20.280 sitting in this interrogation room. Rob Williams is sitting across 0:23:20.400 --> 0:23:23.760 the table from them, and he's clearly confused. And they 0:23:23.840 --> 0:23:27.960 bring out these papers and I'll play it from here. 0:23:29.880 --> 0:23:34.440 December twenty sixth, around one pm? Where were you. 0:23:37.359 --> 0:23:38.359 Home? Home? 0:23:39.080 --> 0:23:43.560 Can you do me a favorite? Is that you? 0:23:43.560 --> 0:23:45.320 You know? No? 0:23:45.960 --> 0:23:53.560