Speaker 1: Welcome to tech Stuff, a production from iHeartRadio.
Speaker 2: Hey there, and welcome to tech Stuff. I'm your host,
Speaker 2: John that Strickland.
Speaker 1: I'm an executive producer with iHeart Podcasts and how the
Speaker 1: tech are you? It's time for another classics episode. This
Speaker 1: episode originally published on May nineteenth, twenty seventeen. It is
Speaker 1: called the National Facial Recognition Database. Pretty, I would say,
Speaker 1: controversial topic. Well, let's listen in now. Before I dive
Speaker 1: into the topic, I want to make a couple of
Speaker 1: things very clear at the very beginning. First is I'm biased.
Speaker 1: I think the use of facial recognition software is problematic
Speaker 1: even if you have regulations in place. But I'm mostly
Speaker 1: talking about unregulated use because really we haven't a establish
Speaker 1: the rules and policies to guide the use of facial
Speaker 1: recognition software in a law enforcement context. So that's problem
Speaker 1: Number one is I have a very strong opinion about
Speaker 1: this and I'm not going to shy away from that.
Speaker 1: It's really unjustifiable to have unregulated use of facial recognition
Speaker 1: software in law enforcement contexts. So I want to make
Speaker 1: that clear out of the gate that I have this bias,
Speaker 1: and if that's an issue, that's fair, But at least
Speaker 1: I'm being honest, right, I'm not presenting this as if
Speaker 1: it's completely objective, unbiased information. I own this. You don't
Speaker 1: have to tell me. I know it already. Next, this
Speaker 1: is largely going to be a US centric discussion so
Speaker 1: that I can talk about details. But please know that
Speaker 1: there are a lot of these types of systems all
Speaker 1: over the world, not just in the United States, and
Speaker 1: a lot of these places have similar issues to the
Speaker 1: ones I'm going to be talking about here in the US.
Speaker 1: I'll just be focusing more on US stories to make
Speaker 1: specific points because this is where I live, and now
Speaker 1: to explain what I'm actually talking about here. So, back
Speaker 1: in twenty ten, the FBI undertook a project that cost
Speaker 1: more than an estimated one point two billion dollars that's
Speaker 1: billion with a B to replace what was called the
Speaker 1: Integrated Automated Fingerprint System or IAFS. Now, if I had
Speaker 1: been in place since nineteen ninety nine, and I've talked
Speaker 1: about fingerprints in a previous episode, IAFS was an attempt
Speaker 1: to create a usye database of fingerprint records so that
Speaker 1: if you were investigating a crime and you had lifted
Speaker 1: some prints from the crime, you could end up consulting
Speaker 1: this database and see if there are any matches in
Speaker 1: place to give you any leads on your investigation. The
Speaker 1: twenty ten project the FBI undertook was meant to vastly
Speaker 1: expand that capability by adding a lot more data to
Speaker 1: the database, not just fingerprints, but other stuff as well,
Speaker 1: and the new system is called the Next Generation Identification
Speaker 1: or NGI. It includes not just fingerprints, but other biographical
Speaker 1: data and biometrics information, including face recognition technology. So a
Speaker 1: lot of images are included in this particular database. So
Speaker 1: as part of this project, the FBI incorporated the Interstate
Speaker 1: Photo System or IPS, so you have NGI IPS it
Speaker 1: typically is how it's referred to now. That system includes
Speaker 1: images from police cases as well as photos from civil
Speaker 1: civic sources that are not necessarily related to crimes. That's
Speaker 1: not the only way the FBI can scan for a
Speaker 1: match of a photograph they've taken that relates to a
Speaker 1: case in some way to this massive database, but more
Speaker 1: on that in a little bit now. The general process
Speaker 1: of searching for a match follows a pretty simple pattern,
Speaker 1: although the details can be vastly different depending upon what
Speaker 1: facial recognition software you are using at the time. So
Speaker 1: you first start with an image related to a case,
Speaker 1: and this is called the probe photo. It is the
Speaker 1: one you are probing for lack of a better term,
Speaker 1: you don't know the identity of the person in the photograph, typically,
Speaker 1: or at least you might have suspicions, but you don't
Speaker 1: necessarily know for sure. So you've got a picture of
Speaker 1: an unknown person in this photograph. You then scan that
Speaker 1: photo and you use facial recognition software to analyze the
Speaker 1: picture and to try and find a match in this
Speaker 1: larger database. It starts searching all of the images in
Speaker 1: the database looking for any that might be a potential match.
Speaker 1: Depending upon the system and the policies that are in use,
Speaker 1: you could end up with a single photo return to you.
Speaker 1: You could end up with dozens of photos, so these
Speaker 1: would all be potential matches with different degrees of certainty
Speaker 1: for a match. You might remember in episodes I've talked
Speaker 1: about things like IBM's Watson that would come up with
Speaker 1: answers to a question and assign a value to each
Speaker 1: potential answer, and the one that had the highest value,
Speaker 1: assuming it's above a certain threshold, would be submitted as
Speaker 1: the answer. So it's not so much that the computer
Speaker 1: quote unquote knows it has a match. It suspects a
Speaker 1: match based upon a certain percentage as long as it's
Speaker 1: over a threshold of certainty, or you might end up
Speaker 1: with no photos at all. If no match was found
Speaker 1: or nothing ended up being above that threshold, the system
Speaker 1: might say, I couldn't match this photo with anyone who's
Speaker 1: in the database. A study performed by researchers at Georgetown
Speaker 1: University found that one in every two American adults has
Speaker 1: their face captured in an image database that is accessible
Speaker 1: by various law enforcement agencies, including but not limited to
Speaker 1: the IPS. In fact, the IPS has a small number
Speaker 1: of photos compared to the overall number represented by databases
Speaker 1: across the US. Now, this involves agencies at all different levels, federal, state,
Speaker 1: even tribal law for Native American tribes. That ends up
Speaker 1: being about one hundred and seventeen million people in these databases,
Speaker 1: many of whom, in fact large percentage of whom have
Speaker 1: no criminal background whatsoever. Their images are also in these databases,
Speaker 1: and this raises some big concerns about privacy and also accountability.
Speaker 1: So in today's episode, we're going to explore how facial
Speaker 1: recognition software works, as well as talk about the implementation
Speaker 1: for law enforcement and the reaction to this technology, and
Speaker 1: will probably listen to me get upset and a little
Speaker 1: head up about the whole thing in general. All right,
Speaker 1: So first, before we leap into the mess of law enforcement,
Speaker 1: because it is a mess, that's just a fact, let's
Speaker 1: talk first about the technology itself. When did facial recognition
Speaker 1: software get started and how does it work? Well, it's
Speaker 1: related to computer vision, which is a subset of artificial
Speaker 1: intelligence research. If you look at artificial intelligence, a lot
Speaker 1: of people simplify that by meaning, oh, this is so
Speaker 1: that you can teach computers how to think like people.
Speaker 1: But that's actually a very specific definition of a very
Speaker 1: specific type of artificial intelligence. When you really look at
Speaker 1: AI and you break it out, it involves a lot
Speaker 1: of subsets of abilities. One of those is the ability
Speaker 1: for machines to analyze imagery and be able to determine
Speaker 1: what that imagery represents. In a way, you could argue
Speaker 1: it's teaching computers how to understand pictures. It's also really challenging,
Speaker 1: and this is one of the object lessons that I
Speaker 1: use to teach people how Artificial intelligence is really tricky.
Speaker 1: It requires more than just pure processing power. I mean,
Speaker 1: processing power is important, but you can't solve all of
Speaker 1: AI's problems just by throwing more processors at it. You
Speaker 1: have to figure out from a software level how to
Speaker 1: leverage that processing power in a way that gives computers
Speaker 1: this ability to identify stuff based upon imagery. So a
Speaker 1: computer might be able to perform far more mathematical operations
Speaker 1: per second than even the cleverest of humans, but without
Speaker 1: the right software, they can't identify the picture of a
Speaker 1: seagull compared to say, a semi truck. You have to
Speaker 1: teach the computer how to do this. So let's say
Speaker 1: you develop a program that can analyze an image and
Speaker 1: break it down into simple data to describe that image,
Speaker 1: and then you essentially teach a computer what a coffee
Speaker 1: mug looks like. You take a picture of a coffee mug,
Speaker 1: you feed it to a computer, and you essentially say
Speaker 1: this data represents a coffee mug. You then would have
Speaker 1: to try and train the computer on what that actually means.
Speaker 1: The computer does not now know what a coffee mug is.
Speaker 1: It will recognize that specific mug in that specific orientation
Speaker 1: under those specific lighting conditions, assuming that you've designed the
Speaker 1: algorithm properly. But it's way more tricky than that. What
Speaker 1: if in the image that you fed the computer, the
Speaker 1: coffee mugs handle was facing to the left with respect
Speaker 1: of the viewer, but in a future picture the handle
Speaker 1: is off to the right instead of to the left,
Speaker 1: or it's turned around so you can't see the handle
Speaker 1: at all. It's behind the coffee mug. Well, if the
Speaker 1: mug is bigger or smaller, or a different shape, well
Speaker 1: if it's a different color. Image recognition is tough because
Speaker 1: computers don't immediately associate different objects within the same category
Speaker 1: as being the same thing. So if you teach me, Jonathan,
Speaker 1: what a coffee mug is, and you show me a
Speaker 1: couple of different examples saying, this is a coffee mug,
Speaker 1: but this is also a coffee mug, even though it's
Speaker 1: a different size and different shape and a different color,
Speaker 1: I'll catch on pretty quickly and it won't take very
Speaker 1: many coffee mugs for me to figure out. All Right,
Speaker 1: I got the basic idea of what a coffee mug is.
Speaker 1: I know what the concept of coffee mug is now,
Speaker 1: But computers aren't like that. You have to feed them
Speaker 1: thousands of images, both of coffee mugs and of not
Speaker 1: coffee mugs, so that the computer starts to be able
Speaker 1: to pick out the various features that are the essence
Speaker 1: of a coffee mug versus things that are not related
Speaker 1: to being a coffee mug. It takes hours and hours
Speaker 1: and hours of work of training these computers to do it,
Speaker 1: so it's a non trivial task, and this is true
Speaker 1: of all types of image recognition, including facial recognition. Now,
Speaker 1: to get around that problem, you end up sending thousands,
Speaker 1: countless thousands, millions maybe of images of what you're interested
Speaker 1: in while you're training the computer. And the nice thing
Speaker 1: is computers can process this information very very quickly, so
Speaker 1: while it takes a lot, it doesn't take relatively that long,
Speaker 1: it's not as laborious a process as it could be
Speaker 1: if computers were slower at analyzing information. So you might
Speaker 1: remember a story that kind of illustrates the point. Back
Speaker 1: in twenty twelve, there was a network of sixteen thousand
Speaker 1: computers that analyzed ten million images, and as a result,
Speaker 1: it could do the most important task any computer connected
Speaker 1: to the Internet should be expected to do. It could
Speaker 1: then identify cat videos because it now knew what a
Speaker 1: cat was, or at least the features that define catness.
Speaker 1: Catness as in the essence of being a cat, not
Speaker 1: a character from Hunger Games. Even then, there were times
Speaker 1: when a computer would get it wrong. Either it would
Speaker 1: not identify a cat as being a cat, or it
Speaker 1: would misidentify something else as being a cat because its
Speaker 1: features were close enough to cat like for it to
Speaker 1: fool the computer algorithm. A major breakthrough in facial recognition
Speaker 1: algorithms happened way back in two thousand and one. That's
Speaker 1: when Paul Viola and Michael Jones unveiled an algorithm for
Speaker 1: face detection, and it worked in real time, which meant
Speaker 1: that it could recognize a face that it would appear
Speaker 1: on a webcam. And by recognized, I mean it recognized
Speaker 1: that it was a face. It didn't assign an identity
Speaker 1: to the face. It didn't say, Oh, that's Bob, It said, oh,
Speaker 1: that is a face that is in front of the
Speaker 1: webcam right now. The algorithm soon found its way into
Speaker 1: open CV, which is an open source computer vision framework,
Speaker 1: and the open source approach allowed other programmers to dive
Speaker 1: into that code and to make changes and improvements, and
Speaker 1: it helped a rapid prototyping of facial recognition software to
Speaker 1: Other computer scientists who helped advance computer vision further were
Speaker 1: Bill Triggs and Navnit de Lal, who published a paper
Speaker 1: in two thousand and five about the histograbs of oriented gradients. Now,
Speaker 1: that was an approach that looked at gradient orientation in
Speaker 1: parts of an image, and essentially it describes the process
Speaker 1: of viewing an image with attention to edge directions and
Speaker 1: intensity gradients. That's a complicated way of saying the technique
Speaker 1: looks at the totality of a person, and then a
Speaker 1: machine learning algorithm determines whether or not that is actually
Speaker 1: a person or not a person. A bit later, computer
Speaker 1: scientists began pairing computer vision algorithms with deep learning and
Speaker 1: convolutional neural networks or CNNs. To go into this would
Speaker 1: require an episode all by itself. Neural networks are fascinating,
Speaker 1: but they're also pretty complicated, and I've got a whole
Speaker 1: lot of topics to cover today, so we can't really
Speaker 1: dive into it. You can think of an artificial neural
Speaker 1: network as designing a computer system that processes information in
Speaker 1: a way that's similar to the way our brains do.
Speaker 1: The computers are not thinking, but they are able to
Speaker 1: process information in a way that mimics how we process information,
Speaker 1: or a semi close approximation thereof that's a really kind
Speaker 1: of weak way of describing it. But again, to really
Speaker 1: go into detail will require a full episode all by itself. Typically,
Speaker 1: facial recognition software uses feature extraction to look for patterns
Speaker 1: in an image relating to facial features. In other words,
Speaker 1: it searches for features that resemble a face, the elements
Speaker 1: you would expect to be present in a typical face,
Speaker 1: So eyes, nose, a mouth, that would be major ones. Right.
Speaker 1: Then the software starts to estimate the relationships between those
Speaker 1: different elements. How wide are the eyes, how far apart
Speaker 1: are they from each other, How wide is the nose,
Speaker 1: how long is the jawline, what shape are the cheekbones?
Speaker 1: These sort of elements all play a part as points
Speaker 1: of data, and different facial recognition software packages weight these
Speaker 1: features in a different way. So it's not like I
Speaker 1: could say all facial recognition software looks at these four
Speaker 1: points of data as its primary source. It varies depending
Speaker 1: on upon the algorithm that's been designed by various companies,
Speaker 1: and part of the problem that we're going to talk
Speaker 1: about is that law enforcement across the United States they
Speaker 1: are not relying on a single facial recognition software approach.
Speaker 1: Different agencies have different vendors that they work with, So
Speaker 1: just because one might work very well doesn't necessarily mean
Speaker 1: it's competitors work just as well. And that's part of
Speaker 1: the problem. Now, all of these little points of data
Speaker 1: I'm talking about, these notle points and how they relate
Speaker 1: to one another, all of that gets boiled down into
Speaker 1: a numeric code that you could think of as a
Speaker 1: face print. This is supposed to be a representation of
Speaker 1: the unique set of data that is a compilation of
Speaker 1: all of these different points boiled down into numeric information itself.
Speaker 1: Then what you would do is you would have a
Speaker 1: database of face So if you wanted to find a match,
Speaker 1: you would feed the image you have, the probe image
Speaker 1: into this database, and the facial recognition software would analyze
Speaker 1: the probe photo. It would end up assigning this numeric
Speaker 1: value and would start looking through the database for other
Speaker 1: numeric values that were as similar to that probe one
Speaker 1: as possible and start returning those images as potential matches
Speaker 1: or candidates. They tend to use the word candidate photos.
Speaker 1: Otherwise you'll either get no match at all or you
Speaker 1: get a false positive. You will end up getting an
Speaker 1: image of someone who looks like the person whose image
Speaker 1: you submitted, but is not the same person. That does happen,
Speaker 1: And that's the basic way that facial recognition software works.
Speaker 1: But keep in mind, different vendors use all their own
Speaker 1: specific approaches, like I said, and some could be less
Speaker 1: accurate than others. Some might be accurate for specific ethnicities
Speaker 1: and not as accurate as other ones. That's a huge problem,
Speaker 1: so it gets complicated. Even when I'm talking in more
Speaker 1: general terms, you have to remember that there are a
Speaker 1: lot of specific incidents and specific implementations of facial recognition
Speaker 1: software that have their own issues. So I'm gonna be
Speaker 1: as general as I can. I'm not going to call
Speaker 1: out any particular facial recognition software vendors out there. I'm
Speaker 1: more going to talk about the overall issues that various
Speaker 1: organizations have had as they've looked into this topic. Now,
Speaker 1: there are plenty of applications for facial recognition that have
Speaker 1: nothing to do with identifying a person. I mentioned that
Speaker 1: earlier that there was the one for a webcam that
Speaker 1: could identify when a face was in front of the webcam.
Speaker 1: This wasn't to identify anybody. It was again just to say, yes,
Speaker 1: there's somebody looking into the webcam at this moment, which
Speaker 1: by itself can be useful and have nothing to do
Speaker 1: with identification. There are plenty of digital cameras out there
Speaker 1: and camera phone apps that can identify when there's a
Speaker 1: face looking at the camera, and again it's not necessarily
Speaker 1: to identify that person, but rather to say, oh, well,
Speaker 1: this is a face. The camera is most likely trying
Speaker 1: to focus on this person, so let's make this person
Speaker 1: the point of focus and not focus on something in
Speaker 1: the background like a tree that's fifty yards back. Instead,
Speaker 1: let's focus on the person who's in the foreground. So
Speaker 1: that's pretty handy, and again there's nothing particularly problematic from
Speaker 1: an identification standpoint, because that's not the purpose of it.
Speaker 1: But then you also have other implementations, like on social media,
Speaker 1: which allow you to do things like tag people based
Speaker 1: upon an algorithm recognizing a person. So Facebook is a
Speaker 1: great example of this. Right, if you upload a picture
Speaker 1: of one of your Facebook friends onto Facebook chances are
Speaker 1: it's giving you a suggestion to tag that photo with
Speaker 1: the specific in mind. That may not be that problematic either,
Speaker 1: depending upon how your friend feels about pictures being uploaded
Speaker 1: to Facebook. Some people are very cautious about that, and
Speaker 1: of course you know, I always recommend you talk to
Speaker 1: anybody before you start tagging folks on Facebook photos, just
Speaker 1: to make sure they're fine with it. I say that
Speaker 1: as a person who has done it, and then notice
Speaker 1: that some of my tags got removed by the people
Speaker 1: I tagged later on, which taught me I should probably
Speaker 1: ask first, rather than give them the feeling that they
Speaker 1: need to go and remove a tag or two. We've
Speaker 1: also seen examples of this simple implementation of facial recognition
Speaker 1: going awry. Google's street View will blur out faces, for example,
Speaker 1: in an effort to protect people's identity while street view
Speaker 1: cars are out and about taking images. This makes sense.
Speaker 1: Let's say that you are in a part of town
Speaker 1: that you normally would not be in. For whatever reason,
Speaker 1: you might not want your picture to be included on
Speaker 1: Google street View, so that whenever anyone looks at that
Speaker 1: street for that point forward, they see your face on there,
Speaker 1: you know, coming out of I don't know a Wendy's.
Speaker 1: Maybe you are a manager for burger King that would
Speaker 1: look bad, or you know, lots of other reasons that
Speaker 1: obviously can spring to mind as well. You don't want
Speaker 1: to violate someone's privacy. But Google StreetView would also blur
Speaker 1: out images that were not real people faces, like images
Speaker 1: on billboards or murals. Sometimes if it had a person's
Speaker 1: face on a mural, the face would be blurred out,
Speaker 1: even though it's not a real person, it's just a
Speaker 1: painting or In September twenty sixteen, c Neet reported on
Speaker 1: an incident in which Google street View blurred out the
Speaker 1: face of a cow. So Google was being very thoughtful
Speaker 1: to protect that cow's privacy. But what about matching faces
Speaker 1: to identities? So in some cases, again seemingly harmless if
Speaker 1: you want to tag your friends, but when it comes
Speaker 1: to law enforcement, things get a bit sticky, particularly as
Speaker 1: you learn more about the specifics. And we'll talk about
Speaker 1: that in just a second, but first let's take a
Speaker 1: quick break to thank our sponsor. All right, let's first
Speaker 1: start with the FBI's Interstate Photos System, or IPS, because
Speaker 1: this one has perhaps the least controversial elements to it
Speaker 1: when you really look at it, it's still problematic, but
Speaker 1: not nearly as much as the larger picture. The system
Speaker 1: contains both images from criminal cases like mugshots and things
Speaker 1: of that nature, but it also includes some photos from
Speaker 1: civil sources like ID applications, that kind of thing. When
Speaker 1: the Government Accountability Office or GAO, they're gonna be a
Speaker 1: lot of acronyms and initializations or initialisms, I should say
Speaker 1: in this episode, so I apologize for that. But Government
Speaker 1: Accountability Office they did a study on this matter just
Speaker 1: in twenty sixteen, so not that long ago. They published
Speaker 1: its report on facial recognition software use among law enforcements,
Speaker 1: specifically the FBI because they're a federal agency, so they
Speaker 1: were concerned with the federal use of this. The database
Speaker 1: contained about thirty million photos at the time of the
Speaker 1: GAO study, so thirty million pictures are in this database.
Speaker 1: Most of those images came from eighteen thousand different law
Speaker 1: enforcement agencies at all levels of government, that includes the
Speaker 1: tribal law enforcement offices. About seventy percent of all the
Speaker 1: photos in the database were mugshots. More than of the
Speaker 1: photos in that database are from criminal cases, so that
Speaker 1: means that less than twenty percent were from civil sources.
Speaker 1: In addition to that, there were some cases, plenty of them,
Speaker 1: where the database had images of people both from a
Speaker 1: civil source and from a criminal source. So I'll give
Speaker 1: you a theoretical example. Let's say that sometime in the
Speaker 1: past I got nabbed by the cops for grand theft
Speaker 1: auto because I play that game. But let's say that
Speaker 1: I stole a car, which we already know is a
Speaker 1: complete fabrication because I don't even drive. But let's say
Speaker 1: I stole a car, and that I had moved the
Speaker 1: car across state lines. It became a federal case. Therefore,
Speaker 1: my criminal information is included. My mugshot would be included
Speaker 1: in this particular database. On related note, my ID also
Speaker 1: is in that database as a civil image, not as
Speaker 1: a criminal image. Well, in my case, they would tie
Speaker 1: those two images together because they refer to the same
Speaker 1: person and I had been involved in a criminal act.
Speaker 1: So while I would have an image in there from
Speaker 1: a civil source, it would be filed under the criminal
Speaker 1: side of things. This is important when we get to
Speaker 1: how the probes work. Now, let's say you have been
Speaker 1: perfectly law abiding this whole time, and that your ID
Speaker 1: is also in this database, but it's just under the
Speaker 1: civil side of things. Since you don't have any criminal background,
Speaker 1: it's not connected to anything on the criminal side, So
Speaker 1: when it comes to probes using the IPS, your information
Speaker 1: will not be referenced because the FBI policy is when
Speaker 1: it's running these potential matches with a photo that's been
Speaker 1: gathered as part of the evidence for an ongoing investigation,
Speaker 1: they can only consult the criminal side, not the civil side,
Speaker 1: with the exception of any civil photos that are connected
Speaker 1: to a criminal case, as in my example, those are
Speaker 1: fair game. So it might run a match and it
Speaker 1: turns out that my photo for my state given identification
Speaker 1: card is a better match than the mugshot is. That's
Speaker 1: going to be fine because those two things were both
Speaker 1: attached to a criminal file in the first place. But
Speaker 1: let's say that it would have matched up against you
Speaker 1: since you didn't have a criminal background, and since the
Speaker 1: only record in there was a civil source, the match
Speaker 1: would completely skip over you. It wouldn't return your picture
Speaker 1: because your image is off limits in that particular use
Speaker 1: very important because it's an effort to try and make
Speaker 1: sure this facial recognition technology is focusing just on the
Speaker 1: criminal side, not putting law abiding citizens in danger of
Speaker 1: being pulled up in a virtual lineup, at least not
Speaker 1: using that approach. That's the problem is that that's not the
Speaker 1: only way the FBI runs searches. In fact, that might
Speaker 1: not be the primary way the FBI runs searches when
Speaker 1: they're looking for a match to a photo that was
Speaker 1: taken as part of evidence gathering in pursuing a case.
Speaker 1: But let's say that you are an FBI agent and
Speaker 1: you've got a photo, a probe photo, and you want
Speaker 1: to run it for a match. What's the procedure. You
Speaker 1: would send off your request to the NGI dash Ips Department,
Speaker 1: and you would have to indicate how many potential photographs
Speaker 1: you want back, how many candidates do you want. You
Speaker 1: can choose between two candidate photos and fifty candidate photos.
Speaker 1: These are photos of different individuals, by the way, not
Speaker 1: just here's a picture of Jonathan on the beach. Here's
Speaker 1: a picture of Jonathan in the woods. No, it's more like,
Speaker 1: here's a picture of Jonathan. Here's a picture of a
Speaker 1: person who's not Jonathan, but also kind of matches this
Speaker 1: particular probe photo you submitted. And here are forty eight others.
Speaker 1: The default is twenty, so if you don't change the
Speaker 1: default at all, you will get back twenty images that
Speaker 1: are potential candidates matching your probe photo, assuming that any
Speaker 1: are found at all. It is possible that you submit
Speaker 1: a probe photo and the system doesn't find any matches
Speaker 1: at all, and which case you'll just get a null.
Speaker 1: You might get less than what you asked for if
Speaker 1: only a few had met the threshold for reliability. Now
Speaker 1: we call them candidate photos because you're supposed to acknowledge
Speaker 1: the fact that these are meant to help you pursue
Speaker 1: a lead of inquiry. In a case, it is not
Speaker 1: meant to be a source of positive identification of a suspect.
Speaker 1: So in other words, you shouldn't run a facial recognition
Speaker 1: software probe, get a result back and say that's our guy,
Speaker 1: let's go pick him up. That's not enough. It's meant
Speaker 1: to be the start of a line of inquiry, and
Speaker 1: whether or not it gets used that way all the
Speaker 1: time is another matter. But the purpose of calling it
Speaker 1: candidate photo is to remind everyone this is not meant
Speaker 1: to be proof of someone's guilt or innocence. The FBI
Speaker 1: also allows certain state authorities to use this same database,
Speaker 1: and different agencies have different preferences. So in the GAO
Speaker 1: report that I talked about earlier, the authors noted that
Speaker 1: law enforcement officials from Michigan, for example, would always ask
Speaker 1: for the maximum number of candidate photos, particularly when they'd
Speaker 1: use probe images that were of low quality. So let's
Speaker 1: say you've got a picture captured from a security camera
Speaker 1: and the lighting is pretty bad and perhaps the person
Speaker 1: wasn't facing dead on into the camera. You might ask
Speaker 1: for the maximum number of candidate photos to re return
Speaker 1: to you, knowing that the image you submitted was low quality,
Speaker 1: and therefore any match is only potentially going to be
Speaker 1: the person you're actually looking for. And again, this is
Speaker 1: all just to help you with the beginning of your investigation.
Speaker 1: It's not meant to be the that's our guy moment
Speaker 1: that you would see and say police procedural that would
Speaker 1: appear on network television in primetime. The FBI I also
Speaker 1: has a policy in that all returned candidate photos must
Speaker 1: first be analyzed by human specialists before being passed on
Speaker 1: to other law enforcement agencies. Up to that point, the
Speaker 1: entire process is automatic, so you don't have people overseeing
Speaker 1: the process once it's probing all of the database, but
Speaker 1: once the results come in, human analysts, who are supposed
Speaker 1: to be trained in this sort of thing, are supposed
Speaker 1: to look at each of those returned candidates and determine
Speaker 1: if whether or not they really do resemble the person
Speaker 1: in the probe photo that was submitted in the first place,
Speaker 1: and if they're not, they are not supposed to be
Speaker 1: passed on any further down the chain. Now, so far,
Speaker 1: this probably doesn't sound too problematic. The FBI has a
Speaker 1: database containing both criminal and civil photographs, but when it
Speaker 1: runs a probe, it can only use the criminal photos
Speaker 1: or the civil ones that are attached to criminal files.
Speaker 1: Candidate photos are supposed to only be used to help
Speaker 1: start a line of inquiry, not to positively identify suspects,
Speaker 1: and everything has to be reviewed by human being. That
Speaker 1: sounds fairly reasonable. But even if you're mostly okay with
Speaker 1: this approach, which still has some problems we'll talk about
Speaker 1: in a bit, things get significantly more dicey as you
Speaker 1: learn more about the FBI's policies. For example, they have
Speaker 1: a unit called the Facial Analysis Comparison and Evaluation Services
Speaker 1: or face FACE. This is a part of the Criminal
Speaker 1: Justice Information Services Department CG. Rather I yeah, I can
Speaker 1: spell justice with a G. It doesn't make sense. No,
Speaker 1: the cjis. This is a department within the FBI, and
Speaker 1: FACE can carry out a search far more wide reaching
Speaker 1: than one that just uses the ngi IPS database. FACE
Speaker 1: uses not only that database but also external databases when
Speaker 1: conducting a search with a probe photo. So let's say again,
Speaker 1: you're an FBI agent and you have an image that
Speaker 1: you want to match. You want to find out who
Speaker 1: this person is. Maybe it's just a person of interest,
Speaker 1: doesn't even necessarily have to be a suspect. Could be that, hey,
Speaker 1: maybe this person can tell us more about this thing
Speaker 1: that happened later on. Well, you could follow the NGIIPS procedure,
Speaker 1: which would focus on those criminal photographs, or you could
Speaker 1: submit your image to face. Face then would search dozens
Speaker 1: of databases holding more than four hundred eleven million photographs,
Speaker 1: many of which are from civil sources. So NGIIPS has
Speaker 1: thirty million, all of them together have four hundred eleven
Speaker 1: million pictures. And again a lot of those pictures just
Speaker 1: come from things like passport ID, driver's licenses, sometimes security clearances,
Speaker 1: that sort of stuff. That's this database has a lot
Speaker 1: of law abiding citizens who have no criminal record, and
Speaker 1: the images have nothing to do with any sort of
Speaker 1: criminal act, but they're in these databases. These external databases
Speaker 1: belong to lots of different agencies, and both at the
Speaker 1: federal level and state level. So you've got state police agencies,
Speaker 1: You've got the Department of Defense, You've got the Department
Speaker 1: of Justice, you have the Department of State, and again
Speaker 1: it contains photos from licenses, passports, security ID cards, and more.
Speaker 1: So your submission would then go to one of twenty
Speaker 1: nine different biometric image specialists. They would take that probe
Speaker 1: photo and run a scan through these various databases and
Speaker 1: they would look for matches. Here's another problem. Each of
Speaker 1: these systems has a different methodology for performing and returning
Speaker 1: search results, which makes this even more complicated. For example,
Speaker 1: I talked about how the ngi IPS system gives you
Speaker 1: a return between two and fifty candidate photos. Right, Well,
Speaker 1: the Department of State will return as many as eighty
Speaker 1: eight candidate photos if they are all from visa applications
Speaker 1: from people who are not US citizens. So you can
Speaker 1: get up to eighty eight pictures from visa applicants, or
Speaker 1: you could just get three images from US citizen passport applicants,
Speaker 1: because that's a hard limit. They can only return three
Speaker 1: candidate photos from US citizens who applied for passports, but
Speaker 1: they can return up to eighty eight visa application photos.
Speaker 1: The Department of Defense will will down all of their
Speaker 1: candidates into a single entry. So, in other words, Diberna Defense,
Speaker 1: if you query that database with your probe photo, you
Speaker 1: will only get one image back, so they will call
Speaker 1: all the other ones and give you the most likely
Speaker 1: match out of all the ones that they find in
Speaker 1: their search. Some states will do similar things where they
Speaker 1: will narrow down which images they will return to you.
Speaker 1: Some of them will just give you everything they've got.
Speaker 1: Every match that comes up, they'll just return it back
Speaker 1: to the FBI. So it's very complicated. You can't really
Speaker 1: be sure what methods people are using to be certain
Speaker 1: that the potential matches they have represent a good match,
Speaker 1: a good chance that the person that they've returned is
Speaker 1: actually the same one who is in the probe photo.
Speaker 1: At any rate, you as an FBI agent, wouldn't get
Speaker 1: all of these at all, all of these photos that
Speaker 1: would come back, They would come back to that biometric
Speaker 1: analyst over at face, So you send your request to
Speaker 1: face face takes care of the rest. They get back
Speaker 1: all these results. Then they go through the results they
Speaker 1: get back and they whittle that down to one or
Speaker 1: two candidate photos and they send those on to you,
Speaker 1: the FBI agent. So by the time you get it,
Speaker 1: you only see one or two out of the potentially
Speaker 1: more than one hundred images that were returned on this search.
Speaker 1: But you might ask, well, how frequently does this happen?
Speaker 1: I mean, how often is the FBI looking at images,
Speaker 1: including pictures of law abiding citizens in these virtual lineups.
Speaker 1: It can't be that frequent, right, Well, again, according to
Speaker 1: that GAO report, the FBI submitted two hundred fifteen thousand
Speaker 1: searches between August twenty eleven, which is pretty much when
Speaker 1: the program went into pilot mode and started to be
Speaker 1: rolled out more widely, through December twenty fifteen two hundred
Speaker 1: and fifteen thousand. From August twenty eleven to December twenty fifteen,
Speaker 1: thirty six thousand of those searches were on state driver's
Speaker 1: licensed databases. So it happens a lot thirty six thousand times.
Speaker 1: Chances are if you are an adult in America, you
Speaker 1: got like a coin flip situation that your image was
Speaker 1: looked at at some time or another by an algorithm
Speaker 1: comparing it to a probe photo in the pursuit of
Speaker 1: information regarding a federal case or in some cases, state cases,
Speaker 1: because the FBI has also allowed certain states law agencies
Speaker 1: access to this approach. Now, according to the rules, the
Speaker 1: FBI should have submitted some important documents to inform the
Speaker 1: public of their policies and to lay down the regulations,
Speaker 1: the rules, the processes that they would have to follow
Speaker 1: in order for this to be fair, for it to
Speaker 1: not encroach on your privacy or to violate civil liberties
Speaker 1: or civil rights. Without those rules, the use of the
Speaker 1: system is largely unread, which can lead to misuse, whether
Speaker 1: it's intentional or otherwise. The Government Accountability Office specifically pointed
Speaker 1: out two different types of notifications that the FBI either
Speaker 1: failed to submit or was just very late in submitting.
Speaker 1: The first is called a Privacy Impact assessment or PIA. Now,
Speaker 1: as that name suggests, a PIA is meant to inform
Speaker 1: the public about any potential conflicts with privacy with regards
Speaker 1: to methods for collecting personal information. The FBI did submit
Speaker 1: a PIA for its next generation system, but they did
Speaker 1: it back in two thousand and eight when they first
Speaker 1: launched the NGIIPS. According to the Government Accountability Office, the
Speaker 1: FBI made enough significant changes to the system to warrant
Speaker 1: another PIA that anytime you make a significant revision to
Speaker 1: your personal information systems, you have to submit a new
Speaker 1: because things have changed, and according to the GAO, the
Speaker 1: FBI failed to do that for way too long. Now
Speaker 1: ultimately the FBI would publish a new PIA, but by
Speaker 1: that point, the Government Accountability Office said they had delayed
Speaker 1: so long that it made it more problematic as a result,
Speaker 1: because during the whole time that they were supposed to
Speaker 1: have submitted this, they were actively using this system. It
Speaker 1: wasn't like this was a system being tested. It was
Speaker 1: actually being put to use in real cases. And that
Speaker 1: kind of violates it, well, it doesn't. Kind of. It
Speaker 1: violates a Privacy Act of nineteen seventy four, which states
Speaker 1: that when you make these revisions, you're supposed to file
Speaker 1: a PIA before you put it into use. According to
Speaker 1: the GAO, the FBI failed to do so, and also
Speaker 1: the longer you wait to file this the more entrenched though,
Speaker 1: those uses come. So if you put a system in place,
Speaker 1: you build everything out, you've actually taken the time to
Speaker 1: do it, and then you publish a PIA any objections
Speaker 1: that are raised, you could say, well, we've got a
Speaker 1: system now, and it costs one point two billion dollars
Speaker 1: to put it in place. It's going to cost more money,
Speaker 1: taxpayer money for us to alter it, to remove it,
Speaker 1: to change it. You could argue against any move to
Speaker 1: amend the situation. And the GAO says, that's not playing
Speaker 1: cricket or playing fair for my fellow Americans. So that's
Speaker 1: a problem. But then there's another one. There's a second
Speaker 1: type of report called a Systems of Records Notice or
Speaker 1: sor in SORN. The Department of Justice was required to
Speaker 1: submit a SORN upon the launch of NGIIPS, but didn't
Speaker 1: do so until May fifth, twenty sixteen. The GAO criticized
Speaker 1: both the FBI and the Department of Justice for failing
Speaker 1: to inform the public of the nature of this technology
Speaker 1: and how it might impact personal privacy. But wait, there's more.
Speaker 1: The GAO report also accused the FBI of failing to
Speaker 1: perform any audits to make certain the use of facial
Speaker 1: recognition software isn't in violation of other policies, or even
Speaker 1: to make sure it doesn't violate the Fourth Amendment rights
Speaker 1: of US citizens. Now, for those of you who are
Speaker 1: not US citizens, you might wonder what does this actually mean. Well,
Speaker 1: the Fourth Amendment is supposed to protect us against unreasonable
Speaker 1: search and seizure, and part of that means law enforcement
Speaker 1: can't just demand to search you for no reason. And
Speaker 1: some have argued that using facial recognition software without a
Speaker 1: person's consent, using it invisibly and widespread essentially amounts to
Speaker 1: crossing that line. Now, in the United States, we've got
Speaker 1: plenty of examples of troublesome policies that seem to overstep
Speaker 1: the bounds that are established by the Fourth Amendment. But
Speaker 1: that's a tirade for an entirely different show, probably not
Speaker 1: a tech stuff, maybe a stuff they don't want you
Speaker 1: to know. There are a couple of laws in the
Speaker 1: United States that are important to take note of here
Speaker 1: besides that Fourth Amendment. One of them I just mentioned
Speaker 1: the Privacy Act of nineteen seventy four, and the other
Speaker 1: one is the e Government Act of two thousand and two.
Speaker 1: The Privacy Act sets limitations on the collection, disclosure, and
Speaker 1: use of personal information maintained in systems of records, including
Speaker 1: the ones that law agencies use. The e Government Act
Speaker 1: is the one that requires government agencies to conduct pias
Speaker 1: to make certain that personal information is handled properly in
Speaker 1: federal systems, and the GAO report alleges that the FBI
Speaker 1: policy wasn't aligned with either of those. Now, part of
Speaker 1: this accusation depends upon the fact that the FBI was
Speaker 1: using face in investigations for years before they updated their SORN.
Speaker 1: They're sworn. According to the Privacy Act, agencies must publish
Speaker 1: a new SORN upon the establishment or revision of the
Speaker 1: system of records. This is what I was talking about earlier,
Speaker 1: except I think I said PIA earlier when actually I
Speaker 1: met sor In. That's entirely my fault because I didn't
Speaker 1: write in my notes and I was talking next to boraneously.
Speaker 1: But SORN is what I should have said. The FBI
Speaker 1: argued that it was continuously updating the database to refine
Speaker 1: the system, but the GAO's argument was that you could
Speaker 1: be continuously updating the system and argue, well, we don't
Speaker 1: want to publish an sor in after every tiny revision
Speaker 1: because it's wasteful and time consuming. The GAOS counter to
Speaker 1: that is, yeah, but you were using this tool in
Speaker 1: actual cases. If you were developing this, let's say, in
Speaker 1: a department where you're not using real cases, you're just
Speaker 1: gradually tweaking the system so that it's more and more
Speaker 1: accurate in a controlled environment. That's one thing. But if
Speaker 1: you're actively making use of the system in real world investigations,
Speaker 1: you absolutely must adhere to these laws, because to do
Speaker 1: otherwise is in violation to laws that are passing the
Speaker 1: United States. So you can't have it both ways. You
Speaker 1: can't continuously tweak a system and put it to official
Speaker 1: use and not also file these reports. You could argue
Speaker 1: the FBI was trying to have its cake and eat
Speaker 1: it too, So the expression that I think I actually
Speaker 1: use properly. All Right, we've got more to talk about,
Speaker 1: but it's time for us to take another quick break
Speaker 1: to thank our sponsor. All right, So, the Government Accountability
Speaker 1: Office criticizes the FBI and various other agencies for failing
Speaker 1: to establish the scope and use of its facial recognition technology.
Speaker 1: But that's just the tip of the iceberg. Because the
Speaker 1: GAO report goes on to make an equally troubling point
Speaker 1: that the FBI had performed only a few studies on
Speaker 1: how accurate these facial recognition systems were in the first place. So,
Speaker 1: in other words, not only was this a poorly defined
Speaker 1: and unregulated tool, but it's a tool of unknown accuracy
Speaker 1: and precision, which is terrifying when you think about it now.
Speaker 1: According to the report, the FBI did perform some initial
Speaker 1: tests before they deployed the ngiibs, and then occasionally did
Speaker 1: a couple of tests when they made some changes. But
Speaker 1: there were problems with these tests. For one thing, they
Speaker 1: were limited in scope and they didn't represent how the
Speaker 1: system might be used out in the real world. When
Speaker 1: they were actually running these tests, they ran on about
Speaker 1: nine hundred thousand photographs in the database, so they took
Speaker 1: a subset of the photos that they had. They took
Speaker 1: nine hundred thousand of them, and they ran probe tests
Speaker 1: using photos that they knew either were or were not
Speaker 1: represented in that group of nine hundred thousand. However, you've
Speaker 1: got to remember the full database is more than thirty
Speaker 1: million images, so something that works on a smaller scale
Speaker 1: may not work once you scale it up for another
Speaker 1: The tests did not specify how often incorrect matches would
Speaker 1: come back, so you didn't know how many false positives
Speaker 1: were there because the FBI wasn't tracking false positives. They
Speaker 1: were only concerned with how frequently they were getting a
Speaker 1: match to an actual image. So the way they test
Speaker 1: this is, you've got nine hundred thousand images, they've got
Speaker 1: a probe image, They know for a fact that the
Speaker 1: probe image is inside that database, and then they run
Speaker 1: the search to see if the system sends that image back.
Speaker 1: And their threshold was an eighty five percent detection rate
Speaker 1: for a positive match. So, in other words, it went
Speaker 1: like this, Let's say you need to conduct a test
Speaker 1: of this system. This is one way you would determine
Speaker 1: whether or not you had that eighty five percent detection rate.
Speaker 1: Let's say you have one hundred probe photos that you've
Speaker 1: taken of one person, and you know this person's face
Speaker 1: is in that database. You know it's going to be
Speaker 1: in among those nine hundred thousand or so images, So
Speaker 1: then you submit your query. If you have an eighty
Speaker 1: five percent detection rate, then eighty five of those probe
Speaker 1: photos should come back with a match, and that match
Speaker 1: should be the actual person you're looking for. That's what
Speaker 1: they meant by an eighty five percent detection rate, that
Speaker 1: eighty five percent of the time an image that is
Speaker 1: in their database would be pulled due to a facial
Speaker 1: recognition software search. Now, during this testing phase, the FBI
Speaker 1: reported that they met this threshold. They used that subset
Speaker 1: of actually was nine hundred and twenty six thousand photos
Speaker 1: as their subset when they were testing it, and they
Speaker 1: said that they had an eighty six percent detection rate,
Speaker 1: So they actually were exceeding what they had set as
Speaker 1: their threshold. But that just meant that eighty six percent
Speaker 1: of the time, the actual match for a probe photos
Speaker 1: showed up in a group of fifty candidate images, so
Speaker 1: you would get forty nine other images that were not
Speaker 1: your match. The match would be there eighty six percent
Speaker 1: of the time along with forty nine other images. So
Speaker 1: we know that the system works if you were asking
Speaker 1: for the maximum number of candidates. Remember in the FBI system,
Speaker 1: you can ask for between two and fifty, but fifty
Speaker 1: is the max. But what happens if you ask for
Speaker 1: fewer images? What if you said, no, I want twenty returns.
Speaker 1: What's the accuracy, then the FBI can't tell you because
Speaker 1: they do not know. According to the FBI, they did
Speaker 1: not run tests to see what would happen if you
Speaker 1: decrease the number of candidate photos you asked for. They
Speaker 1: only ran tests on the maximum number of candidate photos.
Speaker 1: And keep in mind, the default for any search is
Speaker 1: twenty photos, so the default is less than what they tested,
Speaker 1: and they never tried to see if the eighty six
Speaker 1: percent detection rate held true at these lower numbers. That's
Speaker 1: a huge issue. On top of that, the FBI didn't
Speaker 1: go so far to determine how frequently its system would
Speaker 1: return false positives to probes, so they never paid attention
Speaker 1: to how many times they got responses that didn't reflect
Speaker 1: and the actual identity. They didn't keep track of it. So,
Speaker 1: according to the FBI, the purpose of the system is
Speaker 1: to generate leads, not to positively identify persons of interest.
Speaker 1: So it shouldn't come as a big surprise, or you
Speaker 1: shouldn't even care if it returns a lot of false positives,
Speaker 1: because hey, this technology isn't meant to be the smoking
Speaker 1: gun that says, here's the evidence that will put this
Speaker 1: person away. It's meant to just create a lead, So
Speaker 1: why do you care how many false positives it returns?
Speaker 1: As if being looped in on an official inquiry when
Speaker 1: you had nothing to do with it isn't disruptive or
Speaker 1: stressful or provoke anxiety. I don't know about you, guys,
Speaker 1: but if I had a federal agent show up at
Speaker 1: my door asking me weird questions about a case that
Speaker 1: I had no connection to because my image had popped
Speaker 1: up in one of these searches and I have nothing
Speaker 1: to do with it, it just so happens that I
Speaker 1: look enough like a photo that's being used in the
Speaker 1: case to warrant this. I would probably find that pretty
Speaker 1: disruptive in my life, so I would care about false positives. FBI,
Speaker 1: at least according to this GAO report, apparently didn't think
Speaker 1: it was that big a deal. Now, the GAO points
Speaker 1: out that it is a big deal, and that they're
Speaker 1: not the only ones to think so. The National Science
Speaker 1: and Technology Council and the National Institute of Standards and
Speaker 1: Technology both state then, in order to know how accurate
Speaker 1: a system is, you need to know two pieces of information,
Speaker 1: not just the detection rate, which the FBI claims is
Speaker 1: eighty six percent at least when you're asking for fifty candidates,
Speaker 1: but also the false positive rate. You have to know
Speaker 1: both of them in order to understand how accurate a
Speaker 1: system is, So only knowing one of those pieces of
Speaker 1: information isn't enough to state this system is accurate or not.
Speaker 1: You have to know both. So, not only does the
Speaker 1: FBI not have a grasp on how accurate their system
Speaker 1: is if you're asking for fewer than the maximum number
Speaker 1: of candidates, they also don't know how often it returns
Speaker 1: false positives. So the FBI has no way of knowing
Speaker 1: how accurate this facial recognition software is that's being used
Speaker 1: to actually further investigations for official investigations of the FBI
Speaker 1: and also other state agencies that have access to the system,
Speaker 1: That is beyond problematic. If you cannot say that the
Speaker 1: system with any degree of certainty is above a certain
Speaker 1: threshold of accuracy, why are you using it? Because? I mean,
Speaker 1: it has the potential to dramatically impact people's lives and
Speaker 1: potentially lead people down a pathway that could result in
Speaker 1: false accusations and imprisonment. The person who is actually responsible
Speaker 1: might totally get away with something because of this. This
Speaker 1: is a real problem. And the thing is it might
Speaker 1: be a perfectly accurate system, but we don't know that
Speaker 1: because we haven't tested it. So until we test it,
Speaker 1: we cannot just assume that it's accurate enough. That's not
Speaker 1: when people's lives are at staate. This is where that
Speaker 1: my bias doesn't so much creep in as it kicks
Speaker 1: open the door and makes itself at home on your couch.
Speaker 1: But I digress. The GAO report also goes into great
Speaker 1: detail about how this accuracy really can have a clear
Speaker 1: impact on people's privacy, their civil liberties, their civil rights.
Speaker 1: They also cite the Electronic Frontier Foundation the EFF which
Speaker 1: says that if a person is brought up as a
Speaker 1: defendant in a case and it is revealed that they
Speaker 1: were matched by a facial recognition system, it puts a
Speaker 1: burden on the defendant to argue that they are not
Speaker 1: the same person as was seen in a probe photo,
Speaker 1: that they are not the same one that the system
Speaker 1: has identified. And if you cannot reliably state how accurate
Speaker 1: your system is because you don't know how frequently it
Speaker 1: returns false positives, you have unfairly burned And the defendant,
Speaker 1: Like if you were to say, if you're the FBI,
Speaker 1: and you say, we have an eighty six percent detection rate,
Speaker 1: but you don't admit, oh, by the way, we don't
Speaker 1: know how many false positives we get on any given search.
Speaker 1: The implication you have given is that we're pretty sure
Speaker 1: that this is the right guy. And again they argue
Speaker 1: that this is meant to be a point of inquiry,
Speaker 1: but you could easily see how it could also be
Speaker 1: used by a lawyer to argue that a defendant is
Speaker 1: in fact the person responsible for a crime, and they
Speaker 1: may not be. And because you don't know the accuracy
Speaker 1: of the system, you can't using the system to argue
Speaker 1: for it is irresponsible. There's no accountability there. Now. Not
Speaker 1: only has the FBI failed to establish the accuracy of
Speaker 1: its own NGIIPS system, it has also not assessed the
Speaker 1: accuracy of all those external databases that are used whenever
Speaker 1: they use the face approach. There are no accuracy requirements
Speaker 1: for these agencies, so there's not like a threshold they
Speaker 1: have to prove that they meet in order to be
Speaker 1: part of this. That's a huge problem. While each agency
Speaker 1: might be accurate with no testing procedure, in place, it's
Speaker 1: impossible to be certain of that. And since these databases
Speaker 1: include millions of people with no criminal background and they
Speaker 1: all use different facial recognition software products, this is a
Speaker 1: huge issue. You could be put in a virtual lineup
Speaker 1: simply because you look enough like someone else that a
Speaker 1: computer thinks you are in fact the same person. The
Speaker 1: GAO report concludes with a host of recommendations for future actions,
Speaker 1: including addressing the problem of the FBI being so slow
Speaker 1: to publish those updated pias in a timely manner, and
Speaker 1: create a means to assess each system's accuracy. The Department
Speaker 1: of Justice read the report and then responded disagreeing with
Speaker 1: several points that the GOAO report made, including arguing that
Speaker 1: the FBI and the Department of Justice published information when
Speaker 1: it made the most sense, when the system had been
Speaker 1: tweaked and finalized. More or less. However, by that time, again,
Speaker 1: they had been using that system for real world cases
Speaker 1: throughout the entire process, So it seems to me to
Speaker 1: be kind of a weak argument. You can't really say, like, hey,
Speaker 1: it wasn't finished until then, that's when we published it.
Speaker 1: If you also are saying, hey, we use that for
Speaker 1: real zees to go after actual people. You can't have
Speaker 1: it both ways and not maintain accountability at any rate.
Speaker 1: So that kind of gets to the end of the
Speaker 1: Government Accountability Office report, but that's not the end of
Speaker 1: the story. In March twenty seventeen, Congress held some hearings
Speaker 1: about this, and boy howdy, were some congress people very
Speaker 1: upset with the FBI. On both sides of the aisle.
Speaker 1: You had Democrats and Republicans really chastising the FBI for
Speaker 1: their use of facial recognition software and arguing that it
Speaker 1: could amount to an enormous invasion of privacy as well
Speaker 1: as endangering the civil liberties of US citizens. So people
Speaker 1: who have dramatically different political philosophies were agreeing on this point.
Speaker 1: So it wasn't really a partisan issue in this case,
Speaker 1: and it got pretty ugly, but probably not as ugly
Speaker 1: as the Georgetown University report that was published in late
Speaker 1: twenty sixteen. This is an amazing report. Both the Government
Speaker 1: Accountability Office report and the Georgetown University report are available
Speaker 1: for free online. I will warn you collectively, they're about
Speaker 1: two hundred pages, so if you want some light reading
Speaker 1: you can check it out. They are quite good, both
Speaker 1: of them. And they're very accessible. Neither of them are
Speaker 1: written in crazy legallees which will make it impossible to understand.
Speaker 1: They're written in very plain English, as in the Georgetown
Speaker 1: University report that was revealed that one in every two
Speaker 1: American adults has their picture contained in a database connected
Speaker 1: to law enforcement facial recognition systems. And this report goes
Speaker 1: far beyond just that FBI to state all the way
Speaker 1: down to state and local systems that are implementing their
Speaker 1: own facial recognition databases, and many of them have no
Speaker 1: understanding of how it might impact the civil liberties or
Speaker 1: privacy of citizens. The report is the summary of a
Speaker 1: study that lasted a full year with more than one
Speaker 1: hundred records requests to various police departments. They looked at
Speaker 1: fifty two different law enforcement agencies across the United States,
Speaker 1: and the report assessed the risks to civil liberties and
Speaker 1: civil rights because up until this report was filed, no
Speaker 1: such study had been made, which is a huge problem.
Speaker 1: You don't know the impact of the tool that you've
Speaker 1: created until after it's been put in use for a while.
Speaker 1: That's an issue. Ideally, you think all this out before
Speaker 1: you implement the procedure and their findings were pretty upsetting.
Speaker 1: For example, the report found that some agencies limit themselves
Speaker 1: to using facial recognition within the framework of a targeted
Speaker 1: and public use, such as using it on someone who
Speaker 1: has been legally arrested or detained for a crime. And
Speaker 1: in this case, you're talking about totally above board approach.
Speaker 1: You're assuming that everyone is following the law as regards
Speaker 1: to apprehending and charging a suspect with a crime, and
Speaker 1: maybe that person is unwilling or unable to tell you
Speaker 1: what their identity is, and in that case, you would
Speaker 1: use this facial recognition software stuff in order to figure
Speaker 1: out who you are dealing with. That's largely a legitimate
Speaker 1: case the government. The Georgetown University study didn't say that's bad.
Speaker 1: They actually said, no, that makes sense. It's targeted, it's public.
Speaker 1: But you could have a more invisible approach, for example,
Speaker 1: using facial recognition software in real time on a closed
Speaker 1: circuit camera pointed at a city street, where you're literally
Speaker 1: picking up people as they pass by. They're not people
Speaker 1: of interest, they're just people going about their day. And
Speaker 1: if you're running facial recognition software on such a feed,
Speaker 1: you are potentially invading privacy and stepping on civil rights
Speaker 1: and civil liberties.
Speaker 2: Hey, it's modern day, Jonathan here just cutting end to
Speaker 2: say we will have more about the National Facial Recognition
Speaker 2: Database after this break.
Speaker 1: So even if you were to argue that this real
Speaker 1: time use where you're just looking at people as they
Speaker 1: pass by, and maybe a little name pops up every
Speaker 1: now and then as it as the system recognizes a
Speaker 1: person that matches a file in the database, it's easy
Speaker 1: to a scenario in which such a technology could be abused.
Speaker 1: Either it picks up somebody mistakenly, it thinks it identifies someone,
Speaker 1: but in fact it's a totally different person, and then
Speaker 1: you end up establishing a person's location by mistake, like
Speaker 1: it's not really where they were, but because the system
Speaker 1: has identified a person as being at X place at
Speaker 1: why time, you then have established supposedly that person's location,
Speaker 1: when in fact that person might be across town or
Speaker 1: not even in the same state. But it's because of
Speaker 1: a misidentification in the system. That's one problem. But think
Speaker 1: about this. Think of this is a scary scenario. Imagine
Speaker 1: a situation in which a group of people are discriminated
Speaker 1: against by a government agency. Let's say they have a
Speaker 1: legitimate gripe. It's completely legitimate. They're victims of unfair treatment.
Speaker 1: So a group of them at some of their allies
Speaker 1: get together in a public place for peaceful protest, to
Speaker 1: raise awareness of this issue and to confront the government
Speaker 1: agencies that have discriminated against them. This is all perfectly
Speaker 1: legal according to the US Constitution. They're not doing anything legal.
Speaker 1: They're assembling on public grounds in order to practice free speech.
Speaker 1: But it's not hard to imagine a government agency using
Speaker 1: a camera with this sort of facial recognition software to
Speaker 1: identify people who are in the crowd in order to
Speaker 1: use that as leverage in the future for some purpose
Speaker 1: or another, even if it's just to say we know
Speaker 1: you were there, and to put that kind of pressure
Speaker 1: on a person in order to essentially squelch people's freedom
Speaker 1: of speech. So this is a First Amendment issue, not
Speaker 1: just a Fourth Amendment issue. Now that might sound like
Speaker 1: a dramatic scenario like something like Big brother Ish. It's orwellian,
Speaker 1: but it's also entirely within the realm of possibility. From
Speaker 1: a technological standpoint, there's nothing technologically oriented that would prevent
Speaker 1: us from doing this or prevent an agency from doing this,
Speaker 1: and even without the evil empire scenario in place, you
Speaker 1: still have the problematic issue of treading on civil liberties
Speaker 1: just by having such technology available and unregulated. You don't
Speaker 1: have rules to guide this sort of stuff. The Georgetown
Speaker 1: report found that only one agency out of the fifty
Speaker 1: two that they looked at have a specific rule against
Speaker 1: using facial recognition software to identify people participating in public
Speaker 1: demonstrations or free speech in general. So only one agency
Speaker 1: actually has rules against that. Now, that doesn't mean the
Speaker 1: other fifty one agencies are regularly using this technology to
Speaker 1: monitor acts of free speech, but it also doesn't mean
Speaker 1: that they can't. They don't have rules against it. Only
Speaker 1: one a agency out of the fifty two, people are
Speaker 1: being watched and identified without any connection to a crime.
Speaker 1: In these cases, it's pretty terrifying. The Georgetown report also
Speaker 1: found that no state had yet passed a law to
Speaker 1: regulate police use of facial recognition software. No state in
Speaker 1: the US. They're fifty of them, and none of them
Speaker 1: have passed any regulations, any laws to regulate the use
Speaker 1: of facial recognition software. So without rules, how do you
Speaker 1: argue whether someone's misused or abused a system, you have
Speaker 1: to have rules so that you know what is allowed
Speaker 1: and what is not allowed. With no rules, the implication
Speaker 1: is that everything's allowed until it isn't. That's a huge
Speaker 1: dangerous problem. The report also pointed out that most of
Speaker 1: these agencies lacked any sort of methodology to ensure that
Speaker 1: the accuracy of their respective systems was decent. The report
Speaker 1: stated that of all the agencies they investigated, only two,
Speaker 1: the San Francisco Police Department and the South Sound nine
Speaker 1: to one one from Seattle, had made decisions about what
Speaker 1: facial recognition software they were going to incorporate in their
Speaker 1: office based off of accuracy rates. That was not a
Speaker 1: consideration for all of the other agencies, at least not
Speaker 1: the ones that they asked. Moreover, they report points out
Speaker 1: that facial recognition companies are also trying to have it
Speaker 1: both ways. So, for example, they cite a company called
Speaker 1: fat Face First. Now face First advertises that has a
Speaker 1: ninety five percent accuracy rate, but it simultaneously disclaims any
Speaker 1: liability for failing to meet that ninety five percent accuracy rate.
Speaker 1: So it's kind of like saying we guarantee these tires.
Speaker 1: Tires are not guaranteed not quite like that, but similar.
Speaker 1: So again, this is according to the Georgetown University report,
Speaker 1: that's a problem for a company to sell itself on
Speaker 1: a performance threshold, but then say, hey, you can't hold
Speaker 1: us to that performance threshold that we sold you on.
Speaker 1: That's a little dangerous there too. Then the report goes
Speaker 1: on to state that the human analysts, you know, the
Speaker 1: ones I was talking about earlier, that supposed to be
Speaker 1: a safeguard. Human analysts are supposed to take the images
Speaker 1: that are returned by these automated systems and manually review
Speaker 1: them to make sure that they do or do not
Speaker 1: match that probe photo. That was the whole thing to
Speaker 1: begin with. But it turns out, according to this report,
Speaker 1: those human analysts are not that accurate. In fact, they're
Speaker 1: no better than a coin flip. Literally. The report sites
Speaker 1: of study that showed that if analysts did not have
Speaker 1: highly specialized training, they would make the wrong decision for
Speaker 1: a potential match fifty percent of the time. Literally a
Speaker 1: coin flip. That's ridiculous. Now, the report found only eight
Speaker 1: agencies out of the fifty two used specialized personnel to
Speaker 1: review images. In other words, people who presumably have actually
Speaker 1: received that highly specialized training necessary to make more accurate
Speaker 1: decisions regarding these photos, and the report states that there's
Speaker 1: no formal training regime in place for examiners, which is
Speaker 1: a major problem for a system that's already in widespread use.
Speaker 1: So not only do you need highly specialized training, there's
Speaker 1: no formalized approach to give or receive that highly specialized training.
Speaker 1: So we know you need it, but we haven't developed
Speaker 1: the best practices to actually deliver upon that. So meanwhile,
Speaker 1: you've got human analysts who are making mistakes half the
Speaker 1: time while reviewing these photo And if you wonder if
Speaker 1: facial recognition systems would disproportionately affect some ethnicities over others,
Speaker 1: the answer to that is resounding and dismaying yes. The
Speaker 1: report found that African Americans would be affected more than
Speaker 1: other ethnicities. According to an FBI co authored study that
Speaker 1: was cited by this Georgetown University report, several facial recognition
Speaker 1: algorithms are less accurate for Black people than for other ethnicities,
Speaker 1: and there's no independent testing process to determine if there's
Speaker 1: a racial bias in any of these facial recognition systems,
Speaker 1: so no one has developed a test to make certain
Speaker 1: that it is in fact accurate despite a person's age, gender,
Speaker 1: or race, without being able to verify that it is
Speaker 1: accurate across all parameters, you have opened up an enormous
Speaker 1: can of worms, and you are disproportionately affecting people just
Speaker 1: because of the race, because your system does not address
Speaker 1: that properly. The report also points out that the information
Speaker 1: about the systems in use had not been generally available
Speaker 1: to the public. In fact, all of the fifty two
Speaker 1: agencies they contacted, only four had publicly available use policies. So,
Speaker 1: in other words, only four of the fifty two could
Speaker 1: tell you what their general policy was as far as
Speaker 1: facial recognition software goes. That's less than ten percent of
Speaker 1: all of the agencies they looked at, and only one
Speaker 1: of those agencies, which was San Diego's Association of Governments,
Speaker 1: had legislative approval for its policy. All the others were
Speaker 1: just self appointed policies that had not passed through any
Speaker 1: kind of official legislative support. Finally, the report asserted that
Speaker 1: most of these systems did not have an official audit
Speaker 1: process to determine if or when someone misuses the systems.
Speaker 1: Nine agencies were or that they did have a process,
Speaker 1: but only one provided Georgetown with any evidence that they
Speaker 1: had a working audit system, and that was the Michigan
Speaker 1: State Police, by the way, who said, we have an
Speaker 1: audit system, and here's proof that it actually works the
Speaker 1: way we said it did. So good on you, Michigan
Speaker 1: State for our having that system in place and being
Speaker 1: able to back it up now. The Georgetown University report
Speaker 1: also urged some major changes in the way law enforcement
Speaker 1: uses facial recognition, including an appeal to Congress to create
Speaker 1: clear regulations to define the parameters of when such a
Speaker 1: system could be used. They also called for companies to
Speaker 1: publish processes that test their products accuracy regardless of race, gender,
Speaker 1: and age, to remove that possibility of bias. And if
Speaker 1: we're being really super kind and generous toward law enforcement,
Speaker 1: we could say this is just another case where technology
Speaker 1: has clearly outpaced the law. We see that all the time,
Speaker 1: driverless artificial intelligence, lots of different technologies are advancing far
Speaker 1: faster than legislation can keep up with. All right, that's fair,
Speaker 1: we see it happen. However, it's particularly troublesome that this
Speaker 1: is happening within law enforcement that is already employing this
Speaker 1: technology before we've developed the policies to guide it. It's
Speaker 1: one thing to say someone's out here working on a
Speaker 1: driverless car, and we need to start thinking about how
Speaker 1: are we going to regulate that in the future. Maybe
Speaker 1: right now we say you aren't allowed to operate your
Speaker 1: driverless car until we figured this out. That's fair. It's
Speaker 1: another thing to say, there's this technology that could potentially
Speaker 1: impact people's lives and we're allowing law enforcement to use
Speaker 1: it while we try and figure out the rules. That's
Speaker 1: at best a problem. And as I said at the
Speaker 1: top of the show, I'm really just talking about the
Speaker 1: United States with particulars here, but this is happening all
Speaker 1: around the world. There are lots of governments around the
Speaker 1: world that are incorporating facial recognition software along with law enforcement.
Speaker 1: So while I'm using specific US examples in this podcast,
Speaker 1: the same is true for lots of other places. Of course,
Speaker 1: the laws that protect the citizens can be different from
Speaker 1: country to country, and in some cases there might not
Speaker 1: be very many outlets for citizens to voice their concern
Speaker 1: or it might even be dangerous to do so. But
Speaker 1: this is something I think we need to be aware of.
Speaker 1: I'm not generally the kind of person who tells you
Speaker 1: that you're being watched or you know, you should be paranoid.
Speaker 1: But I'm also not the person to just sit back
Speaker 1: and let something go on when I feel like it's
Speaker 1: potentially more of a problem than a solution.
Speaker 2: Well that was it for the episode I did on
Speaker 2: the National Facial Recognition Database back in twenty seventeen. It's
Speaker 2: a topic I should definitely revisit. Obviously, there's so much
Speaker 2: going on here. There are so many concerning things about it,
Speaker 2: from surveillance states, to privacy and security concerns, to the
Speaker 2: fact that we've been seeing lots of companies try and
Speaker 2: use facial recognition to match people against databases to varying
Speaker 2: degrees of success, and that for people of color in particular,
Speaker 2: those degrees of success are not good. And I think
Speaker 2: there's a lot that we need to talk about as
Speaker 2: far as this goes, when it comes to things like,
Speaker 2: you know, individual rights and authoritarian abuse of these kind
Speaker 2: of technologies, and I think we do need to have
Speaker 2: another update on this, so I will put that on
Speaker 2: my list. I hope that you are all well, and
Speaker 2: I'll talk to you again really soon. Tech stuff is
Speaker 2: an iHeartRadio production for more podcasts from iheartradiosit the iHeartRadio app,
Speaker 2: Apple Podcasts, or wherever you listen to your favorite shows.