Speaker 1: Get in text with technology with tech Stuff from how
Speaker 1: stuff works dot com. Hey there, and welcome to text Stuff.
Speaker 1: I'm your host Jonathan Strickland, senior writer with how stuff
Speaker 1: works dot com, and today we're going to examine a
Speaker 1: controversial topic, namely how law enforcement is using facial recognition
Speaker 1: software and the problems that this raises. Now, before I
Speaker 1: dive into the topic, I want to make a couple
Speaker 1: of things very clear at the very beginning. First, is
Speaker 1: I'm biased. I think the use of facial recognition software
Speaker 1: is problematic even if you have regulations in place. But
Speaker 1: I'm mostly talking about unregulated use because really we haven't
Speaker 1: established the rules and policies to guide the use of
Speaker 1: facial recognition software in a law enforcement context. So that's
Speaker 1: problem No. One is. I have a very strong opinion
Speaker 1: about this, and I'm not gonna shy away from that. Um,
Speaker 1: it's really unjustifiable to have unregulated use of facial recognition
Speaker 1: software in law enforcement contexts. So I want to make
Speaker 1: that clear out of the gate, that I have this bias,
Speaker 1: and if that's an issue, that's fair. But at least
Speaker 1: I'm being honest, right, I'm not presenting this as if
Speaker 1: it's completely objective, unbiased information. I own this. You don't
Speaker 1: have to tell me. I know it already. Next, this
Speaker 1: is largely going to be a US centric discussion so
Speaker 1: that I can talk about details. But please know that
Speaker 1: there are a lot of these types of systems all
Speaker 1: over the world, not just in the United States, and
Speaker 1: a lot of these places have similar issues to the
Speaker 1: ones I'm gonna be talking about here in the US.
Speaker 1: I'll just be focusing more on US stories to make
Speaker 1: specific points because this is where I live, and now
Speaker 1: to explain what I'm actually talking about here. So back
Speaker 1: in the FBI undertook a project that cost more than
Speaker 1: an estimated one point two billion dollars that's billion with
Speaker 1: a B to replace what was called the Integrated Automated
Speaker 1: Fingerprint System or I A f S. Have I only
Speaker 1: been in place since nine and I've talked about fingerprints
Speaker 1: in a previous episode. UM. The i F I a
Speaker 1: f S was an attempt to create a A U
Speaker 1: S Y database of fingerprint records so that if you
Speaker 1: were investigating a crime and you had lifted some prints
Speaker 1: from the crime, you could end up UH consulting this
Speaker 1: database and see if there are any matches in place
Speaker 1: to give you any leads on your investigation. The project
Speaker 1: the FBI undertook was meant to vastly expand that capability
Speaker 1: by adding a lot more data to the database, not
Speaker 1: just fingerprints, but other stuff as well, and the news
Speaker 1: system is called the Next Generation Identification or in g I.
Speaker 1: It includes not just fingerprints, but other biographical data and
Speaker 1: biometrics information, including face recognition technology. So a lot of
Speaker 1: images are included in this particular database. So as part
Speaker 1: of this project, the FBI incorporated the Interstate Photo System
Speaker 1: or i p S, so you have n g I
Speaker 1: dash ips that typically is how it's referred to now.
Speaker 1: That system includes images from police cases as well as
Speaker 1: photos from civil civic sources that are not necessarily related
Speaker 1: to crimes. Not that's not the only way the FBI
Speaker 1: can scan for a match of a photograph they've taken
Speaker 1: that relates to a case in some way to this
Speaker 1: massive database, But more on that in a little bit now.
Speaker 1: The general process of searching for a match follows a
Speaker 1: pretty simple pattern, although the details can be vastly different
Speaker 1: depending upon what facial recognition software you are using at
Speaker 1: the time. So you first start with an image related
Speaker 1: to a case, and this is called the probe photo.
Speaker 1: It is the one you are probing for lack of
Speaker 1: a better term, You don't know the identity of the
Speaker 1: person in the photograph, typically, or at least you might
Speaker 1: have suspicions, but you don't necessarily know for sure. So
Speaker 1: you've got a picture of an unknown person in this photograph.
Speaker 1: You then scan that photo and you use facial recognition
Speaker 1: software to analyze the picture and to try and find
Speaker 1: a match in this larger database. It starts searching all
Speaker 1: of the images in this database looking for any that
Speaker 1: might be a potential match. Depending upon the system and
Speaker 1: the policies that are in use, you could end up
Speaker 1: with a single photo do returned to you. You could
Speaker 1: end up with dozens of photos. So these would all
Speaker 1: be potential matches with different degrees of certainty for a match.
Speaker 1: You might remember in episodes I've talked about things like
Speaker 1: IBM S Watson that would come up with answers to
Speaker 1: a question and assign a value to each potential answer,
Speaker 1: and the one that had the highest value, assuming it's
Speaker 1: above a certain threshold, would be submitted as the answer.
Speaker 1: So it's not so much that the computer quote unquote
Speaker 1: knows it has a match. It suspects a match based
Speaker 1: upon a certain percentage as long as it's over a
Speaker 1: threshold of certainty, or you might end up with no
Speaker 1: photos at all. If no match was found or nothing
Speaker 1: ended up being above that threshold, the system might say,
Speaker 1: I couldn't match this photo with anyone who's in the database.
Speaker 1: A study performed by researchers at Georgetown University found that
Speaker 1: one in every two American adults has their face captured
Speaker 1: in an image database that is accessible by various law
Speaker 1: enforcement agencies, including but not limited to, the i p S.
Speaker 1: In fact, the i p S has a small number
Speaker 1: of photos compared to the overall number represented by databases
Speaker 1: across the US. Now, this involves agencies at all different levels, federal, state,
Speaker 1: even tribal law for Native American uh tribes. That ends
Speaker 1: up being about a hundred seventeen million people in these databases,
Speaker 1: many of whom, in fact large personage of whom have
Speaker 1: no criminal background whatsoever. Their images are also in these databases,
Speaker 1: and this raises some big concerns about privacy and also accountability.
Speaker 1: So in today's episode, we're going to explore how facial
Speaker 1: recognition software works. As well as talk about the implement
Speaker 1: implementation for law and forcement and the reaction to this technology,
Speaker 1: and will probably listen to me get upset and a
Speaker 1: little head up about the whole thing in general. All Right,
Speaker 1: So first, before we leap into the mess of law enforcement,
Speaker 1: because it is a mess, that's just a fact, let's
Speaker 1: talk first about the technology itself. When did facial recognition
Speaker 1: software get started and how does it work? Well, it's
Speaker 1: related to computer vision, which is a subset of artificial
Speaker 1: intelligence research. If you look at artificial intelligence, a lot
Speaker 1: of people simplify that by meaning, oh, this is so
Speaker 1: that you can teach computers how to think like people.
Speaker 1: But that's actually a very specific definition of a very
Speaker 1: specific type of artificial intelligence. When you really look at
Speaker 1: a I and you break it out, and involves a
Speaker 1: lot of subsets of abilities, and one of those is
Speaker 1: the ability for machines to analyze imagery and be able
Speaker 1: to determine what that imagery represent. In a way, you
Speaker 1: could argue it's teaching computers how to understand pictures. It's
Speaker 1: also really challenging, and this is one of the object
Speaker 1: lessons that I use to teach people how artificial intelligence
Speaker 1: is really tricky. It requires more than just pure processing power.
Speaker 1: I mean, processing power is important, but you can't solve
Speaker 1: all of AI's problems just by throwing more processors at it.
Speaker 1: You have to figure out from a software level how
Speaker 1: to leverage that processing power in a way that gives
Speaker 1: computers this ability to identify stuff based upon imagery. So
Speaker 1: a computer might be able to perform far more mathematical
Speaker 1: operations per second than even the cleverest of humans, but
Speaker 1: without the right software, they can't identify the picture of
Speaker 1: a seagull compared to say, a semitruck. You have to
Speaker 1: teach the computer how to do this. So let's say
Speaker 1: you develop a program that can analyze an image and
Speaker 1: breaking down into simple data to describe that image, and
Speaker 1: then you essentially teach a computer what a coffee mug
Speaker 1: looks like. You take a picture of a coffee mug,
Speaker 1: you feed it to a computer, and you essentially say
Speaker 1: this data represents a coffee mug. You then would have
Speaker 1: to try and train the computer on what that actually means.
Speaker 1: The computer does not now know what a coffee mug is.
Speaker 1: It will recognize that specific mug in that specific orientation
Speaker 1: under those specific lighting conditions, assuming that you've designed the
Speaker 1: algorithm properly. But it's way more tricky than that. What
Speaker 1: if in the image that you fed the computer, the
Speaker 1: coffee mugs handle was facing to the left with respect
Speaker 1: of the viewer, but in a future picture the handle
Speaker 1: is off to the right instead of to the left,
Speaker 1: or it's turned around so you can't see the hand
Speaker 1: to at all. It's behind the coffee mug. What if
Speaker 1: the mug is bigger or smaller, or a different shape,
Speaker 1: what if it's a different color. Image recognition is tough
Speaker 1: because computers don't immediately associate different objects within the same
Speaker 1: category as being the same thing. So if you teach me, Jonathan,
Speaker 1: what a coffee mug is, and you show me a
Speaker 1: couple of different examples saying, this is a coffee mug,
Speaker 1: but this is also a coffee mug, even though it's
Speaker 1: a different size, and different shape and a different color,
Speaker 1: I'll catch on pretty quickly and it won't take very
Speaker 1: many coffee mugs for me to figure out. All Right,
Speaker 1: I got the basic idea of what a coffee mug is.
Speaker 1: I know what the concept of coffee mug is now,
Speaker 1: but computers aren't like that. You have to feed them
Speaker 1: thousands of images, both of coffee mugs and of not
Speaker 1: coffee mugs, so that the computer starts to be able
Speaker 1: to pick out the various features that are the say
Speaker 1: it's of a coffee mug versus things that are not
Speaker 1: related to being a coffee mug. It takes hours and
Speaker 1: hours and hours of work of training these computers to
Speaker 1: do it, so it's a non trivial task, and this
Speaker 1: is true of all types of image recognition, including facial recognition. Now,
Speaker 1: to get around that problem, you end up sending thousands,
Speaker 1: countless thousands, millions maybe of images of what you're interested
Speaker 1: in while you're training the computer. And the nice thing
Speaker 1: is computers can process this information very very quickly, so
Speaker 1: while it takes a lot, it doesn't take relatively that long.
Speaker 1: It's not as laborious a process as it could be
Speaker 1: if computers were slower at analyzing information. So you might
Speaker 1: remember a story that kind of illustrates this point. Back
Speaker 1: in two thousand twelve, there was a network of sixteen
Speaker 1: thousand computers that analyzed ten million only in images, and
Speaker 1: as a result, it could do the most important task
Speaker 1: any computer connected to the Internet should be expected to do.
Speaker 1: It could then identify cat videos because it now knew
Speaker 1: what a cat was, or at least the features that
Speaker 1: defined cat nous catness as in the essence of being
Speaker 1: a cat, not a character from Hunger Games. Even then,
Speaker 1: there were times when the computer would get it wrong.
Speaker 1: Either it would not identify a cat as being a cat,
Speaker 1: or it misidentify something else as being a cat because
Speaker 1: its features were close enough to cat like for it
Speaker 1: to fool the computer algorithm. A major breakthrough and facial
Speaker 1: recognition algorithms happened way back in two thousand one. That's
Speaker 1: when Paul Viola and Michael Jones unveiled an algorithm for
Speaker 1: face detection, and it worked in real time, which meant
Speaker 1: that it could recognize a face that it would appear
Speaker 1: on a webcam. And but I recognize I mean it
Speaker 1: recognized that it was a face. It didn't assign an
Speaker 1: identity to the face. It didn't say, oh, that's Bob,
Speaker 1: It said, oh, that is a face that is in
Speaker 1: front of the webcam right now. Uh. The algorithm soon
Speaker 1: found its way into open CV, which is an open
Speaker 1: source computer vision framework, and the open source approach allowed
Speaker 1: other programmers to dive into that code and to make
Speaker 1: changes and improvements, and it helped a rapid prototyping of
Speaker 1: facial recognition software to other computer scientists who helped advance
Speaker 1: computer vision further. Where Bill Triggs and Novnique de Lal
Speaker 1: who published the paper in two thousand five about the
Speaker 1: histograms of oriented gradients. Now, that was an approach that
Speaker 1: looked at gradient orientation in parts of an image, and
Speaker 1: essentially it describes the process of viewing an image with
Speaker 1: attention to edge directions and intensity gradients. That's a complicated
Speaker 1: way of saying the technique looks at the totalitary t
Speaker 1: of a person, and then a machine learning algorithm determines
Speaker 1: whether or not that is actually a person or not
Speaker 1: a person, and a bent later, computer scientists began pairing
Speaker 1: computer vision algorithms with deep learning and convolutional neural networks
Speaker 1: or CNNs. To go into this would require an episode
Speaker 1: all by itself. Neural networks are fascinating, but they're also
Speaker 1: pretty complicated, and I've got a whole lot of topics
Speaker 1: to cover today, so we can't really dive into it.
Speaker 1: You can think of an artificial neural network as designing
Speaker 1: a computer system that processes information in a way that's
Speaker 1: similar to the way our brains do. The computers are
Speaker 1: not thinking, but they are able to process information in
Speaker 1: a way that mimics how we process information, or a
Speaker 1: close a semi close approximation thereof that's a really kind
Speaker 1: of weak way of describing it. But again, to really
Speaker 1: go into detail will require a full episode of by itself. Typically,
Speaker 1: facial recognition software uses feature extraction to look for patterns
Speaker 1: in an image relating to facial features. In other words,
Speaker 1: it's searches for for features that resemble a face, the
Speaker 1: elements you would expect to be present in a typical face,
Speaker 1: so eyes, knows, a mouth, that would be major ones. Right.
Speaker 1: Then the software starts to estimate the relationships between those
Speaker 1: different elements. How wide are the eyes, how far apart
Speaker 1: are they from each other, How white is the nose,
Speaker 1: how long is the jawline, what shape are the cheap bones?
Speaker 1: These sort of elements all play apart as points of data,
Speaker 1: and different facial recognition software packages weight these features in
Speaker 1: a different way. So it's it's not like I could
Speaker 1: say all facial recognition software looks at these four points
Speaker 1: of data as its primary source. It varies depending upon
Speaker 1: the algorithm that's been designed by various companies. UH And
Speaker 1: part of the problem that we're going to talk about
Speaker 1: is that law enforcement across the United States, they're not
Speaker 1: relying on a single facial recognition software approach. Different agencies
Speaker 1: have different vendors that they work with, So just because
Speaker 1: one might work very well doesn't necessarily mean it's competitors
Speaker 1: work just as well. And that's part of the problem. Now,
Speaker 1: all of these little points of data I'm talking about,
Speaker 1: these nodal points and how they relate to one another,
Speaker 1: all of that gets boiled down into a numeric code
Speaker 1: that you could think of as a face print. This
Speaker 1: is supposed to be a representation of the unique set
Speaker 1: of data that is a compilation of all of these
Speaker 1: different points boiled down into numeric information itself. Then what
Speaker 1: you would do is you would have a database of faces.
Speaker 1: So if you want to find a match, you would
Speaker 1: feed the image you have, the probe image into this database,
Speaker 1: and the facial recognition software would analyze the probe photo.
Speaker 1: It would end up assigning this numeric value and would
Speaker 1: start looking through the database for other numeric values that
Speaker 1: were as similar to that probe one as possible, and
Speaker 1: start returning those images as potential matches or candidates. They
Speaker 1: tend to use the word candidate photos. Otherwise you'll either
Speaker 1: get no match at all or you get a false positive.
Speaker 1: You will end up getting an image of someone who
Speaker 1: looks like the person whose image you submitted, but is
Speaker 1: not the same person. That does happen, and that's the
Speaker 1: basic way that facial recognition software works. But keep in mind,
Speaker 1: different vendors use all their own specific approaches, like I said,
Speaker 1: and some could be less accurate than others. Some might
Speaker 1: be accurate for specific ethnicities and not as accurate as
Speaker 1: other ones. That's a huge problem. So it gets complicated.
Speaker 1: Even when I'm talking in more general terms, you have
Speaker 1: to remember that there are a lot of specific UH
Speaker 1: incidents and specific implementations of facial recognition software that have
Speaker 1: their own issues. So I'm gonna be as general as
Speaker 1: I can. I'm not going to call out any particular
Speaker 1: facial recognition software vendors out there. I'm more going to
Speaker 1: talk about the overall issues that various organizations have had
Speaker 1: as they've looked into this topic. Now, there are plenty
Speaker 1: of applications for facial recognition that have nothing to do
Speaker 1: with identifying a person. I mentioned that earlier that there
Speaker 1: was the one for a webcam that could identify when
Speaker 1: a face was in front of the webcam. This wasn't
Speaker 1: to identify anybody. It was again just to say, yes,
Speaker 1: there's somebody looking into the webcam at this moment, which
Speaker 1: by itself can be useful and have nothing to do
Speaker 1: with identification. There are plenty of digital cameras out there
Speaker 1: and and camera phone apps that can identify when there's
Speaker 1: a face looking at the camera. Again, it's not necessarily
Speaker 1: to identify that person, but rather to say, oh, well,
Speaker 1: this is a face. The camera is most likely trying
Speaker 1: to focus on this person, so let's make this person
Speaker 1: the point of focus and not focus on something in
Speaker 1: the background like a tree that's fifty yards back. Instead,
Speaker 1: let's focus on the person who's in the foreground. So
Speaker 1: that's pretty handy, and again there's nothing particularly problematic from
Speaker 1: an identification standpoint, because that's not the purpose of it.
Speaker 1: But then you also have other implementations, like on social media,
Speaker 1: which allow you to do things like tag people based
Speaker 1: upon a an algorithm recognizing a person. So Facebook is
Speaker 1: a great example of this. Right, if you upload a
Speaker 1: picture of one of your Facebook friends onto Facebook. Chances
Speaker 1: are it's giving you a suggestion to tag that photo
Speaker 1: with the specific person in mind. That may not be
Speaker 1: that problematic either, depending upon how your friend feels about
Speaker 1: pictures being uploaded to Facebook. Some people are very um
Speaker 1: cautious about that, And of course you know, I always
Speaker 1: recommend you talk to anybody before you start tagging folks
Speaker 1: on Facebook photos, just to make sure they're fine with it. Um.
Speaker 1: I say that as a person who has done it,
Speaker 1: and then notice that some of my tags got removed
Speaker 1: by the people I tagged later on, which taught me
Speaker 1: I should probably ask first, rather than give them the
Speaker 1: feeling that they need to go and remove a tag
Speaker 1: or two. We've also seen examples of this simple implementation
Speaker 1: of facial recognition going awry. Google's street View will blur
Speaker 1: out faces, for example, in an effort to protect people's
Speaker 1: identity while street view cars are out and about taking images.
Speaker 1: This makes sense. Let's let's say that you are in
Speaker 1: a part of town that you normally would not be in.
Speaker 1: For whatever reason, you might not want your picture to
Speaker 1: be included on Google street View, so that whenever anyone
Speaker 1: looks at that street for that point forward. They see
Speaker 1: your face on there, you know, coming out of uh,
Speaker 1: I don't know, a Windy's. Maybe you are a manager
Speaker 1: for Burger King that would look bad, or you know,
Speaker 1: lots of other reasons that obviously can spring to mind
Speaker 1: as well. You don't want to violate someone's privacy. But
Speaker 1: Google StreetView would also blur out images that were not
Speaker 1: real people faces, like images on billboards or murals. Sometimes
Speaker 1: if it had a person's face on a mural, the
Speaker 1: face would be blurred out, even though it's not a
Speaker 1: real person, it's just a painting. Or In September, Set
Speaker 1: reported on an incident in which Google street View blurred
Speaker 1: out the face of a cow. So Google was being
Speaker 1: very thoughtful to protect that cow's privacy. But what about
Speaker 1: matching faces to identities? So in some cases, again seemingly
Speaker 1: harmless if you want to tag your friends, but when
Speaker 1: it comes to law enforcement, things get a bit sticky,
Speaker 1: particularly as you learn more about the specifics. And we'll
Speaker 1: talk about that in just a second, but first let's
Speaker 1: take a quick break to thank our sponsor. All right,
Speaker 1: let's first start with the FBI's Interstate Photos System or
Speaker 1: i p S, because this one has perhaps the least
Speaker 1: controversial elements to it. When you really look at it,
Speaker 1: it's still problematic, but not nearly as much as the
Speaker 1: larger picture. The system contains both images from criminal cases
Speaker 1: like mug shots UH and things of that nature, but
Speaker 1: it also includes some photos from civil sources like UH,
Speaker 1: like I D applications, that kind of thing. When the
Speaker 1: Government Accountability Office or g a O, there's gonna be
Speaker 1: a lot of acronyms and initializations are initialisms, I should
Speaker 1: say in this episodes, so I apologize for that. But
Speaker 1: Government Accountability Office they did a study on this matter
Speaker 1: just in sten So not that long ago. They published
Speaker 1: its report on facial recognition software use among law enforcements,
Speaker 1: specifically the FBI because they're a federal agency, so they
Speaker 1: were concerned with the federal use of this. The database
Speaker 1: contained about thirty million photos at the time of the
Speaker 1: g a O study, so thirty million pictures are in
Speaker 1: this database. Most of those images came from eighteen thousand
Speaker 1: different law enforcement agencies at all levels of government, that
Speaker 1: includes the tribal law enforcement offices. About sevent of all
Speaker 1: the photos and the database were mug shots. UH. More
Speaker 1: than eighty percent of the photos in that database are
Speaker 1: from criminal cases, so that means that less than twenty
Speaker 1: were from civil sources. In addition to that, there were
Speaker 1: some cases, plenty of them, where the database had images
Speaker 1: of people both from a civil source and from a
Speaker 1: criminal source. So I'll give you a theoretical example. Let's
Speaker 1: say that sometime in the past, UH, I got nabbed
Speaker 1: by the cops for for grand theft auto because I
Speaker 1: play that game. But let's say that I stole a car,
Speaker 1: which we already know is a complete fabrication because I
Speaker 1: don't even drive. But let's say I stole a car
Speaker 1: and that I had moved the car across state lines.
Speaker 1: It became a federal case. Therefore, my criminal information is included.
Speaker 1: My mug shot would be included in this particular database. UH.
Speaker 1: On a related note, my I d also is in
Speaker 1: that database as a civil UH image, not as a
Speaker 1: criminal image. Well, in my case, they would tie those
Speaker 1: two images together because they refer to the same person
Speaker 1: and I had been involved in a criminal act. So
Speaker 1: while I would have an image in there from a
Speaker 1: civil source, it would be filed under the criminal side
Speaker 1: of things. This is important when we get to how
Speaker 1: the probes work. Now let's say you have been perfectly
Speaker 1: law abiding this whole time, and that your I D
Speaker 1: is also in this database, but it's just under the
Speaker 1: civil side of things. Since you don't have any criminal background,
Speaker 1: it's not connected to anything on the criminal side. So
Speaker 1: when it comes to probes using the i p S,
Speaker 1: your information will not be referenced. Because the FBI policy
Speaker 1: is when it's running these these potential matches with a
Speaker 1: photo that's been gathered as part of the evidence for
Speaker 1: an ongoing investigation, they can only consult the criminal side,
Speaker 1: not the civil side, with the exception of any civil
Speaker 1: photos that are connected to a criminal case, as in
Speaker 1: my example, those are fair game. So it might run
Speaker 1: a match and it turns out that my photo for
Speaker 1: my state given identification card is a better match than
Speaker 1: the mugshot is. That's gonna be fine, because those two
Speaker 1: things were both attached to a criminal file in the
Speaker 1: first place. But let's say that it would have matched
Speaker 1: up against you since you didn't have a criminal background,
Speaker 1: and since the only record in there was a civil source,
Speaker 1: the match would completely skip over you. It wouldn't return
Speaker 1: your picture because your image is off limits in that
Speaker 1: particular use. Very important because it's an effort to try
Speaker 1: and make sure this facial recognition technology is focusing just
Speaker 1: on the criminal side, not putting law abiding citizens in
Speaker 1: danger of being pulled up in a virtual lineup, at
Speaker 1: least not using that approach. That's the problem is that
Speaker 1: that's not the only way the FBI runs searches. In fact,
Speaker 1: that might not be the primary way the FBI runs
Speaker 1: searches when they're looking for a match to a photo
Speaker 1: that was taken as part of evidence gathering in pursuing
Speaker 1: a case. But let's say that you are an FBI
Speaker 1: agent and you've got a photo, a probe photo, and
Speaker 1: you want to run it for a match. What's the procedure.
Speaker 1: You would send off your request to the n g
Speaker 1: I dash Ips Department, and you would have to indicate
Speaker 1: how many potential photographs you want back, how many candidates
Speaker 1: do you want. You can choose between two candidate photos
Speaker 1: and fifty candidate photos. These are photos of different individuals,
Speaker 1: by the way, not just here's here's a picture of
Speaker 1: Jonathan on the beach. Here's a picture of Jonathan in
Speaker 1: the woods. No, it's more like, here's a picture of Jonathan.
Speaker 1: Here's a picture of a person who's not Jonathan but
Speaker 1: also kind of matches this particular probe photo you advent
Speaker 1: you you submitted, and here are forty eight others. The
Speaker 1: default is twenty, so if you don't change the default
Speaker 1: at all, you will get back twenty images. Uh that
Speaker 1: our potential candidates matching your probe photo, assuming that any
Speaker 1: are found at all. It is possible that you submit
Speaker 1: a probe photo and the system doesn't find any matches
Speaker 1: at all, in which case you'll just get a null. Uh,
Speaker 1: you might get less than what you asked for if
Speaker 1: only a few had met the threshold for reliability. Now
Speaker 1: we call them candidate photos because you're supposed to acknowledge
Speaker 1: the fact that these are meant to help you pursue
Speaker 1: a lead of inquiry. In a case, it is not
Speaker 1: meant to be a source of positive identification of a suspect. So,
Speaker 1: in other words, you shouldn't run a facial recognition software probe,
Speaker 1: get a result back and say that's our guy, let's
Speaker 1: go pick them up. That's not enough. It's meant to
Speaker 1: be the start of a line of inquiry, and uh,
Speaker 1: whether or not it gets used that way all the
Speaker 1: time is another matter. But the purpose of calling it
Speaker 1: candidate photo is to remind everyone this is not meant
Speaker 1: to be proof of someone's guilt or innocence. The FBI
Speaker 1: also allows certain state authorities to use this same database,
Speaker 1: and different agencies have different preferences. So in the g
Speaker 1: a O report that I talked about earlier, the authors
Speaker 1: noted that law enforcement officials from Michigan, for example, would
Speaker 1: always ask for the maximum number of candidate photo is,
Speaker 1: particularly when they'd use probe images that were of low quality.
Speaker 1: So let's say you've got a picture captured from the
Speaker 1: security camera and the lighting is pretty bad and perhaps
Speaker 1: the person wasn't facing dead on into the camera. You
Speaker 1: might ask for the maximum number of candidate photos to
Speaker 1: read returned to you, knowing that the image you submitted
Speaker 1: was low quality and therefore any match is only potentially
Speaker 1: going to be the person you're actually looking for. And again,
Speaker 1: this is all just to help you with the beginning
Speaker 1: of your investigation. It's not meant to be the that's
Speaker 1: our guy moment that you would see and say, uh,
Speaker 1: police procedural that would appear on network television in prime time.
Speaker 1: The FBI also has a policy and that all returned
Speaker 1: candidate photos must first be analyzed by human specialists before
Speaker 1: being passed on to other law enforcement agencies. Up to
Speaker 1: that point, the entire process is automatic, so you don't
Speaker 1: have people overseeing the process once it's probing all of
Speaker 1: the database. But once the results come in, human analysts
Speaker 1: who are supposed to be trained in this sort of
Speaker 1: thing are supposed to look at each of those returned
Speaker 1: candidates and determine if whether or not they really do
Speaker 1: resemble the person in the probe photo that was submitted
Speaker 1: in the first place, and if they're not, they are
Speaker 1: not supposed to be passed on any further down the chain. Now,
Speaker 1: so far, this probably doesn't sound too problematic. The FBI
Speaker 1: has a database containing both criminal and civil photographs, but
Speaker 1: when it runs a probe, it can only use the
Speaker 1: criminal photos or the civil ones that are attached to
Speaker 1: criminal files. Candidate photos are supposed to only be used
Speaker 1: to help start a line of inquiry, not to positively
Speaker 1: identify suspects, and everything has to be reviewed by a
Speaker 1: human being. That sounds fairly reasonable. But even if you're
Speaker 1: mostly okay with this coach, which still has some problems
Speaker 1: will talk about in a bit, things get significantly more
Speaker 1: dicey as you learn more about the FBI's policies. For example,
Speaker 1: they have a unit called the Facial Analysis, Comparison and
Speaker 1: Evaluation Services or FACE FACE. This is a part of
Speaker 1: the Criminal Justice Information Services Department c G c J
Speaker 1: rather I S. Yeah, I can spell justice with the G,
Speaker 1: it doesn't make sense. No, the c J I S.
Speaker 1: This is a department within the FBI, and FACE can
Speaker 1: carry out a search far more wide reaching than one
Speaker 1: that just uses the n G I I p S database.
Speaker 1: FACE uses not only that database but also external databases
Speaker 1: when conducting a search with a probe photo. So let's
Speaker 1: say again, you're an FBI agent and you have an
Speaker 1: image that you want to match. You want to find
Speaker 1: out who this person is. Maybe it's just a person
Speaker 1: of interest and doesn't even necessarily have to be a suspect.
Speaker 1: Could be that, hey, maybe this person can tell us
Speaker 1: more about this thing that happened later on. Well, you
Speaker 1: could follow the n G I I p S procedure,
Speaker 1: which would focus on those criminal photographs, or you could
Speaker 1: submit your image to FACE. FACE then would search dozens
Speaker 1: of databases holding more than four hundred eleven million photographs,
Speaker 1: many of which are from civil sources. So n G
Speaker 1: I I P S has thirty million, all of them
Speaker 1: together a four hundred eleven million pictures. And again a
Speaker 1: lot of those pictures just come from things like passport
Speaker 1: I d U, driver's licenses, sometimes security clearances, that sort
Speaker 1: of stuff. That's the You know, this database is a
Speaker 1: lot of law abiding citizens who have no criminal record
Speaker 1: and the image just have nothing to do with any
Speaker 1: sort of criminal act, but they're in these databases. These
Speaker 1: external databases belong to lots of different agencies, and both
Speaker 1: of the federal level and state level. So you've got
Speaker 1: state police agencies, you've got the Department of Defense, You've
Speaker 1: got the Department of Justice, you have the Department of State,
Speaker 1: and again it contains photos from licenses, passports, security I
Speaker 1: d cards, and more. So your submission would then go
Speaker 1: to one of twenty nine different biometric image specialists. They
Speaker 1: would take that probe photo and run a scan through
Speaker 1: these various databases and they would look for matches. Here's
Speaker 1: another problem. Each of these systems has a different methodology
Speaker 1: for performing and returning search results, which makes this even
Speaker 1: more complicated. For example, I talked about how the n
Speaker 1: G I I P S system gives you a return
Speaker 1: between two and fifty candidate photos. Right, well, the Department
Speaker 1: of State will return as many as a the eight
Speaker 1: candidate photos if they are all from visa applications from
Speaker 1: people who are not US citizens, So you can get
Speaker 1: up to eighty eight pictures from visa applicants, or you
Speaker 1: could just get three images from US citizen passport applicants,
Speaker 1: because that's a hard limit. They can only return three
Speaker 1: candidate photos from US citizens who applied for passports, but
Speaker 1: they can return up to eight visa application photos. The
Speaker 1: Department of Defense will will down all of their candidates
Speaker 1: into a single entry, so, in other words, to burn
Speaker 1: a defense. If you if you query that database with
Speaker 1: your probe photo, you will only get one image back,
Speaker 1: So they will call all the other ones and give
Speaker 1: you the most likely match out of all the ones
Speaker 1: that they find in their search. Some states will do
Speaker 1: similar things where they will narrow down which images they
Speaker 1: will return to you. Some of them will just give
Speaker 1: you everything they've got. Every match that comes up, they'll
Speaker 1: just return it back to the FBI. So it's very complicated.
Speaker 1: You can't really be sure what methods people are using
Speaker 1: to be certain that the the potential matches they have
Speaker 1: represent a good match, a good chance that the person
Speaker 1: that they've returned is actually the same one who was
Speaker 1: in the probe photo. At any rate, you as an
Speaker 1: FBI agent, wouldn't get all of these at all. All
Speaker 1: of these photos that would come back, they would come
Speaker 1: back to that biometric analyst over at face. So your
Speaker 1: you send your request to face face takes care of
Speaker 1: the rest. They get back all these results. Then they
Speaker 1: go through the results they get back and they whittle
Speaker 1: that down to one or two candidate photos and they
Speaker 1: send those on to you, the FBI agent. So by
Speaker 1: the time you get it, you only see one or
Speaker 1: two out of the potentially more than one hundred images
Speaker 1: that were returned on this search. But um, you might ask, well,
Speaker 1: how frequently does this happen? I mean, how how often
Speaker 1: is the FBI looking at images, including pictures of law
Speaker 1: abiding citizens in these virtual lineups. It can't be that frequent, right, Well, again,
Speaker 1: according to that g a O report, the FBI submitted
Speaker 1: two hundred fifteen thousand searches between August two thousand eleven,
Speaker 1: which is pretty much when the program went into pilot
Speaker 1: mode and started to be rolled out more widely through
Speaker 1: December two thousand, fifteen two thousand. From August to December,
Speaker 1: thirty six thousand of those searches we're on state driver's
Speaker 1: licensed databases, So it happens a lot thirty six thousand times.
Speaker 1: Chances are, if you are an adult in America, you had,
Speaker 1: like a coin flip situation, that your image was looked
Speaker 1: at at some time or another by an algorithm comparing
Speaker 1: it to a probe photo in the pursuit of information
Speaker 1: regarding a federal case or in some cases, state cases,
Speaker 1: because the FBI has also allowed certain states law agencies
Speaker 1: access to this approach. Now, according to the rules, the
Speaker 1: FBI should have submitted some important documents to inform the
Speaker 1: public of their policies and to lay down the regulations,
Speaker 1: the rules, the processes that they would have to follow
Speaker 1: in order for this to be fair, for it to
Speaker 1: not encroach on your privacy or to violate civil liberties
Speaker 1: or civil rights. Without those rules, the use of the
Speaker 1: system is largely unregulated, which can lead to misuse, whether
Speaker 1: it's intentional or otherwise. The Government Accountability Office specifically pointed
Speaker 1: out two different types of notific cations that the FBI
Speaker 1: either failed to submit or was just very late in submitting.
Speaker 1: The first is called a Privacy Impact Assessment or p
Speaker 1: i A. Now, as that name suggests, a p i
Speaker 1: A is meant to inform the public about any potential
Speaker 1: conflicts with privacy with regards to methods for collecting personal information.
Speaker 1: The FBI did submit a p i A for its
Speaker 1: next generation system, but they did it back in two
Speaker 1: thousand eight when they first launched the n g I
Speaker 1: I p S. According to the Government Accountability Office, the
Speaker 1: FBI made enough significant changes to the system to warrant
Speaker 1: another p i A that anytime you make a significant
Speaker 1: revision to your personal information systems, you have to submit
Speaker 1: a new p i A because things have changed, and
Speaker 1: according to the g a O, the FBI failed to
Speaker 1: do that for way too long. Now, ultimately the FBI
Speaker 1: would publish a new p i A, but by that point,
Speaker 1: the Government Accountability Office said they had delayed so long
Speaker 1: that it made it more problematic. And as a result,
Speaker 1: because during the whole time that they were supposed to
Speaker 1: have submitted this, they were actively using this system. It
Speaker 1: wasn't like this was a system being tested. It was
Speaker 1: actually being put to use in real cases, and that
Speaker 1: kind of violates it, Well, it doesn't kind of. It
Speaker 1: violates a Privacy Act of nineteen seventy four, which states
Speaker 1: that when you make these revisions, you're supposed to file
Speaker 1: a p i A before you put it into use.
Speaker 1: According to the g a O, the FBI failed to
Speaker 1: do so. And also the longer you wait to file this,
Speaker 1: the more entrenched those uses come. So if you put
Speaker 1: a system in place, you build everything out, you've actually
Speaker 1: taken the time to do it, and then you publish
Speaker 1: a p i A any objections that are raised, you
Speaker 1: could say, well, we've got a system now, and it
Speaker 1: cost one point two billion dollars to put it in place.
Speaker 1: It's gonna cost more money, taxpayer money for us to
Speaker 1: alter it, to remove it, to change it. You could
Speaker 1: argue against any move to amend the situation. And the
Speaker 1: g a O says, that's not playing cricket or playing
Speaker 1: fair for my fellow Americans. So that's a problem. But
Speaker 1: then there's another one. There's a second type of report
Speaker 1: called a Systems of Records Notice or s o R.
Speaker 1: In SORN, the Department of Justice was required to submit
Speaker 1: a sworn upon the launch of n g i I
Speaker 1: P S but didn't do so until May five. The
Speaker 1: g a O criticized both the FBI and the Department
Speaker 1: of Justice for failing to inform the public of the
Speaker 1: nature of this technology and how it might impact personal privacy.
Speaker 1: But wait, there's more. The g O report also accused
Speaker 1: the FBI of failing to perform any audits to make
Speaker 1: certain the use of facial recognition software isn't in violation
Speaker 1: of other policies, or even to make sure it doesn't
Speaker 1: violate the Fourth Amendment rights of U. S citizens. Now,
Speaker 1: for those of you who are not US citizens, you
Speaker 1: might wonder what does this actually mean. Well, the Fourth
Speaker 1: Amendment is supposed to protect us against unreasonable search and seizure,
Speaker 1: and part of that means law enforcement can't just demand
Speaker 1: to search you for no reason. And some have argued
Speaker 1: that using facial recognition software with other person's consent, using
Speaker 1: it invisibly and widespread essentially amounts to crossing that line. Now,
Speaker 1: in the United States, we've got plenty of examples of
Speaker 1: troublesome policies that seem to overstep the bounds that are
Speaker 1: established by the Fourth Amendment. But that's a tirade for
Speaker 1: an entirely different show, probably not a tech stuff, maybe
Speaker 1: a stuff they don't want you to know. There are
Speaker 1: a couple of laws the United States that are important
Speaker 1: to take note of here, besides that Fourth Amendment. One
Speaker 1: of them I just mentioned the Privacy Act of nineteen
Speaker 1: seventy four, and the other one is the e Government
Speaker 1: Act of two thousand two. The Privacy actsts limitations on
Speaker 1: the collection, disclosure, and use of personal information maintained in
Speaker 1: systems of records, including the ones that law agencies use.
Speaker 1: The E Government Act is the one that requires government
Speaker 1: agencies to conduct p I A s to make certain
Speaker 1: that personal information is handled properly in federal systems, and
Speaker 1: the g O report alleges that the FBI policy wasn't
Speaker 1: aligned with either of those. Part of this accusation depends
Speaker 1: upon the fact that the FBI was using FACE in
Speaker 1: investigations for years before they updated their s O r
Speaker 1: N their SORN. According to the Privacy Act, agencies must
Speaker 1: publish a new sworn upon the establishment or revision of
Speaker 1: the system of records. This is what I was talking
Speaker 1: about earlier, except I think I said p I a
Speaker 1: earlier when actually I met s O r N. That's
Speaker 1: entirely my fault because I didn't write in my notes
Speaker 1: and I was talking next to berraneously. But s O
Speaker 1: r N is what I should have said. The FBI
Speaker 1: argued that it was continuously updating the database to refine
Speaker 1: the system. But the g a O S argument was
Speaker 1: that you could be continuously updating the system and argue, well,
Speaker 1: we don't want to publish an s O r N
Speaker 1: after every tiny revision because it's wasteful and time consuming.
Speaker 1: The g A O S counter to that is, yeah,
Speaker 1: but you were using this tool in actual cases. If
Speaker 1: you were developing this, let's say, in a in a
Speaker 1: department where you're not using real cases, you're just gradually
Speaker 1: tweaking the system so that it's more and more accurate
Speaker 1: in a controlled environment. That's one thing. But if you're
Speaker 1: actively making use of this system in real world investigations,
Speaker 1: you absolutely must adhere to these laws, because to do
Speaker 1: otherwise is in violation two laws that are pasting the
Speaker 1: United States. So you can't have it both ways. You
Speaker 1: can't continuously tweak a system and put it to official
Speaker 1: use and not also file these reports. You could argue
Speaker 1: the FBI was trying to have its cake and eat
Speaker 1: it too. It's the expression that I think I actually
Speaker 1: used properly. All Right, we've got more to talk about,
Speaker 1: but it's time for us to take another quick break
Speaker 1: to thank our sponsor. All right. So, the Government Accountability
Speaker 1: Office criticizes the FBI and various other agencies for failing
Speaker 1: to establish the scope and use of its facial recognition technology.
Speaker 1: But that's just the tip of the iceberg, because the
Speaker 1: g O report goes on to make an equally troubling
Speaker 1: point that the FBI had performed only a few studies
Speaker 1: on how accurate these facial recognition systems were in the
Speaker 1: first place. So, in other words, not only was this
Speaker 1: a poor to find an unregulated tool, but it's a
Speaker 1: tool of unknown accuracy and precision, which is terrifying when
Speaker 1: you think about it. Now. According to the report, the
Speaker 1: FBI did perform some initial tests before they deployed the
Speaker 1: n G I I P s, and then occasionally did
Speaker 1: a couple of tests when they made some changes. But
Speaker 1: there were problems with these tests. For one thing, they
Speaker 1: were limited in scope, and they didn't represent how the
Speaker 1: system might be used out in the real world. When
Speaker 1: they were actually running these tests, they ran on about
Speaker 1: nine hundred thousand photographs in the database, so they took
Speaker 1: a subset of the photos that they had. They took
Speaker 1: nine hundred thousand of them, and they ran probe tests
Speaker 1: using photos that they knew either were or were not
Speaker 1: represented in that group of nine hundred thousand. However, you've
Speaker 1: got to remember the full database is more than thirty
Speaker 1: million images, so something that works on a smaller scale
Speaker 1: may not work once you scale it up. For another,
Speaker 1: the tests did not specify how often incorrect matches would
Speaker 1: come back, so you didn't know how many false positives
Speaker 1: were there because the FBI wasn't tracking false positives. They
Speaker 1: were only concerned with how frequently they were getting a
Speaker 1: match too, you know, an actual image. So the way
Speaker 1: they test this is you've got nine D images, they've
Speaker 1: got a probe image. They know for a fact that
Speaker 1: the probe image is inside that database, and then they
Speaker 1: run the search to see if the system sends that
Speaker 1: image back. And their threshold was an eight detection rate
Speaker 1: for a positive match. So, in other words, it went
Speaker 1: like this, Let's say you need to conduct a test
Speaker 1: of this system. This is one way you would determine
Speaker 1: whether or not you had that detection rate. Let's say
Speaker 1: you have a hundred robe photos that you've taken of
Speaker 1: one person, and you know this person's face is in
Speaker 1: that database. You know it's going to be in the
Speaker 1: among those nine hundred thousand or so images, So then
Speaker 1: you submit your query. If you have an eighty five
Speaker 1: percent detection rate, then eighty five of those probe photos
Speaker 1: should come back with a match, and that match should
Speaker 1: be the actual person you're looking for. That's what they met.
Speaker 1: By an eighty five percent detection rate, that eighty five
Speaker 1: percent of the time an image that isn't their database
Speaker 1: would be pulled due to a facial recognition software search. Now,
Speaker 1: during this testing phase, the FBI reported that they met
Speaker 1: this threshold. They used that subset of actually was nine
Speaker 1: twenty six thousand photos as their subset when they were
Speaker 1: testing it, and they said that they had an eighty
Speaker 1: six detection rates, so they actually were exceeding what they
Speaker 1: had set as their threshold. But that just meant that
Speaker 1: six percent of the time the actual match for probe
Speaker 1: photos showed up in a group of fifty candidate images,
Speaker 1: so you would get forty nine other images that were
Speaker 1: not your match. The match would be there of the time,
Speaker 1: along with forty nine other images. So we know that
Speaker 1: the system works if you are asking for the maximum
Speaker 1: number of candidates. Remember in the FBI system, you can
Speaker 1: ask for between two and fifty, but fifty is the max.
Speaker 1: But what happens if you asked for fewer images? What
Speaker 1: if you said, no, I want twenty returns? What's the accuracy?
Speaker 1: Then the FBI can't tell you because they do not know.
Speaker 1: According to the FBI, they did not run tests to
Speaker 1: see what would happen if you decrease the number of
Speaker 1: candidate photos you asked for. They only ran tests on
Speaker 1: the maximum number of candidate photos. And keep in mind,
Speaker 1: the fault for any search is twenty photos, so the
Speaker 1: default is less than what they tested, and they never
Speaker 1: tried to see if the eight six percent detection rate
Speaker 1: held true at these lower numbers. That's a huge issue.
Speaker 1: On top of that, the FBI didn't go so far
Speaker 1: to determine how frequently its system would return false positives
Speaker 1: to probes, so they never paid attention to how many
Speaker 1: times they got responses that didn't reflect and the actual identity.
Speaker 1: They didn't keep track of it. So, according to the FBI,
Speaker 1: the purpose of the system is to generate leads, not
Speaker 1: to positively identify persons of interest, so it shouldn't come
Speaker 1: as a big surprise, or you shouldn't even care if
Speaker 1: it returns a lot of false positives, because hey, this
Speaker 1: technology isn't meant to be the smoking gun that says,
Speaker 1: here's the evidence that will put this person away. It's
Speaker 1: meant to just create a lead, So why do you
Speaker 1: care how many false positives it returns? As if being
Speaker 1: looped in on a an official inquiry when you had
Speaker 1: nothing to do with it isn't disruptive or stressful or
Speaker 1: provoke anxiety. I don't know about you, guys, but if
Speaker 1: I had a federal agent show up at my door
Speaker 1: asking me weird questions about a case that I had
Speaker 1: no connection to because my image had popped up in
Speaker 1: one of these searches and had I have nothing to
Speaker 1: do with it. It just so happens that I look
Speaker 1: enough like a photo that's being used in the case
Speaker 1: to warrant this. I would probably find that pretty disruptive
Speaker 1: in my life, so I would care about false positives. FBI,
Speaker 1: at least according to this g a O report, apparently
Speaker 1: didn't think it was that big a deal. Now, the
Speaker 1: g a O points out that it is a big
Speaker 1: deal and that they're not the only ones to think so.
Speaker 1: The National Science and Technology Council and the National Institute
Speaker 1: of Standards and technolo Ology both state then, in order
Speaker 1: to know how accurate a system is, you need to
Speaker 1: know two pieces of information, not just the detection rate,
Speaker 1: which the FBI claims is at least when you're asking
Speaker 1: for fifty candidates, but also the false positive rate. You
Speaker 1: have to know both of them in order to understand
Speaker 1: how accurate a system is. So only knowing one of
Speaker 1: those pieces of information isn't enough to state this system
Speaker 1: is accurate or not. You have to know both. So
Speaker 1: not only do does the FBI not have a grasp
Speaker 1: on how accurate their system is if you're asking for
Speaker 1: fewer than the maximum number of candidates, they also don't
Speaker 1: know how often it returns false positives. So the FBI
Speaker 1: has no way of knowing how accurate this facial recognition
Speaker 1: software is considering that it's being used to actually further investigations.
Speaker 1: For official investigations of the FBI and also other state
Speaker 1: agencies that have access to the system, that is beyond problematic.
Speaker 1: If you cannot say that the system with any degree
Speaker 1: of certainty is above a certain threshold of accuracy. Why
Speaker 1: are you using it? Because, I mean, it has the
Speaker 1: potential to dramatically impact people's lives and potentially lead people
Speaker 1: down the pathway that could result in in false accusations
Speaker 1: and imprisonment. Uh, the person who is actually responsible might
Speaker 1: totally get away with something because of this. This is
Speaker 1: a real problem. And then the thing is it might
Speaker 1: be a perfectly accurate system, but we don't know that
Speaker 1: because we haven't tested it. So until we test it,
Speaker 1: we cannot just assume that it's accurate enough. That's not
Speaker 1: when people's lives are at stake. This is where the
Speaker 1: my bias doesn't so much creep in as it kicks
Speaker 1: open the door and makes itself at home on your couch,
Speaker 1: but I die gress. The g a O report also
Speaker 1: goes into great detail about how this accuracy really can
Speaker 1: have a clear impact on people's privacy, their civil liberties,
Speaker 1: their civil rights. They also cite the Electronic Frontier Foundation
Speaker 1: the e f F, which says that if a person
Speaker 1: is brought up as a defendant in a case and
Speaker 1: it is revealed that they were matched by a facial
Speaker 1: recognition system. It puts a burden on the defendant to
Speaker 1: argue that they are not the same person as was
Speaker 1: seen in a probe photo, that they are not the
Speaker 1: same one that the system has identified. And if you
Speaker 1: cannot reliably state how accurate your system is because you
Speaker 1: don't know how frequently it returns false positives, you have
Speaker 1: unfairly burdened the defendant. Like if you were to say,
Speaker 1: if you're the FBI, and you say, we have an
Speaker 1: eight six percent detection rate, but you don't admit, oh,
Speaker 1: by the way, we don't know how many false positive
Speaker 1: as we get on any given search. The implication you
Speaker 1: have given is that we're pretty sure that this is
Speaker 1: the right guy. And again they argue that this is
Speaker 1: meant to be a point of inquiry, but you could
Speaker 1: easily see how it could also be used by a
Speaker 1: lawyer to argue that a defendant is in fact the
Speaker 1: person responsible for a crime, and they may not be.
Speaker 1: And because you don't know the accuracy of the system,
Speaker 1: you can't like using the system to argue for it
Speaker 1: is irresponsible. There's no accountability there. Now, not only has
Speaker 1: the FBI failed to establish the accuracy of its own
Speaker 1: n G i I p S system. It has also
Speaker 1: not assessed the accuracy of all those external databases that
Speaker 1: are used whenever they use the face approach. There are
Speaker 1: no accuracy requirements for these agencies, so there's not like
Speaker 1: a threshold they have to prove that they meet in
Speaker 1: order to be part of this. That's a huge problem.
Speaker 1: While each agency might be accurate with no testing procedure
Speaker 1: in place, it's impossible to be certain of that. And
Speaker 1: since these databases include millions of people with no criminal
Speaker 1: background and they all use different facial recognition software products,
Speaker 1: this is a huge issue. You could be put in
Speaker 1: a virtual lineup simply because you look enough like someone
Speaker 1: else that a computer thinks you are in fact the
Speaker 1: same person. The g a O report concludes with a
Speaker 1: host of recommendations for future actions, including addressing the problem
Speaker 1: of the FBI being so slow to publish those updated
Speaker 1: a p I as in a timely manner, and to
Speaker 1: create a means to assess each system's accuracy. The Department
Speaker 1: of Justice read the report and then responded disagreeing with
Speaker 1: several points that the g O report made, uh including
Speaker 1: arguing that the FBI and the Department of Justice published
Speaker 1: information when it made the most sense when the system
Speaker 1: had been tweaked and find analized more or less. However,
Speaker 1: by that time, again, they had been using that system
Speaker 1: for real world cases throughout the entire process, So it
Speaker 1: seems to me to be kind of a weak argument.
Speaker 1: You can't really say, like, hey, it wasn't finished until then,
Speaker 1: that's when we published it. If you allso at saying hey,
Speaker 1: we use that for real zes to go after actual people,
Speaker 1: you can't have it both ways and not not maintain
Speaker 1: accountability at any rate. So that kind of gets to
Speaker 1: the end of the Government Accountability Office report. But that's
Speaker 1: not the end of the story. In March, Congress held
Speaker 1: some hearings about this, and boy haality were some congress
Speaker 1: people very upset with the FBI. On both sides of
Speaker 1: the aisle. You had Democrats and Republicans really chastising the
Speaker 1: FBI for their use of facial recognition software and arguing
Speaker 1: that it could amount to an enormous invasion of privacy
Speaker 1: as well as endangering the civil liberties of U. S citizens.
Speaker 1: So people who have dramatically different political philosophies were agreeing
Speaker 1: on this point. So it wasn't really a partisan issue.
Speaker 1: In this case, and it got pretty ugly, but probably
Speaker 1: not as ugly as the Georgetown University report that was
Speaker 1: published in late This is an amazing report. Both the
Speaker 1: Government Accountability Office report and the Georgetown University report are
Speaker 1: available for free online. I will warn you collectively they're
Speaker 1: about two hundred pages, so if you want some light reading,
Speaker 1: you can check it out there. They are quite good,
Speaker 1: both of them, and they're very accessible. Neither of them
Speaker 1: are written in like crazy legal lease which will make
Speaker 1: it impossible to understand. They're written in very plain English. Now.
Speaker 1: It was in the Georgetown University report that was revealed
Speaker 1: that one in every two American adults has their picture
Speaker 1: contained in a database connected to law enforcement facial recognition systems.
Speaker 1: And this report goes far beyond just that FBI to
Speaker 1: state all the way down to state and local systems
Speaker 1: that are implementing their own facial recognition databases, and many
Speaker 1: of them have no understanding of how it might impact
Speaker 1: the civil liberties or privacy of citizens. The report is
Speaker 1: the summary of a study that lasted a full year
Speaker 1: with more than one records requests to various police departments.
Speaker 1: They looked at fifty two different law enforcement agencies across
Speaker 1: the United States, and the report assessed the risks to
Speaker 1: civil liberties and civil rights because up until this report
Speaker 1: was was filed, no such study had been made, which
Speaker 1: is a huge problem. You don't know the impact of
Speaker 1: the tool that you've created until after it's been put
Speaker 1: in use for a while. That's an issue. Ideally, you
Speaker 1: think all this out before you implement the procedure, and
Speaker 1: their findings were pretty upsetting. For example, the report found
Speaker 1: that some agencies limit themselves to using facial recognition within
Speaker 1: the framework of a targeted and public use, such as
Speaker 1: using it on someone who has been legally arrested or
Speaker 1: detained for a crime. And in this case, you're talking
Speaker 1: about totally above board approach. You're assuming that everyone is
Speaker 1: is following the law as regards to apprehending and charging
Speaker 1: a suspect with a crime, and maybe that person is
Speaker 1: unwilling or unable to to tell you what what their
Speaker 1: identity is, and in that case, you would use this
Speaker 1: facial recognition software stuff in order to figure out who
Speaker 1: you are dealing with. That's largely a legitimate case. You know,
Speaker 1: the government. The Georgetown University study didn't say that's bad.
Speaker 1: They actually said, no, that's that makes sense. It's targeted,
Speaker 1: it's public. But you could have a more general, invisible approach,
Speaker 1: for example, using facial recognition software in real time on
Speaker 1: a closed circuit camera pointed at a city street, where
Speaker 1: you're literally picking up people as they pass by. They're
Speaker 1: not people of interest, they're just people going about their day.
Speaker 1: And if you're running facial recognition software on such a feed,
Speaker 1: you are potentially invading privacy and stepping on civil rights
Speaker 1: and civil liberties. So even if you were to argue
Speaker 1: that this real time use where you're just looking at
Speaker 1: people as they pass by, maybe a little name pops
Speaker 1: up every now and then as it as the system
Speaker 1: recognizes a person that matches a file in the database,
Speaker 1: it's easy to imagine a scenario in which such a
Speaker 1: technology could be abused. Either it picks up somebody mistakenly,
Speaker 1: it thinks that identifies someone, but in fact it's a
Speaker 1: totally different person, and then you end up establishing a
Speaker 1: person's location by mistake, like it it's not really where
Speaker 1: they were, but because the system has identified a person
Speaker 1: as being at X place at Y time, you then
Speaker 1: have established supposedly that person's location. When in fact that
Speaker 1: person might be across town or not even in the
Speaker 1: same state, but it's because of a misidentification in the system.
Speaker 1: That's one problem. But think about this. Think of this
Speaker 1: is a scary scenario. Imagine a situation in which a
Speaker 1: group of people are discriminated against by a government agency.
Speaker 1: Let's say they have a legitimate gripe. It's completely legitimate.
Speaker 1: They're victims of unfair treatment. So a group of them
Speaker 1: and some of their allies get together in a public
Speaker 1: place for peaceful protest, to raise awareness of this issue
Speaker 1: and to confront uh the government agencies that have discriminated
Speaker 1: against them. This is all perfectly legal according to the U.
Speaker 1: S Constitution. They're not doing anything legal. They're assembling on
Speaker 1: public grounds in order to practice free speech. But it's
Speaker 1: not hard to imagine a government agency using a camera
Speaker 1: when this sort of facial recognition software to identify people
Speaker 1: who are in the crowd, in order to use that
Speaker 1: as leverage in the future for some purpose or another,
Speaker 1: even if it's just to say we know you were there,
Speaker 1: and to put that kind of pressure on a person
Speaker 1: in order to essentially squelch people's freedom of speech. So
Speaker 1: this is a first Amendment issue, not just a Fourth
Speaker 1: Amendment issue. Now that might sound like a dramatic scenario
Speaker 1: like something like Big Brother issue, its Orwellian, but it's
Speaker 1: also entirely within the realm of possibility. From a technological standpoint,
Speaker 1: there's nothing technology technologically oriented that would prevent us from
Speaker 1: doing this, or prevent an agency from doing this. And
Speaker 1: even without the evil Empire scenario in place, you still
Speaker 1: have the problematic issue of treading on civil liberties just
Speaker 1: by having such technology available and unregulated. You don't have
Speaker 1: rules to to guide this sort of stuff. The Georgetown
Speaker 1: report found that only one agency out of the fifty
Speaker 1: two that they looked at have a specific rule against
Speaker 1: using facial recognition software to identify people participating in public
Speaker 1: demonstrations or free speech in general. So only one agency
Speaker 1: actually has rules against that. Now, that doesn't mean the
Speaker 1: other fifty one agencies are regularly using this technology to
Speaker 1: monitor acts of free speech, but it also doesn't mean
Speaker 1: that they can't. They don't have rules against it. Only
Speaker 1: one agency out of the fifty two, people are being
Speaker 1: watched and identified without any connection to a crime. In
Speaker 1: these cases, it's pretty terrifying. The Georgetown report also found
Speaker 1: that no state had yet passed a law to regulate
Speaker 1: police use of facial recognition software. No, no state in
Speaker 1: the US. They're fifty of them, and none of them
Speaker 1: have passed any regulations, any laws to regulate the use
Speaker 1: of facial recognition software. So without rules, how do you
Speaker 1: argue whether someone's misused or abused a system. You have
Speaker 1: to have rules so that you know what is allowed
Speaker 1: and what is not allowed. With no rules, the implication
Speaker 1: is that everything is allowed until it isn't. That's a
Speaker 1: that's a huge dangerous problem. The report also pointed out
Speaker 1: that most of these agencies lacked any sort of methodology
Speaker 1: to ensure that the accuracy of their respective systems was
Speaker 1: was decent. The reports stated that out of all the
Speaker 1: agencies they investigated, only to the San Francisco Police Department
Speaker 1: and the South Sound nine one from Seattle had made
Speaker 1: decisions about what facial recognition software they were going to
Speaker 1: incorporate in their office based off of accuracy rates. That
Speaker 1: that was not a consideration for all of the other agencies,
Speaker 1: at least not the ones that they asked. Moreover, the
Speaker 1: report points out that facial recognition companies are also trying
Speaker 1: to have it both ways. So, for example, they cite
Speaker 1: a company called fat face First. Now face First advertises
Speaker 1: that has accuracy rate, but it's simultaneously disclaims any liability
Speaker 1: for failing to meet that accuracy rate. So it's kind
Speaker 1: of like saying we we guarantee these tires. Tires are
Speaker 1: not guaranteed. Not quite like that, but similar. So again,
Speaker 1: this is according to the Georgetown University report. That's a
Speaker 1: problem for a company to to sell itself on a
Speaker 1: on a performance threshold, but then say, hey, you can't
Speaker 1: hold us to that performance threshold that we sold you on.
Speaker 1: That's a little dangerous there too. Then the report goes
Speaker 1: on to state that the human analysts, you know, the
Speaker 1: ones I was talking about earlier, that supposed to be
Speaker 1: a safeguard. Human analysts are supposed to take the images
Speaker 1: that are returned by these automated systems and manually review
Speaker 1: them to make sure that they do or do not
Speaker 1: match that probe photo. That was the whole thing to
Speaker 1: begin with, But it turns out, according to this report,
Speaker 1: those human analysts are not that accurate. In fact, they're
Speaker 1: no better than a coin flip. Literally. The report sites
Speaker 1: of study that showed that if analysts did not have
Speaker 1: highly specialized training, they would make the wrong decision for
Speaker 1: a potential match fifty percent of the time, literally a
Speaker 1: coin flip. That's ridiculous. Now, the report found only eight
Speaker 1: agencies of the fifty two used specialized personnel to review images.
Speaker 1: In other words, people who presumably have actually received that
Speaker 1: highly specialized training necessary to make more accurate decisions regarding
Speaker 1: these photos. And the report states that there's no formal
Speaker 1: training regime in place for examiners, which is a major
Speaker 1: problem for a system that's already in widespread use. So
Speaker 1: not only do you need highly specialized training, there's no
Speaker 1: formalized approach to to give or receive that highly specialized training.
Speaker 1: So we know you need it, but we haven't developed
Speaker 1: the best practices to actually deliver upon that. So meanwhile,
Speaker 1: you've got human analysts who are making mistakes half the
Speaker 1: time while reviewing these photos. And if you wondered if
Speaker 1: facial recognition systems would disproportionately affect some ethnicities over others,
Speaker 1: the answer to that is resounding and dismaying yes. The
Speaker 1: report found that African Americans would be affected more than
Speaker 1: other ethnicities. According to an FBI co authored study, that
Speaker 1: was cited by this Georgetown University report. Several facial recognition
Speaker 1: algorithms are less accurate for black people than for other ethnicities,
Speaker 1: and there's no independent testing process to determine if there's
Speaker 1: a racial bias in any of these facial recognition systems,
Speaker 1: So no one has developed a test to make certain
Speaker 1: that it is in fact accurate despite a person's age, gender,
Speaker 1: or race. Without being able to verify that it is
Speaker 1: accurate across all parameters, you have opened up an enormous
Speaker 1: can of worms, and you are disproportionately affecting people just
Speaker 1: because of their race because your system does not address
Speaker 1: that properly. The report also points out that the information
Speaker 1: about the systems and use had not been generally available
Speaker 1: to the public. In fact, all the fifty two agencies
Speaker 1: that they they contacted, only four had publicly available use policies.
Speaker 1: So in other words, only four of the fifty two
Speaker 1: could tell you what general policy was as far as
Speaker 1: facial recognition software goes. That's less than ten percent of
Speaker 1: all of the agencies they looked at, and only one
Speaker 1: of those agencies, which was San Diego's Association of Governments,
Speaker 1: had legislative approval for its policy. All the others were
Speaker 1: just self appointed policies that had not passed through any
Speaker 1: kind of official legislative support. Finally, the report asserted that
Speaker 1: most of these systems did not have an official audit
Speaker 1: process to determine if or when someone misuses the systems.
Speaker 1: Nine agencies reported that they did have a process, but
Speaker 1: only one provided Georgetown with any evidence that they had
Speaker 1: a working audit system, and that was the Michigan State Police,
Speaker 1: by the way, who said, we have an audit system,
Speaker 1: and here's proof that it actually works the way we
Speaker 1: said it did. So good on you, Michigan State for
Speaker 1: having that system in place and being able to back
Speaker 1: it up now. The Georgetown University report also urge some
Speaker 1: major changes in the way law enforcement uses facial recognition,
Speaker 1: including an appeal to Congress to create clear regulations to
Speaker 1: define the parameters of when such a system could be used.
Speaker 1: They also called for companies to publish processes that test
Speaker 1: their products accuracy regardless of race, gender, and age to
Speaker 1: remove that possibility of bias. And if we're being really
Speaker 1: super kind and generous toward law enforcement, we could say
Speaker 1: this is just another case where technology has clearly outpaced
Speaker 1: the law we see that all the time. Driverless cars,
Speaker 1: artificial intelligence, lots of different technologies are advancing far faster
Speaker 1: than legislation can come up with. All right, that's fair,
Speaker 1: we see it happen. However, it's particularly troublesome that this
Speaker 1: is happening within law enforcement that is already employing this
Speaker 1: technology before we've developed the policies to guide it. It's
Speaker 1: one thing to say, someone's out here working on a
Speaker 1: driverless car, and we need to start thinking about how
Speaker 1: are we going to regulate that in the future. Maybe
Speaker 1: right now we say you aren't allowed to operate your
Speaker 1: driverless car until we figured this out. That's fair. It's
Speaker 1: another thing to say, there's this technology that could potentially
Speaker 1: impact people's lives and we're allowing law enforcement to use
Speaker 1: it while we try and figure out the rules. That's
Speaker 1: at best a problem. And as I said at the
Speaker 1: top of the show, I'm really just talking about the
Speaker 1: United States with particulars here, but this is happening all
Speaker 1: around the world. There are lots of governments around the
Speaker 1: world that are incorporating facial recognition software along with law enforcement.
Speaker 1: So while I'm using specific US examples in this podcast,
Speaker 1: the same is true for lots of other places. Of course,
Speaker 1: the laws that protect the citizens can be different from
Speaker 1: country to country. Um and in some cases there might
Speaker 1: not be very many outlets for citizens to to voice
Speaker 1: their concern, or it might even be dangerous to do so.
Speaker 1: But this is something I think we need to be
Speaker 1: aware of. I'm not generally the kind of person who
Speaker 1: tells you that you're being watched or you know you
Speaker 1: should be paranoid. But I'm also not the person to
Speaker 1: just sit back and let something go on when I
Speaker 1: feel like it's potentially more of a problem than a solution.
Speaker 1: All Right, that's it. I'm done. It's an important topic,
Speaker 1: and it's one that's still developing. Obviously, perhaps once legislation
Speaker 1: has been passed, once regulations are in place, once we
Speaker 1: have more definition about what law agencies can and cannot
Speaker 1: do with this technology, maybe I'll revisit this topic and
Speaker 1: talk about whether or not it works, or whether or
Speaker 1: not it is a still a good idea or a
Speaker 1: bad idea, or are there any other problems that we
Speaker 1: did not anticipate when we had this podcast. But for now,
Speaker 1: we can conclude this, and I hope to do a
Speaker 1: more Zany Happy Fun tech Stuff for our next episode.
Speaker 1: In the meantime, if you have any suggestions for future
Speaker 1: topics for tech Stuff, email me. My email address is
Speaker 1: tech stuff at how stuff works dot com, or you
Speaker 1: can always drop me a line on Twitter or Facebook.
Speaker 1: The handle for that the show is tech Stuff h
Speaker 1: s W at both Facebook and Twitter. You can also
Speaker 1: go to twitch dot tv slash tech stuff to watch
Speaker 1: me live stream this show. If you want to see
Speaker 1: me make mistakes live on camera and hear about these
Speaker 1: podcasts about a month before they actually publish, you can
Speaker 1: check it out a record on Wednesdays and Friday's Twitch
Speaker 1: dot tv slash tech stuff for more details and I
Speaker 1: will talk to you guys again really soon. For more
Speaker 1: on this and thousands of other topics. Is that how
Speaker 1: stuff works dot com.