Speaker 1: My welcome to Stuff to Blow Your Mind, the production
Speaker 1: of My Heart Radio. Hey you welcome to Stuff to
Speaker 1: Blow your Mind. My name is Robert Lamb and I'm
Speaker 1: Joe McCormick. And right before we started recording today, we
Speaker 1: were just talking about that iconic scene and returned to
Speaker 1: the Jedi where the droids are sent to the droid
Speaker 1: torture Chamber. Do you remember there? I guess it's not
Speaker 1: just a droid torture chamber. It's sort of like the uh,
Speaker 1: the droid onboarding center right where the you know, R
Speaker 1: two D two and C three PO have been given
Speaker 1: as gifts to job of the Hut and they go
Speaker 1: meet their new like droid boss and he's like, yeah,
Speaker 1: you're a feisty little one, and he's signing them in,
Speaker 1: but uh, he sees that R two D two is
Speaker 1: a is a bad robot who needs a discipline, and
Speaker 1: R two D two is confronted with these images of
Speaker 1: robots being punished with various corporal punishments, like one is
Speaker 1: getting stretched on a robot rack and another one is
Speaker 1: getting its feet burned. Yes, this is a this is
Speaker 1: a great scene one that that that definitely burns its
Speaker 1: way into your your brain as a as a young viewer,
Speaker 1: and maybe you don't think about it that much for
Speaker 1: a long time, but uh, it's it's still in there.
Speaker 1: It's it takes place in the bowels of Joba's palace
Speaker 1: on tattooing, and it's um, yeah, it's like droid intake
Speaker 1: but also droid corrections. It's there are there are a
Speaker 1: number of different departments that I think are converging here
Speaker 1: and and it it ultimately kind of raises some interesting
Speaker 1: questions about um about ethics and punishment and in crime,
Speaker 1: and certainly as it relates to two robots. Uh. Of course,
Speaker 1: one thing, this important distress here is like none of
Speaker 1: this was really intended in these scenes. This was about
Speaker 1: having droids doing things that humans would be doing to
Speaker 1: each other in other pieces of cinema, certainly things like
Speaker 1: old pirate movies or old and bad movies or what
Speaker 1: have you. I mean, that's kind of that's kind of
Speaker 1: Star Wars in a nutshell, right. Uh. This whole portion
Speaker 1: of Return of the Jedi is essentially a big pirate movie,
Speaker 1: a big swashbuckler set in an alien location. Oh yeah,
Speaker 1: the job of the Hut is a pirate captain yeah,
Speaker 1: but something interesting occurs when you replace the humans in
Speaker 1: these these tropic scenes with machines. Uh, and then you
Speaker 1: think about it, you know, you think about, well, why
Speaker 1: is that robot torturing the other? Like as if it
Speaker 1: makes perfect sense if it's humans doing it. But then
Speaker 1: the things that we create in our image when they're
Speaker 1: doing it, Suddenly we start seeing the flaws in our reasoning.
Speaker 1: Suddenly we start questioning, well, how does this whole system
Speaker 1: supposed to work? And maybe this whole system doesn't work. Well. Yeah,
Speaker 1: there are multiple levels of absurdities in the scene. One
Speaker 1: is the idea that this robot is just sort of
Speaker 1: like coolly telling are two that he is going to
Speaker 1: learn some discipline. But then the image that accompanies that
Speaker 1: is like, clearly just like extreme robot tor sure, like
Speaker 1: it's something way beyond what would have to do with
Speaker 1: with discipline in the real world. But then the other
Speaker 1: level of absurdity is that it's robots in the scene,
Speaker 1: but coming off of the issue of just like barbaric
Speaker 1: pirate torture directly and more to the broader question of
Speaker 1: robots and discipline and punishment. Uh, this is something that
Speaker 1: we actually wanted to talk about today because the issue
Speaker 1: of robot moral and legal agency is something I've been
Speaker 1: interested in for a long time. I've talked about it,
Speaker 1: It's come up on the show in the past and
Speaker 1: in briefer ways, um And today I wanted to come
Speaker 1: back and devote a full episode to the subject. I
Speaker 1: guess actually we're gonna be talking about this for a
Speaker 1: couple of episodes now. The question of as machines AI
Speaker 1: robots become more independent and act more like agents, more
Speaker 1: like humans do, how are we to understand their moral
Speaker 1: and legal culpability when they do something that harms people?
Speaker 1: And is there such a thing as robot punishment robot discipline?
Speaker 1: Do these concepts reflect anything that's achievable in the real
Speaker 1: world and practical and if so, how would any of
Speaker 1: this work? Yeah? I think one of the most interesting
Speaker 1: things about this topic is that it does force us
Speaker 1: to force a face off between what robots and AI
Speaker 1: actually are or will be, and how we think about them, indeed,
Speaker 1: how we anthropomorphize them. Um And perhaps it might be
Speaker 1: helpful to take a step back and think about something
Speaker 1: far less advanced as a robot and think something I
Speaker 1: think about something more like a hammer Okay, so everyone's
Speaker 1: heard the old adage that it's a poor carpenter who
Speaker 1: blames their tools, right, But of course we do this
Speaker 1: all the time. Uh, the hammer slips, it hits our fingers,
Speaker 1: and we may, at least in the heat of the moment,
Speaker 1: blame the hammer for the failure. Now, we may get
Speaker 1: over this quickly, but then again, we may decide that
Speaker 1: the hammer truly is at fault and it should be
Speaker 1: used less. We might also take this idea to a
Speaker 1: number of different extremes. We might decide that the hammer
Speaker 1: is not merely at fault but faulty, and then we're
Speaker 1: entitled to at least a refund for its purchase. Or
Speaker 1: we might decide that the hammer needs to actually be punished.
Speaker 1: And and this, of course is ridiculous. And yet the
Speaker 1: idea of punishing the hammer by say, putting it in
Speaker 1: the corner, or perhaps you have an old toolbox of
Speaker 1: shame that's just for the misbehaving tools, or maybe it's
Speaker 1: it's less thought out and you just throw the hammer
Speaker 1: across the yard as punishment for what it has done
Speaker 1: to you. Um. Again, these are ridiculous things to do,
Speaker 1: but the idea of doing them is not that far
Speaker 1: and from us. Um, those of you listening, you may
Speaker 1: have engaged in this sort of thing as well. You
Speaker 1: might also simply throw the tool away and otherwise perfectly
Speaker 1: good tool. Um. I know that I did this once
Speaker 1: with a knife sharpening gadget that caused me to cut
Speaker 1: my finger. My and my like reaction was this thing
Speaker 1: has now injured me, it has drawn my blood. Uh
Speaker 1: I am, I'm getting rid of it. It goes in
Speaker 1: the trash. It bore malice again to me. Yeah, or
Speaker 1: you know, I you know ultimately it's I mean, I
Speaker 1: mean you you get into arguments about different tools, like
Speaker 1: is this a dangerous tool? And in that case that
Speaker 1: was my reasoning. It's like this tool is dangerous. It's
Speaker 1: not enabling me to do what I want to do
Speaker 1: without drawing blood, So it goes in the trash. Um.
Speaker 1: But then there have been other cases where like I
Speaker 1: had a mandolin for slicing up carrots, and um, I
Speaker 1: like nicked my finger on it. But oh I I
Speaker 1: nicked my fingers. Things are brutal, they can be. But
Speaker 1: I nicked my finger, not using it but going into
Speaker 1: the drawer for something else. So I punished it by
Speaker 1: putting at the very bottom of the drawer, but I
Speaker 1: didn't throw it away. Uh huh. So I think if
Speaker 1: we all think, think back, you know, we have examples
Speaker 1: of this sort of thing from our from our life.
Speaker 1: Well sure, I mean, I'm going to talk in this
Speaker 1: episode about some of the ways that we mindlessly apply
Speaker 1: social rules to robots. But yeah, I think what you're
Speaker 1: illustrating here is that you don't even have to get
Speaker 1: to the robot agency stage of four people start doing that.
Speaker 1: I mean people mindlessly, to a lesser extent, mindlessly apply
Speaker 1: social rules and rules derived for managing human relationships to
Speaker 1: inanimate objects with no moving parts. Yeah. Yeah, you don't
Speaker 1: even have to get into a room ba or anything,
Speaker 1: or you know, you can deal with the hammer, the
Speaker 1: can opener. But we also you know, it's it's also
Speaker 1: a sad fact that many pet owners will punish an
Speaker 1: animal for a transgression transgression, but scientific evidence shows that
Speaker 1: this tends to not actually work, at least in most
Speaker 1: of the circumstances that it's used. Um So, you know,
Speaker 1: even it's not merely with with tools and inanimate objects,
Speaker 1: but even non human entities were liable to engage in
Speaker 1: this kind of discipline based thinking. Now, most of the
Speaker 1: studies I think that necessarily relate to dogs, if I
Speaker 1: remember correctly, And there's a lot going on here that
Speaker 1: doesn't relate directly to inanimate objects and robots, but it
Speaker 1: illustrates how we tend to approach the punishment of other
Speaker 1: agents and perceived agents. Well, yeah, there's a disconnect, and
Speaker 1: this will be highlighted in one of the papers we're
Speaker 1: going to talk about in in this pair of episodes,
Speaker 1: But there's a disconnect in that punishment is often logically
Speaker 1: characterized as serving one type of purpose, but then is
Speaker 1: applied more like it serves another type of purpose. So
Speaker 1: like it is logically explained as say a deterrent, right,
Speaker 1: I mean, if you if you're talking about uh, legal
Speaker 1: theories of punishment, one of the main things that people
Speaker 1: come up with is say, well, the remedy provided by
Speaker 1: the law is in order to punish the person who
Speaker 1: did the bad thing in order to send a message
Speaker 1: that people should not do this bad thing and thus
Speaker 1: maybe discourage other people from doing something similar in the future,
Speaker 1: or discourage the same person from doing it again. And
Speaker 1: if it were to actually serve that purpose, it's debatable
Speaker 1: in what cases it does actually serve that purpose. Maybe
Speaker 1: sometimes it does, but that is a you know, you
Speaker 1: could argue that's a rational, logical thing that prevents harm.
Speaker 1: But the way punishment is actually often inflict it in
Speaker 1: the real world seems to be more consistent with judgments
Speaker 1: based on like emotional satisfaction of the idea of having
Speaker 1: been wronged. Yeah. Yeah. And then also we get into
Speaker 1: this area where, uh, we have a couple of different
Speaker 1: factors encouraging traditions of discipline. Um, particularly if we look
Speaker 1: at a parenthood, which there's some crossover between discipline and
Speaker 1: parenthood and discipline and the criminal justice system. But uh,
Speaker 1: you know, uh, not everything is going to line up
Speaker 1: one to one here. But um, on the childhood example,
Speaker 1: it's been argued that parents use punishment first of all,
Speaker 1: because it's an emotional response out of anger and anger
Speaker 1: that may be mismanaged. But then on top of this,
Speaker 1: it's you know, something that's culturally passed down and punishment
Speaker 1: may seem to work. I was reading about this in
Speaker 1: a Psychology Today article by Michael Carson, pH d j D.
Speaker 1: And uh, this is what they said, quote, because the
Speaker 1: child is an bited in your presence it's easy to
Speaker 1: think they would be inhibited in your absence. Punishment produces politeness,
Speaker 1: not morality. Thus the inhibited, obedient child inadvertently reinforces the
Speaker 1: parents punitive behavior by acting obedient. For the sorts of
Speaker 1: parents who find obedient children reinforcing, Yeah, that raises an
Speaker 1: interesting question. I mean, I've been mainly thinking for this
Speaker 1: episode about about legal punishments, but like when it comes
Speaker 1: down to parenting, that's a very different kind of thing
Speaker 1: because both parenting in the legal system involved punishment, but
Speaker 1: parenting is not subject to a legal system, right, So
Speaker 1: there's no there is no systematized way by which justice
Speaker 1: is administered from a parent. It's just just I mean,
Speaker 1: I think a lot of times it's just sort of
Speaker 1: like whatever the parent can manage to do in the moment,
Speaker 1: because like the kids driving them crazy or something. Yeah,
Speaker 1: usually the child can't take it to a higher court.
Speaker 1: But I mean, I think you're absolutely right that whether
Speaker 1: you're talking about discipline administered by a parent or the
Speaker 1: justice system is whole, I'd say that both are probably
Speaker 1: based more on tradition and philosophy and less on a
Speaker 1: scientifically rigorous study of the most efficient ways to reduce harm.
Speaker 1: And one of the interesting things about thinking about how
Speaker 1: law could potentially be applied to harm caused by autonomous
Speaker 1: machines is that it may help give us some insights
Speaker 1: on ways that the justice system as it exists and
Speaker 1: is applied to humans today tends to behave irrationally already,
Speaker 1: like with respect to humans. Yeah, and again, this is
Speaker 1: what's so interesting about this this paper, I mean, well,
Speaker 1: the papers that we're going to discuss this topic in general, though,
Speaker 1: is if you start, you start comparing machine possibilities to
Speaker 1: human possibilities, and it's on one level of thought experiment
Speaker 1: in how you would hold machines responsible, but then it
Speaker 1: makes you rethink the way humans are held responsible. You know.
Speaker 1: It's like, um, you know, I think you have a
Speaker 1: pretty square away. Like if if an adult sells a
Speaker 1: pack of cigarettes to someone who's underage, right, but then
Speaker 1: one of a machine does the same thing, how do
Speaker 1: you treat the machine? Do you treat a machine like
Speaker 1: an adult? And then in trying to figure out how
Speaker 1: to treat this machine, does it make you rethink how
Speaker 1: you should be treating the adult who engaged in this behavior?
Speaker 1: I don't know. Yeah, And I think a lot of
Speaker 1: that will come down to our understanding of what the
Speaker 1: machine is capable of, like what kind of constraints it has,
Speaker 1: what type of what level of autonomy it seems to
Speaker 1: be operating at. I mean, again, Weirdly, even when people
Speaker 1: set out to define clear rules for what makes a
Speaker 1: machine culpable, there there's still going to be a lot
Speaker 1: of subjectivity in it. I'm looking at like legal definitions
Speaker 1: of what constitutes a robot versus just a machine, and
Speaker 1: some of these definitions involved things like, well, a robot
Speaker 1: feels like a social agent. So there's still, like, you know,
Speaker 1: an element of subjectivity. But I think that's correct in
Speaker 1: how we actually apply the term most of the time, right,
Speaker 1: Like something is like a gut feeling about how this
Speaker 1: machine is behaving in your world. Is it acting more
Speaker 1: like a fixed, you know, brainless machine or is it
Speaker 1: acting a little bit more like a person. So while
Speaker 1: it would be one thing if if it were basically
Speaker 1: a cigarette vending machine that was selling to children, but
Speaker 1: if it were a machine that went door to door
Speaker 1: and and rang the doorbell and then asked for the
Speaker 1: children so they could sell them cigarettes, that would be
Speaker 1: a different matter. I mean, yeah, I mean, I think
Speaker 1: that would require different types of remedies probably, Yeah, I
Speaker 1: mean I think a lot of people would probably look
Speaker 1: at the cigarette vending machine and say, where was the
Speaker 1: vending machine placed? Why was it in a place that
Speaker 1: children could have access to it? Rather than attacking the
Speaker 1: fundamentals of the machine itself. If it's going door to
Speaker 1: door and giving cigarettes to kids, yeah, then people are
Speaker 1: probably going to attack the fundamentals and the moral character
Speaker 1: of the robot. Right, you have not attacked the robot itself.
Speaker 1: It would just rob justice in somebody's front yard. Yeah,
Speaker 1: thank Alright, So I guess I want to introduce one
Speaker 1: of the papers we're gonna be looking at in this
Speaker 1: pair of episodes, and it is by Mark A. Limley
Speaker 1: and Brian Casey called Remedies for Robots, published in the
Speaker 1: University of Chicago Law Review in twenty nineteen. And this
Speaker 1: is a big paper. It's like eighty something pages long
Speaker 1: with a with a lot of different interesting, uh thoughts
Speaker 1: in it. We're not going to be able to cover
Speaker 1: the entire thing in depth, but it's worth looking up.
Speaker 1: You can easily find a full PDF of it if
Speaker 1: you want to read it in depth. And we're gonna
Speaker 1: look at some of the larger framework it lays out,
Speaker 1: and then some interesting thoughts raised to buy it, But
Speaker 1: to kick it off here the the author's right quote,
Speaker 1: what happens when artificially intelligent robots misbehave? The question is
Speaker 1: not just hypothetical. As robotics and artificial intelligence systems increasingly
Speaker 1: integrate into our society, they will do bad things. We
Speaker 1: seek to explore what remedies the all can and should
Speaker 1: provide once a robot has caused harm. Now, obviously we're
Speaker 1: going to be focused less on the like minute particulars
Speaker 1: of US legal precedent here and more on the broader
Speaker 1: issues they raise about robot agency, robot moral decision making
Speaker 1: and how that interacts with harm and morality and justice.
Speaker 1: And the authors start out in their introduction by giving
Speaker 1: what I think is a really fantastic example of how
Speaker 1: an autonomous robot with behaviors guided by machine learning, which
Speaker 1: is how you know increasingly most robots are going to
Speaker 1: be controlled, can end up doing things that are the
Speaker 1: exact opposite of what was intended. So this case that
Speaker 1: they cite is based on a true story from a
Speaker 1: presentation at the eleventh Annual Stanford e Commerce Best Practices
Speaker 1: Conference in June, and it goes like this quote. Engineers
Speaker 1: training and artificially intelligent self flying drone were perplexed. They
Speaker 1: were trying to get the drone to stay within a
Speaker 1: predefined circle and head toward its center. Things were going
Speaker 1: well for a while. The drone received positive reinforcement for
Speaker 1: its successful flights, and it was improving its ability to
Speaker 1: navigate toward the middle quickly and accurately. Then suddenly things changed.
Speaker 1: When the drone near the edge of the circle, it
Speaker 1: would inexplicably turn away from the center, leaving the circle.
Speaker 1: What went wrong. After a long time spent puzzling over
Speaker 1: the problem, the designers realized that whenever the drone left
Speaker 1: the circle during tests they had turned it off. Someone
Speaker 1: would then pick it up and carry it back into
Speaker 1: the circle to start again. From this pattern, the drones
Speaker 1: algorithm had learned correctly that when it was sufficiently far
Speaker 1: from the center, the optimal way to get back to
Speaker 1: the middle was to simply leave it all together. As
Speaker 1: far as the drone was concerned, it had discovered a
Speaker 1: worm hole. Somehow, flying outside of the circle could be
Speaker 1: led upon to magically teleport it closer to the center,
Speaker 1: and far from violating the rules instilled in it by
Speaker 1: its engineers, the drone had actually followed them to a t.
Speaker 1: In doing so, however, it had discovered an unforeseen shortcut,
Speaker 1: one that subverted its designers true intent. That's really good.
Speaker 1: That's that's it's as Yes, I love it. This is
Speaker 1: such a great example of how robots can fail in
Speaker 1: ways that are perfectly logical for the machines themselves, but
Speaker 1: hard for humans to predict in advance, because we're not
Speaker 1: understanding how our you know, our programming or the data
Speaker 1: sets were training it on is biasing its behavior in
Speaker 1: ways that that are strange to us. And in this case,
Speaker 1: of course, such a malfunction is harmless. But as autonomous
Speaker 1: machines become more and more integrated into the broader culture,
Speaker 1: not just in controlled contained locations like factory floors and laboratories,
Speaker 1: but in the wild, so on the streets and in
Speaker 1: our homes and stuff, there will inevitably be cases where
Speaker 1: robots fail like this and fail in ways that cause
Speaker 1: catastrophic harm to people. Yeah, and plus, as an aside,
Speaker 1: we we have to realize that even in cases where
Speaker 1: the machines have not failed, there will be gray areas
Speaker 1: in which it's not completely clear, and an argument could
Speaker 1: be made in these cases for machine culpability with a
Speaker 1: variety of intense and possible biases in place. Oh yeah,
Speaker 1: that's another thing these authors talk about that there can
Speaker 1: be all kinds of ways that that robotics and AI
Speaker 1: could end up causing extreme harm to people without ever
Speaker 1: doing anything that if a human did it would be illegal.
Speaker 1: What one example they give is like if, um, if
Speaker 1: Google were to suddenly change its Google Maps algorithm so
Speaker 1: that it routed all of the city's traffic through your neighborhood. Like,
Speaker 1: nothing illegal about that. It doesn't like commit a crime
Speaker 1: against you, but this is going to drastically negatively in
Speaker 1: hacked your quality of life. And it's a decision that's
Speaker 1: just like a could be a quirk of an algorithm
Speaker 1: in a machine. Now, this paper in particular concerns the
Speaker 1: legal concept of remedies. So I was reading about remedies.
Speaker 1: A common legal definition that I found is quote the
Speaker 1: means to achieve justice in any matter in which legal
Speaker 1: rights are involved. Or in the words of Limely and Casey,
Speaker 1: what do I get when I win? Right? So, if
Speaker 1: you if you take somebody to court because you say
Speaker 1: they have harmed you. Whatever outcome you're seeking from that
Speaker 1: that court case is the remedy. So usually when a
Speaker 1: court case finds that somebody has done something wrong to
Speaker 1: harm somebody else, the court responds to the finding of
Speaker 1: guilt or blame by enforcing this remedy. And common remedies
Speaker 1: would include a payment of money, right a guilty defendant
Speaker 1: has to pay money to the plaintiff, a punishment of
Speaker 1: the offender like maybe they go to jail, or a
Speaker 1: court order to do something or not to do something.
Speaker 1: For example, somebody is ordered not to drive a vehicle,
Speaker 1: or they are ordered not to go within a hundred
Speaker 1: feet of somebody else or something like that. Yeah, or
Speaker 1: their their eyeball is removed, or they have to spend
Speaker 1: a night in a haunted house something like that. Hopefully
Speaker 1: not in modern law. But wait a minute, there are
Speaker 1: some sometimes you do read about some really strange like
Speaker 1: remedies that are ordered by judges like I order you
Speaker 1: to I don't I don't know, to wear chicken suit
Speaker 1: or something right like, Yeah, there's some judges who like
Speaker 1: to get creative. It seems weird. Yeah, I wonder I
Speaker 1: have there have been in cases where someone has to
Speaker 1: spend a night in a haunted house due to a
Speaker 1: court order. I think that would be a good setup
Speaker 1: for a film. But anyway, so when you start looking
Speaker 1: at the idea of remedies, remedies are complicated because they
Speaker 1: involve different types of implied satisfaction on behalf of the
Speaker 1: victim or plaintiff. And some are very clear and material,
Speaker 1: and others are much more abstract. So the very the
Speaker 1: ones that are very clear and material are like, if
Speaker 1: I hit your car with my car and I'm clearly
Speaker 1: at fault, I need to give you a payment of
Speaker 1: cash to offset the material losses to the value of
Speaker 1: your car, right. But then other times it's it's more abstract.
Speaker 1: It's you know, punishment of an offender to give the
Speaker 1: victim a sense of justice or to allegedly discourage someone
Speaker 1: from committing this type of harm or offense in the future.
Speaker 1: And then the authors right that things get way more
Speaker 1: complicated when you bring robots and AI into the picture.
Speaker 1: For example, if you're trying to give a court order
Speaker 1: to a person you know saying like you shall not
Speaker 1: drive a car, you shall not you know, come within
Speaker 1: a hundred feet of this person. You can do so
Speaker 1: in natural language, you can like speak a sentence to
Speaker 1: them and you can expect them to understand. But how
Speaker 1: do you get a court to give an order to
Speaker 1: a robot not to do something? Most robots don't have
Speaker 1: natural language processing, and even if they do, a lot
Speaker 1: of times it's not that good. So you might think, okay,
Speaker 1: well you just you know, you give the court order
Speaker 1: to the robots programmer and then it and then they'll
Speaker 1: have to program the robot to obey. But this is
Speaker 1: also really complicated, like whose responsibility is it the robots
Speaker 1: current owner or the original contractor or creator who made
Speaker 1: the robot? Uh? And what if this is like an
Speaker 1: end user consumer device that the owner doesn't have any
Speaker 1: ability to reprogram, or what if, in the case of
Speaker 1: robots whose behavior is driven by machine learning or some
Speaker 1: other kind of system that is for practical purposes, a
Speaker 1: black box, what if it's not even clear how you
Speaker 1: could reprogram it to reliably obey the rule. Yeah, because
Speaker 1: there's a chance you got to this position because the
Speaker 1: robot misinterpreted what was asked of it. So if you
Speaker 1: then make additional requirements ones that maybe you know, haven't
Speaker 1: actually been tested before, but are just you know that
Speaker 1: that are then brought on by the court that could
Speaker 1: conceivably create new problems, right yeah, yeah, totally, and and
Speaker 1: it keeps getting even more comp located from there like
Speaker 1: Limle and Casey right quote. To complicate matters further, some systems,
Speaker 1: including many self driving cars, distribute responsibility for their robots
Speaker 1: between both designers and downstream operators. For systems of this kind,
Speaker 1: it has already proven extremely difficult to allocate responsibility when
Speaker 1: accidents inevitably occur. It just seems like a real, real
Speaker 1: fast way to get into skynet territory, where it's like
Speaker 1: the robot then decides that the only way to assure
Speaker 1: that it never sells cigarettes to children again is to
Speaker 1: destroy all humans. That sounds like finding a wormhole to me.
Speaker 1: We will be getting into some more wormhole territory as
Speaker 1: we go on, so more complications. Uh. The authors bring
Speaker 1: up the idea of how to courts compel a person
Speaker 1: or a company to obey a court order. Right Like,
Speaker 1: if you know a company is like dumping poison that's
Speaker 1: harming somebody, and the person sues that company, what does
Speaker 1: the court do to get them to stop, Well, the
Speaker 1: there is a threat of contempt of court if they
Speaker 1: don't stop doing it. Right, Courts usually just assume that
Speaker 1: people are motivated by a desire not to pay huge
Speaker 1: monetary damages or a desire not to go to jail.
Speaker 1: Would that have any motivating power on a robot. It
Speaker 1: would only have that power to the extent that the
Speaker 1: robot had been programmed to take that into account. If
Speaker 1: it hadn't, it wouldn't matter at all. Like, you know,
Speaker 1: most robots probably do not have any opinion one way
Speaker 1: or another about whether about going to jail or having
Speaker 1: to pay damages, So you'd have to explicitly program it
Speaker 1: to be disincentivized by potential punishments. Yeah, because take the
Speaker 1: cigarette robot for example, Like it's prime it's prime directive
Speaker 1: is just to sell delicious cigarettes to human beings like
Speaker 1: the the what what else? What kind of leverage do
Speaker 1: you have? Right exactly, So in that case, you'd be
Speaker 1: faced with either you'd be trying to find some kind
Speaker 1: of human who's responsible for its behavior, but you could
Speaker 1: very well run into the problem that like you can't
Speaker 1: really identify any one person who's seems to be at
Speaker 1: fault for what it did, and it's doing this bad thing,
Speaker 1: so so what are you going to do about it?
Speaker 1: And then of course things get even weirder when you
Speaker 1: start getting into that that other side. You know, that's
Speaker 1: like the more like direct and material remedies that can
Speaker 1: be provided by courts, either like a monetary award to
Speaker 1: the victim or in order to stop doing something that
Speaker 1: causes harm. On the other hand, you've got this thing
Speaker 1: that courts often end up engaging in, and people are
Speaker 1: are largely driven and motivated by however, however irrational it
Speaker 1: might be in some cases, and that's the perceived abstract
Speaker 1: value of punishment, you know, not just material damages to
Speaker 1: a victim or in order not to do something, but
Speaker 1: the inflicting of punishments specifically to demonstrate the court's displeasure
Speaker 1: with the original behavior of the defendant. Uh so uh
Speaker 1: They raise a question that's brought up in a paper
Speaker 1: by a professor named Christina Mulligan who explores the subject
Speaker 1: of should you have the right to punch a robot
Speaker 1: that hurts you? Uh lee? In case? He called the
Speaker 1: called this the expressive component of remedies, and though a
Speaker 1: desire to see offenders punished. Maybe an extremely natural and
Speaker 1: nearly universal human drive. It's debatable whether it actually serves
Speaker 1: a purpose in reducing harm, and if it does, in
Speaker 1: what cases it does. I I love this idea because,
Speaker 1: on a very literal level, it makes me think, well,
Speaker 1: why would you punch a robot. They're made out of
Speaker 1: out of metal. You're gonna hurt your hand. All you're
Speaker 1: gonna do is hurt your hand, and you're not going
Speaker 1: to hurt the robot unless first of all, you design
Speaker 1: the robot so that it has at least one punchable
Speaker 1: portion of its anatomy, and then for it to be
Speaker 1: more than just a you know, a cathartic uh uh
Speaker 1: thing for you, then you have to also make sure
Speaker 1: there's some sort of feedback right where punch. Yeah, like
Speaker 1: you punch cigarette bought in it's punchable area, then it
Speaker 1: will say owl, and maybe it will I don't know,
Speaker 1: ottaw inciner rate one packet of reigrettes so that it
Speaker 1: can never sell them that sort of thing. But then, yeah,
Speaker 1: you're having to design your robots to to suffer to
Speaker 1: a certain extent, which I guess that means that goes
Speaker 1: back to what C three po said, right, he said,
Speaker 1: you know about about being made to suffer. It seems
Speaker 1: to be our lot in life. Oh, that's interesting. I
Speaker 1: hadn't thought about that. Yeah. Clearly R two, D two
Speaker 1: and C three p O have have inherent desires to
Speaker 1: avoid pain. They have been programmed with that. Yeah, but
Speaker 1: as we've said that, that's not standard issue for robots.
Speaker 1: Most robots don't care about whether or not they get injured, Like,
Speaker 1: that's not a motivating factor for them. And again it
Speaker 1: raises this bizarre question of like, what are you doing
Speaker 1: when you punch the robot? Like, what is the I
Speaker 1: guess it's making you feel better, But does it make
Speaker 1: you feel better if like you know that the robot
Speaker 1: doesn't actually care? Yeah, and then what then what needs
Speaker 1: to be done to convince you that it does? K Yeah,
Speaker 1: it just gets very sticky, very quickly. And then of
Speaker 1: course turns the mirror back on the way we handle
Speaker 1: human to humans in areas. Right, but anyway, limly in
Speaker 1: casey I guess to to summarize their position, they say, Okay,
Speaker 1: increasingly independent robots and AI are coming. They're they're infiltrating
Speaker 1: more and more into society, and they will inevitably do
Speaker 1: bad things. When that happens, the legal system will try
Speaker 1: to order remedies to make things right in you know,
Speaker 1: when when harm has been caused. Our current legal understanding
Speaker 1: of remedies is based on the assumption of human agents
Speaker 1: and human agents only, and its rules are not suited
Speaker 1: to dealing with robot crime or robot offenses quote. As
Speaker 1: we have shown, failing to recognize those differences could result
Speaker 1: in significant unintended consequences, inadvertently encouraging the wrong behaviors or
Speaker 1: even rendering our most important remedial mechanisms functionally irrelevant. Uh So,
Speaker 1: to take robot agents into account, we're going to have
Speaker 1: to examine and rethink how our systems of remedies work.
Speaker 1: But and this is a point we've been making all ready,
Speaker 1: this could have multiple benefits because it could also lead
Speaker 1: to a better understanding of how we apply these remedies
Speaker 1: to cases dealing exclusively with humans quote. Indeed, one of
Speaker 1: the most pressing challenges raised by the technology is its
Speaker 1: tendency to reveal the trade offs between sidal economic and
Speaker 1: legal values that many of us today make without deeply
Speaker 1: appreciating the downstream consequences. They write, we need a law
Speaker 1: of remedies for robots, but in the final analysis, remedies
Speaker 1: for robots may also end up being remedies for all
Speaker 1: of us. Now, like I said, this is a very
Speaker 1: long paper. We can't do justice to all of the
Speaker 1: subjects they raise, but to focus on some highlights, I
Speaker 1: thought one interesting place to look was when they try
Speaker 1: to get into the definition of what actually makes a
Speaker 1: robot in the legal sense. Obviously, there's going to be
Speaker 1: some difficulty here because think about how differently the term
Speaker 1: is used and how many different things it's applied to
Speaker 1: in the world. Uh. The authors here is cite a
Speaker 1: professor Ryan Hallo, who in the past had written that
Speaker 1: there are three important characteristics that define a robot and
Speaker 1: make it different from any machine, just like a computer
Speaker 1: or phone. And Callo says that these three uh, these
Speaker 1: three qualities are embodiment, emergence, and social valence. So to
Speaker 1: quote from Calo, robotics combines, arguably for the first time,
Speaker 1: the promiscuity of information with the embodied capacity to do
Speaker 1: physical harm. Robots display increasingly emergent behavior, permitting the technology
Speaker 1: to accomplish both useful and unfortunate tasks in unexpected ways.
Speaker 1: I like that idea of unfortunate tasks. Um and robots,
Speaker 1: more so than any technology and history, feel to us
Speaker 1: like social actors, a tendency so strong that soldiers sometimes
Speaker 1: jeopardize themselves to preserve the lives of military robots in
Speaker 1: the field, and lives is in quotes there. Yeah, you
Speaker 1: may remember this from the film it came out a
Speaker 1: few years back, Saving Private Cigarette Robot. It's quite touching.
Speaker 1: I mean, it seems absurd, but it does seem to
Speaker 1: play on our natural biases. I want to talk about
Speaker 1: a couple of examples from a psychology paper in a second,
Speaker 1: but like, uh, we're we're just so ready to look
Speaker 1: at machines like humans and and treat them as such.
Speaker 1: It seems almost impossible to avoid. But anyway to pick
Speaker 1: up with Limly and Casey after after that callo quote
Speaker 1: they say quote. In light of these qualities, Calo argues
Speaker 1: that robots are best thought of as artificial objects or
Speaker 1: systems that sense, process, and act upon the world to
Speaker 1: at least some degree. Thus, a robot in the strongest,
Speaker 1: fullest sense of the term, exists in the world as
Speaker 1: a corporeal object with the capacity to exert itself physically. Though,
Speaker 1: It's interesting to me that even this attempt to give
Speaker 1: a strict and legally useful definition of a robot includes
Speaker 1: a subjective component. I brought this up earlier, the component
Speaker 1: about human feelings, the social valence criterion. The callos sites here.
Speaker 1: This means they feel to us like social actors. Yeah,
Speaker 1: Like I was wondering in all of this, like, where
Speaker 1: does a particularly malicious robo call fit into the scenario? Say,
Speaker 1: a robo call that is not just about trying to
Speaker 1: sell you something, but it's like, you know, actively trying
Speaker 1: to say, get a credit card number out of you
Speaker 1: for nefarious purposes. Yeah, that's a really good point. And
Speaker 1: and along those lines, Limbly and Casey argue that actually,
Speaker 1: they don't think the embodiment criteria of hardware is necessarily
Speaker 1: a good one. That maybe our concept of a robot
Speaker 1: should be less limited to the essentialist quality of being
Speaker 1: embodied and more just apply to anything that exhibits intelligent behavior.
Speaker 1: And exactly things like that robot call would be would
Speaker 1: be a good example. Uh, the things we think of
Speaker 1: as robots probably do. They They're not just like stand
Speaker 1: alone objects. They interact with the broader world in some way,
Speaker 1: but they could be entirely software based. Yeah, I guess certainly.
Speaker 1: The roomba is a great example, or any kind of
Speaker 1: like vacuuming robot where it's it's it's it's in your house,
Speaker 1: or it's in a room in your house. It's it's
Speaker 1: interacting in your environment and it's essentially making decisions about
Speaker 1: how best to move around that space. Sure, but if
Speaker 1: you want to take it out of the embodied space,
Speaker 1: you could have the idea of bots on the Internet.
Speaker 1: There's things out there acting autonomously to some extent and doing,
Speaker 1: you know, executing some behavior, acting almost maliciously. But we
Speaker 1: we are tempted to call them bots meaning short for robots,
Speaker 1: because they have some kind of apparent independent agency and
Speaker 1: they're doing something that seems at least halfway intelligent. Right, Yeah,
Speaker 1: And you can easily imagine how they could they are
Speaker 1: and well they I mean, they are used maliciously in
Speaker 1: some cases, but how something like a social media bot
Speaker 1: that responds to certain comments in a particular way, Like,
Speaker 1: it's very easy to imagine how you how that could
Speaker 1: be utilized in a way that would be like not
Speaker 1: only annoying, but just that outright harmful, even physically harmful.
Speaker 1: Oh yeah, I mean think about some of these, uh say,
Speaker 1: like bots on social media that try to crowdsource information,
Speaker 1: like during a natural disaster or something like that. You
Speaker 1: could imagine uh, intentionally maliciously manipulating a bot of this
Speaker 1: kind to like have you know, bad information on it
Speaker 1: or something. Yeah, yeah, or you know, anything that a
Speaker 1: troll can do on social media, a bought could conceivably
Speaker 1: do as well. So that just you know, opens up
Speaker 1: the door, right. But coming back to this, so there's
Speaker 1: this interesting idea that robots feel to us like social actors,
Speaker 1: and that seems to be, at least by some people's definitions,
Speaker 1: a kind of inextricable quality of what makes a robot
Speaker 1: like it feels like, at least to a small extent,
Speaker 1: like a person somehow. And it reminds me of the
Speaker 1: psychology paper I was looking at just recently on human
Speaker 1: social interaction with robots that is by Elizabeth Broadbent called
Speaker 1: Interactions with Robots The Truths We Reveal about Ourselves, published
Speaker 1: in the Annual Review of Psychology in seventeen. Uh. This
Speaker 1: was a highly cited paper and it seems to be
Speaker 1: it's a big literature review of a lot of different
Speaker 1: stuff about about how humans interact emotionally and socially with robots,
Speaker 1: And the one section I was thinking about was where
Speaker 1: she reviews a bunch of other studies about how we
Speaker 1: mindlessly apply social rules to robots. So there are a
Speaker 1: ton of different examples, but just to cite a couple
Speaker 1: of them, one she writes up his quote. After using
Speaker 1: a computer, people evaluate its performance more highly if the
Speaker 1: same computer delivers the rating scale then if another computer
Speaker 1: delivers the rating scale, or if they rate it with
Speaker 1: pen and paper. So like, if you know, you get
Speaker 1: a thing at the end of a test that says like, hey,
Speaker 1: you know, how did you enjoy interacting with this machine,
Speaker 1: You're more likely to give it a higher score if
Speaker 1: you're still sitting on the same machine. Or at least
Speaker 1: that was what was found by nas at all in
Speaker 1: nine UH and the and Broadbent rights quote. This result
Speaker 1: is similar to experiment or bias, in which people try
Speaker 1: not to offend a human researcher. Another example of social
Speaker 1: behavior is reciprocity. We help others who help us. People
Speaker 1: helped to computer with a task for more time and
Speaker 1: more accurately if the computer first helped them with a task,
Speaker 1: then if it did not, and this was found by
Speaker 1: Fog and nas In. I love that idea of people,
Speaker 1: you know, being more reluctant to rate a computer, uh
Speaker 1: poorly if they're still interacting with the same computer. That
Speaker 1: that's that seems perfectly true to me. But another interesting
Speaker 1: one from the summary is quote research and psychology has
Speaker 1: shown that the presence of an observer can increase people's honesty,
Speaker 1: but incentives for cheating can reduce honesty, and this is
Speaker 1: found by Covey at all in nineteen eighty nine. In
Speaker 1: a robot version of this work, participants given incentives to
Speaker 1: cheat were shown to be less honest when alone compared
Speaker 1: to when they were accompanied by either a human or
Speaker 1: by a simple robot, and that was found by Hoffman
Speaker 1: at all in. This illustrates that the social presence of
Speaker 1: robots may make people feel as though they're being watched
Speaker 1: and increase their honesty in an effect similar to that
Speaker 1: produced by the presence of humans. Now this is inter right.
Speaker 1: This This also reminds me of various studies that have
Speaker 1: gone into sort of the idea of of imagine beings
Speaker 1: or religious beings watching us while we're doing things right,
Speaker 1: or even just like I imagery, like putting some eyes
Speaker 1: imagery on a wall looking at people while they're like,
Speaker 1: I don't know, not supposed to steal from the collection
Speaker 1: plate or something like that. I don't know if it's
Speaker 1: to the same extent, but at least in the same
Speaker 1: direction that the presence of another human is. You know,
Speaker 1: your You might be a little bit worried that are
Speaker 1: who D two is gonna, you know, judge your moral
Speaker 1: character harshly or tattle on you. I'm not as worried
Speaker 1: about our two, but um three po snitch than Coming
Speaker 1: back to Limele and Casey, so they talked for a
Speaker 1: long time about how robots get their intelligence. They talked
Speaker 1: about the importance of machine learning for the modern generations
Speaker 1: of robots and AI, that it's just not practical to
Speaker 1: hard code AI the way we used to imagine. You know,
Speaker 1: you'd be a programmer and you're just like creating a
Speaker 1: lot of strings of if then statements like you know,
Speaker 1: the kind of intelligence that we expect from a modern
Speaker 1: AI or or intelligent robot is too complex for people
Speaker 1: to program in a in a direct way like that. Instead,
Speaker 1: they've got to be trained on natural data sets through
Speaker 1: machine learning, but of course doing so comes at the
Speaker 1: cost of increasing uncertainty about their future behaviors. Behaviors could
Speaker 1: emerge that a conchi and just programmer would never intentionally
Speaker 1: hard code into the system. Uh So, so that brings
Speaker 1: us to like, what types of harms could we expect
Speaker 1: from robots and AI? And the authors here come up
Speaker 1: with what I think are some very useful categories, some
Speaker 1: sort of like cubby holes to slot the different types
Speaker 1: of AI fears into. So the first kind is what
Speaker 1: they call unavoidable harms. These are probably not the main
Speaker 1: ones to be worried about, but they are worth thinking about.
Speaker 1: Uh And this is just the fact that some dangers
Speaker 1: are inherent too many products and services, we just accept
Speaker 1: them as the cost of having those products and services
Speaker 1: in the first place. So like this would just be
Speaker 1: cigarette bought just by virtue of selling cigarettes is doing
Speaker 1: harm to people? Right? Yes, I mean, yeah, the fact
Speaker 1: that you have cigarettes. There is some harm coming from that.
Speaker 1: But there are also ones that are more fully integrated
Speaker 1: into just the way society works, like having cars. It
Speaker 1: is absolutely inevitable that people driving cars are going to
Speaker 1: crash their cars and there will be FATA lalities from that,
Speaker 1: and you can think of ways of reducing it, but
Speaker 1: there's there's really not any expectation that we can have
Speaker 1: a country that has car based transportation and there will
Speaker 1: not be any accidents because there will always be things
Speaker 1: that are that are not even reducible to driver error
Speaker 1: or to malfunction of the cars, right, like a tree
Speaker 1: falls on the road or something, right, birds, wild animals.
Speaker 1: Any So, even though they're I think there's some very
Speaker 1: convincing arguments to be made that uh a switch to
Speaker 1: self driving cars would create a much safer uh travel environment,
Speaker 1: that it would make roads safer. You're not gonna you're
Speaker 1: not gonna get to absolute zero crashes or absolute zero
Speaker 1: road fatalities, right, I mean you wouldn't even if the
Speaker 1: driving algorithms were perfect, right, and they're probably not going
Speaker 1: to be perfect. They may well be and probably are
Speaker 1: going to be better than the average human driver. Yeah, Okay,
Speaker 1: So that's just there's just unavoidable harm that comes from
Speaker 1: using any type of product or service, and when you
Speaker 1: integrate robotics in AI and of that product or service,
Speaker 1: those unavoidable harms will just continue. But that's something we
Speaker 1: already deal with. The next category is deliberate least cost harms.
Speaker 1: This is similar to unavoidable harms, but it's in cases
Speaker 1: where the machine actually is able to make a decision
Speaker 1: with with important ramifications, like it can make a decision
Speaker 1: to act in a way that causes harm, but is
Speaker 1: attempting to cause the least harm possible. So, in a sense,
Speaker 1: this is forcing robots to do the trolley problem. Right,
Speaker 1: do you switch to the track that has one person
Speaker 1: sitting on the train tracks instead of five people? And
Speaker 1: this will be another inevitable capability of autonomous cars, but
Speaker 1: it raises all kinds of thorny questions. If an autonomous
Speaker 1: vehicle can avoid a head on collision that will likely
Speaker 1: kill multiple people by suddenly swerving out of the way
Speaker 1: and hitting one pedestrian, that may indeed avoid a greater harm,
Speaker 1: but that's probably cold comfort to the one person who
Speaker 1: got hit. Right, Yeah, and then when you you have
Speaker 1: a robot or some sort of an AI involved in
Speaker 1: that decision making, I mean, it's it's you can just
Speaker 1: imagine the the the intensity of the arguments and the
Speaker 1: conversations that would ensue. Right, But then the authors raised
Speaker 1: what I think is a very interesting point. They say
Speaker 1: that this kind of life or death trolley problem will
Speaker 1: probably be the exception rather than the rule. Instead, they say, quote, uh,
Speaker 1: far likelier, I'll build albeit subtler scenarios involving least cost
Speaker 1: harms will involve robots that make decisions with seemingly trivial
Speaker 1: implications at an individual level, but which result in non
Speaker 1: trivial impacts at scale. Self driving cars, for example, will
Speaker 1: rarely face a stark choice between killing a child or
Speaker 1: killing two elderly people, but thousands of times a day
Speaker 1: they will have to choose precisely where to change lanes,
Speaker 1: how closely to trail another vehicle, when to accelerate on
Speaker 1: a freeway on ramp, and so forth. Each of these
Speaker 1: decisions will entail some probability of injuring someone, I guess.
Speaker 1: Another thing to keep in mind, like with the trap,
Speaker 1: with the trolley problem, generally, when you're dealing with it,
Speaker 1: there's a there's a lot of emphasis on the problem
Speaker 1: aspect of it, you know, like the trolley problem should
Speaker 1: be a an ethical dilemma. It should it should hurt
Speaker 1: a bit to try and figure out how of what
Speaker 1: to do, and the idea of the trolley problem being
Speaker 1: something that is encountered and decided upon, like as in
Speaker 1: a split second, by a machine, um, by an algorithm
Speaker 1: like that. That feels that that feels a bit worse
Speaker 1: to us. You know, that feels like if if it's
Speaker 1: if it's an easy decision, even if it's just based
Speaker 1: purely on math, you know, it's um, it feels wrong
Speaker 1: on some level. Oh yeah, yeah, so I think you're right.
Speaker 1: But also the thing they're bringing up here is that
Speaker 1: the trolley problem you're actually more often facing is that
Speaker 1: every single day you're your autonomous car is going to
Speaker 1: make you know, hundreds or thousands of trolley problem calls
Speaker 1: where on one track it is getting to your destination
Speaker 1: a few seconds faster, and on the other track is
Speaker 1: a one in a million chance of killing somebody. Yeah yeah,
Speaker 1: and these do. We make these decisions all the time,
Speaker 1: but we don't focus on these. That's but I think
Speaker 1: that's part of the issue, you know, Yeah, exactly, You're like,
Speaker 1: should I, okay, should I take a left on this road? Well,
Speaker 1: there's a chance there's a speeding car just above, just
Speaker 1: over the edge there, and I can't see it, but
Speaker 1: I'm going to take that chance because I want to
Speaker 1: cut three minutes off my drive to work. Yes. Uh,
Speaker 1: this is actually a very good point that we we
Speaker 1: already make these decisions, but we just don't think about
Speaker 1: them in these explicit probability calculations, and there may be
Speaker 1: some consequences to thinking about them this way, which is
Speaker 1: there could be a weird like perceived downside just to
Speaker 1: making these kind of these kind of calculations objective and
Speaker 1: and explicit. Yeah, I mean I've I've run into this
Speaker 1: with some of the map programs that I used to
Speaker 1: drive before, where I want to tell it in some
Speaker 1: cases like like give me the ability and maybe they
Speaker 1: have this now, but there was one left turn in
Speaker 1: particular where I would the ability to to flag this
Speaker 1: left turn. This is a dangerous left turn. You have
Speaker 1: put me in a position to make. Um, I might
Speaker 1: know which left turn you're talking about. If you probably
Speaker 1: in town, it's it's it's in town. It's near our office.
Speaker 1: So yeah, I mean yeah, by the way, if you're
Speaker 1: out there working on programming driving apps, you should absolutely
Speaker 1: include the A toggle key where you can say no
Speaker 1: left turns please. Yes, that that is highly useful. I'm
Speaker 1: to understand this is how one of my aunts got around. Like,
Speaker 1: if they got older and they were less adventurous driving,
Speaker 1: they would only take right turns, and they would do
Speaker 1: all their driving so that no left turns were made.
Speaker 1: I think I have one time read This could be
Speaker 1: totally wrong, but I at least one time I remember
Speaker 1: reading a claim that, like, you know, the traffic efficiency
Speaker 1: would be x percent higher and people would spend x
Speaker 1: number of minutes less time in traffic if there were
Speaker 1: no such thing as left turns, if everybody had to
Speaker 1: get everywhere by only doing you know, full right turns
Speaker 1: to to go around the block. Interesting. I'm sure there
Speaker 1: would be some cases where you can't do that, but
Speaker 1: you know, in a in a grid city, seems to
Speaker 1: make a lot of sense. Maybe you get like one
Speaker 1: left turn of day. It's some sort of card system.
Speaker 1: But like I said, I I cannot confirm that. Okay,
Speaker 1: But anyway, the next categories of harm they talk about
Speaker 1: this one is defect driven harms. Uh, this one is
Speaker 1: very easy to understand. The robot harms someone because of
Speaker 1: a design flaw or a bug or a mistake, or
Speaker 1: it's just broken. You know, a warehouse loading robot is
Speaker 1: designed to only operate when no humans are nearby it.
Speaker 1: But there's a malfunction with one of its sensors and
Speaker 1: it fails to detect the presence of a human operator
Speaker 1: trying to get I don't know, a piece of junk
Speaker 1: out of it out of one of its hinges, and
Speaker 1: it moves and kills them. Okay, this is pretty straightforward.
Speaker 1: Just it's broken for some reason. The authors here do
Speaker 1: point out that this gets even more complicated when there
Speaker 1: is a human in the loop e g. An autonomous
Speaker 1: car with a human driver who is supposed to intervene
Speaker 1: in the event of an emergency. They talked about one
Speaker 1: case where this happened with with I believe it was
Speaker 1: an uber autonomous vehicle where both the machine and the
Speaker 1: human fail, that both of them failed to stop a
Speaker 1: collision that hurts someone, Like what happens here? Yeah, yeah,
Speaker 1: of course, we we have very similar cases in just
Speaker 1: purely human affairs. Right when questions are asked, like where
Speaker 1: was this person's supervisor? Uh, you know, who who were
Speaker 1: the watchers? Who? There should have been some other person,
Speaker 1: there was someone else in the loop here, Why didn't
Speaker 1: they do something to stop this crime from taking place? Right? Okay?
Speaker 1: After that you get into misuse harms. Now, some of
Speaker 1: these are very obvious, very straightforward, like if you program
Speaker 1: a robot directly to go kill someone, or even if
Speaker 1: you program it to wander around at random swinging a machete.
Speaker 1: In these cases, it seems that the human programmer is
Speaker 1: clearly at fault, Right, the robot has just become a
Speaker 1: weapon of murder or of reckless endangerment, and the person
Speaker 1: who told it to do that is the person responsible. Yeah.
Speaker 1: Like if you take an automotive, uh, like oil changed
Speaker 1: robot and you reprogram it to um do appendectomies and
Speaker 1: people die as a result, Like, that's a misuse you
Speaker 1: can you can only blame the the oil changed robots
Speaker 1: so much because it was not ultimately designed to perform appendectomies. Right.
Speaker 1: In this case, this is more like the hammer example
Speaker 1: used at the beginning. This it's not the robot autonomously
Speaker 1: making the decision to do this. Uh, this is somebody
Speaker 1: just using it as a tool of crime. Yeah, But
Speaker 1: the authors point out that there are cases where quote,
Speaker 1: people will misuse robots in a manner that is neither
Speaker 1: negligent nor criminal, but nevertheless threatens to harm others, and
Speaker 1: these types of harm are especially difficult to predict and prevent.
Speaker 1: So one example is just people love to trick robots,
Speaker 1: people like to mess around with robots and AI. I
Speaker 1: would admit to myself finding this amusing and principle we've
Speaker 1: talked about this in uh, you know, the flate of
Speaker 1: sex Mokena episodes. But of course there are times when
Speaker 1: it's not so funny, when when people take it to
Speaker 1: really sinister places. One example the authors bring up here
Speaker 1: is the horrible saga of Microsoft ta Do you remember
Speaker 1: this thing? Oh? This was the this is the robot
Speaker 1: that was traveling across the country. No, no, no, no,
Speaker 1: uh though I know what you're talking about there. No,
Speaker 1: maybe we can come back to that. But Tay was
Speaker 1: a Twitter chat bot created by Microsoft that was supposed
Speaker 1: to learn how to interact on the Internet just by
Speaker 1: learning from conversations it had with real users. So you
Speaker 1: could tweet it Tay and say hey, how are you doing?
Speaker 1: You know, and you could talk about the weather or whatever.
Speaker 1: But of course who who ended up engaging and training
Speaker 1: this AI to speak? It was like the worst trolls
Speaker 1: on the internet. So within a matter of hours, this
Speaker 1: brand new chat bot had been transformed from from a
Speaker 1: from a you know, a lump of clay unformed into
Speaker 1: a pornographic nazi. Yes, I do remember this now. And
Speaker 1: this kind of just gets you thinking about the ways
Speaker 1: that people will be will be able to misuse robots
Speaker 1: in ways that guide their behavior in extremely pernicious directions,
Speaker 1: sometimes without the people guiding this misuse necessarily committing any
Speaker 1: kind of identifiable crime. Yeah, Like people are going to
Speaker 1: look for exploits, They're gonna look for ways, they're gonna
Speaker 1: look for cracks in the system. It's it's like within
Speaker 1: with any kind of like a video game system. You know,
Speaker 1: people are just gonna see what they can get away
Speaker 1: with and and just engage in that kind of action,
Speaker 1: sometimes just for the fun of it, right, And sometimes
Speaker 1: that's harmless, but sometimes that's really awful. Yeah, Okay. Next
Speaker 1: category is unforeseen harms. And here's where we start getting
Speaker 1: into the really really interesting and really difficult cases, types
Speaker 1: of harm that are not unavoidable, not a product of
Speaker 1: defects or miss use, but are still not predicted by creators. Uh.
Speaker 1: And so the authors talk about how, in a way,
Speaker 1: unpredictability is what makes AI potentially useful, right like, it
Speaker 1: can potentially arrive at solutions that humans wouldn't have predicted.
Speaker 1: But sometimes it does so in ways that really miss
Speaker 1: the boat and could be extremely harmful if they were
Speaker 1: embodied in action in the real world. Uh, similar to
Speaker 1: the drone example from the circle that we talked about
Speaker 1: at the beginning. But they signed another fantastic example here.
Speaker 1: That's kind of chilling. So I'm just gonna read from
Speaker 1: Limle and Casey here. In the ninety nineties, a pioneering
Speaker 1: multi institutional study sought to use machine learning techniques to
Speaker 1: predict health related risks prior to hospitalization. After ingesting an
Speaker 1: enormous quantity of data covering patients with pneumonia, the system
Speaker 1: learned the rule has asthma X delivers lower risk X.
Speaker 1: The colloquial translation is patients with pneumonia who have a
Speaker 1: history of asthma have a lower risk of dying from
Speaker 1: pneumonia than the general population. The machine derived rule was curious,
Speaker 1: to say the least, far from being protective, asthma can
Speaker 1: seriously complicate pulmonary illnesses, including pneumonia. Perplexed by this counterintuitive result,
Speaker 1: the researchers dug deeper, and what they found was troubling.
Speaker 1: They discovered that quote, patients with the history of asthma
Speaker 1: who presented with pneumonia usually were admitted not only to
Speaker 1: the hospital but directly to the i c U, the
Speaker 1: intensive care unit. Once in the i c U, asthmatic
Speaker 1: pneumonia patients went on to receive more aggressive care, thereby
Speaker 1: raising their survival rates compared to the general population. The rule,
Speaker 1: in other words, reflected a genuine pattern in the data,
Speaker 1: but the machine had confused correlation with causation quote, incorrectly
Speaker 1: learning that asthma lowers risk when in fact, asthmatics have
Speaker 1: much high fire risk. It seems like we've got another
Speaker 1: wormhole here. And here the authors introduce an idea of
Speaker 1: of a curve of outcomes that they call a leptokurtic curve.
Speaker 1: That's a strange term, but basically what that means is
Speaker 1: if you are um, if you're charting what types of
Speaker 1: outcomes you expect from a traditional system like just you know,
Speaker 1: humans looking at data versus a a complex automated system,
Speaker 1: the sort of the tails of the graph with the
Speaker 1: complex automated system will tend to be fatter, meaning you
Speaker 1: get more extreme events in the positive and negative space,
Speaker 1: rather than a you know, a sort of rounder clustering
Speaker 1: of events in the you know, normal operation space, if
Speaker 1: that makes any sense. So, these kinds of unforeseen harms
Speaker 1: are some of the most worrisome types of things to
Speaker 1: expect coming out of robots and AI. But then the
Speaker 1: other one would be systemic harms And this is the
Speaker 1: last category of of harms they talk about. UH the
Speaker 1: author's right quote. People have long assumed that robots are
Speaker 1: inherently neutral and objective, given that robots simply intake data
Speaker 1: and systematically output results. But they are actually neither. Robots
Speaker 1: are only as neutral as the data they're fed, and
Speaker 1: only as objective as the design choices of those who
Speaker 1: create them. When either bias or subjectivity infiltrates the systems
Speaker 1: inputs or design choices, it is inevitably reflected in the
Speaker 1: system's outputs. This is your classic garbage in, garbage out, problem, right,
Speaker 1: They go on. Accordingly, those responsible for overseeing the deployment
Speaker 1: of robots must anticipate the possibility that algorithmically biased applications
Speaker 1: will cause harms of this systemic nature to third parties.
Speaker 1: So UH, an example that's much discussed in this would
Speaker 1: be an AI trained to make decisions about granting loans
Speaker 1: by studying patterns of which loan applicants got their loans
Speaker 1: granted in the past. And a I like this could
Speaker 1: end up manifesting some type of bias that hurts people,
Speaker 1: like a racial bias in its loan assessments, because there
Speaker 1: was already a bias in the real world data set
Speaker 1: that it was trained on. So, in other words, AI
Speaker 1: that is trained on data from the real world, unless
Speaker 1: it is it is explicitly told not to do this,
Speaker 1: it will tend to reproduce and perpetuate any injustices, any
Speaker 1: inequalities that already exist. And the authors here give an
Speaker 1: example that is based on algorithmically derived insurance premiums that
Speaker 1: I think they're talking about auto insurance quote. A recent
Speaker 1: study by Consumer Reports found that contemporary premiums depended less
Speaker 1: on driving habits and increasingly on socioeconomic factors, including an
Speaker 1: individual's credit score. After analyzing two billion car insurance price
Speaker 1: quotes across approximately seven hundred companies, the study found that
Speaker 1: credit scores actored into insurance algorithms so heavily that perfect
Speaker 1: drivers with low credit scores often paid substantially more than
Speaker 1: terrible drivers with high scores. The studies findings raised widespread
Speaker 1: concerns that AI systems used to generate these quotes could
Speaker 1: create negative feedback loops that are hard to break. According
Speaker 1: to one expert, quote, higher insurance prices for low income
Speaker 1: people can translate to higher debt and plummeting credit scores,
Speaker 1: which can mean reduce job prospects, which allows debt to
Speaker 1: pile up, credit scores to sink lower, and insurance rates
Speaker 1: to increase in a vicious cycle. Uh So this is
Speaker 1: kind of a nightmare scenario, right, Like an AI that
Speaker 1: is too powerful and not explicitly protected against acquiring these
Speaker 1: types of biases could create these kind of computer enforced
Speaker 1: prisons in reality, like a machine code for perpetuating whatever
Speaker 1: state of the world, like whatever state the world was
Speaker 1: in when the AI was first deployed, and then entrenching
Speaker 1: it further and further. Yeah, and that kind of thing
Speaker 1: is especially scary because, like, if there's a human making
Speaker 1: the decision, you can you can call up the human
Speaker 1: to a witness stand or ask them like, hey, why
Speaker 1: did you make the decision this way? But if it's
Speaker 1: an AI doing it, you could say like, hey, why
Speaker 1: why is it? Why are we getting this outcome that's
Speaker 1: you know, creating a sort of like cyclical prison out
Speaker 1: of reality, And they can just say, hey, you know
Speaker 1: it's the machine with the machine. You know it knows
Speaker 1: what it's doing. Yeah, and then yes, the machine it
Speaker 1: says I learned it from watching you dad, and you
Speaker 1: have that moment of shame. So I think these different
Speaker 1: categories that they that they bring up are really important
Speaker 1: for helping us kind of sort our ideas into into
Speaker 1: recognizable types for for ways that AI and robots could
Speaker 1: go wrong and could potentially cause harm that you would
Speaker 1: seek legal remedy for. And also they help identify the
Speaker 1: spaces that there's the most worry. I mean, for me,
Speaker 1: I think that would be like those last two cases, right,
Speaker 1: the unforeseen problems and the systemic problems are the ones
Speaker 1: where there's the most real danger I think and the
Speaker 1: most difficulty in trying to figure out how to solve it. Yeah,
Speaker 1: because we we kind of you know, train ourselves for
Speaker 1: to a certain extent and sort of culturally focus on
Speaker 1: the sky net problems, right, the really obvious um situations
Speaker 1: where the robot car veers off the road in a
Speaker 1: dangerous way. But the situations where it is just perpetuating
Speaker 1: what we're already doing, where it's making choices in getting
Speaker 1: from point A to point B that don't violate anything
Speaker 1: we told it, but just are an uninventive and even
Speaker 1: harmful way of doing it. Uh. Yeah, that's that's that's
Speaker 1: harder to deal with. That's a type of misbehavior that
Speaker 1: you can't solve by just having Dan o'harla Hay stand
Speaker 1: up and bellowing behave yourselves exactly. Um yeah, yeah, yeah,
Speaker 1: I mean I can't remember if that even worked. I
Speaker 1: just remember that was one of my favorite moments, and
Speaker 1: that was RoboCop two, right, was it? Yeah, well, I
Speaker 1: mean RoboCop one. I think we're already dealing with this
Speaker 1: problem of like the sort of like weird dynamics of
Speaker 1: machine culpability. When Ed two oh nine like shoots that
Speaker 1: guy five hundred times in the boardroom during the demonstration,
Speaker 1: and then Dan O'Hurley he's response to it is to
Speaker 1: turn to Ronnie Cox and say, I'm very disappointed, Dick.
Speaker 1: But anyway, well, I guess we're running running kind of long,
Speaker 1: so maybe we should call part one there, But we
Speaker 1: will resume this discussion about about robot justice and robot
Speaker 1: punishment in the next episode. That's right, we'll be back
Speaker 1: with more of this discussion in the meantime. If you
Speaker 1: would like to check out past episodes of Stuff to
Speaker 1: Blow Your Mind Uh, and they're definitely worth checking out
Speaker 1: because we have lots of past episodes that deal with
Speaker 1: robots and AI. We have lots of episodes where we
Speaker 1: make RoboCop references, so they're all they're all there, go
Speaker 1: back and check them out. You can find our podcast
Speaker 1: wherever you get your podcasts. Just look for the Stuff
Speaker 1: to Blow your Mind podcast need uh in that feed
Speaker 1: We put out core episodes of the show on Tuesdays
Speaker 1: and Thursdays. Mondays, we have a little listener mail Wednesdays,
Speaker 1: so that's when we do the artifact shorty uh usually,
Speaker 1: and then on Friday's we do Weird House Cinema. That's
Speaker 1: our chance to sort of set most of the science
Speaker 1: aside and just focus on the films about rampaging robots.
Speaker 1: You just thinks. As always to our excellent audio producer
Speaker 1: Seth Nicholas Johnson. If you would like to get in
Speaker 1: touch with us with feedback on this episode or any
Speaker 1: other to suggest a topic for the future, or just
Speaker 1: to say hello. You can email us at contact at
Speaker 1: stuff to Blow your Mind dot com. Stuff to Blow
Speaker 1: Your Mind is production of I Heart Radio. For more
Speaker 1: podcasts for my heart Radio, visit the iHeart Radio app,
Speaker 1: Apple Podcasts, or wherever you're listening to your favorite shows.
Speaker 1: Pretty Toy pop po