Speaker 1: Hey, welcome to Stuff to Blow Your Mind. My name
Speaker 1: is Robert Lamb and I'm Joe McCormick, and it's Saturday.
Speaker 1: Time for an episode from the Vault. This is the
Speaker 1: beginning of a series that we did about about robot
Speaker 1: culpability and machine punishment. I think we're just calling this
Speaker 1: Punish the Machine Part one, right, So this originally aired
Speaker 1: on April. This one raised a lot of interesting questions
Speaker 1: and so we hope you enjoy it. Welcome to stot
Speaker 1: to Blow Your Mind, production of My Heart Radio. Hey you,
Speaker 1: welcome to Stuff to Blow Your Mind. My name is
Speaker 1: Robert Lamb and I'm Joe McCormick. And right before we
Speaker 1: started recording today, we were just talking about that iconic
Speaker 1: scene and returned to the Jedi where the droids are
Speaker 1: sent to the droid torture Chamber. Do you remember there?
Speaker 1: I guess it's not just a droid torture chamber. It's
Speaker 1: sort of like the uh, the Droid onboarding center right
Speaker 1: where the you know, R two, D two and C
Speaker 1: three po have been given as gifts to job of
Speaker 1: the Hut and they go meet their new like droid
Speaker 1: boss and he's like you you're a feisty little one
Speaker 1: and he's signing them in but uh, he sees that
Speaker 1: R two D two is a is a bad robot
Speaker 1: who needs discipline, and R two D two is confronted
Speaker 1: with these images of robots being punished with various corporal punishments,
Speaker 1: like one is getting stretched on a robot rack and
Speaker 1: another one is getting its feet burned. Yes, this is
Speaker 1: a This is a great scene, one that that definitely
Speaker 1: burns its way into your your brain as a as
Speaker 1: a young viewer, and maybe you don't think about it
Speaker 1: that much for a long time, but uh, it's it's
Speaker 1: still in there. It's it takes place in the bowels
Speaker 1: of job as Palace on tattooing and it's um, yeah,
Speaker 1: it's like Droid intake but also Droid corrections. It's there.
Speaker 1: There are a number of different department that I think
Speaker 1: are converging here and and it ray it ultimately kind
Speaker 1: of raises some interesting questions about um about ethics and
Speaker 1: punishment and in crime, and certainly as it relates to
Speaker 1: two robots. Uh. Of course, one thing, this important distress
Speaker 1: here is like none of this was really intended in
Speaker 1: these scenes. This was about having droids doing things that
Speaker 1: humans would be doing to each other in other pieces
Speaker 1: of cinema, certainly, things like old pirate movies or old
Speaker 1: sin bad movies or what have you. I mean, that's
Speaker 1: kind of that's kind of Star Wars in a nutshell, right. Uh.
Speaker 1: This whole portion of Return of the Jedi is essentially
Speaker 1: a big pirate movie, a big swashbuckler set in an
Speaker 1: alien location. Oh yeah, the job of the hud as
Speaker 1: a pirate captain. Yeah. But something interesting occurs when you
Speaker 1: replace the humans in these these tropic scenes with machines. Uh,
Speaker 1: and then you think about it, you know, you think
Speaker 1: about why is that robot torturing the other as if
Speaker 1: it makes perfect sense if it's humans doing it, But
Speaker 1: then the things that we create in our image when
Speaker 1: they're doing it, suddenly we start seeing the flaws in
Speaker 1: our reasoning. Suddenly we start questioning, well, how does this
Speaker 1: whole system supposed to work? Uh? And maybe this whole
Speaker 1: system doesn't work. Well. Yeah, there are multiple levels of
Speaker 1: absurdities in the scene. One is the idea that this
Speaker 1: robot is just sort of like coolly telling are two,
Speaker 1: that he is going to learn some discipline. But then
Speaker 1: the image that accompanies that is like, clearly just like
Speaker 1: extreme robot torture, Like it's something way beyond what would
Speaker 1: have to do with with discipline in the real world.
Speaker 1: But then the other level of absurdity is that it's
Speaker 1: robots in the scene, but coming off of the issue
Speaker 1: of just like barbaric pirate torture directly and more to
Speaker 1: the broader question of robots and discipline and punishment. Uh,
Speaker 1: this is something that we actually wanted to talk about
Speaker 1: today because the issue of robot moral and legal agency
Speaker 1: is something I've been interested in for a long time.
Speaker 1: I've talked about it, It's come up on the show
Speaker 1: in the past and in briefer ways, um, and today
Speaker 1: I wanted to come back and devote a full episode
Speaker 1: to the subject. I guess actually we're gonna be talking
Speaker 1: about this for a couple of episodes now. The question
Speaker 1: of as machines AI robots become more independent and act
Speaker 1: more like agents more like humans do, how are we
Speaker 1: to understand their moral and legal culpability when they do
Speaker 1: something that harms people? And is there such a thing
Speaker 1: as robot punishment robot discipline? Do these concepts reflect anything
Speaker 1: that's achievable in the real world and practical, and if so,
Speaker 1: how would any of this work? Yeah? I think one
Speaker 1: of the most interesting things about this topic is that
Speaker 1: it does force us to force a face off between
Speaker 1: what robots and ai actually are or will be, and
Speaker 1: how we think about them, indeed, how we anthropomorphize them. Um,
Speaker 1: And perhaps it might be helpful to take a step
Speaker 1: back and think about something far less advanced as a
Speaker 1: robot and think something. I think about something more like
Speaker 1: a hammer. Okay, so everyone's heard the old adage that
Speaker 1: it's a poor carpenter who blames their tools, right, But
Speaker 1: of course we do this all the time. Uh, the
Speaker 1: hammer slips, it hits our fingers, and we may, at
Speaker 1: least in the heat of the moment, blame the hammer
Speaker 1: for the failure. Now, we may get over this quickly,
Speaker 1: but then again, we may decide that the hammer truly
Speaker 1: is at fault and it should be used less. We
Speaker 1: might also take this idea to a number of different extremes.
Speaker 1: We might decide that the hammer is not merely at
Speaker 1: fault but faulty, and then we're entitled to at least
Speaker 1: a refund for its purchase. Or we might decide that
Speaker 1: the hammer needs to actually be punished and and this
Speaker 1: of course is ridiculous. And yet the idea of punishing
Speaker 1: the hammer by say, putting it in the corner, or
Speaker 1: perhaps you have an old toolbox of shame that's just
Speaker 1: for the misbehaving tools, or maybe it's it's less thought
Speaker 1: out and you just throw the hammer across the yard
Speaker 1: as punishment for what it has done to you. Um. Again,
Speaker 1: these are ridiculous things to do, but the idea of
Speaker 1: doing them is not that far and from us. Um,
Speaker 1: those of you listening, you may have engaged in this
Speaker 1: sort of thing as well. You might also simply throw
Speaker 1: the tool away and otherwise perfectly good tool. Um. I
Speaker 1: know that I did this once with a knife sharpening
Speaker 1: gadget that caused me to cut my finger my and
Speaker 1: my like reaction was this thing has now injured me,
Speaker 1: it has drawn my blood. Uh, I am, I'm getting
Speaker 1: rid of it. It goes in the trash. It bore
Speaker 1: malice against me. Yeah, or you know, I you know
Speaker 1: ultimately it's I mean, I mean you you get into
Speaker 1: arguments about different tools, like is this a dangerous tool?
Speaker 1: And in that case that was my reasoning. It's like
Speaker 1: this tool is dangerous, it's not enabling me to do
Speaker 1: what I want to do without drawing blood, so it
Speaker 1: goes in the trash um. But then there have been
Speaker 1: other cases where like I had a mandolin for slicing
Speaker 1: up carrots, and um, I like nicked my finger on it.
Speaker 1: But oh I I nicked my fingers A rule they
Speaker 1: can be. But I nicked my finger not using it,
Speaker 1: but going into the drawer for something else. So I
Speaker 1: punished it by putting at the very bottom of the drawer.
Speaker 1: But I didn't throw it away. Uh huh. So I
Speaker 1: think if we all think, think back, you know, we
Speaker 1: have examples of this sort of thing from our from
Speaker 1: our life. Well sure, I mean I'm going to talk
Speaker 1: in this episode about some of the ways that we
Speaker 1: mindlessly apply social rules to robots. But yeah, I think
Speaker 1: what you're illustrating here is that you don't even have
Speaker 1: to get to the robot agency stage before people start
Speaker 1: doing that. I mean people mindlessly, to a lesser extent,
Speaker 1: mindlessly apply social rules and rules derived for managing human
Speaker 1: relationships to inanimate objects with no moving parts. Yeah. Yeah,
Speaker 1: you don't even have to get into a room ba
Speaker 1: or anything, or you know, you can deal with the hammer,
Speaker 1: the can opener but we also you know, it's it's
Speaker 1: also a sad fact that many pat pet owners will
Speaker 1: punish an animal for a transgression, transgression, but scientific evidence
Speaker 1: shows that this tends to not actually work at least
Speaker 1: in most of the circumstances that it's used. Um so
Speaker 1: you know, even it's not merely with with tools and
Speaker 1: inanimate objects, but even non human entities were liable to
Speaker 1: engage in this kind of discipline based thinking. Now, most
Speaker 1: of the studies I think that necessarily relate to dogs
Speaker 1: if I remember correctly, And there's a lot going on
Speaker 1: here that doesn't relate directly to inanimate objects and robots,
Speaker 1: but it illustrates how we tend to approach the punishment
Speaker 1: of other agents and perceived agents. Well, yeah, there's a disconnect,
Speaker 1: and this will be highlighted in one of the papers
Speaker 1: we're going to talk about in in this pair of episodes,
Speaker 1: But there's a disconnect in that punishment is often logically
Speaker 1: characterized as serving one type of purpose, but then is
Speaker 1: applied more like it serves another type of purpose. So
Speaker 1: like it is logically explained as say a deterrent, right,
Speaker 1: I mean, if you if you're talking about uh, legal theories,
Speaker 1: of punishment. One of the main things that people come
Speaker 1: up with is say, well, the remedy provided by the
Speaker 1: law is in order to punish the person who did
Speaker 1: the bad thing, in order to send a message that
Speaker 1: people should not do this bad thing and thus maybe
Speaker 1: discourage other people from doing something similar in the future,
Speaker 1: or discourage the same person from doing it again. And
Speaker 1: if it were to actually serve that purpose, it's debatable
Speaker 1: in what cases it does actually serve that purpose. Maybe
Speaker 1: sometimes it does, but that is a you know, you
Speaker 1: could argue that's a rational, logical thing that prevents harm.
Speaker 1: But the way punishment is actually often inflicted in the
Speaker 1: real world seems to be more consistent with judgments based
Speaker 1: on like emotional satisfaction of the idea of having been wronged. Yeah.
Speaker 1: And then also we get into this area where, uh,
Speaker 1: we have a couple of different factors encouraging traditions of discipline. Um,
Speaker 1: particularly if we look at a parenthood, which there's some
Speaker 1: crossover between discipline and parents who I didn't discipline and
Speaker 1: the criminal justice system. But you know, uh, not everything
Speaker 1: is going to line up one to one here. But um,
Speaker 1: on the childhood example, it's benn argue that parents use
Speaker 1: punishment first of all, because it's an emotional response out
Speaker 1: of anger and anger that may be mismanaged. But then
Speaker 1: on top of this, it's, you know, something that's culturally
Speaker 1: passed down and punishment may seem to work. I was
Speaker 1: reading about this in a Psychology Today article by Michael Carson,
Speaker 1: pH d j D. And uh, this is what they said. Quote,
Speaker 1: because the child is inhibited in your presence, it's easy
Speaker 1: to think they would be inhibited in your absence. Punishment
Speaker 1: produces politeness, not morality. Thus, the inhibited, obedient child inadvertently
Speaker 1: reinforces the parents punitive behavior by acting obedient. For the
Speaker 1: sorts of parents who find obedient children reinforcing, Yeah, that
Speaker 1: raises an interesting question. I mean, I've been mainly thinking
Speaker 1: for this episode about about legal punishments, but like when
Speaker 1: it comes down to parenting, that's a very different kind
Speaker 1: of thing because both parenting in the legal system involved punishment,
Speaker 1: but parenting is not subject to a legal system, right,
Speaker 1: So there's no there is no systematized way by which
Speaker 1: justice is administered from a parent. It's just just I mean,
Speaker 1: I think a lot of times it's just sort of
Speaker 1: like whatever the parent can manage to do in the
Speaker 1: moment because like the kids driving them crazy or something. Yeah,
Speaker 1: usually the child can't take it to a higher court, right,
Speaker 1: But I mean, I think you're absolutely right that whether
Speaker 1: you're talking about discipline administered by a parent or the
Speaker 1: justice system as a whole, I'd say that both are
Speaker 1: probably based more on tradition and philosophy and less on
Speaker 1: a scientifically rigorous study of the most efficient ways to
Speaker 1: reduce harm. And one of the interesting things about thinking
Speaker 1: about how law could potentially be applied to harm caused
Speaker 1: by autonomous machines is that it may help give us
Speaker 1: some insights on ways that the justice system as it
Speaker 1: exists and is applied to humans today tends to be
Speaker 1: of irrationally already, like with respect to humans. Yeah, and again,
Speaker 1: this is what's so interesting about this this paper, I mean, well,
Speaker 1: the papers that we're going to discuss this topic in general, though,
Speaker 1: is if you start, you start comparing machine possibilities to
Speaker 1: human possibilities, and it's on one level of thought experiment
Speaker 1: in how you would hold machines are responsible, but then
Speaker 1: it makes you rethink the way humans are held responsible.
Speaker 1: You know. It's like, um, you might think you have
Speaker 1: a pretty square a way, like if if an adult
Speaker 1: sells a pack of cigarettes to someone who's underage, right,
Speaker 1: but then one of a machine does the same thing,
Speaker 1: how do you treat the machine? Do you treat the
Speaker 1: machine like an adult? And then in trying to figure
Speaker 1: out how to treat this machine, does it make you
Speaker 1: rethink how you should be treating the adult who engaged
Speaker 1: in this behavior. I don't know, yeah, And I think
Speaker 1: a lot of that will come down to our understanding
Speaker 1: of what the machine is capable of, like what kind
Speaker 1: of constraints it has, what type of what level of
Speaker 1: autonomy it seems to be operating at. I mean, again, Weirdly,
Speaker 1: even when people set out to define clear rules for
Speaker 1: what makes a machine culpable, there there's still going to
Speaker 1: be a lot of subjectivity in it. I'm I'm looking
Speaker 1: at like legal definitions of what constitutes a robot versus
Speaker 1: just a machine, and some of these definitions involved things like, well,
Speaker 1: a robot feels like a social agent. So there's still, like,
Speaker 1: you know, an element of subjectivity. But I think that's
Speaker 1: correct in how we actually apply the term most of
Speaker 1: the time, right, Like something is like a gut feeling
Speaker 1: about how this machine is behaving in your world. Is
Speaker 1: it acting more like a fixed, you know, brainless machine,
Speaker 1: or is it acting a little bit more like a person. So,
Speaker 1: while it would be one thing if if it were
Speaker 1: basically a cigarette vending machine that was selling to children,
Speaker 1: but if it were a machine that went door to
Speaker 1: door and and rang the doorbell and then asked for
Speaker 1: the children so they could sell them cigarettes, that would
Speaker 1: be a different matter. I mean, yeah, I mean I
Speaker 1: think that would require different types of remedies. Probably, Yeah,
Speaker 1: I mean I think a lot of people would probably
Speaker 1: look at the cigarette vending machine and say where was
Speaker 1: the vending machine placed? Why was it in a place
Speaker 1: that children could have access to it? Rather than attacking
Speaker 1: the fundamentals of the machine itself, if it's going door
Speaker 1: to door and giving cigarettes to kids, yeah, then people
Speaker 1: are probably going to attack the fundamentals and the moral
Speaker 1: character of the robot. You have not attacked the robot itself.
Speaker 1: It would just rob justice in somebody's front yard. Yeah, alright,
Speaker 1: So I guess I want to introduce one of the
Speaker 1: papers we're gonna be looking at in this pair of episodes,
Speaker 1: and it is by Mark A. Limley and Brian casey
Speaker 1: called Remedies for Robots published in the University of Chicago
Speaker 1: Law Review in twenty nineteen. And this is a big paper.
Speaker 1: It's like eighty something pages long with a with a
Speaker 1: lot of different interesting, uh thoughts in it. We're not
Speaker 1: going to be able to cover the entire thing in depth,
Speaker 1: but it's worth looking up. You can easily find a
Speaker 1: full PDF of it if you want to read it
Speaker 1: in depth. And we're gonna look at some of the
Speaker 1: larger framework it lays out and then some interesting thoughts
Speaker 1: raised to buy it. But to kick it off here
Speaker 1: the the author's right quote, what happens when artificially intelligent
Speaker 1: robots misbehave? The question is not just hypothetical. As robotics
Speaker 1: and artificial intelligence systems increasingly integrate into our society, they
Speaker 1: will do bad things. We seek to explore what remedies
Speaker 1: the law can and should provide once a robot has
Speaker 1: caused harm. Now, obviously we're going to be focused less
Speaker 1: on the like minute particulars of US legal precedent here
Speaker 1: and more on the broader issues they raise about robot agency,
Speaker 1: robot moral decision making, and how that interacts with harm
Speaker 1: and morality and justice. And the authors start out in
Speaker 1: their introduction by giving what I think is a really
Speaker 1: fantastic example of how an autonomous robot with behave of
Speaker 1: years guided by machine learning, which is how you know,
Speaker 1: increasingly most robots are going to be controlled, can end
Speaker 1: up doing things that are the exact opposite of what
Speaker 1: was intended. So this case that they site is based
Speaker 1: on a true story from a presentation at the eleventh
Speaker 1: annual Stanford e Commerce Best Practices Conference in June, and
Speaker 1: it goes like this quote. Engineers training and artificially intelligent
Speaker 1: self flying drone were perplexed. They were trying to get
Speaker 1: the drone to stay within a predefined circle and head
Speaker 1: toward its center. Things were going well for a while.
Speaker 1: The drone received positive reinforcement for its successful flights, and
Speaker 1: it was improving its ability to navigate toward the middle
Speaker 1: quickly and accurately. Then suddenly things changed. When the drone
Speaker 1: near the edge of the circle, it would inexplicably turn
Speaker 1: away from the center, leaving the circle. What went wrong?
Speaker 1: After a long time spent puzzling over the problem, the
Speaker 1: designers realized that whenever the drone left the circle during tests,
Speaker 1: they had turned it off. Someone would then pick it
Speaker 1: up and carry it back into the circle to start again.
Speaker 1: From this pattern, the drones algorithm had learned correctly that
Speaker 1: when it was sufficiently far from the center, the optimal
Speaker 1: way to get back to the middle was to simply
Speaker 1: leave it all together. As far as the drone was concerned,
Speaker 1: it had discovered a wormhole somehow, flying outside of the
Speaker 1: circle could be relied upon to magically teleport it closer
Speaker 1: to the center, and far from violating the rules instilled
Speaker 1: in it by its engineers, the drone had actually followed
Speaker 1: them to a t. In doing so, however, it had
Speaker 1: discovered an unforeseen shortcut, one that subverted its designer's true intent.
Speaker 1: That's really good, that's that's it's as a yes, I
Speaker 1: love it. This is such a great example of how
Speaker 1: robots can fail in ways that are perfectly logical for
Speaker 1: the machine means themselves, but hard for humans to predict
Speaker 1: in advance, because we're not understanding how our you know,
Speaker 1: our programming or the data sets we're training it on
Speaker 1: is biasing its behavior in ways that that are strange
Speaker 1: to us. And in this case, of of course, such
Speaker 1: a malfunction is harmless, but as autonomous machines become more
Speaker 1: and more integrated into the broader culture, not just in
Speaker 1: controlled contained locations like factory floors and laboratories, but in
Speaker 1: the wild so on the streets and in our homes
Speaker 1: and stuff. There will inevitably be cases where robots fail
Speaker 1: like this and fail in ways that cause catastrophic harm
Speaker 1: to people. Yeah, and plus as an aside, we we
Speaker 1: have to realize that even in cases where the machines
Speaker 1: have not failed, there will be gray areas in which
Speaker 1: it's not completely clear, and an argument could be made
Speaker 1: in these cases for machine culpability with a variety of
Speaker 1: intense and possible biases in place. Oh yeah, that's another
Speaker 1: thing these authors talk about that there can be all
Speaker 1: kinds of ways that, uh, that robotics and AI could
Speaker 1: end up causing extreme harm to people without ever doing
Speaker 1: anything that if a human did, it would be illegal.
Speaker 1: What one example they give is like if, um, if
Speaker 1: Google were to suddenly change its Google Maps algorithm so
Speaker 1: that it routed all of the city's traffic through your neighborhood. Like,
Speaker 1: nothing illegal about that. It doesn't like commit a crime
Speaker 1: against you, but this is going to drastically negatively impact
Speaker 1: your quality of life, and it's a decision that's just
Speaker 1: like a could be a quirk of an algorithm in
Speaker 1: a machine. Now this paper in particular concerns the legal
Speaker 1: concept of remedies. So I was reading about remedies. A
Speaker 1: common legal definition that I found is quote the means
Speaker 1: to achieve justice in any matter in which legal rights
Speaker 1: are involved. Or in the words of Limely and Casey,
Speaker 1: what do I get when I win? Right? So, if
Speaker 1: you if you take somebody to court, be as you
Speaker 1: say they have harmed you, whatever outcome you're seeking from that,
Speaker 1: that court case is the remedy. So usually when a
Speaker 1: court case finds that somebody has done something wrong to
Speaker 1: harm somebody else, the court responds to the finding of
Speaker 1: guilt or blame by enforcing this remedy. And common remedies
Speaker 1: would include a payment of money, right, a guilty defendant
Speaker 1: has to pay money to the plaintiff, a punishment of
Speaker 1: the offender, like maybe they go to jail, or a
Speaker 1: court order to do something or not to do something.
Speaker 1: For example, somebody is ordered not to drive a vehicle,
Speaker 1: or they are ordered not to go within a hundred
Speaker 1: feet of somebody else or something like that. Yeah, or
Speaker 1: their their eyeball is removed, or they have to spend
Speaker 1: a night in a hunted house something like that. Hopefully
Speaker 1: not in modern law. But wait a minute, there are
Speaker 1: some Sometimes you do read about some really strange like
Speaker 1: remedies that are ordered by judges like I order you
Speaker 1: to I don't know, to wear chicken suit or something,
Speaker 1: right like, Yeah, there's some judges who like to get creative.
Speaker 1: It seems weird. Yeah, I wonder I have There have
Speaker 1: been cases where someone has to spend a night and
Speaker 1: a hounded house due to a court order. I think
Speaker 1: that would be a good setup for a film. But anyway,
Speaker 1: so when you start looking at the idea of remedies,
Speaker 1: remedies are complicated because they involve different types of implied
Speaker 1: satisfaction on behalf of the victim or plaintiff. And some
Speaker 1: are very clear and material and others are much more abstract.
Speaker 1: So the very the ones that are very clear and
Speaker 1: material are like, if I hit your car with my
Speaker 1: car and I'm clearly at fault, I need to give
Speaker 1: you a payment of cash to offset the material losses
Speaker 1: to the value of your car. Right. But then other
Speaker 1: times it's it's more abstract. It's you know, punishment of
Speaker 1: an offender, to give the victim a sense of justice
Speaker 1: or to allegedly discourage someone from committing this type of
Speaker 1: harm or offense in the future. And then the authors
Speaker 1: right that things get way more complicated when you bring
Speaker 1: robots and AI into the picture. For example, if you're
Speaker 1: trying to give a court order to a person, you know,
Speaker 1: saying like you shall not drive a car, you shall
Speaker 1: not you know, come within a hundred feet of this person.
Speaker 1: You can do so in natural language. You can like
Speaker 1: speak a sentence to them and you can expect them
Speaker 1: to understand. But how do you get a court to
Speaker 1: give an order to a robot not to do something?
Speaker 1: Most robots don't have natural language processing, and even if
Speaker 1: they do, a lot of times it's not that good.
Speaker 1: So you might think, okay, well you just you know,
Speaker 1: you give the court order to the robots programmer and
Speaker 1: then it will and then they'll have to program the
Speaker 1: robot to obey. But this is also really complicated, like
Speaker 1: whose responsibility is it the robots current owner or the
Speaker 1: original contractor or creator who made the robot? Uh? And
Speaker 1: what if this is like an end user consumer device
Speaker 1: that the owner doesn't have any ability to reprogram or
Speaker 1: what if, in the case of robots whose behavior is
Speaker 1: driven by machine learning or some other kind of system
Speaker 1: that is, for practical purposes, a black box, what if
Speaker 1: it's not even clear how you could reprogram it to
Speaker 1: reliably obey the rule. Yeah, because there's a chance you
Speaker 1: got to this position because the robot misinterpreted what was
Speaker 1: asked of it. So if you then make additional requirements,
Speaker 1: ones that maybe you know, haven't actually been tested before
Speaker 1: but are just you know, that that are then brought
Speaker 1: on by the court, that could conceivably create new problems,
Speaker 1: right yeah, Yeah, totally, And and it keeps getting even
Speaker 1: more complicated from there, like Limle and Casey right quote.
Speaker 1: To complicate matters further, some systems, including many self driving cars,
Speaker 1: distribute responsibility for their robots between both designers and downstream operators.
Speaker 1: For systems of this kind, it has already proven extremely
Speaker 1: difficult to allocate responsibility when accidents inevitably occur. It just
Speaker 1: seems like a real, real fast way to get into
Speaker 1: skynet territory, where it's like the robot then decides that
Speaker 1: only way to assure that it never sells cigarettes to
Speaker 1: children again is to destroy all humans. That sounds like
Speaker 1: finding a wormhole to me. We will be getting into
Speaker 1: some more wormhole territory as we go on, so more complications. Uh.
Speaker 1: The authors bring up the idea of how to courts
Speaker 1: compel a person or a company to obey a court order. Right, Like,
Speaker 1: if you know a company is like dumping poison that's
Speaker 1: harming somebody, and the person sues that company, what does
Speaker 1: the court do to get them to stop? Well, the
Speaker 1: there is a threat of contempt of court if they
Speaker 1: don't stop doing it. Right, Courts usually just assume that
Speaker 1: people are motivated by a desire not to pay huge
Speaker 1: monetary damages or a desire not to go to jail.
Speaker 1: Would that have any motivating power on a robot? It
Speaker 1: would only have that power to the extent that the
Speaker 1: robot had been programmed to take that into account. If
Speaker 1: it hadn't, it wouldn't matter at all. Like, you know,
Speaker 1: most robots probably do not have any opinion one way
Speaker 1: or another about whether about going to jail or having
Speaker 1: to pay damages. So you'd have to explicitly program it
Speaker 1: to be disincentivized by potential punishments. Yeah, because take the
Speaker 1: cigarette robot for example, Like it's prime it's prime directive
Speaker 1: is just to sell delicious cigarettes to human beings, like
Speaker 1: the the what what else? What kind of leverage do
Speaker 1: you have? Right? Exactly? So in that case you'd be
Speaker 1: faced with either you'd be trying to find some kind
Speaker 1: of human who's responsible for its behavior, but you could
Speaker 1: very well run into the problem that like, you can't
Speaker 1: really identify any one person who seems to be at
Speaker 1: fault for what it did, and it's doing this bad thing,
Speaker 1: so so what are you going to do about it? Yeah,
Speaker 1: And then of course things get even weirder when you
Speaker 1: start getting into that that other side. You know, that's
Speaker 1: like the more like direct and material remedies that can
Speaker 1: be provided by courts, either like a monetary award to
Speaker 1: the victim or in order to stop doing something that
Speaker 1: causes harm. On the other hand, you've got this thing
Speaker 1: that courts often end up engaging in, and people are
Speaker 1: are largely driven in motivated by however however irrational it
Speaker 1: might be in some cases, And that's the perceived abstract
Speaker 1: value of punishment, you know, not just material damages to
Speaker 1: a victim or in order not to do something, but
Speaker 1: the inflicting of punishments, specifically to demonstrate the court's displeasure
Speaker 1: with the original behavior of the defendant. Uh So, they
Speaker 1: raise a question that's brought up in a paper by
Speaker 1: a professor named Christina Mulligan, who explores the subject of
Speaker 1: should you have the right to punch a robot that
Speaker 1: hurts you limly in case? He called the call this
Speaker 1: the expressive component of remedies, And though a desire to
Speaker 1: see offenders punished maybe an extremely natural and nearly universal
Speaker 1: human drive, it's debatable whether it actually serves a purpose
Speaker 1: in reducing harm, and if it does, in what cases
Speaker 1: it does. I love this idea because, in a very
Speaker 1: literal level, it makes me think, well, why would you
Speaker 1: punch a robot? They're made out of out of metal.
Speaker 1: You're gonna hurt your hand. All you're gonna do is
Speaker 1: or your hand, and you're not going to hurt the
Speaker 1: robot unless first of all, you design the robots so
Speaker 1: that it has at least one punchable portion of its anatomy,
Speaker 1: and then for it to be more than just you know,
Speaker 1: a cathartic uh uh thing for you, then you have
Speaker 1: to also make sure there's some sort of feedback right
Speaker 1: where Yeah, like you punch cigarette bought in it's punchable area,
Speaker 1: then it will say owl. And maybe it will I
Speaker 1: don't know, ottaw inciner rate one packet of cigarettes so
Speaker 1: that it can never sell them, that sort of thing.
Speaker 1: But then, yeah, you're having to design your robots to
Speaker 1: to suffer to a certain extent, which I guess that
Speaker 1: means that goes back to what C three PO said, Right,
Speaker 1: he said, you know about about being made to suffer.
Speaker 1: It seems to be our lot in life. Oh that's interesting.
Speaker 1: I hadn't thought about that. Yeah, clearly R two, D
Speaker 1: two and C three p O have have inherent desires
Speaker 1: to avoid pain. They have been programmed with that. Yeah,
Speaker 1: but as we've said that, that's not standard issue for robots.
Speaker 1: Most robots don't care about whether or not they get injured,
Speaker 1: Like that's not a motivating factor for them. And again
Speaker 1: it raises this bizarre question of like, what are you
Speaker 1: doing when you punch the robot? Like what is I
Speaker 1: guess it's making you feel better, But does it make
Speaker 1: you feel better if, like you know that the robot
Speaker 1: doesn't actually care? Yeah, and then what then what needs
Speaker 1: to be done to convince you that it does care. Yeah,
Speaker 1: it just gets very sticky, very quickly, and then of
Speaker 1: course turns the mirror back on the way we handle
Speaker 1: human to human scenarios. Right. But anyway, Limbly and Casey,
Speaker 1: I guess to to summarize their position, they say, Okay,
Speaker 1: increasingly independent robots and AI are coming. They're they're infiltrating
Speaker 1: more and more into society, and they will inevitably do
Speaker 1: bad things. When that happens, the legal system will try
Speaker 1: to order remedies to make things right, and you know
Speaker 1: when when harm has been caused. Our current legal understanding
Speaker 1: of remedies is based on the assumption of human agents
Speaker 1: human agents only, and its rules are not suited to
Speaker 1: dealing with robot crime or robot offenses quote. As we
Speaker 1: have shown, Failing to recognize those differences could result in
Speaker 1: significant unintended consequences, inadvertently encouraging the wrong behaviors, or even
Speaker 1: rendering our most important remedial mechanisms functionally irrelevant. Uh So,
Speaker 1: to take robot agents into account, we're going to have
Speaker 1: to examine and rethink how our systems of remedies work.
Speaker 1: But and this is a point we've been making already,
Speaker 1: this could have multiple benefits because it could also lead
Speaker 1: to a better understanding of how we apply these remedies
Speaker 1: to cases dealing exclusively with humans. Quote. Indeed, one of
Speaker 1: the most pressing challenges raised by the technology is its
Speaker 1: tendency to reveal the trade offs between sidal economic and
Speaker 1: legal values that many of us today make without deeply
Speaker 1: appreciating the downstream consequences. They right, we need a law
Speaker 1: of remedies for robots, but in the final analysis, remedies
Speaker 1: for robots may also end up being remedies for all
Speaker 1: of us. Now, like I said, this is a very
Speaker 1: long paper. We can't do justice to all of the
Speaker 1: subjects they raise, but to focus on some highlights, I
Speaker 1: thought one interesting place to look was when they try
Speaker 1: to get into the definition of what actually makes a
Speaker 1: robot in the legal sense. Obviously, there's going to be
Speaker 1: some difficulty here because think about how differently the term
Speaker 1: is used and how many different things it's applied to
Speaker 1: in the world. Uh. The authors here cite a professor
Speaker 1: Ryan Callo, who in the past had written that there
Speaker 1: are three important characteristics that define a robot and make
Speaker 1: it different from any machine, just like a computer or phone,
Speaker 1: and Callo says that these three uh, these three qualities
Speaker 1: are embodiment, emergence, and social valence. So to quote from Calo,
Speaker 1: robotics combines, arguably for the first time, the promiscuity of
Speaker 1: information with the embodied capacity to do physical arm. Robots
Speaker 1: display increasingly emergent behavior, permitting the technology to accomplish both
Speaker 1: useful and unfortunate tasks in unexpected ways. I like that
Speaker 1: idea of unfortunate tasks. Um and robots, more so than
Speaker 1: any technology and history, feel to us like social actors,
Speaker 1: a tendency so strong that soldiers sometimes jeopardize themselves to
Speaker 1: preserve the lives of military robots in the field, and
Speaker 1: lives is in quotes there. Yeah, you may remember this
Speaker 1: from the film that came out a few years back,
Speaker 1: Saving Private Cigarette Robot. It's quite touching. I mean, it
Speaker 1: seems absurd, but it does seem to play on our
Speaker 1: natural biases. I want to talk about a couple of
Speaker 1: examples from a psychology paper in a second, but like, uh,
Speaker 1: we're we're just so ready to look at machines like
Speaker 1: humans and and treat them as such. It seems almost
Speaker 1: impossible to avoid. But anyway to pick up with limely
Speaker 1: and Casey after after that Callo quote they a quote.
Speaker 1: In light of these qualities, Calo argues that robots are
Speaker 1: best thought of as artificial objects or systems that sense, process,
Speaker 1: and act upon the world to at least some degree. Thus,
Speaker 1: a robot, in the strongest, fullest sense of the term,
Speaker 1: exists in the world as a corporeal object with the
Speaker 1: capacity to exert itself physically. Though, it's interesting to me
Speaker 1: that even this attempt to give a strict and legally
Speaker 1: useful definition of a robot includes a subjective component. I
Speaker 1: brought this up earlier, the component about human feelings, the
Speaker 1: social valence criterion. The Calos sites here this means they
Speaker 1: feel to us like social actors. Yeah. Like I was
Speaker 1: wondering in all of this, like, where does a particularly
Speaker 1: malicious robo call fit into the scenario, Say, a robo
Speaker 1: call that is not just about trying to sell you something,
Speaker 1: but it's like, you know, actively trying to say, get
Speaker 1: a credit card number out of you for nefarious purposes. Yeah,
Speaker 1: that's a really good point. And and along those lines,
Speaker 1: Limbly and Casey argue that actually they don't think the
Speaker 1: embodiment criteria of hardware is necessarily a good one that
Speaker 1: maybe our concept of a robot should be less limited
Speaker 1: to the essentialist quality of being embodied and more just
Speaker 1: applied to anything that exhibits intelligent behavior and exactly things
Speaker 1: like that robot call would be would be a good example. Uh,
Speaker 1: the things we think of as robots probably do. They
Speaker 1: they're not just like stand alone objects. They interact with
Speaker 1: the broader world in some way, but they could be
Speaker 1: entirely software based. Yeah, I guess certainly the room is
Speaker 1: a great example, or any kind of like vacuuming robot
Speaker 1: where it's it's it's it's in your house or it's
Speaker 1: in a room in your house. It's a it's interacting
Speaker 1: in your environment and it's essentially making decisions about how
Speaker 1: best to move around that space. Sure, but if you
Speaker 1: want to take it out of the embodied space, you
Speaker 1: could have the idea of bots. On the Internet, there's
Speaker 1: things out there are acting autonomously to some extent and doing,
Speaker 1: you know, executing some behavior, acting almost maliciously. We were
Speaker 1: tempted to call them bots meaning short for robots, because
Speaker 1: they have some kind of apparent independent agency and they're
Speaker 1: doing something that seems at least halfway intelligent. Right, yeah, yeah,
Speaker 1: and you can easily imagine how they could they are
Speaker 1: and well they I mean they are used maliciously in
Speaker 1: some cases. But how something like a social media boat
Speaker 1: that responds to certain comments in a particular way, like,
Speaker 1: it's very easy to imagine how you how that could
Speaker 1: be utilized in a way that would be not only
Speaker 1: annoying but just that outright harmful, even physically harmful. Oh yeah,
Speaker 1: I mean think about some of these, say like bots
Speaker 1: on social media that try to crowdsource information like during
Speaker 1: a natural disaster or something like that. You could imagine uh,
Speaker 1: intentionally maliciously manipulating a bot of this kind to like
Speaker 1: have you know, bad information on it or something, yeah yeah,
Speaker 1: or know, anything that a troll can do on social media,
Speaker 1: a bought could conceivably do as well. So that just
Speaker 1: you know, opens up the door, right. But coming back
Speaker 1: to this, so there's this interesting idea that robots feel
Speaker 1: to us like social actors, and that seems to be,
Speaker 1: at least by some people's definitions, a kind of inextricable
Speaker 1: quality of what makes a robot like it feels like,
Speaker 1: at least to a small extent, like a person somehow,
Speaker 1: and it reminds me of the psychology paper I was
Speaker 1: looking at just recently on human social interaction with robots,
Speaker 1: that is by Elizabeth Broadbent called Interactions with Robots The
Speaker 1: Truths We Reveal About Ourselves, published in the Annual Review
Speaker 1: of Psychology in twenty seventeen. Uh, this was a highly
Speaker 1: cited paper, and it seems to be. It's a big
Speaker 1: literature review of a lot of different stuff about about
Speaker 1: how humans interact emotionally and socially with robots. And the
Speaker 1: one section I was thinking about was where she reviews
Speaker 1: a bunch of other studies about how we mindlessly apply
Speaker 1: social rules to robots. So there are a ton of
Speaker 1: different examples, but just to cite a couple of them,
Speaker 1: once she writes up his quote. After using a computer,
Speaker 1: people evaluate its performance more highly if the same computer
Speaker 1: delivers the rating scale, then if another computer delivers the
Speaker 1: rating scale, or if they rate it with pen and paper.
Speaker 1: So like, if you know, you get a thing at
Speaker 1: the end of a test that says like, hey, you know,
Speaker 1: how did you enjoy interacting with this machine? You're you're
Speaker 1: more likely to give it a higher score if you're
Speaker 1: still sitting on the same machine or at least. That
Speaker 1: was what was found by nas at All in UH
Speaker 1: and the and Broadbent rights quote. This result is similar
Speaker 1: to experiment or bias, in which people try not to
Speaker 1: offend a human researcher. Another example of social behavior is reciprocity.
Speaker 1: We help others who help us. People help to computer
Speaker 1: with a task for more time and more accurately if
Speaker 1: the computer first helped them with a task than if
Speaker 1: it did not, And this was found by Fog and
Speaker 1: nas In. I love that idea of people, you know,
Speaker 1: being more reluctant to rate a computer UH poorly if
Speaker 1: they're still interacting with the same computer that that's that
Speaker 1: seems perfectly true to me. But another interesting one from
Speaker 1: the summary is quote research and psychology has shown that
Speaker 1: the presence of an observer can increase people's honesty, but
Speaker 1: incentives for cheating can reduce honesty, and this is found
Speaker 1: by Covey at All in nineteen eighty nine. In a
Speaker 1: robot version of this work, participants given incentives to cheat
Speaker 1: were shown to be less honest when alone compared to
Speaker 1: when they were accompanied by either a human or by
Speaker 1: a simple robot, and that was found by Hoffman at
Speaker 1: All in this illustrates that the social presence of robots
Speaker 1: may make people feel as though they're being watched and
Speaker 1: increase their honesty in an effect similar to that produced
Speaker 1: by the presence of humans. Now, this is inter right.
Speaker 1: This This also reminds me of various studies that have
Speaker 1: gone into sort of the idea of of imagine beings
Speaker 1: or religious beings watching us while we're doing things right,
Speaker 1: or even just like I imagery, like putting some eyes
Speaker 1: imagery on a wall looking at people while they're like,
Speaker 1: I don't know, not supposed to steal from the collection
Speaker 1: plate or something like that. I don't know if it's
Speaker 1: to the same extent, but at least in the same
Speaker 1: direction that the presence of another human is. You know,
Speaker 1: you're you might be a little bit worried that ARE
Speaker 1: two D two is gonna, you know, judge your moral
Speaker 1: character harshly or tattle on you. I'm not as worried
Speaker 1: about R two, but um three po snitch. Coming back
Speaker 1: to Limele and Casey, so they talked for a long
Speaker 1: time about how robots get their intelligence. They talked about
Speaker 1: the importance of machine learning for the modern generations of
Speaker 1: robots and AI that it's just not practical to hard
Speaker 1: code AI the way we used to imagine. You know,
Speaker 1: you'd be a programmer and you're just like creating a
Speaker 1: lot of strings of if then statements like you know,
Speaker 1: the kind of intelligence that we expect from a modern
Speaker 1: AI or or intelligent robot is too complex for people
Speaker 1: to program in a in a direct way like that. Instead,
Speaker 1: they've got to be trained on natural data sets through
Speaker 1: machine learning. But of course doing so comes at the
Speaker 1: cost of increasing uncertainty about their future behaviors. Behaviors could
Speaker 1: emerge that a conscientious programmer would never intentionally hard code
Speaker 1: into the system. Uh So, so that brings us to like,
Speaker 1: what types of harms could we expect from robots and AI?
Speaker 1: And the authors here come up with what I think
Speaker 1: are some very useful categories, some sort of like cubby holes,
Speaker 1: to slot the different types of AI fears into. So
Speaker 1: the first kind is what they call unavoidable harms. These
Speaker 1: are probably not the main ones to be worried about,
Speaker 1: but they are worth thinking about. Uh And this is
Speaker 1: just the fact that some dangers are inherent too many
Speaker 1: products and services, we just accept them as the cost
Speaker 1: of having those products and services in the first place.
Speaker 1: So like this would just be cigarette bought just by
Speaker 1: virtue of selling cigarettes is doing harm to people, right, Yes,
Speaker 1: I mean the fact that you have cigarettes, there is
Speaker 1: some harm coming from that. But there are also ones
Speaker 1: that are more fully integrated into just the way society works,
Speaker 1: like having cars. It is absolutely inevitable that people driving
Speaker 1: cars are going to crash their cars and there will
Speaker 1: be fatalities from that, and you can think of ways
Speaker 1: of reducing it, but there's there's really not any expectation
Speaker 1: that we can have a country that has car based
Speaker 1: transportation and there will not be any accidents because there
Speaker 1: always be things that are that are not even reducible
Speaker 1: to driver error or to malfunction of the cars, right,
Speaker 1: like a tree falls on the road or something, birds,
Speaker 1: wild animals. Any So, even though they're I think there's
Speaker 1: some very convincing arguments to be made that uh a
Speaker 1: switch to self driving cars would create a much safer
Speaker 1: uh travel environment, that it would make roads safer. You're
Speaker 1: not gonna you're not gonna get to absolute zero crashes
Speaker 1: or absolute zero road fatalities, right, I mean, you wouldn't
Speaker 1: even if the driving algorithms were perfect, right, and they're
Speaker 1: probably not going to be perfect. They may well be
Speaker 1: and probably are going to be better than the average
Speaker 1: human driver. Yeah, okay, so that's just there's just unavoidable
Speaker 1: harm that comes from using any type of product or service,
Speaker 1: and when you integrate robotics and AI into that product
Speaker 1: or service, those unavoidable harms will just continue. But that's
Speaker 1: something we already deal with. The next category is deliberate
Speaker 1: least cost harms. This is similar to unavoidable harms, but
Speaker 1: it's in cases where the machine actually is able to
Speaker 1: make a decision with with important ramifications, like it can
Speaker 1: make a decision to act in a way that causes harm,
Speaker 1: but is attempting to cause the least harm possible. So
Speaker 1: in a sense, this is forcing robots to do the
Speaker 1: trolley problem. Right, do you switch to the track that
Speaker 1: has one person sitting on the train tracks instead of
Speaker 1: five people? Yeah? Yeah, And this will be another inevitable
Speaker 1: capability of autonomous cars, but it raises all kinds of
Speaker 1: thorny questions. If an autonomous vehicle can avoid a head
Speaker 1: on collision that will likely kill multiple people by suddenly
Speaker 1: swerving out of the way and hitting one pedestrian, that
Speaker 1: may indeed avoid a greater harm. But that's probably cold
Speaker 1: comfort to the one person who got hit, right, right. Yeah,
Speaker 1: And then when you have a robot or some sort
Speaker 1: of an AI involved in that decision making, I mean,
Speaker 1: it's it's you can just imagine the the the intensity
Speaker 1: of the arguments and the conversations they would ensue. Right.
Speaker 1: But then the authors raised what I think is a
Speaker 1: very interesting point. They say that this kind of life
Speaker 1: or death trolley problem will probably be the exception rather
Speaker 1: than the rule. Instead, they say, quote, uh, far likelier,
Speaker 1: I'll build albeit subtler scenarios involving least cost harms will
Speaker 1: involve robots that make decisions with seemingly trivial implications at
Speaker 1: an individual level, but which result in non trivial impacts
Speaker 1: at scale. Self driving cars, for example, will rarely face
Speaker 1: a stark choice between killing a child or killing two
Speaker 1: elderly people, but thousands of times a day they will
Speaker 1: have to choose precisely where to change lanes, how closely
Speaker 1: to trail another vehicle, when to accelerate on a freeway,
Speaker 1: on ramp, and so forth. Each of these decisions will
Speaker 1: entail some probability of injuring someone. I guess another thing
Speaker 1: to keep in mind, like with the trap, with the
Speaker 1: trolley problem, generally, when you're dealing with it, there's a
Speaker 1: there's a lot of emphasis on the problem aspect of it,
Speaker 1: you know, like the trolley problem should be a an
Speaker 1: ethical dilemma. It should, it should hurt a bit to
Speaker 1: try and figure out how what to do. And the
Speaker 1: idea of the trolley problem being something that um is
Speaker 1: encountered and decided upon, like as in a split second,
Speaker 1: by a machine, um, by an algorithm like that, that
Speaker 1: feels that that feels a bit worse to us. You know,
Speaker 1: that feels like if if it's if it's an easy decision,
Speaker 1: even if it's just based purely on math, you know,
Speaker 1: it's um it feels wrong on some level. Oh yeah, yeah,
Speaker 1: so I think you're right. But also the thing they're
Speaker 1: bringing up here is that the trolley problem you're actually
Speaker 1: more often facing is that every single day you're your
Speaker 1: autonomous car is gonna make you know, hundreds or thousands
Speaker 1: of trolley problem calls where on one track it is
Speaker 1: getting to your destination a few seconds faster, and on
Speaker 1: the other track is a one in a million chance
Speaker 1: of killing somebody. Yeah yeah, And these do we make
Speaker 1: these decisions all the time, but we don't focus on
Speaker 1: these That's but I think that's part of the issue,
Speaker 1: you know exactly. You're like, should I, okay, should I
Speaker 1: take a left on this road? Well, there's a chance
Speaker 1: there's a speeding car just above, just over the edge there,
Speaker 1: and I can't it, but I'm going to take that
Speaker 1: chance because I want to cut three minutes off my
Speaker 1: drive to work. Yes. Uh, this is actually a very
Speaker 1: good point that we we already make these decisions, but
Speaker 1: we just don't think about them in these explicit probability calculations.
Speaker 1: And there may be some consequences to thinking about them
Speaker 1: this way, which is there could be a weird, like
Speaker 1: perceived downside just to making these kind of uh, these
Speaker 1: kind of calculations objective and and explicit. Yeah, I mean
Speaker 1: I've I've run into this with some of the map
Speaker 1: programs that I used to drive before, where I want
Speaker 1: to tell it in some cases like like give me
Speaker 1: the ability and maybe they have this now, but there
Speaker 1: was one left turn in particular where the ability to
Speaker 1: to flag this left turn, this is a dangerous left turn.
Speaker 1: You have put me in a position to make. Um,
Speaker 1: I might know which left turn you're talking about. If
Speaker 1: you probably in town, it's it's it's in town, it's
Speaker 1: near our office, so yeah, I mean yeah. By the way,
Speaker 1: if you're out there working on programming driving apps, you
Speaker 1: should absolutely include the toggle key where you can say
Speaker 1: no left turns please. Yes, that that is highly useful.
Speaker 1: I'm to understand this is how one of my aunts
Speaker 1: got around, Like if they got older and they were
Speaker 1: less adventurous driving, they would only take right turns, and
Speaker 1: they would do all their driving so that no left
Speaker 1: turns were made. I think I have one time read
Speaker 1: this could be totally wrong, but I at least one
Speaker 1: time I remember reading a claim that, like, you know,
Speaker 1: the traffic efficiency would be x percent higher and people
Speaker 1: would spend x number of minutes less time and traffic
Speaker 1: if there were no such thing as left turns, if
Speaker 1: everybody had to get everywhere by only doing you know,
Speaker 1: full right turns to to go around the block. Interesting,
Speaker 1: I'm sure there would be some cases where you can't
Speaker 1: do that, but you know, in a in a grid city,
Speaker 1: seems to make a lot of sense. Maybe you get
Speaker 1: like one left turn of days, some sort of a
Speaker 1: card system. But like I said, I I cannot confirm that. Okay,
Speaker 1: but anyway, the next categories of harm they talked about
Speaker 1: this one is defect driven harms. Uh. This one is
Speaker 1: very easy to understand. The robot harms someone because of
Speaker 1: a design flaw or a bug or a mistake, or
Speaker 1: it's just broken. You know, A warehouse loading robot is
Speaker 1: designed to only operate when no humans are nearby it.
Speaker 1: But there's a malfunction with one of its sensors and
Speaker 1: it fails to detect the presence of a human operator
Speaker 1: trying to get I don't know, a piece of junk
Speaker 1: out of it, out of one of its hinges, and
Speaker 1: it moves and kills them. Okay, this is pretty straightforward,
Speaker 1: just it's broken for some reason. The authors here do
Speaker 1: point out that this gets even more complicated when there
Speaker 1: is a human in the loop e g. An autonomous
Speaker 1: car with a human driver who is supposed to intervene
Speaker 1: in the event of an emergency. They talked about one
Speaker 1: case where this happened with with I believe it was
Speaker 1: an uber autonomous vehicle where both the machine and the
Speaker 1: human fail, that both of them failed to stop a
Speaker 1: collision that hurts someone, like what happens here. Yeah, yeah,
Speaker 1: of course, we we have very similar cases in just
Speaker 1: purely human affairs. Right when questions are asked like where
Speaker 1: was this person's supervisor? Uh wait, you know who? Who
Speaker 1: were the watchers? Who? There should have been some other person,
Speaker 1: There was someone else in the loop here. Why didn't
Speaker 1: they do something to stop this crime from taking place? Right? Okay,
Speaker 1: After that you get into misuse harms. Now, some of
Speaker 1: these are very obvious, very straightforward, like if you program
Speaker 1: a robot directly to go kill someone, or even if
Speaker 1: you program it to wander around at random swinging a machete.
Speaker 1: In these cases, it seems that the human programmer is
Speaker 1: clearly at fault, right, the robot has just become a
Speaker 1: weapon of murder or of reckless endangerment, and the person
Speaker 1: who told it to do that is the person responsible. Yeah.
Speaker 1: Like if you take an automotive, uh like oil change
Speaker 1: robot and you reprogram it to um do appendectomies and
Speaker 1: people die as a result, Like, that's a misuse. You
Speaker 1: can you can only blame the the oil changed robots
Speaker 1: so much because it was not ultimately designed to perform appendectomies. Right.
Speaker 1: In this case, this is more like the hammer example
Speaker 1: used to the beginning this it's not the robot autonomously
Speaker 1: making the decision to do this. Uh, this is somebody
Speaker 1: just using it as a tool of crime. But the
Speaker 1: authors point out that there are cases where quote, people
Speaker 1: will misuse robots in a manner that is neither negligent
Speaker 1: nor criminal, but nevertheless threatens to harm others. And these
Speaker 1: types of harm are especially difficult to predict and prevent.
Speaker 1: So one example is just people love to trick robots,
Speaker 1: people like to to mess around with robots in AI.
Speaker 1: I would admit to myself finding this amusing and principle,
Speaker 1: we've talked about this in uh, you know, the the
Speaker 1: flated sex make Ina episodes. But of course there are
Speaker 1: times when it's not so funny, when when people take
Speaker 1: it to really sinister places. One example the authors bring
Speaker 1: up here is the horrible saga of Microsoft Tay. Do
Speaker 1: you remember this thing? Oh? This was the this is
Speaker 1: the robot that was traveling across the country. No no,
Speaker 1: no, no no, uh though I know what you're talking about there. No,
Speaker 1: maybe we can come back to that. But Tay was
Speaker 1: a Twitter chat bot created by Microsoft that was supposed
Speaker 1: to learn how to interact on the Internet. Just by
Speaker 1: learning from conversations it had with real users. So you
Speaker 1: could tweet it Tay and say hey, how are you doing,
Speaker 1: you know, and you could talk about the weather or whatever.
Speaker 1: But of course who who ended up engaging and training
Speaker 1: this AI to speak? It was like the worst trolls
Speaker 1: on the Internet. So within a matter of hours, this
Speaker 1: brand new chat bot had been transformed from from a
Speaker 1: from a you know, a lump of clay unformed into
Speaker 1: a pornographic nazi. Yes, I do remember this now. And
Speaker 1: this kind of just gets you thinking about the ways
Speaker 1: that people will be will be able to misuse robots
Speaker 1: in ways that guide their behavior in extremely pernicious directions,
Speaker 1: sometimes without the people guiding this misuse necessarily committing any
Speaker 1: kind of identifiable crime. Like people are going to look
Speaker 1: for exploits, they're going to look for ways, they're gonna
Speaker 1: look for cracks in the system. It's it's like within
Speaker 1: with any kind of like a video game system. You know,
Speaker 1: people are just gonna see what they can get away
Speaker 1: with and and just engage in that kind of action,
Speaker 1: sometimes just for the fun of it, right, And sometimes
Speaker 1: that's harmless, but sometimes that's really awful. Yeah, Okay, next,
Speaker 1: category is unforeseen harms. And here's where we start getting
Speaker 1: into the really the really interesting and really difficult cases,
Speaker 1: types of harm that are not unavoidable, not a product
Speaker 1: of defects or misuse, but are still not predicted by creators. Uh.
Speaker 1: And so the authors talk about how, in a way,
Speaker 1: unpredictability is what makes AI potentially useful, right, Like, it
Speaker 1: can potentially arrive at solutions that humans wouldn't have predicted,
Speaker 1: but sometimes it does so in ways that really miss
Speaker 1: the boat and could be extremely harmful if they were
Speaker 1: embodied in action in the real world. Uh. Similar to
Speaker 1: the drone example from the circle that we talked about
Speaker 1: at the inning, But they signed another fantastic example here
Speaker 1: that's kind of chilling. So I'm just going to read
Speaker 1: from Lemle and Casey here. In the ninet nineties, a
Speaker 1: pioneering multi institutional study sought to use machine learning techniques
Speaker 1: to predict health related risks prior to hospitalization. After ingesting
Speaker 1: an enormous quantity of data covering patients with pneumonia, the
Speaker 1: system learned the rule has asthma X delivers lower risk X.
Speaker 1: The colloquial translation is patients with pneumonia who have a
Speaker 1: history of asthma have a lower risk of dying from
Speaker 1: pneumonia than the general population. The machine derived rule was curious,
Speaker 1: to say the least. Far from being protective, asthma can
Speaker 1: seriously complicate pulmonary illnesses, including pneumonia. Perplexed by this counterintuitive result,
Speaker 1: the researchers dug deeper, and what they found was troubling.
Speaker 1: They discovered that quote patients with the history of asthma
Speaker 1: who presented with pneumonia usually were admitted not only to
Speaker 1: the hospital, but directly to the i c U, the
Speaker 1: intensive care unit. Once in the i c U, asthmatic
Speaker 1: pneumonia patients went on to receive more aggressive care, thereby
Speaker 1: raising their survival rates compared to the general population. The rule,
Speaker 1: in other words, reflected a genuine pattern in the data,
Speaker 1: but the machine had confused correlation with causation quote, incorrectly
Speaker 1: learning that asthma lowers risk when in fact, asthmatics have
Speaker 1: much higher risk. It seems like we've got another wormhole here.
Speaker 1: And here the authors introduce an idea of of a
Speaker 1: curve of outcomes that they call a leptokurtic curve. That's
Speaker 1: a strange term, but basically what that means is if
Speaker 1: you are UM, if you're charting what types of outcomes
Speaker 1: you expect from a traditional system like just you know,
Speaker 1: humans looking at data versus a a complex automated system.
Speaker 1: The uh, the sort of the tails of the graph
Speaker 1: with the complex automated system will tend to be fatter,
Speaker 1: meaning you get more extreme events in the positive and
Speaker 1: negative space rather than a you know, a sort of
Speaker 1: rounder clustering of events in the you know, normal operation space,
Speaker 1: if that makes any sense. So, these kinds of unforeseen
Speaker 1: harms are some of the most worrisome types of things
Speaker 1: to expect coming out of robots and AI. But then
Speaker 1: the other one would be systemic harms and this is
Speaker 1: the last category of of harms they talk about. Uh
Speaker 1: the author's right quote. People have long assumed that robots
Speaker 1: are inherently neutral and objective, given that robots simply intake
Speaker 1: data and systematically output results, But they are actually neither.
Speaker 1: Robots are only as neutral as the data they're fed,
Speaker 1: and only as objective as the design choices of those
Speaker 1: who create them. When either bias or subjectivity infiltrates a
Speaker 1: systems inputs or design choices, it is in inevitably reflected
Speaker 1: in the system's outputs. This is your classic garbage in
Speaker 1: garbage out, problem, right, They go on, Accordingly, those responsible
Speaker 1: for overseeing the deployment of robots must anticipate the possibility
Speaker 1: that algorithmically biased applications will cause harms of this systemic
Speaker 1: nature to third parties. So uh, an example that's much
Speaker 1: discussed in this would be an AI trained to make
Speaker 1: decisions about granting loans by studying patterns of which loan
Speaker 1: applicants got their loans granted in the past. And a
Speaker 1: I like this could end up manifesting some type of
Speaker 1: bias that hurts people, like a racial bias in its
Speaker 1: loan assessments, because there was already a bias in the
Speaker 1: real world data set that it was trained on. So,
Speaker 1: in other words, AI that is trained on data from
Speaker 1: the real world, unless it is it is explicitly told
Speaker 1: not to do this, it will tend to reproduce and
Speaker 1: perpetuate any injustices, any inequalities that already exist. And the
Speaker 1: authors here give an example that is based on algorithmically
Speaker 1: derived insurance premiums that I think they're talking about auto
Speaker 1: insurance quote. A recent study by Consumer Reports found that
Speaker 1: contemporary premiums depended less on driving habits and increasingly on
Speaker 1: socioeconomic factors, including an individual's credit score. After analyzing two
Speaker 1: billion car insurance price quotes across approximately seven hundred companies,
Speaker 1: the study found that credit scores factored into insurance algorithms
Speaker 1: so heavily that perfect drivers with low credit scores often
Speaker 1: paid substantially more than terrible drivers with high scores. The
Speaker 1: studies findings raised widespread concerns that AI systems used to
Speaker 1: generate these quotes could create negative feedback loops that are
Speaker 1: hard to break. According to one expert quote, higher insurance
Speaker 1: prices for low income people can translate to higher debt
Speaker 1: and plummeting credit scores, which can mean use job prospects,
Speaker 1: which allows debt to pile up, credit scores to sink lower,
Speaker 1: and insurance rates to increase in a vicious cycle. Uh so,
Speaker 1: this is kind of a nightmare scenario, right, Like an
Speaker 1: AI that is too powerful and not explicitly protected against
Speaker 1: acquiring these types of biases could create these kind of
Speaker 1: computer enforced prisons in reality, like a machine code for
Speaker 1: perpetuating whatever state of the world, like whatever state the
Speaker 1: world was in when the AI was first deployed and
Speaker 1: then just entrenching it further and further. Yeah, And that
Speaker 1: kind of thing is especially scary because like, if there's
Speaker 1: a human making the decision, you can you can call
Speaker 1: up the human to a witness stand or ask them like, hey,
Speaker 1: why did you make the decision this way? But if
Speaker 1: it's an AI doing it, you could say like, hey,
Speaker 1: why why is it? Why are we getting this outcome
Speaker 1: that's you know, creating a sort of like cyclical prison
Speaker 1: out of reality, And they can just say, hey, you
Speaker 1: know it's the machine with the machine. You know it
Speaker 1: knows what it's doing. Yeah, yes, the machine it says
Speaker 1: I learned it from watching you dad, and you have
Speaker 1: that moment of shame. So I think these different categories
Speaker 1: that they that they bring up are really important for
Speaker 1: helping us kind of sort our ideas into into recognizable
Speaker 1: types for for ways that AI and robots could go
Speaker 1: wrong and could potentially cause harm that you would seek
Speaker 1: legal remedy for. And also they help identify the spaces
Speaker 1: that there's the most worry. I mean, for me, I
Speaker 1: think that would be like those last two cases, right,
Speaker 1: the unforeseen problems and the systemic problems are the ones
Speaker 1: where there's the most real danger, I think, and the
Speaker 1: most difficulty in trying to figure out how to solve it. Yeah,
Speaker 1: because we we kind of you know, train ourselves for
Speaker 1: to a certain extent and sort of culturally focus on
Speaker 1: the sky net problems, right, the really obvious, um uh
Speaker 1: situations where the robot car veers off the road in
Speaker 1: a dangerous way. But the situations where it is just
Speaker 1: perpetuating what we're already doing, where it's making choices in
Speaker 1: getting from point A to point B that don't violate
Speaker 1: anything we told it, but just are an uninventive and
Speaker 1: even harmful way of doing it. Uh. Yeah, that's that's
Speaker 1: that's harder to deal with. That's a type of misbehavior
Speaker 1: that you can't solve by just having Dan o'harla hay
Speaker 1: stand up and bellowing behave yourselves exactly. Um yeah, yeah, yeah,
Speaker 1: I mean I can't remember if that even worked. I
Speaker 1: just remember that was one of my favorite moments in
Speaker 1: that was RoboCop two, right, was it? Yeah? Well, I
Speaker 1: mean RoboCop one. I think we're already dealing with this
Speaker 1: problem of like the sort of like weird dynamics of
Speaker 1: machine culpability when ed two oh, nine like shoots that
Speaker 1: guy five hundred times in the boardroom during the demonstration,
Speaker 1: and then Dan O'Hurley. He's response to it is to
Speaker 1: turn to Ronnie Cox and say, I'm very disappointed, Dick.
Speaker 1: But anyway, well, I guess we're running running kind of long,
Speaker 1: so maybe we should call part one there. But we
Speaker 1: will resume this discussion about about robot justice and robot
Speaker 1: punishment in the next episode. That's right, We'll be back
Speaker 1: with more of this discussion in the meantime. If you
Speaker 1: would like to check out past episodes of Stuff to
Speaker 1: Blow Your Mind, uh, and the definitely worth checking out
Speaker 1: because we have lots of past episodes that deal with
Speaker 1: robots and AI. We have lots of episodes where we
Speaker 1: make RoboCop references, so they're all they're all there, go
Speaker 1: back and check them out. You can find our podcast
Speaker 1: wherever you get your podcasts. Just look for the Stuff
Speaker 1: to Blow your Mind podcast feed. UH. In that feed,
Speaker 1: we put out core episodes of the show on Tuesdays
Speaker 1: and Thursdays. Mondays, we have a little listener mail Wednesdays,
Speaker 1: so that's when we do the artifact shorty uh usually,
Speaker 1: and then on Friday's we do weird house cinema. That's
Speaker 1: our chance to sort of set most of the science
Speaker 1: aside and just focus on the films about rampaging robots.
Speaker 1: You just thinks, as always to our excellent audio producer
Speaker 1: Seth Nicholas Johnson. If you would like to get in
Speaker 1: touch with us with feedback on this episode or any other,
Speaker 1: to suggest a topic for the future, or just to
Speaker 1: say hello, you can email us at contact at stuff
Speaker 1: to Blow your Mind dot com. Stuff to Blow Your
Speaker 1: Mind is production of I Heart Radio. For more podcasts
Speaker 1: for my heart Radio, visit the i heart Radio app,
Speaker 1: Apple Podcasts, or wherever you're listening to your favorite shows.