WEBVTT - Punish the Machine, Part 1 0:00:00.160 --> 0:00:05.240 My welcome to Stuff to Blow Your Mind, the production 0:00:05.240 --> 0:00:13.960 of My Heart Radio. Hey you welcome to Stuff to 0:00:13.960 --> 0:00:16.680 Blow your Mind. My name is Robert Lamb and I'm 0:00:16.760 --> 0:00:19.640 Joe McCormick. And right before we started recording today, we 0:00:19.640 --> 0:00:22.480 were just talking about that iconic scene and returned to 0:00:22.560 --> 0:00:26.680 the Jedi where the droids are sent to the droid 0:00:26.840 --> 0:00:28.920 torture Chamber. Do you remember there? I guess it's not 0:00:29.000 --> 0:00:31.360 just a droid torture chamber. It's sort of like the uh, 0:00:32.440 --> 0:00:36.080 the droid onboarding center right where the you know, R 0:00:36.159 --> 0:00:38.320 two D two and C three PO have been given 0:00:38.320 --> 0:00:40.519 as gifts to job of the Hut and they go 0:00:40.640 --> 0:00:43.400 meet their new like droid boss and he's like, yeah, 0:00:43.479 --> 0:00:45.800 you're a feisty little one, and he's signing them in, 0:00:45.920 --> 0:00:48.760 but uh, he sees that R two D two is 0:00:48.800 --> 0:00:51.880 a is a bad robot who needs a discipline, and 0:00:52.120 --> 0:00:54.560 R two D two is confronted with these images of 0:00:54.640 --> 0:00:58.880 robots being punished with various corporal punishments, like one is 0:00:58.920 --> 0:01:02.000 getting stretched on a robot rack and another one is 0:01:02.040 --> 0:01:05.720 getting its feet burned. Yes, this is a this is 0:01:05.760 --> 0:01:09.399 a great scene one that that that definitely burns its 0:01:09.400 --> 0:01:11.840 way into your your brain as a as a young viewer, 0:01:11.920 --> 0:01:13.600 and maybe you don't think about it that much for 0:01:13.640 --> 0:01:16.160 a long time, but uh, it's it's still in there. 0:01:16.240 --> 0:01:19.720 It's it takes place in the bowels of Joba's palace 0:01:20.080 --> 0:01:23.960 on tattooing, and it's um, yeah, it's like droid intake 0:01:24.040 --> 0:01:27.520 but also droid corrections. It's there are there are a 0:01:27.600 --> 0:01:30.440 number of different departments that I think are converging here 0:01:31.160 --> 0:01:34.920 and and it it ultimately kind of raises some interesting 0:01:35.000 --> 0:01:40.240 questions about um about ethics and punishment and in crime, 0:01:40.319 --> 0:01:44.200 and certainly as it relates to two robots. Uh. Of course, 0:01:44.240 --> 0:01:46.440 one thing, this important distress here is like none of 0:01:46.480 --> 0:01:49.560 this was really intended in these scenes. This was about 0:01:49.840 --> 0:01:53.880 having droids doing things that humans would be doing to 0:01:53.920 --> 0:01:57.480 each other in other pieces of cinema, certainly things like 0:01:57.520 --> 0:02:01.040 old pirate movies or old and bad movies or what 0:02:01.120 --> 0:02:03.080 have you. I mean, that's kind of that's kind of 0:02:03.080 --> 0:02:06.600 Star Wars in a nutshell, right. Uh. This whole portion 0:02:06.680 --> 0:02:09.680 of Return of the Jedi is essentially a big pirate movie, 0:02:09.680 --> 0:02:13.679 a big swashbuckler set in an alien location. Oh yeah, 0:02:13.720 --> 0:02:16.399 the job of the Hut is a pirate captain yeah, 0:02:16.440 --> 0:02:20.320 but something interesting occurs when you replace the humans in 0:02:20.360 --> 0:02:25.320 these these tropic scenes with machines. Uh, and then you 0:02:25.360 --> 0:02:27.119 think about it, you know, you think about, well, why 0:02:27.200 --> 0:02:29.680 is that robot torturing the other? Like as if it 0:02:29.720 --> 0:02:33.040 makes perfect sense if it's humans doing it. But then 0:02:33.400 --> 0:02:35.919 the things that we create in our image when they're 0:02:35.919 --> 0:02:38.720 doing it, Suddenly we start seeing the flaws in our reasoning. 0:02:38.760 --> 0:02:41.959 Suddenly we start questioning, well, how does this whole system 0:02:42.040 --> 0:02:46.560 supposed to work? And maybe this whole system doesn't work. Well. Yeah, 0:02:46.560 --> 0:02:49.280 there are multiple levels of absurdities in the scene. One 0:02:49.360 --> 0:02:52.000 is the idea that this robot is just sort of 0:02:52.000 --> 0:02:54.120 like coolly telling are two that he is going to 0:02:54.200 --> 0:02:57.120 learn some discipline. But then the image that accompanies that 0:02:57.280 --> 0:03:00.400 is like, clearly just like extreme robot tor sure, like 0:03:00.440 --> 0:03:03.240 it's something way beyond what would have to do with 0:03:03.520 --> 0:03:06.160 with discipline in the real world. But then the other 0:03:06.240 --> 0:03:08.959 level of absurdity is that it's robots in the scene, 0:03:09.400 --> 0:03:12.280 but coming off of the issue of just like barbaric 0:03:12.360 --> 0:03:16.840 pirate torture directly and more to the broader question of 0:03:17.040 --> 0:03:21.520 robots and discipline and punishment. Uh, this is something that 0:03:21.560 --> 0:03:24.200 we actually wanted to talk about today because the issue 0:03:24.280 --> 0:03:28.080 of robot moral and legal agency is something I've been 0:03:28.160 --> 0:03:30.680 interested in for a long time. I've talked about it, 0:03:30.680 --> 0:03:32.320 It's come up on the show in the past and 0:03:32.440 --> 0:03:36.040 in briefer ways, um And today I wanted to come 0:03:36.040 --> 0:03:38.760 back and devote a full episode to the subject. I 0:03:38.760 --> 0:03:40.440 guess actually we're gonna be talking about this for a 0:03:40.480 --> 0:03:45.680 couple of episodes now. The question of as machines AI 0:03:45.800 --> 0:03:50.040 robots become more independent and act more like agents, more 0:03:50.120 --> 0:03:54.360 like humans do, how are we to understand their moral 0:03:54.520 --> 0:03:58.480 and legal culpability when they do something that harms people? 0:03:59.000 --> 0:04:03.120 And is there such a thing as robot punishment robot discipline? 0:04:03.160 --> 0:04:06.520 Do these concepts reflect anything that's achievable in the real 0:04:06.560 --> 0:04:09.480 world and practical and if so, how would any of 0:04:09.520 --> 0:04:11.760 this work? Yeah? I think one of the most interesting 0:04:11.800 --> 0:04:14.480 things about this topic is that it does force us 0:04:14.480 --> 0:04:17.520 to force a face off between what robots and AI 0:04:17.680 --> 0:04:21.599 actually are or will be, and how we think about them, indeed, 0:04:21.640 --> 0:04:26.560 how we anthropomorphize them. Um And perhaps it might be 0:04:26.600 --> 0:04:28.760 helpful to take a step back and think about something 0:04:29.240 --> 0:04:32.240 far less advanced as a robot and think something I 0:04:32.320 --> 0:04:36.799 think about something more like a hammer Okay, so everyone's 0:04:36.839 --> 0:04:38.960 heard the old adage that it's a poor carpenter who 0:04:38.960 --> 0:04:41.960 blames their tools, right, But of course we do this 0:04:42.040 --> 0:04:45.800 all the time. Uh, the hammer slips, it hits our fingers, 0:04:46.160 --> 0:04:48.240 and we may, at least in the heat of the moment, 0:04:48.600 --> 0:04:51.839 blame the hammer for the failure. Now, we may get 0:04:51.839 --> 0:04:54.440 over this quickly, but then again, we may decide that 0:04:54.800 --> 0:04:57.480 the hammer truly is at fault and it should be 0:04:57.600 --> 0:05:00.840 used less. We might also take this idea to a 0:05:00.920 --> 0:05:03.440 number of different extremes. We might decide that the hammer 0:05:03.760 --> 0:05:06.560 is not merely at fault but faulty, and then we're 0:05:06.680 --> 0:05:09.720 entitled to at least a refund for its purchase. Or 0:05:09.760 --> 0:05:12.680 we might decide that the hammer needs to actually be punished. 0:05:13.040 --> 0:05:15.600 And and this, of course is ridiculous. And yet the 0:05:15.680 --> 0:05:18.320 idea of punishing the hammer by say, putting it in 0:05:18.360 --> 0:05:21.479 the corner, or perhaps you have an old toolbox of 0:05:21.560 --> 0:05:25.479 shame that's just for the misbehaving tools, or maybe it's 0:05:25.600 --> 0:05:27.880 it's less thought out and you just throw the hammer 0:05:28.360 --> 0:05:31.200 across the yard as punishment for what it has done 0:05:31.240 --> 0:05:34.320 to you. Um. Again, these are ridiculous things to do, 0:05:34.920 --> 0:05:36.920 but the idea of doing them is not that far 0:05:36.960 --> 0:05:38.960 and from us. Um, those of you listening, you may 0:05:38.960 --> 0:05:41.240 have engaged in this sort of thing as well. You 0:05:41.320 --> 0:05:44.360 might also simply throw the tool away and otherwise perfectly 0:05:44.400 --> 0:05:47.479 good tool. Um. I know that I did this once 0:05:47.480 --> 0:05:49.880 with a knife sharpening gadget that caused me to cut 0:05:49.880 --> 0:05:53.160 my finger. My and my like reaction was this thing 0:05:53.240 --> 0:05:56.559 has now injured me, it has drawn my blood. Uh 0:05:56.600 --> 0:05:58.080 I am, I'm getting rid of it. It goes in 0:05:58.120 --> 0:06:01.440 the trash. It bore malice again to me. Yeah, or 0:06:01.480 --> 0:06:04.479 you know, I you know ultimately it's I mean, I 0:06:04.520 --> 0:06:06.960 mean you you get into arguments about different tools, like 0:06:07.120 --> 0:06:09.520 is this a dangerous tool? And in that case that 0:06:09.600 --> 0:06:12.880 was my reasoning. It's like this tool is dangerous. It's 0:06:12.920 --> 0:06:15.360 not enabling me to do what I want to do 0:06:15.440 --> 0:06:19.000 without drawing blood, So it goes in the trash. Um. 0:06:19.040 --> 0:06:20.760 But then there have been other cases where like I 0:06:20.800 --> 0:06:24.599 had a mandolin for slicing up carrots, and um, I 0:06:24.680 --> 0:06:27.440 like nicked my finger on it. But oh I I 0:06:27.560 --> 0:06:30.120 nicked my fingers. Things are brutal, they can be. But 0:06:30.640 --> 0:06:33.920 I nicked my finger, not using it but going into 0:06:33.960 --> 0:06:36.279 the drawer for something else. So I punished it by 0:06:36.320 --> 0:06:39.080 putting at the very bottom of the drawer, but I 0:06:39.120 --> 0:06:41.480 didn't throw it away. Uh huh. So I think if 0:06:41.480 --> 0:06:43.640 we all think, think back, you know, we have examples 0:06:43.640 --> 0:06:45.560 of this sort of thing from our from our life. 0:06:45.880 --> 0:06:48.040 Well sure, I mean, I'm going to talk in this 0:06:48.080 --> 0:06:50.960 episode about some of the ways that we mindlessly apply 0:06:51.279 --> 0:06:55.280 social rules to robots. But yeah, I think what you're 0:06:55.279 --> 0:06:58.120 illustrating here is that you don't even have to get 0:06:58.160 --> 0:07:01.280 to the robot agency stage of four people start doing that. 0:07:01.320 --> 0:07:05.559 I mean people mindlessly, to a lesser extent, mindlessly apply 0:07:05.800 --> 0:07:09.800 social rules and rules derived for managing human relationships to 0:07:10.200 --> 0:07:13.920 inanimate objects with no moving parts. Yeah. Yeah, you don't 0:07:13.920 --> 0:07:16.760 even have to get into a room ba or anything, 0:07:16.880 --> 0:07:18.280 or you know, you can deal with the hammer, the 0:07:18.320 --> 0:07:21.760 can opener. But we also you know, it's it's also 0:07:21.880 --> 0:07:25.120 a sad fact that many pet owners will punish an 0:07:25.160 --> 0:07:29.040 animal for a transgression transgression, but scientific evidence shows that 0:07:29.120 --> 0:07:32.840 this tends to not actually work, at least in most 0:07:32.880 --> 0:07:37.000 of the circumstances that it's used. Um So, you know, 0:07:37.080 --> 0:07:41.760 even it's not merely with with tools and inanimate objects, 0:07:41.760 --> 0:07:44.760 but even non human entities were liable to engage in 0:07:44.800 --> 0:07:49.320 this kind of discipline based thinking. Now, most of the 0:07:49.320 --> 0:07:52.600 studies I think that necessarily relate to dogs, if I 0:07:52.640 --> 0:07:54.480 remember correctly, And there's a lot going on here that 0:07:54.520 --> 0:07:57.520 doesn't relate directly to inanimate objects and robots, but it 0:07:57.560 --> 0:07:59.800 illustrates how we tend to approach the punishment of other 0:07:59.840 --> 0:08:03.120 agents and perceived agents. Well, yeah, there's a disconnect, and 0:08:03.120 --> 0:08:04.920 this will be highlighted in one of the papers we're 0:08:04.920 --> 0:08:07.000 going to talk about in in this pair of episodes, 0:08:07.040 --> 0:08:11.480 But there's a disconnect in that punishment is often logically 0:08:12.160 --> 0:08:15.880 characterized as serving one type of purpose, but then is 0:08:15.960 --> 0:08:19.040 applied more like it serves another type of purpose. So 0:08:19.080 --> 0:08:22.760 like it is logically explained as say a deterrent, right, 0:08:22.800 --> 0:08:25.560 I mean, if you if you're talking about uh, legal 0:08:25.600 --> 0:08:28.000 theories of punishment, one of the main things that people 0:08:28.000 --> 0:08:30.760 come up with is say, well, the remedy provided by 0:08:30.800 --> 0:08:33.640 the law is in order to punish the person who 0:08:33.640 --> 0:08:36.240 did the bad thing in order to send a message 0:08:36.280 --> 0:08:39.280 that people should not do this bad thing and thus 0:08:39.320 --> 0:08:42.600 maybe discourage other people from doing something similar in the future, 0:08:42.679 --> 0:08:45.520 or discourage the same person from doing it again. And 0:08:45.600 --> 0:08:48.120 if it were to actually serve that purpose, it's debatable 0:08:48.120 --> 0:08:50.520 in what cases it does actually serve that purpose. Maybe 0:08:50.600 --> 0:08:53.120 sometimes it does, but that is a you know, you 0:08:53.120 --> 0:08:56.920 could argue that's a rational, logical thing that prevents harm. 0:08:57.240 --> 0:09:00.200 But the way punishment is actually often inflict it in 0:09:00.240 --> 0:09:03.880 the real world seems to be more consistent with judgments 0:09:03.920 --> 0:09:08.199 based on like emotional satisfaction of the idea of having 0:09:08.240 --> 0:09:11.720 been wronged. Yeah. Yeah. And then also we get into 0:09:11.720 --> 0:09:15.520 this area where, uh, we have a couple of different 0:09:15.559 --> 0:09:20.199 factors encouraging traditions of discipline. Um, particularly if we look 0:09:20.200 --> 0:09:25.480 at a parenthood, which there's some crossover between discipline and 0:09:25.480 --> 0:09:28.600 parenthood and discipline and the criminal justice system. But uh, 0:09:28.640 --> 0:09:31.240 you know, uh, not everything is going to line up 0:09:31.280 --> 0:09:34.680 one to one here. But um, on the childhood example, 0:09:34.679 --> 0:09:37.120 it's been argued that parents use punishment first of all, 0:09:37.160 --> 0:09:40.680 because it's an emotional response out of anger and anger 0:09:40.840 --> 0:09:43.920 that may be mismanaged. But then on top of this, 0:09:44.720 --> 0:09:48.760 it's you know, something that's culturally passed down and punishment 0:09:48.840 --> 0:09:52.000 may seem to work. I was reading about this in 0:09:52.040 --> 0:09:56.079 a Psychology Today article by Michael Carson, pH d j D. 0:09:56.760 --> 0:09:59.240 And uh, this is what they said, quote, because the 0:09:59.320 --> 0:10:01.520 child is an bited in your presence it's easy to 0:10:01.559 --> 0:10:05.920 think they would be inhibited in your absence. Punishment produces politeness, 0:10:05.960 --> 0:10:10.880 not morality. Thus the inhibited, obedient child inadvertently reinforces the 0:10:10.920 --> 0:10:14.960 parents punitive behavior by acting obedient. For the sorts of 0:10:15.000 --> 0:10:18.880 parents who find obedient children reinforcing, Yeah, that raises an 0:10:18.880 --> 0:10:21.360 interesting question. I mean, I've been mainly thinking for this 0:10:21.400 --> 0:10:25.720 episode about about legal punishments, but like when it comes 0:10:25.760 --> 0:10:28.320 down to parenting, that's a very different kind of thing 0:10:28.400 --> 0:10:32.520 because both parenting in the legal system involved punishment, but 0:10:32.800 --> 0:10:36.440 parenting is not subject to a legal system, right, So 0:10:36.520 --> 0:10:40.040 there's no there is no systematized way by which justice 0:10:40.120 --> 0:10:42.760 is administered from a parent. It's just just I mean, 0:10:42.800 --> 0:10:44.120 I think a lot of times it's just sort of 0:10:44.160 --> 0:10:46.640 like whatever the parent can manage to do in the moment, 0:10:46.720 --> 0:10:49.880 because like the kids driving them crazy or something. Yeah, 0:10:49.960 --> 0:10:52.559 usually the child can't take it to a higher court. 0:10:53.320 --> 0:10:55.800 But I mean, I think you're absolutely right that whether 0:10:55.880 --> 0:10:58.840 you're talking about discipline administered by a parent or the 0:10:59.000 --> 0:11:02.480 justice system is whole, I'd say that both are probably 0:11:02.520 --> 0:11:07.520 based more on tradition and philosophy and less on a 0:11:07.600 --> 0:11:11.920 scientifically rigorous study of the most efficient ways to reduce harm. 0:11:11.960 --> 0:11:14.360 And one of the interesting things about thinking about how 0:11:14.520 --> 0:11:18.960 law could potentially be applied to harm caused by autonomous 0:11:19.000 --> 0:11:22.599 machines is that it may help give us some insights 0:11:22.640 --> 0:11:26.000 on ways that the justice system as it exists and 0:11:26.120 --> 0:11:30.439 is applied to humans today tends to behave irrationally already, 0:11:30.559 --> 0:11:34.080 like with respect to humans. Yeah, and again, this is 0:11:34.120 --> 0:11:37.079 what's so interesting about this this paper, I mean, well, 0:11:37.120 --> 0:11:39.720 the papers that we're going to discuss this topic in general, though, 0:11:40.160 --> 0:11:43.840 is if you start, you start comparing machine possibilities to 0:11:43.920 --> 0:11:47.880 human possibilities, and it's on one level of thought experiment 0:11:47.960 --> 0:11:50.760 in how you would hold machines responsible, but then it 0:11:50.800 --> 0:11:53.680 makes you rethink the way humans are held responsible. You know. 0:11:54.080 --> 0:11:56.240 It's like, um, you know, I think you have a 0:11:56.240 --> 0:12:00.880 pretty square away. Like if if an adult sells a 0:12:00.880 --> 0:12:03.800 pack of cigarettes to someone who's underage, right, but then 0:12:03.840 --> 0:12:06.680 one of a machine does the same thing, how do 0:12:06.720 --> 0:12:08.319 you treat the machine? Do you treat a machine like 0:12:08.360 --> 0:12:10.480 an adult? And then in trying to figure out how 0:12:10.520 --> 0:12:13.079 to treat this machine, does it make you rethink how 0:12:13.120 --> 0:12:15.720 you should be treating the adult who engaged in this behavior? 0:12:15.800 --> 0:12:17.720 I don't know. Yeah, And I think a lot of 0:12:17.720 --> 0:12:20.400 that will come down to our understanding of what the 0:12:20.480 --> 0:12:24.040 machine is capable of, like what kind of constraints it has, 0:12:24.440 --> 0:12:27.160 what type of what level of autonomy it seems to 0:12:27.200 --> 0:12:30.160 be operating at. I mean, again, Weirdly, even when people 0:12:30.200 --> 0:12:32.800 set out to define clear rules for what makes a 0:12:32.840 --> 0:12:35.679 machine culpable, there there's still going to be a lot 0:12:35.720 --> 0:12:39.880 of subjectivity in it. I'm looking at like legal definitions 0:12:39.920 --> 0:12:43.760 of what constitutes a robot versus just a machine, and 0:12:43.920 --> 0:12:47.120 some of these definitions involved things like, well, a robot 0:12:47.360 --> 0:12:51.080 feels like a social agent. So there's still, like, you know, 0:12:51.120 --> 0:12:53.960 an element of subjectivity. But I think that's correct in 0:12:54.040 --> 0:12:56.480 how we actually apply the term most of the time, right, 0:12:56.520 --> 0:13:00.240 Like something is like a gut feeling about how this 0:13:00.280 --> 0:13:03.720 machine is behaving in your world. Is it acting more 0:13:03.840 --> 0:13:06.800 like a fixed, you know, brainless machine or is it 0:13:06.840 --> 0:13:11.280 acting a little bit more like a person. So while 0:13:12.000 --> 0:13:14.240 it would be one thing if if it were basically 0:13:14.240 --> 0:13:17.280 a cigarette vending machine that was selling to children, but 0:13:17.320 --> 0:13:19.959 if it were a machine that went door to door 0:13:20.440 --> 0:13:23.480 and and rang the doorbell and then asked for the 0:13:23.559 --> 0:13:25.679 children so they could sell them cigarettes, that would be 0:13:25.679 --> 0:13:27.559 a different matter. I mean, yeah, I mean, I think 0:13:27.600 --> 0:13:31.080 that would require different types of remedies probably, Yeah, I 0:13:31.120 --> 0:13:32.760 mean I think a lot of people would probably look 0:13:32.760 --> 0:13:35.480 at the cigarette vending machine and say, where was the 0:13:35.559 --> 0:13:38.200 vending machine placed? Why was it in a place that 0:13:38.320 --> 0:13:41.600 children could have access to it? Rather than attacking the 0:13:41.600 --> 0:13:44.240 fundamentals of the machine itself. If it's going door to 0:13:44.280 --> 0:13:46.600 door and giving cigarettes to kids, yeah, then people are 0:13:46.600 --> 0:13:50.160 probably going to attack the fundamentals and the moral character 0:13:50.559 --> 0:13:53.760 of the robot. Right, you have not attacked the robot itself. 0:13:53.800 --> 0:14:00.319 It would just rob justice in somebody's front yard. Yeah, 0:14:01.200 --> 0:14:05.080 thank Alright, So I guess I want to introduce one 0:14:05.120 --> 0:14:06.720 of the papers we're gonna be looking at in this 0:14:06.760 --> 0:14:10.200 pair of episodes, and it is by Mark A. Limley 0:14:10.360 --> 0:14:14.520 and Brian Casey called Remedies for Robots, published in the 0:14:14.600 --> 0:14:18.920 University of Chicago Law Review in twenty nineteen. And this 0:14:19.000 --> 0:14:22.600 is a big paper. It's like eighty something pages long 0:14:22.640 --> 0:14:26.000 with a with a lot of different interesting, uh thoughts 0:14:26.000 --> 0:14:27.760 in it. We're not going to be able to cover 0:14:27.880 --> 0:14:31.200 the entire thing in depth, but it's worth looking up. 0:14:31.240 --> 0:14:33.240 You can easily find a full PDF of it if 0:14:33.240 --> 0:14:35.920 you want to read it in depth. And we're gonna 0:14:35.920 --> 0:14:38.440 look at some of the larger framework it lays out, 0:14:38.440 --> 0:14:41.320 and then some interesting thoughts raised to buy it, But 0:14:41.400 --> 0:14:43.960 to kick it off here the the author's right quote, 0:14:44.000 --> 0:14:48.960 what happens when artificially intelligent robots misbehave? The question is 0:14:49.000 --> 0:14:54.239 not just hypothetical. As robotics and artificial intelligence systems increasingly 0:14:54.360 --> 0:14:57.920 integrate into our society, they will do bad things. We 0:14:58.000 --> 0:15:01.000 seek to explore what remedies the all can and should 0:15:01.080 --> 0:15:05.480 provide once a robot has caused harm. Now, obviously we're 0:15:05.480 --> 0:15:08.880 going to be focused less on the like minute particulars 0:15:08.920 --> 0:15:11.760 of US legal precedent here and more on the broader 0:15:11.800 --> 0:15:16.520 issues they raise about robot agency, robot moral decision making 0:15:16.640 --> 0:15:19.960 and how that interacts with harm and morality and justice. 0:15:20.600 --> 0:15:23.440 And the authors start out in their introduction by giving 0:15:23.440 --> 0:15:26.520 what I think is a really fantastic example of how 0:15:26.560 --> 0:15:31.360 an autonomous robot with behaviors guided by machine learning, which 0:15:31.400 --> 0:15:34.720 is how you know increasingly most robots are going to 0:15:34.760 --> 0:15:37.840 be controlled, can end up doing things that are the 0:15:37.880 --> 0:15:41.480 exact opposite of what was intended. So this case that 0:15:41.520 --> 0:15:44.000 they cite is based on a true story from a 0:15:44.040 --> 0:15:48.680 presentation at the eleventh Annual Stanford e Commerce Best Practices 0:15:48.800 --> 0:15:54.440 Conference in June, and it goes like this quote. Engineers 0:15:54.520 --> 0:15:59.480 training and artificially intelligent self flying drone were perplexed. They 0:15:59.520 --> 0:16:01.840 were trying to get the drone to stay within a 0:16:01.920 --> 0:16:05.920 predefined circle and head toward its center. Things were going 0:16:05.960 --> 0:16:09.560 well for a while. The drone received positive reinforcement for 0:16:09.600 --> 0:16:12.600 its successful flights, and it was improving its ability to 0:16:12.720 --> 0:16:17.800 navigate toward the middle quickly and accurately. Then suddenly things changed. 0:16:18.160 --> 0:16:20.480 When the drone near the edge of the circle, it 0:16:20.520 --> 0:16:24.360 would inexplicably turn away from the center, leaving the circle. 0:16:25.040 --> 0:16:28.640 What went wrong. After a long time spent puzzling over 0:16:28.680 --> 0:16:32.160 the problem, the designers realized that whenever the drone left 0:16:32.200 --> 0:16:36.200 the circle during tests they had turned it off. Someone 0:16:36.240 --> 0:16:38.920 would then pick it up and carry it back into 0:16:38.920 --> 0:16:42.680 the circle to start again. From this pattern, the drones 0:16:42.760 --> 0:16:46.760 algorithm had learned correctly that when it was sufficiently far 0:16:46.880 --> 0:16:49.600 from the center, the optimal way to get back to 0:16:49.680 --> 0:16:53.080 the middle was to simply leave it all together. As 0:16:53.120 --> 0:16:55.840 far as the drone was concerned, it had discovered a 0:16:55.920 --> 0:16:59.800 worm hole. Somehow, flying outside of the circle could be 0:17:00.080 --> 0:17:03.240 led upon to magically teleport it closer to the center, 0:17:03.680 --> 0:17:06.520 and far from violating the rules instilled in it by 0:17:06.600 --> 0:17:10.080 its engineers, the drone had actually followed them to a t. 0:17:10.840 --> 0:17:14.399 In doing so, however, it had discovered an unforeseen shortcut, 0:17:14.680 --> 0:17:18.840 one that subverted its designers true intent. That's really good. 0:17:18.880 --> 0:17:22.760 That's that's it's as Yes, I love it. This is 0:17:22.800 --> 0:17:26.239 such a great example of how robots can fail in 0:17:26.320 --> 0:17:30.280 ways that are perfectly logical for the machines themselves, but 0:17:30.440 --> 0:17:33.240 hard for humans to predict in advance, because we're not 0:17:33.400 --> 0:17:37.119 understanding how our you know, our programming or the data 0:17:37.160 --> 0:17:40.480 sets were training it on is biasing its behavior in 0:17:40.840 --> 0:17:43.760 ways that that are strange to us. And in this case, 0:17:43.800 --> 0:17:48.080 of course, such a malfunction is harmless. But as autonomous 0:17:48.119 --> 0:17:51.600 machines become more and more integrated into the broader culture, 0:17:52.040 --> 0:17:56.960 not just in controlled contained locations like factory floors and laboratories, 0:17:56.960 --> 0:17:59.320 but in the wild, so on the streets and in 0:17:59.359 --> 0:18:03.720 our homes and stuff, there will inevitably be cases where 0:18:03.800 --> 0:18:07.800 robots fail like this and fail in ways that cause 0:18:08.240 --> 0:18:12.360 catastrophic harm to people. Yeah, and plus, as an aside, 0:18:12.359 --> 0:18:14.440 we we have to realize that even in cases where 0:18:14.440 --> 0:18:17.040 the machines have not failed, there will be gray areas 0:18:17.359 --> 0:18:20.280 in which it's not completely clear, and an argument could 0:18:20.320 --> 0:18:23.040 be made in these cases for machine culpability with a 0:18:23.119 --> 0:18:26.560 variety of intense and possible biases in place. Oh yeah, 0:18:26.560 --> 0:18:28.719 that's another thing these authors talk about that there can 0:18:28.760 --> 0:18:32.880 be all kinds of ways that that robotics and AI 0:18:32.920 --> 0:18:36.440 could end up causing extreme harm to people without ever 0:18:36.520 --> 0:18:39.720 doing anything that if a human did it would be illegal. 0:18:40.320 --> 0:18:43.760 What one example they give is like if, um, if 0:18:43.920 --> 0:18:48.520 Google were to suddenly change its Google Maps algorithm so 0:18:48.560 --> 0:18:52.960 that it routed all of the city's traffic through your neighborhood. Like, 0:18:53.440 --> 0:18:56.520 nothing illegal about that. It doesn't like commit a crime 0:18:56.560 --> 0:18:59.840 against you, but this is going to drastically negatively in 0:19:00.040 --> 0:19:02.480 hacked your quality of life. And it's a decision that's 0:19:02.560 --> 0:19:05.400 just like a could be a quirk of an algorithm 0:19:05.440 --> 0:19:08.960 in a machine. Now, this paper in particular concerns the 0:19:09.080 --> 0:19:13.159 legal concept of remedies. So I was reading about remedies. 0:19:13.320 --> 0:19:16.600 A common legal definition that I found is quote the 0:19:16.680 --> 0:19:20.480 means to achieve justice in any matter in which legal 0:19:20.600 --> 0:19:24.640 rights are involved. Or in the words of Limely and Casey, 0:19:24.680 --> 0:19:27.040 what do I get when I win? Right? So, if 0:19:27.040 --> 0:19:29.360 you if you take somebody to court because you say 0:19:29.359 --> 0:19:33.119 they have harmed you. Whatever outcome you're seeking from that 0:19:33.119 --> 0:19:36.800 that court case is the remedy. So usually when a 0:19:36.800 --> 0:19:39.480 court case finds that somebody has done something wrong to 0:19:39.560 --> 0:19:42.760 harm somebody else, the court responds to the finding of 0:19:42.800 --> 0:19:46.360 guilt or blame by enforcing this remedy. And common remedies 0:19:46.359 --> 0:19:49.560 would include a payment of money, right a guilty defendant 0:19:49.560 --> 0:19:52.600 has to pay money to the plaintiff, a punishment of 0:19:52.640 --> 0:19:55.520 the offender like maybe they go to jail, or a 0:19:55.520 --> 0:19:58.399 court order to do something or not to do something. 0:19:58.520 --> 0:20:01.719 For example, somebody is ordered not to drive a vehicle, 0:20:01.960 --> 0:20:04.119 or they are ordered not to go within a hundred 0:20:04.160 --> 0:20:07.040 feet of somebody else or something like that. Yeah, or 0:20:07.160 --> 0:20:09.720 their their eyeball is removed, or they have to spend 0:20:09.880 --> 0:20:12.440 a night in a haunted house something like that. Hopefully 0:20:12.480 --> 0:20:15.120 not in modern law. But wait a minute, there are 0:20:15.160 --> 0:20:18.280 some sometimes you do read about some really strange like 0:20:18.600 --> 0:20:21.040 remedies that are ordered by judges like I order you 0:20:21.080 --> 0:20:22.880 to I don't I don't know, to wear chicken suit 0:20:22.960 --> 0:20:26.280 or something right like, Yeah, there's some judges who like 0:20:26.359 --> 0:20:30.159 to get creative. It seems weird. Yeah, I wonder I 0:20:30.200 --> 0:20:31.920 have there have been in cases where someone has to 0:20:31.920 --> 0:20:33.639 spend a night in a haunted house due to a 0:20:33.640 --> 0:20:35.479 court order. I think that would be a good setup 0:20:35.520 --> 0:20:37.960 for a film. But anyway, so when you start looking 0:20:38.000 --> 0:20:41.159 at the idea of remedies, remedies are complicated because they 0:20:41.200 --> 0:20:45.480 involve different types of implied satisfaction on behalf of the 0:20:45.560 --> 0:20:49.639 victim or plaintiff. And some are very clear and material, 0:20:49.840 --> 0:20:52.520 and others are much more abstract. So the very the 0:20:52.520 --> 0:20:55.120 ones that are very clear and material are like, if 0:20:55.160 --> 0:20:58.240 I hit your car with my car and I'm clearly 0:20:58.280 --> 0:21:00.800 at fault, I need to give you a payment of 0:21:00.840 --> 0:21:04.000 cash to offset the material losses to the value of 0:21:04.000 --> 0:21:07.440 your car, right. But then other times it's it's more abstract. 0:21:07.520 --> 0:21:10.720 It's you know, punishment of an offender to give the 0:21:10.840 --> 0:21:15.440 victim a sense of justice or to allegedly discourage someone 0:21:15.560 --> 0:21:18.320 from committing this type of harm or offense in the future. 0:21:18.840 --> 0:21:21.040 And then the authors right that things get way more 0:21:21.119 --> 0:21:24.479 complicated when you bring robots and AI into the picture. 0:21:24.880 --> 0:21:27.760 For example, if you're trying to give a court order 0:21:27.760 --> 0:21:30.159 to a person you know saying like you shall not 0:21:30.280 --> 0:21:32.720 drive a car, you shall not you know, come within 0:21:32.760 --> 0:21:34.840 a hundred feet of this person. You can do so 0:21:34.920 --> 0:21:37.600 in natural language, you can like speak a sentence to 0:21:37.680 --> 0:21:41.000 them and you can expect them to understand. But how 0:21:41.040 --> 0:21:44.320 do you get a court to give an order to 0:21:44.440 --> 0:21:49.080 a robot not to do something? Most robots don't have 0:21:49.240 --> 0:21:52.000 natural language processing, and even if they do, a lot 0:21:52.040 --> 0:21:54.880 of times it's not that good. So you might think, okay, 0:21:54.920 --> 0:21:57.280 well you just you know, you give the court order 0:21:57.359 --> 0:22:00.800