Speaker 1: Welcome to Stuff to Blow Your Mind, the production of
Speaker 1: My Heart Radio. Hey, welcome to Stuff to Blow Your Mind.
Speaker 1: My name is Robert Lamb and I'm Joe McCormick, and
Speaker 1: we're back for part two of our talk about punishing
Speaker 1: the robot. We're we're back here to uh to tell
Speaker 1: the robot he's been very bad. Now. In the last episode,
Speaker 1: we talked about the idea of legal agency and culpability
Speaker 1: for robots and other intelligent machines, and for a quick
Speaker 1: refresher on on some of the stuff we went over,
Speaker 1: we talked about the idea that as robots and AI
Speaker 1: become more sophisticated and thus in some ways or in
Speaker 1: some cases more independent and unpredictable, and as they integrate
Speaker 1: more and more into the wild of human society, they
Speaker 1: are just inevitably going to be situations where AI and
Speaker 1: robots do wrong and cause harm to people. Now, of course,
Speaker 1: when a human is wrong and causes harm to another human,
Speaker 1: we have a legal system through which the victim can
Speaker 1: seek various kinds of remedies, And we talked in the
Speaker 1: last episode about the idea of remedies that the simple
Speaker 1: version of that is the remedy is what do I
Speaker 1: get when I win in court? So that can be
Speaker 1: things like monetary rewards. You know, I ran into your
Speaker 1: car with my car, so I pay you money, or
Speaker 1: it can be punishment, or it can be court orders
Speaker 1: like commanding or restricting the behavior of the perpetrator. And
Speaker 1: so we discussed the idea that as robots become more
Speaker 1: unpredictable and more like human agents, more sort of independent,
Speaker 1: and more integrated into society, it might make sense to
Speaker 1: have some kind of system of legal remedies for when
Speaker 1: robots cause harm or commit crimes. But also, as we
Speaker 1: talked about last time, this is much easier said than done.
Speaker 1: It's going to present tons of new problems because our
Speaker 1: legal system is in many ways not equipped to deal
Speaker 1: with with defendants and situations of this kind. And this
Speaker 1: may cause us to ask questions about how we already
Speaker 1: think about culpability and blame and and punishment in the
Speaker 1: legal system. And so in the last episode we talked
Speaker 1: about one big legal paper that we're going to continue
Speaker 1: to explore in this one. It's by Mark A. Limley
Speaker 1: and Brian Casey in the University of Chicago Law Review
Speaker 1: from twenty nineteen called remedies for robots, So I'll be
Speaker 1: referring back to that one a good bit throughout this
Speaker 1: episode too. Now. I think when we left off last time,
Speaker 1: we had mainly been talking about sort of trying to
Speaker 1: categorize the different sorts of harm that could be done
Speaker 1: by robots or AI intelligent machines, and so we talked
Speaker 1: about some things like unavoidable harms and deliberate least cost harms.
Speaker 1: These are sort of going to be unavoidable parts of
Speaker 1: having something like autonomous vehicles, right if you have cars
Speaker 1: driving around on the road, Like, even if they're really
Speaker 1: really good at minimizing harm, there's still going to be
Speaker 1: some cases where there's just no way harm could be
Speaker 1: avoided because their cars. Another would be defect driven harms.
Speaker 1: That's pretty straightforward, that's just where the machine malfunctions or
Speaker 1: breaks in some way. Another would be misuse harms. That's
Speaker 1: where the machine is used in a way that is harmful.
Speaker 1: And in those cases it can be usually pretty clear
Speaker 1: who's at fault. It's the person who misused the machine.
Speaker 1: But then there are a couple of other categories that
Speaker 1: where things get really tricky, which are unforeseen harms and
Speaker 1: systemic harms and in the case of unforeseen harms. One
Speaker 1: example we talked about in the last episode was the
Speaker 1: drone that invented a wormhole. So, you know, people were
Speaker 1: trying to train a drone to move towards like an
Speaker 1: autonomous flying vehicle, to move towards the center of a
Speaker 1: circular area. But the drone started doing a thing where
Speaker 1: when it got sufficiently far away from the center of
Speaker 1: the circle, it would just fly out of the circle altogether.
Speaker 1: And so it seems kind of weird at first, like, Okay,
Speaker 1: why would it be doing that? But then what the
Speaker 1: researchers realized was that whenever it did that, they would
Speaker 1: turn it off off and then they would move it
Speaker 1: back into the circle to start it over again. So,
Speaker 1: from the machine learning point of view of the drone itself,
Speaker 1: it had discovered like a like a time space warp
Speaker 1: that you know. So so it was doing this thing
Speaker 1: that made no sense from a human perspective, but actually
Speaker 1: was it was following its programming exactly. Now for an example, uh,
Speaker 1: sort of a thought experiment of how this could become lethal.
Speaker 1: There's an example that is stuck in my head. I
Speaker 1: can't recall where I heard this who presented this idea?
Speaker 1: And I kind of had it in my head that
Speaker 1: it came from Max tag Mark. But I did some
Speaker 1: searching around in my notes and some searching around and
Speaker 1: one of his books, and I couldn't find it. Perhaps
Speaker 1: you can help refresh me. You remember maybe you remember this, Joe,
Speaker 1: But the idea of the the AI that is running
Speaker 1: deciding how much oxygen needs to be in a train station,
Speaker 1: it didn't given time. Oh this sounds familiar. I don't
Speaker 1: know the answer, but a lot of these thought experiments
Speaker 1: tend to trace back to Nick Bostrom, so I wouldn't
Speaker 1: be surprised if in there. But but go ahead, right, okay,
Speaker 1: as I remember it. The way it works is you have, um,
Speaker 1: you have this AI that's in charge of of making
Speaker 1: sure there's enough oxygen in the train station for when
Speaker 1: humans are there, and it seems to have learned this fine.
Speaker 1: And when humans are there to get on the train,
Speaker 1: everything goes goes well, everybody's breathing fine. And then one day, uh,
Speaker 1: the train arrives a little late or it leaves a
Speaker 1: little right late. You get which whatever it is, and
Speaker 1: there's not enough oxygen and people die and then it
Speaker 1: turns out that the train was not basing its decision
Speaker 1: on when people were there, but it was basing it
Speaker 1: on a clock in the train station, like what train
Speaker 1: it was? Um, And I may be mangling this horribly,
Speaker 1: but you know another way of illustrating the point that
Speaker 1: machine learning could end up, you know, latching onto shortcuts
Speaker 1: or heuristic devices. Uh. That would just seem completely insane
Speaker 1: to a quote unquote logical human mind, but might make
Speaker 1: sense within the framework of the AI. Right. They worked
Speaker 1: in training cases, and it doesn't understand because it doesn't
Speaker 1: have common sense, it doesn't understand why they wouldn't work
Speaker 1: in another case. There was actually a real world case
Speaker 1: that we talked about in Part one where there was
Speaker 1: an attempt to do some machine learning on what risk
Speaker 1: factors would would make a pneumonia case admitted to the
Speaker 1: hospital have a higher or lower chance of survival. And
Speaker 1: one thing that a machine learning algorithm determined was that
Speaker 1: asthma meant that you were you were better off when
Speaker 1: you got pneumonia if you had asthma. But actually the
Speaker 1: reason for that that that isn't true. Actually, the reason
Speaker 1: for that is that if you have asthma, you're a
Speaker 1: higher risk case for pneumonia, so you've got more intensive
Speaker 1: treatment in the hospital and thus had better outcomes on
Speaker 1: the data set that the algorithm was trained on. But
Speaker 1: the algorithm came up with this completely backwards uh failure
Speaker 1: to understand the difference between correlation and causation. There it
Speaker 1: made it look like asthma was a superpower. Now, of course,
Speaker 1: if you take that kind of shortsighted algorithm and you
Speaker 1: make it god, then it will say, oh, I've just
Speaker 1: got to give everybody asthma, so so we'll have a
Speaker 1: better chance of surviving. The point is it can be
Speaker 1: hard to imagine in advance all the cases like this
Speaker 1: that would arise when you've got a world full of
Speaker 1: robots and and AI is running around in it that
Speaker 1: are trained on machine learning. Basically, they're just a number
Speaker 1: of less sort of soft sky nets that you couldn't
Speaker 1: possibly predict, you know, like the skynet scenario being sort
Speaker 1: of like a robots decide that one to end all war,
Speaker 1: and um, you know, humans causal war, therefore end all humans,
Speaker 1: that sort of thing. But there's so many different like
Speaker 1: lesser versions of it. It It could also be destructive or
Speaker 1: annoying or just get in the way of effectively using
Speaker 1: AI for whatever we turn to it for. Uh yeah, yeah.
Speaker 1: To come to an example that is definitely used by
Speaker 1: Nick Bostrom, the paper clip maximizer. You know, a robot
Speaker 1: that is designed to make as many paper clips as
Speaker 1: it can, and it just it looks at your body
Speaker 1: and says, hey, that's full of matter. Those could be
Speaker 1: paper clips. Yeah, yeah, yeah, that would be that would
Speaker 1: be quite an apocalypse. Now, before we get back into
Speaker 1: the main subject and talking about this limly in Casey
Speaker 1: paper with with robots as offenders, there was one thing
Speaker 1: that was interesting I came across. It was just a
Speaker 1: brief footnote in their paper, but about the question of
Speaker 1: what about if the robot is the plaintiff in a case? Uh.
Speaker 1: They said, it's it is possible to imagine a robot
Speaker 1: as a plaintiff in a court case because of course robots,
Speaker 1: you know, can be injured by humans. And they cited
Speaker 1: a bunch of examples of news stories of humans just
Speaker 1: intentionally like torturing and being cruel two robots like that.
Speaker 1: They cited one news article fromen about people just aggressively
Speaker 1: kicking food delivery robots, and then they share another story actually,
Speaker 1: remember this one from the news from about a Silicon
Speaker 1: Valley security robot that was just violently attacked by a
Speaker 1: drunk man in a parking garage. I don't remember this one,
Speaker 1: but I can imagine how it went down. Yeah, exactly.
Speaker 1: So they say that in a case like this, this
Speaker 1: is actually pretty straightforward as a property crime. I mean,
Speaker 1: unless we start getting into a scenario where we're really
Speaker 1: seeing robots as like human beings with their own like
Speaker 1: consciousness and interests and all that, the attacks against robots
Speaker 1: are really probably just property crimes against the owner of
Speaker 1: the robot. It's like, you know, attacking somebody's computer or
Speaker 1: their car or something potentially. But that we'll get into
Speaker 1: some stuff a little later that I think shows some
Speaker 1: other directions that could go in as well, you know, um,
Speaker 1: you know, especially considered the awesome possibility of robots owning themselves. Yeah,
Speaker 1: and and that's obviously a very different world, I mean,
Speaker 1: where you get into the idea like does a robot
Speaker 1: actually have rights? Um, which is not. That's sort of
Speaker 1: beyond the horizon of what's explored in in this paper itself.
Speaker 1: This paper is more focused on like the kinds of
Speaker 1: ro bots that you can practically imagine within the next
Speaker 1: few decades. And and in those cases, it seems like
Speaker 1: all of the really thorny stuff would probably be in
Speaker 1: robots as offenders rather than robots as victims of crimes. Right.
Speaker 1: But to your point, like the the initial crimes against
Speaker 1: robot that we can in the robots that we can
Speaker 1: imagine would be stuff like drunk people pushing them over
Speaker 1: things like that. Yeah, or just like a human driver
Speaker 1: and a human powered vehicle hitting an autonomous vehicle. You know,
Speaker 1: right now, As I mentioned in the last episode, this
Speaker 1: is a very big paper and we're not gonna have
Speaker 1: time to get into every avenue they go down in it.
Speaker 1: But I just wanted to go through uh and and
Speaker 1: mention some ideas that stuck out to me is interesting
Speaker 1: that they discuss. And one thing that really fascinated me
Speaker 1: about this was that the idea of robots as possible
Speaker 1: agents in in a legal context UH brings to the
Speaker 1: for a philosophical argument that has existed in the realm
Speaker 1: of substance of law for a while. Uh. And I'll
Speaker 1: try not to be too dry about this, but I
Speaker 1: think it actually does get to some really interesting philosophical territory. Uh.
Speaker 1: And this is the distinction between what Limbly and Casey
Speaker 1: call the normative versus economic interpretations of substantive law. Again,
Speaker 1: complicated philosophical and legal distinction. I'll try to do my
Speaker 1: best to sum it up simply. So. The normative perspective
Speaker 1: on substantive law says that the law is a prohibition
Speaker 1: against doing something bad. So when something is against the law,
Speaker 1: that means you shouldn't do it, And we would stop
Speaker 1: the offender from doing the thing that's against the law
Speaker 1: if we could. But since we usually can't stop them
Speaker 1: from doing it, often because it already happened, the remedy
Speaker 1: that exists, You know that maybe paying damages to the
Speaker 1: victim or something like that is a is a an
Speaker 1: attempt to right the wrong, in other words, to do
Speaker 1: the next best thing to undoing the harm in the
Speaker 1: first place. So, basically, getting into the idea of negative reinforcement,
Speaker 1: somebody or something did something bad, we can't we couldn't
Speaker 1: stop them from doing something bad, but we can try
Speaker 1: and and give them stimulus that would make them not
Speaker 1: do it again. Be that economic or otherwise. Well, yes,
Speaker 1: but I think what you're saying uh could actually apply
Speaker 1: to both of these conditions I'm going to talk about.
Speaker 1: So I think maybe the distinction comes in about whether
Speaker 1: the whether there is such a thing as an inherent prohibition.
Speaker 1: So the thing that's uh operative in the normative view
Speaker 1: is that the thing that's against the law is a
Speaker 1: thing that should not be done, and thus the remedy
Speaker 1: is an attempt to try to fix the fact that
Speaker 1: it was done in the first place. The the economic
Speaker 1: view is the alternative here, and the way they sum
Speaker 1: that up is there is no such thing as forbidden conduct. Rather,
Speaker 1: a substantive law tells you what the cost of the
Speaker 1: conduct is. Does that distinction make any more sense? Yes, yes,
Speaker 1: So it's basically the first version is doing crimes is bad. Um.
Speaker 1: The second one is doing crimes is expensive. So it's
Speaker 1: the first is crimes should not be done, and the
Speaker 1: second one is crimes can be done if you can
Speaker 1: afford it. Yes, exactly so. In Limley, in cases words,
Speaker 1: quote damages on this view the economic view are simply
Speaker 1: a cost of doing business. One we want defendants to internalize,
Speaker 1: but not necessarily to avoid the conduct altogether. And now
Speaker 1: you might look at this and think, oh, okay, well,
Speaker 1: so the economic view is just like a psychopathic way
Speaker 1: of looking at things, And in a certain sense, you
Speaker 1: could look at it that as like if you're calculating
Speaker 1: what's the economic cost of murder? Then yeah, okay, that
Speaker 1: does just like that's evil, that's like psychopathic. But they're
Speaker 1: actually all kinds of cases we're thinking about. The economic
Speaker 1: view makes more sense of the way we actually behave
Speaker 1: And they use the example of stopping at a traffic light. Yes,
Speaker 1: so read from Limely in casey here quote. Under the
Speaker 1: normative view, a red light stands as a prohibition against
Speaker 1: traveling through an intersection, with the remedy being a ticket
Speaker 1: or a fine against those who are caught breaking the prohibition.
Speaker 1: We would stop you from running the red light if
Speaker 1: we could, but because policing every intersection in the country
Speaker 1: would be impossible, we instead punish those we do catch
Speaker 1: in hopes of deterring others. So in this first case,
Speaker 1: you running a red light is bad, you should not
Speaker 1: do it, and the cost of doing it, you know,
Speaker 1: the punishment you face for doing it is an attempt
Speaker 1: to right that wrong. But then they say, under the
Speaker 1: economic view, however, and absolute prohibition against running red lights
Speaker 1: was never the intention. Rather, the red light merely signals
Speaker 1: a consequence for those who do, in fact choose to
Speaker 1: travel through the intersection. As in the first instance, the
Speaker 1: remedy available is a fine or a ticket. But under
Speaker 1: this view, the choice of whether or not to violate
Speaker 1: the law depends on the willingness of the lawbreaker to
Speaker 1: accept the penalty. So in the case of a red light,
Speaker 1: well that that might make more sense if you're like
Speaker 1: sitting at a red light and you look around and
Speaker 1: there are no other cars anywhere near you, and you've
Speaker 1: you've got a clear view of the entire intersection and
Speaker 1: the red lights not changing, and you think maybe it's broken,
Speaker 1: and you're just like, Okay, I'm I'm just going to
Speaker 1: drive through. Well, if if you reach that point where
Speaker 1: you're like, I think it's broken, that I feel like
Speaker 1: that's a slightly different case. But if you're just like,
Speaker 1: nobody's watching, I'm gonna do it, um and and and
Speaker 1: the light isn't taking an absurd amount of time or
Speaker 1: longer than you're you're accustomed to, Yeah, I don't know
Speaker 1: how the the belief that the light is broken would
Speaker 1: factor into that, but yeah it is. I mean one
Speaker 1: thing that I think is clear that in it's that
Speaker 1: in many cases there are people, especially I think companies
Speaker 1: and corporations that operate on the economic view. And it
Speaker 1: is something that I think people generally look at and say, Okay,
Speaker 1: that that's kind of grimy. Like it like a company
Speaker 1: that says, Okay, there is a fine for not obeying
Speaker 1: this environmental regulation, and we're going to make more money
Speaker 1: by violating the regulation than we would pay in the fine. Anybody,
Speaker 1: So we're just gonna pay it. Yeah, you hear about
Speaker 1: that with factories, for instance, where where there'll, yeah, there'll
Speaker 1: be some situation where that the fine is not significant
Speaker 1: enough to really be at a turrent. It's just a
Speaker 1: fit for them breaking that that mandate being called on it. Occasionally.
Speaker 1: It's just the cost of doing business. Right. Uh. So
Speaker 1: there's a funny way to describe this point of view
Speaker 1: that the authors bring up here that they call it
Speaker 1: the bad man theory. And this comes from Justice Oliver
Speaker 1: Wendell Holmes, who is a U. S. Supreme Court justice. Uh.
Speaker 1: And he's talking about the economic view of substantive law. Uh.
Speaker 1: And Holmes wrote, quote, if you want to know the
Speaker 1: law and nothing else, you must look at it as
Speaker 1: a bad man who cares only for the material consequences
Speaker 1: which such knowledge enables him to predict, not as a
Speaker 1: good one who finds his reasons for conduct, whether inside
Speaker 1: the law or outside of it, in the vaguer sanctions
Speaker 1: of conscience. Uh. And so they write, the measure of
Speaker 1: the substantive law, in other words, is not to be
Speaker 1: mixed up with moral qualms, but is simply coextensive with
Speaker 1: its remedy. No more and no less. It just is
Speaker 1: what the remedy is. It's the cost of doing business. Now.
Speaker 1: Of course, there are plenty of legal scholars and philosophers
Speaker 1: who would dispute how Holmes thinks of this. But the
Speaker 1: interesting question is how does this apply to robots? If
Speaker 1: you're programming a robot to behave well, you actually don't
Speaker 1: get to just sort of like jump over this distinction
Speaker 1: the way humans do when they think about their own
Speaker 1: moral conduct. Right, Like, you're not sitting when you're trying
Speaker 1: to think what's a good way to be a good person.
Speaker 1: You're not sitting around thinking about well, am I going
Speaker 1: by the normative view of morality or the economic view
Speaker 1: of morality? You know? Um, you just sort of act
Speaker 1: a certain way whatever it seems to you the right
Speaker 1: way to do. But if you're trying to program a
Speaker 1: robot to behave well, you have to make a choice
Speaker 1: whether to embrace the normative view or the economic view.
Speaker 1: Does a robot view a red light, say, as a
Speaker 1: firm prohibition against forward movement, it's just a bad thing
Speaker 1: and you shouldn't do it to drive through a red light?
Speaker 1: Or does it just view it as a substantial discouragement
Speaker 1: against forward motion that has a certain cost, and if
Speaker 1: you were to overcome that cost, then you drive on through. Yeah,
Speaker 1: this is a great, great question because I feel like
Speaker 1: with humans, we're probably mixing a match and all the time,
Speaker 1: you know, perhaps you even done the same law breaking behavior.
Speaker 1: You know, we may do both on on one thing,
Speaker 1: and we do one on another thing, and then the
Speaker 1: other one on as still a third thing. But with
Speaker 1: the robot, it seems like you're gonna deal more or
Speaker 1: less with kind of an absolute direction. Either they're going
Speaker 1: to be um either the law is is to be
Speaker 1: obeyed or the law is to be taken into your
Speaker 1: cost analysis. Well, yeah, so they talk about how the
Speaker 1: normative view is actually very much like uh, Isaac Asimov's
Speaker 1: Laws of Robotics, inviolable rules, and the the Asimov story
Speaker 1: is doing a very good job of demonstrating why inviolable
Speaker 1: rules are really difficult to implement in the real world.
Speaker 1: Like you know that they Asimov explored this brilliantly, and
Speaker 1: along these lines, the authors here argued that there there
Speaker 1: are major reasons to think it will just not make
Speaker 1: any practical sense to program robots with a normative view
Speaker 1: of legal remedies. That probably when people make AI s
Speaker 1: and robots that that have to take these kind of
Speaker 1: things into account, they're almost definitely going to program them
Speaker 1: according to the the economic view, right. Uh, They say
Speaker 1: that quote encoding the rule don't run a red light
Speaker 1: as an absolute prohibition, for example, might sometimes conflict with
Speaker 1: the more compelling goal of not letting your driver die
Speaker 1: by being hit by an oncoming truck. So the robots
Speaker 1: are probably going to have to be economically economically motivated
Speaker 1: to an extent like this. Um. But then they talk
Speaker 1: about how you know, this gets very complicated because robots
Speaker 1: will calculate the risks of reward and punishment with different
Speaker 1: biases than humans, or maybe even without the biases that
Speaker 1: humans have that the legal system relies on in order
Speaker 1: to keep us obedient. Humans are highly motivated, usually by
Speaker 1: certain types of punishments that like, you know, humans like
Speaker 1: really don't want to spend a month in jail, you know,
Speaker 1: most of the time. And you can't just rely on
Speaker 1: a robot to be incredibly motivated by something like this,
Speaker 1: first of all, because like it wouldn't even make sense
Speaker 1: to send the robot itself to jail. So you need
Speaker 1: some kind of organized system for making a robot understand
Speaker 1: the cost of bad behavior in a systematized way that
Speaker 1: made sense to the robot as a as a demotivating incentive. Yeah,
Speaker 1: Like shame comes to mind as another aspect of of
Speaker 1: all this, Like how do you shame a robot? Do
Speaker 1: you have to program a robot to feel shamee and
Speaker 1: being uh, you know, made to give a public apology
Speaker 1: or something? Yeah? Uh so, so they are you that
Speaker 1: that it really only makes sense for robots to look
Speaker 1: at legal remedies in an economic way, and then they
Speaker 1: write quote it thus appears that Justice Holmes, archetypical bad man,
Speaker 1: will finally be brought to corporeal form, though ironically not
Speaker 1: as a man at all. And if Justice Holmes metaphorical
Speaker 1: subject is truly morally impoverished and analytically deficient, as some accused,
Speaker 1: it will have significant ramifications for robots. But yeah, thinking
Speaker 1: about these incentives, it gets more and more difficult, Like
Speaker 1: the more you try to imagine the particulars humans have
Speaker 1: self motivations, you know, pre existing motivations that can just
Speaker 1: be assumed. In most cases, humans don't want to pay
Speaker 1: out money, Humans don't want to go to jail. How
Speaker 1: would these costs be instantiated as motivating for robots? You
Speaker 1: would have to you would have to basically force some
Speaker 1: humans I guess meaning the programmers or creators of the robots,
Speaker 1: to instill those costs as motivating on the robot. But
Speaker 1: that's not always going to be easy to do because, okay,
Speaker 1: imagine a robot does violate one of these norms and
Speaker 1: it causes harm to somebody, and as a result, the
Speaker 1: court says, okay, uh, someone, you know, someone has been
Speaker 1: harmed by this negligent or failed autonomous vehicle, and now
Speaker 1: there must be a payout. Who actually pays? Where is
Speaker 1: the pain of the punishment located? A bunch of complications
Speaker 1: to this problem arise like, it gets way more complicated
Speaker 1: than just the programmer or the owner, especially because in
Speaker 1: this age of artificial intelligence, there is a kind of
Speaker 1: there's a kind of distributed responsibility across many parties. The
Speaker 1: authors write, quote robots are composed of many complex components,
Speaker 1: learning from their interactions with thousands, millions, or even billions
Speaker 1: of data points, and they are often designed, operated, least
Speaker 1: or owned by different companies. Which party is to internalize
Speaker 1: these costs, The one that designed the robot or AI
Speaker 1: in the first place, and that might even be multiple companies,
Speaker 1: The one that collected and curated the data set used
Speaker 1: to train and its algorithm in unpredictable ways, the users
Speaker 1: who bought the robot and deployed it in the field.
Speaker 1: And then it gets even more complicated than that, because
Speaker 1: the authors start going into tons of ways that we
Speaker 1: can predict now that it's unlikely that these costs will
Speaker 1: be internalized in commercially produced why robots in ways that
Speaker 1: are socially optimal, Because if if you're you're asking a
Speaker 1: corporation that makes robots to take into account some type
Speaker 1: of economic disincentive against the robot behaving badly, other economic
Speaker 1: incentives are going to be competing with those disincentives, right,
Speaker 1: So the author's right. For instance, if I make it
Speaker 1: clear that my car will kill its driver rather than
Speaker 1: run over a pedestrian, if the issue arises, people might
Speaker 1: not buy my car. The economic costs of lost sales
Speaker 1: may swamp the costs of liability from a contrary choice.
Speaker 1: In the other direction, car companies could run into pr
Speaker 1: problems if their cars run over kids. But simply it
Speaker 1: is aggregate profits, not just profits related to legal sanctions,
Speaker 1: that will drive robot decision making. And then there are
Speaker 1: still a million other things to consider. I mean, one
Speaker 1: thing they talk about is the idea that even within
Speaker 1: corporations that produce UH robots and ai uh, the parts
Speaker 1: of those corporations don't all understand what the other parts
Speaker 1: are doing. You know. They say workers within these corporations
Speaker 1: are likely to be siloed in ways that interfere with
Speaker 1: effective cost internalization. UH. Quote. Machine learning is a specialized
Speaker 1: programming skill, and programmers aren't economists. Uh. And then they
Speaker 1: talk about why in many cases, it's going to be
Speaker 1: really difficult to answer the question of why an AI
Speaker 1: did what it did, So can you even determine that
Speaker 1: the AI say, was was acting in a way that
Speaker 1: wasn't reasonable, Like, how could you ever fundamentally examine the
Speaker 1: state of mind of the AI well enough to to
Speaker 1: prove that the decision it made wasn't the most reasonable
Speaker 1: one from its own perspective of But then another thing
Speaker 1: they raise, I think is a really interesting point, and
Speaker 1: this gets into one of the things we talked about
Speaker 1: in the last episode where thinking about culpability UH for
Speaker 1: AI and robots actually makes us is going to force
Speaker 1: us to re examine our ideas of of culpability and
Speaker 1: blame when it comes to human decision making. Because they
Speaker 1: talk about this the idea that quote, the sheer rationality
Speaker 1: of robot decision making may itself provoke the ire of humans.
Speaker 1: Now how would that be? It seems like we would say, okay, well,
Speaker 1: you know, we want robots to be as rational as possible.
Speaker 1: We don't want them to be irrational. But it is
Speaker 1: often only by carelessly putting costs and risks out of
Speaker 1: mind that we are able to go about our lives.
Speaker 1: For example, people drive cars, and no matter how safe
Speaker 1: of a driver you are, driving a car comes with
Speaker 1: the unavoidable risk that you will harms one uh the
Speaker 1: right quote. Any economist will tell you that the optimal
Speaker 1: number of deaths from many socially beneficial activities is more
Speaker 1: than zero where it. Otherwise, our cars would never go
Speaker 1: more than five miles per hour. Indeed, we would rarely
Speaker 1: leave our homes at all. Even today, we deal with
Speaker 1: those costs and remedies law unevenly. The effective statistical price
Speaker 1: of a human life in court decisions is all over
Speaker 1: the map. The calculation is generally done ad hoc and
Speaker 1: after the fact. That allows us to avoid explicitly discussing
Speaker 1: politically fraught concepts that can lead to accusations of trading
Speaker 1: lives for cash. And it may work acceptably for humans
Speaker 1: because we have instinctive reactions against injuring others that make
Speaker 1: deterrence less important. But in many instances robots will need
Speaker 1: to quantify the value we put on a life if
Speaker 1: they are to modify their behavior at all. Accordingly, the
Speaker 1: companies that make robots will have to figure out how
Speaker 1: much they value you human life, and they will have
Speaker 1: to write it down in the algorithm for all to see,
Speaker 1: at least after extensive discovery UH referring to like you
Speaker 1: know what, the courts will find out by looking into
Speaker 1: how these algorithms are created. And I think this is
Speaker 1: a fantastic point, Like, in order for a robot to
Speaker 1: make ethical decisions about living in the real world, it's
Speaker 1: going to have to do things like put a price
Speaker 1: tag on you know, what kind of risk to human
Speaker 1: life is acceptable in order for it to do anything,
Speaker 1: And we don't, and that seems monstrous to us. It
Speaker 1: does not seem reasonable for any percent chance of harming
Speaker 1: a human, of killing somebody to be the unacceptable risk
Speaker 1: of your day to day activities. And yet it actually
Speaker 1: already is that, you know, it always is that way
Speaker 1: whenever we do anything, but we just like have to
Speaker 1: put it out of mind, like we can't think about it. Yeah,
Speaker 1: I mean, like, what's the alternative, right A programming monstrous
Speaker 1: self delusion into the self driving car where it says
Speaker 1: I will not get into a wreck on my on
Speaker 1: my next route because I cannot That cannot happen to me.
Speaker 1: It has never happened to me before, it will never happen.
Speaker 1: You know, These sorts of you know, ridiculous, not even
Speaker 1: statements that we make in our mind. It's just kind
Speaker 1: of like assumptions, like that's that's the kind of thing
Speaker 1: that happens to other drivers, and it's not going to
Speaker 1: happen to me, even though we we've all seen the
Speaker 1: you know, the statistics before. Yeah, exactly, I mean, I
Speaker 1: think this is a really good point. And uh so,
Speaker 1: in this case, the robot wouldn't even necessarily be doing
Speaker 1: something evil. In fact, you could argue there could be
Speaker 1: cases where the robot is behaving in a way that
Speaker 1: is far safer, far less risky than the average human
Speaker 1: doing the same thing. But the very fact of its
Speaker 1: clearly coded rationality reveals something that is already true about
Speaker 1: human societies, which we can't really bear to look at
Speaker 1: or think about. So another thing that the authors explored
Speaker 1: that I think is really interesting is the idea of
Speaker 1: how robot punishment would make it like directly punishing the
Speaker 1: robot itself, whether how that possibility might make us rethink
Speaker 1: the idea of punishing humans. Uh Now, of course, it's
Speaker 1: just the case that whether or not it actually serves
Speaker 1: as any kind of deterrent, whether or not it actually
Speaker 1: rationally reduces harm, it may just be unavoidable that humans
Speaker 1: sometimes feel they want to inflict direct harm on a perpetrator,
Speaker 1: as punishment for the crime they're alleged to have committed,
Speaker 1: and that may well translate to robots themselves. I mean,
Speaker 1: you can imagine we we've all i think raged against
Speaker 1: an inanimate object before. We wanted to kick a printer
Speaker 1: or something like that. Uh. And we talked in the
Speaker 1: last episode about some of that psychological research about how
Speaker 1: people mindlessly apply social rule to robots. The authors here right,
Speaker 1: Certainly people punch or smash inanimate objects all the time.
Speaker 1: Juries might similarly want to punish a robot not to
Speaker 1: create optimal cost internalization, but because it makes the jury
Speaker 1: and the victim feel better. The authors write later towards
Speaker 1: their conclusion about the idea of directly punishing robots that quote,
Speaker 1: this seems socially wasteful. Punishing robots not to make them
Speaker 1: behave better, but just to punish them is kind of
Speaker 1: like kicking a puppy that can't understand why it's being hurt.
Speaker 1: The same might be true of punishing people to make
Speaker 1: us feel better, but with robots, the punishment is stripped
Speaker 1: of any pretense that it is sending a message to
Speaker 1: make the robot understand the wrongness of its actions. Now
Speaker 1: I'm pretty sympathetic personally to the point of view that
Speaker 1: a lot of punishment that happens in the world is
Speaker 1: not actually uh, is not actually a rational way to
Speaker 1: reduce harm, but just kind of like is uh. You know,
Speaker 1: if it serves any purpose, it is the purpose of
Speaker 1: the emotional satisfaction of people who feel they've been wronged,
Speaker 1: or people who want to demonstrate moral approbrium on on
Speaker 1: the offender. But I understand that. You know, in some cases,
Speaker 1: you could imagine that punishing somebody serves as an object
Speaker 1: example that deters behavior in the future, and to the
Speaker 1: extent that that is ever the case. If it is
Speaker 1: the case, could punishing a robot serve that role, could
Speaker 1: actually inflicting say like like punching a robot or somehow
Speaker 1: otherwise punishing a robot serve as a kind of object
Speaker 1: example that deters behavior in humans, say, say the humans
Speaker 1: who will program the robots of the future. It's a
Speaker 1: weird kind of symbolism to imagine. Yeah, I mean, when
Speaker 1: you start thinking about, you know, the ways to punish robots,
Speaker 1: I mean you think of some of the more ridiculous
Speaker 1: examples that have been brought up in sci fi and
Speaker 1: sci fi comedy like robot Hells and so forth. Um,
Speaker 1: and or the just the idea of even destroying or
Speaker 1: deleting a robot that is faulty or misbehaving. Um. But
Speaker 1: but maybe, you know, maybe it ends up being something
Speaker 1: more like I think of game systems right where say,
Speaker 1: if you accumulate too many of uh say, madness points,
Speaker 1: your I don't know, your movement is cut in half,
Speaker 1: that sort of thing, and then that has a ramification
Speaker 1: on how you play the game and to what extent
Speaker 1: you can play the game well. And therefore, like playing
Speaker 1: into the economic model, you know, it could it could
Speaker 1: have sort of artificially constructed but very real consequences on
Speaker 1: how well a system could behave, you know, but then again,
Speaker 1: you could imagine ways that an AI might find ways
Speaker 1: to to circumvent that, and say, well, if I play
Speaker 1: the game a certain way where I don't need to
Speaker 1: move at normal speed, I can just move at half
Speaker 1: speed but have the benefit of getting to break these rules,
Speaker 1: then who knows, you know, it just I feel like
Speaker 1: there it seems an inescapable maze. Yeah. Well, that's that's
Speaker 1: interesting because is edging toward another thing that the authors
Speaker 1: actually talked about here, which is the idea of a
Speaker 1: robot death penalty. Uh. And this is funny because I
Speaker 1: again because personally, you know, I see a lot of
Speaker 1: flaws in in applying a death penalty to humans. I
Speaker 1: think that is a very flawed judicial remedy. But I
Speaker 1: can understand a death penalty for robots. Like you know,
Speaker 1: robots don't have the same rights as human defendants. If
Speaker 1: a robot is malfunctioning or behaving in a way that
Speaker 1: is so dangerous as to suggest it is likely in
Speaker 1: the future to continue to endanger human lives to an
Speaker 1: unacceptable extent, then yeah, it seems to me reasonable that
Speaker 1: you should just turn off that robot permanently. Okay, But
Speaker 1: but then again, and then it raises the question, well,
Speaker 1: what about what what led us to this malfunction? Is
Speaker 1: there something in the system itself that needs to be
Speaker 1: remedied in order to prevent that from happening? Again, that's
Speaker 1: like very good point, and the authors bring up exactly
Speaker 1: this concern. Yeah, so they say, well, then again, so
Speaker 1: a robot might not have human rights where you would
Speaker 1: be concerned about the death penalty for the robot's own
Speaker 1: good but you might be concerned about what you are
Speaker 1: failing to be able to learn from. Allowing the robot
Speaker 1: to continue to operate like that that could help you
Speaker 1: refine AI in the future. Maybe not letting it continue
Speaker 1: to operate in the wild, but I don't know, keeping
Speaker 1: it operative in some sense because like, whatever it's doing
Speaker 1: is something we need to understand better. So, you know,
Speaker 1: with the robot prison instead of robot death penalty. UM,
Speaker 1: and of course that the human comparison to be made
Speaker 1: is equally as is frustrating because you end up with
Speaker 1: scenarios where you'll have, um, a society that's very pro
Speaker 1: death penalty. But then when it comes to doing the
Speaker 1: same sort of backwork and saying, well, what led to
Speaker 1: this case, what were some of the systematic problems, uh,
Speaker 1: cultural problems, societal problems, I don't know, you know, well,
Speaker 1: whatever it is that that led to this case that
Speaker 1: needed to be remedied with death, should we correct those
Speaker 1: problems too, And in some cases the answer seems to be, oh, no,
Speaker 1: we're not doing that. We'll just we'll just do the
Speaker 1: death penalty as is necessary, even though it doesn't actually
Speaker 1: prevent us from reaching this this point over and over again.
Speaker 1: I mean, I feel like It's one of the most
Speaker 1: common features of the tough on crime mentality that it
Speaker 1: is resistant to the idea of understanding why what led
Speaker 1: a person to commit a crime. I mean, you've heard
Speaker 1: I'm trying to think of an example of somebody, but
Speaker 1: I mean you've heard the person say, oh, uh, you know, oh,
Speaker 1: you're just gonna give some sob story about what happened
Speaker 1: when he was a child or something like that. Yeah, yeah, yeah, yeah,
Speaker 1: I've definitely encountered that that that counter argument before. Yeah,
Speaker 1: but yeah, I mean I think we're probably on the
Speaker 1: same page that it really probably is very useful to
Speaker 1: try to understand what are the common underlying conditions that
Speaker 1: you can detect when people do something bad. And of
Speaker 1: course the same thing would be true of robots, right,
Speaker 1: and it seems like with robots there would potentially be
Speaker 1: room for true rehabilitation with with with these things. If not,
Speaker 1: I mean, certainly you could look at it in a
Speaker 1: software hardware scenario where like, Okay, the software's something's wrong
Speaker 1: with the software, Well delete that put in some some
Speaker 1: healthy software, um, but keep the hardware. Uh you know,
Speaker 1: that's in a way that's rehabilitation right there. It's a
Speaker 1: sort of rehabilitation that's not possible with humans. We can't
Speaker 1: wipe somebody's mental state and replace it with a new,
Speaker 1: factory clean mental state. You know, we can't go back
Speaker 1: and edit someone's memories and traumas and what have you. Uh.
Speaker 1: But with machines, it seems like we would have more
Speaker 1: ability to do something of that nature. Yeah, though, this
Speaker 1: is another thing that comes up, and I mean, of
Speaker 1: course it probably would be useful to try to learn
Speaker 1: from failed AI in order to better perfect AI and robots.
Speaker 1: But on the other hand, in in basically the idea
Speaker 1: of trying to rehabilitate or reprogram robots that do wrong, uh,
Speaker 1: the authors point out that they're probably going to be
Speaker 1: a lot of difficulties in enforcing, say, the the equivalent
Speaker 1: of court orders against robots. So one thing that is
Speaker 1: a common remedy in in legal cases against humans, as
Speaker 1: you might get a restraining order, you know, you need
Speaker 1: to stay fifty feet away from somebody right fifty feet
Speaker 1: away from the plaintiff or something like that, or you
Speaker 1: need to not operate a vehicle or you know something.
Speaker 1: There will be cases where it's probably difficult to enforce
Speaker 1: that same kind of thing on a robot, especially on
Speaker 1: robots whose behavior is determined by a complex interaction of
Speaker 1: rules that are not explicitly coded by humans. So you know,
Speaker 1: most AI these days is not going to be a
Speaker 1: series of if then statements written by humans, but it's
Speaker 1: going to be determined by machine learning, which can to
Speaker 1: some extent be sort of reverse engineered and and somewhat
Speaker 1: understood by humans. But the more complex it is, the
Speaker 1: harder it is to do that. And so there might
Speaker 1: be a lot of cases where you know, you say, okay,
Speaker 1: this robot needs to do X, it needs to obit,
Speaker 1: you know, stay fifty feet away from the plaintiff or something,
Speaker 1: but the person whoever is in charge of the robot
Speaker 1: might say, I don't know how to make it do that.
Speaker 1: Or the possibly more tragic or funnier example would be
Speaker 1: the it discovers the equivalent of the drone with the
Speaker 1: wormhole that we talked about in the last episode, right
Speaker 1: where it's the robot is told to keep fifty feet
Speaker 1: of distance between you and the plaintiff. Robot obeys the
Speaker 1: role by lifting the plaintiff and throwing them fifty feet away.
Speaker 1: So to read another section from Limeley and casey. Here,
Speaker 1: they're right to issue an effective injunction that causes a
Speaker 1: robot to do what we want it to do and
Speaker 1: nothing else, requires both extreme foresight and extreme precision in
Speaker 1: drafting it. If injunctions are to work at all, courts
Speaker 1: will have to spend a lot more time thinking about
Speaker 1: exactly what they want to happen and all the possible
Speaker 1: circumstances that could arise. If past experience is any indication,
Speaker 1: courts are unlikely to do it very well. That's not
Speaker 1: a knock on courts. Rather, the problem is twofold words
Speaker 1: are notoriously bad at conveying our tended meaning, and people
Speaker 1: are notoriously bad at predicting the future. Coders, for their part,
Speaker 1: aren't known for their deep understanding of the law, and
Speaker 1: so we should expect errors in translation even if the
Speaker 1: injunction is flawlessly written. And if we fall into any
Speaker 1: of these traps, the consequences of drafting the injunction incompletely
Speaker 1: maybe quite severe. So I'm imagining you issue a cord
Speaker 1: order to a robot to do something or not do something.
Speaker 1: You're kind of in the situation of like the monkeys
Speaker 1: pawl wish you know, right, like you, Oh, you shouldn't
Speaker 1: have phrased it that way. Now you're in for real trouble.
Speaker 1: Or what's the better example of that? And there's some
Speaker 1: movie we were just talking about recently with like the
Speaker 1: Bad Genie who when you phrase a wish wrong, does
Speaker 1: you know works it out on you in a terrible way. Um,
Speaker 1: I don't know. We were talking about Lepricn or wish
Speaker 1: Master or something. Does LEPrecon grant wishes? I don't remember
Speaker 1: LEPrecon granting any wishes. What's he do? Then? I think
Speaker 1: the only one I've seen is Lepricn in space, so
Speaker 1: it is. I'm a little foggy on the the logic.
Speaker 1: I don't think he grants wishes. He just he just
Speaker 1: like rides around on skateboards and punishes people. He just
Speaker 1: attacks people who try to get his gold and stuff. Well,
Speaker 1: but Lepricans in general are known for this sort of thing. Though,
Speaker 1: where are they? Okay, if you're not precise enough, they'll
Speaker 1: work something in there to cheat you out of your
Speaker 1: your your prize. I'm trying to think, so like, don't
Speaker 1: come within fifty feet of the plaintiff, and so the
Speaker 1: robot I don't know, like it builds a big yard
Speaker 1: stick made out of human feed or something. Yeah, yeah,
Speaker 1: has fifty ft long arms again to lift them into
Speaker 1: the air. Something to that effect. Or say the say
Speaker 1: it's uh, for some reason, schools are just too dangerous
Speaker 1: and this self driving car is not permitted to go
Speaker 1: within um, you know some you know so many blocks
Speaker 1: of an active school, and so it calls in a
Speaker 1: bomb threat on that school every day in order to
Speaker 1: get the kids out so that it can actually go
Speaker 1: buy I don't know, something to that effect. Maybe, well,
Speaker 1: that reminds me of a funny observation that uh, not
Speaker 1: that this is lawful activity, but uh, a funny observation
Speaker 1: that the authors make towards their conclusion. They bring up
Speaker 1: there are cases of of crashes with autonomous vehicles where
Speaker 1: the autonomous vehicle didn't crash into someone. The autonomous vehicle,
Speaker 1: you could argue, caused a crash, but somebody else ran
Speaker 1: into the autonomous vehicle because the autonomous vehicle did something
Speaker 1: that is legal and presumably safe but unexpected. And examples
Speaker 1: here would be driving the speed limit in certain areas
Speaker 1: or coming to a complete stop at an intersection. And
Speaker 1: this is another way that the authors are bringing up
Speaker 1: the idea that, uh, examining robot logic is really going
Speaker 1: to have to cause us to re examine the way
Speaker 1: humans interact with the law, because there are cases where
Speaker 1: people cause problems that lead to harm by a baying
Speaker 1: the rules. Oh yeah, Like I think of this all
Speaker 1: the time, and imagine most people do when when driving
Speaker 1: for any long distance, because you have the speed limit
Speaker 1: as it's posted, you have the speed that the majority
Speaker 1: of people are driving. Um, you know, you have that
Speaker 1: sort of ten mile over zone. Then you have the
Speaker 1: people who are driving exceedingly fast. Then you have that
Speaker 1: minimum speed limit that virtually nobody is driving forty miles
Speaker 1: per hour on the interstate, but it's posted. Uh, and
Speaker 1: therefore it would be legal to drive forty one mile
Speaker 1: per hour if you were a robot and weren't in
Speaker 1: a particular hurry. And perhaps that's you know, maximum efficiency
Speaker 1: for your travel. Uh. There's so many, so many things
Speaker 1: like that to think about, and I think we're probably
Speaker 1: not even very good at at guessing until we encounter
Speaker 1: them through robots. How many other situations there are like
Speaker 1: this in the world, where where you can technically be
Speaker 1: within the bounds of the law, like you're doing what
Speaker 1: by the book you're supposed to be doing. Actually, it's
Speaker 1: really dangerous to be doing it that way. So how
Speaker 1: are you supposed to interrogate a robot state of mind?
Speaker 1: And when it comes to stuff like that. But so anyway,
Speaker 1: this leads to the author's talking about the difficulties in
Speaker 1: in robots state of mind valuation, and they say, quote,
Speaker 1: robots don't seem to be good targets for rules based
Speaker 1: on moral blame or state of mind, but they are
Speaker 1: good at data. So we might consider a legal standard
Speaker 1: that bases liability on how safe the robot is compared
Speaker 1: to others of its type. This would be a sort
Speaker 1: of robotic reasonableness test that could take the form of
Speaker 1: a carrot, such as a safe harbor for self driving
Speaker 1: cars that are significantly safer than average or significantly safer
Speaker 1: than human drivers. Or we could use a stick holding
Speaker 1: robots liable if they lagged behind their peers, or even
Speaker 1: shutting down the worst ten percent of robots in a
Speaker 1: category every year. So I'm not sure if I agree
Speaker 1: with this, but this was an interesting idea to me.
Speaker 1: So instead of like trying to to interrogate the underlying logic,
Speaker 1: of a type of autonomous car, robot or whatever. Because
Speaker 1: it's so difficult to try to understand the underlying logic.
Speaker 1: What if you just compare its outcomes to other machines
Speaker 1: of the same genre as it, or two humans. I mean,
Speaker 1: you can imagine this working better in the case of
Speaker 1: something like autonomous cars, then you can and you know
Speaker 1: other cases where the robot is essentially introducing a sort
Speaker 1: of a new genre of agent into the world. But
Speaker 1: autonomous cars are in many ways going to be roughly
Speaker 1: equivalent in outcomes to human drivers in in regular cars,
Speaker 1: and so would it make more sense to try to
Speaker 1: understand the reasoning behind each autonomous vehicle's decision making when
Speaker 1: it gets into an accident, or uh, to compare its
Speaker 1: behavior to I don't know, some kind of aggregate or
Speaker 1: standard of human driving or other autonomous vehicles, or maybe
Speaker 1: we just we just tell it, Look, most humans I
Speaker 1: have like selfish bastards, So just go do it and
Speaker 1: do what you gotta do well. I mean, I would
Speaker 1: say that there is a downside risk to not taking
Speaker 1: this stuff seriously enough, which is uh, which is something
Speaker 1: like that, I mean something like essentially letting robots go
Speaker 1: hog wild? Because they can well be designed and not
Speaker 1: saying that anybody would be you know, maliciously going wahaha
Speaker 1: and rubbing their hands together while they make it this case.
Speaker 1: But you know, you could imagine a situation where there
Speaker 1: are more and more robots entering the world, where the uh,
Speaker 1: the corporate responsibility for them is so diffuse that nobody
Speaker 1: can locate the one person who's responsible for the robots behavior,
Speaker 1: and thus nobody ever really makes the robot, you know,
Speaker 1: behave morally at all. So robots just sort of like
Speaker 1: become a new class of superhuman psychopaths that are immune
Speaker 1: from all consequences. In fact, I would say that is
Speaker 1: a robot apocalypse scenario I've never seen before done in
Speaker 1: a movie. It's always like when the robots are terrible
Speaker 1: to us, it's always like organized, it's always like that,
Speaker 1: you know, they okay, they decide humans are a cancer
Speaker 1: or something and there so they're going to wipe us out.
Speaker 1: What if instead, it's the problem is just that robots,
Speaker 1: sort of, by corporate negligence and distributed responsibility for their
Speaker 1: behavior among humans, robots just end up being ultimately a
Speaker 1: moral and we're flooded with these a moral critters running
Speaker 1: around all over the place that are pretty smart and
Speaker 1: really powerful. I guess there are You do see some
Speaker 1: shades of this in UM in some futuristic sci fi genres.
Speaker 1: I'm particularly thinking of some of the models of cyberpunk genre,
Speaker 1: where the the corporation model has been has been embraced
Speaker 1: as the way of understanding the future of Aiyes, um,
Speaker 1: but but yeah, I think I think for the most
Speaker 1: part this this scenario hasn't been as explored as much
Speaker 1: we tend to. We tend to want to go for
Speaker 1: the eve overlord or the out of control kilbot rather
Speaker 1: than this, right, yeah, you want you want an identifiable villain,
Speaker 1: just like they do in the courts. But yeah, sometimes uh,
Speaker 1: sometimes corporations or manufacturers can be kind of slippery and
Speaker 1: saying like whose thing is this? Thank so, I was
Speaker 1: thinking about all this, about the idea of you know,
Speaker 1: particularly self driving cars being like the main example we
Speaker 1: we ruminate on with with this sort of thing. UM,
Speaker 1: I decided to to look to the book Life three
Speaker 1: point oh by Max teg Mark. UM, which is a
Speaker 1: is a really great book came out of a couple
Speaker 1: of years back, and Max teg Mark is a Swedish
Speaker 1: American physicist, cosmologist and machine learning researcher. If you've been
Speaker 1: listening to the show for a while, you might remember
Speaker 1: that I briefly interviewed him, had like a mini interview
Speaker 1: with him at the World Science Festival, UH several years back. Yeah,
Speaker 1: and I know I've referenced his book Our Mathematical Universe
Speaker 1: previous episodes. Yeah. So so these are these are both
Speaker 1: books intended for a wide audience, very very readable. Life
Speaker 1: three point oh does a does a fabulous job of
Speaker 1: walking the reader through these various scenarios of UH in
Speaker 1: many cases of of AI scendency and how it could work.
Speaker 1: And he gets into this this topic of UM of
Speaker 1: legality and UM and and AI and self driving cars.
Speaker 1: Now he does not make any allusions to Johnny Cab
Speaker 1: in Total Recall, but I'm going to make allusions to
Speaker 1: Johnny Cab in Total Recall is a way of sort
Speaker 1: of putting a manic face on self driving cars. How
Speaker 1: did I get here? The door opened, you got in,
Speaker 1: it's sound reason, so UM, imagine that you're in a
Speaker 1: self driving Johnny cab and it recks. So the basic
Speaker 1: question you might ask is are you responsible? For this
Speaker 1: wreck as the occupant. That seems ridiculous to think so, right,
Speaker 1: you weren't driving it, You just told it where to go? Um,
Speaker 1: Are the owners of the Johnny Cab responsible? Now? This
Speaker 1: seems more reasonable, right, sure, but again it runs into
Speaker 1: a lot of the problems we were just raising there. Yeah,
Speaker 1: but tag Mark points out that there is this other
Speaker 1: option and that American legal scholar David of Latic has
Speaker 1: pointed out that perhaps it is the Johnny Cab itself
Speaker 1: that should be responsible. Now we've been already been discussing
Speaker 1: a lot of this, like what does that mean? What
Speaker 1: does it mean if a have a Johnny Cab of
Speaker 1: a self driving vehicle is responsible for the wreck that
Speaker 1: it is in? What you know, how do we even
Speaker 1: begin to make sense of that statement? Do you do
Speaker 1: you take the damages out of the Johnny cabs bank account? Well,
Speaker 1: that's the thing. We we kind of end up getting
Speaker 1: into that scenario because if the Johnny Cab has responsibilities
Speaker 1: than than tag Mark rights, why not let it own
Speaker 1: car insurance? Not only would this allow for it to
Speaker 1: financially handle accidents, it would also potentially serve as a
Speaker 1: design incentive and a purchasing incent incentive. So the the
Speaker 1: idea here is the better self driving cars with better
Speaker 1: records will qualify for lower premiums, and the less reliable
Speaker 1: models will have to pay higher premiums. So if the
Speaker 1: Johnny Cab runs into enough stuff and explodes enough, then
Speaker 1: that brand of Johnny Caps simply won't be able to
Speaker 1: take to the streets anymore. Oh this is interesting, okay,
Speaker 1: So in order to mean this is very much the
Speaker 1: economic model that we were discussing earlier. So when Schwarzenegger
Speaker 1: hops in and Johnny Cap says, where would you like
Speaker 1: to go? And he says, drive, just drive anywhere? And
Speaker 1: he says, I don't know where that is. And so
Speaker 1: so his incentive to not just like blindly plow forward
Speaker 1: is how much would it cost if I ran into
Speaker 1: something when I did that? Yeah? Exactly. But but Tamar
Speaker 1: points out that the implications of letting a self driving
Speaker 1: car own car insurance it ultimately goes beyond this situation,
Speaker 1: because how does the Johnny Cab pay for its insurance
Speaker 1: policy that, again it hypothetically owns in this scenario? Should
Speaker 1: we let it own money in order to do this?
Speaker 1: Does it have its own bank account? Like you alluded
Speaker 1: to earlier, especially if it's operating as an independent contractor
Speaker 1: of sorts, perhaps paying back certain percentages or fees to
Speaker 1: a greater cab company. Like maybe that's how it would work.
Speaker 1: And if it can own money, well, can it also
Speaker 1: own property like perhaps at the very least it rents
Speaker 1: garage space, uh, but maybe it owns garage space for itself, um,
Speaker 1: you know, or a maintenance facility or the tools that
Speaker 1: work on it. Does it own those as well? Does
Speaker 1: it own spare parts? Does it own the bottles of
Speaker 1: water that go inside of itself for its customers? Does
Speaker 1: it own the complementary wet towels for your head that
Speaker 1: it keeps on hand? Yeah? Um, I mean if nothing else.
Speaker 1: It seems like if it owned things like, the more
Speaker 1: things it owns, the more things that you could potentially
Speaker 1: um uh invoke a penalty upon through the legal system.
Speaker 1: And if they can own money and property and again
Speaker 1: potentially themselves, then Tegmark takes it a step further. He writes,
Speaker 1: if this is the case, quote, there's nothing legally stopping
Speaker 1: smart computers from making money on the stock market and
Speaker 1: using it to buy online services. Once the computer starts
Speaker 1: paying humans to work for it. It can accomplish anything
Speaker 1: that humans can do. I see. So you might say
Speaker 1: that even if you're skeptical of an AI's ability to
Speaker 1: have say the emotional and cultural intelligence to uh to
Speaker 1: write a popular screenplay or you know, create a popular movie,
Speaker 1: it just doesn't get humans well enough to do that.
Speaker 1: It could, at least if it had its own economic
Speaker 1: agency pay humans to do that, right, right and um.
Speaker 1: Elsewhere in the book, tag Mark gets into a lot
Speaker 1: of this, especially the entertainment idea, presenting a scenario by
Speaker 1: which machines like this could gain the entertainment industry in
Speaker 1: order to to ascend to you know, extreme financial power.
Speaker 1: A lot of it is just like sort of playing
Speaker 1: the algorithms, you know, like doing corporation stuff and then
Speaker 1: hiring humans as necessary to to bring that to fruition.
Speaker 1: You know, I mean, would this be all that different
Speaker 1: from any of our like I don't know, Disney or
Speaker 1: Comic book Studios or whatever exists today. Yeah, yeah, exactly. Um.
Speaker 1: So you know we already know the sort of prowess
Speaker 1: that computers have when it comes to the stock market.
Speaker 1: Tech Mark you know, points out that, you know, you know,
Speaker 1: what we have examples of this in the world already
Speaker 1: where we're using AI, and he writes that it could
Speaker 1: lead to a situation where most of the economy is
Speaker 1: owned and controlled by machines. And this, he warns, is
Speaker 1: not that crazy, considering that we already live in a
Speaker 1: world where non human entities called corporations exert tremendous power
Speaker 1: and hold tremendous wealth. I think there there is a
Speaker 1: large amount of overlap between the concept of corporation and
Speaker 1: the concept of an AI. Yeah, and uh, And then
Speaker 1: there are steps beyond this as well. If if machines
Speaker 1: can do all of these things. So if they can,
Speaker 1: if they can if a machine can own property, if
Speaker 1: it can potentially own itself, if it can if it
Speaker 1: can buy things, if it can invest in the stock market,
Speaker 1: if it can accumulate financial power. If you can do
Speaker 1: all these things, then should they also get the right
Speaker 1: to vote as well? You know, it's it's potentially paying taxes?
Speaker 1: Does it get to vote in addition to that? And
Speaker 1: then if not, why and what becomes the caveat that
Speaker 1: determines the right to vote in the scenario? Now, if
Speaker 1: I understand you, right, I think you're saying the tech
Speaker 1: mark is is exploring these possibilities as stuff that he
Speaker 1: thinks might not be as implausible as people would suspect,
Speaker 1: rather than his stuff where he's like, here's my ideal world,
Speaker 1: right right, He's saying like, look, you know, this is
Speaker 1: already where we are. We know what a I can do,
Speaker 1: and we can easily extrapolate where it might go. These
Speaker 1: are the scenarios we should we should potentially be prepared
Speaker 1: for in much the same way that nobody, nobody really
Speaker 1: at an intuitive level, believes that a corporation is a person,
Speaker 1: like a like a human being as a person. Uh,
Speaker 1: you know, it's at least done well enough at convincing
Speaker 1: the courts that it is a person. So would you
Speaker 1: not be able to expect the same coming out of
Speaker 1: machines that were sophisticated enough right and convincing the court is? Uh,
Speaker 1: I'm glad you brought that up, because that's that's another
Speaker 1: area that tech markets into. So what does it mean
Speaker 1: when judges have to potentially judge aiyes? Um? Would these
Speaker 1: be specialized judges with technical knowledge and understanding of the
Speaker 1: complex systems involved, Uh, you know, or is it going
Speaker 1: to be a human judge judging a machine as if
Speaker 1: it were a human. Um. You know, both of these
Speaker 1: are possibilities. But then here's another idea that Tegmarks discusses
Speaker 1: at length. What if we use robo judges um. And
Speaker 1: this ultimately goes beyond the idea of using robo judges
Speaker 1: to judge at the robots, but potentially using them to
Speaker 1: judge humans as well. Um. Because while human judges have
Speaker 1: limited ability to understand the technical knowledge of cases, robo judges,
Speaker 1: tech Mark points out would in theory have un limitted
Speaker 1: learning and memory capacity. They could also be copied, so
Speaker 1: there would be no staffing shortages you need to judges today,
Speaker 1: We'll just copy and paste, right, uh and simplification. But
Speaker 1: but you know, essentially, once you have one, you can
Speaker 1: have many. Uh. This way justice could be cheaper and
Speaker 1: just maybe a little more just by removing the human equation,
Speaker 1: or at least so the machines would argue, right, but
Speaker 1: then the others. The side of the thing is, we've
Speaker 1: already discussed how human created a I is susceptible to
Speaker 1: to bias, so we could potentially, you know, create we
Speaker 1: could create a robo judge. But if we're not not careful,
Speaker 1: it could be bugged, it could be hacked, it could
Speaker 1: be otherwise compromised, where it just might have these various
Speaker 1: biases that it is um that it is using when
Speaker 1: it's judging humans or machines. And then you'd have to
Speaker 1: have public trust in such a system as well. So
Speaker 1: we run into a lot of the same problems we
Speaker 1: run into when we're talking about trusting the machine to
Speaker 1: drive us across town. Yeah, Like, so if robot judge,
Speaker 1: even if now I'm certainly not granting this because I
Speaker 1: don't necessarily believe this was the case, but even if
Speaker 1: it were true that a robot judge would be better
Speaker 1: at judging cases than a human and like more fair
Speaker 1: and more just, you could run into problems with public
Speaker 1: trust in those kind of judges because, for example, they
Speaker 1: make the calculations explicit, right, the same way we talked about,
Speaker 1: like placing a certain value on a human life. Uh,
Speaker 1: it's something that we all sort of do, but we
Speaker 1: don't like to think about it or acknowledge we do it.
Speaker 1: We just do it at an intuitive level that's sort
Speaker 1: of hidden in the dark recesses of the mind, and
Speaker 1: and and don't think about it. A machine would have
Speaker 1: to like put a number on that and and for
Speaker 1: public transparency reasons, that number would probably need to be
Speaker 1: publicly accessible. Yeah, another area, and this is where this
Speaker 1: is another topic and robotics that you know, we could
Speaker 1: easily discuss at at extreme length, but there's a robotic
Speaker 1: surgery to consider. You know. While we continue to make
Speaker 1: great strides and robotic surgery, and in some cases the
Speaker 1: robotic surgery route is indisputably the safest route, there remains
Speaker 1: a lot of discussion regarding UM. You know, how robot
Speaker 1: surgery is, UM is progressing, where it's headed, and how
Speaker 1: malpractice potentially factors into everything UM. Now to despite the
Speaker 1: advances that we've seen, we're not quite at the medical
Speaker 1: droid level, you know, like the autonomous UH surgical bought.
Speaker 1: But as reported by Dennis Grady in the New York
Speaker 1: Times just last year, AI coupled with new imaging techniques,
Speaker 1: are already showing promise as a means of diagnosing tumors
Speaker 1: as accurately as human physicians, but at far greater speed. UM.
Speaker 1: So it's interesting to to think about these advancements, but
Speaker 1: at the same time realize that particularly in an AI
Speaker 1: we're talking more about AI I mean, particularly in AI
Speaker 1: and medicine. We're talking about AI assisted medicine or AI
Speaker 1: assisted surgery. So the human AI relationship is in these
Speaker 1: cases not one of replacement, but of cooperation, at least
Speaker 1: for the near term. Yeah. Yeah, yeah, I see that
Speaker 1: because I mean, there are many reasons for that, but
Speaker 1: one of the one of the reasons that strikes me
Speaker 1: is it comes back to a perhaps sometimes irrational desire
Speaker 1: to inflict punishment on a person who has done wrong,
Speaker 1: even if it doesn't like help the person who has
Speaker 1: been harmed in the first place. Um. There there are
Speaker 1: certain just like intuitions we have, and I think one
Speaker 1: of them is we we feel more confident if there
Speaker 1: is somebody in the loop who would suffer from the
Speaker 1: consequences of failure, you know, like the fit Like it
Speaker 1: doesn't just help that, Like, oh no, I I assure
Speaker 1: you the surgical robot has, you know, strong incentives within
Speaker 1: its programming not to fail, not to botch this surgery
Speaker 1: and take out your you know, remove one of your
Speaker 1: vital organs. Yeah. Like, on one level, on some level,
Speaker 1: we want that person to know their careers on the line,
Speaker 1: are the reput pation is on the line. You know,
Speaker 1: I think most people would feel better going under surgery
Speaker 1: with the knowledge that if the surgeon were to do
Speaker 1: something bad to you. It's not just enough to know
Speaker 1: that the surgeon surgeon is going to try really hard
Speaker 1: not to do something bad to you. You also want
Speaker 1: the like second order guarantee that, like, if the surgeon
Speaker 1: were to screw up and take take out one of
Speaker 1: your vital organs, something bad would happen to them and
Speaker 1: they would suffer. But with a robot, they wouldn't suffer.
Speaker 1: It's just like, oh, whoops. I wonder if we end
Speaker 1: up reaching a point with this in this discussion where
Speaker 1: you know, we're talking about robots hiring people, do we
Speaker 1: end up in a in a position where aiyes, higher
Speaker 1: humans not so much because they need human um expertise
Speaker 1: or human skills or human senses the ability to feel pain. Yeah,
Speaker 1: and to be culpable, Like they need somebody that will,
Speaker 1: like essentially aies hiring humans to be scapegoats in the
Speaker 1: system or in their in the in the in their
Speaker 1: particular job. Uh So they're like, yeah, we need a
Speaker 1: human in the loop. Not because I need a human
Speaker 1: in the loop. I can do this by myself, but
Speaker 1: if something goes wrong. If you know, then there's always
Speaker 1: a certain chance that something will happen. I need a
Speaker 1: human there that will bear the blame. Every robot essentially
Speaker 1: needs a human co pilot, even in cases where robots
Speaker 1: far outperformed the humans, just because the human copilot has
Speaker 1: to be there to accept responsibility for failure. Oh yeah.
Speaker 1: In the first episode, we talked about the idea of
Speaker 1: there being like a punchable um plate on a robot,
Speaker 1: um for when it for when we feel like we
Speaker 1: need to punish it. It's like that, except instead of
Speaker 1: a specialized plate on the robot itself, it's just a
Speaker 1: person that the robot hired. A whipping boy. Oh this
Speaker 1: is so horrible and and so perversely plausible. I can
Speaker 1: I can kind of see it. It's like in my lifetime,
Speaker 1: I can see it. Well, thanks for the nightmares, Rob, Well, no,
Speaker 1: I think we've had plenty of potential nightmares discuss here.
Speaker 1: But I mean we shouldn't just focus on the nightmares.
Speaker 1: I mean, again, to be clear, Um, you know, so
Speaker 1: the idea of self driving cars, the idea of robot
Speaker 1: assisted surgery, I mean, we're ultimately talking about the aim
Speaker 1: of of of creating safer practices of saving him and lives. So, uh,
Speaker 1: you know it's all it's not all nightmares and um
Speaker 1: robot health scapes. But we have to be realistic about
Speaker 1: the very complex UM scenarios and tasks that we're building
Speaker 1: things around and unleashing machine intelligence upon. Yeah. I mean
Speaker 1: I made this clear in uh in the previous episode.
Speaker 1: I'm not like down on things like autonomous vehicles. I mean,
Speaker 1: ultimately I think autonomous vehicles are are probably a good thing. Um,
Speaker 1: but I do think it's really important for people to
Speaker 1: start paying attention to these, uh, these unbelievably complicated philosophical, moral,
Speaker 1: and legal questions that will inevitably arise as more independent
Speaker 1: and intelligent agents infiltrate our our world. All right, Well,
Speaker 1: on that note, we're gonna go ahead close it out.
Speaker 1: But if you would like to listen to other episodes
Speaker 1: of Stuff to Blow Your Mind, you know where to
Speaker 1: find them. You can find our core episodes on Tuesdays
Speaker 1: and Thursdays in the Stuff to Blow Your Mind podcast feed.
Speaker 1: On Monday's, we tend to do listener mail. On Wednesdays,
Speaker 1: we tend to bust out an artifact shorty episode and
Speaker 1: on Fridays we do a little weird house cinema where
Speaker 1: we don't want to talk about science so much as
Speaker 1: we just talk about one weird movie or another, and
Speaker 1: then we have a little rerun on the weekend. Huge Things.
Speaker 1: As always to our excellent audio producer Seth Nicholas Johnson.
Speaker 1: If you would like to get in touch with us
Speaker 1: with feedback on this episode or any other, to suggest
Speaker 1: a topic for the future, or just to say hello,
Speaker 1: you can email us at contact at stuff to Blow
Speaker 1: your Mind dot com. Stuff to Blow Your Mind is
Speaker 1: production of I Heart Radio. For more podcasts for my
Speaker 1: Heart Radio, visit the i heart Radio app, Apple Podcasts,
Speaker 1: or wherever you listening to your favorite shows.