Speaker 1: Hey, welcome to Stuff to Blow Your Mind. My name
Speaker 1: is Robert Lamb and I'm Joe McCormick, and it's Saturday.
Speaker 1: Time for an episode from the Vault. This is the
Speaker 1: episode Punish the Machine, Part two. It's all about about
Speaker 1: machine moral agency, legal agency and culpability, or what we
Speaker 1: make of this emergent world when when machines are acting
Speaker 1: in an increasingly autonomous way. Let us delay no further.
Speaker 1: Welcome to Stuff to Blow Your Mind production of My
Speaker 1: Heart Radio. Hey, welcome to Stuff to Blow Your Mind.
Speaker 1: My name is Robert Lamb and I'm Joe McCormick, and
Speaker 1: we're back for part two of our talk about Punishing
Speaker 1: the Robot. We are we are back here to uh
Speaker 1: to tell the robot he's been very bad now. In
Speaker 1: the last episode, we talked about the idea of legal
Speaker 1: agency and culpability for robots and other intelligent machines, and
Speaker 1: for a quick refresher on on some of the stuff
Speaker 1: we went over. We talked about the idea that as
Speaker 1: robots and AI become more sophisticated and thus in some
Speaker 1: ways or in some cases more independent and unpredictable, and
Speaker 1: as they integrate more and more into the wild of
Speaker 1: human society, there are just inevitably going to be situations
Speaker 1: where AI and robots do wrong and cause harm to people. Now,
Speaker 1: of course, when a human does wrong and causes harm
Speaker 1: to another human, we have a legal system through which
Speaker 1: the victim can seek various kinds of remedies. And we
Speaker 1: talked in the last episode about the idea of remedies
Speaker 1: that the simple version of that is the remedy is
Speaker 1: what do I get when I win in court? So
Speaker 1: that can be things like monetary rewards. You know, I
Speaker 1: ran into your car with my car, so I pay
Speaker 1: you money, Or it can be punishment, or it can
Speaker 1: be court orders like commanding or restricting the behavior of
Speaker 1: the perpetrator. And so we discussed the idea that as
Speaker 1: roots become more unpredictable and more like human agents, more
Speaker 1: sort of independent, and more integrated into society, it might
Speaker 1: make sense to have some kind of system of legal
Speaker 1: remedies for when robots cause harm or commit crimes. But also,
Speaker 1: as we talked about last time, this is much easier
Speaker 1: said than done. It's going to present tons of new
Speaker 1: problems because our legal system is in many ways not
Speaker 1: equipped to deal with with defendants and situations of this kind,
Speaker 1: and this may cause us to ask questions about how
Speaker 1: we already think about culpability and blame and and punishment
Speaker 1: in the legal system. And so in the last episode
Speaker 1: we talked about one big legal paper that we're going
Speaker 1: to continue to explore in this one. It's by Mark A.
Speaker 1: Limley and Brian Casey in the University of Chicago Law
Speaker 1: Review from twenty nineteen, called Remedies for Robots. So I'll
Speaker 1: be referring back to that one a good bit throughout
Speaker 1: this episode two. Now, I think when we left off
Speaker 1: last time, we had mainly been talking about sort of
Speaker 1: trying to categorize the different sorts of harm that could
Speaker 1: be done by robots or AI intelligent machines. And so
Speaker 1: we talked about some things like unavoidable harms and deliberate
Speaker 1: at least cost harms. These are sort of going to
Speaker 1: be unavoidable parts of having something like autonomous vehicles, right,
Speaker 1: if you have cars driving around on the road, Like,
Speaker 1: even if they're really really good at minimizing harm, there's
Speaker 1: still going to be some cases where there's just no
Speaker 1: way harm could be avoided because their cars. Another would
Speaker 1: be defect driven harms. That's pretty straightforward. That's just where
Speaker 1: the machine malfunctions or breaks in some way. Another would
Speaker 1: be misuse harms. That's where the machine is used in
Speaker 1: a way that is harmful, and in those cases it
Speaker 1: can be usually pretty clear who's at fault. It's the
Speaker 1: person who misused the machine. But then there are a
Speaker 1: couple of other categories that where things get really tricky,
Speaker 1: which are unforeseen harms and systemic harms. And in the
Speaker 1: case of unforeseen harms, one example we talked about in
Speaker 1: the last episode was the drone that invented a wormhole. So,
Speaker 1: you know, people were trying to train a drone to
Speaker 1: move towards like an autonomous flying vehicle, to move towards
Speaker 1: the center of a circular area. But the drone started
Speaker 1: doing a thing where when it got sufficiently far away
Speaker 1: from the center of the circle, it would just fly
Speaker 1: out of the circle altogether. And so it seems kind
Speaker 1: of weird at first, like, Okay, why would it be
Speaker 1: doing that, But then what the researchers realized was that
Speaker 1: whenever it did that, they would turn it off, and
Speaker 1: then they would move it back into the circle to
Speaker 1: start it over again. So from the machine learning point
Speaker 1: of view of the drone itself. It had discovered like
Speaker 1: a like a time space warp that you know. So
Speaker 1: so it was doing this thing that made no sense
Speaker 1: from a human perspective, but actually was it was following
Speaker 1: its programming exactly. Now for an example, uh, sort of
Speaker 1: a thought experiment of how this could become lethal. There's
Speaker 1: an example that is stuck in my head. I can't
Speaker 1: recall where I heard this, who presented this idea? And
Speaker 1: I kind of had it in my head that it
Speaker 1: came from Max tag Mark. But I did some searching
Speaker 1: around in my notes and some searching around and uh,
Speaker 1: one of his books, and I couldn't find it. Perhaps
Speaker 1: you can help refresh me. You remember maybe you remember
Speaker 1: this joke, but the idea of the the AI that
Speaker 1: is running deciding how much oxygen needs to be in
Speaker 1: a train station, it didn't given time. Oh this sounds familiar.
Speaker 1: I don't know the answer, but a lot of these
Speaker 1: thought experiments tend to trace back to Nick Bostrom, so
Speaker 1: I wouldn't be surprised if in there. But but go ahead,
Speaker 1: right okay, as I remember it. The way it works
Speaker 1: is you have you have this AI that's in charge
Speaker 1: of of making sure there's enough oxygen in the train
Speaker 1: station for when humans are there, and it seems to
Speaker 1: have learned this fine, and when humans are there to
Speaker 1: get on the train, everything goes goes well, everybody's breathing fine.
Speaker 1: And then one in one day, Uh, the train arrives
Speaker 1: a little late or it leaves a little right late,
Speaker 1: if you get which whatever it is, and there's not
Speaker 1: enough oxygen and people die. And then it turns out
Speaker 1: that the train was not based saying its decision on
Speaker 1: when people were there, but it was basing it on
Speaker 1: a clock in the train station, like what it was. Um.
Speaker 1: And I may be mangling this horribly, but you know,
Speaker 1: another way of illustrating the point that machine learning could
Speaker 1: end up, you know, latching onto shortcuts or heuristic devices
Speaker 1: that would just seem completely insane to a quote unquote
Speaker 1: logical human mind, but might make sense within the framework
Speaker 1: of the AI. Right, they worked in the training cases,
Speaker 1: and it doesn't understand because it doesn't have common sense,
Speaker 1: it doesn't understand why they wouldn't work. In another case,
Speaker 1: there was actually a real world case that we talked
Speaker 1: about in Part one where there was an attempt to
Speaker 1: do some machine learning on what risk factors would would
Speaker 1: make a pneumonia case admitted to the hospital have a
Speaker 1: higher or lower chance of survival. And one thing that
Speaker 1: a machine learning algorithm determined was that asthma meant that
Speaker 1: you were you better off when you got pneumonia if
Speaker 1: you had asthma. But actually the reason for that that
Speaker 1: that isn't true. Actually the reason for that is that
Speaker 1: if you have asthma, you're a higher risk case for pneumonia,
Speaker 1: so you've got more intensive treatment in the hospital and
Speaker 1: thus had better outcomes on the data set that the
Speaker 1: algorithm was trained on. But the algorithm came up with
Speaker 1: this completely backwards uh failure to understand the difference between
Speaker 1: correlation and causation. There it made it look like asthma
Speaker 1: was a superpower. Now, of course, if you you take
Speaker 1: that kind of shortsighted algorithm and you make it god,
Speaker 1: then it will say, oh, I've just got to give
Speaker 1: everybody asthma, so so we will have a better chance
Speaker 1: of surviving. The point is it can be hard to
Speaker 1: imagine in advance all the cases like this that would
Speaker 1: arise when you've got a world full of robots and
Speaker 1: and AI is running around in it that are trained
Speaker 1: on machine learning. Basically, they're just a number of less
Speaker 1: sort of soft sky nets that you couldn't possibly predict,
Speaker 1: you know, like the sky net scenario being sort of
Speaker 1: like a robots decide that one to end all war
Speaker 1: and um, you know, humans causal war, therefore end all humans,
Speaker 1: that sort of thing. But there's so many different like
Speaker 1: lesser versions of it. It It could also be destructive or
Speaker 1: annoying or just get in the way of effectively using
Speaker 1: AI for whatever we turn to it for. Uh yeah, yeah.
Speaker 1: To come to an example that is definitely used by
Speaker 1: Nick Bostrom, the paper clip maximizer. You know, a robot
Speaker 1: that is designed to make as many paper clips as
Speaker 1: it can, and it just it looks at your body
Speaker 1: and says, hey, that's full of matter. Those could be
Speaker 1: paper clips. Yeah that yeah, yeah, that would be that
Speaker 1: would be quite an apocalypse. Now, before we get back
Speaker 1: into the main subject and talking about this limly in
Speaker 1: case paper with with robots as offenders, there was one
Speaker 1: thing that was interesting I came across. It was just
Speaker 1: a brief footnote in their paper, but about the question
Speaker 1: of what about if the robot is the plaintiff in
Speaker 1: a case? Uh, They said, it's it is possible to
Speaker 1: imagine a robot as a plaintiff in a court case,
Speaker 1: because of course robots, you know, can be injured by humans.
Speaker 1: And they cite a bunch of examples of news stories
Speaker 1: of humans just intentionally like torturing and being cruel two
Speaker 1: robots like that. They cite one news article from eighteen
Speaker 1: about people just aggressively kicking food delivery robots, and then
Speaker 1: they share another story. I actually remember this one from
Speaker 1: the news from about a Silicon Valley security robot that
Speaker 1: was just violently attacked by a drunk man in a
Speaker 1: parking garage. I don't remember this one, but I can
Speaker 1: imagine how it went down. Yeah, exactly. So they say
Speaker 1: that in a case like this, this is actually pretty
Speaker 1: straightforward as a property crime. I mean, unless we start
Speaker 1: getting into a scenario where we're really seeing robots as
Speaker 1: like human beings with their own like consciousness and interests
Speaker 1: and all that, the attacks against robots are really probably
Speaker 1: just property crimes against the owner of the robot. It's like,
Speaker 1: you know, attacking somebody computer or their car or something potentially.
Speaker 1: But we'll get into some stuff a little later that
Speaker 1: I think shows some other directions that could go in
Speaker 1: as well, you know, um, you know, especially consider the
Speaker 1: awesome possibility of robots owning themselves. Yeah, and and that's
Speaker 1: obviously a very different world. I mean, where you get
Speaker 1: into the idea like does a robot actually have rights? Um,
Speaker 1: which is not. That's sort of beyond the horizon of
Speaker 1: what's explored in in this paper itself. This paper is
Speaker 1: more focused on like the kinds of robots that you
Speaker 1: can practically imagine within the next few decades. And and
Speaker 1: in those cases, it seems like all of the really
Speaker 1: thorny stuff would probably be in robots as offenders rather
Speaker 1: than robots as victims of crimes. Right. But to your point,
Speaker 1: like the the initial crimes against robot that we can
Speaker 1: robots that we can imagine would be stuff like drunk
Speaker 1: people pushing them over things like that, Yeah, or just
Speaker 1: like a human driver and a human powered vehicle hitting
Speaker 1: an autonomous vehicle, you know, right. Aw As I mentioned
Speaker 1: in the last episode, this is a very big paper
Speaker 1: and we're not gonna have time to get into every
Speaker 1: avenue they go down in it. But I just wanted
Speaker 1: to go through, uh and and mention some ideas that
Speaker 1: stuck out to me is interesting that they discuss. And
Speaker 1: one thing that really fascinated me about this was that
Speaker 1: the idea of robots as possible agents in in a
Speaker 1: legal context uh brings to the for a philosophical argument
Speaker 1: that has existed in the realm of substantive law for
Speaker 1: a while. UH. And I'll try not to be too
Speaker 1: dry about this, but I think it actually does get
Speaker 1: to some really interesting philosophical territory. UH. And this is
Speaker 1: the distinction between what Limly and Casey call the normative
Speaker 1: versus economic interpretations of substantive law. Again complicated philosophical and
Speaker 1: legal distinction. I'll try to do my best to sum
Speaker 1: it up simply. So. The normative perspective on substantive law
Speaker 1: says that the law is a prohibition against doing something bad.
Speaker 1: So in so thing is against the law. That means
Speaker 1: you shouldn't do it, And we would stop the offender
Speaker 1: from doing the thing that's against the law if we could.
Speaker 1: But since we usually can't stop them from doing it,
Speaker 1: often because it already happened, the remedy that exists. You
Speaker 1: know that maybe paying damages to the victim or something
Speaker 1: like that is a is a an attempt to right
Speaker 1: the wrong in other words, to do the next best
Speaker 1: thing to undoing the harm in the first place. So
Speaker 1: basically getting into the idea of negative reinforcement. Somebody or
Speaker 1: something did something bad, we can't we couldn't stop them
Speaker 1: from doing something bad, but we can try and and
Speaker 1: give them stimulus that would make them not do it again.
Speaker 1: Be that economic or otherwise. Well, yes, but I think
Speaker 1: what you're saying, uh, could actually apply to both of
Speaker 1: these conditions I'm going to talk about. So I think
Speaker 1: maybe the distinction comes in about whether that whether there
Speaker 1: is such a thing as an inherent prohibition. So the
Speaker 1: thing that's operative in the normative view is that the
Speaker 1: thing that's against the law is a thing that should
Speaker 1: not be done, and thus the remedy is an attempt
Speaker 1: to try to fix the fact that it was done
Speaker 1: in the first place. The the economic view is the
Speaker 1: alternative here, and the way they sum that up is
Speaker 1: there is no such thing as forbidden conduct. Rather, a
Speaker 1: substantive law tells you what the cost of the conduct is.
Speaker 1: Does that distinction make any more sense? Yes? Yes, So
Speaker 1: it's basically the first version is doing crimes is bad. Um,
Speaker 1: the second one is doing crimes is expensive. So it's
Speaker 1: the first is crime should not be done, and the
Speaker 1: second one is crimes can be done if you can
Speaker 1: afford it. Yes, exactly so. In Limley and Casey's words, quote,
Speaker 1: damages on this view, the economic view are simply a
Speaker 1: cost of doing business. One we want defendants to internalize,
Speaker 1: but not necessarily to avoid the conduct altogether. And now
Speaker 1: you might look at this and think, oh, okay, well,
Speaker 1: so the economic view is just like a psychopathic way
Speaker 1: of looking at things. And in a certain sense you
Speaker 1: could look at that as like, if you're calculating what's
Speaker 1: the economic cost of murder, then yeah, okay, that does
Speaker 1: just like that's evil, that's like psychopathic. But they're actually
Speaker 1: all kinds of cases we're thinking about. The economic view
Speaker 1: makes more sense of the way we actually behave And
Speaker 1: they use the example of stopping at a traffic light.
Speaker 1: So to read from Limely and Casey here quote Under
Speaker 1: the normative view, a red light stands as a prohibition
Speaker 1: against traveling through an intersection, with the remedy being a
Speaker 1: ticket or a fine against those who are caught breaking
Speaker 1: the prohibition. We would stop you from running the red
Speaker 1: light if we could, But because policing every intersection in
Speaker 1: the country would be impossible. We instead punish those we
Speaker 1: do catch in hopes of deterring others. So in this
Speaker 1: first case, you running a red light is bad, you
Speaker 1: should not do it, and the cost of doing it,
Speaker 1: you know, the punishment you face for doing it is
Speaker 1: an attempt to right that wrong. But then they say,
Speaker 1: under the economic view, however, and absolute prohibition against running
Speaker 1: red lights was never the intention. Rather, the red light
Speaker 1: merely signals a consequence for those who do, in fact
Speaker 1: choose to travel through the intersection. As in the first instance,
Speaker 1: the remedy available is a fine or a ticket. But
Speaker 1: under this view, the choice of whether or not to
Speaker 1: violate the law depends on the willingness of the lawbreaker
Speaker 1: to accept the penalty. So in the case of a
Speaker 1: red light, well that that might make more sense if
Speaker 1: you're like sitting at a red light and you look
Speaker 1: around and there are no other cars anywhere near you,
Speaker 1: and you've you've got a clear view of the entire
Speaker 1: intersection and the red lights not changing, and you think
Speaker 1: maybe it's broken, and you're just like, okay, I'm I'm
Speaker 1: just going to drive through. Well, if if you reach
Speaker 1: that point where you're like, I think it's broken. That
Speaker 1: I feel like that's a slightly different case. But if
Speaker 1: you're just like nobody's watching, I'm gonna do it, um
Speaker 1: and and and the light isn't taking an absurd amount
Speaker 1: of time or longer than you're you're accustomed to. Yeah,
Speaker 1: I don't know how the the belief that the light
Speaker 1: is broken would factor into that, but yeah, it is.
Speaker 1: I Mean, one thing that I think is clear that
Speaker 1: in it's that in many cases there are people, especially
Speaker 1: I think companies and corporations that operate on the economic view,
Speaker 1: and it is something that I think people generally look
Speaker 1: at and say, okay, that that's kind of grimy. Like
Speaker 1: it like a company that says, okay, there is a
Speaker 1: fine for not obeying this environmental regulation, and we're going
Speaker 1: to make more money by violating the regulation than we
Speaker 1: would pay in the fine anybody, So we're just gonna
Speaker 1: pay it. Yeah, you hear about that. With factories, for instance,
Speaker 1: where where there'll be there'll be some situation where that
Speaker 1: the fine is not significant enough to really be at
Speaker 1: a turrent, it's just a for them breaking that that
Speaker 1: mandate being called on it. Occasionally, it's just the cost
Speaker 1: of doing business. Right. Uh. So, there's a funny way
Speaker 1: to describe this point of view. The the authors bring
Speaker 1: up here that they called the bad man theory. And
Speaker 1: this comes from Justice Oliver Wendell Holmes, who is a
Speaker 1: US Supreme Court justice. Uh And he's talking about the
Speaker 1: economic view of substantive law. Uh And. Holmes wrote, quote,
Speaker 1: if you want to know the law and nothing else,
Speaker 1: you must look at it as a bad man who
Speaker 1: cares only for the material consequences which such knowledge enables
Speaker 1: him to predict, not as a good one who finds
Speaker 1: his reasons for conduct, whether inside the law or outside
Speaker 1: of it, in the vaguer sanctions of conscience. Uh. And,
Speaker 1: so they write, the measure of the substantive law, in
Speaker 1: other words, is not to be mixed up with moral qualms,
Speaker 1: but is simply coextensive with its remedy. No more and
Speaker 1: no less. It just is what the remedy is. It's
Speaker 1: the cost of doing business. Now, of course, there are
Speaker 1: plenty of legal scholars and philosophers who would dispute how
Speaker 1: Holmes thinks of this. But the interesting question is how
Speaker 1: does this apply to robots. If you're programming a robot
Speaker 1: to behave well, you actually don't get to just sort
Speaker 1: of like jump over this distinction the way humans do
Speaker 1: when they think about their own moral conduct. Right, Like,
Speaker 1: you're not sitting when you're trying to think what's a
Speaker 1: good way to be a good person. You're not sitting
Speaker 1: around thinking about well, am I going by the normative
Speaker 1: view of morality or the economic view of morality? You know, Um,
Speaker 1: you just sort of act a certain way whatever it
Speaker 1: seems to you the right way to do. But if
Speaker 1: you're trying to program a robot to behave well, you
Speaker 1: have to make a choice whether to embrace the normative
Speaker 1: view or the economic view. Does a robot view a
Speaker 1: red light, say, as a firm prohibition against forward movement,
Speaker 1: it's just a bad thing and you shouldn't do it
Speaker 1: to drive through a red light? Or does it just
Speaker 1: view it as a substantial discouragement against forward motion that
Speaker 1: has a certain cost, and if you were to overcome
Speaker 1: that cost, then you drive on through. Yeah, this is
Speaker 1: a great, great question because I feel like with humans
Speaker 1: we're probably mixing a match and all the time, yes,
Speaker 1: you know, perhaps the same law breaking behavior. You know,
Speaker 1: we may do both on on one thing, and we
Speaker 1: do one on another thing, and then the other one
Speaker 1: on its still a third thing, but with the robot
Speaker 1: it seems like you're gonna deal more or less with
Speaker 1: kind of an absolute direction. Either they're going to be
Speaker 1: um either that the law is is to be obeyed
Speaker 1: or the law is to be taken into your cost analysis. Well, yeah,
Speaker 1: so they talked about how the normative you is actually
Speaker 1: very much like uh Isaac Asimov's Laws of Robotics inviolable rules,
Speaker 1: and the Asimov story is doing a very good job
Speaker 1: of demonstrating why inviolable rules are really difficult to implement
Speaker 1: in the real world. Like you know that they Asimov
Speaker 1: explored this brilliantly. And along these lines, the authors here
Speaker 1: argue that there there are major reasons to think it
Speaker 1: will just not make any practical sense to program robots
Speaker 1: with a normative view of legal remedies. That probably when
Speaker 1: people make ais and robots that that have to take
Speaker 1: these kinds of things into account, they're a most definitely
Speaker 1: going to program them according to the economic view. Right. Uh.
Speaker 1: They say that quote encoding the rule don't run a
Speaker 1: red light as an absolute prohibition, for example, might sometimes
Speaker 1: conflict with the more compelling goal of not letting your
Speaker 1: driver die by being hit by an oncoming truck. So
Speaker 1: the robots are probably going to have to be economically
Speaker 1: economically motivated to an extent like this, um, But then
Speaker 1: they talk about how you know, this gets very complicated
Speaker 1: because robots will calculate the risks of reward and punishment
Speaker 1: with different biases than humans, or maybe even without the
Speaker 1: biases that humans have that the legal system relies on
Speaker 1: in order to keep us obedient. Humans are highly motivated,
Speaker 1: usually by certain types of punishments that like, you know,
Speaker 1: humans like really don't want to spend a month in jail,
Speaker 1: you know, most of the time. And you can't just
Speaker 1: rely on a robot to be incredibly motivated by something
Speaker 1: like this, first of all, because like it wouldn't even
Speaker 1: make sense to send the robot itself to jail. So
Speaker 1: you need some kind of organized system for making a
Speaker 1: robot understand the cost of bad behavior in a systematized
Speaker 1: way that made sense to the robot as a as
Speaker 1: a demotivating incentive. Yeah, Like shame comes to mind as
Speaker 1: another aspect of of all this, Like how do you
Speaker 1: shame a robot? You have to program a robot to
Speaker 1: feel Shane and being uh you know, made to give
Speaker 1: a public apology or something. Yeah. Uh so, so they
Speaker 1: argue that that it really only makes sense for robots
Speaker 1: to look at legal remedies in an economic way, and
Speaker 1: then they write quote, it thus appears that Justice Holmes
Speaker 1: archetypical bad man, will finally be brought to corporeal form,
Speaker 1: though ironically not as a man at all. And if
Speaker 1: Justice Holmes metaphorical subject is truly morally impoverished and analytically deficient,
Speaker 1: as some accused, it will have significant ramifications for robots.
Speaker 1: But yeah, thinking about these incentives, it gets more and
Speaker 1: more difficult. Like the more you try to imagine the
Speaker 1: particular humans have self motivations, you know, pre existing motivations
Speaker 1: that can just be assumed. In most cases, humans don't
Speaker 1: want to pay out money, humans don't want to go
Speaker 1: to jail. How would these costs be instantiated as motivating
Speaker 1: for robots? You would have to you would have to
Speaker 1: basically force some humans I guess meaning the programmers or
Speaker 1: creators of the robots, to instill those costs as motivating
Speaker 1: on the robot. But that's not always going to be
Speaker 1: easy to do because, Okay, imagine a robot does violate
Speaker 1: one of these norms and it causes harm to somebody,
Speaker 1: and as a result, the court says, okay, uh, someone,
Speaker 1: you know, someone has been harmed by this negligent or
Speaker 1: failed autonomous vehicle, and now there must be a payout.
Speaker 1: Who actually pays? Where is the pain of the punishment located?
Speaker 1: A bunch of complications to this problem arise, like it
Speaker 1: gets way more complicated than just the programmer or the owner,
Speaker 1: especially because in this age of artificial intelligence, there is
Speaker 1: a kind of there's a kind of distributed responsibility across
Speaker 1: many parties. The authors write, quote robots are composed of
Speaker 1: many complex components, learning from their interactions with thousands, millions,
Speaker 1: or even billions of data points. And they are often designed, operated,
Speaker 1: least or owned by different companies. Which party is to
Speaker 1: internalize these costs. The one that designed the robot or
Speaker 1: AI in the first place, and that might even be
Speaker 1: multiple companies, The one that collected and curated the data
Speaker 1: set used to train its algorithm in unpredictable ways, the
Speaker 1: users who bought the robot and deployed it in the field.
Speaker 1: And then it gets even more complicated than that, because
Speaker 1: the authors start going into tons of ways that we
Speaker 1: can predict now that it's unlikely that these costs will
Speaker 1: be internalized in commercially produced robots in ways that are
Speaker 1: socially optimal. Because if if you're you're asking a corporation
Speaker 1: that makes robots to take into account some type of
Speaker 1: economic disincent of against the robot behaving badly, other economic
Speaker 1: incentives are going to be competing with those disincentives, right.
Speaker 1: Uh So the author's right. For instance, if I make
Speaker 1: it clear that my car will kill its driver rather
Speaker 1: than run over a pedestrian, if the issue arises, people
Speaker 1: might not buy my car. The economic costs of lost
Speaker 1: sales may swamp the costs of liability from a contrary choice.
Speaker 1: In the other direction, car companies could run into pr
Speaker 1: problems if their cars run over kids. But simply it
Speaker 1: is aggregate profits, not just profits related to legal sanctions,
Speaker 1: that will drive robot decision making. And then there are
Speaker 1: still a million other things to consider. I mean, one
Speaker 1: thing they talk about is the idea that even within
Speaker 1: corporations that produce robots and AI, uh, the parts of
Speaker 1: those corporations don't all understand what the other parts are doing.
Speaker 1: You know, they say, workers within these corporations are likely
Speaker 1: to be siloed in ways that interfere with effective cost internalization. Uh.
Speaker 1: Quote machine learning is a specialized programming skill, and programmers
Speaker 1: aren't economists. Uh. And then they talked about why in
Speaker 1: many cases, it's going to be really difficult to answer
Speaker 1: the question of why an AI did what it did?
Speaker 1: So can you even determine that the AI, say, was,
Speaker 1: was acting in a way that wasn't reasonable? Like, how
Speaker 1: could you ever fundamentally examine the state of mind of
Speaker 1: the AI well enough to to prove that the decision
Speaker 1: it made wasn't the most reasonable one from its own perspective.
Speaker 1: But then another thing they raised, I think is a
Speaker 1: really interesting point, And this gets into one of the
Speaker 1: things we talked about in the last episode where thinking
Speaker 1: about culpability uh for AI and robots actually makes us
Speaker 1: is going to force us to re examine our ideas
Speaker 1: of of culpability and blame when it comes to human
Speaker 1: decision making. Because they talk about this the idea that quote,
Speaker 1: the sheer rationality of robot decision making may itself provoke
Speaker 1: the ire of humans. Now, how would that be? It
Speaker 1: seems like we would say, okay, well, you know we
Speaker 1: want robots to be as rational as possible. We don't
Speaker 1: want them to be irrational. But it is often only
Speaker 1: by carelessly putting costs and risks out of mind that
Speaker 1: we are able to go about our lives. For example,
Speaker 1: people drive cars, and no matter how safe of a
Speaker 1: driver you are, driving a car comes with the unavoidable
Speaker 1: risk that you will harm someone uh the right quote.
Speaker 1: Any economist will tell you that the optimal number of
Speaker 1: deaths from many socially beneficial activities is more than zero
Speaker 1: where it Otherwise, our cars would never go more than
Speaker 1: five miles per hour. Indeed, we would rarely leave our
Speaker 1: homes at all. Even today, we deal with those costs
Speaker 1: and remedies law unevenly. The effective statistical price of a
Speaker 1: human life in court decisions is all over the map.
Speaker 1: The calculation is generally done ad hoc and after the
Speaker 1: fact that it allows us to avoid explicitly discussing politically
Speaker 1: fraught concepts that can lead to accusations of trading lives
Speaker 1: for cash. And it may work acceptably for humans because
Speaker 1: we have instinctive reactions against injuring others that make deterrence
Speaker 1: less important. But in many instances robots will need to
Speaker 1: quantify the value we put on a life if they
Speaker 1: are to modify their behavior at all. Accordingly, the companies
Speaker 1: that make robots will have to figure out how much
Speaker 1: they value human life, and they will have to write
Speaker 1: it down in the algorithm for all to see, at
Speaker 1: least after extensive discovery, uh, referring to like you know
Speaker 1: what the courts will find out by looking into how
Speaker 1: these algorithms are created. And I think this is a
Speaker 1: fantastic point, Like, in order for a robot to make
Speaker 1: ethical decisions about living in the real world, it's going
Speaker 1: to have to do things like put a price tag
Speaker 1: on you know, what kind of risk to human life
Speaker 1: is acceptable in order for it to do anything. And
Speaker 1: we don't. That seems monstrous to us. It does not
Speaker 1: seem reasonable for any percent chance of harming a human,
Speaker 1: of killing somebody to be the unacceptable risk of your
Speaker 1: day to day activities. And yet it actually already is that,
Speaker 1: you know, it always is that way whenever we do anything,
Speaker 1: but we just like have to put it out of mind,
Speaker 1: like we can't think about it. Yeah, I mean, like,
Speaker 1: what's the alternative, right A programming monstrous self delusion into
Speaker 1: the self driving car where it says I will not
Speaker 1: get into a wreck on my on my next route
Speaker 1: because I cannot That cannot happen to me. It has
Speaker 1: never happened to me before, it will never happen. You know,
Speaker 1: these sorts of you know, ridiculous, not even statements that
Speaker 1: we make in our mind. It's just kind of like
Speaker 1: assumptions like that's that's the kind of thing that happens
Speaker 1: to other drivers, and it's not going to happen to me,
Speaker 1: even though we we've all seen the you know, the
Speaker 1: statistics before. Yeah, exactly, I mean, I think this is
Speaker 1: a really good point. And uh so in this case,
Speaker 1: the robot wouldn't even necessarily be doing something evil. In fact,
Speaker 1: you could argue there could be cases where the robot
Speaker 1: is behaving in a way that is far safer, far
Speaker 1: less risky than the average human doing the same thing.
Speaker 1: But the very fact of its clearly coded rationality reveals
Speaker 1: something that is already true about human societies, which we
Speaker 1: can't really bear to look at or think about. So
Speaker 1: another thing that the author has explored that I think
Speaker 1: is really interesting is the idea of how robot punishment
Speaker 1: would make it like directly punishing the robot itself, whether
Speaker 1: how that possibility might make us rethink the idea of
Speaker 1: punishing humans. Uh, Now, of course, it's just the case
Speaker 1: that whether or not it actually serves as any kind
Speaker 1: of deterrent, whether or not it actually rationally reduces is harm.
Speaker 1: It may just be unavoidable that humans sometimes feel they
Speaker 1: want to inflict direct harm on a perpetrator as punishment
Speaker 1: for the crime they're alleged to have committed, and that
Speaker 1: may well translate to robots themselves. I mean, you can
Speaker 1: imagine we we've all I think, raged against an inanimate
Speaker 1: object before. We wanted to kick a printer or something
Speaker 1: like that. Uh. And we talked in the last episode
Speaker 1: about some of that psychological research about how people mindlessly
Speaker 1: apply social rules to robots. The authors here right, Certainly
Speaker 1: people punch or smash inanimate objects all the time. Juries
Speaker 1: might similarly want to punish a robot not to create
Speaker 1: optimal cost internalization, but because it makes the jury and
Speaker 1: the victim feel better. The authors write later towards their
Speaker 1: conclusion about the idea of directly punishing robots that quote,
Speaker 1: this seems socially wasteful. Punishing robots not to make them
Speaker 1: behave better, but just to punish them is kind of
Speaker 1: like kicking a puppy that can't understand why it's being hurt.
Speaker 1: The same might be true of punishing people to make
Speaker 1: us feel better, but with robots, the punishment is stripped
Speaker 1: of any pretense that it is sending a message to
Speaker 1: make the robot understand the wrongness of its actions. Now,
Speaker 1: I'm pretty sympathetic personally to the point of view that
Speaker 1: a lot of punishment that happens in the world is
Speaker 1: not actually uh, is not actually a rational way to
Speaker 1: reduce harm, but just kind of like is uh. You know,
Speaker 1: if it serves any purpose, it is the purpose of
Speaker 1: the emotional satisfaction of people who feel they've been wronged,
Speaker 1: or people who want to demonstrate moral approprium on on
Speaker 1: the offender. But I understand that, you know, in some cases,
Speaker 1: you could imagine that punishing somebody serves as an object
Speaker 1: example that deters behavior in the future, and to the
Speaker 1: extent that that is ever the case. If it is
Speaker 1: the case, could punishing a robot serve that role, could
Speaker 1: actually inflicting say like like punching a robot or somehow
Speaker 1: otherwise punishing a robot serve as a kind of object
Speaker 1: example that deters behavior in humans, say, say the humans
Speaker 1: who will program the robots of the future. It's your
Speaker 1: kind of symbolism to imagine. Yeah, I mean, when you
Speaker 1: start thinking about, you know, the ways to punish robots,
Speaker 1: I mean you think of some of the more ridiculous
Speaker 1: examples that have been brought up in sci fi and
Speaker 1: sci fi comedy, like robot hells and so forth. Um,
Speaker 1: And or just the idea of even destroying or deleting
Speaker 1: a robot that is faulty or misbehaving. Um. But but maybe,
Speaker 1: you know, maybe it ends up being something more like
Speaker 1: I think of game systems right where say, if you
Speaker 1: accumulate too many of uh say madness points, your I
Speaker 1: don't know, your movement is cut in half, that sort
Speaker 1: of thing, and then that has a ramification on how
Speaker 1: you play the game and to what extent you can
Speaker 1: play the game well. And therefore, like playing into the
Speaker 1: economic model, you know, it could it could have sort
Speaker 1: of our officially constructed but very real consequences on how
Speaker 1: well a system could behave, you know, But then again
Speaker 1: you can imagine ways that an AI might find ways
Speaker 1: to to circumvent that and say, well, if I play
Speaker 1: the game a certain way where I don't need to
Speaker 1: move at normal speed, I can just move at half
Speaker 1: speed but have the benefit of getting to break these rules.
Speaker 1: Then who knows, you know, it just I feel like
Speaker 1: there it seems an inescapable maze. Yeah, well that's that's
Speaker 1: interesting because that is edging toward another thing that the
Speaker 1: authors actually talked about here, which is the idea of
Speaker 1: a robot death penalty. Uh. And this is funny because
Speaker 1: I again because personally, you know, I see a lot
Speaker 1: of flaws in in applying a death penalty to humans.
Speaker 1: I think that is a very flawed, uh judicial remedy.
Speaker 1: But I can understand a death penalty for robots, Like
Speaker 1: you know, robots don't have the same rights as human defendants.
Speaker 1: If a robot is malfunctioning or behaving in a way
Speaker 1: that is so dangerous as to suggest it is likely
Speaker 1: in the future to continue to endanger human lives to
Speaker 1: an unacceptable extent, then yeah, it seems to me reasonable
Speaker 1: that you should just turn off that robot permanently. Okay,
Speaker 1: But but then again, and then it raises the question, well,
Speaker 1: what about what what led us to this malfunction? Is
Speaker 1: there something in the system itself that needs to be
Speaker 1: remedied in order to prevent that from happening. Again, that's
Speaker 1: like very good point, and the authors bring up exactly
Speaker 1: this concern. Yeah, so they say, well, then again, so
Speaker 1: a robot might not have human rights where you would
Speaker 1: be concerned about the death penalty for the robot's own good,
Speaker 1: but you might be concerned about what you are failing
Speaker 1: to be able to learn from. Allowing the robot to
Speaker 1: continue to operate like that that could help you refine
Speaker 1: AI in the future. Maybe not letting it continue to
Speaker 1: operate in the wild, but I don't know, keeping it
Speaker 1: operative in some sense because like, whatever it's doing is
Speaker 1: something we need to understand better. With the robot prison
Speaker 1: instead of robot death penalty. Um. And of course that
Speaker 1: the human comparison to be made is equally as is
Speaker 1: frustrating because you end up with scenarios where you'll have, um,
Speaker 1: a society that's very pro death penalty. But then when
Speaker 1: it comes to doing the same sort of backwork and saying, well,
Speaker 1: what led to this case, what were some of the
Speaker 1: systematic problems, uh, cultural problems, societal problems. I don't know,
Speaker 1: you know, well, whatever it is that that led to
Speaker 1: this case that needed to be remedied with death. Should
Speaker 1: we correct those problems too, And in some cases the
Speaker 1: answer seems to be, oh, no, we're not doing that.
Speaker 1: We'll just we'll just do the death penalty as is necessary,
Speaker 1: even though it doesn't actually prevent us from reaching this
Speaker 1: this point over and over again. I mean, I feel
Speaker 1: like it's one of the most common features of the
Speaker 1: tough on crime mentality that it is resistant to the
Speaker 1: idea of understanding why what led a person to commit
Speaker 1: a crime. I mean, you've heard I'm trying to think
Speaker 1: of an example of somebody, but I mean you've heard
Speaker 1: the person say, oh, uh, you know, you're just gonna
Speaker 1: give some sob story about what happened when he was
Speaker 1: a child or something like that. Yeah, yeah, yeah, yeah,
Speaker 1: I've definitely encountered that that that counter argument before. Yeah,
Speaker 1: but yeah, I mean, I think we're probably on the
Speaker 1: same page that it really probably is very useful to
Speaker 1: try to understand what are the common underlying conditions that
Speaker 1: you can detect when people do something bad. And of course,
Speaker 1: the same thing would be true of robots, right, And
Speaker 1: it seems like with robots there would potentially be room
Speaker 1: for true rehabilitation with with with these things. If not,
Speaker 1: I mean, certainly you could look at it in a
Speaker 1: software hardware scenario where like, Okay, the software's something's wrong
Speaker 1: with the software, Well delete that, put in some some
Speaker 1: healthy software, um, but keep the hardware. Uh. You know
Speaker 1: that's in a way, that's rehabilitation right there. It's a
Speaker 1: sort of rehabilitation that's not possible with humans. We can't
Speaker 1: wipe somebody's mental state and replace it with a new,
Speaker 1: factory clean mental state. You know, we can't go back
Speaker 1: and edit someone's memories and traumas and what have you. Uh,
Speaker 1: But with machines, it seems like we would have more
Speaker 1: ability to do something of that nature. Yeah. Though, this
Speaker 1: is another thing that comes up, and I mean, of
Speaker 1: course it probably would be useful to try to learn
Speaker 1: from failed AI in order to better perfect AI and robots.
Speaker 1: But on the other hand, in in basically the idea
Speaker 1: of trying to rehabilitate or reprogram robots that do wrong, uh,
Speaker 1: the authors point out that they're probably going to be
Speaker 1: a lot of difficulties in enforcing say, the equivalent of
Speaker 1: court orders against robots. So one thing that is a
Speaker 1: common remedy in in legal cases against humans, as you
Speaker 1: might get a restraining order, you know, you need to
Speaker 1: stay fifty feet away from somebody right uh, fifty feet
Speaker 1: away from the plantiff or something like that, or you
Speaker 1: need to not operate a vehicle or you know something.
Speaker 1: There will be cases where it's probably difficult to enforce
Speaker 1: that same kind of thing on a robot, especially on
Speaker 1: robots whose behavior is determined by a complex interaction of
Speaker 1: rules that are not explicitly coded by humans. So, you know,
Speaker 1: most AI these days is not going to be a
Speaker 1: series of if then statements written by humans, but it's
Speaker 1: going to be determined by machine learning, which can to
Speaker 1: some extent be sort of reverse engineered and and somewhat
Speaker 1: understood by humans. But the more complex it is, the
Speaker 1: harder it is to do that. And so there might
Speaker 1: be a lot of cases where you know, you say, okay,
Speaker 1: this robot needs to do X, it needs to obit,
Speaker 1: you know, stay fifty feet away from the plaintiff or something,
Speaker 1: but the person you know, whoever is in charge of
Speaker 1: the robot might say, I don't know how to make
Speaker 1: it do that. Or the possibly more tragic or funnier
Speaker 1: example would be the it discovers the equivalent of the
Speaker 1: drone with the wormhole that we talked about in the
Speaker 1: last episode, right where it's the robot is told to
Speaker 1: keep fifty feet of distance between you and the plaintiff.
Speaker 1: Robot obeys the role by lifting the plaintiff and throwing
Speaker 1: them fifty feet away. So, to read another section from
Speaker 1: Limley and Casey here, they're right to issue an effective
Speaker 1: injunction that causes is a robot to do what we
Speaker 1: want it to do and nothing else. Requires both extreme
Speaker 1: foresight and extreme precision in drafting it. If injunctions are
Speaker 1: to work at all, courts will have to spend a
Speaker 1: lot more time thinking about exactly what they want to
Speaker 1: happen and all the possible circumstances that could arise. If
Speaker 1: past experiences any indication courts are unlikely to do it
Speaker 1: very well. That's not a knock on courts. Rather, the
Speaker 1: problem is twofold words are notoriously bad at conveying our
Speaker 1: intended meaning, and people are notoriously bad at predicting the future. Coders,
Speaker 1: for their part, aren't known for their deep understanding of
Speaker 1: the law, and so we should expect errors in translation.
Speaker 1: Even if the injunction is flawlessly written, and if we
Speaker 1: fall into any of these traps, the consequences of drafting
Speaker 1: the injunction incompletely maybe quite severe. So I'm imagining you
Speaker 1: issue a court order to a robot to do something
Speaker 1: or not do something. You're kind of in the situation
Speaker 1: of like the monkeys Paw wish, you know, right, like, oh,
Speaker 1: you shouldn't have phrased it that way, Now you're in
Speaker 1: for real trouble. Or what's the better example of that
Speaker 1: isn't There's some movie we were just talking about recently
Speaker 1: with like the Bad Genie who when you phrase a
Speaker 1: wish wrong, does you know works it out on you
Speaker 1: in a terrible way. Um, I don't know. We were
Speaker 1: talking about Lepricn or wish Master or something. Does LEPrecon
Speaker 1: grant wishes? I don't remember LEPrecon granting any wishes? What's
Speaker 1: he do? Then? I think the only one I've seen
Speaker 1: is Lepricon in space, So I'm a little foggy on
Speaker 1: the the logic. I don't think he grants wishes. He
Speaker 1: just he just like rides around on skateboards and punishes people.
Speaker 1: He just attacks people who try to get his gold
Speaker 1: and stuff. Well, Lepricans in general are known for this
Speaker 1: sort of thing, though, are they. Yeah, Okay, if you're
Speaker 1: not precise enough, they'll work something in there to cheat
Speaker 1: you out of your your your prize. I'm trying to think, so, like,
Speaker 1: don't come within fifty feet of the plaintiff. And so
Speaker 1: the robot I don't know, like it builds a big
Speaker 1: yard stick made out of human feed or something. Yeah, yeah,
Speaker 1: it has fifty ft long arms again to lifted them
Speaker 1: into the air. Something to that effect, or say the
Speaker 1: say it's uh, for some reason, schools are just too dangerous,
Speaker 1: and this self driving car is not permitted to go
Speaker 1: within um, you know some you know, so many blocks
Speaker 1: of an active school, and so it calls in a
Speaker 1: bomb threat on that school every day in order to
Speaker 1: get the kids out so that it can actually go
Speaker 1: by I don't know, something to that effect. Maybe, Well,
Speaker 1: that reminds me of a funny observation that uh, not
Speaker 1: that this is lawful activity, but uh, a funny observation
Speaker 1: that the authors make towards their conclusion. They bring up
Speaker 1: there are cases of of crashes with autonomous vehicles where
Speaker 1: the autonomous vehicle didn't crash into someone the autonomous vehicle,
Speaker 1: you could argue caused a crash, but somebody else ran
Speaker 1: into the autonomous vehicle because the autonomous vehicle did something
Speaker 1: that is legal and presumably safe, but unexpected. And examples
Speaker 1: here would be driving the speed limit in certain areas
Speaker 1: or coming to a complete stop at an intersection. And
Speaker 1: this is another way that the authors are bringing up
Speaker 1: the idea that UH, examining robot logic is really going
Speaker 1: to have to cause us to re examine the way
Speaker 1: humans interact with the law, because there are cases where
Speaker 1: people cause problems that lead to harm by obeying the rules.
Speaker 1: Oh yeah, Like I think of this all the time,
Speaker 1: and imagine most people do when when driving for any
Speaker 1: long distance. Because you have the speed limit as it's posted,
Speaker 1: you have the speed that the majority of people are driving, um,
Speaker 1: you know, you have that sort of ten mile over zone.
Speaker 1: Then you have the people who are driving exceedingly fast.
Speaker 1: Then you have that minimum speed limit that virtually nobody
Speaker 1: is driving forty mile per hour on the interstate, but
Speaker 1: it's posted. Uh, and therefore would be legal to drive
Speaker 1: forty one per hour if you were a robot and
Speaker 1: weren't in a particular hurry, and perhaps that's you know,
Speaker 1: maximum efficiency for your travel. Uh yeah, there's so many,
Speaker 1: so many things like that to think about. And I
Speaker 1: think we're probably not even very good at at guessing
Speaker 1: until we encounter them through robots. How many other situations
Speaker 1: there are like this in the world where where you
Speaker 1: can technically be within the bounds of the law, like
Speaker 1: you're doing what by the book you're supposed to be doing,
Speaker 1: but actually it's really dangerous to be doing it that way.
Speaker 1: So how are you supposed to interrogate a robot state
Speaker 1: of mind? And when it comes to stuff like that.
Speaker 1: But so anyway, this leads to the author's talking about
Speaker 1: the difficulties in in robots state of mind valuation, and
Speaker 1: they say, quote, robots don't seem to be good targets
Speaker 1: for rules based on moral blame or state of mind,
Speaker 1: but they are good at data. So we might consider
Speaker 1: a legal standard that bases liability on how safe the
Speaker 1: robot is compared to others of its type. This would
Speaker 1: be a sort of robotic reasonableness test that could take
Speaker 1: the form of a carrot, such as a safe harbor
Speaker 1: for self driving cars that are significantly safer than average
Speaker 1: or significantly safer than human drivers, or we could use
Speaker 1: a stick holding robots liable if they lagged behind their peers,
Speaker 1: or even shutting down the worst ten percent of robots
Speaker 1: in a category every year. So I'm not sure if
Speaker 1: I agree with this, but this was an interesting idea
Speaker 1: to me. So, instead of like trying to to interrogate
Speaker 1: the underlying logic of a type of autonomous car, robot
Speaker 1: or whatever, because it's so difficult to try to understand
Speaker 1: the underlying logic, what if you just compare its outcomes
Speaker 1: to other machines of the same genre as it, or
Speaker 1: two humans. I mean, you can imagine this working better
Speaker 1: in the case of something like autonomous cars then you can,
Speaker 1: and you know other cases where the robot is essentially
Speaker 1: introducing a sort of a new genre of agent into
Speaker 1: the world. But autonomous cars are in many ways going
Speaker 1: to be roughly equivalent in outcomes to human drivers in
Speaker 1: in regular cars, and so would it make more sense
Speaker 1: to try to understand the reasoning behind each autonomous vehicle's
Speaker 1: decision making when it gets into an accident, or uh
Speaker 1: to compare its behavior to I don't know, some kind
Speaker 1: of aggregate or standard of human driving or other autonomous
Speaker 1: vehicles or maybe we just we just tell it. Look,
Speaker 1: most humans drive like selfish bastards, so just go do it,
Speaker 1: do what you gotta do well. I mean, I would
Speaker 1: say that there is a downside risk to not taking
Speaker 1: this stuff seriously enough, which is uh, which is something
Speaker 1: like that, I mean something like essentially letting robots go
Speaker 1: hog wild because they can well be designed and not
Speaker 1: saying that anybody would be you know, maliciously going wahaha
Speaker 1: and rubbing their hands together while they make it this case.
Speaker 1: But you know, you could imagine a situation where there
Speaker 1: are more and more robots entering the world where the uh,
Speaker 1: the corporate responsibility for them is so diffuse that nobody
Speaker 1: can locate the one person who's responsible for the robots behavior,
Speaker 1: and thus nobody ever really makes the robot, you know,
Speaker 1: behave morally at all. So robots just sort of like
Speaker 1: become a new class of superhuman psychopaths that are immune
Speaker 1: from all consequences. In fact, I would say that is
Speaker 1: a robot apocalypse scenario I've never seen before done in
Speaker 1: a movie. It's always like when the robots are terrible
Speaker 1: to us, it's always like organized, it's always like that,
Speaker 1: you know, they Okay, they decide humans are a cancer
Speaker 1: or something, and so they're going to wipe us out?
Speaker 1: What if instead? It's the problem is just that robots,
Speaker 1: sort of, by corporate negligence and distributed responsibility for their
Speaker 1: behavior among humans, robots just end up being ultimately a
Speaker 1: moral and we're flooded with these a moral critters running
Speaker 1: around all over the place that are pretty smart and
Speaker 1: really powerful. I guess there are You do see some
Speaker 1: shades of this in um in some futuristic sci fi genres.
Speaker 1: I'm particularly thinking of some of the models of cyberpunk genre,
Speaker 1: where the the corporation model has been has been embraced
Speaker 1: as the way of understanding the future of Aiyes, um,
Speaker 1: but but yeah, I think I think for the most
Speaker 1: part this this scenario hasn't been as explored as much.
Speaker 1: We tend to We tend to want to go for
Speaker 1: the evil overlord or the out of control kilbot rather
Speaker 1: than this, right, yeah, you want you want an identifiable villain,
Speaker 1: just like they do in the courts. But yeah, sometimes, uh,
Speaker 1: sometimes corporations or manufacturers can be kind of slippery and
Speaker 1: saying like whose thing is this? Than? So I was
Speaker 1: thinking about all this, about the idea of you know,
Speaker 1: particularly self driving cars being like the main example we
Speaker 1: we ruminate on with this sort of thing. UM, I
Speaker 1: decided to to look to the book Life three point
Speaker 1: oh by Max teg Mark UM, which is a is
Speaker 1: a really great book came out of a couple of
Speaker 1: years back. And Max teg Mark is a Swedish American physicist, cosmologist,
Speaker 1: and machine learning researcher. If you've been listening to the
Speaker 1: show for a while, you might remember that I briefly
Speaker 1: interviewed him, had like a mini interview with him at
Speaker 1: the World Science Festival, UH several years back. Yeah, and
Speaker 1: I know I've referenced his book Our Mathematical Universe in
Speaker 1: previous episodes. Yeah, so so these are These are both
Speaker 1: books intended for a wide audience, very very readable. UH.
Speaker 1: Life three point oh does a does a fabulous job
Speaker 1: of walking the reader through these various scenarios of UH
Speaker 1: in many cases of of AI scendency and how it
Speaker 1: could work. And he gets into this this topic of
Speaker 1: UM of legality and UM and and AI and self
Speaker 1: driving cars. Now, he does not make any allusions to
Speaker 1: Johnny Cab until a recall, but I'm going to make
Speaker 1: allusions to Johnny Cab in total recall is a way
Speaker 1: of sort of putting a manic face on self driving cars.
Speaker 1: How did I get here? The door opened, you got in,
Speaker 1: It's sound reason. So, um, imagine that you're in a
Speaker 1: self driving Johnny Cab and it recks. So the basic
Speaker 1: question you might ask is are you responsible for this
Speaker 1: wreck as the occupant? That seems ridiculous to think, so, right,
Speaker 1: you weren't driving it, You just told it where to go. Um,
Speaker 1: are the owners of the Johnny Cab responsible? Now this
Speaker 1: seems more reasonable, right, but again it runs into a
Speaker 1: lot of the problems we were just raising there. Yeah,
Speaker 1: but tag Mark points out that there is this other
Speaker 1: option and that American legal scholar David of Lattic has
Speaker 1: pointed out that perhaps it is the Johnny Cab itself
Speaker 1: that should be responsible. Now we've been already been discussing
Speaker 1: a lot of this, like what does that mean? What
Speaker 1: does it mean if a you have a Johnny Cab,
Speaker 1: you have a self driving vehicle is responsible for wreck
Speaker 1: than it is in what you know? How do we
Speaker 1: even begin to make sense of that statement? Do you
Speaker 1: take the damages out of the Johnny Cabs? Bank account. Well,
Speaker 1: that's the thing. We we kind of end up getting
Speaker 1: into that scenario because if the Johnny Cab has responsibilities
Speaker 1: than than tech Mark writes, why not let it own
Speaker 1: car insurance? Not only would this allow for it to
Speaker 1: financially handle accidents, it would also potentially serve as a
Speaker 1: design incentive and a purchasing incent incentive. So the the
Speaker 1: idea here is the better self driving cars with better
Speaker 1: records will qualify for lower premiums, and the less reliable
Speaker 1: models will have to pay higher premiums. So if the
Speaker 1: Johnny Cab runs into enough stuff and explodes enough, then
Speaker 1: that brand of Johnny Cab simply won't be able to
Speaker 1: take to the streets anymore. Oh this is interesting, okay,
Speaker 1: So in order to mean this is very much the
Speaker 1: economic model that we were discussing earlier. So when Schwarzenegger
Speaker 1: hops in and Johnny Cab says, where would you like
Speaker 1: to go? And he says, drive, just drive anywhere, and
Speaker 1: he says, I don't know where that is. And so
Speaker 1: so his incentive to not just like blindly plow forward
Speaker 1: is how much would it cost if I ran into
Speaker 1: something when I did that? Yeah? Exactly. But but Temark
Speaker 1: points out that the implications of letting a self driving
Speaker 1: car own car insurance it ultimately goes beyond this situation,
Speaker 1: because how does the Johnny cab pay for its insurance
Speaker 1: policy that, again it hypothetically owns in this scenario? Should
Speaker 1: we let it own money in order to do this?
Speaker 1: Does it have its own bank account like you alluded
Speaker 1: to earlier, especially if it's operating as an independent contractor
Speaker 1: of sorts, perhaps paying back certain percentages or fees to
Speaker 1: a greater cab company, Like maybe that's how it would work.
Speaker 1: And if it can own money, well, can it also
Speaker 1: own property? Like perhaps at the very least it rents
Speaker 1: garage space, Uh, but maybe it owns garage space for itself, um,
Speaker 1: you know, or a maintenance facility or the tools that
Speaker 1: work on it. Does it own those as well? Does
Speaker 1: it own spare parts? Does it own the bottles of
Speaker 1: water that go inside of itself for its customers? Does
Speaker 1: it own the complementary wet towels for your head that
Speaker 1: it keeps on hand? Yeah? Um, I mean, if nothing else,
Speaker 1: it seems like if it owned things, like, the more
Speaker 1: things it owns, the more things that you could potentially
Speaker 1: um uh invoke a penalty upon through the legal system,
Speaker 1: and if they can own money and property and again
Speaker 1: potentially themselves. Then tech Mark takes it a step further.
Speaker 1: He writes, if this is the case, quote, there's nothing
Speaker 1: legally stopping smart computers from making money on the stock
Speaker 1: market and using it to buy online services. Once the
Speaker 1: computer starts paying humans to work for it, it can
Speaker 1: accomplish anything that humans can do. I see. So you
Speaker 1: might say that even if you're skeptical of an AI's
Speaker 1: ability to have say the emotional and cultural intelligence to
Speaker 1: uh to write a popular screenplay or you know, create
Speaker 1: a popular movie, it just doesn't get humans well enough
Speaker 1: to do that. It could, at least if it had
Speaker 1: its own economic agency pay humans to do that, right,
Speaker 1: right and um. Elsewhere in the book, tech Mark gets
Speaker 1: into a lot of this, especially entertainment idea, presenting a
Speaker 1: scenario by which machines like this could gain the entertainment
Speaker 1: industry in order to to ascend to you know, extreme
Speaker 1: financial power. A lot of it is just like sort
Speaker 1: of playing the algorithms right, you know, like doing corporation
Speaker 1: stuff and then hiring humans as necessary to to bring
Speaker 1: that to fruition. You know, I mean, would this be
Speaker 1: all that different from any of our like I don't know,
Speaker 1: Disney or comic book studios or whatever exists today. Yeah, yeah, exactly. Um, so,
Speaker 1: you know, we already know the sort of prowess that
Speaker 1: computers have when it comes to the stock market. Tech
Speaker 1: Mark you know, points out that, you know, you know what,
Speaker 1: we have examples of this in the world already where
Speaker 1: we're using AI, and he writes that it could lead
Speaker 1: to a situation where most of the economy is owned
Speaker 1: and controlled by machines. And this, he warns, is not
Speaker 1: that crazy, considering that we already live in a world
Speaker 1: where non human entities called corporations exert tremendous power and
Speaker 1: hold tremendous wealth. I think there there is a large
Speaker 1: amount of overlap between the concept of corporation and the
Speaker 1: concept of an AI. Yeah and uh. And then there
Speaker 1: are steps beyond this as well. If if machines can
Speaker 1: do all of these things, So if they can, if
Speaker 1: they can if a machine can own property, if it
Speaker 1: can potentially own itself, if it can if it can
Speaker 1: buy things, if it can invest in the stock market,
Speaker 1: if it can accumulate financial power, if it can do
Speaker 1: all these things then should they also get the right
Speaker 1: to vote as well? You know, it's it's potentially paying taxes,
Speaker 1: does it get to to vote in addition to that?
Speaker 1: And then if not, why and what becomes the caveat
Speaker 1: that determines the right to vote in this scenario? Now,
Speaker 1: if I understand you, right, I think you're saying the
Speaker 1: tech mark is is exploring these possibilities as stuff that
Speaker 1: he thinks might not be as implausible as people would suspect,
Speaker 1: rather than his stuff where he's like, here's my ideal world,
Speaker 1: right right, He's saying like, look, you know this is
Speaker 1: already where we are. We know what a I can do,
Speaker 1: and we can easily extrapolate where it might go. These
Speaker 1: are the scenarios we should, we should potentially be prepared
Speaker 1: for in much the same way that nobody, nobody really
Speaker 1: at an intuitive level, believes that a corporation is a
Speaker 1: person like a like a human being as a person. Uh,
Speaker 1: you know, it's at least done well enough at convincing
Speaker 1: the courts that it is a person. So would you
Speaker 1: not be able to expect the same coming out of
Speaker 1: machines that were sophisticated enough, right, and convincing the court
Speaker 1: is Uh. I'm glad you brought that up, because that's
Speaker 1: that's another area that tech markets into. So what does
Speaker 1: it mean when judges have to potentially judge Aiyes? Um?
Speaker 1: Would these be specialized judges with technical knowledge and understanding
Speaker 1: of the complex systems involved? Uh? You know? Or is
Speaker 1: it going to be a human judge judging a machine
Speaker 1: as if it were a human? Um? You know. Both
Speaker 1: of these are possibilities. But then here's another idea that
Speaker 1: tech marks discusses at length. What if we use robo judges? Um?
Speaker 1: And this ultimately goes beyond the idea of using robo
Speaker 1: judges to judge at the robots, but potentially using them
Speaker 1: to judge humans as well. UM because while human judges
Speaker 1: have limited ability to understand the technical knowledge of cases,
Speaker 1: robo judges, tech Mark points out, would in theory have
Speaker 1: unlimited learning and memory capacity. They could also be copied,
Speaker 1: so there would be no staffing shortages you need to
Speaker 1: judges today, We'll just copy and paste, right, uh and simplification.
Speaker 1: But but you know, essentially, once you have one, you
Speaker 1: can have many. Uh. This way justice could be cheaper
Speaker 1: and just maybe a little more just by removing the
Speaker 1: human equation, or at least so the machines would argue right,
Speaker 1: But then the other the side of the thing is,
Speaker 1: we've already discussed how human created AI is susceptible to
Speaker 1: to bias. So we could potentially, you know, we could
Speaker 1: create a robot judge, but if we're not not careful,
Speaker 1: it could be bugged, it could be hacked, it could
Speaker 1: be otherwise compromised where it just might have these various
Speaker 1: biases that it is um that it is using when
Speaker 1: it's judging humans or machines. And then you'd have to
Speaker 1: have public trust in such a system as well, So
Speaker 1: we run into a lot of the same problems we
Speaker 1: run into when we're talking about trusting the machine to
Speaker 1: drive us across town. Yeah, Like so if uh robot judge,
Speaker 1: even if now I'm certainly not granting this because I
Speaker 1: don't necessarily believe this was the case, but even if
Speaker 1: it were true that a robot judge would be better
Speaker 1: at judging cases than a human and like more fair
Speaker 1: and more just, you could run into problems with public
Speaker 1: trust and those kind of judges because, for example, they
Speaker 1: make the calculations explicit, right, the same way we talked
Speaker 1: about like placing a certain value on a human life. Uh,
Speaker 1: it's something that we all sort of do, but we
Speaker 1: don't like to think about it or acknowledge we do it.
Speaker 1: We just do it at an intuitive level that's sort
Speaker 1: of hidden in the dark recesses of the mind. And
Speaker 1: and and don't think about it. A machine would have
Speaker 1: to like put a number on that, and and for
Speaker 1: public transparency reasons, that number would probably need to be
Speaker 1: publicly accessible. Yeah, another area, and this is where this
Speaker 1: is another topic and robotics that you know, we could
Speaker 1: easily discuss at extreme length, but there's a robotic surgery
Speaker 1: to consider. You know. While we continue to make great
Speaker 1: strides and robotic surgery, and in some cases the robotic
Speaker 1: surgery route is indisputably the safest route, there remains a
Speaker 1: lot of discussion regarding UM. You know, how robot surgery is,
Speaker 1: UM is progressing, where it's headed, and how malpractice potentially
Speaker 1: factors into everything UM. Now to despite the advances that
Speaker 1: we've seen, we're not quite at the medical droid level,
Speaker 1: you know, like the autonomous UH surgical bot. But as
Speaker 1: reported by Dennis Grady in the New York Times just
Speaker 1: last year, AI coupled with new imaging techniques are already
Speaker 1: showing promise as a means of diagnosing tumors as accurately
Speaker 1: as human physicians, but at far greater speed. UM. So
Speaker 1: it's interesting to UH to think about these advancements, but
Speaker 1: at the same time realize that particularly in an AI,
Speaker 1: we're talking more about AI, I mean, particularly an AI
Speaker 1: and medicine. We're talking about AI assisted medicine or AI
Speaker 1: assisted surgery. So the human AI relationship is in these
Speaker 1: cases not one of replacement but of cooperation, at least
Speaker 1: for the near term. Yeah, yeah, yeah, I see that,
Speaker 1: because I mean, there are many reasons for that, but
Speaker 1: one of the one of the reasons that strikes me
Speaker 1: is it comes back to a perhaps sometimes irrational desire
Speaker 1: to inflict punishment on a person who has done wrong,
Speaker 1: even if it doesn't like help the person who has
Speaker 1: been harmed in the first place. Um. There there are
Speaker 1: certain just like intuitions we have, and I think one
Speaker 1: of them is we we feel more confident if there
Speaker 1: is somebody in the loop who would suffer from the
Speaker 1: consequences of failure, you know, like the fit, Like it
Speaker 1: doesn't just help that, Like oh no, I assure you
Speaker 1: the surgical robot has you know, strong incentives within its
Speaker 1: programming not to fail, not to botch the surgery and
Speaker 1: take out your you know, remove one of your vital organs. Yeah,
Speaker 1: Like on one level, on some level, we want that
Speaker 1: person to know their career is on the line, or
Speaker 1: the reputation is on the line. You know. I think
Speaker 1: most people would feel better going under surgery with the
Speaker 1: knowledge that if the surgeon were to do something bad
Speaker 1: to you. It's not just enough to know that the
Speaker 1: surgeon and surgeon is going to try really hard not
Speaker 1: to do something bad to you. You also want the
Speaker 1: like second order guarantee that, like, if the surgeon were
Speaker 1: to screw up and take take out one of your
Speaker 1: vital organs, something bad would happen to them and they
Speaker 1: would suffer. But with a robot, they wouldn't suffer. It's
Speaker 1: just like, oh, whoops. I wonder if we end up
Speaker 1: reaching a point with this in this discussion where you know,
Speaker 1: we're talking about robots hiring people, do we end up
Speaker 1: in a in a position where Aiyes, higher humans not
Speaker 1: so much because they need human um expertise or human
Speaker 1: skills or human senses the ability to feel pain. Yeah,
Speaker 1: and to be culpable, Like they need somebody that will,
Speaker 1: like essentially aies hiring humans to be scapegoats in the
Speaker 1: system or in their in the in the in their
Speaker 1: particular job. Uh So they're like, yeah, we need a
Speaker 1: human in the loop. Not because I need a human
Speaker 1: in the loop. I can do this by myself, but
Speaker 1: if something goes wrong, if you know, then there's always
Speaker 1: a certain chance that something will happen, I need a
Speaker 1: human there that will bear the blame. Every robot essentially
Speaker 1: needs a human co pilot, even in cases where robots
Speaker 1: far outperformed the humans, just because the human copilot has
Speaker 1: to be there to ex upt responsibility for failure. Oh yeah.
Speaker 1: In the first episode, we talked about the idea of
Speaker 1: there being like a punchable um plate on a robot
Speaker 1: um for when it for when we feel like we
Speaker 1: need to punish it. It's like that, except instead of
Speaker 1: a specialized plate on the robot itself, it's just a
Speaker 1: person that the robot hired. A whipping boy. Oh this
Speaker 1: is so horrible and and so perversely plausible. I can
Speaker 1: I can kind of see it. It's like in my lifetime,
Speaker 1: I can see it. Well, thanks for the nightmares, Robin, Well, no,
Speaker 1: I think we've had plenty of potential nightmares discussed here.
Speaker 1: But I mean we shouldn't just focus on the nightmares.
Speaker 1: I mean, again, to be clear, Um, you know, so
Speaker 1: the idea of self driving cars, the idea of robot
Speaker 1: assisted surgery. I mean, we're ultimately talking about the aim
Speaker 1: of of of creating safer practices of saving him and live. So, uh,
Speaker 1: you know, it's all it's not all nightmares and um
Speaker 1: robot health scapes. But we have to be realist stick
Speaker 1: about the very complex UM scenarios and casks that we're
Speaker 1: building things around and unleashing machine intelligence upon. Yeah, I
Speaker 1: mean I made this clear in uh in the previous episode.
Speaker 1: I'm not like down on things like autonomous vehicles. I mean, ultimately,
Speaker 1: I think autonomous vehicles are probably a good thing. Um,
Speaker 1: but I do think it's really important for people to
Speaker 1: start paying attention to these, uh, these unbelievably complicated philosophical, moral,
Speaker 1: and legal questions that will inevitably arise as more independent
Speaker 1: and intelligent agents in filth radar our world. All right, Well,
Speaker 1: on that note, we're gonna go and close it out.
Speaker 1: But if you would like to listen to other episodes
Speaker 1: of Stuff to Blow Your Mind, you know where to
Speaker 1: find them. You can find our core episodes on Tuesdays
Speaker 1: and Thursdays in the Stuff to Blow Your Mind podcast feed.
Speaker 1: On Monday's we tend to do listener mail, on Wednesdays
Speaker 1: we tend to bust out an artifact shorty episode, and
Speaker 1: on Fridays we do a little weird house cinema where
Speaker 1: we don't wantly talk about science so much as we
Speaker 1: just talked about one weird movie or another, and then
Speaker 1: we have a little rerun on the weekend. Huge thanks
Speaker 1: as always to our excellent audio producer Seth Nicholas Johnson.
Speaker 1: If you would like to get in touch with us
Speaker 1: with feedback on this episode or any other, to suggest
Speaker 1: a topic for the future, or just to say hello,
Speaker 1: you can email us at contact at stuff to Blow
Speaker 1: your Mind dot com. Stuff to Blow Your Mind is
Speaker 1: production of I Heart Radio. For more podcasts for my
Speaker 1: Heart Radio, visit the iHeart Radio app, Apple Podcasts, or
Speaker 1: wherever you're listening to your favorite shows.