Speaker 1: Welcome to Stuff to Blow Your Mind from how Stuff
Speaker 1: Works dot com. Hey, welcome to Stuff to Blow your Mind.
Speaker 1: My name is Robert Lamb and I'm Joe McCormick. And
Speaker 1: so today we're gonna be talking about a an issue
Speaker 1: in the future of technology, actually not just the future,
Speaker 1: the present of technology. And it's going to have to
Speaker 1: do with how we assess threats based on different types
Speaker 1: of technological regimes that exist in the world. And so
Speaker 1: you and I, Robert, are aware that very often we
Speaker 1: fear the wrong things, right, Oh yes, I mean we
Speaker 1: we fear things that are completely disproportionate to the level
Speaker 1: of threat they represent in our lives. And this is
Speaker 1: obvious in some ways because we have phobias of things
Speaker 1: that are totally harmless. Some people are afraid of balloons
Speaker 1: or something. I guess they're not harmful to birds and fish,
Speaker 1: but harmless to us. And some people are afraid of
Speaker 1: I don't know, public speaking, something that is in some
Speaker 1: ways genuinely threatening to your reputation maybe, but it is
Speaker 1: not is not a threat to your to your body,
Speaker 1: to your body integrity. Yeah, and we have a lot
Speaker 1: of these, uh, what some people called paper tigers. You
Speaker 1: know where, in fact, we have a whole classic episode
Speaker 1: of stuff to build your mind about paper tigers, where
Speaker 1: something something that is ultimately not life threatening or even
Speaker 1: even something that's not even gonna cause you actual physical injury. Uh,
Speaker 1: we build it up in our minds to the level
Speaker 1: it's really on par with some sort of a large
Speaker 1: predatory creature, you know, in our in our primordial past.
Speaker 1: You know, like we give it the same credence we
Speaker 1: would give a tiger leaping out of the bushes at us.
Speaker 1: So those are the obvious ways that we fear the
Speaker 1: wrong things. We fear stuff that isn't even literally going
Speaker 1: to hurt us. But then people are just wrong in
Speaker 1: the ways they assess the relative dangers of actual physical threats,
Speaker 1: like attacks by animals. You know, people are more afraid of, say,
Speaker 1: being attacked and killed by a shark, But shark kill
Speaker 1: almost nobody. They kill like less than ten people on
Speaker 1: average per year worldwide. Uh, you know, you you are
Speaker 1: extremely unlikely to die by shark attack, even if you
Speaker 1: swim a lot. Meanwhile, animals that don't come in nearly
Speaker 1: as much fear and and grip our minds in the
Speaker 1: same way like dogs kill way more people. Dogs kill
Speaker 1: tens of thousands of people every year, and mosquitoes, which
Speaker 1: spread diseases, they literally kill hundreds of thousands of people annually.
Speaker 1: Nobody's got mosquito phobia. I mean I guess maybe some
Speaker 1: people do. Yeah, it's true, Like at least some people
Speaker 1: have dog phobias. I think you can. You can basically
Speaker 1: look to horror cinema and see this played out right,
Speaker 1: how many how many dog horror films can you think of? Yes,
Speaker 1: you can think of some key ones here and there,
Speaker 1: mainly Kujo, right um, and a few others of note
Speaker 1: you can, And then when you think of shark horror films,
Speaker 1: there's just an endless supply. You could spend the rest
Speaker 1: of your life I think watching terrible shark movies. But
Speaker 1: when it comes to mosquitoes, you basically just have man squito.
Speaker 1: And that's it's not even just a straight up mosquito movie.
Speaker 1: It's a it's a human mosquito hybrid. Well, part of
Speaker 1: it has to do with the kinds of imagery that
Speaker 1: excite our brain. I mean, you just it's hard to
Speaker 1: make a mosquito look scary. Mosquitoes look irritating. Yeah, but
Speaker 1: then on the other hand, Uh, ticks are horrifying looking
Speaker 1: and carry illnesses. And as we pointed out in the past,
Speaker 1: there's only one there's really only one Tick horror film,
Speaker 1: and it's terrific, But there's only one of them. You're
Speaker 1: talking about, the one with Seth Green and Clint Howard,
Speaker 1: Ron Howard's brother. Yeah, I'll make sure that we linked
Speaker 1: to the trailer talk video episode on the Landing page
Speaker 1: for this episode of Stuff to Blow Your Mind, in
Speaker 1: which we talk about this cinematic jewel. Well, if you must.
Speaker 1: But people are also totally off base in the way
Speaker 1: they assess the relative dangers of, say, travel threats. Everybody's
Speaker 1: heard this statistic right about travel travel methods. Oh yeah,
Speaker 1: fear of flying being a huge one. Um, you know
Speaker 1: driving is considerably more dangerous, and yet it's it's flying
Speaker 1: that fills so many of us with varying levels of anxiety.
Speaker 1: I can personally relate to this, and I'm continually fascinated
Speaker 1: and frustrated by the way like this deeper irrational fear
Speaker 1: can overpower, or at least sufficiently overpower my rational understanding
Speaker 1: of the risks. Yeah, I know exactly what you're talking about.
Speaker 1: I I have had various levels of fear of flying
Speaker 1: in the past, And yet I totally understand, Like I've
Speaker 1: read all the statistics about how per mile traveled, you
Speaker 1: are so much safer in a in a commercial airliner
Speaker 1: than you are, say, driving yourself somewhere. Yeah, Like, just
Speaker 1: as a rational human being, you think, well, I can
Speaker 1: just I can educate myself out of this fear. Like
Speaker 1: with it with the airplane thing. I think back to
Speaker 1: m was it the escape pot episode where we talked
Speaker 1: about like where why there are no escape pods on airplanes?
Speaker 1: And it doesn't make sense? Yeah, that it doesn't make sense.
Speaker 1: And also if you are going to encounter a dangerous
Speaker 1: situation than an airplane, like, it's gonna be far more off.
Speaker 1: That's going to be the take off of the landing. Um.
Speaker 1: And uh. And yet I'll I'll run through these facts,
Speaker 1: I'll run through the material that we research and uh,
Speaker 1: and it still doesn't quite penetrate the deeper set anxiety
Speaker 1: that sometimes kicks in while flying. Yeah, And this disconnect
Speaker 1: between what we fear and the actual threat is kind
Speaker 1: of troubling. But but it actually gets worse because you
Speaker 1: can point out some scenarios where fearing the wrong thing
Speaker 1: has direct consequences in reality. So I was reading this
Speaker 1: short article on edge dot org from by the psychology
Speaker 1: professor David G. Myers about how humans just do not
Speaker 1: accurately gauge what the real threats they face are. For example,
Speaker 1: people tend to be very afraid of terrorist attacks, and
Speaker 1: of course that makes sense. Terrorist attacks are straightforwardly terrifying
Speaker 1: like that, they are a horrible thing, but statistically they
Speaker 1: are so unlikely to harm you, just as one example, uh,
Speaker 1: Myers points to how much more likely you are to
Speaker 1: be harmed by writing in a motor vehicle, by being
Speaker 1: in a car accident, than by being a victim of
Speaker 1: a terrorist attack. And yet terrorism is designed exactly to
Speaker 1: make us afraid of threats in an outsized way. That's
Speaker 1: sort of the purpose of it, right, It's to grip
Speaker 1: your mind with a horrifying image that will force you
Speaker 1: to behave irrationally in response. Yeah, kind of a forced
Speaker 1: recategorization of a safe or reasonably safe place. Uh. And
Speaker 1: and of course that can be psychologically damaging. That's again
Speaker 1: the whole point of terrorism. Yeah, exactly. Terrorism plays on
Speaker 1: our psychology. And Myers gives an example of how this
Speaker 1: can work exactly against our interests. So after the nine
Speaker 1: eleven attacks in America, a lot of people were very
Speaker 1: worried about terrorist attacks on airplanes, and as a result,
Speaker 1: fewer people were flying. But Myers did the math on
Speaker 1: the effects of this given the statistics about the relative
Speaker 1: safety of these travel methods, and he discovered that if
Speaker 1: people in general flew an average of twenty less and
Speaker 1: didn't just like not travel anywhere, but instead made all
Speaker 1: those same trips and covered the same out of ground
Speaker 1: by driving, then their risk of fatal injury actually went
Speaker 1: up because scheduled airline flights, as we know, are much
Speaker 1: safer than surface travel by car per mile. So if
Speaker 1: Americans flew less in two thousand one and made the
Speaker 1: same trips by car, we could expect about eight hundred
Speaker 1: more people to die in auto fatalities that year. And
Speaker 1: Myers reports that later there was a German psychologist named
Speaker 1: Gerd Gigerinzer who checked this prediction against travel data for
Speaker 1: the year and the year following the attacks, and gird
Speaker 1: determined that an estimated one thousand, five hundred Americans died
Speaker 1: in traffic accidents while trying to avoid the dangerous of
Speaker 1: air travel. So Myers writes, quote, long after nine eleven,
Speaker 1: the terrorists were still killing us. And in a way
Speaker 1: this is true, that they were able to present people
Speaker 1: with horrifying images that made them behave irrationally, made them
Speaker 1: worry about the wrong things, and actually led to more
Speaker 1: dangerous decisions that hurt more people. And of course this
Speaker 1: is limited merely to people's choice of how to travel
Speaker 1: right there. There are obviously other ways that you can
Speaker 1: say terrorism leads to negative consequences that actually harm the
Speaker 1: people who are seeking security and all that. Political decisions
Speaker 1: and stuff like that, Yeah, basically affecting at at various
Speaker 1: levels how you live your life. Yeah, trying to get
Speaker 1: people to operate on the basis of terror management rather
Speaker 1: than rational decision making. And people don't make good decisions
Speaker 1: when they're terrified. And part of the reason is that
Speaker 1: there's this principle exploited by terrorism. It's the cognitive bias
Speaker 1: known as the availability heuristic. And we've talked about this
Speaker 1: on the show before, but basically, what this means is
Speaker 1: that items and events which you are, which you have
Speaker 1: easily accessible in your memory, are given undo weight in
Speaker 1: our considerations. So when we try to think about what's dangerous,
Speaker 1: what are the things we should worry about and protect
Speaker 1: ourselves against. We actually end up not thinking about what
Speaker 1: is statistically the most relevant threat, but we end up
Speaker 1: thinking basically about what's the most scary. And these are
Speaker 1: two very different things. Scary images stick in the mind,
Speaker 1: and real threats very often go unnoticed. Myers writes, quote,
Speaker 1: Thus we remember and fear disasters, tornadoes, air crashes, attacks
Speaker 1: that kill people dramatically in bunches, while fearing too little
Speaker 1: the threats that claim lives one by one. We hardly
Speaker 1: noticed the half million children dying quietly each day from rhodavirus.
Speaker 1: Bill Gates once observed the equivalent of four seven forty
Speaker 1: seven's full of children every day. And we discount the
Speaker 1: future and its future weapon of mass destruction, climate change.
Speaker 1: And so Myers goes on to quote the American security
Speaker 1: and privacy expert Bruce Schneier, who says, quote, if it's
Speaker 1: in the news, don't worry about it. The very definition
Speaker 1: of news is something that hardly ever happens. Now, of course,
Speaker 1: that's not always the case, because, of course you could
Speaker 1: see news reports about things that are are real things
Speaker 1: to be concerned about. But I think what he's talking
Speaker 1: about there is that if you detect your in a
Speaker 1: situation of if it bleeds, it leads, you should do
Speaker 1: your best not to let the scary spectacle of what
Speaker 1: you're seeing become overrepresented as a threat in your brain.
Speaker 1: And of course it's also the case with false news,
Speaker 1: which we've talked about recently, um, in which case this
Speaker 1: is something that never happens and it can end up
Speaker 1: affecting the way we we live our lives or govern ourselves.
Speaker 1: I mean, think to such moral panics at say Satanic
Speaker 1: panic or the or a you know, to a lesser
Speaker 1: degree that the poison candy myth, the idea that Halloween
Speaker 1: candy is going to be tampered with and then handed
Speaker 1: out to children. These are things that that that did
Speaker 1: not happen, but became pervasive ideas. I mean, especially in
Speaker 1: the case of Satanic panic, this is the fiction that
Speaker 1: there is or was an organized effort amongst secret occultists
Speaker 1: to ritually abuse children, a fantasy that resulted in manufactured trauma,
Speaker 1: ruined lives, and a legacy of superstitious persecution and in
Speaker 1: violence in parts of the world, including parts of Africa
Speaker 1: where you kind of see the echo effect of of
Speaker 1: of satanic panic in western nations of predominantly United States
Speaker 1: in the nineteen eighties. Yeah, but we should remember, of course,
Speaker 1: while we talked about the myth, to talk about the
Speaker 1: causal reality. I think a lot of people think what
Speaker 1: was going on when people were coming up with stories
Speaker 1: of Satanic ritual abuse in the eighties was that children
Speaker 1: were being led in interviews and prompted by police and
Speaker 1: therapists and people who had these pre existing ideas in
Speaker 1: their head and children were just sort of like going
Speaker 1: along with what they perceived the adults uh to want
Speaker 1: them to say, right, and and of course it wasn't
Speaker 1: just the children that the children were a major part of.
Speaker 1: You also had adults with these uh these these these
Speaker 1: supposed memories that they were reclaiming through therapy that were
Speaker 1: revealing past satanic abuse. Um. But but yeah, it all,
Speaker 1: it all amounted to a manufactured fear that that so
Speaker 1: many people bought into and the thing that everyone was
Speaker 1: afraid of did not exist and has never existed. You know,
Speaker 1: it's it still floors me there there's an older episode
Speaker 1: of stuff to blow your mind about it. Um what
Speaker 1: to link to that it's it's a fascinating psychological event
Speaker 1: in history that it makes you realize yet again that
Speaker 1: it's the vividness of the imagery and sort of the
Speaker 1: what the what the threat suggests as an idea that
Speaker 1: captures people's fear, not so much the present reality of
Speaker 1: the threat. Yeah, and it drives home just how how
Speaker 1: easy it is, then um psychologically to just to have
Speaker 1: an unbalanced fear of flying or dogs or terrorism or
Speaker 1: what have you. And so for today, I wanted to
Speaker 1: take this principle that we don't exactly fear the right
Speaker 1: thing and and and try to redirect it in one
Speaker 1: particular area, which is the kind of thing we're usually
Speaker 1: worried about when it comes to AI and autonomous weaponry
Speaker 1: and the dangers of technological weapons. I think we are
Speaker 1: worried about the wrong stuff, or more specifically, we're worried
Speaker 1: about stuff that we should be worried about, but we're
Speaker 1: worried almost exclusively about the smaller threat rather than the
Speaker 1: larger threat. The smaller more dramatic. And I think this
Speaker 1: is this is something that came up when you're listening
Speaker 1: the examples earlier. A threat that you could you've given
Speaker 1: the chance physically run from fearing climate change, for instance,
Speaker 1: how do you run from that? How do you hide
Speaker 1: behind a bush? Uh like? But a tornado, on the
Speaker 1: other hand, you could conceivably run into a bunker, right
Speaker 1: you can, you can. You can attach a lot of
Speaker 1: anxiety to it. But then you could conceivably see it
Speaker 1: and and react in real time to its to its threat.
Speaker 1: But if you're in a place where tornadoes don't happen
Speaker 1: very often, say, and you are investing a lot of
Speaker 1: energy and resources into preparing for a tornado threat instead
Speaker 1: of in sting that energy and resources into something that
Speaker 1: will definitely affect you in the future, like say, climate change,
Speaker 1: which will affect most people wherever you are in one
Speaker 1: way or another, especially with all of its secondary downstream effects.
Speaker 1: If you're investing exclusively in the lesser threat, than you
Speaker 1: are doing yourself and your future descendants a disservice. Absolutely.
Speaker 1: All right, Well, let's take a quick break and the
Speaker 1: when we come back, we will talk about different types
Speaker 1: of technological weapon threats. Thank you, thank you. All right,
Speaker 1: we're back. So, Joe, I know that when when we
Speaker 1: when we talk about technological weapon threats, what we're really
Speaker 1: talking about here is of course ED two oh nine.
Speaker 1: We're talking about the terminator, right, We're talking about chopping mall,
Speaker 1: the killer robots of shopping mall that may just go
Speaker 1: out of control and start hunting our teenagers down in
Speaker 1: the malls of the future. Chopping Mall is a fantastic
Speaker 1: piece of eighties trash cinema. But but no, I mean
Speaker 1: that is the thing that captures our mind because you
Speaker 1: can run from it. Yeah, there's a robot with a
Speaker 1: gun on it that that is the standard image. Okay,
Speaker 1: what is the AI autonomous weapon threat of the future.
Speaker 1: It is a terminator. It is those killer robots from
Speaker 1: the Matrix. It is something that is an embodied robot
Speaker 1: that is coming to you know, point a gun at
Speaker 1: you and make you do something, or hunt you down
Speaker 1: for robot sport or something like that. And I want
Speaker 1: to be clear that this is absolutely a real thing
Speaker 1: worth discussing. Not so much I don't know about terminators,
Speaker 1: but the idea of conventional autonomous weapons that I'm not
Speaker 1: saying that is not worth discussing, because it is It's
Speaker 1: been the focus of a lot of international conversation about
Speaker 1: the ethics of warfare. We're actively trying to figure out
Speaker 1: how to regulate this, and there for example United Nations
Speaker 1: Conventions on Discussing Autonomous Conventional Weapons and what we should
Speaker 1: do about them. Yeah, the ideas of Head two oh
Speaker 1: nine terminators, what have you even? Chopping mall Sci five
Speaker 1: has always spoken to our anxieties and our fears, uh,
Speaker 1: and our hopes about technology and where it's going to
Speaker 1: to get us and given us vivid imagery to feed
Speaker 1: our availability heuristic for these types of threats. Yea. And
Speaker 1: of course give us something that we can we can
Speaker 1: we can we can run from you know, we can,
Speaker 1: we can battle, you can fight back against the the
Speaker 1: machines and chopping mall. But to get serious for a minute,
Speaker 1: I mean, we are more and more seeing actual semi
Speaker 1: autonomous weapon technology being introduced into military arsenals around the world. Absolutely.
Speaker 1: I mean you'd have to be under a rock to
Speaker 1: have avoided uh any any coverage of the U. S
Speaker 1: military's use of drone strikes in recent decades. Uh. And
Speaker 1: and that's that's just the tip of the iceberg too.
Speaker 1: I mean I was reading a little bit about autonomous
Speaker 1: weapons in Max tech Mark's recent book Life three point
Speaker 1: oh and he points out that you have. You have
Speaker 1: in the U s military, the US Phalanx system for
Speaker 1: its Ages class cruisers that automatically detect, track, and attack
Speaker 1: threats such as anti ship missiles and aircraft. You may
Speaker 1: have seen images of these these like big dome looking
Speaker 1: devices with the weapon on on the front. Oh yeah,
Speaker 1: I mean we'll think about other types of automatic missile defense,
Speaker 1: the the Israeli Iron Dome system. Yeah. But in particular though,
Speaker 1: this this one system, it's been in service since nineteen eighty,
Speaker 1: still in service today, and it it actually led to
Speaker 1: the nine downing of Iran air Flight six five five,
Speaker 1: a civilian Iranian passenger jet, killed all two nine people
Speaker 1: on board and caused international outrage. There was, however, a
Speaker 1: human in the loop on this system who made the error.
Speaker 1: And that's that's one of the key distinctions between uh,
Speaker 1: some of these contemporary and past autonomous weapons systems and
Speaker 1: the possible future of autonomous weapons systems. Is there a
Speaker 1: human being that is at some point weighing in on
Speaker 1: the decision or having to make a final decision. And
Speaker 1: in pretty much all conventional autonomous weapons systems I can
Speaker 1: think of today, they're they're not fully autonomous there's semi autonomous,
Speaker 1: there's still a human command structure, a human override, there's
Speaker 1: still basically being controlled by humans, but they're making some
Speaker 1: kind of uh, they've got some kind of automatic assistance function, right,
Speaker 1: they'll be they'll be a drone pilot somewhere, or in
Speaker 1: some of these models, I believe it would be you'll
Speaker 1: have a drone pilot and maybe they're looking they're dealing
Speaker 1: with with multiple drones, but there's still a human in
Speaker 1: the loop on that particular weapons system. And just a
Speaker 1: reminder that that Russia is believed to be testing its
Speaker 1: first autonomous nuclear torpedo. The idea with this is that
Speaker 1: it would be guided largely by AI to strike the
Speaker 1: United States even if it lost all communications with Moscow.
Speaker 1: A frightening weapon concept, to be sure, and but it's
Speaker 1: the one that was originally proposed back in the nineties
Speaker 1: sixties by um um Andre Sakarov, and some analysts call
Speaker 1: this a doomsday weapon, and with good reason. That's a
Speaker 1: it's a terrifying concept. It certainly is. I mean, one
Speaker 1: at least hopes that there's there's still human command structure
Speaker 1: there and and but these types of weapons, all these
Speaker 1: things we've been talking about are part of this debate,
Speaker 1: part of this debate that the international community is having
Speaker 1: about what are the ethics of autonomous weapons systems, especially
Speaker 1: if they're not just weapons systems used to say, shoot
Speaker 1: down incoming enemy missiles. I mean that you sort of
Speaker 1: see the difference there right now, Like you can imagine
Speaker 1: missile defense kind of thing is different from something that
Speaker 1: will be aiming at people or aiming at places where
Speaker 1: people could be even though even a missile defense system could,
Speaker 1: of course, as we've seen, go go awry, right right,
Speaker 1: I mean, it becomes increasingly more complicated when you start
Speaker 1: thinking about, um, military engagements, say in a city, if
Speaker 1: you're having to deal with civilians and or or or uh,
Speaker 1: you know, combatants that are not in uniform that sort
Speaker 1: of thing, or any variety of uh, morally complex standoff situation.
Speaker 1: How do you program for that? Yeah, and so that
Speaker 1: is a very important question. But I wanted to focus
Speaker 1: today on a potentially even more dangerous class of weapons
Speaker 1: that can actually hurt many more people, and a class
Speaker 1: of weapons that in fact already exist in some form today.
Speaker 1: And we have a pretty clear vision of how much
Speaker 1: more advanced and how much more dangerous they will continue
Speaker 1: to get in the very near future, even without access
Speaker 1: to heavy weapons or manufacturing capabilities. So I got the
Speaker 1: idea to have this discussion when I read an article
Speaker 1: in Undark magazine, which if Robert you ever read an Undark, Yeah,
Speaker 1: I don't think I've read this one before we can
Speaker 1: mentioned the title. My mind immediately went to various horror
Speaker 1: fiction publications. It's got a horror kind of name. I
Speaker 1: think it's named after We intentionally named after a type
Speaker 1: of radium paint that was used back in the day,
Speaker 1: I think early twentieth century, when before people realize what
Speaker 1: the risks of it were. And that's sort of what
Speaker 1: the magazine explores. That explores science and a lot of
Speaker 1: good long form science writing, exploring the good and the
Speaker 1: bad that science has to offer. But anyway, this article
Speaker 1: was published in July of this year by science journalist
Speaker 1: named Jeremy Sue, and it's called forget killer robots. Autonomous
Speaker 1: weapons are already online. So I was reading this and
Speaker 1: I started thinking about this. I thought this would be
Speaker 1: a good topic for us to talk today. So Sue
Speaker 1: starts by discussing the ways that the problem of autonomous
Speaker 1: weaponry has really captured people's attention worldwide in the conventional
Speaker 1: weapons since and as we've been talking about, there's a
Speaker 1: good reason for this. If we're gonna be increasingly using
Speaker 1: robots and AI programs capable of delivering lethal force on
Speaker 1: the battleground, or I guess to be less uphimistic, if
Speaker 1: we're gonna have machines that can kill people without a
Speaker 1: human taking responsibility and directly making the machine do it,
Speaker 1: we really need to be having serious conversations about the
Speaker 1: ethics of this technology. Should there be international treaties governing
Speaker 1: what kinds of autonomous weapons we allow each other to
Speaker 1: make and so forth? I mean, we've got international treaties
Speaker 1: about nuclear weapons. Shouldn't there be international treaty agreements on
Speaker 1: autonomous weapons? Oh? Yeah, I mean it makes sense. Yet
Speaker 1: to your point, we have we have treaties about nuclear weapons.
Speaker 1: We have we have treaties about biological and chemical agents
Speaker 1: as well. We have treaties about just the way that
Speaker 1: one wages traditional warfare. Their various you know, non nuclear,
Speaker 1: non biological weapons that are also banned under international treaty.
Speaker 1: So it makes sense that we would we would also
Speaker 1: have treaties dealing with autonomous systems. Yeah, it totally makes
Speaker 1: sense in these conversations, as we've been saying, are ongoing.
Speaker 1: Sue writes about how the issue was discussed at a
Speaker 1: United Nations convention in Geneva in April of this year
Speaker 1: in and this was part of an ongoing series of
Speaker 1: conversations with the u N's Convention on Certain Conventional Weapons,
Speaker 1: which has a kind of clunky name. But uh, that's
Speaker 1: a useful kind of forum to be having right now.
Speaker 1: But while we're having this debate about the role of
Speaker 1: AI and autonomous programming and conventional deadly weapons, we really
Speaker 1: tend to miss the fact that autonomous weapons are already
Speaker 1: being used widely in warfare around the world. And we're
Speaker 1: talking about cyber warfare, not autonomous guns and bombs and missiles,
Speaker 1: but little pieces of computer code that autonomously attacks systems
Speaker 1: around the world or in targeted places. Plus, with software,
Speaker 1: there's always the worry of replication. YEA. No matter how
Speaker 1: worried you are about a AI controlled nuclear torpedoes, you
Speaker 1: usually don't have to fret about them mating with each
Speaker 1: other and producing a torpedo offspring. Exactly. I mean that
Speaker 1: it introduces a totally different dynamic of threat and a
Speaker 1: certainly different dynamic of proliferation, and that that's something that
Speaker 1: makes it a whole different ball game when we're considering
Speaker 1: how much of a threat it is and what kind
Speaker 1: of limits we should put on it in international agreements,
Speaker 1: so much the same way that we debate how to
Speaker 1: make air travel safer while people are are dying by
Speaker 1: the thousands on the road in traffic accidents, we're debating
Speaker 1: autonomous conventional weapons, which does matter, while autonomous cyber weapons
Speaker 1: are already here and already fighting and potentially capable of
Speaker 1: doing far more harm and killing more people than a
Speaker 1: robot with a gun could. But I think the availability
Speaker 1: heuristic is at play again here because it's it's just
Speaker 1: that the damage done by autonomous digital weapons like these
Speaker 1: computer viruses and worms and malicious bits of code, the
Speaker 1: the type of damage done by these is less visceral,
Speaker 1: it's harder to picture, and it definitely has fewer movies
Speaker 1: about it, and thus it doesn't get the advantage offered
Speaker 1: by the availability heuristic. You get when you get pictures
Speaker 1: of terminators, that's right, When when you think of an
Speaker 1: infrastructure wrecking, malicious program, uh, It's really hard to think
Speaker 1: of in any horror films that that really line up
Speaker 1: with that concept. The best I can come up with
Speaker 1: is Stephen is maximum Overdrive, which is of course Vigures film.
Speaker 1: So that is the most realistic picture of future technology threat. Yeah,
Speaker 1: it's strange how this works. Right, we were talking about
Speaker 1: this a little before we we started rolling. You know,
Speaker 1: it's it's It's been said that any sufficiently advanced technology
Speaker 1: is indistinguishable from magic, right, and you can point to
Speaker 1: it to various bits of mythology, the myths of say,
Speaker 1: humans creating other rational beings. And in the past these
Speaker 1: were this was pure fantasy. Now in the twenty first century,
Speaker 1: it is. Uh, it is far more realistic when we
Speaker 1: look at models of AI and genetic engineering, etcetera. Uh. Likewise,
Speaker 1: maximum overdrive was ridiculous in the nineteen eighties, but as
Speaker 1: we enter into this age that is increasingly defined by
Speaker 1: research into autonomous vehicles or the Internet of things, Uh,
Speaker 1: maximum overdrive is suddenly not so bonkers. It is crazy.
Speaker 1: In the nineteen eighties, people might have said, Okay, maximum
Speaker 1: overdrive is a silly fantasy about the future technology threat,
Speaker 1: and Terminator is a more realistic movie about future technology
Speaker 1: threat and looking at the wilay of the land today,
Speaker 1: I said, I mean, obviously Maximum Overdrive his cartoony, but
Speaker 1: the general picture outlined by the two of them, maximum
Speaker 1: Overdrive might be the more realistic threat. Any sufficiently advanced
Speaker 1: technology is indistinguishable from Maximum Overdrive. But if you're lost
Speaker 1: right now, okay, why do we say Maximum Overdrive, by
Speaker 1: the way, is basically just a ridiculous eighties Stephen King
Speaker 1: movie where like trucks and appliances and everything every electronic
Speaker 1: thing or sometimes just cars and stuff start trying to
Speaker 1: kill mostly cars, and like really badass trucks come to
Speaker 1: life and start killing things. But but also I think
Speaker 1: like toast drivans and lawn mowers and whatnot. So so
Speaker 1: what autonomous cyber weapons have already been deployed? We said,
Speaker 1: this is already a thing that exists. So in his article,
Speaker 1: Sue quote Scott Borg, director in chief economist of the
Speaker 1: US Cyber Consequences Unit, and Borg says quote malicious computer
Speaker 1: programs that could be described as intelligent autonomous agents are
Speaker 1: what steal people's data, build bot nets, lock people out
Speaker 1: of their systems until they pay a ransom. And do
Speaker 1: most of the other work of cyber criminals. So he's
Speaker 1: talking about the fact that generally when they're cyber crime
Speaker 1: going on, it's not a cyber criminal sitting at a
Speaker 1: computer like manually doing stuff to you. They've they've created
Speaker 1: a program that does it autonomously, so they just put it,
Speaker 1: they let it go, and it does its work and
Speaker 1: then they reap the benefits. But there have been plenty
Speaker 1: of examples of this type of warfare in in actual
Speaker 1: international relations. So there's the ducks networm. This is a
Speaker 1: malicious computer worm that attacked computers controlling nuclear center refuges
Speaker 1: in Iran, and it's believed to have slowed down the
Speaker 1: development of the Iranian nuclear program. I think this happened
Speaker 1: around two thousand ten, and no one's in it responsibility,
Speaker 1: but it appears that this was a cyber weapon created
Speaker 1: by the United States and Israel to try to prevent
Speaker 1: Iran from acquiring nuclear weapons. Another example would be the
Speaker 1: want to Cry ransomware worm, which shuts down computers and
Speaker 1: demands payment of money before allowing the computers to be
Speaker 1: made functional again. And want to Cry has caused real damage.
Speaker 1: It attacked computer systems in hospitals at the UK National
Speaker 1: Health Service, among a bunch of other important infrastructure machines.
Speaker 1: Plus I think you could you could, you could reasonably
Speaker 1: argue that there was UH psychological damage inflicted by that
Speaker 1: attack as well. I mean, maybe not as much as
Speaker 1: like a full blown UH cyber terrorism event, but it's
Speaker 1: certainly captured headlines and it was It was the kind
Speaker 1: of threat that's kind of central to the the appeal
Speaker 1: of a terroristic act. It makes everyone feel like they
Speaker 1: could be a potential victim. Yeah, yeah, totally, and in
Speaker 1: fact a lot of people you could be a potential victim.
Speaker 1: I mean, this is a thing that's that's worth being
Speaker 1: concerned about and thinking about what defensive measures could we
Speaker 1: put in place. Again, no one has plausibly admitted responsibility
Speaker 1: for the want to Cry attack, but a lot of
Speaker 1: analysts thinks science point to it being a project of
Speaker 1: the North Korean government. Now, these examples demonstrate that, like
Speaker 1: any weapon and autonomous UH, an autonomous cyberweapon can be
Speaker 1: used for various types of conflicts, right like in the
Speaker 1: case of the stuck S networm. Whatever you think about
Speaker 1: the U S and Israel, I think most people would
Speaker 1: probably agree that they're glad somebody figured out a non
Speaker 1: violent way to stop an authoritarian regime from making nuclear weapons.
Speaker 1: But then again, the same technology could be used by
Speaker 1: the same actors or by others to attack things that
Speaker 1: are you know, that would get less sympathy from people
Speaker 1: around the world. Attack critical infrastructure anywhere, power, water, security, hospitals, telecommunications, media, banks,
Speaker 1: and financial systems, transportation. I mean, all the these things
Speaker 1: are increasingly connected now. Yeah, and we've only seen partial
Speaker 1: use of these cyber weapons, certainly nothing of the magnitude
Speaker 1: of of a full scale cyber war. Uh, something that
Speaker 1: that you know that some futurists and cybersecurity analysts have
Speaker 1: have written about and discussed like what would this look like?
Speaker 1: Uh though though that though many of them stressed that
Speaker 1: even these limited uses, they can build legitimacy for the
Speaker 1: development of such weapons, as well as for the development
Speaker 1: of national cyber response teams. And it's inevitable too that
Speaker 1: that such attacks will lean increasingly into machine learning and
Speaker 1: the use of AI. So what we've seen so far
Speaker 1: is is sadly just like the tip of the iceberg unfortunately. Yeah,
Speaker 1: and I want to talk more about that, especially the
Speaker 1: role of machine learning and AI later on. But um,
Speaker 1: one of the things I really want to stress is
Speaker 1: that these days in the modern world, attacks on infrastructure
Speaker 1: are not necessarily just inconveniences. It's not like thinking, oh, well,
Speaker 1: I just lost internet for a day. It was actually
Speaker 1: kind of nice because I went outside and took a walk. Yeah.
Speaker 1: I mean, you might have lost power in your neighborhood before,
Speaker 1: and you know it was an inconvenience or something, but
Speaker 1: you were fine. But at a large enough scale, attacks
Speaker 1: like these on infrastructure that are totally plausible under under
Speaker 1: the world we live in today, they're pretty much guaranteed
Speaker 1: to result in people facing serious material loss, injury, and death.
Speaker 1: As an example for comparison, you could look at the
Speaker 1: tragedy that's happened in Puerto Rico following the following the
Speaker 1: landfall of Hurricane Maria in September. Now, when that happened,
Speaker 1: when Puerto Rico was hit by the hurricane, it effectively
Speaker 1: knocked out Puerto Rico's electrical power grid and temporarily put
Speaker 1: a stop to lots of services in its aftermath, including electricity,
Speaker 1: sewage treatment, some types of health and medical care, clean
Speaker 1: tap water, and so forth. And in a civilization that's
Speaker 1: built on the assumption of continued access to service is
Speaker 1: like power and clean water. Sudden interruption of those services
Speaker 1: is devastating and genuinely lethal. And while we we can't
Speaker 1: be sure of the exact number of people who died
Speaker 1: as a result of the devastation caused by the hurricane,
Speaker 1: there have been some estimates and it seems like a
Speaker 1: lot of people survived the initial storm but died in
Speaker 1: the weeks and months afterwards from complications in the aftermath.
Speaker 1: A lot of these possibly due to interruptions and services, healthcare, disease,
Speaker 1: and so forth. Harvard study published in the New England
Speaker 1: Journal of Medicine used survey data from households in Puerto
Speaker 1: Rico to try to estimate the amount of the human impact,
Speaker 1: and they found that among respondents there was a mortality
Speaker 1: rate of about fourteen point three deaths for thousand persons
Speaker 1: in September twenty through the end of seventeen, and this
Speaker 1: yielded somewhere between seven hundred and ninety three and eight thousand,
Speaker 1: four hundred nine excess deaths above the normal rate for
Speaker 1: this period. That the mean of those numbers would be
Speaker 1: like four six hundred people and that was within that
Speaker 1: range with confidence interval. So the authors think the death
Speaker 1: holl could actually be higher because the survey data is
Speaker 1: contaminated by survivor bias. Right, people who died were not
Speaker 1: able to respond to the survey. Now, we don't know
Speaker 1: for sure if that estimate is accurate. There could be
Speaker 1: flaws in the method. But if it's accurate, that's equivalent
Speaker 1: to a sixty two percent increase in the mortality rate
Speaker 1: as compared to the same period in st And we
Speaker 1: should acknowledge there's been a lot of anger about the
Speaker 1: way the US government handled the Hurricane Maria aftermath, with
Speaker 1: charges that it basically didn't do enough to get essential
Speaker 1: infrastructure and services back online fast enough, and that people
Speaker 1: actually suffered and died as a result of not having
Speaker 1: this infrastructure online, arguably in a manner that would have
Speaker 1: been far different had this been, uh, you know, the
Speaker 1: aftermath of a terrorist attack or or something that was
Speaker 1: a little more centralized, personified, or too or tied in
Speaker 1: with these more pervasive fear. Yeah, there there does appear
Speaker 1: to be a weird psychology in the way societies respond
Speaker 1: to threats, and that all different kinds of biases can
Speaker 1: provide differential motivation in how much effort we put into
Speaker 1: fixing the problem afterwards and helping people. So, yeah, it's
Speaker 1: been it's been a horrible tragedy there, But we should
Speaker 1: think about that. This kind of tragedy is what happens
Speaker 1: due to a random attack by the weather. One can
Speaker 1: only wonder what it would look like if, say, infrastructure
Speaker 1: was attacked not randomly by the weather, but by a
Speaker 1: malicious party intentionally trying to do as much damage as
Speaker 1: possible with digital weapons. Yeah, I mean obviously a number
Speaker 1: of possibilities instantly come to mind. Targeting infrastructure during the
Speaker 1: very depths of winter, for example, right, or during a key,
Speaker 1: say politically sensitive times like during an election. Yeah. Um, yeah,
Speaker 1: I know I've read that there there's belief that there
Speaker 1: was Russian digital autonomous agent attacks again to say some
Speaker 1: Ukrainian power infrastructure during times of political upheaval and civil unrest,
Speaker 1: which you know, it only makes things worse the timing
Speaker 1: of those attacks. And a scary part is that we're
Speaker 1: constantly connecting more and more devices to the Internet. Yeah,
Speaker 1: like our refrigerators to the internet, cars, home appliances, medical devices,
Speaker 1: and even medical implants. Every year the connectedness does not shrink,
Speaker 1: it grows, and thus every year the world's connectedness and
Speaker 1: vulnerability to cyber attacks, you could argue, becomes even greater,
Speaker 1: almost as if we really want maximum overdrive to happen,
Speaker 1: as if we were saying, you know, you know that movie,
Speaker 1: that movie that came out in the eighties that was
Speaker 1: so ridiculous, Uh, let's do that. Let's just go ahead
Speaker 1: and put our refrigerator and our toaster on the Internet
Speaker 1: and have them talk to each other. I mean, it's
Speaker 1: hard to predict exactly what attacks would look like, though
Speaker 1: there has been a lot of work on this, and
Speaker 1: maybe if we revisit this topic in the future, we
Speaker 1: could just more explicitly explore scenarios that have been talked
Speaker 1: about by cybersecurity x birds. Yeah, in particular that the
Speaker 1: concept of full blown cyber warfare. What that would look
Speaker 1: like if you had nation states actively and at least
Speaker 1: semi openly engaging in attacks and counterattacks against each other,
Speaker 1: with the possibility then of actual military attacks on top
Speaker 1: of that. Yeah, totally. But I mean, one thing I
Speaker 1: really want to drive home, if you take anything away
Speaker 1: from this episode, I wanted to be that attacks by
Speaker 1: autonomous digital agents just computer viruses. Basically in the common understanding,
Speaker 1: worms and stuff like that, things that don't have a
Speaker 1: gun or a missile or an explosive attacks to them
Speaker 1: can be just as dangerous, just as deadly and probably
Speaker 1: more so than conventional weapons can be. Absolutely and remember
Speaker 1: another thing is that most of these attacks can, at
Speaker 1: least in theory, function without ongoing human input or direction. Right,
Speaker 1: It's the set it and forget it model of warfare.
Speaker 1: Like you, you make an autonomous digital agent, a computer virus,
Speaker 1: a worm, whatever to attack infrastructure out there, and you
Speaker 1: might be able to sign it so that you don't
Speaker 1: need to go back and do anything to it later.
Speaker 1: You don't need to maintain it. Uh directed it functions
Speaker 1: on its own. That's what autonomous means. Yeah, it's it's
Speaker 1: basically the more perfect version of the nuclear torpedo that
Speaker 1: I talked about earlier. Like the the idea there, of course,
Speaker 1: is that it's out there, it's not communicating back, and
Speaker 1: then it could and then it strikes its target, detonates
Speaker 1: and causes you know, lasting radioactive damage to a particular
Speaker 1: area uh and and uh and loss of life um
Speaker 1: and damage to the infrastructure. Etcetera. But we're talking about
Speaker 1: digital agents. It would be able to achieve many of
Speaker 1: those same goals. Uh, but without the same risk of
Speaker 1: components aging and uh and uh and and the torpedo
Speaker 1: itself eventually dying. Right, And the one thing about what
Speaker 1: you just said does make me want to emphasize, I'm
Speaker 1: not suggesting that cyber warfares, say, worse than a nuclear
Speaker 1: attack would be. I don't think that's true. I mean, obviously,
Speaker 1: a full scale attack by conventional or new clear weapons
Speaker 1: would be a worse outcome than a cyber attack. But
Speaker 1: I think a cyber attack is a more a threat
Speaker 1: that we need to be even more concerned about because
Speaker 1: it can be very, very realistically destructive, and it's very
Speaker 1: likely to happen because it's already happening. I mean, it's
Speaker 1: something that you can very easily see being deployed, much
Speaker 1: more so than you can imagine, say, nuclear war between
Speaker 1: currently nuclear armed countries. Right. Yeah, When we try to
Speaker 1: imagine an the anonymous use of a nuclear even a
Speaker 1: powerful enough conventional weapon, um, it's it's far less less
Speaker 1: likely compared to the anonymous use of a particularly volatile
Speaker 1: cyber weapon, because we haven't really seen the former, and
Speaker 1: we've definitely definitely seen the ladder exactly. So a lot
Speaker 1: of this article that I mentioned earlier that got me
Speaker 1: thinking about this by Jeremy Sue it it's simply highlighting
Speaker 1: the fact that governments and international organizations are not having
Speaker 1: enough conversation about sidelines for how to control the threat
Speaker 1: of autonomous cyber weapons. We are having some international conversations
Speaker 1: about what to do about autonomous conventional weapons, the robot
Speaker 1: with a gun, We are not having enough conversations about
Speaker 1: what to do about controlling autonomous cyber weapons. And these
Speaker 1: autonomous cyber weapons are already here and being used. They're
Speaker 1: easier to create and deploy and potentially in many cases
Speaker 1: more destructive than autonomous conventional weapons. One of the experts
Speaker 1: that Sue quotes in his article is Kenneth Anderson, a
Speaker 1: professor of law at American University who specializes in national security.
Speaker 1: One of the experts Sue quotes in his article is
Speaker 1: Kenneth Anderson, a professor uh law professor at American University
Speaker 1: who does national security issues. And Anderson says where is
Speaker 1: the band killer apps? In GEO advocacy campaign demanding a
Speaker 1: sweeping total ban on the use, possession, transferred to transfer,
Speaker 1: or development of cyber weapons all the features found in
Speaker 1: today's Stop Killer Robots campaign. I think that's a good question. Yeah,
Speaker 1: I absolutely agree, and I mean I hope that that
Speaker 1: someone is putting it together. I would very much, especially
Speaker 1: after this episode, like to support such a campaign. Yeah.
Speaker 1: I mean, we've got nuclear non proliferation agreements and stuff
Speaker 1: like that. It seems more than reasonable to be trying
Speaker 1: to work on a similar framework for cyber weapons, and
Speaker 1: and nuclear proliferation treaties have to a large extent worked.
Speaker 1: I mean, for starters, we have not had a nuclear war,
Speaker 1: which some commentators have said is nothing short of a miracle. Um,
Speaker 1: we've seen the nuclear stockpiles of of of the United
Speaker 1: States and Russia uh deplete over the decades, and hopefully
Speaker 1: that will remain the trend. Again, We've seen similar scenarios
Speaker 1: with biological and chemical agents as well. So something could
Speaker 1: be done here if we act and we actually uh,
Speaker 1: you know, push for regulations to be made. You know, Robert,
Speaker 1: one of the things you mentioned earlier is about the
Speaker 1: difficulty in controlling the proliferation of digital agents as compared
Speaker 1: to conventional weapons, right, like digital agents, a computer worm
Speaker 1: or a virus something like that can replicate in the
Speaker 1: wild in some scenarios. So I would say that this
Speaker 1: is also a case where just like with nuclear weapons,
Speaker 1: it's like if if two countries with nuclear weapons go
Speaker 1: to war, it's not just a problem for the people
Speaker 1: in those two countries, it's a problem for the entire world.
Speaker 1: And I would say that software based digital warfare agents
Speaker 1: that operate on the Internet are similarly a problem for
Speaker 1: the entire world because you don't know potentially who they
Speaker 1: could harm on the sidelines. Yeah, I mean we with
Speaker 1: with nuclear, biological and chemical agents. Obviously, we all share
Speaker 1: an atmosphere, we all share a global environment, and when
Speaker 1: we wage devastating war within that environment, we run the
Speaker 1: risk of destabilizing UH everything and harming ourselves in the process.
Speaker 1: And with the digital technology, we have created another environment
Speaker 1: that we have we have made ourselves dependent upon uh
Speaker 1: and and we run the risk of doing the same thing,
Speaker 1: poisoning this new ocean that we've created, especially when we
Speaker 1: consider the possibility of these autonomous agents becoming more adaptive,
Speaker 1: and I think we should explore that after we take
Speaker 1: a break. Thank thank you, thank you, Alright, we're back.
Speaker 1: We're talking about the future, the dangers, the risk, and
Speaker 1: how we should how we should really handle the risk
Speaker 1: and handle anxiety over the prospect of of cyber warfare
Speaker 1: in the future. So here's something I think is really
Speaker 1: worth considering, and it's the convergence of cyber warfare and
Speaker 1: cyber weapons with machine learning and artificial intelligence. Because we've
Speaker 1: already got autonomous cyber weapons, these these malicious bits of
Speaker 1: software out there that you know, enact warfare on on
Speaker 1: infrastructure of opposing forces in the world, and we've got
Speaker 1: machine learning and AI, and there's really no reason these
Speaker 1: capabilities could not be combined. So this is a future
Speaker 1: that combines the devastating capabilities of cyber warfare with the
Speaker 1: attack dynamics of something like biological or germ warfare. This
Speaker 1: is a future that should worry us. Autonomous cyber weapons
Speaker 1: that can learn, adapt and change on their own. And
Speaker 1: I think it's not hard to see how we could
Speaker 1: potentially go down this road of developing dangerous AI cyber
Speaker 1: weapons that alter themselves through machine learning and get out
Speaker 1: of control. I thought of just a couple of scenarios
Speaker 1: that seem plausible to me, at least one would be
Speaker 1: cyber terrorism. You know, some forces are not rational actors
Speaker 1: seeking to limit the harm they cause and preserve their interests.
Speaker 1: But some people are just simply interested in causing harm
Speaker 1: and chaos. I imagine how bad things could be if
Speaker 1: somebody like the Uni bomber had computer skills to wage
Speaker 1: lone wolf cyber warfare of this kind. But then I
Speaker 1: also think about dangerous autonomous AI that begins as a
Speaker 1: defensive measure to protect against cyber attacks. So most harmful
Speaker 1: military technologies, Robert, I bet you would agree, most of
Speaker 1: these harmful technologies and strategies in world history have not
Speaker 1: come from people claiming to be developing offensive weapons to
Speaker 1: maliciously attack on suspecting victims. They've been developed under a mindset,
Speaker 1: whether this is really objectively fair or not, of defense.
Speaker 1: People think like, I'm under threat, I need to do
Speaker 1: something to protect myself exactly. I mean this is that
Speaker 1: this is then the arms race throughout history, right. Uh,
Speaker 1: if the other side has a large slingshot, I need
Speaker 1: an equally size size sling shot. I need something that
Speaker 1: is a deterrent, otherwise they're just going to take advantage
Speaker 1: of me. Yeah, So what what looks like offensive threatening
Speaker 1: behavior from your perspective. From the other person's perspective is like, look,
Speaker 1: I just got to defend myself. Uh So, usually by
Speaker 1: the time you recognize you've been the victim of a
Speaker 1: cyber attack, a lot of damage is already done. So
Speaker 1: what if in the future we decide we need autonomous,
Speaker 1: adaptive defensive cyber weapons to protect us against offensive cyber weapons,
Speaker 1: something like an immune system for our infrastructure, the equivalent
Speaker 1: of deployed white blood cells, you know, T cells and
Speaker 1: B cells and so forth, to autonomously detect, hunt, and
Speaker 1: kill malicious autonomous cyberweapons the same way that white blood
Speaker 1: cells in your body behave kind of like an adaptive,
Speaker 1: independent organism within the body, hunting and killing autonomously in
Speaker 1: the bloodstream. Well that sounds great, Joe, But now you're
Speaker 1: right back to sky net, Like this sounds exactly like
Speaker 1: sky net, except it is like sky net is just
Speaker 1: is the the clunkier metaphor for what the future may become. Well,
Speaker 1: it's not terminators holding guns, but it is a distributed
Speaker 1: defense network. At this point, you're you're talking about like
Speaker 1: an anti virus virus, right, or the possibility of an
Speaker 1: of an immune system turning against itself, turning against its host,
Speaker 1: which which of course we see in the biological Well absolutely, yeah,
Speaker 1: that's exactly where I was going with that. So you've
Speaker 1: got autoimmune diseases, if you've got an immune system, you
Speaker 1: run the risk of the immune system, in say cases
Speaker 1: like arthritis or type one diabetes or MS, misidentifying friendly
Speaker 1: tissue as something that needs to be defended against and
Speaker 1: attacking its own body, you know, turning, turning parts of
Speaker 1: the body into innocent victims. Except the nature of the
Speaker 1: internet and the connected world means that if one system
Speaker 1: develops the digital equivalent of an autoimmune disease, potentially anybody
Speaker 1: could catch it. I'd also pair this this kind of
Speaker 1: scary scenario with our episode last year about neurosecurity. The
Speaker 1: increasing vulnerabilities were accumulating as you make the connections between
Speaker 1: your digital services and your nervous system more robust. Indeed,
Speaker 1: the idea that you could have your your brain implant
Speaker 1: hacked or your pacemaker device hacked, etcetera. So anyway, just
Speaker 1: entertaining these scenarios and thinking about the risks posed makes
Speaker 1: me think, should we have international cyber warfare non proliferation
Speaker 1: treatise the same way we've got nuclear non proliferation treaties.
Speaker 1: I mean, absolutely, as we've been driving home these these
Speaker 1: are legitimate threats and therefore they you know, we should
Speaker 1: be taking steps to prevent um it from happening. As
Speaker 1: far as what countries could do, countries around the world
Speaker 1: could do to protect themselves, I mean, I wonder is
Speaker 1: there an option other than just trying to revert to
Speaker 1: a world of decreased connectedness, And would societies ever do
Speaker 1: that without being forced to by some kind of tragedy
Speaker 1: where Like decreased connectedness of course would mean fewer rather
Speaker 1: than more systems can be accessed by the Internet. Where
Speaker 1: you might have crucial systems for controlling infrastructure kept offline
Speaker 1: or in isolated networks, they're not plugged into the Internet,
Speaker 1: and so it would be a lot harder to infect
Speaker 1: them with some kind of autonomous digital weapon. Uh though,
Speaker 1: I wonder if that's possible even would it takes some
Speaker 1: kind of visceral disaster that calls to mind images through
Speaker 1: the availability heuristic to make people think this is worth doing.
Speaker 1: And it's a good point because I mean, I've certainly
Speaker 1: read predictions for the sort of digital future, both in
Speaker 1: sci fi and just in general futurism, the idea that
Speaker 1: the Internet will become say more national, more regional, more
Speaker 1: more layered, and indeed and they need less worldwide. Um uh.
Speaker 1: The alternative, of course, is as a continuation of what
Speaker 1: we've already been doing, essentially building what cybersecurity expert Bruce
Speaker 1: Schneier referred to as a as a worldwide computer. We're
Speaker 1: building a worldwide computer, and it's uh, it's susceptible to
Speaker 1: attack at every level, from baby monitors to uh to
Speaker 1: nuclear power plants. I get, you know. I would hope
Speaker 1: that we're that that we that we will grow into
Speaker 1: this globally connected world. It will be like there will
Speaker 1: be the people deserving of such a worldwide computer. But
Speaker 1: if not, then yeah, we deserve the regional model. I
Speaker 1: guess well, I don't like the regional model. I mean,
Speaker 1: I understand that some decrease in connectedness might be necessary
Speaker 1: to prevent attacks, but then again, I like the world
Speaker 1: connected model in terms of communication, I mean, all the
Speaker 1: good stuff about connection between cultures. I don't think I
Speaker 1: buy into the nationalist mindset that says we should only
Speaker 1: be talking and interacting with people within our own national
Speaker 1: national boundaries or our own culture that seems very limiting.
Speaker 1: I mean, it's a wonderful thing to be able to
Speaker 1: communicate across borders and with people all around the world.
Speaker 1: That that's something I love about what we get to do, right, Uh,
Speaker 1: And so I don't know that that sounds like a
Speaker 1: horrible thing to do. But then again, I mean I
Speaker 1: wonder if there are ways to to leave open the
Speaker 1: good channels while preventing people from from using digital exchange
Speaker 1: to hurt one another. Well, I mean, part of this
Speaker 1: goes back to our past discussions on just the the
Speaker 1: the the origins of the Internet. Some would argue that
Speaker 1: that that one of the big problems is just uh
Speaker 1: security issues with the Internet itself and the idea that
Speaker 1: you have this thing that was built as a as
Speaker 1: essentially a private network for for developers that has been
Speaker 1: bloated out into this glow system that it was really
Speaker 1: never meant to be. So we we need a new Internet,
Speaker 1: we need a new human race. Uh, just for starters,
Speaker 1: those are two things that would help. Yeah. Well, so
Speaker 1: if there there are any takeaways from this episode, I
Speaker 1: would say I think it's that people should should understand
Speaker 1: the relative severity of different types of autonomous technological weapon
Speaker 1: threats like that that cyber warfare is not just an inconvenience.
Speaker 1: It's not just like, oh darn, the power went out
Speaker 1: for a day, or oh, you know, there was a
Speaker 1: d d O S attack on the website I wanted
Speaker 1: to go to and it went down. These could be
Speaker 1: real serious. This could be the warfare of the future
Speaker 1: and every bit as serious as conventional warfare. And uh
Speaker 1: and so that's worth considering, and it's so it's worth
Speaker 1: promoting people who have good ways of thinking about this.
Speaker 1: If if you know of cybersecurity experts, the kind of
Speaker 1: people who are doing the best thought about this, coming
Speaker 1: up with ways of thinking about defenses, especially as we've
Speaker 1: talked about the kinds of defenses that don't lead to
Speaker 1: a you know, a shutdown of international communication and and
Speaker 1: this worse soft world. Shine a light on those people.
Speaker 1: I want to know what our best options are. Who's
Speaker 1: doing the best thinking about this right now? Yeah, let
Speaker 1: us know. We'll do our part to shine our light
Speaker 1: on those people as well. All right, So there you
Speaker 1: have it. Um as always stuff to blow your mind.
Speaker 1: Dot com that is the mothership, that's where you'll find
Speaker 1: this and other episodes of note, we've had a number
Speaker 1: of them that have dealt with technology and warfare uh
Speaker 1: and likewise, UH. There are a number of issues in
Speaker 1: this episode that we could easily return to, such as
Speaker 1: cyber warfare even the more traditional autonomous weapon designs that
Speaker 1: we could do multiple episodes on that as well. So
Speaker 1: as we said, I think one thing definitely worth exploring
Speaker 1: would be the more specific nitty gritty of the scenarios
Speaker 1: imagined by cyber cyber warfare experts, like what's the most
Speaker 1: plausible thing that could happen and how could it be prevented? Um?
Speaker 1: So hey, yeah, check it out Stuff to blow your
Speaker 1: mind dot com UH, and you'll find links to our
Speaker 1: various social media accounts there as well. And if you
Speaker 1: want to support the show, rate and review us wherever
Speaker 1: you have the ability to do so. Big thanks as
Speaker 1: always to our wonderful audio producers Alex Williams and Torry Harrison.
Speaker 1: If you would like to get in touch with us
Speaker 1: with feedback about this episode or any other to let
Speaker 1: us know topic you'd like us to cover in the future,
Speaker 1: to just send us your your thoughts to say hi,
Speaker 1: let us know where you listen, from, how you found
Speaker 1: out about the show. Any of that, You can always
Speaker 1: email us at Blow the Mind and how stuff Works
Speaker 1: dot com for more on this and thousands of other topics.
Speaker 1: Is it how stuff works dot com. B b P
Speaker 1: has a found back about a protect