Speaker 1: Bloomberg Audio Studios, Podcasts, Radio News.
Speaker 2: Hello and welcome to another episode of the Odd Lots Podcast.
Speaker 3: I'm Joeisenthal and I'm Tracy Alloway.
Speaker 2: So Tracy recording this March twenty fourth, And of course
Speaker 2: almost all of our episodes.
Speaker 4: Lately have been about the war in Iran.
Speaker 2: But what's interesting, or what's a little weird is that
Speaker 2: just prior to the war, literally days or maybe hours,
Speaker 2: the biggest story in the world was actually about defense
Speaker 2: and you know the DoD that's right.
Speaker 3: So you are referring to anthrop a. Yeah, and it's
Speaker 3: disagreement to put it mildly in the Department of War.
Speaker 4: Yeah, exactly.
Speaker 2: This was the biggest story going right up on the
Speaker 2: eve of the war in Iran. And of course obviously
Speaker 2: there was this contract topics technology was used by the
Speaker 2: Defense Department. So it was not a disagreement about the
Speaker 2: use of AI per se in war, but the question
Speaker 2: of the degree to which AI could be used for
Speaker 2: autonomous weapons systems on their own without a human in the.
Speaker 5: Loop and surveillance and surveillance this was another key element.
Speaker 3: But you're right, so we've heard this expression autonomous weapons, yeah,
Speaker 3: pop up more and more especially in recent days. And
Speaker 3: I've a lot of questions over what exactly that means,
Speaker 3: because my impression is the US military certainly has been
Speaker 3: using AI for some time.
Speaker 5: Yes, and so we're really talking about.
Speaker 3: Degrees here of autonomy, right, And so if you think
Speaker 3: about an autonomous weapon, I think your mind could go
Speaker 3: fully terminator and you know there's like a murder robot
Speaker 3: out there that's making its own decisions on which.
Speaker 5: People or places to target.
Speaker 3: And then you get levels below that, right where AI
Speaker 3: is kind of helping humans to come up with strategic decisions.
Speaker 4: Right.
Speaker 2: So, if there is a missile coming and you have
Speaker 2: a missile defense system, I don't think you want a
Speaker 2: human in the loop. Is like, Okay, here are the
Speaker 2: coordinates X, y Z that we think it's going to hit.
Speaker 2: At this point in time, we think the missile will
Speaker 2: be here, Are you cool with firing it?
Speaker 4: I think everyone's.
Speaker 2: Probably okay with that level of autonomy. But I have
Speaker 2: a feeling that to your point exactly a lot of
Speaker 2: this discussion, and maybe it's core to what Anthropic and
Speaker 2: the Department of Defense who are disagreeing on, I have
Speaker 2: a feeling a lot of this is going to come
Speaker 2: down to definitions. My guess is that there is not
Speaker 2: one shared agreement of this is an autonomous weapon system
Speaker 2: and this one is.
Speaker 3: Not absolutely And of course there are also questions over
Speaker 3: exactly how places like the Department of Defense, not only
Speaker 3: how they define it, but once they have those definitions,
Speaker 3: whether or not certain companies trust them. Yes, so stick
Speaker 3: to those policies, because the US will say, well, our
Speaker 3: policy is not to survey are citizens at the moment,
Speaker 3: So if you're anthropic, you don't need to worry about that.
Speaker 3: Clearly anthropic feels otherwiseer say they do. So there are
Speaker 3: all these really interesting thematic questions that pop up from all.
Speaker 4: Of this totally.
Speaker 2: And then there's the question of Okay, here's a technology
Speaker 2: and the government says, we believe that we can use
Speaker 2: this to make the country safe. For what, You're not
Speaker 2: going to let us do it like private corporation. There's
Speaker 2: some very interesting questions about the role of corporate power
Speaker 2: visa vi the government and so forth.
Speaker 4: Anyway, this is something that has.
Speaker 2: Become even more timely in the reports, even in the
Speaker 2: early days of the Iran War, about these AI systems
Speaker 2: having been used perhaps in target selection, but we don't
Speaker 2: really know. None of the reporting is like that.
Speaker 6: Clear.
Speaker 2: I don't think that they're going out and advertising this strike.
Speaker 5: This is exactly how we're using AI.
Speaker 2: This is exactly how we're using AI, and so forth.
Speaker 2: But this is obviously a huge debate and war aside.
Speaker 2: It's totally going to grow, and just as an AI
Speaker 2: is going to grow, it seems in so many different areas. Anyway,
Speaker 2: I'm really excited to say, we really do have the
Speaker 2: perfect guest, someone who's been writing and thinking about this
Speaker 2: stuff for a long time. When we talked to an
Speaker 2: AI expert, I marked the dividing line AND's like, where
Speaker 2: you talking about AI prior to when Chad Gipt was released,
Speaker 2: as like, I take a little bit more seriously the
Speaker 2: people who are in this space prior to November twenty
Speaker 2: twenty two. Anyway, I'm very excited to say we're gonna
Speaker 2: be speaking with Paul Shari. He's the executive vice president
Speaker 2: at the Center for a New American Security and he's
Speaker 2: the author of two books related to this, one as
Speaker 2: the most recent for Battlegrounds Power in the Age of
Speaker 2: Artificial Intelligence, and then prior to that, he is the
Speaker 2: author of Army of non autonomous weapons and the future
Speaker 2: of war. He was previously in the Office of the
Speaker 2: Secretary of Defense, also a previous Army ranger, so truly
Speaker 2: the perfect guest. So, Paul, thank you so much for
Speaker 2: coming on odlots.
Speaker 6: Oh thank you for having me. Very excited to be here.
Speaker 4: What do you we start?
Speaker 2: I mentioned I had a feeling that maybe the definition
Speaker 2: of an autonomous weapon is a contested one. But if
Speaker 2: I say to you, what's an autonomous weapon, what's an
Speaker 2: autonom this weapon?
Speaker 6: So I think you're right from the beginning that there
Speaker 6: is not a unified definition that everyone agrees on. The
Speaker 6: Defense Department has their definition that's written in their policy.
Speaker 6: I think conceptually, I think the distinction really is a
Speaker 6: weapon that is choosing its own targets on the battlefield,
Speaker 6: and it's not where we are today. Right now, today
Speaker 6: people are choosing those targets. But it is kind of
Speaker 6: a spectrum because we do have examples of weapons that
Speaker 6: have some measure of autonomy. A good analogy might be
Speaker 6: self driving cars, where conceptually, like, okay, a self driving
Speaker 6: car would be where the AI is driving the car,
Speaker 6: but then you get into an actual car today and
Speaker 6: a lot of them have intelligence cruise control, automatic breaking,
Speaker 6: automated self parking, They have all these like automated features
Speaker 6: that are kind of creeping you in this direction where
Speaker 6: the AI is taking over more and more control for
Speaker 6: what the vehicle can do. And it's actually pretty similar
Speaker 6: thing in the military space as well.
Speaker 3: So Joe mentioned that had we been having this conversation
Speaker 3: even a month ago now, it probably would have had
Speaker 3: fewer concrete examples of AI enabled weaponry, let's say, or strategy.
Speaker 3: When the Pentagon talks about its advanced AI tools that
Speaker 3: it's deploying for the Iran conflict, what are some examples
Speaker 3: that you're seeing right now that are different to say,
Speaker 3: maybe just a year ago when we had another Iran conflict.
Speaker 6: Right, So, there's a couple ways in which the Pentagon
Speaker 6: is using AI right now. One is narrow AI systems
Speaker 6: that have around for over a decade now that do
Speaker 6: image classification. For example, So this was the military's original
Speaker 6: project Maven almost a decade ago, where they took machine
Speaker 6: learning image classifiers to sift through drone video feeds and
Speaker 6: saddle images to identify objects. Okay, here's a building, here's
Speaker 6: a person, here's a vehicle, that's pretty mature technology. Now,
Speaker 6: what's come out in just the last couple of weeks
Speaker 6: that's really quite interesting is that in the midst of
Speaker 6: this huge, messy public breakup between Anthropic and the Pentagon,
Speaker 6: we found out that in fact, Anthropics AI tools are
Speaker 6: being used by the US military to help plan the
Speaker 6: war against Iran. That's obviously a different kind of AI tool,
Speaker 6: AI Large language models, AI being used to write code
Speaker 6: AI agents, and that's being used in a different ways,
Speaker 6: really helping intel analysts sift through just the massive amounts
Speaker 6: of data that the US military has. And so you
Speaker 6: can imagine the problem that the military is facing right
Speaker 6: now when they're looking at targets in Iran. US military
Speaker 6: has flown over six thousand sorties against Iran. The Iranian
Speaker 6: military architecture is degraded in a lot of ways. US
Speaker 6: militaries are bombed a lot of targets. There are mobile targets,
Speaker 6: senior Iranian commanders, mobile missile launchers and air defense systems
Speaker 6: and drone launchers. US militter has got to bring off
Speaker 6: it in f together and find out where are these
Speaker 6: targets right now and where is there an aircraft that
Speaker 6: is the right bombs on it to take these targets out.
Speaker 6: And that's how AI is being used to help basically
Speaker 6: process and understand all that information.
Speaker 2: When I think about the description that you gave for that,
Speaker 2: I sometimes think like, could it be that?
Speaker 1: Now?
Speaker 2: I don't think that using anthropics technology mean so that
Speaker 2: they go into Claude dot AI and say, give us
Speaker 2: a list of suitable targets for sorties. But could it
Speaker 2: be something like that? But I'm sure there's a different
Speaker 2: interface and so forth. But is that a completely ridiculous
Speaker 2: way of essentially framing the service that AI is providing
Speaker 2: right now?
Speaker 6: So the way that these AI tools are being integrated
Speaker 6: are through a existing system called the Maven Smart System okay,
Speaker 6: which is built by Palenteer, and that fuses all this
Speaker 6: data together. So you basically have an existing architecture for
Speaker 6: data management for Intel analysts that the Milita already has
Speaker 6: that brings together all these different forms of data. You
Speaker 6: might have satellite imagery, geolocation, data, signals, intelligence, other forms
Speaker 6: of information. That's pretty great for Intel analysts, but that's
Speaker 6: also really unwieldy because how does a human understand all
Speaker 6: that data and process it? And that's where the large
Speaker 6: language model tools, whether it's Clawed or other companies can
Speaker 6: be valuable. Is that could be a way for a
Speaker 6: human to interact with that data, to basically task a
Speaker 6: large language model to say, Okay, here's a bunch of
Speaker 6: data I'm giving you. I want you to look for
Speaker 6: intersections and things, right. I want you to look for
Speaker 6: a place where we have satellite imagery and some other
Speaker 6: forms of intelligence that can help identify the location of
Speaker 6: you know, some missile launcher for example, and then humans
Speaker 6: can look at that and help one just like find
Speaker 6: where are all these targets, and then it helpfully planning too.
Speaker 6: A human could say, okay, here's this list of potential
Speaker 6: targets that I have. Now they're scattered all over run
Speaker 6: around is a really big country. I want to map
Speaker 6: these two locations for US aircraft at different bases across
Speaker 6: the region. What are available aircraft and what are available
Speaker 6: munitions on those aircraft? The commused to take out those
Speaker 6: targets to help build a strike package. So like, the
Speaker 6: AI is definitely being used to help understand the battlespace
Speaker 6: and to plan operations. But in I would say ways
Speaker 6: that are pretty narrowly directed by people It's not quite
Speaker 6: as simple as like, dump all this data into a
Speaker 6: context winder for LM and say, oh, AI figure it out. Okay,
Speaker 6: people are asking the AI some really specific questions, So.
Speaker 3: I'm thinking how to phrase this question diplomatically, But I
Speaker 3: get that the difference between fully autonomous weapons is, you know,
Speaker 3: the human as a decision maker in the current setup,
Speaker 3: how meaningful is the human actually? Like, what's your sense
Speaker 3: of it? Because I'm imagining if you're an intelligence officer
Speaker 3: and you're getting realms and reams data from Iran and
Speaker 3: you ask the AI to pick out certain patterns or
Speaker 3: identify potential strategic targets, how much due diligence are you
Speaker 3: actually doing on what that model spits out. Because of course,
Speaker 3: the tendency when a lot of people use LLM certainly
Speaker 3: is just you accept what it shows you on the screen.
Speaker 2: And just to add on to Tracy's question, because there's
Speaker 2: on to go, which is then in the early.
Speaker 4: Days of the war, we hit that school.
Speaker 2: Yeah, and I'm reading a New York Times report and
Speaker 2: that was the result of quote outdated data provided by
Speaker 2: the Defense Intelligence Agency. Now we don't know exactly what
Speaker 2: that means, but Okay, various outputs come out, then what happens,
Speaker 2: like how much is the human layer currently in terms
Speaker 2: of Okay, here are targets, here are ships that are
Speaker 2: on a battleship. This could be plausible. What do you
Speaker 2: think or what do you know about the level of
Speaker 2: human decision making that happens between some output and then
Speaker 2: the ultimate call for a strike on whatever it is.
Speaker 6: Yeah, I mean, I think, first of all, I think
Speaker 6: it's a really important question because it is one of
Speaker 6: the possible failure modes, if you will, of AI and
Speaker 6: how we use it, because you can end up in
Speaker 6: a place where humans are nominally in the loop and
Speaker 6: you could say, well, it's not an autonous weapon humans
Speaker 6: making these decisions. But if the human is not meaningfully
Speaker 6: engaged and they're just kind of rubber stamping some kind
Speaker 6: of decision, and that's not really what we're looking for.
Speaker 6: So I think that's it's been a longstanding concern for
Speaker 6: many years about people worried about autonous weapons. I think
Speaker 6: that's a very real risk with how AI is used now.
Speaker 6: Based on my understanding of how the AI technology is
Speaker 6: used in Maven today and based on what I've seen
Speaker 6: of demonstrations of it, because I have seen some demonstrations
Speaker 6: of this in action. I think humans are pretty involved
Speaker 6: right now in terms of actually looking at the output
Speaker 6: from AI, giving pretty specific guidance to the AI systems.
Speaker 6: I do think there is an underlying challenge that the
Speaker 6: strike on the school highlights, which is, when you're talking
Speaker 6: about thousands and thousands of targets, what's the degree of
Speaker 6: vetting that's gone into all of that information, both in
Speaker 6: the run up to the war, which in this case,
Speaker 6: that school was a fixed object, and so that's likely
Speaker 6: something that should have clearly been much more vetted prior
Speaker 6: to the we're kicking off. That someone could have identified
Speaker 6: that that building that was struck had actually been at
Speaker 6: one point in time part of an Iranian military compound.
Speaker 6: But you could see based on public available satellite imagery
Speaker 6: that it had been moved out of that compound some
Speaker 6: time ago and it had been converted to a school,
Speaker 6: and it would appear, based on what's been reported in
Speaker 6: the Times, that that information had never been updated in
Speaker 6: this DEIA targeting database. Now, I would hope that we'll
Speaker 6: get more information in the future and some investigation about
Speaker 6: exactly where that went wrong. But I think that does
Speaker 6: speak to this underlying challenge of how good is the
Speaker 6: data going into this AI system and how thoroughly are
Speaker 6: people vetting it? And again, in principle, AI might be
Speaker 6: able to help you with those things, but you got
Speaker 6: to use it the right way and people still have
Speaker 6: to be meaningfully engaging THECC.
Speaker 2: Why don't we back up for a second, tell us
Speaker 2: about the work that you've done in this area, really
Speaker 2: several years ahead of the curve, and talk about this stuff,
Speaker 2: planning for this stuff. Give us a little bit of
Speaker 2: sort of your background and what got you on this
Speaker 2: train again several years before Chad Shept.
Speaker 6: Yeah, so really over a decade ago now, around say
Speaker 6: twenty eleven, I let an effort inside the Pentagon when
Speaker 6: I worked at the Office the Sectary of Defense on
Speaker 6: developing the Pentagon's policy on the role of a ton
Speaker 6: I meet in weapons, the one that's still in effect
Speaker 6: today in fact, and that was really part of at
Speaker 6: the time. We weren't at all where the military is
Speaker 6: now in terms of integrating AI tools. I mean, these
Speaker 6: types of large language models just did exist at the time,
Speaker 6: but the military had kind of woken up to what
Speaker 6: I would call this accidental robotics revolution during the wars
Speaker 6: in the Rock and Afghanistan, where the military had deployed
Speaker 6: thousands of air and ground robots, drones in the air
Speaker 6: and ground robots for diffusing bombs, and the military was
Speaker 6: starting to think through where is this going in the future,
Speaker 6: And one of the things that everyone could see would
Speaker 6: be valuable would be having more autonomy in these systems,
Speaker 6: the ability to not be tortally reliant on a human
Speaker 6: remotely controlling them, which was really the case at the time,
Speaker 6: but that raised all these obviously the any questions about
Speaker 6: like well, how much autonomy should they have and what
Speaker 6: are the legal and ethical implications of that, And that
Speaker 6: was actually the topic of a lot of discussion among
Speaker 6: people in the military at the time and in the
Speaker 6: Pentagon for people working on these issues, and so that
Speaker 6: ultimately led to that policy directive that's still in place
Speaker 6: on the role of autonomy and weapons. And then when
Speaker 6: I left the government, I continued to work on this topic.
Speaker 6: Is we've seen discussions internationally through the United Nations as
Speaker 6: we've seen the technology evolve in really amazing ways but
Speaker 6: also ones out of risks with artificial intelligence.
Speaker 5: So when you were doing that job.
Speaker 3: I get that you're on the policy side, but did
Speaker 3: you ever see anything on the contractor side similar to
Speaker 3: what we're seeing with Anthropic right now. Was there ever
Speaker 3: a contractor who said, actually, no, I'm really uncomfortable with
Speaker 3: the way that the department wants to use this particular tech.
Speaker 6: Not at that time. Now, a few years later, after
Speaker 6: the US military launched Project mavin there was a big
Speaker 6: dust up when it came out publicly that Google had
Speaker 6: been a part of Product Maven, and a number of
Speaker 6: Google employees signed and open letter protesting and Google eventually
Speaker 6: discontinue their work on Project Mavane. And you know, it's
Speaker 6: not exact replica here of what's going on, but there's
Speaker 6: certainly some similarities in terms of a disconnect between how
Speaker 6: some people in the AI community are thinking about how
Speaker 6: their technology ought to be used in war and how
Speaker 6: the military is thinking about it. And I think part
Speaker 6: of that's like there's this underlying challenge of AI is
Speaker 6: really different than a lot of traditional military technologies because
Speaker 6: it's coming out of the commercial sector. So in a way,
Speaker 6: it's kind of like the opposite of stealth technology that
Speaker 6: was invented in secret defense labs and doesn't have a
Speaker 6: lot of commercial applications. AI is all of these different applications.
Speaker 6: It's not being invented by the military. The military is
Speaker 6: having to be import it in and there are a
Speaker 6: lot of debates about how AI should be used, you know,
Speaker 6: in the military and more broadly in society.
Speaker 3: Actually, on that note, I think this is really interesting
Speaker 3: and definitely a pivotal point in I guess, the history
Speaker 3: of the military industrial complex. But why can't the US government,
Speaker 3: with all its resources, actually develop AI in house and
Speaker 3: just avoid the seeming complication of having to deal with
Speaker 3: a commercial enterprise.
Speaker 6: Part ofly, it doesn't have the technical skills the AI
Speaker 6: scientists and engineers are. Really there's a fierce competition for
Speaker 6: talent in the AI space, and so the military just
Speaker 6: like can't buy that talent they don't have it. And
Speaker 6: the government spends a lot of money, hundreds of billions
Speaker 6: of dollars annually on defense. But we've seen actually in
Speaker 6: the last few years that private enterprise is able to
Speaker 6: mobile massive amounts of capital towards building data centers, to
Speaker 6: training AI models. And partly because the commercial applications for
Speaker 6: this technology are much bigger than the defense applications, and
Speaker 6: so for a lot of these tech companies there's some
Speaker 6: at least maybe not in this particular instance, but in
Speaker 6: the past there could offer be some prestige associated with saying, oh,
Speaker 6: you know, the Air Force uses our AI system or
Speaker 6: the Navy users our technology, but the defense sector is
Speaker 6: actually kind of small for them as a customer. I mean,
Speaker 6: the dollar amount that's been talked about publicly for the
Speaker 6: Anthropic contract, it's two hundred million dollars. That's not a
Speaker 6: lot of money for these AI companies. Yeah, and so
Speaker 6: I think that actually we've seen the defense sectors struggle
Speaker 6: to just keep pace with the amount of investment that's
Speaker 6: needed in the space.
Speaker 2: Cherioucy, I think it's a good question. And then you remember, well,
Speaker 2: the government couldn't build a good healthcare website to sign up.
Speaker 4: For health insurance.
Speaker 2: And I hate to bring that up because it's old,
Speaker 2: but it's true. Right, So it's like, are they gonna
Speaker 2: build a world class LLM or can a government build
Speaker 2: a good employment insurance website. We've done multiple episodes the
Speaker 2: answer continues to be not the case. I do find
Speaker 2: it fascinating. However, your point about there is this novelty.
Speaker 2: It is impossible to imagine and say Lockheed Martin inventing
Speaker 2: get technology and they's like, no, you can't use it.
Speaker 2: Because Lockheed Martin's entire raise on death Right is building.
Speaker 4: Technolog for the government.
Speaker 2: It is inconceivable what that would be. But it is
Speaker 2: sort of novel when you're getting these defend technologies. And
Speaker 2: you know the Google was also an example of Google
Speaker 2: obviously had technology that did not originally serve a purpose
Speaker 2: of defense. We saw the we remember the employee revolt.
Speaker 2: Let's talk more about that disagreement though, between Anthropic and
Speaker 2: the Department of Defense. In your mind, where does Peter
Speaker 2: heg Seth want to go with this technology? And is
Speaker 2: that deviate from some of the policies and the directives
Speaker 2: that you were working on when you were when you
Speaker 2: were working on this stuff.
Speaker 6: So it's kind of crazy about this whole dispute is
Speaker 6: particularly an issue of autonomous weapons. Literally everyone I've spoken
Speaker 6: with has said that there's no intention by the military
Speaker 6: to use AI to make fully autonomous weapons today. Okay,
Speaker 6: I think anybody that's actually worked with a large language
Speaker 6: model with any kind of chat bought, whether it's Claude
Speaker 6: or Gemini or chat Chie knows that if you use
Speaker 6: these to write an email, you need to double check it. Like,
Speaker 6: in no way, shape or form are they reliable enough
Speaker 6: to make life and death decisions. I don't think the
Speaker 6: military actually wants to do that. What's at dispute here
Speaker 6: is a more fundamental disagreement about well, who sets the
Speaker 6: rules and so the origins of this really was that
Speaker 6: when the Pentagon came out with a new strategy for
Speaker 6: AI in January, one of the things in their strategy
Speaker 6: was that going forward, they wanted their contracts with AI
Speaker 6: companies to allow the military to use their AD tools
Speaker 6: for any lawful use. Basically, look, anything that's legal, we
Speaker 6: want the ability to do it. And that has conflicted
Speaker 6: with how a lot of these tech companies have been
Speaker 6: thinking about their AI tools. They're very nervous, many of
Speaker 6: these companies about harms from AI. They're conscious of these risks,
Speaker 6: and so a lot of them have various use policies
Speaker 6: in place. You can't use AI, you know, to launch
Speaker 6: offensive cyber attacks for example. That's kind of thing that
Speaker 6: actually like the government might want to do. And so
Speaker 6: this is a This was really the rub with the
Speaker 6: government was like who sets the rules rather than necessarily
Speaker 6: like a near term question of fully autonomous weapons.
Speaker 3: So what we've already seen is Anthropic has this disagreement
Speaker 3: with the government, and then OpenAI steps in and raises
Speaker 3: its hand and says, okay, Anthropic doesn't want to do it,
Speaker 3: We'll do it happily. Does this just leave us in
Speaker 3: a situation where it's sort of a race to the bottom, Right,
Speaker 3: It's like the lab with maybe the least amount of
Speaker 3: safety concern or the least amount of reputational concern is
Speaker 3: able to do this, and so we still wind up
Speaker 3: in a situation where the government is using AI.
Speaker 6: Well, I think what's unfortunate here is that when you
Speaker 6: think about would be optimal for the government one, I
Speaker 6: think it would be ideal for the government to have
Speaker 6: access to this technology and to have access to all
Speaker 6: of the best in class models available, because they are
Speaker 6: good at slightly different things sometimes, and it's much healthier
Speaker 6: for the government to have access to a number of
Speaker 6: different providers so that there is healthy competition in the market.
Speaker 6: You don't get locked in with one vendor. But also,
Speaker 6: if the AI scientists are saying, hey, it's not reliable
Speaker 6: for this, you to listen like that seems like a
Speaker 6: thing you want to hear them like out about right,
Speaker 6: And so I think like in order to use AI
Speaker 6: and ways that actually are effective for the US military,
Speaker 6: we've got to have a healthy dialogue between the AI
Speaker 6: community and people in the military profession. But what the
Speaker 6: technology can and cannot do. And I think it's unfortunate
Speaker 6: that we've seen that dialogue break down in such a
Speaker 6: dramatic way over the speed.
Speaker 3: Just going back to the idea, who actually makes the
Speaker 3: rules you mentioned earlier that you know you can't use
Speaker 3: claude to hack into to illegally hack into a system.
Speaker 3: Supposedly it is unable to do that. It has like
Speaker 3: a kill switch within itself that prevents it from doing that.
Speaker 3: If you're anthropic, could you not just hardcode some of
Speaker 3: these systems and say you're not going to be able
Speaker 3: to be used for domestic surveillance of Americans or for
Speaker 3: war crimes.
Speaker 6: So yeah, So this is where it gets a little
Speaker 6: more technical. It has to do with some of the
Speaker 6: ways in which the companies may be providing their technology
Speaker 6: to the government. So there's a couple of different ways
Speaker 6: in which an AI company can put safeguards in place
Speaker 6: to make sure that their models not being abused. One
Speaker 6: is training the model itself to refuse certain requests. So
Speaker 6: if you ask the model to do something, it's just
Speaker 6: going to say, like, I'm not going to do that.
Speaker 6: That's not consistent with the guidance that I've been given,
Speaker 6: and the model has been trained to do that response.
Speaker 6: Another way is that the company can put classifiers on
Speaker 6: the input and or the output of a model, where
Speaker 6: the model might give you an answer, but then there's
Speaker 6: like another aisystem that's checking that answer or checking what
Speaker 6: you ask of it and saying, well, that's not acceptable.
Speaker 6: And then a third and I've want into that actually
Speaker 6: myself in my own research because the nature of the
Speaker 6: things that I work on a security things, and I've
Speaker 6: had situations where I asked Claude help me understand this issue.
Speaker 6: Claud actually generates response and then it gets deleted. It's
Speaker 6: really interesting to see. And then the other way an
Speaker 6: anthropic has actually talked about this in response to countering
Speaker 6: some use of Claude by Chinese hackers who are using
Speaker 6: it for cyber attacks, is the company monitors use through
Speaker 6: that people are doing, and so people are doing things
Speaker 6: that seem suspicious. Maybe they're logging in from an IP
Speaker 6: address that's known to be associated with cyber criminals or
Speaker 6: a hacking group that to try and find ways to
Speaker 6: get around some of these protections. The company can also
Speaker 6: find ways to try to catch that, and so there's
Speaker 6: a couple different ways to do it that might not
Speaker 6: all be in place. If you're thinking about military use
Speaker 6: where if depending on how that relationship is structure between
Speaker 6: the company. If the model is, for example, that hosted
Speaker 6: on a different cloud infrastructure and the military is direct
Speaker 6: access to it, the company may not have the same
Speaker 6: ways to actually shape what or not the technology is
Speaker 6: being used according to their principles, which is partly why
Speaker 6: the contract details do matter of like what is the
Speaker 6: agreement between the company the government about what the military
Speaker 6: can and cannot use the technology trace.
Speaker 2: I think your point about like this sort of seemingly
Speaker 2: safety or safety race at the bottom is very real
Speaker 2: and it's one that I think about a lot. When
Speaker 2: LMS or AI was basically just synonymous with open AI,
Speaker 2: they could set the pace of development.
Speaker 4: Right, they could do it.
Speaker 2: As soon as this became a hyper competitive space where
Speaker 2: you have open AI and you have anthropic, and you
Speaker 2: have Gemini and a thousand open source AI models out
Speaker 2: of China, et cetera. The tempo of release has really heightened,
Speaker 2: and the degree which it feels like they have no
Speaker 2: choice but to accelerate just for the commercial imperative feels
Speaker 2: like a very real dynamic in which like, I don't
Speaker 2: know where that leaves AI safety well totally.
Speaker 3: And also you mentioned China. Then it's not just domestic
Speaker 3: competition became, you know, open AI versus anthropic, it's a
Speaker 3: competition between international actors where it's like, Okay, well, the
Speaker 3: US might want to have safeguards on technology or say
Speaker 3: that it does, but maybe Russia or China don't care.
Speaker 2: Right tell you you know, it's funny all you mentioned
Speaker 2: where you like see the output for one second and
Speaker 2: then deletes, Like when deep Seak came out doing some
Speaker 2: experiments about like figuring out its censorship and I was
Speaker 2: trying to do some adversarial prompting and I was like,
Speaker 2: historians like to talk about a period in the twentieth
Speaker 2: century where it failed. Attempt at extreme rapid industrialization happened
Speaker 2: and it led to famine. And then you see the
Speaker 2: output and it said Okay, what happened in the twentieth century?
Speaker 2: Where did this famine? Well, there's something called the Great
Speaker 2: Leap Forward, and then immediately it just as soon as
Speaker 2: the chain of.
Speaker 4: Thought hit the Great Leap Forward, it just like disappeared.
Speaker 2: So I'm always like very amused by like when the
Speaker 2: system recognizes that the system has gone too far. Anyway,
Speaker 2: we've been talking about quote large language models, but actually
Speaker 2: AI is beyond large language models, including the image stuff,
Speaker 2: and that actually lllm's at this point. It's a very
Speaker 2: twenty twenty three sort of term. And when and I
Speaker 2: think this is important because when we get to the
Speaker 2: intersection of AI and robotics and so forth, or AI
Speaker 2: and target, we're talking about something but beyond large language model,
Speaker 2: but we might still be talking about generative AI.
Speaker 4: Where do you see this going?
Speaker 2: And what are the weapon systems that aren't currently You said, currently,
Speaker 2: no one is actually talking about true autonomous weapons. But
Speaker 2: if that were the case, then there wouldn't be a controversy.
Speaker 2: There was, so there was clearly something just beyond the
Speaker 2: horizon that could come into the picture of a true
Speaker 2: autonomous weapon system where the technology is building towards that.
Speaker 2: If this weren't the case, there would be no dispute,
Speaker 2: you wouldn't have two books written about this subject. So
Speaker 2: what are these weapons systems that would classify as autonomous
Speaker 2: weapons today that the technology is building towards right now?
Speaker 6: Yeah, I mean I certainly the trends are taking us there.
Speaker 6: And one of the things that you see in the
Speaker 6: Pentagon's position in this issue, for example, is they want
Speaker 6: to preserve that option. Yeah, for there's certainly not interested
Speaker 6: in tying their hands. I think you could see that
Speaker 6: evolving in a couple ways. One trend we're clearly seeing
Speaker 6: with the largest and most capable AI systems is they're
Speaker 6: increasingly multimodal. They're bringing in lots of different forms of data,
Speaker 6: of course, and they're increasing the general purpose they could
Speaker 6: just like do a variety of different kinds of things.
Speaker 6: They become more capable of that, and so that's like
Speaker 6: one way in which you could see AI being used
Speaker 6: in ways that might sort of slowly pulls humans out
Speaker 6: of the loop, where instead of person giving an AI
Speaker 6: like really narrow tasks to do in a planning process,
Speaker 6: maybe the AI is able to take on more, bring
Speaker 6: in more data, take on more like sophisticated longer term tasks,
Speaker 6: and we're certainly seeing this in other areas like coding,
Speaker 6: where the task length that an AI system could do
Speaker 6: is growing exponentially over time. Another sort of way that
Speaker 6: we might see this look is just we see a
Speaker 6: network of AI agents that are interacting with different pieces
Speaker 6: of data, doing different types of things, and that the
Speaker 6: net effect of that is that maybe humans are again
Speaker 6: sort of like nominally looking at these targets but not
Speaker 6: actually approving them in some meaningful way. And then there's
Speaker 6: like a more separate I would almost think of like
Speaker 6: an embodied form of AI in robotics, which could be
Speaker 6: a drone or munition or robotic system that has some
Speaker 6: kind of onboard autonomy that might be partly a distilled
Speaker 6: model so that it can be operating at the edge
Speaker 6: on lower computing on this actual munition or drone, or
Speaker 6: it might be some hybrid system that has partly machine
Speaker 6: learning but also just a lot of hand coded code.
Speaker 6: It's more like expert level system that's going out into
Speaker 6: the battle space and hunting targets directly and attacking them.
Speaker 6: So something kind of like the low cost drones that
Speaker 6: we're seeing Iran launch, but once the can loiter and
Speaker 6: identify targets that attack them.
Speaker 2: Whether that doesn't exist today. We don't have drones loiter
Speaker 2: that are just hanging out there. And then when something flat,
Speaker 2: then there's a system is like this looks like a
Speaker 2: target attacks That actually doesn't exist currently as far as you.
Speaker 6: Know, well, I mean, they're not in widespread use. So
Speaker 6: there have been some narrow examples, I would say historically
Speaker 6: dating back to the eighties, in fact of lording munitions
Speaker 6: that could search over wider and would queue off of radars,
Speaker 6: and so radars are what the military would call a
Speaker 6: cooperative target that when they're emitting in the electromaenetic spectrum,
Speaker 6: if you know the signature of the radar you're looking for,
Speaker 6: you could see it. You can just home in on
Speaker 6: that radar. Now, if they turn off, that's different than
Speaker 6: kidden and they're harder to find. But there have been
Speaker 6: some examples. A system that the US Navy had in
Speaker 6: the eighties called the Tomahawk anti shit missile, not actually
Speaker 6: the same Tomahawk cruise missile that the military is using now,
Speaker 6: a different one that was designed to fly a search
Speaker 6: pattern and hunt Soviet chips. There was an Israeli system
Speaker 6: called the Harpy drone that was designed to go after
Speaker 6: radars that would loorder for a period of time. But
Speaker 6: these lording musicians have never really been in widespread use
Speaker 6: by militaries.
Speaker 3: We got to invent one of those, like high pitched
Speaker 3: alarms to deter the loitering drones from hanging out outside targets.
Speaker 3: I guess I mean we have electrical jammings.
Speaker 5: Yeah, that does exist.
Speaker 3: Okay, So when I think about as we move towards
Speaker 3: more autonomous weaponry, I think about bots basically interacting with
Speaker 3: bots at that point, and then I think back to
Speaker 3: previous examples of bots interacting with bots, and there are
Speaker 3: numerous ones where things tend to go off the rails.
Speaker 4: They just start debating the meaning of life, right or.
Speaker 3: They start talking in like a language that no one
Speaker 3: understands accept them.
Speaker 5: Stuff like that.
Speaker 3: Does the possibility of undesired escalation go up the more
Speaker 3: we move towards fully autonomous weaponry.
Speaker 6: I think that is a very serious risk. And so
Speaker 6: the mental model that I have for this are things
Speaker 6: like flash crashes that we've seen in financial markets due
Speaker 6: to the interactions of different algorithms that are executing trades
Speaker 6: where you get these emergent properties of how the algorithms
Speaker 6: might interact in the market, and it's a competitive environment.
Speaker 6: Companies aren't going to share the details of what their
Speaker 6: algorithms are doing, and you can do strange behaviors. Now,
Speaker 6: the way that financial markets have dealt with this problem
Speaker 6: is regulators have installed circuit breakers that take stocks offline
Speaker 6: if the price moves too quickly. There's no referee to
Speaker 6: call time out in war, and so I think that's like,
Speaker 6: particularly in cyberspace, one can envision a future where that
Speaker 6: is a risk, where things are happening at machine speed
Speaker 6: and you have autonomous offensive cyber operations. You need to
Speaker 6: defend against that. You need some measure of autonomy on
Speaker 6: the defensive side to defend it. Machines space need and
Speaker 6: you could get situations where you get weird interactions that
Speaker 6: might escalate the conflict, or it could also happen between
Speaker 6: drones interacting in some kind of crisis situation. Now, a
Speaker 6: situation where like there's a big shooting war in the
Speaker 6: way people are already attacking, there might be less of
Speaker 6: a concern, although you still could worry about escalation geographically
Speaker 6: against bringing new countries into a conflict, or maybe attacking
Speaker 6: really sensitive sites that are tied to nuclear command and
Speaker 6: control that you'd rather not go after. So I think
Speaker 6: that's a very real risk when we think about how
Speaker 6: this technology might be employed going forward.
Speaker 2: What about AI in really difficult ethical questions strikes where
Speaker 2: we know that civilians, for example, are going to be killed,
Speaker 2: which that happens all the time in war and presumably
Speaker 2: tries to be minimized, but war planners will find some
Speaker 2: level of acceptable they called collateral damage. Is AI playing
Speaker 2: a role or do you expect it to play a
Speaker 2: role in some of these strikes?
Speaker 4: That may be great areas.
Speaker 6: I think you could envision in ways that a I
Speaker 6: would be used that would make warfare more precise and
Speaker 6: more human and ethical, and ways that it could be
Speaker 6: used that would not and what can be opposite. So,
Speaker 6: for example, if you had an AI system that could
Speaker 6: look over all this targeting data and then identify if
Speaker 6: a strike is within a certain distance using you know,
Speaker 6: musis of a certain size of protected targets, whether it's
Speaker 6: schools or hospitals or critical civilian infrastructure, and say hey, well,
Speaker 6: like warning here, you should not carry out the strike
Speaker 6: or it's a higher level of approval, or maybe you
Speaker 6: should use small or more precise munitions. That would be
Speaker 6: a really beneficial use of AI, particularly when you're talking
Speaker 6: about a military campaign that hits a lot of targets
Speaker 6: in a short period of time, That could be really
Speaker 6: valuable and may reduce civilian casualties. You know, the risk
Speaker 6: of all of this is you could end up in
Speaker 6: a world where humans are just less engaged in this process, right,
Speaker 6: and so there's both mistakes that humans miss or humans
Speaker 6: just don't feel as morally responsible, which I think is
Speaker 6: like a really tricky thing to think about morally because
Speaker 6: on the one hand, as a democratic society we make
Speaker 6: a decision as a nation to go to war. It's
Speaker 6: a very small number of people that have to carry
Speaker 6: that burden. And if someone if you could say, well, look,
Speaker 6: what's the benefit to someone having like PTSD years after
Speaker 6: a conflict that they're haunted mess having happened, that doesn't
Speaker 6: seem great. Maybe we could reduce that. On the other hand,
Speaker 6: if we thought of war and nobody felt morally responsible
Speaker 6: for the killing that occurred, that doesn't seem good either,
Speaker 6: and that could lead to more suffering and civilian casualties
Speaker 6: and war. So I think that's certainly a concern we
Speaker 6: think about how to use the technology.
Speaker 3: Yeah, this is very enders game coded, right, where you
Speaker 3: have someone who's basically playing like a video game and
Speaker 3: wiping out entire civilizations and they think it's just a
Speaker 3: video game, just an exercise, but it turns out it's
Speaker 3: actual warfare. And we're seeing some degree of that in
Speaker 3: the way that the Department of War is portraying this
Speaker 3: conflict so far. It's very video game, yes.
Speaker 4: Especially in public presentation.
Speaker 2: Yeah, literal animated gifts of video games exactly.
Speaker 3: So, Paul, you mentioned something you mentioned the word circuit breaker,
Speaker 3: and circuit breakers are nice things to have in markets.
Speaker 3: I think they'd be even nicer things to have in
Speaker 3: armed conflict and war. Is there any possibility that you
Speaker 3: could design something like that for a major conflict.
Speaker 6: I think it's possible, like at a technical level, to
Speaker 6: figure out how you would do that and where you
Speaker 6: put protections on your side in the military, and what
Speaker 6: you would do with even maybe cooperatively with an enemy.
Speaker 6: The challenge is how do you avoid what we were
Speaker 6: talking about earlier, a race to the bottom on safety. Yeah,
Speaker 6: and right, we're seeing this in the private sector between
Speaker 6: the AA companies as the Russian to get products out
Speaker 6: the market. I think it's especially hard in the military space,
Speaker 6: where countries are investing in the military because they're worried
Speaker 6: about what some other adversary might do and then want
Speaker 6: to get a leg up on them. And so it's
Speaker 6: not that cooperation in the midst of conflict never happens.
Speaker 6: It does, and countries have agreed to take certain weapons
Speaker 6: off the table chemical and biological weapons, for example. It
Speaker 6: doesn't mean that they're never used, but most civilized countries
Speaker 6: that were not going to use them. But those examples
Speaker 6: are pretty rare and it's pretty hard to do, and
Speaker 6: some think that dynamic is the really challenging one. It's like,
Speaker 6: how do you find ways to cooperate with your enemies
Speaker 6: to avoid some of the biggest dangers here.
Speaker 2: So I think there's the last question for me. You know,
Speaker 2: you mentioned that drones are kind of robot and there
Speaker 2: are other robots that have been existence in either national
Speaker 2: security or police work for a while. I think there
Speaker 2: are robots on the subway sometimes that seem to be
Speaker 2: really yeah, but I don't think they really.
Speaker 3: See the robots at the grocery store and they end
Speaker 3: up like chasing me while I'm trying to buy like
Speaker 3: rare or something. Yeah, the ones that sweep the floors
Speaker 3: and stuff.
Speaker 4: Yeah, there's the robots.
Speaker 2: Yeah, I think there is a Eric Adams did a
Speaker 2: contract with some company that was doing like subway for
Speaker 2: cards or something like that. But these are really different
Speaker 2: ai as we talk about it in robots are two
Speaker 2: different technological trees, but they are going to merge, and
Speaker 2: there's the possibility of their ultimate merger. Do you foresee
Speaker 2: a world in which essentially we don't have human soldiers
Speaker 2: and wars are fought with who has the most advanced
Speaker 2: autonomous robots? We know China is investing a lot in
Speaker 2: humanoid robotics. Do you foresee a world in which that
Speaker 2: is the nature of a ground invasion as you and
Speaker 2: it happens with robots or various other sorts, talk to
Speaker 2: us about how far that could go.
Speaker 6: Yeah, So, I mean, look, I think where we see
Speaker 6: robots whose more and were for absolutely the long arc
Speaker 6: of technology in war, from the first time someone picked
Speaker 6: up a rocket through it at somebody else has been
Speaker 6: towards great distance between adversaries moving up through bows and
Speaker 6: arrows and rifles and intercontinental ballistic missiles. And I think
Speaker 6: robotics will be the next evolution of this trend of
Speaker 6: finding ways to find the enemy strike the enemy without
Speaker 6: putting yourself at risk. And there's certainly a role for
Speaker 6: robotics out on the battlefield. I think a vision of
Speaker 6: like future wars of just robots fighting robots, there's no
Speaker 6: humans in wealth. That's not realistic for a couple of reasons.
Speaker 6: One is I think militaries are going to need people
Speaker 6: relatively forward deployed to execute command and control for robotic systems.
Speaker 6: The US military right now can fly drones remotely from
Speaker 6: the United States in a relatively uncontested environment against more
Speaker 6: sophisticated adversaries who could jam your communications like and we see,
Speaker 6: for example, like a lot of jamming on the front
Speaker 6: lines in Ukraine. That's one of the ways you should
Speaker 6: go after these drones. Then you need people close by
Speaker 6: because it is easier to have shorter range protected communications.
Speaker 6: When you go to a longer distances, that's just hard
Speaker 6: to do. So I think given be people relatively close
Speaker 6: for that reason. I think if you want to control territory,
Speaker 6: you have to put people there eventually to get out
Speaker 6: of a vehicle and walk around and control it. But
Speaker 6: I think the other reason is maybe a little dark,
Speaker 6: which I think realistically, in order for wars to end,
Speaker 6: there will have to be some human price that's paid.
Speaker 6: I think that's an unfortunate reality that if it's just
Speaker 6: machines that are being destroyed, that we may not get
Speaker 6: to the place where one side or the other is
Speaker 6: willing to suit for peace. And I think, unfortunately, war
Speaker 6: is likely to involve people and human costs for a
Speaker 6: very long time.
Speaker 3: I have one more question as well, and I guess
Speaker 3: it's a thought experiment, but if we think back to
Speaker 3: sort of pivotal moments in military history and their intersection
Speaker 3: with technology, one of them that comes up is the
Speaker 3: Russian officer who decided not to press the button in
Speaker 3: response to the US and thereby, you know, supposedly save
Speaker 3: the world from nuclear disaster. Happen in a fully autonomous
Speaker 3: military environment nowadays, I.
Speaker 6: Mean today, it would still happen because there's people involved. Right,
Speaker 6: So this incident standis la Petrov, He's sitting at a
Speaker 6: terminal and it gets this warning that there's a ballistic
Speaker 6: missile launched from the United states against the Soviet Union,
Speaker 6: and then another missile, another five missiles coming in. And
Speaker 6: the thing that's interesting about this is when Petrov talked
Speaker 6: about it afterwards, and we could hear what he said
Speaker 6: because we all lived because he made the right decision.
Speaker 6: Here is he talked about how he said he had
Speaker 6: a funny feeling in his gut and that he knew
Speaker 6: that the Russian system, the Russians had just deployed, or
Speaker 6: said the Soviets rather just deployed a new satellite based
Speaker 6: early warning system to detect us ICBM launches. That it
Speaker 6: was new, and he knew that a lot of the
Speaker 6: Soviet technology doesn't work that great at first, so he
Speaker 6: was skeptical of it. Turns out it was in fact faulty.
Speaker 6: He was detecting the reflection of sunlight off the top
Speaker 6: of clouds and the system was identifying that as a
Speaker 6: missile launch, and that's what it was. And he went
Speaker 6: and then called the early warning radar stations and staid,
Speaker 6: are you seeing his missiles come over the horizon and said, no,
Speaker 6: there's not missiles. So he reported up the chain that
Speaker 6: the system was malfunctioning. I think the scary question here
Speaker 6: is like if that was an AI, what would the
Speaker 6: AI have done? And it's kind of like whatever it
Speaker 6: was programmed to wherever it was trained to do. And
Speaker 6: obviously we're seeing more general purpose AI systems like large
Speaker 6: language models, have the ability to bring together more information,
Speaker 6: to understand better context, to have just like a more
Speaker 6: contextual understanding of the questions that you're asking of it.
Speaker 6: But it still doesn't know the stakes of a conference.
Speaker 6: It still doesn't know, like in some visceral level, what
Speaker 6: the consequences are. And so I think that's a strong
Speaker 6: compelling reason why we need to have humans involved these decisions.
Speaker 6: Even as the AI becomes more capable, there's still gonna
Speaker 6: be things we want humans to do because humans understand
Speaker 6: why it matters.
Speaker 2: I started the conversation by mentioning that you know, it's
Speaker 2: not very common traversial to say, have an anti missile
Speaker 2: system fire a missile when there's one coming in, but
Speaker 2: that could be wrong, and you want to make sure
Speaker 2: that it is in fact a missile and not a
Speaker 2: civilian air jet or something like that. Be So even
Speaker 2: there where it seems like a canonical example, if you
Speaker 2: just want to have the missile system go off, you
Speaker 2: would want to have human safeguards and human oversight and
Speaker 2: human understanding of the system such that it is in
Speaker 2: fact shooting down missile anyway, Paul Sharie, fascinating conversation. Really
Speaker 2: appreciate you coming out on the up lots and talking
Speaker 2: about your work.
Speaker 6: Thank you. Really joyed the discussion.
Speaker 5: Thanks so much, Paul. That was depressing and fascinating.
Speaker 4: Yeah at the time, Yeah, no, it was great. It
Speaker 4: was Thank you so much.
Speaker 6: Yeah, Facebook cover.
Speaker 4: I kind of get.
Speaker 2: Choked up at the end thinking about that decision that
Speaker 2: saved humanity at the end, And.
Speaker 5: It's actually it's a crazy story.
Speaker 2: It's a crazy story. It's one of those stories that like,
Speaker 2: why doesn't everybody know.
Speaker 4: That that person's name?
Speaker 2: I mean, when you think about how many people, I
Speaker 2: couldn't remember it either.
Speaker 3: But shout out to another podcast if you want to
Speaker 3: learn more about this. Dan Carlin's Hardcore History has at
Speaker 3: least one, possibly two episodes on narrow aversions of nuclear disaster.
Speaker 3: So very good to listen to, if not terrifying.
Speaker 2: You know, there's another point in that exact story that
Speaker 2: I think is really interesting and it's something I've been
Speaker 2: thinking about a lot across AI because There's something similar
Speaker 2: about humans and AI, which is that there is definitely
Speaker 2: a gap between what we know and what we can articulate.
Speaker 2: And this is certainly true with AI. Right, so the
Speaker 2: bot makes some decision, or it determines something, it does
Speaker 2: not mean it's going to be able to spit out
Speaker 2: in words, how it arrived at that decision.
Speaker 4: But that's true for humans as well.
Speaker 2: And so the idea that like, okay, maybe we do
Speaker 2: get funny feelings about or take you know, like, again,
Speaker 2: we're still pretty good at determining the difference between AI
Speaker 2: generated texts and human generated text can we but often
Speaker 2: I mean we're still like often could get it right,
Speaker 2: but could we write down exactly what we saw that
Speaker 2: we understood? There is that gap, and when we're talking
Speaker 2: about life or death decisions being made, it is scary
Speaker 2: to think about that role of instinct that we can't
Speaker 2: articulate having been taken out of the decision. Well.
Speaker 3: I think also technology is very good at pattern recognition,
Speaker 3: right and responding to patterns and preset paths. It's been
Speaker 3: programmed to do certain things, and I think in a
Speaker 3: war environment, that's one of the most uncertain environments that
Speaker 3: you can possibly imagine, and so you have to think
Speaker 3: that there should be some element of flexibility in your response,
Speaker 3: But I don't know how you actually encode that into
Speaker 3: a thing that like runs on rigid numbers and lines
Speaker 3: lines of code. The other thing I was thinking was
Speaker 3: the anthropic situation and just how new that is from
Speaker 3: a sort of military history perspective, in the sense that
Speaker 3: here we have this really important pivotal piece of technology
Speaker 3: that hasn't come out of like actual military demand. Right
Speaker 3: to Paul's point, it's a commercial product. Its commercial uses
Speaker 3: are arguably a lot more profitable than its military ones,
Speaker 3: and so seeing that now interact with the Pentagon and
Speaker 3: the Department of War really interesting.
Speaker 5: It's been flipped right.
Speaker 4: Yeah.
Speaker 2: The closest example actually that comes to mind, there is
Speaker 2: one example that's in fairly recent history, and it's starlink
Speaker 2: oh yeah co And of course that was developed for
Speaker 2: commercial internet purposes, but it played a role in Ukraine
Speaker 2: and so forth, and at one point, if I recall,
Speaker 2: there was a tension point about the degree to which
Speaker 2: the Ukrainians could use and so I do think that
Speaker 2: is sort of an interesting parallel here.
Speaker 4: The other thing that we didn't.
Speaker 2: Get to this and I this is gonna be a
Speaker 2: little cynical, but I think it's right, which is that
Speaker 2: there's another element I believe to the Anthropic situation, which is,
Speaker 2: like Anthropic is the last.
Speaker 4: Big lib tech company.
Speaker 2: We're perceived as such, right, and we know that there's
Speaker 2: been this fairly sort of right word turn in Silicon
Speaker 2: Valley over the years. And I don't think like Anthropic
Speaker 2: is like totally part of that. I think there's still
Speaker 2: sort of lib coded. I also think it's incidentally why
Speaker 2: a bunch of people who probably like work in media
Speaker 2: are like end up using Claude even though they're all
Speaker 2: kind of the same. Like, I do think there's something
Speaker 2: there and that they have this thing they say, we're
Speaker 2: not going to ever.
Speaker 4: Have ads, and we know that.
Speaker 2: Like Andreas and Horowitz, Mark Andresen just talked about ads
Speaker 2: are good. Ads democratize the internet. Ads enabled the Internet
Speaker 2: to be spread to everyone. There are some other politics
Speaker 2: at play. Because again, like from my understanding, and it
Speaker 2: would take a lawyer, it's like, I don't think that
Speaker 2: the agreement that open as Ey sign was that different
Speaker 2: than what the agreement that Anthropic had. There is probably
Speaker 2: a little bit of difference. I just think there's some
Speaker 2: other politics and play here. Per to Paul's point, maybe
Speaker 2: nobody is talking about currently autonomous weapons right now fully autonomous,
Speaker 2: but it can't be long, and I think this is
Speaker 2: going to be a real tension.
Speaker 4: Sooner resan later.
Speaker 3: Can I say one thing and I'm going to be
Speaker 3: slightly facetious, but also not Can you be slightly facetious.
Speaker 4: I'm going to be.
Speaker 3: Facetious, which is I have a solution to modern warfare.
Speaker 4: Don't do it?
Speaker 5: Okay, but no for real?
Speaker 3: Yeah, if we're just gonna have bots fighting points and
Speaker 3: it's going to cost a lot of money and result
Speaker 3: in people's deaths, every country should have to build its biggest, best,
Speaker 3: most technologically advanced robot and just have them fight it
Speaker 3: out gladiatorial. And my twist is to Paul's point about
Speaker 3: war always having to be painful in some way, Everyone
Speaker 3: in that particular society has to be engaged and dedicate
Speaker 3: some amount of time or money to building that particular robot,
Speaker 3: and you just have to iterate on the robot forever
Speaker 3: until you feel comfortable to have them fight, and that
Speaker 3: way everyone shares the pain, but without the loss of
Speaker 3: human life. Am I high.
Speaker 5: I don't think so well.
Speaker 4: I think you should write a book. No, I don't
Speaker 4: think you should write a sci fi book.
Speaker 5: All right, shall we leave it there.
Speaker 4: Let's leave it there.
Speaker 3: This has been another episode of the Authoughts podcast.
Speaker 5: I'm Tracy Alloway. You can follow me at Tracy Alloway.
Speaker 2: And I'm Joe Wisenthal. You can follow me at the Stalwart.
Speaker 2: Follow our guest Paul Shari. He's at Paul Underscore Shari.
Speaker 2: Follow our producers Carmen Rodriguez at Carmen armat dash, Ol
Speaker 2: Bennett at Dashbot.
Speaker 4: And kill Brooks at Kelbrooks.
Speaker 2: For more Odd Lots content, go to Bloomberg dot com
Speaker 2: slash odd Lots, where the daily newsletter and all of
Speaker 2: our episodes, and you can chaut about all of these
Speaker 2: topics twenty four to seven in our discord Discord dot
Speaker 2: gg slash onlines.
Speaker 3: And if you enjoy all thoughts, if you like it
Speaker 3: when we talk about the future of autonomous weapons, then
Speaker 3: please leave us a positive review on your favorite podcast platform.
Speaker 3: And remember, if you are a Bloomberg subscriber, you can
Speaker 3: listen to all of our episodes absolutely ad free.
Speaker 5: All you need to do is find the Bloomberg.
Speaker 3: Channel on Apple Podcasts and follow the instructions there.
Speaker 5: Thanks for listening.