WEBVTT - Here's Why And How AI Could Kill Us All

0:00:02.450 --> 0:00:04.010
<v Speaker 1>Bloomberg Audio Studios.

0:00:04.530 --> 0:00:05.250
<v Speaker 2>Podcasts.

0:00:05.590 --> 0:00:06.030
<v Speaker 3>Radio.

0:00:06.390 --> 0:00:06.730
<v Speaker 2>News.

0:00:08.539 --> 0:00:11.100
<v Speaker 1>I'm Stephen Carroll and this is Here's Why, where we

0:00:11.140 --> 0:00:13.299
<v Speaker 1>take one news story and explain it in just a

0:00:13.320 --> 0:00:15.920
<v Speaker 1>few minutes with our experts here at Bloomberg.

0:00:19.360 --> 0:00:22.930
<v Speaker 2>I'm worried about misuse of AI models. Instead of talking

0:00:22.960 --> 0:00:26.850
<v Speaker 2>about the probabilities, let's talk about what we can do.

0:00:27.310 --> 0:00:29.880
<v Speaker 1>It's not like we're just rolling the dice. I think

0:00:29.920 --> 0:00:33.699
<v Speaker 1>there are genuine safety concerns, but I also think there's

0:00:33.780 --> 0:00:36.180
<v Speaker 1>probably a bit of self-interest involved.

0:00:36.360 --> 0:00:40.320
<v Speaker 3>We need agreement from Republicans and Democrats in Congress to

0:00:40.400 --> 0:00:44.449
<v Speaker 3>put aside our partisan pre-election bickering and put in place

0:00:44.470 --> 0:00:47.650
<v Speaker 3>some real guardrails before AI goes off the rails.

0:00:50.590 --> 0:00:52.690
<v Speaker 1>All of a sudden, a lot of people are really

0:00:52.729 --> 0:00:56.470
<v Speaker 1>worried about AI. Donald Trump is a notable exception.

0:00:57.150 --> 0:00:59.430
<v Speaker 3>They're just playing right into the hands of a lot

0:00:59.450 --> 0:01:01.870
<v Speaker 3>of people that don't want to see it happen.

0:01:01.970 --> 0:01:03.940
<v Speaker 1>And that could be political people.

0:01:03.950 --> 0:01:05.100
<v Speaker 2>It could also be China.

0:01:05.920 --> 0:01:07.240
<v Speaker 1>And we're not going to let that happen.

0:01:07.380 --> 0:01:08.620
<v Speaker 2>It's a hoax.

0:01:09.620 --> 0:01:11.760
<v Speaker 1>This debate was blown open by a warning from a

0:01:11.819 --> 0:01:15.560
<v Speaker 1>former anthropic and OpenAI researcher about the risks posed by

0:01:15.620 --> 0:01:19.840
<v Speaker 1>artificial intelligence. It came after the startling cyber attack by

0:01:20.000 --> 0:01:24.050
<v Speaker 1>OpenAI's rogue agents on fellow tech firm Hugging Face. The

0:01:24.090 --> 0:01:27.430
<v Speaker 1>result has been widespread alarm, not just about what AI

0:01:27.490 --> 0:01:30.520
<v Speaker 1>can do, but also the ability of top developers to

0:01:30.580 --> 0:01:35.520
<v Speaker 1>control their creations. Here's why and how AI could kill

0:01:35.580 --> 0:01:41.310
<v Speaker 1>us all. Bloomberg Tech Europe anchor Tom McKenzie joins us

0:01:41.370 --> 0:01:44.510
<v Speaker 1>now for more. Tom, the researcher Jacob Coxon had warned

0:01:44.750 --> 0:01:49.210
<v Speaker 1>his former employers, Anthropic and OpenAI were gambling with our lives.

0:01:49.650 --> 0:01:53.950
<v Speaker 1>in that social media post. What exactly is he warning of,

0:01:53.990 --> 0:01:57.529
<v Speaker 1>is he worried about? How could this technology actually kill people?

0:01:58.390 --> 0:02:01.700
<v Speaker 2>Well, he and other researchers from other frontier labs, current

0:02:01.720 --> 0:02:05.820
<v Speaker 2>researchers and those who have recently left, point primarily to

0:02:05.880 --> 0:02:10.720
<v Speaker 2>the developments around something called recursive self-improvement or RSI, which

0:02:10.760 --> 0:02:15.280
<v Speaker 2>is essentially the idea that agents, AI agents are getting

0:02:15.370 --> 0:02:18.280
<v Speaker 2>so good that they will soon be able to do

0:02:18.300 --> 0:02:20.900
<v Speaker 2>their own AI research and build their own agents. And

0:02:20.940 --> 0:02:22.620
<v Speaker 2>those agents will be better than them. And then those

0:02:22.700 --> 0:02:26.620
<v Speaker 2>agents will build other agents. And that will continue. That

0:02:26.680 --> 0:02:29.680
<v Speaker 2>research has started. And a lot of researchers say that

0:02:29.720 --> 0:02:34.900
<v Speaker 2>the results are more extreme than many maybe had expected.

0:02:35.300 --> 0:02:37.300
<v Speaker 2>There hasn't been confirmation that it's exactly going to play

0:02:37.340 --> 0:02:39.370
<v Speaker 2>out the way I described, but that is theoretically how

0:02:39.410 --> 0:02:43.370
<v Speaker 2>it could play out. And in that world, there are

0:02:43.389 --> 0:02:48.389
<v Speaker 2>deep questions then for these researchers about agency and how

0:02:48.450 --> 0:02:51.240
<v Speaker 2>humans control those agents if they are continuing to build

0:02:51.250 --> 0:02:54.459
<v Speaker 2>their own agents, and how you understand what is going

0:02:54.540 --> 0:02:58.460
<v Speaker 2>on under the hood, essentially, something called interpretability. It's still

0:02:58.740 --> 0:03:01.180
<v Speaker 2>not clear to a lot at the frontier of AI

0:03:01.380 --> 0:03:03.660
<v Speaker 2>how these models work, and there's still a lot of

0:03:03.720 --> 0:03:05.950
<v Speaker 2>work being done to try and understand that. And it's

0:03:05.970 --> 0:03:08.770
<v Speaker 2>the pace of all of this innovation on top of

0:03:08.810 --> 0:03:11.800
<v Speaker 2>those developments that is leading the a number of prominent

0:03:11.820 --> 0:03:16.060
<v Speaker 2>researchers to highlight these concerns and very, very real concerns

0:03:16.080 --> 0:03:19.420
<v Speaker 2>that they are pointing to potentially the very existence of

0:03:19.460 --> 0:03:20.160
<v Speaker 2>humanity itself.

0:03:20.620 --> 0:03:23.179
<v Speaker 1>To think through the causality of this, the thought is

0:03:23.220 --> 0:03:26.920
<v Speaker 1>that if AI is out of human control, that it's,

0:03:27.240 --> 0:03:30.220
<v Speaker 1>you know, creating its own, developing its own agents, that

0:03:30.240 --> 0:03:34.440
<v Speaker 1>it could take control of critical systems or what exactly

0:03:34.520 --> 0:03:37.680
<v Speaker 1>is the concrete risk if you follow that chain? And

0:03:37.720 --> 0:03:39.100
<v Speaker 1>as you say, it is a theoretical chain.

0:03:39.830 --> 0:03:44.410
<v Speaker 2>So you highlighted the Hugging Face open AI incident, hacking incident,

0:03:44.470 --> 0:03:47.530
<v Speaker 2>which stretched from May to July of this year. And

0:03:47.590 --> 0:03:50.870
<v Speaker 2>what that showed was that agents that are currently active,

0:03:50.910 --> 0:03:53.190
<v Speaker 2>that they have currently built at these frontier labs, are

0:03:53.290 --> 0:03:56.839
<v Speaker 2>already capable of breaking out of so-called sandboxes where they're

0:03:56.860 --> 0:04:00.200
<v Speaker 2>cut off for the internet. They conspired, about 1,200 of

0:04:00.240 --> 0:04:03.040
<v Speaker 2>these agents conspired with each other to break out of

0:04:03.080 --> 0:04:07.480
<v Speaker 2>these so-called sandboxes to hack into Hugging Face, which is

0:04:07.540 --> 0:04:11.110
<v Speaker 2>a depository of AI code. and other things that go

0:04:11.170 --> 0:04:13.970
<v Speaker 2>into how we use AI, but largely a depository of

0:04:14.010 --> 0:04:17.550
<v Speaker 2>AI code. They managed to break in. That showed agents

0:04:17.630 --> 0:04:20.210
<v Speaker 2>coordinating at a level that many had said we've never

0:04:20.250 --> 0:04:22.409
<v Speaker 2>seen before. And if you put on top of that,

0:04:22.730 --> 0:04:25.070
<v Speaker 2>this recursive self-improvement where the agents are getting better and

0:04:25.089 --> 0:04:27.530
<v Speaker 2>better and better, then all sorts of potential scenarios could

0:04:27.570 --> 0:04:29.909
<v Speaker 2>map out. You could, for example, as you said, get

0:04:30.110 --> 0:04:32.150
<v Speaker 2>to the point where these agents are running industries and

0:04:32.180 --> 0:04:35.160
<v Speaker 2>taking those over and that we lose control as they

0:04:35.200 --> 0:04:36.679
<v Speaker 2>took over the energy system.

0:04:37.360 --> 0:04:37.420
<v Speaker 3>Or.

0:04:38.100 --> 0:04:39.940
<v Speaker 2>you could get to a point where these agents are

0:04:39.980 --> 0:04:44.850
<v Speaker 2>being used by nefarious actors to, for example, build biological weapons.

0:04:45.350 --> 0:04:48.150
<v Speaker 2>That's a risk that's been flagged by researchers in AI

0:04:48.170 --> 0:04:50.870
<v Speaker 2>for a long time. And then there's the question of

0:04:51.790 --> 0:04:54.969
<v Speaker 2>the nuclear deterrent. And if AI gets embedded or gets

0:04:55.070 --> 0:04:58.650
<v Speaker 2>into our nuclear systems, what that potentially could mean. So

0:04:58.690 --> 0:05:04.010
<v Speaker 2>there's all sorts of different scenarios under which agents, training agents,

0:05:04.440 --> 0:05:07.279
<v Speaker 2>we don't understand to what extent they are aligned with

0:05:07.360 --> 0:05:11.280
<v Speaker 2>our own aims and human aligned, which means they are

0:05:11.300 --> 0:05:13.600
<v Speaker 2>working on our best interest. We don't know if they are,

0:05:14.140 --> 0:05:16.000
<v Speaker 2>because that's still an area of research that is yet

0:05:16.040 --> 0:05:19.349
<v Speaker 2>to be fully resolved and fully understood. And we don't

0:05:19.440 --> 0:05:21.970
<v Speaker 2>exactly know what's going on under the hood of these models.

0:05:22.029 --> 0:05:26.770
<v Speaker 2>That's the interpretability part of it. Those things combined pose

0:05:26.810 --> 0:05:30.390
<v Speaker 2>this potential very, very significant risk, is what these researchers

0:05:30.410 --> 0:05:31.229
<v Speaker 2>are suggesting.

0:05:31.750 --> 0:05:34.450
<v Speaker 1>So how seriously are these risks being taken? We've heard

0:05:34.470 --> 0:05:36.770
<v Speaker 1>a little bit about the alarm that's been raised, but

0:05:36.970 --> 0:05:40.520
<v Speaker 1>within the industry, outside the industry, how much are these

0:05:40.560 --> 0:05:41.659
<v Speaker 1>risks being taken seriously?

0:05:42.000 --> 0:05:44.580
<v Speaker 2>Well, they are being taken very seriously to the point

0:05:44.620 --> 0:05:48.560
<v Speaker 2>where Dario Amadei, the CEO and the founder of Anthropic,

0:05:48.960 --> 0:05:50.790
<v Speaker 2>one of the leading frontier labs, of course, posted a

0:05:50.810 --> 0:05:53.950
<v Speaker 2>more than 3,000-word essay over the weekend calling for a

0:05:53.990 --> 0:05:57.850
<v Speaker 2>three-step approach to pacing, as he described it, pacing the frontier,

0:05:57.890 --> 0:06:00.469
<v Speaker 2>essentially slowing some of the development of making sure that

0:06:00.490 --> 0:06:03.850
<v Speaker 2>they have independent auditors essentially inside their own business. They

0:06:03.890 --> 0:06:06.570
<v Speaker 2>say they're committed to that already. He laid out a

0:06:06.610 --> 0:06:09.070
<v Speaker 2>number of measures where he wants to see leading frontier

0:06:09.100 --> 0:06:11.500
<v Speaker 2>and AI labs working together, coordinating, but he also wants

0:06:11.540 --> 0:06:14.360
<v Speaker 2>a regulatory framework imposed by the government as well to

0:06:14.400 --> 0:06:16.620
<v Speaker 2>ensure there are those checks and balances. And he wants

0:06:16.940 --> 0:06:20.380
<v Speaker 2>work at the international level to try to mitigate the

0:06:20.420 --> 0:06:24.900
<v Speaker 2>risks as he sees them. That essay was supported by

0:06:25.040 --> 0:06:28.710
<v Speaker 2>Sam Altman of OpenAI.com. But also, Elon Musk seemed to

0:06:28.750 --> 0:06:32.070
<v Speaker 2>give it his endorsement by saying, Dario is right, and

0:06:32.120 --> 0:06:35.140
<v Speaker 2>posting that on X. We've had people like Demis Hassabis

0:06:35.360 --> 0:06:39.480
<v Speaker 2>from Google DeepMind, who has previously put together a plan

0:06:40.260 --> 0:06:43.320
<v Speaker 2>that he says could help Swage and address some of

0:06:43.360 --> 0:06:46.280
<v Speaker 2>those risks, a regulatory framework along the lines of FINRA,

0:06:46.400 --> 0:06:49.000
<v Speaker 2>which is used in the United States, which is a

0:06:49.080 --> 0:06:51.560
<v Speaker 2>self-regulating body for the world of finance. He says something

0:06:51.580 --> 0:06:55.590
<v Speaker 2>like that could be appropriate. And shortly after that Hugging

0:06:55.610 --> 0:06:58.810
<v Speaker 2>Face OpenAI hacking event, that happened and ended in July,

0:06:59.270 --> 0:07:02.610
<v Speaker 2>about 1,200 researchers from across different frontier labs came out

0:07:02.630 --> 0:07:05.410
<v Speaker 2>together and wrote an open letter calling for regulation and

0:07:05.450 --> 0:07:09.190
<v Speaker 2>better safeguards. So it is being taken very seriously, though

0:07:09.210 --> 0:07:12.770
<v Speaker 2>there is a divide within the AI community and indeed

0:07:12.810 --> 0:07:16.290
<v Speaker 2>with the Trump administration itself about what the risks really are,

0:07:16.330 --> 0:07:18.370
<v Speaker 2>how dangerous this technology really could be.

0:07:19.110 --> 0:07:21.560
<v Speaker 1>You mentioned the FINRA model there, but in terms of

0:07:21.620 --> 0:07:24.600
<v Speaker 1>putting guardrails on technology, it's a debate we've seen play out,

0:07:24.640 --> 0:07:27.930
<v Speaker 1>for example, in social media as well Is there a

0:07:27.970 --> 0:07:31.230
<v Speaker 1>model that actually tells us how we can regulate a

0:07:31.250 --> 0:07:34.440
<v Speaker 1>technology like AI, given its nature and the risks of

0:07:34.480 --> 0:07:35.440
<v Speaker 1>it going rogue?

0:07:35.980 --> 0:07:38.640
<v Speaker 2>Someone like Mark Zuckerberg from Meta has come out just

0:07:38.680 --> 0:07:40.540
<v Speaker 2>in the last couple of days and said, what you

0:07:40.580 --> 0:07:44.780
<v Speaker 2>need is independent evaluators inside the organization to evaluate these

0:07:44.820 --> 0:07:47.210
<v Speaker 2>models before they're released. He doesn't seem to be making

0:07:47.870 --> 0:07:50.630
<v Speaker 2>the case for a new regulatory framework. So he seems

0:07:50.670 --> 0:07:53.610
<v Speaker 2>to be taking a slightly different approach to Dario Modi

0:07:54.050 --> 0:07:57.140
<v Speaker 2>and Anthropic and others. You could look to different industries,

0:07:57.200 --> 0:07:59.739
<v Speaker 2>the airline industry, for example. You can't put out Airbus

0:07:59.760 --> 0:08:03.320
<v Speaker 2>and Boeing can't put aircraft out before they've been rigorously

0:08:03.400 --> 0:08:06.720
<v Speaker 2>checked and before they have appropriate licenses and credentials and

0:08:06.740 --> 0:08:10.620
<v Speaker 2>the sign-off. And the same for the car makers as well.

0:08:10.640 --> 0:08:13.340
<v Speaker 2>And both of those industries started off putting out products

0:08:13.380 --> 0:08:16.060
<v Speaker 2>that were very dangerous. And obviously, we've evolved and come

0:08:16.080 --> 0:08:18.400
<v Speaker 2>a long way since then. And there are now regulatory

0:08:18.420 --> 0:08:21.560
<v Speaker 2>frameworks around both of those industries to mitigate the risks

0:08:22.190 --> 0:08:25.650
<v Speaker 2>and very effectively. So, some in the AI world and

0:08:25.670 --> 0:08:29.210
<v Speaker 2>some in this space point to regulatory regimes like that.

0:08:29.250 --> 0:08:33.710
<v Speaker 2>But again, there is debate because One of the contentions

0:08:33.850 --> 0:08:36.770
<v Speaker 2>from those within the Trump administration is that this is

0:08:36.830 --> 0:08:41.230
<v Speaker 2>essentially Anthropic and OpenAI, a duopoly calling for regulations so

0:08:41.270 --> 0:08:43.750
<v Speaker 2>that they can lock in their market advantage and stop

0:08:43.790 --> 0:08:46.929
<v Speaker 2>some of their competitors catching up because their competitors will

0:08:47.170 --> 0:08:50.850
<v Speaker 2>be wrapped up in this regulatory burden. And Anthropic and

0:08:50.929 --> 0:08:53.710
<v Speaker 2>OpenAI will be able to, because they've got such market share,

0:08:54.270 --> 0:08:57.130
<v Speaker 2>work through that regulatory burden and have a clear advantage

0:08:57.150 --> 0:08:59.300
<v Speaker 2>that will be locked in. That is one of the

0:08:59.340 --> 0:09:03.500
<v Speaker 2>counters to the calls for a government regulatory body or

0:09:03.540 --> 0:09:05.440
<v Speaker 2>an international regulatory body for this technology.

0:09:06.160 --> 0:09:08.500
<v Speaker 1>You've mentioned some of the previous challenges that we've seen

0:09:08.520 --> 0:09:10.989
<v Speaker 1>with technology and that we've overcome in something like the

0:09:11.450 --> 0:09:14.950
<v Speaker 1>car industry where there have been questions of safety. But

0:09:14.990 --> 0:09:17.510
<v Speaker 1>do we need to rely on the companies here to

0:09:18.190 --> 0:09:21.130
<v Speaker 1>move us towards a model that's more protected or to

0:09:21.170 --> 0:09:24.240
<v Speaker 1>have the safeguards in place? Or does this need some

0:09:24.300 --> 0:09:27.760
<v Speaker 1>sort of overarching government or otherwise regulations?

0:09:28.790 --> 0:09:31.220
<v Speaker 2>Well, that is a question that is getting a lot

0:09:31.240 --> 0:09:35.140
<v Speaker 2>of traction in the US and some bipartisan traction as well.

0:09:35.200 --> 0:09:39.900
<v Speaker 2>Both Democrats, independents like Bernie Sanders and Republicans working together

0:09:39.980 --> 0:09:43.390
<v Speaker 2>to call for greater scrutiny and a regulatory framework. And

0:09:43.870 --> 0:09:47.300
<v Speaker 2>laws in place, regulatory framework in place to make sure

0:09:47.340 --> 0:09:50.910
<v Speaker 2>that the risks around this technology are fully accounted for

0:09:50.910 --> 0:09:53.890
<v Speaker 2>and that this can be safely put out into the world.

0:09:53.890 --> 0:09:56.740
<v Speaker 2>Others say that the current laws current laws in place,

0:09:57.340 --> 0:10:00.920
<v Speaker 2>company liability, for example, mean that should an anthropic or

0:10:00.940 --> 0:10:03.520
<v Speaker 2>an open AI put a model out that proves dangerous

0:10:03.559 --> 0:10:06.520
<v Speaker 2>and ends up harming people, then they will face litigation.

0:10:06.640 --> 0:10:08.989
<v Speaker 2>And that in itself will be a curve, that existing

0:10:09.050 --> 0:10:11.370
<v Speaker 2>laws are enough. And that's something that the NVIDIA boss

0:10:11.790 --> 0:10:15.130
<v Speaker 2>and CEO, Jensen Huang, that's the case that he has made.

0:10:15.610 --> 0:10:19.100
<v Speaker 2>There are examples of the industry itself slowing the pace

0:10:19.120 --> 0:10:22.060
<v Speaker 2>of this innovation and doing it in a way that

0:10:22.080 --> 0:10:25.059
<v Speaker 2>they would describe as mindful and thoughtful. Anthropic, for example,

0:10:25.120 --> 0:10:28.440
<v Speaker 2>with its Project Glasswing and one of its most powerful models,

0:10:29.020 --> 0:10:32.460
<v Speaker 2>only putting that out to a select handful of companies,

0:10:33.200 --> 0:10:36.540
<v Speaker 2>Mythos model, so that they could check their cyber resilience

0:10:36.580 --> 0:10:39.819
<v Speaker 2>to this model. And that has still not been widely distributed.

0:10:39.840 --> 0:10:41.740
<v Speaker 2>So that's an example of one of the frontier labs

0:10:42.080 --> 0:10:45.000
<v Speaker 2>taking what they would say is a responsible approach to

0:10:45.020 --> 0:10:48.010
<v Speaker 2>this technology. Mark Zuckerberg says that they have also taken

0:10:48.050 --> 0:10:50.330
<v Speaker 2>similar measures and slowed the release of some of their

0:10:50.370 --> 0:10:53.350
<v Speaker 2>models for similar reasons. Sam Altman has also said that

0:10:53.390 --> 0:10:55.810
<v Speaker 2>they have put in place similar checks and balances to

0:10:55.850 --> 0:10:58.250
<v Speaker 2>make sure that the models they put out don't pose

0:10:58.350 --> 0:11:00.449
<v Speaker 2>a risk and don't pose a danger. And certainly there

0:11:00.470 --> 0:11:02.790
<v Speaker 2>are big teams in all of these frontier labs that

0:11:02.820 --> 0:11:05.720
<v Speaker 2>are working on that alignment question, making sure the models

0:11:05.740 --> 0:11:09.090
<v Speaker 2>are do align with our own human interests, though there

0:11:09.130 --> 0:11:12.070
<v Speaker 2>are still grey areas around that, and putting and baking

0:11:12.170 --> 0:11:15.850
<v Speaker 2>in guardrails into these models so that, for example, the

0:11:15.870 --> 0:11:17.809
<v Speaker 2>bad actor can't take them and use them to develop

0:11:18.150 --> 0:11:22.569
<v Speaker 2>biological weapons. But those measures are by no means conclusive.

0:11:23.309 --> 0:11:26.240
<v Speaker 1>Okay. Well, giving us plenty to watch in this space

0:11:26.270 --> 0:11:29.140
<v Speaker 1>going forward. Tom McKenzie, anchor of Bloomberg Tech Europe on

0:11:29.160 --> 0:11:31.730
<v Speaker 1>Bloomberg Television. Thank you very much for joining us. For

0:11:31.770 --> 0:11:35.070
<v Speaker 1>more explanations like this from our team of 3,000 journalists

0:11:35.170 --> 0:11:38.920
<v Speaker 1>and analysts around the world, go to bloomberg.com slash explainers.

0:11:41.000 --> 0:11:43.559
<v Speaker 1>I'm Stephen Carroll. This is Here's Why. I'll be back

0:11:43.600 --> 0:11:45.370
<v Speaker 1>next week with more. Thanks for listening.