Speaker 1: Bloomberg Audio Studios, podcasts, radio news. We are here because
Speaker 1: on July twenty second, Open Ai and Hugging Face disclosed
Speaker 1: that two powerful AI models went out of a sandbox
Speaker 1: or closed testing environment, gained Internet access, and then were
Speaker 1: able to hack hugging Faces systems in what was seen
Speaker 1: at the time as a watershed moment in AI safety.
Speaker 1: And now an investigation has been in part concluded and
Speaker 1: there's been time to assess what happened. Hugging Face CEO
Speaker 1: Clem DeLong is here to discuss exactly that, and I
Speaker 1: think Clem, that's probably the best place to start. What happened.
Speaker 2: A lot happens right in the past few weeks, we
Speaker 2: noticed this closed as you mentioned, the first kind of
Speaker 2: like publique instance of an autonomous AI cyber attack. We
Speaker 2: we defended ourselves against it with with an open model,
Speaker 2: interestingly coming from from China.
Speaker 3: UH.
Speaker 2: And we learned that this came from from Open Ai. Literally,
Speaker 2: when you when you think about cyber attacks, you think
Speaker 2: about nation states, you think about UH, you know, hacker groups,
Speaker 2: you don't really think about you know, one of the
Speaker 2: most prominent American AI company UH.
Speaker 3: And obviously since.
Speaker 2: Then, we we learned that Entropic was was also facing
Speaker 2: some of the similar issues. So really, uh, you know,
Speaker 2: unprecedented kind fla like event happening here new development in
Speaker 2: in the Saga.
Speaker 1: Of AI, we will discuss what has been highlighted by this,
Speaker 1: which is closed versus open debate China's work on AI
Speaker 1: America's response to this. But going back to the very basics,
Speaker 1: this was two powerful models from Open Ai right one
Speaker 1: least one unreleased, but they had their guard rails lowered
Speaker 1: for the purposes of evaluation Open Air. I said to
Speaker 1: the models. They instructed the models go out and do something,
Speaker 1: and I think it would be useful to the audience for
Speaker 1: you to explain that part, whether the Open Air models
Speaker 1: were just following instructions or if they were ais that
Speaker 1: went rogue.
Speaker 3: Yeah, I mean I.
Speaker 2: Joked with the team a few days after it was
Speaker 2: announced that sometimes we ask agents to think outside of
Speaker 2: the box, but we don't want them to think outside
Speaker 2: of the sandbox in that In that case, I think,
Speaker 2: I think it was a mix of kind of like
Speaker 2: you know, mistakes and kind of like the systems Internity.
Speaker 2: I think, weren't you know, good enough to prevent this
Speaker 2: to happen?
Speaker 3: After that. On our side, the attack was really interesting.
Speaker 2: Over seventeen thousand different actions taken over four and a
Speaker 2: half days, So the speed and the volume of the
Speaker 2: actions taken were nowhere close to what human cyber attacks
Speaker 2: could be. It wasn't particularly smart or sophisticated. It was
Speaker 2: more kind of like someone described it as a bear
Speaker 2: robbing really everything in the system to try to find
Speaker 2: the honeypot in a way. But obviously because it's an
Speaker 2: autonomous AI system, it does that pretty well.
Speaker 1: After the disclosure on July twenty second, you got on
Speaker 1: an aeroplane and flew from Miami to San Francisco in
Speaker 1: part to get with the Open AI team and do
Speaker 1: this investigation. You just said that the systems were not
Speaker 1: in place to prevent this happening. More about that prevent
Speaker 1: what happening? Where were the points of failure?
Speaker 2: Well, I can't talk for for them, but from what
Speaker 2: I've understood from what they released, and I think they're
Speaker 2: going to release more in the coming days, which I'm
Speaker 2: excited about. You know, they had kind of like an
Speaker 2: evaluation sandbox, right, which was supposed to be like a
Speaker 2: contained environment for for the I models. Uh and and
Speaker 2: unfortunately you know, there were some weaknesses that led the
Speaker 2: agents to be able to get out of it, get
Speaker 2: access to to internet, and decide to run a cyber
Speaker 2: attack against hiking faces. So that that's one one first
Speaker 2: fact things that I think we can we can improve
Speaker 2: in in the future now that we understand that these
Speaker 2: systems are capable of that. Obviously, I think that the
Speaker 2: monitoring part is important, right, because the faster you can
Speaker 2: detect that, the better it is. Right, we learn about
Speaker 2: some entropic instances that they haven't you know, seen or
Speaker 2: detected that happened three months ago or something like that,
Speaker 2: So obviously we want to monitors these systems better. And
Speaker 2: then one last interesting thing, as I mentioned, we defended
Speaker 2: ourselves with an open model, right, and some of the
Speaker 2: guard rails prevented us from using frontier APIs to defend ourselves.
Speaker 3: So kind of like improving the tools or.
Speaker 2: Defenders, you know, instead of oblissing about not giving them
Speaker 2: to attackers, I think would be a good thing in
Speaker 2: the future to make sure these incidents are not too harmful.
Speaker 1: Okay, you've taken us there, so we'll go there. In
Speaker 1: this incident, it was too powerful, but closed open AI
Speaker 1: models where open AI lowered the guardrails in place to
Speaker 1: test their full cyber capabilities. They escaped the sandbox or
Speaker 1: testing environment, gained the Internet access, and were able to
Speaker 1: access your platforms mistakenly. You defended yourself using an open model,
Speaker 1: but it was a Chinese model, But the abilities of
Speaker 1: that model to defend you were also diminished by their guardrails.
Speaker 1: Is that a fair statement?
Speaker 2: Yeah? I mean the open model fortunately was kind of
Speaker 2: like a flexible enough that we could use it to
Speaker 2: defend ourselves. So that's kind of like the kind of
Speaker 2: like efficient tools for for defenders.
Speaker 1: We're live on Bloomberg Television and Radio on Balance of Power.
Speaker 1: There'll be people in America and around the world listening
Speaker 1: to this and thinking, okay, very powerful AI models were
Speaker 1: able to escape a testing environment, access the Internet, and
Speaker 1: then access mistakenly and other companies platform What was the
Speaker 1: net result? Did something bad happen as a result of this?
Speaker 2: Well, I think not as bad as it could have been.
Speaker 2: It was bad for us for our team, right, I mean,
Speaker 2: we're all security and for a teamwork worked on this
Speaker 2: for for quite a while. Uh, you know, it created
Speaker 2: some some challenges for us, obviously, but you know it
Speaker 2: could have been way worse. Right, Like, for example, we're
Speaker 2: in the I platform, right, so we have kind of
Speaker 2: like good ways to defend ourselves, but a lot of
Speaker 2: other organizations companies don't have kind of like the same defenses. Obviously,
Speaker 2: some some domains are you know, more at risks than others.
Speaker 2: But I think it could have could have been much worse.
Speaker 2: But we need to take this kind of like seriously
Speaker 2: as a wake up call for us to kind of
Speaker 2: like work more on getting these systems secure and giving
Speaker 2: more tools to to all defenders, uh and, and generally
Speaker 2: creating more transparency and more monitoring of these systems.
Speaker 1: Has the US government taken this as a wake up call?
Speaker 1: Which branches of government have reached out to you since
Speaker 1: the event was disclosed.
Speaker 2: I'm not gonna talk to kind of like more and
Speaker 2: more the private conversations that we've had, but you know,
Speaker 2: we obviously reported that TOKLF like the relevant authorities and
Speaker 2: have had kept like a bunch of conversations with different
Speaker 2: different organizations, different kinds like Congress members, uh and and
Speaker 2: people from from from government. I mean, I mean, there
Speaker 2: are a couple of things that I think we need
Speaker 2: to do and get right in in the next few months. First,
Speaker 2: in my opinion, we have to make sure that cyber attacks,
Speaker 2: even when they're done by agents, stay a crime and
Speaker 2: illegal and make sure to enforce that. But otherwise we're
Speaker 2: going to end up in the world where everyone is
Speaker 2: cyber attacking everyone with really no this incentive to it.
Speaker 2: A Second, and we need to create more transparency, like
Speaker 2: for example, why not doing disclosure Manda to read disclosure
Speaker 2: when agent cyber attack is happening.
Speaker 3: Uh. And Third, we have.
Speaker 2: To come like give more tools to defenders, like we
Speaker 2: we mentioned open models. These are like some of the
Speaker 2: tools that cyber attack defenders need to make sure that
Speaker 2: they can defend themselves.
Speaker 1: To recap, you're saying that the illegality of an AI agent,
Speaker 1: a non human an AI agent carrying out a hack
Speaker 1: needs to be enforced and basically broader regulations. The pathway
Speaker 1: to that is legislative. To your mind, you always it's
Speaker 1: not really for me for me to say.
Speaker 2: Obviously, I'm not like a legal legal expert or policy maker,
Speaker 2: and I think that's the topic that we need.
Speaker 3: To think about. If you kind of like, uh, take
Speaker 3: can like higher view.
Speaker 2: We already have some autonomous systems in our life, right,
Speaker 2: like a self driving car, you know, and many others.
Speaker 2: We made sure there's kind of like liability, right if
Speaker 2: you fall asleep at the wheel of your self driving car,
Speaker 2: then you're responsible if you're hitting another car in a
Speaker 2: similar way, we need to make sure that the legal
Speaker 2: framework for autonomous agents is clear and defines and useful
Speaker 2: for the field and the American society.
Speaker 1: We're live on Bloomberg Television and Bloomberg Radio. This is
Speaker 1: balance of power, and we're speaking to Hugging Face CEO
Speaker 1: clemed Along, whose company disclosed on July twenty second that
Speaker 1: two powerful open AI models without authorization or mistakenly accessed
Speaker 1: or breached his company systems as part of a cyber
Speaker 1: evaluation that opening I was doing, you know, Clem, timing
Speaker 1: is everything. Within days, one of the most important people
Speaker 1: in the world of technology came out with a letter
Speaker 1: backing America focusing on open models. I'm talking about Sati
Speaker 1: Nadella Jensen one. More recently, awsc or Matt Garman has
Speaker 1: joined many others. You know, do you think that the
Speaker 1: timing of that was the sort of catalyzed or a
Speaker 1: direct response to the July twenty second breach.
Speaker 3: Well, I think it was related.
Speaker 2: I mean, this cyber attack showed that, you know, you
Speaker 2: can create risks with models behind closed doors that are unreleased,
Speaker 2: so just working on you know, preventing the models to
Speaker 2: be released is not the solution, and that you know,
Speaker 2: defenders need open models because you know, to defend yourself
Speaker 2: against cyber attack, you you need to run it on
Speaker 2: your own infrastructure because it's usually on your private data,
Speaker 2: and you need you need to control that. Open models
Speaker 2: gives you that Sometimes corporatory APIs don't give you because
Speaker 2: of gad wills, because of limitations, because of also how
Speaker 2: much it costs. Right, So I think that was a
Speaker 2: perfect perfect example and perfect valuation or for this notion
Speaker 2: of you know, the world's small companies, startups, researchers, but
Speaker 2: also large companies like needing open models wherever they come from.
Speaker 1: I would just point out claim that you know, in
Speaker 1: the course of this conversation, we've talked a lot about
Speaker 1: it was open AI's models. You conducted a joint disclosure
Speaker 1: and then joint investigation with open ai, and of course,
Speaker 1: like we've made every effort to invite open ai onto
Speaker 1: the network, onto the show to discuss their side of
Speaker 1: the story and the investigation. But that open weighted letter
Speaker 1: in part was supposed to sort of outline the best
Speaker 1: benefits of open models, the economic benefit, the business consideration.
Speaker 1: That is very much hugging faces. Realm Right hosts models
Speaker 1: big and small open on its platform. The other thing
Speaker 1: was the concern that there would be over regulation of
Speaker 1: open models by the US government kind of bring that
Speaker 1: full circle for us and why you think that that
Speaker 1: concern is valid or not?
Speaker 2: Yeah, I mean there are there are a lot of
Speaker 2: different topics in the I and and I think the
Speaker 2: role of a lot of people is to is to
Speaker 2: set the priorities. And I think when what this later
Speaker 2: kind of like was was outlining too, is that probably
Speaker 2: one of the biggest risks in the eye is concentration
Speaker 2: of power, right Like it's it's it's concentration of powers, capabilities,
Speaker 2: wealth in a fuel organizations. Why when like everyone else
Speaker 2: would be kind of like lagging behind and be kind
Speaker 2: of like left out in a way, right and open
Speaker 2: models are kind of like a counter force to that, right,
Speaker 2: Like they empower small companies, startups, you know, organizations that
Speaker 2: are not necessarily kind of like Frontieri Labs to build
Speaker 2: the eye themselves to own their intelligence, as some people said.
Speaker 2: And I think that's that's one topic that you know,
Speaker 2: policy makers need to focus on a little bit more
Speaker 2: than they've been focusing on so far. So hopefully this
Speaker 2: letter kind of like puts the topic more prominently on
Speaker 2: their desk.
Speaker 1: After you and an open a I made the disclosure
Speaker 1: in July twenty second that this had happened. You know,
Speaker 1: I spoke to lots of your peers in industry, and
Speaker 1: there was also the kind of acceptance that this wasn't
Speaker 1: some major scandal. You know, open ai has a lot
Speaker 1: of power. The closed models themselves are powerful, but generally speaking,
Speaker 1: industry said, we want to see the frontier labs doing this,
Speaker 1: doing these tests and evaluations of their capabilities, and then
Speaker 1: disclosing when something goes wrong. Just react to that, you know,
Speaker 1: in that sentiment towards the events of that week, I don't.
Speaker 3: Agree at all with that.
Speaker 2: It like, we don't we don't want any company in
Speaker 2: the US running cyber attacks against other companies. This is
Speaker 2: this is a crime. This is illegal for a good
Speaker 2: reason because if you if you create a world where
Speaker 2: everyone is allowed to run cyber attacks against everyone, we're
Speaker 2: in for a lot of trouble, right we You have
Speaker 2: to remember that in our society, most damages and most
Speaker 2: hurts isn't prevented because it is hard to do, right,
Speaker 2: Like if if you think of it, I can you know,
Speaker 2: go across the street and steal a grocery shop and
Speaker 2: it's not that hard to do, right. It is prevented
Speaker 2: because this is immoral and this is illegal, right, So
Speaker 2: I don't do it because because of that.
Speaker 3: And it's the same thing for for cyber attacks. Right.
Speaker 3: You know, if if we end up in the world where.
Speaker 2: These are normalized, cyber attacks are normalized, I think we
Speaker 2: can end up in a very dangerous world the same
Speaker 2: way as you know, I made the comferison with with
Speaker 2: self driving.
Speaker 3: Cars, right.
Speaker 2: You don't want to end up in the world where
Speaker 2: you know, bumping into another car is normalized is okay
Speaker 2: because you're driving a self driving car. Like, we want
Speaker 2: to make sure that our society stays kind of like
Speaker 2: healthy by you know, keeping things that need to be
Speaker 2: must be but legal illegal.
Speaker 1: This brings us back to the central concern of the
Speaker 1: American people, frankly, right, which is open. AI did not
Speaker 1: instruct those two models to go out and hack hugging Face.
Speaker 1: They instructed the two models to undergo the site of
Speaker 1: the evaluation right to do the test. They were able
Speaker 1: to escape the sandboxed environment, gain internet access, and they
Speaker 1: went to hugging faces platforms. In part, I understand because
Speaker 1: the models determined they could find information on hugging Faces
Speaker 1: platform that would help them to pass that test carry
Speaker 1: out that invaluation, but open air I didn't instruct them
Speaker 1: to hack hugging Face. Nevertheless, here we are. You know,
Speaker 1: that's the bit that people are worried about.
Speaker 2: Yeah, yeah, I mean, you know the same way if
Speaker 2: I'm Tesla and I have self drubbing and I fall
Speaker 2: asleep at the wheel, you know, my intention is not
Speaker 2: to bump into another car. But you know, you still
Speaker 2: have kind of like think I think liability. I think
Speaker 2: they're they're shared themselves that you know, there was some
Speaker 2: kind of like a mistakes the turn that there's kind
Speaker 2: of like a ways you know, they could fix some
Speaker 2: of the bugs that led or some of the weaknesses
Speaker 2: in their systems that led the agentic system to be
Speaker 2: able to escape. And I think they're hopeful to actually
Speaker 2: fix this and make sure it doesn't happen again. I
Speaker 2: don't think they're going to try in the future to
Speaker 2: build a system that that kind of like regularly runs
Speaker 2: cyber attacks against I guess other companies.
Speaker 3: I think they're they're going.
Speaker 2: To be old systems that are kind of like a
Speaker 2: stronger better to to make sure these things don't happen
Speaker 2: in the future.
Speaker 1: How you face here, climbed along. We just we just
Speaker 1: have about sixty seconds left in the conversation. You've talked
Speaker 1: about what needs to happen, but what's the net result
Speaker 1: of all of this and what needs to happen next?
Speaker 2: Yeah, I think, uh, you know, we need more transparency.
Speaker 2: I think it was a good wake up call that
Speaker 2: you know, preventing releases is not enough. I think we
Speaker 2: need more transparency on on these systems.
Speaker 3: Uh.
Speaker 2: And second, we need to kind of like equip defenders better, right,
Speaker 2: like for example, with with open models that that we
Speaker 2: used to defend ourselves. So we need more powerful open
Speaker 2: models for for all defenders, so that we prepared for
Speaker 2: for what's coming next.
Speaker 1: Hugging face here, Clem DeLong joining us live on Balance
Speaker 1: of Power. Thank you very much.