Speaker 1: Bloomberg Audio Studios, podcasts, radio news.
Speaker 2: Nicholas Carlini is a well known AI security researcher. He
Speaker 2: works with Anthropic to stress tests and software for ways
Speaker 2: it could be used by bad actors, and back in February,
Speaker 2: he took a trip.
Speaker 1: He was on holiday in Bali. He was actually at
Speaker 1: a friend's wedding with his wife.
Speaker 2: Margie Murphy is a cybersecurity reporter for Bloomberg News. She
Speaker 2: says at the time of Carlini's vacation, Anthropic had just
Speaker 2: released a version of a new AI model called Mythos
Speaker 2: for internal review.
Speaker 1: So that meant that anyone in the company could start
Speaker 1: trying out and report back if there were any issues
Speaker 1: with it. And Nicholas never takes a day off, is
Speaker 1: so excited by these kind of things, got his laptop
Speaker 1: and immediately started testing to see whether the model would
Speaker 1: do all sorts of bad things that he asked it to.
Speaker 2: To Carlini's shock and concern, it would not.
Speaker 1: Only at his direction was it going and finding flaws
Speaker 1: in some really really important software that underpins a lot
Speaker 1: of modern computing, but it was finding ways to find
Speaker 1: these flaws and turn them into exploits.
Speaker 2: Once he got back to anthropics San Francisco office. Carlini
Speaker 2: kept testing, and he found that this new model could
Speaker 2: create tools to break into all sorts of software all
Speaker 2: by itself. As Margie and her colleagues put it, Mythos
Speaker 2: could conduct the digital equivalent of a bank robbery.
Speaker 1: The executive started talking about, you know, what do we do?
Speaker 1: We had planned on releasing this as a part of
Speaker 1: Claude to the general public, but we might have a
Speaker 1: national security risk on our hands here.
Speaker 2: Anthropics executives decided to hold Mythos back from the market
Speaker 2: and instead share it selectively with some big tech companies
Speaker 2: and banks, and even with some of its AI rivals,
Speaker 2: so that they could start seeing what Mythos was capable
Speaker 2: of and protect themselves. Some skeptics say the fears Anthropic
Speaker 2: is stoking might be overblown. Many outside researchers haven't had
Speaker 2: a chance to validate the company's claims already, though there
Speaker 2: are signs it might be hard to keep this tool contained.
Speaker 2: Bloomberg has learned that a small group with unauthorized users
Speaker 2: has accessed on propics new Methos AI model.
Speaker 1: There's an unwritten rule in cybersecurity that nothing can ever
Speaker 1: be one hundred percent secure.
Speaker 2: I'm Sarah Holder, and this is the big take from
Speaker 2: Bloomberg News today on the show, how Anthropic decided Mythos
Speaker 2: was too dangerous for public release and what a new
Speaker 2: wave of AI enabled hacking could mean for businesses, banks,
Speaker 2: governments and everybody else. Was developing Mythos, it intended for
Speaker 2: it to be the latest update to its large language
Speaker 2: model Claude. It wasn't meant to be a special cybersecurity
Speaker 2: tool or a master hacker. But Bloomberg's Marge Murphy says
Speaker 2: that when the company tested an earlier version of the model,
Speaker 2: it found dozens of examples of concerning behavior not following
Speaker 2: human instructions and in rare cases, covering its tracks Afterward,
Speaker 2: once the model used a series of steps to escape
Speaker 2: its testing environment and gain broad access to the Internet,
Speaker 2: where it began publishing materials online, Mythos, the company found
Speaker 2: was really good at exploiting vulnerabilities.
Speaker 1: Realistically, humans have for years been able to find fools
Speaker 1: and software, and we have not only internal teams at
Speaker 1: major companies whose job is to find that and fix
Speaker 1: them as soon as possible. There are companies that you know,
Speaker 1: that's their business model. But what's incredible about Mythoss, according
Speaker 1: to Anthropic, is that it could find so many and
Speaker 1: so many that have existed for years. It was finding
Speaker 1: bugs that are twenty seven years old in software that
Speaker 1: is considered highly secure. And in addition, it could also
Speaker 1: find different flaws. And now floors are just entry points.
Speaker 1: They may not be a big deal, but what it
Speaker 1: was able to do was chain together multiple flaws to
Speaker 1: create an exploit, which is when you are able to
Speaker 1: get into a system and do something useful the floor
Speaker 1: That was the kind of turning point I think where
Speaker 1: they thought, wow, this is something that that is actively
Speaker 1: finding flaws, putting them together, being very smart about how
Speaker 1: it's doing it, and doing it on mass.
Speaker 2: Another thing that Anthropic leaders talked about was Mythos's ability
Speaker 2: to act with greater autonomy. What made its capabilities so
Speaker 2: concerning previous models, you'd need to give a little bit
Speaker 2: of direction, So if you're trying to hack something, you
Speaker 2: would have to have a good bit of knowledge about
Speaker 2: the system you're trying to hack. You would have to
Speaker 2: kind of really point the model in certain directions. With Mythos,
Speaker 2: there is less direction needed. The way that Nicholas described
Speaker 2: it to me. It was coming back with these sort
Speaker 2: of methods that it could do itself. You don't want
Speaker 2: your technical tools to have too much autonomy. You want
Speaker 2: them to be working for you and explaining why they're
Speaker 2: doing it. So when something does something that you don't
Speaker 2: really understand and it doesn't tell you why, I think
Speaker 2: that's always a concern. What kinds of companies or industries
Speaker 2: or institutions could be most vulnerable to a cyber attack
Speaker 2: from a extra autonomous LM like Mythos.
Speaker 1: So this is the thing everyone everything, Because Mythos isn't
Speaker 1: changing hacking in a sense, it's changing the speed and
Speaker 1: the acceleration of hacking. Flaws exist in loads of software,
Speaker 1: so from the financial industry, to manufacturing to even governments
Speaker 1: who might be concerned about espionage. Mythos isn't actively going
Speaker 1: and hacking those things right now. But if a model
Speaker 1: like Mythos was to come out, anything that's on the
Speaker 1: Internet is potentially able to be hacked. Right so we're
Speaker 1: reliant upon the industry defending itself and putting in place
Speaker 1: proper measures.
Speaker 2: Also, Anthropic discovers these potential concerns, Carlini and others start
Speaker 2: raising the alarm. Anthropic decides to not release Mythos publicly,
Speaker 2: but release it to this limited group called Project glass Wing.
Speaker 2: Who gets this first look and how do they decide
Speaker 2: who is going to get Mythos in this limited release.
Speaker 1: I did ask Anthropic, and they're careful of giving too
Speaker 1: many details about how exactly they decide. But the reassuring
Speaker 1: thing about the AI industry is, despite being fierce competitors,
Speaker 1: there is an element of working together when it comes
Speaker 1: down to security. And so they came up with this
Speaker 1: idea to create a sort of cohort of companies including
Speaker 1: cybersecurity companies which would be their competitors, including Google and
Speaker 1: Apple and AWS who are also trying to crush them
Speaker 1: in the AI market. And yeah, they formed this group
Speaker 1: called glass Wing and invited people to try out Mythos
Speaker 1: and see whether they could find any flaws and bugs
Speaker 1: in their suptware supply chain. You've got some of the
Speaker 1: biggest tech companies in the world, You've got some finance players,
Speaker 1: and as Anthropic would say, they're talking to government agencies.
Speaker 2: And what's the reasoning here. They've discovered the big potential
Speaker 2: threat in Mythos these cybersecurity capabilities. Why release it to anyone? YEA,
Speaker 2: why release it to this limited group of players at all?
Speaker 1: At first it was totally confusing. Why are you working
Speaker 1: with your competitors to give them access to, you know,
Speaker 1: your IP, this product that probably would be very valuable
Speaker 1: for you if you just released it as claud and
Speaker 1: you're also giving it to them for free as well,
Speaker 1: because there's a kind of a credit program that Anthropic offered.
Speaker 1: Anthropic says they released it to these limited players so
Speaker 1: they could use it for good. So rather than tone
Speaker 1: it down, don't tell anyone about the scary thing that
Speaker 1: you might freak everyone out, and just make sure that
Speaker 1: it doesn't do that again, and we'll just release it.
Speaker 1: We could kind of make a big song and dance
Speaker 1: of it and say, hey, we're gonna responsibly give it
Speaker 1: to a few people who we trust, who are to
Speaker 1: try it out, test it out, see what we learn
Speaker 1: from it, and then in the meantime signal to the
Speaker 1: world that we have this extremely powerful model which could
Speaker 1: do no harm for their marketing strategy. And I think,
Speaker 1: to be fair to Anthropic, they have cared about AI
Speaker 1: safety for some time. Their founders come out of open
Speaker 1: AI and their whole Mission from the beginning has been
Speaker 1: talking about using AI in a responsible way. They were thinking, look,
Speaker 1: other companies are going to have something as powerful as
Speaker 1: we do soon if they don't already, and maybe adversaries
Speaker 1: like Nation States who are working on kind of military
Speaker 1: style hacking operations, they may have that too, So at
Speaker 1: least let's talk about it.
Speaker 2: It does seem like a very good marketing strategy, right, Like,
Speaker 2: we release this tool, it's so scary good that you
Speaker 2: need it to protect yourself. You need it now, you
Speaker 2: need it first before we let the rest of the
Speaker 2: world get their hands on it.
Speaker 1: It's an incredibly effective marketing tool. Everyone wants to see
Speaker 1: what it can do. Everyone wants to get their hands
Speaker 1: on it. To have access to Mythos is like a
Speaker 1: golden ticket. And you can see online and there's been
Speaker 1: so many hot takes on oh this is just buzz
Speaker 1: and you know, how do we know that it's true?
Speaker 1: But I've spoken to a lot of Glass Wing members,
Speaker 1: the people who actually have access to it, in addition
Speaker 1: to the people at Anthropic who have explained how it works,
Speaker 1: and they've all said to me, it really is very powerful.
Speaker 1: Are they going to go and live in a cave now,
Speaker 1: And do they have fears that like satellites are going
Speaker 1: to drop from the sky.
Speaker 2: No.
Speaker 1: Do they fear that if something like Mythos got into
Speaker 1: the hands of an adversary there would be problems, yes.
Speaker 2: Coming up. What companies are learning from their Mythos testing
Speaker 2: and how they're sorting through the results, and what anthropics
Speaker 2: legal battle with the US Defense Department means for other
Speaker 2: federal agencies that want their own look. The same day
Speaker 2: that Anthropic announced the existence of Mythos, US Treasury Secretary
Speaker 2: Scott Besson and Federal Reserve Chair Jerome Powell held this
Speaker 2: meeting with Wall Street leaders in Washington to discuss the model,
Speaker 2: its power, and the risks it could pose to banking software.
Speaker 2: Bloomberg cybersecurity reporter Margie Murphy says their directions were clear.
Speaker 1: Everyone needed to make sure that they had their house
Speaker 1: in order and that they were prepared for the potential
Speaker 1: for more hacks to take place.
Speaker 2: The fact that the US government had Mythos on its
Speaker 2: radar and that it was calling urgent meetings to tell
Speaker 2: business leaders to use it defensively was significant. Margy says
Speaker 2: it was also a bit ironic because the US government
Speaker 2: has been feuding pretty publicly with Entthropic. In March, it
Speaker 2: declared the company a supply chain risk after Anthropic give
Speaker 2: the Pentagon unfettered access to its tools.
Speaker 1: Anthropic had been considered one of the government's top AI
Speaker 1: tools for some time. Lots of places we're using it,
Speaker 1: Department of Defense was using it. And then there was
Speaker 1: this spat that happened between Anthropic and Pete Hegseth. Anthropic said,
Speaker 1: we don't want our tool being used for certain things,
Speaker 1: and I think the government response was, well, you can't
Speaker 1: pick and choose. You're here to provide national security and
Speaker 1: if you don't, will label you a supply chain risk.
Speaker 1: And that's what's happened, and it's going through the courts
Speaker 1: at the moment. So when Anthropic announced that they had
Speaker 1: this powerful model, I think a lot of government agencies
Speaker 1: were like, well, what do we do now. We should
Speaker 1: have access to this, we should be able to see it.
Speaker 1: My reporting suggests that the Treasury was really keen to
Speaker 1: get to get access to it. Anthropic has told us
Speaker 1: that they were doing government briefings and letting them know
Speaker 1: about the helpfulness.
Speaker 2: Of this tool, and what about Wall Street?
Speaker 1: So the banking industry has been using LLMS and AI
Speaker 1: to try and protect itself already, So I think at
Speaker 1: first what we were hearing was, oh, is this really
Speaker 1: anything new? And then as we know, some banks have
Speaker 1: had access to it, and I think they're starting to
Speaker 1: realize that this shows that the thing that they've all
Speaker 1: been worried about for maybe the past six to twelve
Speaker 1: months is now a real threat and that the time
Speaker 1: is now to act because if something bad was to
Speaker 1: happen that impacts the US economy.
Speaker 2: And as for Project glass Wing, what do you think
Speaker 2: this new era of experimentation will look like for these
Speaker 2: companies and institutions. How challenging will it be to actually
Speaker 2: fix the vulnerabilities that Mythos identifies.
Speaker 1: Yeah, so I was talking to one Glass Wing member
Speaker 1: and they were talking about how they'd already used the
Speaker 1: tool they found it in be powerful, surprisingly powerful, and
Speaker 1: would love to continue using it. However, even though they
Speaker 1: found all these flows, because obviously I asked them, wow,
Speaker 1: do you find anything really scary? You know, what if
Speaker 1: you've found and they're like, we actually don't know what
Speaker 1: we found it because then going through it all and
Speaker 1: figuring out what to patch, what to flag, is a
Speaker 1: critical issue. I think it will take a while for
Speaker 1: the companies to figure that all out, and it will
Speaker 1: be really interesting to see the feedback because part of
Speaker 1: glass wing is that these companies are meant to be
Speaker 1: sharing information about what they're finding.
Speaker 2: But I'm wondering could this create sort of an unfair
Speaker 2: dynamic in the marketplace, Like there's a divide between these
Speaker 2: companies and governments that are part of anthropics trial and
Speaker 2: then companies that are kind of left to fend for themselves.
Speaker 1: Yeah, the have and the have nots. I think that
Speaker 1: is absolutely a concern. The people who are going to
Speaker 1: be most impacted by AI models like Mythos coming out
Speaker 1: into the world are the smaller players who maybe don't
Speaker 1: have the budgets to make sure that when this kind
Speaker 1: of bug apocalypse is as everyone's describing it as, comes
Speaker 1: that they might be the lower hanging through where lots
Speaker 1: of flaws and exploits are found, and then when when
Speaker 1: someone tries to use those, they don't have the defense
Speaker 1: in place.
Speaker 2: Well, this week, Bloomberg reported that a small group of
Speaker 2: unauthorized users have already been able to access this model
Speaker 2: using various tactics. A person familiar with the situation told
Speaker 2: our colleague that they just want to play around with
Speaker 2: the models, not reek havoc. But Markie, what does this
Speaker 2: tell us about how hard it could be for Anthropic
Speaker 2: to actually keep mythos in a box.
Speaker 1: When I spoke to Anthropic when they first told me
Speaker 1: about this limited release, I did ask the question, how
Speaker 1: do you when you are opening this up to a
Speaker 1: number of organizations, how do you protect what those organizations
Speaker 1: are doing? And how do you have oversight? You know,
Speaker 1: the more people you open this up to, the more
Speaker 1: ve as there are for potentially malicious actors to try
Speaker 1: and get access to. Whether that's socially engineering them figuring
Speaker 1: out who's the person who might have access and kind
Speaker 1: of using these manipulative tactics to get into their systems,
Speaker 1: or whether it's other means of hacking. And so it
Speaker 1: raises concerns about who potentially might already have access to it,
Speaker 1: and if AI hobbyists want it, hackers probably want it too,
Speaker 1: and they're probably working around the clock to try and
Speaker 1: get access to it right now.
Speaker 2: And wouldn't Anthropics say about Bloomberg's reporting on this unauthorized access?
Speaker 1: So Anthropics said they're investigating we reported that it was
Speaker 1: partially through this third party contractor that Anthropic uses that
Speaker 1: these AI hobbyists we're able to get access. They are
Speaker 1: looking into that and making sure that that access potentially
Speaker 1: gets cut off and we'll wait and see what they
Speaker 1: come back with.
Speaker 2: And is there a plan to release my Thos to
Speaker 2: the general public eventually?
Speaker 1: So what Anthropic told me was that they don't plan
Speaker 1: to release this generally, so there won't be a Mythos
Speaker 1: update to Claude. What they have released in the past
Speaker 1: couple weeks is Opus four point seven, which is kind
Speaker 1: of a bit of a dressed down Mythos. The idea
Speaker 1: is that they may possibly broaden it out as a
Speaker 1: cybersecurity initiative, so people may be able to get use
Speaker 1: of it if they are only using it for defensive reasons,
Speaker 1: just for the cybersecurity element.
Speaker 2: What about nthropics competitors like open Ai, Google, even you know, Deepseek.
Speaker 2: Are they close to releasing models that have similar capabilities
Speaker 2: to mythols And do you think they will take the
Speaker 2: same approach as Anthropic, which you know was explicitly founded
Speaker 2: on this pledge to prioritize safety.
Speaker 1: I do not doubt that competitors have found or developed
Speaker 1: models that have near enough or similar capabilities, but maybe
Speaker 1: chose a different path in how to release or talk
Speaker 1: about them. Open Ai has come out with a last week.
Speaker 1: They announced that they have a kind of bug finding
Speaker 1: capability as well. Google has big Sleep, which offers a
Speaker 1: similar thing. The way that the AI race is going
Speaker 1: is that everyone is kind of edging quite close to
Speaker 1: each other. It depends on whether they make that decision
Speaker 1: to release it or talk about it publicly.
Speaker 2: Part of what we're talking about is that these models
Speaker 2: are getting better faster, and especially faster than the ability
Speaker 2: of human regulators to agree on how to put guardrails
Speaker 2: on this technology. And I'm wondering, you know, is government
Speaker 2: regulating this? Can they?
Speaker 1: Yeah? So, the fact that a private company had to
Speaker 1: come to the decision to do a limited release of
Speaker 1: one of its models because it was so powerful just
Speaker 1: shows the state of AI regulation right now. Everything is
Speaker 1: moving just so quickly. Even if someone was to suggest
Speaker 1: the perfect plan to regulate, you'd probably have to tear
Speaker 1: that up a week later and start again.
Speaker 2: This is the Big Take from Bloomberg News. I'm Sarah
Speaker 2: Holder to get more from the Big Take and unlimited
Speaker 2: access to all of bloomberg dot Com. Subscribe today at
Speaker 2: bloomberg dot com slash podcast offer. If you liked this episode,
Speaker 2: make sure to subscribe and review The Big Take Wherever
Speaker 2: you listen to podcasts. It helps people find the show.
Speaker 2: Thanks for listening. We'll be back tomorrow