Speaker 1: Bloomberg Audio Studios, podcasts, radio news.
Speaker 2: The last week of August isn't exactly known as a
Speaker 2: busy time of year unless you work in California's legislature.
Speaker 3: I'm Senator Scott Wiener, and I am calling in from
Speaker 3: the state capitol in California.
Speaker 2: When we talked recently, Senator Wiener told me he was
Speaker 2: running from vote to vote on everything from regulating hedge
Speaker 2: funds to allowing outdoor cannabis sales, very California stuff.
Speaker 3: We are in our last week of our legislative session,
Speaker 3: so every day is marathon voting. We've already voted on
Speaker 3: a few hundred bills and we have one hundreds more
Speaker 3: before our session ends on Saturday.
Speaker 2: But I wanted to speak to Senator Wiener about one
Speaker 2: very specific bill that was being voted on that day.
Speaker 2: It was one he introduced back in February called SB
Speaker 2: ten forty seven, the Safe and Secure Innovation for Frontier
Speaker 2: Artificial Intelligence Models Act. The bill would hold AI companies
Speaker 2: to new safety standards and hold them legally liable if
Speaker 2: their tools end up causing catastrophic harm. The bill would
Speaker 2: only apply to the biggest AI players, the ones that
Speaker 2: are spending upwards of one hundred million dollars on training
Speaker 2: their AI models or spending ten million dollars fine tuning them.
Speaker 3: We have an obligation in California, as the tech leader
Speaker 3: on the planet, to make good, forward looking, pro innovation,
Speaker 3: pro safety tech policy.
Speaker 2: Now. Senator Wiener has never been one to shy away
Speaker 2: from controversial policy moves. A Democrat, he's been one of
Speaker 2: the biggest champions of zoning reform in the state. But
Speaker 2: introducing this bill in the place where chat, GPT and
Speaker 2: Claude were first unleashed and where OpenAI and anthropic are
Speaker 2: racing towards superintelligence hasn't exactly been popular.
Speaker 3: I knew that there would be significant pushback just because
Speaker 3: of the nature of this subject, and I knew it
Speaker 3: would be forceable, but it was even more than I
Speaker 3: thought that it would be.
Speaker 2: Open AI opposes the bill, so does Nancy Pelosi and
Speaker 2: Andresen Horowitz. But Weiener believes that even with that pushback,
Speaker 2: no state is better positioned to be forward looking in
Speaker 2: this realm.
Speaker 3: California has led on tech policy, on climate policy, on
Speaker 3: so many issues where we are ahead of Congress and
Speaker 3: we set the trend, and we should do that here
Speaker 3: as well.
Speaker 2: Today on the show. The debate over the bill to
Speaker 2: curb AI harms has become a lightning rod for the
Speaker 2: tech community. Could it be the future of tech regulation
Speaker 2: or a cautionary tale? This is the big take from
Speaker 2: Bloomberg News. I'm Sarah Holder. The idea for SB ten
Speaker 2: forty seven came to California State Senator Scott Wiener about
Speaker 2: a year and a half ago. That's when he was
Speaker 2: approached by a group of experts who wanted him to
Speaker 2: take on AI regulation.
Speaker 3: Sometimes in the early part of twenty twenty three, some
Speaker 3: AI technologists came to me and expressed that we're overdue
Speaker 3: for a conversation about AI safety. As we have all
Speaker 3: this amazing, brilliant innovation, how do we make sure that
Speaker 3: as AI scales, that it happens in a responsible way,
Speaker 3: because we have a history of not getting out in
Speaker 3: front of the risks posed by technology and just letting
Speaker 3: things happen and then having to clean up the mess later.
Speaker 2: Now, there are a lot of AI bills floating around
Speaker 2: in California. In fact, during that August session, legislators passed
Speaker 2: another bill on deep fakes, but SB ten forty seven
Speaker 2: focuses on a whole different universe of potential AI harms,
Speaker 2: catastrophic ones think property damage over five hundred million dollars
Speaker 2: or a massive loss of human life. And Wiener says
Speaker 2: he chose to focus on this particular aspect for a reason.
Speaker 3: In the debate on SB ten forty seven, there's a
Speaker 3: lot of what about ism? So what about deep fakes?
Speaker 3: And what about algorithmic discrimination? What about misinformation? Why are
Speaker 3: you focusing on the risk of catastrophic harm? And my
Speaker 3: answer is, well, we need to adjust all of these issues.
Speaker 3: But in addition to those potential harms, we know that
Speaker 3: there is a risk of larger harms, for example, shutting
Speaker 3: down the grid, melting down the banking system and making
Speaker 3: it easier, or facilitating the creation of a chemical, biological,
Speaker 3: and nuclear weapon destroying critical infrastructure. These are all harms
Speaker 3: that you don't have to have the best imagination on
Speaker 3: the planet in order to visualize.
Speaker 2: These are worst case scenarios, and Wiener has been criticized
Speaker 2: for even imagining them.
Speaker 3: Sometimes the opponents of the bill, they try to say
Speaker 3: that anyone who focuses on AI safety must be a
Speaker 3: crazy doomer, which is not true.
Speaker 2: You don't consider yourself a crazy doomer.
Speaker 3: I am not a crazy doomer. I am very pro
Speaker 3: AI and pro innovation. I just wanted to be done responsibly.
Speaker 2: Still, the risks outlined in the legislation are mostly hypothetical,
Speaker 2: at least as of now. I asked Serene Gafari, a
Speaker 2: reporter who covers AI for Bloomberg News, about how imminent
Speaker 2: the risk of an AI fueled catastrophe really is.
Speaker 4: Okay, this is where you know, it really depends you
Speaker 4: talk to, because even the experts, even the quote unquote
Speaker 4: godfathers of AI who basically invented this field of the
Speaker 4: current type of AI models we're using today, wildly disagree. So,
Speaker 4: you know, someone like Yoshua Benjo, for example, leading computer
Speaker 4: scientists who won the Touring Award, which is like the
Speaker 4: Nobel Prize in math for deep learning, which again is
Speaker 4: like the foundation of AI that we're using today. He
Speaker 4: is a big supporter of this bill. He's come out
Speaker 4: and said, you know, these risks could be coming. He
Speaker 4: doesn't know exactly when, but he says the risk is
Speaker 4: enough that we should be trying to mitigate it. Then
Speaker 4: you have someone else who also won the Tory Award
Speaker 4: with him that year. Another big scientist, Jan Lucuhn, head
Speaker 4: of AI at Facebook. Right now a meta, He says,
Speaker 4: this is ridiculous. You know, we shouldn't be worried about
Speaker 4: these catastrophic risks. These AI tools today are nowhere near
Speaker 4: as smart as human beings, and we're all way too
Speaker 4: overly paranoid here.
Speaker 2: Overly paranoid or not. Another question is whether state legislation
Speaker 2: like this could actually prevent the worst from happening. I
Speaker 2: asked Serene how exactly SB ten forty seven approaches AI
Speaker 2: harm reduction.
Speaker 4: So what it would actually do is mandate that companies
Speaker 4: do a couple things. One is that they have a
Speaker 4: so called kill switch that they have a way to
Speaker 4: like turn off the machine if things go horribly wrong.
Speaker 4: Another is that they would have to create what's called
Speaker 4: SSPs Safety and Security Protocols. And what that would do
Speaker 4: is essentially be like a document that outlines how they're
Speaker 4: adhering to risk mitigation, which includes things like having external testing. Right,
Speaker 4: so they say you have to get a third party
Speaker 4: to go and test your systems and make sure that
Speaker 4: they're not high risk. Also just through the threat of
Speaker 4: being held legally liable, right, is kind of putting into
Speaker 4: writing and spelling out exactly how the state of California
Speaker 4: could come after you if you are not at least
Speaker 4: following a basic set of principles around minimizing risk.
Speaker 2: It's that threat of legal liability that spooked some tech
Speaker 2: companies and help fuel a fierce lobbying effort against the legislation.
Speaker 4: VC firms and tech companies have brought in the big guns.
Speaker 4: They've hired top California lobbyists andres and Horowitz has hired
Speaker 4: someone who has known Governor Newsom for a long time
Speaker 4: and is a very well known and influential lobbyist in California.
Speaker 4: Open aither companies have also hired lobbyists who are working
Speaker 4: on this bill.
Speaker 2: And already this tech industry pushback has led to tangible changes.
Speaker 2: Take the way SB ten forty seven deals with open
Speaker 2: source startups. For example, Sharin says there were concerns that
Speaker 2: even though the bill only applies to these AI giants,
Speaker 2: some of them, like Meta, make their AI models open source.
Speaker 2: That makes it easier for smaller startups to use them
Speaker 2: in their work.
Speaker 4: And the worry is that this bill, if it does
Speaker 4: apply to companies like Meta, whether it be a downstream
Speaker 4: effect where Meta says we're not going to open source
Speaker 4: our models anymore because we don't want to deal with
Speaker 4: worrying about potential liability coming from other developers misusing our technology.
Speaker 2: The latest version of the legislation tries to address those
Speaker 2: potential downstream consequences.
Speaker 3: Wiener told me, for example, making very clear that the
Speaker 3: obligation to be able to shut down an AI model
Speaker 3: only applies if you have that model in your possession.
Speaker 3: So if you open source a model and someone significantly
Speaker 3: changes the model, it's no longer your responsibility.
Speaker 2: There's another overarching critique that tech companies like OpenAI have
Speaker 2: made about SB ten forty seven, though, one that's harder
Speaker 2: to solve with an amendment. It's the fact that the
Speaker 2: legislation comes from the state of California, not Congress.
Speaker 4: A lot of critics of the bill say we should
Speaker 4: get federal legislation. They don't want this patchwork.
Speaker 2: A patchwork meaning different AI regulations set state by state,
Speaker 2: potentially overlapping or contradicting one another, instead of one national
Speaker 2: framework all tech companies have to follow. But Senator Wiener
Speaker 2: says part of the reason California is acting is because
Speaker 2: Congress hasn't.
Speaker 3: People would say, hey, this should be handled by Congress.
Speaker 3: It should be handled at the federal level. It's better
Speaker 3: to have a consistent national standard. And my answer is absolutely,
Speaker 3: I would love to be able to close up shop
Speaker 3: because Congress stepped in and passed a strong AI safety law.
Speaker 3: That would be music to my ears.
Speaker 2: Not all California politicians are on board with this trailblazing
Speaker 2: policy agenda, including Senator Nancy Pelosi.
Speaker 3: This bill is well intentioned but ill informed.
Speaker 2: But prominent supporters have also emerged.
Speaker 1: Elon Musk now coming out in support of a California
Speaker 1: bill to regulate the development of AI, a stark contrast
Speaker 1: to other big names including open Ai and Meta, both
Speaker 1: of which have come out against the bill.
Speaker 2: You're Sharen wasn't entirely surprised by Musk's take.
Speaker 4: Musk has been someone who for a long time, this
Speaker 4: has been his big fear. He has worried about the
Speaker 4: long term, potentially catastrophic effects of AI. That's why he
Speaker 4: helped fund open Ai to be a counter force to Google,
Speaker 4: because he thought that AI is so important that more
Speaker 4: people should be in control of it. Now, if you're cynical,
Speaker 4: I think you could look at it and say, well,
Speaker 4: Elon has more incentive to be support of this bill
Speaker 4: because his AI company isn't number one Open AI or
Speaker 4: anthropic or these other companies are perceived as being more ahead.
Speaker 2: As for Wiener, he told me he'll take the support
Speaker 2: where he can get it. Do you think his tweet
Speaker 2: helps you?
Speaker 3: I think that his tweet really shook things up because
Speaker 3: there are you know, folks who I know, we're very
Speaker 3: critical of the bill, and once they saw the tweet
Speaker 3: and made them think about it more.
Speaker 2: The drama within the AI community over this bill hasn't
Speaker 2: stopped state legislators. They passed SB ten forty seven in
Speaker 2: the last week in August. Now it's fate rests in
Speaker 2: California Governor Gavin Newsom's hands, and if he signs it
Speaker 2: into law, the reapercussions could be felt far beyond California.
Speaker 2: Coming up will these AI regulations developed in the industry's
Speaker 2: backyard have global reach. The Golden State is often ahead
Speaker 2: of the curve when it comes to technology, both building
Speaker 2: it and regulating it, and Scharene Gafari, a Bloomberg Tech reporter, says,
Speaker 2: what happens in California very rarely stays in California.
Speaker 4: It could very well be something that turns into a
Speaker 4: template for other states. This has happened in the past
Speaker 4: where California said some kind of tech legislation and then
Speaker 4: other people followed through. We are already seeing other states
Speaker 4: like Colorado looking at AI legislation, and I do think
Speaker 4: we could see more.
Speaker 2: And it's not just other states that could be taking
Speaker 2: notes on what California is doing.
Speaker 4: It could even at one day become a template for
Speaker 4: federal or even international kind of regulations.
Speaker 2: Sharene says SB ten forty seven already hewes closely to
Speaker 2: international agreements that have come out of recent global safety
Speaker 2: summits in the EU and in Korea. And though the
Speaker 2: US has no national AI regulation in place, Open AI
Speaker 2: and Anthropic recently voluntarily agreed to share their AI models
Speaker 2: before they're released with the US Commerce Department's new AI
Speaker 2: Safety Institute.
Speaker 4: So proponents of the bill say, look, you're already doing
Speaker 4: some of this work. Why are you so against it?
Speaker 4: But critics of the bill say, well, these voluntary agreements
Speaker 4: are working. We're already seeing companies voluntarily come on board,
Speaker 4: So why do we need to have this hammer.
Speaker 2: It's a common tension between businesses and policymakers. They both
Speaker 2: think they should be the ones holding the hammer. But
Speaker 2: for the AI industry, Sharin says, now is a particularly
Speaker 2: tense time to be having this debate.
Speaker 4: People think that there is a race to the finish
Speaker 4: line to reach superintelligence right now. It's this exciting moment
Speaker 4: where we're seeing this kind of Cambrian explosion and this
Speaker 4: rapid advancement in AI, and they worry that this is
Speaker 4: going to just stifle that. And I think that's kind
Speaker 4: of a classic tech versus government fear.
Speaker 2: To AI regulators slowing down that blowsive growth just long
Speaker 2: enough to put up some guardrails might not be the
Speaker 2: worst thing. And while Serene says we likely won't see
Speaker 2: any changes to the AI tools we use in our
Speaker 2: everyday lives just yet, the biggest change could be in
Speaker 2: how developers go about rolling out new technology.
Speaker 4: They may have to take more of a beat right
Speaker 4: before putting out certain models and make sure that, Okay,
Speaker 4: are we in compliance with these guidelines and if not,
Speaker 4: are we taking the risk of being sued. So it
Speaker 4: would just kind of create some more administrative overhead for
Speaker 4: these companies to make sure that they are following these.
Speaker 2: Rules, and for California State Senator Scott Wiener that administrative
Speaker 2: overhead is a small price to pay to ensure future safety.
Speaker 3: This is not about being an AI doomer. It's about
Speaker 3: being a responsible grown up, where we say, let's do
Speaker 3: all these good things and let's also protect against significant.
Speaker 2: Whether you agree with him or not, Charine says that
Speaker 2: the debate over the bill has already informed what questions
Speaker 2: we all might need to be asking about AI development
Speaker 2: and AI regulation in the coming years.
Speaker 4: I think what this bill shows is that it's very
Speaker 4: easy to say, oh, yeah, there could be some kind
Speaker 4: of catastrophic risks from AI. We're all in agreement on that,
Speaker 4: but it's very hard to actually agree. Okay, now, what
Speaker 4: is the percentage chance you think that risk is going
Speaker 4: to come, How urgent is it, What is the best
Speaker 4: way to manage it? Once you get into the details,
Speaker 4: all of a sudden, managing AI becomes really really complicated,
Speaker 4: especially because it's so new, it's so technologically difficult. So
Speaker 4: I think it's probably one of the hardest kinds of
Speaker 4: technology in some ways to regulate.
Speaker 2: This is the big take from Bloomberg News I'm Sarah Holder.
Speaker 2: This episode was produced by Adrian AA Tapia, who also
Speaker 2: fact checked this episode. It was mixed by Blake. Our
Speaker 2: senior producers are Naomi Shaven and Kim Gittelson, who also
Speaker 2: edited this episode. Our senior editor is Elizabeth Ponso. Nicole
Speaker 2: Bumster Borr is our executive producer. Sage Bauman is Bloomberg's
Speaker 2: head of podcasts. If you liked the episode, make sure
Speaker 2: to subscribe and review The Big Take wherever you listen
Speaker 2: to podcasts. It helps people find the show. Thank you
Speaker 2: so much for listening. We'll be back on Monday.