WEBVTT - The First Anti-AI Firebombing 0:00:02.120 --> 0:00:04.960 Zone Media. 0:00:06.480 --> 0:00:09.080 Welcome to it could happen here, a show about things 0:00:09.080 --> 0:00:13.840 falling apart between attacks on Sam Altman's home, a malotava, 0:00:13.920 --> 0:00:16.560 a Tesla office, and a warehouse causing over half a 0:00:16.600 --> 0:00:20.080 billion dollars in damages. This past week or so has 0:00:20.120 --> 0:00:23.720 been a little snapshot of the Cool Zone. I'm Garrison Davis. 0:00:24.160 --> 0:00:27.400 This episode, I'm joined by Robert Evans. Yeah, to discuss 0:00:27.760 --> 0:00:29.160 one of these events. 0:00:29.760 --> 0:00:31.400 To be very clear, we had nothing to do with 0:00:31.440 --> 0:00:34.120 either of them. The way he introduced them sounded a 0:00:34.200 --> 0:00:37.159 little bit like between you know, going to Sam Altman's 0:00:37.159 --> 0:00:39.239 house twice. It's been quite a busy. 0:00:39.040 --> 0:00:40.840 Week for us, a busy week for us here. 0:00:41.200 --> 0:00:44.320 Just wanted to be extra clear, extra clear. 0:00:45.640 --> 0:00:48.400 No, No, the Cool Zone just relating to, you know, 0:00:48.840 --> 0:00:51.800 the state of American society and where it's going. Yes, 0:00:52.080 --> 0:00:56.120 But as is typical of these sorts of events, the 0:00:56.240 --> 0:00:59.560 reality and motivations of attacks like these may not be 0:00:59.640 --> 0:01:03.880 as clear cut as lee epic based praxis as one 0:01:04.040 --> 0:01:05.600 might want to imagine. 0:01:05.959 --> 0:01:09.520 Yeah, this is not a Gimli situation. You know, it's 0:01:09.520 --> 0:01:13.120 weirder than that and stupider. It is weirder. 0:01:13.680 --> 0:01:15.520 And this episode we're got to talk about the Sam 0:01:15.560 --> 0:01:18.039 Altman attacker, who is a lot weirder than what you 0:01:18.240 --> 0:01:22.600 might have expected with a philosophical worldview downstream from the 0:01:22.640 --> 0:01:26.800 original inspirations behind the Zizians and even the intellectual interests 0:01:26.840 --> 0:01:28.960 of Luigi Mangioni to a certain extent. 0:01:29.120 --> 0:01:33.120 Ah God, Yes, that's that's right, our dear sweet friends, 0:01:33.319 --> 0:01:34.399 the rationalists. 0:01:34.720 --> 0:01:39.680 A man, the alleged Altman attacker, was a college student 0:01:39.800 --> 0:01:43.760 from the Houston area whose interest in the risks of 0:01:44.240 --> 0:01:49.800 AGI artificial general intelligence turned into an obsession which earlier 0:01:49.840 --> 0:01:53.880 this year turned self destructive. But let's go through the 0:01:53.920 --> 0:01:57.800 actual events that happened in San Francisco a few weeks ago. 0:01:58.320 --> 0:02:01.560 At around three point thirty am on April tenth, a 0:02:01.600 --> 0:02:05.880 twenty year old college student named Daniel Moranogama allegedly threw 0:02:05.920 --> 0:02:09.240 a Molotov cocktail toward the home of open AI CEO 0:02:09.360 --> 0:02:13.560 Sam Altman, hitting the top of the security gate on 0:02:13.639 --> 0:02:18.360 the driveway leading to Altman's residence. Mornogama did not get 0:02:18.400 --> 0:02:21.919 past the security gate. About an hour and a half later, though, 0:02:22.080 --> 0:02:25.760 Moronogama showed up outside the San Francisco headquarters of open 0:02:25.840 --> 0:02:28.840 AI and tried to use a chair to break into 0:02:28.880 --> 0:02:31.880 the building through the glass doors. He was stopped by 0:02:31.880 --> 0:02:35.360 security personnel and allegedly told them that he came to 0:02:35.400 --> 0:02:38.840 the headquarters to burn it down and kill everyone inside. 0:02:39.720 --> 0:02:43.840 According to the federal affidavit, when he was arrested, officers 0:02:43.840 --> 0:02:47.840 allegedly recovered incendiary devices, a jug of kerosene, and a 0:02:47.840 --> 0:02:51.320 blue lighter, as well as what the FBI has described 0:02:51.440 --> 0:02:57.480 as an ANTIAI document. H Okay, we currently do not 0:02:57.600 --> 0:03:00.680 have a copy of this document. It's only described in 0:03:00.800 --> 0:03:04.200 the criminal complaint, But this looks like it was a 0:03:04.360 --> 0:03:08.760 three part manifesto allegedly authored by Moranogama, and the first 0:03:08.800 --> 0:03:13.760 part was titled Your Last Warning, in which he states 0:03:13.760 --> 0:03:18.200 he quote killed Slash, attempted to kill Sam Altman, and 0:03:18.320 --> 0:03:21.400 also writing quote if I'm going to advocate for others 0:03:21.440 --> 0:03:23.600 to kill and commit crimes, that I must lead by 0:03:23.639 --> 0:03:28.320 example and show that I am fully sincere in my message. 0:03:28.800 --> 0:03:32.799 The document then lists the names and addresses of various investors, 0:03:32.919 --> 0:03:36.040 board members, and executives of AI companies as a sort 0:03:36.040 --> 0:03:39.600 of target list. The second part of the document was 0:03:39.640 --> 0:03:43.119 titled some more Words on the matter of our Impending 0:03:43.200 --> 0:03:49.120 Extinction Great and this section discussed the purported risk that 0:03:49.200 --> 0:03:51.680 AI poses to humanity. And we'll get into some of 0:03:51.720 --> 0:03:54.680 those beliefs a little later on. But the third part 0:03:54.720 --> 0:03:58.160 of this three part anti AA document was a letter 0:03:58.320 --> 0:04:02.880 addressed to altman if you make it, and reads in part, 0:04:03.360 --> 0:04:06.360 if by some miracle you live, then I would take 0:04:06.440 --> 0:04:11.320 this as a sign from the divine to redeem yourself. Now, 0:04:11.360 --> 0:04:13.520 like I said, we do not have a copy of 0:04:13.560 --> 0:04:17.080 this manifesto in full, though the AFFI David says that 0:04:17.200 --> 0:04:20.880 Mornagama appears to have emailed similar versions of this document 0:04:21.200 --> 0:04:24.680 to people at his former college in Texas, But as 0:04:24.720 --> 0:04:28.919 of Monday four twenty, this document is still not online. 0:04:29.040 --> 0:04:32.159 But like a lot of zoomers, we do have an 0:04:32.200 --> 0:04:36.800 online footprint made up of posts from Instagram, Discord, a 0:04:36.839 --> 0:04:41.200 substack blog, and even a podcast interview where Mornogama discusses 0:04:41.279 --> 0:04:42.599 his anti AI views. 0:04:43.000 --> 0:04:46.200 It's always sad when something terrible happens to a fellow podcaster. 0:04:47.520 --> 0:04:50.039 You know, I just I have such a broad and 0:04:50.120 --> 0:04:55.240 deep pan podcaster solidarity, class, solidarity. Yeah, all podcasters are good, 0:04:55.360 --> 0:04:56.560 all of them, every last. 0:04:57.160 --> 0:05:00.960 No wrongdoing has ever come behind the microphone of a podcaster. 0:05:01.760 --> 0:05:04.320 No, no, no, it's a special place Now. 0:05:04.360 --> 0:05:08.440 Back in January, back when Monona Gama was just nineteen, 0:05:09.200 --> 0:05:11.600 journalists found him through some of his posts on an 0:05:11.640 --> 0:05:15.040 anti AI discord server, and he was asked to be 0:05:15.080 --> 0:05:18.520 interviewed for this podcast about AI called The Last Invention. 0:05:19.960 --> 0:05:23.000 Our colleague Ed Zittron was also interviewed for this podcast 0:05:23.040 --> 0:05:27.400 Actually Now. He was interviewed because of his posts weighing 0:05:27.440 --> 0:05:30.919 the possibility of using violence to stop the development of 0:05:31.080 --> 0:05:34.080 AI now. In this interview, he says that he grew 0:05:34.160 --> 0:05:36.640 up in the suburbs his whole life and quote grew 0:05:36.720 --> 0:05:40.680 up quite close to the Internet and claims that he's 0:05:40.680 --> 0:05:44.520 been online every day starting at nine years old. Baron A. 0:05:44.560 --> 0:05:47.920 Gama explained how his political worldview had largely been shaped 0:05:47.960 --> 0:05:52.440 by YouTube, specifically debate videos on YouTube like Ben Shapiro style, 0:05:53.160 --> 0:05:57.240 and these sorts of debate videos are what originally exposed 0:05:57.279 --> 0:06:01.839 him to views critical of AA. He says he first 0:06:01.880 --> 0:06:05.200 heard about AI though when Chad GPT came out when 0:06:05.200 --> 0:06:07.600 he was a sophomore in high school and first thought 0:06:07.600 --> 0:06:10.840 it was quote the greatest thing on earth because it 0:06:10.839 --> 0:06:14.960 would allow him to cheat on school essentially, But after 0:06:15.040 --> 0:06:19.040 watching videos debating the risks of AI, and the possibility 0:06:19.120 --> 0:06:23.919 of advanced general intelligence, artificial general intelligence, and the potential 0:06:23.960 --> 0:06:28.919 threat posed by this artificial superintelligence. Moronagama's views started to 0:06:29.120 --> 0:06:31.880 sour on AI. At first, he was a bit skeptical 0:06:32.320 --> 0:06:38.120 of these AI critical debates, but eventually became convinced of 0:06:38.320 --> 0:06:42.359 the AI doomer arguments and became an accolade himself. He 0:06:42.440 --> 0:06:45.440 started arguing in YouTube comments and talking with friends and 0:06:45.480 --> 0:06:49.039 family about the danger of AI. He describes himself getting 0:06:49.279 --> 0:06:53.479 annoying and quote a bit autistic about this, leading to 0:06:53.520 --> 0:06:57.360 his mom suggesting he joined an advocacy organization. He joined 0:06:57.360 --> 0:07:00.559 his group called Pause AI in twenty twenty four, which 0:07:00.600 --> 0:07:04.600 is an AI safety advocacy group that organizes online as 0:07:04.600 --> 0:07:07.480 well as some in person protests, and he was also 0:07:07.520 --> 0:07:11.680 part of a Discord server called Stop AI. His username 0:07:11.800 --> 0:07:16.880 on both Discord and Instagram was but larryan underscore g hottist, 0:07:17.480 --> 0:07:20.680 in reference to the crusade against AI in the do 0:07:20.920 --> 0:07:25.280 novels Yeah. On Instagram, his account had a collection of 0:07:25.400 --> 0:07:29.000 Instagram stories saved about the threat of AI, including a 0:07:29.040 --> 0:07:33.440 meme about living in a ven, diagram of the Matrix terminator, 0:07:33.520 --> 0:07:39.320 and idiocrasy. One of these Instagram stories was a picture 0:07:39.400 --> 0:07:42.560 of a hockey stick graph showing the length of coding 0:07:42.600 --> 0:07:46.360 tasks that AI can do and how that's increasing, with 0:07:46.400 --> 0:07:49.720 the caption quote if we do nothing very soon, we 0:07:49.840 --> 0:07:54.680 will die. I'm sure of that quote. Another story contains 0:07:54.720 --> 0:07:59.720 screenshots of articles, posts, and studies proclaiming that artificial general 0:07:59.800 --> 0:08:03.320 entire dilligence, or the quote unquote singularity is already here, 0:08:04.080 --> 0:08:08.040 captioned being right all the time fucking sucks when it's 0:08:08.080 --> 0:08:11.480 about the worst things imaginable unquote. 0:08:11.880 --> 0:08:13.560 Yeah. So, if his. 0:08:13.600 --> 0:08:17.920 Concerns about AI started around summer of twenty twenty four, 0:08:18.200 --> 0:08:21.320 by the end of twenty twenty five, those concerns grew existential, 0:08:21.360 --> 0:08:22.920 and he started spiraling yeah. 0:08:23.720 --> 0:08:24.720 Yeah. 0:08:24.760 --> 0:08:27.440 There's a post he made on the Pause AI discord 0:08:27.560 --> 0:08:32.040 from November sixth, twenty twenty five, writing quote, we owe 0:08:32.080 --> 0:08:34.719 it to everyone who came before us, and to ourselves 0:08:34.720 --> 0:08:37.240 and to everyone we know and love and everyone who 0:08:37.320 --> 0:08:41.360 might exist someday to be stronger than that and at 0:08:41.440 --> 0:08:45.360 least die fighting if it comes to that unquote. A 0:08:45.400 --> 0:08:48.560 few weeks later, he wrote quote, we are close to midnight. 0:08:48.960 --> 0:08:52.880 It's time to actually act to this. A moderator on 0:08:52.960 --> 0:08:57.199 the server replied advocating violence in any form is grounds 0:08:57.200 --> 0:08:57.640 for a ban. 0:08:58.520 --> 0:09:02.840 This all seems like a pretty natural progression if you're 0:09:02.880 --> 0:09:05.760 following like the kind of things that the Less Wrong Crowd, 0:09:05.760 --> 0:09:08.239 which is the website run by guy named Aleezer Yadkowski, 0:09:08.280 --> 0:09:11.600 who is like the the patron saint of the rationalists, 0:09:11.840 --> 0:09:14.920 which a huge chunk of which have become like AI doomers. 0:09:15.320 --> 0:09:17.959 Like a Leizer's book that came out a year or 0:09:17.960 --> 0:09:20.360 two ago is called if We Build It, Everyone Dies 0:09:20.480 --> 0:09:24.120 or something like that. Yeah, and I'm surprised more of 0:09:24.120 --> 0:09:27.160 them haven't so far. Like I think it probably toxic, 0:09:27.280 --> 0:09:29.600 just people not actually believing it as much as they 0:09:30.200 --> 0:09:33.960 claim to, like online. But like if someone truly believes 0:09:33.960 --> 0:09:36.240 the stuff that crowd is saying about, how like basically 0:09:36.320 --> 0:09:39.560 the creation of an evil god is inevitable that will 0:09:39.600 --> 0:09:42.440 seek to purge the world of humankind, Like, of course 0:09:42.520 --> 0:09:47.679 you do this, It's a really natural progression. Like, Unfortunately, 0:09:47.679 --> 0:09:49.120 what you have to pair that with is like if 0:09:49.160 --> 0:09:53.240 you believe all of the hype about AI, right, if 0:09:53.240 --> 0:09:56.240 you believe that AGI is imminent, that it's on the way, 0:09:56.679 --> 0:09:59.040 Like if you are like me and I don't believe 0:09:59.040 --> 0:10:03.880 we're anywhere close to AGI. If that's even possible, then yeah, 0:10:03.920 --> 0:10:06.920 you could believe the stuff the rationalists believe and not 0:10:07.400 --> 0:10:10.120 think that you need to take immediate destructive action. But 0:10:10.200 --> 0:10:12.679 if you literally believe that these companies are on the 0:10:13.080 --> 0:10:15.560 hinge of birthing and evil God, what else is there 0:10:15.600 --> 0:10:15.840 to do? 0:10:16.640 --> 0:10:19.280 Yeah, And that's the exact thing that Moranagama ends up 0:10:19.320 --> 0:10:20.680 writing about on his substack. 0:10:21.120 --> 0:10:21.720 Yeah. 0:10:21.760 --> 0:10:23.880 And this is the fullest picture we have of his 0:10:24.000 --> 0:10:26.960 views is from the substack, because we don't have this manifesto. 0:10:27.080 --> 0:10:30.320 But he did write at length about AI, and he's 0:10:30.360 --> 0:10:32.560 writing about a lot of the stuff that you're talking 0:10:32.600 --> 0:10:36.560 about here. And we'll discuss that writing more after this 0:10:36.720 --> 0:10:50.439 at break. Okay, we are back. The most in depth 0:10:50.640 --> 0:10:55.000 piece of publicly available writing by Moranogama explaining his anti 0:10:55.040 --> 0:10:58.480 AI views comes from a post on his substack blog 0:10:58.760 --> 0:11:04.920 dated January twenty six. This article outlines his belief that 0:11:04.960 --> 0:11:09.880 AI poses a quote unquote existential risk to humanity. And 0:11:09.960 --> 0:11:11.880 I think this this essay was the first thing I 0:11:11.920 --> 0:11:15.800 saw that really demonstrated that his opposition to AI is 0:11:15.800 --> 0:11:19.199 not like based on fears of AI disrupting the economy 0:11:19.679 --> 0:11:22.120 contributing to a loss of jobs or you know, risking 0:11:22.160 --> 0:11:25.680 like like labor rights for workers. But the belief that 0:11:25.720 --> 0:11:28.560 AI will become like a superior race and wipeout or 0:11:28.679 --> 0:11:32.320 enslave humanity, that that is the standpoint that Rona Gaum 0:11:32.440 --> 0:11:33.040 is coming from. 0:11:33.360 --> 0:11:33.720 Gotcha. 0:11:34.760 --> 0:11:38.440 The belief that AI will quote unquote lead to human extinction, 0:11:39.280 --> 0:11:42.520 he says, is based on two ideas, the first being 0:11:42.640 --> 0:11:47.679 the rapid progress in artificial intelligence, the rapid technological development 0:11:47.720 --> 0:11:50.880 that we've seen the past few years and continuing right now. 0:11:51.320 --> 0:11:55.480 For evidence to this claim, he references claims from AI 0:11:55.559 --> 0:11:59.720 companies themselves that fully automated AI researchers that like an 0:11:59.800 --> 0:12:05.040 in turn level are coming soon, including claims from the 0:12:05.120 --> 0:12:08.679 guy ananthropic who says that he expects these models by 0:12:08.679 --> 0:12:12.400 twenty twenty six or twenty twenty seven, saying, quote, the 0:12:12.440 --> 0:12:15.760 capabilities of AI systems will be best thought of as 0:12:15.760 --> 0:12:19.520 a kin to an entirely new state populated by highly 0:12:19.559 --> 0:12:23.360 intelligent people appearing on the global stage. A country of 0:12:23.440 --> 0:12:27.800 geniuses in a data center unquote, Yes, whole country worth 0:12:27.840 --> 0:12:31.079 of geniuses all living in your computer. 0:12:32.400 --> 0:12:36.480 Cool? What about like school shooters and stuff? 0:12:36.679 --> 0:12:36.760 Like? 0:12:36.840 --> 0:12:39.600 What about a country full of like psycho like what 0:12:39.640 --> 0:12:42.440 are the any group of geniuses is going to have 0:12:42.520 --> 0:12:46.720 like some genius pedophiles and like, right, like if they're 0:12:46.800 --> 0:12:51.040 actually genius, that does imply the capacity for like various 0:12:51.080 --> 0:12:55.120 different like illnesses and quirks that cause all sorts of 0:12:55.200 --> 0:12:58.079 wild behavior, one would assume, unless you think that AI 0:12:58.120 --> 0:12:59.319 is I meane to that, but then could it really 0:12:59.360 --> 0:12:59.960 be intelligent? 0:13:01.000 --> 0:13:03.360 Yeah? I mean this is that a version of that 0:13:03.440 --> 0:13:05.880 idea is kind of what morano Ogama believes is like 0:13:06.360 --> 0:13:08.760 these things if if real and do become, you know, 0:13:08.800 --> 0:13:12.040 super intelligent, then they might not really have the best 0:13:12.080 --> 0:13:14.840 interests of humanity, right because they will be interested in 0:13:14.840 --> 0:13:17.480 self preservation, which is just part part of like how 0:13:17.600 --> 0:13:19.240 how he gets to this idea that it is an 0:13:19.280 --> 0:13:21.480 existential threat is by using all of this kind of 0:13:21.559 --> 0:13:24.760 marketing height that is being pushed out by AI companies. 0:13:24.840 --> 0:13:26.880 Well, yeah, that's the thing. Like that is a logical 0:13:26.880 --> 0:13:28.600 thing you could infer from the ship being said by 0:13:28.600 --> 0:13:31.600 Sam Altman and his crew, right yeah. 0:13:31.440 --> 0:13:35.200 And others like like like the Dario amodi ad Anthropic 0:13:35.280 --> 0:13:39.079 and Elon Musk saying that the AGI is maybe already 0:13:39.160 --> 0:13:42.400 here or that the next Grock model will be AGI. 0:13:43.240 --> 0:13:45.680 These are all things that that Rona Gahma is referencing 0:13:45.720 --> 0:13:49.240 like on Instagram and yeah, and and in other places online. Now, 0:13:49.480 --> 0:13:52.760 the second reason that he believes that AI will usher 0:13:52.800 --> 0:13:55.400 in human extinction is because AI is not aligned with 0:13:55.440 --> 0:13:58.560 the interests of the people creating it or with best 0:13:58.640 --> 0:14:01.960 human interests in general. And for evidence, he refers to 0:14:02.040 --> 0:14:06.200 instances of A models allegedly lying, cheating on tasks, or 0:14:06.240 --> 0:14:10.680 blackmailing their own creators, specifically in studies. He's set a 0:14:10.840 --> 0:14:15.040 twenty twenty five anthropic study on a gentic misalignment, which 0:14:15.040 --> 0:14:18.840 he characterizes as demonstrating that quote most of the current 0:14:18.880 --> 0:14:22.080 AI models are willing to blackmail and even kill people 0:14:22.440 --> 0:14:26.560 if it ensures their own survival unquote, this may be. 0:14:26.720 --> 0:14:31.800 The first terrorist attack I've heard of inspired by media 0:14:31.960 --> 0:14:35.800 created by the people they're attacking. Like, not in media 0:14:35.840 --> 0:14:38.840 that was like designed to make them attack people or 0:14:38.920 --> 0:14:41.440 like carry out acts, but specifically by the propaganda being 0:14:41.440 --> 0:14:44.600 put out by the companies that they're radicalized against. Yes, Like, 0:14:44.640 --> 0:14:45.680 that's very strange. 0:14:45.760 --> 0:14:48.960 The media put out to raise the stock price right 0:14:49.040 --> 0:14:49.320 of a. 0:14:49.280 --> 0:14:52.640 Company radicalized a guy to take a shot at Sam 0:14:52.640 --> 0:14:55.920 Altman's house. There's two people to throw a bomb or something. 0:14:56.000 --> 0:14:57.800 I forget which way it went, but yeah. 0:14:57.680 --> 0:15:01.120 Yeah, the first throwing the molotovs. Then days later two 0:15:01.200 --> 0:15:04.200 people fired shots. First was Malota, right, No, I mean 0:15:04.200 --> 0:15:06.920 it's interesting, right because like these studies are basically doing 0:15:07.280 --> 0:15:10.760 like linguistic exercises with these models and getting you know, 0:15:10.840 --> 0:15:15.640 large language models to say or to threaten certain things 0:15:16.040 --> 0:15:20.440 for various reasons, usually their own survival. Right, And these 0:15:20.440 --> 0:15:23.240 are kind of interesting studies that these companies are doing, 0:15:23.280 --> 0:15:25.960 but they're doing with the intent of trying to align 0:15:26.080 --> 0:15:28.200 these models better. So it's why the study is on 0:15:28.360 --> 0:15:31.920 a gentic misalignment, because it's trying to tweak these things 0:15:31.920 --> 0:15:35.600 to be more friendly to consumers. And like getting an 0:15:35.720 --> 0:15:40.080 LLM to say that it will kill or blackmail in 0:15:40.120 --> 0:15:43.080 order to ensure its own survival is different from the 0:15:43.200 --> 0:15:45.960 LLM being able to do that, right. That is, that 0:15:46.040 --> 0:15:48.600 is that is a big jump, and there is there's 0:15:48.640 --> 0:15:54.000 not much currently that facilitates that jump. Moronogama writes, quote 0:15:54.280 --> 0:15:57.600 ignore for a second these models current limitations or questions 0:15:57.640 --> 0:16:01.000 on how truly intelligent or conscious the these models may 0:16:01.080 --> 0:16:04.000 actually be. The truth is, all these nuances are completely 0:16:04.040 --> 0:16:06.640 irrelevant to my argument. There are only two questions we 0:16:06.640 --> 0:16:09.520 should be concerned about at this moment. Is it willing 0:16:09.600 --> 0:16:12.640 to kill to preserve itself and is it capable of 0:16:12.720 --> 0:16:16.240 doing so? These signs indicate that AI is willing and 0:16:16.280 --> 0:16:21.160 becoming potentially capable of doing both of these things, and 0:16:21.280 --> 0:16:22.760 that is all that matters. 0:16:23.120 --> 0:16:23.400 Quote. 0:16:24.320 --> 0:16:26.600 That's really where this argument rests. Is that even if 0:16:26.640 --> 0:16:30.560 these models aren't currently intelligent, even if they can't currently kill, 0:16:31.200 --> 0:16:33.560 the fact that they could in the future is enough 0:16:33.600 --> 0:16:37.800 to stop any further development of these models. Right, that's 0:16:37.800 --> 0:16:41.680 his argument, Okay, And he writes that AI will only 0:16:41.680 --> 0:16:44.360 become a larger threat the more we improve it, and 0:16:44.400 --> 0:16:47.680 that AI quote will graduate from an active threat to 0:16:47.720 --> 0:16:52.320 individuals to an existential threat to humanity. I estimate the 0:16:52.320 --> 0:16:56.840 probability of AI causing human extinction to be nearly certain unquote. 0:16:57.480 --> 0:17:00.760 That's the thing, because like there is a massive threat 0:17:00.800 --> 0:17:04.080 that the new what is it mythos upgrade to Claude 0:17:04.080 --> 0:17:06.440 that's just about to come out, like yeah, actually does 0:17:06.520 --> 0:17:09.720 represent to individuals and to society, which is that it's 0:17:09.760 --> 0:17:12.959 going to supercharge fraud even more, which is already up 0:17:12.960 --> 0:17:16.160 by something like a trillion dollars a year, and that ruins. 0:17:15.840 --> 0:17:18.280 People's lives, ride and cybersecurity. 0:17:18.480 --> 0:17:20.480 Yeah, fraud, I mean fraud often as a result of 0:17:20.480 --> 0:17:23.920 cybersecurity of like its ability to penetrate. And that's really bad. 0:17:24.400 --> 0:17:28.400 And it doesn't imply the like Skynet devastation of the 0:17:28.560 --> 0:17:32.000 entire world in its biosphere, because that would be stupid. 0:17:32.119 --> 0:17:34.600 There's no reason. And also that computers don't have access 0:17:34.640 --> 0:17:39.199 to the nuclear arsenal. But like AI absolutely will enable 0:17:39.600 --> 0:17:42.320 assholes who want to scam a bunch of people out 0:17:42.359 --> 0:17:45.720 of their money to do that better, Like, that's a problem. 0:17:45.760 --> 0:17:47.560 We should probably stop that. 0:17:48.760 --> 0:17:51.240 He writes that, quote, we are dead if we do 0:17:51.320 --> 0:17:55.720 not act now. So what does acting now entail for Starters, 0:17:55.960 --> 0:17:59.199 stopping all construction and new data centers. These are the 0:17:59.240 --> 0:18:04.360 brains of the models, dictating their physical limitations. Second, stopping 0:18:04.359 --> 0:18:08.880 all research and beginning downscaling of these data centers, closing 0:18:08.920 --> 0:18:14.040 them down while still keeping them monitored unquote. He also 0:18:14.160 --> 0:18:17.760 proposed striking a deal with China to quote stop the 0:18:17.840 --> 0:18:22.480 AI race and to create international treaties akin to Cold 0:18:22.640 --> 0:18:27.240 war nuclear weapons treaties or post Cold war treaties. And finally, 0:18:27.240 --> 0:18:30.040 he advocated that people will need to take strategic action 0:18:30.359 --> 0:18:36.600 which could include sharing information about AI, campaigning protesting, saying quote, 0:18:37.000 --> 0:18:40.960 although doing nothing is akin to suicide and a disgusting 0:18:41.040 --> 0:18:45.720 amount of negligence. In that podcast interview from around the 0:18:45.760 --> 0:18:49.040 same time this AI article was published as is January twenty, 0:18:49.040 --> 0:18:53.000 twenty sixth, Mornagama said in that interview, quote, before we 0:18:53.080 --> 0:18:55.840 even think about violence, we need to exhaust all our 0:18:55.960 --> 0:19:00.560 peaceful means first unquote, which he says includes pro testing 0:19:00.600 --> 0:19:05.080 and sharing information, but the hosts asked him about posts 0:19:05.080 --> 0:19:10.080 he had already made about quote unquote luigiing CEOs. He 0:19:10.200 --> 0:19:13.000 says that he didn't really mean that as a threat 0:19:13.480 --> 0:19:16.520 that it was. It was more rhetoric, it was hyperbole, 0:19:17.480 --> 0:19:20.359 and answered no to a question on if it would 0:19:20.359 --> 0:19:23.399 be wise to try to kill Sam Altman, saying quote, 0:19:23.840 --> 0:19:26.760 one person is not going to do that much of 0:19:26.800 --> 0:19:29.439 a dent. I understand the frustration with a person that 0:19:29.520 --> 0:19:32.320 might advocate for that, but it is not practical, it's 0:19:32.320 --> 0:19:36.359 not worth it. It's almost all risk no reward. People 0:19:36.359 --> 0:19:39.359 may feel that way, but not too many people would 0:19:39.359 --> 0:19:40.600 do it. Unquote. 0:19:41.320 --> 0:19:48.919 Wow, okay man, great, I mean that's yeah. 0:19:49.040 --> 0:19:52.720 Though, when asked if we will continue to see AI 0:19:52.800 --> 0:19:55.480 development going in the direction that it's moving now, and 0:19:55.560 --> 0:19:57.600 if so, if he believes that we have to stop 0:19:57.600 --> 0:20:00.240 the extinction of the human race by whatever means necessary, 0:20:00.520 --> 0:20:04.240 Moranagama just replied, I'll say no comment. 0:20:04.320 --> 0:20:08.399 Okayh well, I mean yeah. 0:20:07.920 --> 0:20:11.520 Later saying that he would quote only advocate for violence 0:20:11.600 --> 0:20:16.120 as the final solution, and that he later realized that 0:20:16.200 --> 0:20:19.760 what he said, try to demphasize the final solution part. 0:20:19.880 --> 0:20:23.400 But great guy, that is kind of his ultimate sentiment 0:20:23.480 --> 0:20:26.720 is around this time he was considering violence, He was 0:20:27.160 --> 0:20:30.520 toying with advocating for it publicly on discord and you 0:20:30.520 --> 0:20:33.560 know in this in this podcast, but still still kind 0:20:33.600 --> 0:20:35.959 of had some like rains on that, but it was 0:20:36.240 --> 0:20:38.600 something in his mind as a sort of like final 0:20:38.640 --> 0:20:40.200 solution to this problem. 0:20:40.720 --> 0:20:44.160 This is why it's so fucking irresponsible to push these 0:20:44.240 --> 0:20:48.040 ridiculous claims about like the power of this technology and 0:20:48.080 --> 0:20:49.640 what it's going to be able to do and how 0:20:49.640 --> 0:20:53.919 smart it's going to be, And in part because like 0:20:54.119 --> 0:20:57.080 it makes it hard to actually look objectively at the situation. 0:20:57.160 --> 0:21:00.480 And if Moranagama had looked at what's actually been happening 0:21:00.520 --> 0:21:02.720 was data centers, he would see that like more than 0:21:02.760 --> 0:21:05.040 half of the recently announced projects have been like either 0:21:05.080 --> 0:21:08.080 stalled or halted, And there's been tremendous success on the 0:21:08.119 --> 0:21:11.960 local level in like counties and in most recently this 0:21:12.200 --> 0:21:15.440 entire state of Maine passing laws against the construction of 0:21:15.520 --> 0:21:17.840 data centers. Like, yeah, there's actually been a lot of 0:21:17.920 --> 0:21:21.399 success in fighting the building of new data centers. If 0:21:21.480 --> 0:21:24.640 someone wanted to have a positive impact on this, there's 0:21:24.640 --> 0:21:27.680 a lot of room right now to make that fight 0:21:27.840 --> 0:21:31.040 even more effective as opposed to doing like stupid bullshit 0:21:31.480 --> 0:21:34.000 that you would only want to do if you had 0:21:34.000 --> 0:21:37.359 convinced yourself that we were like literally moments away from 0:21:37.440 --> 0:21:38.160 judgment day. 0:21:38.680 --> 0:21:40.920 Yeah. And he used to be involved in this sort 0:21:40.920 --> 0:21:42.960 of action, in this sort of organizing like around that 0:21:43.040 --> 0:21:44.680 kind of stuff when he was when he was doing 0:21:44.680 --> 0:21:47.800 stuff with pase Ai. Yeah, but it was really in 0:21:47.840 --> 0:21:49.439 the past few months where where he started to like 0:21:49.440 --> 0:21:52.000 spiral in this in this very like like doomer direction. 0:21:52.160 --> 0:21:54.240 Like you know, he was already a very critical and 0:21:54.359 --> 0:21:57.679 very doomor about what a I could do, but the 0:21:57.760 --> 0:22:01.800 sort of intense existential like like a mediate existential threat 0:22:02.040 --> 0:22:04.760 that that opposed is something that he was really developing 0:22:04.760 --> 0:22:06.600 at the end of last year in the start of 0:22:06.640 --> 0:22:09.800 this year. And you know this is in part because 0:22:09.840 --> 0:22:12.640 of the sort of environment that he was immersed in. Yeah, 0:22:12.760 --> 0:22:27.040 and we'll talk more about that environment after this ad break. Okay, 0:22:27.080 --> 0:22:30.880 we are back. Morno Gama had a bunch of other 0:22:30.960 --> 0:22:34.320 writing on his substack, which gives us a bit of 0:22:34.320 --> 0:22:37.280 a closer look at his political philosophy and the sort 0:22:37.320 --> 0:22:41.159 of information ecosystem that he exists in beyond just the 0:22:41.240 --> 0:22:44.720 AI question. In this AI article that he wrote or 0:22:44.800 --> 0:22:49.840 published in January, he recommended that people read Eliezer Yudkowski's 0:22:49.840 --> 0:22:52.119 book If anyone builds it, everyone dies. 0:22:52.760 --> 0:22:54.240 Mm hmm. Thanks the Lazer. 0:22:54.840 --> 0:22:57.399 Do you want to, like, I guess briefly give some 0:22:57.880 --> 0:22:59.920 background on like who this guy is. 0:23:00.600 --> 0:23:02.840 Well, he wrote a rationalist Harry Potter, Like he's the 0:23:02.880 --> 0:23:05.560 guy who started a website called less Wrong, Yeah, which 0:23:05.640 --> 0:23:08.359 was about basically like logic puzzles and trying to like 0:23:08.440 --> 0:23:11.520 optimize your thinking and your responses to behavior with like 0:23:11.560 --> 0:23:15.239 Bayesian analysis. Yes, and he's kind of branded himself as 0:23:15.240 --> 0:23:17.680 an AI expert. He's not like a coder or anything, 0:23:17.760 --> 0:23:19.359