WEBVTT - Nick Bostrom on What Happens if AI Solves All of Our Problems 0:00:03.840 --> 0:00:05.320 Hello, Odd Loads listeners. 0:00:05.360 --> 0:00:07.560 I'm Joe Wisenthal and I'm Tracy Alloway. 0:00:07.720 --> 0:00:10.280 We're the hosts of the odd Loads podcast, and we've 0:00:10.280 --> 0:00:11.560 got something exciting for you. 0:00:11.760 --> 0:00:12.160 That's right. 0:00:12.200 --> 0:00:14.800 So one of the best parts of hosting our podcast 0:00:14.920 --> 0:00:17.840 is we get to actually meet and interact with our listeners. 0:00:17.960 --> 0:00:21.680 And we know we have some listeners over in Los Angeles. 0:00:21.160 --> 0:00:21.560 That's right. 0:00:21.600 --> 0:00:23.840 So if you're in LA, we are going to be 0:00:23.920 --> 0:00:27.760 recording a live show, some live recordings at the Vermont 0:00:27.800 --> 0:00:30.160 Theater in Hollywood on September seventeenth. 0:00:30.240 --> 0:00:33.040 We have some really exciting guests lined up, have some 0:00:33.120 --> 0:00:36.479 really great conversations planned, so go ahead and get your tickets. 0:00:36.479 --> 0:00:39.040 You can find those over at bloomberg dot com forward 0:00:39.080 --> 0:00:41.760 slash odd Lots, or click the link below in the 0:00:41.800 --> 0:00:43.800 show notes and come and say hi when you're there. 0:00:49.440 --> 0:01:03.200 Bloomberg Audio Studios, Podcasts, Radio News. 0:01:05.280 --> 0:01:08.840 Hello and welcome to another episode of the Odd Lots podcast. 0:01:08.920 --> 0:01:10.319 I'm Joeisenthal and. 0:01:10.280 --> 0:01:11.240 I'm Tracy Alloway. 0:01:11.520 --> 0:01:15.160 Tracy, there's so much AI news, obviously, every single day 0:01:15.200 --> 0:01:17.360 for the rest of our lives. If we're lucky in 0:01:17.440 --> 0:01:20.960 the good scenario, there's two big events or sort of things, 0:01:21.040 --> 0:01:22.920 though that have really struck out to me lately that 0:01:23.000 --> 0:01:27.640 to my mind, represent either the very good version or 0:01:27.680 --> 0:01:30.120 the very bad version, depending on how you look at 0:01:30.520 --> 0:01:33.679 the scary version, and we're seeing more and more headlines 0:01:33.720 --> 0:01:36.200 like this. Several weeks ago, we had the open ai 0:01:36.360 --> 0:01:39.680 hugging face incident where the model quote escaped from its 0:01:39.680 --> 0:01:42.680 sandbox and did some hacking and people are working, you know, 0:01:43.040 --> 0:01:45.520 some light hacking, light hacking erases all these fears of 0:01:45.560 --> 0:01:47.280 Skynit type scenarios, etc. 0:01:47.560 --> 0:01:48.919 Where we can't control the model. 0:01:49.360 --> 0:01:52.800 And then in the sort of quote good version, we 0:01:52.880 --> 0:01:55.720 had open AI recently coming out and saying it had 0:01:55.760 --> 0:01:59.360 made all of these breakthroughs and theoretical mathematicians, which is 0:01:59.440 --> 0:02:02.640 good in the sense that it's like it's working, but 0:02:02.720 --> 0:02:06.160 seems to be creating an existential crisis for the most 0:02:06.200 --> 0:02:07.800 some of the most brilliant people on earth who work 0:02:07.800 --> 0:02:09.000 in theoretical mathematics. 0:02:09.080 --> 0:02:11.079 Right, so we've talked about this before. When it comes 0:02:11.160 --> 0:02:16.359 to AI, the range of possibilities seems more extreme than 0:02:16.520 --> 0:02:19.280 some previous technologies. Right, on the one hand, you have 0:02:19.400 --> 0:02:22.440 death and dystopia. Yeah, the robots basically kill us all. 0:02:22.720 --> 0:02:25.120 Now on the other hand, you have this idea that 0:02:25.240 --> 0:02:29.760 maybe super intelligence AGI is going to solve literally all 0:02:29.880 --> 0:02:34.239 the world's problems to the extent that you and I 0:02:34.280 --> 0:02:36.720 won't know what to do with ourselves anymore, right, Like. 0:02:36.639 --> 0:02:40.160 That is the that's the good scenario? Is that? 0:02:40.400 --> 0:02:43.000 And then the whole range of outcomes in between the. 0:02:42.919 --> 0:02:46.320 Good scenarios, we've literally solved everything, in which case the 0:02:46.400 --> 0:02:47.000 question is in. 0:02:47.080 --> 0:02:50.520 Case we're sad and miserable, yeah, or grappling for a 0:02:50.560 --> 0:02:51.920 meaning to our lives. 0:02:52.200 --> 0:02:55.200 Yeah, So is there a good scenario with AI actually 0:02:56.200 --> 0:02:57.639 like it doesn't know or. 0:02:57.720 --> 0:03:00.760 Maybe AGI will figure it out for us. 0:03:00.840 --> 0:03:03.280 Maybe the middle scenarios, we just have a huge financial 0:03:03.320 --> 0:03:06.400 crisis of all the debt, and that's something between human 0:03:06.440 --> 0:03:10.120 extinction and meaninglessness in our lives, and we just have 0:03:10.200 --> 0:03:12.839 something that resembles like the two thousand and eight Maybe 0:03:12.880 --> 0:03:16.520 even two thousand and eight is the is actually the 0:03:16.520 --> 0:03:18.320 best of all suboptimal world. 0:03:18.400 --> 0:03:19.799 So the AI excellent. 0:03:19.960 --> 0:03:22.280 This will be our most uplifting conversation ever. 0:03:22.520 --> 0:03:25.959 Anyway, we've been doing more AI episode seemingly every year 0:03:26.000 --> 0:03:28.760 since chat GPT came out, But people smarter than us 0:03:28.919 --> 0:03:31.000 have been anticipating all of these things for a very 0:03:31.040 --> 0:03:34.960 long time. Some of these exact scenarios, particularly the existential 0:03:34.960 --> 0:03:39.320 crisis facing mathematicians that the model escaping from its sandbox 0:03:39.400 --> 0:03:42.360 and circumvent misalignment. 0:03:41.640 --> 0:03:42.520 With the creators. 0:03:42.680 --> 0:03:45.520 People have been thinking about this and anticipating this for 0:03:45.600 --> 0:03:49.160 literally decades, even before and even before any of us 0:03:49.160 --> 0:03:49.720 were talking about this. 0:03:49.800 --> 0:03:52.800 It's funny how like the ven diagram of sci Fi 0:03:52.880 --> 0:03:59.080 writers and like legitimate academic featurerologists is just converging right totally. 0:03:59.080 --> 0:04:01.680 And then there's the question of the models bad because 0:04:01.720 --> 0:04:03.520 they read all of the sci Fi and that's what 0:04:03.560 --> 0:04:06.640 they were trained on. Anyway, I'm very excited to say 0:04:06.640 --> 0:04:08.480 we really do have the perfect guest, someone who has 0:04:08.640 --> 0:04:11.840 been thinking about the whole range of possible questions and 0:04:11.880 --> 0:04:14.840 outcomes with AI for a very long time. We're gonna 0:04:14.840 --> 0:04:17.520 be speaking with Nick Bostrom. He's a professor, a philosopher, 0:04:17.760 --> 0:04:20.240 and his most recent book is Deep Utopias, which sort 0:04:20.240 --> 0:04:23.839 of attempts to answer the question what happens if AI 0:04:23.920 --> 0:04:26.560 goes right? So, Professor Bostrom, thank you so much for 0:04:26.760 --> 0:04:27.640 coming on Odd. 0:04:27.400 --> 0:04:28.760 Lots, happy to join you. 0:04:29.320 --> 0:04:30.280 What do you make of. 0:04:30.200 --> 0:04:35.120 The existential crisis of mathematicians their sort of realization that 0:04:35.560 --> 0:04:38.000 maybe there will just be no more puzzles left to 0:04:38.080 --> 0:04:41.640 solve because the AI is just very good at them 0:04:41.680 --> 0:04:43.800 and can find them all cheaply and answer all of 0:04:43.839 --> 0:04:44.440 our questions. 0:04:44.839 --> 0:04:48.280 Yeah, I hopefully we'll all be in that position at 0:04:48.320 --> 0:04:52.960 some point. I guess we could compare it to what 0:04:53.120 --> 0:04:58.960 happened with chess, where computers have also exceeded humans, and 0:04:59.040 --> 0:05:01.880 yet a lot of pe ball are still involved in 0:05:02.440 --> 0:05:06.119 training themselves in chess and competing in chess and taking 0:05:06.120 --> 0:05:09.400 pleasure in that. So maybe mathematics will ultimately become more 0:05:09.440 --> 0:05:10.360 like a hobby pursuit. 0:05:10.680 --> 0:05:13.159 Wait, Joe, you still play chess, right, Yeah? I was. 0:05:13.320 --> 0:05:15.480 I'm glad you brought up chess because I still love 0:05:15.560 --> 0:05:15.880 chess and. 0:05:16.000 --> 0:05:17.679 Why it's fun. 0:05:18.040 --> 0:05:20.480 It's fun, you know, it's still a puzzle. It's still 0:05:20.600 --> 0:05:23.719 mental exercise for me. It's still something to do to 0:05:23.800 --> 0:05:26.120 pass the time while I'm waiting for the subway playing 0:05:26.120 --> 0:05:29.160 a three minute game online. I know that I knew, 0:05:29.440 --> 0:05:31.919 even before stockfishould so forth, I knew I would never 0:05:32.000 --> 0:05:32.679 be a grand master. 0:05:33.000 --> 0:05:33.960 Still fun, all right? 0:05:34.160 --> 0:05:38.240 So, Nick, you have basically two books that deal with 0:05:39.080 --> 0:05:43.360 these tail risk scenarios. Now, So one is this idea 0:05:43.360 --> 0:05:46.240 of deep utopia, and AI is going to solve all 0:05:46.240 --> 0:05:49.120 our problems. And then you also have super intelligence, which 0:05:49.160 --> 0:05:53.400 seemed to paint a different scenario where AI poses a 0:05:53.440 --> 0:05:58.359 physical and possibly existential danger to humanity. Which one do 0:05:58.400 --> 0:06:01.160 you think we're closer to right now? 0:06:01.800 --> 0:06:04.680 We're somewhere in the middle maybe. So the first one 0:06:04.839 --> 0:06:08.479 was super Intelligence. It came out in twenty fourteen, and 0:06:08.520 --> 0:06:11.480 I had been working on it for six years before that, 0:06:11.680 --> 0:06:15.760 and at that time, the whole notion of having human 0:06:15.880 --> 0:06:20.600 level or even super intelligent machines was generally just ignored 0:06:20.800 --> 0:06:24.039 or people dismissed it as science fiction. And it seemed 0:06:24.040 --> 0:06:27.720 to me clear that this would one day happen if 0:06:27.839 --> 0:06:31.200 science and technology were allowed to continue on wide front. 0:06:31.240 --> 0:06:34.800 We would eventually figure out how to make smart algorithms 0:06:34.800 --> 0:06:38.960 and build big computers, and that once we succeeded with that, 0:06:39.160 --> 0:06:42.279 we would then be faced with a bunch of challenges, 0:06:42.600 --> 0:06:45.840 including this alignment problem of how you can sort of 0:06:45.880 --> 0:06:50.520 design a mind more brilliant than any human and still 0:06:50.760 --> 0:06:53.120 steer it so that it will actually have an impact 0:06:53.160 --> 0:06:57.159 on the world that is beneficial. So that, I guess, 0:06:57.920 --> 0:07:01.760 like with many other plopments, helped eventually build a kind 0:07:01.760 --> 0:07:04.480 of global conversation, and so now all the frontier labs 0:07:04.480 --> 0:07:08.800 have research teams trying to develop scalable methods for AI control, 0:07:09.040 --> 0:07:12.480 and the conversation really has kind of swung, and there's 0:07:12.560 --> 0:07:16.880 like this rise of people really worried about what will happen. 0:07:17.600 --> 0:07:20.120 And I thought it might be time to have a 0:07:20.160 --> 0:07:21.880 look at the other side of the coin, like what 0:07:21.920 --> 0:07:24.960 happens if things actually go well if we solve this 0:07:25.080 --> 0:07:29.480 alignment problem. Let's also imagine we solve to whatever extent 0:07:29.520 --> 0:07:31.800 it can be solved, like the governance challenge, like we 0:07:31.840 --> 0:07:35.360 don't use this technology to oppress each other or whach war, 0:07:35.400 --> 0:07:38.040 and we get like the best possible outcome, like what 0:07:38.040 --> 0:07:40.440 would the human life actually look like? In this kind 0:07:40.440 --> 0:07:43.080 of I call it a solve the world where all 0:07:43.080 --> 0:07:45.960 practical problems either have already been solved, or to the 0:07:46.000 --> 0:07:48.880 extent that they remain, are better worked on by these 0:07:49.120 --> 0:07:52.640 super intelligent machines and robots, and whether there's no longer 0:07:52.680 --> 0:07:56.200 any need for human economic labor. And in fact, if 0:07:56.200 --> 0:07:59.160 you think more deeply about it, you realize it's not 0:07:59.280 --> 0:08:03.240 just earning wages that becomes unnecessary for humans to do, 0:08:03.320 --> 0:08:06.400 but a lot of other instrumentally motivated efforts as well 0:08:06.480 --> 0:08:10.600 would seem to lose their point. And so that's the 0:08:10.720 --> 0:08:14.880 kind of topic of this Deep Utopia book. 0:08:14.960 --> 0:08:18.239 Actually talk about some of that other things that could 0:08:18.240 --> 0:08:20.400 be solved. Because when I was like a third of 0:08:20.400 --> 0:08:22.920 the way through your book, I was like, you know what, 0:08:23.080 --> 0:08:25.679 it's fine, if all of our economic problems are solved, 0:08:25.720 --> 0:08:29.720 I'll just derive satisfaction from raising my kids and then 0:08:29.960 --> 0:08:32.840 maybe helping to raise their kids, etc. But as you 0:08:32.920 --> 0:08:37.280 point out, even that in the truly solved world, maybe 0:08:37.320 --> 0:08:41.800 even something like raising children might not be an available 0:08:41.880 --> 0:08:46.520 avenue to us to find, at least our conventional notions 0:08:46.520 --> 0:08:47.240 of satisfaction. 0:08:47.559 --> 0:08:50.959 Yeah, so we need to, I guess, make some distinctions here. 0:08:51.040 --> 0:08:55.040 So when you say satisfaction, that could mean a subjective 0:08:56.160 --> 0:09:00.440 state of mind, like you're feeling happy and pleased and 0:09:00.760 --> 0:09:03.920 relax and joyful and so forth. I think that would 0:09:03.920 --> 0:09:10.599 be easy to achieve in this condition of technological maturity. 0:09:10.640 --> 0:09:14.240 It could also mean that satisfaction in the sense of 0:09:14.280 --> 0:09:17.560 our preferences having been mad, like all the things we 0:09:17.640 --> 0:09:20.200 want from life. And there it's a little bit more 0:09:20.240 --> 0:09:22.560 tricky because one of the things that people might want 0:09:22.720 --> 0:09:27.080 is purpose, like having worthwhile things to work on and 0:09:27.080 --> 0:09:30.480 struggle with. And there is more of a question mark 0:09:30.480 --> 0:09:32.600 which we can sort of dive into. But if we 0:09:32.640 --> 0:09:36.120 take it sort of at first, like medical automate economy, okay, 0:09:36.160 --> 0:09:38.040 so you wouldn't need to sort of commute into the 0:09:38.040 --> 0:09:40.280 office every day just to sort of get your paycheck. 0:09:40.920 --> 0:09:44.080 Would require a big adjustment in society and in culture 0:09:44.200 --> 0:09:46.960 because right now, like the whole education system is geared 0:09:46.960 --> 0:09:51.560 towards producing workers, like children are coming in right and 0:09:51.600 --> 0:09:54.760 then they are processed, learned to sit at their desk, 0:09:55.280 --> 0:09:58.640 receive assignments, they are sort of quality graded, and then 0:09:58.920 --> 0:10:00.960 spit out at the other end, and then you have 0:10:01.000 --> 0:10:04.240 somebody who can go into the economy and perform some 0:10:04.960 --> 0:10:07.199 economically useful time. So that that would no longer make 0:10:07.240 --> 0:10:10.360 sense in this scenario where machines can do all of 0:10:10.400 --> 0:10:12.840 these jobs, and you might want to have an education 0:10:12.920 --> 0:10:17.320 system that instead focused on developing skills for enjoying life, 0:10:17.360 --> 0:10:23.439 like cultivating the art of conversation, appreciation for nature and beauty, sports, hobbies, 0:10:24.640 --> 0:10:28.240 the ability to form deep friendships, spirituality, all of that 0:10:28.320 --> 0:10:32.280 kind of stuff. But then, as you alluded to, you 0:10:32.320 --> 0:10:35.439 think through what it really would mean to have such 0:10:35.760 --> 0:10:38.200 immense affordances. And because here it's not just that we 0:10:38.200 --> 0:10:41.800 would have super intelligent machines, but the superintelligence could also 0:10:42.120 --> 0:10:45.280 bring along a host of other technological advances. Right, you 0:10:45.280 --> 0:10:48.600 have science and technology running now at machine speed rather 0:10:48.640 --> 0:10:51.560 than human speed. So you get the kind of compression 0:10:51.600 --> 0:10:54.720 of the future, all these science fiction like technologies and 0:10:54.760 --> 0:10:57.360 inventions that maybe we would make if we had twenty 0:10:57.440 --> 0:10:59.360 thousand years to work on it. We would maybe have 0:10:59.400 --> 0:11:03.360 space colon, some perfect virtual realities, cures for aging, like 0:11:03.400 --> 0:11:06.079 all of that might happen quite soon after super intelligence. 0:11:06.320 --> 0:11:08.680 And so then if you ask somebody now who doesn't 0:11:08.720 --> 0:11:10.800 have to work for a living, you know, oftentimes they 0:11:10.840 --> 0:11:13.320 have very busy lives, but maybe what do they do else? 0:11:13.360 --> 0:11:16.280 It's for somebody maybe they like to they like fitness, 0:11:16.280 --> 0:11:18.720 and like maybe they spend an hour a day running 0:11:18.800 --> 0:11:21.120 or at the gym. Right, but at technological maturity, you 0:11:21.120 --> 0:11:23.480 could just pop a pill that would achieve the same 0:11:23.920 --> 0:11:28.600 physiological effects and like the state of mental clarity and 0:11:28.640 --> 0:11:31.280 relaxation that some people get, right, And so then you 0:11:31.280 --> 0:11:34.280 can start to work through the different things that like 0:11:34.320 --> 0:11:38.320 maybe for somebody else they like, oh, like decorating the 0:11:38.360 --> 0:11:40.760 home could be a fun project that people might do 0:11:40.840 --> 0:11:43.079 right if they don't have to work. So like they 0:11:43.080 --> 0:11:45.800 go through the catalogs, they go to the store, they 0:11:45.840 --> 0:11:48.240 pick out the perfect curtain, they do all of these 0:11:48.240 --> 0:11:51.480 things that could be But then like with super intelligence, 0:11:51.559 --> 0:11:53.640 you could have this kind of system that really is 0:11:53.679 --> 0:11:56.840 good at knowing or inferring your preferences and could do 0:11:56.880 --> 0:11:59.520 a much better job at you at picking out all 0:11:59.559 --> 0:12:05.040 the perfect items and purchasing them and like installing them 0:12:05.080 --> 0:12:07.040 with robots and so that the end result would be 0:12:07.080 --> 0:12:09.680 much better if you just press the button and say, AI, 0:12:09.800 --> 0:12:12.000 decorate my home. Then if you went through the whole 0:12:12.000 --> 0:12:13.840 trouble of doing it yourself, so you could still do 0:12:13.880 --> 0:12:16.200 it yourself, but is there really a point to it? 0:12:16.280 --> 0:12:18.760 Like that would seem to be this kind of purposelessness, 0:12:19.040 --> 0:12:21.839 and you can sort of walk through the different things 0:12:21.880 --> 0:12:24.120 that one might fill one's life with if one is 0:12:24.400 --> 0:12:27.000 like on an extended holiday, are on retirement, or you're 0:12:27.080 --> 0:12:29.720 like sufficiently wealthy you have just decided to quit working, 0:12:30.240 --> 0:12:32.880 and a lot of those also, it seems could be automated, 0:12:33.160 --> 0:12:35.000 so you would have this it's like more like a 0:12:35.000 --> 0:12:38.320 post instrumental condition rather than just a post work condition. 0:12:38.800 --> 0:12:42.840 Some of this sounds nice, and I myself enjoy a 0:12:42.840 --> 0:12:45.480 little bit of interior decorating and going to the shops 0:12:45.520 --> 0:12:49.120 and buying stuff in the utopian vision of the future, 0:12:49.440 --> 0:12:54.839 How are economic gains from AI actually distributed, because again, 0:12:54.960 --> 0:12:56.920 like all of this sounds very nice, but it also 0:12:56.960 --> 0:13:00.319 costs money, and if we're not working because the Rowts 0:13:00.480 --> 0:13:03.360 have replaced us, I'm not sure where that money comes from. 0:13:03.559 --> 0:13:06.600 Yeah, so that's a more practical problem, shall we say, 0:13:06.640 --> 0:13:10.680 which is like deliberately set aside in this book The 0:13:10.760 --> 0:13:13.360 Deep Utopa, because I wanted in that book actually to 0:13:13.400 --> 0:13:15.600 get to the question and the more philosophical questions of 0:13:15.640 --> 0:13:18.600 what ultimately gives value. But it is of course an 0:13:18.720 --> 0:13:22.040 important question, I think. So a lot depends on how 0:13:22.120 --> 0:13:24.760 it pans out. But I mean, we do have a 0:13:24.800 --> 0:13:28.439 taxation system, so you could imagine, like in the simplest 0:13:28.440 --> 0:13:31.840 scenario where things just continue as they are with better technology, 0:13:31.920 --> 0:13:36.080 maybe some companies make windfall profits from like serving the 0:13:36.120 --> 0:13:40.040 latest models or building the infrastructure the data centers, et cetera. 0:13:40.600 --> 0:13:44.760 Companies achieving efficiency gains from not being needing to hire 0:13:44.800 --> 0:13:48.280 all of these humans, and then some of those capital gains, 0:13:48.640 --> 0:13:51.240 you know, or dividend income gets taxed, and that could 0:13:51.320 --> 0:13:56.280 be redistributed. The good thing about these scenarios where there 0:13:56.320 --> 0:14:00.680 is massive automation and hence massive on a employment is 0:14:00.720 --> 0:14:04.559 that those would also be scenarios with extremely rapid economic growth. 0:14:05.240 --> 0:14:10.319 So the pie would be expanding humongously, which means that 0:14:10.400 --> 0:14:14.480 you'd still maybe need some mechanism whereby people who don't 0:14:14.559 --> 0:14:20.040 have assets can still benefit, but even a small slice 0:14:20.080 --> 0:14:22.160 of a really enormous pie could go a long way. 0:14:22.240 --> 0:14:24.320 So I think that could be this tsunami of wealth 0:14:24.360 --> 0:14:28.320 that would sort of be flowing through society and many 0:14:28.360 --> 0:14:31.320 services that currently people have to pay a lot of money. 0:14:31.320 --> 0:14:33.720 It might become almost free. I mean, we'll already see 0:14:33.720 --> 0:14:37.040 it with some like medical advice for example, right maybe before, 0:14:37.640 --> 0:14:39.640 if you don't have public healthcare, you would need to 0:14:39.640 --> 0:14:42.760 maybe pay a doctor four hundred dollars to get the consultation. 0:14:42.880 --> 0:14:45.840 Now you can ask CHATTYPT and get you know, in 0:14:45.840 --> 0:14:49.080 some cases a better opinion basically for free. And that 0:14:49.080 --> 0:14:51.240 that could happen, you know, in many other fields as well, 0:14:51.320 --> 0:14:53.720 legal advice. You know, you maybe have cheap robots you 0:14:53.760 --> 0:14:57.480 could do your cleaning and you know, gardening and run errants. 0:14:58.000 --> 0:15:01.960 So yeah, some combination of sort of deflation effects and 0:15:01.960 --> 0:15:04.120 then maybe some sort of general spill over, either from 0:15:04.120 --> 0:15:06.560 philanthropy or taxation and government redistribution. 0:15:22.240 --> 0:15:25.240 I found the twenty fourteen book to be sort of 0:15:26.160 --> 0:15:28.440 it was very interesting. It was clearly ahead of its time. 0:15:28.720 --> 0:15:31.200 I also found it to be sort of like unrelentingly 0:15:31.680 --> 0:15:34.800 grim and you sort of there's this very narrow path 0:15:34.880 --> 0:15:36.560 that we seem to have and we'll talk more about 0:15:36.560 --> 0:15:40.480 it maybe later, of like avoiding the doom fate. The 0:15:40.600 --> 0:15:44.040 newer book I found to be very fun. You seem 0:15:44.080 --> 0:15:46.560 to make fun of yourself a little bit in it, 0:15:46.920 --> 0:15:48.920 you seem to make fun of the students. You seem 0:15:49.000 --> 0:15:51.920 to be having a good time writing it. I'm curious 0:15:51.960 --> 0:15:53.360 of like, in the like the ten. 0:15:53.280 --> 0:15:56.000 Years between the two books. 0:15:55.720 --> 0:15:58.840 Did something change such that I don't know, this is 0:15:58.840 --> 0:16:00.320 the new book just seemed like you were having more 0:16:00.360 --> 0:16:00.760 fun writing. 0:16:00.960 --> 0:16:03.120 Yes, it's not so much that my views have changed, 0:16:03.200 --> 0:16:05.760 it's just that all along for me or these two 0:16:05.800 --> 0:16:09.200 sides of the coin, and back in twenty fourteen, it 0:16:09.240 --> 0:16:14.480 seemed more pressing and relevant to study that downside, because 0:16:14.600 --> 0:16:17.400 that was very much neglected, and it seemed important to 0:16:17.400 --> 0:16:19.640 try to introduce some concepts so people could start working 0:16:19.800 --> 0:16:23.720 on figuring out solutions. And now there's much more awareness 0:16:23.880 --> 0:16:26.280 and understanding of that, it seemed time to also look 0:16:26.320 --> 0:16:29.320 at the other side of the coin. So it's more 0:16:29.480 --> 0:16:32.160 just like both sides where they're all along in my thinking, 0:16:32.200 --> 0:16:36.960 but that the style is actually suggest different in Deep Topia. 0:16:37.000 --> 0:16:40.280 And that's partly I wanted, in addition to talking about 0:16:40.360 --> 0:16:44.080 this utopian condition, maybe also sort of illustrate a little 0:16:44.080 --> 0:16:47.200 bit of spirit with which I think we should approach 0:16:47.960 --> 0:16:51.760 these choices that would open up to us if we 0:16:52.000 --> 0:16:54.880 make it through this transition, a spirit of sort of 0:16:55.640 --> 0:17:01.320 playful generosity that I think might make us more likely 0:17:01.360 --> 0:17:05.200 to sort of explore this large space of wonderful new 0:17:05.520 --> 0:17:08.080 modes of being and the wisdom. And I think so 0:17:08.160 --> 0:17:11.520 I wanted to rub you illustrate in the right are 0:17:11.560 --> 0:17:14.400 these fictional characters, for example, in the book, and they're 0:17:14.440 --> 0:17:16.480 kind of leaving a little mini version of a utopia 0:17:16.680 --> 0:17:19.920 of sorts as they are sort of discussing and debating 0:17:19.960 --> 0:17:21.360 these topics. 0:17:21.480 --> 0:17:26.840 I mean, playful generosity is not necessarily a characterization that 0:17:26.880 --> 0:17:30.280 I would associate with a lot of Silicon Valley and 0:17:30.720 --> 0:17:35.439 hyperscalers and the people who are actually creating these models. 0:17:36.560 --> 0:17:37.240 I don't know, Like. 0:17:37.200 --> 0:17:41.159 There seems to be this weird tension between futurologists and 0:17:41.240 --> 0:17:45.840 transhumanists who seem to be very focused on saving humanity 0:17:45.920 --> 0:17:50.040 but actually don't seem to like people that much at 0:17:50.040 --> 0:17:51.359 the moment, you know what I. 0:17:51.240 --> 0:17:54.120 Mean, Absolutely they like humanity, but not human. 0:17:54.040 --> 0:17:59.840 Yeah, exactly, putting on your transhumanist representational hat. Can you explain, 0:18:00.119 --> 0:18:04.080 like I guess, the attitudes towards humanity of Silicon Valley, 0:18:04.800 --> 0:18:05.359 big question? 0:18:05.880 --> 0:18:09.000 Yeah, well, I mean I should state first that this 0:18:09.160 --> 0:18:13.040 playful generosity is like after we have solved these more 0:18:13.080 --> 0:18:15.600 pressing practical problems that we first we've first got to 0:18:15.600 --> 0:18:17.600 get there right now. I think there is a lot 0:18:17.640 --> 0:18:23.679 of need for earnest effort and like straining ourselves to 0:18:23.680 --> 0:18:25.920 do really do our best on this and even so 0:18:26.080 --> 0:18:28.399 we might not make it. But yeah, I don't know, 0:18:28.480 --> 0:18:31.119 I think me you know, it might just be partially 0:18:31.480 --> 0:18:34.760 there are personality traits, like some people are more I guess, 0:18:34.840 --> 0:18:38.040 towards the autistic side of the spectrum, or introvert or 0:18:38.080 --> 0:18:40.199 interested in text, and maybe they don't like to be 0:18:40.240 --> 0:18:43.520 outmoosing with people all day long. It doesn't mean that 0:18:43.560 --> 0:18:47.560 they don't like people in the sense of wanting well 0:18:47.600 --> 0:18:50.879 for people. It might just be that they prefer to 0:18:50.880 --> 0:18:53.439 spend more hours of the day solving technical problems and 0:18:53.480 --> 0:18:56.760 thinking about puzzles and you know, stuff like that. So 0:18:56.960 --> 0:18:58.840 I don't I don't think there's like a necessary connection 0:18:58.920 --> 0:19:01.000 between like being a people person who just like to 0:19:01.000 --> 0:19:03.240 be around people, you know, twenty four to seven and 0:19:03.480 --> 0:19:06.919 therefore being sort of altruistic or kind or benevolent. So 0:19:06.960 --> 0:19:09.239 that's one thing. Another I guess is that it's an 0:19:09.280 --> 0:19:14.640 extremely competitive landscape. So whatever the personal preferences of different protagonists, 0:19:15.080 --> 0:19:18.639 their action space is constrained. You know, unless you really 0:19:18.920 --> 0:19:21.920 push forward with maximum speed and efficiency, he you're likely 0:19:22.040 --> 0:19:25.120 to just fall out of the race and become kind 0:19:25.160 --> 0:19:29.400 of irrelevant if you're one of these frontier labs. So yeah, 0:19:29.440 --> 0:19:31.920 it's like and it's just like the hugely complex situation, 0:19:32.040 --> 0:19:34.440 so many pressures and people who have different opinions about 0:19:34.480 --> 0:19:35.560 what they should do or shouldn't do. 0:19:35.960 --> 0:19:39.840 In the new book, it's presented as you delivering a 0:19:40.000 --> 0:19:41.960 series of lectures over a course of a week at 0:19:41.960 --> 0:19:44.160 some point in the future. In the new book, it's 0:19:44.160 --> 0:19:47.760 interspersed with some fit with a fable, a series of 0:19:47.760 --> 0:19:52.879 fables or epistles about animals trying to achieve utopia in 0:19:52.920 --> 0:19:57.960 the woods, and in their efforts to achieve utopia, they 0:19:58.000 --> 0:20:01.359 conceive of things like taking second delic mushrooms in order 0:20:01.440 --> 0:20:05.000 to make everyone more generous. They come up with a 0:20:05.080 --> 0:20:08.959 way to rationalize I guess what would say, like harems 0:20:09.000 --> 0:20:11.560 and so forth. And then you know you also like 0:20:11.600 --> 0:20:13.640 have these student characters and some of them are smart, 0:20:13.680 --> 0:20:16.240 et cetera. Oh, and you also have a story about 0:20:16.280 --> 0:20:18.879 a company in the book yet another like sort of 0:20:18.920 --> 0:20:22.040 short story where an AI company where it forms a 0:20:22.080 --> 0:20:23.280 cult around the model. 0:20:23.320 --> 0:20:24.040 And of course I. 0:20:23.920 --> 0:20:28.440 Immediately think about the way Anthropic is frequently characterized as 0:20:28.480 --> 0:20:32.159 a cult to claw that Claude is the character around 0:20:32.200 --> 0:20:36.400 which the company Anthropic is formed. All of this sounds 0:20:36.520 --> 0:20:41.800 like a certain AI subculture, et cetera. Are you impressed 0:20:42.280 --> 0:20:48.440 with the current generation of either AI builders, effective altruists, rationalists, 0:20:48.480 --> 0:20:51.199 and so forth? Like, do you have an assessment of 0:20:51.240 --> 0:20:52.360 the contemporary cohort? 0:20:53.160 --> 0:20:54.040 Am I impressed with it? 0:20:54.160 --> 0:20:54.320 Yeah? 0:20:54.359 --> 0:20:56.600 I mean a broadly speaking, I think it's a feel 0:20:56.640 --> 0:21:00.600 that is attracting a huge amount of talent right now, sure, 0:21:00.840 --> 0:21:03.880 And a lot of like the smartest young people I know, 0:21:03.920 --> 0:21:07.800 they're like all basically going into working on some aspect 0:21:08.200 --> 0:21:10.320 of this AI, think, like for some reason, like try 0:21:10.359 --> 0:21:13.119 to solve a safety you know, for others are like 0:21:13.160 --> 0:21:15.679 maybe trying to work on the governance challenges or the 0:21:15.720 --> 0:21:19.160 ethics of digital minds. So it's just a huge magnet 0:21:19.280 --> 0:21:21.199 right now, Like if you're young and brilliant, like, what 0:21:21.240 --> 0:21:22.960 are you going to work on with your life? Like, 0:21:23.200 --> 0:21:25.840