WEBVTT - The Story: Ask Lots of Questions w/ Meredith Whittaker 0:00:00.040 --> 0:00:02.680 Thanks for tuney into tech Stuff. If you don't recognize 0:00:02.680 --> 0:00:05.440 my voice, my name is Oz Valoshian and I'm here 0:00:05.480 --> 0:00:08.639 because the inimitable Jonathan Strickland has passed the baton to 0:00:08.720 --> 0:00:12.160 Cara Price and myself to host Tech Stuff. The show 0:00:12.160 --> 0:00:15.040 will remain your home for all things tech, and all 0:00:15.080 --> 0:00:18.400 the old episodes will remain available in this feed. Thanks 0:00:18.400 --> 0:00:22.200 for listening. Welcome to Tech Stuff. I'm Os Valoshian and 0:00:22.239 --> 0:00:23.080 I'm Cara Price. 0:00:23.239 --> 0:00:26.960 And this is the story our weekly episode, dropping each Wednesday, 0:00:27.160 --> 0:00:30.160 sharing an in depth conversation with a fascinating person in 0:00:30.160 --> 0:00:33.159 and around tech. This week, Oz tells me he's bringing 0:00:33.200 --> 0:00:35.400 me a conversation with Meredith Whittaker. 0:00:37.760 --> 0:00:41.800 That's right. Meredith leads the Signal Foundation, which is a 0:00:41.800 --> 0:00:47.360 nonprofit that oversees the encrypted messaging app Signal, beloved by journalists, 0:00:47.360 --> 0:00:50.320 deal makers, drug dealers, and privacy buffs alike. 0:00:50.720 --> 0:00:54.320 It takes wanting to keep a really really really big 0:00:54.360 --> 0:00:57.080 secret to get on Signal, but we should also think 0:00:57.080 --> 0:00:59.640 about everything we say as a really big secret. If 0:00:59.640 --> 0:01:02.600 you can who has access to all of our information. 0:01:02.280 --> 0:01:04.679 One hundred percent, Well, you don't sometimes know what's a 0:01:04.720 --> 0:01:08.400 really really big secret until it's too late. Meredith has 0:01:08.480 --> 0:01:11.640 a fascinating background. She started at Google back in the 0:01:11.720 --> 0:01:14.640 day when it famously boasted that don't be Evil slogan, 0:01:15.000 --> 0:01:17.320 and she actually started as a temp in a kind 0:01:17.360 --> 0:01:20.880 of customer service adjacent gig that was despite being a 0:01:20.920 --> 0:01:24.240 Berkeley grad in literature and rhetoric. Eventually she worked her 0:01:24.280 --> 0:01:28.280 way up into leadership, founding Google Open Research Group and 0:01:28.319 --> 0:01:32.640 then separately founding something called the AI Now Institute at NYU. 0:01:32.880 --> 0:01:35.480 And after she left Google, she became head of Signal 0:01:35.480 --> 0:01:39.000 in twenty twenty two. But the circumstances of her departure 0:01:39.000 --> 0:01:40.880 from Google were memorable, to say the least. 0:01:41.000 --> 0:01:42.399 I think it's fair to say that she got on 0:01:42.440 --> 0:01:45.720 everyone's radar back in twenty eighteen when she led the 0:01:45.760 --> 0:01:48.640 walkouts at Google over the experience of women in tech 0:01:48.680 --> 0:01:52.760 and at Google specifically over things like pay inequality and 0:01:52.760 --> 0:01:55.120 the handling of sexual harassment at the company. 0:01:55.400 --> 0:01:59.640 And then there's also Project Maven, which was Google's decision 0:01:59.640 --> 0:02:02.800 to take part in a Department of Defense program to 0:02:03.000 --> 0:02:07.960 use machine learning and data for battlefield target identification, which 0:02:08.040 --> 0:02:11.120 was very very controversial internally and which in part is 0:02:11.120 --> 0:02:14.040 what led her to her role today at signal. Merediths 0:02:14.080 --> 0:02:16.440 won a prize in Germany last year called the Helmut 0:02:16.440 --> 0:02:19.280 Schmidt Future Award, where she was honored to quote her 0:02:19.280 --> 0:02:22.960 commitment to the development of AI technology oriented towards the 0:02:23.000 --> 0:02:25.639 common good. So we talk about that too, and why 0:02:25.720 --> 0:02:29.640 Meredith fundamentally describes herself as an optimist. But we start 0:02:29.639 --> 0:02:31.119 by geeking out over literature. 0:02:31.560 --> 0:02:32.160 Let's hear it. 0:02:33.040 --> 0:02:35.200 So I was reading your bio and one of the 0:02:35.240 --> 0:02:37.959 things that stuck out to me was your background as 0:02:38.000 --> 0:02:41.440 a English literature and rhetoric major in college. I was 0:02:41.480 --> 0:02:44.280 also in English major. As I looked at your kind 0:02:44.320 --> 0:02:49.040 of story, there's this arc from arriving at Google as 0:02:49.040 --> 0:02:51.760 a graduate in two thousand and six in the Don't 0:02:51.800 --> 0:02:55.280 Be Evil error, seeing all of these technologies developed, many 0:02:55.280 --> 0:02:57.480 of which went on to be kind of part of 0:02:57.520 --> 0:03:00.640 the foundation of what we now call AI, leading a 0:03:00.680 --> 0:03:05.120 protest movement and ultimately leaving and it made me think 0:03:05.160 --> 0:03:08.359 of Pilgrim's Progress or a buildings Room. 0:03:08.400 --> 0:03:11.959 Then yeah, I mean, maybe a Jeremiah is more accurate. 0:03:14.800 --> 0:03:17.840 So that was a joke for the four English literature 0:03:18.120 --> 0:03:22.400 majors who are listening. I think for me, I was 0:03:22.600 --> 0:03:26.440 very sincere. I still am very sincere, and I came 0:03:26.480 --> 0:03:28.320 in and took a lot of things at face value, 0:03:28.400 --> 0:03:32.000 in part because I had no idea about tech. I 0:03:32.040 --> 0:03:35.240 was stepping in and I was asking very basic questions. 0:03:34.960 --> 0:03:36.320 You did in two thousand and six. 0:03:36.400 --> 0:03:39.720 I mean, well, you know, it wasn't the money occupation 0:03:40.160 --> 0:03:43.160 when I got into Google. It was it was actually, 0:03:43.240 --> 0:03:46.240 it was a fairly lovely environment in some ways because 0:03:46.280 --> 0:03:50.520 a lot of the people there were people who had 0:03:50.520 --> 0:03:53.800 a special interest, like nerdy people like myself, but I 0:03:53.920 --> 0:03:56.320 was a book nerd, and they were sort of math 0:03:56.400 --> 0:04:01.880 and science nerds. And suddenly their special interest, their sweet 0:04:01.920 --> 0:04:04.960 passion that they spent so much time thinking about, was 0:04:05.000 --> 0:04:09.000 also profitable. It was also important for the world. It 0:04:09.040 --> 0:04:12.400 put them in a position of being a meaningful contributor. 0:04:12.800 --> 0:04:15.800 And there was a genuineness to that. And we're in 0:04:15.840 --> 0:04:18.840 an era now, of course, where you know, no one's 0:04:18.920 --> 0:04:21.000 mom is saying be a doctor or a lawyer. They're 0:04:21.040 --> 0:04:24.080 saying be an engineer. And so it was, you know, 0:04:24.240 --> 0:04:26.799 it was simply a very different world. 0:04:26.800 --> 0:04:29.000 And what about don't be evil? When was that generation? 0:04:29.080 --> 0:04:29.200 Then? 0:04:29.200 --> 0:04:31.040 When was that abandoned? Was that part of the kind 0:04:31.080 --> 0:04:33.360 of discourse when you went to Google, that. 0:04:33.440 --> 0:04:35.760 Was generated before my time, and it was core to 0:04:35.800 --> 0:04:39.640 the discourse at Google. I think it was removed by 0:04:39.800 --> 0:04:43.200 lawyers in like twenty fifteen, twenty sixteen, I don't actually 0:04:43.240 --> 0:04:46.400 remember the date, but look, it's a slogan. 0:04:47.440 --> 0:04:47.640 You know. 0:04:47.680 --> 0:04:51.440 The structure of Google was still up into the right 0:04:51.560 --> 0:04:55.960 forever so profits and growth worthy objective functions. That's the 0:04:56.000 --> 0:05:02.200 sort of perpetual motion machine that Google adhere. However, I 0:05:02.279 --> 0:05:06.359 do think don't Be Evil had some power in that 0:05:06.520 --> 0:05:10.000 it was a touchstone that we all shared where socially 0:05:10.040 --> 0:05:12.880 awkward people are people who might not quite understand how 0:05:12.920 --> 0:05:15.400 to formulate a discomfort could kind of point to it 0:05:15.440 --> 0:05:17.720 and say like, hey, this is making me think that 0:05:17.760 --> 0:05:20.480 we should examine this because we said don't be evil. 0:05:21.040 --> 0:05:23.480 And most of this, you know, throughout Google's history was 0:05:23.520 --> 0:05:27.479 at a time where let's say, the horizons that would 0:05:27.600 --> 0:05:31.760 challenge that were very far in the distance. So you 0:05:31.760 --> 0:05:34.960 could still make a lot of quarterly returns doing a 0:05:35.000 --> 0:05:38.320 lot of things that were understood and we can debate 0:05:38.360 --> 0:05:42.800 that as as good right, and things like building a 0:05:42.920 --> 0:05:46.560 search engine for the Chinese market that would include surveillance 0:05:46.600 --> 0:05:49.200 functions for the government was not something you even had 0:05:49.200 --> 0:05:51.880 to consider because there was so much low hanging fruit 0:05:51.960 --> 0:05:55.080 in terms of continuing to grow, continuing to profit. And 0:05:55.120 --> 0:05:57.480 so I think that combination of like it was, it 0:05:57.520 --> 0:06:00.760 was sincerely held, and that the actual temptation to do 0:06:00.839 --> 0:06:04.239 the things that really got into that sort of hot 0:06:04.279 --> 0:06:08.200 water potential evil were not on the table at that 0:06:08.240 --> 0:06:10.400 point because there was so much else that could be done. 0:06:10.440 --> 0:06:12.920 And you know, of course, the ratchet turns, the ratchet turns, 0:06:12.960 --> 0:06:16.520 the ratchet turns, and then suddenly those horizons are much closer. 0:06:17.360 --> 0:06:23.880 The company is swollen by orders of magnitude, and that 0:06:24.080 --> 0:06:28.880 little slogan is quietly removed by a team of lawyers 0:06:29.880 --> 0:06:32.760 who probably bought some summer houses with the fees. 0:06:34.000 --> 0:06:36.800 Did you believe that slogan? And was there a moment 0:06:36.920 --> 0:06:40.120 along the way where you definitively started not to believe it. 0:06:42.360 --> 0:06:45.200 I liked having that slogan as a tool. Let me 0:06:45.240 --> 0:06:46.200 say that, right. 0:06:46.560 --> 0:06:49.400 It was useful. It did work in rooms. 0:06:49.480 --> 0:06:53.000 It you know, allowed us to frame debates in a 0:06:53.040 --> 0:06:58.040 way that had at at least as their reference port 0:06:58.560 --> 0:07:04.840 a core and ethical consideration. Like I was young, there 0:07:04.880 --> 0:07:07.920 was about a million different learning curves I was scaling 0:07:07.920 --> 0:07:10.040 at once. Yes, what does it mean to have a 0:07:10.080 --> 0:07:10.600 real job. 0:07:10.680 --> 0:07:10.880 Right. 0:07:11.160 --> 0:07:12.680 I didn't come from this class. 0:07:13.160 --> 0:07:16.440 I didn't have familiarity with these cultures or environments because 0:07:16.480 --> 0:07:19.840 I'd literally never been in an environment that looked anything 0:07:20.000 --> 0:07:23.640 like you know what my family still jokingly refers to 0:07:23.720 --> 0:07:24.640 as a business job. 0:07:24.920 --> 0:07:25.120 Right. 0:07:25.800 --> 0:07:28.320 I didn't know what was normal and what was not normal. 0:07:28.400 --> 0:07:30.880 So for me, Google was normal, and so I just 0:07:30.920 --> 0:07:32.280 want to be care Like. I don't know if I 0:07:32.400 --> 0:07:34.800 believed it or not. I found it useful and I 0:07:34.800 --> 0:07:39.440 saw it do work in the world, and I liked 0:07:39.520 --> 0:07:42.200 the idea, Like, it sounds great that we're giving people 0:07:42.280 --> 0:07:44.760 ads that match their interests. What could be wrong with 0:07:44.800 --> 0:07:48.600 that We're organizing the world's information. Wow, all of this 0:07:48.680 --> 0:07:49.120 is cool. 0:07:49.360 --> 0:07:53.160 Like I was like, Okay, what is tech? What is Google? Hey, 0:07:53.240 --> 0:07:56.320 sit me down and tell me what is information? I 0:07:56.360 --> 0:07:58.640 was a master at asking the dumbest questions, but sometimes 0:07:58.680 --> 0:08:01.200 they would unlock whole world for me, and then I 0:08:01.240 --> 0:08:05.160 would realize, like, oh, information is just websites. Okay, but 0:08:05.240 --> 0:08:07.440 you can't make a website if you don't have a computer, 0:08:09.040 --> 0:08:12.480 and you have to be able to code HTML markup 0:08:12.520 --> 0:08:16.040 to do that. Okay, So it's more limited than I thought. Okay, 0:08:16.080 --> 0:08:18.040 well what is an index? Oh, it's ranking it. But 0:08:18.200 --> 0:08:19.600 isn't that a value judgment? 0:08:20.040 --> 0:08:20.160 Well? 0:08:20.200 --> 0:08:21.960 I guess it's not if it's an algorithm, but like 0:08:22.000 --> 0:08:24.800 who makes the algorithm? And so it was butting my 0:08:24.880 --> 0:08:30.040 head up against what are kind of dumb, very basic 0:08:30.200 --> 0:08:34.640 questions and sort of sensing an insufficiency of the responses, 0:08:34.679 --> 0:08:39.319 and like, you know, over almost twenty years now, continuing 0:08:39.360 --> 0:08:41.760 that process of trying to, you know, just figure out 0:08:41.920 --> 0:08:43.360 like what are we doing here? 0:08:44.600 --> 0:08:48.520 And how do I understand it? And that was basically 0:08:48.559 --> 0:08:49.240 my education. 0:08:49.760 --> 0:08:54.000 So fust forward, you know, thirteen years from two thousand 0:08:54.000 --> 0:08:56.440 and six to twenty nineteen. How did you get to 0:08:56.480 --> 0:09:00.720 that point where it became impossible for you to stay? 0:09:01.559 --> 0:09:03.839 I don't have a pat answer, But how did I 0:09:03.880 --> 0:09:06.320 get to that point? Well? 0:09:06.520 --> 0:09:10.360 Again, I was a sincere person and I had done 0:09:10.360 --> 0:09:11.720 a lot in my career. I had a lot of 0:09:11.720 --> 0:09:14.600 opportunities and then I took them, and you know, I 0:09:14.600 --> 0:09:16.640 built a research group inside of Google. 0:09:17.400 --> 0:09:18.840 I was involved in. 0:09:18.800 --> 0:09:24.800 Efforts to spread privacy and security technologies across the tech ecosystem. 0:09:24.880 --> 0:09:28.040 I was a champion for privacy within Google. I had 0:09:28.040 --> 0:09:32.360 built you know, open source measurement infrastructures to help inform 0:09:32.400 --> 0:09:35.920 the net neutrality debate. I had always tried to be 0:09:36.040 --> 0:09:41.040 on the side of doing social good, to. 0:09:40.559 --> 0:09:42.240 Be very bland about it. 0:09:42.520 --> 0:09:46.160 Yeah, And I had been able to do very very 0:09:46.200 --> 0:09:50.840 well doing that. And at the point where I became frustrated, 0:09:51.679 --> 0:09:57.080 I had already established the AI Now Institute, co founding 0:09:57.160 --> 0:10:01.800 that at NYU, looking at some of the present day 0:10:02.760 --> 0:10:06.520 real term harms and social implications of AI technologies. This 0:10:06.720 --> 0:10:12.200 was twenty sixteen. I had become known through the academic 0:10:12.280 --> 0:10:15.720 world and the technical world, and particularly within Google itself 0:10:16.360 --> 0:10:19.920 as an authority and a speaker and an expert on 0:10:20.000 --> 0:10:23.680 those issues. So talking about some of the harms that 0:10:24.120 --> 0:10:30.160 these technologies could cause, criticizing within, essentially criticizing within. I 0:10:30.200 --> 0:10:33.080 saw myself as a resource, and actually many many people did. 0:10:33.080 --> 0:10:35.600 I would be brought in when teams were struggling with things, 0:10:35.640 --> 0:10:39.119 I would advise them. I would often give academic lectures 0:10:39.640 --> 0:10:42.440 that went against the party line at Google, but we're 0:10:42.920 --> 0:10:47.960 very well cited, were very empirically documented, and I had 0:10:48.080 --> 0:10:51.640 felt that I was making some headway right. 0:10:51.679 --> 0:10:54.040 People were listening to me. I was at the table, 0:10:54.559 --> 0:10:55.240 so to speak. 0:10:55.400 --> 0:11:00.240 And then in twenty seventeen Fall twenty seventeen, and I 0:11:00.320 --> 0:11:04.520 learned about a military contract that Google had secretively signed 0:11:04.520 --> 0:11:08.480 to build for the US drone program. And that was 0:11:08.760 --> 0:11:11.920 at a time and still today, when those systems were 0:11:11.960 --> 0:11:14.800 not purpose built right, they were not safe. 0:11:14.840 --> 0:11:18.440 There were ethical concerns we had. Sergey Brin, the co 0:11:18.520 --> 0:11:20.800 founder of Google, had made. 0:11:20.640 --> 0:11:23.120 Remarks that were very clear in the past about keeping 0:11:23.120 --> 0:11:26.840 away from military contracting. Given Google's business model, it's public interest. 0:11:26.920 --> 0:11:28.199 It serves the world, not just. 0:11:28.160 --> 0:11:30.360 The US business model, meaning that it knows a lot 0:11:30.400 --> 0:11:32.960 about people. Essentially, yeah that it is. 0:11:33.040 --> 0:11:36.160 Ultimately it's a surveillance company. It collects huge amounts of 0:11:36.280 --> 0:11:39.760 data about people, and then it creates models from that 0:11:39.880 --> 0:11:45.119 data that it sells to advertisers, and advertisers can leverage 0:11:45.160 --> 0:11:50.079 Google's models and ultimately the surveillance that they're built on 0:11:50.480 --> 0:11:53.680 in order to precision target their message and thus reach 0:11:53.800 --> 0:11:56.800 more customers. And that remains the core of the business model, 0:11:56.800 --> 0:11:58.960 that remains the core of the Internet economy. 0:11:59.480 --> 0:12:01.920 And putting that type. 0:12:01.640 --> 0:12:05.360 Of information at the service of one or another government 0:12:05.720 --> 0:12:08.400 is in my view, it is dangerous. And the first 0:12:08.400 --> 0:12:10.480 thing I did is I think start a signal thread 0:12:10.480 --> 0:12:12.040 with some people and say what should we do? 0:12:12.559 --> 0:12:14.800 This was Project Maven. It was to do with helping 0:12:14.800 --> 0:12:18.920 the US government target enemy combatants on the battlefield. 0:12:18.920 --> 0:12:23.160 Though well, it was building surveillance and targeting AI for 0:12:23.679 --> 0:12:24.600 the drone program. 0:12:24.679 --> 0:12:26.760 And what was the result of me used so talking 0:12:26.760 --> 0:12:29.679 to people on a signal. What happened next? 0:12:30.200 --> 0:12:33.120 You know, it grew and it grew, and then I 0:12:33.160 --> 0:12:37.560 wrote an open letter. That letter got signed by thousands 0:12:37.559 --> 0:12:40.520 of people. Eventually someone leaked it to the New York Times. 0:12:41.120 --> 0:12:46.000 It snowballed, it grew into a movement, and ultimately Google 0:12:46.160 --> 0:12:49.479 dropped that contract. Now you know, these efforts are ongoing. 0:12:49.960 --> 0:12:53.280 I want to be clear that this was a fairly 0:12:53.320 --> 0:12:57.280 different time. I think now there's a lot more comfort 0:12:57.320 --> 0:13:00.440 with military technology, but I think those issue shoes have 0:13:00.559 --> 0:13:03.720 yet to be addressed. The issues of the danger of 0:13:04.280 --> 0:13:10.480 yoking the interests of any one country irrespective to a 0:13:10.679 --> 0:13:13.560 massive surveillance apparatus, the types of which we haven't seen 0:13:13.600 --> 0:13:16.199 in human history. That's a real concern, and I think 0:13:16.200 --> 0:13:19.760 that is a concern across political spectrum. We should all 0:13:19.800 --> 0:13:23.000 be concerned about that, and frankly, I think it speaks 0:13:23.040 --> 0:13:26.959 to the need to question that form altogether. The form 0:13:27.000 --> 0:13:30.440 of sort of the surveillance business model and the collection 0:13:30.600 --> 0:13:36.239 of massive amounts of information that can be used to harm, control, oppress, 0:13:36.480 --> 0:13:36.959 et cetera. 0:13:37.240 --> 0:13:40.840 How did you end up finally deciding what's time to leave? 0:13:41.880 --> 0:13:45.280 There's not one answer to that, you know, I realized 0:13:45.280 --> 0:13:47.679 pretty quickly that I had poked a bear. I had 0:13:47.679 --> 0:13:50.000 a huge amount of support inside the company. 0:13:50.040 --> 0:13:53.600 I still do. I don't think anyone's saying I'm wrong. 0:13:53.679 --> 0:13:55.720 I mean, there are twenty thousand employees at one point 0:13:55.840 --> 0:13:59.000 right involved in Yeah, and it's a huge no. I mean, 0:14:00.120 --> 0:14:01.319 the employees are there. 0:14:01.800 --> 0:14:05.800 At that point. I think it was like two hundred thousand. Yeah, 0:14:05.840 --> 0:14:07.160 So it's it's significant. 0:14:07.200 --> 0:14:11.440 It was the largest labor action in tech history, which 0:14:11.520 --> 0:14:15.439 speaks not to the militancy of people who work in tech, 0:14:15.960 --> 0:14:18.480 but I think speaks to the fact that this was 0:14:18.520 --> 0:14:21.280 really a rupture of a lot of discomfort that had 0:14:21.280 --> 0:14:24.360 been building for a long time. Right, Like, people. 0:14:24.040 --> 0:14:27.680 Who often got into this industry because of their ethical compass, 0:14:27.680 --> 0:14:30.360 because they really, you know, wanted to quote unquote change 0:14:30.440 --> 0:14:34.440 the world, were feeling betrayed and feeling like their hands 0:14:34.440 --> 0:14:36.400 were no longer on the rudder in a way that 0:14:36.440 --> 0:14:37.720 they felt comfortable. 0:14:37.320 --> 0:14:43.600 With coming up. We discussed why signal broke through the noise, 0:14:44.120 --> 0:14:44.560 stay with. 0:14:44.600 --> 0:14:53.400 Us, so you poked the bear and it became time 0:14:53.440 --> 0:14:56.080 to leave. I mean, not for nothing. The kind of 0:14:56.360 --> 0:14:58.880 one of the wedge issues was this use of Google's 0:14:58.880 --> 0:15:02.920 technology for military purposes. And I think twenty twenty four 0:15:03.040 --> 0:15:05.080 was the year. I mean, of course we will remember, 0:15:05.720 --> 0:15:08.480 you know, the strikes, the drone strikes during the Obama years. 0:15:08.480 --> 0:15:11.720 And this is not per se a new phenomenon, but 0:15:11.840 --> 0:15:16.240 something about Ukraine and Gaza has really elevated into mainstream 0:15:16.280 --> 0:15:20.560 consciousness what it means to have autonomous weapon systems. 0:15:20.880 --> 0:15:24.080 Yeah, I think the environment has changed a lot. Look 0:15:24.160 --> 0:15:27.040 on one side, there's a very real argument that I'm 0:15:27.040 --> 0:15:33.480 deeply sympathetic to, like military infrastructure and technology is often outdated. 0:15:33.520 --> 0:15:39.760 They're amortizing yanky old systems that are not up to snuff. 0:15:40.160 --> 0:15:44.400 Logistics and processes are threadbare, all of that being true, 0:15:45.040 --> 0:15:47.880 and some of the new technologies would make that a 0:15:47.880 --> 0:15:53.840 lot easier. On the other hand, I think we're often 0:15:53.920 --> 0:15:57.000 not looking at these dangers. What does it mean to 0:15:57.040 --> 0:16:00.360 automate this process? What does it mean to predicate kate? 0:16:00.840 --> 0:16:06.560 Many of these technologies on surveillance data on infrastructure that 0:16:06.680 --> 0:16:09.840 is ultimately controlled by a handful of companies based in 0:16:09.880 --> 0:16:13.240 the US. How do we make sure the very real 0:16:13.360 --> 0:16:18.560 dangers of such reliance are actually surfaced and quick fixes 0:16:18.720 --> 0:16:24.440 to bad practices accrued over many, many years of grifty 0:16:24.520 --> 0:16:29.320 military contracting don't come at the expense of, you know, 0:16:29.880 --> 0:16:33.400 civilian life. Don't come at the expense of handing the 0:16:33.480 --> 0:16:36.920 keys to military operations to for profit companies based in 0:16:36.920 --> 0:16:37.360 the US. 0:16:38.120 --> 0:16:41.280 And you talked about Project Lavender and your Helmut Schmidt speech, 0:16:41.840 --> 0:16:44.120 which I found fascinating. Why do you think it was 0:16:44.120 --> 0:16:45.240 important to start there? 0:16:47.080 --> 0:16:51.600 Well, Lavender, Gospel and Where's Daddy are three systems that 0:16:52.440 --> 0:16:58.440 were revealed to be used in Gaza by the Israeli 0:16:58.560 --> 0:17:07.120 military to identify targets and then, you know, basically kill them. 0:17:07.960 --> 0:17:11.520 I looked at those because they are happening now, and 0:17:11.600 --> 0:17:16.800 because they were the type of danger that the thousands 0:17:16.840 --> 0:17:20.399 of people who were pushing back on Google's role in 0:17:20.440 --> 0:17:26.119 military contracting were raising as a hypothetical, and because I 0:17:26.160 --> 0:17:29.760 think it shows the way that the logics of the 0:17:29.800 --> 0:17:35.640 Obama era drone war have been supercharged and exacerbated by 0:17:35.880 --> 0:17:41.000 the introduction of these automated AI systems. So for folks 0:17:41.040 --> 0:17:44.159 who might not remember, the drone war was the theater 0:17:44.680 --> 0:17:48.119 to use the military term in which the signature strike 0:17:48.240 --> 0:17:53.760 was introduced, and the signature strike was it's killing people 0:17:53.840 --> 0:17:57.639 based on their data fingerprint, not based on any real 0:17:57.720 --> 0:18:00.800 knowledge about who they are, what they've done, or anything 0:18:00.840 --> 0:18:04.800 that would resemble what we think of as evidence. So, 0:18:05.560 --> 0:18:11.680 because someone visited five locations that are flagged as terrorist locations, 0:18:11.760 --> 0:18:15.880 because they have family here, because they are in these 0:18:16.000 --> 0:18:20.280 four group chats, whatever it is, I'm making a possible data. 0:18:20.320 --> 0:18:23.000 But you could model what a terrorist looks like in 0:18:23.040 --> 0:18:26.679 any way you want. And then if if my data 0:18:26.720 --> 0:18:32.080 patterns look similar enough to that model, you deem me 0:18:32.280 --> 0:18:35.920 to be a terrorist and you kill me. And you 0:18:36.000 --> 0:18:38.640 know it is exactly the logic of ad targeting. 0:18:39.040 --> 0:18:39.240 Right. 0:18:39.560 --> 0:18:44.160 I scroll through Instagram. I see an ad for athletic greens. 0:18:44.960 --> 0:18:46.760 Because you know they have my credit card data. You 0:18:46.760 --> 0:18:49.440 buy healthy stuff. You're this age group, you live in 0:18:49.480 --> 0:18:52.520 New York, whatever it is, we assume you will buy 0:18:52.920 --> 0:18:55.200 athletic greens or be likely to click on this ad 0:18:55.240 --> 0:19:00.840 and will get paid. It's that but for death and 0:19:00.920 --> 0:19:07.080 the lavender, Where's Daddy? And gospel systems are basically that 0:19:07.760 --> 0:19:10.600 for death supercharged. 0:19:12.840 --> 0:19:17.480 And is it a coincidence the resemblance between how AD 0:19:17.480 --> 0:19:21.680 targeting works and how these signature strikes on the battlefield 0:19:22.119 --> 0:19:26.080 or in war work. Was the military inspired in some 0:19:26.200 --> 0:19:29.080 way by the ad industry or what's the kind of 0:19:29.080 --> 0:19:30.119 source of the connections. 0:19:31.840 --> 0:19:34.439 I wouldn't be surprised. But then again, you know, it 0:19:34.480 --> 0:19:36.639 wasn't just the ad industry. You know, there was an 0:19:36.680 --> 0:19:40.840 Obama era and kind of a neoliberal faith in data. 0:19:41.359 --> 0:19:44.720 Now there wasn't much of a definition of what data is, 0:19:44.760 --> 0:19:48.560 but the idea that data was more objective, less fallible, 0:19:49.160 --> 0:19:53.200 more scientific. These are almost like mythologized versions of data, 0:19:53.960 --> 0:19:56.919 and that if we relied on the data instead of 0:19:57.240 --> 0:20:02.560 subjective decision making, we would re each determinations that were better, 0:20:02.640 --> 0:20:07.359 We would make the right choices. Now, part of my 0:20:07.359 --> 0:20:10.680 background is I came up doing large scale measurement system 0:20:10.720 --> 0:20:12.359 so I was in the business of making data. 0:20:12.680 --> 0:20:14.439 That's interesting. I mean, I just want to pause on that, 0:20:14.480 --> 0:20:18.160 because that's an interesting way. I've never heard it expressed before. 0:20:18.160 --> 0:20:21.080 But essentially making data equals measuring stuff. 0:20:21.920 --> 0:20:25.960 Yeah, data is a very rough proxy for a complex reality. 0:20:26.200 --> 0:20:27.600 Right, So you. 0:20:27.520 --> 0:20:32.320 Figure out, like, what's the way we're gonna create those proxies. 0:20:32.359 --> 0:20:34.040 How are we going to measure it? Right, whether we 0:20:34.119 --> 0:20:38.679 measure human emotion, or measure the timing of street lights, 0:20:38.800 --> 0:20:41.080 or measure the temperature in the morning based on a 0:20:41.119 --> 0:20:44.080 thermometer that's calibrated in a certain way. We log all 0:20:44.119 --> 0:20:46.800 of those as become data, right, And of course those 0:20:46.840 --> 0:20:50.480 methodologies are generally created by people with an interest in 0:20:50.520 --> 0:20:52.840 answering a certain set of questions, maybe not another set 0:20:52.840 --> 0:20:56.320 of questions. In the case of Google or another company, 0:20:56.520 --> 0:21:00.760 they were interested in measuring consumer behavior in servis of 0:21:01.600 --> 0:21:05.119 giving advertisers access to a market. This is, in effect, 0:21:05.200 --> 0:21:08.439 like a rough proxy created to answer very particular questions 0:21:08.480 --> 0:21:12.000 by a very particular industry at a very particular historical conjuncture, 0:21:12.760 --> 0:21:15.800 and it is then leveraged for all these other things. 0:21:15.840 --> 0:21:20.920 And I think shrouding that data construction process in sort 0:21:20.960 --> 0:21:24.679 of scientific language, you know, assuming that like data is 0:21:24.720 --> 0:21:29.400 simply a synonym for objective fact, that's a very convenient 0:21:30.119 --> 0:21:33.359 way of strouding some of those intentions, right, Shrouding the 0:21:33.560 --> 0:21:36.960 distinction between those who get to measure and thus get 0:21:37.000 --> 0:21:40.120 to decide, and those who are measured and are thus 0:21:40.200 --> 0:21:41.800 subject to those decisions. 0:21:42.400 --> 0:21:44.920 I mean, twenty twenty four was the year of this 0:21:45.920 --> 0:21:51.359 hypothetical of what was raised at Google becoming real in Gaza. 0:21:51.560 --> 0:21:54.000 It was also the year that two Nobel Prizes, one 0:21:54.040 --> 0:21:56.119 in chemistry and one in physics, went to one former 0:21:56.160 --> 0:22:00.560 and one current Google employee, and both in the realm 0:22:00.760 --> 0:22:05.480 of pushing fundamental science forward. So how do you see 0:22:05.520 --> 0:22:07.760 the moment in an AI and both this kind of 0:22:07.880 --> 0:22:11.960 enormous hope and something drug discovery and the obvious peril 0:22:12.080 --> 0:22:14.200 in getting people based on their metadata. 0:22:15.320 --> 0:22:18.960 Well, I think there's a lot of things AI could 0:22:19.000 --> 0:22:22.879 do very well right. Drug discovery is one, it's very interesting, 0:22:23.000 --> 0:22:25.200 and the idea of being able to create drugs for 0:22:25.640 --> 0:22:29.320 things that affect smaller numbers of people thus aren't generally incentivized. 0:22:29.359 --> 0:22:32.200 All of that, like, yes, let's leverage what we. 0:22:32.200 --> 0:22:34.399 Have to make life as beautiful as possible for as 0:22:34.400 --> 0:22:38.720 many people as possible. Amen, The issue comes down to 0:22:38.760 --> 0:22:42.720 the political economy of AI and what we're actually talking about. 0:22:43.359 --> 0:22:47.400 And I think when we look at the AI industry 0:22:47.480 --> 0:22:50.919 right now, and when we look at the technologies that 0:22:51.000 --> 0:22:55.000 the Nobel Prize was awarded for either creating or leveraging, 0:22:56.440 --> 0:23:01.200 we begin to recognize that AI is an incredibly centralized 0:23:01.400 --> 0:23:07.720 set of technologies. It relies on huge amounts of computational resources, 0:23:08.280 --> 0:23:11.680 and then there is data that's required to train these models. 0:23:11.720 --> 0:23:14.040 And so the paradigm we're in right now is the 0:23:14.080 --> 0:23:17.400