Speaker 1: Thanks for tuney into tech Stuff. If you don't recognize
Speaker 1: my voice, my name is Oz Valoshian and I'm here
Speaker 1: because the inimitable Jonathan Strickland has passed the baton to
Speaker 1: Cara Price and myself to host Tech Stuff. The show
Speaker 1: will remain your home for all things tech, and all
Speaker 1: the old episodes will remain available in this feed. Thanks
Speaker 1: for listening. Welcome to Tech Stuff. I'm Os Valoshian and
Speaker 1: I'm Cara Price.
Speaker 2: And this is the story our weekly episode, dropping each Wednesday,
Speaker 2: sharing an in depth conversation with a fascinating person in
Speaker 2: and around tech. This week, Oz tells me he's bringing
Speaker 2: me a conversation with Meredith Whittaker.
Speaker 1: That's right. Meredith leads the Signal Foundation, which is a
Speaker 1: nonprofit that oversees the encrypted messaging app Signal, beloved by journalists,
Speaker 1: deal makers, drug dealers, and privacy buffs alike.
Speaker 2: It takes wanting to keep a really really really big
Speaker 2: secret to get on Signal, but we should also think
Speaker 2: about everything we say as a really big secret. If
Speaker 2: you can who has access to all of our information.
Speaker 1: One hundred percent, Well, you don't sometimes know what's a
Speaker 1: really really big secret until it's too late. Meredith has
Speaker 1: a fascinating background. She started at Google back in the
Speaker 1: day when it famously boasted that don't be Evil slogan,
Speaker 1: and she actually started as a temp in a kind
Speaker 1: of customer service adjacent gig that was despite being a
Speaker 1: Berkeley grad in literature and rhetoric. Eventually she worked her
Speaker 1: way up into leadership, founding Google Open Research Group and
Speaker 1: then separately founding something called the AI Now Institute at NYU.
Speaker 1: And after she left Google, she became head of Signal
Speaker 1: in twenty twenty two. But the circumstances of her departure
Speaker 1: from Google were memorable, to say the least.
Speaker 2: I think it's fair to say that she got on
Speaker 2: everyone's radar back in twenty eighteen when she led the
Speaker 2: walkouts at Google over the experience of women in tech
Speaker 2: and at Google specifically over things like pay inequality and
Speaker 2: the handling of sexual harassment at the company.
Speaker 1: And then there's also Project Maven, which was Google's decision
Speaker 1: to take part in a Department of Defense program to
Speaker 1: use machine learning and data for battlefield target identification, which
Speaker 1: was very very controversial internally and which in part is
Speaker 1: what led her to her role today at signal. Merediths
Speaker 1: won a prize in Germany last year called the Helmut
Speaker 1: Schmidt Future Award, where she was honored to quote her
Speaker 1: commitment to the development of AI technology oriented towards the
Speaker 1: common good. So we talk about that too, and why
Speaker 1: Meredith fundamentally describes herself as an optimist. But we start
Speaker 1: by geeking out over literature.
Speaker 2: Let's hear it.
Speaker 1: So I was reading your bio and one of the
Speaker 1: things that stuck out to me was your background as
Speaker 1: a English literature and rhetoric major in college. I was
Speaker 1: also in English major. As I looked at your kind
Speaker 1: of story, there's this arc from arriving at Google as
Speaker 1: a graduate in two thousand and six in the Don't
Speaker 1: Be Evil error, seeing all of these technologies developed, many
Speaker 1: of which went on to be kind of part of
Speaker 1: the foundation of what we now call AI, leading a
Speaker 1: protest movement and ultimately leaving and it made me think
Speaker 1: of Pilgrim's Progress or a buildings Room.
Speaker 3: Then yeah, I mean, maybe a Jeremiah is more accurate.
Speaker 4: So that was a joke for the four English literature
Speaker 4: majors who are listening. I think for me, I was
Speaker 4: very sincere. I still am very sincere, and I came
Speaker 4: in and took a lot of things at face value,
Speaker 4: in part because I had no idea about tech. I
Speaker 4: was stepping in and I was asking very basic questions.
Speaker 1: You did in two thousand and six.
Speaker 4: I mean, well, you know, it wasn't the money occupation
Speaker 4: when I got into Google. It was it was actually,
Speaker 4: it was a fairly lovely environment in some ways because
Speaker 4: a lot of the people there were people who had
Speaker 4: a special interest, like nerdy people like myself, but I
Speaker 4: was a book nerd, and they were sort of math
Speaker 4: and science nerds. And suddenly their special interest, their sweet
Speaker 4: passion that they spent so much time thinking about, was
Speaker 4: also profitable. It was also important for the world. It
Speaker 4: put them in a position of being a meaningful contributor.
Speaker 4: And there was a genuineness to that. And we're in
Speaker 4: an era now, of course, where you know, no one's
Speaker 4: mom is saying be a doctor or a lawyer. They're
Speaker 4: saying be an engineer. And so it was, you know,
Speaker 4: it was simply a very different world.
Speaker 1: And what about don't be evil? When was that generation?
Speaker 3: Then?
Speaker 1: When was that abandoned? Was that part of the kind
Speaker 1: of discourse when you went to Google, that.
Speaker 4: Was generated before my time, and it was core to
Speaker 4: the discourse at Google. I think it was removed by
Speaker 4: lawyers in like twenty fifteen, twenty sixteen, I don't actually
Speaker 4: remember the date, but look, it's a slogan.
Speaker 3: You know.
Speaker 4: The structure of Google was still up into the right
Speaker 4: forever so profits and growth worthy objective functions. That's the
Speaker 4: sort of perpetual motion machine that Google adhere. However, I
Speaker 4: do think don't Be Evil had some power in that
Speaker 4: it was a touchstone that we all shared where socially
Speaker 4: awkward people are people who might not quite understand how
Speaker 4: to formulate a discomfort could kind of point to it
Speaker 4: and say like, hey, this is making me think that
Speaker 4: we should examine this because we said don't be evil.
Speaker 4: And most of this, you know, throughout Google's history was
Speaker 4: at a time where let's say, the horizons that would
Speaker 4: challenge that were very far in the distance. So you
Speaker 4: could still make a lot of quarterly returns doing a
Speaker 4: lot of things that were understood and we can debate
Speaker 4: that as as good right, and things like building a
Speaker 4: search engine for the Chinese market that would include surveillance
Speaker 4: functions for the government was not something you even had
Speaker 4: to consider because there was so much low hanging fruit
Speaker 4: in terms of continuing to grow, continuing to profit. And
Speaker 4: so I think that combination of like it was, it
Speaker 4: was sincerely held, and that the actual temptation to do
Speaker 4: the things that really got into that sort of hot
Speaker 4: water potential evil were not on the table at that
Speaker 4: point because there was so much else that could be done.
Speaker 4: And you know, of course, the ratchet turns, the ratchet turns,
Speaker 4: the ratchet turns, and then suddenly those horizons are much closer.
Speaker 4: The company is swollen by orders of magnitude, and that
Speaker 4: little slogan is quietly removed by a team of lawyers
Speaker 4: who probably bought some summer houses with the fees.
Speaker 1: Did you believe that slogan? And was there a moment
Speaker 1: along the way where you definitively started not to believe it.
Speaker 4: I liked having that slogan as a tool. Let me
Speaker 4: say that, right.
Speaker 3: It was useful. It did work in rooms.
Speaker 4: It you know, allowed us to frame debates in a
Speaker 4: way that had at at least as their reference port
Speaker 4: a core and ethical consideration. Like I was young, there
Speaker 4: was about a million different learning curves I was scaling
Speaker 4: at once. Yes, what does it mean to have a
Speaker 4: real job.
Speaker 2: Right.
Speaker 3: I didn't come from this class.
Speaker 4: I didn't have familiarity with these cultures or environments because
Speaker 4: I'd literally never been in an environment that looked anything
Speaker 4: like you know what my family still jokingly refers to
Speaker 4: as a business job.
Speaker 3: Right.
Speaker 4: I didn't know what was normal and what was not normal.
Speaker 4: So for me, Google was normal, and so I just
Speaker 4: want to be care Like. I don't know if I
Speaker 4: believed it or not. I found it useful and I
Speaker 4: saw it do work in the world, and I liked
Speaker 4: the idea, Like, it sounds great that we're giving people
Speaker 4: ads that match their interests. What could be wrong with
Speaker 4: that We're organizing the world's information. Wow, all of this
Speaker 4: is cool.
Speaker 3: Like I was like, Okay, what is tech? What is Google? Hey,
Speaker 3: sit me down and tell me what is information? I
Speaker 3: was a master at asking the dumbest questions, but sometimes
Speaker 3: they would unlock whole world for me, and then I
Speaker 3: would realize, like, oh, information is just websites. Okay, but
Speaker 3: you can't make a website if you don't have a computer,
Speaker 3: and you have to be able to code HTML markup
Speaker 3: to do that. Okay, So it's more limited than I thought. Okay,
Speaker 3: well what is an index? Oh, it's ranking it. But
Speaker 3: isn't that a value judgment?
Speaker 2: Well?
Speaker 4: I guess it's not if it's an algorithm, but like
Speaker 4: who makes the algorithm? And so it was butting my
Speaker 4: head up against what are kind of dumb, very basic
Speaker 4: questions and sort of sensing an insufficiency of the responses,
Speaker 4: and like, you know, over almost twenty years now, continuing
Speaker 4: that process of trying to, you know, just figure out
Speaker 4: like what are we doing here?
Speaker 3: And how do I understand it? And that was basically
Speaker 3: my education.
Speaker 1: So fust forward, you know, thirteen years from two thousand
Speaker 1: and six to twenty nineteen. How did you get to
Speaker 1: that point where it became impossible for you to stay?
Speaker 3: I don't have a pat answer, But how did I
Speaker 3: get to that point? Well?
Speaker 4: Again, I was a sincere person and I had done
Speaker 4: a lot in my career. I had a lot of
Speaker 4: opportunities and then I took them, and you know, I
Speaker 4: built a research group inside of Google.
Speaker 3: I was involved in.
Speaker 4: Efforts to spread privacy and security technologies across the tech ecosystem.
Speaker 4: I was a champion for privacy within Google. I had
Speaker 4: built you know, open source measurement infrastructures to help inform
Speaker 4: the net neutrality debate. I had always tried to be
Speaker 4: on the side of doing social good, to.
Speaker 3: Be very bland about it.
Speaker 4: Yeah, And I had been able to do very very
Speaker 4: well doing that. And at the point where I became frustrated,
Speaker 4: I had already established the AI Now Institute, co founding
Speaker 4: that at NYU, looking at some of the present day
Speaker 4: real term harms and social implications of AI technologies. This
Speaker 4: was twenty sixteen. I had become known through the academic
Speaker 4: world and the technical world, and particularly within Google itself
Speaker 4: as an authority and a speaker and an expert on
Speaker 4: those issues. So talking about some of the harms that
Speaker 4: these technologies could cause, criticizing within, essentially criticizing within. I
Speaker 4: saw myself as a resource, and actually many many people did.
Speaker 4: I would be brought in when teams were struggling with things,
Speaker 4: I would advise them. I would often give academic lectures
Speaker 4: that went against the party line at Google, but we're
Speaker 4: very well cited, were very empirically documented, and I had
Speaker 4: felt that I was making some headway right.
Speaker 3: People were listening to me. I was at the table,
Speaker 3: so to speak.
Speaker 4: And then in twenty seventeen Fall twenty seventeen, and I
Speaker 4: learned about a military contract that Google had secretively signed
Speaker 4: to build for the US drone program. And that was
Speaker 4: at a time and still today, when those systems were
Speaker 4: not purpose built right, they were not safe.
Speaker 3: There were ethical concerns we had. Sergey Brin, the co
Speaker 3: founder of Google, had made.
Speaker 4: Remarks that were very clear in the past about keeping
Speaker 4: away from military contracting. Given Google's business model, it's public interest.
Speaker 4: It serves the world, not just.
Speaker 1: The US business model, meaning that it knows a lot
Speaker 1: about people. Essentially, yeah that it is.
Speaker 4: Ultimately it's a surveillance company. It collects huge amounts of
Speaker 4: data about people, and then it creates models from that
Speaker 4: data that it sells to advertisers, and advertisers can leverage
Speaker 4: Google's models and ultimately the surveillance that they're built on
Speaker 4: in order to precision target their message and thus reach
Speaker 4: more customers. And that remains the core of the business model,
Speaker 4: that remains the core of the Internet economy.
Speaker 3: And putting that type.
Speaker 4: Of information at the service of one or another government
Speaker 4: is in my view, it is dangerous. And the first
Speaker 4: thing I did is I think start a signal thread
Speaker 4: with some people and say what should we do?
Speaker 1: This was Project Maven. It was to do with helping
Speaker 1: the US government target enemy combatants on the battlefield.
Speaker 4: Though well, it was building surveillance and targeting AI for
Speaker 4: the drone program.
Speaker 1: And what was the result of me used so talking
Speaker 1: to people on a signal. What happened next?
Speaker 4: You know, it grew and it grew, and then I
Speaker 4: wrote an open letter. That letter got signed by thousands
Speaker 4: of people. Eventually someone leaked it to the New York Times.
Speaker 4: It snowballed, it grew into a movement, and ultimately Google
Speaker 4: dropped that contract. Now you know, these efforts are ongoing.
Speaker 4: I want to be clear that this was a fairly
Speaker 4: different time. I think now there's a lot more comfort
Speaker 4: with military technology, but I think those issue shoes have
Speaker 4: yet to be addressed. The issues of the danger of
Speaker 4: yoking the interests of any one country irrespective to a
Speaker 4: massive surveillance apparatus, the types of which we haven't seen
Speaker 4: in human history. That's a real concern, and I think
Speaker 4: that is a concern across political spectrum. We should all
Speaker 4: be concerned about that, and frankly, I think it speaks
Speaker 4: to the need to question that form altogether. The form
Speaker 4: of sort of the surveillance business model and the collection
Speaker 4: of massive amounts of information that can be used to harm, control, oppress,
Speaker 4: et cetera.
Speaker 1: How did you end up finally deciding what's time to leave?
Speaker 4: There's not one answer to that, you know, I realized
Speaker 4: pretty quickly that I had poked a bear. I had
Speaker 4: a huge amount of support inside the company.
Speaker 3: I still do. I don't think anyone's saying I'm wrong.
Speaker 1: I mean, there are twenty thousand employees at one point
Speaker 1: right involved in Yeah, and it's a huge no. I mean,
Speaker 1: the employees are there.
Speaker 4: At that point. I think it was like two hundred thousand. Yeah,
Speaker 4: So it's it's significant.
Speaker 3: It was the largest labor action in tech history, which
Speaker 3: speaks not to the militancy of people who work in tech,
Speaker 3: but I think speaks to the fact that this was
Speaker 3: really a rupture of a lot of discomfort that had
Speaker 3: been building for a long time. Right, Like, people.
Speaker 4: Who often got into this industry because of their ethical compass,
Speaker 4: because they really, you know, wanted to quote unquote change
Speaker 4: the world, were feeling betrayed and feeling like their hands
Speaker 4: were no longer on the rudder in a way that
Speaker 4: they felt comfortable.
Speaker 2: With coming up. We discussed why signal broke through the noise,
Speaker 2: stay with.
Speaker 1: Us, so you poked the bear and it became time
Speaker 1: to leave. I mean, not for nothing. The kind of
Speaker 1: one of the wedge issues was this use of Google's
Speaker 1: technology for military purposes. And I think twenty twenty four
Speaker 1: was the year. I mean, of course we will remember,
Speaker 1: you know, the strikes, the drone strikes during the Obama years.
Speaker 1: And this is not per se a new phenomenon, but
Speaker 1: something about Ukraine and Gaza has really elevated into mainstream
Speaker 1: consciousness what it means to have autonomous weapon systems.
Speaker 4: Yeah, I think the environment has changed a lot. Look
Speaker 4: on one side, there's a very real argument that I'm
Speaker 4: deeply sympathetic to, like military infrastructure and technology is often outdated.
Speaker 4: They're amortizing yanky old systems that are not up to snuff.
Speaker 4: Logistics and processes are threadbare, all of that being true,
Speaker 4: and some of the new technologies would make that a
Speaker 4: lot easier. On the other hand, I think we're often
Speaker 4: not looking at these dangers. What does it mean to
Speaker 4: automate this process? What does it mean to predicate kate?
Speaker 4: Many of these technologies on surveillance data on infrastructure that
Speaker 4: is ultimately controlled by a handful of companies based in
Speaker 4: the US. How do we make sure the very real
Speaker 4: dangers of such reliance are actually surfaced and quick fixes
Speaker 4: to bad practices accrued over many, many years of grifty
Speaker 4: military contracting don't come at the expense of, you know,
Speaker 4: civilian life. Don't come at the expense of handing the
Speaker 4: keys to military operations to for profit companies based in
Speaker 4: the US.
Speaker 1: And you talked about Project Lavender and your Helmut Schmidt speech,
Speaker 1: which I found fascinating. Why do you think it was
Speaker 1: important to start there?
Speaker 4: Well, Lavender, Gospel and Where's Daddy are three systems that
Speaker 4: were revealed to be used in Gaza by the Israeli
Speaker 4: military to identify targets and then, you know, basically kill them.
Speaker 4: I looked at those because they are happening now, and
Speaker 4: because they were the type of danger that the thousands
Speaker 4: of people who were pushing back on Google's role in
Speaker 4: military contracting were raising as a hypothetical, and because I
Speaker 4: think it shows the way that the logics of the
Speaker 4: Obama era drone war have been supercharged and exacerbated by
Speaker 4: the introduction of these automated AI systems. So for folks
Speaker 4: who might not remember, the drone war was the theater
Speaker 4: to use the military term in which the signature strike
Speaker 4: was introduced, and the signature strike was it's killing people
Speaker 4: based on their data fingerprint, not based on any real
Speaker 4: knowledge about who they are, what they've done, or anything
Speaker 4: that would resemble what we think of as evidence. So,
Speaker 4: because someone visited five locations that are flagged as terrorist locations,
Speaker 4: because they have family here, because they are in these
Speaker 4: four group chats, whatever it is, I'm making a possible data.
Speaker 4: But you could model what a terrorist looks like in
Speaker 4: any way you want. And then if if my data
Speaker 4: patterns look similar enough to that model, you deem me
Speaker 4: to be a terrorist and you kill me. And you
Speaker 4: know it is exactly the logic of ad targeting.
Speaker 3: Right.
Speaker 4: I scroll through Instagram. I see an ad for athletic greens.
Speaker 4: Because you know they have my credit card data. You
Speaker 4: buy healthy stuff. You're this age group, you live in
Speaker 4: New York, whatever it is, we assume you will buy
Speaker 4: athletic greens or be likely to click on this ad
Speaker 4: and will get paid. It's that but for death and
Speaker 4: the lavender, Where's Daddy? And gospel systems are basically that
Speaker 4: for death supercharged.
Speaker 1: And is it a coincidence the resemblance between how AD
Speaker 1: targeting works and how these signature strikes on the battlefield
Speaker 1: or in war work. Was the military inspired in some
Speaker 1: way by the ad industry or what's the kind of
Speaker 1: source of the connections.
Speaker 4: I wouldn't be surprised. But then again, you know, it
Speaker 4: wasn't just the ad industry. You know, there was an
Speaker 4: Obama era and kind of a neoliberal faith in data.
Speaker 4: Now there wasn't much of a definition of what data is,
Speaker 4: but the idea that data was more objective, less fallible,
Speaker 4: more scientific. These are almost like mythologized versions of data,
Speaker 4: and that if we relied on the data instead of
Speaker 4: subjective decision making, we would re each determinations that were better,
Speaker 4: We would make the right choices. Now, part of my
Speaker 4: background is I came up doing large scale measurement system
Speaker 4: so I was in the business of making data.
Speaker 1: That's interesting. I mean, I just want to pause on that,
Speaker 1: because that's an interesting way. I've never heard it expressed before.
Speaker 1: But essentially making data equals measuring stuff.
Speaker 4: Yeah, data is a very rough proxy for a complex reality.
Speaker 3: Right, So you.
Speaker 4: Figure out, like, what's the way we're gonna create those proxies.
Speaker 4: How are we going to measure it? Right, whether we
Speaker 4: measure human emotion, or measure the timing of street lights,
Speaker 4: or measure the temperature in the morning based on a
Speaker 4: thermometer that's calibrated in a certain way. We log all
Speaker 4: of those as become data, right, And of course those
Speaker 4: methodologies are generally created by people with an interest in
Speaker 4: answering a certain set of questions, maybe not another set
Speaker 4: of questions. In the case of Google or another company,
Speaker 4: they were interested in measuring consumer behavior in servis of
Speaker 4: giving advertisers access to a market. This is, in effect,
Speaker 4: like a rough proxy created to answer very particular questions
Speaker 4: by a very particular industry at a very particular historical conjuncture,
Speaker 4: and it is then leveraged for all these other things.
Speaker 4: And I think shrouding that data construction process in sort
Speaker 4: of scientific language, you know, assuming that like data is
Speaker 4: simply a synonym for objective fact, that's a very convenient
Speaker 4: way of strouding some of those intentions, right, Shrouding the
Speaker 4: distinction between those who get to measure and thus get
Speaker 4: to decide, and those who are measured and are thus
Speaker 4: subject to those decisions.
Speaker 1: I mean, twenty twenty four was the year of this
Speaker 1: hypothetical of what was raised at Google becoming real in Gaza.
Speaker 1: It was also the year that two Nobel Prizes, one
Speaker 1: in chemistry and one in physics, went to one former
Speaker 1: and one current Google employee, and both in the realm
Speaker 1: of pushing fundamental science forward. So how do you see
Speaker 1: the moment in an AI and both this kind of
Speaker 1: enormous hope and something drug discovery and the obvious peril
Speaker 1: in getting people based on their metadata.
Speaker 4: Well, I think there's a lot of things AI could
Speaker 4: do very well right. Drug discovery is one, it's very interesting,
Speaker 4: and the idea of being able to create drugs for
Speaker 4: things that affect smaller numbers of people thus aren't generally incentivized.
Speaker 3: All of that, like, yes, let's leverage what we.
Speaker 4: Have to make life as beautiful as possible for as
Speaker 4: many people as possible. Amen, The issue comes down to
Speaker 4: the political economy of AI and what we're actually talking about.
Speaker 4: And I think when we look at the AI industry
Speaker 4: right now, and when we look at the technologies that
Speaker 4: the Nobel Prize was awarded for either creating or leveraging,
Speaker 4: we begin to recognize that AI is an incredibly centralized
Speaker 4: set of technologies. It relies on huge amounts of computational resources,
Speaker 4: and then there is data that's required to train these models.
Speaker 4: And so the paradigm we're in right now is the
Speaker 4: bigger is better. The more chips, the more compute, the
Speaker 4: more data, and you combine those in whatever architecture and
Speaker 4: you create a model, and then that model is purportedly
Speaker 4: better performing and therefore better, and therefore we're advancing science.
Speaker 1: And that's a useful argument, I guess for very very
Speaker 1: large companies whose inherent logic is to grow larger. Right,
Speaker 1: I mean, if you Amazon or Googo, little Microsoft, you
Speaker 1: what is there to do for growth other than popularizing
Speaker 1: this model of how the progress of AI works?
Speaker 4: Yeah, I mean it would be very bad for these companies'
Speaker 4: bottom lines if it turned out that much smaller models
Speaker 4: using much more bespoke data were in fact much better, right,
Speaker 4: Because they've thrown a lot of money behind this. Narratives
Speaker 4: we hear about tech, and the narratives we hear about
Speaker 4: progress are not always born of scientific and I think
Speaker 4: when you look at that, then you have to realize,
Speaker 4: like AI is not a it's not a product of
Speaker 4: scientific innovation that everyone could leverage and these guys just
Speaker 4: figured it out first, right, It's a product of this
Speaker 4: concentration of resources, and in fact, in the early twenty
Speaker 4: tens when the sort of current AI moment took off,
Speaker 4: the algorithms that were animating that boom were created in
Speaker 4: the late nineteen eighties. But what was new were the
Speaker 4: computational resources, the chips and the ability to use them
Speaker 4: in certain ways, and the data and the data is
Speaker 4: that product of this surveillance business model that Google, Meta
Speaker 4: at Alia sort of participated in and had built their
Speaker 4: infrastructures and businesses around. So of course, who has the
Speaker 4: access to those infrastructures, who can meaningfully apply these old
Speaker 4: algorithms and begin to fund research sort of optimizing them,
Speaker 4: building new approaches to this big data, big compute AI paradigm.
Speaker 4: It's the same surveillance platform companies, And so all of
Speaker 4: this comes back to the question of like, well, the
Speaker 4: Nobel is awarded for AI this year, don't we want
Speaker 4: better drug discoveries? And it's great, yes, we absolutely do,
Speaker 4: but if you look at the market conditions, it's unclear
Speaker 4: how that better drug discovery is actually going to lead
Speaker 4: to better health outcomes, and so who will the customers
Speaker 4: of that AI be?
Speaker 2: Right?
Speaker 4: Most likely it will be pharmaceutical companies, not altruistic organization.
Speaker 4: It'll be insurance companies who may actually want to limit care. Right,
Speaker 4: So you have a diagnostic algorithm, but it's not used
Speaker 4: to make sure that I'm given the care I need,
Speaker 4: even if it's expensive early so I live a long life.
Speaker 4: It's used to make sure I'm excluded from the pool
Speaker 4: so that I am.
Speaker 3: Not a cost going forward.
Speaker 4: So I think that's the lens that we need to
Speaker 4: put on some of this, because the excitement is very warranted.
Speaker 4: But in the world we have, it's pretty clear without
Speaker 4: other changes to our systems, that AI, given the cost,
Speaker 4: given the concentrated power, given the fact that these entities
Speaker 4: need to recoup massive investment, is not going to be
Speaker 4: an altruistic resource, but will likely be ramifying the power
Speaker 4: of actors who have already proven themselves pretty problematic in
Speaker 4: terms of fomenting public good so to speak.
Speaker 2: Coming up, we'll hear about Meredith Whitaker's speech upon winning
Speaker 2: the Helmet Schmidt Future Prize and why we should rethink
Speaker 2: our preconceptions about data stayed with us.
Speaker 1: Talk about Signal and not for nothing. Was a was
Speaker 1: the first place you went when you learned about Project Maven,
Speaker 1: and is now where you spend a lot more time.
Speaker 1: So for those who don't know what it is and
Speaker 1: how it works and why it's important, I mean, just
Speaker 1: tell us a bit about it.
Speaker 3: Yeah, Well, you know, Signal is incredible.
Speaker 4: I'm really honored to work here and it's been along
Speaker 4: on the journey and various capacities for a very long time.
Speaker 3: It is the.
Speaker 4: World's largest actually private communications network in an environment as
Speaker 4: we just discussed, where almost every one of our activities,
Speaker 4: digital and increasingly analog are surveilled in one way or
Speaker 4: another by a large tech company, government, or some ad mixture.
Speaker 4: So it's incredibly important infrastructure. We hear human rights workers
Speaker 4: rely on signal. Journalists rely on signal to protect their sources.
Speaker 4: We have militaries rely on signal to get communications out
Speaker 4: when they're necessary. We have board rooms, you know, most
Speaker 4: bour room chatter, most CEOs I meet use signal religiously.
Speaker 3: Governments use signals. So anywhere where the confidentiality of communications
Speaker 3: is necessary, certainly anywhere where the stakes of privacy are
Speaker 3: life or death. We know that people rely on Signal
Speaker 3: and our core encryption protocol was released in twenty thirteen
Speaker 3: and it has stood the test of time. It is
Speaker 3: now the gold standard across the industry.
Speaker 1: And it's used by WhatsApp as well.
Speaker 3: Right, yeah, it's used by WhatsApp.
Speaker 4: It's used across the tech industry because one of the
Speaker 4: things about encryption is it's very hard to get right.
Speaker 4: It's very easy to get wrong, and so if someone
Speaker 4: gets it right, you want to reuse that. You want
Speaker 4: to use that formula. You don't want to DIY because
Speaker 4: it's almost certain you'll have some small error in there,
Speaker 4: and when we're talking about life or death consequences, you
Speaker 4: can't afford that.
Speaker 1: But why if what seven Signal used the same open
Speaker 1: source code, am I so much safe for using Signal
Speaker 1: and WhatsApp?
Speaker 4: Because the Signal protocol is one slice of a very
Speaker 4: large stack, and WhatsApp uses that slice to protect what
Speaker 4: you say. But Signal protects way more than what you say.
Speaker 4: We protect who you talk to. We don't have any
Speaker 4: idea who's talking to whom. We don't know your profile name,
Speaker 4: we don't know your profile photo. We don't know your
Speaker 4: contact list, and it goes on and on and on.
Speaker 4: So everything we do we sweat the details in order
Speaker 4: to get as close to collecting as little data as possible.
Speaker 1: Those things that you're describing that Signal doesn't collector are
Speaker 1: the inputs to the signature strikes we were discussing earlier.
Speaker 3: Could be? Could be.
Speaker 4: I mean one of the issues with proprietary technology or
Speaker 4: classified information is we don't totally know, but it's that
Speaker 4: type of information that has been reported and mentioned as
Speaker 4: inputs to those strikes.
Speaker 1: Yeah, and you're not the CEO of Signal, you're the
Speaker 1: president of the Signal Foundation. Can you kind of explain
Speaker 1: why that is and what it means?
Speaker 4: Well, I think in this moment in the tech industry,
Speaker 4: that would be dangerous. Although it's not profits that we
Speaker 4: are opposed to, it is the particular incentive structure of
Speaker 4: the tech industry which ultimately you either collect data and
Speaker 4: monetize that you're creating models to sell ads as we discussed,
Speaker 4: or training AI models or selling its data brokers or
Speaker 4: what have you, so surveillance, or you provide sort of
Speaker 4: goods and services to those who do the picks and
Speaker 4: shovels of the surveillance industry. So you can imagine a
Speaker 4: board with a fiduciary duty governing a for profit signal.
Speaker 4: At some point that fiduciary duty is going to take
Speaker 4: precedence to the obsessive focus on privacy.
Speaker 3: So that's really the crux of it.
Speaker 4: And we're in an industry where consumer apps are quote
Speaker 4: unquote free, right there is not a norm of paying
Speaker 4: for these things upfront by the consumer. Is not for
Speaker 4: lack of innovation that we don't have many many more
Speaker 4: signals or signal like products that are focused on democracy,
Speaker 4: focused on privacy, focused on ensuring fundamental rights, focused on
Speaker 4: the public good. It's really because many of those things
Speaker 4: cannot be subject to the surveillance business model and thus
Speaker 4: don't really exist because capitalizing them is such an issue.
Speaker 1: You mentioned that you know there's a consumer expectation that
Speaker 1: tech is free, or at least free at the point
Speaker 1: of delivery. You also have heard a piece of Wired
Speaker 1: recently kind of arguing twenty twenty five might be but
Speaker 1: the beginning of the end of big tech. But I
Speaker 1: guess my question is big tech of so successfully kind
Speaker 1: of created this expectation on the consumer side that tech
Speaker 1: products are free and that essentially I pay for the
Speaker 1: product by allowing myself to be surveiled. Can there be
Speaker 1: a fundamental shift away from the big tech hegemony un
Speaker 1: lessen until consumers are willing to pay with money rather
Speaker 1: than data for infrastructure services.
Speaker 3: I mean, I absolutely believe there can be. This is
Speaker 3: what we have.
Speaker 4: Thirty years of this, twenty years of this. This is
Speaker 4: hyper novel in terms of human history. This is in
Speaker 4: no way natural. There's you know, there's a longer history
Speaker 4: of the sort of particular regulatory and political environment this
Speaker 4: came out of. We now have the FBI advising people
Speaker 4: not to send SMS messages following a massive hack that
Speaker 4: enabled China and I don't know who else to access
Speaker 4: telephony networks and surveil US citizens, including many high ranking citizens.
Speaker 4: And so we're in a geopolitically volatile moment where the
Speaker 4: lines between nations and jurisdictions and interests are getting a
Speaker 4: bit more brittle, a bit more crisp, and I think
Speaker 4: the imperative of moving away from this centralized surveillance tech
Speaker 4: business model is really becoming clear to almost everyone. And
Speaker 4: that was what animated the piece and wired you know,
Speaker 4: I would say it's more of a wish casting into
Speaker 4: the future than necessarily a prediction of what will happen.
Speaker 3: But insofar as like hope.
Speaker 4: Is an invitation for action, I really think there's a
Speaker 4: lot of creative work that could be done to undo
Speaker 4: that business model. Right, the question is really where do
Speaker 4: the resources come from? And that's a question we can
Speaker 4: begin to answer. Okay, are there endowments, Are there certain
Speaker 4: technologies like signal, like some of the cloud and core
Speaker 4: infrastructure that should be more openly governed that really, you know,
Speaker 4: the more dependent we are on these infrastructures, the more
Speaker 4: these infrastructures are controlled by you know, a small number
Speaker 4: of actors, the more perilous those single points of failure become.
Speaker 4: And I think the more attentive people are becoming to
Speaker 4: that peril, and the more appetite there is for solutions
Speaker 4: that may not have been on the table even a
Speaker 4: couple of years ago.
Speaker 1: Yeah, it is interesting. I mean I was a talk
Speaker 1: the other day and somebody's talking about cloud computing and
Speaker 1: how the metaphor kind of suggests that this kind of
Speaker 1: decentralized model and it's kind of around all of us.
Speaker 1: But like what cloud computing has actually meant it's this
Speaker 1: incredible kind of centralization of computational power in the hands
Speaker 1: of basically two nations and ten companies.
Speaker 4: Yep, yeah, exactly, and you know, two nations, but the
Speaker 4: US being dominant there it's Amazon, Microsoft, and Google have
Speaker 4: seventy percent of the global market seven zero insane. So
Speaker 4: that means other nation states are running on Amazon, running
Speaker 4: on Google, right, And there is no way to simply
Speaker 4: create a competitor because you are talking about a form
Speaker 4: that was understood in the utilities context, particularly in telephony,
Speaker 4: as a natural monopoly for a very long time, and
Speaker 4: that has you know, ultimately been kind of entered into
Speaker 4: the gravitational pull of these handful of companies. So it's a.
Speaker 3: Very big problem.
Speaker 4: But again, it doesn't mean that things like signal don't exist.
Speaker 3: Signal exists. Swimming upstream signal is proving that one extremely
Speaker 3: innovative technologies can and actually do exist, that even swimming
Speaker 3: up against stream, we can create something that has set
Speaker 3: the gold standard and proven that yes, there's a ton
Speaker 3: of innovation left to do in tech, in privacy, in
Speaker 3: rights preservation, in creating.
Speaker 4: Tech that actually serves a social need. You know, it
Speaker 4: is popular, and Okay, now let's solve the issue of
Speaker 4: this toxic business model to actually foment innovation, to actually
Speaker 4: foment progress, whatever the claims of the marketing on the
Speaker 4: tan of big tech may be.
Speaker 1: Yeah, and just to close, I've really was taken by
Speaker 1: your Helmut Schmidt speech and you had a line, make
Speaker 1: no mistake, I'm optimistic, but my optimism is an invitation
Speaker 1: to analysis in action, not a ticket to complacency. Yes,
Speaker 1: talk about.
Speaker 4: That, Well, that is true, and I think that was
Speaker 4: me also pushing back a little bit on the idea
Speaker 4: that a grim diagnosis is somehow pessimism, right, uh huh, Like,
Speaker 4: we know it's not great, we know it's not great.
Speaker 4: Along a lot of axis tech is not the only
Speaker 4: one we know things aren't working. But the most dangerous,
Speaker 4: and frankly, the most pessimistic place to be is pushing
Speaker 4: that aside, disavowing that, diluting that analysis in service of
Speaker 4: immediate comfort.
Speaker 1: Stoping to bother to critique because you think there's no point. Essentially,
Speaker 1: that's the ultimes.
Speaker 4: Well, I would say that's almost nihilism. I would say
Speaker 4: the pessimism is where we paint a rosy picture so
Speaker 4: we don't have to feel uncomfortable and then base our
Speaker 4: strategy on that rosy picture in a way that is
Speaker 4: wholly insufficient to actually tackle the problem, because we never
Speaker 4: scope the problem, because we were unwilling to carry the
Speaker 4: intellectual emotional responsibility of recognizing what we're actually up against.
Speaker 4: So I think the most optimistic thing we can be
Speaker 4: doing right now is really understanding where we stand, really
Speaker 4: understanding what is the tech industry, how does it function,
Speaker 4: how does money move? And okay, what are the alternatives
Speaker 4: that exist, how do we support them, and how do
Speaker 4: we recognize the power that we have to shift things right?
Speaker 4: But that has to be based on a realistic picture.
Speaker 4: It can't be based on delusion.
Speaker 1: And part of that, as you know firsthand, coming back
Speaker 1: to the beginning of the conversation, is acting as a collective.
Speaker 3: Yeah.
Speaker 4: Absolutely, I think no one person is going to do
Speaker 4: this alone ever ever has ever you know, one person
Speaker 4: may have taken credit at some point or another, but ultimately,
Speaker 4: like the Internet was a collective effort, the open source
Speaker 4: Library is you know, every single big tech company that
Speaker 4: has one CEO, it has millions and millions of contributors.
Speaker 4: And so we change this together. The issue is will
Speaker 4: and the issue is resources. The issue is not ideas right.
Speaker 4: We're not waiting for one genius to figure it out.
Speaker 4: We're waiting for a clear map and some space to
Speaker 4: examine it together and share insights and then figure out
Speaker 4: how to push forward to a world where it's you know,
Speaker 4: thousands of interesting projects that are all thinking together about
Speaker 4: creating much better tech and reshaping the industry and its
Speaker 4: incentives in order to nurture that.
Speaker 1: Us.
Speaker 2: First of all, Meredith said about fifteen things that I
Speaker 2: would put on a small office couch pillow if I
Speaker 2: knew how to needle point.
Speaker 1: Well, start with the thing you would love most to
Speaker 1: put on a needle point.
Speaker 2: A grim diagnosis is not an invitation to pessimism.
Speaker 1: I love that too.
Speaker 2: I just think that, especially right now, it's easy and
Speaker 2: right to say that there's a lot to be wary of. Yeah,
Speaker 2: but it also is not an invitation to be weary,
Speaker 2: ad infinitum.
Speaker 1: No. No, I totally agree with you. I mean, I
Speaker 1: think often if you're critical, you'll called out for being pessimistic.
Speaker 1: But I thought the way that Meredith reframed that and
Speaker 1: was basically no, no hold on a second it's optimistic
Speaker 1: to be critical was really fascinating.
Speaker 2: Yeah. I think my favorite line in the whole interview,
Speaker 2: and maybe one of my just favorite lines I've heard
Speaker 2: in reporting on technology ever, is that data is a
Speaker 2: rough proxy for complex reality.
Speaker 1: Sounds like you're going to need two pillows or just a.
Speaker 2: Very big one. I actually want to talk about something
Speaker 2: that we've talked about in our production meetings for this
Speaker 2: show is that we don't want this show to be
Speaker 2: all doom and gloom. And you know, when I found
Speaker 2: out that you were interviewing Meredith, I was interested to
Speaker 2: see what she would say outside of a sort of
Speaker 2: technopessimist viewpoint, given that she has been quite critical of
Speaker 2: Google and also of surveillance capitalism. And yet when you
Speaker 2: hear her talk about Signal, I was sort of optimistic
Speaker 2: in terms of, oh, here's a tool that we actually
Speaker 2: have been encouraged by the FBI to use, yes, yet
Speaker 2: still don't use. But here's a tool that is actually
Speaker 2: answering a real fear that a lot of people have
Speaker 2: that Meredith has, of privacy and doing some actionable stuff
Speaker 2: to make us feel more secure.
Speaker 1: Well, Meredith, ultimately the buck stops with her at signal,
Speaker 1: and like, if you're a product lead or if you're
Speaker 1: running a company whose main thing is a product, you
Speaker 1: sure as hell better be optimistic because you're not going
Speaker 1: to get through your day.
Speaker 3: That's right, that's right.
Speaker 1: But but I agree with you. The thing that's really
Speaker 1: stuck with me was this concept of technology being dual use.
Speaker 1: And Meredith, you know, used this phrase signature strike, which
Speaker 1: it turns out can apply both to serving you with
Speaker 1: an AD to make you want to buy something and
Speaker 1: using your metadata to make a calculation that you're most
Speaker 1: probably an enemy combatant or a terrorist and kill you.
Speaker 1: I mean, this is the idea that it's the same
Speaker 1: basically set of analysis that can apply to shopping or
Speaker 1: life and death. I don't know, I just I have
Speaker 1: no sort thinking about that.
Speaker 2: What certainly makes me want to protect that data set
Speaker 2: that's it protects uff today. This episode was produced by
Speaker 2: Victoria Dominguez, Lizzie Jacobs, and Eliza Dennis. It was executive
Speaker 2: produced by me care Price, Oswalashan and Kate Osborne for
Speaker 2: Kaleidoscope and Katrina Norvel for iHeart Podcasts. Our Engineer is
Speaker 2: Bihit Frasier. Al Murdoch wrote our theme song.
Speaker 1: Join us on Friday for tech Stuff's the Weekend tech
Speaker 1: We'll run through our favorite headlines, talk with our friends
Speaker 1: at four or four media, and try to tackle a question,
Speaker 1: when did this become a thing. Please rate and review
Speaker 1: on Apple Podcasts, Spotify, or wherever you get your podcasts,
Speaker 1: and reach out to us at tech stuff podcast at
Speaker 1: gmail dot com with your thoughts and feedback. Thank you,