Speaker 1: So you've all likely heard about Tilly Norwood, the first
Speaker 1: ai A lister. Her creator just announced that Tilly is
Speaker 1: about to star in her first feature film. Taylor and Natasha,
Speaker 1: your face is a quite a picture.
Speaker 2: This is just a fake PR thing.
Speaker 3: Wait, Taylor, do you remember when all the editors wanted
Speaker 3: you to like interview Little Michayla.
Speaker 1: Yeah, I've met Lit Michayla's creator, Trevor.
Speaker 2: I know Trevor.
Speaker 4: I've known Trevor for a long time. I have the
Speaker 4: girl that Little Michaela was designed off of. I But
Speaker 4: with all due respect, love Trevor. But guys, this is
Speaker 4: just PR.
Speaker 2: It's just PR.
Speaker 1: Welcome to tech stuff. I'm Maza Velosa and this is
Speaker 1: the Weekend Tech, where I'm joined by the world's most
Speaker 1: plugged in report is to break down what's really happening
Speaker 1: in tech right now. Today, we're joined by Taylor Lorenz
Speaker 1: of User mag Read Abrigotti, tech editor for Semiphore, and
Speaker 1: Natasha Tku, tech reporter at the Washington Post.
Speaker 5: Welcome, Well, how's it going.
Speaker 2: Thanks for having us.
Speaker 3: Nice to be back.
Speaker 1: Natasha. You recently published an article that got to number one,
Speaker 1: I think in the in the Washington post algorithm. The
Speaker 1: subheads of the story is anthropic. Google and Meta have
Speaker 1: hired computer scientists, neuroscientists, and philosophers to study what some
Speaker 1: in the industry think may become a moral crisis. The
Speaker 1: story is about AI consciousness.
Speaker 5: What is that?
Speaker 3: Oh, start with the hard one. Huh. You know, it's
Speaker 3: this idea that has percolated in the back of the
Speaker 3: AI industry, like on the fringes for decades. You know,
Speaker 3: the same people who hope to upload their consciousness one
Speaker 3: day or you know, become transhuman and merge with machine
Speaker 3: also anticipated this idea that you know, sufficiently advanced AI
Speaker 3: would have subjective experiences and goals and preferences, just like
Speaker 3: human beings. And you know, I first got interested in
Speaker 3: this subject four years ago, Like it was a few
Speaker 3: months before chat chipet came out, and I wrote a
Speaker 3: story about this Google engineer who believed the company's large
Speaker 3: language model was sentient, was conscious in some way, and
Speaker 3: he got fired.
Speaker 1: Right, he was a whistleblower or is that right?
Speaker 3: I think that's fair to say, Yeah, I'm.
Speaker 1: Not a whistleblower per se. He viewed himself as a whistleblower.
Speaker 1: Others viewed him as crazy.
Speaker 3: I think that's also fair to say. Yeah, so he
Speaker 3: you know, at the time, none of us had like
Speaker 3: it wasn't public, You couldn't like talk to these very
Speaker 3: human like machines. So I thought, you know, it was
Speaker 3: a kind of response to this technology designed to sound
Speaker 3: like a human and you know, often used first person
Speaker 3: and refer to its own experiences. And I think after
Speaker 3: that the tech companies stepped back a little bit. You know,
Speaker 3: they wanted people to adopt their very productive tool. They
Speaker 3: were showing all the practical use cases. And I would
Speaker 3: say over the last year or so for Anthropic and
Speaker 3: then the last few months for other companies, Google, Meta
Speaker 3: and Anthropic have really embraced this idea in public. I mean,
Speaker 3: we've talked on the show before about Chris Ola, the
Speaker 3: Anthropic co founder, talking about, you know, can chat pots
Speaker 3: have experience emotions and you know, joy and despair on
Speaker 3: stage at the Vatican a few seats away from the Pope, and.
Speaker 1: As you point out your story that was a point
Speaker 1: of difference between him and the Pope because the Pope
Speaker 1: expelicitically said no, right.
Speaker 3: Yeah, exactly, Like it's still unclear who was who got
Speaker 3: the upper hand there. And who was using who you
Speaker 3: know to push their their message. But yeah, I mean
Speaker 3: they really just don't shy away from it now. Like
Speaker 3: Anthropic talks about its AI psychiatry team. You know, they
Speaker 3: hired this guy, Kyle Fish, who actually used to work
Speaker 3: in like bioengineering for meat replacement, but now he's their
Speaker 3: head of model welfare and thinking about like changes that
Speaker 3: the company could potentially make to ensure that if they
Speaker 3: find out that the models are conscious, they're meeting their
Speaker 3: like moral obligation. It just very quickly goes to analogies
Speaker 3: around slavery and genocide or factory farming. So so you know,
Speaker 3: and from the mouth of these executives at the tech companies.
Speaker 6: So I just he's still in the meat replacement industry.
Speaker 6: Apparently I just thrown a dad joke.
Speaker 1: Farewell to us some meat puppets read tailor I mean,
Speaker 1: what do you do you think this is on behalf
Speaker 1: of the technology companies are kind of legitimate and even
Speaker 1: necessary line of inquiry or do you think this is
Speaker 1: kind of you know, or a boros kind of snaky
Speaker 1: in its own tail of falling in love without creations?
Speaker 5: Yet again, I.
Speaker 4: Think it's deranged. I don't think there's any world in
Speaker 4: which an LM is conscious. It reminds me of that
Speaker 4: rain Fisher Kwandswait after a judge of et came out
Speaker 4: where she typed, you know, typing I'm evil in a
Speaker 4: Google doc and getting scared. Basically, I think this is
Speaker 4: very silly, and I think it's going to blow up
Speaker 4: in their faces. I think this is completely going to
Speaker 4: give consumers the wrong idea. The more that people consider
Speaker 4: this thing sentient, the more they're going to sort of
Speaker 4: basically expect it to have a morality and a moral compass,
Speaker 4: and it's never going to be able to do that.
Speaker 6: Yeah, these these companies have now, you know, put a
Speaker 6: lot of guardrails around these things so that they don't
Speaker 6: actually sort of seem as human like, as sentient as
Speaker 6: as they probably would if you use these things with
Speaker 6: like you know, with the raw model, they sort of
Speaker 6: they're a lot more human like and a lot crazier honestly.
Speaker 6: So they I think it's it's like, I think, hard
Speaker 6: for us to understand like what people like the guy
Speaker 6: you wrote about four years ago we're seeing, I think,
Speaker 6: But it is ultimately just you know, text prediction, right,
Speaker 6: There's nothing sentient about it. I think that I think
Speaker 6: these companies are I mean a lot of these people
Speaker 6: inside them are. They're very genuine people and they do
Speaker 6: have these concerns, but they're also like, in a way,
Speaker 6: it's marketing because it's like these things are going to
Speaker 6: end up being so intelligent, they're going to get to
Speaker 6: human like intelligent, what do we do? What do we
Speaker 6: do when we get there? And like we may never
Speaker 6: get there, right, it's not ultimately it may not ultimately
Speaker 6: be possible to even build what they're sort of concerned about.
Speaker 6: I think that's another way of looking at it.
Speaker 1: Natasha, you write a real piece at the search for
Speaker 1: machine consciousness is practically mainstream. Do you mean mainstream within
Speaker 1: the type of technology companies or yes.
Speaker 3: Yeah, yeah, I mean but the fact that it's happening
Speaker 3: in peril well, with so many people asking whether or not,
Speaker 3: you know, the chatbot that they're talking to every day
Speaker 3: is alive in some way or human in some way,
Speaker 3: I just find that remarkable. I mean, I think you
Speaker 3: know what Taylor said about it, coming back to bite
Speaker 3: the companies is super interesting because I wanted to write
Speaker 3: about this because it is a prelude to an argument
Speaker 3: about personhood and rights for the AI. So I think
Speaker 3: actually the companies are hoping that they won't be accountable
Speaker 3: for some of the actions of their chatbots in the future.
Speaker 3: You know, they're really like, this is only this argument
Speaker 3: is only going to get weirder.
Speaker 1: But they want to push the liability onto the chatbots.
Speaker 1: I mean you would you would think, I mean, I
Speaker 1: would think anyway that it would be bad for them
Speaker 1: for the chatbots to be considered conscious or having any
Speaker 1: kind of self of because that would open them up
Speaker 1: to a different type of liability.
Speaker 3: Right, Yes, unless they're able to confer personhood on a chatbot,
Speaker 3: in which case you can't. Yeah, you can't go after
Speaker 3: the anthropic engineer.
Speaker 5: Has anyone actually told you that?
Speaker 3: Has anyone told me that about personhood?
Speaker 2: Yes?
Speaker 6: Has anyone at these companies said their strategy is like,
Speaker 6: this is going to be our way out of liability.
Speaker 3: No, no, no, no, I'm going to give them more liability.
Speaker 2: They're going to have more liability.
Speaker 5: If anybody thinks that they're so delusional.
Speaker 3: No, no one said that this is a strategy. That's
Speaker 3: more coming from the critics who see this argument that
Speaker 3: has been building. I mean, there's many many people actually, Like,
Speaker 3: right after I wrote about that Google guy, there was
Speaker 3: this ethicist Giada Pistili at Hugging Face who was like,
Speaker 3: I am not going to any more conferences where I
Speaker 3: have to talk to a man about robot rights, Like
Speaker 3: this is ridiculous, Like, do not make me do this anymore.
Speaker 3: So I'm just letting you know, like where I think
Speaker 3: it's heading.
Speaker 1: Well, I guess in phase one, you need to get
Speaker 1: normal people to want to engage with the product, right,
Speaker 1: and people like engaging with stuff that reminds them of
Speaker 1: stuff that already do, which is like human engagement or
Speaker 1: the uncannoness of talking to a chatbot. But once these
Speaker 1: things start to pick up their momentum in terms of
Speaker 1: having more functional use cases, I guess the incentives for
Speaker 1: the tech companies to seduce people with human like experiences
Speaker 1: get lower.
Speaker 3: Right.
Speaker 1: I mean, none of the tech companies are making money
Speaker 1: from you know, late night chatbot conversation.
Speaker 3: That's not true. No, it's been one of the big
Speaker 3: use cases.
Speaker 1: So people pay, people pay subscriptions to get more access
Speaker 1: to do like personal chats or how do they character.
Speaker 3: Unlimited chats with the chatbot or you know, they'll respond
Speaker 3: to you or you can change your avatar or change theirs.
Speaker 1: That must be a fractional a fractional amount of the
Speaker 1: money that goes into the space compared to like Cisco
Speaker 1: doing enterprise AI tethers and stuff, right like.
Speaker 3: Well, I think the open AI and others they know
Speaker 3: that people. They can see the engagement, They can see
Speaker 3: what people are are doing with the chatbots. And I
Speaker 3: think it has been true that they like the you know,
Speaker 3: the back and forth has been getting longer and longer,
Speaker 3: you know, with the really engage users. But I just
Speaker 3: wanted to disagree with Read on one point. This is wrong.
Speaker 2: Read.
Speaker 3: No, I think that they actually, I mean, part of
Speaker 3: why I find this topic really interesting is because I
Speaker 3: think they do enormous amounts of work to make them
Speaker 3: sound more human. Like you if you look at like
Speaker 3: GPT two, for example, and you said a man walks
Speaker 3: into a bar, it would just like finish the sentence.
Speaker 3: You couldn't you couldn't ask, you know, there was no
Speaker 3: instruction fine tuning to follow what you wanted. There was
Speaker 3: not you know, there wasn't that like reinforcement learning from
Speaker 3: human feedback so that like, oh, people like it when
Speaker 3: you phrase something in this way, and I just find
Speaker 3: it wild that anthropic has you know, has said on
Speaker 3: some of these blog posts like we can't there's no
Speaker 3: way we could possibly make this human assistant less human.
Speaker 3: You have like a multi thousand word constitution where you're
Speaker 3: talking to it about morality and it's potential conciousness, like
Speaker 3: and you're apologizing to claude about you know, its moral
Speaker 3: personhood in case it has one, like you are doing
Speaker 3: a lot to make it sound human and then to
Speaker 3: kind of yeah, I guess then do tests where you're.
Speaker 6: Just an extent, what do you mean? It's like, but
Speaker 6: if you ask it, but if you ask these things,
Speaker 6: are you conscious?
Speaker 5: Are you human? I'd be like, I am just an
Speaker 5: l lem.
Speaker 3: That's different they're saying. They tweak the responses to those answers,
Speaker 3: but in terms of how human it sounds, where you
Speaker 3: can very easily like start to get annoyed with it
Speaker 3: or start to like.
Speaker 4: They just add the ability to interrupt someone, which is
Speaker 4: very human.
Speaker 5: The opening eye voice model.
Speaker 6: Yeah, I mean I appreciate that clarification at that point
Speaker 6: that like, yes, of course they want it to sound
Speaker 6: intelligent like a human to some extent, but like right,
Speaker 6: now they sound like a like a person communicating yes,
Speaker 6: but they don't really sound human, if that makes sense.
Speaker 6: They don't do what humans do, which is I.
Speaker 4: Think they sound very human to a lot of people.
Speaker 4: I agree, unfortunately, and I think that I think that
Speaker 4: the more that we give people that are extremely uneducated
Speaker 4: about this technology, the idea that these things might be conscious,
Speaker 4: the more agency they give them, and that that worries me.
Speaker 1: Natasha, I wanted to pick up on what the research
Speaker 1: actually looks like. So, you know, you mentioned that all
Speaker 1: these anthropic and others are trying to like correct the
Speaker 1: code on whether or not AI is a consciousness. I mean,
Speaker 1: it's hard enough to solve or in slash, impossible to
Speaker 1: solve the conundrum consciousness in humans let loan machines. But
Speaker 1: how what does the research look like?
Speaker 3: You know, A lot of the research is, you know,
Speaker 3: which they caveat. You know, they say, we're not really
Speaker 3: sure how to interpret this results. But a lot of
Speaker 3: it involves applying the same techniques you would use for
Speaker 3: human psychology. A lot of it just involves asking the
Speaker 3: chatbot questions about itself. And that's where I find it.
Speaker 3: Soon we're interesting that this is happening in parallel because
Speaker 3: they had Anthropic for example, in one of its first
Speaker 3: like model welfare reports, which they include in like Mythos too.
Speaker 3: It's like here's a cyber weapon, and here's its internal feelings.
Speaker 3: But it had it let two chapbops talk to themselves,
Speaker 3: and naturally the conversation flowed to consciousness. But then in
Speaker 3: some cases it went to something they called a spiritual
Speaker 3: bliss attractor state, and it just started having like the
Speaker 3: cyclone emoji, and they were like, all all is one piece.
Speaker 3: And another version was like Nambas Day with the prayer
Speaker 3: hands emoji Numbas Day. And it looks a lot like
Speaker 3: the conversations I see in lawsuits, you know, like about
Speaker 3: people believing that they're having a very meaningful conversation with
Speaker 3: the chatbot.
Speaker 6: But I think they published hasn't been written on the
Speaker 6: internet before, right, So yeah, that's where it all comes from.
Speaker 1: But but what was the conversations you have with neuroscientists,
Speaker 1: what was the what was the kind of counter counter
Speaker 1: take on all of this.
Speaker 3: They were saying. I guess it's what's called like the
Speaker 3: hard problem of this, which is if you are using
Speaker 3: tests that are that are still in development. You know,
Speaker 3: it's not like anyone has found a way to prove
Speaker 3: that humans are conscious, but we know that they are.
Speaker 3: So if you use the tests that people use on
Speaker 3: humans on chatbots, like you're just it's just kind of
Speaker 3: conflating one thing for the other. It can sound human,
Speaker 3: but we don't know if it has any kind of
Speaker 3: internal state. And there they are trying different ways, you know,
Speaker 3: they're trying to make it a more rigorous science. One
Speaker 3: thing that Anthropic did was like kind of perturb the model,
Speaker 3: like inject a thought into it, into its activation, and
Speaker 3: then see if it recognized it. But yeah, the neuroscientists
Speaker 3: were just extremely skeptical about the methods and really uncomfortable
Speaker 3: about how quickly the discussion can go to analogies of
Speaker 3: factory farming or slavery. And you know, they a lot
Speaker 3: of the proponents of studying this say like, you know,
Speaker 3: if they are sentient or conscious in some way, or
Speaker 3: can experience, you know, suffering in some way, then we
Speaker 3: have this massive moral responsibility to the trillions of chatbots.
Speaker 3: But the neuroscientists and consciousness experts I spoke to are like,
Speaker 3: there is also a great moral peril in going the
Speaker 3: wrong direction and ascribing personhood and sentience to something that
Speaker 3: is not.
Speaker 1: You know, you're nodding vigorously.
Speaker 4: Yeah, I mean, pigs are incredibly conscious. We know that
Speaker 4: pigs have rich in our lives, family relationships, friendships. They
Speaker 4: experience a full range of emotions. And look at what
Speaker 4: we do to pigs, look at how we treat pigs,
Speaker 4: like I think, you know, it's it's interesting to hear
Speaker 4: the comparison to factory farming when we don't do anything
Speaker 4: about factory farming anyway, Like to put to put the
Speaker 4: needs of these models, even if they were conscious, ahead
Speaker 4: of actual living, conscious beings who we treat terribly, not
Speaker 4: to mention all the other you know, the comparisons to slavery.
Speaker 4: Slavery is still alive and while in parts of the world,
Speaker 4: you know, like I just I don't know, I think
Speaker 4: it's a little bit ridiculous and problematic.
Speaker 2: I guess, for lack of a better word, I don't know.
Speaker 3: Well, I'll just say for context that a lot of
Speaker 3: the funding is a coming from anthropic but also from
Speaker 3: the effective altruism movement, and they are, you know, like
Speaker 3: one of their causes is animal welfare.
Speaker 1: Potechnically, you just remind us what effective altruism isn't and
Speaker 1: why the cults, why would you to send.
Speaker 3: This effective altruism. It's it's like a philosophical or some
Speaker 3: people say, uh, philanthropic movement to try to do the
Speaker 3: most good in the world that uses this really utilitarian
Speaker 3: argument of like, you know, being able to calculate, you know,
Speaker 3: how far your money goes or how many people you
Speaker 3: can help. And you know, during the AI boom has
Speaker 3: come to be pretty influential because so many people who
Speaker 3: work at anthropic and at the labs and early proponents
Speaker 3: of this kind of like human like technology come from
Speaker 3: that movement and are funded by that movement.
Speaker 6: And they're about to have a lot more money. I
Speaker 6: mean they've always been funded by these billionaires, but but
Speaker 6: now as as these companies go public and people cash out,
Speaker 6: this movement is about to have like the most money
Speaker 6: of probably any you know, philanthropic movement in the world.
Speaker 6: So it's going to be super interesting how they spread
Speaker 6: these messages. And like when you see animal welfare people
Speaker 6: moving over to AI welfare, I mean clearly that's a
Speaker 6: that's that's where the money is, right, So you know.
Speaker 1: Well there is a stone iron I was thinking about
Speaker 1: that earlier. Like it's like, is it case closed on
Speaker 1: animal welfare? Like we succeeded there and it's time to
Speaker 1: move on.
Speaker 3: I know many people, many young people, come into this
Speaker 3: movement through animal welfare and then they end up you know.
Speaker 3: The argument is that, like to do the most good,
Speaker 3: you have to siffly build super intelligence.
Speaker 1: Well either heady topics. When we come back, Taylor tells
Speaker 1: us about a week long anti big tech festival in
Speaker 1: New York City. Stick around to hear what she saw.
Speaker 1: Welcome back, Taylor. You came all the way to New
Speaker 1: York from LA to attend something I'd only heard whispers
Speaker 1: of on a recent trip to the Russian bath house,
Speaker 1: this being the Summer of Blood for those unfamiliar or
Speaker 1: not denizens of the eighty eight Wall Street baths. What
Speaker 1: was the event?
Speaker 4: Yeah, so this was an eight day event with hundreds
Speaker 4: of related events around New York City. It was called
Speaker 4: Summer of Blood, referring to the Luddites, the famous group
Speaker 4: that smashed the automated looms back in the day. They've
Speaker 4: sort of become this like avatar of you know, workers'
Speaker 4: rights and resistance. Technology, and so these young people have
Speaker 4: kind of embraced the ludite I guess designation, and they
Speaker 4: hosted this festival. So there was lots of things. I mean,
Speaker 4: there were like lots of talks on how to quit
Speaker 4: big technology, how to leave Apple, leave Google, you know,
Speaker 4: how to embrace indie tech. I went to a tech
Speaker 4: cleansing ceremony where they saged my computer and attempted to
Speaker 4: remove all the evil energy and we all said some mantras.
Speaker 4: And then there was this ship phone protest where they
Speaker 4: all marched around New York City and nomats and put
Speaker 4: open AI and Meta on trial. Obviously they were found
Speaker 4: guilty of being terrible so and sentenced to death. And
Speaker 4: so yeah, it was interesting. I mean, the people came
Speaker 4: from all over the country for this. It's hwed very young.
Speaker 4: It was a lot of college students and young gen zs.
Speaker 1: What made you decide it was worth with the cross
Speaker 1: country trip and what do you find most interesting being there?
Speaker 4: Well, I am trying I do a lot of reporting
Speaker 4: on the so called child safety movement and resistance to
Speaker 4: technology and a lot of these policies, and some of
Speaker 4: these young people that were involved on the planning committee
Speaker 4: of the Summer of Lood actually were originally involved in
Speaker 4: the sort of child safety movement against big tech. So
Speaker 4: they saw these alleged gen Z group, these fake AstroTurf
Speaker 4: groups like Design It for Us, you know, which purports
Speaker 4: to be like gen zs against big tech, and then
Speaker 4: they became very quickly disillusioned. They realized, oh, Design It
Speaker 4: for Us is actually not led by gen Z's. It's
Speaker 4: actually just this front for you know, these other billionaire
Speaker 4: donors that just want to push mass surveillance. And so
Speaker 4: they've tried to build this sort of third path where
Speaker 4: they want to resist technology, but not through enacting massurveillance
Speaker 4: and censorship laws.
Speaker 1: You know, I was interested in I mean I saw
Speaker 1: some of the pictures and stuff. Although I know, I
Speaker 1: know picture there was kind of designed to be not captured.
Speaker 1: I mean, there are a few people who broke the rules.
Speaker 1: As always, if you.
Speaker 4: Tried to record, they would come up to you with
Speaker 4: one of these little like cardboard iPhones and say put
Speaker 4: down your phone, beloved.
Speaker 1: Some a few a few images slipped through the cracks
Speaker 1: there seem to be I mean, there were puppets. There
Speaker 1: was a kind of a you know, a staged play
Speaker 1: with the you know, which recreated the original ludite movement
Speaker 1: in Victorian England.
Speaker 4: They were also doing a crowdsource reading of from some
Speaker 4: iconic scenes from the social network as well.
Speaker 1: I mean, it sounds fun. It also sounds very different
Speaker 1: to what we talked about a couple of weeks ago,
Speaker 1: which was the campus protests against you know AI AI
Speaker 1: talks like how do you?
Speaker 5: How do you?
Speaker 1: I mean talk about the characterized the difference between the
Speaker 1: mood and the energy and what you think explains.
Speaker 4: That it was really fun. I had a great time.
Speaker 4: They've had so many funny activities. I went to so
Speaker 4: many interesting talks like these are people that are really
Speaker 4: I think, thoughtfully engaged on the discourse. And what I
Speaker 4: found interesting is they're not like a lot I mean,
Speaker 4: some of them are sort of anti technology in the
Speaker 4: sense that like they they don't like their you know,
Speaker 4: they don't have they have flip phones maybe, or they
Speaker 4: a lot of them use the like light phone, the
Speaker 4: four hundred dollars you know, stripped down phone that just
Speaker 4: has like Google maps and stuff. But a lot of
Speaker 4: them are just interested in alternative technology. So they love
Speaker 4: the open source movement. A bunch of them had like Linux.
Speaker 4: They were talking about graphene os. You know, they use signal,
Speaker 4: they use they really want to fight to protect encrypted messaging.
Speaker 4: There were some a woman there who was from the
Speaker 4: Internet Archive. You know, there's people affiliated with the EFF.
Speaker 2: So it was.
Speaker 4: It was like very much like my kind of technologists,
Speaker 4: where it was felt like indie tech and kind of
Speaker 4: grassroots tech and really getting away from like corporate tech.
Speaker 4: And so I think their criticism is around how corporate
Speaker 4: power is warping the tech industry. And I think that
Speaker 4: those are much more valid arguments than you know, what
Speaker 4: all of these billionaire backed nonsense big tech groups are.
Speaker 4: You know, big tech backed groups are making on the
Speaker 4: hill about the child safety stuff, and I think some
Speaker 4: of this, like the anti AI stuff, it's just like
Speaker 4: it's not like sort of thoughtful criticism. It's just the
Speaker 4: sort of like reflexive, you know, hate of new technology.
Speaker 5: I can get behind that.
Speaker 1: Read in Touch Food you follow this.
Speaker 2: They are pretty anti capitalists. Read. I don't know if
Speaker 2: you would totally.
Speaker 3: Agree capitalism as.
Speaker 6: Somebody who went into took a vowed poverty related to journalism.
Speaker 5: But I'm not no.
Speaker 2: I thought you said one time on the plot.
Speaker 6: Pro capitalism for sure, I just not for myself personally,
Speaker 6: But I think that's I think that's how I feel
Speaker 6: about a lot of things, right Like I'm not I'm
Speaker 6: also I love technology. I think it's fascinating and fun
Speaker 6: to write about. I love using it. I also am
Speaker 6: most happy when technology is far away and I'm you know,
Speaker 6: backpacking in the wilderness with no cell signal like I
Speaker 6: I'm all for, like in your personal life, just getting
Speaker 6: away from technology.
Speaker 1: But Tita that this was about something something Yes, that,
Speaker 1: but also a more kind of like strident critique. I
Speaker 1: mean you mentioned the the shit foam, which I think
Speaker 1: is that chroninfiscating hatred information technology and the passionate hemorrhaging
Speaker 1: of on neoliberal experience.
Speaker 4: Yes, yes, that's what it stands for exactly, So I
Speaker 4: think these people have a more politicized critique of the system. Yes,
Speaker 4: a lot of them do embrace like an offline lifestyle
Speaker 4: in a lot of ways. And it was great to
Speaker 4: honestly be in going to these events and having no
Speaker 4: one on their phone for so much like it does
Speaker 4: change the vibe and the experience of it.
Speaker 1: By the way, That's why I like about the Russian
Speaker 1: babs because you contact your phone in there because it breaks,
Speaker 1: you have to spend like two and a half hours
Speaker 1: without your phone, and that's in many ways the best
Speaker 1: part about it done. It's also it's a great it's
Speaker 1: the best place to go with your friends because no
Speaker 1: one's sort of like any half that.
Speaker 4: I liked not feeling like somebody was surveilling me and
Speaker 4: taking pictures of me and like videoing. And I feel like,
Speaker 4: you know a lot of times in these public events,
Speaker 4: like the videos are out.
Speaker 1: You don't think NYPD. I'm definitely the NYPD will keep
Speaker 1: an eye on things daya well.
Speaker 4: I'm sure right, I'm sure we're surveilled by flock cameras
Speaker 4: all over. But it's just nice to like, you know,
Speaker 4: not immediately have a camera in your face.
Speaker 6: I love I Also, I love the fact that they
Speaker 6: are like what you taught, Like Taylor, I think we're
Speaker 6: very aligned here. Like this open source, you know, indie
Speaker 6: tech thing is really important because I think ironically, like
Speaker 6: what big tech actually does is it slows the development
Speaker 6: of technology, Like it's in a lot of ways it's
Speaker 6: anti innovation. And I think I wrote this like line
Speaker 6: somewhere maybe when I was at the post about Apple,
Speaker 6: which is like it's a lot more profitable to prevent
Speaker 6: innovation than to innovate. And I really think that's true.
Speaker 6: So I mean, if you're really anti technology, you should
Speaker 6: be pro big tech.
Speaker 3: Ironically, well, I love that they that they prohibited like
Speaker 3: taking photos of it, because I feel like, no matter what, usually,
Speaker 3: you know, there's a stage moment so that you can
Speaker 3: go viral on you know, social media, of course, And
Speaker 3: I'm so happy to hear about this tailor, like I
Speaker 3: hadn't heard about Summer of Flood before. And I feel
Speaker 3: like this critique that separates, you know, this more nuanced
Speaker 3: approach where you can love technology and hate the business model,
Speaker 3: I think is is just a like a delineation that
Speaker 3: will be so important coming forward because like one of
Speaker 3: the biggest pushbacks against any kind of political action against
Speaker 3: concentration of power in like the hands of a UAI
Speaker 3: companies is well, people are using it, people love to
Speaker 3: use it. Like, yes, I also I love to use it,
Speaker 3: but you know you can they don't have the monopoly
Speaker 3: on the technology. They are not the sole inventors or
Speaker 3: the only people who can push this stuff. And if
Speaker 3: young people are getting involved in thinking about open source
Speaker 3: models for it. I think that's awesome.
Speaker 4: I just want to see them exert more political power,
Speaker 4: and I wish that these kids, you know, it's frustrating
Speaker 4: because to read the media coverage of a lot of
Speaker 4: this stuff, it's like a reporter shows up for one
Speaker 4: day they're.
Speaker 2: Like, Wow, these kids hate tech, they like the flip phones.
Speaker 4: Like it's just this very surface level coverage that doesn't
Speaker 4: really interrogate their ideology or what they're trying to achieve,
Speaker 4: and it doesn't mention that they are actually opposed to,
Speaker 4: extremely opposed to this entire online safety slop about protect
Speaker 4: the children that claims to be anti big tech, Like
Speaker 4: these these kids are very anti surveillance. And part of
Speaker 4: the reason they didn't want it all, like document it
Speaker 4: was because they feel like they've lived their entire entire
Speaker 4: lives where like everything is packaged and documented and shared
Speaker 4: and surveilled, and so yeah, I just like, I don't know,
Speaker 4: I'm heartened some of them are gonna apparently protest in
Speaker 4: DC soon. I just want them to, Like, I want
Speaker 4: these people to like get quoted more, you know, instead
Speaker 4: of these like fake groups like design It for Us
Speaker 4: that are just fake gen z groups that just like
Speaker 4: pretend to be anti big tech but exists to push
Speaker 4: metas policies.
Speaker 1: But it's sort of hard to quote them because many
Speaker 1: of them didn't reveal their identities, right, I mean talk
Speaker 1: about who are the organizers and do you think it's
Speaker 1: strategic benefit or strategic mistake not to allow it? I mean,
Speaker 1: I remember the Occupy wool Street days, Right, there's like.
Speaker 4: They reveal their idea. They're not They're not like masked
Speaker 4: or secretive or anything. I mean you could talk to
Speaker 4: them the during the actual event. They did have a
Speaker 4: press puppet, so they had this puppet that would speak
Speaker 4: for them to the media just because they didn't. It
Speaker 4: was like this massive decentralized event and they didn't want
Speaker 4: media just like going around like talking to random people
Speaker 4: instead of like hearing sort of from them, like the organizers.
Speaker 4: But they're not like hard to find. I mean I
Speaker 4: spoke to these people. You could show up four or
Speaker 4: four Media was there, Wired was there, The Economists was
Speaker 4: allegedly there, so they're you know, again, this is a
Speaker 4: massive public event in New York City with hundreds, if
Speaker 4: not thousands of people participating.
Speaker 1: Taylor, do you have any sense of where this where
Speaker 1: this goes this this movement, I mean, does does this
Speaker 1: ladder up to be like a policy platform for the Democrats?
Speaker 4: For not for the Democrats. They don't like the Democratic Party.
Speaker 4: These kids, you know, their mom Donnie supporters, they're veryed
Speaker 4: to the left. I think they're part of this faction
Speaker 4: of the left that's that's exerting pressure on the Democrats,
Speaker 4: you know, and you see the Democrats struggling. This is
Speaker 4: part of why they want to control the internet so badly.
Speaker 4: It is because the establishment democrat sort of policies are
Speaker 4: not popular among their base, and especially not with young
Speaker 4: people like these who hate mass surveillance.
Speaker 1: When we come back read experiences that downside of using
Speaker 1: AI agents stay with us, Welcome back.
Speaker 5: Read.
Speaker 1: We heard in the early part of this year about
Speaker 1: token maxing. There's now token minning happening for Sudden eight
Speaker 1: companies A being their employees to cut back on over
Speaker 1: use of AI books of the cost of tokens. This
Speaker 1: is something you, in a very human way, experience yourself.
Speaker 6: Yeah, well, while while you and I were in Can
Speaker 6: hanging out, you know till the wee hours of the
Speaker 6: morning or whatever, jet you know, my codex.
Speaker 1: My I hasten to add at a at a work
Speaker 1: at a work adjacent to it was Creator Economy, slash Marketing.
Speaker 6: It was for sure, it was all at work, but
Speaker 6: it was you know, while I was gone, I had
Speaker 6: noticed a couple of bills for like five dollars in
Speaker 6: my in my email. And you know, when you're in Europe,
Speaker 6: I don't know, it's just I don't know about you, OZ,
Speaker 6: but I always find it hard to like actually get online.
Speaker 6: And then I was on vacation, you know, in the
Speaker 6: Alps with the family and everything.
Speaker 5: It was great.
Speaker 6: But I come back and I'm like, it's like Sunday morning,
Speaker 6: and I'm jet lagged, and I'm at my computer. I'm
Speaker 6: trying to figure out what these bills are on my
Speaker 6: you know, from my codex, which I hadn't been using.
Speaker 1: A codex read is is is open Ay's platform or
Speaker 1: how do you define codex?
Speaker 5: Yeah?
Speaker 6: I mean, it's it's this Mac app that basically lets
Speaker 6: you be pretend to be a software developer. It just
Speaker 6: does everything for you and we'll you know, you can
Speaker 6: basically start as many AI agents as you want and
Speaker 6: have them work on these on these projects for you.
Speaker 6: I love it. It's fantastic. Like we've I've built so many
Speaker 6: cool things with it, Like I've gone from zero software
Speaker 6: development to like using it all the time. But like
Speaker 6: I hadn't set any limits on this thing, and it
Speaker 6: I go to my credit card bill and it's just
Speaker 6: all open AI bill, like for five dollars, just like
Speaker 6: a huge It just goes on forever, and I'm like,
Speaker 6: oh god, what so like this is partially me like
Speaker 6: making fun of myself for the benefit of readers and
Speaker 6: hopefully humanity.
Speaker 1: Well, what was the project that you would you were
Speaker 1: trying to do and how did how did how did
Speaker 1: open ai codex go rogue in terms of these repeated charges?
Speaker 6: Yeah, well what it What it was was that all
Speaker 6: my projects got deleted and I had to reinstall the
Speaker 6: software and then I but I before I deleted Codex
Speaker 6: form my computer, I asked Codex to make a prompt
Speaker 6: for me to restore my old projects once I reinstalled it,
Speaker 6: because like the files are there, just like all the
Speaker 6: chats are gone. But I had I had one of
Speaker 6: these projects was particularly complicated and I won't go into
Speaker 6: the details, but I tried to restore it right before
Speaker 6: I left on this vacation, and I wasn't monitoring it,
Speaker 6: and it somehow got stuck in this loop where it
Speaker 6: was just reading all of this data over and over again,
Speaker 6: like every day, and then running out of credits and
Speaker 6: then like top up my my five dollars credits. So
Speaker 6: like there were some days where there were like twenty
Speaker 6: five dollars charges in a single day, and I just
Speaker 6: had no idea this was happening for two weeks. It
Speaker 6: turned out it wasn't that much. It was only like
Speaker 6: five hundred bucks, I mean, which is still a lot,
Speaker 6: but like it could have been thousands, you know, it
Speaker 6: was it was like.
Speaker 1: Hey, you got and you've got lucky. What was the
Speaker 1: five dollar how come it was in five dollars increment?
Speaker 5: It just tops itself up in five dollars increments? I
Speaker 5: don't know.
Speaker 6: I like, I must have checked that box at some
Speaker 6: point and totally forgot about it. But what was also
Speaker 6: sort of funny about this is like, so to try
Speaker 6: to figure out what had happened, of course, like I'm
Speaker 6: going to ask Codex, right, So I'm like, I'm like,
Speaker 6: can you figure out why you're billing me five dollars?
Speaker 6: And it goes and does like an investigation and comes
Speaker 6: back and it's like, here's here's what happened. And I'm like, okay, well,
Speaker 6: then help me write an email to open AI to
Speaker 6: get a refund, like I want to a refund for
Speaker 6: my five rebox. So it's like, okay, here's I'll draft
Speaker 6: this email. And here's like all the bill and everything.
Speaker 6: It even went to my I had it go to
Speaker 6: my credit card and like, you know, cross reference it
Speaker 6: and make sure it all matched up. And then it's like,
Speaker 6: but you shouldn't send an email.
Speaker 5: You should.
Speaker 6: You should send that basically their their customer service chatbot.
Speaker 6: You should start a chat with it. So I started
Speaker 6: chat and which of course is just an Opening Eye chatbot.
Speaker 6: So now the chatbot is responding to me, and I'm
Speaker 6: copying that response and pasting it into Codex, and then
Speaker 6: Codex is responding and I'm pasting it into the chat
Speaker 6: and I'm like, I'm just a go between between one
Speaker 6: of Open a Eye's products and another Opening Eye product.
Speaker 5: Now like I'm the It was just hilarious.
Speaker 1: When it coast you is five dollars everything, right, But I.
Speaker 5: Got a nice little story out of it.
Speaker 4: Read this is this, this sounds like a nightmare scenario.
Speaker 6: It was a It was a total nightmare scenario, but
Speaker 6: like also fascinating and and and still other than telling
Speaker 6: Opening Eyes pr you know, one of their what what
Speaker 6: did those people call it?
Speaker 5: The puppet? The media puppet or the media puppet.
Speaker 6: About this, like I never interacted with the human Like
Speaker 6: I was the only human involved in this whole, Like
Speaker 6: I still have not heard from like a Open. I
Speaker 6: don't even know if Opening has human customer service reps.
Speaker 6: So it was, you know, it was it was a
Speaker 6: learning experience for me. And also just like I thought
Speaker 6: it was relevant because we're having this big discussion now
Speaker 6: in one of the big discussions in AI is the
Speaker 6: cost of all this stuff, and companies are trying to
Speaker 6: figure this out, and I'm like this must be happening
Speaker 6: just constantly every day at these companies, Like people are
Speaker 6: just on purpose or not on purpose, like putting these
Speaker 6: things into these loops and wasting a bunch of just
Speaker 6: you know, and it's it's a point in time, like
Speaker 6: I think this stuff will all get solved, But I'm
Speaker 6: also kind of like somebody's got to figure this out
Speaker 6: for consumers. It's just not it's not a consumer product yet,
Speaker 6: and I really think in the next like year or two,
Speaker 6: there's going to be some company that just comes out
Speaker 6: of nowhere and makes like go for ai that's just
Speaker 6: like the ultimate consumer product, and everyone's like, holy shit,
Speaker 6: this is it, and I don't. I mean, I used
Speaker 6: to think it would be Opening Eye, it would be
Speaker 6: Google or somebody, but I'm starting.
Speaker 4: To think, well, else, it's in the market. I mean,
Speaker 4: they should be building now, we would know about it.
Speaker 6: Someone's working in stealth mode or something. But there's a
Speaker 6: there's a Larry and Serge who are probably like just
Speaker 6: writing their application to Stanford, who are going to create
Speaker 6: this product?
Speaker 5: Is my guess.
Speaker 1: Natascha Taita, what does Read's Quoti retail make you think?
Speaker 4: I'm very parent, I'm very into like cloud code and
Speaker 4: I try. I like made agents and stuff, and I
Speaker 4: have little automated things. But I am so paranoid about
Speaker 4: exactly what we described because I feel like I hear
Speaker 4: from friends about like similar experiences. I just think it's
Speaker 4: really easy to like spend money and not realize it
Speaker 4: if you're not super adept at these tools, which I
Speaker 4: certainly am not, and most consumers are not so yeah,
Speaker 4: I do think that this is a problem that the
Speaker 4: companies are going to have to solve. Ultimately, there's just
Speaker 4: gonna be a lot of angry customers, if you know,
Speaker 4: especially when you're starting to use chattupet for shopping, which
Speaker 4: I do all the time now, Like I'm very careful
Speaker 4: to go to like third party services. But you know,
Speaker 4: I just readd my living room and I was taking
Speaker 4: pictures of products and saying, hey, you know, chattupt find
Speaker 4: me dupes. Basically I can't, you know, cheaper versions of everything.
Speaker 4: But there's a world where I could just say, like,
Speaker 4: here's a picture, find cheaper versions of everything, go off
Speaker 4: and buy it and order it. I would actually love that.
Speaker 4: That would save me so much time. But without safeguards
Speaker 4: in place, like I would be so nervous. It'd probably
Speaker 4: ordered me five hundred chairs, and yeah, I'd broke.
Speaker 3: For some reason, it just made me think. I feel
Speaker 3: like there was some kid who racked up a massive
Speaker 3: candy crush bill, like without his parents knowing, and there
Speaker 3: must be so many of those happening right now, except
Speaker 3: the cap is like insanely high. I mean, yeah, I
Speaker 3: think like the lack of fine brain controls. Like even
Speaker 3: just hearing that it went to your credit card read
Speaker 3: is just like you know, because you hear about like
Speaker 3: you hear about things going awry all the time. But
Speaker 3: now I've got to go find the kids who racked
Speaker 3: up like a million dollar token max bill. Maybe Larry
Speaker 3: and Surge the future Larry and Sergey are actually just
Speaker 3: in debt.
Speaker 5: Now that would be a great story.
Speaker 6: I mean, but yeah, I think it's like, but I'm
Speaker 6: also kind of like, yeah, but what's the alternative? The
Speaker 6: alternative is like you as a tech reporter, like you
Speaker 6: just don't use this stuff, and like you're not you know,
Speaker 6: you're not You're not seeing it. Like if you don't
Speaker 6: I think, if you don't use it, and you're not.
Speaker 1: Able to expense the five hundred bucks the sem before
Speaker 1: read in service of your general.
Speaker 5: Expensing the five hundred bucks because I wrote about it.
Speaker 5: Now it's for a story.
Speaker 1: No, I'm just curious, what where does this? I mean,
Speaker 1: just as we close, like, where does this fit in
Speaker 1: the in the larger kind of token maxing to token minning?
Speaker 5: Ceasaw?
Speaker 1: I mean, there was a a New York Times story
Speaker 1: recently with the headline tech workers maxed out their AI use.
Speaker 1: Now they're trying to minimize it. And it's basically about
Speaker 1: perverse corporate rewards and how employees were like basically promoted
Speaker 1: on the basis of how many tokens they've burned. And
Speaker 1: then the tech companies realize this is an extremely bad idea,
Speaker 1: And now tech companies like Meta are trying to encourage
Speaker 1: employees to use their internal models and not always use
Speaker 1: the most advanced available you know models, because that max
Speaker 1: is up too many tokens. Uber I think burnt through
Speaker 1: all of its AI budget in the first four months
Speaker 1: of this year. Like how does your experience kind of
Speaker 1: map to the larger discourse about token token maxing? And
Speaker 1: and I guess finally, is this a like a headwind
Speaker 1: for open ai? And you know, Anthropic and Google in
Speaker 1: terms of people come much more posimonious with wanting to
Speaker 1: use this stuff.
Speaker 5: Yeah.
Speaker 6: I mean, so there's two there's two questions there, and
Speaker 6: I think one is like there's a there's a pattern
Speaker 6: in all of this where a new a new capability
Speaker 6: or new product comes out and then it's very inefficient,
Speaker 6: it's very expensive, and then it and then it quickly
Speaker 6: you know, becomes about efficiency, right, and I think we
Speaker 6: had the new thing here was was the harnesses the
Speaker 6: ability to use like way more tokens, which started sort
Speaker 6: of at the end of last year, and people got
Speaker 6: very excited about it. Companies set up these leader boards
Speaker 6: that you're talking about, and which of course is going
Speaker 6: to just incentivize people to waste as much tokens as possible.
Speaker 5: And now we're already.
Speaker 6: At the point where the companies are like, okay, we
Speaker 6: kind of understand this, now, now let's make it efficient.
Speaker 6: So it's a very natural kind of up and down cycle.
Speaker 6: And there will probably be something that happens, like at
Speaker 6: the end of this year. You know that that causes
Speaker 6: the same thing, the token usage will spike again. And
Speaker 6: I don't know what that is, but maybe Natasha knows.
Speaker 3: But like, no, I think that. I mean, first of all,
Speaker 3: everyone told them the leader boards were a bad idea
Speaker 3: when they did them, Like this was not like a say, and.
Speaker 5: They knew it. They knew it, they knew that would happen.
Speaker 3: But I'll just say I think that this is I'm
Speaker 3: seeing at least more debate around like you know read
Speaker 3: you've called it like the token economy. I think I've
Speaker 3: seen more debate around whether that is the way to
Speaker 3: charge for these chatbots. I mean you even see like
Speaker 3: Alex Karp, the CEO of Palenteer, talking about like why
Speaker 3: would you pay for that when they are going to
Speaker 3: you know, why would you pay for using this intelligence
Speaker 3: when they can then take your data and you know,
Speaker 3: like replicate what your company does or you know, if
Speaker 3: you're a biotech like take that kind of information. And
Speaker 3: I think that's really interesting, like pushing for a different
Speaker 3: kind of payment or business model.
Speaker 6: Yeah, I mean there's also something happening here, which is
Speaker 6: like there's what companies use internally with their software development teams,
Speaker 6: and then what the wider teams use, and then and
Speaker 6: then what they use for their customer facing products, and
Speaker 6: of course the software development teams, I think they still
Speaker 6: have all the best models. They're they're not skimping. Nobody
Speaker 6: that I talked to is like skimping on that really,
Speaker 6: and then on the customer facing stuff, Like there was
Speaker 6: a CTO of like a major consumer brand who told
Speaker 6: me that, you know, they are using all these Chinese models,
Speaker 6: but they don't even want to tell the world that
Speaker 6: because they'll get a letter from the commerce department. So
Speaker 6: it's just I think it's like this is not like
Speaker 6: one blanket thing. There's a lot of nuance happening there.
Speaker 6: So I think, you know, Alex Krp's point was was right.
Speaker 6: It's just that, like the the market is so large
Speaker 6: that I don't think it matters for these companies like Anthropic,
Speaker 6: like the the they have only started to really scratch
Speaker 6: the surface in terms of like penetration, so there's gonna
Speaker 6: be the pie is growing so big that they're fine. However,
Speaker 6: I do think this is a headwind for like the
Speaker 6: larger the company. You're more at a disadvantage here because
Speaker 6: a small startup can come in and they can they
Speaker 6: don't this doesn't matter, Like these these startups in Silicon
Speaker 6: Valley will use as much as many tokens as they
Speaker 6: want of the best models because they're they're first of all,
Speaker 6: they're venture backed. Like it doesn't you know, they're not
Speaker 6: like looking at profitability at this point. They're looking at growth,
Speaker 6: and so they're just gonna max out on the tokens.
Speaker 6: And a big company with forty fifty thousand employees or
Speaker 6: more like they can't do that. The costs just go
Speaker 6: so they're ultimately like they're more susceptible for disruption, and
Speaker 6: I think you're going to see a lot of that.
Speaker 6: Like the regulated industries, maybe that happens more slowly, but
Speaker 6: in non regulated spaces it's going to happen fast.
Speaker 1: That's all we have time for today. Thank you all
Speaker 1: so much for joining us.
Speaker 2: Thank you guys, thanks for having.
Speaker 1: Us for tech stuff. I'm mos Voloshin. This episode was
Speaker 1: produced by Eliza Dennis and Sina Ozaki. It was executive
Speaker 1: produced by me and Julia Nutta for Kaleidoscope and Katrina
Speaker 1: Novel for iHeart Podcasts. Our engineer today was Eli Bronstein.
Speaker 1: Jack Insley makes this episode and Kyle Murdoch wrote our
Speaker 1: theme song. A special thank you to Taylor Lorenz, Natasha
Speaker 1: Tiku and read Albergotti. Please check out all the work
Speaker 1: they put out into the world. We're lucky to call
Speaker 1: them friends at the pod