Speaker 1: Also media, Hello and welcome to Better Offline. I'm, of
Speaker 1: course your host ed Zitron. That's right, I'm back all
Speaker 1: of you. And to be clear, when I said I
Speaker 1: was having wisdom teeth surgery, I meant I was having
Speaker 1: more of them added to make me more powerful. The
Speaker 1: operation was the success. I apologize also for doing this
Speaker 1: week in reverse with the longer episode on the Friday
Speaker 1: and the monologue on the welth I guess Tuesday. I
Speaker 1: guess I had to put out open ayes financials at
Speaker 1: the last minute, and well, here we are. Subscribe to
Speaker 1: the newsletter ideally the premium support your local zitron and
Speaker 1: as ever, if you have anyone else's financials, you can
Speaker 1: catch me on ezitron Dot seventy six on signal. It's
Speaker 1: going to be in the episode notes. Anyway, Today, now
Speaker 1: that I'm done drinking milkshakes, I'm joined by the wonderful
Speaker 1: computer science professor and writer Cal Newport to talk about AI,
Speaker 1: but specifically to talk about what he calls doom trolling
Speaker 1: in one of the best pieces I have read on
Speaker 1: AI ever.
Speaker 2: Cal Well, first of all, I'm disappointed I've sent you
Speaker 2: my financials multiple times in your signal chat and yet
Speaker 2: you have not written ten thousand words about them yet.
Speaker 2: So I don't know what you're doing here.
Speaker 1: You haven't lost eleven billion dollars or anything. You need
Speaker 1: to get those numbers down. No, but I write about
Speaker 1: bad companies.
Speaker 2: I wrote it off though.
Speaker 1: It's okay, Yeah, you moved it to an SPV.
Speaker 2: That's right. I may have spent eleven billion dollars, but
Speaker 2: really I was profitable. It's just a matter exactly you
Speaker 2: look at it.
Speaker 1: But let's talk doom trolling, because you have this fantastic
Speaker 1: piece that talks about the fact that all these companies Anthropic,
Speaker 1: Open AI, even Deep Mind to an extent, just keep
Speaker 1: selling things based on the destruction that AI will allegedly cause,
Speaker 1: though they appear to be talking about another kind of
Speaker 1: technology entirely when you think about him.
Speaker 2: Yeah, I mean, look, I think we have forgotten are
Speaker 2: overlooking how strange this is, right, because what we have here,
Speaker 2: let's just let's abstract away from the particular products here
Speaker 2: and just say what do we have here? More abstractly,
Speaker 2: as a business story, we have consumer product companies who
Speaker 2: are on a regular basis through interviews and official white
Speaker 2: paper releases trying to terrify their consumers about the harms
Speaker 2: that their products are going to cause. And if this
Speaker 2: was any of the industry, this would be completely mind boggling.
Speaker 2: The example I had in the in the New York
Speaker 2: Times op ed is like if the CEO of Ford,
Speaker 2: You'll put out a white paper and was like, we're
Speaker 2: really worried that our f one fifties are going to
Speaker 2: start catching on fire and burning people to death. But
Speaker 2: you know, automotive technology, we can't impede it, and we're
Speaker 2: just going to get back to making them.
Speaker 1: We're like, we've got to go two thousand miles an
Speaker 1: hour at some point in the future. Yeah, we would
Speaker 1: just ban Yeah, we'd ban it.
Speaker 2: No, what's consumer say, No one would buy?
Speaker 1: Yeah, just it's really it is something that's gen I
Speaker 1: loved the term doom trolling as well because it really
Speaker 1: he frames it well because it's it's all part of
Speaker 1: a larger complex than never talking about what they're building today.
Speaker 1: It's always this theoretical, scary thing that might it's always
Speaker 1: two years or two minutes away, depending on who you're
Speaker 1: talking to.
Speaker 2: Yeah, So what I did was, I said, let's actually
Speaker 2: think this through because I'm also a digital ethicist. It's like,
Speaker 2: let's actually think this through with our ethics had on, right, Like,
Speaker 2: let's think through what particularly anthropic I think is the
Speaker 2: most systematic in this propaganda campaign that you know, I
Speaker 2: analogized that to a cat leaving a dead bird on
Speaker 2: your doorstep and walking away. They just every two to
Speaker 2: three weeks want to drop some sort of doom troll
Speaker 2: document to get people freaked out, and then they just
Speaker 2: back away and go back to what they're doing. And
Speaker 2: I kind of got fed up with this, so I said,
Speaker 2: let's do a moral analysis about this strategy. And I said, look,
Speaker 2: there's only two options here. Option number one. They really
Speaker 2: do think that technology they're building has put them on
Speaker 2: a trajectory to and the best case, destroying the economy
Speaker 2: or in the worst case, wiping out humanity, in which
Speaker 2: case every ethical system of you know, any merit would
Speaker 2: say you have no other choice but to stop immediately
Speaker 2: what you're doing and use all of your time, energy
Speaker 2: and treasure to try to stop other people from doing
Speaker 2: the same. The second case is you don't fully believe it,
Speaker 2: which means you are laundering the anxiety of hundreds of
Speaker 2: millions of people to try to make the bank balances
Speaker 2: of a vanishingly small number of original stockholders larger, which
Speaker 2: I think is equally as morally monstrous. There is no
Speaker 2: moral analysis that says trying to frequently terrify people about
Speaker 2: the things you're building while making no adjustments to the
Speaker 2: things you're building, there's no moral calculus that says that
Speaker 2: makes sense. But because AI is new and people don't
Speaker 2: understand the technology, and it's shiny, and we saw James
Speaker 2: Cameron's movies, we somehow treat this different. And I say
Speaker 2: it's not different. This is a product that they are selling.
Speaker 2: They are not the stewards of an inevitable technology. They
Speaker 2: are building products according to business plans and trying to
Speaker 2: sell them. They need to act like consumer product companies.
Speaker 2: We need to demand that they act like consumer product companies.
Speaker 1: Well, another thing that you have in the piece is
Speaker 1: that it's unclear, like that dichotomy is, it isn't clear
Speaker 1: why they're even doing this to other than to make money.
Speaker 1: And also they're not acting as if any of this
Speaker 1: is true. They're not doing anything that suggests that they're
Speaker 1: actually concerned. You mentioned Daria Amida's thing about how oh
Speaker 1: we might need to pause in AI or we should
Speaker 1: have that option. But that isn't actually what he was suggesting.
Speaker 1: He was suggesting something entirely different.
Speaker 2: Well, what they said was and they got headlines I
Speaker 2: had to look up for the factchecker. They got every
Speaker 2: major publication to say anthropic calls for worldwide pause and
Speaker 2: AI development. You read that section of their white paper,
Speaker 2: which by the way, reads like sci fi that a
Speaker 2: ninth grader wrote, right, it's not. It's pseudo scientific at best.
Speaker 2: It's not even really scientific. It's it's I find their
Speaker 2: white paper sort of laughable. But they said in it,
Speaker 2: we should do a worldwide pause because the thing we're
Speaker 2: building is so dangerous dot dot dot. But we can't
Speaker 2: do that unless everybody in the world does the same thing,
Speaker 2: including China essentially, and if that's not the case, we
Speaker 2: have to just keep going full speed. So what they
Speaker 2: said was we should do worldwide pause, but of course
Speaker 2: we're not going to. It's really what they were saying there.
Speaker 2: But the point is, all right. So the report we're
Speaker 2: talking about here came out a few weeks ago. If
Speaker 2: you haven't read it, this is basically the report where
Speaker 2: they just out of nowhere wrote a white paper with
Speaker 2: a bunch of charts. They always have charts where they
Speaker 2: were saying, yeah, pictures and an animation of.
Speaker 1: I was just thinking of the little replicating thing. We'll
Speaker 1: get to self learning AI in a second.
Speaker 2: But the report basically said, huh, cloud code, which is
Speaker 2: a harness on top of an LLM for for helping
Speaker 2: to run computer program.
Speaker 1: Just to be clear, a harness is basically a series
Speaker 1: of scripts that runs on top of the LM.
Speaker 2: It's a computer program that prompts an LLM and then
Speaker 2: parses the response to LM gives and then actually takes
Speaker 2: actions on your computer based on what it reads. Right,
Speaker 2: So it's yeah, and that's a whole other conversation about
Speaker 2: the coding world and how much is the harness and
Speaker 2: how much of the LM. But they basically said, this tool,
Speaker 2: which is a a harness, it's an LM, it's some
Speaker 2: other things. This tool that computer software developers are using
Speaker 2: is getting really good, and you know that could be.
Speaker 2: We think that it might be. Looks like it's getting
Speaker 2: closer to being able to improve itself. And if it
Speaker 2: does recursive self improvement, which by the way, is that
Speaker 2: it's at the core of philosophical superintelligence, right, the sort
Speaker 2: of a Silicon Valley cult, like thinking about computers replacing us.
Speaker 2: If it does, hey, maybe that'll be good for humanity.
Speaker 2: It'll cure all diseases. But also we could lose control
Speaker 2: of it. Thank you, have a nice day. And they
Speaker 2: walked away. They just put out that report. That's the report.
Speaker 1: And the thing that I like about it as well
Speaker 1: is I think that we're in the next big thing
Speaker 1: is going to be RSI. Because so just to be
Speaker 1: clear for the listeners, our recursive self improvement is just
Speaker 1: AI that builds itself. It is a theoretical concept. It
Speaker 1: has not happened yet. But mysteriously within the same six
Speaker 1: seven day period this came out and Sam Moltman said
Speaker 1: to his staff in a Slack message he knew would leak,
Speaker 1: Oh yeah, yeah, we might delay our IPO if we
Speaker 1: have signs of RSI, if we have signs of AI
Speaker 1: that can build itself. Meta a year ago said in
Speaker 1: Personal Superintelligence, we're seeing the early signs. And they're doing
Speaker 1: this because, in my opinion, maybe disagreek how they've run
Speaker 1: out of ideas to the point that they're literally going
Speaker 1: back to what Sam Moltman said in twenty nineteen. Yeah,
Speaker 1: we'll just ask the AI what to do. Yeah, lo'st
Speaker 1: the AI out build. These guys have so few ideas that,
Speaker 1: just like Christ, can't we just make the AI.
Speaker 2: Do it OURSI is to get out of jail free
Speaker 2: card from a technical expertise standpoint. So you know, we
Speaker 2: saw a lot of this a year or two ago
Speaker 2: when we had like Project twenty twenty seven. We had
Speaker 2: these sort of post GPT for warnings of you know,
Speaker 2: existential crisises. It's a get out of jail free card
Speaker 2: because if you ask, like the authors of twenty twenty seven, well,
Speaker 2: explain to me what a system that is smarter than
Speaker 2: humans and can fire the missiles and over what does
Speaker 2: it look like? How you architect it? Like, what what
Speaker 2: allows it to do it? And their answer is we
Speaker 2: don't have to figure that out. The AI will right now.
Speaker 2: This is perfect not the history problem. We could history
Speaker 2: lesson this this although this goes all the way back
Speaker 2: to the nineteen sixties. So there's a there's a statistician
Speaker 2: applied mathematician named I. J. Good and he wrote a paper.
Speaker 2: It's really like an information theory paper, but he wrote
Speaker 2: a paper where he said, we, if we think about it,
Speaker 2: called it ultra intelligence. Really, all you have to do
Speaker 2: is build a machine smart enough to improve itself, and
Speaker 2: then that machine will improve itsself. That machin will improve itself,
Speaker 2: and then we'll get to ultra intelligence. So we don't
Speaker 2: have to figure out how to do it. The danger
Speaker 2: lies in just making a machine fast enough. So it's
Speaker 2: like a bit of a semi thought experiment that became
Speaker 2: the er document of what I call like the philosophical
Speaker 2: superintelligence cultists. So this sort of where the philosophers took
Speaker 2: this idea away from the technologist and began turning it
Speaker 2: into a sort of digital erascholasticism, where they sort of
Speaker 2: abstracted away all technical details and just started building list
Speaker 2: of list of list of lists of all the different
Speaker 2: things that could happen. They just did a bunch of
Speaker 2: thought experiments about super intelligence and recursive self improvement. They
Speaker 2: then merged with the utopian singularity people that were coming
Speaker 2: out of sort of Kurzweil's orbit, and you kind of
Speaker 2: created this group that was non technical. This is not engineers,
Speaker 2: this philosopher's ethicist and sort of gadflies and you know,
Speaker 2: you know, speculative fiction writers and fan fiction writers like Yukowski,
Speaker 2: and they kind of created this sort of dumorous utopian cult.
Speaker 2: And then what happened is that group was like, we
Speaker 2: should create a nonprofit to look at these ideas, and
Speaker 2: we'll take sam Altman and put them in charge. We'll
Speaker 2: call this Open Ai. And then suddenly they were like,
Speaker 2: oh my god, this jin Ai thing is working better
Speaker 2: than we thought, and we have now a one of
Speaker 2: the you know, someone coming out of this general kind
Speaker 2: of cultish let's eschatological you know, in times could come
Speaker 2: through computers, was suddenly and accidentally in charge of the
Speaker 2: most important company in AI. And one of his you know,
Speaker 2: lieutenants left to start his own company. And his name
Speaker 2: is Dario Amaday. So these the two major players we
Speaker 2: have in AI right now, come out of philosophical superintelligence,
Speaker 2: which uses this idea of RSI, and it's like a
Speaker 2: it's like a parable, it's like an abstraction. This is
Speaker 2: not like a serious it wasn't like a technical concern.
Speaker 2: It was a like a thought experiment, and they are
Speaker 2: completely influencing the like the daily lives and psychological well
Speaker 2: being of hundreds of millions, and like the economic health
Speaker 2: of the entire world. So it's it's a really an
Speaker 2: interesting world we've ended up in now. And all of
Speaker 2: which is to say, I don't think we're talking enough
Speaker 2: about how weird these people are. I think we keep
Speaker 2: steering the conversation back to the technology itself. We keep
Speaker 2: steering the conversation back to like, well, could it take
Speaker 2: over or not? And we need to spend some more
Speaker 2: time saying, wait, who are these people and what are
Speaker 2: they doing? They're doom trolling. This makes no sense. No
Speaker 2: company ever in the history of companies was trying to
Speaker 2: troll their own customers to be afraid of their products.
Speaker 2: They're treating their technology like a religious object. Like, we
Speaker 2: are underestimating the weirdness of these people, and when we
Speaker 2: put that into focus, it completely changes I think our
Speaker 2: own psychological relationship with what's going on right now.
Speaker 1: Yeah, and I fully agree that it is almost I
Speaker 1: think it's a distraction as well from the actual reality
Speaker 1: of what these things are, and I think it's a
Speaker 1: deliberate one. I naturally fall on the side of just
Speaker 1: thinking these people are deeply cynical. I've met people who
Speaker 1: think that Dario amade believes this stuff. I actually don't.
Speaker 1: I don't think they believe a fucking thing. I think
Speaker 1: that however you feel is however you feel. But I
Speaker 1: think that they correctly realize that they could very easily
Speaker 1: make people talk about something that doesn't exist just by
Speaker 1: mentioning it. How I saw a bunch of headlines that
Speaker 1: were like, Wow, anthropics seize the path to self building AI.
Speaker 2: No, they don't.
Speaker 1: Other than the fact they had a single set and
Speaker 1: saying I see the path to this. They don't have
Speaker 1: any proof, they don't have anything, and yeah, you're right.
Speaker 1: These people are weird. I have seen people on XT,
Speaker 1: everything app Twitter actually posting that we are in AI
Speaker 1: twenty twenty seven, that everything matches up with AI twenty
Speaker 1: twenty seven. I want to be clear that and the
Speaker 1: reason I mentioned this is I just brought this up,
Speaker 1: is that in early twenty twenty six, in AI twenty
Speaker 1: twenty seven, they say that Agent one has already started
Speaker 1: doing its own research. That's not happening. None of that's happening.
Speaker 1: In fact, none of this is happening. These people are
Speaker 1: just weird fantasists. I think. While there's the poison of
Speaker 1: the philosophers in them. I just think that Warrio Ama
Speaker 1: Day and Sam Altman both I think that they both
Speaker 1: realize that journalists series ly con the investors reasonly con
Speaker 1: that you can just say shit and anyone will print
Speaker 1: it and believe you because you're a CEO.
Speaker 2: I don't know how to answer that question, because the
Speaker 2: more I look into how quasi religious and cultish this
Speaker 2: sort of philosophical superintelligence community is, the more I get
Speaker 2: confused about what these people actually believe. So go spend
Speaker 2: time in Silicon Valley, Go spend time.
Speaker 1: To be clear, when I say they, I'm referring specifically
Speaker 1: to Amidei and Olman. Well, so they could they fully
Speaker 1: believe that there are tons of people in the valley
Speaker 1: that believe is that they believe that this is the
Speaker 1: path to that Yeah, I think just to be abundantly clear,
Speaker 1: I don't think everyone is cynical. I think they've been
Speaker 1: easily conned. I believe think that they are cynical.
Speaker 2: Now I could I could believe that as and I
Speaker 2: could believe that as they their companies got larger and
Speaker 2: their leadership roles got larger, that they have stepped away
Speaker 2: like oh this is this was a little naive. That's possible.
Speaker 2: But also I saw a news report today that Dario
Speaker 2: I don't know if this is true, so I'm just
Speaker 2: this a headline. I haven't read yet that he has
Speaker 2: like one direct report. Yes, so I don't think you know,
Speaker 2: he's he's necessarily in the war room with a thousand
Speaker 2: engineers and like doing a lot of business strategy here too.
Speaker 2: So it's possible that these guys are more isolated than
Speaker 2: than we think. I don't know, but I mean, if
Speaker 2: he has.
Speaker 1: One direct report, he's not doing anything. He's just what
Speaker 1: he's going to meetings. Sorry what you're saying.
Speaker 2: Yeah, but but but coming back to it, like what
Speaker 2: what are the whether they how much they believe or not?
Speaker 2: And I hope they don't because again, I think it
Speaker 2: would be morally monstrous if you really believe this that
Speaker 2: you're like, yeah, but I want to win an IPO,
Speaker 2: like I want to make more exactly, Yeah, I think
Speaker 2: that would be morally monstrous, right, But like if you
Speaker 2: think about what the benefits would would be. They've this
Speaker 2: has led to a sort of discourse environment essentially, where
Speaker 2: you have a normal technology that's that's advancing along a
Speaker 2: jagged frontier, which is what happens with like new technologies.
Speaker 2: There's like certain applications where they feel like they're making
Speaker 2: progress and a lot of other applications where they're stuck.
Speaker 2: And so like they've been making progress with these these
Speaker 2: coding systems that that's kind of a scenario which there's
Speaker 2: a lot of room for play with these type of systems.
Speaker 2: A lot of other things like non coding agents, et
Speaker 2: Cetera's not going so well. In math, they're making progress,
Speaker 2: but that's such a narrow field that it's not like
Speaker 2: it's not economically relevant to them. So you could see
Speaker 2: this as just like a normal technology. And in chat
Speaker 2: chat search which is basically you know, running Google search
Speaker 2: results through a language model, the natural language search basically,
Speaker 2: so like places they're making progress, other places they're stuck.
Speaker 2: That's like an interesting normal technology story. But if you
Speaker 2: layer on this other sort of eschological story about we
Speaker 2: are on our way to utopia or dystopia and we
Speaker 2: keep trying to keep you like upset about that. Think
Speaker 2: about the way now when you're online, how people think
Speaker 2: and talk about AI. It's all about if something made,
Speaker 2: if there's some new feature that works, if there's some
Speaker 2: new chart that goes better, that's evidence for the prophecy
Speaker 2: is true. And then so then that that puts so
Speaker 2: much more power. Like if you otherwise just came to
Speaker 2: me and said like, hey, we have this like advance
Speaker 2: in our coding harnesses, I'd be like, I don't care.
Speaker 2: I'm not a software developer. But if you say, oh,
Speaker 2: that means that the prophecy of the RSI gods is true,
Speaker 2: Now suddenly that is really eventful, and I'm really excited
Speaker 2: or scared. But I'm giving a lot of import to
Speaker 2: your company. So I do think it generates way more
Speaker 2: sense of import on these otherwise, like I think very
Speaker 2: often very minor and jagged improvements. I mean, I think
Speaker 2: with other companies, if you kept putting out graphs that
Speaker 2: you created on measures you created, I was like, I
Speaker 2: don't know what that means. Tell me when like, my
Speaker 2: business is going to be doing better? Right, So there
Speaker 2: is a huge advantage regardless of intention. There is a
Speaker 2: huge advantage to have a doom troll because as long
Speaker 2: as people are are terrified or extremely excited about this future,
Speaker 2: and then there's a secondary advantage. This is cynical, but
Speaker 2: I just want to throw it out here ed because
Speaker 2: I'm sensing this more. I think there's a whole other
Speaker 2: substrata of sort of like tech adjacent people who are
Speaker 2: who are you know, online or not, who got caught
Speaker 2: up a little bit in that type of hype type
Speaker 2: vision and now their pride is engaged, and so now
Speaker 2: they will interpret I encountered this all the time, not
Speaker 2: even skepticism, just literally saying oh, I don't think that tool.
Speaker 2: People are struggling to use that tool well, or this
Speaker 2: might this is not a very profitable model here, They'll
Speaker 2: have to find another model. They interpret it as like, well,
Speaker 2: this is embarrassing because I was just telling my siblings
Speaker 2: and everyone at dinner last night about because I used
Speaker 2: cloud code, and I think it's how nine thousand and
Speaker 2: I was just so excited about it. It's cognitive dissidents,
Speaker 2: and I'm embarrassed. And so now I am a warrior
Speaker 2: for the prophecy. So I think you also have that
Speaker 2: substrate who are like, yeah, but what about and then
Speaker 2: you know some metric that went up or something like that,
Speaker 2: and that's way and their benefit as well. You have
Speaker 2: a soldiers of marketers that are trying to basically protect
Speaker 2: their pride.
Speaker 1: Yeah, you've absolutely nailed it. It's the pride thing again,
Speaker 1: hadn't thought of that, But you're right the way that
Speaker 1: they react when you give that. I mean, I just
Speaker 1: published Open Ais Financials and the Pretzels. They're contorting themselves
Speaker 1: into the ways in which it's Silicon value has always
Speaker 1: been this proudly atheistic, meritocratic, rugged pragmatism world, or at
Speaker 1: least this is how they sold themselves. It's not about
Speaker 1: your feelings, it's about can you ship co? Is your
Speaker 1: software good? Does it make money? But something about this
Speaker 1: is really entered into the realm of religiosity, not even faith,
Speaker 1: because faith is a much stronger and more fundamentally sound thing.
Speaker 1: This is religious cultism. But the God is capital Like
Speaker 1: you can say it's Ai, but the God is capitalism.
Speaker 1: It is this idea that the big money people who
Speaker 1: are constantly threatening you, like they just they're like, yeah,
Speaker 1: it's gonna kill you or it's gonna take everything. I
Speaker 1: guess it's Pascal's wager as well. It's well, if I
Speaker 1: don't know if this is the case, but if they're
Speaker 1: all saying it, and if I'm wrong, the Roco's bassilisk
Speaker 1: will drop an anvil on me. I guess it's it's frustrating.
Speaker 1: But also I don't know what these people will do
Speaker 1: if the bubble, when the bubble burst. I don't know
Speaker 1: why I'm saying if when it bursts, I don't know
Speaker 1: what these people do because their worldview is built around
Speaker 1: existing outside of reality, almost like that just this chatbot
Speaker 1: will become God will become autonomous, and because I was
Speaker 1: there saying it will become autonomous, I will be given something.
Speaker 1: It's unclear what the reward is other than simpering, and
Speaker 1: there's something in it for them, that's the thing. It's like,
Speaker 1: I guess maybe it's community. They're finding. It's replace AI
Speaker 1: with God, just like with church, and they would mock
Speaker 1: whoever was talking about it. It would mark anyone who
Speaker 1: took a sign of well, this is proof that God exists.
Speaker 1: But they would look at a meter study and they go,
Speaker 1: well that's proof that AGI is coming. Yeah, and it's
Speaker 1: so strange.
Speaker 2: Yeah, it's like, hey, look at how complicated the structure
Speaker 2: of the leaf is. It's Paaley intelligent design, like God exists.
Speaker 2: Because of the argue from the design, they would mock that, right, Yeah,
Speaker 2: now they're two seconds. Yeah, but like, hey, we can
Speaker 2: draw a unicorn in ticks using like a GBT four prompt. Yeah,
Speaker 2: the AI god is coming.
Speaker 1: But it's it's so strange as well. I've been writing
Speaker 1: about Silicon Valley bubble recently, this idea that basically the
Speaker 1: valley itself is a bubble inflating that will burst going
Speaker 1: back ten to fifteen years because they ran out of
Speaker 1: new ideas, that they've not had anything good, not had
Speaker 1: a new smartphone or a new app store or a
Speaker 1: new cloud computing in a while. And I think that
Speaker 1: that's what's fed into this as well, because you've got
Speaker 1: that kind of desperation that we've run out of big
Speaker 1: ideas and then added this layer of fearmongering plus childish philosophy. Ye,
Speaker 1: very like, very much like this shit must have hit
Speaker 1: so this would hit so hard if I was twelve. Yeah,
Speaker 1: it goes to it like holy shit. Yeah, Oh I'm
Speaker 1: using this computer today and tomorrow it will be ultra intelligent.
Speaker 1: I'm so smart for finding this first. It's deeply sad.
Speaker 1: I didn't think I could be more embarrassed by Silicon Valley,
Speaker 1: but the fact that these people are being this weird.
Speaker 1: Is that I like, we need to call them weird.
Speaker 1: It's actually kind of unsettling as well. It's unsettling the
Speaker 1: also that you are one of the few people just
Speaker 1: saying it that everyone else is acting like this is
Speaker 1: normal or rational or in any way logical.
Speaker 2: Well, I'm trying to activate a bigger crowd here, right
Speaker 2: and good. Here's what I think we need. I think
Speaker 2: you're you're used to the word embarrassing is key because
Speaker 2: I'm the trend I'm seeing which I don't like. Is
Speaker 2: this idea of the people who instinctually do not like
Speaker 2: what's going on with these AI companies are trying to
Speaker 2: match the intensity and high of what they're saying, but
Speaker 2: going the other direction, right, And so they're like they're
Speaker 2: destroying everything, right, Like the environment will all be destroyed
Speaker 2: by the data centers, and all education is going to end.
Speaker 2: And the idea being is like, I need to be
Speaker 2: as like as intense about all of the terribleness that
Speaker 2: your technology will cause as you are about like the
Speaker 2: good it will cause. Where I don't think that's playing
Speaker 2: into their hands. I think the right reaction is ridicule
Speaker 2: and embarrassment. This is where things need be if you
Speaker 2: read anthropic white paper, you read these things, it's we're
Speaker 2: now going to go through we envision four possible futures
Speaker 2: for where this technology could come, and then they just
Speaker 2: make up like four or twelve year old sci fi.
Speaker 2: Maybe the technology will take over, but it will be
Speaker 2: in more of a sort of a church Hillian mold,
Speaker 2: and we'll wear top hats and that we could imagine
Speaker 2: a council of humans, but they would probably be like
Speaker 2: one third humans and two third robots. And you know,
Speaker 2: like just like work, it's just it's just nonsense, right,
Speaker 2: Like this's goofy. It's not it's not philosophy, it's not
Speaker 2: it's not technology, it's not political science. It's just goofy. Right,
Speaker 2: And I think that, and I said this in the
Speaker 2: op ed. I said, the zeitgeist on this should could
Speaker 2: change really quickly, right where it's like, all right, Dario,
Speaker 2: is the agi in the room with you right now?
Speaker 2: Can you see the agi in the room with your right?
Speaker 2: Cause it's that's like, that's the reaction I think we
Speaker 2: need plus forget that justify why I'm spending a thousand
Speaker 2: dollars a month on tokens, Like I don't want to
Speaker 2: hear your report about you know, hey Ford, I don't
Speaker 2: need your report about the future in which we'll have
Speaker 2: transformers that will help us police our streets. I am
Speaker 2: spending this much money on gas for my f one fifty, Like,
Speaker 2: tell me, like why that's worth it?
Speaker 1: Right?
Speaker 2: Like, just continually bring it back to you have really
Speaker 2: three products you're selling right now. Great, sell a product,
Speaker 2: it's interest, it's interesting technology, You're make it a good
Speaker 2: product out of it. Sell me what are the benefits?
Speaker 2: Why is it worth the money? And I will take
Speaker 2: for and if you better say it's not going to
Speaker 2: harm anyone, because otherwise fix it. So it doesn't like
Speaker 2: that should.
Speaker 1: Just be a yeah, that's your job to fix.
Speaker 2: That's your job to fix, and you don't. And just
Speaker 2: to throw in another moral argument I put in the
Speaker 2: New York Times up ed it is not actually a
Speaker 2: moral argument to say, there's this thing we want to
Speaker 2: do that's going to cause potentially a lot of harm
Speaker 2: and destruction. But that guy's going to do it, and
Speaker 2: as long as he's doing it, I get to do
Speaker 2: it too. That is what my eight year old says, right,
Speaker 2: he's like, I know, but like he stole the cookies.
Speaker 2: Why can't I steal it too? So I do not
Speaker 2: understand this argument. It was in the latest anthropic paper.
Speaker 2: We're building this technology, it might come alive and we
Speaker 2: might lose control of it. But how can we stop
Speaker 2: because China is also building the technology? And what if
Speaker 2: they build it and it comes alive and comes control.
Speaker 2: Why don't we get to do it too? What's the
Speaker 2: moral argument there? This is not like a nuclear deterrence
Speaker 2: argument that if you somehow destroy humanity first, that like
Speaker 2: what's the advantage there?
Speaker 1: Right?
Speaker 2: Like what game have you want? So I don't understand that.
Speaker 2: But my bigger point here is like, yeah, I think
Speaker 2: we need to activate. I'm working on a big article
Speaker 2: now for a trade magazine for engineers. Actually where I'm
Speaker 2: making this case. It's a cred of cure. You need
Speaker 2: to stand up and say that's nonsense. Let's talk about
Speaker 2: your real technology. I know about this because I think
Speaker 2: there's a lot of people like me who are on
Speaker 2: the wrong coast. We're not on the West coast. We
Speaker 2: don't we haven't gone to those cult meetings who are like,
Speaker 2: there's interesting technology here, and I want to hear about
Speaker 2: the products. And this is actually you and I maybe
Speaker 2: even differ a little bit. I en that piece by
Speaker 2: saying I'm pissed off because this should have been for
Speaker 2: a nerd like me, an exciting time. This is a
Speaker 2: cool technology. It's hard to make products out of it,
Speaker 2: but we should have been like it, kind of excited about.
Speaker 2: Oh I never thought about doing that with this. Oh
Speaker 2: isn't that cool? What's going on? Instead? Everyone is terrified,
Speaker 2: like you ruin this moment because of God knows why.
Speaker 2: We don't even have to get into it. So I
Speaker 2: think the right response from like actual engineers who are
Speaker 2: not part of that cult. And you know, you got
Speaker 2: to swallow your pride. I get it. You were really
Speaker 2: excited to your cousin after you use claude code like
Speaker 2: that's okay, swallow your pride a little bit and say
Speaker 2: this is doom trolling and it's embarrassing. Can we talk
Speaker 2: about the real technology, like a normal product. There's cool
Speaker 2: things you can build. Some stuff isn't working. I want
Speaker 2: to know if my stock market portfolio is going to crash, Like,
Speaker 2: let's just be normal. Stop the weird stuff. Just stop it,
Speaker 2: like you're a real company trying to sell real things.
Speaker 2: Let's talk about products like why I want to spark
Speaker 2: a revolution to bring things back to normal.
Speaker 1: I fully agree, and I even said it was on
Speaker 1: Chris Hayes his show a couple of weeks ago, and
Speaker 1: I said, I think we should legally ban boosters from
Speaker 1: using the future tents. I don't think they should be
Speaker 1: allowed to say if either. I don't think they should
Speaker 1: be able to say will. I think they should only
Speaker 1: be allowed to talk about what's coming out today. You
Speaker 1: can go two weeks in the future, Max.
Speaker 2: Yeah.
Speaker 1: I think that because when you read any story about
Speaker 1: AI that's positive, it always speaks in the future tents.
Speaker 1: It does not speak about what's happening now. And when
Speaker 1: they do speak about what's happening now, it's the veguest
Speaker 1: stuff in the world. It's like it can write or
Speaker 1: it can autonomously write software. Is the software good? No?
Speaker 1: Is it stable? No? Can you rely? Can you even
Speaker 1: actually do anything with it if you don't know what
Speaker 1: you're doing? No? Is the code overlea for Bose, Yes, Like,
Speaker 1: it's like there are tons of problems with it, but
Speaker 1: one sentence allows the generalities to grow. And also, I
Speaker 1: think there's a very obvious reason they don't talk about
Speaker 1: the present because it's kind of boring. It is like,
Speaker 1: it's just in comparison to a big, sexy autonomous robot.
Speaker 1: It's incredibly boring to say. But also the business doesn't work.
Speaker 1: The business is like, the business doesn't function if they
Speaker 1: have to talk about why it costs one thousand dollars
Speaker 1: of API tokens to do something. They have to talk
Speaker 1: about their business and they have to talk about what
Speaker 1: it does today, and the problem is with that is
Speaker 1: so my position on this is I don't mind on
Speaker 1: device models. I actually think that's all that's going to
Speaker 1: remain of LM's I think maybe we'll get some rag
Speaker 1: search stuff here and there. But I don't have an
Speaker 1: animous with the non data center ones. But I do
Speaker 1: agree that there needs to be that people do need
Speaker 1: to stop fighting fire with fire. As far as being
Speaker 1: like if they're gonna dooma, we're gonna say that all
Speaker 1: data centers are the worst, which they are. The water
Speaker 1: story is murky, but the environmental story is very real,
Speaker 1: especially with the behind the meat of gas turbines.
Speaker 2: But and the economic story, which you've you've been doing
Speaker 2: to is there's so much more money coming into building
Speaker 2: these centers and their centers being built and that's going
Speaker 2: to be its own is.
Speaker 1: But also just just fundamentally, these weirdos are trying to
Speaker 1: make you scared of cloud software. Like that's that's my
Speaker 1: thing that I keep up. It's like if such an
Speaker 1: adela every two weeks is like, You'll never fucking believe
Speaker 1: what Microsoft Work's going to do next mate, You're never
Speaker 1: gonna believe what powerbi is going to do with the dashboard.
Speaker 1: I saw a dashboard on Powerbi and I'm pretty sure
Speaker 1: next week it's gonna it's gonna try and kill someone.
Speaker 1: You'd be like, you sound like a lunatic man. You
Speaker 1: sound like a crazy person. This is an insane Sundhar Peshai. Yeah,
Speaker 1: Google Search is oh any in any week now? You
Speaker 1: just laugh at it. You would laugh at him.
Speaker 2: But the two in the Microsoft guys are are he's
Speaker 2: trying to because he's not He's trying to use the
Speaker 2: Dario sam playbook, but he's not good at it, and
Speaker 2: so it's kind of he's like, I think, uh, you know,
Speaker 2: Ai is going to It's it's not only going to
Speaker 2: take over all jobs, it's going to create new jobs
Speaker 2: just to automate them, and it's going to do it Thursday.
Speaker 2: He'll do a Thursday. How about that? Is that? Guys?
Speaker 2: That is that?
Speaker 1: I mean, it's like, do you like that? He's turning
Speaker 1: a big dial, this is AI on it. It's just like,
Speaker 1: I do you like this? I love it well because
Speaker 1: he'll he'll even talk and be like, yeah, twelve to
Speaker 1: eighteen months away, we're going to have super intelligence, but
Speaker 1: also anthropics too expense if we can't pay the money more.
Speaker 1: But also this is normal and it's not going to
Speaker 1: take jobs, but it's going to take all the jobs
Speaker 1: in two years, but not today, but not next week,
Speaker 1: maybe in six months. And it's just the Yeah, they're
Speaker 1: using the Dario playbook with everything because they're just making
Speaker 1: the same things and they don't work particularly well. It's
Speaker 1: just so these people are so weird. Now you've said it.
Speaker 1: Now that's all I can think about.
Speaker 2: So here's the logic. I also want people to avoid
Speaker 2: it because I came up into what you were saying
Speaker 2: there and I feel like maybe Chris did this a
Speaker 2: little bit in your interview. I have to go back
Speaker 2: and listen to it. But this this logic, and I
Speaker 2: get it, and I'll give an explanation for it from
Speaker 2: a psychological perspective in a second. But there's this logic
Speaker 2: of if there's something I can point to where AI
Speaker 2: did something that I thought was like impressive or interesting
Speaker 2: or better than I thought, every prophecy is true, yes,
Speaker 2: and I think I think it's I don't know psychological
Speaker 2: self preservation or I don't know what it is. But
Speaker 2: I don't under this. This is again from like a
Speaker 2: logic point of view. I teach logic to undergraduates, like
Speaker 2: propositional logic. I don't understand. This is a it's an
Speaker 2: argument that's missing premises. Right, So they're going from the
Speaker 2: premise of hey, it's and it's almost always the same thing.
Speaker 2: It's almost always a non engineer using cloud code because
Speaker 2: it's really fun. It's like a slot machine. It especially
Speaker 2: your you know, it just like creates code. It works
Speaker 2: for simple things, and you're like this, it's really fun.
Speaker 2: Like I've talked I know people who are like they're
Speaker 2: they're like, this is my new hobby. Like it's it's
Speaker 2: really fun if you're into that type of thing. Right,
Speaker 2: we talked about this earlier. It's like model trains for engineers.
Speaker 2: You do that like that was really cool, and then
Speaker 2: you translate that experience of like I really enjoyed that
Speaker 2: to you know, I wonder what salt mind the AI
Speaker 2: overlords are going to assign me to. And it's a
Speaker 2: weird that that leap of logic. How can we just
Speaker 2: keep it as like, Okay, this was really cool that
Speaker 2: you built this dashboard. We don't have to leap from
Speaker 2: that to therefore everything Dario Amadae said must be true.
Speaker 2: It's not this binary balance on a folk rum where
Speaker 2: we're just looking for like one one little piece of
Speaker 2: evidence will tip it one way or the other. So
Speaker 2: I think we need to be careful about these elaborations.
Speaker 2: It's ad jagged technology. Some things make more progress than others.
Speaker 2: Some things are cool, some things are more narrow than
Speaker 2: you would think. But like, you know, does see I
Speaker 2: built a really cool web based game in Fable five,
Speaker 2: which actually is probably just copying the source code of
Speaker 2: a similar game. Yeah, does that mean like recursive self
Speaker 2: improvement is coming? So what I'm hoping we can do
Speaker 2: is stop making these leaps of logic where if you
Speaker 2: encounter any sort of feature or have any sort of
Speaker 2: positive experience with an AI product, that you then leap
Speaker 2: immediately to the conclusion that everything involved in the prophecy
Speaker 2: about the future of AI must be true, or to
Speaker 2: put someone like you and me now it has to
Speaker 2: be put on the defense of of I did something
Speaker 2: positive with AI, all right?
Speaker 1: Ed?
Speaker 2: Doesn't that now mean everything anyone has ever said about
Speaker 2: AI is true? Convinced me I'm wrong. I think that
Speaker 2: logic is back words. That these grand proclamations of we're
Speaker 2: going to have self improving AI, we're going to automate
Speaker 2: all jobs use to Carl Sagan quote, those remarkable claims
Speaker 2: require a remarkable amount of evidence. Why is everyone so
Speaker 2: quick of like this product, what you're working on and
Speaker 2: putting billions of dollars on a course? You're going to
Speaker 2: keep adding new features or finding things to improve? That
Speaker 2: is a given a course? What technology in the history
Speaker 2: of world would you have no improvements on? If you're
Speaker 2: spending you know, ten to twenty billion dollars a year
Speaker 2: in R and D, of course it's going to have improvements.
Speaker 2: Why are we leaping from that?
Speaker 1: That?
Speaker 2: Like that means that anything you say, any speculative future
Speaker 2: you give about, you know, some sci fi future, is
Speaker 2: now the default. Unless we can prove otherwise, so we
Speaker 2: got to stop that logic too. We should be able
Speaker 2: to discuss, enjoy, or get around the implications of particular
Speaker 2: AI features without having to have that shift to a
Speaker 2: conversation about RSI or widespread job automation.
Speaker 1: I think what it is. It comes back to the
Speaker 1: pride thing. I think the the There's also something about
Speaker 1: lms and the way they make people feel like I'm
Speaker 1: on the spectrum autistic people have probably worked that out
Speaker 1: from the third ten thousand word article. But I don't
Speaker 1: feel anything when they're nice to me. I feel absolutely nothing.
Speaker 1: When the chat bot is like, Wow, that was a
Speaker 1: great idea. I don't have I don't respect it. I
Speaker 1: don't care what it thinks because it doesn't think. I
Speaker 1: don't see anything about what it says. If it tells
Speaker 1: me I have a good idea, I'm like, I don't care.
Speaker 1: I feel nothing. I think that a lot of people
Speaker 1: do feel something. I think when they see these things
Speaker 1: compliment them and do an impression of brainstorming with them,
Speaker 1: which is really just talking to yourself in the mirror,
Speaker 1: which can help. That can help. I've done it. I
Speaker 1: think that it makes them feel good and also your
Speaker 1: claud code example. I think that people feel very good
Speaker 1: about that, and they feel very proficient, and it makes
Speaker 1: them feel capable at something perhaps they were incapable of before.
Speaker 1: And thus if you say to them, well, okay, that's
Speaker 1: just that's fine, they go, no, no, I'm a big boy.
Speaker 1: I think there's something kind of almost childish about it,
Speaker 1: like no, I'm special, I'm special. Why aren't you impressed with?
Speaker 1: And it's something that models are built to do. The ingratiation,
Speaker 1: I think is important. But also I think that the
Speaker 1: tech industry has done a really good job over the
Speaker 1: years of convincing us that everything gets better and better
Speaker 1: and better, even though it doesn't. Even though a lot
Speaker 1: of Google Search has got worse, Facebook's got worse, Twitter's
Speaker 1: got worse. Honestly, argue, many parts of Blue Sky have
Speaker 1: got worse. It's less reliable. At least, the experience of
Speaker 1: using the computer has got worse. We've hit a wall
Speaker 1: with innovation in general. But this thing, because it can
Speaker 1: kind of resemble anything if you want it to, I
Speaker 1: think touches something dark in some people, not evil, but
Speaker 1: just like something very vulnerable and kind of weak, kind
Speaker 1: of gullible, where they say well, if I'm getting this
Speaker 1: out of this, then I'm part of the future and
Speaker 1: I need someone to be impressed. But it's this is
Speaker 1: now a part of my identity, and that's I think
Speaker 1: that that might also be the differentiator here as well.
Speaker 1: I don't remember. Take sure you had four or wars
Speaker 1: about gaming consoles. I'm sure you had, you have Android
Speaker 1: versus iOS whatever, not like this, nothing like this, And
Speaker 1: I think that it touches something in people.
Speaker 2: Well, so I think there's a whole class of people,
Speaker 2: or that's what's going on, especially the sort of bro
Speaker 2: adjacent tech proadjacent people on you know X, who are
Speaker 2: who are always yelling well, but what about this? What
Speaker 2: about this?
Speaker 1: Right?
Speaker 2: And then I think the the other thing that happens
Speaker 2: for a lot of people, especially who are a little
Speaker 2: bit more tech distance, is it's a way of dealing
Speaker 2: with anxiety. Right, Yeah, And if you think about this,
Speaker 2: and like we've talked about COVID before, you and I,
Speaker 2: but I think it offers some good you know, analogies
Speaker 2: here is like one of the ways when you know
Speaker 2: the virus was spreading but hadn't quite got to the US,
Speaker 2: or it was just coming or the hospitals were overloaded
Speaker 2: in Italy or whatever. It's like one of the ways
Speaker 2: to deal with it psychologically was to get out in
Speaker 2: front of it and be like, this is going to
Speaker 2: be really bad, Like I you know, I'm on it,
Speaker 2: and you know, you feel more prepared, like I'm not.
Speaker 2: I'm not going to be caught off guard. I'll be
Speaker 2: the one telling people how bad this is. And like
Speaker 2: somehow that does help you a little bit deal with
Speaker 2: like the anxiety of like impending dread. And I think
Speaker 2: there's a lot of that going on with this as well,
Speaker 2: is like I'm anxious about this doomtroll lamp and doomtrolled
Speaker 2: so many times, I'm going to kind of get out
Speaker 2: in front of it. Now if I'm the one saying,
Speaker 2: well wait a second, I get so many I am
Speaker 2: for a lot of people now, they're like stress pacifier
Speaker 2: of like well what about this go right? Like I
Speaker 2: saw this thing that upset me, Like doesn't that mean
Speaker 2: that means? I think that means it is going to
Speaker 2: take it over? And then they're kind of hoping that
Speaker 2: me or someone else will come in and be like
Speaker 2: you don't really it's okay, Like that's not as big
Speaker 2: of a deal as it seems, where so I think
Speaker 2: both are going on. It's either like this gives me
Speaker 2: meaning like I'm a part of the future if you're
Speaker 2: like using cloud code, and if you're just using like
Speaker 2: chat GPT for natural language search, you might just be
Speaker 2: really anxious because you know, you hear these earnest podcasts
Speaker 2: where people are interviewing people about you know, we'll have
Speaker 2: to eat our dogs when the jobs are gone, and
Speaker 2: you're like, well, if I get out in front of it,
Speaker 2: at least I feel like I have some autonomy in
Speaker 2: this dark thing that's happening. And so it's like common psychology.
Speaker 2: Both of the psychologies equal dollar and cents for a Again,
Speaker 2: I keep coming back to remember is a vanishingly small
Speaker 2: number of early investors who have large equity positions in
Speaker 2: these companies. That is, who benefits from all of that.
Speaker 2: They will benefit immensely because no matter what that psychology is,
Speaker 2: this company is at the full crumb of the future,
Speaker 2: and that's worthy of investing in, even if their revenue
Speaker 2: story is murky, even if their product story is there's
Speaker 2: no moat and it is I mean, I don't want
Speaker 2: to get into the economic weeds. But this is like
Speaker 2: a Clayton Christiansen fever dream what is happening with these
Speaker 2: companies and their products. So he had this idea of,
Speaker 2: you know, talking about disruption from below that the problem
Speaker 2: is is you're working on high end version of a
Speaker 2: product and you don't want to undermine your market, so
Speaker 2: you try to keep it expensive and high end, and
Speaker 2: then someone comes in from below with a different way
Speaker 2: of doing it this much cheaper, and you're screwed because
Speaker 2: you ignored that market. All of these use the use
Speaker 2: cases that are useful now that people are spending money on,
Speaker 2: like software development tools and natural language search. Here's the problem.
Speaker 2: You don't need a ten trillion parameter language model to
Speaker 2: do this, and this is why they're so these companies
Speaker 2: are so screwed, and I think are trying to IPO
Speaker 2: as fast as they can. Is that like someone's going
Speaker 2: to come along and say, we're using a twenty billion
Speaker 2: parameter open source model with a really smart coding harness
Speaker 2: that we just coded ourselves on top of it and
Speaker 2: tweaked for the last two years, and you know what,
Speaker 2: people like it better than claud code. And it's a
Speaker 2: hundredth of the price right or natural language search. You
Speaker 2: don't need claud fable five to summarize Google search results right.
Speaker 2: And the economics make no sense. But a small model could,
Speaker 2: a non chip model could. That is why they're in
Speaker 2: trouble economically.
Speaker 1: I also think that there is and I do think
Speaker 1: there is a deeply cynical thing here. Well, let me
Speaker 1: reframe them. I think that people have to find ways
Speaker 1: to rationalize capitalism. I think that the only way that
Speaker 1: they could the amount of times I hear a week
Speaker 1: of just like Amazon Web service is just like Uber's,
Speaker 1: just like this. I have restated that point a hundred times.
Speaker 1: People sometimes listen, people sometimes don't. But people need to
Speaker 1: find a way, I think to rationalize the massive expenditures,
Speaker 1: and the current products do not provide that rationalization. Nothing
Speaker 1: about chat, GBT or Claude or clock code or fable
Speaker 1: or whatever actually rationalizes this. Doesn't make it. A trillion
Speaker 1: dollars is too much money for anything outside of like
Speaker 1: healthcare and housing. In fact, probably could have taken care
Speaker 1: of that really easily. But nevertheless, people see that and
Speaker 1: they don't want to deal with the natural kind of
Speaker 1: dissonance they get, which is wow, are all the companies
Speaker 1: run by people who don't know what's going on, and
Speaker 1: the answer is yes, Like, I don't think that there's
Speaker 1: actually a grand conspiracy. Oh my god. The emails are
Speaker 1: get every week, people being like, it's surveillance. It's a
Speaker 1: secret surveillance state. It's secret. It's yeah, it's so secret.
Speaker 1: It's on CNBC every day. Oh oh, they're all they're
Speaker 1: doing secret data centers. Yeah, it's called Oracle, it's being
Speaker 1: it's called Oracle. That's what Oracle does. Oracle has a
Speaker 1: huge class. Like, that's very well known. What do you like?
Speaker 1: I think people need there to be a conspiracy because
Speaker 1: otherwise the answer is which is what I believe, which
Speaker 1: is none of these companies had a plan. I think
Speaker 1: Sam Mortman and Daria Amedy thought we're gonna throw a
Speaker 1: bunch of money at this and then God will come out. Yeah, money, money,
Speaker 1: so much money, and then they will have asi agi
Speaker 1: whatever they call it. And then it happened, and then
Speaker 1: ye didn't. They hit the diminishing returns at the end
Speaker 1: twenty twenty four, and since then they've been kind of
Speaker 1: harnessing non LLM stuff on top of LLMS in an
Speaker 1: attempt to make them do stuff.
Speaker 2: But that's been so successful. By the way, think about
Speaker 2: the doom trolling. This is another advantage of it. They've
Speaker 2: completely offfuscated exactly that point that you just made, which
Speaker 2: is that pre training scaling again had We've talked about
Speaker 2: this on the show, but has had diminishing returns, you know,
Speaker 2: starting after twenty twenty four, and so all of the
Speaker 2: improvements that have come since then has been post training
Speaker 2: pre train models and building hand coding harnesses with more
Speaker 2: and more special cases plugged into it to find more
Speaker 2: places for them to be useful, which is a completely
Speaker 2: different trajectory than twenty twenty three where they were like, no,
Speaker 2: we'll just keep pre trained scaling these things till we
Speaker 2: have AGI. And they've created a sense of like, well
Speaker 2: AI has been drastically improving, hasn't it. And you said, well,
Speaker 2: wait a second, Like if you told me in twenty
Speaker 2: twenty two, you know when you're like, we're going to
Speaker 2: scale our way to AGI, and our vision of AGI
Speaker 2: is software development tools.
Speaker 1: Yeah, Like it's just dreadfully boring and compact for.
Speaker 2: Natural language Google. But you're like, but it's not just
Speaker 2: like a feature that like Google would just kind of
Speaker 2: otherwise would have just kept adding and tweaking in a
Speaker 2: very like economically reasonable way.
Speaker 1: I'm saying forever. It's like, yeah, I thought, like the
Speaker 1: reason that people like LLLM search is pretty simple that
Speaker 1: it responds to search how people have always used search,
Speaker 1: which is questions you're like, hey, what about this and this.
Speaker 1: I've used AI mode with Google maybe three times, and
Speaker 1: I always use it as like a trawling expedition to say, like,
Speaker 1: has anyone ever said anything like this? And then it
Speaker 1: would give me a bunch of fake citations like just
Speaker 1: that happens every time. But then I'll find one weird
Speaker 1: PDF and It'll be like, oh cool, But isn't that
Speaker 1: if you didn't if elms didn't exist, I would expect
Speaker 1: Google Search to do that by now, like that would
Speaker 1: that feels like a natural progression, But it's just normal
Speaker 1: kind of boring. So I always say it's like, oh,
Speaker 1: if they called it library models, it wouldn't be that
Speaker 1: big deal, but it would be a more accurate, like
Speaker 1: you just choose a regular name.
Speaker 3: This.
Speaker 1: People weren't freaking out about platform as a service tools,
Speaker 1: then they hear all this bullshit about Kubernettes, which is
Speaker 1: way more fun to say in large language models. People
Speaker 1: have taken this. And the thing is, I don't think
Speaker 1: that the strategy was always we're going to doom troll like.
Speaker 1: I don't think they sat and thought about it very much.
Speaker 1: I think they just kind of oath maxed their way.
Speaker 1: They just bumbled into it and they're like, it's scary,
Speaker 1: and then they saw how well it worked. I think
Speaker 1: I genuinely think on an ethical level, it's one of
Speaker 1: the most disgraceful things anyone's ever done technology Technologically. I
Speaker 1: think it's just on top of the obfuscation of what
Speaker 1: these things actually do, it's horrible. It's a it's a
Speaker 1: cruel thing to do. I hear from people all the
Speaker 1: times being like, am I going to lose my job
Speaker 1: to this? I'm like, why is my boss telling me
Speaker 1: I'm going to lose my job to this?
Speaker 2: Why?
Speaker 1: What is this? Why is it different? Why? Am And
Speaker 1: I think when they hear all the doomerism and they
Speaker 1: use the product and they use claw code and they go, oh,
Speaker 1: it boffed out an open source CRM, they're like, oh god,
Speaker 1: they're right. Imagine what someone who kind of knew what
Speaker 1: they were doing could do. Ooh, I'm scared. Maybe I
Speaker 1: don't think that. I think that they shouldn't be able
Speaker 1: to sleep at night. I think it's an evil way
Speaker 1: to market.
Speaker 2: I would say the way I would think of I
Speaker 2: put on like my like my Bentham hat. Here, right,
Speaker 2: let's use like a utilitarian calculus the net harm that
Speaker 2: doom trolling as a communication strategy is caused in terms
Speaker 2: of mental health, right, just like people feeling bad, The
Speaker 2: net harm caused by doom trolling far far, far, far
Speaker 2: far exceeds any the grand total of benefits that this
Speaker 2: AI technology is produced so far, Like, I don't see
Speaker 2: any other way for that calculus to work out. You know,
Speaker 2: I was just reading one of the mini papers that
Speaker 2: are actually now you know, surveying and studying AI in
Speaker 2: the workforce, and they they looked at this paper and
Speaker 2: they asked people that these software developers, like, estimate how
Speaker 2: much time you've saved or whatever, and it was, you know,
Speaker 2: eleven hours a week we're saving. And then they asked them,
Speaker 2: is your company performing better at all? Because this made
Speaker 2: any impact on your bottom line? And it was like
Speaker 2: a vanishingly small percent that was like, yeah, there's anything
Speaker 2: different on the performance was that worth or making like
Speaker 2: software developers maybe by some measures one point two times
Speaker 2: more efficient, Yeah, worth causing. I think the negative toll
Speaker 2: of this psychologically is like getting close to like COVID territory.
Speaker 2: It's like population wide, you are stressing out everybody day
Speaker 2: after day. They've made the last two years for anyone
Speaker 2: who's following the news psychologically miserable. That is a real harm.
Speaker 2: That is a real measurable harm. Like I really hate
Speaker 2: that I have to deal with this like time and again.
Speaker 2: It reminds me of having to deal with the pandemic,
Speaker 2: which was like a like a psychologically fraught time because
Speaker 2: every day you were being bombarded. But at this time,
Speaker 2: you know, a, there's not a virus that's infecting a
Speaker 2: lot of people, and be the people who are like
Speaker 2: warning about all the doom. In this analogy, it would
Speaker 2: be as if like they were the ones in the
Speaker 2: lab doing the research that was producing the virus. Like
Speaker 2: that's the craziest thing about it, right, I mean imagine
Speaker 2: if in like twenty nineteen someone was like these viruses,
Speaker 2: coronaviruses doing gaina function research could really really like infect
Speaker 2: the whole world Like a course, they're going to get out.
Speaker 2: We can't contain them. Oh, by the way, we're curiously
Speaker 2: doing this gain of function research on these viruses, and
Speaker 2: we're not going to stop because you know the Chinese
Speaker 2: well in this case, but whatever, yeah, someone else is direction,
Speaker 2: someone else might do it. So like what can we do,
Speaker 2: like stop building the viruses then? Anyways, So I think
Speaker 2: we have to account for the net psychological harm, and
Speaker 2: I think, I mean, I don't think our current government
Speaker 2: can do this, but I think that that's something where
Speaker 2: the government should, in an abstract world be involved, Like, oh,
Speaker 2: there's a small number of people who are causing a
Speaker 2: huge amount of distress and anxiety around our entire country.
Speaker 2: Of course we're going to get in the game and
Speaker 2: we're going to pause all your model deployments and tell
Speaker 2: like you either convince us that that's not the case
Speaker 2: and you admit like, yeah, we were kind of making
Speaker 2: that up, or if you don't want to say thathing
Speaker 2: like no, you don't get a release's product. You're causing
Speaker 2: mass pro mass unhappiness and damage right to our entire country.
Speaker 2: You have to account for that. You don't get to
Speaker 2: just do that. So does I think the harms? So
Speaker 2: far have exponentially outweighed the tangible benefits.
Speaker 1: I'm just also going to be honest, Yeah, the model
Speaker 1: AI companies are terrible. I also think the media is
Speaker 1: categorically failed here and should feel fucking ashamed of themselves.
Speaker 1: I think the media is as much.
Speaker 2: To blame because I think I'm gonna put it. I'm
Speaker 2: gonna put a fundamental blame on venture capital. I'll give
Speaker 2: you my quick pitch on that. Yes, it's because, I
Speaker 2: mean you have to understand the economics of especially high
Speaker 2: tech venture capital coming out of Silicon Valley. The whole model,
Speaker 2: and again this is all just because you have, like
Speaker 2: some institutional investors in rich people that you want to
Speaker 2: make Richard. That's it, Like, that's the whole underlying goal, right,
Speaker 2: And the their whole model is we have to find
Speaker 2: new areas where there's something to be monetized, in which
Speaker 2: there's possibilities for unicorns. So we have to find new
Speaker 2: territories where you have the possibility of getting in early
Speaker 2: and having an investment turn out to be a multi
Speaker 2: billion dollar company.
Speaker 1: Right.
Speaker 2: The web that's what started in the nineties with the web,
Speaker 2: and most of those bets failed because of the first
Speaker 2: dot Com crash, a couple paid off like Amazon. Then
Speaker 2: you had Web two. It turned out attention was a
Speaker 2: resource you could monetize, so it's like discovering a new
Speaker 2: minroller oil. There was a huge fortune to be had.
Speaker 2: The venture capitals could get in there early. They had
Speaker 2: massive returns betting on meta, betting on you know, the
Speaker 2: social media companies. Right then they're like, we need another
Speaker 2: because you have to keep finding every time you raise
Speaker 2: a new fund, you need a giant new resource that
Speaker 2: you can monetize to get to early. So the next
Speaker 2: thing they tried was Crypto. Crypto was just you know,
Speaker 2: Andreessen Horowitz saying, there's no more unicorns left in social
Speaker 2: We're going to try to create a new field in
Speaker 2: which we can create unicorns. And they really flogged crypto.
Speaker 2: This is the future, This is the future. Yeah, yeah,
Speaker 2: and it wasn't, especially for software. And I got yelled
Speaker 2: at for saying this, but I'm literally an expert on this.
Speaker 2: My doctorate's in the theory of distributed systems. It's a
Speaker 2: stupid way to build a distributed system, and I was right.
Speaker 2: And then no one built the systems that way, so
Speaker 2: that kind of failed. This was the next territory. And
Speaker 2: it doesn't say this technology is not interesting or potentially useful,
Speaker 2: but as you said, it's not trillion dollar useful, not yet.
Speaker 2: So why did all that money come in is because
Speaker 2: we're talking about one street in one city. We're talking
Speaker 2: like sand Hill Road in Palo Alto. Like the people
Speaker 2: on that one street are like, we need to find
Speaker 2: a new territory that can produce, you know, one hundred
Speaker 2: billion dollar companies so that we can get the returns
Speaker 2: that we expect to keep our lifestyle up it And.
Speaker 1: That's just what I've I've been saying this for years.
Speaker 1: It's the rock combobbles. They're out of big ideas, and
Speaker 1: AI was the panacea. It was the It was the
Speaker 1: API would be the way to build new startups, be
Speaker 1: the way for enterprise companies to sell more software with
Speaker 1: new add ons. It would be the way for business
Speaker 1: software to take off consumer software. It was meant to
Speaker 1: be the solution because they're doing exactly the same thing
Speaker 1: is that AI companies are doing, saying yeah, the AI
Speaker 1: will just fix this, what are we going to build
Speaker 1: with this? Or the AI will work out what's the product?
Speaker 1: AI will work it out. It's honestly a kind of
Speaker 1: a pathetic culture.
Speaker 2: I don't think they even care about that. I think, like, okay,
Speaker 2: if we put the money in this, now, what's the
Speaker 2: strike price going to be at the IPO.
Speaker 1: That's whine I'm getting at that.
Speaker 2: Works from the end or not they're they're like, we
Speaker 2: want a SpaceX, look at that.
Speaker 1: But the theoretical that I'm saying is that the idea
Speaker 1: of large language models as an API or a consumer
Speaker 1: product was what they were excited about. Because you can
Speaker 1: just extrapolate the various things. But I must, I must
Speaker 1: get back to it. I think the media is responsible.
Speaker 1: I one hundred percent believe the media is responsible. They
Speaker 1: are the ones that at the Times Ezra Cline, how
Speaker 1: much AGI bullshit is that guy shoveled? How many time
Speaker 1: Bernie Sanders going out there and talking to Claude. I
Speaker 1: realized that's politics. But they put on YouTube for a reason.
Speaker 1: How many different outlets have talked about the job apocalypse
Speaker 1: that does not exist or the saspocalypse claiming that software
Speaker 1: companies are being sold off because of AI disruption that
Speaker 1: doesn't exist. The media bought and sold this. In twenty
Speaker 1: twenty three, there was Guardian headline that said Samuel was
Speaker 1: a little bit scared. It's just without that, none of
Speaker 1: this was possible. And perhaps they were just being a
Speaker 1: megaphone for real anxieties from these companies. Well, perhaps they
Speaker 1: were just looking for a narrative that would get clicks.
Speaker 2: I just think, yeah, I don't think it's clix. I
Speaker 2: think they underestimated the weirdness of these people, and they
Speaker 2: took them at phase. This is my you know, having
Speaker 2: mini feet within the world of media. What I really
Speaker 2: think happened is it didn't cross their minds that these
Speaker 2: people are super strange cultist weirdos. They don't know about Yukowski,
Speaker 2: they don't know about the rationalists. They don't know about
Speaker 2: the excescummuit that came out of the rationalists and then
Speaker 2: merged with the technological singularities, and that they also think
Speaker 2: they're going to live forever by harvesting the blood of
Speaker 2: children and all. They were just like, these are engineers.
Speaker 2: I don't know about technology.
Speaker 1: They do.
Speaker 2: They're telling me this. Why would they lie? I've heard
Speaker 2: that word for word from multiple early on, multiple journalists,
Speaker 2: and I think it was all being seen through the
Speaker 2: frame of COVID, where that was in the recent history
Speaker 2: if the viral I don't know anything about viruses, but
Speaker 2: the virologists were like trusted, Like this is something I
Speaker 2: can't directly observe right because there's not very many people sick,
Speaker 2: but like trust us, this is going to spread and
Speaker 2: this is a really big deal. And like all trucks,
Speaker 2: the virologists know what they talk about. Why would they lie?
Speaker 2: And I don't want to miss that story. I think
Speaker 2: there was a lot of that going on. Of these
Speaker 2: are the people building the technic because I heard this
Speaker 2: phrase from so many people. These are the people building
Speaker 2: the technology. They know about it best. I'm not a
Speaker 2: computer scientist. I mean I am, but if I'm being
Speaker 2: the journalist, and this is why, like the people who
Speaker 2: were not, where is the resistance is coming from is
Speaker 2: computer scientists like me who don't live near Silicon Valley,
Speaker 2: And well, I do understand this technology and what the
Speaker 2: hell are you talking about? But I think for the
Speaker 2: journalist it was and now they're just and this is
Speaker 2: why I wrote that off ed. I'm trying to spread
Speaker 2: the word more. These guys are weird. They're weird, and
Speaker 2: by the way, they're mainly guys too, because I think
Speaker 2: women were gonna put up with this weird You know,
Speaker 2: I'm gonna go in my room and put with my
Speaker 2: Star Wars toys nonsense, right, I don't want to Who
Speaker 2: knows what that tells us about our our gender. But
Speaker 2: it's not great that it's all men doing this craziness.
Speaker 1: It's tim Tim ree Gamber who isn't particularly sold on this,
Speaker 1: for example, like.
Speaker 2: Emily Bender, like a lot of the hardcore skeptics are
Speaker 2: like you boys with.
Speaker 1: Your Star Wars.
Speaker 2: Yeah, you don't have friends, do you? Is this is
Speaker 2: this related the fantasy football?
Speaker 1: What?
Speaker 2: Guys? What is this?
Speaker 1: Right? Which? What's the funny thing is, though there is
Speaker 1: something to that where it's is a very male technology,
Speaker 1: and I do wonder like it's not something I think
Speaker 1: two white guys can necessarily get into in any any
Speaker 1: depth without a woman present, so that we could actually
Speaker 1: have a rounded conversation. But it is interesting that pretty
Speaker 1: much everyone running this stuff is a guy, and the
Speaker 1: majority of them are white fellas, and they're all claiming
Speaker 1: that everything they build is amazing. I don't know, perhaps
Speaker 1: there's something going on that too, It's just.
Speaker 2: Is there something Yeah, white guys are good at? Is
Speaker 2: thinking everything we do is amazing. We are pretty well
Speaker 2: trained or we are pretty well trained at that. But
Speaker 2: that's why I do think it's important why I wrote
Speaker 2: that op ed. It's like, I want to name what
Speaker 2: they're doing. This is doom trolley. I think you need
Speaker 2: to understand these people are weird and we can't take
Speaker 2: what they're saying at face value. We just can't because
Speaker 2: they're too weird. They've they've proven time and again. But
Speaker 2: the key message I would also want like, if you're
Speaker 2: listening to this pod, if you're listening to this podcast, now,
Speaker 2: do not interpret this conversation as I'm going to categorize
Speaker 2: this in a claim that AI doesn't work. And therefore,
Speaker 2: if I see something cool that AI does, Uh, this
Speaker 2: conversation isn't relevant. You know, cal was wrong. This is
Speaker 2: it's really not about that is about what I'm talking
Speaker 2: about right now. I am talking about the way that
Speaker 2: we actually talk about AI, how we treat it as
Speaker 2: a business, what is morally appropriate and what is morally
Speaker 2: not And these are normal products and I don't I
Speaker 2: don't to me, this isn't this not vaporware. I mean,
Speaker 2: I think the ability to parse natural language like a
Speaker 2: language model can do is like it's fascinating, and I'm
Speaker 2: also surprised by how hard it's been to come up
Speaker 2: with a lot of good, useful, economically viable products off
Speaker 2: of that ability. Like we were still with software development
Speaker 2: tools and and uh, you know, rewriting Google search results.
Speaker 2: I actually thought we would have a lot more interesting
Speaker 2: tools that we were using in our everyday life right now.
Speaker 2: And because there's been some problems that we didn't think about.
Speaker 2: So this whole conversation is about stop treating this like
Speaker 2: a demon that's been summoned. Make fun of doom trolling
Speaker 2: because it's kind of embarrassing. This is the you know,
Speaker 2: this is the magic the gathering club at the high
Speaker 2: school table, and just treat this like a normal product.
Speaker 2: And like I don't want to hear. Let's just what
Speaker 2: are the products you're selling, what are the benefits, how
Speaker 2: much do they cost? Is it worth the money? And uh,
Speaker 2: what are the products are you working on? Like, let's
Speaker 2: just have normal conversation. So so do not as a
Speaker 2: listener categorize this in this in this, in this sort
Speaker 2: of light vers dark is like, okay, this means AI
Speaker 2: doesn't work. And this team says it does work, and
Speaker 2: if if and then I'm going to try to assess
Speaker 2: in my mind is AI useful or not? And if
Speaker 2: it is, then I have to be ready to mate
Speaker 2: with machines. And if it doesn't, then, you know, stop
Speaker 2: all that sort of extreme thinking. I just want us
Speaker 2: to talk about this like a normal, interesting but normal technology,
Speaker 2: and normal people should look at the way that these
Speaker 2: companies are doom trolling us and say that's strange. Please stop.
Speaker 2: And in the meantime, I am going to have to
Speaker 2: really make fun of you, like I think that's where
Speaker 2: we need to be.
Speaker 1: And I think all of the large language models in
Speaker 1: a vacuum in very interesting technology. The problem is they
Speaker 1: tried to scale them like a giant, multi trillion dollar industry,
Speaker 1: which I don't think they fit into the future of
Speaker 1: LMS in whatever form they have is on device. They
Speaker 1: never should have been scaled at this scale, like it
Speaker 1: never made sense, never once, And in the end, it
Speaker 1: is normal technology. And that is the thing to always
Speaker 1: take away from this, which is you wouldn't freak out
Speaker 1: about Kubernetes. You wouldn't Microsoft word doing autocomplete, shouldn't scare you, grammarly,
Speaker 1: shouldn't scare you. This is the actual technology itself is
Speaker 1: not scaring you. It's the people around it and the
Speaker 1: people's extrapolations. And also, like you said, the willingness. We
Speaker 1: have to believe people who are working on something, who
Speaker 1: have supposed expertise, as if expertise guarantees that you're correct,
Speaker 1: or indeed guarantees that you know what you're talking about,
Speaker 1: or indeed that you can see the future.
Speaker 2: Yeah, so there we go. I mean I think that's yeah, yeah,
Speaker 2: I mean I don't know who knows. I'm already seeing
Speaker 2: Like the comments on my New York Times piece is
Speaker 2: just falling into the like uh oh you you you
Speaker 2: said somewhere in there there could be a benefit of AI.
Speaker 2: You apostle AI is terrible, or like the other side
Speaker 2: is is like, well, well, how do you explain the fact,
Speaker 2: you know, I'm oh my god, it's the most is
Speaker 2: this written by AI? These comments like this synth this
Speaker 2: all the time, these these comments that are like I
Speaker 2: work for a such and such company and what used
Speaker 2: to take me three weeks I now do in like
Speaker 2: six seconds. And you know, it's just like falling to
Speaker 2: People are just falling in their camps about like either
Speaker 2: you're one of the priests of the new religion or
Speaker 2: you know you're the you're the the the the Canaanites
Speaker 2: who are who were kicked out of the land or
Speaker 2: something right like, it's like you got to be in
Speaker 2: like one of these two like extremes, and don't do that.
Speaker 2: Just say, like, treat it like a normal technology. Stop
Speaker 2: doom trolling is stupid. Stop it like, treat it like
Speaker 2: a normal technology. Either stop if you really think you're
Speaker 2: destroying the world, or start talking about it like pickup trucks. Right,
Speaker 2: we don't need this in between, So hopefully, hopefully we
Speaker 2: can convince a few more people about that.
Speaker 1: Ope, so cal thank you so much for joining me.
Speaker 2: Oh it's a pleasure.
Speaker 1: Thank you for listening to Better Offline.
Speaker 3: The editor and composer of the Better Offline theme song
Speaker 3: is Matasowski. You can check out more of his music
Speaker 3: and audio projects at Matasowski dot com, M A T
Speaker 3: T O S O W s KI dot com. You
Speaker 3: can email me at easy at Better offline dot com,
Speaker 3: or visit Better Offline dot com to find more podcast
Speaker 3: links and of course my newsletter. I also really recommend
Speaker 3: you go to chat dot Where's youreaed dot at to
Speaker 3: visit the discord and go to our slash.
Speaker 1: Better Offline to check out our reddit. Thank you so
Speaker 1: much for listening.
Speaker 2: Better Offline is a production of cool Zone Media.
Speaker 3: For more from cool Zone Media, visit our website cool
Speaker 3: Zonemedia dot com. Check us out on the iHeartRadio app,
Speaker 3: Apple
Speaker 2: Podcasts, or wherever you get your podcasts