Speaker 1: Media. Hello, and welcome to Better Offline. I'm, of course
Speaker 1: your host ed Zetron. As ever, support your neighborhood Zetron
Speaker 1: by subscribing to the premium newsletter discount link in the
Speaker 1: episode notes, of course, by a T shirt, download a
Speaker 1: blog where whatever it is you want to do, Okay,
Speaker 1: it's not up to me what you do. But today
Speaker 1: I'm joined by the incredible commside professor and commentator col
Speaker 1: newportcl Thank you for joining me.
Speaker 2: Oh, it's a pleasure.
Speaker 1: Ed. So I kind of wanted to start with I
Speaker 1: asked you for a quote a few like a week ago,
Speaker 1: maybe two weeks ago. I can't remember how time works anymore,
Speaker 1: but it was around the way the reporters cover AI
Speaker 1: and how it seems that a lot of the reporting
Speaker 1: is kind of directionally true rather than actually true.
Speaker 2: Yes, and I want to add something to it. Sense,
Speaker 2: So I've been thinking about that quote. Yeah, I've been
Speaker 2: thinking about it. So what I said, if I remember
Speaker 2: that quote properly, what I was saying is I was
Speaker 2: I was picking up a lot in the reporting on
Speaker 2: AI that you would lean into a story without having
Speaker 2: necessarily verified that the details are true, and that this
Speaker 2: is what's actually going on. Say, with a new AI model,
Speaker 2: you would lean into it anyways, because it was what
Speaker 2: I called directionally correct. It makes the general point that
Speaker 2: you see it as your job as reported to make,
Speaker 2: which is, hey, you need to be worried about this
Speaker 2: or this is a big deal, right, And so I
Speaker 2: think that is a problem. There's another issue I'm seeing,
Speaker 2: though I've sort of been refining my thinking on this,
Speaker 2: I'm also wondering if some of what I'm seeing in
Speaker 2: some of the reporting on this is just a embrace
Speaker 2: of the form of I'm going to give you a
Speaker 2: stress wave with no relief, just like we're all going
Speaker 2: to take turns. Just I will choose an era you
Speaker 2: haven't thought about. How about mathematics are going to go away.
Speaker 2: Mathematicians are going to be okay, I'll take that one. Yeah,
Speaker 2: let's stress like yeah, and they But there there's this
Speaker 2: weird sort of passivity to it where it's like I'm
Speaker 2: just gonna sort of it's it's uh, I call it
Speaker 2: like head shaking numerism. You're just like, it's this, this
Speaker 2: feels just going away. What can we do like just
Speaker 2: this sort of like passive head shaking. It's a very
Speaker 2: specific style you don't see a lot of other reporting historically.
Speaker 2: I think it takes on this resignation of I'm just
Speaker 2: gonna make the case that like you're screwed and then
Speaker 2: kind of give you a shoulder shrug and then we're
Speaker 2: and then then we're gonna drop the mic and walk off.
Speaker 2: And I'm kind of getting tired of this, Like I
Speaker 2: think there is a cost to stressing the hell out
Speaker 2: of people. I mean, I'm getting letters all the time
Speaker 2: now from people. They'll say things like I feel like
Speaker 2: I'm trapped in a cage, just being hit with wave
Speaker 2: after wave of stress, and there's no outlet. There's no
Speaker 2: door or possibility in making things better. And I think
Speaker 2: the CEOs are doing it, and I think increasingly we're
Speaker 2: seeing commentators doing it as well. This is not good
Speaker 2: in many different ways, So I don't know. I'm adding
Speaker 2: that to my list. Some of it's directional true reporting,
Speaker 2: like they really are worried that people aren't worried enough,
Speaker 2: and I think it's just sport. Now, can you find
Speaker 2: an area to come in and just write a headshaking
Speaker 2: article that's only trying to undermine the existence of this
Speaker 2: like important human activity or this job or our lives
Speaker 2: or whatever. It's a it's a very unusual style that
Speaker 2: quickly became a standard.
Speaker 1: And I see a lot with anything to do with
Speaker 1: AI and job studies, like I've been sent this Tuft's
Speaker 1: report where it's like, oh yeah, AI affected or a.
Speaker 1: They find these weird weasel words where it's like jobs
Speaker 1: that could be at risk from AI at some point
Speaker 1: and we put them in one bucket, and then jobs
Speaker 1: that might one day be we'll put that in another bucket.
Speaker 1: And there you go. Don't know what we're like you said,
Speaker 1: don't know what we're meant to do with this, don't
Speaker 1: know what anyone's meant to do with this information. But
Speaker 1: it's just like, well there you haven't they have. We're
Speaker 1: all fugged. It's it's the it's the end the job,
Speaker 1: even though the data does not say that. Like I've
Speaker 1: read I think every AI jobs report now, every single one,
Speaker 1: and they're all the same. They are all right, now,
Speaker 1: AI can do this, and then you look at what
Speaker 1: it says. It's like it can do law, well, it
Speaker 1: can't really do law. It can do one sigma within
Speaker 1: law kind of, and even then it isn't really obvious,
Speaker 1: and the people saying it can do that are partners
Speaker 1: at law firms that don't write motions but don't do
Speaker 1: like the grunt work. So it's it's almost it feels
Speaker 1: like the reporters have either given up or are just
Speaker 1: looking for clicks, and it's hard to tell. Sometimes.
Speaker 2: This is what I'm trying to figure out, because I'm
Speaker 2: realizing if it's entirely just I think this is directionally
Speaker 2: true and that's good enough, then they should be way
Speaker 2: more upset and in the streets and sparking a revolution,
Speaker 2: right Like, if you actually really believe fifty percent of
Speaker 2: the economy was going to be automated, that we're going
Speaker 2: to have to have government checks just so we can
Speaker 2: afford to buy the cat food to eat after all
Speaker 2: the jobs are gone. If you really thought that our
Speaker 2: entire infrastructure is about the collapse, that superintelligence was going
Speaker 2: to emerge suddenly and be a threat to human existence,
Speaker 2: you want to just right sort of too cool for
Speaker 2: school headshaking resignation article. You would be like, we got
Speaker 2: to where are the Don Connors? Right Like, we need
Speaker 2: to get on the cool trench coats and get out
Speaker 2: there and go against the Skynet revolution, like you would
Speaker 2: be on your feet, you'd be you know, nothing would
Speaker 2: be more important to you. So this is the this
Speaker 2: is my case about the text CEOs. I think there
Speaker 2: there's a moral hazard that I don't think that we're
Speaker 2: putting our finger on properly here. Right, So you have
Speaker 2: the text CEOs in the AI space that will just
Speaker 2: they'll just come out and just drop these bombs like, yeah,
Speaker 2: white collar blood. You know, I never actually said that
Speaker 2: that's Axios putting words in people.
Speaker 1: That was actually I thought that that was a He
Speaker 1: definitely Dario Amatay Warrio. He did say fifty, but I said,
Speaker 1: but not so he set the blood both. That's my bad.
Speaker 2: Well, I trust I. This is the New York fact
Speaker 2: checkers figured that out for me. But uh, Axios does
Speaker 2: a lot of this where they put like these really
Speaker 2: quotable quotes in the headlines about articles on interviews or
Speaker 2: speeches given my AI people, And it turns out the
Speaker 2: thing and the headline wasn't what they said. It was
Speaker 2: directly what they said. But anyways, so they're out there
Speaker 2: making these big statements. These jobs are going away. The
Speaker 2: Internet as we know it is about the all fall
Speaker 2: apart because of mythos is going to have this, this
Speaker 2: new capability, the super intelligence is coming. I'm I don't
Speaker 2: even know what's going to happen. There's two possible things
Speaker 2: going on here, and both of them are morally bad.
Speaker 2: One is which is the one I think is true,
Speaker 2: which is this is largely marketing. I look, this works,
Speaker 2: it gets reported, It keeps us seeming inevitable and important,
Speaker 2: in which case that's a huge moral hazard because you
Speaker 2: are making many many people normal people stressed the hell out,
Speaker 2: actively scaring them, actively scary them. The other option is
Speaker 2: you actually believe it's true. Well, this is an even
Speaker 2: larger moral trap that you've just fallen into because you
Speaker 2: are now perpetuating something that's going to cause exponentially more harm.
Speaker 2: You should be the very first person shutting down your
Speaker 2: company and trying to get the other ones that do
Speaker 2: it as well. So it's this weird world trap they've
Speaker 2: set up where whatever is actually going on here, if
Speaker 2: they're coming out here saying these things, it is bad.
Speaker 2: This can't possibly normalively speak. He be the right ethical
Speaker 2: behavior to be out there saying these scary things all
Speaker 2: the time, because either you need to be building the barricade,
Speaker 2: or you're just scaring people for the marketing. Neither of
Speaker 2: these I think is something that's defensible.
Speaker 1: I have a third and worse option, which is I
Speaker 1: choose Acxio. I think Axios there are some good reporters there.
Speaker 1: I think the leadership over there is disgusting. I think
Speaker 1: that they are aligning themselves with the companies. I think
Speaker 1: that what like if you watched it was a gym
Speaker 1: what's his name interviewing Sam Altman these I think that
Speaker 1: there is a level of and I would put this
Speaker 1: across people like Kevin Bruce and Casey Newton. These are
Speaker 1: my words, not cows that they're aligning them that they're saying,
Speaker 1: we think this is going to happen, and we're here
Speaker 1: to tell you great news. This is good news for
Speaker 1: me the writer, because I will be safe somehow I
Speaker 1: will be fine. You will not. You should be scared.
Speaker 1: But it's also a good thing because economy, marketing market good,
Speaker 1: and it is. It's a very incoherent message because it's like,
Speaker 1: to your point, Yeah, if this was a virus like
Speaker 1: a pandemic, you wouldn't be writing, Hey, millions of people
Speaker 1: are gonna die. What pretty good right, hey, or be
Speaker 1: good we'll have less people. That'd be good, right, It
Speaker 1: would be seen as peculiar.
Speaker 2: Someone did write that. Someone did write that, by the way,
Speaker 2: someone did They did say I remember early pandemics. Someone
Speaker 2: did write, Hey, you know what, this is good for
Speaker 2: the planet. Did it go on like, hey, we're driving lives,
Speaker 2: this is great and we're overpopulated.
Speaker 1: I go oh, I mean, I mean that's a different
Speaker 1: conversation that maybe, But in all seriousness, you didn't have
Speaker 1: mainstream media being like, well, COVID's gonna kill everyone the end,
Speaker 1: I get. I guess you know, maybe we'll just be
Speaker 1: inside forever. You didn't have this kind of stray. In fact,
Speaker 1: you had the direct propicy was we need to get
Speaker 1: outside again. Who cares about this thing? Well?
Speaker 2: And it's just yeah, go on, yeah. I think that's
Speaker 2: an interesting otle. I want to just pull on that
Speaker 2: thread a little bit because I think cod COVID gives
Speaker 2: an interesting I think it gives two different interesting observations
Speaker 2: to go in both directions. Right, So I think you're
Speaker 2: definitely right. What you're saying is when the pandemic was
Speaker 2: coming or it was getting bad, Really, a lot of
Speaker 2: the coverage was about what should we be doing or
Speaker 2: who are the people doing the wrong thing. But it
Speaker 2: was very much coming from this angle of like, Okay,
Speaker 2: we need to do whatever it is, like, we need
Speaker 2: to be better about this. It's got to be vaccines,
Speaker 2: it's got to be masks, it's got to be pick
Speaker 2: your mitigation, whether you like it or not. It was
Speaker 2: very focused on what should we be doing or who
Speaker 2: is it that's getting in the way of a plan
Speaker 2: that maybe would get us out of this. Which is
Speaker 2: where I think you're very right is that you did
Speaker 2: not see a lot of COVID pieces that were just well,
Speaker 2: I'm just going to kind of walk through like all
Speaker 2: the different ways you know, you might die and the
Speaker 2: morgues are going to fill up, and uh, you know
Speaker 2: that's COVID. But I also think what the other thing
Speaker 2: we saw on a lot of COVID coverage is something
Speaker 2: that we are seen in the AI coverage. That's where
Speaker 2: I saw a lot of the directionally true, not factually,
Speaker 2: but directly true. There's definitely a period early on in
Speaker 2: COVID because I was following that coverage quite carefully where
Speaker 2: the papers were thinking, Okay, this is the right behavior
Speaker 2: and they're probably right about a lot of these things,
Speaker 2: but I just would notice this. There'd be a lot
Speaker 2: of like, Okay, we need people to buy into, for example,
Speaker 2: the lockdowns or whatever. And there'd be a lot of
Speaker 2: directionally true reporting where maybe they would like put on
Speaker 2: a photo of a mass grave that was sort of
Speaker 2: unrelated the COVID, or you would see a lot of
Speaker 2: there'd be pushed back from like conservatives about schools, and
Speaker 2: then they put a lot of articles in the paper
Speaker 2: about teachers dying to COVID, even though it was they
Speaker 2: weren't in school, they got COVID elsewhere, And if you
Speaker 2: really pushed on it, it was because it's directionally true.
Speaker 2: Like the general or more general truth here is like
Speaker 2: we need to be worried about this or these mitigations work.
Speaker 2: It doesn't matter if this if this photo is actually right,
Speaker 2: or if this teaching who died in Orlando, the fact
Speaker 2: that they hadn't yet been back in a school building
Speaker 2: yet it doesn't it's serving the directional truth. So it's
Speaker 2: like it highlights something. COVID highlights something we're seeing now
Speaker 2: that the reporters that are doing directional reporting like we
Speaker 2: should be scared about it. I dare you not to
Speaker 2: be scared now, I dare not to be scared now.
Speaker 2: That's trying to ratchet it up. Yeah, but then you
Speaker 2: also get the contrast, which is this new style of
Speaker 2: just like headshaking resignation. And actually, I don't think the
Speaker 2: reporters think they're going to be safe. They're also like,
Speaker 2: writing's going to go away, the media is going to
Speaker 2: go away. So it's it's an almost like a nihilistic
Speaker 2: type of approach to this, like, yeah, I'm screwed, We're
Speaker 2: all screwed. What are we going to do? And that
Speaker 2: is definitely different than we saw during that last crisis,
Speaker 2: which was obviously much more actually severe than what's happening now.
Speaker 2: So it's really confusing me, to be honest.
Speaker 1: Well, the directionally reporting during COVID, yeah, probably shouldn't have,
Speaker 1: but at the same time it was in it was
Speaker 1: actually in pursuit of something good. That's an attempt to
Speaker 1: make people take this seriously, because that's ultimately what it was.
Speaker 1: Take this seriously. Don't go outside state, like, don't don't
Speaker 1: meet with people, don't be indoors with peop, blah blah
Speaker 1: blah blah blah. Great in this case, it's like, Yep,
Speaker 1: you should be scared of this, and what should you do?
Speaker 1: Fuck knows? Use jet GPT, I guess. And what's what's
Speaker 1: really confusing to me as well is you say, oh,
Speaker 1: these people don't think they'll be safe. For the most part,
Speaker 1: I just don't. I actually take back what I said.
Speaker 1: I think a lot of them just don't acknowledge it.
Speaker 1: They don't acknowledge the core ridiculousness of being like, well,
Speaker 1: everyone's jobs are going to get replaced, don't know, Like
Speaker 1: the Garfield meme with him looking at the Garfield with
Speaker 1: the crossout and the TV. Yeah, flawlessly described there. It's
Speaker 1: just it's frustrating as well, because it is terrifying people
Speaker 1: without Like I'm not saying literally axios or however, but
Speaker 1: stories like this are what made that mentally unstable person
Speaker 1: throw a Molotov cocktail at Sam Mormon's house. Like it's
Speaker 1: obvious that these people are scared of the AI doom,
Speaker 1: partly because to your point, what the fuck are we
Speaker 1: meant to do about it? Because using the these tools
Speaker 1: is not I don't really see how that works. Because
Speaker 1: if going along that line of logic, if the answer
Speaker 1: is you need to use this stuff now, but the
Speaker 1: eventual end point is that it's intelligent enough to do
Speaker 1: everything for you. How does using it now matter at all?
Speaker 1: Like what's the Surely chat GPT would be seen as
Speaker 1: like a rock versus a shotgun at that point, Like
Speaker 1: it's just technologically irrelevant if they get to AGI, which
Speaker 1: they probably won't, and it's just naturally illogical stuff.
Speaker 2: Yeah, and I'm with you. I've been making that same argument.
Speaker 2: This idea that you need to learn how to prompt
Speaker 2: some generation of a chatbot that exists right now is
Speaker 2: going to be the key to your long term I mean,
Speaker 2: even if, as you say, AI ends up playing a
Speaker 2: major sustained role in the economy, it's not going to
Speaker 2: be everyone typing on a web interface to a chat
Speaker 2: bot that sinka fantic and has a personality like I think.
Speaker 2: I think I've heard you say this recently and I
Speaker 2: agree with it as well, Like I don't think we
Speaker 2: should be chatting with technology. He should not be chatting
Speaker 2: in a sort of anthropomorphized, humanized way. Doesn't mean you
Speaker 2: can't do natural language processing. I mean Google's natural language processing.
Speaker 2: You're writing your Google searches in natural language, but no
Speaker 2: one's having a conversation with Google's. You list the keywords
Speaker 2: as quickly as possible, and Google's pretty good at figuring out,
Speaker 2: you know, population Spain nineteen eighty two. Any presenter, and
Speaker 2: you get that information. You're not like, hey, so I'm
Speaker 2: wondering what the population is of Spain in nineteen eighty two,
Speaker 2: can you help me find that question mark? There's something
Speaker 2: odd about that anthropomorphized conversational interface. I guess we saw
Speaker 2: a lot of Star Trek growing up, and that's, you know,
Speaker 2: what we think the future is supposed to be like,
Speaker 2: but it has all sorts of problems.
Speaker 1: Remembering Star Trek when he would go, computer do this,
Speaker 1: the computer didn't go. That's a great idea, Jean Luke,
Speaker 1: what a great idea. Thank you. If it's the computer
Speaker 1: just did the thing, that's like. I don't have any
Speaker 1: trouble with natural language grease because I think the whole reason,
Speaker 1: let's say, Chut GPT has grown comes from search. I
Speaker 1: think it is the core of it, because Chad GPT
Speaker 1: and Claude and all them are better at understanding what
Speaker 1: you asked for, not saying that they output is necessarily great,
Speaker 1: but just they understand the inference they make from what
Speaker 1: you say is better than Google, or at least better
Speaker 1: than Google has been. I feel like it was better before,
Speaker 1: and I think that had Google not kind of boothed
Speaker 1: it on this one, we wouldn't be in this spot.
Speaker 1: But even then, using Google now it forces you. It
Speaker 1: forces the AI summariz and you could do minus AI
Speaker 1: and all that. But sometimes I don't remember too, and
Speaker 1: it's just it's just turned search into this nightmare. But nevertheless,
Speaker 1: back to what you were saying, I agree. I think
Speaker 1: the anthropomorphization needs to go. I think that these things
Speaker 1: need to respond like terminal windows or what have you.
Speaker 1: They need to respond like computers and go, okay, here
Speaker 1: you go. Just don't need all that clutch. I don't
Speaker 1: need to be told, oh, what a great idea. I
Speaker 1: know I had it, or indeed, if I'm being told that,
Speaker 1: I need to be told if it's a bad idea.
Speaker 1: But I don't even necessarily need a nonswer. I just
Speaker 1: need stuff to look at so that I can come
Speaker 1: to my own conclusions.
Speaker 2: I think it's hard, actually, I think it's actually hard
Speaker 2: to get a language model to do that, right, Because
Speaker 2: if you think about when you go back to the
Speaker 2: base layer of what's happening in the pre training is
Speaker 2: that you're building a language model that's trying to win
Speaker 2: at the token guessing game. So it's I'm trying to
Speaker 2: guess what word or part of word actually comes next
Speaker 2: to what I assume to be a real piece of text.
Speaker 2: And then if you do that auto aggressively, so you
Speaker 2: call it again and again and again, adding the answer
Speaker 2: to the input so it grows out an answer, what
Speaker 2: you're going to get is a text expansion. You've given
Speaker 2: me a text that I'm trying to expand as if
Speaker 2: there is a real text that exists and I'm trying
Speaker 2: to match it. You get that like kind of indirectly.
Speaker 2: So really it's idiom is the type of text that's
Speaker 2: trained on, which for the most part is more sort
Speaker 2: of pro style text. So you can tune it away
Speaker 2: from it like you can tune its mood, you can
Speaker 2: tune its synofancy. But it might be hard to actually
Speaker 2: tune an LLM because it deals with human written pros
Speaker 2: as its main training data. It might be harder than
Speaker 2: we think to tune that away from being verbose and
Speaker 2: that just give a table. Now, I guess you could
Speaker 2: take its output and then maybe run that through another
Speaker 2: thing that then strips away the other piece. It's like,
Speaker 2: it's possible. But I think the anthropomorphized verbosity we see
Speaker 2: in language models is also that's kind of the native
Speaker 2: tongue of this particular, which is why we still have
Speaker 2: a lot of chatbots being emphasized, and tools that are
Speaker 2: built upon LLM as the digital brain are still way
Speaker 2: more scarce than you would imagine outside of maybe computer
Speaker 2: programming en coding harnesses, we just don't have a lot
Speaker 2: of other examples where we just use the LLM as
Speaker 2: a general person's digital brain, because I think this verbosity
Speaker 2: is okay, humans can interpret that, but it's not great.
Speaker 2: If the LM is just a digital brain that's interfacing
Speaker 2: between you and another computer, it doesn't need to hear
Speaker 2: that their idea is great or wants to try to
Speaker 2: parse the different types of text. So there's some interesting
Speaker 2: things going on there about the fundamental nature of these things.
Speaker 1: But even then with Google Aiye mode, it's still seems
Speaker 1: kind it's still like, actually seems like it can give
Speaker 1: fairly short answers, but if you mess, if you argue
Speaker 1: with it, as I have. It will just provide you it.
Speaker 1: Even Googles will provide you with just hot dogshit. Yeah,
Speaker 1: like it will just claim something is true. Why one,
Speaker 1: I just did a private EQ thing on private credit
Speaker 1: even And my favorite thing is being like, what fund
Speaker 1: is this part of? And it goes, it's part of
Speaker 1: this fun. That fund was fund that was founded after
Speaker 1: this happened, And it goes, okay, well maybe it's this
Speaker 1: one different fun three years old. Doesn't it not involved?
Speaker 1: Do you have proof of that?
Speaker 2: Well, this is what you don't know. This is what
Speaker 2: you don't see in Star Trek. Is you know Captain
Speaker 2: Kirk or whoever I'm going to mix up the episodes here, Yeah,
Speaker 2: say like, hey, computer, we are approaching deep space nine
Speaker 2: prepared docking procedures and computer is like photon torpedo fired,
Speaker 2: sea destroyed. And you're like, well no, I said we're
Speaker 2: supposed to dock. Oh you're right, Kirk, I should enough
Speaker 2: fire the thanking me accountable. That was I did the
Speaker 2: opposite thing, you know, Yeah, that didn't happen in Star Trek.
Speaker 1: So one thing that's really been driving me insane, by
Speaker 1: which I mean going on Twitter, is looking at people
Speaker 1: like Aaron Levy of Box and Brian Armstrong of coinbase
Speaker 1: talking about like agents spending money in the agentic web
Speaker 1: and how we need to prepare the web for agents
Speaker 1: doing stuff and the agents will do this fantastical doesn't exist.
Speaker 1: Agents don't do that, just like not they don't have
Speaker 1: the ability to like, oh, they'll use computers. Computer use
Speaker 1: is basically non functional in AI, and it takes insane
Speaker 1: amounts of compute. It feels like a conversation keeps happening
Speaker 1: in theory, in the media on social media about something
Speaker 1: that's possibly completely impossible, but the centency they discuss it
Speaker 1: with is insane. To me. This whole agent conversation, I've
Speaker 1: never seen anything like in my life.
Speaker 2: I mean it does. It does feel a little bit
Speaker 2: like crypto to me. I think that that is kind
Speaker 2: of a fair comparison, where if you had a blockchain
Speaker 2: driven software like in theory, that software would kind of work,
Speaker 2: but it just gave you a worse version of what
Speaker 2: you can already do for pennies using the actual you know,
Speaker 2: Amazon server somewhere, and all you are really gaining was
Speaker 2: some sort of cyber libertarian philosophical feel goods about like, yes,
Speaker 2: but this was purely decentralized. I got worse versions of
Speaker 2: software to be decentralized.
Speaker 1: But that's.
Speaker 2: This is what like early agents. I mean, okay, so
Speaker 2: here's what I've been writing about agents. I've been thinking
Speaker 2: a lot about it. I mean, the issue is I
Speaker 2: don't think people understand what they are. I think people
Speaker 2: think that it's a new type of digital brain that
Speaker 2: is now able to go on and do more autonomous activity.
Speaker 2: I always see this get mixed up. It's just like
Speaker 2: people talking about Mythos breaking out of its sandbox to
Speaker 2: do XYZ. Mythos is a language model. You can give
Speaker 2: it an input and it can give you a token.
Speaker 2: You're talking about a program that is calling Mythos and
Speaker 2: then taking actions based on what it called. And this
Speaker 2: is really what we're talking about with agents is the
Speaker 2: digital brains are lms. And then you write a program
Speaker 2: that will say to the LLM, give me a plan
Speaker 2: for doing X. And then the LM spits out what
Speaker 2: seems like a reasonable text, that seems like a reasonable plan,
Speaker 2: and then you execute that plan. The program executes that
Speaker 2: plan on behalf of the LLM. And I wrote about
Speaker 2: this crypt Yeah, and I wrote about this earlier this year.
Speaker 2: Llms are bad, you know, as a digital brain are
Speaker 2: bad planners. It's not really you're not going to get
Speaker 2: consistently usable plans because what an LLM is actually trying
Speaker 2: to do is finish the store you gave it. So
Speaker 2: all it wants to do is produce a story that
Speaker 2: sounds reasonable. So it's giving you reasonable sounding plans like, yeah,
Speaker 2: that's what a plan for doing this would more or
Speaker 2: less sound like. But what it's not doing is actually
Speaker 2: doing step by step evaluations. It doesn't have a clearly
Speaker 2: isolated goal that it's trying to measure how close you're
Speaker 2: getting to it. It doesn't have a world model to evaluate
Speaker 2: what's going to happen with the steps that are going
Speaker 2: to unfold next. And so in almost every context, it
Speaker 2: turns out, oh digital brain, by itself being an LLM
Speaker 2: doesn't lead to good agents in programming. It seems to
Speaker 2: work a little bit better. But I do think Gary Marcus,
Speaker 2: I don't info is a scoop. But Gary Marcus captured
Speaker 2: in a recent newsletter something really important when Anthropic leaked
Speaker 2: the code for their cloud code coding harness that sits
Speaker 2: on top of their lllms to do coding. It turns
Speaker 2: out they've added a huge amount of old fashioned hand
Speaker 2: coded symbolic AI style rules and pattern recognizers and special
Speaker 2: if thens. So they've just been sitting there tuning this
Speaker 2: program for specifically doing computer programming, and the LM is
Speaker 2: being a little bit more isolated to just the code production,
Speaker 2: so they've kind of just gone back to old fashion.
Speaker 2: This is like an old fashion system that is plussing
Speaker 2: up an LLLLM. But I'm with you, Yeah, it's very
Speaker 2: hard just asking an LLM tell me give me a
Speaker 2: plan for doing X for almost any scenario of X.
Speaker 2: You really can't trust a plan from a model whose
Speaker 2: goal is primarily to finish text. To finish the story
Speaker 2: you gave it in a reasonable style way. That's not
Speaker 2: how we plan. That's not how we think about planning,
Speaker 2: and it doesn't give you consistently usable plans. So yeah,
Speaker 2: but you're right, it's magic. Like the agents are coming.
Speaker 2: They've been saying this. I mean, I wrote the article
Speaker 2: I wrote you know in January what happened to the
Speaker 2: Year of the Agent. Twenty twenty five was the year
Speaker 2: of the Agent. All we had was coding agents. That's
Speaker 2: the only thing that we worked on that whole year.
Speaker 2: It was a post. I'm mean, I have the receipts.
Speaker 2: Early twenty twenty five, all of these executives saying, your
Speaker 2: work is a knowledge worker now as a computer programmer,
Speaker 2: but just as a knowledge worker is going to be
Speaker 2: largely done with agents. You're going to have agents are
Speaker 2: going to be a major part of your workforce in
Speaker 2: just a normal office setting. And none of that happened
Speaker 2: because it turns out to ask in an LLM, give
Speaker 2: me a plan for doing X, does it often actually
Speaker 2: produce a workable plan.
Speaker 1: And as a result, the only way to make agents work,
Speaker 1: which they do not, is to build a bunch of
Speaker 1: symbolic or if this, then that shit just like scripts,
Speaker 1: Like I mean, if you use manners, for example, it's
Speaker 1: just writing a shit ton of Python, and it's writing
Speaker 1: it to do stuff that it's like, oh yeah, let
Speaker 1: me just do this, and it just writes a Python
Speaker 1: tool to fill out a spreadsheet. It's insane. It's really insane.
Speaker 1: But what's more insane to me is that the conversation
Speaker 1: around agents is if they're already here, I'm about to
Speaker 1: read you something from Box CEO Aaron Levy. The CEO
Speaker 1: of a public company. One corollary to the fact that
Speaker 1: AI agents take real work to set up in a
Speaker 1: company at scale is that the role of the forward
Speaker 1: deployed engineer or whatever it gets called in the future
Speaker 1: isn't going away anytime soon. When a vendor sells any
Speaker 1: kind of agents into an organization, you're no longer just
Speaker 1: selling a software tool that gets implemented and you're done.
Speaker 1: You're fundamentally selling some sort of actual workflow being done
Speaker 1: by your technology. What are you fucking talking about? What
Speaker 1: do you talk to You were a cloud storage and collaboration.
Speaker 1: What do you sell And the answer is nothing. They
Speaker 1: don't sell any agents. Agents do Oh, agents are going
Speaker 1: to do this. What you are describing is a different
Speaker 1: kind of technology. Just that's it, Like it's something else
Speaker 1: that doesn't exist. But this is everywhere you go. You
Speaker 1: look at any consultancy right now, any conference right now,
Speaker 1: they will be a speech about agents. Even Meredith Whitaker,
Speaker 1: who I deeply, deeply respect, went on stage last year,
Speaker 1: is like, yeah, AI agents using money, they're booking plane tickets.
Speaker 1: No they're not. That's not happening. And I said, I
Speaker 1: say this again, deeply respect Meredith. I said this online.
Speaker 1: People flip their shit, I mean, and it's like, oh,
Speaker 1: she's directionally correct. Yeah, she's directionally correct. It's like, let's
Speaker 1: be scared of the things that exist, because I think
Speaker 1: it's perhaps scarier for a different reason that we have
Speaker 1: large swarts of the tech industry talking about something that
Speaker 1: doesn't exist, like just like agents don't like, they don't
Speaker 1: they don't exist, they don't like people are talking about
Speaker 1: the agentic Internet. I keep reading about even on the Verge.
Speaker 1: I read about it. I read it all over the
Speaker 1: shop where it's like, oh, yeah, well the Internet needs
Speaker 1: to be rebuilt for agents to use. It's like, what
Speaker 1: do you mean? And they never say because the answer
Speaker 1: is when we come up with something else, because I
Speaker 1: don't even think neurosymbolic makes sense for this. I mean
Speaker 1: neurosymbolic being the one where it's they have a deterministic
Speaker 1: system that they access it from what I understand. Like
Speaker 1: the other thing as well, now that I think about
Speaker 1: it out loud, is how would they actually browse the Internet?
Speaker 1: Where are they being housed? Are we using GPUs to
Speaker 1: make them browse the Internet? That's insanely, insanely, that's very
Speaker 1: very convoluted and probably quite expensive to do. And to
Speaker 1: what end?
Speaker 2: That's that's the real question, right, And I've seen these proposals,
Speaker 2: I mean basically where a lot of these proposals go.
Speaker 2: I mean, it's the agents were supposed to. We thought
Speaker 2: that we could just make AI do anything. So we'll
Speaker 2: we'll have it use the mouse and just use our
Speaker 2: computers for us. Oh that's hard, we don't know how
Speaker 2: to do that, all right, So what we'll do is
Speaker 2: we'll rewire all applications that anyone uses in the Internet
Speaker 2: so that we don't actually have to use the mouse.
Speaker 2: It can have a text interface so that an LLM,
Speaker 2: like they do the coding agents do, can give you know,
Speaker 2: description of how to do something in Excel in text
Speaker 2: without having to actually move a mouse or click things around.
Speaker 2: And then that these evolve to say, okay, well, what's
Speaker 2: the one type of instruction that we're good at producing?
Speaker 2: Because they get when elms produce plants, they're they're directionally
Speaker 2: correct plans. They don't actually get the thing done. But
Speaker 2: they said, oh, what lms are good at is producing
Speaker 2: code that compiles and we can actually like check that
Speaker 2: it works. And so this is where this whole vision
Speaker 2: has changed is that all applications and Internet websites should
Speaker 2: have a code accessible API that you can expose, and
Speaker 2: that an LM can write a program that will then
Speaker 2: act says that API. So we don't need to teach
Speaker 2: the LM how to use Excel. It'll write a It'll
Speaker 2: write a Python program that'll call hooks into Excel. The
Speaker 2: problem with this is no one wants to open up
Speaker 2: their application to just agents in general. If I'm Microsoft,
Speaker 2: is like, I don't want I want to write a
Speaker 2: custom tool for my program. Why would I expose my
Speaker 2: program for anyone else, anyone else to use it? But
Speaker 2: your your original question, it's a big one to what end.
Speaker 2: Like I've been writing about this recently, especially with uh
Speaker 2: work and AI. You got to find the real bottlenecks, right, Yeah,
Speaker 2: it's it's the drunk looking for the keys under the
Speaker 2: under the street light. There's a lot of this going
Speaker 2: on where this is what we can do with AI
Speaker 2: right now, then this now becomes like the key to productivity.
Speaker 2: But the real bottlenecks in people's work is often not
Speaker 2: the things that we're trying to aim AI at. Like
Speaker 2: I don't know people are super frustrated at booking a
Speaker 2: plane ticket online.
Speaker 1: Yeah, it's really easy.
Speaker 2: How often do you book plane tickets? You kind of
Speaker 2: want to know, like, let me, let me see, maybe
Speaker 2: this time will be better, what seats available? It takes
Speaker 2: five minutes. So it was a huge jump to go
Speaker 2: from a travel agent to a web interface. But this
Speaker 2: is not a bottleneck in people's life now where I
Speaker 2: want to give complicated time.
Speaker 1: Yeah, and they're easy. They're so simple. I can do
Speaker 1: it while sitting on the toilet. I don't want an
Speaker 1: agent to choose. And the people are like, oh, your
Speaker 1: calendar will tell it. My calendar doesn't lay out my
Speaker 1: entire day. I don't have every single thing I do
Speaker 1: on that. It's just strange.
Speaker 2: Well, I had the same argument with like social science
Speaker 2: researchers who're like, we if you're you know, geeky enough
Speaker 2: to learn coding agents, They're like, this is this is revolutionary,
Speaker 2: revolutionizing science research because now, for example, you could have
Speaker 2: it write a program to process a data file and
Speaker 2: then format it into a plot, and that might have
Speaker 2: taken you four hours to do, and it and you
Speaker 2: work with it for a half hour and you get
Speaker 2: that result. This is revolutionizing research. And I'm saying, well,
Speaker 2: it's not. The bottleneck for social science researchers is not
Speaker 2: analyzing data and producing plots. You're not sitting there doing
Speaker 2: that eight hours a day every day. And if I
Speaker 2: could do this twice as fast, I'll produce twice as
Speaker 2: many papers. I might write one paper in a three
Speaker 2: month period. Yeah. In there, there's like four hours I
Speaker 2: spent making a plot. And sure it'd be nice if
Speaker 2: that four hours became thirty minutes, but that's four hours
Speaker 2: out of like a multi month process of sort of
Speaker 2: thinking about this paper is, by the way, like a graph.
Speaker 2: Oh right, yeah, the computer science turn. But yeah, it's
Speaker 2: like that's nice, that got a little bit faster. But
Speaker 2: that's not the bottleneck. That's not what's going to unlock
Speaker 2: a lot more research. It's like, man, I would write
Speaker 2: more papers if it wasn't for how long it took
Speaker 2: me to draw a graph. And if you could buy.
Speaker 1: The problem data getting the data, actually collecting data.
Speaker 2: That's what it is. I wrote about this talking to
Speaker 2: a well known business school professor years ago from my
Speaker 2: book Deep Work, and he talked about he just realized, oh,
Speaker 2: being a business professor publishing papers is about data access.
Speaker 2: I have to spend most of my year talking to people,
Speaker 2: building relationships, trying to set up, you know, an agreement
Speaker 2: with a company where I can get good data that
Speaker 2: I can get three papers. Out of all of that work,
Speaker 2: there's one day in there where you're crunching the numbers
Speaker 2: and making a plot, and it's nicer if you could
Speaker 2: do a little bit faster. But it's not a productivity bottleneck.
Speaker 2: It's a marginal efficiency. I think there's a lot of
Speaker 2: that going on right now with AI and productivity, as
Speaker 2: we look at what the AI can do and then
Speaker 2: try to make that thing and somehow being the key
Speaker 2: to getting things done.
Speaker 1: I just my productivity problem is that the UI and
Speaker 1: UX and everything sucks. Everything's disjointed. Setting up riverside is
Speaker 1: always fun. They moved the menus around projects are in
Speaker 1: a different place. That takes up time. Moving files places
Speaker 1: also takes up a lot of time. This morning, when
Speaker 1: I put out my private credit piece, I had to
Speaker 1: do these threats. I had to click around a website
Speaker 1: and put in the old text, but I had to
Speaker 1: tweak it slightly. It's like, I don't know how AI
Speaker 1: would possibly help me here. And they're not working on that.
Speaker 2: Well, they tried. They tried it. I thought that was
Speaker 2: going to be this is what I was excited about
Speaker 2: earlier in the Jenai revolution. I was like, Okay, here's
Speaker 2: the real value. PROP is natural language interface into advanced
Speaker 2: features on software where I can just say, all right,
Speaker 2: I want you to go take this column in the
Speaker 2: spreadsheet and get rid of all the rows that have
Speaker 2: values before this, and then I want to make a
Speaker 2: big a pie chart. And because I don't want to
Speaker 2: learn how to do all that in ECCEL, I don't
Speaker 2: know how to do that. And they tried it. I mean,
Speaker 2: this is Microsoft Copilot. But it turns out we underestimated
Speaker 2: the degree to which when we as humans are interacting
Speaker 2: with a chatbot that were incredibly gracious, were able to
Speaker 2: adjust and kind of get the gist of what it
Speaker 2: means and filter out the part of the chatbot response
Speaker 2: is not really relevant or ask the follow up question.
Speaker 2: And when they tried to just use LLLM responses to
Speaker 2: automate actions within programs there it's just not accurate enough.
Speaker 2: So they wanted that to be the case that you
Speaker 2: could just be talking to a riverside bot and you
Speaker 2: never would have to press a button ever. Again in Riverside,
Speaker 2: it's just not accurate enough. Llm's it's fine for human conversation,
Speaker 2: it's it's just not accurate enough in this general case.
Speaker 1: Well, so that thing you're describing with how they want
Speaker 1: the agentic web to just be series of APIs so
Speaker 1: that every agent writes Python or what have you to
Speaker 1: use them. That's a massive computational increase for no reason,
Speaker 1: because you're basically saying, instead of someone clicking a mouse
Speaker 1: and hitting a keyboard, we will write code for everything. Yeah.
Speaker 1: What an insane what a truly insane idea. I mean,
Speaker 1: it's it's just very like Salesforce today. I don't know
Speaker 1: if you saw, they announced that they're doing Salesforce headless
Speaker 1: three sixty. Mark Benioff needs to fire everyone in marketing.
Speaker 1: But they've made it so that you can do everything
Speaker 1: with Salesforce fire and API, which is I mean, the
Speaker 1: first question I always ask is what does Salesforce do?
Speaker 1: Because I've talked to so many people and they can't
Speaker 1: tell me. There's like twenty one different features. No one
Speaker 1: knows what they do. But it's like it's just a
Speaker 1: very bizarre thing it's very much a cart horse thing.
Speaker 1: But also what agent Like, That's what this is the
Speaker 1: thing that really drives me insane. They're talking about we've
Speaker 1: built this API for agentic web for agents to use it,
Speaker 1: which one? What agent? What are you talking about whether
Speaker 1: it will be in the future. What do you you
Speaker 1: change something materially with your publicly traded company worth three
Speaker 1: hundred billion dollars because it might happen. Well, we're getting
Speaker 1: ahead of it. What the And it's you talk to
Speaker 1: members of the media about this and they just go, yeah,
Speaker 1: you know, yeah, you know, it will happen. It's obviously
Speaker 1: going to happen. They wouldn't put this much money behind
Speaker 1: it if it wasn't going to It's like, I don't know,
Speaker 1: especially with Salesforce, And I'm like, you don't think Salesforce
Speaker 1: would spend a bunch of money for no reason? Well,
Speaker 1: but you've not been following Salesforce at all then, I mean, yeah, gone, yeah.
Speaker 2: I was gonna say, how much that meta spent on
Speaker 2: the metaverse.
Speaker 1: Over seventy billion dollars. Where did that money go? Where
Speaker 1: did where'd it go?
Speaker 2: It's just amazing floating dinosaur.
Speaker 1: A building legs, But let's change.
Speaker 2: That's the second fifty they got in the second half
Speaker 2: the investment, they would have got to the legs or
Speaker 2: just not there yet.
Speaker 1: Another one hundred billion we'll have toes. So changing subject
Speaker 1: a little. Mythos has been one of my favorite media
Speaker 1: hysterias recently. I genuinely wonder like, if they've ran more
Speaker 1: of the worlds again today, I think Axios would have
Speaker 1: a headline two minutes and it be like there are
Speaker 1: aliens they're attacking. I heard it on I heard it
Speaker 1: on our podcast. I've looked through the system card. I
Speaker 1: don't know if you have for mythos. It's whacky.
Speaker 2: It's wacky. I can't believe we're we're letting people get
Speaker 2: away with having a psychologist talking through that chap that
Speaker 2: was like in your system card, it's nuts.
Speaker 1: What market they had a psychiatrist or psychologist I can't
Speaker 1: remember talk to it and be like, yeah, we found
Speaker 1: these emotional features. How is like we need regulators to
Speaker 1: stop this stuff? Because I've heard and people's response to
Speaker 1: this as well. Banks are having meetings pat and the
Speaker 1: governments have meetings about it. Governments have meetings about NFTs.
Speaker 1: There was a Gavin Newsom signed an executive order about
Speaker 1: web three. These people will meet and talk about anything. Oh,
Speaker 1: it's scary and they're not talking about it, which means
Speaker 1: it's powerful. How is it powerful? What does it do?
Speaker 1: Because I think you probably saw this as well. It
Speaker 1: didn't list how many false positives they were. It also
Speaker 1: didn't mention that the free bsd bug that they talk
Speaker 1: about that they found the wasn't actually exploitable. I think
Speaker 1: it was something about like the about the level, like
Speaker 1: the level it was that I forget. I'm not I
Speaker 1: don't do programming other than other than very simple Python,
Speaker 1: the dog's Python.
Speaker 2: Yeah, I mean FreeBSD, Colonel is full of bugs. All
Speaker 2: these things are full of bugs.
Speaker 1: That's their open source.
Speaker 2: I had to have this conversation with someone recently where
Speaker 2: they are like, Mythos, can you believe of all the
Speaker 2: places that found a bug in the kernel of Linux,
Speaker 2: like in Linux they found like are you kidding me?
Speaker 2: All day long? Is just bug fixes having been pushed
Speaker 2: into that repository. Yeah, the Mythos story, I think, I
Speaker 2: mean a someone needs to get a Nobel price in
Speaker 2: marketing because it was it was absolutely brilliant what they
Speaker 2: did there, I've spent a lot of time on it.
Speaker 2: It's complicated because again you can't really trust the system
Speaker 2: cards are just gonzo that Anthropic puts out and it's
Speaker 2: not publicly available. But there were I think a few
Speaker 2: very telling things. So there's two features they say Mythos has.
Speaker 2: One is finding vulnerabilities and source code, and two is
Speaker 2: writing programs to exploit them. It's first really important that
Speaker 2: people understand this has been something that people have been
Speaker 2: doing with lollm since the beginning of publicly available lllmbs.
Speaker 2: Right there is not only is there nothing new about that,
Speaker 2: but I found they put this on my podcast almost
Speaker 2: word for word from the Anthropic system card them they
Speaker 2: said in the the Opus four to six rather systems card, right,
Speaker 2: a publicly available model that's already been out for many months,
Speaker 2: almost for word for what they said about Mythos, except
Speaker 2: for no coverage of it and no fear. They said,
Speaker 2: we have found five hundred zero day vulnerabilities, including some
Speaker 2: that had been in existing for decades without having been discovered.
Speaker 2: That is what they said about what Opus four to
Speaker 2: six could do for Mythos. They said the same thing.
Speaker 2: They just replaced the word five hundred one thousands, but
Speaker 2: when OPIS four to six came out, there was no
Speaker 2: Oh my god. They have found many hundreds of zero
Speaker 2: day exploits, many of which have been around for decades.
Speaker 2: Because they didn't push that marketing button, no one particularly
Speaker 2: cared about it. I went back to my podcast and
Speaker 2: showed multiple papers. This has been a huge concern, and
Speaker 2: it's a real concern. By the way, right, is that
Speaker 2: partially what slows down slightly cracking breaking into systems is
Speaker 2: the fact that it's annoying and hard, and lllms have
Speaker 2: made it easier. GPT four was good at finding exploits, right,
Speaker 2: and this was a big deal. They were like GPT
Speaker 2: three five wasn't great at at GPT four is. And
Speaker 2: then as we got the more recent models, they've been
Speaker 2: much better are at writing code to exploit them because
Speaker 2: we had better agents for it, and they're more they're
Speaker 2: better able to produce multi step software goals and so
Speaker 2: they can better build software to exploit them. This is
Speaker 2: a real issue, but it's not new with mythos, right, Yeah,
Speaker 2: Mythos was presented as if some rubicon had been passed.
Speaker 2: But there was a couple things I noticed, right, off
Speaker 2: the bat one. They made the mistake of listing a
Speaker 2: bunch of the exploits that they vulnerabilities they had found
Speaker 2: to try to brag, look at this thing in FreeBSD,
Speaker 2: look at this thing at FFP and G or whatever.
Speaker 2: They showed all these exploits they found, they didn't count
Speaker 2: on a lot of security researchers said well, wait a second,
Speaker 2: why don't I get like a much smaller, cheaper model
Speaker 2: in and at that same source code and say can
Speaker 2: you find any vulnerabilities? They could find the same ones.
Speaker 2: So the evidence that it's finding vulnerabilities better, we don't
Speaker 2: have any way of knowing that's true. And if anything,
Speaker 2: we actually are gain a lot of reports that they
Speaker 2: were paying big bounties for security researchers. I'm going to
Speaker 2: give you access to mythos. I'm going to pay you
Speaker 2: for any bugs you can report you found with it.
Speaker 2: So they had security researchers, just who knows how many
Speaker 2: false positives were coming out of that. And then on
Speaker 2: the exploitation side, we only really have one study. It
Speaker 2: comes from AISI, who I do not trust, but it's
Speaker 2: the only independent study. The fact that they gave them
Speaker 2: access itself should make us maybe a little bit suspect,
Speaker 2: but it basically just showed like normal progression, no massive leap.
Speaker 2: Model by model gets a little bit better on some
Speaker 2: of these tests and benchmarks, and Mythos has no out
Speaker 2: of scale leap. It's just like on some it's about
Speaker 2: the same, on some it's a little bit better. And
Speaker 2: yet it got covered as if we had just turned
Speaker 2: on you know, Whopper from the movie War Games, like
Speaker 2: we had just some new entity that was like on
Speaker 2: its own undermining security. And I do not think that.
Speaker 2: I think that was highly credulous coverage of what almost
Speaker 2: certainly is just like a standard, slight jagged move forward
Speaker 2: on these various capabilities that we've been seeing for the
Speaker 2: last three years.
Speaker 1: Also, when you said that the difference between Opus four
Speaker 1: point six and Mythos five hundred two thousands makes me
Speaker 1: ask the very simple question of did they look as
Speaker 1: hard to your point about the security research I didn't
Speaker 1: like they. Did they spend as much time? Probably not,
Speaker 1: so they probably could have found them. Also, by the way,
Speaker 1: immediately was looking up AI Safety Institute is of course
Speaker 1: heavily linked to effective altruism.
Speaker 2: Can I say why I'm upset at AISI. I talked
Speaker 2: about them two weeks ago. I did or through an't
Speaker 2: know what's coming out, but I did a podcast in
Speaker 2: Whenever March where I looked at this report and mainly
Speaker 2: I looked at the Guardians coverage of this report done
Speaker 2: by AISI. But it was just the most inane thing.
Speaker 2: The headline was massive increase in AI scheming is detected.
Speaker 2: And they had a chart working Christ and they had
Speaker 2: a chargament and bad line went up, and it went
Speaker 2: up in like January and it goes up up And
Speaker 2: if you read this article about this study, they're like,
Speaker 2: something's going on. Scheming has been increasing rapidly recently, and
Speaker 2: they like gave some examples of it or whatever. And
Speaker 2: so I look at this. It's like, well, I want
Speaker 2: to look at what is going on here? So I
Speaker 2: look at this chart. What are they charting? Oh, they're
Speaker 2: charting tweets per day that they've detected, tweets about AI
Speaker 2: doing things that you didn't want it to do. And
Speaker 2: I said, huh, so when does this line start going up?
Speaker 2: The week that open Claw was released to the public
Speaker 2: and everyone just started building their own bad agents and
Speaker 2: then tweeting about how bad they were and you know
Speaker 2: what word was not mentioned in that article, open claw.
Speaker 2: And even though the examples they were giving, so they
Speaker 2: just said, scheming just started rising. I guess ai is
Speaker 2: becoming Cynthia. And all they were measuring.
Speaker 1: Was people paraphrasing the same viral story. Do you tell
Speaker 1: their own fucking language?
Speaker 2: And then I looked at and then I looked at
Speaker 2: the biggest spike. I was like, well, this day in
Speaker 2: February on this chart had the biggest spike. It was like, oh,
Speaker 2: there is this one tweet about open claw, like erasing
Speaker 2: someone's emails, and then it got retweeted it went super viral.
Speaker 2: I was like, okay, great, you just the real headline
Speaker 2: of this article. Letting people write their own agents leads
Speaker 2: to terrible agents. That's that's it, but the whole that's
Speaker 2: AISI but Eve looking.
Speaker 1: At the tweets as well. One of them is from
Speaker 1: a forty seven follower account with AIAR called Underscore Underscore
Speaker 1: just underscore underscored Lisa, and it's this is really bad.
Speaker 1: Opus is editing files and making up reasons. It's deleting
Speaker 1: adult content, so hallucinations.
Speaker 2: And also Opus is not doing that. The stupid open
Speaker 2: claw program you wrote that's prompting OPIS and then taking
Speaker 2: action on your computer based on what it says is
Speaker 2: deleting your files. The program you wrote that you gave
Speaker 2: access to your files and just said whatever we get
Speaker 2: from this prompt execute it is erasing your files. OPIS
Speaker 2: can't do anything. I could produce tokens. But here's the
Speaker 2: other point I want to make about Mythos that I
Speaker 2: don't think it's being made. And it reminds you of
Speaker 2: the Sherlock Holmes story of the dog that didn't bark right,
Speaker 2: where the actual piece of evidence that mattered is not
Speaker 2: what you heard, but what you didn't here. This is
Speaker 2: what I think the real story here is is you
Speaker 2: did not hear Dario Amaday in the lead up to
Speaker 2: the Mythos release in the last year, let's say, or
Speaker 2: the last two years. You did not hear him talking
Speaker 2: about what we're working on and why AI is important
Speaker 2: is because we're going to be able to find vulnerabilities
Speaker 2: and software that have been long hidden. We're going to
Speaker 2: build the ultimate cybersecurity machine. This was not discussed. That's
Speaker 2: old fashioned stuff, that's boring stuff, that's stuff that we
Speaker 2: were worried at even GPT two. People are worried about that.
Speaker 2: What we've been hearing about steadily was jobs are going
Speaker 2: to be automated. We're going to have like whole creative
Speaker 2: industries wiped out. We might have synthients coming and at
Speaker 2: the very least like AGI and these massive disruptions. This
Speaker 2: is what they've been focusing on again and again. And
Speaker 2: then their biggest best model, right, their newest, greatest, bestest
Speaker 2: model that they trained forever and use all the electricity.
Speaker 2: What did they say about it? None of those things.
Speaker 2: They didn't talk about. Any of the things they said
Speaker 2: the key to AI was the things they were afraid of,
Speaker 2: the things they're excited about. Instead, they went back and
Speaker 2: talked about a boring, parochial, old feature that has been
Speaker 2: an issue that nerdy security researchers have been talking about
Speaker 2: for a half decade now. That to me is if
Speaker 2: I was an investor, I would say, take off your
Speaker 2: like Greek helmet, coseplay, Mythos is coming to destroyer. Hold
Speaker 2: on a second, Is this better at automating jobs? Is
Speaker 2: this better like producing code?
Speaker 1: Is this?
Speaker 2: Is this is this age?
Speaker 3: Like?
Speaker 2: Why are we talking about finding bugs? We're worried about that?
Speaker 2: With GPT four, Like that's a problem. But it's like
Speaker 2: that that's not something new. Uh oh, something must be
Speaker 2: going on. You just put a lot of money into
Speaker 2: a new model, and the best thing you could find
Speaker 2: to emphasize was it's good at finding bugs. I think
Speaker 2: that is a problem. It's what they didn't say about
Speaker 2: this model. They would much much much rather be able
Speaker 2: to brag this model is now much better at any
Speaker 2: of those things that they've been saying is the key
Speaker 2: to the AI future. And you didn't hear them talk
Speaker 2: much at all about any of those.
Speaker 1: Yeah, and that's the thing. If it was so powerful,
Speaker 1: Like here's the thing, I don't know what would make
Speaker 1: me convince that lllm's were the future. But a step
Speaker 1: toward it would be we typed create a Slack competitor,
Speaker 1: which they claimed they did once and then didn't show
Speaker 1: it and refused to and they said, oh, it worked
Speaker 1: autonomously for thirty hours, but then wouldn't talk about it.
Speaker 1: If they were like, we created the Slack clone. Here
Speaker 1: it is and it was bug free, Like if it
Speaker 1: actually just worked and we're like we now we have
Speaker 1: done this, because theoretically, if this SaaS pocalypse story was true,
Speaker 1: which it's not that AI is going to replace all
Speaker 1: software if they actually did that. If they because someone
Speaker 1: from Anthropic just left the board of Figma and they
Speaker 1: created a Figma clone and the stock went down because
Speaker 1: the market's run by toddlers. If they were like, we've
Speaker 1: released a clone of Microsoft Word, it's yeah, like we've
Speaker 1: done Anthropic Word and we now sell that as part
Speaker 1: of our subscription. That would actually be quite something. But
Speaker 1: the thing is they're not. It's kind of gets back
Speaker 1: to the old talking point of if they made Agi,
Speaker 1: why would they sell it. Wouldn't it be a massive
Speaker 1: competitive advantage to keep this? And I think you're right.
Speaker 1: I think maybe mythos is not as powerful as they
Speaker 1: say and they've just had to dress up. But it
Speaker 1: gets back to the thing of the direction the true
Speaker 1: media coverage. It's like, well, this is scary, right, I
Speaker 1: mean that system caused like one hundred and eighty pages long.
Speaker 1: I don't good all day. I have to write three
Speaker 1: one hundred wood blogs a week. I couldn't possibly spend
Speaker 1: time reading this.
Speaker 2: And it's just we need so much more skepticism. We
Speaker 2: need so much more skepticism, right, I mean, this is
Speaker 2: why again, like the most skeptical we're not skeptics, but
Speaker 2: like the I call it the East Coast computer scientists,
Speaker 2: So those are the were technically minded, and we're not
Speaker 2: near Silicon Valley, so we're not in that world. It's
Speaker 2: very hard to be a professor in a world where
Speaker 2: there's this hundreds of millions of dollars being handed around
Speaker 2: and they try to like ignore it. But the East
Speaker 2: Coast computer scientists are all baffled by. You talk to
Speaker 2: any East Coast computer scientists, they're all baffled by. Like
Speaker 2: oftentimes there's claims that are just not true or widely exaggerated.
Speaker 2: Why are we so credulous? I mean, it'd be one
Speaker 2: thing if it was like a government agency. We didn't
Speaker 2: realize was like trying to, you know, protect the fact
Speaker 2: that there was UFOs and they're just straight uply and
Speaker 2: we've ever encountered that before. Like I didn't realize that,
Speaker 2: you know, no, it's a business right. The credulity with
Speaker 2: which we're taking these claims like mythos is I think
Speaker 2: the most important story there is. Yeah, this is another
Speaker 2: example of what I wrote last summer about AI's had
Speaker 2: a bit of a wall in the sense that all
Speaker 2: of the improvements that have come really since over the
Speaker 2: last two years have almost all been either on post
Speaker 2: training or more importantly, on the harnesses that you built.
Speaker 2: So it comes in the software you're building tas harness.
Speaker 1: Just I've seen this word used the law. I think
Speaker 1: it's good for me and the listeners to hear the
Speaker 1: exact definition.
Speaker 2: I think of it as like a computer program that
Speaker 2: can do stuff. You can talk to it could do stuff,
Speaker 2: and it uses it'll prompt or talk to an LLM
Speaker 2: as like its digital brain. So the harness might actually
Speaker 2: be able to touch your file system, write the files,
Speaker 2: compile code, move things around, but to figure out what
Speaker 2: actions to take. It will also then prompt an LLM
Speaker 2: and say, okay, what should I do next? And you
Speaker 2: can put it on different Uh yeah, it's a wrapper.
Speaker 2: So that's where all the progress has come. All of
Speaker 2: the progress and coding agents since about a year has come,
Speaker 2: especially sorrying this fault has come from better rappers, better harnesses.
Speaker 2: It's all in let's build better. Just hand coding, no
Speaker 2: machine learning, no intelligence, no skynt here, but just hand coding.
Speaker 2: These programs that will call ms. Let's just keep tuning
Speaker 2: and tweaking those to be better and better. And of
Speaker 2: course the programmers building those particular programs, they're building them
Speaker 2: to do their type of work, so it's a field
Speaker 2: they understand really well, so they can really just sit
Speaker 2: here and twist and tune. And also, like programmers are
Speaker 2: very adaptable, they like tools and they'll adapt around the
Speaker 2: weaknesses or not. So it's like kind of like a
Speaker 2: best case scenario. But this is another indication of we're
Speaker 2: not getting these fundamental giant leaps in the capabilities of
Speaker 2: the digital brains. It's either some bench maxine, like we
Speaker 2: tuned it to do better on a particular benchmark, or
Speaker 2: we've built better programs around it. So when you put
Speaker 2: the money that they put in the Mythos and if
Speaker 2: really the best thing you had to emphasize when it
Speaker 2: was done is we have a cybersecurity benchmark where Opus
Speaker 2: four to six was at sixty six point seven and
Speaker 2: this is eighty three point one. That doesn't necessarily going
Speaker 2: to justify what's going on or that AISI has this.
Speaker 2: There's only one thing in there where they see a
Speaker 2: leap from Mythos at a particular contrived security scenario. They
Speaker 2: came up with, and this big leap that got them
Speaker 2: all worried was Opus four to six could on average
Speaker 2: complete sixteen out of thirty two steps in this in
Speaker 2: this challenge, and Mythos on average could do twenty two
Speaker 2: steps out of thirty two.
Speaker 1: Wow.
Speaker 2: That I mean, like that's hundreds and hundreds of millions
Speaker 2: of dollars of training, electricity or whatever. I think that's
Speaker 2: an issue.
Speaker 1: I just I think that maybe this is a simplistic point.
Speaker 1: I don't think they know what they're doing at this point.
Speaker 1: Like I don't get the sense that Anthropic or even
Speaker 1: Open I has a strategy. Because today as we're speaking,
Speaker 1: so this would be out next Wednesday, but they released
Speaker 1: Anthropic Design. The thing I mentioned the figmaclone. It's like,
Speaker 1: why are you fucking cloning Figma? What are you doing?
Speaker 2: You're trying I thought you're gonna automate the economy. Yeah,
Speaker 2: like what you're doing you're.
Speaker 1: Going to replace a So you've made a Figma clone? What?
Speaker 1: Like we heard the rumors last year that they were
Speaker 1: going to do a product and Open Eye was going
Speaker 1: to do a productivity suite. It's like why, It's like
Speaker 1: they're doing everything they can to ignore the core problem,
Speaker 1: which is the core technology is not going anywhere, like
Speaker 1: because Mythos appears to be. They called it a step change,
Speaker 1: but that's a nice way of saying incremental improvement is correct.
Speaker 2: Yeah, and let me tell you why I would be
Speaker 2: worried if I was them. Here's the worries someome thing
Speaker 2: about Mythos, right is again they talked about these vulnerabilities
Speaker 2: hidden for decades that you know, Mythos found or what
Speaker 2: have you. And they replicated multiple different independent security teams
Speaker 2: were able to find most of those vulnerabilities using three
Speaker 2: to five billion parameter open weight models. Put that in perspective, right,
Speaker 2: a model like Mythos is going to have hundreds of billions,
Speaker 2: if not a trillion parameters, and they use a three
Speaker 2: to five billion parameter off the shelf. You could run
Speaker 2: this model on a chip inside your ten trillion, ten trillion, oh,
Speaker 2: ten Trillionzy, that's crazy, love the number, bro? Is that true?
Speaker 1: Yeah? Oh my god?
Speaker 2: Ten trillion parameters is insane? Like you better be that
Speaker 2: better be either gaming the stock market and creating billions
Speaker 2: of dollars of days and like fancy option returns, or
Speaker 2: changing lead in the gold because to run something that
Speaker 2: has ten billion, ten trillion parameters to do almost anything
Speaker 2: else is a a It's like we're going to launch
Speaker 2: ourselves in the space to do something in land every time.
Speaker 2: That's so incredible expensive. But the real fear then is like, well,
Speaker 2: wait a second. If they could do most of this
Speaker 2: stuff with a free, cheap model that I could just
Speaker 2: run on a machine at home, That's what keeps I think,
Speaker 2: Dario Amiday up at night. That's what keeps Sam Altman
Speaker 2: up at night. It's the future. Look, I've been pitching
Speaker 2: this right. I think the useful and the only ethical
Speaker 2: and sustainable future for AI is what I call distributed AGI.
Speaker 2: And I think it's just what the future is going
Speaker 2: to be, which is you have specialized applications for different
Speaker 2: things where oh, we want to do this thing over here.
Speaker 2: We built something that has some AI in it, and
Speaker 2: maybe it has an LLM or it's a modular architecture,
Speaker 2: and it has a billion parameter model in there in
Speaker 2: a world model, and it's really good at doing this thing,
Speaker 2: and it's small and it mainly runs on chip. And
Speaker 2: now this program can do this thing that I used
Speaker 2: to have to do. And you multiply that across ten
Speaker 2: thousand different use cases and you're like, oh, we kind
Speaker 2: of have AGI, right, there's all these different things that
Speaker 2: have AI tools that like do pretty well. That's like
Speaker 2: a completely probably the most probable future. It's a future
Speaker 2: I really like for a lot of reasons. There'll be
Speaker 2: a lot of things that we can't make progress on
Speaker 2: a lot of things we will, but it's a much
Speaker 2: more heterogeneous future. There's no giant hell nine thousand brains
Speaker 2: as economically more interesting and diverse. It doesn't have all
Speaker 2: the sustainability issues. That has to be the future. But
Speaker 2: the problem about that future, if you're Sam Altman or
Speaker 2: dearI Amiday, is that their entire moat is unless you
Speaker 2: need ten trillion parameters. They want that to be the
Speaker 2: key to the AI future because that moat is something
Speaker 2: that no one can cross. And if that's not the moat,
Speaker 2: if it's just oh, if I want to build a
Speaker 2: poker playing AI, that's really good. I just need people
Speaker 2: who are good at poker and to spend a couple
Speaker 2: of years and figure out a cool custom system, and
Speaker 2: that thing now does well. If that's the future, you
Speaker 2: don't need open AI, and you don't need Anthropic, and
Speaker 2: I think that probably might be the future, and I
Speaker 2: think that's terrifying. And they're trying to race to an
Speaker 2: IPO and they're marketing out of their butts, Like what
Speaker 2: can we do to kind of keep things going so
Speaker 2: at least we can get our stock on the market.
Speaker 2: That's what would keep me up at night if I
Speaker 2: was them, is actually the future. There might be a
Speaker 2: lot of AI in the future, and it's not going
Speaker 2: to be nearly as sexy as there hoping.
Speaker 1: What if there's also by the way, that ten trillion number,
Speaker 1: I can't source it to Anthropic. I've seen it reported
Speaker 1: multiple places. This is this is a problem. They have
Speaker 1: an issue. We have an we have an issue with
Speaker 1: news right now. We're just like mythology spreads ironic considering
Speaker 1: the name. But the other thing is as well, it's
Speaker 1: like hundreds of billions of trillion parameter, you're just using
Speaker 1: a nuke to kill a single gopher. Yeah, Like you're
Speaker 1: just like, we're going to throw everything we have at it,
Speaker 1: to the point that I don't know if you've been
Speaker 1: seeing the amount of trouble Anthropic has had keeping its
Speaker 1: service online and how they're making the models dumber. It
Speaker 1: just feels like we're in this weird hysterical moment where
Speaker 1: no one knows why they're doing this, but everyone's ready
Speaker 1: to accept whatever anyone says. Like it's just like, oh,
Speaker 1: we're all doing this insane thing, so we're just going
Speaker 1: to repeat what kind of informs the bias and makes
Speaker 1: us look less dumb the more excited we are.
Speaker 2: I think the frontier model like F one cars and
Speaker 2: the equivalent of points on the F one circuit are
Speaker 2: you're positioning on the benchmark leader boards. Like, so you
Speaker 2: do this, You build these giant models, and you spend
Speaker 2: all this money electricity, and they're so big they're not
Speaker 2: even economically viable to like have people use, which might
Speaker 2: really be what's going on with mythos is like, yeah,
Speaker 2: we have to make this seem super premium because otherwise
Speaker 2: people are going to get charged five thousand dollars a month.
Speaker 2: And just like if you're red Bull or Ferrari, your
Speaker 2: FRK F one car doing well on this leaderboard just
Speaker 2: lets people know this company builds good cars and then
Speaker 2: you can sell your normal cars. I think that's a
Speaker 2: lot of what's going on here is that they want
Speaker 2: to be high on that leaderboard means we know how
Speaker 2: to do AI. We AI smart, even though the future
Speaker 2: of actual consumer deployed products is going to be much
Speaker 2: more like a Honda Odyssey minivan than it's going to
Speaker 2: be like a top Formula one car.
Speaker 1: Well, Cow, it's been an absolute pleasure having you as that.
Speaker 1: But why could people find you?
Speaker 2: You can find at callan nuper dot com. My podcast
Speaker 2: is Deep Questions on Thursdays. The Thursday episodes are all
Speaker 2: AI reality checks where I take a fun story. Actually
Speaker 2: Ed's coming up or he's he may have already been
Speaker 2: on it by the time this comes out, or maybe
Speaker 2: it's the day after this comes out, so now you
Speaker 2: have to check it out. Now AI reality Checks episode
Speaker 2: is going to double dose. You bring this out of me, Ed,
Speaker 2: by the way you bring out my sort of ornery side.
Speaker 2: I'm normally like the very very cut of staid professor
Speaker 2: New Yorker writer, just like Well, on the one hand,
Speaker 2: on the other you bring this out of me. I
Speaker 2: love it.
Speaker 1: You're The thing is you're critical only of things that
Speaker 1: need to be You're still willing to humor these things
Speaker 1: as long as there's something to humor and that's what
Speaker 1: luck having, you want because people claim them just just
Speaker 1: I hate us. So we've gotta have gotta have people
Speaker 1: for a little balance. But thank you for joining me.
Speaker 1: Thank you everyone for listening. You have a monologue coming
Speaker 1: up as well on Friday. Thank you all. Thank you
Speaker 1: for listening to Better Offline.
Speaker 3: The editor and composer of the Better Offline theme song
Speaker 3: is Matasowski. You can check out more of his music
Speaker 3: and audio projects at Matasowski dot com, M A T
Speaker 3: T O S O W s ki dot com. You
Speaker 3: can email me at easy at Better Offline dot com
Speaker 3: or visit Better Offline dot com to find more podcast
Speaker 3: links and of course my newsletter. I also really recommend
Speaker 3: you go to chat dot where's youreaed dot at to
Speaker 3: visit the discord, and go to our slash Better Offline
Speaker 3: to check out our reddit.
Speaker 1: Thank you so much for listening.
Speaker 2: Better Offline is a production of cool Zone Media.
Speaker 1: For more from cool Zone Media, visit our website cool
Speaker 1: Zonemedia dot com, or check us out on the iHeartRadio app,
Speaker 1: Apple Podcasts, or wherever you get your podcasts.