S1: All right. What I'm going to do today is we're
S1: going to have a debate between myself and Marcus Hutchins.
S1: If you don't know Marcus, he is probably the most
S1: famous malware researcher in the world. He's been on the
S1: front page of wired, and he has a dramatically different
S1: view of AI than I do. So I am in
S1: sort of the pro AI camp and believing that it
S1: is net positive and that it is extremely useful today,
S1: and also getting more useful and most importantly, that it's
S1: going to start affecting jobs in a very serious way.
S1: Marcus has called it basically, autocomplete does not believe that
S1: it's intelligence, and believes that the impact is dramatically overplayed,
S1: mostly by people trying to scam people and make money.
S1: So again, we are on pretty much opposite sides of this,
S1: but we are also both security people and, you know,
S1: fairly well known in the security space, and we've both
S1: done a lot and accomplished a lot. And this is
S1: really showing that you can have two people who are
S1: in kind of the same space, can have dramatically different opinions.
S1: And what turned out to be fantastic here, because Marcus
S1: and I have been friends for some amount of time now,
S1: and we had this debate previously in text and it
S1: was fine in text. I was a little worried that
S1: we're both a bit snarky online, and I was worried
S1: that we were going to be snarky to each other
S1: in this debate that turned out not to happen. It
S1: was a very positive experience, and I think we made
S1: our points. Well, not only that, but I think I
S1: made a number of points that he really enjoyed. He
S1: made a number of points that I really enjoyed. I
S1: feel like we both learned something from this, and we've
S1: already committed to doing a follow up at some point
S1: in the future, but this one is hopefully going to
S1: be useful to anybody who is kind of agnostic or
S1: really just kind of wants to hear strong points being
S1: made on both sides of the conversation and also in
S1: a civil way. So that's what we're going to do here.
S1: I do apologize a little bit for the audio quality.
S1: We actually did this on zoom, and we should have
S1: done it on Riverside because you can't actually do 4K
S1: in zoom yet. So anyway, the next one is going
S1: to be higher quality. But um, let's jump into it.
S1: This is a debate between myself and Marcus Hutchins on AI.
S1: I think it's wrong to call what is happening now
S1: with AI hype and to tell people not to worry. Um,
S1: because I think it is a significant threat to people's
S1: jobs and, um, overall, just like a massive thing, uh, in,
S1: in much more of a way than crypto was, I
S1: think crypto was a giant. Nothing. Um, I wasn't excited
S1: about crypto at the time. I didn't see why why
S1: it was like everyone was taking it so seriously. But
S1: I see this as very different. Um, and so what?
S1: I see people like yourself or someone like another person.
S1: I have a lot of respect for, uh, Christopher Hoff,
S1: who I've been friends with for a very long time, actually,
S1: for him, like 15 years or something. 20 years. And
S1: he's frequently putting out similar stuff to you, whereas like, look,
S1: what are you guys talking about? This thing is like,
S1: not anything. It's going to be just like crypto. It's
S1: going to be gone soon. It's massively overhyped. So, um,
S1: when I see you posting those things, I haven't responded
S1: to you and I'm sure you're, like, itching to respond
S1: to me as well. Uh, we haven't done that because
S1: we planned on doing this. And this is a better,
S1: I think, a better way to start that off. But, um,
S1: I would respond to Chris Hoff and he would respond
S1: back and it would get kind of like adversarial, snarky
S1: or whatever. Um, but I mean, all that to say
S1: that I, I think that position is steering people in a,
S1: in a bad direction. It's telling people not to worry.
S1: And I'm not saying like panic. I'm saying this thing
S1: is real and we have to get ready for it.
S1: I think it's wrong to call what is happening now
S1: with AI hype and to tell people not to worry. Um,
S1: because I think it is.
S2: But when you say it's staring people in the direction of, uh,
S2: I guess the, for a lack of a better term
S2: is harmful. What do you think the harm is there?
S2: So if I say now, this isn't my personal position,
S2: but let's say I say AI is completely useless. Like
S2: beyond useless. It does nothing of value whatsoever. What is
S2: inherently bad about people believing that.
S1: Uh, they will not, uh, think about their own skills
S1: and their own capabilities and their own value and how
S1: that is potentially going to be impacted by AI if
S1: they don't see AI as a threat because it's a
S1: nothing burger, and it's basically all hype and marketing and
S1: a bunch of rich people trying to get rich, uh,
S1: even more rich, then they will ignore it. So to
S1: answer your question. The threat, the risk of that is
S1: them ignoring something that is potentially harmful to their career.
S2: But I think you could make that argument for, uh,
S2: we use the the example of blockchain or crypto specifically.
S2: You could have made that same argument for crypto. We
S2: could have said, uh, this is going to be this
S2: future big, impactful technology, which is something many people did say.
S2: They were like, everything is going to be blockchain, crypto.
S2: There were people arguing that Doge was going to replace
S2: the US currency. And you could have made the argument
S2: back then that if you don't go out right now
S2: and spend hours and hours of your time learning blockchain,
S2: you're going to be unemployed. You're not going to be
S2: able to, uh, make it in the future. So we're
S2: sort of trying to come up with this reason for
S2: why AI is different from blockchain. Now, I'm not saying
S2: it's similar. I, I think most people will agree large
S2: language models have infinitely more usefulness than blockchain did. But
S2: then it still comes down to what are we thinking
S2: that where do we think that this technology is going
S2: that is going to have such a big impact that
S2: people need to be paying as much attention as possible?
S2: And like, what even is the level of attention people
S2: need to pay? Like, do I just need to know, okay,
S2: ChatGPT is a thing and here's what it can do.
S2: Or do I need to spend hours and hours of
S2: my day learning how to use ChatGPT in order to
S2: avert whatever problem we're going to say is going to
S2: be the thing in the future.
S1: Yeah, sure. I mean. Yeah. Well, first of all, I mean,
S1: ChatGPT is just one app from one group, but I
S1: think I think I see what you mean, just AI
S1: in general. Um, no, I mean, what I advocate is
S1: people focus more on first principles thinking and like, understanding
S1: their own opinions and forming their own opinions. Uh, because
S1: what I'm trying to get to is like, what comes
S1: after the job replacement, which is like more human to
S1: human interaction. So I think people need to be more, um,
S1: thoughtful internally about who they are and what they are
S1: and like, what they're about actually having opinions. I think
S1: too many people who are in danger of being replaced
S1: by this technology, um, simply had a job executing tasks
S1: that someone else gave them, and that didn't require that
S1: much intellect, but it required enough intelligence. And this is
S1: another point we can debate. Like, is AI actually intelligence? Um,
S1: but it has enough intelligence for all these hundreds of
S1: millions of knowledge worker jobs. It has enough intelligence required
S1: that you couldn't automate it. So which is why they
S1: have the job today, and that if the bar is
S1: getting to your question, if the bar for AI raises
S1: above that level. So we could do basic intelligence tasks,
S1: which is, you know, 90% or 99% of knowledge worker, uh,
S1: Capabilities is included in that. Then those jobs go away.
S1: Hundreds of millions. I don't know the actual numbers for
S1: knowledge worker total employment, but I would say that vast,
S1: vast majority of those jobs go away and they are
S1: left not doing anything. And it's not like they could
S1: just pivot to the next thing, like previous technology trends, because, um, those,
S1: those pivots that they will make likely are also taken
S1: over by that.
S2: I, I understand that line of thinking of, okay, it's
S2: not like the printing press or, uh, early machine learning
S2: where I can just go into something that is not
S2: covered by the current models. But I think the idea
S2: of this just sweeping replacement, where llms are just going
S2: to take out 90% of the workforce, I don't personally
S2: believe that that is rooted in reality. I think, uh,
S2: we're basically extrapolating with, uh, it's almost like a new car, right?
S2: If I, If I've just invented the car, it only
S2: goes 15mph. And then if I do some tweaks to it,
S2: I spend like a couple of months doing some tweaks.
S2: It goes a little bit faster. And now we could
S2: say that if I doubled the speed in three months
S2: than in six months, I can double the speed again.
S2: And then you can just keep extrapolating, extrapolating, extrapolating. So
S2: at some point this car will break the speed of light. Um,
S2: whereas I think that's what we've been seeing with Llms
S2: is the LLM technology was very new and it had
S2: a lot of issues in the beginning. And that gave
S2: the illusion of a lot of very fast progress. We
S2: weren't actually seeing these models rapidly getting like better and
S2: better and better. All that was happening is we made
S2: a new technology that was a little bit, uh, like finicky.
S2: It was it had problems. And we slowly ironed out
S2: some of those problems, which made it look like it
S2: was advancing at this rapid pace. But it seems now
S2: that it is sort of capped out. Um, I would
S2: say the leap from GPT two to GPT three and
S2: 3.5 was pretty groundbreaking. But then as I look at
S2: the newer models, when we go from 3.5 to whatever
S2: they've called it now, like 001, I don't even know
S2: what the naming convention is anymore. Um, we haven't really
S2: seen any improvements. We're just seeing very slight increases in, oh,
S2: it's better at coding now. It's a little bit better
S2: at math. And what it looks like is happening is
S2: they've made the base large language model, and they've refined
S2: that almost as much as they can. And then they're
S2: building these sub models to then make up the slack
S2: where it can't do math or it can't do programming
S2: or whatever other areas it struggles. So I think what
S2: we're actually going to see is we're going to see
S2: large language models as a technology just cap out like
S2: they're just going to hit a wall. And then what
S2: we're going to see is them start building out sub
S2: models that make them better at a specific job. Now
S2: that would just be normal technological evolution like I as
S2: a malware researcher, I got into malware analysis when it
S2: was already machine learning automated. There was still enough space
S2: in my industry where I didn't need to become an
S2: expert in machine learning. I didn't even need to touch
S2: those models. I still to this day, have no idea
S2: what the virus classification machine learning models do, because there
S2: was still enough space for me to operate in that
S2: field as a human, and I don't see large language
S2: models as any different from that. I think what we're
S2: going to just see is people making sub models that make, say,
S2: malware analysis better. And it's like, okay, maybe I can't
S2: do manual virus classification anymore, but I can do this
S2: other thing and I think it's going to move slowly
S2: enough as they do that, that people can just pivot
S2: into not even different fields, but into different, different like
S2: roles in the same field where a lot of their
S2: skills are transferable. But rather than doing something that maybe
S2: is very pattern recognition and is very, uh, lm Them
S2: automatable they could move into something that takes the same
S2: skills but actually requires more critical thinking.
S1: Yeah, yeah. I mean, I, I definitely agree there's going
S1: to be pivots. Um, still possible. Unfortunately, I think those
S1: pivots are going to be mostly available to the really sharp,
S1: exceptional people who can pivot up to something much higher
S1: up in the stack, uh, which requires a lot more
S1: critical thinking that the AI can't do yet. Um, you
S1: had the analogy there of speed and approaching the speed
S1: of light, uh, a car approaching the speed of light.
S1: So here, here's what I think might be a fundamental difference, uh,
S1: in our opinions here, um, the speed of light that
S1: you're talking about, I agree it would be hard to say.
S1: Went from 30mph to 35mph. And next year we're going
S1: to be at the speed of light. That's a crazy statement.
S1: The difference is the speed of light that we're trying
S1: to get here, um, that I'm talking about is actually approaching.
S1: Cannot do the work of a knowledge worker. An average
S1: knowledge worker on earth. Okay. And so my argument to
S1: you is that that bar is not high at all.
S1: So it is basically and this is something we talked
S1: about in text. Um, right in the previous version of this,
S1: this discussion, it's like the average knowledge worker. They, they
S1: clock in and clock out. And I'm talking about mean,
S1: you know, like, uh, average just average across the globe.
S1: It's like, hey, got some new emails, respond to the email. Uh,
S1: the email says, hey, go, um, make a list of
S1: all the documents that are related to this project. Okay, cool.
S1: Here's your list. They send out the list. Um, someone says, hey,
S1: I need you to write a report on how our
S1: finances are doing. So what do they do? Is they
S1: pull a wiki, uh, article, they review this other document,
S1: they talk to Sarah in this other department that takes
S1: them 4 hours or 5 hours or two days or
S1: however long it takes, and they write a four page report,
S1: and then they take that report and they put it
S1: in an email and they send it out to the company.
S1: They are doing work. They're being paid 50 grand, 100
S1: grand for this work. And I'm claiming to you that
S1: this is a low bar, not for I in the future,
S1: but for I, we already had a year ago, in
S1: some cases two years ago. So my concern here is
S1: not about the technical definition of I. I don't give
S1: two shits about it. People are going to argue about, oh,
S1: AGI means this technically, specifically, I don't care. All I
S1: care about is impact to humans. So I'm claiming that
S1: the AI that we have a year ago already could
S1: do if it were properly orchestrated. A lot of the
S1: jobs that I just described. So customer service, um, responding
S1: to emails, setting up like a lunch meeting, just just
S1: basic tasks like that that could not be automated with
S1: regular tech, but can be automated with with this tech.
S2: Uh. Well, you see, I kind of disagree with that
S2: because I've unfortunately had to interface with LM based customer
S2: service agents. And they do have the the flaws that
S2: both of us are familiar with with LMS. They are
S2: not deterministic. If I give it the same question 50 times,
S2: it does not have the same response 50 times. Like
S2: depending on, uh, I don't know, the, the star alignment,
S2: how the electricity is going that day, who knows. It
S2: comes out with a different response to each, um, to
S2: each input. And that is fundamentally a problem because if
S2: you need a machine to do a deterministic task like
S2: finance reports or customer service, you cannot have a model
S2: that comes out with completely different answers every single time
S2: you ask the same question. And that is one of
S2: the major limitations that has been, uh, currently holding back
S2: large language models is that by their very nature they
S2: are not deterministic. So all of those tasks you are
S2: talking about are tasks which, uh, when you say knowledge task, you,
S2: you're kind of, I think, sort of talking about tasks
S2: that don't require critical thinking. They don't really require much intelligence.
S2: They're just doing like tasks that will be the same
S2: every time you do them. And if the large language
S2: model cannot even do tasks the same way multiple times,
S2: how can we then go and automate all of those jobs?
S1: Yeah, I think I think it's fairly easy. I mean,
S1: I've been doing this for a long time. Everyone I
S1: know who's building apps here, it's fairly easy to actually
S1: give somebody, uh, give an AI a list and say,
S1: you can only pick from these, like, this is the
S1: thing that was solved, you know, two and a half
S1: years ago, probably before that. Um, but you can make
S1: a system fairly deterministic. Um, and keep in mind, like
S1: customer service is already horrible. Everyone already hated customer service
S1: because it was so bad. Before I came out. Right.
S1: So it's not like customer service is the high bar,
S1: and we're seeing if we can get there. No, it's
S1: a horrible bar. Like, everyone hates it already. And oftentimes
S1: it has the exact same characteristics you were talking about,
S1: where it's like you're not getting good answers. The good
S1: answer is actually somewhere there in the knowledge base. But
S1: somehow this tier one or tier two person you're talking to,
S1: they didn't find the answer. Um, so a couple of,
S1: you know, objections to that. I don't think, um, current
S1: execution of a lot of these jobs is at a
S1: high level. And that's why I think it's possible to
S1: beat it. The other thing is, um, you're not actually
S1: when your boss says, hey, go make a write a
S1: report for this thing that is fundamentally a non-deterministic thing
S1: because the problem is always different that the human is given.
S1: So it definitely does require intelligence, right? You can't give
S1: that to a script because the script doesn't have like
S1: an if, then it doesn't have a deterministic thing to
S1: go look up the thing because the problem is always different.
S1: And this this is another thing that you and I
S1: talked about before, which I thought was super interesting, is like,
S1: what actually is intelligence and how much deviation from a
S1: standard slotted thing does it require to be called intelligence.
S1: So I would argue if someone is thrown a giant
S1: batch of emails and a giant batch of documents and
S1: they're like, hey, Chris. Hey, Sarah, um, go write a
S1: report on this. And I need it by today at 4:00.
S1: No one person or two people or no combination of
S1: humans is going to make the same report. They're going
S1: to make vastly different reports based on how they choose
S1: to go through the documents, based on how they choose
S1: to form the thing. How long is the report going
S1: to be? If you ask a thousand people to make
S1: a report, they're all going to be different, just like
S1: with an LLM. So and the other thing is just
S1: because that doesn't require like PhD level invention or creativity
S1: that still requires human intelligence. No, no tech prior to
S1: AI could do that job.
S2: So I'd actually would like to go back to the
S2: customer service example, because I think I would argue customer
S2: service is horrible for the exact same reason that the
S2: solution you proposed would be horrible. They are given these
S2: limited scripts of answers where if customer asks this, then
S2: you must answer that and they cannot go beyond their script.
S2: So if I come up with a unique problem that
S2: requires some kind of critical thinking, although the human on
S2: the phone has intelligence and is capable of critical thinking,
S2: they are not allowed to go outside of my script.
S2: And essentially you would run into the same problem with
S2: trying to do that with a large language model is
S2: you've just given it a list of canned answers it's
S2: not allowed to go outside of. And you're going to
S2: get the exact same results where there is no critical thinking.
S2: We can get the argument that I don't think I
S2: can think like full stop. But you're essentially just recreating
S2: the same awful system where it's just it's just parroting
S2: back canned responses. And I'm trying to ask my question,
S2: and it doesn't match up to some canned response that
S2: they have. So they're just repeating the same closest line
S2: that they have in their script. And then we both
S2: leave the interaction completely annoyed. And, um, I think that's
S2: exactly what is going to happen with large language models.
S2: And it's already happening. They're just feeding their dogs or
S2: their scripts into the LLM, and it's just making a
S2: worse version of a customer service representative, because not only
S2: does it still have the canned responses, but it doesn't
S2: have the critical thinking to go, oh, maybe I should
S2: escalate this to my boss.
S1: Yeah, no, it's an interesting point. I, I like that argument. Um,
S1: I would say the best argument I have against that
S1: is that, um, companies have been trying this customer service
S1: thing for decades, Like, uh, human customer service with the
S1: different levels. And they have tried like millions of different things.
S1: They've tried to give people autonomy. They've tried to not
S1: give people autonomy. The system they came up with is, look,
S1: follow this script, use this document. So I would argue
S1: to you that the reason that we see this in
S1: the industry is because it's the thing that works best.
S1: If what worked best was to just hire people who
S1: are somewhat smart and give them free rein, then that's
S1: what companies would be doing. Uh, and I don't think
S1: they're doing that because it doesn't work. Now, I would
S1: argue that as you move into tier three, tier four
S1: or whatever, whatever the structure is for these, um, like
S1: more senior customer service people, I feel like that thing
S1: you're describing is exactly what's happening. They are given free
S1: rein because all the known answers have already been tried,
S1: and now they're, um, now now they're going to have
S1: to use critical thinking.
S2: But I think a lot more jobs than you maybe
S2: estimate are they fall into that category of like, sure,
S2: a lot of it is automatable, but the reason they
S2: need a human in that role and not a script
S2: or a machine or whatever we want to call it,
S2: is because there is some aspect that requires critical thinking, um, uh,
S2: like the one I will push back against endlessly until
S2: my last breath is software engineers. I think there is
S2: no point in any of the AI's evolution, at which
S2: point you will be able to replace software engineers. And
S2: I think the disconnect that happens is a lot of
S2: the non software engineers or even the coders, the people
S2: who just don't fully understand software engineering don't understand the
S2: difference between writing code and designing software. So they're looking
S2: at ChatGPT and Cursor and Gemini or whatever, and they're looking, oh,
S2: I can put in a query and it spits out
S2: all of this code. And they're like, that is what
S2: coding is. So like that very easily could replace software engineering.
S2: But because they're not a software engineer, they don't understand
S2: the amount of critical thinking and design decisions that actually
S2: go into the software. I think, uh, a lot of
S2: software engineers will say that writing code is like 10%
S2: of software engineering. But then if you're not a software engineer,
S2: you're looking at that, that a small percentage that the AI, okay,
S2: sort of can do that and then ignoring the rest.
S2: And I think the same is true for every role. Um,
S2: until you actually have like spent a year in any
S2: given job, you don't really understand how much is automatable
S2: versus how much actually does require a human with critical thinking.
S2: So my personal belief is, I think for the large,
S2: like the overwhelming majority of all roles, there will never,
S2: ever be a point in which AI replaces that person.
S2: It might be able to make the person faster. You
S2: might be able to take one person and have them
S2: do work at a slightly more productive rate, but I
S2: don't think there is any point in which I actually
S2: starts replacing like, like large swaths of the job market.
S1: Yeah. Interesting. Um, so let me give you a counterpoint.
S1: And this this is anecdotal, of course, but I have
S1: a close friend who is a cardiologist, like actual MD
S1: actually sits in the, you know, the doctor's office every day.
S1: He also happens to be a bug bounty hunter and like,
S1: a programmer. So he he is, uh, you know, using
S1: whatever tools are approved at his particular job. And, um,
S1: he is now finding as of like this is this
S1: is six months ago. This is a year ago. Um,
S1: the analysis that this, this thing is doing is as
S1: good or better in some cases than the thing that
S1: he was going to write. So this is at the
S1: cardiologist level. Not not even like a nursing level. This
S1: is really, really advanced stuff. So it's it's taking all
S1: the inputs that came from like the patient and also
S1: from his notes. It's combining all that together with, of course,
S1: the knowledge of the model itself. And then, you know,
S1: giving out recommendations, recommending the actual drugs. And so my
S1: thing is like if this thing is actually doing the
S1: job and it's not just one person, right? A lot
S1: of doctors are actually saying that the output is matching
S1: their capabilities as well. We've already seen this with analyzing moles.
S1: We've seen this like in lots of different places. Um,
S1: and that brings me back to this one other point
S1: that you made earlier, which I thought was really interesting, um,
S1: how they're going to hit a wall. The Elon's hit
S1: a wall and they basically start building subsystems. This is
S1: exactly how I define AGI. I don't think AGI is
S1: going to be some breakthrough, generally intelligent thing. I think
S1: that is more of an AI breakthrough than we've actually seen.
S1: I don't think it's happened yet. What I'm arguing is
S1: going to replace the average knowledge worker is actually one
S1: of these, uh, Frankenstein systems where the LLM is like
S1: an orchestrator, and it's basically spinning up these individual things
S1: to go and accomplish individual tasks like write the email,
S1: organize the events, organize the conference, uh, summarize this or whatever.
S1: So it's going to have at its disposal dozens or
S1: hundreds of these little workers. But the problem is that
S1: you're going to buy this thing, let's call it a
S1: synth worker or whatever the stupid AI company is going
S1: to be called. You buy, you buy synth worker, synth
S1: worker shows up to work on Monday. It goes to
S1: the onboarding meeting. It talks to the manager. It talks
S1: to all its team members. It reads slack, it reads
S1: the wiki. And it starts working just like a regular employee.
S1: And it produces the work of a regular employee. And
S1: I would argue in a lot of cases, even better. Um,
S1: but if you look under the hood, it's not some
S1: all intelligent AI system. It's actually a whole bunch of tricks.
S1: It's like this big agent orchestrating a whole bunch of
S1: other ones. But functionally, what you end up with is
S1: this new AI named Julie actually does the work. And
S1: guess what? We didn't hire a human to do that job.
S2: Um, so I guess the first thing I'd say is
S2: that is not the definition of AGI that most people
S2: are going with. Um, but it is a fair point. Um,
S2: a lot of people have this idea of, uh, if
S2: we give the llms enough data, at some point, they
S2: start learning for themselves, and at some point they can
S2: do all of these things at the Submodels are doing, um,
S2: which I think most likely we both agree that that's
S2: that's horseshit. That's that's never happening. Um, but then the
S2: problem is when it comes to customizing these models, um,
S2: now you need a custom model for everything. You, uh,
S2: you need your model to identify tumors. And that is
S2: no different from where we've been like. That is what
S2: we've been doing for the past. Uh, I don't know.
S2: Probably longer than I've been alive, like 30 years, maybe more.
S2: And that hasn't replaced humans. Um, I would go back
S2: to my own personal example, which is machine learning based
S2: malware detection. Those machines have massively improved, uh, malware detection. Like, uh,
S2: back in the old days, you would have to open
S2: up a virus. You would have to manually determine that
S2: this is actually malicious code, and then enter a signature
S2: into a database. And can you imagine, like a human
S2: trying to do that with billions and billions of new
S2: files every single day? It's impossible. Um, so we built
S2: machine learning models to do that. But then the question is, well,
S2: how many malware analysts got laid off as a result?
S2: And the answer is there's actually more malware analysts today
S2: than there was in those days before the machine learning
S2: based virus classification. And the reason was the classification could
S2: only do so much, and you still needed people to
S2: check false positives. You still needed people to write reports,
S2: and there was always a place for humans and it
S2: wasn't even different skills like, uh, my job has been
S2: automated three times now, and in none of those three
S2: times have I had to, uh, even learn like, some
S2: massively different skill set. I've always been able to pivot,
S2: and I think that is all we're going to see
S2: with that is like, okay, maybe our cardiologist, I can
S2: automate a lot of that. So the cardiologists end up
S2: doing something else. I can't say what else they might
S2: end up doing because that's not my field and I
S2: don't know anything about it. But I do know that
S2: in my field, we have just been automating every single thing.
S2: We've been building sub models for malware classification. We've built
S2: some models for, uh, security alerts. We've built uh, sub
S2: models for analyzing code. Um, and we built up all
S2: of these sub models. But there's two problems there. The
S2: first is there is no evidence of it ever actually
S2: replacing humans. In fact, we've, uh, the industry has grown
S2: by a factor of ten since these machine learning models
S2: started coming out. And the second thing is they're very
S2: expensive to produce and maintain. So you're not actually getting
S2: the thing, which is the AI companies are trying to
S2: sell people on, which is, oh, you can get rid
S2: of employees, get rid of Greg, get rid of Simon,
S2: get rid of Sandra, and just keep all their salaries.
S2: Just pay $20 a month for this AI subscription. And
S2: now you're saving all of this, all of this money.
S2: But then when it comes to the actual who's maintaining
S2: these models, who's updating them for the latest, uh, whatever
S2: it is they're doing, because, um, like, obviously the technologies
S2: that they're working on are always changing. So someone has
S2: to maintain these models and they're phenomenally expensive to run.
S2: Like any kind of pattern matching model requires insane amounts
S2: of computational power. And what's actually just happening is you're
S2: taking those salaries, and I don't think you're even cutting them.
S2: You're actually spending more running these models or paying to
S2: have the models run than you would be having employees
S2: do it. It only makes sense when you're operating at scale,
S2: where employees physically like there aren't enough employees to do
S2: this thing. Like, imagine if you had to hire, uh,
S2: humans to manually categorize a billion malware samples. Like, it
S2: just wouldn't be possible. Uh, but when it actually comes to, hey,
S2: let's replace a human with this AI, uh, it usually
S2: ends up just working out more expensive. And the only
S2: reason we haven't started seeing that with Llms yet is
S2: because they're being subsidized by VCs. We don't see how much. Uh,
S2: I'm just going to say ChatGPT, because that's the one
S2: most people are familiar with. Um, we don't really see
S2: how much ChatGPT costs to actually operate. We don't know
S2: how much they're spending on acquiring the training data, training
S2: the model, how much computational power is being used, how
S2: much electricity is being used. And then when you think
S2: about an actual human being, it's just some cells. They
S2: need a little bit of water, a little bit of glucose,
S2: the amount of money it costs to operate a actual
S2: human being is almost zero. The only reason it's not
S2: zero is because capitalism, we've made houses so expensive, so
S2: now they've got to pay rent and we want to
S2: make a profit selling food. So now they've got to buy, uh,
S2: like this $0.01 banana is now a dollar. Um, so
S2: we basically just increased the cost of running humans to
S2: the point where these machines now look compatible. But then
S2: when you try and actually replace the humans, you're just
S2: going to run into the exact same problem, which is
S2: these machines are expensive. They may not need to pay rent. Actually,
S2: technically they do need to pay rent. You need a
S2: data centre, but they may not need to eat. They
S2: may not need water, they may not need plumbing. Uh,
S2: but they need a billion, trillion dollars worth of GPUs,
S2: a ton of electricity, a ton of cooling. And I
S2: actually don't think we are going to reach a point
S2: where we can make, uh, like computational based intelligence more
S2: cheaper than the equivalent human intelligence. I think right now
S2: we're in, uh, I call it sort of the Utopia era.
S2: Even though I don't feel like we're in Utopia, I
S2: think everything is terrible. I hate what's going on with AI.
S2: It makes me miserable. But it reminds me of, um, uh,
S2: in LA, we had this thing called, uh. Well, it's
S2: actually it's quite widespread now, but we had this program
S2: called the Bird Scooter, and it was actually the very
S2: first bird scooter was in LA, and I was there
S2: for the pilot program. It's like, okay, free public transport.
S2: You pay like maybe a dollar at most. You can
S2: go anywhere, you can leave the scooter anywhere you want.
S2: And I was like, this is incredible. Like, this is
S2: going to revolutionize transport. I can go from point A
S2: to point B, I don't need to find parking. It
S2: cost me a dollar. Like this is perfect. But what
S2: was actually happening on the back end is VCs were
S2: pouring billions and billions of dollars into this system to
S2: try and, like, build out this company. And what was
S2: happening is people were getting hit by cars. They were
S2: throwing the scooters into rivers and oceans, and they were
S2: actually losing more money, uh, to running this program than
S2: they were ever going to make. And I think we're
S2: seeing the same thing start to happen with AI is
S2: it's being funded by VCs, but it's there's not really
S2: yet any certain path where it's even going to be profitable.
S2: Like they are currently operating at a loss, and they're
S2: selling us on this idea that at some point you
S2: will be able to run like a genetic AI or
S2: whatever for less than the equivalent of an employee. But
S2: I actually don't think that's going to happen. I don't
S2: think we're going to reach a point where computers are
S2: cheap enough to run, that we can replace humans with these, uh,
S2: with these AI agents. And I think the more that
S2: we start, uh, going from this idea, which is never
S2: going to work of a single AI that is just
S2: able to do everything to building out sub models, I
S2: think that's just going to it's going to compound the cost. Uh,
S2: we're just going to get more and more and more
S2: cost to the point where eventually we'll come to the
S2: realization that actually, it's cheaper to just hire humans.
S1: Okay. I like this argument. I find that interesting. I
S1: don't believe it's true. My intuition is that it's not true.
S1: And the reason for that is just the the amount
S1: that the, um, the costs are falling. And the fact
S1: that I think we're so bad at what we're doing
S1: right now with artificial intelligence. I think there's so much
S1: slack in the rope that I think we're going to
S1: end up getting, you know, 99% or whatever, some, some
S1: dramatic amount of the cost that we're paying now just
S1: will keep falling out and falling out. But I would
S1: say that's an empirical question. And we'll just have to
S1: like see how that works out. Because I do agree
S1: with you that there is a the current state is
S1: like a state of confusion because you don't know how
S1: much it's being propped up. So so I like your argument,
S1: and I definitely think the there will be an effect
S1: from what you're saying. Um, I, um, I want to
S1: I want to touch on this, this whole concept of
S1: intelligence itself in like, because I feel like, um, we
S1: were trying to define it before, uh, when we had
S1: the previous conversation, we were trying to define like what, uh,
S1: what exactly do we mean by it? So I think
S1: the way that I defined it before was, um, or
S1: at least the way that I wanted to define it now,
S1: which which I think is roughly similar, the ability to
S1: solve everyday human problems using knowledge about the world. So
S1: I think that I try to hit it from a
S1: bunch of angles, and I think that accounts for why
S1: an average knowledge worker can't be automated. Um, and I
S1: put every day in there because it's basic stuff like,
S1: should I break up with my boyfriend? Um, and keep
S1: in mind, just just the case of, like, how useful
S1: this stuff is. I think the current numbers are OpenAI
S1: or ChatGPT has like a billion users per day. So
S1: I feel like for everyday problems, It's if we're using
S1: that as a as a metric doesn't that to me
S1: indicates that it's very useful. Um, but but I guess
S1: that's a red herring because you're, you're already acknowledged that
S1: it's useful. Um, but I've got some examples of, like,
S1: things that ChatGPT is really good at. So write an
S1: essay about a book that you read. It could already
S1: write pretty good essays. Oh, and by the way, this
S1: is unrelated to the debate, but we actually blew by
S1: the Turing test because most people can't tell the difference
S1: between AI and humans at this point, because it's already.
S1: I just thought it was an interesting piece of trivia
S1: that that used to be the gold standard for AI,
S1: and now no one even cares that we passed it. Um, managing.
S2: That one was actually passed the with the very early GPT. Um,
S2: I thought I.
S1: Think it might.
S2: Have. Yeah. Because I mean, so as, like a, like
S2: putting my, uh, like my philosophy hat on instead of
S2: my scientist hat on. I have never, ever seen the
S2: Turing test as a reliable metric for anything. Because essentially,
S2: for anyone who doesn't know what that is, it is
S2: just you put a machine and you put a person, uh,
S2: like basically concealed from the person they're talking to. And
S2: the question is, can that person distinguish between a machine
S2: or a human conversation? But the bar for that is
S2: very low, because your average person will spend like 30
S2: hours arguing with a troll bot on Twitter, not realizing
S2: that that is a Python script. Um, so I've always
S2: felt that Turing test is more a test of humans
S2: lack of ability to distinguish a real human, rather than, uh,
S2: AI being successful at what it does. Now, granted, I
S2: will concede that AI is, or at least Llms are very,
S2: very good at imitating human like conversation. I will give
S2: them that. But there is a huge difference between imitating
S2: human like conversation and human intelligence, which is why I
S2: believe that people aren't, uh, that they're not seeing it
S2: passing the Turing test as this amazing feat because it's
S2: not showing that the AI can mimic human intelligence. It's
S2: showing that it can mimic human conversation, which is a
S2: very different thing.
S1: Yeah, that makes sense. I think we agree there. Um,
S1: so what about this definition that I'm using? Um, and
S1: I can't remember the exact definitions that we used before.
S1: We had a slight disagreement there. But what do you
S1: think about this? The ability to solve everyday human problems
S1: using knowledge about the world. So it's kind of general. Right.
S1: And it's general in the sense that a knowledge worker
S1: could do it. It's general in the sense that AGI
S1: would be able to do it. AC obviously would be
S1: able to do it. But I've got examples here managing
S1: someone's daily schedule, writing performance review for an employee, reading
S1: and answering emails, writing code for an application at a company. Um,
S1: by the way, I completely agreed with, um, your your
S1: differentiation between writing code and doing software engineering or software architecture.
S1: Completely agree with you there. I think that's a big
S1: thing that coding is missing out on. Um, writing a
S1: requirements document based on conversations, writing a work status update
S1: for your boss plan, a four day trip to Switzerland.
S1: So these are my examples of like everyday human problems
S1: that are nowhere near free, like automation could possibly even
S1: try to do versus, um, with AI, these are kind
S1: of like trivial. And I would argue that these are
S1: the types of things that are going to make up
S1: a human replacing AI in the workforce.
S2: I'm not sure I agree because, um, uh, the the
S2: point on, uh, I mean, it goes back to your
S2: general knowledge worker point, the question of can it replace
S2: a average worker? Um, and we've talked about that already. Uh,
S2: but I don't think that is the definition of intelligence.
S2: For me, the definition of intelligence is not the ability
S2: to perform a task that wasn't performable by previous automation.
S2: It is the ability to work with incomplete information. Like
S2: if I were to, I don't know. Let's say I
S2: build you a puzzle and I take away some of
S2: the pieces and let's say I take away five of
S2: the pieces. Um, and I give you the puzzle with the, uh,
S2: without those pieces that I took away, you could probably
S2: at some level redraw the rest of the puzzle. Right?
S2: You could use your, uh, critical thinking and your intelligence
S2: to see. Oh, okay. I don't have all the pieces here,
S2: but I can see this is clearly a photo of
S2: a waterfall. Right? It's that ability to work with. Not
S2: what we know, but what we don't know. That is
S2: my definition of intelligence. Um, because I think I think
S2: it gets a little confusing because at least in the
S2: early schooling era, they're not really teaching you intelligence. They're
S2: teaching you knowledge. Uh, the teacher will just give you
S2: a fact, and you memorize that fact, then they'll give
S2: you a test and you repeat back the fact. And
S2: that is the metric that a lot of people are
S2: using for large language models. Yeah, they're doing like bullshit, like, oh,
S2: can it do the SATs? Can it do the bar exam? Like,
S2: of course it can do the bar exam. It's a
S2: database of all the answers. It can just regurgitate the answers. Now,
S2: my definition of intelligence would be okay. What happens when
S2: we put things that, like, we ask it to do
S2: things that aren't in its database, which is admittedly very
S2: hard for large language models because, well, they have all
S2: of human knowledge pretty much in there. They have like
S2: every book, every movie, every web page. So it's very
S2: hard to find a task where the AI cannot just
S2: use basic pattern matching and be. It isn't in their
S2: data set. And this is where I say AI is
S2: not in any like any definition of the word intelligent. Um,
S2: and the way I would assess this is by product. Um,
S2: if we ask the AI to do something that it
S2: already has in its database. It's very easily going to
S2: be able to do it because it already knows the answer.
S2: But then we have a lot of problems, like we
S2: as humans have a lot of unanswered questions. We have
S2: mathematical problems that have not yet been solved. We have
S2: incomplete scientific theories. Now, uh, I like to go to
S2: Einstein for this example, because Einstein lived before computers. Uh,
S2: in his day, he could read as many books as
S2: he could read. He couldn't Google things. He couldn't control
S2: F through a book. He would have to read a
S2: book or a scientific paper. And back then I think
S2: they were published in like, actual physical journals. So he
S2: would have had to gone and got a scientific journal
S2: and read through it to learn a little bit about
S2: a thing. And with as little like, I'm not going
S2: to say Einstein wasn't knowledgeable, but with as little knowledge
S2: as he had access to, he was able to create
S2: theories like special and general relativity, which revolutionized physics. Now,
S2: in physics classes today we work with a lot of
S2: his theories. And your average like, not very bright college
S2: student can can work with Einstein's theories because we already
S2: have the complete picture. He's already come up with them.
S2: He's already given us the answers and we can now
S2: work with them. But what made Einstein so amazing is
S2: he did not have the answers. He came up with
S2: them initially. So, um, kind of my point there would be, okay,
S2: so Einstein could come up with this amazing theory with very,
S2: very limited access to information. We have these machines that have,
S2: I don't even know how much trillions and trillions and
S2: trillions and trillions of data points like every book, every
S2: scientific paper, absolutely everything is in their knowledge database. But like,
S2: what have they come up with? If I ask it
S2: for a theory that unifies classical physics and quantum physics,
S2: it just shits out some existing theories that someone else
S2: came up with, and if it had any intelligence at all,
S2: with the sheer amount of knowledge it has access to
S2: and the sheer amount of computing power it has access to,
S2: I would expect it would be able to complete at
S2: least one of those theories. Like the fact that it
S2: just has so much knowledge, yet seems to have done
S2: nothing novel whatsoever. And in my opinion says that not
S2: only does it lack intelligence, it isn't intelligent at all.
S1: Yeah. So. So this is fascinating. This is where I
S1: wanted to get to. So I think a couple of
S1: times you've made this like horrible, horrible error. And this
S1: is why. This is why it's actually a risk to
S1: regular people. So when we were talking about computer science
S1: and programming, you used yourself as an example of being
S1: able to pivot. You have been on the front page
S1: of wired. You. You're an exceptional person. Um, malware analysts
S1: are also exceptional people within computer science and within security.
S1: So very few people in security can do malware analysis.
S1: So you're already exceptional and then exceptional within that group.
S1: And then you just defined intelligence giving the example of Einstein.
S1: Einstein is one of the smartest people, arguably the smartest
S1: person I would argue Newton. Uh, but anyway, um, one
S1: of the smartest people in the entire world that's ever existed.
S1: If you put the bar there, I agree with you.
S1: And I also want to just grant you the overall
S1: point of like, where are the novel discoveries? Here's my
S1: problem with this, Marcus. Like, that is not the bar
S1: that matters. The bar that matters is what's going to
S1: change society. The bar that matters is taking normal people's
S1: level of intelligence and getting above that bar. So you have,
S1: you know, John Smith, you know, working at some job. Again,
S1: I go down the list writing an essay about the
S1: book that you read, reading and answering emails, organizing a conference.
S1: Hundreds of millions of people are being paid a full
S1: living salary to do these jobs. Not very well. That
S1: level of intelligence, which I think you and I can agree,
S1: like it's not that great. And they are not Einstein.
S1: They are not writing malware. They are. The standard here
S1: is extremely low. The amount of critical thinking needed for
S1: that is extremely low. So what I'm talking about is
S1: an AI that can replace that level of intelligence. And
S1: I just think the the definition of inventing net new
S1: things is an unbelievably high like bar for intelligence. To
S1: me it has to be can you be hit with
S1: regular everyday problems and can you solve them using your
S1: knowledge of how the world works? Because another thing, just
S1: one last thing to say here. Every single one of
S1: those mundane tasks that I just gave, They're not actually
S1: just knowledge lookup. They actually require critical thinking for every
S1: single one of them because the email is not the same.
S1: The report that they're going to write is not the same.
S1: Each each one of those, even though it's relatively simple
S1: for a human, it's impossible for pre-human technology or pre
S1: AI technology that wasn't human. So it's in my mind
S1: it's 100% intelligence because it can't be automated.
S2: I think that's an entirely different argument. Well, there's always
S2: 2 or 3 separate arguments going on here. There's first like,
S2: is it enough to replace the average worker and which
S2: you're sort of extrapolating and saying that is a definition
S2: of intelligence. But I would argue the average worker is
S2: not using their intelligence. They're working primarily with knowledge. Like
S2: if we were to take your average like John Smith
S2: office worker, and apply them to a task that was
S2: novel and did require intelligence, they would probably be able
S2: to do that because they have intelligence that is going
S2: to waste in that job, like they're being made to
S2: do essentially busy work, which is primarily knowledge based, not
S2: intelligence based. Now, I think what um, the kind of, uh,
S2: the miscommunication or the misunderstanding here is trying to attribute, uh,
S2: what requires intelligence for a human as intelligence for a
S2: large language model, because these large language models are doing
S2: pattern matching. While, sure, every email isn't the same, it's
S2: not thinking about the differences between emails. It simply just
S2: has a big database of here's all the different conversations
S2: and every conversation that has pretty much ever occurred. Like,
S2: almost every single conversation I've ever had in my life
S2: is a slight variation of a conversation that has already
S2: been had by someone else. Like there is nothing novel
S2: going on. So I would argue that that is not
S2: a sign of, uh, AI intelligence. It is a sign
S2: that you can, uh, you can almost emulate a a
S2: not fully applied intelligence with a very advanced pattern matching algorithm.
S2: And the reason?
S1: Can I jump in? Can I jump in real quick? Yeah.
S1: Sorry to interrupt. Um, I want to give you an
S1: example here. Couples therapy. So a couples therapist studies for
S1: whatever they have, a master's degree or a PhD, whatever
S1: they have. And a couple comes to them and says,
S1: you know, we're we're about to break up our marriage
S1: because of so and so problem. And they, they help
S1: them for 2 or 3 years listening to all their
S1: different problems. And then they're coaching them through all the psychology,
S1: the sociology, all the trauma stuff, whatever, whatever they're talking
S1: to them about cognitive behavior, behavioral therapy, all this stuff, Marcus,
S1: all that stuff is just knowledge. It's just knowledge. And
S1: guess what? That conversation with this couple is kind of
S1: just like the other conversation with the other couple. Like,
S1: there's not really anything net new. Can we really argue
S1: that what that marriage therapist is doing is not isn't
S1: requiring intelligence? Of course it does. Of course it does.
S1: And what I'm arguing is all this knowledge work. It
S1: does to a lesser degree, of course, and to a
S1: lesser degree than my friend who's a cardiologist, because those
S1: are like really difficult things. But when you were trying
S1: to plan a four day trip to Switzerland with a
S1: different type of family, and one has different food needs,
S1: and the weather is also different in Switzerland, it is
S1: a new it is a net new problem each time,
S1: even though the problem specifics look different than previous versions.
S2: I would say it's a net new problem to a
S2: specific human, but not humanity as a whole, because, um, but.
S1: That doesn't matter. Because that's that's what that's what we
S1: solve every day. It doesn't matter what's new to to
S1: humanity overall, day to day, we're being hit with regular
S1: everyday problems. And that that's the work that we have
S1: to do. We we can't leverage all of humanity for that.
S1: We have to solve it ourselves.
S2: Well, the I guess the problem is, are you trying
S2: to argue that AI is intelligent, or are you trying
S2: to argue that it could do the average or like
S2: any of the jobs that you've given? Because my argument.
S1: I'm arguing.
S2: Both. Okay. So I would agree with you that. Yeah. Um,
S2: a lot of therapy is just pattern matching. Uh, there
S2: are very, uh, I would use like an example like
S2: attachment theory. There's four different types of attachment style. Um,
S2: and everyone fits into one of those four groups. Um,
S2: and there's, uh, some very clearly defined rules of what
S2: tends to cause someone to become a certain attachment style.
S2: But every single situation is going to be slightly different. Um,
S2: but all of those situations map to a single rule
S2: that then maps to your specific attachment style. So essentially
S2: what a therapist is doing there is they're listening to
S2: you and they're trying to pick out like, okay, what
S2: are what are their traumas? Like how did their parents
S2: teach them? And then they're mapping that just to a rule. Now,
S2: as a human, that requires a lot more intelligence than
S2: it would require for a large language model, because the
S2: large language model has way more examples to go on.
S2: So what you're basically taking is um, uh, almost it's
S2: like a, a kind of hand in hand relationship. You can,
S2: to an extent, replace intelligence with knowledge and knowledge with intelligence.
S2: And with the large language model, you have infinitely more knowledge,
S2: which means it needs infinitely less intelligence to replace, uh,
S2: whatever job you want to say it's going to replace.
S2: But my argument is that's not a sign of AI intelligence.
S2: That is a sign of the AI's knowledge. I think
S2: in order to, uh, even demonstrate a small amount of intelligence,
S2: the bar should be a lot higher than for a human.
S2: And that's the reason why I went with the Einstein example. Because, sure,
S2: a lot of people consider Einstein to be the smartest
S2: human on earth. But he was working with a very,
S2: very small amount of information, like a very small amount
S2: of knowledge and a very small amount of data compared
S2: to an AI. So knowledge sort of acts as this
S2: sort of amplifier for intelligence. So if Einstein, with his
S2: amazing intelligence but very limited access to data, could accomplish
S2: such amazing feats, why can something with trillions and trillions
S2: and trillions of data points and books not really accomplish
S2: more than your average Joe, who's not even fully applying
S2: all the intelligence they have. They're just doing data entry
S2: or pattern matching. So my argument there is I'm not
S2: saying that current generation large language models might not be
S2: able to replace certain roles. I'm saying it is not
S2: evidence that they are intelligent. It is intelligent. It's evidence
S2: that their level of knowledge can act in place of intelligence.
S2: And this becomes important because there is a ceiling, there
S2: is a cap where you can't just shove in more
S2: and more knowledge and it just gets more and more
S2: intelligent or sorry, it's not getting intelligent. It's able to
S2: emulate intelligence more and more. At some point you hit
S2: a ceiling and we've already hit that ceiling because we
S2: have these machines that have all of this, like all
S2: of science in their database, and they can't do shit.
S2: Like they cannot complete a single scientific theory.
S1: Yeah. No, no, I love this. First of all, I
S1: just want to say I love this line that you
S1: were on. This is actually not being talked about anywhere. Um, well,
S1: I want to say in all the places that I'm
S1: looking and I'm looking a lot of different places, I
S1: love this point that you just made about how more
S1: and more knowledge is a cheat code to intelligence. It
S1: requires less intelligence. I don't think it matters. And and
S1: here's why. So. So first of all, there's, um, there's
S1: a bunch of post training stuff that gets done. And
S1: I'm not an expert on post training, but there's a
S1: bunch of stuff that you do with these models afterwards,
S1: and I believe that we are making massive inroads on that,
S1: what I call tricks. Um, basically tricking AI as a
S1: system overall to understand how to make these jumps. In
S1: other words, I think it is possible to teach AI
S1: how to go from Newtonian physics to Einsteinian theories, and
S1: I think it's possible to teach it, to generalize, to
S1: be able to do that. And there's already some evidence
S1: of this working. I had a thing in the show recently. Um, basically,
S1: researchers have been working on this one problem, um, with bacteria, bacteriophages,
S1: which are basically viruses that propagate and try to take
S1: over bacteria. And so, um, they've been struggling with this
S1: for like over a decade or maybe two decades. The
S1: absolute pinnacle researchers in this particular thing, they gave it
S1: to this new, uh, Google model, which is specifically designed,
S1: designed to do exactly what you're talking about, Marcus. To
S1: actually find net new things. And it came back and said,
S1: here's my hypothesis for why this is happening. They looked
S1: at it and they said, Holy crap. That is the answer.
S1: It's a net new answer. They went and tested it.
S1: It was 100% confirmed. And there's a bunch of more
S1: examples of this. There's companies that are doing this basically
S1: harvesting like research that dormant research, raw research and coming
S1: up with net new hypotheses. So I think that's just
S1: a matter of like, we just haven't done the work
S1: of the scaffolding of teaching the AI how to make jumps, right.
S1: Because like you said, it's been so easy to just
S1: pull from the knowledge. But I think this is a
S1: training step that we could do. And this is an
S1: empirical thing. Like it'll either work or it doesn't. And
S1: I can send you over the stuff. But the bigger
S1: point here for me is that, um, this is why
S1: this matters to me is because of humans. So if
S1: we go back to the point of like, don't worry
S1: about I if I could, you know, summarize your thing
S1: about that saying there's always going to be a place
S1: for humans. This thing is not a big deal. Um,
S1: I think you've even called it like, um, auto autocorrect.
S1: No autocomplete. Right? You call it autocomplete? I'm like, no,
S1: it is not autocomplete. It's actually doing this work. My
S1: point to you, Marcus, is my friends cardiology practice is
S1: nothing but pattern matching. Okay. Most high level work, um,
S1: is just pattern matching. This, um, couples therapist. They are
S1: also doing pattern matching. And if you look at the
S1: average job, they are also doing pattern matching like okay,
S1: it's we're taking emails. We're writing a report. You can
S1: reduce all this work that hundreds of millions of people
S1: are doing. You could reduce that to pattern matching if
S1: you want. But we're talking about hundreds of millions of jobs, Right?
S1: So if I could do that work, then it still
S1: has the impact that I'm concerned about, whatever we want
S1: to call it.
S2: I don't disagree, but then the question is like, why
S2: hasn't it? Are we are we not there yet? Um.
S2: Are we?
S1: Because it's just now getting started. It's just now getting started.
S1: Like the tech is just now getting to the companies.
S1: These companies don't even know what AI is. They don't
S1: know what it isn't. It takes time to spin it up.
S1: Like we see the experiment starting. We see thousands of
S1: companies trying to adopt it, but they're trying to figure
S1: out what it is at the same time that they're
S1: trying to adopt it. So it's just a matter of
S1: it's just a matter of one, two, three, 4 or
S1: 5 years in my opinion. And it's already happening. We
S1: already have evidence that it's happening. It's just a matter
S1: of time.
S2: I'm not sure I agree because like companies have, as
S2: you said, they've been in a rush to roll out AI.
S2: But then the question is like, if it is so
S2: close to being able to replace the average knowledge worker.
S2: Why aren't we seeing any massive replacement? Why aren't we
S2: seeing any change in the productivity of companies? Because, like,
S2: we have objective metrics for those things. Um, and one
S2: sort of tangent I'd want to go on here is
S2: a lot of industries aren't finite. Like if you think
S2: of something like, say, farming, there's only so much food
S2: we need. Uh, if we were able to automate all
S2: the farming, sure, farmers could go away. Uh, we don't.
S2: There's no value to. Hey, let's re allocate the farmers
S2: to figuring out how to make even more food. Like,
S2: once we've got enough food to feed everyone, we're good. Um,
S2: but then you have industries which is also overlap with
S2: the industries that are most invested in AI and are
S2: most trying to replace workers with AI like tech. Now,
S2: tech is not a finite industry. There is infinite software
S2: you can write. There are infinite, uh, feature improvements. There's
S2: infinite patches now, uh, like, uh, I guess the example
S2: I give is Microsoft is not a search engine company,
S2: but they felt the need to come out with a
S2: search engine. Google is not an operating system company, but
S2: they felt the need to come up with an operating system.
S2: And why? Because the more products you make, the more
S2: market share you can capture, and the more you can
S2: compete in, uh, in the global market. So from a
S2: tech company perspective, uh, let's say I can right now
S2: replace all of my software engineers with AI. Um, the
S2: illogical thing to do would be to lay off all
S2: my engineers and do the same thing that I'm already doing,
S2: but with AI, because all of the companies that are
S2: smart are just going to do more. They're going to
S2: just start breaking into all the other industries that all
S2: their competitors are in, and they're going to dominate every space.
S2: So the logical path, uh, if I could replace employees,
S2: would not be laying off your employees. It would be
S2: to just expand. It's to have your employees do more
S2: and more things, do more and more products, and capture
S2: more and more market share. Now, if that was happening,
S2: we would see an increase in GDP, because GDP is
S2: the total value of all of the goods and products
S2: produced within the country. Yet we're not seeing any any
S2: change in GDP. We're not seeing any change in productivity.
S2: All of these companies who are claiming that they're going
S2: to replace all their engineers with AI, we've not seen
S2: any change in anything. Um, so firstly, I wouldn't especially
S2: in tech, I would not be worried about employees being
S2: just replaced, uh, because, um, well, firstly, AI doesn't fully
S2: replace employee. It accelerates their productivity. So, uh, doing the
S2: same amount of productivity but with less employees is far
S2: less desirable than keeping your employees and just doing more. Um,
S2: but we've not really seen either. We haven't seen employees
S2: being replaced with AI, and we haven't seen this massive
S2: explosion in productivity from tech companies. In fact, they're doing
S2: the opposite. They're laying off their employees to focus on AI.
S2: They're like, oh, we don't even have enough money to do, uh,
S2: whatever it was. Like whatever products Google has been, they're
S2: like binning half of their products and services to go
S2: and focus on AI. Whereas if I was actually capable
S2: of what they say it's capable of, they would be
S2: going in the opposite direction. They'd be making a gaming console,
S2: they'd be making a desktop operating system better. They would
S2: be cornering like every single market. And they're not. They're
S2: actually cutting down.
S1: Yeah, yeah, yeah. Interesting point, I hear you. Um, I
S1: think the reason it's not affecting GDP yet is because
S1: it's it's not rolled out. I mean, this is just
S1: starting like last year. We weren't even talking about agents yet,
S1: which is kind of like the the way we try
S1: to get into, um, all this automation. Um, and now
S1: agents are just now starting to get serious. My, my
S1: estimate for this has always been, uh, before 29. So
S1: I my I think this is going to be like
S1: a 20, 27 thing when this AGI, um, worker replacement
S1: technology is good enough to actually start replacing workers and
S1: it could be way sooner than that. It could be
S1: this year. It could be whenever, um, that it's not
S1: until that thing gets rolled out and it starts getting
S1: implemented by the thousands, by many, many different companies that
S1: we're actually going to see a GDP bump. So I
S1: would anticipate that being in like 28, 29, 30 and
S1: into the 30s, because that's just a slow like massive
S1: ramp up. Um, and the other thing, um, I can't
S1: remember your other point. What was the other point that
S1: you made about, uh, other than the GDP.
S2: Um, that it would make sense to, uh, not lay
S2: off employees, but to increase productivity?
S1: Yeah, yeah. So the difference there, the reason that doesn't
S1: work is because, um, an employee might cost like, let's
S1: just call it $200,000 with benefits, but for Depending on
S1: the level, that could be 3 or $400,000. Well, their
S1: entire contract with the AI company might be $200,000 if
S1: they can spin up 40,000 of these employees for $200,000
S1: instead of the 500 human employees. Then I think they
S1: would start with getting rid of the the previous ones. Obviously,
S1: they should do a slow thing, but I think the
S1: natural thing they're going to do, just based on what
S1: the CFO says is, yeah, we have these 500 people
S1: making 400 grand or 300 grand. Yeah, we need to
S1: phase them out. Keep the top 1%, keep the top 10%,
S1: and they're going to be the ones, you know, helping
S1: us move into other areas. But all. Net new actual, um,
S1: software engineers are going to be this other thing which
S1: we're already paying for the subscription, which. So it's it's
S1: like no marginal or. Yeah, the marginal cost is virtually
S1: zero to add new software engineers.
S2: But I think that rests on the assumption that the, uh,
S2: the AI agent can wholly replace an engineer, which is,
S2: as I said, is something I don't believe will ever,
S2: ever happen. I think what we will see is maybe
S2: I'm not even convinced of this yet, but I think
S2: maybe we will see these AI models get to the
S2: point where they can actually accelerate the productivity of a
S2: software engineer. But at the end of the day, there
S2: is always going to be a human on the end.
S2: It's just an abstraction. It's like, uh, how when we
S2: used to write things in machine code, it was super
S2: slow scrolling ones and zeros on a punch card. And
S2: then we made assembly language, and that made things a
S2: bit more productive. And then we made a C, and
S2: then we made Python. And now you can write whole
S2: software suites with these like point and click applications. But
S2: at the end of the day there is always a
S2: human on the end of that, it is just an
S2: like an abstraction and an acceleration of a human employee.
S2: Whereas I think the assumption you're working from here is
S2: that we get to a point where we can just
S2: wholly replace the human, which, uh, for the same reason
S2: I said AI is not intelligent. I don't think we
S2: can do, because it will always lack that ability to
S2: fill in the gaps, to come up with answers when
S2: not all the pieces are there. Like, I don't know
S2: how much you've done software engineering, but a lot of
S2: it is like the client doesn't even know what they want. Like,
S2: how do you prompt ChatGPT to build code when the
S2: client isn't even sure what it is they want to build? Um,
S2: and that's always been a large part of software engineering.
S2: It's not the writing code, which, uh, admittedly it's iffy,
S2: but you can to an extent do with large language models.
S2: It's the making the design decisions. It's like, what kind
S2: of server infrastructure do you want? How do you want
S2: the applications to talk to each other? And those are
S2: not decisions that an agent can make. Their decisions that
S2: need to be made by a human and the human
S2: making the decisions doesn't even know what decisions they want
S2: to make.
S1: Yeah, but so so check this out. Um, I want
S1: to use your previous points kind of to counter that argument,
S1: making those server choices and making those customer choices, those
S1: conversations are exactly the same as the marriage therapists, okay.
S1: They're they're basing that on good fundamental principles of building
S1: good applications with servers and applications and network connectivity and
S1: authentication and security and privacy and all these different things.
S1: These are fundamental rules. These are like written down and
S1: you can debate them or whatever. But fundamentally, AI is
S1: going to be doing the same exact thing that they're
S1: doing for like writing emails and sending emails and summarizing documents. Fundamentally,
S1: we're not making a Einsteinian jump here. We're talking about
S1: teaching an AI. What a fundamental building of an application
S1: infrastructure looks like, and what good software engineering looks like.
S1: That is a knowledge base. So the fact that we
S1: haven't done that yet and we're still stuck in this
S1: vibe coding land, that's because vibe coding started about 48
S1: seconds ago in I time, right? It started just now.
S1: So the vibe coding or the the eyes are not
S1: good at doing the software engineering piece yet. But it is.
S1: It is the same. It's not any harder than the cardiologist,
S1: which is also a knowledge base, or the marriage therapist,
S1: which is also a knowledge base. It's just a matter
S1: of time before this stuff becomes more capable. Now in
S1: terms of like, well, there'll always be a human in
S1: the loop. I mean, I think the higher level you go,
S1: the more advanced you go. Like, I think there's going
S1: to be humans in the loop over agent farms that
S1: are doing things, and the human's going to be applying
S1: what I believe to be the most irreplaceable thing, which
S1: is like taste and judgment and like preference, uh, because
S1: eyes are not alive. They don't have their own opinions.
S1: And so I think that's going to be the kind
S1: of like spiritual shaping these things are going to be
S1: putting into these agent farms. But in terms of execution,
S1: of writing the code and making sure it's on a
S1: good infrastructure that's secure. That, to me is all knowledge
S1: based stuff.
S2: I would actually push back and say that I completely
S2: disagree because unlike with the past examples you've given me,
S2: tech moves insanely fast. Like we have, uh, there's probably
S2: been 15 new database, uh, frameworks that have come out
S2: since I started programming. And back in the day, it
S2: would be buying everything into an SQL database. And that
S2: was horrible for the most the majority of applications. And
S2: then we started coming out with all these new types
S2: of databases. We came out with NoSQL and all of
S2: these unstructured, um, like query based. Um, uh, I don't
S2: even want, I just want to call it like a
S2: data bin. And then we had like AWS buckets, and
S2: then we're constantly, uh, inventing new technologies. And the problem
S2: there is twofold. The first is there is no set
S2: of rules because we're inventing new technologies at a rate
S2: in which there it's it's basically it is opinions, it's decisions.
S2: And there's no hard and fast rule of if this
S2: then that for database or server design. And also uh,
S2: unlike with, say, the marriage therapist, almost everything that has
S2: ever happened to you has happened to someone else. Like,
S2: like a lot of people would like to believe that
S2: their experience is, like, unique and no one else has
S2: lived their life and no one else has their problems.
S2: But in reality, your average human is like a hundred.
S2: Other people have been through the same stuff that they
S2: have been through. Whereas with technology we're trying to build new,
S2: new applications. We're trying to build stuff that hasn't already
S2: been built. So if we would say just, uh, confined
S2: within only the software that already exists, sure. You could
S2: just use rules, but then what would be the value
S2: of that? Like what is the value of, hey, we
S2: can just build software we already have like, oh, I
S2: can build a HTTP server, why don't I just download
S2: Apache or nginx. Um, so that is a problem. And
S2: then the second thing is, uh, in order for the
S2: large language models to actually make, uh, even emulate making
S2: those decisions, they need knowledge of those frameworks, which means
S2: they have to be trained on that data. So someone
S2: has to go and they have to create massive amounts
S2: of data to feed into the large language model, because
S2: it's almost like, uh, I'd call it like a lossy
S2: compression algorithm. Uh, you can't just put like a really
S2: good paper on database design into a large language model.
S2: And now your large language model is really good at
S2: database design. You need like, thousands and millions of data
S2: points in order for it to even be mediocre at
S2: that decision making. So now we're going to have a
S2: lag where like a new technology will come out. And
S2: as a human with like actual intelligence, but a lack
S2: of knowledge, because this technology is new and I'm not
S2: an expert in it, I can read the documentation and
S2: I can understand it. I can fill in the missing pieces.
S2: I can make decisions about how that might pertain to
S2: my software. but the large language model. Now, it doesn't
S2: have any intelligence to begin with, and it doesn't have
S2: the knowledge because it's not being put out there on
S2: the internet yet for it to be trained on. So
S2: now we just have this massive lag where an actual
S2: human developer is going to be better than the AI
S2: because they're going to know more and they're going to
S2: be able to work with newer technologies better. So I
S2: would really push back on that, especially with software engineering,
S2: because a lot of fields are very static. It's like
S2: things haven't changed in like decades or millennia. Whereas technology
S2: like in the time we've been recording this video, there's
S2: probably like 15 new technologies. I'm going to have to
S2: go and learn. So I would.
S1: Disagree. I hear you there. I hear you there. Um,
S1: let me respond to that. Um, so essentially, who do
S1: you think is going to be better at learning a
S1: new tech? Uh, so so I think you're incorrect about
S1: you need to go and get, um, you know, thousands
S1: or millions of examples, uh, what people are currently doing, uh,
S1: in state of the art of building this stuff now,
S1: and I'm pretty sure you and I could just test
S1: this offline afterwards. Um, we could potentially make, like, a
S1: fake language, a fake new programming language, also using AI,
S1: and then say, rewrite this working application in this new
S1: application in this new, um, programming language, and you give
S1: it a full programming language spec and it could just
S1: port automatically over. So I think you're making my point
S1: for me. So if you think about the concept of
S1: creating a. Net new application, um, going back to human psychology, um,
S1: and going back to philosophical principles, think about this. There
S1: are only so many net new things that you can make.
S1: And they're going to be oriented. They're going to have
S1: the shape, the pothole shape of human problems. So like
S1: you're not going to make something up that is net
S1: new to a computer that it that the AI has
S1: never heard of before. You're going to make things like,
S1: oh it's going to be a game. Oh it's going
S1: to be an application where I submit this and get
S1: this back. And when you show it a new programming language,
S1: it's not going to be like, Holy crap, I've never
S1: seen that before. You've blown my mind. It's going to
S1: be like, are you kidding me? I know all programming languages,
S1: and I see what you've done here with the spec. Okay, cool. Yeah,
S1: I could use that. So if there are 25 new
S1: programming paradigms and programming languages and programming specs that come
S1: out while we're doing this, the advantage of these new
S1: software engineers, uh, things, they're they're parsing those all the time.
S1: That's just part of the agent infrastructure to constantly be
S1: re ingesting these new languages. And here's the crazy part.
S1: If it hadn't done that, then when you say just
S1: I want you to program in, uh, you know, booga
S1: booga or whatever it is, it's like, I don't know
S1: what that is. Hold on. Give me a second, okay?
S1: I just consumed everything anyone's ever said about it because
S1: I live crawled the internet about it. I read all
S1: the forums, I read the entire programming spec. I brought
S1: that in. That's now part of my knowledge. Would you
S1: like me to rewrite this in? Booga booga.
S2: I don't, uh, so I don't agree for multiple reasons.
S2: The first is that I think we're talking more about, um, uh,
S2: I don't know what actually is the correct term for it,
S2: but you have two functionalities to a large language model.
S2: You have the actual training database where they've ingested the data,
S2: they've run it through their training system, they've done their, uh,
S2: human like reinforcement learning on the data. And that's how
S2: you get the AI being able to do something. Now,
S2: on top of that, it can search the internet, it
S2: can grab some new data. And then based on its
S2: existing knowledge base, it can to an extent parse that data.
S2: But I would argue that firstly, like even something as
S2: simple as a new programming language would not be something
S2: that it could just go out, fetch the spec, and
S2: then using its existing database, be able to write in
S2: that language. I think it would have to be not
S2: just retrained, but also it would have to go through
S2: the human, uh, reinforcement learning process. And then my second
S2: point is I've actually done this, um, not even with
S2: a new programming language, but, uh, there's a niche programming
S2: language that I have to use quite a lot. And
S2: for the life of me, I cannot get any large
S2: language model to write functional code in that language, because
S2: there's just not enough data points about it. I can
S2: go on to a forum, I can grab a script
S2: and it'll work. But whenever I ask the large language
S2: model to make me something in that language, it just
S2: falls flat on its face like it cannot do it.
S2: So like based on my experience and we're talking something
S2: as simple as a programming language, which can be 1
S2: to 1 mapped to an existing programming language, when we
S2: talk about like brand new database frameworks, it's not like, oh,
S2: this is just SQL, but the words are different. Like,
S2: this is an entirely new framework with entirely new, uh,
S2: design decisions and things to comprehend. Um, and if I
S2: can't even get these large language models to write me
S2: a language where it literally is a 1 to 1
S2: map of existing languages, I can't imagine it being able
S2: to just on the fly, consume, uh, say like a
S2: documentation or a press release about a new technology and
S2: then be able to immediately work with that technology.
S1: Yeah. I mean, I would agree it's not possible. Now, um,
S1: I've actually tried this as well. I mean, there's also
S1: just languages that a given model is better with, and
S1: everyone kind of knows that. So it's not as simple
S1: as just go grab the spec for the language. And
S1: now it's suddenly better because of the previous point that
S1: you made. But this is the type of thing that
S1: billions of dollars are being spent on, like, uh, cursor and,
S1: you know, um, well, I guess all the pinnacle models
S1: are obviously working on this, but the kind of the
S1: biggest challenge that everyone's trying to solve right now is
S1: how to get that context into the current working model.
S1: And that is accelerating at like a crazy amount. I
S1: don't think it requires retraining. Um, I think fine tuning
S1: is largely kind of blown up and just not a
S1: great thing. Um, I expect that to be very possible, uh,
S1: very soon. And I'm not saying, like, um, an entire
S1: new database structure, entirely new programming language. Um, but it
S1: takes time for a human to learn a programming language
S1: as well. It's not like the human just go reads
S1: the spec and now they can suddenly do it. That's
S1: a thing that also takes a human years to learn.
S1: I think AIS are going to be way faster, um,
S1: at doing that. And I think that'll speed up, um,
S1: pretty dramatically over the next. I don't know who knows
S1: how fast 1 to 3 years.
S2: Yeah. So that's kind of where I think we our
S2: core disagreement is, which is basically, I think in order
S2: to be able to do that, you need actual functional intelligence,
S2: which I do not believe a machine can ever have.
S2: So I think we can only keep building these models
S2: that they emulate intelligence on specific tasks. But I actually
S2: think it's going to take longer to build those models
S2: for these new technologies than it is for a bunch
S2: of humans to just learn about the tech. So I
S2: think we're actually just going to we're going to hit
S2: a wall where a lot of people are thinking that, um,
S2: these models are just get more and more advanced, more
S2: and more intelligent, and at some point they will reach
S2: human level intelligence. Whereas my personal belief is that we
S2: have already capped out and we're now at the stage
S2: of we will just build some models that can do
S2: this specific task. So now you're, uh, essentially now your
S2: question is, can I learn a new framework or a
S2: new language or a new technology, then understand it enough
S2: to build a submodel, build the Submodel, make sure it
S2: actually works functionally faster than someone could, just learn it
S2: and write code in it. And I, I don't see
S2: any way in which without getting to a model that
S2: has actual human level intelligence, not knowledge, actual intelligence, we
S2: would ever get to the point where it can learn
S2: that quickly without retraining.
S1: Well, keep in mind it only has to do it once, right?
S1: Because then it just becomes like an MCP that somebody
S1: can call so the whole world can use it, right?
S1: So I think that's just the scale of the the
S1: scale of the benefit of doing it this way, um,
S1: is extraordinary. The other thing is like, this is not
S1: even talk about like actual ASI, which I consider to
S1: be like true general intelligence, which I'm still a bit
S1: agnostic on. I would say I'm like 90% that it
S1: gets there eventually, but I have no idea when. But
S1: more importantly, like, I tend to like, listen, when somebody like, uh,
S1: Karpathy or, uh, Ilya or people like this, people who
S1: are like knee deep into this and are the true
S1: experts on what the frontier looks like. Um, plus a
S1: lot of the people that I follow, actually, at the
S1: labs who are not like the marketing people, not the CEO,
S1: but the actual researchers talking about how fast they're making
S1: these jumps. Um, I think this whole thing of, like,
S1: learn a new framework for programming is going to be
S1: relatively small in the difficulty, um, category. Um, but I
S1: do hear your main point. I would say that this
S1: whole thing we've been talking about here is not even
S1: my main point. My main point is that what I
S1: care about is the impact of humans, and that if,
S1: you know, cardiologists and marriage therapists and really advanced, like,
S1: highly trained people are largely doing rote knowledge and having
S1: an an interaction about rote knowledge that could be defined,
S1: that could be applied to software engineering. Okay, what if
S1: we just didn't start using brand new tech all the
S1: time to build new tech? So let's say, for example,
S1: we actually locked in on TypeScript and a certain back
S1: end technology and a certain, you know, database and a
S1: certain server structure. And then the AI got really, really
S1: good at that. And then the whole goal became, let's
S1: maximize GDP for the planet or whatever it is that
S1: that would potentially make a software engineer extremely an AI
S1: based software engineer way more productive and way lower marginal
S1: cost than hiring a human right. There's no rule that
S1: says we must adopt new technologies all the time as
S1: they come out the following day, even though that is
S1: like the way humans do it. But I just don't
S1: think we can look at the top tier of someone
S1: like a Marcus, someone like myself, someone like a malware engineer,
S1: someone like, you know, a cardiologist or whatever, and say,
S1: you know, there's parts of their job that they're going
S1: to be able to pivot because they're exceptional. There's parts
S1: of their job and I might not be able to do.
S1: I'm worried about the other 99% who are not doing
S1: anything extraordinarily exceptional. All those examples that I gave of,
S1: like regular tasks, that's what most people are being paid
S1: money to do. And that's what I can already do.
S1: It just hasn't been like systematized and like brought into
S1: companies where it's actually running as an engine, where they
S1: can start laying everyone off and hiring more of those things.
S1: So if you could replace doctors like this, if you
S1: could replace, you know, um, marriage counselors, then it's going
S1: to head knowledge work. It's going to hit it really bad.
S2: Um, I mean, I guess it's a big if, but
S2: if it could, then. Sure. Yeah. Um, but then there's
S2: there's two, two points I would make. The first is,
S2: does it all happen at once? Does it happen so
S2: fast that all of these people are just out of work? Um,
S2: and two, is that, like, do we not find a
S2: way to deal with that? Um, I would argue that
S2: there's always going to be critical thinking work in any
S2: role that cannot be automated with large language models. So
S2: there is always going to be a place for someone
S2: who has the knowledge of that domain and the critical thinking. Um,
S2: I see this a lot in security, actually. Uh, we,
S2: we try and secure medical systems. It's like, I don't
S2: know shit about medical systems. I can be like, oh,
S2: let's fire a wall it off from the internet. And
S2: it's like, oh no, that person is dead now. It
S2: needed to talk to the cloud to set their heartbeat
S2: pace or some shit. Um, so then we have medical, uh,
S2: professionals come in and they understand these technologies, they understand
S2: the patient and they understand the needs of the patient,
S2: and they, uh, they communicate them to us, and then
S2: we do the security. And I think there's always going
S2: to be an equivalent of that for anything you can
S2: do with AI. There is always going to be a
S2: need for someone who has the same knowledge as the AI,
S2: but also the, uh, the critical thinking, the intelligence to
S2: actually work out the problems. So it might be we just, uh,
S2: everyone becomes project managers, like, instead of having, uh, humans doing, like,
S2: data entry and busywork, everyone becomes a product, uh, project manager.
S2: And the AI is now the low level employee. Now, personally,
S2: I would love if that was possible. If we could
S2: take the entire working class and move everyone up a
S2: level and have all those people make, uh, good salaries
S2: like project manager salaries, and then have the AI do
S2: all the grunt work that no one wants to do.
S2: I think that would be amazing, but I also don't
S2: think it's realistic. I don't think we're actually going to
S2: get to the point where it can even do enough
S2: to get us to that level.
S1: Don't you think the AIS are going to be okay
S1: if the AI could do cardiologist work, don't you think
S1: it could be a better project manager? Nope. Isn't project management?
S1: Isn't all the things that you just described? That is
S1: knowledge that that thing of like, well, we didn't know
S1: how to secure it because it's a medical thing. That
S1: medical thing is knowledge that is captured somewhere. Like, this
S1: is all just a matter of orchestration. I mean, you
S1: could feed all this to a project manager and it
S1: would know, hey, Marcus, don't, like, set up that firewall
S1: rule because so-and-so is going to die like this. This is, um.
S1: This is not net new stuff. This goes back to
S1: your previous point of like. Well, now you're just calling
S1: up knowledge. Yeah, that's that's the whole game. The whole
S1: game is calling up knowledge.
S2: Well I meant that example, not as an example of
S2: something I couldn't do, but as an example of when
S2: we need to bring in people with one skill set
S2: to another. Um, and the same would be true for
S2: the skill set essentially just being intelligence, like critical thinking,
S2: the the parts that are missing from AI, there would
S2: exist that in every field. And you can't take someone
S2: who has, uh, like what I was trying to get
S2: at is you can't take someone who has the intelligence
S2: and the knowledge, sorry, the intelligence and the critical thinking,
S2: but not the knowledge, and then give them the knowledge
S2: of the AI and make it whole. Uh, because that
S2: person now has no ability to fact check the AI.
S2: They have no ability to work with that information because
S2: they don't know if it's true or not. They just
S2: know what this model is telling them. Which is why
S2: I say, like with the medical professionals, I don't know
S2: about how medical devices work. I could ask ChatGPT. It's
S2: going to give me a horrible answer and I'm going
S2: to end up killing someone. So we need someone who understands,
S2: who has both the knowledge in the medical field and
S2: the critical thinking and intelligence. So my argument is that, uh,
S2: these large language models will always be able to replace
S2: the knowledge. They will always be able to, to an extent,
S2: substitute knowledge, but they can never do the critical thinking.
S2: So you always need a human for that. And it
S2: can't just be any human. It's going to have to
S2: be someone who also has some of the knowledge to
S2: actually work with the model.
S1: Yeah. So here's what I think the fundamental issue is
S1: that you gave an example of like when AI is
S1: using its knowledge and its vast knowledge, it's not actually
S1: being intelligent. Well guess what? When you bring over Sarah
S1: Meier to be the medical expert to help us not
S1: do the wrong firewall rule, she is not being Einstein right?
S1: Then she's just calling on her knowledge. She is also
S1: not invoking her creativity and her massive critical thinking. She's
S1: calling a database inside her own brain. That database can
S1: be in the context that is given when this AI
S1: tries to write the firewall rule. When the AI goes
S1: to consider writing the firewall rule, it will pull from
S1: its knowledge, which includes Sarah Meyer's knowledge, which is you
S1: don't put firewall rules for egress traffic inside of medical,
S1: you know, um, high criticality systems because you might block
S1: the heartbeat monitor that that was not critical thinking that
S1: Sarah did she she called on her knowledge. So the
S1: same exact thing that how how is that how is that?
S2: Because essentially, when you're trying to bridge these two different fields,
S2: like we're trying to secure a medical system that's taking
S2: security and medical systems, there is a gap there. There
S2: is a gap where let's say we have Sarah, who
S2: knows like a lot about medical systems. And then there's
S2: me who knows a lot about cybersecurity. Well, there needs
S2: to be communication. There needs to be thought, there needs
S2: to be problem solving. And that is where the intelligence
S2: comes in. If we just sit there with our collective
S2: knowledge and we sit in a room together and we're like,
S2: I'm knowledgeable and you're knowledgeable, nothing's going to happen. We
S2: have to discuss and we have to reason and we
S2: have to think. And that is the part that large
S2: language models cannot do. They cannot reason. They cannot think.
S2: They just have the knowledge. Now you need the intelligence
S2: to apply the knowledge. And that is where we come in.
S1: So so yeah, I love this example by the way.
S1: This is great. So imagine you're in the room with Sarah.
S1: And again we're going back to the fact that intelligence
S1: is needed when you don't have the knowledge. Right. So
S1: so what's this? You're sitting in the, in the room
S1: with all the knowledge about cybersecurity. She's sitting in the
S1: room with all the knowledge about medical systems, and you're
S1: having a conversation. It turns out you're just exchanging knowledge,
S1: because the fact of what should be done is already known. Right?
S2: I disagree. Yeah. Like you're making a situation where, like,
S2: whatever it is we're trying to solve, there is already
S2: a known solution. Um, whereas if that was the case,
S2: then obviously security would be solved, right? Like, if there
S2: was a known solution to every problem, we wouldn't have jobs. Um,
S2: I actually don't know if you work in cyber security still. Um, but.
S1: So so check this out. That, that that kind of
S1: is the case, though, isn't it? Marcus doesn't. Has anyone
S1: come up with an answer, like, have you coming up
S1: with an answer for a customer where? Um, it hadn't
S1: been talked about, like basically doing that solution hadn't been
S1: talked about a million times throughout the history of cybersecurity.
S2: Yes, I personally have, um, but a lot of the
S2: time it isn't just like, it's kind of hard to
S2: explain in a way that the non-security viewers will will
S2: easily grasp. but a lot of security is known solutions.
S2: They're just not good solutions. Like, let's say, password theft.
S2: Your password gets stolen. What's the everyone's solution to that
S2: two factor authentication? You send an SMS to their phone. Um,
S2: why doesn't everything have two factor authentication? Well, because there's
S2: problems with that. Okay. Let's say we choose phone based
S2: two factor authentication. What if I lose my phone?
S1: Sure.
S2: Now I have to go down to it and I
S2: have to get my password reset. And I'm losing productivity,
S2: and it has to somehow verify my identity and know
S2: that I'm not a scammer pretending to be me. So
S2: there's a huge productivity loss there. And then what happens
S2: when I pick up my phone to to get my
S2: toufar code? And I see there's a notification from my girlfriend.
S2: So I start talking to my girlfriend and we've just
S2: now lost a bunch of productivity. And then you amplify
S2: that over an entire office space and to whatever the
S2: cost of password theft was. Um, being hacked was you
S2: probably lost more in productivity loss. So while we have
S2: existing solutions, we actually don't have good solutions. Everything is
S2: just trying to get the best possible, uh, the best
S2: possible idea for our specific scenario, which is really what
S2: cybersecurity is about. And that is the critical thinking. That's
S2: the intelligence. And I can just go, are your passwords
S2: are being stolen? Use Toofar. Okay. What? Toofar. Yeah. How
S2: do I handle password reset?
S1: But that's not what an AI is doing. Okay. So
S1: modern AIS like because I do this with risk assessment
S1: all the time, I do. Yeah. To answer your question,
S1: I'm doing this all the time for actual customers when
S1: you give the thing proper context. So if you give
S1: it the thing that look toofar doesn't work in this
S1: situation because of this, um, you don't even have to
S1: give it that. You could just say, this situation exists.
S1: This situation exists. We have this complexity over here. We
S1: have this complexity over here. What do you think the
S1: security control should be. It will say things like, well,
S1: based on your current state of your cloud infrastructure and
S1: based on the fact that you do business in France
S1: and based on the fact that tufa won't work for
S1: this and you can't actually do SMS because, uh, whatever,
S1: it's it's too easy to do, um, spoofing of, you know,
S1: cell phones. So we're going to rule that one out,
S1: especially in this scenario because of this jurisdiction whatever. And
S1: you start giving it the more context you give it,
S1: the more it's going to navigate all those special situations
S1: just the way that that you would or that I would.
S1: My argument to you is that when you look at
S1: an average pen test report and again, we can't use
S1: me or you or some other person who's been doing
S1: this forever as the example, because, yeah, maybe we can
S1: come up with a novel thing that's never been thought
S1: of before. But if you go take all the pen
S1: test reports produced last week and we go step by
S1: step through them, what kind of novelty is in there?
S1: What kind of new solutions are being proposed? Aren't they
S1: mostly saying, um, this thing was wide open. You've got
S1: a config problem here. Uh, that config should not be
S1: in that way. Oh, by the way, you should review
S1: your changes before you actually publish them. Oh, by the way,
S1: you have, you know, your credentials are open. Uh, like,
S1: it's going to be very common stuff. Like you've got
S1: to patch this stuff, you've got to patch.
S2: But who's giving it the context?
S1: The point is people are being paid to do this work.
S1: They don't have to give any context. They can just
S1: give the list of vulnerabilities. They can give the list
S1: of vulnerabilities. They can say, I was able to get
S1: in through this. Um, you have to patch that system
S1: because it's critical. And the people read it and they're like,
S1: oh yeah, yeah, it's a critical system. This is what
S1: my team has been telling me for 14 years. Everything
S1: in this report, my team has already told me a
S1: million times, I just needed to get this report because
S1: now I can justify the security spend. Like how much
S1: of novel, quote unquote novel things that very highly paid
S1: people are being paid for is actually exactly the thing
S1: that you described before, which is it's very much known.
S1: Things just applied in a specific way for a specific customer.
S1: It's not always the same. It's non-deterministic because it's a
S1: particular tester doing it. But the output looks remarkably similar,
S1: almost identical to the output of another pentester doing the
S1: same exact thing.
S2: Yeah, but now we're switching from the reports to the
S2: decision making. Because, Ali, you said about the context, like, okay,
S2: the company is operating in this jurisdiction, blah blah, blah, blah,
S2: blah blah blah, but who is giving it that context? Like,
S2: surely there has to be a person who is gathering
S2: and giving the model that context.
S1: Sure, sure. The agent, an agent is crawling all the docs.
S1: It's having a chat, interviews with people. It's doing voice
S1: calls and interviewing people. Like these are this is all
S1: just like common common stakes for like something an AI
S1: could do. It could do an interview, it could talk
S1: to you and extract information. It could read an entire wiki.
S1: It could read every doctrine on, uh, you know, Google
S1: Docs and Confluence. Like, it can go and gather that context.
S2: You see, I don't think it can. Like, I don't
S2: think it has the capability or the intelligence to gather
S2: that kind of context. I think maybe there is an
S2: argument that in the future it could, uh, but given
S2: any current generation model I have used, we're not even
S2: close to being able to gather the context. We can't
S2: even answer the questions. Like I can, I've had multiple
S2: times where I've asked the AI stuff, and because it's
S2: been trained on all of the past knowledge, which, as
S2: I mentioned, is constantly changing, it'll be like, oh, we
S2: should do password rotation. And then like we've disproven password
S2: rotation like 20 years ago. Like, why are you suggesting this? Um,
S2: and so we're right now, we're still running into the
S2: problem of, oh, the best practices have changed in the
S2: last five years. Maybe it's even the last one year
S2: and the model hasn't even updated to to account for that.
S2: And now you're saying we could have the model that
S2: can barely even keep up with the current state of
S2: the world? Now, actually, like interviewing people and like profiling
S2: my company and like the local laws and the documentations.
S2: And I think it may be in ten, 20 years
S2: is something that I could do. Uh, but there is
S2: no current generation model I've seen that even comes close
S2: to being able to do that.
S1: But but it's not a model, though, Marcus. It's like.
S1: It's not like it's not like, um, O3 or some
S1: model has has capabilities like that. It's more like we
S1: hired this, I don't know. Securus. This Securus company comes
S1: out and it's like our model Securus is the smartest
S1: model ever, and it will do a security assessment for
S1: you when you give the thing to Securus. It's actually
S1: smoke and mirrors. It's actually a whole bunch of agents
S1: sitting behind it. So it's really this really smart orchestrator
S1: in the front. It has a million different gatherers that
S1: go and answer questions that haven't been talked about. So
S1: you handed us this thing that's like, hey, we're building this, uh,
S1: you know, whatever it is, um, this special new application.
S1: And the top level agent is like. I have no
S1: idea what that is. Let me go crawl all the documentation.
S1: So it's all smoke and mirrors. Orchestration of multiple things
S1: going together. It's like, hey, what about this application? Well,
S1: it's being developed by so-and-so team. Okay, who's on that team? Okay,
S1: here's the list of people. Okay. Set up meetings and
S1: go talk to them or send them this form. Have
S1: them fill it out and bring it back to us,
S1: and we'll parse that. So it's nothing about those individual
S1: steps are difficult. What's difficult is just combining all those
S1: steps up into this overall product, so that it looks
S1: like with a customer, when you're talking to it, it's
S1: figuring it all out when behind the scenes, it's like
S1: you were saying before, it's a whole bunch of subtasks
S1: that are being doled out to this thing. And ultimately,
S1: that's just orchestration. That's just like all these little pieces
S1: being combined to produce the illusion of an overall intelligence.
S1: And my argument to you is that that capability is
S1: going to be functionally the same as the capability of
S1: you hired someone else to do it. You hire an
S1: actual person to do it. Because at the end of
S1: the day, it's going to be like the Turing test.
S1: You're going to look at two different pen test reports,
S1: and one was just you or me doing this assessment
S1: for three weeks where we had to do the interviews,
S1: and this other one launched off 780 agents and came back.
S1: And because the cost of inference is so short, you know,
S1: the whole thing cost them $13, whereas they would have
S1: paid you or me 90 grand to do it.
S2: I think that only works if you can build the
S2: system with the system, because if you're making all of
S2: these sub models to handle all of these, like niche
S2: edge cases and these like specific domains of expertise, you
S2: need people in those domains of expertise to build those systems.
S2: So now essentially all you have is rather than a
S2: bunch of pen test companies, you have the pen test company,
S2: which is every pen tester now working on building some
S2: model that can, uh, roughly emulate what a human pen
S2: tester would do. Um, but that I would not see
S2: as a job replacement. I would see as a job shift. Uh,
S2: you would essentially just end up with this horrible late
S2: stage capitalist bullshit where everyone works for a single AI company. Um,
S2: and no one has been replaced. We still need the
S2: same number of people just there, building models to automate
S2: things instead of doing the things. Um, but I don't
S2: see it as such a productivity boost that we would
S2: actually see like massive amounts of employees being lost. We
S2: would just see them going over to these, uh, these AGI,
S2: I guess we would call it companies to make these
S2: fake Agis by like, cobbling together a bunch of, uh,
S2: agents and sub models. But the amount of humans that
S2: you would need to build and maintain such a system
S2: would be like, phenomenal. It would be, uh, it would
S2: be like the biggest company on earth. So I wouldn't
S2: see that as people are going to be losing their jobs.
S2: I would see it as problematic because now you just
S2: have a monopoly that just it does everything. Um, and
S2: that's going to be horrible for working conditions. Um, but
S2: I wouldn't see that as a risk of, okay, now
S2: pen testers are obsolete. There's no more pen testers. Uh, okay.
S2: Medical professionals obsolete. There's no more medical professionals. It would
S2: just be. Everyone would just be like a medical professional
S2: working for an AI company.
S1: Yeah, but aren't these all just modules, though? So you
S1: have a module for going to collect more information when
S1: you don't know something, you have a module for summarizing
S1: that information and bringing it into the context. You have
S1: a module for um, you know. Net learning some new thing, right?
S1: I mean, let's go back to the cardiologist. Think of
S1: how much schooling that cardiologist had to have, right. And
S1: how different the different scenarios are. I would argue it's
S1: not that much more complex than, um, or less complex
S1: than pen testing. So I think what you have a
S1: module for understanding pen testing. Really well, a lot of
S1: these concepts are general enough. They are abstracted enough that
S1: when an AI understands them deeply enough, it is then
S1: using your analogy here or your concept, which I really like,
S1: it becomes knowledge. It becomes knowledge as opposed to intelligence.
S1: I really like this distinction. So you when it understands.
S1: And by the way, I would argue that we've already
S1: reached this with security. Um, it's already really, really good
S1: at cybersecurity. We have. Let me give you an example here. Okay.
S1: Let me give you an example of an extremely dynamic situation. Um,
S1: right now on hacker one, a completely automated, completely automated
S1: AI system has the As the number one spot. That
S1: means it's dealing with all sorts of random situations it's
S1: never seen before. It's doing its own research. It's finding
S1: new information on new technologies that's never seen before. It's
S1: launching all the attacks. It's actually showing proof and evidence
S1: that it can exploit it. It's actually submitting the reports
S1: and getting back the points with humans hand off. So
S1: I mean, if that's not the level of complexity that
S1: could replace a knowledge worker and let alone do a
S1: pen test, I mean, we already have all the evidence here,
S1: like the models are already good at security. And if
S1: you could just gather more knowledge to put into the context,
S1: that's all you need.
S2: But I think you have to ask yourself, like how
S2: many people were working on that model? Like, is this
S2: now a model that is just on autopilot forever that
S2: requires no human interaction whatsoever. And it just sits. And
S2: it does pen tests. Or does someone keep having to
S2: update systems behind the scenes because, uh, it kind of
S2: reminds me of that. I don't remember the name of
S2: the specific company, but there was this one company that was, uh,
S2: they were selling Vibe coding, and they went bankrupt after
S2: it was found that there was no I it was
S2: just a bunch of people in a, in an impoverished
S2: country writing the code for them. And essentially, this is
S2: all you're doing is an abstraction of this is you're
S2: having you don't know how many people are behind the
S2: scenes working the AI. Um, and I think that's kind
S2: of where the metrics get to get a bit skewed,
S2: because when we say someone was replaced with AI, we
S2: don't count how many jobs were gained by the creation
S2: of that AI. We're just like, oh, Jeff has been
S2: replaced with Pentester AI. Um, that means one person has
S2: now lost their job to AI, and what you don't
S2: see is on the back end. There's 500 people making this.
S2: I actually do anything at all. Um, so I think
S2: that's kind of the that's my issue with this argument
S2: of like the job replacement, that people being unemployed because, um,
S2: at least from what I'm seeing, the amount of work
S2: that goes into actually maintaining and running these systems is
S2: equivalent to doing it manually still. So I don't see
S2: any world in which we're going to see something as
S2: crazy as like 90% job loss. Um, and then that
S2: also assumes that everything is capped, right? That maybe pen
S2: testing is capped, like maybe there is only so much
S2: security we can do. But then there's other industries where
S2: it's like, oh, we gained some productivity. We can do
S2: more now. Um, so I would think that we would
S2: just see, uh, assuming we can get an AI model
S2: that just for the sake of argument, let's say it
S2: doesn't require any people. Like somehow we built an AI
S2: that just pilots itself, doesn't need any programmers, doesn't need
S2: any tweaks. It just works on its own. Would we
S2: not just take those systems and then focus into doing
S2: something more like, um, maybe we go in mined materials
S2: from asteroids. Maybe, uh, we start sending probes to, uh,
S2: to other planets within the galaxy. Like, there's always more
S2: work to do. Um, and the idea of, like, job
S2: replacement and job loss kind of hinges on this idea
S2: that there is finite work, there's finite productivity, um, and
S2: that a, the productivity isn't just being transferred to the
S2: AI side, and those people are just doing the same
S2: thing they were already doing. But for an AI company
S2: and that like there is just a ceiling and we
S2: hit this ceiling, it's like, okay, we don't need people anymore.
S2: Like we've done all the things we need to do. Um,
S2: and I just I don't see either.
S1: Yeah. So, so, so I think let's say it costs.
S1: Let's say it took, um, I actually don't know how
S1: big this company is that actually did the thing that
S1: I'm describing. I don't know how much, you know, human
S1: effort went into building the agents, but the thing running is,
S1: is not involving humans. It's actually the agents running that
S1: are doing the thing. Now, you have a good point
S1: of like, well, those agents start to rot, and we
S1: have to maintain the agent. Sure, but let's say that's
S1: only ten people or 100 people. The point is that
S1: is now replaceable infrastructure that the entire planet can use.
S1: So the planet can go and now run this, this
S1: company's application and pointed at targets, and it could find
S1: vulnerabilities and submit reports. Now that, um, the scaling of
S1: that infrastructure there to run that I don't think compares
S1: at all to, let's say, um, a million tests, let's
S1: say a million pen tests, a million bug bounties or
S1: pen tests or whatever. You're going to stick this thing
S1: on scaling up to how many humans are going to
S1: run that thing and how much they're going to cost,
S1: versus a million instances of this AI, especially given the
S1: I mean, look at the inference costs of AI and
S1: how they come down over like the last two years.
S1: I think we're paying like 0.01% or something of the
S1: inference cost of what we were paying to two years ago.
S1: It's something extraordinarily like that. So there's no reason to
S1: believe that that's not going to keep falling. Who knows
S1: what the bottom level is? But the marginal cost for
S1: doing a pen test using this automated system is going
S1: to be way lower than adding yet another tester. Not
S1: only that, but the testers, they're humans, they move on.
S1: They stop being testers and move on to they become
S1: managers or whatever. And so I think the scalability there's
S1: like no comparison.
S2: Um, I could like, given your assumptions, I could maybe
S2: see that, um, I don't know about, uh, obviously the
S2: first thing I would steer away from is the whole
S2: prior performance, future results thing. Like, if the costs are
S2: falling that we can't just expect, they're going to keep
S2: falling forever. Sure, sure.
S1: Um, that's why I said I know where the bottom is.
S2: In fact, we may actually see the opposite happen as
S2: we start to, uh, as, like, resources start to get constrained,
S2: we might actually see the costs start going back up
S2: because that's true. Silicon is pretty hard to get. Um,
S2: energy is um, we're kind of hitting our cap on
S2: energy right now. So we're having to build out more
S2: power plants, and those will end up getting factored into
S2: the costs of running these models. Um, but with something
S2: like that, say Pentesting, I could see in that specific example. Yeah,
S2: I think we could probably automate it to make it cheaper. Um,
S2: that's not because I think there's something fundamentally wrong with, uh,
S2: with people or the AI is fundamentally good. It's because
S2: the industry is kind of a scam. We go and
S2: we bill like $100,000 for some dude to, like, spend
S2: 15 minutes running Nessus or something. Um, so I think
S2: in a lot of those spaces we could see, uh,
S2: I guess full AI automation, but a lot of that
S2: stuff is really just it was always automatable like, you
S2: could have scripted a lot of that. It's just that
S2: no one was doing it. The real work, which is
S2: not being AI automated, is the stuff that is actually dynamic. Um,
S2: because like one of the things with bug bounties is
S2: it is very, very, uh replicatable. Like, you can reproduce it. Um,
S2: there's only a very few classes of vulnerabilities. And if
S2: you go for something like low hanging fruit, like cross-site scripting,
S2: you could write a Python script to go and find
S2: a million of those. And a lot of the older
S2: pen testers did, sorry, bug bounty testers did do that.
S2: They were also automated, just not with AI. So that
S2: is a, uh, like that is something that has always
S2: been automatable. Like, I don't think AI has really made
S2: any amazing breakthroughs there. Um, it might have maybe increased
S2: the scope of vulnerabilities that are automatable to a, I
S2: would say a very small degree, but I don't think
S2: we're going to start seeing like large language models who
S2: are like writing novel exploits. Um, I think we're just
S2: going to see stuff that like, um, was already automated,
S2: like fuzzing, Like fuzzing. I guess you could call automation like, it's, uh,
S2: you basically just the equivalent of shaking a tree and
S2: seeing what falls out. Uh, you could put slap AI
S2: on that and say, oh, look, we have AI, automated
S2: pen testing, but a lot of those things were already
S2: automated or automatable. Um, so I think I'd caution like the, uh,
S2: going from, hey, look, this thing is doing very well
S2: in an automated sense to cybersecurity is already like, automatable. Um,
S2: because I, I personally stay away from pen testing. Um,
S2: but I would say that's a very, very small portion
S2: of cybersecurity. And most of the parts of cybersecurity, uh,
S2: that like really do need doing isn't automatable. Like, it's
S2: not stuff we could just set, uh, pen test bot
S2: on and it'll it'll fix the system. It's human problems. It's, uh,
S2: psychological problems. Um, it's like actual complex problems that require
S2: critical thinking versus. Let's scan this code and look for
S2: XSS or integer overflows. Um, I think the what seems
S2: so significant there is that we just weren't doing it.
S2: Like we have had the capability to automate a lot
S2: of this for a lot of time, but we've just
S2: decided not to.
S1: Um, doesn't that doesn't that apply to hundreds of millions
S1: of jobs?
S2: Um, I don't know. That's the thing is, the issue
S2: is kind of that the people aren't doing the jobs. Um,
S2: like one thing I've noticed, uh, with vulnerability research is
S2: a lot of times what will happen is a company will, uh,
S2: they'll fix a vulnerability in their code base, like a
S2: very specific type of vulnerability, and they won't go and
S2: look for that same vulnerability in other parts of their
S2: own code base. 100% agree they will. Yeah. So they
S2: will fix one very specific instance of vulnerability. And could
S2: we make a large language model that goes and finds
S2: the rest? Sure. But they could they could have and
S2: they should have done that. And essentially all we're doing
S2: is we're making this huge, expensive model to pick up
S2: the slack. That was just we just decided we could
S2: have done it. We just decided not to.
S1: So. So yeah, this that's a perfect example of what
S1: I'm saying. So check this out. That thing I told
S1: you about that bacteriophages like for researchers, the best in
S1: the entire world. It turns out that they had a
S1: bias in their brain. They thought that, um, I'm going
S1: to mess this up because I don't know anything about
S1: the science, but there's something about a head and a
S1: tail of a bacteriophage. And like they combine in a
S1: certain way to be able to move and go spread.
S1: And so they were making this thing in their mind
S1: of like, well, it can't be this and it can't
S1: be this because heads can only go on tails the
S1: following way. So the I was like, can't you just
S1: do this instead? That's probably why it's happening. When they
S1: both looked at it, they were like, oh my God,
S1: that is so simple. So a mental human human bias
S1: caused them to miss this, which would have propelled science
S1: forward for so long? So it's exactly the same thing.
S1: So basically, did I make a discovery there? It actually
S1: just found a human error. But but at scale. So
S1: so now everyone in the in the world can use
S1: this model. And they can now do the same thing
S1: to all this hidden research that's sitting inside this, this, um, this, um,
S1: archive of, like, raw data that's just sitting there. This
S1: raw data, in my opinion, contains, hey, you know, if
S1: you put this molecule to to this thing in this
S1: part of the cell, it will actually just do this
S1: and like, it'll change aging completely. And anybody who sees
S1: that when the AI says it, they'll be like, that
S1: was obvious. That was completely obvious. So somebody like yourself
S1: could say, well, that wasn't actually intelligence. That was actually
S1: just noticing something and applying a rule. And what I'm
S1: saying is that that mentality is missing the point of
S1: the benefit of the AI, because whether it's pentesting or
S1: finding new drugs, or finding new interactions or helping couples
S1: or solving, you know, helping people get healthier heart, it
S1: doesn't matter if it's basic, if it should be basic
S1: or simple for a human. I agree it should be,
S1: but it's not so. So actually doing these complex things
S1: of pen testing and marriage counseling and cardiac cardiology, doing
S1: those things at scale as good or better than most
S1: people do them, most humans do them is still a
S1: massive boon to society.
S2: I, I agree with, uh, like what you just said
S2: in a vacuum. The issue I have, which I think
S2: we we discussed previously on text, is that in order
S2: for these systems to exist and sustain that, human knowledge
S2: has to remain right. Like, we can't set and forget
S2: this AI, and then suddenly we don't need to learn
S2: this entire subject. Right. Um, and one of the really
S2: big problems, which I've actually blogged about a lot, is
S2: the amount of data these models need in order to even, like,
S2: understand at a third grade level, a area of expertise
S2: is phenomenal. And what happens is once we start relying
S2: on the systems, we lose our own knowledge because we're
S2: now just deferring it to this system. But the system
S2: cannot sustain without this constant, uh, incoming flow of knowledge.
S2: And my big worry is that the more we automate
S2: with AI, the more of the, the more skill loss
S2: we get on our end, which then down the line
S2: impacts our ability to feed it back into the AI
S2: and keep these AIS, uh, like in tip top shape. Um,
S2: which is where we would need like an actual genuine AGI.
S2: We would need an AI that at some point it
S2: can learn for itself. It can just, um, it can
S2: teach itself new skills, which I don't think is possible.
S2: So then we run the risk of. sure in the
S2: short term. Can we, like, automate these bug bounties? Sure.
S2: Can we, uh, like, find the solution to this bacteria thing? Sure.
S2: But in like five, ten years, when we now no
S2: longer have any pen testers and, uh, no longer any
S2: biologists or whatever the word is for people who do bacteria,
S2: then what? What happens now when, like a new problem arises,
S2: our AI needs to be trained on it. It needs
S2: a model built to address that problem. And suddenly, like,
S2: no one can do the thing anymore. It's actually, uh. Yeah,
S2: it reminds me of. I don't remember the name of
S2: the movie, but there's a movie where they're, uh. I
S2: believe the aliens, they become so advanced that they forget
S2: how to do all the basics, and they have to
S2: contact humanity and be like, yo, we actually kind of
S2: forgot how to do, like, this basic shit. Can you, uh,
S2: can you help us? Can you, like, bail us out, please?
S1: Yeah, no, I agree. Yeah. It's an interesting variation of, uh, thing.
S1: I don't know if you saw the MIT paper that
S1: basically said people who are using AI and kind of
S1: relying on it, their brains actually changed. Like, you could
S1: physically see in their brains that they were less smart
S1: as a result of like. Using it as a crutch. Yeah, yeah,
S1: it was crazy. The sample size was really small. It
S1: was only 54 people, but it was the first example of, um.
S1: Like tangible difference of, like somebody who's thinking for themselves versus, uh,
S1: versus using AI. So I do take your point there. Um, well,
S1: I think we've been going for a while. Uh, any
S1: any final thoughts? I got to go get someone from Bart, but, um,
S1: any final thoughts? We could also do a second version, too. So.
S2: Yeah, I'm down to do a follow up. Yeah. No, my.
S2: I did not actually see that paper, but that that
S2: is terrifying. Like, if people are actually losing, like, critical
S2: thinking ability, that's even worse than just, like, knowledge loss.
S2: That's like full on. We're just going to end up
S2: with Idiocracy. So that is that is very concerning. Um,
S2: but yeah, like my, my primary concerns have been purely
S2: like the economic model of it. If you've got these
S2: eyes just ingesting all this data, they're essentially stealing it. Like,
S2: people don't like me using that term, but they are
S2: just taking copyrighted content. And the people who make that content,
S2: they make money from posting it from ad revenue. Um,
S2: so my belief has always been that they would just,
S2: just collapse under their own weight, they would cannibalize their
S2: source of information, and they would fail that way. But
S2: if they're also making humans dumber, that is an even
S2: more pressing problem, because then we're just I mean, we're
S2: just doomed at that point. So I'm I'm hoping that
S2: turns out to just be like, the sample size was
S2: too small, and maybe there was like some sampling bias
S2: where there's like a, a correlation between unintelligent people and
S2: just deferring their critical thinking to AI. Uh, but if
S2: that is the case that like, it's actually making intelligent
S2: people dumber. Um, um, I think we're going to have
S2: to unite to, to do something about that.
S1: Yeah. Yeah. I unfortunately, I don't think it's going to
S1: be a sample size issue. I think that's going to
S1: be reproducible. Uh, my explanation for it, though, is that
S1: there are some people who will do this. Arguably even
S1: most people will do this. They'll just like rely, rely, rely.
S1: And pretty soon they won't be able to think for themselves.
S1: And that's that's extremely troubling. But there is also some
S1: other group, um, which I hopefully will include myself in,
S1: who is going to like, train my eye to constantly
S1: badger me in, like sort of a Socratic type of
S1: way to make sure that never happens. Right? Because because
S1: I'm going to be actively like defending against this. Um,
S1: so I think people like that, they're going to accelerate,
S1: they're going to get smarter, even better at critical thinking.
S1: But that doesn't speak to the issue of like the other, um,
S1: larger percent.
S2: Yeah, I think that's going to be a very, very
S2: large percentage because it's the same. It's the same thing
S2: that makes social media work and. Yeah, and makes, uh,
S2: llms so attractive. It's the the instant gratification that I
S2: don't have to put in too much work to get
S2: the results that I want. Um, so I think that
S2: it's almost it's basically dopamine addiction to an extent. It's like, yeah,
S2: people just love the idea of I don't have to
S2: go and read a hundred books on programming. I can
S2: just make an app. So I think, uh, what's going
S2: to happen is sure there will be people like you
S2: who who do make their AI like, keep them in,
S2: like just challenge them. But I think for the overwhelming
S2: majority of the population that like instant gratification is just
S2: going to take them down that rabbit hole of I'm
S2: just going to I'm not even going to think about
S2: the question I was just asked. I'm just going to
S2: type it into ChatGPT. Yeah. Um, which now is giving
S2: me a lot of anxiety.
S1: Yeah. Well, Marcus, this has been super fun. Uh, I
S1: think the conversation was great. And, um. Yeah, we should
S1: talk about doing a follow up.
S2: Maybe just challenge them, but I think for them.
S1: And, uh, talk to you soon.
S2: Awesome. Thanks so much for having me on.
S1: All right. See you.