S1: Andrej Karpathy came on Dwarkesh. S podcast recently and I
S1: have a number of thoughts. The consensus seems to be
S1: that Karpathy thinks AGI is ten years away and therefore
S1: Gary Marcus is right. And people like myself and Sholto
S1: and all the other people saying AGI is within a
S1: few years have just basically lost the war. It's a
S1: compelling narrative, but that's not really what happened. He did, however,
S1: say that he thinks AGI is ten years out. But
S1: the AGI debate has always hinged on definitions, and I
S1: think the one that Karpathy is using is the reason
S1: he's wrong. It came from back when he was at OpenAI,
S1: and it basically goes like this. An AI that can
S1: do any economically valuable work as good or better than
S1: a human. And again, that goes all the way back, like,
S1: I don't know, whenever Karpathy was at OpenAI. This is
S1: over five years ago. I simply don't think this is
S1: the best definition to use. I quite like it as
S1: a pure definition or as a computer science definition, but
S1: I think we should use one that focuses more on
S1: practically and directly helping humans and avoiding bad outcomes for humans,
S1: as opposed to talking about what's interesting and valuable to
S1: AI people like us. I'm worried about human worker replacement,
S1: specifically human knowledge work, and that's why I've been using
S1: this definition since 2023. And Dwarkesh is now using this
S1: definition as well, which is an AI system that can
S1: replace an average knowledge worker. For me, this is a
S1: better definition for two reasons. One, it focuses on the
S1: fact that it's an AI system and not one particular
S1: component of a system like a model. Two it provides
S1: a more direct benchmark for the thing we care about,
S1: which is our companies actually replacing workers with the system.
S1: Yes or no. And this system part is extremely key.
S1: I have no reason or even ability to disagree with
S1: Karpathy on the limitations of pure llms. He recently wrote
S1: yet another LLM from scratch by hand A thousand lines
S1: of code. He is the actual sensei here. Like I
S1: know 0.0017% of what he knows about Llms. The problem is,
S1: AI systems aren't just the llms themselves, they're not naked
S1: neural nets. When you go to ChatGPT and you're talking
S1: with GPT five, you're not talking to a base neural net,
S1: you're talking to an AI system. You're talking to the
S1: result of that initial LLM being shaped and molded with
S1: colossal amounts of extra scaffolding and engineering to be the
S1: best possible system it can be for doing that particular task.
S1: In this case, being a chatbot or an assistant. This
S1: distinction is crucial because replacing human jobs will also be
S1: done through composite, stitched together systems that are many times
S1: more powerful than their parts. To replace a project manager
S1: or an executive assistant, the company's building human worker replacement
S1: aren't going to wait for GPT nine or Gemini 7.5
S1: to maybe solve their problems. Human worker replacement will happen
S1: through AI products and systems that work around the pure
S1: limitations of llms and of individual model intelligence like Rag.
S1: Expanding context, windows, context management, things like that. And the
S1: best example of this is actually Claude code. It's just
S1: a brilliant example. Just throwing out estimates when Cloud Code
S1: came out, which was earlier in 25, in like basically
S1: March of 25 when it launched, it was like five
S1: times better than opus, which was its best model at
S1: the time for doing coding tasks and stuff like that. Well,
S1: it's less than ten months later and it's already gotten
S1: many times better than that already. It's like a night
S1: and day difference. Yes, the models got better, but that's
S1: not what made the difference. It was constant iterative improvements,
S1: grinding towards improving how the AI talks to itself and
S1: how humans interact with the AI coordination, context management, context engineering.
S1: And just now they added skills, which takes the whole
S1: thing to like completely different tier. This is exactly the
S1: type of efficiency ratchet that will apply to human work replacement,
S1: where we don't have enough context window to read all
S1: the company's docs. Companies will have or invent systems to
S1: do that, whether or not general enough to match human flexibility.
S1: They'll just add so many great use cases and capabilities
S1: based roughly around like the agent skills thing from anthropic
S1: that they just released that we eventually won't notice because
S1: it'll cover most use cases. The part that concerns me
S1: most about the speed of progress towards AI replacing human
S1: knowledge workers is not the speed of the AI system improvement.
S1: It's also the fact that the bar is so low.
S1: A good portion of our culture's comedy is based on
S1: the utter incompetence of, like, half of our workforce. We're
S1: talking about the worst possible customer service, people bragging about
S1: how little work they do, making a sport of doing
S1: the bare minimum, showing up the bare minimum amount of time,
S1: not doing hardly any work and getting away with it
S1: and getting paid. People absolutely detesting their jobs. Even decent
S1: workers just mindlessly punch in and out a lot of
S1: the time. Mediocrity is the baseline, almost by definition. That
S1: is what multibillion dollar human worker replacement startups are competing with,
S1: not the top 10% performers that you know, a lot
S1: of us know, at least for now. Think of it
S1: this way in the time that we went from cloud
S1: code not existing to getting really, really good to now
S1: having shareable work task replacement skills, the bottom 50% of
S1: knowledge workers improved by how much? Zero in the time
S1: since ChatGPT came out. Right. So we're talking about late 22.
S1: So we're talking about what is that over three years
S1: in the time since ChatGPT came out, we're talking about
S1: a stark difference in AI before then and now, three
S1: full years go by, the bottom 50% of knowledge workers
S1: improved their capabilities. By how much? Again, 0%. The bar
S1: for human work replacement is not moving, while the capabilities
S1: of AI systems are going absolutely apeshit. Now, you might
S1: push back saying this is only for the people not
S1: trying very hard or who aren't that smart or whatever. True.
S1: But it doesn't matter. You and me and Dwarkesh and
S1: Karpathy are going to be fine. So what? I'm worried
S1: about everyone else. If AI only eats the absolute worst
S1: bottom 50% of knowledge workers in the next 5 or
S1: 10 years, we're still talking about hundreds of millions of jobs,
S1: or even 25%. So basically a bunch of I just
S1: did a bunch of research on this, and the total
S1: number of knowledge workers worldwide is right around a billion.
S1: 1 billion knowledge workers. So half is a big percentage.
S1: That's 500 million people, but let's just say it's 10%.
S1: Let's just say it's 25%. And we've already established that
S1: these are the least competent people at the job. So no,
S1: they won't be pivoting easily to another knowledge work position.
S1: This is why I disagree with Karpathy on AGI. It's
S1: not because he's wrong about Llms having severe limitations. He's not,
S1: but he's focused on the wrong thing. If the thing
S1: we care about is AI's near-term and practical impact on humanity,
S1: the thing to watch is not the pure LLM tech
S1: or the specific technical limitations of RL to achieving continuous learning.
S1: It's the trillions of dollars being invested in replacing the
S1: worst performing human workers, who will likely never get better
S1: than they already are. Those trillions are being spent on
S1: scaffolding workarounds to LLM limitations that provide us just general
S1: enough AGI to start replacing people and from there it
S1: will only improve. Given what we've seen in systems like
S1: cloud code cursor codecs that dramatically magnify model capability. While
S1: the models continue to improve along their own axis as well,
S1: do you really want to bet that good enough generality
S1: won't be hit in the next couple of years? I
S1: wouldn't take that bet. And this is why I think
S1: AGI will arrive before 2028. Like a 70% chance. A
S1: rough guess who really knows. And before 2030, I'm guessing 95%.
S1: Not because all the stuff Karpathy is talking about will
S1: be solved by then, but because it won't matter if
S1: it's solved with trillions of dollars in funding and trillions
S1: of dollars in market opportunity, we're almost guaranteed to cloud
S1: code our way past a very low bar of millions
S1: of barely there employees.