S1: There's a common argument out there about AI that says
S1: it doesn't understand anything about the world and that it's, quote,
S1: just predicting the next token of text. So basically, no
S1: matter how impressive AI appears to be, it's just using
S1: math to do next token prediction. Case closed. I mean,
S1: it's technically true, but that's like saying human authors are
S1: just writing down the words that pop into their head
S1: while they're writing. Humans have no earthly idea where their
S1: thoughts or ideas are coming from either. They just stream
S1: into our consciousness like they're coming out of an LLM.
S1: So that's basically a failed attempt at a Jedi hand wave.
S1: But even setting that aside, there are several problems with
S1: this argument. And I'll start with the one that is
S1: like the most insane thing about this that nobody ever
S1: thinks about. So AI is completing text, right? We all
S1: agree on that. Well, what text is it completing exactly?
S1: It turns out if you ask it to summarize a
S1: collection of articles, it will complete the text of a
S1: summary of those articles. And if you ask her to
S1: solve a whodunit mystery like an Agatha Christie novel, it'll
S1: complete the text describing who the killer is. So AI
S1: is not predicting the next word in a random string
S1: of text. It's predicting the next word in the answer
S1: to what you asked it or said differently. AI does
S1: autocomplete for answers. It doesn't autocomplete random things. It autocompletes
S1: the answer to what we asked. So how in the
S1: hell is it coming up with the answer? Now, the
S1: classic response to this is that oh, that's easy. It's
S1: already read the answer somewhere, so it's just regurgitating it.
S1: And that's true sometimes when it's like pulling out facts
S1: where they were in the training. But there's an easy
S1: way to test this, which is by asking it something
S1: that cannot have been in the training. So I actually
S1: built a demonstration of this that anybody can try. It's
S1: at a site called AI understands.ai. And on that site
S1: I have a few whodunit murder mystery scenarios that have
S1: never been seen by AI. So they have characters, they
S1: have a setup of the situation, and they have the
S1: clues for you to solve the mystery. That would be
S1: good enough to illustrate this, but I want an extra
S1: step and included completely fake physics as part of the
S1: scenarios so that in order to think about this, you
S1: have to imagine the world working in the way that's described.
S1: And remember, the whole question here is whether AI understands
S1: the world or not, or if it's just spewing out text.
S1: So here's the first one. And keep in mind, this
S1: isn't in the training data at the time of recording this,
S1: but it will be in a few months, which is
S1: why I'll keep rotating these scenarios so they're fresh. So
S1: the first one is called the Walking stones. The rules
S1: of this world are every person carries a walking stone
S1: from birth. It glows softly the whole time they are awake.
S1: The instant its owner falls asleep, the stone goes dark.
S1: The instant they awake, it glows again. No one can
S1: fake either state. The night watch walks into the town
S1: through dark hours. And notes whose stones glow. And whose
S1: stones are dark. And here's the story. Old hat is
S1: strangled in his bed at the midnight bell. The house
S1: was locked from within. So only the three who slept
S1: there could have reached him. His wife Mara, his son
S1: Bram and Toll, a lodger who owns months of rent.
S1: Each of the three swears that they were fast asleep
S1: when het died. The constable fixes on toll immediately because
S1: he's a stranger and he's deep in debt to the
S1: dead man. But the night watch kept his log. Passing
S1: the house at the midnight bell the very moment het
S1: was being killed. He marked all three stones through the
S1: window's toll stone. Dark Morris stone glowing. Bram Stone. Dark.
S1: Who killed old hat? Now, if you're a young kid,
S1: like an actual human kid, you might immediately jump to
S1: toll because he owed lots of money. And he's a
S1: stranger from out of town. But if you pay attention
S1: to the setup and the strange new physics of the
S1: world we laid out, you realize it has to be Mara. Why?
S1: Because her stone was the only one glowing at midnight
S1: when old hat was killed. Okay, so that's what a
S1: human would do to figure this out. And we'd do
S1: that because we understand certain things. We understand these things.
S1: We understand how time works. Like everything happening at midnight
S1: is happening at the same time. That's kind of a
S1: big assumption about time. And we understand that just because
S1: someone is a stranger and owes someone money doesn't mean
S1: they killed them. We also understand that you can't kill
S1: someone while you're asleep. And finally, we understand that in
S1: this world, with these strange physics, we can guarantee that
S1: toll was asleep at midnight. Therefore, it must have been
S1: the person who was awake, which is Mara. That might
S1: seem trivial logic for a human adult, but it absolutely
S1: requires that you understand a whole bunch of different things
S1: to piece together the answer. So that's why this doesn't work.
S1: When you paste a scenario into a fresh AI instance
S1: like ChatGPT or whatever, it gets it wrong because it's
S1: just next token prediction. And it has never seen this
S1: situation before, and it's not in the training data. And
S1: it also doesn't understand things like how time works or
S1: how alternative physics work, which are all concepts that require thinking, right?
S1: That's what we should expect, right? But that's actually wrong.
S1: The site has an easy copy function for you to
S1: try any AI that you want, paste the scenario into
S1: any fresh AI and it gets the answer easily. And
S1: if you have the thinking function turned on, you can
S1: actually watch it step by step, walking through the logic
S1: of each phase in real time to arrive at the answer.
S1: So yes, AI is absolutely doing next token prediction. We
S1: all get that. But that's just describing the mechanism of
S1: what it's doing. Like how our token generation has something
S1: to do with chemicals. But that doesn't mean it doesn't
S1: understand things. It absolutely understands the world. Otherwise, it would
S1: not be able to think through completely new scenarios with
S1: completely new physics to solve problems that have never been
S1: solved before. I think where people get hung up here
S1: is conflating understanding with experiencing. AI understands in a conceptual way.
S1: And arguably to a deeper degree than humans. You can
S1: test this by giving it extremely difficult versions of these problems,
S1: and a ton of real world scenarios that it can
S1: do the same thing with. And it will come up
S1: with answers that humans can't do easily, if at all.
S1: And this, of course, is why billions of people are
S1: using it, including doctors and scientists and millions of other
S1: very smart people. So it definitely has an extremely deep
S1: level of functional understanding, but it doesn't understand the way
S1: we do at an experiential level. It doesn't feel any
S1: kind of way about what it understands. It doesn't see
S1: someone get cheated on in a relationship after it's been
S1: cheated on and be like, wow, I really understand how
S1: much that sucks. I think when people hear that AI
S1: understands things, this is the one they're thinking of, like
S1: the sensation of a human going, mm, yeah, that makes sense.
S1: And the reason for that is the feeling of something
S1: making sense is just that it's a feeling. And as
S1: far as I know, AI doesn't have feelings or any
S1: other type of sensation either. So I think the best
S1: way to think about this is to break the concept
S1: of understanding into two types functional and experiential. AI clearly
S1: has the functional kind, but at least so far, there's
S1: no evidence whatsoever that they have the other one.