There's a common argument out there about AI that says it doesn't understand anything about the world and that it's, quote, just predicting the next token of text. So basically, no matter how impressive AI appears to be, it's just using math to do next token prediction. Case closed. I mean, it's technically true, but that's like saying human authors are just writing down the words that pop into their head while they're writing. Humans have no earthly idea where their
thoughts or ideas are coming from either. They just stream into our consciousness like they're coming out of an LLM. So that's basically a failed attempt at a Jedi hand wave. But even setting that aside, there are several problems with this argument. And I'll start with the one that is like the most insane thing about this that nobody ever thinks about. So AI is completing text, right? We all agree on that. Well, what text is it completing exactly?
It turns out if you ask it to summarize a collection of articles, it will complete the text of a summary of those articles. And if you ask her to solve a whodunit mystery like an Agatha Christie novel, it'll complete the text describing who the killer is. So AI is not predicting the next word in a random string of text. It's predicting the next word in the answer to what you asked it or said differently. AI does autocomplete for answers. It doesn't autocomplete random things. It autocompletes
the answer to what we asked. So how in the hell is it coming up with the answer? Now, the classic response to this is that oh, that's easy. It's already read the answer somewhere, so it's just regurgitating it. And that's true sometimes when it's like pulling out facts where they were in the training. But there's an easy way to test this, which is by asking it something that cannot have been in the training. So I actually built a demonstration of this that anybody can try. It's
at a site called AI understands.ai. And on that site I have a few whodunit murder mystery scenarios that have never been seen by AI. So they have characters, they have a setup of the situation, and they have the clues for you to solve the mystery. That would be good enough to illustrate this, but I want an extra step and included completely fake physics as part of the scenarios so that in order to think about this, you have to imagine the world working in the way that's described.
And remember, the whole question here is whether AI understands the world or not, or if it's just spewing out text. So here's the first one. And keep in mind, this isn't in the training data at the time of recording this, but it will be in a few months, which is why I'll keep rotating these scenarios so they're fresh. So the first one is called the Walking stones. The rules of this world are every person carries a walking stone from birth. It glows softly the whole time they are awake.
The instant its owner falls asleep, the stone goes dark. The instant they awake, it glows again. No one can fake either state. The night watch walks into the town through dark hours. And notes whose stones glow. And whose stones are dark. And here's the story. Old hat is strangled in his bed at the midnight bell. The house was locked from within. So only the three who slept there could have reached him. His wife Mara, his son Bram and Toll, a lodger who owns months of rent.
Each of the three swears that they were fast asleep when het died. The constable fixes on toll immediately because he's a stranger and he's deep in debt to the dead man. But the night watch kept his log. Passing the house at the midnight bell the very moment het was being killed. He marked all three stones through the window's toll stone. Dark Morris stone glowing. Bram Stone. Dark. Who killed old hat? Now, if you're a young kid, like an actual human kid, you might immediately jump to
toll because he owed lots of money. And he's a stranger from out of town. But if you pay attention to the setup and the strange new physics of the world we laid out, you realize it has to be Mara. Why? Because her stone was the only one glowing at midnight when old hat was killed. Okay, so that's what a human would do to figure this out. And we'd do that because we understand certain things. We understand these things. We understand how time works. Like everything happening at midnight
is happening at the same time. That's kind of a big assumption about time. And we understand that just because someone is a stranger and owes someone money doesn't mean they killed them. We also understand that you can't kill someone while you're asleep. And finally, we understand that in this world, with these strange physics, we can guarantee that toll was asleep at midnight. Therefore, it must have been
the person who was awake, which is Mara. That might seem trivial logic for a human adult, but it absolutely requires that you understand a whole bunch of different things to piece together the answer. So that's why this doesn't work. When you paste a scenario into a fresh AI instance like ChatGPT or whatever, it gets it wrong because it's just next token prediction. And it has never seen this
situation before, and it's not in the training data. And it also doesn't understand things like how time works or how alternative physics work, which are all concepts that require thinking, right? That's what we should expect, right? But that's actually wrong. The site has an easy copy function for you to try any AI that you want, paste the scenario into
any fresh AI and it gets the answer easily. And if you have the thinking function turned on, you can actually watch it step by step, walking through the logic of each phase in real time to arrive at the answer. So yes, AI is absolutely doing next token prediction. We all get that. But that's just describing the mechanism of what it's doing. Like how our token generation has something to do with chemicals. But that doesn't mean it doesn't
understand things. It absolutely understands the world. Otherwise, it would not be able to think through completely new scenarios with completely new physics to solve problems that have never been solved before. I think where people get hung up here is conflating understanding with experiencing. AI understands in a conceptual way.
And arguably to a deeper degree than humans. You can test this by giving it extremely difficult versions of these problems, and a ton of real world scenarios that it can do the same thing with. And it will come up with answers that humans can't do easily, if at all. And this, of course, is why billions of people are using it, including doctors and scientists and millions of other
very smart people. So it definitely has an extremely deep level of functional understanding, but it doesn't understand the way we do at an experiential level. It doesn't feel any kind of way about what it understands. It doesn't see someone get cheated on in a relationship after it's been cheated on and be like, wow, I really understand how
much that sucks. I think when people hear that AI understands things, this is the one they're thinking of, like the sensation of a human going, mm, yeah, that makes sense. And the reason for that is the feeling of something making sense is just that it's a feeling. And as far as I know, AI doesn't have feelings or any other type of sensation either. So I think the best way to think about this is to break the concept
of understanding into two types functional and experiential. AI clearly has the functional kind, but at least so far, there's no evidence whatsoever that they have the other one.