Why Every Brain Metaphor in History Has Been Wrong [SPECIAL EDITION]

Carl grew up to be Professor Carl Fristen, one of the most cited neuroscientists alive. He's been on this channel before more times than I can count, and that childhood observation about wood lice It never left him. He spent decades developing what he calls the free energy principle. which tries to explain all of behavior with one equation. Perception, action, learning, why you scratch your nose, all of it, Friston claims,

comes down to minimizing a single mathematical quantity. There's an old physics joke. Assume that we can model a spherical cowl in a vacuum. The joke is about how scientists grotesquely simplify messy reality to tame it. The free energy principle might be the ultimate sphere of the. cow. It promises to explain self-organization, this bewilderingly complicated phenomenon, with something so emaciated we might as well call it tautological.

Even Priston himself agrees with this by the way. This is what he said to us last time we spoke with him.

Her book, The Brain Abstracted, is basically about what happens when neuroscientists simplify brains to study them. What gets captured? What gets lost.

If those simplifications sort of allow you to achieve your technological goals, there's no in principle problem with oversimplification if you're gonna say, I'm not just interested in nature for its own sake, I just want applied science.

So here's her starting point. Scientists have to simplify. We're limited creatures trying to wrap our heads around systems way more complex than we can actually comprehend. Our working memory holds maybe seven items. Our attention is more scattered than a group of toddlers with iPads. Um we die after eighty years if we're lucky.

So we build models, right? We leave stuff out on purpose. We tell ourselves stories about how the world works. But the question is, why does any of this even work at all?

And in the blue corner, Ignorantio. He thinks we simplify because we're too dumb to do otherwise. Our models work well enough for our purposes, but they're approximations, just useful fictions, if you like. The map, not the territory. Now both of them agree that scientists need to simplify, but where they disagree is what that means about reality. Simplicius had history on his side.

or at least a certain type of history. Galileo, Newton, Einstein, they all believed pretty explicitly that nature was fundamentally orderly and that finding simple laws meant you'd found something true. Einstein famously said, God doesn't play dice and no, he didn't actually think God had anything to do with it, but he was expressing faith that the universe is at the very bottom

Legible. Now Chiramuta has gone all in on Ignorantio's position. She thinks successful science tells us we've become good at building useful simplifications, and that doesn't prove that nature is simple. The philosopher Nicholas of Cusa had a phrase for this attitude doctor ignorantia basically learned ignorance. You study hard, you learn a lot, and what you learn includes

What you don't know. Now when we interviewed Chirimut, she'd been following Francois Cholet's videos, and for those of you who don't know, Francois is a friend of the channel, he's our mascot, he's one of my heroes, and um he's got this idea called the kaleidoscope hypothesis, which is basically that

The universe is made out of code and underneath all of the apparent gnarly mess that we see there is intrinsic underlying structure.

Of complex patterns. You know, it's it's beautiful. The kaleidoscope hypothesis is this idea that the world in general and any domain in particular, follows the same structure that it appears on the surface to be uh extremely rich and complex and uh infinitely novel with every passing moment, but in reality it is made from the repetition and composition of just a few atoms of meaning.

A big part of intelligence is the process of mining your experience of the world to identify bits that are repeated. um and to extract them, extract these unique atoms of meaning. Uh when we extract them we call them abstractions.

Now the biggest simplification in the 21st century, the final boss of simplifications, is this idea that the mind is a computer or that the mind is running a software program. So we have inputs, we have processing, we have an output. This metaphor has become so established in the collective zeitgeist that no one even questions it anymore. It barely even registers in our brains as a metaphor. So is it or isn't it?

Isn't it a little bit weird that computation is this abstract formalism, like, you know, an an automata that makes these state transitions, something completely non physical, and we're describing the mind as if it is that abstract thing.

That sounds a little bit weird. There are many movies made about this who talk about uploading their minds into the matrix. Neuralink talks about interfacing with your brain's software. Yosha Bach thinks that consciousness is a software program running on your brain.

But you have to look at these abstract patterns to make sense of them. Every other explanation is going to be more complicated in the same way as money is going to be impossibly complicated if you try to reduce it to atoms. So you have to look at these causal invariances and spirits are actually such causal invariances. They are actually disembodied, right? They they're not bodies. They're not stuff in space.

And they're not mechanisms in the same way, but they're causal mechanisms, abstract mechanisms. And so we put the spirit back into nature. Using the concept of software. A lot of people think that's metaphorical, but I don't think it's metaphorical at all. It's the literal truth. Software is spirit.

And it's not going to have any information about the nature of the CPU that it's on, the color of the casing of the computer, the voltage that the computer is running on, the place that the computer is standing in in the parent universe, but our universe. So the um computer is insulating this world of Minecraft from our world. It makes it possible that an arbitrary world is happening inside of this box.

And our brain is also such a causal insulator. It's possible for us to have thoughts that are independent of what happens around us. Right? We can envision a future that is not much tainted by the present. we can remember a past that is independent from the present in which we are. And that's necessary for us. Our compu uh brain has evolved as such a causal insulator as well to allow us to

give rise to universes that are different from this one. For instance, future worlds. So we can plan for being in them.

And that pattern has causal power because it controls whatever substrate it's running on. The same algorithm produces the same effects regardless of what physical stuff implements it. So the invariance. That sameness across substrates is the causal mechanism, the pattern itself, at least according to Yosha.

He even accepts that physics is causally closed. He says that the abstract description and the physical description are two ways of looking at the same causal structure. Neither is reducible to the other. Both are real.

But I'm pretty sure Chiramuta would ask who identifies that invariance? When we say the same algorithm runs on different chips. Completely different things are actually physically happening, right? Different voltages, different electrons doing different things. The sameness is something that we impose. It exists in our description, not in nature.

And as for the money example, money only works because of human interpretive practices, right? If you take away the humans and their agreements, it's just paper, right? Money is just paper. And the causal power is actually in the social substrate that participates in it. Now I think Yosha has taken a useful way of talking about complex systems and promoted it to metaphysics.

And that's simplicious all over again, right? Mistaking the elegance of our descriptions for the structure of reality itself. I mean maybe information really is more fundamental than matter, but that's another philosophical wager. And we've made these bets many, many times before. Just look at the history of all of this, so Descartes thought that the nervous system worked like the Automata in French Royal Gardens, fluids pumping through tubes, pushing levers.

That was the high-tech metaphor of his day. Later, when scientists figured out that nerves carry electrical signals, the brain became a telegraph network. Then it was a telephone switchboard, signals traveling down wires. operators routing calls. And now in our era, the brain is a computer.

think about it as non physical because it is a thing that is not a thing, which is uh the same as temperature. You wake up, you look at your phone and you see the temperature and you decide how you're gonna dress and nobody has any doubt that temperature is something that can be measured.

But it took about like two thousand years for us, you know, as a species to figure out, you know, what temperature was and the fact that it could be measured. And there were two fundamental difficulties that I would say made it difficult for us to understand you know uh temperature. The first one is that first people thought that hot and cold were two separate things. Okay, so that temperature was like a mixture of the two, like when you make green out of blue and yellow.

The other thing that was very difficult to understand is that people thought that temperature was a thing. Was some sort of fluid that grabbed onto things. So let's say if you had a steel uh rod that is hot is that steel rod kind of like has this sort of invisible fluid that is heat

And they had good reasons to believe that it was an invisible fluid because it could flow, let's say, you could connect that rod to something that was cold and that cold thing was gonna warm up because that fluid was gonna be flowing in that direction and so forth. So they thought that it had a physicality as a thing.

A brilliant Englishman, Jowl, basically f figures out that that is not the case, that, you know, temperature is not a thing. And the way that they do it is through this observation which I don't know if you know how cannons used to be built, you know? So if you just grab a piece of sheet metal and you make it into a cylinder and you try to make a cannon out of that

The moment exactly that you that you shoot the cannon, that's gonna open up like a flower in a cartoon, you know, like like you know, like a Looney Tunes type of situation. So what they would do is they would make this solid, you know uh cylinders of metal and they would bore a hole in it, you know, uh to create the cannons. And boring those holes released an enormous amount of heat. So Jal thought, well,

How come all of that heat is there? It's like an infinite amount of heat. If I continue to bore a hole in a piece of metal for an infinite amount of time, I'm not gonna it cannot be a thing then. And that, you know, leads him to realize that Temperature is actually something that has to live in things.

But it's not a thing itself is related to the kinetic energy of the particles in the thing. But it's not a thing itself. It doesn't have its own particle. There isn't kinda like a temperature particle. Temperature is kind of like a property that matter has and that holds on to things. Knowledge is similar. You know, in that it holds on to you and to me, you know, and and and to the collective to exist. But it doesn't have kind of like a physicality in itself.

But it always exists in some sort of physical medium or substrate. So in that sense, it's always gonna be physical, no matter how virtual it gets. It has maybe a different type of physicality, but even electromagnetic waves that are transmitting, you know, data uh from your wifi router to your laptop are technically a physical embodiment.

Reality in itself, call it system. Description of reality as we perceive it, enjoy it, conceptualize it, live through, model of the system. Ontology to me is the ontology of the model, it's not the metaphysics of the system. I hope I haven't uh no made a complete mess here, okay? So metaphysics, no menum system, whatever the source of the data that we get, fantastic. The data don't speak about the source. The music of the radio is not about the radio. But there is a radio.

Of course the music is what we perceive. The music has its own ontology, structure, etc. The models. The model is at that point what we enjoy. Why digit the digital revolution has changed the the nature of the world around us, not metaphysically, but ontologically, so the reontologizing. Because some of the things that we have inherited from modernity, a sense of the world that Is now being restructured and a certain understanding of the world, so re-epistemologizing as well, of that world.

We go back to this temptation of talking about reality as if it were something that we need to grasp and portray, uh hook, uh spears. Um when in fact uh the the the way I prefer to uh understand it is as malleable, understandable in a variety of ways, um, something that provides constraints. It doesn't mean that you can interpret in any possible way.

but leaves room for different kind of interpretations. So if the flaw of data that comes from whatever is out there, uh and again I'd rather be sort of uh agnostic about it, can be modelled in a variety of ways. Um one way is to especially twenty first century, given the technology we have, etcetera, to interpret that as no an enormous computational kind of uh environment. It's perfectly fine as long as we don't think that there is our right metaphysics.

is the correct ontology for the twenty first century. Now this is not relativism, because on the other hand, Different models of the same system are comparable depending on why you're developing that particular model. And let me give you a completely trivial example. Suppose you ask me whether that building is the same building. That question has no real answer because it depends on why you're asking their question. If your question is asked because you want to have directions,

I'm gonna say, oh yeah, that's the same building, sir. The same building, yeah, absolutely. No. Go there, turn left, no, traffic lights, uh. But if your question is like same function, as I know it's a completely different building. It was a school, now it's a hospital. Next question. So is it or is it not the same?

You're doomed man. I mean there is no way you can play any Oh, I change every plank that you're gonna pay that tax. It's the same ship. I don't care. But if it is a collector, Their ship is worth zero. You change all the planks, you must be joking. It's worthless. So is it or is it not the same?

depends on why you're asking that particular question. Tell me why, and I can give you the answer. No why, in other words no frame within which we have chosen the interface that provides the model of the system No potential answer. So the question is like: is the universe a computational, gigantic yes or no, meaningless? modeling the universe as a gigantia for the purpose of making sense of our digital life. Oh yes, definitely. Because we are informational organisms. Aha, so metaphysics.

No, I meant in the 21st century the best way of understanding human beings today is as information organisms. Last century we thought that biologically not made much more sense. A lot of water and sprinkle a little bit of that extra. And so on. Mechanism, big art time, etcetera. Not Absolute answers, not relativistic answers, but relational answers, the relation between the question, the purpose, and the actual answer. But it takes three, not two.

but they were using it as a kind of functional description. But somewhere along the way, the metaphor hardened. A lot of neuroscientists today don't say that the brain is like a computer. They say it is one and the metaphor became the thing itself. Now Chiramuta, uh borrowing from Whitehead, by the way, she said that this is the fallacy of misplaced concreteness.

Uh this is another one of those leaky abstractions I was talking about. By the way, there's a great book called um The Brain Abstracted by uh Marvitz Chirimuto I I interviewed recently. And she said that one of the most pervasive myths in neuroscience is that we use these leaky abstractions and idealizations to talk about cognition. And usually it it's using the the most recent technology at the time. So, you know, a few hundred years ago we were describing the brain in terms of pulleys and

Don't be ridiculous. Why that was the m you know, i at some point in the Middle Ages it became humors, right? Because fluid dynamics was like the you know, w was the kind of technol you know, the technology that was like the most advanced or uh technology that took advantage of Of water power was like the most advanced technology that we had. Now the most advanced technology is computers. So duh, that's exactly how the brain works.

I feel that there's been more interesting stuff happening in the world of software development in the last six months than there has been in the previous twenty years. This this technology is genuinely amazing. But it is automation technology. It's it's not really intelligence, which means it it's only really as good as your ability to specify and supervise and delegate to the system. But it is absolutely amazing. But why do we have this view?

which are very congenial to thinking that whatever is going on in animals like us, in terms of the processes which lead to cognition, they're just mechanisms anyway. So why couldn't you put them into an actual machine and have that actual machine do what we do.

So with all that all of that mechanistic history in the background, AI could seem very inevitable, but if that mechanistic hypothesis is is actually wrong, then these claims for the inevitability of a biological like AI would not actually be well founded, but we could be subject to a kind of cultural historical illusion that this is just going to happen.

Now another thing that um Marvitsa has inspired me to think about a lot is the difference between prediction and understanding. Indeed when I interviewed the Nobel Prize winner John Jumper at Google Deep Mind a couple of months ago Um this was the question I asked and he had quite an interesting way of distinguishing those two things. It's almost like it's at any point learning how to refine and optimize the structure.

Control is I want to measure this thing in the future and I want it to come out 17. Right? That's control. Understand is a lot like predict, except there's a human in the loop. Understand means that I have such a small collection of facts. That you will predict and you will do it with facts that I can communicate to another human Um in kind of this compact fix fits on an index card. That's almost understand. And so I think these machines led us to

As predictive models of neuronal responses which don't have that mathematical legibility that originally, so when I was trained in the field, that people aspired to have. And so you have this um m possible conflicts, you can either pursue that goal of understanding or you can pursue the goal of prediction, but it seems like you can't have both at the same time.

So LLMs are getting unreasonably good. They are winning math Olympiads. They are I mean, as of last week actually, GPT five point two apparently um discovered a new theoret well, it's so it solved one of these problems that Terence Tao had on on his website. This is insane. But does it actually understand anything? And does it matter if it does or doesn't, as long as it works?

Chomsky had an amazing commentary on this a few years ago when we spoke, and I think it's still as relevant today as it was then.

anything goes. Okay? That includes all the laws of nature. The ones we know. The ones we do not know yet. Everything. What's the problem? The problem is they're not going to accept the paper because when you have a theory, There are two kinds of questions you have to ask me. Why are things this way? Why are things not that way? If you don't get the second question, you've done nothing. GPT three has done nothing.

I spoke with philosopher Anna Tunica uh about this recently and she had a beautiful way of describing it.

the physical instantiation, the stuff that the mountain is made out of. So Mike's view is that if something produces intelligent outputs, why does the substrate matter? Silicon, neurons, it doesn't make any difference. It's all information processing. Needless to say, he pushed back hard.

Helicopter can climb a mountain much better than you, does not have the ability to reason and abstractly and plan and predict things at all.

So individual minds are limited. But what about collective minds? What about humanity as a whole? We've built this incredible thing over centuries, right? Libraries, universities, Wikipedia, an expanding store of knowledge that no single person could ever hold. Doesn't that escape our individual limitations? So there's this dream of universal knowledge, accessible anywhere, perspective free.

only can go to work when it's embodied. You cannot throw like, you know, a bunch of engineering manuals and cement into a gorge and expect to get a bridge. Because the books don't have knowledge. Teams have knowledge, organizations have knowledge.

a universal repository of knowledge. But what this perspectivalist position actually sort of points us to is actually knowledge is inherently of a place, of a community, we acquire knowledge not by s being like completely open minded to everything that's possible to know. But actually by sort of narrowing our view, discounting possibilities actually is what allows you to pursue a line of inquiry and actually pin down

Um, some information about, say, the natural world which is humanly achievable. So the contrast I'm trying to make here is between a view which says that. Knowledge is perspectival. It's inherently from a human point of view, which means that it's inherently finite. We cannot aspire to this sort of universal free-floating knowledge.

Because as finite human beings, we can only achieve knowledge of the world through recognizing our limitations. And this notion of like you can have non perspectival knowledge like everything in the internet. Based on like all of the different possible perspectives, all blended together, that this somehow gives us a God's eye view. LLMs aspire to be this.

like every person voice, but it's precisely because they don't have a particular so s socialization into a finite community that they're not reliable, that we can't pin them down to actually um what would be a sort of honest, trustworthy perspective.

ultimately change them in order for us to acquire knowledge of them. So you cannot discount the fact that we're kind of meddling th with things in the process of um bringing about our our knowledge.

Minimize free energy. Everything else supposedly follows from that. Now Simplicius loves this. I mean finally the simple truth, the one principle to explain it all. The Ignorantio says Wait a minute. The math is elegant, the framework is unified, but does that mean it's captured what brains actually are?

Or did we just build another beautiful simplification and started forgetting that it was a simplification? So Chiramuta said to me that we should ask different questions, right? Not is this true, but What does this help us do? What does this light up? What does it leave in the darkness? And the other thing, of course, is that we are finite biological creatures, right? We there are limits to our cognition.

And Chomsky spoke about this fascinating concept of a cognitive horizon when we when we chatted with him.

Well, I suspect there's reasons to suppose we're like rats. We have capacities, we have a nature, we have a structure, they yield all sorts of uh extensive range of things that we can do, but they probably impose limits. Then I think we could even make some guess about what these limits are.

But simplification has risks, right? You forget your model is a model, you mistake elegance for truth, and you think you found solid ground when really you're just building another floor. So look at Opus four point five, right? Foundation models today, they are artifacts of staggering complexity. We've trained them on everything humans have ever written. We treat their outputs like they came from somewhere authoritative, somewhere outside of us, somewhere that knows.

But the knowing was ours all along, right? Just compressed, refracted, reflected back to us from the silicon. Whether that reflection captures the actual thing, that is a question that we're barely starting to ask.

You can use powerful frameworks like the free energy principle, but just remember they're frameworks, right? They're tools for building. They're not the final word. So the brain is not a hydraulic pump. It's not a computer. It's not a telephone network. It's probably not a free energy minimizer either. I mean at least not in some like literal way. What the brain actually is, we will only ever cap glimpses of, right? That is through our limited instruments and theories.

And that's okay, because that's what it means to be finite. So Chiramutus, you had this amazing example from uh Greek mythology, uh called Proteus, right? And if you could pin him down, he'd have to answer your question correctly. But if you let go and you let him get away, then he would shape shift and shape shift. Nature is like that, right? You can pin it down, you can ask questions, but it's always perspectival. As soon as you let go, there's always a myriad of other perspectives.

that can be interpreted from reality. Carl Friston's woodlice, they were doing something very similar, right? So slow down in the sun, move faster in the shade. But Fristan isn't a woodlouse, and neither are you.