Have you ever seen those pictures of blobs on a page and it doesn't look like anything to you until you're told what it is, and then you suddenly see it. Why does that give us a great clue about the wiring of the brain. And why are neuroscientists so magnetically attracted to those visual illusions that you scroll through on social media? What is the deep trick about the way your visual system works that you were never taught in school?
And what does any of this have to do with catching a baseball or zooming down the road in New York City or the warp drive in Star Trek. Welcome to Inner Cosmos with me David Eagleman. I'm a neuroscientist and author at Stanford and in these episodes we sail deeply into our three pound universe to understand why and how our lives look the way they do.
Today.
I'm going to start with the notion of visual illusions. Elementary school students love these and they stare at them for about a minute and then they're on to the next thing, because why not. The illusion is just an interesting trick. There's nothing further to do about it. It's only later when you grow up to be a neuroscientist or a fan of a neuroscience podcast, that you might even return to one of these illusions to ask, wait, why exactly does that happen?
Does that tell us.
Something deep and fundamental about the way my consciousness constructs the world for me?
What does it reveal? So?
Have you ever seen the illusion where you're looking at lines like bicycle spokes, and then there's some straight lines drawn on top of that, and they don't look straight, they look bent. Why does that happen? Seems like it shouldn't be hard to answer, but it's actually taken well over a century to figure this out, and the answer is gonna blow your mind. I promise you that. But in order to get us there, I'm gonna start with something completely different. I'm gonna start with those pictures that
look like just a bunch of blobs. Probably you've seen one of these before. There's just a bunch of random looking splotches of black and white on a page. If your brain doesn't have a prior expectation about what it's seeing, about what the blobs mean, then you simply see black and white blobs and there's no particular meaning to the picture.
I'm gonna link an example of this on the show notes at eagleman dot com slash podcast, and I want you to stare at it for a few moments and then scroll down to the very bottom of the page for the hint. What you'll see is that you can't make heads or tails of these blobs. But then I only change one thing, and it has nothing to do with what's on the screen. I give you a hint, and as soon as you have a notion about how to interpret what is on your retinas, then you say,
oh yeah, I see it now. Now the exact same blobs that confused you a moment ago make perfect sense. But again, nothing changed out there in the world. The only thing that changed is something in your neural networks. So what's the lesson from this? There has to be a match between incoming data and your expectations for you to see anything. But wait, what, That's not how vision
is supposed to work, is it. I mean, after all, you look at any basic biology textbook and it will tell you that photons hit the retina and the information is carried on back to the visual cortex, and then you just see what's out there. The visual cortex is like a television screen, So what's going on?
Why can't you see.
The image in the blobs until you've got the right expectation.
This ties into.
A concept that you hear me refer to all the time on this podcast, and that is the concept of the internal model. Remember that your brain is isolated in soundless and lightless solitude inside your skull, and its single mission is to construct a loud, colorful mental model of the outside world. In other words, it builds an inner reality that tries to accurately reflect the outside. The key is that you don't see by capturing television pixels from
the world. Instead, all you ever see is your internal model, and your internal model only perceives some thing when its expectations are sufficiently supported by the sensory data coming in. Now, this isn't really a widely known idea. I think you'll find if you ask people about it on airplanes, as I often do. This isn't really the way that most people are used to thinking about the brain. So it's a bit surprising that the basic conceptualization of this idea
is almost seventy years old. One of the earliest examples of this framework that I know of came from the neuroscientist Donald McKay, who in nineteen fifty six said, Look, the job of the visual cortex is to construct an internal model, and it's always trying to anticipate the data coming up from the retina. Now, just as a reminder, the retina is the part of the back of your eye that captures light, and the visual cortex is all the way at the back of your head, on the
far side of the brain. But here's the surprise. The information doesn't just shoot from the retina to the visual cortex. Instead, there's a train station in between, a structure called the thalamus. The thalamus sits right in the middle. So information doesn't go straight from the eye to the visual cortex, but instead it makes a stop and changes trains halfway at the thalamus.
Okay, well that's weird. Why is there the setup?
Well, to understand this, we need to understand that the model of vision in introductory textbooks isn't just misleading, it's dead wrong. The brain isn't built on straight lines, it's built with loops. So what McKay suggested is that the
retina sends its data to the thalamus. In other words, what the eye is capturing about the world out there, and the cortex sends its predictions to the thalamus what the cortex is expecting to see next, and all that ever comes out of the thalamus back to the cortex is the difference, the difference between what you expected and
what you got. In other words, the information that goes from the thalamis to the visual cortex is just that little bit which was unanticipated, the difference between what's out there and what was already expected. The thalamus sends to the cortex only that difference signal, because that's the only part that wasn't predicted away. And then, by the way, this unpredicted information adjusts the internal model so there will
be less of a mismatch in the future. That way, the brain refines its model of the world by paying attention to its mistakes. Okay, so the idea here is that the brain is always trying to anticipate what it's seeing out there, and McKay pointed out that this is consistent with the anatomical fact that there are ten times as many fibers projecting from the visual cortex back to the thalamis as there are going from thalamis to visual cortex,
which no one would have guessed. But that's just what you'd expect if detailed predictions are going from the cortex to the thalamis, and the little signal from thalamis back to cortex is just carrying the difference.
Between what was expected and what was seen.
Okay, So why am I telling you this level of detail because it exposes a giant idea. It means that what you perceive about the world emerges from an active comparison of sensory data with your internal predictions. Again, think about those blobs. If you don't have a prediction of
what you're seeing out there, there's really nothing there. As soon as you have a close enough expectation because you've been given a hint, then that lights up a forest fire in your brain and you see the thing because there's a match.
Now. So what this.
Means is that the brain is always trying to predict everything that is coming or expected. And here's one way that the brain helps itself along. Whenever it sends out a signal to your body, like move your head or move your arm, it also sends copies of that command internally all around the brain. These are called efference copies.
So now your movement isn't just happening in the outside world and then you react to it, but there's also a simulation of that movement happening inside your internal model, so that you can predict the outcome of that action. And this, by the way, is the reason you can't tickle yourself. Other people can tickle you because they're tickling
maneuvers are not predictable to you. But you can't tickle you because your brain moves your fingers into the tickle position and it already expects the resulting sensations, that already knows what's come. Now, by the way, if you'd really like to tickle yourself, there is a way to do it, and this just involves taking predictability away from your own actions.
So what you do is you control the position of a feather with a joystick that inserts a random time delay, so when you move the joystick, at least a second passes before the feather moves accordingly, so that takes away the predictability and now you can self tickle. By the way, related to this, I described in episode forty four how people with schizophrenia can tickle themselves, and this is because of a problem with their internal timing that doesn't allow
their motor actions and resulting sensations to be correctly sequenced. Okay, so back to this issue about having a brain that's not just moving signals down a one way assembly line, but instead has all these internal loops so that it can always be feeding its internal model and guessing what's going to happen next. What is the advantage of this, Well, it allows us to transcend stimulus response behavior. In other words, we don't have to just observe the world and then
react to it. Instead, a brain with an internal model gives us the ability to make predictions ahead of actual sensory input, like predicting what your fingers will feel like in your underarm. So our brains build these predictive internal models that tell us how things are likely to go in the world. And this way our brains don't work solely from the latest sensory data, but instead they're always
guessing ahead to the next moment. Now, why do we need a complicated brain like this because our perception is massively delayed from reality. Why is it delayed because signals from the world, like something you see or a touch on your toe. These signals have to travel along nerve cells, and they move about a meter per second in the cortex, which is, by the way, about three hundred million times
slower than electricity moving through your laptop. We are giant systems of cells, and it takes time for impulses and cells to travel around. Yes, they use electricity, but it's not like a signal propagating along a wire. Instead, with a cell, you've got these long extensions called axons, and the signals travel by causing little channels to open in the membrane, which allows little charged particles to flow through and change the voltage locally, and that propagates down the axon.
So this is a very cool way that mother nature discovered how to run a signal down a cell. But it ain't fast, and the consequence is that it just takes a long time for signals to propagate through the system and eventually come together and settle into a coherent pattern. So by the time you become consciously aware of something in the outside world, the event has already happened a while ago. We live in the past. For example, clap
your hands in front of you. By the time you see and feel and hear that it's already happened a tiny little while ago. Whatever conscious movie you're seeing right now, now, that world is already gone.
Now.
We don't often think about this, but this delay from reality, the fact that we're living in the past, is a major problem because you need to operate in the present, but your brain is always working with old news. All your sensory inputs like vision and hearing and touch, these take time to travel to the brain to get processed, and finally the brain croaks out a response. And even though this delay is less than a second, that's plenty of time to create issues. So just think about trying
to catch a baseball that someone throws to you. If you were merely an assembly line device, you couldn't do it. Why because there would be a delay of hundreds of milliseconds from the time the light strikes your eyes until you could put up your glove in the right spot. And the problem is that by the time the image of the ball reaches your brain and gets processed, the ball has moved. Your hand would always be reaching for a place where the ball used to be.
So how do you catch a baseball?
It's because of these deeply hardwired internal models. Your internal model generates expectations about when and where the ball's going to hit, given momentum and gravity and so on. Your brain is not just passively processing information. It's predicting. It's not reactive, it's.
Constantly guessing ahead.
It predicts where the ball is going to be based on clues about its trajectory and speed, and that's what allows you to catch it. By the way, as a side note, these predictive internal models you have are trained up by lifelong exposure in your normal experience. If your great grandkids grow up on Mars, their internal models will get trained up with different parameters of physics, and they'll put up their glove at a different time the moment
that's right for a Martian pop fly. Okay, But the critical point I want to make here is that we have these predictive internal models, and these things tell us from experience how things are likely to move in the world. And this way our brains don't work solely from the latest sensory information, but instead they construct predictions about where the ball is going to be. The same idea is in play when you're walking through a busy airport, when you have a flow of people moving in all directions
around you. If you had access to only outdated information from photons a few hundred milliseconds ago, you'd be constantly crashing into people.
But you don't. Your brain solves this.
Your brain is constantly forecasting where the people are going to be based on their speed and direction, and that's what allows you to smoothly navigate without crashing despite the neural the and processing the visual information. So I want to summarize where we are so far. The foundation we're establishing here is that the brain is not just reacting to the world. Instead, it's a machine that continuously makes
educated guesses. Prediction is how we compensate for our signal processing delays, and from an evolutionary standpoint, this ability to predict was absolutely critical for survival because animals who wanted any chance of living how to anticipate the movements of predators or prey to react quickly enough. You have to somehow operate in real time if you want to evade
a thread or catch the running animal. So whenever you are next catching a ball or moving through the airport, think about how much you rely on your brain's predictive abilities to act without having to wait for all the signals to dribble their way in there. Okay, now we're finally ready to return to the issue that I started with. The illusions where you have some lines that are straight but they look bent. These fall into the category of geometric illusions. So what in the world do they have
to do with what we've been talking about so far. Well, what I told you is that the visual system has developed these predictive mechanisms to deal with the signal delays so that it can see something at this moment in time and make a really good guess where that thing is going to be in say, one hundred milliseconds. So some of my colleagues proposed a framework called perceiving the present, and the idea is that your brain sees what is likely to be the case, rather than to perceive the
recent past. So the first examples of this framework came out in the early nineteen nineties. So imagine you're looking at a small horizontal line on a computer screen and you're trying to judge its exact position, but there's a field of dots drifting continuously in the upward direction behind that line. In this case, you'll judge the line to be higher up on the screen. This is called motion capture.
So by the beginning of the two thousands, my colleague Mark Chengizi started proposing that the explanation for this motion capture was the perceiving the present framework, which is that your brain sees the line, and it sees the motion and decides that in the next moment the line is probably going to be pushed up by the motion. So it's actually perceiving the line in a different place where it expects the line to be in the next moment. And besides that, he argued, he could explain the classical
geometric illusion. What are these classical geometric illusions. Well, let's take what's known as the Herring illusion. You almost certainly saw this as a kid. There are a bunch of lines coming out of the center, like the spokes on a bicycle wheel. Okay, now you put two parallel lines on that bicycle wheel, let's say, a vertical line to the right of center and one to the left. You could do this by taping two pencils on the bicycle spokes.
Now here's the illusion to two pencils. Although they are straight, they don't look that way anymore. Instead, it looks like the pencils are curving, they look slightly bent. Their middles are bowing outwards slightly. So this is an illusion that was first described by the physiologist Ewald Herring in eighteen sixty one.
But why in the world does it happen?
Well, Herring proposed that this has to do with the brain overestimating angles where the lines are meeting. And then other people proposed different things in the brain that might explain that angle overestimation. But Changhizi proposed a new explanation
when which was quite stunning. He said, look, when you're looking at these radial lines, in other words, the lines like the bicycle spokes coming out from a central point, your brain might think that it's just looking at the convergence of lines to the vanishing point, like imagine you're looking straight ahead on a street in New York City and everything converges in the middle. But equally, he said, these lines are what a scene looks like to your
visual system when you are moving forward. For example, imagine that you're driving down the road in New York City and up ahead on the left there's a hot dog stand and that zips by you on your left side, and at the same time, there's a street juggler on your right.
Side, and he gets bigger and he zips by you on that.
Side, and there's an overhead street sign that at a distance starts essentially in the middle in front of you, but as you get closer and closer, it moves over your head. So everything is streaking past you like radial lines, and this is known as optic flow. So one place you've seen this before is on Star Trek, where they yank down the lever and put the ship into warp drive and all the stars suddenly shoot past them, all moving away from the center, like the radial spokes of
the bicycle wheel. So Tanghizi said, when you see radial lines like that, it's typically a visual signature of you moving forward towards the vanishing point. And certainly when you're moving fast, there's a radial smear, like the way that the stars and Star Trek smear into lines. And he said, look, Herring's radial lines essentially mimic this. It's like you are in the spaceship moving directly ahead. Now here's the key.
Let's come back to the de lays in the visual system and how they can be accounted for by the brain making projections where things are about to be. Imagine that you're in New York City and driving and there are two skyscrapers up ahead of you, one on the left and one on the right. Now, as you race forward in your car, those two buildings will loom closer.
But now something interesting is happening. The parts of the buildings closer to you will seem farther apart, because if you look up, the tips of the buildings are coming closer together, way up in the sky.
So the point is that.
Even though you see essentially straight skyscrapers when they're at a distance, as you approach, they are bending away from you. Their centers are bowing out. And Shannghizi's idea was that when you look at the radial lines the bicycle spokes, your brain thinks this might be a clue that I'm moving forward, and I don't want there to be delays in my perception, so I'm going to see the world as it will be a moment later. And so you see the two parallel lines bowed outward from the center.
In other words, when you look at the radio lines on the piece of paper, even though nothing is moving. Your brain thinks this is what movement looks like, and so it predicts the next moment, and that's what you perceived. In other words, you perceive the lines just as they would project in the next moment if you were moving forward toward the vanishing point. So, as Changhesi wrote in this paper, evolution has seen to it that geometric drawings
like this elicit in us premonitions of the near future. Okay, so the framework by Changizi and colleagues suggests that several geometric illusions are caused by temporal delays with which the visual system must cope. The idea is that the visual system extrapolates its current information to perceive the present. Instead of providing a conscious image of how the world was a few hundred milliseconds ago when the signals first struck the retina, the visual system estimates how the world is
likely to look in the next moment. But how would we get at clues to the possible neural basis? In other words, how does the brain actually pull this off? So in my laboratory we wanted to figure this out. So my student Don Vaughan and I had people look at the herring illusion on a screen. You've got a background of radial lines like the bicycle spokes, and we flashed two vertical lines on top of this. And I'll just take a quick second to give you a sense
of how we quantify illusions. In the laboratory, A person sits in front of the computer and they use let's say the right and left arrows on the keyboard to change the curvature of those two lines. So at one end of the range, they're actually physically bending the lines outward, and at the other extreme they're bending them inward, and somewhere in the middle they're physically straight. And what the person does is adjust the curvature of the line until
it looks straight to them. But of course with the hairing illusion, you need to actually adjust the lines so the middles are curving inward in order for it to look straight. In other words, we see how much curvature it takes to cancel out the illusion, and that's the way we can quantify the size of the illusion. Okay, so we measure the heiring illusion, and no surprises there. But now what we do is we replaced the radio
lines with an actual star field. We have dots moving in an expanding pattern like the stars and Star Trek, And now people are judging the size of the hering illusion against the background of expanding dots.
And what we find is that.
The illusion still happens. The lines still appear bent, which is just what you'd expect from the perceiving the present hypothesis. Okay, but here's the really wacky part, the part that uncovers an unexpected secret in the brain. We now measure the size of the hering illusion over a field of contracting dots. So picture that Star Trek footage running backwards. Now everything is moving from the outside to the inside.
And here's the surprise.
We find that the size of the herring illusion was exactly the same here. In other words, the lines still curve outward, just like in the other two cases.
So what does that mean. We can get this.
Illusion by having radial lines or dots expanding or dots contracting, and you find that the bars bend out in the same direction. Now, at first glance, the bending of the bars during contracting motion would seem to refute the perceiving the present framework. If your brain is doing an active of temporal extrapolation of the scene, it should make the bars bend in the other direction. But the key thing to note is that backward motion is ecologically quite rare.
Unless you're a dog looking out the back window of a car. Most animals probably never see backward optic flow in their lives. Okay, so we did a lot of other experiments in this paper, but just this first result that the Herring illusion happens the same with expanding or contracting optic flow tells us some critical things. First, it tells us that this spatial warping we see isn't a
sophisticated online computation. Instead, it's a basic mechanism that just acts to get it right in the most common scenario of forward motion. And because backward motion essentially never happens, it doesn't matter that the mechanism is so unrefined, and it gives us a big clue about the the underlying neural mechanisms. The most parsimonious explanation. In other words, the simplest idea would look for something in the brain that's equally sensitive to lines like the bicycle spoke and also
to motion in either direction along that line. And it turns out there are very simple neurons in primary visual cortex that do exactly this. They're called orientation selective neurons, and they respond to lines, and they also respond to dots moving along the same trajectory as the line in
either direction. So what this means is that our findings are consistent with the perceiving the present hypothesis, with the caveat that the spatial warping to counteract neural delays is not a smart active neural process, but instead it's just a simple mechanism that succeeds only under forward moving circumstances, which is most of the time. Now, as I said, we did a lot of other experiments, and I'm gonna skip the detail because they're not as important as this
main point. But all the other experiments supported this hypothesis that the bars look bent because the brain is extrapolating ahead what things would look like in the next moment. And by the way, our findings weren't consistent with older theories like angle over estimation, as Herring had suggested. So if you want to dive deeper into the paper, I'm linking it to the show notes. So what we saw today is the issue that it always takes time for
signals to move through the brain. And although people got excited when they discovered that the signals were carried by electricity, it's nothing like the way that electricity runs on wires, which is close to the speed of light. In the brain, we're talking hundreds of millions of times slower than that. So you have these signals limping along in the brain, and that means we are always living in the past. Our brains sometimes see fake news about the world out there,
but we always see old news. Your brain is always getting information about an outdated version of the world, one that no longer exists. And so one of the many incredible things your brain does is make predictions so that you can perceive things as they probably are right now, rather than perceiving the stale version of the data, which is much less useful by the time you see it.
This is the brain's very bold move to compensate for its own delays, and that wacky fact explains a lot about how we catch balls and walk through crowded airports, but it also seems to explain surprisingly why we see this basic geometric illusion of straight lines not looking straight, discovered by Heiring almost one hundred and sixty five years ago. Now, I wrote an article in Nature Reviews Neuroscience in which I covered all kinds of visual illusions, and that's in
the show notes. And each one of these illusions opens a new mind shaft into the brain, teaching us why it's happening. And once we understand that, we can usually construct new illusions, which is why amazing new illusions are coming out each year.
Now.
Sometimes people see all these illusions on their social media feeds and they might become tempted to say that everything is an illusion. We don't accurately see what's really out there. But that's probably not the right conclusion, because often we do see what's out there accurately, as verified by our
other senses and by our objective machines. For example, if I put a cup of steaming coffee on the table in front of you, you see that, and you heard the clunk when I set it down, and you can verify the heat with your fingers and smell the coffee and pick it up and taste it, and you can measure the presence of the cup with video, or the heat with an infrared camera, and the coffee with spectroscopy and so on. So it's not that everything you see
is illusory. Instead, we see what is maximally useful to us. And every once in a while we can throw an unexpected wrench in the system by showing the brain something that happens to tickle the right receptors. And in those very special cases we can catch the system coming to the wrong conclusion. But it's precisely because the brain has evolved to do the optimal thing in all the other cases.
And traditionally, when we find these special cases, we just laugh and enjoy and keep scrolling through our feed But the endeavor of figuring ourselves out encourages us to pause, to ask why, to dig deeper, and although these things sometimes take centuries to answer, they typically yield deep insights into who we are and what is actually going on. Go to Eagleman dot com slash podcast for more information
and to find further reading. Send me an email at podcasts at eagleman dot com with questions or discussion, and check out and subscribe to Inner Cosmos on YouTube for videos of each episode and to leave comments until next time. I'm David Eagleman, and this is Inner Cosmos.