Ep63 "Why do brains love faces?"

Tell us Welcome to Inner Cosmos with me David Eagleman. I'm a neuroscientist and an author at Stanford, and in these episodes we dive deeply into our three pound universe to understand some of the most surprising aspects of our lives. Today's episode is about faces. Now, let's say I posed the question to you, how does the brain recognize faces? You may reasonably think, why is that even a question? Recognizing people's faces isn't really that hard, But think about

how similar faces are to one another. They're insanely similar. You have two eyes, nose, mouth, chin, ears, everyone's face looks pretty much the same people's faces are pretty much very tiny variations on a theme. The only reason you are able to distinguish faces, literally thousands of faces from one another, is because your brain devotes an enormous amount of circuitry to it. It puts so much effort in that it feels effortless to you. And we have all

this circuitry because face recognition is extremely salient. It's how we recognize this package of identity from this other. Oh that's my mother, that's my neighbor, that's my wife. There's Tom Cruise and his face is similar but different from Dwayne Johnson, and that's different from Joe Biden. It's massively important for us as a social species to keep track of identities. Your brain wants to know who it is and everything that entails, like is that a friend or

an enemy? What are the risks and opportunities here? And the answer to that is massively different. If you're looking at the face of Ted Bundy versus your best pal, and keep in mind that to a space alien and these faces would be essentially indistinguishable. Now, when you're trying to distinguish faces, there are lots of other cues like hairline and hairstyle and height, and details of their skin from color to smoothness, and even cues like how they

walk and their clothing choice. And context also plays a large role. You've probably experienced some time when you saw somebody you know well, like a neighbor or a colleague, but it's in an unfamiliar setting and you struggle to recognize them immediately. Why does that happen Because our brains use contextual cues when identifying people, So our brains rely not just on facial features, but also on surrounding context

of all sorts to identify people. But despite all these other cues that we use, just note that facial recognition is possible even over zoom, where many of those other cues are gone. And even if you've got a bunch of movie stars to put on wigs and change their hairlines and so on, you'd still be pretty good at distinguishing them just based on the details of the face itself. And I think there are at least two ways to appreciate how hard a challenge face recognition is to the brain.

The first is to understand how difficult it is to get computers to recognize and distinguish human faces. Now you might think, aren't we pretty good at this with modern technology. Yes, but this has been a sixty year slog by very smart people in the computer science world. Starting in the nineteen sixties, people realized that face recognition was a really hard problem, and they've been climbing that hill ever since.

So the way that researchers have gotten face recognition to work on computers nowadays is by busting the problem up into different computational stages. So first, you have algorithms just for detecting a face, like is there a face out there? Then you have a second step where you segment this face from the background and figure out how the face is aligned to tell you about the size of the face and the illumination on the face and even the

pose of the face. Then you have a third step where you extract the facial features, like here's the length of the nose and the distance between the eyes, and the shape of the mouth and the position of the ears. All these things you pinpoint and you measure them, and then in the fourth step, you match those numbers against a database of faces to see if you can identify this one. So this is a really complicated algorithm and

it's instructive to see how computer get things wrong. I saw a great photograph where it's a family sitting on their porch at Halloween, and the computer has identified each of their faces and put a little label next to it, like Henry and John and Sarah, and then it identifies another face and says unknown person. But that face is the Jack O lantern, the pumpkin that's sitting there on the porch. Now, a human observer would never make that kind of mistake. But this goes to show that it's

a very computationally challenging problem. So some version of this very complicated algorithm, probably not exactly the same, but equally complex. This is what your brain is doing under the hood when you just glanced at some billboard and you think, oh,

that's Paul Giamati. I love that guy. And we'll come back to what the brain is doing in just a moment, but I just want to say for now that quite often we find that the tasks that seem most effortless to us are the things underpinned by the most brain circuitry. And I want to look at a second way to appreciate how hard a challenge face recognition is for the brain.

Just look at a bunch of Golden Retriever faces. If you were in a situation where you were hanging out with hundreds or thousands of Golden Retriever dogs, do they really look that different to you? Or consider horse faces

or squirrel faces or cowfaces. Obviously we can imagine some faces at the extreme that look a little bit different, But in general, if I dropped you into the middle of a giant ranch and you looked at thousands of cows, you'd say, okay, well, each face has two eyes and a snout and a mouth, and there's just not that

much difference from one face to another. You wouldn't meet recognized if one of those cows had been in a movie like oh, that's that famous cow, or if one of those cows was wanted by the legal system, like wait a minute, I recognize that cow from the poster of the post office. And of course, if one of these cows was looking at you and a bunch of other humans, she would feel exactly the same way. There's not that much difference between human faces. Now side note,

which we'll return to. If you were the rancher, you could get better at distinguishing cow faces. This is because of brain plasticity and because distinguishing the cows is salient to your rancher brain. But for now, what I want to surface is just how similar faces are before you get good at them from experience. Faces are unbelievably similar.

And one expression of this difficulty in distinguishing faces comes from a psychological effe fact called the other race effect, and this refers to the tendency of people to have more difficulty recognizing faces that belong to races that are not their own race. This phenomenon is also known as the cross race effect or the own race bias, and it's been widely studied and you see it across all cultures. Now people sometimes hear this and they take this as

evidence for racism. It's not racism in the sense of you treating your own group better and other groups more poorly. Instead, the other race effect simply results from the fact that your brain gets trained up on the faces around you, and you are better at recognizing those faces than other faces. So if you grew up in Cambodia, you will be excellent and distinguishing Cambodian faces, and maybe not so great

at distinguishing Norwegian faces. If you grew up in Norway, you're excellent and distinguishing Norwegian f but not so great at distinguishing Zimbabwean faces. And if you grew up in Zimbabwe, you're not so good at Intuit Eskimo faces and so on. So your brain becomes attuned to the type of faces that you see most often, and this heightened sensitivity makes us better at distinguishing among those familiar faces. As a side note, can you mitigate the other race effect? Sure,

it's just about exposure. You can improve the ability to recognize other race faces through training and exposure. Now, Hollywood has been on the forefront of this for a long time, making sure that there's a mixture that we all get exposed to different sorts of faces, and that's presumably very useful. It could be noted that Hollywood has tended to be mostly in love with white faces and black faces, and that leaves a lot of people out of the mix.

So we the viewing audience don't necessarily get much better at Cambodian face or Norwegian faces, or Inuit Eskimo faces or whatever. But it's a start to training our brains on a broader diet of faces. So the other race effect underscores how our brains can get tuned to the social world around us, and it highlights our natural tendency to recognize familiar faces more easily. Okay, so where are

we so far. It's really hard to distinguish faces unless your brain has had lots and lots of practice at it. But again it seems effortless to us now. Even though we're not always great at distinguishing faces from one another, unless we've had lots of practice, we are highly programmed to identify that there is a face there and to

zoom in on it. Why because faces carry so much information for us, And this is why you sometimes look at the electrical plug in your wall and you see a little face, or you see a pattern of burn marks on a piece of toast and you think, oh, that looks just like a face, or you look at the surface of the moon and you think, oh, there's a man there looking at me. In science, this is

called paradolia. Paradolia refers to our tendency to perceive specific and often meaningful images in random patterns, and the most common usage of this word is in the case of perceiving the pattern of a face in random unrelated objects. And we've all been there where your brain says, oh, that looks like a face when you're looking at a cloud or a rock formation, or some arrangement of fruits on a table. And I don't know if you've ever seen this image from the surface of Mars. I'll put

it on the show. But on the bumpy landscape of Mars, there are some bumps that happen to fall into the pattern that look like two y's and a nose and a mouth. And many people go nuts about this image because they suggest that it's a sculpture of a human face planted there by aliens. But probably not. This is merely our brains experiencing garden variety paradolia. We assign patterns to random inputs, and in particular, we love to see faces. Okay,

but why does this happen to us so commonly. Well, it's because we are hardwired to see faces everywhere, and the reason is that faces are crucial for social interaction. Seeing a face tells us about a person's identity, their emotional state, whether they're a friend or a foe. This ability helped our ancestors to survive by recognize is saying oh, that's a tribe member. Oh, that's a potential threat. In fact, even newborn babies prefer to look at faces. They look

at face like stimuli more than other stimuli. So take two little circles and a little square beneath that, and a little rectangle underneath that, they'll stare at that as opposed to the same shapes in a different configuration. And this underscores that brains are primed to focus on faces. Paridolia is just an extension of this bias to recognize faces. It's our brain's way of staying vigilant because in the wild, it is safer to mistakenly see a face where there

isn't one than to miss a real face. So this hypersensitivity to faces means the difference between life and death in some situations, like spotting a lurking predator or an aggressive intruder. By the way, I'll just mention that paradolia is something that artists have always exploited. They turn random patterns or abstract shapes into recognizable faces. Just as one example, one of my favorite artists, Salvador Dhli. He very often takes advantage of paradolia to create double images that play

with the viewer's perception. You can see this as a rock formation or as a face. So paradolia highlights how our brains are constantly trying to make sense of the world around us, even when there isn't anything immediately recognizable. And I want to give just one other angle on paradolia here. If you're a regular listener to this podcast, you know that I often talk about how the brain's main job is to build an internal model of the

outside world, and our model influences our perception. So whatever you specialize in, you'll see lots of that world. We're all specialists at faces, and we often see those where there isn't actually one, but it generalizes beyond that. So for me personally, when I look around the world, I tend to impose shapes that look like brains. I think, oh, there's a brain, or there's a midsadgital section, or there's a cerebellum. My friends who are pilots tend to see

airplane shapes and runways. My collogist friends see mushroom shapes, my dermatologists friends see melanomas, and so on. Whatever you specialize in makes you an expert in detecting that in the world, and you impose that interpretation on lots of things. It's paridolia writ large. Okay, so back to faces in particular. Now we're ready to turn to how the brain does its magic with face recognition. One of the key areas in our brain that does this is in the temporal lobe.

It's a little region called the fusiform face area or FFA fusiform face area. Now, we can see in brain imaging that this area is particularly sensitive to faces. But what's interesting is it's not just any part of the face that gets attention from the FFA, it's the holistic view of the face. So the FFA responds strongly to the overall configuration of a face rather than individual features.

So that allows us to quickly recognize faces and familiar faces, even if we only get a quick glimpse of them. When you look at these brain imaging studies, for example, with fMRI, you find that this fusiform face area responds more to familiar faces than to unfamiliar faces. And given what we saw a moment ago with the other race effect, the FFA is more active when you're viewing faces of

your own race compared to faces of other races. So again, just like recognizing the distinction between different cows if you're the rancher, or distinguishing faces that you grew up with as opposed to other cultures that you rarely see. Your visual system is specialized to efficiently process faces that you see a lot more. Now back to this issue about

recognizing the face holistically rather than detail by detail. This is what underlies the phenomenon that's called the inversion effect, which is when you see a face that is upside down, it's really hard to recognize. So try this. Just open a magazine upside down and flip through the pages and see if you can recognize the people in it. And then when you flip it right side up, it's a

whole different experience. And this is because this part of the brain FFA is super specialized on faces, and it's always seen faces in a particular way, and it's recognizing the whole big picture with two eyes above the nose, above the mouth, and if you throw a ranch in that, it doesn't function so well. The key thing is that this inversion effect is much stronger for faces than for

other objects, which highlights this very specialized nature of face processing. Now, what's wild is that this part of the brain is trying to see faces everywhere, as we saw in paridolia. And so if someone puts an electrode into this part of your brain, then you might look at, let's say, a soccer ball, and when the electrode turns on and stimulates this area with electricity, the hexagons of the soccer ball seem to you to become the eyes and the

mouth of a face. So activity in this part of the brain is always working over time to detect faces in the world, and sometimes it does this by imposing the interpretation onto the canvas of the world. Now, consistent with what I mentioned before, the FFA isn't specialized just for faces. It also plays a role in recognizing other things that we're highly familiar with, like cars for car enthusiasts,

or birds for bird watchers. So that tells us slightly more generally that this part of the brain specializes in recognizing very detailed things that are relevant and that we have a lot of experience seeing. So back to faces, the FFA, it's not just that it's sensitive to faces. It's actually critical if you want to see a face. If you get damaged to the FFA or to its connections, then you become unable to recognize faces. And will return to that in a moment, but first I just want

to make one more thing clear. Like almost everything in the brain, it's not just one area that's involved. Face recognition uses a broader, specialized network. Beyond the FFA, you have other areas involved, like the occipital face area and the superior temporal sulcus. Don't worry about the details, but the thing I want to surface here is that some areas are more focused on the detailed features of the face and others are sensitive to facial expressions and movements.

And the key is that together these areas form a larger network that allows us to recognize faces and interpret their expressions well. Your brain's also doing a lot of work to gauge a person's emotions and then navigating social interactions by carefully watching the reactions of someone's facial expressions. We humans are super specialists at this because our well being generally depends on our ability to recognize and understand

the people around us. And just to give you a sense of how all these networks interact, there are studies showing that you're interpretation of facial expressions is influenced by the broader context. So just as an example, in one study, you're shown faces with neutral expressions, but you're given different contextual information about the person. So if you're told the person is happy, you perceive the neutral face as slightly smiling, and if you're told this person is sad, you perceive

that same face as slightly frowning. So, in other words, your brain combines the facial information with the context of the larger situation whenever you're interpreting emotions and for that matter, intentions. So putting together the giant networks in the brain involved in identifying and interpreting faces, the thing that becomes clear is that we are massively finely tuned to process faces

in a way that benefits are social interactions and are survival. Now, if you've been listening to other episodes of this podcast, you'll know that I'm obsessed by the differences in the internal life from person to person, from head to head. And one place this really comes up is with individual variability in face recognition. So, while most people are fairly good at recognizing faces, there's a shocking amount of variation in this ability. Some people are exceptionally good at it,

and other people struggle. And this brings us to the concepts of super recognizers on the one hand, and face blindness on the other hand. So let's start with the super recognizers. These are people who have an extraordinary ability to recognize faces. They can remember and identify faces even after just brief encounters or after long periods of time. Super recognizers are terrific at picking out their acquaintances and really large crowds, and they sometimes end up getting employed

in security or law enforcement to identify perpetrators. Like they'll be hired to watch the video of let's say a subway entrance and you'll see thousands of people flowing in and out, and they'll just stare at the feed for hours or days, and then they'll say, oh, there's the guy. Now. On the other end of the spectrum, we have what's known as face blindness, and this is known as prosopagnosia. This word comes from the Greek pro sopon meaning face,

and agnosia meaning lack of knowledge. So people with prosopagnosia have a very hard time recognizing faces, even faces of close family and friends. Now, by the way, this isn't all or none. The condition ranges from mild to severe, like difficulty recognizing familiar faces to an inability to distinguish any faces at all. And at that far end of the spectrum, just imagine not being able to recognize your own spouse or child if you see them out of context.

And this is the reality for many people with prosopagnosia. Now, how does prosopagnosia happen? Usually you're born with it. This is called congenital prosopagnosia, just meaning it's present from birth. In rarer cases, you can have acquired prosopagnosia, which just means that you get it because of a brain injury later in life, like a stroke or a traumatic brain injury.

And in these acquired cases, people notice the sudden change in their ability to recognize faces, which is obviously distressing and isolating. But what's fascinating is that people born with this usually don't realize they have it because they've never known anything different, and so they just get along by using other cues like people's voices or their clothing or the way they walk, and this is how they recognize people.

One example of a person who had prosopagnosia was the neurologist Oliver Sacks, and he wrote an article in The New Yorker in twenty ten on this. He talked about his personal lifelong struggle with face blindness, but he never realized there was something unusual about this until his middle age. He'd always just been really bad with recognizing faces, but

that was just the way it was. But he went to visit an older brother in Australia and they got to talking and he realized that his brother had the same problems with faces that he did, and it dawned on him that this was something beyond normal variation, and that they both had this trait of face blindness, and he guessed that there was probably some distinctive genetic basis

to it. And very informally, I've noticed among people that I know that those with prosopagnosia often don't enjoy movies as much as other people, because in order to follow a plot, especially if it's switching between an A story and a B story and a C story, in order to follow that plot, you really need to get it when this actor comes into the scene who you haven't seen for fifteen minutes and who's now wearing different clothes, but this is just a continuation of his plot, and

everyone else in the audience recognizes him and immediately remembers what was going on with him in the last act. But just try to imagine how difficult it makes things to follow along if you don't immediately recognize the person. And by the way, if you don't have prosopagnosia, you might be thinking what would that be like? How could you not recognize a person? A common analogy is to think about what it's like to walk through a forest and just imagine if you had to remember and distinguish

every tree that you see. Imagine you had to have a name for every tree, and you were quizzed on it later. That sounds impossible to most of us, but this is what daily life is like for the person with face blindness. There wandering through a forest of people,

all of whom look essentially indistinguish. Now, I just want to note beyond issues of following movie plots, prosopagnosia can have pretty significant social effects like anxiety and embarrassment and frustration because of and inability to recognize other people, And for children with face blindness, social development can be tough because you need to recognize your peers and your teacher, and occasionally kids will get misdiagnosed with other learning or

behavioral disorders, which complicates things for them and an adults, face blindness can lead to social isolation or in difficulties in forming and maintaining relationships. So super recognizers and people with prosopagnosia they represent extreme ends of the face recognition spectrum. Most people fall somewhere in between, but with varying degrees of ability and understanding the spectrum in face recognition. This is how we can get insight into the differences between

people's realities. Okay, now I just want to say lest you think that these issues about recognizing faces just show up in podcasts, In fact, they show up in courtrooms all the time, most notably in the very stormy world of eyewitness identification. If you check out my episode nineteen, I talked all about the difficulties of eyewitness identification, which is generally very difficult because of fundamental flaws in memory.

But there are other challenges to this as well, and the main one of interest here is when people are asked to reproduce faces, as in who did you see at the scene of the crime. So what police did for centuries was to have a trained artist who sketches what you describe. But of course it's really hard to describe a face, and so by the late nineteen fifties, the Los Angeles Police Department introduced a new way of doing this where you line up individual strips for the eyes,

the eyebrows, the nose, the mouths, and so on. So instead of me asking you, hey, can you describe that guy's face? Instead, you now get a bunch of possible eyes and possible noses and mouths and ears and chins,

and you try to reconstruct it that way. And this ended up spreading from Los Angeles to Scotland Yard by the nineteen sixties, and they had some successes, and eventually this became a computerized system where people can piece this together on a computer, and so this was considered a real success about how you can use things beyond just

unreliable verbal descriptions to identify perpetrators. But when researchers subjected this to more careful study, it turned out that this approach of piecing faces together this is pretty imperfect, and the fundamental problem is that the composite is built from pieces and parts, like the guy's eyes look like that, and his mouth looks like that, his nose like that, and so on. But that's not the way the human

visual system works. It recognizes faces based on the whole picture, the gestalt, where the whole is perceived as more than the sum of the parts. And so it turns out that reconstructing a face from pieces and parts is not really so easy. I'll also mention another face recognition issue which shows up in courts all the time, and that's

the other race effect which I mentioned earlier. People are more likely to misidentify individuals of other races, for example, a Hispanic person identifying a Japanese person and so on. This is not racism, it's just neuroscience. Your brain comes to represent what you see around you, and so in many ways, the legal system has to chew on the issue of how brain recognized faces and wear brains don't

do it so well. Okay, So moving beyond courts into the broader society, we're now entering a brave new world

of face recognition technology. There was a big moment in twenty eighteen in China where new face recognition technology picked out a person that the authorities wanted out of a crowd of sixty thousand people at a concert, and apparently when they came up and grabbed the guy, he was infinitely surprised because he assumed that being in a giant crowd made him safe, that no one would ever be able to spot him, and he was of course right

that no one could, but the computer did. Now, this was an example where face recognition technology performed at a superhuman level, but note that the technology often messes up and also does something that's very human paradolia. We always see algorithms mistakenly idea identifying faces where there aren't any, which mimics the same tendencies that we have as humans.

For example, remember the jack O lantern that I mentioned earlier that the Facebook algorithm mistakenly identified as a human face. Computers tend to impose patterns as much as we do. And I just want to say that even though we're often worried about new technologies and its face recognition abilities, what's also going to grow out of this are increasingly

sophisticated assist of devices. You have AI driven tools that are going to offer real time face recognition support which is going to allow you to recognize your friend when you see him totally out of context. But more importantly, this has the potential to enhance the quality of life for people with prosopagnosia. So let's wrap up. Face recognition is a remarkably computationally intensive thing that we do, and even though it seems effortless, we have these massive, specialized

neural networks that underpin our ability to do this. This ability highlights the very special role that faces play in our social lives and in our interactions, and it's so important that we all experience things like paradolia, where we impose an interpretation of faces on other patterns all around us, and this underscores our brain's need to make sense of the world, to impose order on chaos, and to connect

what we see with what we know. And finally, we saw that skill in face recognition varies widely from person to person. We can measure this from brain imaging in people's performances, and we find a spectrum from super recognizers to those who are face blind. And understanding how different reality can be on the inside is critical for living in a society and understanding one another as it stands now, most people on the planet are unaware of this spectrum.

They've never heard of something like prosapagnosia, and as a result, they don't recognize it in themselves or in a loved one. So this is one of the things we gain from a deeper understanding of the brain, a broader empathy that allows societies to interact more richly. So the next time you see a newborn baby, lock onto its mother's face and track the face and study it, just remember that you're not just seeing something cute, you are catching insight

into deep circuitry of the inner cosmos. Go to Eagleman dot com slash podcast for more information and to find further reading. Send me an email at podcasts at egleman dot com with questions or discussion, and check out and subscribe to Inner Cosmos on YouTube for video of each episode and to leave comments. Until next time. I'm David Eagleman, and this is Inner Cosmos.