I have a conviction that the brain is understandable. You look inside these AI models, and there's some clarity coming up, but it's also super confusing. So I believe that there's a huge landscape of different architectures that are able to solve these tasks, and we have two points. There's Chachi PT and Dali, and there's the brain. I think the brain is on one extreme of efficiency and understandability, and that's what I want.
That's what I care about. I'm John Foxe, Director of the Del Monte Institute for Neuroscience here at the University of Rochester, and I'd like to welcome you to another episode of Neuroscience Perspectives. I'm absolutely thrilled to be joined by renowned neuroscientist Professor Doris Tsao. Her pioneering research has transformed our understanding of how the brain processes visual information. And her lab helped us discover that it takes only a few hundred neurons to encode any given
face. For reference, as many of you will know, there are billions of neurons in the brain. Dr. Tsao is a professor of neurobiology. at University of California, Berkeley, and she's been the recipient of numerous awards, including most recently the Kavli Prize, one of the most prestigious international
science prizes there is. She's also been elected as a very young scientist to the National Academy of Sciences and was named an investigator of the Howard Hughes Medical Institute, which is among the highest of distinctions for biomedical research. Doris, thank you so much for joining us here on Neuroscience Perspective. I'm dying to get into things with you and ask you a bunch of questions about yourself. But let's kick off with your research. You know, why faces? Why
the visual system? Where did that all come from? probably high school when I realized that the visual system, the neurons are what generates our perception of everything. But I came to faces kind of by accident. You know, I was a graduate student. I was recording in V1. I was initially interested in this question of how the brain represents space. and I wasn't getting anywhere, so I got into fMRI, and I read about Nancy Kanwisher's work that reported a face area in the human brain,
and it just seemed like a fun thing to do. Let's just show faces and objects to a monkey and see if monkeys also might have a face area. And then you found it. they didn't just have one face area. They had a whole system of face areas. Yes, yes. And so it took off on its own. You know, every question kind of naturally led to the next question. It's like, yeah, so there's six face areas. What are they each doing? Are they each doing something different? Yeah, yeah,
yeah. Tell us about these six face areas. You
know, why six in a monkey? for faces yeah we we wondered about that for a long time and um a few years ago i think we discovered the answer basically there's a whole map of object space so object space you know same way we have 3d space we can represent you know different points and locations so you This part of the brain called IT cortex represents objects also in this more abstract space that's also two -dimensional, and there's repeated copies of it, and the representation
becomes more and more invariant, and so you have these multiple stages, and that's why you have these multiple copies of this map, and faces are one part of that map, and so when you just look at faces, you see these six face patches. I got you. Okay, so it's six levels of complexity? Is that the way to think about it? Yeah, I think that's... But highly... They're part of a circuit,
right? Yes, yes. So they're interacting to evolve, and eventually then... an individual face I mean I think we all remember these stories about like the Jennifer Aniston neuron and that you know when you get down to the level of like dissociating two very similar faces from your life The representation there is actually very limited in terms of the number of neurons that are involved. Is that fair to say? Or is that not the right way to think about it? Yeah, so face carries so much
information. There's information about the physical features. There's information about the identity, the expression. And I should say this is still, we're still figuring all this out, right? It's still early days. But indeed, you don't need... that many cells to represent the identity, which is kind of surprising. You look at a face, it just, you know, it feels so complex, right? It's like ineffable, like who knows how many neurons
that would take. But it turns out that with just 50 numbers, and this is something that computer vision people discovered with like 50 numbers. you can, like, recreate the face, right? So you can really compress the information down. And it turns out that the neurons use a code that is very similar to one that the computer vision people figured out for how to compress the information.
Gotcha, yeah. So this sort of dimensionality reduction is instantiated neurons in much the same way that one can do it with a formula in silico in a computer. Exactly. And, you know, in recent years, like with Dali, right, where you can, like, you know, specify any prompt and
it can create a picture. of anything that you can possibly imagine um that idea of being able to compress reality has really taken off right and they're kind of an existence proof that you know like like the space of all possible for example 500 by 500 you know rgb images it's like it's more than the number of atoms you know in the universe right but you know with just this like one gigabyte you know neural network like you can create any like any picture of reality
and the reason is because reality is much lower dimensional than the space of all possible images right so much of that space is just like random noise right but then there's like Yeah, the space of reality, of objects. And there's beautiful ways to discover those spaces that these machine learning people are figuring out. And obviously the brain has figured out how to do that with its limited neurons. And so that's what we're trying to figure out. You know, and all the buzz
is about AI at the moment. And you made a case yesterday, and I'd be interested to hear you sort of extrapolate on that a bit, that actually it's still very important for us to understand the neurobiology of vision because we have a lot of insight. of how that's achieved to provide to AI folks and that there's still a place for neuroscience in the face of AI. I mean, it is
incredible. You know, our brain runs on 30 watts while you hear about them, like, OpenAI building nuclear reactor farms to power, you know, next generation chat GPT. So I think the brain is so much more efficient. I have a conviction that the brain is understandable. And, you know, you look inside these... AI models, there's some clarity coming up, but it's also super confusing. So I believe that there's a huge landscape of different architectures that are able to solve
these tasks. We have two points, you know, there's Chachi PT and Dali and there's like the brain. I think the brain is on one extreme of efficiency and understandability. Right, right. And that's what I want. Right, these AIs are built on the part that you can take brute force, super fast computers and a lot of energy and just churn through billions of calculations. Exactly. And they just impose this one arbitrary architecture. You know, what is the likelihood that that is
the architecture, right? Like, I just really believe that the evolution figured out something.
tricks that we can still learn from. Here's a thought I was thinking about technologies and the way in which you sort of have pulled together multiple technologies to address your question and we're sort of living in this like extraordinary time right particularly over the last couple of decades as you've sort of built out your career where you know fMRI came online high field magnets extraordinary electrophysiological techniques how much is that driving and do you see Do you
see AI, for example, now interfacing with that and really driving forward science in a whole new way? Oh, yeah. I mean, absolutely. I'm sure you know this quote from Sidney Brenner, right? Progress in science is driven by new tools, new phenomena, and new ideas in that order of importance. That's good. Sadly, it's true. And I was just so lucky when I was in Boston as a grad student.
landed at Mass General Hospital where there were all these experts on fMRI, you know, the Martinez Imaging Center, and that was, like, so important, right? Because these face patches, like, just sticking an electrode into the temporal lobe, you would never find them. Or you might, but you couldn't then find it again. Whereas this was, like, you know, it just gives you a systematic
way, right? fMRI gives you this bird's -eye view of the brain so you can see all the, you know, loci that are important, then you can go in and study the single neurons in detail. Right, right. Around those same times. people like Greg McCarty and Ina Puse, Truett Allison, were using ECOG and seeing fusiform face area. Yes, they were the pioneers that really suggested there was something profound down there. It's amazing.
I mean, having done some of those recordings myself, when you have an electrode sitting over the fusiform face area and you show it a face, I mean, it announces itself. It's really amazing. Okay, good. So we share that. Yeah, yeah. When did you decide you wanted to be a scientist or you think this is where I'm going to end up? You know, I wasn't interested in science very much as a kid. I was not really a curious child. I played with Barbie dolls. Really? Yeah. I love
to read. And so I always found the stories, the biographies of scientists very inspiring. These were people who came from humble backgrounds, but just by working hard. they could you know achieve glory so that seemed like a very accessible path to glory to me um and then It was after I got accepted to Caltech. And, you know, I was worried I was going to fall behind. And so my senior year, I started reading the Feynman Lectures on Physics, Volume 1. And that was when I understood
what science was about. It was so beautiful. It's just like this freedom to ask any question you want and use your brain and think through it logically and come to an insight. Those Feynman Lectures, you can get them on YouTube. And he has this fantastic Brooklyn accent. And they're really wonderful. He's such an entertainer. Recommend them to people. You don't need to be a physicist to enjoy Richard Feynman telling his stories. Amazing. Now, but your mom and dad were academics?
So, yeah, we came to the U .S. So they were graduate students at the University of Maryland. My mom was a computer programmer. I wouldn't call her an academic. I think she was my mom. And by my dad, you know, he was like he, you know, during the Cultural Revolution, he was sent off to Manchuria to like chop trees. And he always knew that that period would end and there would be a new day. And so he, you know, brought a thick stack of math books with him, like studying. functional
analysis in the forest. And so, you know, after the end of the Cultural Revolution, you know, he got into Peking University and then we came to the U .S. And so, yeah, he's a mathematician and always, like, just deeply curious. You know, he's 82 years old now. He works harder than I do. Like, whenever I go in his room, he's, like, watching a YouTube video, like, learning a new... Recently, he got excited about this area of math I've never heard of called sheaf theory. And
it's, like... Apparently, it's about how you can, everything is defined by its relation to other things. And he's telling me how that's like the essence of Marxism. And so that's the kind of person he is. So mathematics to political theory. Yeah. Just a brilliant mind. Yes. And an inspiration then to you both, both the practical
part of computer science that your mom. brings to the table and the inspiration yes yes you know my mom actually like programmed some stimuli for me when I was yeah setting these bases yeah so yeah so a big move though to come from China to the US in the in the 80s was it yeah 1980 1980 so right yes yeah and uh how much of that do you remember you must have been extraordinarily young at that point I remember yeah I remember this night we were in Hong Kong I remember tasting
chocolate for the first time on that journey somehow um yeah the U .S. you know when we first came my grandfather he had been here for many years and so he would have these like American culture evenings, apparently. He would teach, because I came with my cousins and their family, so he would teach them about all the American traditions, like this is football and here's the rules and stuff. Wow. Were you five or six or something? I was four. You were four, right.
And yet, very profound memories of that, obviously, because it was such a big change. And how much of your very early childhood in China do you remember? Yeah, vague, vague glimmerings. And you studied mathematics, though, in university, right, as well? Yeah, I studied math and biology. I really love math. Yeah, I wish I could, like, if I had a secret wish, it would be, like, if I could go do a math PhD, yeah. Look at that. Well, maybe you will, like your dad at 82. Yeah.
That's outstanding, yeah. So then you went to Caltech, you did your undergraduate there, and then you went to Harvard for graduate school. And then you ended up in a very important lab or very famous lab, right, with Marge Livingston. Tell us a little bit about that and the impact Marge had on you. Marge is an incredible scientist. She's such a true scientist. You know, the first time I met her, like during the graduate school interviews, she was recording and she kind of
like. had to finish her experiment then she looked up at me and said hello and then she was like so um just direct you know she said what are you interested in i told her something vague about you know i want to understand neural circuits and she said well if i were interested in neural circuits i would go to larry katz's lab maybe this is not going so well but you know she just like intuitively understood me you know it was yeah i am very shy i was very shy i remember
i was like Marge, can I talk to you about something? Because I wanted to join her lab. And I didn't have to say anything more. She said, I ordered an amplifier for you already. Yes. So it was wonderful. She just does her own experiments every day. And she's taught us by example. Yeah, true. one of the great leading vision scientists
in the history. She's incredible. She started as a Drosophila neuroscientist, discovered these learning mutants, and then she went on to, she became so interested in vision, she felt like she found her calling. And she's also done incredible work understanding the relationship between art and vision, how artists... And so her mentorship style, she's sleeves rolled up, in the lab making recordings. Oh, yeah, yeah. And have you taken that into your own lab? How do you approach it?
Not as much as Marge. Yeah, I don't know how. I think I'd like to. Yeah, I do try, but just, yeah, time management. That's an amazing trajectory. And then you did something. A little strange, too, because, you know, most people, you know, the American science engine. But you took off for Germany, right? And I went to, was it Bremen? Bremen, yeah. And what was the, like, why? What was the decision there and how did that work? I got a fellowship from the Humboldt Foundation
to set up my own little lab. This was, you know, I got my PhD and I spent two more years in Marge's lab. But then, yeah, it was like an amazing opportunity. So you didn't do an official postdoc. Yeah, yeah. So, you know, we found these space patches and it just like there's so many questions. And so that was like an amazing opportunity to keep studying this on my own. Yeah. So off to Germany. And I'd always been interested in German culture. You were. Okay. Where does that come from? I
think it started with music. Like my father, he was always listening to Beethoven. I see. Yeah, yeah, yeah. And was that culture shock? Was it different? Did you settle easily? I loved Germany. It was like a much simpler place. You know, Bremen's a small town. There's like one department store in Karstadt. So there wasn't much to do. So I found a violin teacher. I fell in love with the violin again. I loved it. Yeah,
yeah, yeah. It's a brave move. I mean, to get out of your PhD and your own milieu and the place that you've sort of trained and to take off for
Europe to a foreign country. launch a science career I mean and are there lessons in that for example for for your trainees is there something about that streak in you that's willing to take a chance that's part of how you approach science I mean every path is unique but I think um yeah I've never worried too much about I've just never worried too much you know about the future I've always had this like irrational exuberance and so I try to like tell my trainees yeah yeah just
like follow follow your heart what you want to do everything else will take care of itself yeah yeah you gave a wonderful talk yesterday and I really appreciated the part where you made sure to point out exactly who the folks were that did the work, the graduate students and the postdocs, and give credit where credit's due. And it's obvious that you've put some very, very talented people through your lab. Oh, yeah.
Everything is due to them. It is such a privilege to be able to work with these brilliant young people. Right. It begins and ends with that sort of vibrancy in the ranks and training people. There's nothing better, right? There's nothing better than to be slacking furiously about a new idea. with a student and just fantasizing, it's, yeah, it doesn't get better than that.
I love it, love it. Here, I read a quote, just going back to mentorship, and we were talking about some of these leading elders in the field, you know, and this is a quote, I love this. It said, all of us desire to find a mentor who sees us as we wish to be rather than as we are. That's quite profound. Would you unpack that for us? What's that? What drove that? That was about
Marge, my PhD advisor. I felt that's how she... treated me like i was like a child not her like you know oars women for her grant yeah right right um not just yeah the engine room right yeah she like even when i was a young faculty member like you know caltech was starting out and she she said i said something about i was like still studying math and she she sent me an email and she's like you know i i you should keep doing that i believe you can do something
great you know in that realm and it's like she's always encouraged me like this side of me that's not like maybe conventional I don't know maybe is it fair to say then you know there's there's a pragmatics part of mentorship in a PhD you need to learn this material you need to know how to do this technique you need to be able to and then there's the the humanocentric piece of it there's the development of the individual through these you know early 20s into a fully
formed human and and that real mentorship sees those two things equally yes like I think that like you know everyone in our field comes in with this like ambition right this raw ambition and like you have to nurture that like yeah and i think like somehow you know like on the outside i think it's like this like soup crazy shy you know girl like just shoveled hair and all that and it was like but when march like saw this like you know this is like inner flame and she
like nurtured that and i i i hope i i do that for my trainees because they yeah they have their wild dreams and i think it's so important right yeah yeah developing the human as well The shyness. Tell me about the shyness. Is it gone or is it still there? It's totally still there. But you've learned to conquer it. I've learned to cope with it. We're talking right now somehow. I've seen you give in talks so many times and you're a
very confident presenter. And I think if somebody just saw the talk... they wouldn't know the shy Doris. Oh yeah. No, when I go to a conference, like there's always like this moment when hotel lobby, I see all these colleagues and I just like feel this dread. Oh my God. Even the most extroverted people. Well, I think even the most extroverted people get that. Oh really? I do. Yeah. And then, yeah, I do. That's true. Oh, I certainly do. Yeah. I was very, very shy youngster.
I sometimes say that to people and they're like, no, that's ridiculous. But, but it was, you know, but, and I, and I, I must say that's a, that's, I often have that, like, you know, you walk into the reception at one of those meetings and I go, oh, gosh, what if I'm just standing there alone as well? Just like, you know, you wade in, I suppose. Yeah. So you've had to conquer
that as part of it. Because obviously the other pieces, you know, there's the doing the science, but the communicating part is equally important, right? Yeah, yeah. I have to. Yeah, again, thank my mentors for that, like teaching me how to give talks and so on. You know, we're living
in strange political times for science. And they sort of, you know, the public perception of science as, you know, there was a time I think I read recently, you know, if you go back 10, 15 years, public trust in scientists was in the high 90%. 98 % of people had complete trust in the system. I think those numbers would not be there today. What are we doing wrong and how do we get at that? How do we get at science communication that brings our public along with us? Do you
have thoughts around that? Yes, I think we need to tell people about our science, make many more efforts to do that. And I think this podcast is a great way to do that. I myself am like writing a book now. I'm trying to link neuroscience, philosophy, and self -help. Because I think that there's this whole genre of neuroscience -based self -help that's around how you can sleep better, how you can eat better, and so on. But I'm interested.
I believe that the mind -body problem, insights into consciousness, into free will, those insights. don't live in some pure realm, right? They're about the brain and the brain is what we used to behave with. And they have like profound implications for how we should live our life. And I want to work that out. And I, so yeah. So you're waiting. solidly into this space to bring neuroscience.
Yes, I think yes. And I hope then to go out into the Oakland Unified School District where there are many underrepresented students who may not be thinking about careers in neuroscience, but they do care about how should I live my day tomorrow? Do I have free will? I think they would be totally captivated by that question if they could discover that there's a link between that and electrical activity in the brain. Maybe they will change their career trajectory. Outstanding. Thank you
so much. Thanks so much for spending some time with us and allowing us to get to know you a little bit better, Doris. You're a major star in the field. Look forward to watching you in the decades ahead and see how the work evolves. Thank you so much. This was fun.