Is it possible that we're thinking about intelligence in the wrong way? Instead of being something inside individual brains, is intelligence instead something that emerges from lots of brains that are constantly working to align with one another. And if we take on that lens, what does this mean about the way that we can build AI agents or the way that they can make us better? What is the
difference between information and information with a purpose? Today we're going to speak with Daniel Persick, a cognitive scientist who leads the Human Computer Interaction team at Amazon's AGI Lab. So get ready for a great brain stretch. Welcome to Intercosmos with me David Eagleman. I'm a neuroscientist and an author at Stanford and in these episodes, as we sail deeply into our three pound universe to understand how we see the world and soon how AI might come to
understand the world with us. Let's think about the word intelligence. You might justifiably assume that neuroscientists have an agreed upon definition for this, but we actually don't. However one thinks about intelligence, I think it's a fair assumption that most of us, when we think about it, assume that intelligence is something that happens inside a single head, in other words,
a brain processing information. This statement seems so obvious that it hardly invites inspection, but if you step back and look at how intelligence actually unfolds in a human life, a different picture can start to emerge. Our thinking is shaped by other people from the very beginning. We learn by watching, by imitating, by trying to communicate, and eventually
by negotiating meaning with the people around us. Even our most private thoughts are built from tools that are fundamentally social, things like language and symbols and shared concepts and cultural norms. So this may sound strange, but this is what we're going to talk about today, and the idea will become very clear. Most of humanity's greatest achievements didn't come from lone geniuses working in isolation, but from really dense networks
of minds interacting over time. When we look at things like science or art, or morality or technology, it almost never makes sense to interpret these as products of individual intelligence, but instead they are collective processes that allow ideas to collide and to form into something and to continuously evolve. So intelligence in this sense may be less like a
thing we possess and more like something that emerges between us. Now, this broader perspective becomes especially important as we find ourselves flinging headlong into the era of artificial intelligence. With every passing week, we're getting AI acting more like a participant. We're getting systems that communicate but also agents that act on our behalf to do things in the world. And soon these agents will collaborate with each other at their
time scales and spatial scales. So if intelligence is social by nature, then building the future world of AI might end up requiring more than just dumping billions into scaling up the training data for these systems. It may instead require understanding how minds relate to one another in the first place. And that's where today's conversation begins. Today I'm joined by Danielle Persk. She's a cognitive scientist who leads
the Human Computer Interaction team at Amazon's AGI Lab. Danielle uses insights from the evolution and development of human intelligence to inform how we can not only make AI smarter, but build AI that also makes us smarter. Here's my conversation with Danielle Persk.
Intelligence in humans is really social, and that is the thing that differentiates our intelligence from other species. Even other species that are closely related to us have similar brain structures and function similar genetics. And what we are really optimizing for is representing other minds. So not only are infants human infants inferring the existence of other minds, but once this thing exists, we are optimized for aligning our minds. Evolutionarily,
we had to cooperate to survive. Infants need to be able to have their caretaker's attention on them to survive, and in terms of being able to learn about the world, once infants have a model of other minds, then they can manipulate it. They can direct their caretaker's attention point what's that, and magically they'll have a label for.
This thing that they're looking at in their environment.
So they're doing prompt engineering.
Great technology. Yeah, okay, so we know that.
You know, throughout the course of human evolution, we became increasingly dependent upon cooperating to to stay alive and adapt to new environment. So it makes sense that there'd be this extreme pressure on being able to predict each other's behaviors to understand our minds, and then with infants, developmentally, we have also the benefit of being able to learn much more efficiently even language itself, from representing other minds.
Okay, so it turns out that we can do a much better job of predicting if we can imagine what it's like to be inside other people's heads. Right, So, if I want to know what some non player character is going to do in a video game whatever, they have certain behaviors. But if I want to know, let's say what you're going to do next, or say next, if I have a model of your mind and what you know and you don't know and all that stuff, I can make a better prediction.
And so you've said that there's information and then information with a purpose, and that information with a purpose really matters.
So you've used the example of like the.
Land rover on Mars not being able to fix itself, and like a wolf that gets its like trap.
Right, the Curiosity Rover went up to Mars. We had spent like a billion something dollars on it. It did a great job on Mars, but eventually it got its right front wheel stuck in the Martian soil and it died couldn't get out. But if you can trast that with a wolf who gets its leg cond of trap. It'll chew its leg off and then figure out how to walk on three legs, which is extraordinary because a wolf's brain didn't evolve for three legs. But it can
figure it out because it's live wired. It has brain plasticity and figure out, Okay, how do I adjust everything.
So that I can survival?
Depends upon it exactly. That's the key. It has relevance to the animal.
Right, So all animals have a drive to survive, a drive to reproduce, But humans also have a drive to align our minds because it helps us cooperate, it helps us survive, and it helps us to learn extremely efficiently. So we don't just model other minds. That would just be the information part. We are optimized for aligning our minds. So it's information with a purpose.
Okay, so aligning our minds this is the key thing and at the center of your interests. And so then you went into looking into AGI. So first of all, tell us what artificial general intelligence is to you.
Well, I think most of the labs that are trying to build something like AGI, they all have their own definitions. None of them are really very good. But the one thing that unifies all of them is that they are all benchmarked to human intelligence. And this goes all the way back to the origin of the field of AI. So in nineteen fifty six, a group of these engineers
and mathematicians got together. They were going to solve intelligence and build thinking machines, and the idea is that these thinking machines would think like us.
It obviously took a.
Very long time to realize, Oh, that's a lot harder than we thought that it was. But now we are back to aiming for something like that original goal of building thinking machines that think like us.
We call it AGI.
Again, have slightly different operationalizations. But I think that we're all running towards the wrong thing. And that's because I don't think that intelligence can exist in a machine. It doesn't exist in individual humans. It's something that emerges from our interactions because we have this drive to align our representations, and of course we all have very different representations.
Right When I used to teach.
Cognitive science, I would teach about this condition called a fantasia, and once every couple of classes a student would come up to.
Me quick In fantations where you can't imagine, you don't have any visual representation on the Yes.
Yes, a student would come up to me and say, wait, you mean there are people who can actually imagine things.
Their mind's eye is not just a metaphor. It's a thing that.
People experience, and they wouldn't know because they don't suffer from other types of death. It's just one of the many ways in which human cognition and experience can very And when I imagine in apple, it's different than when
you imagine an apple. We all have different associations. So when we come together and we have to use words to try to align our minds, there's necessarily going to be friction, especially when we're trying to talk about abstract things, especially when we're talking about things at the bleeding edge of our knowledge, like science.
How do you align.
Your representations when there's not even a word for something. So intelligence emerges as a function of trying to align our minds and oftentimes creating new concepts to achieve that.
Okay, so when you're talking about aligning minds, it's because I've got my whole internal world. You've got your whole internal world that is built by each of our our trajectories through space time. We've had different experiences all these things. So we come together and we've got completely different worlds running on the inside. And that's what conversation is about. We're trying to align things that way.
And there are neuroscientists who measure when people are either communicating in real time or if they're listening to a story, if they're watching something on a screen, you can measure the degree of neuralsynchrony, how close they are to be on the same wavelength, and that predicts all sorts of things like how much they like each other, how much they understood the story, and how much they liked the story, how similarly they remember things.
Okay, so this is what humans do. We get together in conversation all the time and we try to achieve that synchrony in terms of oh, okay, wait, you have a different view than I do on this, here's how we can make progress. This is the Socratic dialectic, right. This is what Socrates love to do, is have these conversations where the truth emerges, something bigger than either person knew when they started the conversation.
And on that point too, I think a lot of us think that we know things, but actually when we're forced to describe something we realized we don't.
Yeah. Actually, in my next book, I'm talking about this as a Potempkin village. Yeah, so you know. The Potemkin village, for anyone doesn't remember, is when it was Catherine the Great of Russia was heading down the river with a bunch of dignitaries that she was trying to impress. She
hired this skuy Potempkin. Actually he was her lover as well as a military general, but she got him to go down the river a long way and build what looked like a facade of a village so that when the ship went by, all the dignitaries would be impressed that there was this village. And he got all these peasants like walk around happily and stuff, But there were no buildings. It was just the front face of the building.
And then when the ship passed, he deconstructed this and went ahead and built another village so that they passed another great village, so it looked like things were really happening there. Anyway, Cognition is often like this, where we think, oh, yeah, I got it. Here's an example that I often use is for anybody listening, take out a piece of paper, and draw a bicycle, draw a bike.
I've tried this, yeah hard, Yeah.
Exactly, it turns out, I mean something as simple as a bike what you see every day. Yeah, you start realizing, wait, actually I don't know exactly where this goes and what's the thing and so anyway, Yes, this is an example of where we think we have deep knowledge and sometimes it's just the facade of something that we know.
Yeah, and you can apply it on all different levels. So you're describing, like the visual imagery might not be very stable, but a lot of concepts are not stable either, and we invent ways of making them more stable. Words are a classic example of that. Once you have a word for something, you can more easily trigger it, you can more easily remember it, you can use it and manipulate it and apply it to different things. But there's a whole class of things that we're constantly inventing to
better align our minds. They're called cognitive technologies. So writing would be one of the original ones. But symbols like math, logic, Yeah.
So unpack that. What's an example of this?
So literally, any word that you learn. Let's go back to apples.
So children see apples, they don't have a word associated with it, so the likelihood that it's going to spontaneously sort of emerge in their mind is very low. Maybe if they've seen a couple then there will be some sort of increased likelihood or lowered threshold. But once they have a word for that thing, that they reliably associate it with it, then anybody who says the word anytime they hear it, now their brain will elicit that activity
and it becomes a more stable representation. That's an obvious example where where there's actually a physical thing that the word can refer to.
But what about.
Concepts like love and justice that you can't see you're saying By assigning a word to it, then we make that stable, and then it become it can become associated with a whole web of other concepts, and that web becomes increasingly stable when when we can.
Make the associations.
More robust, more reliable, and then further when we can invent things like science where we can really validate causal relationships between things, and that makes our representations even more stable.
I see, so human brains interact with one another and work on how do we make these representations stable? How do we get knowledge coming out like a Socratic dialectic, but with with everybody all involved and so on. And so your idea when you moved into this field of AGI, howeveryone wants to define it? What was your idea?
Well, so I left academia, which I absolutely loved, but I felt an urgency to validate this theory.
I don't I mean, it's just a theory at this point.
How do we know whether we are optimized to align our minds?
There's so much evidence to suggest that we do.
But as Richard Feyman said, you don't really know if you understand something until you can build it.
And so I thought, well, maybe I could build this thing. And the moment is just right because AI is taking off again.
It's waking up from one of the winters, and I was watching the scaling have really impressive results with a deep learning, which felt really good because as somebody who had a background in neuroscience, just like, oh yeah, inspiration from brains is actually proving to be really effective. So I moved into tech and started collaborating with the engineers
who were trying to build ever more capable intelligence. I'm now in one of these frontier AGI labs and the thing that we are going to be doing, which I think is really differentiated from other approaches, is try to build the communicative drive. Can we build agents that are optimized for understanding each other's perspectives? And from that, can we get emergent behaviors, emergent capabilities that we wouldn't get from a single model on its own.
So I just want to slow that down. So communicative drive, that's the first time we've heard the term, So tell us what that means.
So communicative drive is the phrase that I use to describe this compulsion that we have to align our minds to establish representational alignment. You can imagine how the communicative drive would interact with other dispositions that humans have. And importantly, you have to think at the population level. So again we have variation for every trait, and some of us are more open, some of us are more closed to experience.
But you can imagine. Okay, so in the case of somebody who's really closed, Let's say that they are.
In some communicative exchange and they detect a mismatch. So somebody is clearly not understanding what they are saying.
They have two choices. They can update.
Their perspectives to the other person's, or they can try to get the other person's perspective to look more like theirs. What would it take to get another person to come to your perspective.
You'd have to create an artifact. You'd have to create.
A word or a piece of art or a theory to get them to really understand and take on your perspective and.
Close that gap.
But if you're a very open person, if you're creative, that might be your default. But if you're a little bit more reserved, maybe you just take on the other person's perspective.
Is it that way or the other way? Sorry? If I'm very open, I feel like I would take the other person's perspective, so you.
Can actually imagine both situations. Yes, So I score very high on openness, and when I'm in communicative exchanges, I often feel like, oh, wow, yeah, that's I've never thought of it that way, or maybe I have, and I want to add all these things and like it's a very cooperative thing. I'm more thinking about the dynamics of people who want to maintain tradition and status quo versus people who want to challenge that. So oftentimes that maps
onto the dimension of openness. So if you see that everybody around you seems to hold a different perspective than you do, you're more likely to conform to their perspectives. If you're somebody who might be a little bit more conservative, not wanting to ruffle feathers that kind of thing.
Well, I'm just trying to stand why I use the word conservative there, because conservative meaning are like iinin exactly. Oh but you're saying, maintain the group traditions. Yeah, okay, got it.
As opposed to being iconoclastic and innovative.
I see, I see how you're using it. Okay, great. So this is the idea is that people are always talking, and depending on your personality type, what you're trying to do is either align yourself with them or them with you or whatever, or meet in the middle. But this, this you feel, is the key to what human societies bring as opposed to looking at individual brains. You know,
the history of neuroscience is all about looking at individual brains. Oh, this is how the visual system works, as how decision making works, how hearing works, whatever. But there's this new feel that's been growing for the last twenty or thirty years, which is called social neuroscience, which is all about, gosh, we've got a lot of circuitry in our brains that care about other brains. So this is the heart of your interest. Is what happens when people are talking and aligning?
And why are we so driven to communicate instead of let's imagine that you and I set down on a bus next to each other, we'd probably chat as opposed to just sit there and deal with our own brains. Okay, so how does this map onto what you're interested in doing in AI?
Yes, so I am concerned about building something that resembles our own intelligence, or something that resembles us because we have all sorts of flaws and biases.
The variability, I.
Think is very useful, and we wouldn't be intelligent in the way that we are without the variability. And you might call some of that variability the bias, the unique biases that we have.
But I think if we try to reproduce.
All of that, we're going to get a mirror of ourselves, and that's not always the most effective way to augment our intelligence. And I should back up and say, why are we doing any of this? Why do we want to build intelligence that looks like us. I think the assumption that a lot of these the people, the engineers, and these labs have is that, oh, of course it's going to be extremely useful for us. It's going to
unlock this unprecedented era of human flourishing. But the assumption that it's going to be really useful for us, I think is taken for granted, and if you really think about it, well, how because a lot of the examples that we have from recent technology and algorithms is that they actually take away our agency. We lose hours to scrolling, we get stuck in echo chambers, we have autocomplete takeaway our thinking, and we're starting to see.
The same kinds of things with chatbots.
We're also seeing that people are using these technologies and very much augmenting their their own intelligence. I feel sometimes like I'm having entirely new thoughts at an unprecedented pace when I'm going back and forth, just like when you were having amazing conversations with other people. We use each other's minds as tools, but you can just do that
at a more rapid pace. So it's not a foregone inclusion that giving the AI more capabilities, making it smarter and giving it more agency is going to be good for us. I think we have to turn that on its head and say, what would it take to make AI that makes us smarter and gives us more agency? And that would be, by definition, something that is good for us. So how do we do that? I don't think that we want to have agents that have their own drives to survive and manipulate us and have all
of the status seeking U situations that we have. But if they were motivated to align their representations with ours, that could actually be really useful for unlocking our potential and for helping us learn. And as we're giving them these capabilities to do that, they have to figure out.
One of the ways that we are able to generalize and continually learn is that we are constantly negotiating meaning and coming up and with the friction of the interactions with each other, we are able to do continual learning because we're not optimizing for one thing, one niche, one environment, one particular problem. We are optimizing for aligning our minds
with many minds, and all of them. These targets are all moving targets, so it's kind of an escape velocity from really focusing on one thing and our ability to do that not only allows us to continually learn, but it gives us superpowers.
So what does this look like for you?
Though?
If you had a world five years in the future that you were able to sort of define where this is going, what's ok Does it mean that there are lots of AI agents and they are talking with humans and they're trying to align their thinking with humans and the humans a with the AI or its look like there's one AI give us a sense of this world?
Okay, So there's at least two important things here. One is that right now agents are not reliable, so they're not useful. And I think the idea there is that they are fundamentally different than llms. They are embodied in some kind of environment, even if it's the digital environment, but we can't yet get them to do long horizon you know, actions in a reliable way, and so they're not yet useful.
Right now, you're talking about AI agents.
Most people have interacted with chatbots and that's what they think AI is, or that's what they think generative AI is. Maybe they know of you know, the image generators too, but a lot of us are interacting with chatbots. Those are llms that are predicting the next text token. Large language models, yes, large language models, but they don't actually do things. Agents, in contrast, can actually take actions and do things on our behalf and in order.
So are lab is.
Working on building computer use agents. So if I want an agent to book me a flight or order me a dinner, I can say that and then it can go off and use whatever websites or software tools to do those things.
That's the hope.
You've seen a couple of these agents, computer use agents come out, and it's really exciting to see them start to do things, but they're not reliable. And because they're not reliable, they might do the thing that you ask them to do one out of ten times, and again that's exciting, but that's not very useful, right.
You mean it's because they make mistakes. It's not the way we would say an employee is not reliable because he's out back smoking a cigarette. Is that they're trying to do stuff is just a clicking on the wrong thing and getting it.
Wrong, that's right y, Yes, So working on making these agents reliable is necessary for making them useful. But we can imagine that unlocking a whole new set of capabilities and ways that they would augment humans because rather than just having a conversation, and conversations can be very useful. All of the things that you do in your daily life, all the things that you're using a computer for, the vast majority of them are probably not worthy of your time.
You're doing things on the computer to actually achieve other things in the real world. So what if you could have agents reliably execute a lot of the things that you're doing.
And in.
Knowledge work, you know, we're using a ton of different tools I call them arbitrary skills, processing invoices or something that everybody does using doing their taxes.
Like, do you really have to becommon expert.
At using these tools or is that maybe not the best use of our human potential, our cognitive potential. If we could have agents that knew how to use all of the tools that we did, that would save us a ton of time, and it would have cascading implications for how people collaborate with other people in the real world. Human collaboration would be different because we'd be freed up
to focus on more creative things, more strategic decisions. Having the sorts of debates that we have to advance whatever shared goals that we have. So this is the sort of first part. We have to just get the agents to reliably click or scroll when we need them to. But if you play that forward, what does reliability actually mean when we have higher level goals.
It's not just.
Knowing where to click or knowing when to scroll. It's actually understanding the goal. And that goal might require breaking the breaking it down into subtasks, and then going and doing all of those things, and there might be many ways of doing it, and there's not necessarily a right or a wrong way of doing it. So at the end of the day, reliability ends up becoming about understanding our minds.
So the idea is if I could have an AI agent that understands my mind, that has a model of me, including what I know and don't know, and what my goals are long term and short term, then it could do a better job at what needs to be done. Because when it comes to a choice point is is oh,
I know what Eagleman wants. He likes this kind of thing, and that might be something that emerges not just from patterns of looking at my behavior, but actually understanding internally, having some theory of my mind.
I think that it.
Would need that, yes, and I think that we would need to be able to interact with it in the way that we interact with other teammates, where we're negotiating, meaning in real time, where we're going back to earlier in our conversation. Sometimes we don't realize that we're not clearly thinking about something, and so having that reflected back and being able to go through this exchange a dialectic. It can refine our thinking, sharpen our thinking.
So this is a thing I've been wondering about for a while, which is if you're looking at something from the outside, you can actually get a lot of data about it. And by outside I mean as opposed to from the inside. If I have a theory of your mind, the question is if I, just if I could observe all of your behavior without knowing anything about what's in your mind, could I nonetheless do just as good a job.
Well, I think that this is what our models of other mind essentially are. It's making sense of behavior. Yeah, it's just that the behavior, again is multiply redundant, and there are many different cues that we can attend to, and we're prioritizing some cues over others, and then it is more efficient for us to represent that there's a mind behind the eyes.
Yeah, very good. Do you see a world where we would have lots of AI agents that are also speeding with one another in terms of communicative drive of saying, hey, this is what I've learned, and I know and blah blah blah, and they develop a better, something bigger as a result of the communication. They have a better understanding of the world because they're talking with one another, because just like humans, each of them is going to have some trajectory through space time.
Okay, this gets into the second thing that I think is really important about how agents can be useful. So the first thing is they can do the digital drudgery for us. They can save time. I call this They can become our collective subconscious because we won't have to spend our conscious time attending to things. We can relegate so the agents can be doing in parallel all of this stuff, so we don't have to become expert in
these arbitrary skills. But they are also as they are doing that for a lot of people, they're learning a bunch of different skills they're learning how to navigate different
websites and use different software tools. And so if I need to for my job learn something really quickly, they can redistribute the skills that they've learned from my teammates, and they can give me the context that I need, not to become expert in that tool, but to be able to establish representational alignment with my teammate who uses
that tool. So they can help coordinate a team's behavior by understanding all of the things that we do, all the goals that we have, all the tools that we use to achieve our goals.
Now, normally that would happen. You go up to Susie and you say, hey, you know, I need to talk to you, and Susie says, no, I use a different word than you're using.
It for ten minutes to even establish common ground.
Got it. But you're saying that AI could help with that interaction like the third person in the room and say, hey, you know what, Danielle, this is what you need to know about and Susie, this is what you need to know. And I noticed you guys are using this same word, but you mean different things by it's that kind of thing.
Yes, that's so yes to your point, I do think that these agents that are working in parallel and understanding our context will probably detect a ton of inefficiencies and how we're doing things, and they will come up with better ways of doing I would hope that they would.
Do that great.
One of the things that we learned is we were trying to train our agent how to use Gmail, is that wow, most people are actually really bad at using Gmail.
In what way?
So we don't know how to do.
The search queries effectively. We mostly just stumble through. And it's the power users who sometimes build hold businesses around you know, the Google Suite and Gmail. They they know exactly how it was designed, all of the affordances that it has, all of the new features, how that can make things actually more efficient.
They get it.
But most people don't have the time to keep up with all of the new things that you can do, and when they sit down to look at their email, they just want to you know, send that email off, or just want to find that thing, and so they're not deeply engaged with learning all of the things you can do.
So if you had this AI agent sitting on the shoulder in that sense, they would let's Sayea, they would teach you. Hey, look, here's the thing you need to know today, Daniel, is that the idea is that it would help you to be a better Gmail user in this particular example.
Well, so, I actually think the longer term goal is that humans are spending far.
Less time looking at screens.
Excellent.
Yes, there are exceptions which include when the actual tool scaffolds your thinking. So I use the example of Adobe Creative Suite. If you want to edit a photo or create a podcast or a video, if you didn't have all of the UIs and all of the dropdowns, yes, then you probably wouldn't even know where to start. You wouldn't even know what was possible.
So some of the tools.
Are actually really helpful in scaffolding your understanding of what's possible, whereas other tools just distract so much from the actual goal. We have to learn how to use the tools to do the actual research that we that we care about. So having agents take over the things that we don't care about is great, and then we can focus on the interactions that really do matter, the visualizations that really do matter.
So let me just understand that. So you're saying, let's say future software are ten years from now. I open it, there's just a few simple things on the menu, but there's lots of hidden power which my AI agent can help me expose and uncover through time.
Yes, and I'm imagining things where Okay, the agent understands my goal, has my context and can generate on the fly only the UI, the button or the search field that I need in that moment. Every time I open my computer, I feel anxious. I swipe to another tab and it's like it's.
Swiping my memory. What was I doing again? And that is that is constant.
You know, you've got most people have a lot of tabs open, They've got a lot of tools open, and it's just our cognition is not meant for all of that. There's a lot of cognitive loads. So if we could simplify that, that would augment our cognition. The other way that agents could really augment our potential is by helping us learn. So this kind of flows from it has a model of my mind, my teammate's mind, It can help us communicate at the right level of abstraction, save
us time. But also if it has a model of my mind and it has my context, it knows the things that I know and don't know, the skills that I have, and the things that I care about. Then say I want to learn something totally new. Maybe it's not just a software tool. Maybe it's something like quantum mechanics, And that's really difficult to understand. You need analogies, But what are the right analogies that are going to work
for me? Well, if it has a model of my mind, then it can, in a personalized way help me come to the understanding that I need to get the big picture, and it can sort of follow that in a way create a curriculum for me that gives me the information that's not too challenging, not too easy, that's right in that sweet.
Spot between frustrating and achievable. Yes, yeah, that's really interesting. That'll have clear implications in education as well, keeping people right at their right spot there. Okay, so let me return to a question, because I just want to make sure I understood coming back to this idea of communicative drive and agents like you and I learning from one another. Will AI agents learn from one another, not just between agent and human, but agent to agent.
Yes, And now you're really bringing it all together. So I think that agents interacting with other agents will be able to learn all sorts of patterns that maybe we haven't yet learned, and detect all sorts of inefficiencies and be more efficient in some ways.
But if they also have a.
Not only the ability to model our minds, but a motivation to then that's not going to be restricted knowledge to them. They're not going to go off and speciate and have all of this you know, intell that we don't have. They're going to try to communicate their insights to us. So chess players are now so much better after we've built AI that's really good at chess. So we can co evolve with this new species of intelligence.
And if it's motivated to bring us along to establish representational alignment, then I think we will continue to get smarter.
Do you see a situation where representational alignment just isn't possible? For example, let's say I came to you and said, hey, Danielle, I really want to teach you about these really important pieces of Mongolian history. And let's say you just don't care about Mongolian history, and I'm trying to tell you the state in this emperor or whatever. It's not going to go very far.
That is a really good question. I think you nailed it when you said just don't care. In the same way that it's really hard to teach a child something that they don't care about, it's not relevant to them. I think it will be hard to establish representational alignment with anybody if they don't care. But in principle, I think it's possible, given that there is ant It might just be a matter of finding the right analogies. And again this goes to your work. There's so much plasticity in the brain.
I think.
Correct me if I'm wrong, But in principle, there's no limit to what we could learn to not only understand, but even have a phenomenological experience of if there is structure to that information.
Yes, but all of brain plasticity is driven by relevance. In other words, do I care about it? I can't think right. So if my AI agent comes to me and says, look, I just realized this great thing about how you could redesign this computer chip in this way, and maybe it starts telling me all this detailed stuff and I just don't care. It's not achieving representational alignments.
That's true, And maybe in some cases, Okay, some people don't care, other people do care.
Go to the people who do care.
But also I guess this also begs the question why do some people care about things? Because it's relevant? So make it relevant. Tell the person as an agent, lead the person to the insights that they need to have
to about something. One of the things that is associated with becoming expert in something is that you really like the things that you become expert in because it is satisfying to understand, and the more bits and pieces of information that you can integrate into a holistic web of knowledge, the better it feels.
So I think that this is my vision for a very happy future, is that.
We're all learning all the time, and the more we learn and the more sort of accurate our shared world models map onto reality, the more satisfying it will be to continue to learn.
I totally agree, and I love thinking about this from the point of view of education, because if you think of this sphere of humankind's knowledge, we know more than any of us could possibly learn in a lifetime. Yeah, So the key is to find out what are the doors on the outside of this sphere that you love or that I love? Given our totally different backgrounds and whatever. Certain things really fascinate me and other things I wish
but they don't. Okay. So if I can enter this door and you enter this other door and we end up learning the same kind of stuff, it's really great. Yeah.
And Isaac Asimov actually really cared about this topic way back in the day, and he did an interview on LERR, which was this PBS talk show thing way back in the eighties, and he envisioned this was before the Internet, and he said, I envisioned a day where there's a huge supercomputer that has all of human kinds of knowledge, and everyone has a cable running from this computer to their home and you can ask the computer any question you want. But his point was you could take your
own inroad into this sphere of knowledge. Okay. So your point is if these AI agents have some theory about our minds, as in your mind, in my mind, everybody is an individual, and then can cast things in a certain way like look, here's a way that you might care about it, then we're all going to learn faster and better.
Yeah.
Yeah, And I also think that this is not just you know, academic stuff, intellectual stuff.
Anything can be interesting.
You know, people in their knitting communities can become really passionate about innovating new ways and in sharing that.
Within a community.
And I think that this also gets to one of the sort of essences of what we care about as humans. It's this idea of optimal distinctiveness. So we simultaneously need communities for belonging.
This is an evolutionarily ancient thing.
It's not just humans, but we need to feel like we belong within a community.
We've got our in group, we've got our tribe.
Not always bad, and maybe it's just something that we will always need as being human, but we also need to feel like we're unique and that we have something to contribute to the community. We are optimally distinct in our contributions. So I imagine a future where agents with models of our minds really allow us to be diverse. They're not flattening the experiences or the capabilities, but they're encouraging diversity and variability. But they're also building mechanisms for
us to align our minds. They're building bridges throughout the various experiences and capabilities so great.
So this leads me to this question that I've been wondering about which is. You know, in political debates that we see lots nowadays, there isn't a representational alignment. If you know someone's on the left and someone's on the right, they end up saying we're not going to have Thanksgiving dinner together instead of saying, oh, let me understand. How can I understand? So where does representational alignment break down? And might AI help us there someday?
I think representational alignment breaks down when two minds have such different starting points, such different sets of maybe analogies, experiences, and also motivations, things.
That they care about, that it's just really hard.
And let's assume that they can get past the initial emotional friction of just knowing that they come from different groups.
That's not trivial.
Sometimes just knowing that somebody is from a different group prevents any sort of establishing of common ground. But assuming that you can get over that, then you just might not have enough overlap in your representations. I suspect that that's very rare. Just by nature of being a human embodied in the way that we are having experiences in the world and the way that we do, there's gonna be enough overlap as an entry point into being able to establish some kind of alignment.
So sorry, you're saying it's rare that people would have such different pints view that can't get there, because it is the case that people don't get there or won't get there. But you're saying if people tried harder, let's say, with political arguments. Yes.
I also think that as we interact with this hypothetical agent of the future that has models of our minds, it will be modeling for us behavior that helps us establish alignment. And so in the same way that we learn from others who model good behavior and then we start to reflect that subconsciously, I think it could nudge us into more pro social interaction.
I totally agree with this. I did a podcast episode on this about this AI research group in Europe that released some chat bots onto a Reddit channel that does debate, and they didn't tell anybody that these were aibots, so they got in big trouble. Everyone was mad about it.
But what happened was these AI bots would come in and take someone who had a particular point of view, and they would take the other point of view and they would discuss, and on this particular channel, you get points if you successfully convince somebody of your point of view. And so it turns out these bots did six times better than humans do on average in terms of changing the other person's mind. So everyone freaked out about this and said, oh my god, these AI debate bots can
manipulate us. And but it turns out the really amazing part is they weren't doing anything manipulative. They weren't lying, they weren't doing anything. They were just better debaters in the sense that they were empathic, they were calm, they presented their arguments well, And I thought, God, we can really learn from that if we teach our children to be better discussings.
Totally.
Yeah, So I love that. I love that point. But what you're saying is, so you think people with differentints of view can do representational alignment. But that's a separate issue, which is that how do we do things culturally and teaching them to be better at conversation.
Yeah, and even having an awareness that other people are different.
It's not personal.
If you disagree or if you have different rituals, different ways of doing things, that's fine. So I think even just exposure to variability diversity can get us part of the way there.
Oh, that's fascinating. Cool. The other thing I was going to ask you about is you mentioned earlier just tangentially said something about archaeology.
Ah, yes, okay, So there over the past two decades, there's been an update in our understanding of the evolution of the types of sophisticated reasoning and thinking symbolic thinking capabilities that humans have.
We used to think.
That they emerged suddenly in what we call the cognitive revolution that happened thirty to forty thousand years ago, and that's because.
The so humans were like other primates and then suddenly, thirty four thousand years ago something happened.
Okay, yeah, and archaeologists were looking at evidence in Europe and the caves and you know, the art and things like that, and it just it seemed like there was a discontinuity. But then they started exploring throughout Africa and using more nuanced methods, and a lot more of them started doing this, and they started finding evidence from as early as three hundred thousand years ago that we were cognitively modern. But what seems to be the important thing
was population density and contact with other groups. So the idea is that our sophisticated cognitive capabilities are latent until we come into contact with each other, until we poke each other's brains. And you see that this evidence ebbs and flows, It appears and disappears as a function of these group donaities.
Yeah, oh fascinating. Okay, So when humans come into contact with other humans, but not just their own tribe, presumably other tribes, bigger and bigger civilizations, slightly.
Different ways of making their tools and their weapons and their jewelry.
Yes, oh excellent, Oh that's beautiful. So, by the way, is it thought that there was some discontinuity where that became possible. In other words, if you stick a bunch of capuchin monkeys together.
They will never they'll never get there, right.
Right, So there's so there's something different.
And I mean, I think it's this stability of our the models of our minds, and I think it's this communicative drive. But you need a critical density of people and you also need the variability.
Okay, so let's just summarize it. So it's having theory of mind in other words, knowing Okay, Danielle has her own thoughts, her own representations in there, and then you add that to the density of people.
And the variability of different groups coming together ability.
That's excellent.
Yeah.
So the takeaway from this archaeological evidence is that becoming cognitively modern was this slow, gradual process over the course of the last couple hundred thousand years.
But it was predicated on pop density yeah, and people coming together.
Yeah.
Oh excellent. Oh I love that. That's so interesting, And it.
Goes along with this idea that we're not just biologically evolving, we're culturally evolving. And cultural evolution does a lot of the work in explaining human behavior.
Yeah, and biological evolution, of course, is super slow, but cultural evolution is so rapid.
Yes.
And actually I think that agents that have models of our minds can help reconcile some of the tensions that we're seeing because cultural evolution is outpacing biological evolution. So what happens when you've succeeded in society, you've done well in education, you go to the workplace and you're supposed to, you know, contribute your intelligence. You end up staring at a screen for most of the day and not interacting
with other people. The infrastructure actually doesn't really support unlocking our potential because of all of the sort of arbitrary things that have happened culturally. We've built these incredible devices and we've co evolved with them, and now they're extensions of our intelligence. But they're also we're also conforming to them rather than the other way around.
True, But they are social, as in social media and so on. I mean, when I'm staring at my screen, I'm interacting with thousands of people in various ways, whether I'm looking at extra Instagram or I'm doing emails. In a sense, it's more social than humans ever could have been.
What this is the problem when the algorithms are not aligned with our well being, with our potential. So I see there are so many mistakes that we've made over the past ten fifteen years that we can learn from and hopefully not repeat with more capable AI.
But out of curiosity, if I'm on X and I see that there are different points of view about this political thing that I happen to care about, then I'm getting exposed to lots of points of view.
Right.
Well, I'm not saying it's all bad. Certainly, Yes, I think.
That having online communities is absolutely a step in the right direction for connecting us. It's fantastic but the way that they are optimized and the attention economy not serving us but serving advertisers is a problem.
But what would you change about let's say something like X. What would you change about social media to make it so it's more optimized?
Well, I think optimization is the problem.
So sorry, I met more optimized for a communicative drive.
I wouldn't think of it that way because I think that you have the agents, but then you also have how they are dynamically interacting with each other. And X is or any social media platform, is one narrow way of facilitating interactions. In the case of X, it's very short form blurbs, and it appeals to the fact of human nature that we are more interested in things that have shock value and things that are negative or disgusting,
and how do you work against that. It's not about informational exchange, although some people lean more in terms of caring about that, and so you do see some of that.
Yeah, so that's interesting. So we are in the sense of population density for three hundred thousand year old tribes. It is the case that you see on X all kinds of points of view that you didn't know existed. And well, I don't know if you do.
Because you're in your echo chamber, you typically tend to not be exposed to a lot of variability.
Well, you know what's interesting, what you're exposed to is the most extreme views of the other side because people in your echo chamber say, look at what this idiot is on the other side of the aisle said.
And the polarize it.
Yeah exactly, Yeah, okay, So wrapping up for today, the key thing is that intelligence is not just about what's happening in an individual brain, but it's social.
Yes, this is true about all intelligences. All intelligences emerge as a function of their environments and interacting with their environments the problems within the environments that the organisms have to solve. The most challenging problems in humans environments are understanding other humans because we have to figure out how to cooperate, We have to figure out.
How to align our minds.
So our intelligence emerges from our interactions with each other, and we continually ratchet up our intelligence by co evolving with each other.
That was my conversation with Danielle Persik. One of the main threads was that maybe we shouldn't be thinking of intelligence as something that's packaged up inside a single head. Another way to look at it is as something that emerges through interaction, through the friction, through the shared effort of understanding another mind. Human intelligence has always been shaped by this social dimension. You can see this from the way that infants learn about the world to the way
that societies build knowledge over generations. What today's conversation invites us to reconsider is the idea that learning and understanding, and fundamentally alignment are really the central features of intelligence. Our ability to model other minds, to recognize that other people see the world differently, that they know different things, that they care about different things. This is what allows cooperation and culture and cumulative progress. Through this lens, intelligence
is about negotiating meaning. Now, if we take on that lens, the future of AI looks very different from what most people are thinking about now, because this shifts the conversation away from only asking how capable AI systems are going to be. Now we're pushed to ask how they participate
in our cognitive ecosystems. In other words, in Danielle's view, how can we develop agents who help us think better by reducing friction and clarifying misunderstandings between people and supporting learning at the right level of abstraction, instead of merely replacing us, which is the doomsayers version of the future. Could AI agents serve as mediators and translators and collaborators. And there's another issue I found fascinating. Alignment is something
humans have always learned through interactions. So perhaps instead of just viewing AI alignment as a technical problem to be solved, we could see it as a behavior to be modeled. Systems that reflect our values back to us might teach us how to communicate more effectively with one another, even
in moments of disagreement. If you're a regular listener to this podcast, you know that I'm obsessed with issues about the brain and polarization, and so this possibility that AI might actually be able to mediate between us and help us get our curiosity back that feels especially consequential. So as AI agents become more embedded in our daily life, the choices that we make now about their design are going to shape how we relate to them and to
our fellow humans. The question is whether they will help intelligence continue to sprout in new directions. In other words, maybe will be even more intelligent as a species thanks to our machines. Go to eagleman dot com slash podcast for more information and to find further reading. Join the weekly discussions on my substack, and check out and subscribe to Inner Cosmos on YouTube for videos of each episode and to leave comments. Until next time, I'm David Eagleman, and this is Inner Cosmos.