Human Intelligence - Best of Coast to Coast AM - 10/25/23

size of a grain of rice. And in that whole ecosystem, most animals that were large, like cnm andes which are like coral, didn't move, so they didn't actually pursue food. The way they got food is they just waited for particles to go buy them, and then they would grab the particles out of the water. But our ancestor was the first creatures that actually found a way to move

towards food and move away from danger. And what's so interesting is this first creature had no eyes, it had no ears, it had none of the standard sensory organs that we have today, and yet it could still navigate. And this is a big clue as to all of the foundational things about how brains work, and the way

we know this is twofold. We can see fossil remnants of these early creatures, and there are modern animals that are quite similar to these early animals, like a modern nematode, which is a really really tiny little worm that has only about three hundred and two neurons in this entire nervous system. So how did this thing navigate without having any eyes or ears. Well, it was a clever trick

called steering that these creatures us to navigate. And what happened is they had these tie little sensory neurons around their heads, and when a smell of something good increased, it would trigger them to steer in that direction. And when the smell of something good, like a food smell, decreased.

The entire brain was based on this foundation of steering, towards good things and steering away from bad things, which means the foundation of brains is based on the idea of classifying things in the world into good and bad. And so that was the very first brain. There was a whole suite of really interesting things that came with this. In that brain, the standard use of dopamine, serotonin, and adrenaline emerged, which has a really interesting relationship to our

modern brains. So there's a whole long journey of how we've tried to understand what dopamine does, for example, and so for a long time people thought dopamine was the pleasure chemical, and has been a lot of research that shows if you stimulate dopamine neurons in a rat, for example, every time it pushes the lever, the rat will endlessly push the lever. And that makes people think, oh, it must be the pleasure chemical.

lip smacks? And the answer is no, they will eat a lot more, but if anything, it looks like they don't like the food anymore. And in fact, if you remove dopamine so they have no dopamine in their brain, they'll stop eating. But if you put food in their mouths, they'll enjoy the food. That's what does this mean. It means dopamine is not about pleasure. It's about craving. Dopamine is about pursuing things that we want. It's not about the actual enjoyment of things when we get it. Let's

go all the way back to a nematode. Now, what are the dopamine neurons In a nematode. There are little neurons that stick out of their head that detect the presence of food around the animal. And then there's another set of neurons and the nemotod's throat that are serotonin neurons that detect when it's actually eating food. So in the very first brain, dopamine was the signal for good things are nearby, and serotonin was the signal for good

things are actually happening. And that basic template between these two neuromodulators exists all the way throughout the animal kingdom, and it's served a whole purpose for steering because and you see this in a roomba. I don't know if you know what a rumba is, the little robot vacum, right, So rumba's work very similarly to the first frame. So

there's something called dirt detect in a roomba. When a room but passes a patch of dirt, it assumes that it must be nearby patches of dirt, so it changes its repertoire. So if it passes a little piece of dirt, it starts turning randomly in the local area because it predicts that if there's a little dirt in one location, there's probably dirt nearby. This is exactly what dopamine did

and does nematode brains. If it detects a little bit of food, it floods the brain with dopamine, which triggers it into an emotional state of what could be called exploitation when they start turning around looking for food for a while, so dopamine levels drop and then it keeps going elsewhere. So dopamine is this initial seeking chemical. Serotonin is the statiation chemical, it makes them stop and rest,

an adrenaline is this big fear chemical. So the basic template of emotion and stress and categorizing the world into good and bad with all present in the steering brain of the very first animal that was break through one.

think about when we look at a fish. And most of the creatures in the Cambrian period were arthropods, so they were huge insect like creatures, and our ancestors were very humble, tiny maybe one to two inch long fish in this whole world, but if you looked in their brain, you would see a brain that has most of the same structures as a human brain, which is exactly what we see when we look at modern fish, which is kind of mind blowing to realize that even in a

tiny fish brain, you can see most of the same brain structures as a human brain. And so the question is what does this brain do. Well, it's a really cool story that is connected to AI where it goes a little bit back to dopamine, where I'm gonna tell a little bit of a story of AI, and then

I'll connect it back to brands. So in the nineteen fifties, people have this grand idea of what if we could train a neural network with reinforcement learning, In other words, we just tell it when it does something right, and then over time I should learn the right decisions to keep doing the thing that quote unquote is right. And perhaps we could teach it to play chess if we just say, hey, AI system, play human in chess, and

when you win, I'll give you a reinforcer. When you lose, I'll punish the system, and if you do it enough times, eventually should get good of chess. And this guy, Marvin Minsky tried this, and it didn't work. And the reason it didn't work is because of something called the credit assignment problem. When you win a game of chess, which previous moves should get credit for being good? You don't know because it could have been the first moves that

were really good. It could have been moved in the middle, it could have been moved at the very end. And so when you get a reward when you win a game, it's actually hard to know what actions should actually be reinforced. So this guy, Richard Sutton, came up with this really clever strategy, which is how all modern reinforcement learning systems work.

In your self driving car, that's using the exact same technique he discovered, which is then called temporal difference learning, where the reinforcer is actually when you think the world just got better. In other words, when you're playing a game of chess and you do a move and you realize, oh, the thing I just did increased my likelihood of winning

by a lot, that's when you get reinforced. And so when we look in fish brains, we actually see exactly this algorithm being implemented, which is it has an expectation of future rewards and it's reinforcing itself again with dopamine when things seemed to be doing better. This all happens in the structure called the basal ganglia. So breakthrough two is reinforcement learning. It was this ability to learn through trial and error, and we see this in fish all

the time. You can teach a fish to do very very clever things through error. You can teach fish to jump through hoops. You can teach fish to remember random shapes of animals and they'll run away from them or they run towards them. So reinforcement learning was break through two. We go forward a bunch further, all the way to

about one hundred million years ago. Our ancestors were early mammals and they resemble most resembled in early creature that would look like today's squirrels, and they lived in a world surrounded by dinosaurs. They were very low on the

totem pole. They were hiding in burrows. The world of sinus our life and our little mammal ancestors evolved a new brain region called the neocortex, and the neocortext gave them a very remarkable ability, which is to imagine the world as it is not in other words, to simulate things, so a mammal can imagine futures and remember paths and can consider alternative decisions. You see this in rast today.

So if you put a ratinum aze and you record neurons and a hippocampus which has these things called place cells, you can literally wash a mouse imagine going down different paths, and you can watch it imagining eating one type of treat versus another before it makes a decision, which is unbelievable. We can literally go into mouse brains and see them imagining things and so simulating. It's this incredible ability to not only learn by actual trial and error, but to

learn by vicarious trial and error. In other words, I can learn before I take an action. I can imagine doing things and then decides to them. It's an incredible break forward, and this is one thing that's really missing in aisystems, which has important consequences. Break Through four emerge in early primates something called mentalizing, which is the ability to think about thinking, and this is another key thing missing in AI systems. So if you go into a primate,

there's some cool stories where primates trick each other. So one famous one is this guy Mlo Menzel was trying to teach a chimpanzee to find locations of food. And you would place a little piece of a treat under a rock or in a bush until and show one champanzee in this group where to find it, and that

chimpanzee would very easily find the tree. But then he noticed something that he did not expect to find, which is when Belle, one of the chimpanzees, ran to the tree, she would share it with other chimps in the group. But then what's started happening is Rock, the top chimp, would take the treat from her for himself. He was kind of a jerk. So what Belle started doing is she wouldn't go to the tree until Rock was not looking, so she was trying to vent him from seeing it.

But then Rock, in exchange, began looking away, pretending he wasn't paying attention until Bell would go to the tree. Then he turned around and run towards it. So then Bell started trying to lead Rock in the wrong direction

and then run back. So the cycle of deception and counter deception is really amazing evidence that chimpanzees and other primates we see the same thing can actually think about each other's thinking, because in order to do that, she needs to understand well, I can trick rock into thinking the rock is not here or the treat is not here if I bring him in the other direction or if I look away. So mentalizing is this really amazing

feat that emerge in primates. It comes from variety of brain structures like the granular prefrontal cortex, and that's a really crucial thing missing in the A systems. For reasons, I can go into and then break through five with early humans was language. And language is only really possible because the mentalizing, because when we're talking, when we talk to each other, we're always inferring what each other mean

when we speak to each other. So this is a big problem with AI, where when we say things to them, if they don't know what we mean, there's potentially dangerous consequences which I can talk about. So anyway, those are the five breakthroughs very fast. I can go into any of them in much more detail. But steering, reinforcing, simulating, mentalizing, and then speaking.

are signals throughout the animal kingdom. But language is unique on two fronts. Language contains what's called declarative labels. And the declarative label is saying you can point to an object and say that is a cup, and now on your head, you know the thing is called cup, an arbitrary sort of symbol. And then we have grammar. That's the second thing. I can start assembling these symbols into

novel phrases, like the words I'm saying right now. And it's really controversial whether any other animals do that, which is assigned symbols and phrase them into novel ways. There's been many studies trying to get chimpanzees and bonobo's to do this with pretty lackluster results. So you know, there's still emerging studies and other animals. But for the most part, it seems like language, at least the extent to which we use it, as pretty unique to humans.

avoid red snakes. It was impossible. But with language, I can share, Hey, this thing happened in the past, and now I'm going to give you that information about it. Or together we can plan a hunt. I can say, hey, let's go to this location. You go left, I go right. When I whistle three times, we're going to start hunting together. And all that type of coordination was not possible with that language. Unbelievably powerful.