Hey everyone. Welcome to Non Trivial. I'm your host, Sean mcclure. There are two main approaches to A I and they are very different in how they think about producing intelligence in the machine. One approach believes in programming, what we know about the world into the machine and the other ignores any pre existing knowledge by relying on large data sets and correlation. Now there's a long standing debate between which approach is best despite only one of these approaches having much success.
The reason this debate still exists is because it gets to the heart of a deeper discussion on the role of knowledge and intelligence. In this episode, I use the A I debate to uncover what I believe to be a deeper problem in how people think about learning and relate this to implications for our everyday lives. Let's get started. OK, let's talk about artificial intelligence in this episode. Now it's not about the reason why I'm going to talk about. This is not so much.
I'm trying to, you know, teach my readers about artificial intelligence. Uh I think there's a deep I'm going to use it as an example to what I think is a deeper pattern at play here. Something that has to do with the way that we think about intelligence, the way that we think about complexity, the way that we think about knowledge.
Uh There's, I'm gonna, you know, talk about two approaches to go about trying to create A I the two main camps that have existed since A I as a field started and how one is taking off more than the other and how people tend to kind of get surprised that this uh particular approach is working so well because it's got this black box as black box aspect to it. We don't really know how it works.
And I'm gonna say that it doesn't really make sense to be surprised because if we actually look at how complexity works and how nature happens that this wouldn't really be so surprising.
And, and the reason why I'm gonna have this conversation uh about how this plays out in A I is because I think it, it, it, it has a deeper truth to it, a deeper pattern at play here that people tend to get surprised at just how complexity actually does work and just what intelligence is and what the role of knowledge actually is in our life.
And then I'm gonna fold that back into something more practical that, you know, even if you're not interested in A I, you could use in your everyday life, a proper understanding of knowledge of skill of intelligence, things like this can help you. I think, navigate through life. So I'm gonna bring this back to a more practical thing. But uh I think the A I conversation in and of itself is pretty interesting.
So, but uh you know, but if you're not into A, I don't worry, it's not really about the A I, I'm gonna use that as an example. Uh It is important for this conversation for A I, I think, I think it's important to uh to think about methods that we could use in A I going forward. But there's a deeper pattern here that I think is uh is more fundamental that can help us in our everyday lives. OK. So let's begin by just talking about A I in general.
So ever since computing started, ever since we could create these machines that could process information, there's always been this kind of analogy to the human brain, right? I mean, people seem to take in information, they do some kind of processing and then they produce these intelligent outputs. So there's always been this question about, well, how smart could a computer be, right? Could we make software that seems to approximate or approach genuine intelligence?
Of course, you know, Alan Turing has talked about things like this. And really since the whole field of artificial intelligence began over half a century ago, there's always been this push to make computers as intelligent as we can and there's always been two approaches to try to do this two main core approaches. One is the symbolic A I approach and the other is called connection is OK.
In symbolic A I, what uh what engineers try to do, what researchers try to do is basically put knowledge about the world into the machine. OK. And this makes sense if I told you, I want you to go make uh an intelligent piece of software assuming you know how to program. And uh uh and I want you to, yeah, to create a program that seems intelligent.
So maybe it's gonna have a conversation with somebody or maybe it's going to be, you know, used in a self driving car or maybe it's gonna be facial recognition. Maybe I remember I used this example a few episodes back. So go ahead and you try to do that. Well, at face value, it seems to make sense that what you might want to do is basically hard code, a bunch of knowledge about the world, right?
Because this piece of into the machine, because this machine, this piece of software is going to have to do something that is intelligent. So it's going to seem to have to, you know, navigate through the world is gonna have to produce intelligent output. So doesn't it need to know about the world? And that's kind of how we think about people, right?
If I'm talking to you right now on this podcast, presumably I'm able to do that because I have like knowledge in my head I have gained in, you know, an understanding about the world and that exists somewhere in my brain and I'm using that to have this conversation. So if I told you to go make an intelligent piece of software, it kind of makes sense that you would want to write a bunch of things into that software that have to do with the task at hand. OK?
If, if you're going to go make a piece of software that does self driving for a car, it would make sense that you might try to pre program, you know, a bunch of things into the machine into the software about roads, about making turns about what a stop sign looks like about what a stop sign means about what you're supposed to do when you see a stop sign and things like that, that would seem to make sense.
The other camp is called connection is which says, look that's too fragile, it's too brittle, it's not flexible enough, it doesn't really approximate intelligent behavior. So what we're going to do is instead just show our piece of software a ton of data about the things that it's supposed to do. And through essentially trial and error, we're going to code kind of the trial and error process into software. We're going to allow it to look at a bunch of things.
A bunch of data, bunch of day budget and millions, if not billions of times and show it the thing that it needs to accomplish. And through that, it will build kind of a statistical model about how to convert inputs to outputs. OK. So it's a very different approach. It's not trying to code explicit instructions into the machine. It's basically just using a kind of a soft statistical model to try to understand how inputs can generally produce outputs. And it's a very much a black box approach.
Nobody in connection is really knows how the black box produces its outputs. They can just code the high level pieces so that it tries a lot of different things. It iterates many times and it just selects the best the best the, you know, wherever it's at, it selects the best piece, let's say of that data to produce the output that it needs. OK. It's kind of a way of thinking about it. So it's a very different approach. The connection is, is very data driven.
It's, it's it's kind of empirical if you will, you just expose it to a lot of things and then it eventually figures out somehow and it really is somehow because no engineer truly knows how to produce the output.
Whereas the symbolic AI approach is more take knowledge that we know about the world, about space, about time, about containers, about, you know objects about, you know, uh the the way that we categorize information, it could be about, you know, well, roads and stop signs and turns about lines and just program a lot of that information into the machine. So that when it, when the software looks at that situation, it kind of knows what to do.
So you got the symbolic A I versus the connection is, and this is very much a difference between, you know, using knowledge to go do the task versus don't use knowledge and just use a very much trial and error approach until you land on the result. And if we look through the history uh of A I connection is, has had far more success than symbolic A I. OK. Um The symbolic AI approach is not that they've had no success, but they've been very brittle.
They're really the causes of the, you know, the A I winter or A I winters that have occurred where there's a lack of funding in A I because it's all hyped up and it doesn't really produce many results and, and symbolic A I has just largely failed to produce truly intelligent outputs in a machine in a piece of software. Whereas connection is, although it's taken a while, has had far more success.
This approach of not trying to code a bunch of knowledge into the machine, but just using large amounts of data, passing it through and allowing the machine to basically figure out correlations between the data and the outputs that it wants to produce in ways that we don't exactly know how, but because it's able to iterate millions of times and we're able to create these statistical models, it's able to put something in place that essentially captures the essence of what it is. Right?
Remember I use that facial recognition a few episodes back as an example. I said, if I task you with trying to go make a face, uh you know, a facial recognition piece of software, you might think OK, I'll take knowledge about the face. I'll take distance between the eyes and the size of the eyes and the size of the lips and the distance between the lips of the nose and the nose and try to get all these rules kind of baked in to the machine. And I said that's going to fail.
It's been tried, it never works. Something about the essence of the face cannot be captured by just collecting a bunch of knowledge by just collecting a bunch of rules about what face is or what we think a face is. But if you instead take the connections approach and expose your software to millions and millions and millions of different faces and just basically iterate it iterate and allow the machine to converge on what it thinks.
The essence of a face is not by constructing knowledge, not by constructing rules, but just finding correlations between the raw data and the output that it knows it needs to achieve that it's able to somehow figure out that connection. OK. That connection is approach has been far more successful and is really the reason why we even talk about A I today beyond just you know, tight academic circles. OK. Um You know, 2011, 2012, we had some competitions that took place is image net competitions.
And these kind of uh uh you know, kind of drug discovery competitions or deep learning was appro was used and deep learning is the approach that that really embraces this connection is uses large amounts of data finds, correlations able to do it in this black box fashion. Uh as opposed to using any kind of pre baked knowledge, as opposed to using rules. And so that's where we are right now with a is connection is, is, is, is definitely winning, is definitely an approach that works.
And yet this kind of rubs a lot, a lot of people the wrong way and it rubs a lot of people. It might be rubbing you the wrong way just from my explanation of the difference between symbolic A I and connection is because if I if you, if you listen to how I'm talking about the connection is approach, it almost seems a bit wishy washy, right? I'm saying? Well, you just take this piece of software and you expose it to a bunch of data and it somehow figures out how to map the inputs of the outputs.
And that, that that one's work. And if you hear that and you don't have any exposure to how A I works. You're kind of like, uh what does that mean? Like, what, what do you mean? It kind of figures out, like if it, it seems kind of hand wavy, it's very opaque what I'm saying.
Whereas if I talk, talk about the symbolic A I approach, I'm like, OK, you take knowledge about the world and then you program that into the machine so that the machine knows something about the world when it goes to do the task. I mean, that one seems to make a lot more sense, right? Uh That's how we think about software. We we code the explicit instructions into the machine and it's able to produce the output.
But the connection is one is like, it almost seems kind of hand wavy, you know I'm saying, no engineer really know how knows how it works. Well, what does that mean? How can an engineer not know how it works? Right. I mean, that's not how we think of engineering.
We think of engineering as you know, designing the system, we know what the different components do, we bring them together in this way for a particular purpose because everything kind of bumps up to, to, you know, into something else causally, right? If you think about an internal combustion engine, you know, or a train, you know, uh uh you know, anything, anything, you know especially industrial revolution type machines, right?
You can, you can peel back the layers, even a rocket engine, even if it's very complicated, you can always peel back the layers and see what part does, what you can see how the thing works.
And what I'm saying is with connection is it's not like that with connection is you're only programming the outer layer, scaffolding, almost kind of like meta level information about like we know we want it to iterate, we have targets in place, you know, we can have a bunch of faces and those faces maybe have names on them or let's say we have a bunch of pictures of school buses and I want it to recognize school buses.
So it has a lot of a lot of pictures that the the software can look at by look at. I mean, it converts it into data, right? And if it looks at a lot of pictures that aren't school buses, it has a label on that data that says this is not a school bus, right? And then if you look at a bunch of ones that do have school buses, it says this is a school bus and so it has labels on that data and so it can keep making guesses, keep making guesses in a trial and error fashion.
And basically it can build a statistical model as it keeps making guesses, right? It's not explicitly using knowledge about what school buses are it's just trying again and again and it keeps changing parameters, changing parameters within the statistical model until it changes them in just the right way. So that every time it tries to guess what a school bus is, it seems to get it right.
OK. So hopefully that makes sense and you can go and look up deep learning and approaches, but it's very much coding in the process of trial and error as opposed to coding in knowledge. OK. So we got symbolic A I versus connection connection is, is the one that's working. And if connection is sounds like it may be is a little bit hand wavy.
That's because you're not doing what we normally do when we think of problem solving, which is to use explicit facts about a situation, use explicit knowledge. And we're not doing what you normally think of in terms of engineering, which is to engineer explicit rules or specific steps that, that, that executed a definite process.
Instead, we're just doing these high level pieces that expose it to a lot of data, uh change a bunch of parameters in a statistical model and do it again and again and again and again, until finally, those parameters just happen to have the settings that produce the output that we want. And that's very much the connection approach, but that makes a lot of people uncomfortable, it makes researchers uncomfortable. Um You can go look at debates that are online, right.
Now between uh connections and symbolic A I, right, probably the most popular would be the one between uh Gary Marcus and Joshua Benzo. I think it was just a few years back at the University of Montreal. And they, they held this uh debate between the two Yau Yau Benzo. Sorry, he is a Canadian computer scientist. Um and he's noted for a bunch of work on artificial intelligence now works uh particularly the deep learning approach.
And he's a professor at the Department of Computer Science and Operations at the University of Montreal. He's very much about the connection. OK. And he's very, he, he, he's laid a lot of the foundational work in that direction. Gary Marcus is an Emeritus Professor Emeritus of Psychology and Neuroscience at New York University. And he's much more uh uh about the symbolic A I approach, right?
So Gary Marcus looks at connection sees things that he doesn't like and then thinks that we need more symbolic A I and Yu Ben is more like, no, it's really, you know, connection is, is the right approach and so let's do it that way. So, so why do we have these debates? Right? Why do we have these debates?
Well, I kind of already, you know, between symbolic A I and connection is which is, which is more than just what's happening in A I, you know, this is really getting into, you know, you could say uh empiricism versus rationalism, right? Or empiricism. It's kind of what you're doing with the connection as a, I just exposing it to a lot of information and you're learning that way.
And rationalism is more what you're doing with symbolic A I where you have specific information about how the world works and you're using that kind of to deductively whittle your way down to truth. This has kind of been an ongoing debate since the beginning of, you know, philosophy in a way or at least it's been around for a long time. But why do we get into these debates? And especially if connection is, is just taking off and winning? I mean, why don't we just focus on connections? Right?
And say, look, regardless of what you might think about how the world works. Obviously, there's something true about connections, let's just focus on that. Well, I think the reason why we get into these debates is because of of how uncomfortable it makes people with, how intelligence seems to work with how complexity seems to work with how knowledge seems to get used or not get used.
I think a lot of people would kind of like symbolic a I to be the approach because it seems at face value to make more sense, right? We're more comfortable with the role of using knowledge. And in fact, if we think about how we go through school, right? We're told, you know, to, to learn this foundation of knowledge, right?
We read these textbooks, we take these tests, professors teach us this foundation of knowledge and then they tell us, ok, then you go out in the real world and you're going to use that knowledge to do great things. And so it seems kind of almost built into us to just think like we use knowledge to do things in this fashion. But I think this is a misunderstanding of how knowledge gets used.
And so even though, you know, when you have someone like Gary Marcus who realizes the limitations of A I and it is limited because it is brittle. I've talked about this before. It works really, really well in a narrow sense. And then when you try to apply it to a different task, uh it can fail pretty catastrophically, right? I talked about adversarial attack examples and, and things like this, but really this is about transfer learning. OK?
This is the idea that if I train a model on one particular task and then I go to do it on a different task. A I doesn't seem to work that well. And yet people do this extremely well, right?
If you teach a child how to play hide and seek and then you say now let's play a game like I don't know, kick the can or sardines or cops and robbers or something that's kind of similar to hide and seek, but a little bit different, they're able to adapt to that new game very well and they can even creatively come up with their own games that again are maybe kind of a lot like hide and seek but have these differences.
Whereas if you look at artificial intelligence today, it's not able to do this, transferring to a different task very well. It seems very brittle. It works really, really well and it's a narrow task that it's designed for, made for. But then you go to give it to a new task and it seems to fail. So people like Gary Marcus will look at that and say, obviously, whatever we're doing with A I right now, even though it has a lot of success, it seems to be missing something fundamental.
And what people like Gary Marcus think is that what it's missing is the symbolic A I approach. It's missing the priors, the prior knowledge that needs to be baked in. And maybe, you know, people like Gary Marcus would say, you know, maybe what we're doing as, as babies, as Children is we come into the world with, with, with this kind of pre baked knowledge about how it works. And that's why we're able to make the transfer to new things, you know, we call that nativism, for example, right?
We're born into the world with an understanding of space and time and translational invariants and different things that we kind of uh think about the world. And, and maybe even beyond that, and we use that to, to go to go to to different tasks, right? In other words, it's not just uh connection is it seems right because if you look at the current connections approach, it seems to be well, it is very data hungry. And if I want to go to a new task, I need a new data set for the new task.
So if I train a model to do hide and seek really well, and I use a hide and seek data set to train my model. And now I want to go do kick the can and well, now I need to kick the can training set. And if I want to go do cops and robbers, now I need a cops and robbers training set and it doesn't transfer just seamlessly from different tasks similar to how people do. It said it just totally needs more data. It's very data hungry, it's very data hungry.
And so and so Gary Marcus is right when he says he looks at that and says like that's not what people are doing. So what's going on here, right? But the idea that the answer would be symbolic. A I that we would code in the knowledge to the machine. That that's the answer that that's the piece that, that the piece that's missing. I disagree with that because I think that's a misunderstanding of how knowledge works when we as humans transfer to a new task.
What we're doing is we're making these kind of analog connections between what we already know and the new thing. So I'm going to maybe as a child learn all about hide and seek and there's going to be something in the essence of hide and seek that I pick up on. But it's not explicit knowledge about hide and seek in as much as it is what I like to call, you know, the essence of hide and seek.
And I get that, that, that that sounds a bit hand wavy but bear with me remember when we were talking about facial recognition, I said, you know, the the statistical models are not learning specific knowledgeable faces. They seem to be learning about the essence of a face, right? And, and even though the word essence seems a little bit wishy washy, I just mean it's extremely high dimensional.
Uh it's, it's it's a very Multivariate kind of, you know, millions, if not billions, if not trillions of parameters to understand what is what is essentially very complex. In other words, we cannot paint an explicit picture of what the definition of a face is.
It's something that lives in a very high dimensional space and only very complex high dimensional solutions will be able to figure out which is of course what deep learning is doing with its very, very high dimensional, extremely high parameter models. OK. So that's what I mean when I say essence. So if I learn as a child hide and seek. I'm learning some kind of not explicit but high dimensional kind of essence of what hide and seek means.
And I'm able to take that model if you will or that, that um pattern of whatever I picked up on. And I'm able to, able to apply to other games like Cops and robbers or kick the can and things like that because those other games also have something similar about that underlying pattern. In other words, I think what people are doing is they're picking up on universal patterns that are shared among many different tasks but do not reside in any one task. OK. I can recognize many different faces.
Not because I have explicit knowledge about what a face is, but because I have some universal understanding or pattern that I've picked up on and what makes a face to face. And I can use that for many different faces and many different objects and many different things. If I learn how to play hide and seek, I'm picking up on some connective tissue that exists between hide and seek and cops and robbers and kick the can and other games.
There is some unseen kind of universal structure that we pick up on. And we do this all the time when we make analogies, right? Things that might seem different or disparate on the surface kind of superficially actually have a deeper connective tissue between them that we tend to pick up on really, really well as humans.
And so the knowledge is not explicit, the way it would need to be with symbolic A I because symbolic A I is trying to, to take specific knowledge about something about the world and then code it into the machine. And then use that. In other words, it's almost instead of you having task specific data set, the way connection is might do it. We're using task specific knowledge to code into the machine.
But what I'm saying is knowledge isn't task specific knowledge is deeper, it's universal, it's not anything that exists for a particular task. It's something that exists for many, many different tasks. And I think that's why Children are able to transfer their learning. Humans are able to transfer over. They're learning, humans are able to make these analogical connections between things. They don't require, you know, a, a huge amount of different data sets for each task.
And they don't require explicit knowledge about what needs to be accomplished. OK? So now, now you might step back and say, OK, but what if symbolic A I tried to program in the universal knowledge? In other words, maybe the the the symbolic aspect is that universal structure. In other words, don't try to hard code explicit knowledge about the world like task specific knowledge but try to code in uh you know, more universal knowledge about how things work.
I mean, after all, you know, scientists and researchers are always learning about universal knowledge, right? They understand, you know, they they build taxonomies about the world, there's higher levels of abstraction in there and many things will subsume into those higher levels of abstraction. So what if that was the knowledge that you coded in? But this is still a kind of explicit knowledge.
The problem with this is that the role that priors play in learning is that they're never really supposed to be correct. In other words, you're not resting your knowledge on the priors, the priors are there so that they can always be updated. We can think of this in kind of the bees in sense, right?
The priors, you know, and again, when I say priors, I mean, the prior knowledge that you have the knowledge that you go into a situation with, it's not there to be correct, it's there to be destroyed, it's there to be constantly updated. It's the process of learning that matters. So whatever knowledge you have in place is there to be dynamic, is there to be ephemeral. It's there to be killed, it's there to be destroyed and constantly updated. That's the role of priors in learning.
So what I think symbolic A I gets wrong is that it assumes priors are these more static things that we know supposedly know about the world. And that if you were to put that into a machine, then it could use that knowledge to navigate around and to produce intelligent outputs. But I don't think that's right. I think priors are supposed to be always updated, always changed.
They're only there as, as a kind of placeholder in order to strike a juxtaposition between what we think we know, which is naive and wrong and then what constantly gets revealed to us and it's the delta between what we think is right, what is wrong and then what is actually correct that constant juxtaposition that we use to update and update and update. And yeah, we could say, you know what, but but don't those priors get better and better and better.
But when are we going to do, when can we really say we know what that prior is or what it looks like or what even constitutes the, the the array of priors that you would have to code into the machine? I don't think you can ever get there. I think what you need is the dynamism. I think what you need is a constant trial and error so that the machine is always able to land on a set of naive priors that constantly get updated as you go.
And that's much more in line with connection is because connection is, is putting in place the trial and error process. It, it, it, it almost has implicit priors if you will as opposed to explicit. But those implicit priors are always being updated, they're always being changed and there are always things that we can't ever point to because again, we've got that cause of opacity.
It, it you know, the the connection is if working correctly would be able to learn the universal underlying pattern that isn't explicit, that can't be, you know, defined explicitly that most definitely could not be coded because it's constantly in flux, it's always being updated. It is the process of trial and error that uses structure on an ongoing dynamic, ever changing basis. OK. So I hope that makes sense.
So I think, you know this, this idea of symbolic A I like I, I understand why people would think, hey, we know about the world and we must use that knowledge in order to, to go do great things, you know, similar to kind of that academic narrative, right? We learn things in textbooks, we learn fundamental concepts and then we go use that knowledge to do things. I think that's backwards in terms of directionality. I don't think that's really how it works.
I'm not saying it's not worth learning things about the world, but I don't think we really use that knowledge to then explicitly piece things together to go make decisions. I think it's much more ad hoc much more trial and error. And that trial and error process allows us to pick up on deeper patterns that are not explicitly contained in any one thing. But they'll allow us to make connected connections to find connective tissues between different tasks. And that's why we can transfer learning.
OK. Now, to be clear, the current uh you know, level of connection is, is, is nowhere near good enough. We could argue, right? Even with things like chat GP T being very successful, it, it still makes a lot of dumb mistakes. Um And, and the truth is, is that connection as it stands right now is still very data hungry because what it needs to do is every time it needs to go to a new task, it needs a new data set. Well, that's also not correct.
We shouldn't need to go to a bunch of different data sets because as per the mechanism I'm talking about, if you can find the universal underlying pattern, you don't need a new data set for every new task. You're using this, this kind of knowledge about underlying core patterns that are shared among many different things to go transfer to many different tasks. So it's not, it's not data hungry in the sense that you need a bunch of new data sets.
But it's also not explicit knowledge that's getting coded because it's constantly in flux and it's always changing. But you could say it is still kind of data hungry because the way that you find underlying universal patterns is you look at many, many, many different things and you notice what does not change. I've talked about invariants in other uh episodes.
So the way that you would pick up on those fundamental underlying universal patterns is you would as per trial and error, you'd be trying a lot of different things. And then you, you know, so let's say we're, we're playing hide and seek and sometimes I'm hiding, sometimes I'm the one doing the searching. Sometimes, maybe there's teams and there's not.
And there's all this different variety and among all that kind of changing, there's certain things that don't change certain things that are invariant and it's those invariant pieces that constitutes learning. That's what learning is and what you're learning is what is truly universal. Those things that don't change that much when everything else does are the pieces that you could then transfer to other tasks. And so this means there is still a decent data requirement. OK?
I, you know, sometimes people in a, I really like to say uh you know, humans don't use a lot of data. So there must be something wrong with the data hungry approach. I don't really get that. I think humans do use a lot of data. There's a ton of stimuli coming our eyes all the time and we have to filter and compress it. We have to create abstractions in the mind to deal with just that amount of data. I think there's a ton of data ever since we're right from the time we're born that we use.
But I don't think we're using a lot of data. The way it's currently used in connection is currently where a different data set is needed for every different task. I think we're using a lot of data because we need a lot of different examples and variation uh in order to spot the invariance is in order to spot the parts that don't change and everything else does. And that's the way that we pick up on the fundamental truths, right?
Uh The fundamental kind of parts that don't move that can then get transferred to other tasks. So it is using a lot of data, but it's not because we need a lot of different data for every different task. We need a lot of different data to note what doesn't move to find something universal and then we can use that to transfer to other tasks.
OK. So just to wrap that part up, um let's just step back a little bit, you know, we're talking about A I as an example of kind of, you know, intelligence, the use of knowledge, all these things that have deeper implications. And we've got the two main accounts to A I, we've got the, the symbolic A I which tries to bake in explicit knowledge about a situation.
And then we've got connections which takes more of a kind of a trial and error approach to run over many examples and try to converge on something, right? And looking at, you know, throughout the history of the last half century of A I, you know, connections has definitely come out on top. It definitely seems like the approach.
And uh I think this rubs a lot of people the wrong, the wrong way because it doesn't, you know, align with kind of how we're taught in, you know, that that kind of academic narrative of a foundation of knowledge is how you go do things. It doesn't align with the way that we're told about problem solving, about how we supposedly use deduction to whittle things down to specifics and things like this.
And, but I think, you know, this should not be surprising that connection is, is the approach that works better. I think it's much more commensurate with how nature works with trial and error with the fact that there is no use of explicit knowledge. OK. Uh We should not be surprised about this. And I think this is uh you know, a bigger comment about things beyond just A I, I think, you know, this is the empiricism versus the rationalism debate.
I think this is the misapplication of rationalism where people tend to think things are more analytically or should be more analytical than they are. We kind of pretend that we have access to information. We don't, we like the little causal stories that add up, but that's not how complexity works. So I think connection is uh in A I is far more aligned with that, but it is also brittle.
There are also um aspects to it that uh you know, it's nowhere near A G I, you could say, despite uh you know, the, the, the major advances over the last couple of years, you know, things like chat GP T and stuff like that, it still makes a really dumb mistakes. And uh and, and really the dumbness is centered around this poor transfer learning. It's not very general, it can't go to a new task.
But I don't think it's because we need symbolic A I I don't think it's because we need to take explicit knowledge about the world and bake that into the machine. I think we have to allow for the trial and error but allow the trial and error to land on more universal patterns that can then be used across different tasks.
And uh and in some sense, that might still be kind of data hungry and that you still need a lot of examples to spot the invariants, but it shouldn't be so data hungry that we're using a different data set for every task, which is kind of what we're doing currently with connections. So connection is still has a lot of problems, but I still think it is much more in line with how nature works with how complexity works.
And we should not be surprised even though it might sound black boxy and hand wavy at times, we should not be surprised that this is the approach to work towards intelligence and more broadly speaking, that we should not be surprised, uh, in everyday life about these things. Right.
When we solve hard problems, when we find out the successful person, um, maybe didn't have the university background and just kind of, you know, scraped by and did a lot of trial and error and then all of a sudden somehow figured it out. You see this in the workplace all the time, you see individuals who, uh, maybe try it, it took a lot of different uh you know, non kind of standard approaches to trying to solve something. And that's what ended up working out.
You know, people get surprised by these things all the time. They get surprised by trial and error. They think trial and error is almost like a less rigorous way to go about problem solving. And it's not the only way to make truly hard problems tractable. It's not about explicit knowledge, it's not about deduction, it's not about analysis, those those only work in very simple systems.
So I think the A I debate between symbolic A I and connection is just kind of smacks of the same old repeating surprise pattern that people have uh of of not appreciating how nature actually works. OK? And, and we can go on and on about that. But of course, I talk about this all the time and non-trivial. So that's why I want to use A I as an example now to wrap it all up. What are the practical aspects of this, you know, what does this mean for everyday life?
Well, with a proper understanding of, of how, you know, knowledge actually gets used and how hard problems actually get solved. You know, I think this comes down to people tend to think that in order to do a thing that's quite specific, you must need specific knowledge to do that thing, right? And again, this kind of goes back to that academic narrative, right? Like if you want to go learn uh you know, physics, I guess you go studies physics.
If you want to go study chemistry, chemistry, biology, biology, if you want to go do something even more specific within that, of course, you know, it's always more specific, you gotta go learn that specific thing. If I want to go paint in a certain style, I guess I better go learn how to paint that style.
If I want to go, you know, uh you know, exercise a certain way, build a certain type of muscle or be athletic in a certain type of fashion, I better go just learn how to do uh you know, exercise in that fashion, whatever it is, we all have to do specific outputs in life. We can't be just complete generalists right? There are, there's always a a level of specificity in what we want to do in the task we want to accomplish and what we want to become. But we tend to think that in order to do that.
I guess that's the specific thing we must learn. And that's analogous to kind of thinking that you need explicit knowledge to do the thing. It's also kind of analogous to uh thinking you need a certain data set, right to train to do the task. In both cases. It's, it's an incorrect understanding. In my opinion of how knowledge works to create specific outputs in life does not mean you need specific inputs.
Remember uh a while back a few episodes, you know, I was talking about you could find, you know, let's say calcium or something in, in a healthy individual and say, oh, so I guess calcium plays a role in health and let's say, OK, that's all true. And then the the the conclusion that you draw from that is, I guess I should just ingest a bunch of calcium. OK?
That, that is a big leap and that's assuming that the inputs, that map to outputs is extremely kind of one dimensional or kind of very deterministic, right? Just because you see something in the output does not mean that was the path to get there. I talked about the pattern is not the path and things like this, right? Uh And I think that's what we're doing. If you want to go paint in a certain style, you think you need to go learn that painting style.
If I need to, you know, just learn a new language, I guess I should just go learn that language. We, we tend to think that we have to do the explicit thing to produce the explicit output. And I don't think that's how nature works. I think you can do a lot of different things from a lot of different angles and that, that's a better way to do it because that allows you to pick up on the fundamental universal pattern that you need to then go do the explicit thing. OK?
Um If I want to go, you know, paint in a certain style, maybe eventually you really hone in on the, on the, on the very specifics of that style. But especially when you're beginning and learning, you should be painting all different kinds of styles because whatever you need to do something in the explicit sense in, in the, in the, in the narrow sense must be learned in the very non narrow sense. If that makes sense, I like to call this multi multiple angle learning.
Sometimes ma l where you need to kind of hit it from a lot of different angles. You're not trying to hone in on anything too specific other than just let's say this is about painting or this is about exercising or this is about learning languages, but you're doing it from a lot of different angles because all those different pieces have something to contribute to the thing that you need that universal pattern that you're trying to pick up on OK.
So I think the very practical application of understanding, uh you know, let me wrap the whole thing up. So we, I used a I as an example. We've got the symbolic A I and we've got the connection is we've got the connections in the way that it is definitely working and taking off. Although it surprises a lot of people because we kind of have this idea that knowledge is supposed to work in this 1 to 1 fashion. It's supposed to be kind of explicit.
And so maybe the symbolic A I was a thing that we're missing to kind of get out of the brittle aspect of A I and improve it. This should not be surprising, it should not be surprising that connection is, is the way. And then I said, you know, look, it's not really about A I, it's about learning in general, it's about intelligence in general. It's about complexity in general.
Complexity works by uh you know, sampling from many different possible angles to arrive at kind of a universal structure that doesn't exist in any one thing but exists in many different things. And that's how we make those analogical connections between things and do a lot of different things. Great, even if specific. So the real world practical application of this is to stop thinking in terms of 1 to 1, stop thinking, you need a specific type of knowledge to do a specific thing.
Again, I'm still I'm still talking about doing a specific task. And when you look at someone do a specific task, you assume that they must have specific knowledge about that thing. But they actually think they have a lot of varied knowledge, varied experience that came from many different angles that converged on something universal that then got applied to that specific thing.
OK. So no matter what it is, no matter how specific the task you want to accomplish, we all want to accomplish specific things. I think you need to do it from many different angles. Stop thinking of a of a kind of 1 to 1 mapping between input and output, you know, lower dimensional specific outputs get created from high dimensional, many different outputs. OK? And I think the A I example of the use of knowledge is is an example of that. OK? So I hope that made sense.
I think there's a practical reason. I used A I as an example, but then I funneled into a real life application because I think this is about knowledge and its use in general. OK. So as always, thanks so much for listening. Uh I said it last time, I'll say it again. Uh If anybody could just take a quick look over at Apple Podcast and give a five star rating if they like what they heard. It really helps a lot, go ahead and do that.
But as always, thank you so much for listening until the next episode. Take care.