#276 - AI, Metacognition, and Neuroscience

Hello and welcome to the AI in education series on the EdTech Podcast, sponsored by Nordang. In this series we're looking at how we ensure that of gravity when it comes to artificial intelligence. My name is Rose Luckin. Professor of the Learner Centre of Design at UCL's Institute.

Research, an organization that uses AI ethically to make education and training better for organisations and individuals. In today's episode, we're going to look at the complex intersection of artificial intelligence and neuroscience. two domains in education that completely fascinate me and form a strong through line in the work that Educate Mentors Research and Nord Anglia Education have been doing together with respect to metacognition.

So metacognition, neuroscience and AI aren't just buzzwords, but areas of intense research and innovation that when we understand more about them, mean that we can help learners in ways that until now have been unavailable to the vast majority of people. The technologies and approaches that study these domains and help us to unlock a greater understanding of what's happening in terms of our brains when it comes to learning.

a'r metacognition a'r ffordd yw'r ffordd yw'r ffordd yw'r ffordd yw'r ffordd yw'r ffordd yw'r ffordd yw'r ffordd that we collect as people learn can be informed by what we understand about metacognition. And so we can have neuroscience informing the way we do analysis with AI, but I know Steve, when I come to you, you've got some very interesting thoughts that you've written about when it comes to artificial intelligence and metacognition. So I want to pick up on those with you in just a moment.

So I should say something about who I've got in the Zoom studio with me today because I'm very excited. um to have two people with me who I know are going to um provide a really interesting discussion. I have Dr. Steve Fleming, Professor of Cognitive Neuroscience at University College London, and Jessica Schulz. Academic and curriculum director at San Roberto International School.

And so let's get straight down into it. Neuroscience and AI are well-respected fields and there's a huge amount of research underpinning their investigation practices. what we can learn from these fields. But they're also potentially quite buzzwordish.

And I know several years ago I worked on a paper with my colleague Murtlu Kukarova, also from University College London, looking at the impact of language on the way that And it was informed by previous research where there'd been a significant uptick in people's respect for a piece of work if it talked about neuroscience. And so we look to see whether artificial intelligence, so this is obviously pre chat GPT, would would bring about a similar

increase in people's respect and and the believability of what was being said. And interestingly enough, it did, but not to the same extent as neuroscience. the mention of neuroscience and in particular having some kind of image of the brain definitely had an impact on the way that people reacted to papers, as did AI, but not to the same extent. So I think, you know, we have to be careful because

Both artificial intelligence and neuroscience are terms that within education one does see misused. People talk about using AI when they're not, and people talk about products that have evidence from neuroscience when often it's perhaps not. quite as clear cut as we're being told. So, you know, it's not it's not all Totally transparent when it comes to these particular areas. But Steve, you published your book, Know Thyself, The Science of Self-Awareness in 2021.

with its lovely tagline about not being about psychology but about the science of human excellence, which I love. I also love the book. I recommend it to people wherever I go. You know, this is a great read. If you want to understand more about metacognition, if you want to understand more about its roots, this is a really readable book. So I am recommending it to our listeners as well. It's definitely worth dipping in. So Steve, I'd really like to hear from you.

A little bit about yourself and the work that you do, because I think our listeners will find that particularly interesting. But I'd also like to know a bit more about Where you see the crossover or the intersection between artificial intelligence and neuroscience. when it comes to understanding issues like metacognition. Yeah, well, thank you for the introduction. And I confess I have been guilty of producing some of the brain images that were perhaps being used to impress um

people in other fields. So what we do in my lab, which is at UCL at the Wellcome Center for Human Neuroimaging and the Department of Experimental Psychology there. We are a cognitive neuroscience lab, so cognitive neuroscience is the intersection between psychology and neuroscience. And we're very interested in understanding how the mind works, how we learn things, how we perceive the world. How knowledge is represented, um, how we form beliefs. About ourselves and other people.

And a particular focus in my lab is on metacognition. So the ability to think about your own thought processes, to reflect on your own skills and abilities, and to know what you know, know what you don't know. And we're interested in how the brain supports this. Like it's kind of a remarkable thing that the human mind does.

this ability to turn our this this capacity to think on ourselves. So we're able to self-reflect. There's debate over whether any other animals can do this. They probably can do it to some degree, but humans seem to be especially good um metacognition. And as you alluded to in the introduction, Rose, uh metacognition is this

It's kind of like a superpower because it allows you to do much more in other domains as well. So if you have good metacognition, that means you're aware of gaps in your knowledge. You can go and seek out new information. You can essentially improve yourself. ac mae'n ymwneud â'r metacognition yn ymwneudol yn ymwneudol yn ymwneudol yn ymwneudol. Felly, os ydych chi'n ymwneud â'r metacognitive'n ymwneudol, mae'n ymwneudol. to self guide their learning, to know what to study, what not to study.

structure their own um study time and so on. And so we think it has very important intrapersonal effects. So kind of controlling yourself, guiding your own learning and and study. But we also think it has important social consequences as well. Because if you have good metacognition, that means you'll be able to work well with other people.

because you'll be able to communicate your confidence in your knowledge appropriately, you'll be able to seek help when you need it, to realise you might have made a mistake and be able to admit that to colleagues and collaborators. So we think metacognition is quite connected to this idea of intellectual humility. So kind of being aware of your weaknesses and seeking out those two um seeking out those areas where you need to improve. So that's the psychology side.

But what we're what kind of makes our lab, I think, uh unique in this space is that we're also taking that side of the psychology and combining it with tools that we can um harness from brain imaging to understand how metacognition is supported by the human brain. So we've made a number of discoveries along this line where we've isolated networks in particularly in the prefrontal cortex, which we know develops um takes quite a long time to develop during childhood and adolescence.

Networks in that in that area of the brain are important for self reflection, for understanding your own skills and abilities. And in particular, if you have if you have damage or if you get put in situations which impair the functioning of the prefrontal cortex. then that can impair your metacognition. So we're particularly interested in why that kind of thing happens and trying to develop models of how how metacognition works. And for that, that then is intersects quite closely with

AI, because to come to your specific question about the crossover here. So I think there's for me, there's really two main crossovers. So one between neuroscience and AI. So one is using AI tools as a model of how the human brain might be working. So people have done this quite successfully in the case of vision. So the best computer models of vision, where they're able to essentially categorize images, generate images themselves. So all these amazing generative AI tools that we can

prompt them to generate any cat on the moon and so on, and they just do it. That kind of architecture, convolutional neural networks that involve generative processes, seem to be quite actually a good model for how the human brain might be perceiving the world and doing things like imagination and dreaming and so on. So that's quite an interesting correspondence because progress in AI has helped us understand how the human brain might work in that domain.

But that so that's one crossover and that's um been more successful in some areas than others. It's been quite successful in this area of vision and perception, but perhaps less successful in explaining thought. And then I guess the other crossover is how humans interact with AI, which is completely different. It's rather it rather than saying AI is somehow a model of how the brain works, now we're thinking, okay, how do humans perceive, um, interact with, exchange information with AI.

And we've got r uh quite interest in this recently in our group and I'll say one thing about it and then I'll stop talking because I realize I've been talking for a while. But um essentially um we've been very interested in this this process of how people perceive

information that's coming back from an AI assistant, how confident are they about the information they're being being told essentially? And it feels to me that that has quite big implications for how students are interacting with tools like Chat GPT. Because what we're finding in a lot of that research is that people tend to think the AI is more confident than an equivalent human making the same statement. And that has consequences for trust. People seem to be trusting that

artificial assistant more than a the uh the a human making a similar uh giving them a similar piece of advice. So I think there's a lot of work to be done on that front, essentially kind of understanding how If these are things are going to become more and more prevalent in the classroom and on campus and so on, like how are students perceiving the the

the the origins of this information. Are they just trusting it at face value or are they questioning it? And that's where metacognition comes into play. Because if you're able to question what you're being told, then you're perhaps putting yourself in a more critical, uh self-critical position and not just getting essentially just parroting what these AI systems are telling us.

So I think that's a whole different area of intersection which is probably relevant for what we're going to talk about today. That's super interesting and you're right. That's definitely highly relevant. There's a lot of things I'd love to pick up on in what you've said.

But before doing that, I'm just going to bring Jessica in. Jessica, we'd love to hear a little bit about you and the work that you do. I know you have a background in psychology and in uh educational counseling for students. So Uh but you're a a a principal in a school and I'd love to hear a little bit more about what you've been doing and also

um any work you've been doing in metacognition since that's clearly an area of connection here and it's one that I think the audience would be really interested to hear a bit more about as well. Oh, sure. So the work that I do at school at this moment, I'm currently the academic and curriculum director at San Roberto International School in Monterey, Mexico. And um I was a middle school principal for nine years before that and I've taught I B English. So

Um, I've been really interested in, you know, language and just being able to be part of the teaching and learning process and just with that psychology background I got really engaged with how that intersection where we see these kind of crossovers. And when you realize that a lot of these skills that we need to be successful, if you look at, you know, the successful habits of students, there, you know, there's a lot of writings and readings about these pieces.

And these are teachable skills. Our school had adopted The 16 Habits of Mind by Arthur El Costa when I had first joined the school. And we talked a lot about this being visible and being intentional in the thinking and being able to teach that first to the adults so they can model that and have students do that as well. And the thing is is that we noticed an incredible impact.

for students to be able to verbalize, identify. I have some amazing examples of this and I want to build on that and I think AI is

is one tool in a teacher's toolkit to be able to do that. I think that the more intentional we can be, the more we're able to inform the chatbots. They're only as good as the prompts that we give it. That comes from humans. And some of the ways that we use that in education is we can give them the prompt.

We can teach students how to do this and then we can ask them to analyze, is that prompt accurate based on what we've learned in class? It's not meant to replace education or teachers or the practice. It's meant to enhance what we do. And I love this question because I explore it so much in my work. And I think that the core of this intersection is this deep human need to know why and what implement uh what tools we can implement.

to arrive at solutions. It's one of the first questions kids ask when they require language. Why is the sky blue or anything new? And humans are naturally curious. And I think that curiosity is crucial for meaningful and sustained learning over time. And I do believe that AI tools can enhance that because it can spark creativity. It can give us models based on what we input and then re-energize that creativity as we explore it.

Um several years ago our school adopted a one for one program, meaning that our students bring their own device at school from third grade up and we use iPad carts for the lower lower grades. So little by little we are encouraging an understanding uh through our digital nation program, the program that we had created that um

not only engages with technology but talks about the responsible use of both equipment and software. I think we need to set parents, students, all stakeholders up for success and support clear communication. It's our responsibility to teach students. how to use these tools and the potential that it has for learning and reflection. And this is our third year being part of the Digital North Anglia Metacognition Research Project.

and we're exploring how metacognition uh in interacting with students and teaching these fixed learner ambitions, which were a natural transition from the habit of mind. um that are really helping us to to what is going to be setting students up for success at the next level, be it in their interactions within the world, in the job market. And so six learner ambitions um are helping them

uh better connect with their own social, personal, academic outcomes. And that's being collaborative, compassionate, curious, creative, committed, and critical. So we pair those up with the tools that are online, the teachings that students um uh engage in with the different lessons that we've intentionally planned for them.

using one of um the Harvard's uh University Project Zero's thinking routine. So we pair each one of those up and then we collect data from different tools. I think one thing is being able to share these incredible anecdotes of what we're seeing. Another thing is to be able to measure that.

So I'm really excited about all these different things that we can talk about today because there is a really important need for us to blend these two fields so that At the end of it, we have a product that we're not only proud of, but that it reduces any misconceptions. that we have about AI and any misunderstandings of how positive it can be when there's a lot of fear surrounding it. Um, and that's based on um maybe missed opportunities to know more.

That's really interesting. And I think the um importance of communicating accurately. to people, particularly in areas of education, where they don't necessarily have a huge understanding about AI. Why would they? It's not something that they they've needed to understand until now, and yet now they do, and we need to make sure we communicate it very clearly to them in terms of what AI can do and what it can't do. And and I want to come onto that for sure. But before doing that, I just want to

loop back a little bit on something both of you have said. So obviously, Steve, you know, I've read your book, you wrote your book. So my my understanding of your book is tiny in comparison to to to yours, obviously. But I was very struck by a couple of things that I think are are really relevant to this discussion and and you've you've alluded to one already and that was I was fascinated by the proximity of

The way in which, and I'm not going to express this correctly, I'm sure. So please correct me if I'm wrong. But basically, the overlap. within the brain of the structures that support social interaction and metacognition. So knowing other people and knowing ourselves, but interconnected, which I was fascinated by and it makes so much sense.

that that's the case. And and that feeds into what you've said about being better at yourself, understanding yourself, but also better understanding other people. But it's also really nice in the context of the work that Nord Anglia have been doing around those six ambitions, one of which is collaboration. So not only is it about ourselves, it's about collaborating with other people. So and the other thing that comes to mind about my experience of reading the book, Steve, is

towards the end of the book, you do talk a bit about how you could improve metacognition and how you might teach metacognition. So I'd love to come back to you and and then um and and then back to you again, Jessica as well. About this, that that that really important relationship between social and understanding oneself. And then how do we best go about increasing our capacity for something like metacognition.

Yeah, I I mean you characterize it very well, this link between um self-directed metacognition and thinking about others. And this idea has uh origins in philosophy of mind going back quite a few decades now. So the general proposal that the way human self-awareness works is that essentially at some point in our evolutionary history we learnt to turn the skills we have for thinking about others Which presumably we needed in social groups and so on.

on ourselves. And so there's a kind of idea that in a sense, metacognition about ourselves. may well be secondary to a broader social skill. And People like there's a a philosopher Peter Carruthers that have b has been defending this view for a number of years now, but it's only recently, probably the past few years, that we've got some empirical evidence to start to supporting this idea.

And so I would say that the jury is still out to some extent. There's some people who say that there are different processes that You know, they recruit they do recruit somewhat

overlapping, but also somewhat distinct brain networks. So there's you can make the case both ways. I think that there are good reasons at a more computational level. So if we think of the mind as a system in implementing various computations, then We've done some work showing that the kind of problem that the mind needs to solve to reflect on its own behaviour, its own skills and abilities is a similar computational problem that it needs to solve to think about others.

And there's also really nice developmental evidence, which is also obviously very relevant for education, that when kids start passing tests of um thinking about others, so the classic theory of mind tack tests. then that's around the same age that they also start passing metacogni uh tests of metacognition around the age of four or five. Rydyn ni'n ymwneud â'r hynny'n ymwneud â'r hynny'n ymwneud â'r hynny'n ymwneud â'r hynny'n ymwneud â'r hyn.

In a sense, you could bootstrap up a process of collaboration with others by increasing your metacognition. Because if you increase your metacognition and then you also increase a little bit how you're able to think about someone else, or vice versa, if you increase your capacities for theory of mind and that also feeds back in in how you think about yourself, then that's just all net positive for working together with other people.

Um, essentially being aware of your own failings, but also being very sensitive to the potential failings of others, their, their, you know, their um Weak points and and strong points.

So that really leads on to the second part of your question, which is on, you know, how do you go about doing that? How do you improve either of these things really? And so one interesting insight to that is some work we've done on Trying to think from a from a first principles perspective, like why does metacognition take so long to develop?

lots of other things that babies are born with that they just do pretty much. I mean, you know, they they perceive, they move, they they obviously need it takes them a little while to to gain things like language and being able to walk and talk. But essentially, I mean, by the time you're around two years old, you're pretty good at all that. relatively speaking. Whereas things like metacognition Really the the the k kids are just flatlining on these tests uh around the age of three.

even when they're very verbal. Even four, five, still not great. And we've done some work in collaboration with Sarah Jane Blakemore in adolescence that continue metacognition just continues to can tick up until you're 18, 19. And only then does it start to plateau. And I should say maybe very briefly the way we measure this in the lab is by asking people to um make a whole series of metacognitive judgments. How well do they think they're doing on a task?

And when we have hundreds of these judgments, but you really need hundreds to measure it properly, you really need to get a really detailed p um picture of how people how people are doing on self-judgment. When you have those kind of judgments, you can build up a really detailed statistical picture. of how their metacognitive judgments map onto their actual performance on a tap.

So we we can then derive from those data, we can a lot of the work we do in my lab is really developing better and better metrics for different parameters of metacognitive function. And so we can derive numbers from those data that reflect For you at this time point in your life, how good is your metacognition in this area? And so we've studied that across the lifespan and in various And what we find is our in our in adolescence it continues to increase, which is pretty striking'cause

It's hard to kind of see that from the outside, but the data show that it does continue to increase. And so one um implication of that is that there must be other things that go into your metacognition that are not. You're not born with them. And one hypothesis is that they're culturally inherited. They're they're really so I think one implication perhaps for the conversation tonight is Rather than thinking about can we boost metal cognition e in education?

Perhaps the better way of thinking about is that education is all about increasing your metacognition. So it's not as if you're just going to have like one little intervention that'll get in there and go like that and your metal condition will go up. It's like it's the whole process, it's culturally.

I guess what I'm trying to say is that compared to other psychological faculties, metacognition seems unusually sensitive to an extended period of sociocultural development and that's going throughout the whole school career. So I think that maybe I I can also say other things about interventions of metacognition, but maybe I'll stop there and let let Jessica come in as well. That's brilliant. Absolutely fascinating in so many ways.

But I am gonna bring Jessica straight in because I can see you're nodding and then you were wanting to to come in because I think it it's it's so relevant to the work that you've been doing. So yeah, let's get some reaction from you, Jessica, and then we can come back and and and probe a bit more.

No, I was so excited to listen to Steven talk a little bit more about um just, you know, the what what I was getting from this is that sense of intentionality um that we we have a responsibility and then just how you're mentioning how some of these these skills kind of flatline at the age of three. They are teachable and learnable skills. And so the thing is is that The more we, you know, kind of uncover, not just discover, but uncover, you know, strategies that students can access.

The goal is to take ownership for learning. And so if we can support known strategies and research that is able to help students. you know, understand their brain, why does it have to be a mystery? Why can't we get them involved in that process when it is an expectation of them? But what can we do as leaders and educators? to bring that out. And I I I'm really passionate about this idea of intentionality. I talked a lot about this in um a TEDx talk that I did.

at the UAN L, which is the Universidad Autónoma de Nuevo León. And in the talk I I discussed what I think are really important intentions. There are five intentions. One is to intentionally teach adaptability, one is to intentionally teach empathy. exploring passions and lots of them, personal branding and investing in teachers. And I think that in order for us to really maximize the potential of metacognition in our students.

We need to take a collective approach to that. We need to be able to connect what we know is best practice and research, but people don't just believe research because it's research. we tend to water down that term a lot. We use it in in different contexts and influencers and and none of that is good or bad. It just is. And I think that We need to come down to having this opportunity to create intentional learning experiences.

So that it's not just being done to someone, but being done with someone, with students, with parents, with stakeholders. So that they're experiencing what it is to think about their thinking. What does that look like? What does that sound like in a classroom and outside? And some of these um examples that I've had the the privilege to experience.

as a middle school principal, I walked downstairs at one point um, you know, on my rounds and and I didn't have the opportunity to interact as much as uh with the little ones as I did with the older students. And I remember uh walking by first grade classroom and I saw first grader six or seven years old and standing outside his classroom and I just decided to engage in conversation and I said,

What are you doing outside of your classroom? And the child literally said to me, Well, I didn't manage my impulsivity and now I have to reflect. And I was so moved and I thought, well, was that teacher language or is that child language? So I probed further and I said, well, can you tell me what you're reflecting on and and what are you going to do next? Well I

was sitting at the table with one of my peers and I distracted him, so I'm going to apologize. And I said, And then what? Well next time I probably should just change places. I thought it was so moving. that is a perfect example and anecdote. But then coming back to what we're talking about in our conversation today.

That's one anecdote. How do we put all of those pieces together? And as Steven was mentioning earlier, is how do we now translate that information, that data into something that we can put into numbers to track it, to track progress, how do I know for certain that that child got better at managing impulsivity and what will that look like? What will those pieces of data look like? And that's really exciting to me. I think it's really important that we continue

exploring this because there's no one right answer. Otherwise everybody would be doing the same thing. We need to take all of these just uh resources from um diverse measures and and include them all together in order to be able to, you know, come up with systems and share them because we know, you know, collective efficacy is what gets us, you know

maximize learning potential. So I'm really excited about some of these pieces. I think it's exciting too. I agree. And I think the potential of helping students understand themselves. better as learners. It's so important to their ability to cope with a world of work that we're not certain about, that is going to be full of changes and you are going to need to be adaptable and to be good at learning. I I think all of this is extremely important for the future.

I want to come back to a couple of things.

One is this second way in which you, Steve, talked about the relationship between AI and neuroscience. and this issue of how humans interact with AI. And that also relates to something you said early on, Jessica, about AI within education. And I'm really concerned about there's a risk, and I know Steve, you and I have spoken about this before. There's a there's a risk that we inappropriately offload some of our cognitive processes to our AI. And it worries me.

Because and what you said, Steve, about people trusting the AI more, believing the AI. Is b the confidence exuded by these AI systems and trusting more adds to that worry. So I guess my question for both of you is how do we help people develop considered and critical perspectives about The AI that they are interacting with. And it does relate to metacognition. It relates to epistemic cognition as well, but it does relate to having a sophisticated understanding of the world, of ourselves, of

the way in which thinking happens at a in a in a non-technical sense. So actually going to come to Jessica first this time to switch it around. And and it I was struck by the situation that you just described in one of your schools and the perceptiveness of that that young student. But how do you think we best equip our students in this instance, but it's also educators? to be appropriately critical of what an AI, for example, is able to do.

Yeah, um those are some really important questions that we've been grappling with because it's it's still fairly new and like a lot of new things, there's this tendency for us to be more comfortable with things that we know. And so a part of it is exposure and it's guidance and it's conversation. And again, um, I think it's really important that we explore and teach that idea of curiosity so that we can explore it in a way as adults.

that we can make some decisions about, okay, well, how do we want to use this in the educational context? And it doesn't have to be everything all at once. And we're not expected as uh I mean, we're living this alongside students. It's a lot different from previous teaching practices where, you know, as as teachers we had all the information and we gave it to students. Now we're learning at the same side of students. And so what we need to be able to do is

think about what is the purpose of the tool that we're using. What do we want to accomplish with it in order to not use it for the sake of using it, but address the questions that students have. We don't want them to be unguided using all of these tools simply because as adults, you know, we're afraid of it or we don't know how it works. we need to put some strategies into place, like explore a new tool once a week.

Get involved in it and then talk with students about it. Like how do you see this being used? Asking them, how do you see this being misused? What should we do about it? If we have these open conversations. we're more likely to come to something that we can all be comfortable with. I think ethics and technology is crucial. I think guidelines are crucial.

And I think it's not something that we just need to wait for someone or experts to tell us what's the right way to use it. I think that is a construct that we need to build together and it can't be built if we don't talk about it and get informed. It's really easy to be afraid of something that you don't have any information about and it's easy to make assumptions.

And I think that that's what's happening with technology now. And it's not a new process. We lived this when computers first came out. We lived this, you know, earlier on in centuries. I mean, there's, you know, a story about uh Yale University students leaving their um their their university because you know the new technology had come in and it was the chalkboard and it was around the 1820s and 30s. So the thing is is that

This is not a new thing. What we need to do is be able to embrace it, be open to it. take it on little by little and yeah focus on what we value most. We want to be able to have success with these experiences and the way to do it. is not only by being open as I mentioned earlier, but I think It's creating guidelines for use of it. How and when do we want to use it? Why do we want to explore it?

And what are some really creative ways to do it? I mean, I've seen this in education already. Um, we learned, for example, how to use the tool of Minecraft and now we have students. using Minecraft simulations to solve a crisis.

I mean, this is an excellent way to address the UN seventeen sustainable development global goals. And I think that curriculum needs to evolve in order to incorporate that as well. So I think The more we engage learners in their learning and we use AI tools to help connect and spark that creativity, I think we're going to be better equipped to reduce the fear, engage

the stakeholders, inform the parents and be able to connect it. I'm very much an optimist as you can see. I'm a glass half-bowl kind of gal and I'm not immune to the risk. And you know, the the the dangers, but that is also what we do. in the real world. We protect our children. We we talk to them about the risks. We involve them. We create scenarios. We need to do this not only with AI. This is a life skill. And so I think that

we can't we can't eliminate the potential risks uh by not including AI. I think that would be a disservice um to our students. So I see deep connections there and we're still exploring what is right and what's next. That's very interesting. I personally um see a real similarity to the situation when the internet first became available in schools. And I can remember talking to head teachers at the time

And then saying to me, Oh, I wish we could just shut it out. It's causing us so many problems. You know, problems are overspilling from outside school, into school, and I just don't I just I just want it to go away, you know, and of course it never was going to. And in a way, you know, we have to face up to that situation with AI. I mean, the internet brings huge benefits, AI can bring huge benefits, but it does bring challenges as well.

And interestingly, as I've been going to various groups of educators and AI developers, tech developers in different countries in the last few months. There's a consistent perspective from the educators that they are concerned about safeguarding issues because they don't feel they understand enough about the technology. And some reflect that. They're worried that they might be doing it wrong. だったいま アンデフォー

I think what you're saying about transparency is so important and and and enabling an educator, for example, to feel okay about saying, Oh, I tried this thing and maybe they've done something that they shouldn't have done. They've put some personal data from a student into a large language model or something like that.

You know, but we need to give them the opportunity to talk about that and and say, actually that's not a great idea, and here's the reason why that's not a great idea. But you need that transparency and that that Safe space where people can talk about these things before they do something wrong and and something that that

that is really significantly problematic. Obviously the example of putting personal data into an II is problematic, but people don't necessarily understand that. So I think there's a lot that needs to be learned. And and taking the approach that you're suggesting, Jessica, I think is very important in enabling people to learn together about something that in many instances does seem mysterious. and can be portrayed as quite magical, let's be honest.

Steve, I I want to come back to you and and and particularly about this point that you were talking about with respect to helping people be more critical in their perceptions of of what the AI is genuinely able to produce.

Yeah, and I think I mean I first of all, I very much agree with Jessica about the optimism here. I think these tools are amazing. I mean, we're even in my lab, we're using them now pretty regularly for help with writing code, asking simple advice on, you know, if we want to write this particular algorithm to analyze our data, how would you go about it? It is remarkable what i in terms of the boost of productivity and being a kind of external prop for your thought process.

And I think I'm kind of envious of all these teenagers who have it at school now. I mean, it's kind of amazing, right? If you're like doing a physics class and you can just ask it a quick question about the solar system or whatever you need to learn. It's it's quite remarkable. Um so I think we shouldn't lose sight of that. Um but then at the same time, I think one way it differs

I thought the internet analogy you made, Rose, was very interesting. But I think one way it differs from the internet, which potentially has profound consequences for trust and and critical thought, is that it's very easy to fall into ymwneud â phobl ymwneud â phobl ymwneud â phobl ymwneud â phobl. wired up to trust, very confident people, and

I think that that can somehow sometimes create problems with a lack of critical thought. You kind of interact with these systems and you take at what it tells you at face value and as we know. They are getting better, but they still can fabricate things. They still can not necessarily always give you the right answer at their.

They're only as good as the data they're trained on. And so I think one um, you know, one thing we've been very interested in is first of all just studying how people perceive these systems. And so one is the data that we mentioned I mentioned earlier, which is a line of work being led by uh a postdoc of mine, Clara Columbato, who's just starting

her own lab in Canada at University of Waterloo soon. And she's very interested in social perception more generally. How do we perceive other people? How do we perceive artificial systems? And what she's found in a number of studies is that if we give people identical judgments or advice from humans and AI, then people tend to find the AI advice a bit more confident, a bit more convincing.

Um, and this is across a range of settings. And the other thing we've been doing recently is asking people pr trying to understand how people think about the mental state of an AI agent. And from the point of view of the AI engineers developing these large language models, they don't, you know, the the internal workings don't have what we think of as human mental states. And we don't know how the human brain works. So, you know, it might turn out to be more similar than we we originally thought.

But what we found is that a a pretty decent chunk of the general population think that something like Chat GPT has emotional states, is to some extent conscious. It they're just treating it like another person. And why and why wouldn't you? It's so compelling. And my so my I do get a little worried, not so much about like, adults who are now kind of have grown up through the system without these the these things. But imagine so my son's four.

Imagine that, you know, in the next three or four years, these tools just become part of everyday life. But the difference is that his theory of mind is still developing, as we talked about earlier. So he starts now thinking of these as real people, essentially. And so what's the developmental consequence of you now essentially thinking you've got a real person, but that person has the knowledge of the internet, which might well be wrong in certain ways?

What's the consequence of that for how kids form knowledge and beliefs? I don't think we know. I think we have no idea. So I think there is it's really exciting, but I also think there's real concern about how you deal with this shift towards essentially like a an ecosystem where we've now got all these personal assistants that and and not just like the internet because we actually treat them in some sense as real individuals and we trust them to perhaps to the same degree.

Um, and so that raises, I think, interesting questions about both how you how you improve your own metacognition to make sure you're thinking critically, but I think also interesting questions about How do the developers of these tools perhaps bake in some signs of

uncertainty, doubt. And we've been, we've got a collaboration with the Oxford Robotics Institute on some of these questions where these are now real robots rather than um disembodied software. But the question is for things like interacting with robots in a care home or safe safety critical robots.

How can you start putting in subtle signs that the robot knows what it's doing or doesn't know what it's doing? So we've been thinking of things like, could you make the robot glow a different color if it's not sure what it's doing? And that's the kind of thing humans do all the time. Like we hesitate, we We're we're, you know, we give we give the impression we're not very confident in this situation. So that's a kind of implicit sign of metacognition to others.

And that will mean others might then defer to someone else or do it themselves or take over. And we just do this thing we just do this all the time without really thinking about it. But when it comes to interacting with the artificial systems, By and large, they don't have that implicit metacognition, at least to the extent that it's visible to us as the human counterpart.

And so I think we then this leads us to thinking they're either terrible and that the robot falls over and we just disregard it, or when it starts to become good, like they are doing now. We start to really trust it. We're like, wow, okay, this thing is telling me this answer, and it's also telling me in a way that has no doubt. And I'm just gonna go with it. So I think this idea of trying to bake in a bit of metacognition.

on the machine side is is an interesting one. That is fascinating and I'd love to have a much longer conversation about artificial intelligence ability to do metacognition, which I think is fairly rudimentary at the moment, isn't it? But I think I find this a really interesting situation. So I've been working in the area of artificial intelligence and education for about 30 years, and for 29 of those.

People didn't really want to know, if I'm honest. A few people did, but it wasn't exactly easy to talk to educators about artificial intelligence. The last year the situation has completely changed and obviously I'm extremely grateful about that because she can start having conversations she've been wanting to have for decades.

And so it's great. And I am, like the two of you, very optimistic about the possibilities these tools afford. But I am worried because of The hubris of these AI systems and the persuadability that they come with. And I find that really challenging in terms of helping. people get the right balance.

when they're dealing with these systems because you want them to be enthusiastic, just as you were saying, Jessica, you want them to test them out, you want them to play with the systems, you want them to feel but at the same time you're also saying, oh, but be careful because It actually, not everything they say is true and and and they exude more confidence than it. So I think it's challenging times for us.

And I think the work that you're talking about, Steve, there is really interesting in terms of thinking about not just how we as humans perhaps change the way we perceive these AI systems, but also how the developers of AI systems can bake in some some signalling. that might help us along the way.

I am on fire right now. In my my brain is on fire listening to all of this. Um this is exactly the conversations that took place when we um started exploring some of these tools first with teachers because, you know, of course it was coming up.

in conversations and you know, to do we have a policy and to what extent can students use these tools? And what I thought was interesting is uh we challenged our staff to say, okay, well thinking like one of your students and one of your, you know, your biggest fears. What what are your concerns, you know, with with this chat bot? And so one of our teachers actually inputted

something personal about themselves and the chatbot came back and said, I cannot tell you any personal information about you. I only what's available on the internet. Um we did the same thing with um a couple of other tools. uh whether it was HN or um, you know, some of the other, you know, virtual uh virtual assistants and, you know, certain likenesses they couldn't use. So you're starting to see

that there is at some point, you know, d certain companies are taking this into account. I know the Center of Humane Technology, the co founder Tristan Harris, I listened to a lot of his work. you know, talking about the the importance of ethics as we build it in.

But again, I go back to that collaborative mindset. We need to be having these conversations with kids because it is exciting. They don't know to what extent they should or shouldn't be doing something. They just know that they get that hit of dopamine and it's exciting in the moment.

when they figure something out. So how do we uh there's no one right formula for it, but I do believe these continuous open conversations, this exploration piece is just so key. And also talking about what guidelines, what makes sense when. And it does have to change curriculum. I mean curriculum has to change. We can't be asking the same questions we did when they can get it from putting it into

any given tool, you know, that comes to to mind, right? We we want to be able to um encourage the thinking. And if we go back to what we value, and I ask you a question about something historical. Do I want you to be able to analyze that or do I do you want to do a quick fact check? So it changes the way that we present information to students and um the way that we talk about learning in general. And the thing is is the more aware that we can be.

the more mindful we are about how we use these tools and in what ways. And I think that is where the power and potential really, really connect. And I think that's what we're looking for.

w I had a conversation recently with someone about, you know, again, curriculum talking about art and how, you know, Dolly Three is coming up with just these incredible, you know, text to images and I mean so many of these other tools I know w we would be forever naming them, they they they're exponential every week. But uh, you know, how is this gonna change art and is art real anymore?

And I have a personal story about that where, you know, my dad's an artist and we had this conversation and at first he was quite disappointed with it. And then at the same time is I said, okay, well tell me about this next painting you're working on. And he started to talk to me about it and he said, I can't quite get this piece right. And as he was talking, I was inputting it into the chat bot. It came out with something. And he goes, that's similar, not quite, but now I know what to do next.

I brought it to the art teachers and they did the same thing, make your interpretation of it. So I think that Some of these fears are just that fears.

There might not um they do have a root in something. Again, I'm not naive to the potential dangers. There's always going to be a dark side to anything that we want to do, you know, for good. There's going to be inappropriate pages on the internet, but it's about these talking talking with students about when and where and how, what the dangers are, why they're there, getting them to think about, you know, how are their brains processing this information.

And getting them involved in that stage of development that they're at. If you know this about, you know, higher risk taking in teenage years, let's talk to them about that. Let's talk to them the wives. Let's talk to them about why we care about them. Messages get through when we connect that learning is social piece.

with caring about each individual. And I think that that's what resonates most. As long as everyone knows they're being listened to, we're in a good pi good place. Yeah, social psychologists do have a lot of expertise on You know, how we interact with other people, how we trust them, how we

form beliefs based on information we get from different sources. And I wonder whether they're going to become more important as we go forward. So I I'm not really a social psychologist, but the the postdoc I had in my group was. And in a sense, as I was saying before, these systems, we're going to start treating them like people. Like other individuals, whether we like it or not, I think it's just going to come down the tracks. They're going to get more and more sophisticated, domain general.

They're probably at some stage going to get embodied in a s in a form that we don't necessarily predict yet. It's probably not going to be like a humanoid robot, but it might be like a little tabletop device that we interact with and, you know, we already have Alexa and everything. So So I think that the social psychology of how we interact with them and how those interactions develop, as particularly in the early years.

I think are gonna become more and more important in terms of shaping how kids and adults you know, operate in this ecosystem of information that is now not us searching for something, it is being told something. And so I think if we can somehow form a partnership between education, AI, and social psychology broadly construed and and try and optimize that, that would be great. I mean, I I I am optimistic. I'm also slightly worried that the

Commercial incentives are not going to be there for baking in doubt and uncertainty into these things. I i it it's not quite clear to me like whether this will be pursued with any vigor on the on the commercial side. Now the only the only um silver lining I can see there, and that's one thing we're trying to work on as well, is there does seem to be some interesting work suggesting that if you can

Essentially create the impression of having good metacognition as a human, then people seem to like you a bit more, they collaborate with you more, they tend to just gel with you more. So if the AI industry realizes that Essentially, to for us to like using these things, we just want it to have some form of metacognition. It has metacognition internally. It's generating probabilities and confidences all the time in its workings. It's just not showing it to us.

So I think that's something that if we can align commercial incentives on developing some form of artificial metacognition, I think we'll be in a best place. That's really interesting. Yeah, and that's a fascinating prospect and a great place to finish.

I very much appreciate Steve and Jessica joining me today. It's been great to have you in the Zoom studio and having you contribute to this series. Thank you so much to Nord Anglia Education. And I hope wherever you're listening. You found our discussion informative and practical on how to with your teams in the coming days. If you want more information on the series and our wonderful guests,

Visit the EdTech Podcast website at theedtechpodcast.com and connect with us via social media. To see how Educate Ventures Research can help your organisation harness the potential of A and With strategies, roadmaps. Ways of using tools, continuing professional development and training, please have a look at our website, educateventures.com, or join the conversation on LinkedIn. You've been listening to the AI in education, our data-driven future.

Performed in collaboration with the EdTech Podcast, presented by myself, Professor Rose Luckett. Have a great week wherever you are.