Hi, everybody. My name is Mike Wooldridge and I'm a professor of computer science at the University of Oxford and I'm currently head of Department of Computer Science at the University of Oxford. And it's my very great pleasure to welcome you to our Hillary term stretchy lecture. The second straight lecture that we've done online, we hope it will be only the second or third and that by November we will, with luck, be back to being able to do them face to face.
So before I begin, I would just like to I would just like to thank our sponsors and our sponsors are Oxford Asset Management. They've been supporting these lectures now for six years and they've completely transformed what we've been able to do with these lectures.
Obviously, we've been in a lockdown now for the past year. But when we went into lockdown, they enabled us to bring speakers that we couldn't have imagined bringing from across the world and completely transformed our structured lecture series. So we are tremendously grateful to you all at Oxford Asset Management. I hope you will be. I hope you will be tuning in and joining us today and and for the future, I should say.
They always ask me to say this. They are hiring. They are based in Oxford. If you're interested in machine learning. If you're interested in advanced computational techniques in Oxford, Asset Management is definitely worth checking out. And I'd given you the you are out there. So onto today's lecture. So the hour straight you lecture today is Subbarao come home Bartee. He is a professor of computer science. Arizona State University.
And he studies fundamental problems in planning and decision making and is motivated particularly by the challenges of human aware A.I. systems. He holds a number of prestigious fellowships, including a fellowship from the Association for the Advancement of Artificial Intelligence, triple-A AI, the American Association for the Advancement of Science, the Association for Computing Machinery.
And he was a NSF young investigator. He served as president of triple-A on the Association for Advancement of A.I. and has served as a trustee of the International Conference on A.I. and has been a founding board member of the Partnership for A.I. His views on research, as well as progress and societal impact on A.I. are featured in many multiple international media outlets.
And in particular, he writes a column, a regular column full on artificial intelligence for the Hill, which is both very entertaining and extremely informative. And I recommend it if you're interested in learning more about the reality of A.I. today and in seeing some commentary on any AI developments. He's also on Twitter and is an extremely entertaining person to follow on Twitter, making lots of interesting remarks about current developments in A.I. so raw.
It's all very great. Pleasure to welcome you. I'm sorry we can't welcome you face to face, but as I just said to you off line, we hope that we will be able to do so at some point in the future. And you would be most welcome. So, Ralph, over to you. Thank you. So thank you all for coming for this lecture from wherever you are. And thanks, Mike. And Oxford as this department for inviting me to give this lectured.
So. I am going to be talking to you today about how to get a chance to interact and collaborate with us on our terms. So let me get started by thinking about how we basically CEI being shown being represented in popular culture. Typically, AI systems already from the beginning from the get go.
I see it as things that are like companions to humans. Whether you start from like 2001, A Space Odyssey, a HAL, you know, hanging around with Dave, Samantha hanging around in HUD and also, of course, our in ex machina even. Obviously, the popular culture, the novels, you know, Ian McKellen and most recently Cosla should go to talking about robots that work, you know, the way they imagine the baby imagining agencies. They will be working with us.
They'll be sort of companions. And, you know, you will almost treat them almost as if that humans at some level. So this is the I in popular culture. So you would think that, you know, instead of the gloom and doom about it, I see the areas you'd be seeing this kind of breaking news in popular culture. Technically, I that is helping old lady across the street. It plays with kids, cooks, food hacks at home and sense of drama.
In case you're wondering, how come we never saw this kind of breaking news item on CNN? It's because I made it up. It's fake news. Basically, it turns out that we typically keep hearing about a I not in the sense of agents working with humans. So there's been essentially a sort of a curious ambivalence that my research has had with humans. Our system seems to be happy as to either far away from humans, like the spirit, opportunity, curiosity.
I'd basically in some adversarial stems with humans essentially claiming the heck out of us in various kinds of board games. It's almost as if we wanted to help humanity. It's the people that we idee, such as just gonna stand paraphrasing something that John Lennon said. Right. So I was sort of. This is something. Been something that has been weighing on me for several years.
And I've seen sort of my research programme in for the last several years has been in terms of what the challenges of human activity ISIS stems. How do you get human systems to come? You know, I collaborate with us as companions.
So I give a talk at the U.S. back at Tripoli Eye that sort of set up as a magazine article on the challenges of Human A very I. And most recently, that whole thing that Mike mentioned, I wrote a column sort of giving the current state of the area in the I fight like Lape Oblique. What I want to do in today's talk is sort of give you some of the technical background. And as much as my group has been working on in some of making some of these challenges come to fruition.
So I was sort of asking in the in my trip, I talk back in 2000. Why isn't human having any AI all over the place already, given that pretty much everybody thinks about the, I think, SBI agents as working with humans, at least outside of it, and why we should be pursuing it. And of course, you know, some of the challenges and one of the points I kind of made there, and that's what sort of repeating here is having for high research to take humans into account.
So we have our agents work with humans in the loop. It's not sort of a dilution of our goals. It's really sort of an expansion of our goals of the scope and reach of the enterprise. After all, it's been said that we had developed evolutionarily these crazy large breeds. We have not to run away from the lands of the Savannah. Are the title staff loyal Bengal. But to really strategize deal with each other. Also, this social interaction is what really potently led to some of the brain size.
And please, at least speculatively, often from an evolutionary theory. If you don't believe that, of course, we do know that we that we are always modelling each other's mental states and what the other people are thinking about me and what would I do to counteract that? There's a great piece from like French, the show that some of you may have, Watchmen, Phoebe, saying they don't know that, we know that they know that vino.
So you can actually play with these mental models both to cooperate and to strategize against, you know, adversarial stances. I mean, obviously, human eye systems are like needed in many places from the very beginning. Actually, people didn't realise that things like intelligent tutoring systems on the social robotics, the researchers in those areas have certainly taken humans into consideration in designing their systems because that's the whole point of those systems.
But then that's not the only pieces where you have you want your as systems to work with humans, pretty much even quotidian interactions between systems for humans, for example, assistance, human digital, personal assistants, office hospital assistants and AFCO in teaming.
You know, some of that is possible, Miles, some probably more so in the near future, elbow to elbow teaming more cognitive deeming, such as search and rescue scenarios and even like factory robotics, which can actually explain their operation to new humans in the loop. OK. So in some sense, increasingly, if in fact, that you believe the hype that yeye is everything and I actually each computer science, then human computer interaction really will become human interaction.
And so we really should understand some of the challenges there and how to solve those in within the United States that I'm pretty familiar with the U.S. national priorities for. I include like 10 areas. And number two is your money AI interaction. And then there, like several research programmes, for example, Darbar has assessed ADAPT and a more recent one, BGT and so on. Many other programmes looking at these kinds of challenges. So getting to this stock, what do I want to say?
What does it. What I really want to get across is what does it take for any AI agent to show interpretable behaviour in the prisms of human agents? And the simple answer that it turns out is managing mental model. Some of you have said, obviously, that's kind of obvious, but some of you may not have thought about this. This is something that psychologists have always been talking about. You know, something like Sallyanne past that some of you may be familiar with.
It points out that kids recognise that what they believe about the world is different from what others believe about the world. And this is when they realise that they actually can lie. You know, not only they can cooperate by actually making clear what they believe in, but also lie.
And, you know, I joke around that. I was extremely happy when my son started telling his fossilise like 25 years back because I knew he had a working brain which essentially is able to model other people's mind, mental states and realise that what he believes in is different from what others believe in. So managing mental models becomes that important aspect.
So that gets me to the overview of the talk, which is sort of a perspective on how to do this from a bunch of research that we have been doing in the past. You know, six or seven years. So so the all you have to talk is I basically want to get you to see that effective human interaction requires systems to be able to manage human mental models. These include include not only the humans model of their task, but also humans model of the agents of that robot.
So this is at least a second level. In general, mental models can have infinite regress. But psychologists have basically talked about the fact that humans tend to use about three level nesting. And I will specifically focus today on the second level of nesting because that winds up leading to explicable behaviour as well as explanations. I'm going to get to that in a minute. And then managing mental models brings up both influence and learning challenges.
I'll talk about how we deal with them and then frameworks for human interaction, because we are a bunch of computer scientists and we might think we are humans. We know what humans want. And so here is a solution that is not going to be up to snuff, because essentially, you know, you really need to have a systematic race of evaluating whether your activities offer assistance and so on. Are actually that acceptable to the humans through systematic.
Subject studies, and then finally, I'll end it with by pointing out that mental modelling capabilities allow agents not only to cooperate, just like they allow kids not only to cooperate with the humans, but also to manipulate the humans in the loop. In fact, people are worried about deep six. Eventually, we have to start worrying about head fakes, which is the agent making you believe something that's not actually true. And using that to an advantage that's kind of not beneficial to you.
Again, I'm not saying that you're going to be out there trying to design any agent that are. But they can be hijacked by somebody else. In fact, I would argue that huge amounts of problems with Facebook and other social media are caused by the fact that people didn't quite realise that they are talking to something like something and an agent that is actually profiling them.
When we talk to other humans spent already on guard, people who are putting their hearts out to these Facebook, giving away lots of information. And then, of course, I got used misused in multiple ways as we found out. So it introduces several new ethical challenges that we need to be worried about. Before I go into details, let me show the people who actually do most of the work that I'm talking about, which is my group of students right here.
And so most of the work that I'm talking about has done by one of them, in particular on a car who just graduated this different year at this is this week and shuddered and thought that that fellow who got in I tend to watch award this year. He graduated a couple of years back that they're actually done a lot of this work that I'll be discussing. Other people also. OK. So let me start with how we handle this sort of human interaction.
And it starts with, as I say, it's about mental models. So let's start with three. A tale of three models. When I say I know for the rest of the talk, when I talk about models, I just want you to understand that. I'm not wedded to the specific language, although a lot of our work was done with relational representations in PDL are planning domain description language.
They've also done work with other kinds of models, such as dynamic programming models have been used in MGP are not all communities. These are all connected. Anyway, the important thing is what's the content of these models? You would think you can think of the contents of these models essentially as including Inácio state of the world. The goal state the actions, the observation capabilities, as well as the current plan.
If any of either of the agents and we have two agents that they're talking about, the robot and the human asset, said robot. But that doesn't. I don't particularly care about embodied A.I. systems alone. I will actually show you that we work also with us. Basically, our software AI systems, software artefacts which are supporting us may not be embedded. So I'll just use the word robot. And in what general sense? OK. So that's what you have to keep in mind when we're talking about models.
This can be presented, obviously, PDA. But they can also be represented in your favourite languages. So let's start talking about, like, the simplest single agent which is living in the world. And then as the agent is actually just by itself, it really has some approximate model, an executable model of the world, what it hopes is the executable model of the world. OK. We'll call that and you can use that to make its own course of action and live in that world.
So you basically given its reward metrics, given its goals. It can actually come up with courses of action that are consistent. I didn't do it on a model and then hope that they can be executed in the world and start executing them. So this is something that we know in a one on one. The moment you put a human in the loop, which is another agent. But it's a not another artificial agent. Is this human agent with all the human fables.
So one of the first things that would happen is that the human race ends up having a model, MFH, for the task. That means they have their own goals, their intentions and their observational constraints and so on. And so the robot to be able to at least sort of it, you know, at least not get they may not get into there. We are hopefully try to help them. It needs to have some idea of this image and they'll call that image.
OK. So this image is a robot of the eye injuries approximation of the humans model. That's the first of the mental models we're talking about. And you heard about this in many places. When you talk about basically robots, are agents anticipating. Human behaviour. This is what we mean. We are anticipating human behaviour by using MHR. This can lead to assistance that keeps getting out of the way. So why do the humans so that you don't get out of it. Don't get into that.
And sometimes also to support Oeming, et cetera. So this has been actually done quite a bit in the community. In fact, I was looking at the strategy lectures, and I know that I think a couple of years back I started Glowers, gave it OK, and she was mostly a lot more interested in just sort of the human behaviour learning. So that is the cut off. You can think of it in terms of MHR. We are done some of the work in those directions.
Do we know, for example, you can do intention, recognition of the human model, basically vete in this particular case anymore to brain computer interface? This is nowhere near as fancy as the neural link that Elon Musk is trying. But even a simple thing which can work. Leslie know without actual explicit communication, there are what is actually able to see in this?
The one in the lower window that that such in was the subject that actually wanted to reserve one of the blocks and that it realised that it was actually using only the other blocks. So things of this kind. So basically intention, recognition and anticipating what people want and wide and getting out of the way can be done. We have looked at that, you know, in the same direction, actually, robots can project their intentions in to the humans either by actually speaking.
But in this particular case, we're using augmented reality holo lens, which sort of provides to that human not only what they're seeing, but also additional information that the robot is projecting about its goals by pointing out two particular blocks and so on. OK. So these are things that are possible. But I'm not actually going to be talking as much about that part.
The one that I want to talk more about is essentially the fact that the moment humans are interacting with any AI system, they will wind up making their own model of daddy AI system. So if they see a robot, they will essentially have some model of the robot MRI. Which is the task model that they attribute to the robot. Some of this can be very magical, some of them, if they're experts in the loop. They might have some really good sense of what the work schools are, what it's actually capable of.
So there is MRI edge, which is that humans model and the agent, the robot essentially now needs to have an approximation of that, too. This is the second mental model that I will spend most of my time in the dark talking about how to exploit and leverage that. So the MRI it here allows the agent to anticipate human expectations in order to either conform to those expectations. If it cannot confirm, then explain its own behaviour in terms by by sort of modifying a model.
So tell the human you have to change your expectations on me in the following ways so that my behaviour makes sense to you. So to me, explanation really is this dialogue that the agent has with the human changing their model, which is their model. Updated. OK. I will mention that MHR and marriage are different from MH and Emaar. Imagine my father, human and father Robot Sackhoff execution models.
These are models that they're hoping at executable in the world that actions without real preconditions. And, you know, hopefully, in fact, the application is satisfied. There should be a high probability that the action actually works. Unlike those image. Ah. And I'm on the edge of expectations. That really expectations on models. We mostly for this part of the stock. We'll talk about like the maximum likelihood expectations maybe.
But in fact, they can even be like a distribution or what, the bardos. And they don't have to have any execution semantics at all. They are just being used by the agent to make sense of the other agents behaviour. So it's worth keeping that in mind. So given doors, it's basically the V, the longitudinal human A.I. Interaction Cycle works in its real assume. And I'm mostly focussing in this particular case in terms of the M.R. Edge, which is the human model of the robot's task model.
And so you start with an initial estimate of haemorrhage. That is, the robot starts with the initial estimate of haemorrhage. And I should mention here that from my perspective, from any AI researchers perspective, if an agent is working with a human, the only controllable variable really is the eye agent. The most obviously controllable variable is the. And because you can't you can possibly tell the human, please do these.
But they don't necessarily have to actually take your advice. So in essence, much of what I would be discussing is from the perspective of the robot. So not only does it needs to anticipate what the humans are thinking of for its capabilities, it also needs to. And I understand what the humans think of its capabilities. And I'm so used that that is our marriage and we assume that we start with our initial estimate of our marriage.
And I talk to you in some cases, they're stocked with a shared model initially. Other cases you can actually learn from existing behaviour traces. I'll get to that part little later in the talk and then you sort of have this loop. There are two pieces in the loop. One is the model following where if the agent thinks that there is a sort of an understanding between what the human thinks its model is, that then it can just conform to it and marriage and conforming to it is really explicable.
And I talk about it and I'll have it in a minute. So an agent basically making its behaviour such that it is in accordance with what a human agent expects it to do. That is explicable. There is another notion that other notions related to model following, which such as a predictability, which is sort of locally this behaviour should be consistent with what the human expects.
Then that is the model. Communication is sometimes actually conforming to the human model, winds up being extremely costly. And so in those cases, essentially, that robot basically decides to do what is optimal for it. I'll give you an example of this in a minute. And then it realises upfront that since this behaviour is different from what the human would be expecting, given their model of its behaviour, it needs to change their model.
And so that's what explicable the explanation part starts explanation really is communicating the changes to our marriage, either as it heads up it upfront or as a post facto explanation after the behaviour is done. There are other ideas such as legibility, which is sort of the explanation, except typically with implicit actions and design, which is sort of changing the environment in such a way that the environment itself gives you a heads up to the human.
As to what you are. A marriage is also touch on a little bit on the design factor. So that's basically the longitudinal human interaction cycle. And I will sort of show this flash to slide a couple of times in the next few sites. So let's talk about model following part. So as I mentioned, I we actually use two kinds of use cases. One is on the left hand side where basically you have embodied robots.
And in this particular case, where the robot, the Fitch robot, is going around in a mock search and rescue scenario with the human commander, are safely ensconced in some some other place and having like a camera, a dialogue with their robot on the other side. On the right side, we are looking at a planning decision, support system. If you have a mission plan, the Mars mission planner or somebody, they are trying to make the plans themselves.
And the attention support system provides suggestions about how to complete their plans. So these two are the two general kinds of scenarios that we look at, the challenges that they look very different syntactically. What the challenge is about, how an agent interacts with the human wind up being quite similar in both cases as it is. I'll show you as we go forward. OK. Does other use cases.
So and then, you know, given that I love social, mostly the robot use cases in the demos because those are more fun to watch. So in the case of model differences with the human in the loop, that means that emerick's that the human has is different from MRV, that the robots actual model. So that what that means is the plans that are optimal to the robot according to its model, Imod might actually not be optimal according to the human's expectations of a robot model.
So I am on edge. So what the plan that is optimal in that model may not be optimal in commodities. That's all I'm saying. And when that happens, there's a surprise on the human side because they're basically seeing behaviour by the robot that they didn't expect. So this leads to inexplicability are inexplicable behaviour. At that point, the robot has two options. It can from in the model following is to change the model that the humans have. That is a marriage, which is the explanation part.
And I'll talk to both of you, talk about both of them in the context of this search and rescue scenario. So imagine you have the you know, in this scenario, you have the human and the agent, the robot essentially start with the correct map of the environment in this particular case. This is the fifth floor right outside my door. And in computer science department right here is you. And presumably that is sort of a. So the model of the map of the environment is known upfront.
But. It turns out that there is possibly some sort of a disaster. And so there possibly are rubble, there may be parts that are closed, maybe actually be passed out are opening up, which means that the Internet age and the robot, which is on the floor, might actually wind up realising that the map has changed. But the human still thinks that the map is the same as before. And so essentially, there's already a difference between marriage and a model.
And then that needs to be reconciled through, you know, through this interaction process, either by the robot being conforming to what the human expects or by changing the model. So in the context of conforming to the human model in this particular case, in this picture, what's happening is that particular path that would have been the shortest path is closed because there is an obstacle act.
Think of it as some kind of some huge double date. So if you don't want the human to be surprised at your behaviour because they're actually expecting perhaps that you would be coming out of this particular kind of dud, then you have to conform to the humans model. So in this particular case, it wound up removing the rubble. So it's costlier than just taking the next best path. That's what is explicable behaviour.
So to be able to provide explicable security, given a goal, the objective normally would have been to just find a plan that is optimal with respect to its own model. But now it needs to find a plan that's not only opportunity inspectorates model, but it's close to what is expected by the human based on dead model, which is there at marriage. And so it becomes sort of a multi-modal planning problem. And then this can be done.
I'll talk about this become sort of a more complex optimisation problem, you know, to actually come up with this kind of an explicable plan. And what we're trying to do is sort of reduce the distance as much as possible. Sometimes making the distance zero actually might be impossible because you must might be expecting magical properties that the robot doesn't have, such as completely removing the rubble from a particular candidate. OK. So now conforming to the explanations is costly.
As I said something, you actually have to remove rubble or something in this particular example. So then you might want to communicate changes to am on edge to the human. And that would be the explanation. But in this case. If you could hear it, it was basically yelling at the top of the wires that a particular part is actually blocked. And because it's this other plot that is being taken.
So in essence, it's just changed the model, just partially Delta changed to the model that the human and if the human actually pays attention and takes that into account, then all of a sudden the robot's behaviour is as explicable with respect to the changed model. So the explanations really has been talked about a lot in explainable A.I. systems and explanations have been talked about a lot.
But when human A.I. agents are interacting, really it's best to think of explanations essentially as this constellation constellation of models. This is sort of close to psychological value here. You do as I talk about it later. But in essence, what we are seeing is that an explanation. Epsilon, given a plan by that robot, has essentially decided to act on is is such that if you add this Epsilon to current Modrich of the human, then this plan by will be actually optimal in this changed model.
And it's also already optimised with respect to the robots model. That's why the robot again, I'm using the word optimal. It's easier to think in terms of essentially optimality from a theoretical analysis point of view. But in practise, essentially, you're more or less trying to say, I'm doing this and you're expecting a different plan. And this is the changes that I need to make to your model such that my plan would be in the Top Gear.
Plans that you would see me doing about optimality allows for a much better theoretical analysis. OK, so that's the explanation. Size model reconciliation. It turns out that, in fact, computing these explanation's winds up being a metasearch in the space of the models themselves actually skipped a little bit about the representations of the models values.
You know, in much of this, what we wound up using planning domain description languages which have actions and with the preconditions and effort in symbolic terms. And, um, so you if you have a model with a bunch of actions and the implications and effects of the model that the human has with a, you know, either similar odd different actions with their preconditions, what the robot is trying to do is essentially modify the human model, Emerich, that they are attributing to the robot.
This will involve, for example, changing the preconditions, changing the effects of changing the costs of the actions that the humans have attributed to the robot. And this search essentially has all sorts of interesting properties. Using this model space search.
You can come up with different kinds of explanations, both minimal explanations that would just make the human understand why this behaviour is fine in this case, but might question this explanation little later because you do something that's inconsistent with this explanation. You can also come up with monotonic explanations which will essentially not cross that confusion. Monotonic explanations will be costlier to compute.
And one of the cases you are searching from the human model in the model space works, the robots model. In either case, you're searching from the other side. The technical details of this search are actually in this HK 2000 17 paper. That would be you might want to look at it, even though I sort of mentioned explicable. And explanation as if one is the model following and one is the model communication.
You can actually combine both of them from a planning perspective, from a reasoning agent's perspective into a single planning framework. You know, we have a paper. Actually, I did this last year just before everything closed in play 20. There's a paper on expectation we're planning which essentially considers both on peak actions, the kinds of actions that change the environment.
Mostly those are climate effects are changing the environment. Epistemic actions, which are mostly their effect, is to change the mental models. That means that communicative actions and sometimes you have actions and take actions can have epistemic effects. That's how you wind up doing implicit communication just by doing actions that have physical effects and then on rent. But they also are providing mental effects, a mental model changes to the human side.
So the robot has both the standard Ontake actions and explanatory actions. It can model the effects of its actions on both states as well as the human's mental state. It sort of becomes for those of you and especially people like Mike. I mean, this is essentially epistemic planning, epistemic reasoning. So here, planning becomes a multi model and. Is it uses its both its model and the human's expectation model to generate a course of action that contains both explicitly unwanted actions.
And so the explanatory actions are such that those are making changes to the marriage, that the human has an offer. Those changes are made. Then the rest of the Ontake actions that the plan that the robot is actually exhibiting, that behaviour is exhibiting would be optimal in this changed model. So you're doing both in the same environment.
And the difference between applicability and explanation just becomes whether that's planetary message is a the E side pot is empty, in which case it's just explicable planning. And if it's not empty, then you are asking that you want to change that mental model first before understanding your behaviour. So that becomes explanation. But it turns out, actually epistemic planning has been studied before.
Do one of the nice things is for just the two level nesting that we're considering that are efficient compilations to single agent planning. And so that's the trick that we use to make this doable, tractable. So the details of how the compilation works here are industry 2020. Although I looked at the robot changes, basically that the embodied robot scenarios, as I said, the same kinds of explanations can be used in the decision systems.
So this is a radar system that we use to kind of provide vision, planning support to humans in the loop. We had some work at NASA as well as some building work. And now when our Office of Naval Research, in both cases, people are interested in making plans and people want to be in charge and the systems should be helping them, the A.I. system should be helping them. And the explanations, when you provide a solution, why did you provide this solution?
If that human were to ask, you should be able to provide an explanation. And that is something that systems are able to do using these kinds of theories. OK, so coming back to the longitudinal human interaction cycle. Now, what I want to do is that having talked about Explicable Ety an explanation, I want to talk about the learning bot. So basically, where did that what did you want and what which model come. Sometimes it is as I said, it maybe did as a shared motivationally.
Sometimes it has to be learnt. So in some cases, such as that USAF scenario human and the agent will both start with the same shared model. So all that is needed is tracking the model drift. Even if the robot doesn't know the model and Modrich with certainty, it can jism with multiple possible models. If you are not sure which is the specific model the robot a human has. If you don't know Emerich specifically, do you think that it's one of these scale models?
You know, in the most general case, you have some Bayesian distribution of what these models, especially the case where you have kear different models. We can actually reason with these different models and provide conferment explanations of conditional explanations. With respect to these models, six models that there's a paper back in 2008 didn't come back. In other cases, in the end, you actually have to let alone these models from scratch.
Some behaviour traces. If it's not being provided by upfront, by knowledge engineering aspects. So in that sense, the agent would need to learn the human and mental models from the traces. One thing that we do want to keep in mind is unlike MHR, which is anticipating humans, which can be learnt from human behaviour, traces that have been cached or at large amounts of time. You've just seen too many humans acting on the problem in over a long period of time.
And you just use that odd edge which is requiring basic that is humans take on what the robot is doing. And so that doesn't just come from the human behaviour. Oftentimes we have different expectations of the agent than what we are capable of doing. So, for example, if you're walking to a room and you see a robot that starts dancing, you would be surprised because you don't expect robots to be able to dance, except maybe in a couple of, you know, videos that you may have seen.
So whereas humans, you do expect them to be able to dance. So that MRI really requires places that human likes or the traces of the robot's behaviour. So that's a slightly harder learning problem. And sometimes you may actually have to deal with vocabulary differences, too. And I'll talk about that. I mean it with respect to one of the points that I want to mention here is that, as I mentioned, MHR and Edmar age are both expectations on models.
And so they really don't have to be in any specific format, such as being little Fotomat that a modern MHR. Rewards. Because they actually have to plan with them. But that much out and am I right? I only SRF expectations and they can be represented in many more flexible ways. One of the things we did essentially is to learn human preferences of the robot behaviour as labelling functions during a training phase and use these labelling functions internally.
But the robot then learns this labelling function. There's a paper in 2017 which sort of treats learns this as a sort of CRF and uses this labelling function as part of its behaviour, its planning process. So it computes this distance between the plan it is making and the plan the human expects in terms of this labelling procedure that it learnt. And so that is a way you can be explicable. It turns out that the same idea for learning.
So you're essentially in this particular case, you're learning a marriage not by learning it in explicit actions, clear conditions affect state, but in terms of just a labelling procedure. You could do the same thing even when you are not doing explicable. But you're also providing explanations.
In that case, all you need to do is improve the training scenario, expand the training scenario such that you are showing a behaviour as well as some explanatory messages next to them and asking you humans to say whether or not the behaviour makes sense. Which parts of the behaviour is making sense. Which doesn't. And this is the training phase.
And having used this, then you can then have a labelling procedure which can then be used to compute the explanations on demand without having a an explicit EMERICH, which is in terms of actions, pretensions and effects. So this is why it's important to understand that Modrich and my child are just expectations.
They don't have to be executable models. Having said all of this about explanation, some of you probably have heard a lot about explain ability and explain will I in the context of machine learning systems. I want to kind of make a connexion between all I discussed and some of the body of the work. They're both similarities and differences. And it's what understanding XY is hot, but mostly as a debugging tool for inscrutable representations.
So, for example, oftentimes you lined up the system basically says this particular particular dog is an Alaskan husky. And you ask, why do you think it is an Alaskan husky? And the system points out that the snow part of the pig cells that the saliency region is, that's not big cells. And then you might say, aha, you don't really understand Alaskan husky at all. You're just understanding snow and the correlation of snow being present next to the dog.
But notice, first of all, that this is a debugging explanation and it's a pointing explanation. And really, you can't actually point to many things. You know, oftentimes the kinds of behaviour I'm talking about, the sequential behaviour as to why did you do this decision? You you want to point out, you know, do a pointing explanation there.
You need to point to a space time, too, which is extremely hard and very unwieldy, which is why civilisation progress by sort of developing these symbolic vocabularies through which we provide explanations. And that's the kind of model based explanations that I've been talking about till now.
In fact, if you have looked at this adversarial example scenario that, you know, many people have probably seen about that school bus on the left hand side of a little bit of noise becomes an ostrich on the right side, you and you don't see it as an ostrich. You know, our system, a classification system might very well say with high confidence that it is an ostrich. And if you ask it, tell me which part of this makes it. Moscovitch, point to me what parts makes an ostrich.
That would be a useless explanation because it'll just show a couple of several pixels, you know, all over the picture, which don't have any rhyme or reason from your point of view. But that's what the system is concerned that made that class for this on ostrich. So the point being that pointing explanations of the primitive explanations that are mostly useful only for debugging explanations are critical for collaboration,
but they're not really a solid luckly by the agent. It's not just talking to itself, which is trying to make sense to the human agent in the loop. And that would require a lot more than pointing pixels. And the model, the Constellation view that I've been talking about here views closer to psychological theories of explanation, such as the ones that not Labruzzo uncle talked about.
That's a useful thing to keep in mind. So. One other thing I want to mention when I'm talking about here is that handling differing vocabulary's. Is another important issue that could wind up happening. We know we did not talk about whether the models are on completely different vocabularies at modern Modrich. Then you have an inscrutable system which essentially, basically is learning its own representations. And so there is no direct connexion to any concepts that the human understands.
This is where you were hoping that you'd be able to just point to the pixels of something that is common between us and point to that particular substrate. But really, if you have things like windshear vision problems, you will need to provide the explanation in terms of concepts that humans can understand. And so ongoing work that we have in a lab, as well as some other work that some other people like Kim have done, involves essentially.
Translating the explanation that you have into the vocabulary that the humans understand and to do that translation, you need to first learn the translation so you understand mappings from simple to big cells in this particular case. And then can work, can hold the explanation in to these symbols. And when you do that, it will be possibly an approximate explanation. But that's way better than just sort of throwing a big place off your reasoning to the humans and say this is the reason I did it.
Actually, I should mention that that's trivial. Violist farm off explanations. Are those where you say you will understand what I did. If you take my brain here, take it. And that is one of the cubicle farms. And we understand we assume that that that's not a good explanation at all. That's why we actually search for smaller pieces of changes to the marriage. That will be enough that human to understand your decisions, just to put this in a classification kind of scenario.
We have some recent work which shows that sort of thinking about mental models, even if it's not sequential additions, making problems can be quite useful. So, for example, in many classification systems are typically evaluated only in terms of their accuracy. How often? Not correct. But I would argue that really it's equally important to see when they are wrong, how badly off are they? So a system which makes an egregious misclassification. Humans, we lose trust in it.
By the way, the word plussed is the first time I mentioned it in the dog, but. The interaction between an agent and a human humans might kind of, you know, might engender trust in humans if in fact they can make sense of the decisions that the agent is making.
So in this particular pictures that they're showing, we essentially about this system where it not only takes into account the classification accuracy, but that misclassification egregiousness and using that, you know, our system, for example, will be able to come up with better kinds of these classifications, even if it is asked to misclassify than the existing system.
So, for example, some of the examples there are showing that, you know, if you have something like a Dabby on the very first top left gardener, you know, existing systems might just when they make a mistake with this, they might just say it's a remote control. And, you know, our system will say it's a tiger cat. So because in essence, it's also looking for the cost of that misclassification.
And that's something that you can do with respect to mental models. So there's a bunch of things that are that can be extended from this. For example, communicating the model. Myard may not guarantee portability unless the human has the inferential capacity to compute optimal behaviour from the model. I sort of, you know, pushed out of the bag earlier. It turns out that you can use file based interactions with a bunch of what that we have done their way into humans.
Just ask why not this other behaviour? How do you only provide reason as to why that other behaviour is not as good as the behaviour you have sheets shown. And this will be easier start off to understand than just dealt us to the model with respect to which you need to be able to see that the behaviour is actually optimal.
And similarly, when you provide information to the humans, the real question is also a question is also whether the agent is actually humans are paying attention to the communication and that involves two parts. Is the communication actually perceivable, at least in theory, at least?
So, for example, if I was getting this talk on radio talk radio, if I started showing you slides, holding them in my hands, the joke is on me because I'm not reasoning about the fact that you won't be able to see what I am sure you. So that is basically reasoning with the communication, possible perception modalities. The other is, even if you see, you may not be attention to it. That is attention management. That is something we haven't done much.
But in fact, it's been done by others, such as the incorporates know in the context of the all Clippy, the Microsoft assistant, which is basically figuring out what is the best time to tell humans anything so that they're likely to pay attention to it. So in terms of the file based explanations, for example, we can you know, here's a bunch of papers that we have made.
The humans basically ask, why not this behaviour? That is, they'll provide a plan, a partial plan of the company then and say, why aren't you doing this one against the one second is the one you are currently doing. And you can use that question itself to figure out what level of abstraction and the abstraction hierarchy of models that human side in and use that knowledge to provide that kind of explanations to them.
That's like a file based explanation. We actually did this in the context of this sub radar system that we're talking about, where we provide a longitudinal dialogue between the agent and the humans and the human actually provides the file. Not only can we provide an explanation as to why that file won't work, but we also realise that that question means that the humans really would like the plan to be closer to the file.
So then we do have a planning scenario where we will bind up coming up with a plan that's closer to the file, even if it is a little less optimal. And there's a demo CHIPLEY. I know that that talks about this. We also extended these models to also consider multiple humans in the loop. So essentially providing customised explanations for different humans at different times based on their mental models. And then I talked about this controllability of zero controlled observability planning problem.
This is something that Unida has done there. Essentially, the system can reason about what can and cannot be seen by the human and use that both to make sure that they see that the help is being given. And also, if it if it's so, please, it's up Outfest gate to people in the loop for it doesn't want them to understand that this help, it can obfuscate. So that's also possible. That's actually a 19 paper.
And then more recently, it has been looking at this proactive assistance part where essentially you are reasoning about the fact that help should not only be given, but should be seen to be given. So at least you should be able to reason about the possible upsetter. Aspects of the model observability aspects and make sure that the human understands that you have provided to help either by explicit communication in Oneida's work,
it's actually by implicit communication. So you wind up showing that you are transporting second objects and that will sort of, you know, provide to communicate certain messages to the humans. So I just want to quickly go what this and maybe in the couple of minutes, I know that they're very close to hand. So the framework for human interaction really cannot just be me. I'm saying I did what because we are humans and we think it works for us.
So our solution for this has been interdisciplinary collaboration with the human factors such as Nancy Cook. Sean Did is a colleague here, and she has actually been she was a past president of the Human Factor Society. And so we work with hard and hard group in setting up systematic human subjects, studies to actually cheque whether the kinds of explicable behaviour as well as explanations actually are coming up with makes sense to realise humans in the loop and doing human subjects.
That is pretty painful. But I would also say that anybody who says that into human interaction but haven't heard of the word I ought to be, which is institutional review board certification, I'm probably making it all up because without that, you have no clue as to whether or not these things are actually working.
So, in fact, there's papers in both H and I. And it's a journal that show how that kind of explanation and that we come up with are seen to be useful in collaborative behaviour with humans. A very lasting mental model capabilities allow agents to manipulate the humans in the loop. As I mentioned, they can allow for head fakes.
This sort of leads to a bunch of ethical quandaries far any work which starts thinking in loving human agent agents, true mental model humans evolutionarily mental modelling allowed us to both cooperate as well as compete about each other. But really, you can't just stop your agents from having that ability because without the mental model, they won't cooperate with you.
Even so, you will have to essentially deal with the Pandora's box like that has to be open you and systems with mental more linkable. It is being ethical boundaries, additional ethical quandaries beyond the usual. Oh my God. Autonomous robots are going to kill us all. Such as, for example, automated negotiating agents that misrepresent their intentions to gain material advantage. Your personal assistant that tells you white lies to get you to eat healthy.
I end up, as I already mentioned, people would have been actually more guarded with social media if they realised that those platforms that actually actively profiling that we are already unmoored off our guard when we're dealing with other humans. But sometimes we don't realise that Facebook is actually an agent. It just doesn't look like a human. But it has some of the same capabilities in terms of profiling.
So, you know, in particular that with the social intelligence scenarios, humans example, closer sentences are far more pronounced. And so we need to be very careful about how to deploy these technologies. As I said, head fakes really are a lot more powerful than deep fakes, and we don't need to worry about them. But then again, every tool is actually a weapon, too. If you just hold it right. And that's true also for these kinds of things.
You know what what we actually have shown that model, the Constellation, can be used to tell lies, lies of commission as well as less commission. And then this work by Unagi that I mentioned earlier, that actually shows, for example, that that. In this particular case, in one case, the robot is making sure that the guy sees what it is doing. Watch what the robot is doing. Novik, is it actually dyspraxia to do something else?
And so basically it's about manipulating the perception model. And so this is like signalling France and taking to enemies simultaneously. These are things that can be done. The purpose on this in 2009 interview a monster 10 20. Finally, you know what we did when I mentioned that lying robot study saying when can I box like we started under what conditions? People are willing to be told how light lies. And there's a paper on that. Yes. System conference. So I'll stop here.
So somebody basically what I was trying to get to you is that effective human interaction requires human systems to be able to manage your own mental models. But the model that you want to model up their task as well as the humans model of the robot, at the least you could do crazy things with infinite regress. But in fact, you can do a lot of useful things with just the second level models.
That's what I've tried to tell you. Managing mental models brings up inference as well as learning challenges. And these frameworks for human interaction have to be evaluated with actual human subjects. Studies and then mental modelling capabilities do open up new ethical problems. And we have to be careful about them. There's a paper in my magazine which sort of gives an overview of some of these issues.
And then we also just wrote sort of a draught of a book that's under review, not explainable human interaction, planning perspective. With that, I'm going to stop and thank you for your attention. Thank you so much for that, rather, it was fantastic. So we are ready for questions, so please enter some questions in the Q&A. There is if you look on your teams, there is a button on the top right foot, which is cute. And there is a little question mark of speech bubble with a question mark on.
If you click on that, that will get you to the questions and answers. Let me kick things off. So I absolutely see where you're coming from or where you want to go with this. But how far can you get with this towards, you know, to kind of human level modelling? How far will this take you? Otherwise, there's something that's missing. What do you think? This is all that we're going to need? Meaning what? Human level modelling. And this is the way we deal with each other. Yeah, yeah.
I mean, this is partial steps towards it. And it's very useful in actual quotidian interactions between human and A.I. systems. I do realise that we do allow far sort of large amount of nesting when we design with each other. You know, oftentimes, in fact, one of the interesting things which I didn't talk about it is you can engender trust in somebody by doing what they expect you to do. And once they start trusting you, you can actually use it.
You are at one day. So, in fact, we have a paper where this sort of reason about how much trust the labour written or robot should engender. So that it's not required to be explicable in the sense are not being monitored as closely. So that it turns out that given that people understand this already, you know, when you and I interact, if you always do things that I expect you to do, I start wondering, you know, stuff not being surprised.
I mean, in the beginning, I'm not surprised because you're doing what I expecting you to do, but you're always doing what I'm expecting you to do. Then I wonder that you how other al-Qaeda goals and then so that will basically bring in additional nesting off the mental models, which we don't deal with right now. I would suddenly think that is something that will be important in bringing it closer. The other thing, of course, is really how we wound up human to human communication.
We wound up developing this sort of a common shared vocabulary, you know, over a period of evolution as well as our number going up. And this is something that was very important. Far, far. Yeah, yeah. Agents in particular, those which are learning their own representations, in particular because the representation they learn may not have any connexions to what humans understand. And so this idea of translation, this is something that's going to be quite important.
So I don't really know if this is still a big issue as to whether my thinking is in the same words that I'm talking to. You are I'm thinking in a different way and I'm talking to you in words that you will understand. And this is something that the agents have to deal with, like more. And this translation problem is something that I mentioned some of it earlier, but that's going to be another big issue in getting there. OK. So we have a question that's come in from Max, rankly. Max.
Yes. Thank you for a brilliant lecture. How is this approach similar to the ways that intelligent tutoring systems model learning skills and understandings and then came to lessons and follow on skills reinforcement appropriately other existing similarities? And you think your approach will enable intelligent tutoring systems to be better tutors?
Yeah, that's very much online. So in fact, I should have mentioned that when I said human interaction, I didn't quite explain who is helping whom and who is trying to get out of who's way in the way we do the work. Actually, the robot could be the teacher, the robot could be the student. That or what could be peer to peer and ideas basically is the robot as the tutor to the human.
And in many of these issues, actually, in fact, as I mentioned in the very beginning on the human eye, very AI application slide, people who have always given let's speak to humans in the loop are intelligent tutoring systems, people who like out on land. My colleague in that field here and other ideas, people, though, always that they didn't have the luxury of saying let's ignore humans, basically. The whole point is trying to teach the humans.
And so, yes, there's a very significant connexion between the ideas of literature and the work that we are doing. We actually have papers on applying some of this model reconciliation framework in teaching scenarios that, you know, this is an AI caps workshop paper that's available from our homepage. In fact, one of my students, our current students, Suchin, was actually in one of the earlier videos where he tries to get away from the robot hand he worked with Good.
And he's working with me. And so there is this obvious kind of. That idea systems and that human interaction are very much connected. One typical issue has been that idea systems tended to be sort of more samite automated oftentimes. And the level of explanations that they provide are somewhat different from the kinds of explanations we tended to provide in the what. But they're very much complementary and it would be less than that.
We try to learn from the ideas, literature and also fought back. Yeah. And suddenly that's a clear cut case. In fact, I why has this new project coming up got into a perception guided task training? That's something where the robot will teach humans how to do a factory. What does any agent teach factory worker? How to do new physical activities by example. So it's a robot coach and that winds up having all these issues.
You know, you need to have a marriage. You need to also have event shot and use them to provide these kinds of support. Okay. So. Well. So everybody out there please ask your questions. We've got another few minutes while we're waiting Sittler. Any other questions. So one other one for me. So so I was really pleased to see you allude to the work on the social brain hypothesis, which is this idea that we have big brains, which enables everything because we have large social groups.
So we need to keep track of complex social relationships. And the famous work there, I think is work by Robin Dunbar at Oxford. Evolutionary psychologist, which is lovely, lovely work. It's the kind of work that I look at and think, well, I wish I did that kind of research. You know, that's. So what do I think about that work? I mean, the the example that I use to illustrate it is this six word dialogue that's due to Steven Pinker where where Bob says,
I'm leaving you and says, who is she? Right. And I love it because it's so crisp. Six words. And again, it's incredibly rich. And we all understand what's going on there. And we can all fill in all the details about what's going on in the relationship between, you know, we can draw rich, rich, rich pictures of these two individuals based on just those six words. And that a very important part of that understanding is our kind of common sense theory of mind.
We all have. So what I'm puzzling over and this is entirely independently of your view of your talk is is it a target? Should it be a target of a A.I. to build machines that can understand that dialogue? I mean, you imagined as a robot in the room with with ball and should it understand should what should it do modelling and understand that dialogue? I think it's certainly my say. I said basic.
I mean, so what would the worry, presumably, that you have is that it might lead to other unethical uses. I mean, I think that ship has sailed my sentences and everything. I mean, that's what I think. I mean, every technology is dual use technology and intelligence is the ultimate technology. So everything we do in the context of intelligence is going to be dual use. You cannot stop developing it. You have to dial up with complete care.
They can to make sure that, you know, some some sorts of ethical framework is being sort of followed, even flying them. So my science and my sense of as an AI problem, as a problem of not understanding intelligent interaction. This is very much relevant to actually fill up these blanks. I mean, you know, using the significant amount of common sense background knowledge, what actually has happened as to how those sorts of things lined up, becoming a problem, of course.
Everything I talked about almost can be misused. And so that's that's just the issue. Yeah. So I would say it suddenly is what worth doing and in fact, part of the idea. They will be dealing with inferences in the context of rich background knowledge. I've sort of made it simpler by saying the model only with respect to the model. The inference is being done, but model plus everything else you know about the word will come into the picture. Doing those influences. OK. OK. So we go.
I think one last question. So the question is human working relationships require time to establish after a few months with a new colleague. One starts to learn whether competence can be relied upon. I think there's a backstory there. And for example, jargon develops. To what degree are your eyes learning about individuals of different categories of human behaviour and building these relationships over time?
So that's a fantastic question. So as I mentioned, this one slide about longitudinal picture of model following and model communication. This is something that, you know, I've sort of used to motivate a lot of our work, more stuff or beginning stuff. I work up until, let's say, last two years back was in the fast shot explanation. But longitudinal explanations lined up being changing your mental model of the human as well as JD noticing that data on commodities changing.
A classic example is this idea that if the robot is doing something inexplicable in front of it, who won the first time? They'll be surprised. The third time they will still be supplies by 17th time. The have nots are placed on by the 18th time and the robot actually realises that it's making a mistake and starts doing the more explicable thing that humans do against uprights, because by this time they went around changing their model of how the human robot marriage.
And this is something that happens in longitudinal scenarios. In that cluster work that I mentioned, we are actually trying to use that to see that if you engender enough trust, then you will basically not how to continually show explicable behaviour. So if you're shown enough explicable behaviour, some number of times, humans, basically how I trust them. So they have less of a monitoring going on on you, at which point you can live your life and do more things that are more optimal.
Again, when you're doing this, whether in fact that's going to be unsafe or not, that's it. Actual separate issues or whether that robot is doing something deliberately unsafe with a human. Is that something that we have to still deal with? No, but the idea that our mental models will be changing during the interaction and so explanations will be changing as the mental models change is very much part of the work that we are doing.
The more recent papers, in fact, is one that I didn't get into, which talks in terms of a Bayesian account of all these things where you think of what the human essentially has at the Bayesian prior or what a marriage that is a D of a possible different models that are what has and then Ventilla that humans are looking at what the robot is doing effectively, that the prior is being updated to be posted here. And so there are new probability masses, all of this.
And in fact, there's always this possibility that one of the models that the humans have is, oh, my God, I have no idea what the robot is doing, which is what we call this Amistad, which means this is you admit to yourself that I have no clue what this person is doing. And I just hope that that would be the least likely explanations for the behaviour. And the more that becomes the most likely explanation, the more the inexplicable the behaviour is.
And so updating this distribution or that longitudinal interaction is a great way to answer this gentleman's question as to what how do you learn that the robot model is that marriage is changing or the interaction? Okay. We do. We do still have like a minute. We have one other question that's come in. Could this kind of modelling be used for machine to machine interactions? If so, would you need to change the way you form models of what the other agent expects and have written?
I'm really sorry if this is a stupid question. We know exactly is our life is spent dealing with stupid questions. That's fine. Actually, it's actually a great question and I thank you for asking this. Yes, it can be done, but no, I won't do it that way because machine and machine interaction, I would say that the thing that we are missing really and I hope Elon Musk won't be successful in changing this is we don't have a USP connected to our brains.
So there is no brain to brain Grantland's. Okay, so you can only interact with these humans in this sort of just like we can't eat pizza or e-mail. We can't learn each other's mindsets by anything other than sort of watching their behaviour and making hypotheses. And so that's. What makes that human interaction more complex than human human interaction also has been more complex because of that. If I was designing robots, in fact, I would say they will have a hive mind.
You know, if I have a whole bunch of robots that have to walk in the factory, there's no real reason why they have to deal with it. Am I being extra able to do that? I guess I'll have the same mind. I'll just make sure that you have the same mind. And then maybe somebody from North Korea will have a different hive mind. You just have to deal with that back. But there are cheaper solutions if it is machine to machine interaction. That's some pops up. This is all relevant.
But there's a lot more that is really because of human in the loop here, because I still think that I'm confused. I'm not. This is the problem, right? What? The pandemic. We did everything on Zoome except eating pizza, which we had to do physically. And we are sort of inefficient that way. And same thing in terms of understanding each other. We have to do this in this inadequate way. So that makes things harder.
But machine and machine, you can still do it in the hardware. But they are also easier ways because you can design the machine. And I'm hoping that we won't design that human ward. And I'm hoping that neural link doesn't become successful, at least not in my lifetime. I don't think I thing. I'm with you on that one. Thank you so much for a wonderful lecture. We very much appreciated it. And let me just say again, you are most welcome physically in Oxford when all this is over.
We would love to have you. And I think you've you've now got a whole bunch of people who would love to meet you in person. So you'll be most welcome. Now, in the meantime, stay safe. And thank you again for those that haven't. I really urge you to look round pieces in the hill and also follow him on Twitter if you're interested in A.I. and for a humorous and informed commentary on the current state of A.I., there's no better place to look, I think. So thanks again, Raph, and take it.
Thanks again for the invite. Mike.