Algorithms Eliminate Noise (and That Is Very Good)

Welcome, everyone, to this online event, which is part of the Colloquium on Ethics presented by Oxford's Institute for Ethics in I. My name is John Tissue this and I'm the director of the institute. Our topic today is the ethics of algorithmic decision making. And I'm delighted to welcome our main speaker, Professor Cass Sunstein from Harvard Law School.

Cass will be well known to many of you, is the most cited American legal scholar and an exemplary practitioner of interdisciplinary research, drawing on fields such as behavioural economics, philosophy, amongst others. He's written in on innumerable topics, including in defence of socio economic rights, on legal reasoning, on the rule of law, as shown by his recent book, Law and the Vieth and with Adrian the New.

And he is a leading proponent of nudge, theory and regulatory policy, developing that idea within a framework that he describes as libertarian paternalism. His intellectual range is so great, it extends to a galaxy far, far away, as illustrated by his book The World, according to Star Wars. In addition to his many academic accomplishments, Katz is also a prominent figure in American public life.

He was, for example, head of the Office of Information and Regulatory Affairs in the Obama administration. Our topic today, as I said, is the place of algorithms in public decision making. If we think about legal adjudication, for example, the allure of automated decision making is evident we face throughout the world. Massive problems of access to justice rights. People have, as a matter of law, remain merely paper rights.

If they cannot be effectively enforced, for example, because of prohibitive costs or excessive delays, algorithms present themselves as tools for achieving faster, cheaper justice. But of course, there are also challenges if algorithms operate on the basis of big data, a vast mass of prior human decisions. Isn't there a risk that they will reproduce and amplify the biases inherent in those past decisions?

And even if they are free from bias, can algorithms be devised that are sensitive enough to balance all the considerations a judge must take into account when, for example, sentencing a defendant? And even if they can reproduce just outcomes, can algorithms do this via a just process? Often the workings of algorithms are opaque even to their makers.

Which strikes against the demand for transparency. And even if we have transparency about how an algorithm reaches a decision, it's process of reaching that decision is unlikely to resemble the reasoning process engaged in by human judges. So we might end up with a justification of the wrong kind. Finally, even if all these problems can be sorted, many will find it creepy and dehumanising.

Tough decisions that bear on our most basic interests being taken not by our fellow citizens, but by machines. Doesn't this undermine the reciprocity and solidarity amongst citizens that we hope for in a healthy, democratic society? In the face of this kind of many scepticism, Kass has been a leading advocate for the positive case for incorporating algorithms and human decision making. He sees them as potential cures for the bias and inconsistency exhibited by human decision makers.

So we are very pleased to have Cass with us today, and we are also delighted to have two distinguished commentators. First, Professor Ruth Chang, who is the chair of jurisprudence in the Oxford Faculty of Law. Ruth's approach to law is through a deeper, original theory of the nature of practical reason that she has developed.

So she is an ideal commentator for this event. Our other commentator, no less ideal, is Professor Sir Nigel Shadbolt, who is principal of Jesus College and a professor of computer science at the university. Amongst his many other roles is chairman of the Open Data Institute, and he is the co-author, along with Roger Hampson of The Digital Ape How to Live in Peace with Smart Machines. Now, I should say there will be an opportunity for Q&A later on.

So please put your questions in the YouTube comments section, and I hope to get round to as many of them as possible. So let me now invite casts to explain to us why, as he puts it, algorithms eliminate noise. And that is very good. Yes. Great. Thank you, John. It's a great honour to get to speak to you all at this amazing set of sessions. And John's new organisation and to have these commentators is also an honour.

I confess that I am particularly thrilled to be doing this because this is the first time I'm going to be presenting on a multi-year research project with Daniel Kahneman and Olivier Sabantuy. We've been working on this subject, which I was sorry that some noise in a very different sense on this topic. We've been working for many years and algorithms are central to what we're exploring. I think to orient the discussion, I'd like to present a figure that's called the Dark Rabbit image.

We get that up. There you go, see the duck rabbit. OK. I'd like each one of you to see, say, whether you see a duck or a rabbit inside your own head. Can you do that? See the dark rabbit? Or just see the dark. Or just see the rabbit. Think to yourself, if you would, what is that you see? You see both, OK. We can take it down. Thank you. And what I'm going to suggest is that the duck rabbit exemplifies both bias and noise around Easter time. People tend to see a rabbit and not a duck.

People generally see ducks more than they see rabbits. And the best theory for this is bias. That is what you see is a product of your preconceptions. And around the time of Easter, people are thinking bunnies. And evidently around the time of, let's say, January or June, people are thinking ducks because they think about ducks more than they do about rabbits.

So let's just think that preconceptions are a determinant of perception and this is associated with it consigns use of the dog grap duck rabbit image. I'm very confident that in addition to bias of the sort just described, there was scatter amongst those of you who thought what it is that you saw. There was scatter in the sense that some of you saw Duck and some of you saw Rabbit.

And that suggests that you as a group were demonstrating system noise where system noise is a system that is showing an unwanted variability. I want to stipulate on wanted and say a few things about what that means. If you go on a scale and weigh yourself in the morning, the scale could be biased, but not noisy. My scale always shows me actually six pounds heavier than I am. I don't know why, but it just happens to do that.

I know for a fact there are other scales that are noisy in the sense that they will show you a little heavier than you are, a little lighter than you are. They are all around the true value to get clear on the difference between noise and bias one more time. You could imagine a team of shooters who are archers, who are constantly going northeast of the target. That would mean that they are biased in a predictable way.

Or you could imagine a team of shooters that is scattered all around the target and they are noisy. These are different forms of error where one is systematic and the other is random. We know from the medical practise that doctors show both intrapersonal and interpersonal noise, and this is associated with error where some doctors will be likely to do certain things, let's say in the morning, but not in the afternoon.

That's intrapersonal noise and in domains where you wouldn't expect to see noise, doctors show interpersonal noise with respect to reading x rays, for example, or diagnosing heart disease or lung cancer. The medical profession is actually quite noisy and the noise can be ascertained without knowing what the true value is or meaning we detect scatter. It might be a product of a bias in the sense that some doctors might be particularly inclined to read an X-ray in a certain way.

And that may be erroneous. But to detect noise, we don't need to be clear on what bias is at work or how the magnitude of bias. OK. To get a little more specific about the operation of noise, this algorithm enthusiastic project actually becan when cognomen was doing a an assortment of work for an insurance company, asking the underwriters at the company to come up with an assortment of premiums for places that wanted to insure with the company and what was discovered at the firm.

This is real people whose job it is to be underwriters is that they were extremely noisy in the premiums with which they on which they settled. That is the magnitude of variance across similarly situated underwriters was really high. Now, given what we know about interpersonal noise, that itself is surprising, but not stunning. What was stunning was that the executives at the company were startled at the magnitude of noise.

They predicted that the level of noise they would see would be a very fraction of the level of noise that they actually observed. Which gives rise to and I'm building up to algorithm's a mantra, which is wherever there is judgement, there is noise. And more than you think, Oh, Daddy. To get clarity on the nature of noise, I think we have to make distinctions amongst different kinds of toys.

And to do that and to see some of the comparative advantages of algorithms, I'd like you to play a little game with me, a little exercise in which no one's going to be asked to write or say anything, but in which you'll get the intuition. Imagine that we all share certain friends. Maybe that's even true. Everyone is John's friend. Let's just assume that. But we're not going to do this with respect to John.

Let's assume that we are ranking our friends along three dimensions on a scale of one to five with respect to kindness, diligence and intelligence. OK, so we're taking our friends, Carol and Carl, and we're ranking them in terms of kindness, diligence and intelligence. And let's suppose all of us know Carol and Carl very well. I've actually engaged in this exercise with my spouse and there is going to be a lot of noise.

That is Carol. My God. A four from you and a two from someone else who knows her. With respect to kindness. And this will be somewhat startling because you both know Carol and Carl really well. OK, what's going on in these distinctions? Which map on, I suggest, to multiple domains of human judgement? OK, a first possibility is occasion noise. Let's call occasion noise, intrapersonal noise in which a human being will make a different judgement depending on the occasion.

So it may be that in hot weather, someone is going to get a higher rating in terms of diligence than in cold weather, which is to say that it may be that a judge the day after his favourite football team won, will be more lenient on a criminal defendant than the day his football team lost. And there's actually data suggestive of exactly that.

Or it might be that a judge on a day in which something has happened in the world that's very cheering will be more lenient than on a day in which something happened in the world. That's the opposite of cheering. This is just a way of saying that one of the reasons the exercise involving Carol and Carol produces noise is that people are in different moods because of different external circumstances and that produces intra personal noise,

which will vary from person to person. The second source of noise to which algorithms are not going to be subject, let's call it level noise. It might be with respect to kindness. That person A is just more lenient. Greater than person B, such that a person who's kind of kind will, on a scale of zero to five, let's say, typically get a four from one person where someone else will give that person a three.

With respect to punishment judgements and with respect to judgements about how to rate applicants, let's say for university or for a job, some people will be different from others in the sense that they will show level noise. They are systematically tougher or more lenient than others where the words toughness and leniency are placed. Place holders for a broader away array of evaluative judgements. The dominant factor in noise we're finding empirically is not occasion, noise or level noise.

We call it now pattern noise by which there's an interaction between people's assessment and the particular category that they are assessing. It's not a systematic level noise difference. So a judge, let's say in the criminal context, might be really tough on auto theft, but not really tough on financial crimes.

Or the opposite. Or a recruiter at a firm might be really upbeat, let's say, about people who came from socio economically challenging backgrounds and not very excited about people who went to Cambridge or Oxford. And there might be someone who has an opposite set of patterns. The idiosyncratic nature of individual judgement and a source of noise as a source of noise is often a product of pattern noise.

And this can be found in medicine and law, as well as in Olympic evaluation of, say, ice skaters, et cetera. And in the award of damages for wrongdoing, there's a fourth source of noise, which I think you'll get immediately with respect to that Carol. And Carl exercise, which is scaling noise, how you use a scale putting occasion noise and level noise and pattern noise to one side, whether the scale is pounds or numbers. Some people will think, well, Carol is a four.

She's pretty kind. And someone else will think Carol is a three. She's pretty kind. They don't disagree about anything except how to use the scale. And I found with my little exercise with spouse that scaling noise was a significant contributor to our different numbers. OK. I hope at this point minds are focussing on the likelihood that wherever there is human judgement, occasion, noise, level, noise, pattern, noise or scaling noise might be the culprit.

And I hope you're also thinking that group deliberations can be a magnifier of noise. So if you have a group of people who show, let's say, a high level of punitive intention with respect or wrongdoing and another group that shows a somewhat lower level, the likelihood that you're going to see system noise because of the differences between the two groups is actually really high. As groups magnify the effects of their antecedent inclinations.

OK. For examples of both interpersonal and intrapersonal noise that will reflect all four of these things. Let's just notice that for criminal, the criminal justice system, for hiring, for medical judgements, for judgements of underwriters and judgements of juries, for judgements of whether to admit people to certain programmes or judgements about whether to promote people to certain positions, we are going to see noise may be produced by an unshared bias, but it might be very hard.

And deceivingly to identify the bias that's predictive of noise, you might see the scatter without knowing exactly what is the source of the scatter. I'm smiling a little bit as I talk because I'm thinking of something I hadn't thought about for about eight months, which is at an early stage of this project. I discussed someone on the admissions committee of a very distinguished university in the United States that shall go nameless.

And the person on the admissions committee said basically went like this and said, you have no idea. Our admissions decisions are so noisy. It really depends on which person tends to happens to be looking at the file. And this is a hidden secret. It has nothing to do with affirmative action or the standard things that are discussed. It's just we're really noisy and we don't do anything about it.

OK. Given the omnipresence of noise and part of our work over the last years has been in investigating noise in different domains that have been taken separately and just uncovering the scandal, that is noise, I would be a very good idea to think, given the possible solution, that algorithms might be. What's wrong with it? Let's notice. What's obviously wrong with noise is unfairness.

We're similarly situated. People are treated differently and that might involve the criminal justice system or who gets disability benefits or who gets hired or who gets promoted. It might involve costs. So the intuition about. Noise is that the costs, that the errors will cancel themselves out if they're scatter. And that's a very hard intuition to get past it. They will cancel out if we take the average or the median. But in life, typically, the costs are additive.

They don't cancel out. They compound each other, which means e.g. in the insurance company, the fact that there's so much noise, just the computation suggests produces massive economic harm to the firm. And it would be very nice to be able to isolate the detached domains in which in which unfairnesses the problem and the domains in which spiralling costs is the problem. OK, how algorithms potentially will eliminate bias.

And by definition, will eliminate noise. I'm just going to give two examples. One thing that is kind of a primitive algorithm, which is the Apgar score for little kids appearance, Paul scrimps activity and respiration. It doesn't involve computers, but it involves something like an algorithm. And the effect of the Apgar score for health assessments and helping little kids is huge. It eliminates bias on the part of doctors of all kinds.

And given the Apgar score, the level of noise doesn't quite vanish. But it comes really close to vanishing. The most dramatic study of kind of the algorithm, literal with respect to bias and noise reduction, involves Bayle studies where replacement of human decisions by an algorithm shows that crime can be reduced by up to 25 percent.

With no change in jailing rates, if we substitute for the human decision an algorithm, a simple algorithm, or you can reduce jail populations by up to 42 two percent with no change in it would no increase in crime rates so long either margin. You can do a whole lot better. We know that the two sources of human inferiority to algorithms are first bias, not racial bias, but current offence bias.

Let's call it. By which human beings overvalue the current defence in deciding whether to give people bail. Algorithms don't. That's eliminated by algorithms and judges are really noisy. Judges show intrapersonal and interpersonal noise and algorithms eliminate that. OK. A puzzle is you could have a really dumb algorithm that is not noisy and that would be inferior to human judgement.

So, as John suggests, if an algorithm is trained on some characteristic X where X is a product of race or sex for race, let's say arrest records. It may be that the algorithm would be noise free, but racially inflected, potentially more so than human beings would be. If you said that everyone convicted of a certain crime gets a five year sentence or no one gets a promotion in 2020, it would not be a noisy system, but it would be a biased system and therefore an erroneous system.

The basic idea here is that the choice between algorithms and, let's say human decision making in situations in which the algorithm is kind of dumb is a rule standards problem, which involves the costs of decisions of costs and costs of errors, where it's potentially the case that the algorithm will be inferior to human decision making. That suggests that every algorithm has to be interrogated both along the dimension of bias.

And along the dimension of noise, always the algorithm will do really well along the dimension of noise. And that's very good. Thank you. I really appreciate that. So now we're going to turn to our first commentator, who is Ruth Chang. Thank you. Thank you. It's a great honour to be here, and I really appreciate the opportunity to be part of this conversation with Carson Nigel, about ethics and A.I. Thanks to cast for typically brilliantly interesting and thought provoking talk.

And to John and Torch for inviting me to take part and thinking about it. So here's a picture of the lay of the land on work, on algorithms on one side over here. You have computer scientists and engineers led mostly by industry dollars, excitedly working away at developing algorithms and applications for them, such as radiological diagnoses, decisions on bank loans, facial recognition, autonomous vehicles and much more.

Over here, you have the academy, especially humanist and social scientists, writing articles and books about how dangerous algorithms and A.I. in general are trying to slow down a AI until we know what we're doing. Now, here's an interesting fact, the two camps agree on one thing, that a guy needs regulation and it needs it.

Yesterday. Even the leading computer scientists who can see the great dangers inherent in current development and application of algorithms regard what's happening over here in the development and application of algorithms as a bit of a bit of the Wild West. And they're begging the folks over in the academy to help bring a sheriff to town. And I'll just show my cards. I think we all have a moral imperative. Those of us in the academy to think. You know, this is what we should be doing.

We should be figuring out ways to make sure that guy we build serves rather than harms the human condition. So it's against this background that cast who is about as expert on the subject of regulation as anyone on Earth could be, offers us a somewhat surprising range of work. He's a cheerleader for algorithm's. So instead of giving us a recipe for how to regulate algorithms, the offices hope for algorithms that they can help solve bias and most importantly, noise.

And there's another point and plussing what he said, that we could emphasise that even if we can't solve the bias problem, we can use algorithms to solve the noise problem. And here are two things I wrote down for what he just said. He said, to detect noise. We have to solve. Oh, sorry. Yeah, to detect noise, he said, we don't have to solve the problem of bias. And later on, he said. Algorithms for move.

Noise by definition. So if algorithms can help us be more consistent and say insurance rider evaluations and beyond. That's a very good thing. So maybe we have a use here for algorithms that everyone can agree on as being a very good thing. And we don't have to get all nervous about regulating it. But I worry that the focus on noise obscures the fact that you can't identify noise independently of tackling the headline or problem for eye,

which is at least for now, bias. Now, bias follows readily from the fact that our social structures advantage some over others. Carson is the first to recognise this. And the question is how? The questions we put to algorithms are framed is very much a reflection of an reproduction of those background conditions. This is what John was referring to when he talked about the training data.

So just to take a familiar example, new mothers must choose between going back to work and putting their isn't in day-care. On the one hand or staying at home to care for their infant child and losing their place on the career ladder. Now, you can imagine devising an algorithm that helped a mother determine the probability of her being able to re-enter the workforce after a prolonged spell of caretaking and evaluated against the probability that her child would.

I don't know, turn out to be a drug addict or a rebellious teenager because she spent her early years with a paid worker. By setting up an algorithm to answer this question, which alternative should the new mother choose? We believe in the bias against new mothers inherent in our current social structures, right in today's social structure. You can't both caretaker is in and remain where you would have been on the career ladder.

So algorithms can quite easily obscure rather than make more transparent the underlying bias in our social structures. The question we should be asking about the algorithm is not how should we think about the Trade-Off between leaving work and not raising your own child, but how can we arrange our social structures so that mothers can be with their infants during the workday if they so choose without any sacrifice to their position on the work ladder?

The danger of algorithms is that they just further entrench our existing biases. Now, the same might be said also about the case of bail bond hearings cast suggests that what matters to a judge in a bail bond hearing are two things. The probability that they will commit crime and their flight risk. But why should we accept or assume that only these probabilistic judgements are what bail bond hearings are about or should be about?

A good judge will care about so much more. The safety of the defendant. Out on the streets. For example, he is flipped on his fellow gang members. The fact that a figure of authority has shown trust towards him and the effect that might have on his life going forward and so on. John Cassilis has recent work cataloguing these and other ways in which bail bond questions involve multiple factors and much more complexity than the experiments.

The cast refers to allow, in this case, the use of algorithms again simply entrenches and reproduces biases in framing the question that leave out the human element, if you like, in decision making. OK, but what about noise? Surely you want our judgements and decisions to be consistent. And if algorithms can help us with that, we should be enthusiastic about it.

Is noise different from bias, if it is and algorithms can help reduce noise, then maybe we can sidestep all the worries about bias and find a safe place where algorithms can be used with gusto. Unfortunately, I don't think this is the case. This is because, in short, one person's noise is another person's bias. Or put differently, as lawyers put it. Treating like cases alike. At the heart of the rule of law is itself a substantive normative matter.

You can't determine where the two cases are being treated alike without already first ascertaining that the differences between them are irrelevant to the question at hand. And that is a normative judgement about whether there is bias in treating a difference as relevant or not. Here's an example, suppose last Friday your teenager asked you whether he could stay up with his friends past midnight. Socially distance and wearing a mask, of course. You just read that Trump came down.

So you say no. Three weeks later, your teenager asks you again. Well, now case counts are diminishing. He's been cooped up in his room and you're worried he's losing his connexions with his friends. So you say yes. You've made two facially inconsistent judgements. But, of course, there is no noise here. There are differences that, normatively speaking, make a difference and justify different judgements.

We can't make a judgement that two judgements are inconsistent without first ascertaining that the difference in the circumstances of the two cases are differences that do not make a normative difference to the judgement. I think the same can be said about Kassin example of Reinking France according to criteria of kindness, diligence and intelligence. So the general point I'd like to raise is that we need to restrict our use of algorithms and I generally.

To help solve only one. Quite restricted type of problem. The computationally tractable. I suggested that you can't ascertain something as a case of noise unless you first solve the problem of bias. So noise isn't a separate, safe niche for algorithms. We're back to the headline problem for algorithms, which is bias. But you might say, well, let's just be extra special, careful in formulating the correct questions for algorithms to minimise bias as much as possible.

And then we're off. We can use algorithms every. But I think the real problem is that algorithms are limited to computationally tractable problems. Only. And deciding whether to stay home and care for your infant. And I would submit granting or denying bail or breaking your friends. These are not computationally tractable problems. So what is it to be computationally tractable?

Well, this is a large question, but at a minimum, to be computations tractable, a problem has to have an adequate numerical representation of the factors involved in the decision, along with a numerical representation of how the various factors weigh against one another. But there's a large literature and philosophy that shows convincingly to my mind that most of the problems we care about are not even in pieces computationally tractable problems.

Is there really a scale of kindness according to which we can rank our friends with respect to their kindness? And if there is such a scale, can you really put the kindness of your good friend Carol on it as opposed to your good friend Carl? Many philosophers argue that is constitutive of being friends. That you can't do this. So I'm gonna end by considering the duck rabbit case.

It's an odd case. And I want to suggest that there's some reason to think, well, it's really not a case of bias or noise. The truth we all know is that the picture is a duck rabbit, it's neither a duck. Neither sorry. Neither just a duck. And it's neither just a rabbit. So one moment I look at it and I judge it's a duck. And then the next moment I look at it and I judge is the rabbit. That appears to be a noisy set of judgements.

But now we look at the differences in the circumstances in which I make those judgements and we see that those differences are relevant to justify the judgement. So neuroscientists tell us that the explanation of why we sometimes see a duck and why we sort of see a rabbit has to do with a pattern of our eye movements of our eye, a light at a certain spot, and then travels through another spot. We'll conceptualise a duck. And, you know, similarly for a rabbit.

So when I judge that, I see a duck and then a minute later I judge that I see a rabbit. Those judgements are facially inconsistent, but they're not really inconsistent. They're not really noisy because there are relevant factors that explain and in some sense justify the judgements I make. So I'm going to end with a little point about transparency. One thing about algorithms is it seems like they they list all of the factors that are in play.

And then we can see how they should be weighed up against one another. That allows us to say if we decrease the importance of one factor and increase the importance of another, look at the outcome. And that's pretty cool. We can learn things about trade Trade-Off between different factors. But I I don't think that anything that an algorithm does these be transparency is something that I don't know how else to put it.

But a philosopher. A person who is actually thinking about the various factors can't do and can't do better. And I'll leave it at that. But thank you cast for that really interesting talk. Thank you so much, root for a great set of comments. And now I'll pass over to Nigel, Shantal, Nigel. Thank you very much and thank you, John, and thank you to torch for organising this. And of course, it's a huge pleasure to be responding to Karzai's excellent talk and compelling observations.

A really powerful enumeration, I think, of our susceptibilities to different kinds of noise, bias, etc in human judgements and how algorithms might help. And I really enjoyed was brilliant insights also. So so one of my roles here at Oxford has been to help establish the institute and helped develop the original case and and now Change Institute steering group, the founding principle was to locate the institute in the Faculty of Philosophy.

We wanted the institute to confront substantive and real ethical challenges using the best philosophical arguments, insights and minds we could muster. Now, of course, the internships got to reach out and be informed by a wide range of other disciplines computer science, a whole range of mathematical, physical engineering, medical and social sciences. And the law, politics and economics. It must reach out widely because the questions that it must address are both technical and societal.

And tonight's discussion is really excellent, excellent example of such a set of questions. So a set of ethical questions. And they either revolve precisely around the issues of when to use A.I. and when not to use it around whether its results are fair or equitable, or whether there is sufficiency of access to the A.I. systems themselves.

And I'm also a professor of computer science here at Oxford, and I've been researching I for almost 40 years, and in that time I've seen the various high points and low points of the subject, the various enthusiasms and disappointments that have followed in its wake.

And I was reflecting back to the 1980s when, as a young researcher, we were busy building knowledge based systems, sometimes called expert systems, and these were rule based engines using a range of technologies to explicitly reason and represent over a problem domain.

So from designing computer hardware to diagnosing medical conditions, from planning logistics to improving the design of aircraft, some of my colleagues I remember even attempted to codify the British Nationality Act that turned out not to be amenable to logic based reasoning at all. Perhaps no surprise that, but many of the systems were built by taking the expertise of humans and codifying them through a process of knowledge.

Engineering and the resulting systems often raise significant questions. Where are they going to replace the experts whose knowledge they codified? Could we trust their results? Were they able to explain or account for their results? Could you ever hope to accommodate the rich range of context that humans are able to marshal when solving problems, the challenges of A.I. are those not new, but with the scale and power of new hardware and software, the availability of data?

There's a new urgency in the questions. So back in the 80s, one common approach to help people not to worry so much was to reframe the system as a decision support system. It was to be seen as augmenting, not replacing human experts and decision makers. And under this rubric, algorithms would help where people succumb to bias or functional fixity. Or else we're locked into a particular way of looking at the data.

Indeed, there was a style of knowledge based system that was referenced as critiquing systems, and they were explicitly designed not to replace the expert, but to ensure that the full range of factors have been taken into account when reaching a decision. That alternative search path had been pursued and other possible outcomes evaluated.

Now, the tension between human problem solving and algorithmic reasoning is not a new one, and certainly we could do quite well, I think, to see what our modern generation of deployment's today. I can learn from the lessons of the past. And one of the most interesting was this recognition that we need a marrying of human and machine capabilities.

So, as I've noted, Modern Eye, in fact, comprises a broad suite of methods, some using explicit reasoning, role based, rule based reasoning and others various forms of neural network systems where the ability to look inside and interrogate the internal state of the algorithm is more problematic. Now, what are the compelling arguments that both Cass Sunstein and in other background papers provided for this event?

Daniel Kahneman paper makes is that sometimes effective decision making doesn't have to be as complex as people think and that relatively simple algorithms can do better than humans. That decision making in humans is variable in many contexts, that individuals are inconsistent both within and between themselves, and that they're susceptible to a range of bias and produce inherently noisy behaviour.

I can't diminish from that wonderful work. The seminal work of researchers such as Kahneman and Daniel Kahneman, Paul's Paul Slovic, Amos Persky, who showed just how pervasive cognitive bias was, how the use of particular heuristics overwhelmed the natural statistics of situations. And I remember reading the heroic attempt to rescue statistically normative models by psychologists of human decision making.

If we weren't real Bayesian, as perhaps we are just yeah, perhaps we weren't able to modify our estimates in a statistically defensible way, then perhaps we were degraded. Bayesian is defeated by the inherent noise and uncertainty of the world. But it's pretty clear, as CAS and people like Daniel Kahneman have been saying for a long time, that in most cases humans, even statisticians, succumb to the bias of availability, anchoring representativeness and so on.

So the elimination of some of these tendencies might seem to be a good thing, particularly if much that's touted as advanced. AI is relatively straightforward predictive statistics and that it can help. But then the question revert back to the ones that really Ruth has already alluded to. It's how far is the reality and context of decision making captured in the impoverished context of the algorithm? And what are the challenges that arises in applications like sentencing and bail?

Particularly if automated, is also this sense of what it is to be explicable. Some of the work in my own group has carried out studies on on how System X flick ability should work and there are clear choices here. Do I offer up as an explanation the inputs to the algorithm or the comparison to similar cases?

Or a contextualisation within the distribution of cases learnt over the Dimmock, the demographics of the distributional demographics of the cases learnt over or the extent to which an outcome is sensitive to particular features. This aspect of the application of algorithms in context, such as bail and sentencing, call out another fundamental concern in a AI ethics. The reasonable expectation for contextualised explanation.

Of course, there might be an expectation that some explanation relates to streetball machine processing. How does the machine do it? And we know that in the case of many modern machine learning systems, this can be very opaque.

There is no constant that can be offered up that we would recognise and the systems do not represent the world in the way we do and in many Pann recognition systems, we literally see this in so-called generalised adversarial networks which generate counterfeits that don't look anything like the confidently predicted outcomes that the original networks predicted.

Just an interesting remark on noise. There's actually an interesting feature of some computational algorithms to literally eliminate it. But there's some others noise is actually used constructively. For example, the fluctuation of signals can be used as a searching mechanism that drives computation and the search for a solution. But that's not the noise we're talking about here. But I thought that was an interesting point.

Can we be confident that the noise and bias in the cases that we do talk about here, that train these algorithms that we seek to deploy amply dealt with? And again, Ruth made the very good point that. Do we really know where we're looking at the source and nature of this place? Do we know that effective sampling and representation might not be an issue in itself?

And is there even a gold standard in the decision making itself? Can we be certain that algorithms that we apply our context to, where are we content that the nature of the explanation the systems used in the decision making are reasonable? But again, perhaps the question that we come back to is, what are the goals and whose goals are we affecting in the algorithms we deploy and what kind of bias is this?

Whose objective function is being optimised? And the essential question of the social embedding of these systems. You can make a judgement that two decisions that appear to be inconsistent can be contextualised to be appropriate for the situation. But all of this to one side, I'd just like to say how much I've enjoyed the observations and insights that both Cass and Ruth have shared with us tonight. Thank you. Thank you so much for your insights as well.

Nischelle, that was fantastic. There's a very rich array of comments that and I hope you won't feel obliged to respond to everything, but perhaps you might want to pick out some and sort of take five minutes, because we also want to get the audience involved as well. So these are fantastic comments. So I'm grateful, Ruth, to you and Nigel to you for for them. Let me try to be. I'm going to have a lot of words permitted, I think, because there's so much there for philosophers, lawyers and others.

Bias has charisma, noise, not so much bias as a star noises off stage. The basic goal of this is to restore the balance there with respect to everything we care about. Noise is really important and fully neglected. Take the following questions with someone. Have a heart attack. Will this product sell? Will a candidate be elected? Will the defendant flee or commit a crime? For all of those, four relevant people are noisy, extraordinarily noisy.

That can create extreme unfairness and produce systematic error. That is very costly, whatever values we care about. We can speculate and maybe even demonstrate that a bias accounts for the noise. That is, that some people are optimistically biased and others not with respect to all of those questions. But we don't need to do that in order to identify noise with respect to Ruth's point, one person's noises and others bias.

I want to be very careful about that, because these studies depend on on noise audits in which the same people are seeing exactly the same case. And some judges, because their sports team won or because it's cold outside, end up being more punitive. So it would be hard to say that they have some normatively relevant reason to be stricter or less strict than others. If Underwriter's turn out to have had a really happy morning, so they quote a low amount.

And what I'm just describing that is dominant every domain I've described, it's not as if there's a morally or other relevant difference. That accounts for the noise. There's not a normative foundation that me shows that we have wanted rather than unwanted variability in which one person is right. There's no evidence, zero evidence. After careful investigation that the differences amongst the judges in the bail study depend on judges caring about such things as Ruth referred to.

It's an empirical hypothesis which is quite plausible. It just turned out not to be true. There's no evidence that some of the factors that humans might care about and that intuitively they ought to care about which, by the way, are legally prohibited. So a judge who's doing that is engaging in civil disobedience. There's no evidence that judges are doing that. Every factor tested. Turns out not to account for what judges do. It's noise.

Ranking friends might be not the nicest thing to do. What's it stative about? The example is not that people think. I think Carol's really kind. I think she's not. No. It's that people use the scales differently. Ah, some people are in better moods than others. That's what's interesting about the exercise. And it's not about ranking friends. It's a heuristic for thinking about hiring. Admitting, suspending. Giving people Social Security benefits. Finding people for environmental harms.

Imposing damage. Awards for pain and suffering. Those are cases in which the unwanted variability not normally normatively relevant fine-tuned human beings. Unwanted variability is the driver. And the question is, what are we going to do about that? I think some of the concerns about algorithms are in the grip of a picture of human beings seising on relative factors to which algorithms are indifferent.

That sometimes happens, sure. But often the factors that are driving human differences aren't normally very relevant things. They're just scaling noise or occasional noise or something else that we can't discern. Should we reformulate the question for bail such that we consider those other things? Maybe. And then an algorithm can do that. If if we get an algorithm to do that, it won't be noisy. That's good. Nigel made a bunch of really interesting points about about seven different things.

I'm going to just mention one, which is it's tempting to think algorithms should just augment and not replace human beings to ensure a full range of variables are considered that the algorithm is alert to, while allowing the human being to be alert to things that the algorithm might be alert to in certain circumstances. That's exactly the right way to go. And a checklist is often a way to do that.

There are domains in which that will replicate the problem, where the human being which is been augmented, will be biased and noisy compared to the algorithm. So with respect to that, yes, sometimes no. Sometimes I think with respect to race and sex discrimination, it's absolutely right and indeed fundamental to see that race or sex and equality might be in the background and the algorithm might be piggybacking on that or aggravating that. That's really, really important.

But I think it wrong Foote's analysis, because to see race and sex as the first thing that comes to mind when we think of algorithms is to narrow, truncate really radically the domain of things to which algorithms are relevant, which was why I started with will someone have a heart attack? Will this product sell? Will a product the elected? Will the defendant flee or commit a crime? Done. Thanks. Thank you, sir. Thank you so much, Cash, that was excellent. We've got a series of questions.

Can I just ask you one question just to clarify a little bit about noise. So noise is not mere inconsistency, but has to be in some sense, unwanted inconsistency. It's unwanted variability, unwanted variability. So one of the difficult questions and I think Ruth kind of alluded to this, is you're gonna have to make a substantive judgement about what is unwonted variability.

So say you think, for example, in the sentencing case that there are pervasive, incommensurate abilities in the values in place such that there isn't a particular sentence? That is the correct sentence, typically, but they'll be usually be a range of sentences that could be imposed upon a defendant. And there you have situations will arise where different judges will impose the same same sort of case, different levels of punishment, but they're all falling within the range.

And we think that maybe this could be a good thing because it's a kind of human interaction, sentencing. So I'm interacting with this particular judge who is someone of a more lenient disposition. Let's say. Let's put aside the possibility that his team won the grand final yesterday and the other judge is somewhat of a more strict disposition. But within the range, the eligible range. Are you suggesting at this point that that kind of inconsistency is unwanted?

Or could that be an acceptable sort? My view is it's a scandal. If Tamma, let's say, gets a five year sentence and Tom gets a three year sentence and there's not no difference between time and 10. That's not merely undesirable, it's a scandal. Now, it's no doubt that there's some characteristic of Tom such that the judge is stricter or more lenient than with Cam. It's fair. So Ruth is completely right on that point. So you're putting a lot of emphasis on this, what they call horizontal equity.

That that is such an important consideration is not the consideration that I'm putting toward allowing judges to make a decision within the range gets sort of comprehensively trumped. Well, a range is helpful because it reduces the magnitude of noise. If one person gets probation and the other one gets life imprisonment, that's a very, very scandalous situation. But if one person gets five years and another gets three, that's deeply unfair.

All it depends on is that it's a lottery. So a way to put it is that the the existence of noise suggests that there are there's a lot of lottery entry out here and there are involuntary lotteries where whether you get asylum in the United States, it turns out, depends on a lottery. The judge to whom you're assigned. And let's stipulate that there are some people who definitely without asylum and some people definitely won't.

So it's parallel to your hypothetical. Still, it's deeply unfair that similarly situated people have their lives either up ended in a bad way or sort it out in a good way just by virtue of the lottery. And I think the human mind doesn't naturally see things the way I am suggesting it, which is one possibility. I'm just wrong. Another possibility is that bias is something which makes our lives light up, our eyes light up noise. They kind of glaze over. OK, I'm going to let Ruth come back.

She wants to raise something in response to what you've said, but we you could do this very succinctly. So we can add onto the Q&A. So just quickly. Even a case where take the Israeli judge case. After having lunch, the judges had a good move. And so he's more likely to grant parole. Is that noise when the counterfactuals. He wouldn't have granted parole before lunch for the very same. I'm not sure. Right. So I. I hear your picture. A walking that's kind of strict. Everything by the book.

It's better, but life is not like that, right, Cass? We know that life is full of luck. There's more luck. Whether or not the child runs in front of your car, you kill it. And we're part of human institutions, not machine institutions, where human interactions like John Sayegh is part of the system. If there is a small range, the discretion of a judge seems to me to be important because the bias he shows after lunch may actually be more accurate.

Right. So it'll tell us whether or not the bias is that the noise is wanted or unwanted. That's all. I take your point. So this is a real standards issue where it might be that the judges before lunch are systematically too harsh. So they are inaccurate by reference to normative standard, let's say, but noiseless. And it might be better to have system noise with leniency shown after lunch. So the unfairness we get is outweighed by the absence of noise free, let's say, harshness.

So I completely get that. Still, the idea that we should be satisfied with an institution in which whether you get a harsh sentence or lenient sentence or whether you got scheduled after a lunch, the judges eat. That's really unfair. OK, let me ask you two questions from the audience, which I think is kind of similar. I'm going to try to run them together. So the first question is, would are algorithms worse than group polarisation?

Is polarisation actually a manifestation of reducing interpersonal noise? And the second question is, when the training data comes, we've consistently biased human decision makers willing to trade-offs be reduced by noise and increased bias is completely in response to the second. So if you have a noise free algorithm that's either stupid or biased, we might want the noisy human beings more completely.

And that's really important. I think often the critics of algorithms find that as a trump card against algorithms rather than an invitation to discuss how to make either human beings through Nigel's suggestion, maybe as advised by the algorithms or better or make the algorithms better. That would be one possibility. Now, group polarisation means that groups tend up ending up in a more extreme point in line with their pre deliberation tendencies,

and that can create a lot of system noise. If you have several groups. So think of three groups of people deciding what to do about some, whether to bring a lawsuit. Some who are inclined might get really excited. Let's do it. Some are disinclined. Might in groups get really disinclined? No, let's settle. And the algorithm. If it's a good algorithm, will be more accurate. Won't run into the trap that group deliberation sometimes creates.

If it's about algorithm, we'll have noise free law firms. But they might be Ruths point more a room error prone. Let me ask you the question, which also I think maybe applies to Ruth. The question is, Ling's shouldn't algorithm's be judged by comparing them to the status quo? I think this is an interesting what's the baseline of comparison? The mother status quo. This is alluding to Roof's example is to ask other people, which is equally biased.

An algorithm could be transparent about how this works, how that process works. I'm thinking that, you know, some of us are teachers grading, and when some of us grade A paper, we don't ask other people. We just do our best. And the grading is noisy. And some of us are aware of our own interpersonal noise with respect to grading. Start moving toward an algorithm with more checklist, the approaches which more like a statistical thing. And that's less noisy and probably less biased.

The point is right, that if you have a decision maker who's resource is a single other person, might be more or less biased and more or less noisy. And do we need to know what the algorithm is aiming to maximise? I think that's part of Ruths points and yours. Also, John, so if you're trying to maximise crime reduction, it might be that's not what you should be after. And I say then have an algorithm that tries to figure out what you actually are after.

Or have the algorithm be an adviser, which will tell you Nigel's point about flight risk and then you can consider other things. Ruth, did you want to respond to the status quo issue? And just quickly. Yeah. So you ask your friend what to do and your friend says X. Then you ask your friend some more questions. Right. So that's what you have to do, is you have to figure out why your friend said X and that X algorithms are only as good as the inputs we we give them.

So can I just end with quick thought experiment? Let's suppose we get, you know, all the flatterers, lawyers and like together and we figure out all the relevant factors to a question. And, you know, we're pretty sure that the weighting is within this range and so on.

OK. I think the deepest difference between people who think, yeah, we need have a lot of algorithms take over human life, and the people who are worried about that is that it really goes back to what John said, that do we want to live in a world where there are human interactions that are subject to a bit of luck, contingency and so on? It's part of the human condition. Or do we want to live in a world where, you know, by the way, it's constrained?

Right. The luck is constrained. So it's not like it's death penalty or you're free to walk. It's five years or just four and a half years. Do we want to live in that kind of world or do we want to live in a world where we know you're gonna get five years, right? Five years, five years, five years. And I want. And I for one, I want to live in the other world, the messier world.

It's just just just a point on that as well. It's just just the sense that sometimes the focus on our task, achieving algorithms, we shouldn't forget that actually all of our institutions exist in these complex milia where there are many things going on. We have this reference to what we sometimes called social machines, components that have algorithms, components that are flawed human decision makers.

And then understanding those system properties is also important because actually you can be doing quite a good job. And I don't think this is necessarily contentious point on one of your algorithms. Make noise free, bias free elements and still have the system overturned by some real absurdities in the structural dynamics between systems. So I think that that's what that's worth considering. I think to Ruth Point, I agree with her in many contexts. So like friendship or something.

But with respect to flying a plane or driving a car, if you have an algorithm that is noise less and really good, it's certainly an empirical possibility that the fact that it's not a pilot making choices or a driver is going to save lives. I think also just might not forget that algorithms are not entirely determined simply on inputs,

that the extraordinary thing about modern plus about rhythms is. Yes, they are in some weak sense, but in a really interesting sense, they're exploring a landscape of solutions which is well beyond a simple set of input parameters. Can I ask another question from the audience? I think there's a number of these questions, but here's his one version of it. Wanting to reduce noise assumes we know the correct answer.

For example, university applicants, aren't we fooling ourselves by imagining a true ranking? Isn't a noisy selection the most fair country to not claim that fairness is one of the key driving factors? Noise elimination. OK, it's a good and I think deep question. So let's take cases where people get admitted based on the random draw of who the person who sees their file is.

If we know that some people are getting admitted who are along relevant dimensions, not as good as people are being rejected. Then there's unfairness. And I guess that's implicit in the question. Let's suppose we just don't know. It may be that the point is right that that kind of lottery is OK. But I want to think more about that in the case of the underwriters that the insurance company, the lottery, costing a lot of money now.

Does that mean we know something at least about the true value? Maybe. I bet we can find cases where we don't know the true value, where we wouldn't celebrate the existence of variability. Like if some doctors are saying, you know, go home, you don't have a problem. And the same doctors and different doctors or the same doctor and different day are saying surgery now.

I don't think we think that that's better. Can I ask you about your Harvard colleague, Michael Sandel, who has recently written a book proposing an element of randomness in admission to elite universities? So the idea is that there should be a threshold of achievement. But beyond that, of course, it's absurd to suppose we could sort of really have an ordinal ranking of different candidates.

So why not just throw them down the stairs and then pick up, you know, which are the two hundred, etc. randomly? And part of the thought here is that there's something quite oppressive about entertaining the idea that we can make these strict rankings. OK, so that's a great question and there's a lot of work on lotteries as just and cases in which you think that the criteria that would be used are morally irrelevant or something.

Maybe one's for a lottery, but I doubt Professor Sandel would think that grades should be given out on the basis of lotteries or book contracts should be given out on the basis of lotteries. Maybe he believes the latter. So that's a signal of a possible account of when, which and when not. And I think there's something about dessert and there's something about welfare.

And he may be right about universities, but that wouldn't mean that for other things like fines, you should have a lottery for fines. Nigel Ruth, is there anything either of you want to. No, I don't think that is.

That's an interesting view, again, about what what contextualises the situation such that you feel that there's some level of reasonableness in in a lottery based approach and that that really does come into some very interesting questions about what we feel is just and where the materiality on decisions are and whether or not we with our out within we're being suggested a kind of a false precision in anybody's making of.

Ruthie Johnson, interject at this point. Just that, you know, fairness is not the same as sameness. So I think we just have to be careful. OK. Let me ask you one other thing, which I think kind of relates a little bit to Nigel's thought that we somehow have to integrate the algorithm with some kind of human in the loop situation. Isn't there a risk Say, you're right about these things, say that even in the bail case.

We can reconfigure the algorithm so it's not so focussed on prediction of flight risk or recidivism, but takes into account other relevant considerations. What are the wider ramifications, though? And this is picking up on a question from the audience for human agency that we kind of become. Become a culture reliant upon the machines taking these decisions. And even if we in theory, have some kind of override, the tendency will be more and more, No.

One, to rely on those sorts of considerations which are amenable to algorithms and a number to just sort of depart from the scene because they acquire a certain kind of prestige or social authority, partly for the reasons you've given. And that puts us in a kind of abject position as autonomous reasoning beings. Well, I mean, my association is. Should we outlaw the G.P.S. device, just like philosophers use it as they need it the most? The best device is basically right there, isn't it?

It's something that reduces our own agency in the sense that it displaces our own judgement about I get from here to there consistent with Nigel's point, it is something we can override so we can think I like the scenic route. I want to be very careful about overriding the G.P.S. If the grounds for overriding it is, I know a faster way noisy and biased human beings, but if it's about seen, it could go for it.

So I think where we are now, one phrase this a little more starkly than I actually think it's that life is nasty, brutish and short. And one reason is bias and noise and algorithms have fantastic promise for making the person you most love have more years on the planet. And that that should not be taken lightly. Well, that is a very positive note on which to end. I want to thank everyone who's joined us today for what I thought was a really fascinating discussion.

I'm sorry I couldn't get to all your questions, but there were a lot of them. I'd like to thank Professor Cass Sunstein for being our main speaker today and introducing us to this very exciting new line of research that he's embarked upon. And I'd also like to thank our two wonderful commentators, Ruth Chang and Nigel Shadbolt.

Thank you very much, everyone, and look out for our next event, which is on November the 13th, where we'll have a lend lend more talking about A.I. and its implications for democracy. Thanks a lot, everyone. Oh.