Well, thanks very much. So this slide is like a happy medium between the first one and most slides at all. And so we just have a few bullet points for you. So I'm at the public school. I'm not authorised to speak on behalf of the Pivarnick School, but I will sample some of the things that we are doing there. So Peter asked me to talk about the work that's going on inside are in school and I haven't gone round and ask everyone.
We don't have a single group of people. We've got lots of people following their own research interests. And many of these research interests do have connexions with AI, machine learning and ethical issues of a number of different sorts. So this first slide is just a sample of some of the things I know that is going on in the school. And many of them have already been mentioned, although not all first, social media and democracy.
So as a school of government, we're very concerned about the quality of the democracy we're in. We're very concerned about the way in which Facebook and other companies have perhaps been affecting the way in which elections are conducted and the way in which information is getting out. And there are a couple of things that have been discussed in the school. I'm not sure anyone's produced any research on it yet, but these are developing areas. One is the obvious area we've already heard about.
I think a type of deliberate manipulation by which Cambridge Analytica has been accused of manipulating marginal voters towards the extremes. We don't know how serious that is as an issue. Views differ about it.
But there's another issue perhaps we're more concerned about, which is how this can happen in an accidental way, and that the business model of Facebook and others, the click driven module model where you got money for diverting traffic to particular sites, more outrageous site, the more money you'll get. And so there is no connexion between truth and money. In fact, may be the reverse. And so the quality of the debates is being corrupted by a business model.
And we need to think about the ways in which that can be done with. And the second area may be related in a way, but all around the world there is a move to have digital governments one way or another. We see this with Gulf War UK. Are we seeing all around the world? Is it a good thing or is it a bad thing? A while ago, people would have discussed in primarily the digital divide and the way in which poorer, unrepresented groups just don't have access to the internet and therefore not services.
And it's still a concern. But the introduction of machine learning, deep learning into the provision of government services could be a very serious problem. One area where it's already being discussed, it was mentioned in the last session is recruitment in government services. But there are probably many other areas where bias can create then and reinforce existing disadvantage. This, of course, takes third. The private sector as well.
Notion of automating inequality again machine learning picks up vulnerabilities groups that already disadvantaged, perhaps regarded as high risk. Well. Become even higher risk through the way they're labelled by machines and something I've only just begun to understand. Well, I don't understand. I've been introduced to is the way in which.
Companies are typically not building their own machine learning systems from scratch because the buying in modules or using modules that they don't understand. Now if you have different companies buying in the same modules and learning the same way. Is it not likely the same biases will appear? Time and time again? So if we have if you simply have a bias St. John manager, then OK, that's going to be unfair there.
But somewhere else that's going to be a different bias. And so to some degree, the biases will cancel out. But if we got the same systems being used over and over again, we see it with credit check agencies or only a couple of agencies. Now, if you're forced to be blacklisted on one agency, you're going to be screwed everywhere. The same thing is quite likely to happen in recruitment and in provision of other services.
So it's not only the bias, but it's the same bias everywhere. It's going to be a big problem. So this is something obviously we want to look into as well. Finally, something I don't think has come up in this seminar before we got a couple of colleagues working on autonomous weapons and machine learning and the army. And one of my colleagues, Tom Simpson, was formerly a soldier. Dapo Akanbi works on the ethics of war.
We already know about drones changing war. What about drones and instruct themselves? What about some weapons that would pick their own targets? Are we ready for that? We regulate it. Should we make sure it doesn't happen? So these are some of the issues of going around in the school at the moment. And I, my own work does relate to the first three, although in terms of outputs, I have very little to show, which is why I say nothing to show for that.
So the work I've been doing, some of my interest in this area comes from work I've been doing for maybe two decades on the ethics of risk,
and I'm particularly interested in the question of risk and regulation. Now, if you think about why do we regulate anything, I positive regulation is typically about harmony of standards or making information available to industry, we have first responders European standards so that people know what they're getting and you can have a type of interaction between items of different manufacturers, for example, if they're all using the same standard.
So now you can replace your shower unit quite easily because they're all the same size or the same size because they're regulated. So positive regulation is about harmony and ease. But negative regulation or negative regulation I regard as a reaction to risk. So we regulate because there are risks out there. I hear I'm very influenced by something I read by the. What was once upon a time called the Better Regulation Task Force.
And it was just one line from the better regulation task force about universities. And we were complaining the universities about how heavily we were being regulated and the Better Regulation Task Force said it's a surprise that the universities be being regulated to such a degree because there has never been a university failure. And there has never been a VC sacked for fraud or embezzlement. So the risk in this one of the risks in the sector.
Well, the main risks in the sector, it will be someone who deserves a two one gets a two two, roughly speaking. That's our main risk other than financial collapse. And so the level of regulation around the universities, given those minor risks, appears to be completely disproportionate. So I found this fascinating because it opens up a world of thinking about risk and regulation because what it suggests is that negative regulation before you regulate,
first of all, I can find your risk. So what is your risk? What is it you're regulating again? So we tend to have the opposite view. We need some regulation. Let's get some regulation the other way around. What's the risk and what can we do? Well, first of all, what is the risk? That's the first question. If you haven't identify the risk you're regulating in the dark. Second thing is that you should make sure your regulation addresses the risk.
Sounds obvious, but it rarely happens, actually, that quite often we regulate because something needs to be done. We have to be seen to be doing something. Someone has identified this problem. We've got to run around and we come up with the six principles of regulation for this area. And now we've got our principles regulation. What do we do? Nothing. We've spent a lot of time and money coming up with our rules for regulation.
Then next thing is, if you've got regulation, you've got to make sure that your regulation actually reduces the risk because there are times when regulation increases the risk. That's also very common. How does that happen? Well, sometimes it's just really bad regulation. More often it's ineffective regulation and roughly speaking, a belief that you're safe is very dangerous. That is, if you believe the area is well regulated, you'll drop your guard.
And that's what we've seen in the aviation industry lately. That's what we've seen in the building around building regulations. Who would have thought we could have had unregulated cladding pretty much on buildings, but that's what we had. We all assumed this was properly regulated. We looked at it and it wasn't. So if you think an area is regulated, then you relax and you don't take your own personal god.
So it's actually better to have no regulation than bad regulation, Mary, because then we can all take care for ourselves. So these are the background, some of the background assumptions I'm making. We have to think about second round effects. We also have to think about the fact that generally speaking, the agencies we're regulating have a financial interest in avoiding that regulation if we can.
Regulation is typically ineffective. If you're sceptical about this, consider most of us work in universities. We're getting regulation all the time coming from a level above. What do you do when you get some regulation that stops you that would stop you doing something you want to do? One of your first instincts and mine is to work out how I can carry on doing what I want to do. Compliant with the regulation.
OK, so that is our instinct when we are regulated and we don't identify with the goals of the regulation, we all do it all the time. So most regulation is useless and is counterproductive. Doesn't mean we shouldn't have it means we should be very, very careful about how we regulate. But a second point. Regulatory drift, negligence, gifting countries and markets, what I mean here.
Well, so I'm also very interested in the regulator once we've got a regulation and how the regulator behaves, particularly over time. Because if we have a new area and this is not just any area of new technology, quite often when it comes in as a lot of fuss, a lot of action, a lot of money goes, a lot of people declare themselves the experts and we and some of them are. And we can get some OK regulation to begin with. What happens over time?
Well, if nothing bad happens, we have a regulation, nothing bad happens, we begin to lose interest, something else comes along. We take our eye off the ball. So there's been a negligence that will come in that we will think this area is OK and we will begin to neglect its. Later on, we'll have a government that says there's too much red tape, we've got to cut red tape, two regulations for every everyone that comes in.
Doesn't matter what they are. OK, this is an area that hasn't been to be affected, so maybe we will too. Well, my friend Judy Brown in Cambridge calls regulatory gifting where the regulations giving given back to the industry as a form of self-regulation. Normally, the industry promises something in return, but it never gets it right. So regulation is reduced and the benefit is entirely for the industry itself rather than others.
Third, regulatory capture. Well, so this is a way in which corporations cosy up to the regulator will entertain the regulators. They will put pressure on the government and regulation becomes more and more favourable to the industry. Finally, one thin market so this is from our colleague Karthick Ramana, who has written I'm not sure if he said this or I said it, but I think the sense in which what he works on is the most boring thing you could possibly work on, which is the.
Regulatory standards in the accountants accountancy industry and what he says is this this is incredibly boring, he wants to talk about regulating accountants. But in fact, his business school, no, this is one of the most important thing that goes on at the moment because so many auditors have failed in the work. So what is happening? Why is auditing accountancy found so difficult to get right? Well, it's because hardly anyone can do it, and they're all employed by accountancy firms.
And so you have thin markets in these very specialised areas. So areas of new high technology, that's almost no one in the world with the knowledge to regulate the area. So if you think about financial markets and derivatives or some of these markets where almost no one understands that if you want to regulate, you've got to bring the. Same thing is happening in solar radiation management, I'm sure the same thing is happening.
I, as a small number of people, the usual suspects turn up every conference they give a keynotes set on the panels they probably once worked for organisations or they're going to work for those organisations in the future, even if they believe they're completely unbiased. They will see the world the particular way they will likely to. Be much more sympathetic to the corporate interests and less sympathetic to consumer interest, for example, what do we do about that?
Do we have massive regulation and even high regulation of the regulation? And what about corporate capture for that? I think we have to go in the other direction and in some areas around about environmental policy. For example, civil society groups have been assigned to check on the regulator. I saw this years ago with gene watch when genetic engineering came in and they always made what you might think the most extreme annoying arguments.
But we need people in the world to irritates the regulator, to get them to take things, to look for the middle ground. So we need civil society groups to stir things up and never give anyone a moment's rest. Nothing. It the remains best. And that's probably what we need in this area, too. I'm not sure it's happening yet. We've got people writing very good books, but I'm not sure I'm seeing a lot of civil society follow up from that, so I don't know where we can.
All the money's on the other side, of course, in this case. The question of values of what values should or are embedded in technology is, of course, a perennial question. Technologists and society at large, however, have often taken this relationship for granted and sometimes falsely assume that technology is value neutral. A normative conception of value, of course, has deep philosophical roots and outside philosophy.
There is a considerable body of work on the relationship between technology and values. For instance, the interdisciplinary field of science and technology studies, or stress, builds on the insight that values are embedded in technological choices and specific technologies. Far from being independent of human desire and intention, Sheila just enough explains technologies are, quote, subservient to social forces all the way through. In his seminal 1980 article, Do Artefacts Have Politics?
Langdon winner argued that technologies can have political properties, either by influencing power relations or corresponding to certain structural arrangements. Technologies can reinforce unjust and unequal relations, and technical choices can lock in particular values. He cited the example of Robert Moses. His 20th century designs of New York City reportedly contributed to deepening racial hierarchy through infrastructure that restricted public transport flows.
There's also been constrict, constructive, considerable constructive responses to these kinds of worries since the early 1990s, for example, an approach known as value sensitive design or vesti drawing on fields such as anthropology, philosophy and engineering, has sought to bring values into technological processes via a range of theoretical, methodological and practical proposals.
The study attempts to find intersections between technical innovation and human flourishing through methods such as stakeholder analysis. I'd like to structure the rest of my remarks in three parts, so that's two to four. First, I'll introduce the problem of value alignment in AI ethics and given time restraint constraints, I'll just stick to some high level normative points rather than try to cover a lot of the innovative technical research that's going on in that area.
I then want to highlight what I will call a political turn in value alignment research. And finally, I will situate the question of value alignment in what I call a social ecology of AI, an ethical AI development. I should acknowledge that my remarks draw heavily from work I've done with my collaborator, Yasmin Gabriel, a philosopher and a senior research scientist at DeepMind. But the views expressed should not be attributed to DeepMind.
I will return to your son's own work in a moment. OK, so the second point on value alignment and AI. There has long been a view expressed in science fiction since the field emerged in the 1950s that AI poses a distinctive moral challenge. The value alignment challenge refers to the one of aligning powerful technologies with human values has risen to the fore in light of the potential for something that has been called Artificial General Intelligence,
or AGI. This is A.I. which would match or exceed human level intelligence across different domains. A prominent A.I. researcher referred to earlier, Stuart Russell, describes a failure of value alignment as arising when quote We perhaps inadvertently imbue machines with objective objectives that are imperfectly aligned with our own. Since the 1990s, A.I. researchers have recognised this as a specific area in need of attention.
More recently, it has become a theme in specialised discourses such as philosophy and computer science and increasingly discussed in the public sphere. A major concern is that an AGI incentivised around a particular reward might pursue that reward without respecting the wider values of its designers users or affect other affected persons, generating systematic pressure towards value misalignment.
Relatedly, Nick Bostrom has pointed to the existential risks of a superintelligence that is one that greatly exceeds the cognitive performance of humans in virtually all domains of interest. But existing A.I. systems already exhibit a degree of value misalignment by amplifying social disadvantage in ways that diverge from the purported values of equality or justice, for instance, held by their designers or wider stakeholders.
For example, algorithmic bias and unfairness have been documented widely now in criminal justice systems, social welfare and insurance markets. Social media algorithms have come under intense scrutiny for their counterproductive consequences, and these misalignment risks are all exasperated, exacerbated by a lack of accountability in what Frank Pasquale has called the Black Box Society.
Such challenges have led to the emergence of a vibrant fairness, accountability and transparency or fat star research community. So value alignment, we can say, has two dimensions. It has has technical dimensions, and that is how to align AI with human values, as well as normative dimensions, deciding what values to align artificial agents with. And I'll focus here just on the latter.
There are features of AI's their potential speed, scale and complexity, which makes value alignment in this area distinctive. I think from other technologies, as artificial intelligence become more advanced, they have a wider range of decisions open to them and as they make such decisions in areas formerly reserved for human judgement and control. This increased. You might call it autonomy becomes morally significant.
And but normatively, there has been a range of responses to this to to this in the literature on safety and control of advanced A.I. The standard approach uses instructions to ensure value aligned outcomes. Here, the idea is to include as much safety or other value. Preserving criteria as possible in the design is instructions. But unsatisfied with the risks of the standard model, some are focussed on creating agents that behave in accordance with the user's intentions.
A third approach advanced by Stuart Russell in his book, a recent book called Human Compatible Advocates, shifting the onus to artificial agents to infer human preferences. In this approach, a utility function would capture preferences in an algorithmic form that a machine can process. Importantly, machines would be designed so that they defer to humans.
He calls it a kind of humility and ask them they would ask permissions and act cautiously when guidance is unclear and allow themselves to be switched off the switched off being important for that AGI or superintelligence worry. But each of these approaches has limitations. For one thing, they struggle to generate a legitimate basis to discuss or decide what value should be embedded in our systems. So now I'd like to turn to my third point on the political turn.
Johannes Zimmer has recently made the case for political philosophy as distinct from moral philosophy or social science to be given greater consideration in technology ethics. And he uses the specific example of self-driving cars to illustrate his point. In his view, relying on moral theories is problematic, since it does not always. It is not always clear which theory is correct and society's, he thinks should generally aim to preserve human agency and autonomy.
Tabulating preferences, on the other hand, which would be a more social scientific approach, perhaps can include discriminatory and unfair preference preferences. A political philosophy approach has three advantages, according to him the British respecting value pluralism. The fact that people have different values. Respecting human agency in a time of me and being sensitive to issues of legitimate authority or why we should abide by certain decisions.
And this thesis fits with what I would like to characterise as an early political turn in value alignment research. And the pathbreaking contribution along these lines was recently made by Yasmin Gabriel in an article published last month. Gabriel disentangled A.I. that aligns with instructions, intentions revealed preferences, ideal preferences, interests and values.
And drawing on with this analytical clarity enhancer, he advocates for a principle based approach to alignment, drawing on the great liberal philosopher John rules this insight on the fact of reasonable pluralism. He suggests that the key challenge is not to identify true moral principles, but rather to identify fair principles for alignment that receive reflective endorsement despite widespread variation in people's moral moral beliefs.
Gabriel suggests three possibilities for deriving such fair principles based on what Rawls called an overlapping consensus. These are global human rights law and discourse, hypothetical agreement from behind the veil of ignorance and social choice theory, given time constraints, I can't elaborate on the specifics. Instead, I just want to point to five reasons why I think Gabriel's approach is immensely generative for future thinking about A.I. ethics.
First, as calls for democratic oversight of tech A.I. technology growth, this approach offers the basis to engage constructively with the diverse values people hold and to channel these into the design and development of AI technologies. Second, the proposals offer technologists a new conceptual mapping for their work, grounded in the perspective of ordinary citizens with proper regard for vulnerable people and groups in particular.
Third, as numerous public and private bodies begin to specify principles for the governance and use of A.I., his approach offers new ways to evaluate these initiatives. Fourth, crucially, Gabriel's approach recognises the transnational nature of the value alignment challenge, given that the potential global reach of AI technology, and he builds this into his proposals from the outset.
Finally, it demonstrates how the tools in a methodological sense of political philosophy and political theory, can inform public policy and deliberation on transformative technologies like A.I., as well as technical choices involved. So finally, I want to turn to this point. My last point, which is inspired by the political turn.
I want to say that I want to suggest that future A.I. ethics research should pay more attention to the evolution of what I call the social ecology that's geared towards solving problems, addressing public problems and learning from the failures of existing technologies. And this reflects some of my own research interests. Prominent AI researchers have suggested innovating in how we conceive and analyse the ethical impact of A.I.
For instance, Kate Crawford and Ryan Keller have promoted the use of what they call social systems analysis to account for all the possible effects of AI systems on all parties in design, deployment and regulation to meet, Gebru says. Rightly, I think that AI researchers should learn about the ways in which their technology is being used, question the direction institutions are moving in and engage with other disciplines to learn from their approaches.
She says those studying fairness, accountability, transparency and ethics in AI should forge collaborations across disciplinary, geographic, demographic, institutional and social and socioeconomic boundaries, and help lift the voices of those who are marginalised. So apart from deriving a ranking of values, then we must additionally ask how to engender what a philosopher Alex Springer calls critical responsiveness amongst agents, given their situated ness within a social ecology.
In her book Communicating Moral Concern, Springer argues critical engagement is a kind of moral work, and we bring different perceptions, talents and social relations to it, rather than trying to build an account of generic agents through philosophical moral theory to promote this. She suggests at best, we can illuminate the process by which we each give varying weights to the concerns that emerge around and within us.
The social ecology. So, so is this terrain that, she says, quote, We struggle to make a distinctive and viable space place. Sorry for our evolving projects. Typically, we use normative theory to narrow the relevant agents who have specific duties or responsibilities. I'm interested in how it can also assist us to develop complementary ways of thinking about pluralism.
The scope of Mali engaged agents not only as victims of injustice or inequality, but as constructive protagonists in collective projects of human flourishing. The fact that A.I. research is largely concentrated in particular parts of rich Western countries means certain background assumptions are likely to define the parameters of value,
line research and practical mechanisms. There is a worry that high profile and privileged voices may stifle the possibility of alternative viewpoints that mask underlying contentiousness or conflicting interests. Indeed, there is also growing recognition. The value alignment efforts must guard against distorting effects of Western biased ideological blinkers and neglect of feminist perspectives.
As one researcher, Shakir Muhammad notes, A.I. is currently localised and quote within restricted geographies and people. He continues, We are talking about A.I. Researchers like himself rely on inherited thinking and sets of unquestioned values. We reinforce selective histories. We fail to consider our technology's impacts and the possibilities of alternative paths. We consider our work to be universally beneficial, needed and welcome.
So an account of social ecology can help evaluate the interactions the social and moral learning processes that grapple with these epistemic and legitimacy challenges. Through this lens, many agents have standing to contribute to the generation of relevant practical knowledge. I can think of at least seven sets of agents. There are more.
These are technologists, technology companies, the media, universities, policymakers and regulators, social movements and community organisations and all individuals, I think conceived as citizens in the Republican sense. So Social Ecology Lens considers how to motivate these agents participation in creating and sustaining apt institutional structures and social processes.
After all, any mechanism for determining legitimate values, I think, depends to some extent, at least on the emergence of ethically robust and trustworthy institutions. The development of social practises that facilitate the effective communication and moral concern, and individuals with sufficient allegiance to realising ideals like justice in the face of competing pressures, such as self seeking economic gain.
And I'll just give a couple of very brief comments and to sort of flesh it out a little bit before I finish. So consider how technology firms can become subject to greater democratic oversight. They must find openings for genuine, creative and constructive engagement with all communities and people affected by their work. They can also innovate with corporate governance arrangements, recognising that such regimes are more malleable than commonly assumed in Orthodox models of capitalism.
A social ecology lens can encompass such questions of political normative political economy on things like the Constitution, incentives of firms, the social function of finance and the dynamics of market competition and cooperation and through an ecological lens. Universities, for example, can have a crucial role in terms of educating future technologists and policymakers, as well as promoting interdisciplinary approaches to value alignment.
As a relatively new area of enquiry, there is an opportunity to harness different branches of knowledge as part of a unified research agenda. But interdisciplinary alignment research requires enhanced trust and collaboration across intellectual bound.
Including the development of shared vocabularies. So finally, citizens social movements and ultimately the state should ensure that both the public sphere and in policymaking that value alignment efforts are subject to continued scrutiny from the perspective of the common good.
So when we begin to desegregate agency in these kinds of ways, I think it's possible to envision not only how to improve value alignment in the design of specific technologies, but also how society at large can support that goal.