Ethics in AI Seminar: Responsible Research and Publication in AI

Welcome to a discussion on responsible research and publication in artificial intelligence. This is part of an ongoing series of seminars associated with the new Institute for Ethics in EHI at the University of Oxford. A series which began over a year ago in that wonderful world before Kobe.

If you're interested in finding links to other related events both past and forthcoming, together with recordings and podcasts categorised by topic, then go to the Oxford Philosophy Faculty homepage and click on the ethics in I Link. I'm Peter Milliken, Gilbert Ryle fellow and professor of philosophy at Hartford College. And head of education and outreach of the new institute. I and related technologies are having an increasing impact on the lives of individuals as well as society as a whole.

Alongside many current and potential future benefits, there's been an expanding catalogue of actual and potential harms arising from deployed systems and raising questions about fairness and equality, privacy, exploitation of workers, environmental impact and much more. In addition, there have been increasing numbers of research publications that have caused an outcry over ethical concerns and possible negative impacts on society.

In response, many are now asking whether the technical A.I. research community itself needs to do more to ensure ethical research conduct and to ensure beneficial outcomes from deployed systems. But how should individual researchers and the research community more broadly respond to the existing and potential impacts from research and technology? How should we balance academic freedom against the impact of research on society?

And where should we draw the line between openness and caution in publication? One key question here is whether technical researchers are themselves well placed to grapple with such issues. Well, who else needs to be involved? What can we learn from other fields to help us navigate forward in this new area that promises to have such high stakes for our collective future?

I'm delighted to be joined by three researchers who will be discussing these issues, drawing on examples such as conference impact statements, released strategies for large language models and responsible research innovation in practise. In order of speaking, they are Rosie Campbell, who leads the safety critical aid programme at the Partnership on.

Carolyn Ashurst, who is a senior research scholar at the Future of Humanity Institute in Oxford and research affiliate with the Centre for the Governance of EHI. And Helena Webb, who is a senior researcher in the Department of Computer Science at. Welcome to all of you and thank you very much for joining me. Each of our speakers will give a short talk and the events, the whole will last for around an hour and a quarter.

So we'll have plenty of time for discussion, and you're very welcome to offer your own questions to the speakers. Feel free to do this at any time by typing them into the comments box on YouTube. I'll be noting these as we go and proposing them to the speakers at various points, so the sooner you get your questions in, the more opportunities there will be for having them address.

First up of our speakers is Rosie Campbell. As I mentioned, Rosie leads the safety critical aid programme at the Partnership On. She's currently focussed on responsible publication and deployment practises for increasingly advanced A.I. and recently CO organised a major international workshop on navigating the broader impact of A.I. research.

Previously, Rosie was assistant director of the Centre for Human Compatable A.I., a technical AI safety research group at Berkeley working towards provably beneficial A.I. Before that, Rosie worked as a research engineer at BBC R&D, a multidisciplinary research lab based in the U.K. There she worked on emerging technologies for media and broadcasting, including an award winning project exploring the use of A.I. in media production.

Rosie holds degrees in computer science and physics and also has academic experience in philosophy and machine learning. She co-founded a futurist community group in the UK to explore the social implications of emerging tech and was recently named one of 100 brilliant women to follow in A.I. ethics. Welcome, Rosie, and over to you. Thank you so much, Peter.

I hope you can all see my screen, OK? It's great to be here. I'm going to be talking today about considerations for responsible publication norms. As Peter mentioned, I lead the Partnership on Ice Safety Critical Eye Programme. And this work is one stream that we're doing under that. In case you're not aware of the partnership, it's a multistakeholder non-profit made up of a variety of institutions, including non-profit organisations, industry companies and academic institutions.

And so a lot of the work we do involves convening and talking with partners from all of these different areas and trying to understand what we need to do as a community to move towards more responsible I. The goal of this talk is to lay out some of the key questions and considerations we've been exploring as part of this work, and I'm also going to suggest some tentative lessons that we might be able to learn from other high stakes fields.

And I'd also like to try to set the scene for further discussion on this, including possible interventions, which I'm sure Carolyn and the Helinet will also be able to cover in their talks. So the initial premise is that our research can be both high stakes and dual use and buy high stakes. I mean that the outcomes can either be really, really great, but they could also be potentially very bad if we get things wrong or act in an irresponsible way.

And it's dual use in the sense that I can be applied to many socially beneficial purposes, but it also could be maliciously used or otherwise otherwise applied in ways that are more harmful to society. And unfortunately, often a default mode of tech companies, at least in the past, has been to move fast and break things, and that's not always the best way to deal with technology that is high stakes and also dual use.

So my focus has been thinking about the question of how can we conduct our research responsibly to maximise the benefits and minimise the risks, and in particular what research practises and publication norms do we need to do that effectively? And I want to start by talking about a particular example of this that happened in early twenty nineteen, so Open I, which is a company that is developing advanced AI models, created a language model called GP2.

And the way that this works is you give it a sentence and it will write an essay in the style of that sentence. If you give it something that sounds like the first line of the president's speech, it will write an essay that sounds like the president's speech.

If you give it something that sounds like a five year old's homework assignment and will write something that sounds like a five year homework assignment, and it's pretty convincing, at least the initial model, you could generally tell that it wasn't written by a human, but it was it was not bad. It was pretty convincing. And basically when they developed this, they were worried that this technology could be used for all sorts of problematic purposes.

So as some basic examples, potentially, it could be used by spammers, it could be used for very targeted fishing. It could be used to generate fake news. It might obviously help people cheat in their coursework or homework assignments. So even if I were a bit worried about releasing this technology out into the world so that it could be used in all these different ways.

So what they did was to experiment with a slightly unusual release model, which was to release a smaller version of the of the technology first. So rather than releasing the full, fully trained, very large language model, they released a smaller version of it that was less capable. And so what that meant is that you could you can feed it a sentence and it would give you something back. But it would be like not overwhelmingly convincing that it was written by a human.

And the plan was to release things in increasingly large steps so that once they could see how people were actually using the model and make any mitigations that needed to be put in place to combat malicious use, they could increasingly release the more capable versions. And this caused quite a few different reactions in the community, some people were very impressed with this.

So a tweet here says, Going so far as to think ahead to malicious uses and check in with stakeholders sets a new bar for ethics. And I will play it open. I saw some people thought this was a necessary step towards thinking about releasing research in a responsible way. Whereas others felt very differently. An example here says what you what you were doing is the opposite of open. It is unfortunate that you hype up and propagate fear and thwart reproducibility and scientific endeavour.

And so this represents what became a pretty intense conversation in the community, which has been going on for the last couple of years, and I think raises some really interesting questions and important tensions that we are facing as a community as we realise the impact our technology is having and could continue to have on the world. And I want to cover a few particular questions that have arisen from this conversation.

One is, how do we balance openness with caution? On one hand, it's very important for science to be as open as possible. A lot of the advances we all enjoy in our lives today are due to the scientific norms of open enquiry and all the benefits that that has brought. It's also very important for reproducibility to be able to release research as open as possible so that other scientists can reproduce the work and confirm the findings and build on them.

And then, of course, there's the opportunity cost of potentially not releasing research in the sense of missing out on beneficial applications of that technology, which is also a very real cost that we need to take into consideration. However, on the other side of things, we may want to take a more cautious approach for a few reasons.

One example I've heard for to illustrate this is if you knew how to make a nuclear bomb from kitchen supplies, most people would probably agree is not a good idea to post that information on the Internet where anyone can see it just because someone is going to do that and it's going to result in devastating consequences. So most people agree that there is a line to draw somewhere, but there's a lot of discussion about where we draw that line.

And an example that occurred in the life science community a few years ago was some researchers discovered how to synthesise the horsebox virus and they were going to publish the steps that they took to do that. But this resulted in a big backlash because people were worried that it could cause people to essentially bring back the smallpox disease, which we worked as a species somehow to eliminate.

So there was a big conversation in the life sciences community about whether that kind of information should be distributed or not. And I think we're now starting to see similar conversations take place in I. Another question is, whose responsibility is it to anticipate and mitigate these risks? On one hand, researchers are experts in their own work, but they may not be able to anticipate second and third order consequences that rely on knowledge and knowledge about economics,

politics, history, all sorts of things. Maybe this is the job of teams, research teams. Maybe it should be part of the peer review process. Is there a role for government to play or society at large? And there's this Trade-Off between individual versus collective responsibility. To what extent should these be things individuals are supposed to be worried about versus infrastructure that we put in place to support the community at large?

And I think Carolyn is going to talk a little bit more about this question and her talk. And then finally, another question I've been asking myself is how do we effectively equip ourselves to navigate responsible publication? What resources do we need as a community and what infrastructure needs to exist?

Potentially, we need to create frameworks for thinking about the risks of our of our work taxonomies, to categorise different types of research, to help us understand how risky it might be, different services, to help researchers anticipate the potential impacts of their work guidelines, institutions. All of these sorts of things are things that we're exploring with our partner community that we might need to develop.

OK, now I'm just going to touch on a few things I've learnt throughout this work, and the first is to try and disambiguate the different terms that often get used in this conversation. So. Firstly, people talk about research integrity, which really refers to the kinds of responsibility a scientist or researcher has in order to make sure that their findings are.

Legitimate and as close to truth as possible, so things like avoiding hacking, not falsifying data, those sorts of things, then we have research ethics, which is primarily, at least traditionally, it's been primarily focussed on the welfare of human participants and things like how you have gathered your data. Have you done that in a responsible way? And then we have the issue of downstream consequences.

So this has been more of what I've been focussed on, which is once research has been conducted and it's been released into the world, what are the downstream effects of that research, whether that's something like technological unemployment or fakes and misinformation?

What are the effects of that on society? And then there's another term that has been used more frequently, which is the idea of broad impact, so not just the potential negative consequences, but the impacts at large from from the work. And I just wanted to disambiguate these because I think we sometimes use the term broad impacts, for example, to cover both the downstream consequences and the research ethics aspects of it.

And I think it can be useful to try and untangle these a little bit to to know what we're focussing on. And as I said, my my work is primarily focussed on downstream consequences, but there are certainly ways that the different areas here overlap. So, for example, the the environmental effects of how much computation you need to train a model might be part of research ethics, but also has relevance to the downstream consequences if that model is then going to be used in a deployed at large.

And a lot of people are going to be that environmental effect is magnified. Another consideration is the fact that a lot of research happens beyond academia and happens within industry, and not only that, but a lot of my research I researched goes directly to places like archival blog posts and bypasses the peer review system completely. So we're dealing with a slightly unusual field here in that we can't just think only about the academic research process.

We have to think more broadly about the community at large. And in addition, the line between research and product is quite blurry. If you think of something like Jupiter, too, like I mentioned earlier in the talk, that was positioned as a research endeavour, but it is also now being turned into products. And so I think we're seeing in I a much sort of type of turnaround between the

research that's done and the products that are deployed based on that research. So it's quite difficult to untangle those two things. That's a wide spectrum of views in the community, so as I mentioned, there's this tension between openness and caution, and some people very much see themselves as on the open side of things, very pro openness, no matter what the costs.

And other people see themselves as very much on the coalition side of things are trying to really be sure that you've mitigated any risks and that you can be pretty confident that your research is safe before it's deployed. And you can imagine most people fall somewhere along that spectrum. It's currently not that uncommon for researchers to discuss the risks of their work, and there are a couple of reasons for this.

Some want to do that and they think it is within the remit, but they just don't feel equipped. They don't understand how to anticipate the second and third order effects. But some researchers see their role as pushing the boundaries of knowledge and science and feel like it's not within them to consider the impacts of that work and it's outside of their responsibility. So we have different views in the community on that. I've also noticed a few different clusters of values in the community.

We have people who are primarily focussed on the science of climate science. Those people in terms of pushing the boundaries of knowledge and not worrying too much about the impacts on society, we then have people who are primarily focussed on social justice issues and are very much interested in examining the impacts that we see today on our systems that are having effects on marginalised communities.

And then we have people who are more focussed on the long term future and the way that these technologies may continue to develop. So they might be worried more about things like automation or superintelligence. And so depending on which cluster you fall into, people tend to have very different views on this issue. And I have identified here a few ways that these groups tend to have things in common.

But I do think it's helpful to understand when you're talking to someone about this, where they're coming from. There are also there's also two major target audiences for this one are practitioners. So these are the people who are the researchers and engineers actually building the systems and doing the research. And they hold a lot of market power because researchers are in high demand. And so they can effect widespread culture changes.

They can put pressure on their employers, they can change the culture within their research labs from a grassroots level. And then you have what I'm calling gatekeepers, who are the grant makers and publishers, such as general editors and conference chairs who hold a lot more direct power so they can mandate a policy change, like requiring all authors to include a broader impact statement with their work, which has wide sweeping effects across the community in a very quick way.

The problem is that if you focus too much on the top down policy changes, you can end up with a backlash from the practitioner group who who may not be on board with those changes. But if you don't if you focus too much on the grassroots changes, then things can happen very slowly. So I see it as needing to to to to sort of work with both of these groups in tandem to try and make progress. And then finally, there are a few coordination challenges that we are seeing on this issue.

So, for example, if if a research lab wants to not fully release a certain piece of work because there are some ethical or broader impact considerations there, often that is within policy of conferences or conferences, often have like open data, open access policies. And both parties here are trying to act in a responsible way. But because there aren't standardised norms in the field, they end up kind of clashing.

And we also have a problem where authors who may want to delay their the publication of their work to try and do some risk mitigation before it's released, may miss out on getting the credit for that discovery. A lot of if you're not going to e-mail, you're probably aware of the publish or perish mindset. And I think a lot of people feel pressure to publish things as soon as possible so that they can get that credit and it's good for their career.

And so anything that talks about restricting publication or delaying publication can be a problem. So that's another coordination challenge we need to think about. And so then just to briefly touch on some things we can learn from other fields in the life sciences, it's very it's the norm to include in any research on, for example, new drugs, the side effects that those drugs might have.

It would be very strange to publish results of clinical trials without talking about the potential downsides of the drug. And so that's an analogy to what we're trying to advocate for here in the world of I. In the last census, there was also something called the National Science Advisory Board for Biosecurity, which was established in the wake of the anthrax attacks.

And the this board can advise on different publication decisions and provide some level of expertise and oversight to those sorts of decisions. And again, that's something that we could think about for. And then finally, in the life sciences, they have what's sometimes referred to as a culture of responsibility. And often this is mostly about how you handle yourself around dangerous and hazardous substances. But it also applies more broadly to thinking about the impacts of of bio research.

And so, again, that's something that we could think about adopting. And I put a link to a case study that we have published on the H5N1 virus and what we can learn from that incident. In cybersecurity, there is a norm of coordinated vulnerability, disclosure, and what that means is that if someone discovers a bug in a software system, rather than making that bug public immediately, they will often notify the vendor of that bugs.

If you find a bug in Google Chrome, you would notify Google before making that public so that then Google have a fixed amount of time to be able to fix that bug before before it's made known to the public. And so what that does is allow a coordinated effort to fix vulnerabilities before they can be exploited by malicious actors. And again, that's something we might want to think about, how that might work in the world of AI.

And they also cybersecurity also has some interesting approaches to accreditation. So trying to get around this issue of how do you delay publication while also giving people credit, there are some interesting things we can learn that. And again, I put a link to a case study that we've written on this topic. And then finally, nuclear research also has some interesting analogies.

Again, it is a dual use and a high stakes feel, just like I but it also gives us an example of how having too much secrecy can actually lead to problems. So there are some arguments that some of the the the result of the devastation in Chernobyl could have been avoided if there was much more of a culture of openness and learning from mistakes in the in the nuclear field. So that's something we also want to be very mindful of with an eye.

And we have a bunch of possible interventions we're exploring, which I'm not going to go into detail on now. I'm running out of time. And also I know Carolyn and potentially Helena are also going to cover some of these.

But just to run through them quickly, we're thinking about things like including broader impact statements in papers, trying to establish institutional review processes to think ahead about the potential impacts of research before it's conducted, looking at ways we could consider partial or restricted publication and advisory and expert advisory board along the lines of the NSA that I mentioned

and also developing informational resources that can assist researchers who are having trouble navigating the potential impacts of that work. And then just to finish off, I wanted to share some upcoming work we have in the pipeline, which is going to be a white paper for with recommendations and considerations for individual researchers, research institutions and teams, and also conferences and journals for how we can navigate this issue responsibly as a community.

And the sort of heuristic that we are using is that the more impressive the contribution of your paper, the greater responsibility, the greater the responsibility you have to consider the potential impacts of your work. And I'm happy to talk more about that in the discussion. Thank you very much. Thank you very much, Rosie. That was that was really interesting. I mean, a great overview of what's clearly quite a complicated landscape.

Yeah. Can I ask you just before we move on, I'd be interested to know your personal view of a kind of two different perspectives that someone might have on this. I mean, on the one hand, somebody might say, look, in general, sunlight is the best disinfectant. But the way to be safe is to have things out in public where everybody can see them.

On the other hand, somebody might say, I mean, you mentioned things like impact statements when researchers are consciously thinking about the risks and seeing what they are. Some might say that's just a way of getting bad guys ideas, but a bit of a conflict there. Do you feel both of those temptations or do you go one way or the other, or what do you think about it?

Yeah, I think these are both really good points and this illustrates why this is a complicated topic and we need to have these conversations now. So all the sunlight is this is the best disinfectant position. I think in general that's probably true. And for most research, the responsible thing to do is going to be to publish it as openly as possible, disseminate as widely as possible. However, I think there are going to be cases where that's not necessarily the case.

And we need to think about what's sometimes called the offence, defence, balance of scientific knowledge. And there's a paper on this and I would recommend people go and read where sometimes releasing the knowledge is going to be more helpful to people who want to cause harm so adversaries than it is going to be to the people who want to help mitigate that harm.

And that's going to be a judgement call. And we need to think about what he thinks we can use to try and anticipate in advance whether research is more is more likely to help sort of social researchers or those who are going to misuse it. So so, yeah, overall, I think sometimes the best disinfectant is generally true, but there are going to be exceptions to that that we want to be mindful of.

And then the other question you had around broader impact statements and will it just give bad guys ideas? Yeah. So I think this is a really interesting problem to have, because if we are trying to encourage researchers to write more about the potential ways that their research could be used to cause harm, you end up in a situation where someone who does want to misuse that research can just go and read all the latest papers and see a nice menu of ways that they can use that to cause trouble.

So I think this is, again, another coordination challenge. Maybe there's a way where we asked researchers to write a broad impact statement, but we don't necessarily publish it. Or maybe there are certain things that it makes much more sense to have out in the open. Again, going back to the sunlight is a disinfectant thing, but some some types of impacts could be better kept as like redacted or something like that.

So I don't really have a strong view on that other than to say that I think it is a valid concern and one that I want to hear from people in the community and think about potential creative solutions to solve it sure shows how complex these problems are. You've got better coordination. You've also got issues of whose responsibility is to make the various judgements and so on. Yeah, well, thank you very much, Rosie. And it's been really interesting. Um, and now we move to Dr. Carolyn Ashurst.

Carolyn is a senior research scholar at the Future of Humanity Institute. Hello, Carolyn, welcome. Thank you. And research affiliate with the Centre for the Government of EHI, both of those in Oxford. Her research focuses on improving the societal impacts of machine learning and related technologies, including topics in governance, responsible machine learning and algorithmic fairness.

The Technical Fairness Research focuses on using causal models to formalise incentives for fairness related behaviours. On the question of responsible and research and publication, Carolynn recently co-authored a major guide to writing impact statements and was Coorg organiser with Rosie of the recent workshop on navigating the broader impacts of our research.

Previously, she worked as a data and research scientist in various roles within government and finance, and you started off with both the master's and doctoral degree in mathematics. So welcome, Carolyn, over to you. Brilliant. Thank you so much. And hopefully you can see my screen do let me know if not, but brilliant. So thank you very much, Peter. So my talk is really a continuation of a lot of the themes that the Raisi already introduced in her excellent talk.

So I'll be talking about self governance in machine learning and using impact statements and ethical review as a as a case study for thinking about that. So here I'll be concentrating on the role of the technical machine learning research community, both individual researchers, research groups, research conferences.

But of course, I ethics and questions around societal impact has to be an interdisciplinary endeavour and has to involve many stakeholders, regulators, tech firms in impacted communities. So whilst I'm focussing on just one important actor, of course, this is just one piece of the puzzle. So there are lots of examples of the technical research community taking action in response to existing and potential harms around technology.

So, for example, technical researchers such as Joy and Wienie shown here have done extensive work to highlight the limitations and resulting harms from A.I. systems. For example, exposing algorithmic bias in commercial systems and work on these topics are spawned new research disciplines concerned with the societal impacts of AI, for example, fairness, accountability and transparency.

And this year's fact conference begins tomorrow, for example, and mainstream conferences such as Europe's now, except technical papers related to societal aspects such as AI, safety, fairness and privacy. Certain data scientists, some machine learning researchers have also been very active in raising awareness amongst the public through books and other activities.

And there are also many groups, workshops and teams who wish to promote the use of A.I. for beneficial purposes, often under the head of A.I. for social good or particular areas such as climate change. But whilst there is a lot of proactive initiatives in this space like these, many are still asking whether the research community needs to do more.

After all, the world is still full of problematic applications, business models and applications that bring benefits to some whilst bringing material harms to others.

And in addition, every year we see research papers that provoke an outcry over ethical concerns, for example, over problematic data sets or categorisations of people, applications that could be used against populations such as surveillance technologies, weapons research and applications that could be used for misinformation such as fake image generation. So in response to this, then, Europe's conference introduced new mechanisms this past year.

So now Europe's is the largest machine learning conference in twenty nineteen thirteen thousand people attended in 2020. The conference was held in line with around twenty two thousand attendees and having a paper published in Europe. It's very competitive. It's very much a top tier conference and has a lot of focus on on theory, on methods, more so than on developing and deploying specific systems in specific contexts.

So in 2020, Europe's introduced two new mechanisms, which I'll talk about in this in this talk. So as part of peer review, they introduce an ethical review process, the technical reviewers could flag papers for ethical concerns and papers that receives strong technical reviews were flagged for ethical reasons, which were assessed by a pool of ethics reviewers. So what happened? Around 10000 papers were submitted, generic IPS, about 2000 were accepted.

Of these, 13 were passed to the ethics reviewers, resulting in four rejection on ethical grounds, and seven authors were asked to make changes to their papers. So this affected just a very small number of papers in the end. On the other hand, the initiative that affected everybody was the broader impact statement.

So it's required that all authors include a broader impact section at the end of their papers, including its ethical aspects and future societal consequences, including both positive and negative outcomes. And this is actually announced before they explain the changes to the review process. And the immediate reaction was very mixed.

So how did people respond? Well, on social media, many came out in favour of this requirement, both from the ethics community and from the technical community as a step in the right direction that will encourage broader dialogue. But many also criticise this initiative with concerns about whether technical authors have the right expertise and whether this is even meaningful for a lot of the work and Europe's.

But of course, social media isn't always the best place to find out what the majority really think. So some research is an element. I survey 50 researchers to find out about their attitudes and how they gone, about the requirements. And again, they found a really mixed response. So the survey authors stated that there seems to be a general feeling that assessing broad impact is important, but some uncertainty regarding who should do it and how. And some liked that.

It forced researchers to reflect on the impact on their work, whereas others felt that it was too broad and that they didn't feel qualified. One respondent said, If I liked writing fiction, I'd be writing novels, and another described it as one more burden that falls on the shoulders of already overworked researchers.

So really a mixed response and lots of concerns. We also know that the vast majority of respondents spent less than two hours on their statement and that many felt there was a lack of clarity and examples and guidance. So because the official guidance was very brief, little more than what I showed on that first slide, a group of us, the FBI and elsewhere put together an unofficial guide for my research, as could years.

And as we learn more about what researchers find challenging, we hope others will build on this so that we can iterate towards a better state of affairs. So I've given a flavour of some of the divided attitudes towards this requirement, but what did researchers actually write as so researchers, including a group of us, are starting to analyse the statements. Back in the autumn, Marguerita Boyarsky and tell analyse the first preference to be put on archive.

And they did find some encouraging trends. They found that some considered a variety of stakeholders, that some were very clear about that uncertainty and some even deliberated about the limitations of mitigation strategies and some gave concrete examples of tasks, failure scenarios and situations of homes.

But they also highlighted some trends that they found concerning, for example, neglecting stakeholders, e.g., assuming the benefits that means benefits to companies and taking positive impacts to mean technical advances. They also felt there examples of people outsourcing ethical responsibility and letting the research topic around the scope of enquiry, for example, fairness papers failing to acknowledge unintended negative effects of fairness work.

And they also felt that some overemphasised the net impact and some were overconfident in their claims. So further analysis is still needed, but it's already clear that the quality of statements was highly variable. So given what we know so far from the immediate reaction from the surveys, from the statements themselves, what is the state of play regarding self governance in I, as I was Rasi pointed to in her talk and has been shown to be the case.

But for impact statements, there's certainly a lack of consensus on some of the underlying questions around whose responsibility there should be around the trade offs, for example, between openness and caution, between scientific freedom and responsibility to do work to benefit society, and on beliefs about scientific endeavour, for example, whether research should be considered beneficial until proven otherwise.

And even amongst those who agree on those questions, there is still a lack of consensus about which concrete mechanisms to adopt and best practise implement them, implementing them. So where do we go from here? So in our recent work in this paper led by Carina Frankel, we outlined some of the potential benefits of impact statements, increased anticipation of risks, reflection and awareness and assisting coordination.

But we also outlined some of their risks that impact statements risk being low quality, that such initiatives can trivialise the task of ethical reflection, that they can provoke negative attitudes or give a false sense of security if particular risks and harms are understated. They can also unintentionally signal that researchers alone should be the ones to judge the ethics of their work.

And we do risk a polarisation of the research community along political lines and along institutional lines. And writing impact statements like this is challenging the potential impacts of a piece of work are very complex, and in this case there was a lack of explanation and transparency.

Also, researchers are under time, pressure, and perhaps most importantly, there are institutional, social and cognitive biases and pressures that can incentivise researchers to focus on the positive impacts and not the negatives. So how can we address these challenges and risks? Well, where initiatives like impact statements are used, we recommend focussing on the following.

So firstly, some straightforward first steps, improving the transparency around the task and process and improving the guidance available, as well as providing links to ethical and societal expertise.

But on the more challenging side, we also need to think very carefully about how to improve the incentives to address some of the challenges, for example, using peer review and expert involvement to ensure that the standard is met, perhaps encouraging researchers to cite other impact statements to provide an incentive to write them well or prestigious prises for well written statements and finally,

deliberation. So in order to move towards a shared understanding and shared norms, we need to continue to create forums for deliberation, providing evidence where possible, and to continue to discuss how to address challenges that push researchers towards understating harms like reputational and legal costs. And wherever self-government mechanisms more broadly are used, we would urge people to consider these suggestions.

And while there are a lot of challenges and risks, I do think that if done well, things like impact statements could be really beneficial in raising awareness and encourage encouraging reflection about societal impacts. But we do need to think carefully about how to get there. So with that, I'll end there. And thank you very much for listening. Thank you very much, Caroline, can I ask you a couple of questions about impact statements?

I mean, it sounds like a quite a promising idea, but it must be quite difficult. I mean, with a lot of eye research, you're dealing with stuff that's actually very theoretical, very, if you like, high level. I mean, developing fundamental algorithmic techniques, for example. That could have zillions of possible applications all over the place. So is it realistic to expect impact statements there?

Yeah, thank you. This is a really good question. Yeah, this is certainly my reaction when the announcement came out, particularly because NeuroPace is such a kind of a theoretical and methods based conference. I kind of thought, what are these researchers going to going to write and say?

The first thing to note is that shortly after the announcement, NeuroPace did clarify that if your research is very theoretical, you do have to include a statement, but you are allowed to say something along the lines of my research is very theoretical and therefore a broader impact statement does not imply it does not apply.

I guess for me personally, I would still encourage theoretical researchers to use the opportunity just to think about their kind of their research field more broadly and the impacts that it does have, which which their theoretical research is is, after all, contributing towards. And in terms of your your other point, you mentioned about having a very large range of potential applications.

This is this is really true for a lot of work, even even work. That's not really theoretical because many of these outcomes are so general-purpose if you train them on a new dataset that can be used for completely different tasks. So I think this is really I think this is really challenging one. And so I encourage researchers to think very broadly across the spectrum because that is that is useful for the discussion.

It is useful for for policymakers. But ultimately, we do need experts from other fields to to help think about some of these kind of these how the impacts could pan out in very different situations, in very different applications. Yeah, I mean, I'm struck by the way the big developments in deep learning over the last decade or so have come in and suddenly impacted all over the place. You know, suddenly it becomes possible to fake pictures writing.

Even suddenly it becomes possible to play chess and go at a high level. And you wouldn't have thought of those as being similar. So the big theoretical innovations somewhere can have impacts all over the place. Yeah, definitely. I mean, we often refer to it as being a general purpose technology. And that's why that thinking about the ethics and the societal impact is is so difficult, but also ultimately why it's so important. Yeah. Yeah. Well, thank you very much.

You've you've emphasised some of the techniques that we're trying to use to keep this beast under control, but it's obviously controlling. It is a very, very complicated matter. Thank you very much indeed, Carolyn. OK, next, we're we're moving on to Helena, Helena Webb. Helena is a senior researcher in the Department of Computer Science at Oxford. She's an interdisciplinary researcher and specialises in projects that bridge social science and computational analysis.

She's interested in the ways that users interact with technologies in different kinds of settings and how social action both shapes and is shaped by innovation. She works on projects that seek to identify mechanisms for the improved design, responsible development and effective regulation of technology. While at Oxford, she's worked on projects relating to, amongst others, harmful content on social media, algorithmic bias, resources and STEM education and responsible robotics.

Helen is the research lead at the newly formed Responsible Technology Institute within the Department of Computer Science here at Oxford. She also convened student multiples in the department on computers, in society and ethics and responsible innovation. Oh, welcome. Thank you. Thank you very much. I pressed a button that I shouldn't have earlier, so I just need to check that my slides are still going to show. So can you see my slides? Okay, they're excellent.

Thank you. So thank you very much. I'm really pleased to have the opportunity to join this discussion today. I'm going to talk about the notion of responsibility and open up what it means in the context of A.I. research, specifically, really about our research in academia. And I think what I'm going to say is really going to touch on a lot of the things that Rosie and Carolyn have already mentioned.

In particular, what I want to do is draw on a well known responsibility initiative that's gained quite a lot of traction in research, industry and policy. And I want to argue that we can use this notion of responsibility to foster ethical practise all the way through the processes of our research and innovation. Before I go on to that, I will just introduce myself a little bit further. So as Peter says, I'm an interdisciplinary researcher.

I trained initially in the social sciences and I've been working now in the Department of Computer Science at Oxford for six years. I'm part of a team called Human Centred Computing, and we're an interdisciplinary group. And as the name suggests, we put humans at the heart of computing. So the kind of projects that we do examine the impacts that computer based innovations have on individuals, communities and societies.

And we often seek to identify ways in which these kind of innovations can be ethical and better support human flourishing so that they can aid empowerment of users, that they can be trustworthy, that they can be safe, that they can address societal inequalities and so on. A number of the projects that I am involved in follow this initiative called Responsible Research and Innovation also sometimes just referred to as responsible innovation.

And this initiative spans across policy, academia and industry and emerged about 20 years ago and has gained a great deal of influence over that period, in particular in the U.K. and in the EU as well as elsewhere. And this initiative, ORER, I began with an aim to identify and address uncertainties and risks associated with novel areas of research such as nanotechnology and geo engineering. And now it has moved into ICT as well.

And really the core idea behind responsible research innovation is to bring together actors across society so researchers, citizens, policymakers, businesses, third sector organisations and so on to work together across the whole research and innovation process. And the idea is that by doing this, we can have better outcomes of research and innovation because we are aligning these process with societal values, needs and interests.

So we will produce better results. For that reason, responsible research and innovation is often characterised as doing science with and for society because it brings society and together with processes of research and innovation.

A longer definition on the slide here comes from Renee von Eschenbach, who talks about R.I as a transparent, interactive process by which societal actors and innovators become mutually responsive to each other with a view on the ethical, acceptability, sustainability and societal desirability of the innovation process and its marketable products in order to allow a proper embedding of scientific and technological advances in our society.

So what we're really talking about here is, is a broadening out of traditional research and innovation processes to include the involvement of society very upstream. And I'll talk a little bit in a second about some frameworks for achieving this kind of work. But first, I'll just mention the ways in which this idea of responsible evasion or responsibility more generally is becoming quite embedded in the ways that people are talking about A.I.

So just a couple of examples on the slide here. So responsibility is often talked about in industry. So one example is Google, which has its own responsible innovation team. Of course, not without any problems or without controversy. And also the other examples from the slide here relates to UK policy. So there is a great deal of drive to see responsible innovation as a factor in the pipeline of A.I. from research to development right the way through into industry.

So so the UK and the UK government wants to make the UK a leader of AI and sees this being achieved by really creating a lot more opportunities for A.I. in research and then driving that through into the development process and into industry.

And the way this is often talked about is combining this with approaches that apply responsible innovation as well to in order to make sure that whilst being a leader in AI, that UK is also creating a situation in which A.I. achieves social good in various different ways. And this kind of interest in responsible innovation also carries over into academia and the kind of places where we might often see.

Funding for the work that we do, so two examples on the slide here, the first one comes from the EPSA Sea, which provides funding work for research in engineering and physical sciences and funds a lot of work that's going on in. The says recently launched a very large programme on trusted autonomous systems.

For instance, as it states on their website, the IPCC is committed to promoting responsible innovation, so it expects those who apply for funding to carry out research using using its money to engage with responsible innovation in some way and to embed these approaches into their projects.

Also, Horizon 2020, which is the other part on the slide here, is the biggest EU research and innovation programme ever, with nearly 80 billion euros of funding available over a period of seven years and responsible innovation, response, research and innovation is very heavily embedded into the Horizon 2020 programme as well, in particular with the science with and for society objective.

So again, we have this expectation that researchers who are getting money from these sources will be engaging with the ideas of responsible innovation as well. So I think it's worth spending a bit of time to look at the kind of frameworks that are available for knowing how you might carry out these kind of responsible research and innovation processes. So we'll look again at these two funding bodies. So first of all, with the EPSA, see, they have what's called an area framework.

And so this area framework sets out the kinds of actions that researchers can do to embed responsible research and innovation into the work that they're doing. And this framework is composed of four parts anticipate, reflect, engage and act.

So the first thing that you do within the area framework is to anticipate and this is something that's already mentioned a couple of times in the seminar, anticipation involves thinking about the possible outcomes and impacts of the research that's being done. So it's identifying the positive and negative impacts and also the intended and unintended ones that might arise from the work that's going on.

And importantly, this isn't the same as predicting there's no need to know for sure what the outcomes might be because they can be very difficult to pin down. But it's an awareness of all the different potential outcomes so that you can then act on them later. The next phase is one of reflection. So it's reflecting about the process of research itself. What kind of assumptions are being carried into it? What kinds of uncertainties need to be addressed?

And then you move on to engage. And this is a highly important stage and it involves opening up the research process to a very inclusive dialogue. So typically, engage involves bringing in the perspectives of different kinds of stakeholders in the research process.

So this might be users, policy makers, people from different professional environments and so on, and genuinely listening, inviting their perspectives and listening to the kind of concerns they might have about the research process and its outcomes, what kind of interests they have, and the different kind of values that they have associated with it. And then the final stage, once you've done these three, is to act.

So you take what you've learnt from the first three stages and then you act on them in certain ways. And what you're trying to do here is to positively influence the trajectory of this research process. So it might be making changes to the research planning, it might be making changes to the research team, making changes to suggested outputs and so on.

Based on what you've learnt across the three earlier phases so that you are moving towards making something that is more likely to be acceptable in society, more likely to provide value, more likely to align with societal needs. So that's the IPCC area framework. And then the EU has the 2014 Rome Declaration on Responsible Research and Innovation, which is rather broader and is based on six pillars.

And this relates to open access, government, governance, ethics, science, communication, public engagement and gender equality. So once again, we can see here that engagement, public engagement is very important and it's certainly something that we found very useful in the projects that we do here in Oxford.

So, as I mentioned, we often use this responsible research innovation approach in the work that we do in our projects, and we find it extremely helpful to get us to think about all the different kinds of societal values related to different technologies and how we can understand those different kinds of values and genuinely listen to the concerns of our stakeholders and then use that in a very constructive way to influence our own research processes.

So just to give a very brief example here, I was involved in a project called Unbias, which looked at the user experience of algorithm driven online platforms and also issues of algorithmic bias.

And when we. Carried out this project, we are very committed to various forms of stakeholder engagement, so we ran professional stakeholder workshops in which we brought together professionals with an interest in the kind of issues of project was exploring and encourage them to have dialogue with us and with each other. So we brought together people from industry, from policy, special interest groups, education, the media and so on.

And we talk through issues of Alcalay bias filter bubbles, the ways in which fake news processes can be driven by the mechanisms of online platforms and so on, and discuss together possible ways to tackle these kind of issues and to really understand the different kind of issues that were in play in these kind of phenomena.

We also spent a lot of time on public engagement as well. So going to different kind of events and talking to members of the public about the research that we were doing and engaging with them to get their responses to it, to see how how they understood the work that we were doing and and what kind of values they associated with it. And then finally, we engaged in creation processes as well for our material outputs.

So we worked with different kinds of stakeholder to produce outputs that could be used beyond the lifespan of the study. So in the slide here, I've shown a picture of our unbias awareness cards, and these are sets of cards that you can use to play kind of games to help understand issues of algorithmic bias and their impacts and to think through what fairness in algorithms might mean. And these concepts were created with groups of school children.

So we worked with them to design activities that would be engaging and also carry over the kinds of ideas that we wanted to communicate. So we find responsible research and innovation highly useful as an approach to embed into our work. And I can understand that this might seem less immediately useful to people who engage in far more technical work. Is the work that we do is already very sort of societal facing and so on.

But I do think there are lots of ways in which we can take this idea of responsibility and the overall aims of the response. Will research and innovation approach and apply it to any form of research. So to include this more technically focussed, I work and I'll just mention a few kind of important characteristics that we might think relating to responsibility to see how it can apply to this form of more technical research.

So one of them is to understand that responsibility isn't just about thinking of the impacts of research. It's also thinking of the research process as well. So it's understanding that when you're doing research, that's part of society, too. So you need to think about responsibilities within it. And this goes beyond the traditional kind of, you know, following the guidance set out by your ethics committee to thinking about different issues as well.

So thinking back to the Rome Declaration on Responsible Research and Innovation, they talk about open access to science, so so reaching different kinds of audiences in making publications available to different audiences, making them understandable to different audiences. Rosie mentioned the carbon footprints of projects as well. And I think this is hugely important when we think about the research process as we know that the A.I. requires a lot of computational power.

So it actually contributes to climate change in various ways. So as a research process, you can think about how you might manage that, how you'd manage the carbon footprint of the project that you're doing more generally with all of us learnt over the last year that we don't necessarily need to travel by plane thousands of miles to go to conferences so we can all start to think about how we might reduce the carbon footprint of our projects in that sense as well.

And then also the Rome declaration mentions gender and exclusivity. And I think this is a highly important point when we think about research processes. And I know that in industry has often come under attack for industries not necessarily being particularly inclusive or balanced in terms of gender and the kind of impacts it has in perhaps making some of the datasets that are used within industry practise less than diverse as well.

And that kind of attitude, this neglect of diversity carrying through. And I think the same can apply in some senses to to university research as well. Computer science departments are not bastions of gender diversity or exclusivity. More generally, there's lots of work that can be done in this side of things. And it's something that we can think about as a responsibility of researchers and research institutions in carrying out this kind of work.

The next two factors, I think, relate to each other very closely, so I put them together. So one is understanding that responsibility is not just about liability. So it's not just about finding who is to blame when something goes wrong. It's something quite different from that because it also involves being forward looking. It's not just looking back after something has gone wrong and saying, oh, well, you know, let's work out who to blame.

It's more about a kind of proactive caretaking, and this is where this idea of anticipation comes in, that you look forward and think about, you know, what are the potential implications of this process, the positive ones, the negative ones, intended and unintended. And rather than just you know, you don't just sort of raise awareness of them, you know, in the sense of kind of creating fear and so on.

You anticipate so that you can then take action, you can take steps to try to mitigate those potentially negative consequences. So by being responsible, you're looking forward and you're taking care of the future in that sense and thinking far more broadly than just simple liability. And then the final point to make is that responsibility is also shared. It's shared as a process is distributed across different groups of people across society, really.

So, you know, we've spoken about the concerns that researchers have about taking all of this on. And I think it is very true to say that they shouldn't be expected to take on all of these kind of considerations themselves.

And in fact, it's much better if they don't is much better if we think of responsibility as a kind of collective process, one in which we bring in these different kind of perspectives and understanding the perspectives of stakeholders and understanding the role that the different groups and that different people play across these research and innovation processes.

So we can understand that the responsibility is really distributed across this landscape rather than just being placed into the laps of poor, overworked researchers in that sense. So I think if we put these to put all these things together, the understanding that responsibility is about the research process, it goes beyond liability, it's very forward looking and it's shared is shared with different stakeholders.

We can really open up this notion of responsibility and we can use it to foster ethical practise all the way through the process of A.I. research and innovation and perhaps overcome some of the concerns about what it means to to ask researchers to take on all of these considerations and just do them by themselves. It's a far more kind of a collective approach and a very Forward-Looking one as well that I think is really crucial.

You know, we've mentioned a number of times the kind of impacts that that I can have on society. And it's really crucial that we think forwards about those impacts and don't just react to them retrospectively. And there are many other things I could say at this point, but I'm going to stop there so that we can move on to the discussion. Thank you very much. Thank you very much, Eleanor.

One question I'd like to ask you about, I mean, you're very interested in the issues of training and education. And you were talking about researchers taking responsibility and caretaking, I mean, just caring about it. Some people might be rather rather sceptical about whether training can actually make people care.

And I mean, if a research is in a context where, you know, progression in their job or in publication or whatever depends primarily on just getting on and producing research and getting there faster than the other guy and so forth. How far can training make people care? Well, I think training is really vital and sort of early training so so that people learn it sort of right from from the start is really important.

So so we see, for instance, in department he computer science here, we now teach our first year undergraduates ethics and responsible innovation with the idea that they kind of get the basics of it in the first year and they carry it through all the way through their research careers. And the EPSA see the funding body that I mentioned where they fund incentives for doctoral training, which is the four year programmes for doctoral students.

They require the centres for doctor training to include responsible innovation training as well. So again, you sort of have these perspectives brought in very early on so that, you know, people learn them quickly and just assume that they're kind of part of the work that they're doing, not an extra thing that's, you know, terrible that they have to add to the workload, but just a central thing of the work that they're doing.

And I can only speak for the students that I've been in contact with, but they certainly take it on very responsibly, actually. And I think, you know, they can really see the need for the need for this kind of ethical thinking about A.I. But I think more broadly, you know, I do take your point that it's really about sort of what Rosie was saying about a culture of responsibilities, like everybody does it.

It's not like, you know, I do it and therefore I matter. You know, I'm at a disadvantage position next to the next person. It's we all do it. Everybody does it. And we just kind of take it for granted that we do it because we see it as central to the work that's going on.

Thank you very much, and it's getting a culture going in a place like Oxford amongst our students and so on, it's probably a relatively easy task getting it over an international community of researchers that Sakey extremely difficult. But I guess let's bring the. Thank you very much.

That's been really interesting. And let's bring the other two back and we'll talk about about these broader issues, because I guess part of the part of this broader culture is a matter of enshrining things like these research impact statements and and so forth in in things like these big international conferences. By the way, the workshop that Rosie and Carolyn were involved with, that was in Europe's workshop. You were doing movement. So that's that's a pretty big deal.

Yes. Well, thank you very much. A couple of questions I'd like to ask all of you or invite you to discuss amongst yourselves. First of all, of meaningful differences between responsible research publication. And responsible deployment. So, I mean, how much some people might think, well, you know, let's take. Don't worry about the researchers, let them get on with their stuff. What we should be doing is putting the checks on the deployment.

Coming up, I'll jump in quickly, so I think this relates to something I mentioned in my talk around the blurriness in our research in particular when it comes to the distinction between research and products and how we're seeing a lot of research that sort of advances capabilities actually happening within industry in the process of building products out of those developments.

So I've thought a lot about, you know, should we be thinking about changing the sort of research culture or is this actually a wider cultural issue in the whole field of AI that includes research and development and deployment as well? And I think that might be slightly different considerations. But overall, they're so entangled that I do think we need some kind of common stances here.

And I think this idea of the culture of responsibility is an example, like if it does, if everyone starts to acknowledge that, yes, I have become something that has these large effects on the world, that puts us in a position of responsibility towards it, as much as I would like to be able to do research in a vacuum and not have to worry about these things, we are all citizens of society and we have to play a role in that.

So I think I think there are possibly some meaningful differences between research and deployment. And in particular, I'm thinking there's a lot of concrete work that has happened in safety engineering fields. So fields like aviation or vehicles have to have very concrete measures of safety. And I think that we could potentially learn from fields like that when we're talking about deploying AI systems.

However, I do think that, yes, we are because of this tight loop between the research and the product deployment, some of that thinking also needs to apply to the research itself. But this gets close to another question that's been asked. Actually, many other fields have established processes for research. Is A.I. trying to reinvent the wheel? Can we learn from other fields like psychology and medicine and so on? So maybe put that into your into your thinking as well.

Carolyn, do you see that? Yeah. Yeah, sure. And so I think I think as has been mentioned a little bit in the seminar, I think I think to an extent, yes. And to an extent no. So I think research ethics, as it concerns you know, I'm doing a piece of research that's going to affect some human participants. How do I need to make sure that that's ethical and that's all that OK in situations like that.

This is, you know, from other fields are whole, you know, institutional review boards and things like that set up to to to make sure that goes well. And so, yes, 100 percent that stuff should be and and I believe is being pulled into the space. Of course, I often does not have human subjects. Often it's more likely to deal with kind of personal data. But then again, in that case, there's a lot of stuff that can be drawn in from the kind of the data governance space.

So to an extent, yes, I think with things like impact statements, though, we are in a slightly different world where we're not just asking researchers to talk about their sorts of research conduct, you know, people being affected during the research itself. We're also asking bigger questions about know downstream impacts and a impacts, as we called them, and often research, research, ethics, things like institutional review boards don't have quite that scope.

And so in spaces like that, and again, particularly because machine learning is so is so sort of general purpose that could be used in so many different contexts, then that does look quite different. And then and then, of course, there is stuff that you can pull over from responsible research and innovation and such and such things like this. But we need to make sure that it kind of fits fits the machine learning setting.

Well, I'm sure Helena might also have thought, oh, can I just bring something else in there and then I'll come to you next. But I just the thought that occurs in reaction to what Carolyn said and one of our our viewers has raised the issue of responsibility as opposed to liability. And I think that could mix in here. So one thing that comes to mind is that in areas like medicine.

It's relatively easy to establish control over what he's done because the research is typically being done in government or large institutions. A medical research institution can't hide away, can't be somebody in their basement working away on a machine, but researching I can be. So it's extremely difficult to control. You can't control it in the same way by controlling large institutions. And there's this issue of liability. And that's a really tricky one.

You know, with big institutions, you can say, well, if your researchers misbehave, we're going to charge huge fines or whatever. If it's someone someone hiding under a cloak of anonymity, working away in their basement, and if it's possible for them to make big leaps in research, it may be different. But one can see that there is much more of a problem than there is in the case of something like Lightman's. So, Helen, you're on any of the issues we've just been raising here.

Yes, I think with the sort of the liability question is absolutely right, you know, where our research is going on, it's much harder to get the sense of what's being done if you compare it to where it's taking part in sort of big, well-established institutions, which sort of have their own regulations and so on. And I think that our community at the moment is sort of facing these questions about kind of like self governance.

So is it enough to have sort of professional codes of ethics and sort of values and so on? Is it enough to sort of expects people to kind of voluntarily follow them and that will be enough to secure good practise? Or do you need some sort of much more kind of external mechanisms in place?

And I think, again, the kind of the comparison with medicine is a very good one, because, of course, I, I I'm a much younger field and it hasn't had a chance to sort of develop those professional values and so on, sort of as as a field where compared to medicine, obviously you have these kind of external mechanisms in place, but you also have to do very, very well embedded values that that individuals have about sort of what is and isn't good practise in research.

So I think it's kind of an ongoing question, I think for the community, the extent to which is it going to to rally itself to sort of get those values in place, or are some more external mechanisms going to be needed in that? Yes. And what could be I mean, the cultural issue that was raised earlier comes in here, doesn't it? Because people standardly go into medicine because they want to cure people.

They're already pretty well motivated. They're not usually going in primarily from a financial motive to medical research, plastic surgery. But but in a I. Yeah, that's that's that's that's not the same. I mean, another question that's been raised by the problem is who decides what is responsible research, not easy or self evident. And another comment, stakeholder approach is interesting, but whole groups of stakeholders can be sadly misinformed.

But this general question of who decides what is responsible, what is acceptable and so forth to. Any of you have any anything particular to say on the. There's the sort of the ideal view that, you know, you have all of the stakeholders come together and you have this process of engagement and through that consensus emerges. And sort of one of the things that we talk about in the sense of engagement is that it is kind of like a creative process in that sense.

And disagreements can be a place where, you know, creativity can emerge. It can be highly constructive and sort of leading to new pathways and so on. So so the ideal form is who decides it's everybody, because it's kind of a collective understanding that that's reached through these processes. Of course, that the practise might be quite different. And you might find you you might find that ideals in or general cultures in different countries are quite different.

Yes, absolutely. Absolutely. So then you sort of have questions where that where the boundaries mean, because we talk about societal values as if they're their single thing, but obviously that they're not there. They can be very different, they can be competing and so on. And then you face very difficult questions about, well, whose values are you going to, you know, take is important and someone who's values wins.

And and just to finish off with this, I mean, another viewer has remarked that a lot of research, a lot of our societal values have an anthropocentric bias. They're biased in favour of human goods rather than good for the environment. And that throws in yet more issues about who decides and how do we decide. So Rosie and Carolyn, do you want to finish off with the gist of your view on how we decide and who decides?

Yeah, so. I think that this is a question that humanity has faced since the dawn of time. We all have to get along with each other. We might all have different intuitions about what is ethical behaviour. Different cultures have different perceptions of morality. And I think it is valid to ask this question with respect to I. But I do think that this is this is definitely not something that is unique to as a field.

And we somehow have managed to kind of muddle along and learn how to cooperate roughly obviously not smooth sailing, but like we've developed institutions and systems that have allowed us to navigate the fact that we still all have differences in values and intuitions and things. And I think my main sort of the thing I'm advocating for here is not necessarily that we need to right now pick exactly whose values we're going to respect and what those values should be.

And it's more that we need to start building those institutions and thinking through those processes within the specific context of I. So I think we can still make progress on this without having a concrete answer about who's values, who is responsible. We can still sort of nudge things. We can kind of iterate, we can try things and we can nudge things in a more positive direction and hopefully slowly over time sort of build our capacity for wrestling with these more difficult questions.

Thank you, Carolyn. Yeah, I guess I can add add a couple of things, yeah, firstly, firstly, on the on the on the excellent point about only worrying about human values. Yes, certainly, certainly things like environmental impact.

I think there's been increasing recognition within the sort of machine learning community that the that we do need to also think about environmental and environmental impacts of systems, particularly systems that require huge, huge amounts of compute for training, which are just very, very energy intensive. So, yeah, lots to do, but at least there's kind of a recognition that this is something that should be thought about.

Yeah. In terms of the question of of who decides this is that this is a really big question that could get into some really meaty philosophical, philosophical debates. I'm sure. I think I think to an extent, you know, so obviously, obviously a lot of people have very different moral views. But I think across the board, there's a lot of stuff that we can agree on in terms of of kind of certain certain human suffering and certain inequalities that we want to avoid.

At least we have a very good starting point in terms of some of the stuff that we don't want to see happening. And yet, sadly, sometimes do can start from things that we do agree on, both positive and negative. Yeah, sure. And some people would also say things like, you know, we have, you know, for example, that human rights, for example, you know, this is this has been a collaborative endeavour to get things written down.

And we have starting points like that that we can think about what that means for AI systems and kind of building, building, building that then and then. I guess the final comment I want to make is a lot of people really are really thinking about how to get sort of impacted communities at the centre of these conversations. So particularly where you're having systems that are creating real material harms for people and their lives and their decisions.

Now, how do we kind of send to those voices who have historically not been not been necessarily listened to that well in the development of these systems? So I think that is that's a really important a really important place to start with.

And as we think about kind of future impacts as well, thinking about how we can meaningfully kind of translate the possibilities of the technologies and where we might be headed and translate that into a meaningful way such that we can have kind of useful conversations with people who might be impacted and to let people kind of have a say and think about what it is that they do and do not want from these systems, I think is really important. Thank you very much. And only Helena.

Oh, yes. So just very building on what Carolyn was saying, I think. Yeah, we absolutely have sort of, you know, a task to work out what are the fundamental values that we have here. And I think Carolyn's point about, you know, addressing the inequalities in society that can be furthered by, you know, we might want to be able to to reach out to disadvantaged communities and sort of correct out some of these inequalities.

So that could be one of the fundamental values that we're looking at when we're talking about responsible research and A.I. And then sometimes it's a matter of a trade off, as well as sort of understanding that maybe the ideal isn't always possible. So it's hard to see whether balances is so, for instance, a balance between that, the profit motive, which drives the huge amounts of A.I. and protecting the environment, for instance, you probably can't have both of those things together.

So where is the balance between them? So we kind of have a combination. What's the fundamental and where are the trade-offs? Thank you very much. Yeah, I'm struck by, as you gave your answers, that one answer that doesn't come out is, oh, we listen to moral philosophers. They will decide where the truth lies in all these things. Well, we can dream. And I think they even agree amongst themselves. Do they? No, no, no, I. Well, absolutely not.

All right. Thank you very much indeed, Rosie. Carolyn. And that's been extremely interesting, giving us all an insight into a lot of difficult but really important issues raised by research and which is certain to become even more important in the future of funding and research structures and things like that aren't the sorts of things that typically attract a huge amount of public attention. But it's clear that they could potentially have a massive effect over time.

I'm feeling rather conflicted myself between the typical academic prejudice in favour of openness and worries about giving bad guys ideas. But I'm in the fortunate position that no ideas I produce are likely to be of any use to bad guys. The recession has been recorded. It will be added to the rich collection of resources that we're building up at Oxford.

As I said at the beginning, you can find links to past and forthcoming events and the growing set of recordings and podcasts categorised by topic. If you go to the Oxford Philosophy Faculty homepage and click on the Ethics in Link. Before saying goodbye, I'd like to thank Wes Williams, Vicki McGuinness and the whole team of Torch for helping with the organisational and technical arrangements for the seminar. They've made everything much easier for the four of us, and we hugely appreciate that.

Thank you again to our three speakers. Thank you for watching, especially those who've added comments or questions. Do you look out for our future events from the link I mentioned? Our next seminar will be on artificial intelligence and mental integrity and takes place in three weeks time, five p.m. on March the 21st. Until then, thank you again and goodbye.