And commercial organisations, well, thank you and thank you for having me along today, it's a pleasure to talk about a topic that's slightly outside of my usual comfort zone. I'm going to address questions that are in the air business school and the business people that we're talking about in the context of a I. But I am acutely aware that in doing so, I'm doing so as a specialist in the governance, the behaviour of investment banks.
So I'm going to step a little bit outside the boundaries of my expertise at times today. And I apologise in advance if you think that at some stage I'm teaching you how to suck eggs. I'm kind of in terms of my remit as I put about what I might see today as covering business in general and finance in particular.
And I've used this seminar as an opportunity to to think about the speculates upon some of the things that I and my colleagues at the business school are worrying about as we contemplate the extension, the extension of A.I. into business life. I don't really have any answers to the problems I'm going to to raise. I'm just going to generate questions and the answers are going to be very far from clear.
So the the the the first thing I think arises in this context is the extent to which businesses know their employees and to which they know their clients. So to illustrate the point I'm making, I'm conducting a bunch of interviews at the moment with a co-author from the states with very elderly investment bankers. People in their late 70s have been in the business 50 odd years to find out how the business has changed or what sort of cultural standards drive.
Investment banking and what received ethical wisdom is in investment banking, and it's probably unsurprising to you that everything has changed dramatically, particularly since the 1980s. One of the things that came up has come up multiple times in conversations with banks is the enormous power of the compliance systems that they have now. So one bank told us that they have a compliance system that logs every phone call people make every day.
That's not surprising. Every keystroke they make on their computers is when they go to lunch and when they get back in. There is how long they take a cigarette break and then when they get for coffee.
And this system tells them when people are in danger of leaving the bank. So they get a flag saying this person might be about to resign, and then they start watching that person very closely to make sure that he or she doesn't leave with customers and with confidential information and that sort of thing.
And that's a very striking fact. Oh, always teaching the banks how to double guess the decision of their employees and me has already indicated that it's probably better double guessing those decisions than the partners, those employees. Sort of interesting question to me is whether it's better a double guessing those questions than the employees themselves. And I kind of suspect it is. I mean, my party can test my decisions better than me.
So if Facebook can be my partner, it can surely beat me a match, thus a striking thing. And that means that businesses already understand more about their employees and their employees do. And I think quite some quite difficult questions arise as we think what the relationship between the business and the employee is on what the business should do with that information. And it's not just about employees, of course. That sort of data gathering applies also to customers.
And that leads broadly to questions about how markets are going to look in the future. We're not just selling products, we're selling knowledge of people and product information about what those people are likely to do. So there are some sort of trivial questions that everyone has anticipated about data integrity and data ownership. It's collectively or individually or organisations outside of me understand me as well as I do.
Surely, at the very least, I to Thomas, perhaps we ought to be able to assert some property rights over that. Information of the institutions that would allow it to do so don't exist at the moment. Although one of my colleagues at the business school is leading an effort in the commercial world to try and establish those rights, which is an interesting thing.
There's also a question of whether the organisations that understands better than we do ought to tell us what we don't know about ourselves, whether that's actually even a meaningful thing to contemplate doing. So maybe corporations that have gathered information about others that have, you know, 200 likes could help us reflexively to think about what we are, what we ought to be. And those seem to be quite deep questions that haven't been properly teased out at all.
There's also a bunch of questions about once a corporation understands as better than we do, whether we can manipulate us and whether a committee plays in ways that are good for us in ways that are bad for us. For example, one of my. As opposed to research called Alex, Peter Blank is analysing the ways that a large bank, I know which bank it is, but I can't remember if it's public uses, I talk to clients in its online stockbroking arm.
So one of the things that comes out of her research is strong evidence that the clients of this bank, or at least by the likes of finance research, profoundly irrational. So they don't do what finance theory says they should do. Whether that's really rational or whether we just have the wrong definition of rationality is another question. But for example, they they have reference points that are almost random. You can manipulate the reference points by giving them information.
And of course, once they understand this, they know which people are most susceptible to what sorts of information, which is the sense in which those people's quote bounded rationality, unquote, is the knowledge of the bank and is something manipulated by the bank to have to alter the way that they trade. Now, there's no suggestion that people at the bank in question is doing this.
There's also absolutely a banking question. If it chose to could do this, it could take advantage of what's called the disposition effect, and it could use that to alter the way that its clients behave. That's interesting. Is it OK? Probably not. If it isn't, what should we do about it? And when I say these firms are irrational, what do I what? These people are irrational? Do I even have the right version of rationality in mind?
Well, to make sense of that, perhaps I should think about the reasons people have for taking actions. The reason giving is, of course, important for people care about ethics. The fact that a value systems uncover and respond to those biases doesn't mean that any person operating the system understands that this is happened upset, well-known and obvious fact. And that means that no one can actually discern the reasons the choices that are being made.
So this is well understood in the context of things like discrimination over gender and discrimination over racial origin. It's also well known that it's incredibly hard to correct that sort of bias. You can go through and score out. All the words are obviously gender specific. And it turns those all sorts of words that you don't immediately think of as being gendered or gendered.
And we know that because the computer programme correctly identifies the gender of the person who's speaking well, that's problematic is problematic because we don't know the reasons that these things are happening. And usually when we think about ethics, we think about reasons. So there are obvious questions here that people have already bloviating about at some length. What should we do about this sort of bias? How can we agree, even though this is just a definition of bias in this context?
But I think the more interesting or is the deeper questions I worry about myself at the moment. Twenty seven minutes relates to accountability. If a customer calls out a financial firm or any other firm for bias and recruitment, or for way it behaves when it gives financial advice, it's not clear what it means the firm to be accountable. Because if a firm is relying on a system like this, it cannot give good reasons for what it's doing.
And if you can't do that, it's not clear that the firm is an accountable agent is not firm, even clear how we should regulate the firm's activities. And I think that is a concern both for people and business and for the people who regulate business. There's also a concern for me, and this kind of starts to tell. It's like back to work. I'm doing myself for what it teaches us about moral learning and the acquisition of moral wisdom in business to use a slightly loaded.
So if we allow corporations or on our talk a bit about what it means to allow a corporation, but if we allow corporations to pass responsibility onto algorithms, in some cases, perhaps even unwittingly to pass that responsibility on, I think it's interesting to ask whether the people who work for those corporations become less morally aware, rather appealing here to literature.
That suggests that the skills you need to make ethically complex choices in business and the professions is a learnt skill, is highly contextual and it's acquired through up through use. You can't learn how to what an ethical choices in investment banking or in the law APR, right? You actually have to engage in those choices. You have to talk to the people who are engaged in the choices.
You have to receive feedback and you have to engage in dialogue. And if we deploy even an absolutely excellent first class A.I. system, I wonder if we run the risk that we create a cadre of moral imbeciles that simply cannot exercise complex judgement and cannot post their values in a meaningful sense onto later generations? And one doesn't want to point fingers. But in the industry, I've spent the last decade worrying about investment banking.
I think this sort of thing has happened. I think it's unfair to describe all investment bankers as moral and. But it's also true to say that some areas of investment banking that used to be characterised by the exercise of quite deep judgement, like giving advice have now been turned over to activities that are to the markets that are very observant, that are very contractual, right, data driven. That's a great thing.
It has actually reduced to less of the sort of biases I was talking about was also reduced in less acquisition of judgement because this judgement is required of those businesses. And that's a concern for bankers. It's a concern for regulators. I don't think it should be a concern for all of us as this sort of system codifies and takes out of human agency called important decisions. It also raises questions that relate to sociology that came out of Stanford in the mid-1980s.
People that will talk about all of this, they are taking moral decisions and being subject to public discourse, not only taking away our capacity for judgement, but for. Is it also perhaps unthinkingly steering is into common ways of thinking generating unthinking consensus? And if it is, I suspect that's a bad thing. We're going to talk about two more things, one of which is obvious, one of which I suspect is not obvious. One is this the effect of this stuff on free speech?
Some firms are starting to use A.I. to police speech. One of my colleagues, the business school, Natalia Efimova, who originally was a computer science faculty member, is now on the road working in the future marketing initiative. She is attempting to understand how Facebook and other actors might identify misogynistic language in chat rooms and other fora. Using artificial intelligence is remarkably hard to do when you can identify some obvious words.
But the tenor of some you actually have to internalise the tone of language to make any sense of this. So Uber is interesting and important work because no single person can police this. The team of people can probably police the volume of messages that was promulgated on Facebook, but we generate some obvious questions about where you draw lines, and we also generate some quite important institutional questions about whether we're happy to allow Facebook to draw those lines.
Right now, the default answer appears to be yes. Right now, we allow Facebook to decide where the line is drawn between unacceptable misogynistic language and just provocative language.
And if they can't readily give us the reasons for their decisions, which they can't, if they're relying on a system that they don't properly understand, and perhaps no one understands that deep learning system, how do we decide what the proper division of responsibilities is on this sort of line drawing between the state and the corporation? And how do we design the institutions that will allow us to draw those lines? There's a hard questions that I think we haven't really got properly into.
I want to talk about relates to moral agency, which is one of my own research interests. We talk a lot in the business school and we talk a lot generally in society about the notion of CSR corporate social responsibility. What do you sort of think about that phrase? It's far from clear what it means. What does it mean to hold a corporation responsible? I am responsible for my actions, at least to some extent.
Is it meaningful to say that IBM is responsible for anything, or is it just the actors at IBM? And this is something that people have been arguing about in business and in philosophy since an influential school 40 years ago by a guy called Peter French. And I think we all took the arguments that were going to be considerably muddied by the arrival of artificial intelligence.
So on the one hand, the list nearly double one of the literature on this stuff is concerned with the extent to which you could be a corporation's being autonomous. So can I view the goals and the ideas and the baby, the decisions and the propositions of corporation views as true as independent of those of its individual members? And there's a consensus that on a reasonably weak assumptions, you can.
Well, if that's the case, presumably A.I. renders corporations even more autonomous because no one person, no bunch of people making decisions on sales. What decisions on some level? Perhaps we have to stop being a corporation is governed partly by Illinois as being more morally or homeless. But I have quite profound problems with that.
My approach to this problem derives from a novel literature, which dates back to a 1962 article by a philosopher, Straus and Straus Beatles Dawson, who at the time was more college under his argument, a more recent work by a Yale philosophical Stephen Dole but identifies moral agency in terms of firstly, the extent to which reform attitudes like outrage or indignation towards one another, and the ways in which we respond to a properly expressed attitudes of that sort of moral agent is someone we
can feel outraged about on a moral agent and someone who responds appropriately when we feel outrage towards. The moral agent, so I'm kind of interested myself on whether we could be a corporation in that way without worrying too much about deep metaphysical questions about autonomy. But I think the presence of AI makes these questions very, very difficult. So how can we can reform an attitude of outrage towards a computer system that no one really understands?
I don't know. I think we probably can buy the new and if we form an attitude of outrage towards an artificial intelligence system that, for example, is introducing regularly but consistently biased decisions and recruitment. Can we meaningfully hold corporations accountable? Because I've already argued that the corporation control reasons for its doing, they call apologise, meaning meaningfully for what it's doing because it's just a computer system.
And if we can't do any of those things, what happens to legal notions of corporate accountability and corporate responsibility? I have no idea what the answer to these questions is, but I think they're really important. I think we're going to care more and more about them in the future. Thank you.