The Institute of Internal Auditors presents all things internal audit. In today's episode, to kick off the new year, we're counting down your top five favorite episodes from 2024 from tackling AI risks and ethics to uncovering misleading greenwashing claims in exploring the next frontier of quantum computing. These episodes highlight the key issues shaping the internal audit profession.
So whether you're a longtime listener or new to all things internal audit, grab a seat and join us as we revisit the episodes that made the biggest impact in 2024. Let the countdown begin. First up is our most streamed episode of the year. Robert Perez and George Barham unpack the i's updated AI auditing framework, giving internal auditors the tools to tackle the challenges of generative ai.
So, as you mentioned, it is in four parts, and I'd I'd like to break those down in specifics of, for, for each one. So it begins with an overview. Uh, is that correct? Yes, yes. Overview, introduction. Um, we spent a little bit of time talking about the history of ai. Mm-hmm. It's, uh, it's certainly not all encompassing, but it's a look through the decades of, uh, different developments.
And even before it was called artificial intelligence, you know, some of the theories and some of the, the applications, um, it goes into machine learning. It is certainly something that's been around a little bit longer. So we spend a little bit of time in the framework in that first, uh, section with the introduction overview, making sure that there's a definition, uh, making sure that there are examples.
And so we kind of, you know, got a, uh, the reader through, uh, you know, how AI has developed through the years. So, in part two, the framework is all about how to build an understanding of where you're at in your organization. Correct? Right. And, uh, part two, it, uh, like we, you know, we talked about a second ago, it starts to build on the foundational knowledge that, uh, you would've acquired in, in part one. So, um, it, it's really the next step.
So it's okay, now that I know what AI is, I understand it, I understand examples. Now I want to start having those discussions in my organization. And, uh, also leading up to that, um, as an auditor, what can I go ahead and be proactive in gathering?
So, policies, procedures, um, you know, board minutes, things like that, that are maybe already existing that you have access to from your other audits, um, to start pulling that information, start having those discussions with, uh, some of the folks in your organization you work with, such as the CFO or if you're an IT auditor, maybe the director of it. But it, it's, it's very, uh, basic in its approach.
But, um, again, it's, it's how you would, it's getting you more comfortable in what topic should I ask about and how should I go about it? So, yeah. Part two is all about, uh, the engagement piece and leveraging the relationships that you already have. And, and, and part three gets us to sort of the meat of the framework itself. That's right. That's right. Part three is actually called the framework. Um, so we break it down into three parts. So three domains.
Uh, it's governance management, and internal audit. Um, the governance piece, you know, we like to start there. I think if you were looking at really any topic, um, that an internal auditor would focus on, you'd wanna understand the, the oversight piece. You know, who's, at the end of the day, who's responsible for making sure that the organization is, uh, is, is doing what they say they're gonna do.
And then making sure it ties to those, uh, strategic initiatives, um, that the organization is trying to accomplish. So we start with, uh, we start with governance, then we go into the management piece, which would be more of the day-to-day, um, how are we monitoring, how are we managing it? What, uh, key performance indicators have we identified to make sure the AI is, is doing what we'd want? And then also the, the risk management and control environment.
So, um, those are things that are, um, if you look at our three lines model that are the first and and second lines, um, and then the internal audit domain, um, would be that last piece, uh, of the puzzle. And that's gonna be focused on the advisory piece and the assurance piece. And, uh, we think that there's really an opportunity for auditors to be an advisor with ai, maybe more so than, uh, jumping into audits.
But, uh, certainly some of the organizations who are further along, um, that can continuum of AI development. Certainly there'd be some opportunities to provide assurance services too, but we're really, we're trying to encourage our members, uh, and those in the profession to try to partner with managements and, and be a resource. And typically that's in an advisory capacity. Yeah, and I, I'm glad you mentioned the, the three lines model.
'cause it is part of the framework itself, the, the sort of bring that familiarity for folks who are familiar with the three lines model. Yeah, absolutely. We feel like the three lines model is a, a tried and true and pretty widely adopted, um, model that, um, that people in the profession, um, look at every day, not just for artificial intelligence, but uh, really provides, uh, uh, the, the basis and understanding of how a control environment is organized, right?
So you have those first and second lines that we talked about, you know, management, risk management compliance. You have internal audit as your third line, right? Your independent and objective, uh, party. And then you have, uh, internal audit working with both management and providing, uh, input to the, to the board or the audit committee or whatever the governing body is.
So they're really relying on us as internal auditors to help them, um, you know, govern and, and have the information they need to provide that oversight. Excellent. So part four is, uh, uh, the framework provides some great tools, uh, to help practitioners actually get started. Tell us a little bit about that.
Yeah, so part four was really, uh, the intention there is to, uh, to give our members to give practitioners, uh, something that they can just, you know, print off and get started or have it on their tablets and, uh, and start to have those discussions. And, uh, we wanted to give them, you know, not just, uh, you know, the knowledge and the information within the framework. We wanted to give them a tool. Mm-hmm.
So, uh, it's, it's certainly something that I think you can, you could customize it and it's not gonna be a one size fits all, but I think if you, if you use that in initial checklist that's in that, uh, practitioner's guide in, in part four, I think that's gonna get you off and running.
And that was our goal. And then we also feel like that, um, you know, the, the, the practitioner guide, the framework itself, we reserve the right to update that and, uh, and make it more valuable, um, especially as the technology changes. So, um, I think there could be additional checklist or maybe additional considerations that we, we update through the, you know, through the months and, and coming years on that.
So really just a jumping off point for an auditor, AI is powerful, but with great power comes great responsibility. Our second episode features experts in the field sharing insights into governance, ethical dilemmas, and strategies for safeguarding sensitive data in an AI driven landscape. Yeah. Let me ask, uh, for a second, if your organizations are using artificial intelligence, have your audit functions, uh, tested or verified?
The, the ethical risks, uh, related to the, the usage of, of artificial Intelligence? So, AI has been in existence, especially in our organization for many years. So, and ethics, uh, and bias and all of that has already been pretested in the first place. So with gen AI is just a new different type of technology, but you know, the same baseline. So it's been tested and, and,
and many, many times all over. Yes. So, Charles, are you seeing more broadly in, in, uh, your, your clients that, uh, that is a, a concern? And, and, uh, are, are people looking to some sort of like standardized, uh, you know, good seal of a, you know, how housekeeping or good housekeeping seal of a Proof or something like that? Well, look, I think, I think, uh, generative AI and artificial intelligence in general has always been an area where we are relying on data from various sources, right?
And data, every data set has bias in it, right? Every data set has errors in it. And the people that are making the models, like the big technology companies, they are building, you know, I think very rigorous controls into the models that they commercialize.
And, you know, as consumers of those models, I think our biggest concern is not necessarily the internals of the model or the guardrails on the model itself, but more how do we talk to our teams and our employees about what the model is and what it isn't, so that there's clear understanding of how the model is producing what it's producing, and that we have a process to think about whether the use cases that we are applying those technologies to make sense in the context of our business.
If you are a lender, you don't wanna be discriminatory in your lending, regardless of whether you're using AI to do that discrimination or have old manual processes, right? I don't think the, our fundamental desire to have a fair business that is free of bias doesn't change because we introduce AI into the picture.
We just need to be aware that AI has the potential to increase that bias, um, in some cases, and to make sure that we are thinking about that, that we have controls and processes in place to identify those issues when, when they're relevant to monitor them and control for them. And then in the, in the event that we see problems that we remediate them. But I don't, you know, I don't think that privacy or ethics exists in an AI vacuum.
I think we have to do that with all tools, all processes, all software, because manual processes have the potential to be biased and unfair as well. Right. But another concern, uh, or that occurs to me is that if these models are learning, are they unstable? Do they change over time? And, and how do you, uh, test or, or verify that it's still operating as intended?
So typically in the ecosystem, in, in the ecosystem from a controlled perspective, uh, you have monitoring controls at the front end, preventative controls, and you also have backend controls to make sure the least, if there's anything or any anomaly that comes out of it, it's corrected as quickly as possible.
So in simple terms, when you build a model, you wanna build it in an ecosystem with the right level of controls from the front end and the backend, and also the preventative and the detective type of controls. Are you seeing anything similar at your organization? Yeah, so it's definitely a challenge. The velocity at which models are being produced these days, it's, is insane. It is really fast. One of the key principles we, we stick to is the governance at the firm level.
So the standards and the monitoring independent of the functions that are implementing the solutions has to exist. So, for example, whenever we implement a solution, we will have to have touch points with the governing body of AI and reaffirm our use of the solution and being still able to produce the results that we want.
Uh, and also you are responsible for tracking the areas where, and this is difficult to do, it's not easy of where actually the model may not be behaving, uh, you know, at 90 or 95% the way you expected it. So, so I think the instrumentation that you need to observe that and having a third party governing structure within the firm, kind of asking you on a periodic basis, are you monitoring your models, you know, this new thing came up.
'cause you know, there will be bugs also on things that, you know, could be found. So, so I think, I think the balance has to exist. It's almost like that having that independent function checking is, is, is crucial. Taking third spot on our list is this eye-opening episode on Greenwashing host, David Petski speaks with Edith Wong and Brian Wilmont about exposing deceptive environmental claims from financial risks to practical strategies.
They reveal what internal auditors need to know about ESG challenges. Today we're talking about greenwashing, and that's a pretty specific term, uh, and many people may not be familiar with. Can you tell me a little bit about, you know, what do you, what do you mean when we talk about greenwashing? Sure. So, um, greenwashing is a specific term, but it would be a term that we will also say doesn't really have a universally accepted, um, legal definition yet.
So for things like fraud and corruption, you have a definition, you can say exactly what those are, but for now, greenwashing is more of an umbrella term that just generally refers to the practice of where companies are. Perhaps, um, you know, enga make certain environmental claims, but don't have the, the data to necessarily substantiate those.
Some of those claims may be more aspirational in nature, um, but in real life, in practice, they're actually just kind of going on with business as usual and don't really, haven't, haven't really changed their approach in their practices to actually substantiate those, those claims. And Are you seeing, uh, those types of cases come up where, uh, people are, um, bringing actions against companies for their, uh, environmental claims?
Yeah. This has, this has been a pretty active area and it's, it's likely to get, I think, more active as well. Uh, a lot of the earlier litigation activity, regulatory activity, I think is focused more around kinda the more adv avatar advertising marketing space and what people are saying. But now, uh, there's a lot more activity and a lot more the greater obligations that are being imposed on companies regarding disclosure and reporting.
So what I expect, you start to see more about, you know, what people are saying to investors, you know, what they're saying in their, their securities filings, uh, you know, what, whether they're, you know, holistically reporting about what their operations are and, and how they're addressing sustainability issues. Yeah.
And are, are there any like significant examples of, you know, um, that you can point to to say, well, here's one, you know, well-known, uh, case that was brought or one well-known, uh, you know, incident of greenwashing? Sure. So there is one that we were talking about earlier in our panel where it's not necessarily greenwashing. I mean, it, it's, it's kind of falls within fraud generally, but it does involve environmental claims.
But it involved a company who was working in the automobile industry and, um, they were apparently providing false data and misrepresenting their, um, carbon emissions within their vehicles. And so, um, you know, this was discovered and it, it, it resulted in a very large fine for the company. Um, but since then the company has, you know, identified what went wrong.
And it's also, um, done a great job in terms of now, um, you know, being able to implement their compliance programs and set up so that they're actually now making sure they're identifying these risks and have become a really, you know, a good example of how to kind of turn it around and make sure that their, their claims, um, really are verifiable and have been fully vetted. Yeah. And that's, you know, pretty well known example. I think I know what you're talking about there.
Um, are, are there any other types of, uh, you know, possible actions or, or statements that companies make that could, uh, be leave them to some exposure for greenwash? I think a big category that we've seen a lot of activity around is where companies are making kind of vague statements about what they're doing to address sustainability concerns.
Uh, you might see that in a circumstance where, you know, they may be a big emitter of greenhouse gases and they wanna be seen to be doing something to address that.
And then questions become, you know, whether, whatever they say they're doing, whether it's investing in renewable energy or, uh, trying to reduce some of the climate impact of their, the products that they manufacture, that sometimes it's seen as, oh, you're trying to hide or distract from the other part of your business, where a lot of those emissions are, are kind of on an ongoing basis. So I think that's one area you see a lot of, a lot of focus on, you know, is those types of situations.
I think what's also interesting about greenwashing is you can kind of see a range. So for example, you know, there could be a, a company that can make a certain claim about how much of their product is made with recyclable or recycled materials. That's something that you can very much test. 'cause you can say, alright, X percentage of this product is made out of recycled materials. You can test it's, it's either correct or it's either incorrect.
Um, but there are gonna be other claims where you can say, you know, we are environmentally friendly, we are green, we're natural. So those ones are a little bit more, you know, ambiguous and vague in nature. And those ones are harder to test, but at the same time would be subject to the same level of scrutiny.
Because if you are misleading consumers towards making choices to purchase or use services by your company based on these environmentally friendly kinds of statements, but in reality, you're, you're not really doing much to substantiate these, that that is a concern and opens you up for all types of risks and negative impact to your company. Ranked forth this episode, explores the next frontier in technology.
Host Lynn Mole and guest Nick Reese discuss quantum computing's game changing potential for internal auditing. So, Nick, tell me, what is quantum computing? So quantum computing is a new kind of computation that is entirely different, not better, not worse than the traditional computers that you and I use every day.
And we can talk about exactly why that is, but it really gets down to, uh, the ability to use the strange and quirky and weird, uh, properties of quantum particles, uh, to affect computation. How will quantum computing change the field of internal auditing in the next few years, do you think? Well, quantum computing is gonna change a lot of fields in the next few years. And, and that change is actually already underway.
And depending on how you look at it, we might actually be already a little bit behind. And so it'll change internal auditing in a lot of probably very specific ways. But really it's gonna change, uh, a lot of, a lot of different areas chiefly, because this is probably going to be one of the most important cybersecurity challenges that any of us face in our lifetime.
And it really is gonna get down to the difference in how we protect data and what privacy is going to mean in the future in a world where we have quantum computers. Okay. So what I hear you saying is organizations are gonna be using quantum computing and internal audit has to be aware of the risks around quantum computing 'cause their risk to their organization? Well,
Yeah. So they might be depending on the different organizations, but I, I think the first thing we have to think about is, uh, actually kind of malicious use. And, you know, I, I hate starting at a place of risk, right? Because there are risks and there are opportunities. Mm-hmm. And I wanna touch on the opportunities, but if we're talking about the things that internal auditors are gonna have to eventually look at, it's gonna really come from that risk side of things.
And right now the technology is at a place where we are going to need significant investment to be able to have a quantum computer that's capable of threatening our cybersecurity. And so really we're thinking about, uh, large multinational corporations, maybe in partnership with, uh, nation states. And so, uh, we've already, we're already seeing activity in this area.
And so for the nation state purpose, uh, we're already thinking about how they might try to use a quantum computer to disrupt steel, um, or otherwise kind of decrypt sensitive information from other organizations. So I think that's the place that we should probably start. Okay. So what, what industries are now using quantum computing? Well, quantum computers aren't actually, uh, mature enough to this point that they can be really impacting value for, uh, for, for, you know, organizations.
And so, you know, what we are seeing right now in this state of quantum computing is a lot of, uh, kind of laboratory level testing, proof of concept, things like that. Um, mathematically, and by the physics, we know what a quantum computer of sufficient capacity, which is kind of the right way to say it, some people say big enough, which is not the right way to say it, of sufficient capacity, can threaten our current, uh, asymmetric public key encryption.
And that's really, you know, the, the, the heart of it is the ability to decrypt what we thought was practically at least, uh, kind of uncrackable can now be, well not now, but it will be cracked in a matter of a few minutes with a quantum computer. What are some security risks linked to quantum computing that auditors need to know about? So this is, this is really the crux of the matter, and I think that auditors need to know kind of, kind of two things.
They need to know what the threat is, but they also need to know why it's a threat. And it's, it's kind of not good enough for me to just say to you, well, there's a threat and you know, you as an auditor say, well, I'll be on the lookout, right? Like, that's not, that's not good, good enough, right? We have to, we have to talk about why. So I think there's a huge kind of workforce training component to this. And then there's the, you know, what does this mean for, for auditing?
And so where I'll start with this is, you know, quantum computers are fundamentally different. And the way that I would describe that is, you know, the world we live in is deterministic. What does deterministic mean? Well, it means that, you know, if I, you know, drop a book from my hand, you know what's gonna happen mm-hmm. Before I even do it, right? Does that mean you're clairvoyant?
No. It means that there are very specific rules, uh, in physics that govern our world, like gravity, like air speed and things like that. And the, the same applies to co to our traditional computers. When we do computation, we co we do computation kind of linearly, binary code ones and zeros, and we read those ones and zeros. But in quantum computers, it's kind of a, the best way I can say it is two plus two doesn't necessarily equal four.
Sometimes it does, but it only does with a certain probability. And so what a quantum computer can do is instead of looking at answers, so to speak, linearly, it can actually look at all possible answers at the same time and then collapse into the correct answer. And what that ultimately means is that if you're trying to decrypt, uh, an, an encrypted message, instead of having to try every possibility one at a time, you can actually try them all at the same time and then find the answer.
And so this, this is what, this is the threat that we have right now to our public key encryption. Public key encryption is one really, really big number that goes out as a public key. And then the prime number factors of that number are the private keys.
And it's, and, and it's, it is increasingly, well, it's, it's very difficult for a traditional computer to guess what that factor is because it has to try them one at a time, and it'll take literal billions of years, but now it'll just take a couple of minutes. And so that is, that means that effectively all of our encrypted traffic on the internet, communications, finances, all of that stuff would be effectively in the open.
Rounding out our top five, this episode A mess, the high stakes world of Financial crime host Ricardo Martinez sits down with Antonio Kadi and Alessandro Kadi to discuss how criminals are leveraging AI and how internal auditors can use it to fight back. My question to you is, how do an AI-based control, um, improve the detection of financial crimes? Okay. There, as I said, we are also at the early stage.
We recently run a survey, uh, with 400 selected clients, uh, in, uh, throughout the Europe, middle East and Africa, practices of pwc. Uh, and, uh, through that of course we touch, uh, ours, the A ML market and our A ML controls were going towards, and we touch also how AI was, uh, implemented into that is the hype. And of course, we touch also on that topic and was very interesting because basically there was a wide majority, almost 80% that is really focused on transaction monitoring.
There is where they see the biggest potentiality. And I, I think it's quite clear there because what is the benefit of ai, benefit of AI is in the prediction, in the prediction analysis. And therefore where you can limit the risk is to start to predict better your, uh, transaction, your fraud, the exposure to transaction, and therefore how that can be, uh, tackled and how can be mitigated towards a better predictive analysis that AI can do.
And actually it was interesting because it was absolutely confirmed by, we did the same poll during the intervention today. And indeed transaction monitoring was, uh, the biggest was picked as, uh, the key topic. So indeed there is where probably the market is, uh, seeing it as a key, uh, point for implementing their controls. There are other potential, I mean, for sure, uh, screening systems, screening against sanctions, for instance. There you can reduce a lot the false positive.
And so helping, uh, the analyst to focus more on, uh, the quality side of the review, and at the same time also customary diligence on that side. It can help to, for instance, digest big volume of analysis. You know, when you do complex products like in Luxembourg, las such securitization funds, and you have maybe to review, uh, 600 plus pages. I mean, if done by, we did some proof of concept where, uh, these 600 and plus pages were rather in one hour by the machine.
So with, you know, you come out with analysis and then you can really focus on what are the risk that you need to focus. And that's where, uh, it can give a great added value. Ai. Yeah. Uh, I think in order to reduce the false positive, you need to, uh, training, you need to train the system, the AI system, and you also need to train yourself on using ai.
Of course, at the beginning you will have a large amount of positive and then keep training, and you will see that the positive will go down the, to the tax on suspicious transaction. Yeah, you need to training the system. And of course you need a cybersecurity professional. Of course, you, you mentioned the back test. This is a, these are the traditional way to test, but in this case, as is a new kind of risk, we don't know too much about that.
And so in this case, I would say also to go with a simple risk assessment and then to compare the, uh, residual risk with the risk appetite of the organization. But it's important to keep training the system because it's a machine learning system. So if you learn, if you teach the system, the system can produce better result. If you like this podcast, please subscribe and rate us. You can subscribe wherever you get your podcasts. You can also catch other episodes on YouTube or@theiiaa.org.
That's THE iia a.org.