The Institute of Internal Auditors presents all things internal audit tech. In this episode, Robert Finley talks with Lynn Moll about the evolving role of data analytics in ai. In internal auditing, Finley shares strategies for overcoming challenges, best practices for integrating data analytics, use cases in the importance of clear communication with stakeholders.
They discuss the differences between basic data analytics and ai, the skills required for each and how advanced tools can enhance audit processes. Can you briefly explain the difference between basic data analytics and AI in the context of internal auditing? Yeah, so the basic data analytics is often a response to a specific query. So quite often I'll be asked to go and just say, find duplicate invoices or something where we've been invoiced a certain amount but the delivery was for less.
So it's often an answer to a very specific question. With AI coming in, it is almost searching for other issues that we hadn't thought of. So I think the AI is going to expand the horizon if we use it judiciously. Um, alternatively it might take us down rabbit holes that we didn't really want to go so it could be less focused. So I think that's really the core difference is the focus on the actual data analytics and what they're trying to achieve.
Would you say that the fundamental skills or knowledge that internal auditors need to have for traditional data analytics are different than with ai? With ai? Does it become easier? Possibly could. So we've just started putting in a tool for our business auditors, which takes away a lot of the pain of extracting the data knowledge of the systems technically. Um, it allows them put into almost English terms what queries they want to run and then they can interpret the results much more easily.
But it's taking away a level of technical expertise required. Now this is deliberate. We've done this because the technical auditors, uh, finding it a bit of a struggle to find the time to do all these technical elements for them. So the more that we go down tech, the technological route where AI is doing the work for us, it's gonna take away the requirement for some of the skills.
Now you still need the inquiring mind of an auditor to interpret the results of course, and the methodology will much remain the same, but some of the, how can I put it? The donkey work may well be done. Gotcha. Um, can you provide examples of specific audit scenarios where basic analytic tools like Excel or ACL are particularly effective? Yeah, so actually on almost all the audits I've done, it's been done with Excel and ACL or Idea or our Buddhist.
These tools are really effective, especially when you already know what the question is, but the key is to do the data extraction correctly in the first place. So they're very effective and cheap. If you get your data extraction, get the exact data you want because you know what the question is you're trying to ask. If it's more random, uh, you might need a more complicated tool or if you're putting together much different data sets for more than one system.
So I've used things like ClickView in the past, uh, where you need a more complicated system than ACL or idea. How have more advanced tools like Python scripts or Tableau provided additional benefit over those basic tools like ACL and Excel? Yeah, So we've used them things like Python quite a lot. Um, and the reason is it gives us flexibility in what we're trying to achieve.
So we can actually write a lot more complex scripts if we start using a programming language ourselves in a way, something like ACL is like pre-written queries in itself. You can still tailor them and change them and extraordinarily useful tool With Python, it's taking us to another dimension. We've used it all kinds of technical reviews where we just go, it'll be too hard actually pass this into ACL and then run the queries we want.
So depending on the technology as a general rule, if I go a slight tangent here, um, I tend to employ people with lots of different skills. I like to recruit people with diverse skills 'cause you don't know what situation you're gonna respond to technically. So in the world of IT audit, um, you need to use the right tool for the right job. Something like Python can be the right tool where ACL isn't the right tool. Do you need that flexibility in hand? You also talked about access to the data.
Does your organization let you run Python against the databases directly? Generally, no. Um, so we're not totally reckless. I am allowed to use, uh, native queries in SAP. So I've had to show that I won't bring down the whole database by doing that. So I was do my down data extracts where I can from our ERP. So most of the people on my team were actually quite adept at doing these data queries and we've done them for many years.
Um, if you don't have that expertise, you're gonna have to ask somebody to do it. For you to start running queries. It's almost like you're putting in a new application. So you would need somebody to go and test it properly before they would allow you to do it. And that's just too much work, too dangerous. I don't know that they would trust us to do stuff that wasn't under the control of SAPs controls itself.
It's possible on some databases generally we've got them to get, give us the data, then we run the Python script. What are some criteria you use to determine which tool is best suited for a particular audit task? Probably complexity of the query. So I would be looking, say the example I gave a click view. I was trying to merge three different databases at once and I just needed a be.
And what occurred to me, it was not only it would be easier and a more complex tool, oddly enough, but also that would handle the presentation layer back to management better. You have a really simple tool, um, you just got a simple result. Whereas ClickView has enabled me to give a, a model to management that they could drill down into. There was a lot more work involved. So fundamentally the criteria is how complex is the initial starting point.
Could you share a particularly challenging audit scenario you faced and how data analytics helped overcome it? I was facing the s if I got it wrong. Um, and it was a date benefits realization review. And these are challenging because quite often in a business case management make all kinds of claims what benefits they're going to get from a system. So in this particular instance, it made a, a very tenuous claim to the benefits they got by putting an HR system in place.
So I went to go and check that. Now without the analytics, I couldn't actually possibly prove it one way or another, but I needed all the data. So it was particularly challenging to get all the data from all the previous databases, pull it all into a data analytics and then review it. But it was worth it because I had a hundred percent coverage of what had actually happened and at the end I was able to demonstrate the results.
Now politically, this is where another angle of all for all auditors is, isn't it? Sometimes the result's not politically palatable. So they um, they kicked off, they fought at tooth and nail and ended up in the chief execs office. Uh, in a very, I can only describe it as an extraordinarily tense meeting, uh, which you've probably been in many yourself. What was one of the most surprising insights or benefits you gained from using data analytics in an audit?
Um, I think the very first time we tried to do, uh, measure best practice. So this is a very tenuous concept and I was asked basically by the audit committee, uh, have we done best practice and it's a very tenuous thing. So I had to try and codify what I thought best practice could be. And the results were fantastic. I got a great model to give to management, but the results actually, we cleaned up our access. We were much more efficient.
So tasks were taking less time, we could measure the time taken. Uh, we got rid of segregation of duties, conflicts, which is a major thing for us in auditor, isn't it, to get control over that. And just everything across the board improved. I couldn't believe it the first time I did it. What a great business result it was. And that's, that's what we're about really is getting business results.
That's interesting because that's not what I would typically think of with data analytics, but you were able to get that type of data out of SAP in order to Yeah, so I think, yeah, so I had to go to multiple data sources to do that, but I'm actually always looking for the benefits from data analytics and I'm always looking to do it where I can't quite get a handle on how I'm going to check a control is working.
And if I think, oh, this sounds a bit tenuous and difficult, I go back, well what is the data going to tell us? Let's look, go back to the data. 'cause there is a source of truth somewhere. And at that point then you've gotta be from uh, like the astronaut come up with a quick tool, you know, work, start thinking. You have to start thinking about what your solution is gonna be. We're we're also like the astronauts, aren't we? We're trying to solve problems.
Yes, we're trying to solve problems. Yes We are. Can you provide an example of a pitfall you encountered while using data analytics and how you mitigated it? First time I forgot to copy the database. I wasn't really thinking and I didn't take an initial copy of the first extract. So I played around with the first one. I got made a total mess of it, which can easily happen. And then I actually didn't have anything to go back to.
So I had to go cap in hand back to the IT guys and go, you know, that extract you did for me, can you do it again because I've made a total mess. But then what was worse was the total mess I made somehow got released, somebody else got hold of it and then started using the data from my total mess. And I was going, I was having to make calls going, no, bring that back. That's, that's not the truth. That's my co total mess up of the data. So that was a horrific pitfall.
I have never ever since forgotten to take a copy of the data. So. So that leads into what strategies or controls you recommend to mitigate Number one, take that. Well, number one, first control is be absolutely clear what you're trying to do. So if you're not doing that, um, you're already lost. You're just doing data analytics for the sake of it and hoping to, you're on a fishing trip. But if you're really clear you will then go and get the data you want.
I treat it almost like writing a new application. I'll go and test that. Does that data come back what I expect it to be? Um, and if I've got access to say SAP, I can go back and just check some of the things and just sense check it. If I've written a really particularly tough query, I'll get my colleagues to actually check what I've done. Then we'll go and take a copy instantly, never mess with the original.
And every time we do something right, we write it down what we have done and then we take another copy. And so we end up with multiple copies following the journey of what our process has been. So at any stage somebody can go, you did this. I go, yes, it took it from here to here and we know why we did it. So that's the kind of control I do Everett, you can't be too careful because somewhere down the path it ends up in a fraud sometimes.
And you really want to be sure of your facts before you accuse anybody of anything fraudulent as you probably well know. Yeah, That's very good advice. How can internal auditors effectively communicate the benefits of data analytics to the organization? Stakeholders? Oh nice. Pretty charts money. Um, I know those are simplistic answers, but I'm afraid that's probably what I've had to resort to.
So I was trying to show in the presentation that these charts work and simple things, color coded boxes and things like this. 'cause quite often you're going to an audit committee or an exec team who don't really understand what you're talking about. They might say they do, but very quickly they don't. But if you can go, actually what you're really interested in is we found 20 million. That's a fact. That's a figure that they can get hold of.
So actual facts which are backed up by a hundred percent coverage, they love that because then they can quote that anything that's like, um, traffic lighted, they like that as well. So it's about thinking about what your presentation layer is gonna be as well, which is not a great thing for auditors. We're not particularly creative, inventive, I would include myself in this, but a bit of effort put into the presentation layer as well. Uh, really doesn't go amiss.
Very good. What best practices can you share for integrating data analytics into the internal audit process? Maybe say an internal audit function that doesn't have a savvy IT department or IT audit shop. First of all, don't be scared. Go for it. A lot of these tools, once you start using them, are going the fancy Excel. So that's the the first thing. Uh, somebody actually asked a good question at the end of the session, which was, you know, how do you bake it in?
And we start every audit with a session planning session way before the audit started really. And we will say, does data analytics apply? Is there, is there some data involved we can look at? And if we do, we go, well what question would we want? What, what do we think could be going wrong? Can we actually analyze the data in that? The answer is almost always yes, we just need to plan well in advance and bake it in. So we baked it into every audit. Is there a requirement here for data analytics?
It's just too useful a tool not to use for people who don't really have the skills, I'd actually say invest in yourself. You can all do it. Everybody's mathematically capable in an audit department. I've almost never met anybody who isn't so that everybody can do it. I really have faith in that and once you see the tool, you go, what was I even scared of? Yeah. How do you measure the success or impact of data analytic initiatives in your audits?
Well actually I'd almost go back to the fact that there's some normally a number at the end of it. And that is the beauty of it is especially when you're presenting to A CFO, they get numbers, don't they? So I'd say a large number of them have ended up in monetary findings. If it's a technical audit, I can actually show the number of things we put right. Uh, things like that. Again, we're not actually normally challenged on that fully enough.
It quite often really it's just the fact that we found something. Um, if you present it right, they're happy enough with that. I don't normally have to justify it. Now if I put in a big spend the time, I used ClickView actually quite an expensive tool, but I needed it to show best practice and ACL and idea weren't gonna cut it, but they knew what I was trying to achieve. So I had to explain what the expense is about.
At the end, when you give them a model, they don't even ask what was that all about? You know? So it's, is there a good product at the end Caveats or guidance for using AI for data analytics? Would you like to review some of those? I would. So the, so the very first thing is by its very nature the data analytics you're using is sensitive data. Sometimes it's intellectual property, sometimes it's personal data and it's quite often financial data.
The question you're asking yourself is, where is this data going? When I apply ai, if it's a cloud model, you have now no idea where you've sent this really sensitive data. We can't just be sending out data and going, I don't even know which country it's gone to. Nevermind who servers it on just because I send it to a company like Microsoft. Is it their servers? I don't even know if they've got a third party behind that. Who has access to it? What are they using it for?
So I've got real concerns about doing this. So before I put AI into anything, and I'm not against AI per se, um, I'd be saying know exactly what you're doing with it and where it's going. You need a map of where that data is going. I personally wouldn't use it if it went into the cloud. That's just my personal opinion. Uh, second of all, test the living death out of it. So just like any other application, it's not necessarily right. Just 'cause it's AI doesn't mean it's right.
So test it, put in inputs and outputs and what you expect the answer to be and see if it's correct. But test it like anything else. But I'd really test it to make sure. 'cause if it goes wrong, can, as we've seen in the, in the media, it can go horribly wrong and it can be embarrassing if nothing else. Um, but if people are making decisions on this, it's even worse. Mm-Hmm. Thank you so much for joining us today, Robert. It was an absolute pleasure. Thank you very much for having me.
If you like this podcast, please subscribe and rate us. You can subscribe wherever you get your podcast. You can also catch other episodes on YouTube or@theiia.org. That's TH iia DO org.