All Things Internal Audit: Been There, Done That, Seen It All

00:00:02

The Institute of Internal Auditors presents all things internal audit tech in this episode, Robert Finley talks with Lynn Mole about the evolving role of data analytics and AI in internal auditing. Finley shares strategies for overcoming challenges, best practices for integrating data analytics, use cases in the importance of clear communication with stakeholders.

00:00:24

They discussed the differences between basic data analytics and AI, the skills required for each, and how advanced tools can enhance audit processes.

00:00:34

Can you briefly explain the difference between basic data analytics and AI in the context of internal auditing?

00:00:41

Yes, so the basic data analytics is often a response to a specific query. So quite often I'll be asked to go and just say find duplicate invoices or something where it's we've been invoiced a certain amount but the delivery was for less. So it's often an answer to a very specific.

00:00:59

Question with AI coming in, it is almost searching for other issues that we hadn't thought of. So I think the AI is going to expand the horizon if we use it judiciously, and alternatively it might take us down rabbit holes that we didn't really want to go. So it could be less focused. So I think that's really the core difference is the focus on the.

00:01:19

Actual data analytics and what they're trying to.

00:01:21

Achieve would you say that the fundamental skills or knowledge that internal auditors need to have for traditional data analytics are different than with AI with AI? Does it become easier?

00:01:33

Possibly could. So we've just started putting in a tool for our business auditors, which takes away a lot of the pain of extracting the data knowledge of the systems. Technically, it allows them put into almost English terms what queries they want to run and then they can interpret the results much more easily. But it's taking away a level of technical expertise required. Now this is deliberate. We've done this because.

00:01:53

The technical auditors, finding it a bit of a struggle to find the time to do all these technical elements for them.

00:02:00

So the more that we go down the technological route where AI is doing the.

00:02:03

For us, it's going to take away the requirement for some of the skills now. You'll still need the inquiring mind of an auditor to interpret the results, of course, and the methodology will much remain the same. But some of the how can I put the donkey? Work? May well be done.

00:02:17

Gotcha. Can you provide examples of specific audit scenarios where basic analytic tools like Excel or ACL are particularly effective?

00:02:26

Yeah. So actually almost all the odds I've done. It's been done with Excel and ACL or IDR or Arbutus. These tools are really effective, especially when you already know what the question is.

00:02:37

But the key is to do the data extraction correctly in the 1st place, so they're very effective and cheap. If you get your data extraction, get the exact data you want because you know what the question is you're trying to ask. If it's more random, yeah, you might need a more complicated tool. Or if you're putting together much different data sets from more than one system. So I've used things like click view in the past.

00:02:59

Where you need a more complicated system than ACL or idea.

00:03:03

How have more advanced tools like Python scripts or Tableau provided additional benefit over those basic tools like ACL and Excel?

00:03:11

Yeah. So we've used them things like Python quite a lot. And the reason is it gives us flexibility in what we're trying to achieve. So we can actually write a lot more complex scripts. If we start using a programming language ourselves in a way something like ACL is like pre written queries in itself, you can still tailor them and change them. An extraordinarily useful tool.

00:03:32

With Python, it's taking us to another dimension, so we've used it all kinds of technical reviews where we just go. It will be too hard actually. Pass this into ACL and then run the queries we want.

00:03:43

So it's depending on the technology. As a general rule, if I go slight tangent here, I tend to employ people with lots of different skills. I like to recruit people with diverse skills because you don't know what situation you can respond to technically. So in the world of IT audit, you need to use the right tool for the right job. Something like Python can be the right tool.

00:04:03

Where ACL isn't the right tool. Do you need that flexibility in hand?

00:04:07

You also talked about access to the data. Does your organization let you run Python against the databases directly?

00:04:14

Generally, no. So we're not totally reckless. I am allowed to use our native queries in SAP, so I've had to show that I won't bring down the whole database by doing that. So I will do my own data extracts where I can from our ERP. So most of the people on my team were actually quite adept at doing these data queries and we've done them for many years.

00:04:33

If you don't have that expertise, you'll have to ask somebody to do it for you to start running queries. It's almost like you're putting in a new application, so you need somebody to go and test it properly before they would allow you to do it. And that's just too much work. Too dangerous. I don't know that they would trust us to do stuff that wasn't under the control of SAP's controls itself, but it's possible on some data.

00:04:53

Cases generally we've got them to get that, give us the data, then we run.

00:04:56

The Python script what are some criteria you use to determine which tool is best suited for a particular audit task?

00:05:02

Probably complexity of.

00:05:04

The query so I would be looking say the example like ever click view I was trying to merge 3 different databases at once and I just needed a bit and what it occurred to me was not only be easier and a more complex tool oddly enough. But also that would handle the presentation layer back to management better if you have a really simple tool you just get a simple result whereas.

00:05:24

Quick view has enabled me to give it a model to management that they could drill down into. There's a lot more work in.

00:05:31

So fundamentally the criteria is how complex is the initial starting point.

00:05:35

Could you share a particularly challenging audit scenario you faced and how data analytics helped overcome it?

00:05:41

I was facing the sack. If I got it wrong and it was a date, benefits realization review and these are challenging because quite often in a business case management make all kinds of claims what benefits they're going to get from a system. So in this particular instance, it made a a very.

00:05:56

Tenuous claim to the benefits they got by putting an HR system in place. So I went to go and check that now, without the analytics, I couldn't actually possibly prove it one way or another, but I needed all the data, so it's particularly challenging. Get all the data from all the previous databases, pull it all into a data analytics and then review it. But it was worth it because I had 100%.

00:06:16

Coverage of what had actually had.

00:06:17

Happened and at the end I was able to demonstrate the result. Now politically this is where another angle of all for all auditors is, isn't it? Sometimes the results not politically palatable, so they they kicked off, they fought it tooth and nail and ended up in the chief execs office in a very I can only describe it as an extraordinarily tense meeting.

00:06:37

Which you've probably been in many yourself.

00:06:43

What was one of the most surprising insights or benefits you gained from using data analytics in an audit?

00:06:48

I think the very first time we tried to do measure best practice, so this is a very tenuous concept and I was asked basically by the committee. Have we done best practice and it's a very tenuous thing. So I had to try and codify what I thought best practice could be and the results were fantastic. I've got a great model to give to management, but the results actually we cleaned up our access.

00:07:09

Do much more efficient so tasks were taking less time. We could measure the time taken. We got rid of segregation of duties conflicts which is that major thing for us in audit isn't.

00:07:18

To get control over that and justice everything across the board improved, I couldn't believe it the first time I did it. What a great business result it was and that's that's what we're about.

00:07:27

Really.

00:07:27

Is getting business results. That's interesting because that's not what I would typically think of with data analytics. But you were able to get that type of data out of SAP in order to.

00:07:37

Yeah. So I think, yeah. So I had to go to multiple data sources to do that.

00:07:42

But I'm actually always looking for the benefits from data analytics and I was looking to do it where I can't quite get a handle on how I'm going to check a control is working and if I think oh, this sounds a bit tenuous and difficult, I go back. Well, what does the data going to tell us? Let's look go back to the data because there is a source of truth somewhere and at that point, then you've got to be come like the astronaut.

00:08:03

Come up with a quick tool you know, start thinking you have to start thinking about what your solution is going to be. We're we're also like the astronauts, aren't we? We're trying to solve problems. They're trying to solve problem.

00:08:13

Yes, we.

00:08:13

Are can you provide an example of a pitfall you encountered while using data analytics and how you mitigated it?

00:08:19

First time I forgot to copy the database. I wasn't really thinking and I didn't take an initial copy of the first extract. So I played around with the first one I got made a total mess of it, which can easily happen, and then I actually didn't have anything to go back to, so I had to go cap in hand back to the IT guys and go, you know, that extract you did for me, can you do it again? Because I've made a total mess.

00:08:39

But then what was worse was the total mess I made somehow got released. Somebody else got hold of it and then started using the data from my total mess and that I was going. I was going to make calls going.

00:08:50

No, bring that back. That's that's not the truth. That's my total mess up of the data. So that was a horrific pitfall I have never, ever since forgotten to take a copy of the.

00:09:00

Data so so that leads into what strategies or controls you recommend to mitigate?

00:09:06

Number one, take that well, number one, first control is be absolutely clear what you're trying to do. So if you're not doing that, you're already lost. You're just doing data analytics for the sake of it and hoping to, you're on a fishing trip, but if you're really clear, you will then go and get the data you want. I treat it almost like writing a new application. I'm going to test that there's that data come back, what I expect.

00:09:26

To be and if I've got access to say SAP, I can go back and just check some of the things and just sense check it. If I've written a really particularly tough query, I'll get my colleagues to actually check what I've done. Then we'll go and take a copy instantly. Never mess with the original, and every time we do something right, we write it down. What we have done, and then we take another copy. And so we end up with multiple copies following the journey.

00:09:48

Of what our process has been so at any stage somebody can go. You did this. I go. Yes. It took her from here to here. And we know why we did it. So that's the kind of control I do over it. You can't be too careful. Because somewhere down the path it ends up in a fraud sometimes. And you really want to be sure of your.

00:10:03

Before you accuse anybody of anything fraudulent, as you probably well know, yeah.

00:10:08

That's very good advice. How can internal auditors effectively communicate the benefits of data analytics to the organization stakeholders?

00:10:15

Ohh, nice pretty charts money. I know there's a simplistic answers, but I'm afraid that's probably what I've had to resort to. So I was trying to show in the presentation that these charts work and simple things colour coded boxes and things like.

00:10:29

Paul.

00:10:33

Quite often you're going to an audit committee on exact team who don't really understand what you're talking about. They might say they do, but very quickly they don't. But if you can go actually what you're really interested in is we found 20 million, that's a fact. That's a figure that they can get hold of. So actual facts which are backed up by 100% coverage, they love that because they they can.

00:10:53

Quote that anything that's like traffic lighted, they like that as well. So it's about thinking about what your presentation layer is going to be as well, which is not a great thing for auditors, not particularly creative and inventive. I would include myself in this, but a bit of effort put into the presentation layer as well. It really doesn't go amiss.

00:11:11

Very good. What best practices can you share for integrating data analytics into the internal audit process? Maybe say an internal audit function that doesn't have a savvy IT department or IT audit shop?

00:11:24

First of all, don't be scared. Go for it. A lot of these tools, once you start using them again the fancy Excel. So that's the first thing somebody actually asked a good question at the end of the session, which was, you know, how do you bake it in? And we start every audit with their session planning session way before the audit started really. And we will say.

00:11:43

This data analytics apply here is this. Is there some data involved we can look at and if we do what, what question would we want? What what do we think could be going wrong?

00:11:51

Can we actually analyze the data and that the answer is almost always yes. We just need to plan well in advance and bake it in. So we baked it into every audit. Is there a requirement here for data analytics? It's just too useful a tool not to use for people who don't really have the skills I'd actually say invest in yourself. You can all do it. Everybody's mathematically capable in an audit.

00:12:12

And I've almost never met anybody who isn't so that everybody can do it. I really have faith in that. Yeah. And once you see the tool, you go, why was.

00:12:18

I even scared of.

00:12:19

Yeah. How do you measure the success or impact of data analytic initiatives in your audit?

00:12:25

Well, actually, I'd almost go back to the fact that there's some normally a number at the end.

00:12:28

It and that is the beauty of it is, especially when you presenting to a CFO, they get numbers, don't they? So I'd say a large number of them have ended up in monetary findings. If it's a technical audit, I can actually show the number of things we put right, things like that. Again, we're not actually normally challenged on that. Funnily enough, it quite often really it's just the fact that we found.

00:12:49

Something if you present it right, they're happy enough with that. I don't normally have to justify it. Now if I put in a big spend, the time I use click view, it's actually quite an expensive tool, but I needed it to show best practice and ACL an idea when going to cut it.

00:13:04

But they knew what I was trying to achieve. So how to explain what the expense was about at the end when you give them a model they don't even ask what was that all about? You know? So it's, is there a good product at?

00:13:15

The end caveats or guidance for using AI for analytics would you like to review?

00:13:19

Some of those.

00:13:20

I would so the so the very first thing is by its very nature the data analytics you're using is sensitive data. Sometimes it's intellectual property, sometimes it's personal data and it's quite often find.

00:13:32

Initial data, the question you're asking yourself is where is this data going? When I apply AI? If it's a cloud model you have now no idea where you've sent this really sensitive data. We can't just be sending out data and going. I don't even know which country it's gone to, never mind whose servers it on. Just because I send it to a company like Microsoft. Is it their servers? I don't even know if they've got a third party behind that.

00:13:52

Who has access to it? What are they using it for? So I've got real concerns about doing this, so before I put AI into anything and I'm not against AI per say, I'd be saying know exactly what you're doing with it and where it's going. You need a map of where that data is going. I personally wouldn't use it if it went into the cloud. That's just my personal opinion.

00:14:12

Second level test the living death out of it. So just like any other application, it's not necessarily right just because it's AI doesn't mean it's right. So test it, put in inputs and outputs and what you expect the answer to be and see if it's correct to test it like anything else. But I'd really tested to make sure because if it goes wrong and as we've seen in the.

00:14:32

In the media.

00:14:33

It can go horribly wrong and it can be embarrassing if nothing else, but if people are making decisions on this, it's even worse.

00:14:43

Thank you so much for joining us today.

00:14:45

Robert, it was an absolute pleasure. Thank you very much for having.

00:14:49

If you like this podcast, please subscribe and rate US. You can subscribe wherever you get your podcast. You can also catch other episodes on YouTube or at thecia.org. That's theiia.org.