For audit managers who feel less sharp since relying more on AI tools
A Question That Wasn't Supposed to Be Hard
Have you noticed your own technical instincts feel a little rusty since you started relying more on AI tools in audit? Read on to see how one audit manager was caught out by that exact gap.
Devi was in the partner's office, watching her finalise the audit report on a mid-sized software company. It was meant to be a quick sign-off conversation, the kind that happens at the very end, almost as a formality. Devi was pleased. The previous team had taken two months to complete the audit. Under her leadership, it had taken just five weeks.
Partner: I want to be sure that the huge increase in software revenue is properly supported with robust documentation and that the work done was sufficient. Can you pull out our audit issues memo and tell me how our team addressed the risk of premature revenue recognition?
Devi had a ready answer. She had read the memo in the file more than once. She started explaining it confidently, the way you explain something you've genuinely understood.
Devi: The licence is treated as distinct from the implementation and support. The customer can use the software on its own without our client doing any further work, so recognising that portion of the revenue upfront is the right call.
Partner: Okay, but distinct according to what, exactly? Did we check that against the contract's actual delivery terms, or is that just the usual answer for this kind of deal? When you say implementation, does that require heavy customisation before the software can actually be used?
Devi opened her mouth to answer, and nothing came out for a second. She realised she didn't know. She knew what the memo said. But she didn't know, with any certainty, whether the reasoning in the memo actually reflected the terms of this customer's contract. For the first time in years, she found herself wondering whether she still knew what deserved her attention.
Partner: Get back to me on that before we file.
What Devi Found When She Went Looking
That evening, Devi pulled the actual contracts and read them herself, properly, for the first time.
The memo in the file started as a junior's first draft, built with an AI tool to speed up the analysis. That part isn't unusual, plenty of people at her firm do this now. Callum, one of her seniors, reviewed it and moved it forward. What Devi found was that the memo's reasoning was well written. Genuinely well written. It used the right terms, in the right order, with the kind of specific, confident language a competent auditor would use. It just didn't quite match what the Meridian contract actually said about when the customer could use the software independently of the implementation work.
Here was the part that unsettled Devi most: nothing about the memo looked wrong. That's because AI doesn't produce something that looks wrong. It produces something that reads exactly like what a competent auditor would write: same tone, same structure, same confident specificity, while being quietly disconnected from the actual contract terms.
Think of it like a student answering an exam question beautifully, in perfect, well-organised paragraphs, using all the right vocabulary, except they've answered a slightly different question to the one that's actually asked. Nothing about the writing gives it away. You only catch it if you go back and check it against the original question, word for word.
That's what happened here. A junior used AI to draft an explanation that sounds exactly like the kind of explanation this situation usually gets. It just hadn't been checked against the actual terms of the contract.
Three People, Three Gaps
Devi's first instinct was to be frustrated with Callum. But sitting with it longer, she realised the problem wasn't really about any one person.
Callum reviewed the junior's first draft, saw a well-reasoned, professional-sounding memo, and moved it forward. He didn't personally verify every sentence back to the contract, because it read like the kind of memo that usually has already been checked. Devi then reviewed Callum's version, and did exactly the same thing, for exactly the same reason. Three people, junior, senior, and manager, each looked at polished, confident work and had no obvious signal telling them to look harder.
This was the part Devi kept coming back to. For years, her way of deciding what to double check hadn't really been about reading every word of every memo. It was instinct, built up over time: a memo that felt rushed got a closer look. A junior who hesitated on a call got followed up on. An explanation that felt thin or generic got questioned. Those signals used to be reliable. A rushed or uncertain piece of work usually looked rushed or uncertain.
That signal doesn't reliably exist anymore. A memo can be coherent, well-structured, and completely fluent, regardless of whether the person, or the tool, behind it actually verified anything. Devi's radar for where to look more closely was trained on a world where confidence and accuracy mostly travelled together. They don't have to anymore, and nobody had ever told her that.
What This Cost Devi
The immediate cost was a late-night re-check of work that should already have been solid, and an uncomfortable follow-up conversation with the partner the next morning.
The quieter cost was that the question from the partner's office hadn't left her.
She had always thought of herself as someone who knew where the risk usually hides in a file, even without reading every line personally. That instinct was part of what made her good at her job, part of what let her manage several engagements at once without drowning in the detail. The doubt that had caught her off guard in that meeting, whether she still knew what deserved her attention, wasn't a passing thought anymore. It sat there as an open question she now had to answer, and she wasn't sure how many other files, across her other engagements, had the same kind of gap sitting quietly inside them, unnoticed for the same reason.
There wasn't a rulebook telling her what to do differently. Standards and firms are still catching up to exactly this problem, and Devi knew that from conversations with peers at other firms. Nobody had handed her a policy that solved this. She was going to have to work one out herself.
What Devi Starts Doing Differently
Devi doesn't try to read every piece of working paper in full going forward. That was never realistic, and it was never actually the job, even before AI existed.
What changes isn't the amount of attention she gives. It's what determines where that attention goes. Instead of letting a memo's tone or confidence guide her, she identifies the areas on every engagement that involve the greatest judgement and the greatest audit risk, and makes it a rule that those analyses and conclusions are challenged against the underlying audit evidence every time, regardless of how polished the explanation looks. Because "it sounds convincing" has stopped being a reason not to check.
She also raises it with her team directly, not as a criticism of Callum or the junior specifically, but as a shift in what "ready for review" means from now on. The message she gives them is simple: if a conclusion involves significant judgement, it must clearly show how that conclusion was reached and what evidence supports it. A well-written memo is no longer a reason to ask fewer questions. Nobody argues with it. A few people look a little uneasy, the same unease Devi sat with herself two nights before.
Devi hasn't lost her instinct for audit work. What she's lost is her trust in the specific signals that instinct used to run on. She is still working out what the new signals should be. For now, the honest answer is that she is building the map as she goes, one engagement at a time, rather than relying on one that no longer fits the terrain.
Your Audit Mentor
Comments
Post a Comment