Evaluating audit software for an Indian CA firm

Evaluate audit software on five things, in this order: whether it handles your actual client data without weeks of preparation, whether every figure it produces stays linked to the document it came from, whether review and sign-off work the way your firm actually reviews, what happens to client data and where it lives, and what it honestly cannot do. Run the evaluation on your most awkward client file rather than on the vendor's demonstration data, because the two behave nothing alike.

Test on your worst file, not the demonstration

Every product demonstrates well on data prepared for it. The question that matters is what happens to a real Tally export with blank ledger names, three spellings of the same vendor, a company file that was split mid-year, and a purchase register where the tax component is posted separately.

Pick a client with messy data and a known answer — ideally a prior-year engagement that is already complete — and run it through. The right comparison is not whether the software found things; it is whether it found what your team found, plus or minus what, and how long the preparation took.

Indian data and the Indian calendar

• Does it read the export formats your clients actually produce, including Tally XML with masters, and what does it do with rows that are incomplete rather than rejecting the file?
• Does it handle multiple GST registrations for one entity separately, rather than pooling them? Pooling is a correctness problem, not a convenience one.
• Does it understand that reverse charge, imports and composition purchases do not appear in GSTR-2B, or will your team be explaining several thousand false differences?
• Does it group vendor masters by PAN and GSTIN rather than by name?
• Does it work in the Indian financial year, and handle invoice references with financial-year suffixes?
• Does it cover the statutory reporting annexures the engagement requires, or is that still a separate exercise?
• Is it maintained as rules change? GST rules in particular have changed repeatedly, and software encoding a superseded rule is worse than no software.

Evidence and traceability

This is where products differ most and where demonstrations reveal least, because the failure only shows up when a reviewer tries to follow a figure back.

Review, sign-off and the audit trail

• Can a paper be approved with no written conclusion? If yes, the tool will let your file fail the experienced-auditor test.
• Can a paper be approved with an open review point?
• What happens to an approval when the content changes afterwards? An approval that silently survives a rewritten conclusion puts a partner's name against text they never saw.
• Is sign-off attributable to a person rather than a shared login?
• Is the trail append-only, or can entries be edited or deleted?
• Can the trail be exported, and is it readable outside the product?
• Are roles enforced, so that only the person whose responsibility it is can give the final sign-off?

Data, residency and confidentiality

• Where is data stored and processed, and is residency architecturally true or a stated policy?
• Is client data used to train any model, including a sub-processor's? Get it in the contract, not in an email.
• Which sub-processors have access, and are they named and notified when they change?
• What is the retention period after an engagement closes, and can deletion be requested?
• Is access scoped so staff see only the engagements they are assigned to?
• What certifications does the vendor hold, and which do they merely intend to obtain? A vendor who distinguishes the two clearly is telling you something useful about how they answer other questions.
• What happens to your data if the vendor is acquired or ceases trading?

The questions vendors find hardest

These are worth asking precisely because they are uncomfortable. The quality of the answer matters more than its content.

Commercials and exit

• How does pricing scale: per user, per engagement, per entity, or per transaction volume? Per-transaction pricing on an audit tool creates an incentive to test less.
• What happens in the off season, when a seasonal practice has low usage for four months?
• Is pricing published? Opacity is usually a sign of price discrimination rather than complexity.
• Can you export everything, in a form usable without the product, and have you tested that export rather than been told about it?
• What is the notice period, and what happens to files already in the system?

How to run the evaluation

1. 01Write down the three problems you are trying to solveBefore seeing anything. Firms that skip this buy the product with the best demonstration rather than the one that fixes their bottleneck.
2. 02Re-run one completed prior-year engagementWith a known answer to compare against, using your own data in the state it actually arrives.
3. 03Put a senior and an article on it, not a partnerThey are the people who will use it, and they will find the friction a partner never encounters.
4. 04Test the failure paths deliberatelyBad data, a revised conclusion after approval, a replaced document, an export. Products are designed around the happy path and differentiate on the rest.
5. 05Decide what you will measure in the first seasonAnd accept that the first season is slower. A firm expecting a return in year one will abandon the tool in month three.