How to audit an AI system that produces analysis

Most AI testing asks whether a system works. An audit of an analytical system asks a harder question: would its output survive an expert reviewer who wants to know where each number came from, what it was compared with, and what would change the answer?

The steps below apply whether the system is a single model with tools, a multi-agent setup or a vendor product you can only see from the outside. They test what the system delivers, not how it was built.

1. Define the tasks and what a right answer looks like

Start from the decisions the system supports. List the tasks it actually performs, such as a monthly forecast, a cost estimate or a program summary. For each one, collect cases where the right answer, and the right report, are known. Past work reviewed by experts is the usual source.

Without known answers, an audit can only describe the system. With them, it can measure it.

2. Compare with baselines

Run the same cases through at least two baselines: a qualified expert, and the simplest method that could reasonably do the job. A system that beats neither adds cost and risk.

Do not assume that a more elaborate setup is better. In one clinical benchmark, single-agent systems outperformed naive multi-agent systems on intensive-care prediction and were better calibrated [1].

3. Remove parts and see what changes

Switch off one component at a time, such as a retrieval step, a reviewing step or an extra agent, and rerun the cases. Each part should earn its place in accuracy or reliability. Parts that add cost and nothing else are candidates for removal. Parts whose removal causes failures are the ones to monitor.

4. Trace every number

Pick figures from the output at random and trace each one back to the calculation and data that produced it. A number that a model wrote, rather than one a calculation produced, is a red flag.

Published system designs show this can be built in. One framework for economic and financial analysis has registered quantitative models generate the numerical results, and predefined tests decide whether intermediate results may be used in the final report [2]. It is a design demonstration, not a tested performance result. An audit checks whether a system behaves that way in practice, whatever its design documents say.

5. Test whether reasonable choices change the answer

Run key cases again under other defensible choices: a different measure, sample or definition. When 150 autonomous AI agents tested the same six hypotheses on the same financial data, their answers spread widely, mainly because they chose different measures [3]. AI peer critiques barely narrowed the spread. If your system’s conclusion flips under a reasonable alternative, decision-makers need to see both answers.

6. Check causal and statistical claims

Find every sentence that says one thing caused, drove or reduced another. Each should state what was compared with what, and the assumption that makes the comparison fair. Check that uncertainty is reported, and that correlations are not presented as effects.

7. Check sources

Open every citation in a sample of outputs. Confirm that the source exists, says what the output claims, and is dated. Fabricated or misread references are a known failure, and they are cheap to catch.

8. Rerun for reproducibility

Run the same inputs again. Material differences between runs mean the system’s answer partly depends on chance. That is not always disqualifying, but it has to be measured and reported.

9. Check the human gates

Identify where people approve assumptions, value judgments and recommendations, and whether that approval is recorded. A gate that nobody can show was used is not a gate.

For systems that take actions, the finance industry is working on the same idea. Version 1.0 of an industry white paper published by the Monetary Authority of Singapore with financial institutions describes checkpoints that verify and record an agent’s proposed actions before they run, plus audit records [4].

10. Measure the claimed benefits

If the system is meant to save time or raise quality, measure that with a comparison, not a survey. In one randomized study, experienced developers expected AI tools to speed them up by 24%. With early-2025 tools they took 19% longer [5].

The output of an audit

A useful audit ends with four things:

• a score against a clear standard, such as our Evidence-Gated AI Checklist, with the evidence behind each score;
• the failures found, ranked by what they would cost the decisions the system supports;
• specific fixes and the sign-off points to put in place;
• the test cases, so the audit can be rerun when the system or its data changes.

Sources

1. Al Jorf, B. and Shamout, F. E. (2026), "AgentRx: A Benchmark Study of LLM Agents for Multimodal Clinical Prediction Tasks", arXiv:2605.10286, accepted at the AHLI Conference on Health, Inference, and Learning 2026. https://arxiv.org/abs/2605.10286 (accessed 2026-10-27)
2. Kato, M. (2026), "AI Economist Agent: An Agentic Framework for Evidence-Based Economic and Financial Analysis with RAG, Knowledge Graphs, and Large Language Models", arXiv:2606.20041. https://arxiv.org/abs/2606.20041 (accessed 2026-10-27)
3. Gao, R. and Xiao, S. C. (2026), "Nonstandard Errors in AI Agents", arXiv:2603.16744. https://arxiv.org/abs/2603.16744 (accessed 2026-10-27)
4. Monetary Authority of Singapore (2026), "Safeguards for Agentic Finance at Runtime (SAFR)", industry white paper v1.0, and media release of 3 July 2026. https://www.mas.gov.sg/publications/monographs-or-information-paper/2026/safeguards-for-agentic-finance-at-runtime (accessed 2026-10-27)
5. Becker, J., Rush, N., Barnes, E. and Rein, D. (2025), "Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity", METR, arXiv:2507.09089. https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/ (accessed 2026-10-27)