AI Audit Checklist: 40 Questions to Answer Before an Auditor Asks

Every AI system in production will eventually be audited by someone: a regulator, a customer's procurement team, an insurer, or your own board after an incident. The companies that come through those reviews cleanly are not the ones with the thickest policies. They are the ones who can answer concrete questions with evidence. This checklist is the set of questions we work through with clients, grouped the way an auditor thinks.

Work through it one system at a time. For each question, the honest answers are yes with evidence, yes without evidence, or no. Anything in the second and third buckets is your gap list.

What an AI Audit Covers

An AI audit is a structured review of a system across five areas: the data it learns from and runs on, the model and its behaviour, security and privacy, regulatory fit, and the documentation that proves all of the above. It is not a vendor comparison and not a strategy exercise. The output is a gap list ranked by risk, with owners and dates.

Before the questions, inventory what you actually run. Most audits stall on day one because nobody can list the AI systems in production, including the ones that arrived inside other products or on a team's credit card.

Data Questions

Can you name the source system of every input the model uses?
Do you know the legal basis for processing each personal data field involved?
Is training data versioned, so you can say what any model version learned from?
Are there documented quality checks on input data, and alerts when distributions shift?
Can you reproduce the dataset behind the current production model?
Is there a retention policy for training data, predictions and logs?
Are data access rights in the AI pipeline aligned with your existing access model?
If a customer invoked their GDPR rights, could you locate their data in the pipeline?

Model Questions

Which exact model version is in production, and who approved its deployment?
Is there a documented accuracy baseline, and do you measure against it regularly?
Is performance broken out by relevant subgroups, not just on average?
Do you know the model's failure modes, and are they written down?
Is there a defined threshold where the system hands off to a human?
Can you reconstruct why the model produced a specific output on a specific date?
Is there a monitoring alert for drift, and has it ever fired?
When did you last retrain, and what triggered it?
Is there a rollback path to the previous model version?
Do humans who review model outputs have real authority to override them?

Security and Privacy Questions

Are model endpoints authenticated and rate-limited like any other API?
Have you tested the system against prompt injection and adversarial inputs?
Can the model leak training data or retrieved documents to the wrong user?
Do retrieval systems respect document-level permissions from source systems?
Are secrets and credentials out of prompts, logs and training data?
Do AI agents run on their own identities with scoped permissions, not shared accounts?
Is there an incident response path that covers AI-specific failures?
Are third-party model providers covered by data processing agreements?
Do you know which country your data is processed in, for every provider in the chain?

EU AI Act Fit

Have you classified each system against the Act's risk categories, and documented the reasoning?
For anything plausibly high-risk under Annex III: have you started conformity work against the December 2027 deadline?
Do customer-facing AI systems disclose that they are AI, as Article 50 requires since August 2026?
Is AI-generated content marked machine-readably where the Act requires it?
Have you taken documented measures to support AI literacy among staff working with AI systems, as Article 4 asks?
If you build on a general-purpose model, do you hold the provider's technical documentation?
Does anything you run touch the prohibited practices list, including the additions that apply from December 2026?

Our EU AI Act guidance for 2026 covers the timeline and obligations in depth.

Documentation Questions

Is there a model card for each production system, with an owner and a date?
Does the risk assessment exist as a living document, not a slide from the kickoff?
Are decisions logged with enough detail to answer a regulator's "show me" request within days?
Is there a named owner for each AI system, who knows they own it?
Would the documentation survive the departure of the person who wrote it?
When did anyone last actually test retrieving the evidence, end to end?

Scoring and Prioritising the Gaps

Do not fix the list top to bottom. Score each gap on impact and effort, the same way we prioritise AI use cases: a missing decision log on a credit model outranks a missing model card on an internal search tool. Across the audits we run, most companies find two or three gaps that are genuinely urgent, a dozen that are a sprint of work, and a tail that can wait for the next quarter. Ranking is the point; precision is not.

The same discipline applies before you build anything new: our note on where AI actually moves the needle covers the opportunity side of auditing.

Running It for Real

Forty questions answered honestly takes a few days of one person's time plus access to the teams involved. If you would rather have an outside view, with benchmark data from other audits and a gap list your board can act on, that is what our AI audit service delivers: a structured review, completed in weeks, with evidence standards an external auditor would accept.