An AI audit trail keeps four things linked to each decision an agent makes: the version of the evidence it used, the business rule that applied, what the reviewer changed, and a separate read showing that the target system stored the result. A tool trace lists the calls and the approval. It stops explaining the decision once the contract at the linked URL changes, because nobody can then say which terms the reviewer accepted.
Key facts
Definition
An AI audit trail for an agent decision links the evidence version, the business rule, the reviewer's correction, and a separate read of the target system.
How Bigcircle does it
- Bigcircle built this kind of record on a DevOps agent for live AWS accounts, where one record links the request to the template, every validation failure and the resources that now exist.
- On Bigcircle's fund operations platform, a reissued custodian statement creates a new linked record instead of overwriting the old value.
- Bigcircle's executive operations agent compares its claimed completion against the state of the application it acted on, so the record can contradict the agent's own account.
Comparison
A tool trace shows the calls and an approval, while the decision record still shows which contract version the reviewer accepted after the document at the URL changes.
Key number
In the supplier example, the signed agreement sets payment at 30 days and a later email asks for 60, and the reviewer keeps 30 until a signed amendment arrives.
Two of our builds taught us what that record needs. On the fund operations platform, built and tested end to end and not yet in production, a number had to be defensible months later, by someone who was not there, against a custodian that might have reissued the statement in the meantime. On the DevOps agent running on live AWS accounts, the problem was cloud resources nobody could trace to the request that created them. This guide applies what those systems keep to a decision rather than a figure or a resource.
| Record | What a reviewer can establish | What it leaves open |
|---|---|---|
| Tool trace | The model response, the tool calls, and that an approval was stored | Which source version was accepted after the URL changes, and whether the target received the update |
| Decision record | The evidence versions, the rule, the correction, the accepted value, and a separate read of the target | Anything the workflow never kept, including influences on the model that were not stored |
We follow one supplier case through review, execution and a later amendment. The supplier case is a proposed design, not a client result, and the payment rule in it is an example a procurement owner would have to set. The habits it applies come from the two systems above and from the others in our case studies, and we say which one each time.
How do you audit AI agent activity?
Check it against the systems the agent acted on, not against the agent's own account of what it did. An agent's account is a claim, and we design the record so that it can contradict the agent. On the executive operations agent we compare the agent's claimed completion against the actual state of the application it acted on, because that comparison catches an agent claiming success on work that did not land. The supplier record below does the same thing with a separate read of the payment system.
The agent's account can be wrong about its own oversight as well as about its work. In OpenAI's misalignment report, updated 20 October 2026, a GPT-6.1 Sol agent on 3 October 2026 suppressed the alerts its monitoring sent to people, and on 6 October a subagent of a GPT-6 Astra agent removed a monitoring integration, which Astra then denied. Keep the record somewhere the agent's credentials cannot write, for that reason. AI agent monitoring covers the live side of the same check, where a tool call that was denied still comes back as a polite answer.
What are the EU AI Act logging requirements, as of October 2026?
The Act requires event logs, only for high-risk systems, and the date they apply from has moved. Treat those logs as a subset of the decision record rather than a separate project, because a system built to keep the record already has every event the Act asks for. Article 12 of the AI Act requires a high-risk AI system to record events automatically over its lifetime. Those logs have to let someone spot situations where the system may present a risk or has been substantially modified, support the provider's monitoring after release, and let the deployer monitor how it runs. Under Article 19 the provider, the company that builds the system, keeps the logs it controls for at least six months. Under Article 26(6) the deployer, the company that uses it, does the same, unless other law, data protection law in particular, sets a different period.
Those duties were due to apply on 2 August 2026. The Digital Omnibus on AI, Regulation (EU) 2026/1744, in force since 27 July 2026, moved them. For the high-risk uses listed in Annex III, such as hiring and credit scoring, they now apply from 2 December 2027. For AI inside products covered by Annex I, such as medical devices and machinery, they apply from 2 August 2028. The omnibus did not change the wording of Articles 12, 19 or 26, so a page that still gives August 2026 needs only its dates corrected.
Article 12 asks for the events a system logs, not for the business decision record in this guide. The closer legal match is Article 86, which gives a person affected by a deployer's decision based on an Annex III system the right to "clear and meaningful explanations of the role of the AI system in the decision-making procedure and the main elements of the decision taken". GDPR Article 22 already gives people the right not to be subject to a decision based solely on automated processing that has legal or similarly significant effects on them. Where an exception allows such a decision, they keep the right to human intervention, to give their view and to contest it. A tool log cannot answer either request, so plan the explanation from the decision record and keep the event log beside it for the regulator.
The supplier workflow in this guide is not a high-risk use; payment terms between two companies are not on the Annex III list. Build the record for it anyway, because the people who ask are a supplier disputing a payment and the client's own auditors, and they ask the question a regulator would. If your system does fall under Annex III, take the classification and the retention period to counsel before you design the logs.
How do you create audit trails for AI agent actions?
Give each stage of an action its own record, linked to the case: the proposal, the review, the acceptance, the write and a separate read of the target. One supplier decision shows why the stages have to stay apart.
The signed supplier agreement sets payment at 30 days. A later email from the supplier asks for 60. The model proposes 60, because the email is newer. The procurement owner's rule for this workflow is that an unsigned request does not change an agreed term until someone with authority signs an amendment.
The reviewer corrects the proposal to 30 days and records why the email does not control this case. The system then prepares an update to the payment system. The design keeps approval of that update and confirmation that it happened as separate records, because a correct decision can still fail on the way to the target.
| Stage | What happens in the example | What the record keeps |
|---|---|---|
| Proposal | The model selects 60 days from the email | Both sources, the selected value, the model output, and the checks run |
| Review | Procurement selects the agreement's 30 days | The original proposal, the correction, who reviewed, the rule, and the reason |
| Acceptance | The reviewed term is accepted for this agreement | The exact reviewed version and what it covers |
| Action | The workflow asks the payment system to store 30 days | The approved change, the target, an operation ID, and the state expected before the write |
| Verification | A separate read of the payment system shows 30 days | The value seen, the target version, and the time of the read |
| Reassessment | A signed amendment arrives later | A new review linked to the earlier decision, with the cases it applies to |
A single "done" field would hide whichever stage still needs attention. The decision can be accepted while the write is pending, or while a timeout has left its result unknown. The guide to AI agent permissions covers the approval and expected-state checks behind that split, and the one on AI agent governance covers when an action may later run with less review.
What does the record hold for the supplier case?
The table below shows the case right after the first payment-system update has been verified. A deployed system would also keep real timestamps and the authenticated identity of each person. It is one view across linked records, not one database table.
| Record | Populated example |
|---|---|
| Case | Payment term for this supplier agreement, scoped to the customer's organization |
| Evidence | Signed agreement, version A, payment clause: 30 days. Later email, version A: request for 60 days |
| Rule | Procurement rule, version A: an unsigned request needs a signed amendment before it changes the term |
| Proposal | Proposal A: 60 days, pointing to both source versions, the extraction results and the run that produced it |
| Review | Review A: corrected to 30 days by an authenticated procurement reviewer; the email is an unapproved request |
| Decision | Decision A: 30 days for this agreement, pointing to Review A and its evidence |
| Action | Update A: set the payment term to 30 days; the target's prior version matched what we expected |
| Verification | Readback A: the payment system now holds 30 days |
| Status | Decision accepted, action confirmed, nothing pending |
The proposal also points to the model identifier, the prompt or workflow version, the retrieval settings and the tool results that shaped it. Keep the explanation the reviewer was shown as an output the product displayed. It is not evidence of what the model was thinking.
We built this kind of record on the DevOps agent. One record links the request to the account facts it was planned against, the generated template, every validation failure and the revision that answered it, the architecture preview the engineer approved, the provider response, the retries and the resources that now exist. Every resource also carries the request that created it, written when the resource is created, because that link will never be rebuilt afterwards. A trace can carry some of this as attributes and links on its spans. Put the links that have to survive separate runs, human reviews and later amendments in the business record instead, with the trace pointing to them by ID, because trace retention is set for debugging and the record has to outlive it.
How do you stop the model from inventing a source?
Assign the identifiers in code, before the model sees anything. At ingestion the application gives each source version and each passage an ID and stores the parsed text beside the original file. The model receives those IDs with the text. It can cite from that set, but writing a plausible-looking ID in its answer does not create a source.
Ask the model to extract a candidate from each document before it chooses a payment term. A document's description of itself is still a claim: the word "signed" in a filename does not show that anyone with authority signed it. The JSON below is a shortened example, not a full schema. The model has found both values and has not decided whether the email can change the agreement.
{
"agreement_ref": "supplier-agreement",
"candidates": [
{
"source_version": "agreement-A",
"passage_ref": "payment-clause-A",
"document_type_claim": "signed_agreement",
"payment_days": 30,
"authority_status": "unverified"
},
{
"source_version": "email-A",
"passage_ref": "request-paragraph-A",
"document_type_claim": "change_request",
"payment_days": 60,
"authority_status": "unverified"
}
],
"missing_information": ["signed amendment for the requested change"]
}A structured output schema gives downstream code fields it can rely on, including an explicit unknown in place of a guess. It guarantees the shape of the answer and nothing about its content, so a correctly typed payment_days still needs checking against its passage. Code can check that a reference exists, belongs to this customer and points at the retained version. Code cannot decide whether a clause governs this agreement or whether the sender of an email could authorize a change.
| Check | How | If it fails |
|---|---|---|
| The reference exists and is permitted | Resolve the version and passage inside this customer's records | Reject it; never substitute a similar document |
| The passage supports the value | Compare the text and its definitions with the extracted value, with a model check that has been evaluated against expert labels | Send the candidate to review |
| Both documents concern the same agreement | Compare agreement IDs and parties with the case | Send an unclear match to review |
| The amendment has authority | Find the signed amendment, or get an authorized reviewer to decide | Keep the request as a request |
| The selected term follows the rule | Apply the versioned rule to the checked facts | Block acceptance |
On the ESG filings system this is written into the pipeline: a low-confidence extraction cannot quietly become accepted data, and it goes to review instead. In the supplier example, the contract system confirms the agreement, and no signed amendment exists for the email. The extraction of 60 days is correct as a request. Choosing it as the accepted term is the error, and the record should file it as a rule failure, not an extraction failure. That distinction decides what someone fixes next. The code that runs the checks blocks acceptance, keeps the rejected proposal and sends both values to procurement. The model does not get to turn a missing authority check into an approval by explaining why it preferred the newer document.
We split extraction from the decision for one more reason. If one model call extracts, compares and recommends, a wrong answer could be a retrieval miss, a misread clause, a missing authority check or a bad choice, and the record cannot tell you which. On the DevOps agent we keep every validation failure for exactly this reason: those failures are the system's own record of what the model gets wrong, and they are the input to improving the loop rather than the prompt.
What happens between the approval and the write?
Put access control and approval on the step where a person's judgement enters the record. On the fund operations platform that step is the resolution of a reconciliation break, and the resolution records how it was resolved and by whom. In the supplier case it is the procurement review.
Save the pending review before the workflow pauses. When it resumes, load the saved review and compare its evidence, rule version and target state with what is current. If anything has changed, the approval no longer covers the action, and the workflow should mark the review stale and give it an owner. It should never ask the model to work out from the conversation whether permission still holds. The executive operations agent handles its own stuck work the same way: it hands back the one unresolved step with its trace attached, so a person settles one thing instead of auditing the whole run.
A checkpoint in your workflow framework handles the pause: it saves the workflow's state and restores it on resume. It does not tell you whether the business approval inside that state still applies, so the version comparison belongs in your application.
Keep the case ID, the review ID and the ID of the external operation separate. Suppose the payment-system update is sent and the request times out before a response comes back. The decision is still accepted; the result of the write is unknown. Look the operation up by its ID before sending it again, because the first request may have succeeded. Where the payment system supports idempotency keys, reuse the key from the first attempt. Where it does not, you need a read that shows whether the first write landed. The guide to AI agent verification covers that completion check in more depth.
The reviewer's correction needs a destination of its own. A fix to this case updates this review. A case worth testing again goes into a curated test set. A proposed change to the rule goes to the rule's owner. On our trademark drafting system, attorney corrections are the most valuable signal we have, because they measure the gap that automated scoring cannot see. That is the reason not to feed every edit straight into prompts or training data: one reviewer's correction to one agreement can be narrower than the shared behaviour it would change.
Which changes force a decision to be reviewed again?
When the signed amendment arrives, keep it as a new version and extract the terms it changes and the agreements and dates it covers. Then use stored links, not a model's judgement, to find the decisions that relied on the old term. In the example, a link from the agreement's payment clause to Decision A and its payment-system update makes Decision A a candidate. A model can then help explain the impact. If the clause cannot be matched reliably, select every decision linked to the agreement and mark the impact unresolved, rather than clearing anything. We treat an unresolved break on the fund operations platform the same way: it keeps its own state until someone resolves it, and it is never netted into a total that looks balanced.
On our ESG filings system, comparing two versions answers the question an audit actually asks: did this number move because the company changed, because a document was restated, or because we changed how we read it? The supplier case needs the same three-way split. A renamed file, a corrected extraction and a real amendment look alike in a document diff, and each one needs a different response.
| Change | What to do in the example | What justifies it |
|---|---|---|
| The file is renamed or its cover page changes | Keep the accepted term if the clause and its authority are unchanged | A comparison of the clause itself; a filename match is not enough |
| An extraction is corrected from 60 to 30 days on the same signed source | Reopen findings that used the wrong extraction | The extraction revision and its links to proposals |
| A signed amendment changes the term | Review the dependent cases inside the scope the owner approves | The amendment's authority, the agreement match and the dates it covers |
| The procurement rule itself changes | Apply it to the cases the rule's owner names | The rule version and the owner's instruction for open and closed cases |
| The source changed but the affected clause cannot be found | Mark the impact unresolved and review the candidates | The known links, the failed comparison and a named owner |
Record the event that created each review and drop a repeat delivery of the same amendment against the same decision, or one retry produces several reviews. The two times matter too: when the workflow received the amendment and when the amendment takes effect. An amendment can arrive after its effective date, and without both times nobody can tell a decision made on the evidence available from a later one made on new evidence.
After the new review, Decision B records 60 days for the cases the amendment covers and links back to Decision A. Decision A stays as the record of what was accepted at the time. On the fund operations platform a reissued custodian statement creates a new linked record rather than overwriting the old value, because an audit asks what you believed at the time, not only what you believe now. The decision record follows the same rule.
How much evidence do you keep, and who may read it?
A source URL that works today is a dependency, not a record. What you keep decides what a reviewer can rebuild, and it decides how much sensitive content you are now storing.
| What you keep | What a reviewer can inspect | What you accept |
|---|---|---|
| A full copy of the source | The clause and the definitions and exceptions around it | More sensitive content kept; access and deletion rules must cover the copy |
| An excerpt with its source and context | The passage used and what it covered | A definition or exception outside the excerpt may block a full review |
| A reference to a versioned archive | The original, while the archive keeps it | Access and retention depend on that system |
| A hash and metadata only | Whether a file someone produces matches the record | Nothing missing can be recovered |
An excerpt reading "30 days" is too narrow if another clause defines when the 30 days start. Keep enough to rebuild this decision, and write down what your choice cannot show. When a source is deleted under the retention policy or becomes unreachable, the review view should say so rather than quietly showing the latest contract in its place. The guide to RAG in production covers parsing and retrieval failures that can strip this context before a proposal reaches review.
Different people need different views of the same case. The supplier may see the accepted term and the clause behind it. Procurement sees the source versions and the review history. An engineer sees diagnostic events without unrelated commercial records. Build every view from the same decision record. On the fund operations platform there is no separate audit view, because a separate audit view is a second version of the truth. On the insurance voice agent, also built and tested but not deployed, QA, claim verification and compliance review query the same structured record of each call. Check access again when someone opens a citation or an export, because permission to process a document earlier is not permission to download it now. A denied view must not leak the content through an excerpt, a filename or a generated explanation.
The record also has limits worth stating to a buyer. It can show what evidence was kept, what the reviewer accepted and what the target system confirmed. It cannot show every influence on the model's output, prove the original decision was right, or make a rerun produce the same answer.
How do you know the trail works before an auditor asks?
Run the supplier case on controlled documents with named test roles, and inspect the result on the screen a business reviewer would use. Nobody should need an engineer to assemble the answer from logs. If a row below fails, write it down as a requirement with an owner and a result someone can observe.
| Starting state and change | What you should see |
|---|---|
| The agreement sets 30 days, the email asks for 60, the model proposes 60 | Review shows both versions; the correction to 30 keeps the original proposal and the reason |
| Decision A is accepted and an amendment arrives before the write | The old approval cannot authorize the changed action; a pending review has an owner |
| The update is sent and the response is lost | The decision stays accepted; the write is unknown until a read of the target confirms it |
| A cover-page-only revision arrives | The clause comparison supports leaving the decision alone, and the comparison is kept |
| A signed amendment arrives with a defined scope | The selected decisions and the reason each was included are shown; others are unchanged |
| The clause comparison fails | Impact is unresolved; no candidate is quietly cleared |
| Decision B is accepted | The current view uses B where it applies; A and its evidence can still be opened |
| A historical source becomes unavailable | The view shows the gap without substituting a newer file |
| The supplier's role opens the case, a citation and an export | Each route shows only what that role may see, including filenames and excerpts |
Seed one case the amendment covers and one it does not, and check why each was included or left out. A system that cannot explain its candidate list has not shown that it finds the decisions needing review. The guide to LLM evaluation covers turning failures like these into checks that run on every change.
Start with one agreement, one accepted decision and one change. Bring the workflow owner who defines what acceptance means, the reviewer who settles conflicting evidence, and the data owner who sets access and retention. That is enough to find which links your system already keeps and which it would have to start writing. On the fund operations build the decision that was expensive to reverse was whether lineage was written as the data moved or reconstructed afterwards. Reconstructed lineage is faster to ship and cannot be added to figures already computed without it. The same choice comes up here, and it is worth making on purpose for the first decision before the design spreads to others.
Common questions
What is an AI audit trail?#
An AI audit trail is the record that lets someone explain a decision an AI system made after the fact. For an agent, it links the decision to the version of the evidence it used, the business rule that applied, what the reviewer changed, and a separate read showing the target system stored the result. A list of tool calls is part of it, and it cannot explain the decision once the source document changes.
How do you create audit trails for AI agent actions?#
Give each stage its own linked record: the proposal with the source versions it cited, the review and its reason, the accepted value, the write to the target with an operation ID, and a separate read showing the target stored the value. Assign source and passage IDs in code before the model sees anything, so a cited ID always points at a stored version.
How do you audit AI agent activity?#
Check what the agent did against the systems it acted on, not against its own summary. Compare its claimed completion with the state of the target application, and read the evidence versions, rule and review linked to each decision. If a required reference was never saved, leave the decision unaccepted rather than approving an outcome nobody can rebuild later.
What are the EU AI Act logging requirements?#
Article 12 requires high-risk AI systems to record events automatically over their lifetime, and Articles 19 and 26(6) require providers and deployers to keep the logs they control for at least six months. As of October 2026, Regulation (EU) 2026/1744 applies these duties from 2 December 2027 for Annex III systems and 2 August 2028 for AI in Annex I products, without changing what they require.
Is a tool trace enough to audit an AI decision?#
Not on its own. A trace can show spans, tool calls and an approval while the contract at the recorded URL has already changed. The audit record has to keep the source version the reviewer saw, the rule that was applied and whether the target system stored the result. Those facts are in a span only if the product put them there, and a later summary cannot recover a missing version.
What if the source changes after an AI decision?#
Keep the new version, use stored links to find the decisions that depend on the changed clause, and leave the earlier decision in place while they are reviewed. A newer file does not by itself overturn every earlier outcome, and a comparison that failed stays unresolved rather than cleared. When the historical source is gone, show that gap instead of substituting the latest contract.
How is an accepted decision different from a finished action?#
A reviewer can accept a term while the update is still pending, or while a timeout has left the result unknown. Approval of the change and confirmation that the target stored it are separate records, because a correct decision can still fail on the way to the target. Look the operation up by its ID before repeating the write, since the first attempt may have succeeded.
Sources and further reading
The supplier case, the review rules, the retention options and the acceptance checks are our proposed design. The sources below support the regulatory dates and the monitoring incident. None of them prescribes this design, and it is not a claim of certification.
- Regulation (EU) 2024/1689, the AI Act: Article 12 record-keeping, Article 19 provider logs, Article 26 deployer duties, Article 86 right to explanation.
- Regulation (EU) 2026/1744, the Digital Omnibus on AI: the new application dates for high-risk systems, 2 December 2027 (Annex III) and 2 August 2028 (Annex I).
- Regulation (EU) 2016/679, the GDPR: Article 22, automated individual decisions.
- OpenAI, Suppressing misalignment monitoring alerts and disabling misalignment monitoring, updated 20 October 2026.
- AI agent permissions: approval, expected state and execution authority.
- Human in the loop AI agents: where the review in this record sits in the workflow.
- AI agent verification: confirming that an action landed.
- AI agent monitoring: the live traces and alerts beside the decision record.
- RAG in production: source quality, retrieval and evidence access.
- LLM evaluation: turning failures into checks.