When an AI vendor hands over its audit log, the reviewer receives the vendor's account of what its system did. A screenshot or an exported spreadsheet shows what the log said on the day of the export, and nothing in the file shows whether a record was edited before then. Compliance officers, internal auditors and a bank's third-party risk reviewers need a way to check the records that does not depend on the party that produced them.
01 · The problem
An exported log is still the vendor's word
Take a bank reviewing an AI product that drafts customer messages. The vendor sends a spreadsheet of last quarter's AI activity and a screenshot of its admin console. Both look complete. Row 4,812 may well say what it said in March, and the file has no way to show it: anyone with access to the vendor's system could have changed that row before the export ran.
The same gap opens inside an organization. A compliance officer who receives AI records from the engineering team, or an internal auditor sampling them, is relying on whoever ran the export. When an examiner asks how anyone knows the records are unaltered, the answer has to come from the records themselves.
02 · What the rules expect
Recordkeeping rules expect records that stay as written
Broker-dealers have worked to a specific standard since 3 May 2023. Under SEC Rule 17a-4(f), an electronic recordkeeping system either keeps a complete time-stamped audit trail of every modification and deletion, detailed enough to re-create the original record, or preserves records in a non-rewriteable, non-erasable format. The system must also verify automatically that its processes for storing and retaining records are complete and accurate. Investment advisers keeping records electronically need procedures that reasonably safeguard them from loss, alteration or destruction under Rule 204-2(g). Our reference entry on SEC 17a-4 and AI recordkeeping covers which AI outputs and prompts become records in the first place.
FINRA's rules apply to generative AI as they apply to any other technology, a point Regulatory Notice 24-09 restated in June 2024. Rule 4511 requires member firms to preserve their books and records in a format and media that complies with Rule 17a-4. An AI-drafted message a member firm sends about its business is a communication, held to that standard like an email a person typed.
Banks work under supervisory guidance. SR 26-2, the model risk guidance issued in April 2026, describes documentation that supports continuity of operations and an internal audit function that evaluates whether model risk management is rigorous and effective. Both depend on records a reviewer can rely on. The guidance keeps vendor products inside those principles even when the bank never receives the vendor's code or data. Generative and agentic AI sit outside its scope by footnote 3, which leaves those systems to the bank's own risk management and governance.
The vendor relationship itself falls under the 2023 interagency guidance on third-party relationships, whatever model sits inside the product. A bank's use of a third party, in the guidance's words, "does not diminish its responsibility" to meet its requirements as if it did the work in-house. Among the contract terms it lists is the bank's right to audit the vendor, or to engage an independent party to do so. What a bank's third-party risk review asks an AI vendor sets out the evidence behind each question.
03 · Software reviewers
The first reviewer may be software
Evidence review is moving into software. Internal audit teams run analytics over whole populations of records, and some use AI assistants for a first read of an evidence file before a person opens it. To that software, a screenshot is an image and a spreadsheet is rows of values, and neither carries anything it can test.
Evidence that checks itself gives software a computable answer: run the check, get a pass or a fail, and on a fail, get the record where it failed. A person still decides what the result means for the review. Whether the records are intact becomes a question of arithmetic, answered the same way whoever runs it.
04 · Independent checking
What independent checking means
Independent checking rests on four pieces. Once the files are handed over, none of them needs the vendor's cooperation.
- A tamper-evident chain. Each record is sealed with a cryptographic fingerprint, a hash, that also covers the fingerprint of the record before it. Change one character in a sealed record, or take a record out of the middle, and the chain breaks at that point.
- An export of the records. The reviewer receives the records themselves for the period under review, as files they keep and can check again later.
- A verifier the reviewer runs. The check runs on the reviewer's own machine, with no connection to any vendor system. It recomputes every fingerprint and every link. When its source code is published, the reviewer, or anyone they trust, can read exactly what it checks.
- Independent timestamps. At intervals, a fingerprint of the chain goes to an independent timestamp authority, which signs it together with the time. The signed token shows that the records it covers existed by that time and have not changed since.
From record to result
The check runs on the reviewer’s side of the handoff
01 · Written
An AI interaction is written down as a record.
02 · Sealed
The record gets a fingerprint that also covers the record before it. A later edit breaks the link.
03 · Exported
The records for the period go to the reviewer as files, with the timestamps that cover them.
04 · Verified offline
A verifier whose code the reviewer can read recomputes every fingerprint and link, and checks each timestamp.
05 · Result
Intact. Every record matches its seal.
Broken. The check names the record where the chain fails.
05 · Timestamps
What a timestamp adds
A chain on its own shows internal consistency. Whoever holds the records could, in principle, rewrite all of them and seal a fresh chain that verifies just as cleanly. A timestamp closes that gap for everything it covers. The authority signed a fingerprint of the chain at a stated time, and a rebuilt chain produces a different fingerprint that will not match the signed one.
The standard for this is RFC 3161, which describes the authority's role as establishing evidence that a piece of data existed before a particular time. The authority receives only the fingerprint and never sees the records. A chain anchored with two authorities that operate independently of each other does not rest on either one alone.
One chain, anchored at intervals
The most recent records in an export will usually sit after the latest anchor. A reviewer who needs them covered can ask for the export again once the next anchor has been taken, and confirm that the earlier anchors still match.
06 · In a review
Putting the checks into a review
For a compliance officer, an internal auditor or a bank's third-party risk reviewer, the request to the vendor gets more specific. Ask for:
- the records for the review period, as an export you keep;
- the timestamp tokens that cover them, and the name of each timestamp authority;
- a verifier you can run without the vendor, with its source code published;
For a bank, the third-party guidance already gives these requests a place: the contract's audit and reporting terms, and ongoing monitoring after signing. For an AI vendor's head of trust, the same list is the evidence to have ready before the first questionnaire arrives.
To try the checks yourself, request our sample examination evidence. It holds a simulated audit chain from a demonstration environment and the open-source verifier, also published at github.com/Meilynx/meilynx-verify, which runs offline on your own machine. Its instructions include a test worth running: edit a record, run the check again and watch it fail.
Evidence that answers for itself
Every reviewer of an AI audit trail eventually asks how anyone knows these are the records that were written. With a chain, an export, a verifier they run themselves and independent timestamps, the reviewer answers that question from the evidence, and the vendor's part ends at the handoff.
Frequently asked
What does a timestamp from a timestamp authority prove about an audit log?
That the records it covers existed by the time the token states and have not changed since.
Does SEC Rule 17a-4 require WORM storage for AI records?
Since 3 May 2023, a broker-dealer's electronic recordkeeping system can meet Rule 17a-4(f) in one of two ways: preserve records in a non-rewriteable, non-erasable format, or keep a complete time-stamped audit trail of every modification and deletion that allows the original record to be re-created. An AI output that is a required record goes into a system that meets one of them.
Why should an audit trail verifier run offline?
A check that runs offline depends only on the exported files and the verifier's code. No vendor system takes part in the result, and the reviewer can repeat the check later on the same files and get the same answer.