Pick a date six months ago and one AI system your organization runs in production. Could you produce, today, what that system was asked on that date and what it answered? Could you say which model answered, who was using it, and which controls were in force?
I ask this in most of my conversations with compliance, risk and engineering leaders in financial services, insurance and healthcare, and the answers vary widely. We turned the question into a short survey. It takes about three minutes, and we'll publish the results in aggregate.
01 · The examiner's question
Why reconstruction is what gets asked
A reviewer picks a past period, samples activity from it, and asks the organization to show what happened and how it was supervised. For a human process the evidence is the email that went out, the approval, the closed ticket. For an AI system it's the prompt, the response, the model that produced it, and what your controls decided at the time.
Examiners give more weight to records made at the time than to ones assembled after the request. We've written about two questions that follow from that: whether you can show a guardrail actually ran, and how a reviewer can check the record without trusting the vendor.
02 · What we hear
"Our AI vendor has the logs"
The answer I hear most often is some version of "OpenAI has the logs," or Bedrock, or whichever platform the team builds on. The assumption is that the provider records every call, and that compliance can pull what it needs once the examiner asks.
In practice there are two problems. First, the logs may not exist. On Amazon Bedrock, model invocation logging stays off until someone turns it on, and it only covers calls made through the runtime endpoint. OpenAI keeps API inputs and outputs for up to 30 days for its own abuse monitoring. Neither default leaves you a record from six months ago.
Second, where the logs do exist, they don't cover what the examiner asks for. A provider's log can show that a call happened, which model handled it, and what text went back and forth. It can't show that your controls worked: that a client's account number was redacted before the prompt left your network, or that a message carrying material nonpublic information was stopped. It also can't tell you who asked. The provider sees your API key or your application's service role, so every adviser, underwriter and nurse behind that application shows up as the same caller. It can't say which adviser drafted a given client email.
The gap is larger for agents, which read and write your business systems: the CRM, the claims platform, the document store. The provider sees the tool calls the model asked for and whatever your application passed back into the next prompt. The records those tools opened and the changes they made stay in your systems, outside the provider's logs. FINRA's 2026 oversight report makes a related point: multi-step agent reasoning can make outcomes hard to trace or explain, which complicates auditability.
Only if someone turned logging on and kept it. Some platforms log nothing by default; others keep a short-lived copy for their own abuse monitoring.
Usually returned with each call.
The provider sees your API key or your application's service role. Every person behind it looks like the same caller.
A redaction or block that happens on your side leaves no trace in the provider's log.
The provider sees the tool calls the model requested. The records the tools then read or changed live in your systems.
03 · The rules
What the rules already ask for
Existing recordkeeping and audit rules already cover AI activity.
Broker-dealers
Communications relating to the business must be kept for at least three years, the first two in an easily accessible place. A client message drafted by an AI tool and sent by the broker-dealer is one of them. Our reference on SEC recordkeeping for AI covers the periods for broker-dealers and advisers.
SEC Rule 17a-4(b)(4)
FINRA member firms
Generative AI can implicate the rules on supervision, communications, recordkeeping and fair dealing. The practices the report describes include keeping prompt and output logs and tracking which model version was used, and when. Our summary has the detail.
FINRA 2026 oversight report
Healthcare
The HIPAA audit controls standard requires mechanisms that record and examine activity in systems that contain or use electronic protected health information. An AI system that reads or writes patient data is one of those systems.
45 CFR 164.312(b)
Banks
SR 26-2 leaves generative and agentic AI outside its scope and says a bank's own risk management practices should guide the controls for them. Whatever approach a bank takes, the evidence that it operates comes from records of what the systems did.
SR 26-2
04 · Where it breaks
Where reconstruction tends to break
Producing the record for one date depends on six things. If any one is missing, the record can't be produced.
Most production prompts are assembled at run time from a template, the user's input and retrieved documents, so the record needs the assembled prompt the model received. Ownership should be settled in advance: a named person who knows where the records are and has access to produce them.
05 · The survey
Take the survey
The survey has 12 short questions. It asks about your role, sector and size, which AI uses you run in production, whether prompts and responses are kept and for how long, and whether you could produce them for a given date. It also asks who would get the request, how long it would take, whether an examiner or auditor has already asked, and how confident you are. The last question is an optional comment.
We don't ask for your name, and email is optional. Leave one if you'd like the aggregate results. We publish results in aggregate only and never share individual responses, and our Privacy Policy explains how submissions are handled.
If you've already been through an exam where this came up, I'd especially like to hear how it went. Take the survey.