An AI agent that passed review last quarter can behave differently today. A routine update or a vendor change is enough, and nobody is notified. The approval on file still describes the old agent.
01 · Two forms
The agent you approved may not be the agent running today
Practitioners call this agent drift. It shows up in two forms:
- Permission drift. An agent's scope changes and it can start accessing new tools.
- Prompt drift. An agent's instructions change and it can start behaving differently.
02 · Field input
What we heard from banks, insurers and AI vendors
We have been talking with practitioners and auditors at banks, insurers and the vendors who sell into them. Three themes came up again and again.
Reviewers want to know what tools can be accessed by the agent.
When the bank's audit team samples an agent's work, it wants to know which tools were available for the agent to use.
Auditor, large US bank
Limits only help while they hold.
A regional bank limits each agent to the tools it needs. One extra tool added later undoes that.
Practitioner, regional bank
Prompt approvals go stale.
A large US bank approves agents on fixed prompts only and does not allow free-form prompting yet. Once the prompt changes, the approval no longer matches what is running.
Auditor, large US bank
03 · Examples
Drift in practice
An unreviewed prompt edit. A developer improves the agent's standing instructions (the system prompt) and ships it. Prompt edits rarely go through the same change review as code, so compliance never sees it.
A guardrail disappears. Someone accidentally deletes a line like "Do not provide investment advice". Content checks scan the data going in and out, and a deleted instruction leaves nothing in that data to flag. The problem only shows up later, when the agent does something it used to refuse.
A tool quietly gains power. An agent approved to look up account balances calls a tool that someone later extends to move money. The agent's name, its prompt and most of its answers stay the same, so nothing in a routine output review points to the change.
A vendor changes its AI. A fintech, insurtech or healthtech vendor builds an AI model into its product. Its customer needs proof the instructions have not changed since the last review. The vendor does not want to hand over the prompt, because it is their intellectual property.
When an auditor asks what changed and when, someone should be able to pull up a record written at the moment it changed.
04 · Drift type 1
Permission drift
Permission drift starts when an agent gets a tool it was never approved to use, or when an approved tool is changed. The agent's answers can look the same, so reviewing outputs will not catch it. Resolving it takes three things:
- An approved list of tools. Record exactly which tools the agent may use and what each one is allowed to do.
- A check on every request. Compare the tools the agent can reach now to the approved list, so a change is caught right away.
- A human review. A person looks at any new or changed tool and decides whether to approve it before it is trusted.
05 · Drift type 2
Prompt drift
Prompt drift happens when the standing instructions behind an agent change after approval. An edit meant to improve tone can also drop a safety instruction, and the agent keeps running as if nothing happened. Resolving it takes five things:
- A record of the approved prompt. Keep the approved version, so every later version has something to be compared against.
- A check on every request. Compare the live prompt to the approved record each time, so a change is caught right away.
- The exact wording of what changed. Show the reviewer the lines that were added or removed, so they can judge the change itself.
- A human review. A person reads the change and decides whether to approve the new prompt or investigate.
- Protected prompt content. The prompt text should not leave the organization's own environment, and wherever it is stored it should be secured.
06 · Meilynx
How we catch drift at Meilynx
Keeping an approval true
An approval describes one agent at one point in time. For vendors selling agents into regulated firms, and for the institutions running them, drift checks answer the question auditors already ask: is this the agent we approved?