A nurse pastes a discharge summary into a public AI chatbot to tidy the wording. A prior-authorization assistant sends a member's whole chart to a model provider for a request that needed three fields. Privacy and compliance teams we talk to describe versions of both. Once one of them reaches the privacy office, the questions that follow come mostly from HIPAA's breach notification rule, and it helps to know them before they arrive.
01 · The presumption
An impermissible disclosure to an AI service is presumed a breach
HIPAA has no AI provision. An assistant that reads a chart or drafts a patient message handles protected health information. The Privacy, Security and Breach Notification Rules apply to it as they apply to any other system. Our reference page on the Security Rule and AI workloads covers those rules in full.
Start with the definition of breach in 45 CFR 164.402. A use or disclosure of PHI that the Privacy Rule does not permit is presumed to be a breach. The covered entity or business associate can rebut that by showing a low probability that the PHI was compromised, based on a risk assessment of at least four factors, and under 45 CFR 164.414(b) the burden of showing it sits with the organization.
The clock runs while you assess. Individuals are notified of a breach of unsecured PHI without unreasonable delay and no later than 60 calendar days after discovery (164.404), and a business associate notifies the covered entity on the same terms (164.410). Discovery is the first day the breach is known, or would have been known with reasonable diligence. On day one we tell teams to write down the date and how they found out, because every other date counts from it.
60
calendar days from discovery, at most, to notify individuals
4
factors, at a minimum, in the risk assessment
The clock runs while the assessment is under way
Day 0
Discovery: the first day the breach is known, or would have been known with reasonable diligence.
Day 60
Outer limit for notice to individuals. The rule says without unreasonable delay, so earlier can be required.
What the assessment decides
Starting point
A use or disclosure the Privacy Rule does not permit is presumed a breach
A business associate notifies the covered entity on the same 60-day terms.
Low probability of compromise shown
The presumption is rebutted. The documented assessment is the evidence.
Not shown, and the PHI was unsecured
The presumption stands and individuals are notified.
02 · The recipient
The first facts are the vendor and the agreement
Take the nurse. The privacy officer's first calls have little to do with what the summary said. Which chatbot was it, and who runs it? Which model sits behind it? Does the clinic have an agreement with that company, and does that company have one with the model provider?
If she used a free consumer account under its standard terms, there is no business associate agreement behind it. The disclosure was impermissible and the four-factor assessment starts. Now take the clinic's approved scribe, where the vendor signed a business associate agreement, the vendor has a subcontractor agreement with its model provider, and the use is one the agreements cover. That may involve no impermissible disclosure at all, and the file stays short.
The rules behind this sit in HHS's business associate guidance, which lists a patient-portal AI chatbot among its examples of a business associate. A covered entity may disclose PHI to a business associate once an agreement gives satisfactory assurance that it will be safeguarded, and a business associate needs the same kind of agreement with any subcontractor. A model provider that receives prompts from an AI vendor is that subcontractor. HHS limits the conduit exception to services that only transmit data, so it does not reach a vendor that works with the PHI to perform a service.
In our experience the second link is what slows teams down. They can usually name the tools they have approved. The model provider behind each one, and the agreement that covers it, takes longer to find. A one-page list of approved tools, each with its vendor, model provider and agreement, turns that first call into a lookup.
Before the four factors: was the disclosure permitted?
Does the AI vendor sit under a business associate agreement?
HHS names a third-party AI chatbot on a patient portal as a business associate. The conduit exception covers only entities that provide transmission services.
Does the model provider have a subcontractor agreement with that vendor?
A business associate needs one before passing PHI on. The model provider stands in that position.
Is the use one those agreements permit?
An agreement covers the purposes it names, and the minimum necessary standard still applies.
Path ends here
Clinical copilot under contract
Vendor agreement, subcontractor agreement and a permitted purpose all in place.
May involve no impermissible disclosure.
Exits at gate 1
Consumer chatbot, standard terms
No agreement behind the disclosure.
The four-factor assessment begins.
03 · Four factors
The risk assessment weighs at least four factors
The factors are listed in 45 CFR 164.402 and explained in the preamble to the 2013 Omnibus Rule (78 FR 5566). We read each one as a question an organization should be able to answer about any AI request on the day it is made.
Nature and extent of the PHI. Which identifiers, and how much clinical detail? HHS asks entities to weigh treatment plans, diagnoses, medications, medical history and test results, and a prompt built from a chart can carry all of them in one request. Pull the categories of PHI the request carried and which of them were removed before it left.
The unauthorized person. Who received it. HHS notes that the probability of compromise may be lower when the recipient is itself bound by HIPAA. Pull the provider, the model, and the terms that govern what the provider can keep and use.
Whether the PHI was actually acquired or viewed. HHS's example is a stolen laptop that was later recovered, where forensic analysis showed the data on it was never accessed. A request a model answered did reach the provider's systems. A request that was blocked, or redacted before it left, is a different case, as long as a record shows the decision.
The extent of mitigation. HHS points to the recipient's satisfactory assurances that the information will not be further used or disclosed, or will be destroyed, and cautions that such assurances from some third parties may not be enough. With an AI vendor we would ask for written confirmation of deletion in the first week, change the workflow so it cannot recur, and keep evidence that the change held.
Presumed a breach unless shown otherwise
To rebut the presumption, the covered entity or business associate shows a low probability that the PHI was compromised, based on a risk assessment of at least these four factors.
Factor, 45 CFR 164.402
Asked of an AI request
The record that answers it
Nature and extent of the PHI
The types of identifiers and the likelihood of re-identification.
Asked of an AI request
Which identifiers, and how much clinical detail, did the request carry?
The record that answers it
The categories of PHI detected in the request, and which of them were redacted before it left.
The unauthorized person
Who used the PHI, or to whom it was disclosed.
Asked of an AI request
Which tool, vendor and model received it, and under which agreement?
The record that answers it
The model and provider for each request, matched to the signed business associate agreements.
Actually acquired or viewed
Or whether only the opportunity existed.
Asked of an AI request
Did the PHI reach the model, or was the request blocked or redacted first?
The record that answers it
The decision on the request (delivered, redacted or blocked) and the time it was made.
Extent of mitigation
Assurances of no further use or disclosure, or of destruction.
Asked of an AI request
What changed once the disclosure was found, and did the change hold?
The record that answers it
The vendor's written assurance, the date the workflow changed, and the requests recorded after it.
Three of the four factors turn on facts about the request: what it carried, where it went and whether it got there. An organization that cannot establish those facts has little to put in the assessment, and the presumption stands.
04 · Audit controls
The answers come from audit controls
The Security Rule's audit controls standard, 45 CFR 164.312(b), requires mechanisms that record and examine activity in systems that contain or use ePHI. An AI workflow that reads a chart is one of those systems. HHS's January 2025 proposal to update the rule goes further for AI: its preamble says a risk analysis must consider the type and amount of ePHI the AI tool accesses, to whom the data is disclosed, and to whom the output goes. The proposal is not final and the current rule stands. The first two items on that list are the first two breach factors, asked before an incident.
Minimum necessary (45 CFR 164.502) applies to what goes into the tool. A disclosure to a business associate is limited, with reasonable efforts, to the minimum necessary for its purpose, and the treatment exception for disclosures to health care providers does not cover an AI vendor. Sending the whole chart for a task that needed three fields raises a minimum-necessary question before any breach question arises.
Evidence the organization does not hold is weak evidence. A vendor's retention and access terms govern its logs, and screenshots gathered after the fact read as reconstruction. When we ask teams how they would answer these questions today, a vendor's logs and screenshots are usually the first things they mention. The record that works is written when the request is made, kept by the organization, and shows whether anyone has changed it since.
Inside the organization’s environment
Clinician or patient
PHI entersAsks a question, pastes a note, or speaks during a visit.
AI tool
PHI in the promptA clinical copilot, ambient scribe, message drafter or prior-authorization assistant builds the prompt, often from the chart.
Boundary check
Last point of controlPHI is detected and redacted in flight, before the request leaves. The decision is written to a tamper-evident record.
What the record holds
- The workflow that sent the request, and the model and provider that received it
- The categories of PHI detected, and what was redacted
- Whether the request was delivered, redacted or blocked
- Sealed in a tamper-evident chain in a write-once store
The raw prompt stays inside the organization’s environment.
The request leaves the organization
Model provider
Detected PHI redactedReceives the request as it crossed the boundary.
05 · Beyond HIPAA
Section 1557 and state laws ask their own questions
HHS's Office for Civil Rights enforces both HIPAA and Section 1557 of the Affordable Care Act. Under 45 CFR 92.210, a covered entity may not discriminate on the basis of race, color, national origin, sex, age or disability through any automated or non-automated tool it uses to support clinical decision-making, and a copilot that suggests a diagnosis is one. Since 1 May 2025, covered entities have also had to make reasonable efforts to identify the tools whose inputs measure those characteristics, and to mitigate the risk of discrimination from each. The court orders against the 2024 rule do not reach section 92.210.
Our reference pages on Section 1557 and on ONC HTI-1 cover the detail. HTI-1 is the EHR certification rule for decision support tools, and ONC finalized HTI-5 in October 2026, which narrows it.
Three states have added duties that reach the same workflows. The timeline sets them beside the federal Section 1557 duty in date order (details and penalties).
Health AI duties, in order of effect
Not yet in force
1 Jan2025
CAAB 3030
Binds: Provider
Generative AI patient messages about clinical information carry a disclaimer and a way to reach a human, unless a licensed or certified provider read and reviewed them.
1 Jan2025
CASB 1120
Binds: Plan
AI may not deny, delay or modify care on medical necessity. A licensed physician or other qualified professional decides.
1 May2025
USSection 1557, 45 CFR 92.210
Binds: Covered entity
Make reasonable efforts to identify the decision support tools whose inputs measure protected characteristics, and mitigate the risk of discrimination from each.
1 Sep2025
TXSB 1188
Binds: Provider
Review all records created with diagnostic AI and tell patients about the use. EHRs stay physically in the United States, including at cloud vendors.
1 Jan2026
TXHB 149
Binds: Provider
Disclose AI use in a health care service or treatment to the patient by the date it is first provided.
1 Jan2027
COHB 26-1139
Binds: Plan
AI in utilization review produces and keeps documentation, audit logs and model-governance records that show compliance.
Texas SB 1188 adds a location rule, so where an AI service stores what it receives belongs in the same vendor review.
06 · Meilynx
Where the record comes from
Before the next incident
If we had to pick where a privacy office should start, it would be two things: the list of approved AI tools with the vendor, model provider and agreement for each, and a record of each request that the organization keeps itself. The breach rule gives at most 60 days from discovery to notify individuals and puts the burden of proof on the organization. A team that has both starts the assessment with the answers in hand.
This post summarizes 45 CFR Parts 92 and 164 and the state laws named above. It is not legal advice; verify against the primary sources and consult counsel.
Frequently asked
Is PHI pasted into a public AI chatbot automatically a reportable breach?
It is presumed to be a breach. Under 45 CFR 164.402, the covered entity or business associate can rebut the presumption by showing a low probability that the PHI was compromised, based on a documented assessment of at least four factors. If the presumption stands and the PHI was unsecured, individuals are notified without unreasonable delay and no later than 60 calendar days after discovery.
Does an AI vendor that handles PHI need a business associate agreement?
HHS lists a third-party AI chatbot on a provider's patient portal that handles patients' PHI as an example of a business associate. A business associate in turn needs an agreement with any subcontractor that receives the PHI. A model provider that receives prompts from an AI vendor stands in that subcontractor position.
Does Section 1557 apply to generative AI in clinical care?
Section 92.210 covers any automated or non-automated tool a covered entity uses to support clinical decision-making. This includes a generative AI tool that suggests a diagnosis or a course of treatment. Since 1 May 2025, covered entities have had to make reasonable efforts to identify such tools whose inputs measure protected characteristics, and to mitigate the risk of discrimination from each.