Jun 23, 2026
5 Min
HIPAA Compliant Generative AI: Core Concepts Explained
Can generative AI be HIPAA compliant? Yes, with a BAA, no training on your data, encryption, and de-identification. Plus what to ask before signing.
TL;DR: Generative AI is HIPAA compliant when the deployment meets HIPAA's conditions: the vendor signs a business associate agreement (BAA), protected health information (PHI) is encrypted in transit and at rest, the vendor commits in writing to never training models on your data, and access controls plus audit logs cover every interaction. Free consumer chatbots meet none of those conditions, so patient information cannot go into them. The concepts that decide compliance are PHI in prompts, the BAA chain, training-data policy, and de-identification.
Generative AI moved into healthcare faster than any technology HIPAA has met. Staff draft patient emails with it, vendors wire it into the phones and chat, and assistants keep appearing inside software the practice already owns.
HIPAA never mentions generative AI, and it does not need to. The law follows the data: if protected health information touches a system, the rules apply to that system. This guide covers the core concepts a practice manager needs to put generative AI in healthcare to work safely. (Not legal advice; run your specifics past your privacy officer or a healthcare attorney.)
What Counts as Generative AI in Healthcare
Generative AI means models that compose new content, whether text, speech, or images, in response to a prompt. That is the technical definition. The operational one matters more for compliance: generative AI is any tool where staff or patients say or type something free-form and the system writes its own reply.
In a practice it shows up in four places:
Drafting. Patient replies, referral letters, prior-auth narratives, review responses.
Conversation. Chatbots on the website and voice agents on the phones.
Summarization. Call summaries, visit recaps, inbox triage.
Embedded assistants. AI features switched on inside document editors, browsers, and office suites you already pay for.
The free-form quality is what creates the compliance question. A rules-based reminder system can only send the template it was given. A generative system will accept anything a user puts into it, including a patient's name, diagnosis, and insurance ID.
PHI in Prompts: Where Compliance Is Won or Lost
Start with the most important concept: a prompt is a disclosure. The moment someone types patient details into a generative AI tool, or a voice agent hears them on a call, PHI has been transmitted to that vendor's systems. Under HIPAA, that transmission is lawful only when the vendor is acting as your business associate under a signed BAA; the definition covers any vendor that creates, receives, maintains, or transmits PHI on a covered entity's behalf, per 45 CFR 160.103.
PHI does not stop at the prompt. Outputs inherit it: an AI summary of a patient call is PHI just like the recording underneath it. So do the byproducts, meaning chat histories, transcripts, logs, and cached context.
The riskiest flow in most practices is shadow AI: staff pasting patient details into free chatbots nobody approved. No BAA covers those tools. The fix is a bright-line policy (nothing identifying a patient goes into any tool off the approved list) backed by training, which we cover in our HIPAA-compliant AI evaluation framework.
The BAA: The Contract That Makes Generative AI Usable
A business associate agreement makes an AI vendor legally responsible for your PHI: safeguard it, use it only to deliver the contracted service, report breaches. Every generative AI vendor whose systems touch PHI needs one signed before the first prompt, recording, or upload.
With generative AI, the chain behind the BAA matters as much as the BAA itself. Most products are assembled from other companies' services: a foundation-model provider, cloud hosting, telephony, transcription. HIPAA requires business associates to obtain the same satisfactory assurances from any subcontractor that handles PHI on their behalf, per 45 CFR 164.502(e). Ask any vendor to name its model provider and confirm the agreements run end to end. A vendor who cannot answer has never checked.
This is also where the "HIPAA GPT" question lands. Compliance attaches to a deployment rather than to a model, so no version of GPT or any other large language model is compliant by itself. The consumer versions of ChatGPT do not come with a BAA, and neither does ChatGPT Business. OpenAI offers BAAs for its API platform and for sales-managed Enterprise and Edu accounts, per OpenAI's BAA documentation. The same split runs through other model providers. The underlying model may be identical; the contract, data handling, and retention terms are what change.
Training-Data Policy: The Question That Sorts Vendors
Ask every generative AI vendor one question and get the answer in writing: do you train models on our data?
Consumer tools frequently reserve that right by default: OpenAI may use individual ChatGPT users' content to train its models unless they opt out, per OpenAI's data-usage documentation, and Google uses Gemini Apps activity to improve its generative models unless the setting is turned off, per Google's Gemini Apps Privacy Hub. For patient data, that is the fastest fail in any evaluation. Large models can memorize fragments of training data and reproduce them for other users, and once PHI is in the weights there is no practical way to pull it back out.
Two answers are acceptable. Either the contract states the vendor does not train on your data, full stop, or training happens only on data de-identified under one of HIPAA's recognized methods, covered next. Central (our product) answers the first way: data is encrypted in transit and at rest, and we don't train AI models on your data.
Retention belongs in the same clause: how long prompts, recordings, and transcripts are kept, whether you can shorten the window, and what gets deleted when you leave.
Want the compliance stack already assembled? Central answers every patient call and chat 24/7, verifies insurance on the line, and books into your EHR, with a signed BAA, SOC 2 and ISO 27001 certification, encryption in transit and at rest, and no model training on your data. Book a demo, or hear it live: +1 (833) 545-5994.
De-Identification: When Data Stops Being PHI
HIPAA stops applying to data that has been properly de-identified, which is why the concept carries so much weight in AI contracts. Properly is the operative word. HIPAA recognizes exactly two methods: Safe Harbor, which removes 18 enumerated categories of identifiers (names, contact details, dates, record numbers, and more), and Expert Determination, in which a qualified expert certifies that the risk of re-identifying anyone is very small, per 45 CFR 164.514(b).
Safe Harbor: The Checklist Method
Safe Harbor is mechanical. Remove all 18 categories of identifiers for the patient and for the patient's relatives, employers, and household members, and the data is de-identified, provided you have no actual knowledge that what remains could still identify someone, per the same section of the rule.
The list runs broader than most people expect. Names, obviously. Also every geographic unit smaller than a state, with one exception that keeps the first three digits of a ZIP code when the three-digit ZIP area holds more than 20,000 people. Every date tied to the person except the year: birth dates, admission and discharge dates, date of death. Ages over 89 collapse into a single "90 or older" bucket. Then the long tail: phone and fax numbers, email addresses, Social Security numbers, medical record and account numbers, health plan IDs, license and certificate numbers, vehicle and device identifiers, URLs, IP addresses, biometrics, full-face photos, and any other unique code or characteristic, per 45 CFR 164.514(b).
Safe Harbor's appeal is that a practice can audit it: either the field is gone or it is not. Its weakness for generative AI is free text. The method was written for structured records, and running an 18-item checklist against call transcripts and chat histories takes software that can spot identifiers inside sentences, where every miss is still a disclosure.
One allowance inside the rule is worth knowing, because vendors reach for it. A covered entity may keep a code that lets it re-identify its own de-identified records, but the code cannot be derived from or related to information about the individual, cannot otherwise be translated back to them, and the re-identification mechanism cannot be disclosed, per 45 CFR 164.514(c). A token generated by hashing the medical record number fails that test. When a vendor calls data tokenized or pseudonymized, ask how the token is produced and who holds the key.
Two more questions separate a working Safe Harbor pipeline from a claimed one. What method does the redaction actually use, and what miss rate has the vendor measured against its own transcripts? And what happens to the copy that existed before redaction, since raw audio and raw transcripts usually survive somewhere in the system.
Expert Determination: The Statistical Method
Expert Determination trades the checklist for judgment. A person with appropriate knowledge of statistical and scientific methods for de-identification analyzes the dataset, determines that the risk of identifying an individual is very small, alone or in combination with other reasonably available information, and documents the methods and the result. That documentation is the artifact. Ask to see it, or at minimum to see who produced it and when.
The method exists because Safe Harbor destroys utility. A vendor that wants dates, geography, or rare-condition detail in its training data will usually go this route. Two practical points matter for contracts. No specific professional degree or certification program designates who counts as an expert; the expertise is gained through various routes of education and experience, and experts turn up in statistical, mathematical, or other scientific domains, per HHS's de-identification guidance. And nothing in the Privacy Rule puts an expiration date on a determination. Some de-identification practitioners issue time-limited certifications instead, and recurring releases of the same data need an expert to examine whether current conditions call for a different process, per the same guidance. A certification produced once, for one historical extract, says nothing about the live feed of next month's call recordings. If a vendor trains on expertly determined data, the contract should say who the expert is, what the determination covers, and how often it is refreshed.
The Limited Data Set: The Middle Option
A limited data set sits between full PHI and de-identified data, and vendor paperwork blurs the two often enough that it needs naming. It strips the direct identifiers, meaning names, address detail below city level, phone and email, record and account and plan numbers, device and vehicle identifiers, biometrics and full-face photos. It keeps town or city, state, ZIP code, and dates tied to the individual, including birth, admission, discharge, and death, per 45 CFR 164.514(e).
Those retained dates and geography are what make the data useful for analysis, and they are also why HIPAA still applies to it. A limited data set is not de-identified data. It can move only for research, public health, or health care operations, and only under a data use agreement in which the recipient commits to those limits and to safeguarding what it receives.
Read AI contracts with that distinction in hand. A data exhibit promising to strip identifiers while keeping dates is describing a limited data set, which calls for a BAA plus a data use agreement rather than an exemption from either.
"Anonymized" Is Not a Method
"Anonymized" appears in a lot of AI vendor copy. It is not a HIPAA method. When a vendor says training data is anonymized, ask which of the two methods applies, then make the contract name it.
Free-form data raises the stakes here too. A call transcript can identify a patient through a nickname, an employer, or a rare condition plus a town, even after the obvious fields are scrubbed. Treat de-identification claims about conversational data with extra suspicion.
The Safeguards That Still Apply
Everything HIPAA already expected of software still applies to generative AI. Four safeguards do most of the work.
Encryption in transit and at rest, covering the unglamorous copies: recordings, transcripts, logs, backups.
Access control. Unique logins, role-based permissions, multi-factor authentication. Least privilege includes the AI itself: a voice agent's EHR credentials should reach only the fields it needs to book and verify, never the full chart. The Privacy Rule's minimum necessary standard points the same direction, requiring reasonable efforts to limit PHI to the minimum needed for the purpose, per 45 CFR 164.502(b).
Audit logging. Every prompt, every output, every record the AI read or wrote, every human who opened a transcript. Logs are how you investigate an incident and how you prove the other safeguards are holding.
Human oversight. Generative models produce fluent, confident errors. A person owns anything that reaches a patient or a chart, and the system needs a defined path for handing a conversation to a human when it meets something it should not handle.
What to Ask a Vendor Before You Sign the BAA
The BAA is where these concepts turn into terms you can hold someone to. Put the answers in the agreement, not in an email thread.
Who provides the underlying model, and where does it run?
Is every subcontractor that touches PHI covered, including hosting, telephony, and transcription, and does any of that work happen outside the United States? Ask for the subprocessor list and for notice before it changes.
Do you train, fine-tune, or evaluate models on our data? Get all three verbs in writing. "To improve our services" is the clause that quietly swallows the commitment.
If you use de-identified data, which method, and who made the determination? Safe Harbor or Expert Determination, named, with the documentation available to read.
How long do you keep prompts, recordings, transcripts, and logs, and can we shorten it? Retention should be a setting you control.
Do your employees read our transcripts? Human quality review is common and defensible. Undisclosed human review is not.
Have you had a reportable incident, and how quickly does the BAA say you must tell us? A business associate has to notify the covered entity of a breach without unreasonable delay and in no case later than 60 calendar days after discovery, per 45 CFR 164.410. Treat that as the outer limit and negotiate down.
A vendor that answers these in one call is running a real compliance program. The rest of the review, what the AI can reach in your EHR, what the audit log captures, when a conversation goes to a person, and how to read a SOC 2 report, runs through our framework for evaluating AI tools under HIPAA.
What Compliant Generative AI Looks Like at the Front Desk
The clearest working example is patient access: the phones, the chat widget, reminders, intake. That is where generative AI already runs at scale in healthcare.
An AI front desk for healthcare answers calls and website chat around the clock, verifies insurance and copay on the call, books directly into the EHR, texts intake forms, and calls missed leads back. Every one of those interactions carries PHI, so it has to clear the same stack described above, with a person available to take the call over when the conversation calls for one. Full disclosure: Central is our product, and that is the bar we build to, with 1,000+ practices running on it.
Patient-facing messages carry rules of their own about what a text or voicemail may say. Our guide to HIPAA-compliant appointment reminders covers those specifics.
FAQ: HIPAA-Compliant Generative AI
Is ChatGPT HIPAA compliant?
No, and that includes ChatGPT Business: OpenAI does not offer a BAA for the consumer tiers or for Business, per OpenAI's BAA documentation. Its API platform and sales-managed Enterprise and Edu accounts can form part of a compliant deployment with a signed BAA, and everything else in this guide still has to be true around them.
Is there such a thing as a "HIPAA GPT"?
No model is compliant on its own. Compliance describes a deployment: the BAA chain, training-data terms, encryption, audit logging, and human oversight wrapped around a model. A vendor selling a "HIPAA GPT" should be ready to answer questions about all five.
Does HIPAA prohibit generative AI in healthcare?
No. HIPAA regulates how PHI is used and protected, whatever the technology. A practice can run generative AI on patient data with a BAA and safeguards in place, and can use it freely for work that contains no patient information, like marketing copy or internal templates.
Can we use generative AI on de-identified data without a BAA?
Yes. HIPAA does not govern properly de-identified data. Confirm which recognized method was used, Safe Harbor or Expert Determination, and be cautious with conversational data, since transcripts resist clean de-identification.
Is a limited data set the same as de-identified data?
No. A limited data set still contains dates and geography, so HIPAA still applies. It moves only under a data use agreement, and it never substitutes for a BAA.
Do patients have to consent before we use generative AI?
HIPAA generally permits PHI use for treatment, payment, and operations through a business associate without separate patient authorization, per 45 CFR 164.506, but AI-specific disclosure duties are developing at the state level. California requires a disclaimer and a route to a human on generative AI patient communications about clinical information unless a licensed provider reviews them first, per California AB 3030, and Utah's Artificial Intelligence Policy Act requires generative AI disclosure on request and in high-risk interactions, a category that includes medical contexts, per the Future of Privacy Forum. Many practices disclose anyway. Patients handle "you're chatting with our AI assistant" far better than discovering it later.
What about AI features inside software we already use?
Every embedded assistant is a new PHI flow that needs the same evaluation: BAA, training terms, retention. Document tools are the common blind spot; our breakdown of whether Adobe Acrobat is HIPAA compliant shows how to assess one.
Put the Concepts to Work
Know where PHI enters prompts, outputs, and logs. Sign the BAA and check the chain behind it. Get the training-data answer in writing. Accept only named de-identification methods. Hold those lines and the practice gets what generative AI is genuinely good for, answered phones and finished paperwork, without new exposure. Vendors who clear the bar show their evidence in the first call. A vendor who stalls has told you what you needed to know.
See a compliant deployment on your own phones. Central's AI front desk for healthcare answers every call and chat 24/7, verifies insurance and copay on the line, and books straight into your EHR. Average go-live is 4 days, and it runs HIPAA compliant with a signed BAA. Book a demo, or hear it live: +1 (833) 545-5994.
