- 56% of medical groups have no formal AI governance policies and aren’t working on them.
- 72% of United States adults say provider disclosure of AI use is very or extremely important.
- Shadow AI is AI that staff use without the practice’s knowledge, approval, or oversight.
- No single federal AI governance law exists, but HIPAA and state laws still apply.
- The NIST AI Risk Management Framework has four functions: Govern, Map, Measure, and Manage.
- A business associate agreement may be required when an AI vendor handles PHI.
TL;DR:
With 83% of medical groups using AI in patient visits but 56% lacking formal governance policies, your practice doesn’t need a dedicated oversight team — it needs a few simple, repeatable safeguards for how AI is selected, reviewed, and monitored. Start with the shadow AI your staff may already be using.
A physician turns on an ambient scribe. The EHR adds an AI feature. A biller uses AI to help draft an appeal. A receptionist uses a generative AI tool to rewrite a patient message. Each decision can seem small in isolation; however, eventually, a medical practice may begin to use AI in multiple workflows without ever establishing basic rules for usage. We’re talking about AI governance, and without it, small practices may be especially vulnerable to privacy and security breaches, inaccurate AI-generated information, compliance problems, patient safety risks, and costly errors. Why? AI adoption often moves faster than a small practice’s ability to put safeguards around it.
In some cases, small practices may not even realize they’re using AI because it’s hidden inside a familiar technology. Here’s the good news: With the right AI governance in place, small practices can use AI technology to improve efficiency and patient care while minimizing potential risks.
What is AI governance?
AI governance refers to the policies, roles, and controls around:
- Which AI tools employees can use
- How employees can use AI tools
- What patient or practice data the AI tools can access
- Who reviews AI-generated output
- How the practice responds when something goes wrong
The overarching question at the root of an AI governance framework is this: How can a medical practice benefit from AI without creating unnecessary risks for patients, employees, or the business?
The cost of skipping AI governance for medical practices
Skipping AI governance can cost practices through:
- Compliance problems
- Errors
- Financial losses
- Loss of trust (Note: About seven-in-ten U.S. adults [72%] say it’s extremely or very important that a doctor or other healthcare provider tells them if they’re using AI in their healthcare).
- Patient harm
- Rework
A lack of governance can also prevent practices from getting the full value of AI because there is no safe framework for using it.
Most practices aren’t governing AI yet
More than half (56%) of medical groups do not have formal AI governance policies, nor are they working on them. This is despite the fact that 83% of medical groups use some form of AI for patient visits. This governance gap can carry real costs.
| Governance gap | Potential cost to the practice |
|---|---|
| AI produces inaccurate clinical information | Patient harm, corrective work and potential malpractice exposure |
| AI scribe creates inaccurate documentation | Time spent correcting records, downstream clinical errors and potential billing problems |
| Staff enter PHI into an inappropriate AI tool | Privacy/security incidents, investigation and remediation costs |
| AI generates inaccurate coding or billing information | Denials, rework, overpayments, repayments or audits |
| Practice doesn't adequately vet an AI vendor | Security, privacy, contractual or performance problems |
| AI introduces or reinforces bias | Patient safety, discrimination and compliance concerns |
| Employees use unapproved AI tools | Loss of control over sensitive information and inconsistent workflows |
| No one reviews AI output | Errors can make their way into records, claims or patient communications |
| No one monitors AI performance | Problems may continue — and potentially affect multiple patients — before the practice recognizes them |
| Patients lose confidence in the practice's use of AI | Complaints and erosion of patient trust |
Where AI is already hiding in your practice
AI can hide in plain sight throughout a medical practice, embedded in EHRs, patient portals, scheduling systems, billing tools, and other everyday technologies. Shadow AI, more specifically, usually refers to AI tools that employees use without the practice's knowledge, approval, or oversight. For example, a staff member using a public generative AI tool to summarize a patient record.
Where shadow AI shows up most
Shadow AI shows up when clinicians and staff use unapproved AI tools for everyday tasks, such as:
- Creating appeal letters
- Drafting patient messages
- Preparing administrative documents
- Researching clinical or billing questions
- Summarizing records
It can also appear through AI features built into software employees already use without the practice formally reviewing or approving those features.
Why staff turn to unapproved tools
Staff often turn to unapproved AI tools because these tools:
- Are easy to access and often free or inexpensive
- Reduce workload when staff are stretched thin
- Save time on repetitive or administrative tasks
- Solve an immediate problem
Staff may also use them because:
- The practice hasn't established clear AI policies.
- The practice hasn't provided an approved AI alternative.
- They assume publicly available or familiar AI tools are safe to use for work-related tasks.
However, here’s what staff need to keep in mind: Even seemingly harmless AI use can create privacy, security, accuracy, and compliance risks, particularly when it involves patient or practice information.
What AI governance covers in a medical practice
AI governance for medical practices should cover the entire lifecycle of AI use, from deciding whether to use a tool to monitoring it once it's in place. Each area of an AI governance framework seeks to answer important questions about the use of AI-generated information.
| Governance area | Questions AI governance seeks to answer |
|---|---|
| AI use | Where are we using AI today? Are employees using AI tools we don't know about? |
| Purpose | Why are we using AI? What problem is it supposed to solve? |
| Tool selection | Which AI tools are appropriate for our practice? Who approves them? |
| Patient data | What information can employees enter into AI tools? Can the tool access or process PHI? |
| Privacy & security | How does the AI tool collect, store, use, and protect patient and practice data? |
| Vendor oversight | Has the vendor adequately addressed privacy, security, accuracy, compliance, and data use? |
| Human oversight | Who reviews AI-generated information before someone acts on it or sends it to a patient? |
| Accuracy | How do we know the AI is producing reliable information? How do we identify errors? |
| Patient safety | Could an AI error harm a patient? Which uses require greater safeguards? |
| Transparency | When should patients know that AI is being used in their care or communications? |
| Accountability | Who is ultimately responsible for decisions or actions involving AI? |
| Staff use | For what purposes can employees use AI — and what uses are prohibited? |
| Training | What do clinicians and staff need to know before using an AI tool? |
| Monitoring | How will we know whether an AI tool continues to work as intended over time? |
| Problems & incidents | What should an employee do if AI produces an error, exposes sensitive information, or behaves unexpectedly? |
| Changing technology | When should we reevaluate an AI tool, particularly after the vendor changes or updates it? |
Tool selection, human review, and data privacy and security controls are particularly important because these areas of governance address some of AI's most consequential risks—from choosing the wrong technology to acting on inaccurate output or exposing sensitive patient information.
Tool selection and vendor vetting
Before adopting an AI tool, practices should determine what problem they are trying to solve, what information the tool will access, how much risk its intended use creates, and whether the vendor can support its claims. Again, the goal is to choose AI tools that provide real value without introducing unnecessary risks to patients, data, or the practice.
Human review of AI output
Practices should define which AI-generated outputs require review, who is qualified to review them, and what that person needs to verify before the output is used. The goal of human oversight is to catch AI errors before they affect a patient's care, medical record, claim, or communication — and to ensure that people, not technology, remain accountable for consequential decisions.
Data privacy and security controls
Practices should establish rules for what information AI tools can access, how that information can be used, and how the practice protects it. The goal is to ensure practices know where sensitive information goes when AI uses it, how vendors protect and use that information, and what employees can — and cannot — share with AI tools.
Rules and AI governance frameworks that apply to your practice
There isn't currently one single federal “AI governance law” that tells a small medical practice exactly how to govern AI. Instead, practices need to think about a combination of existing healthcare AI governance laws, rules, and best practices that apply to particular AI uses, state requirements, and voluntary AI governance frameworks. Here are some AI governance framework examples that can help practices get started.
| Rule, guidance, or framework | What it means for small practices |
|---|---|
| HIPAA Privacy, Security & Breach Notification Rules — Federal requirements | When AI involves PHI, practices must consider how that information is used, disclosed, stored, and protected, including whether an AI vendor is a business associate and a BAA is required. |
| State AI, privacy and healthcare laws — State requirements vary | Practices should determine whether their state imposes additional requirements around AI, privacy, patient notification or consent, healthcare decision-making, or other AI uses. |
| NIST AI Risk Management Framework — Voluntary AI governance framework | Gives practices a structured way to identify, assess, and manage AI risks using the functions Govern, Map, Measure, and Manage. |
| NIST Generative AI Profile — Voluntary guidance | Helps practices address risks specific to generative AI, including general-purpose AI tools used for documentation, communication, research, and administrative tasks. |
| Joint Commission/CHAI Responsible Use of AI in Healthcare Guidance — Voluntary healthcare guidance | Offers healthcare-specific guidance on AI governance, privacy, security, transparency, risk and bias assessment, monitoring, safety-event reporting, and staff training. |
| CHAI AI Governance Playbooks — Voluntary healthcare AI governance guidance | Provides more detailed guidance on AI policies, risk assessment, lifecycle management, data governance, vendor oversight, and employee education. |
| Joint Commission Responsible Use of AI in Healthcare Certification — Voluntary certification | Gives healthcare organizations a formal benchmark for governance, data management, risk and bias reduction, monitoring and validation, transparency, and training. |
Setting up lightweight AI governance in a small practice
When it comes to an AI governance framework, the goal isn't to create a complicated structure; it's to put a few simple, repeatable processes in place so the practice knows how AI is being used and can manage it responsibly and successfully.
List every AI tool the practice already uses
Make a simple list of AI tools already in use — including AI embedded in the EHR, ambient scribes, scheduling and billing technology, and general-purpose AI tools employees may be using independently. For each one, document what it does, who uses it, and what data it accesses.
Put basic AI rules in writing
An AI governance committee can create a short acceptable-use policy that tells employees which tools they can use, what they can use them for, what information they can enter, and what uses are off-limits. Describe permitted-use cases, such as using AI for developing drafts of marketing materials and research summaries as well as prohibited AI uses, such as entering patients’ personal health information into publicly available AI tools.
Add a human-in-the-loop review step for AI output
Require a qualified person to review AI-generated information before the practice uses it to make or support important decisions. Human review helps prevent the practice from automatically accepting AI-generated information as accurate or appropriate.
Set minimum review standards for AI tools
Before adopting a new AI tool, require a short, standardized review. Consider what problem the tool is intended to solve, what patient or practice data it will access, whether the vendor can use that data to train its models, what security and privacy protections are in place, and whether a business associate agreement is required. Also determine where human review is necessary before AI-generated information is used to make clinical, coding, billing, or patient-facing decisions.
Monitor AI tools after they go live
AI governance for medical practices shouldn’t end when a tool is approved. Establish a simple process for periodically checking whether each tool is working as intended. Track issues such as inaccurate outputs, workflow problems, patient or staff complaints, privacy or security incidents, and unexpected effects on quality or efficiency. Give employees an easy way to report AI-related problems and define when a tool should be reevaluated, restricted, or discontinued.
AI can help private practices reduce administrative burden and create more efficient workflows —but getting value from it requires the right guardrails. Tebra gives practices technology designed to support smarter, more streamlined operations while keeping people at the center of care. Learn how Tebra can help your practice navigate the next generation of healthcare technology.
Frequently asked questions
- List every AI tool the practice already uses.
- Put basic AI rules in writing.
- Add a human-in-the-loop review step for AI output.
- Set minimum review standards for AI tools.
- Monitor AI tools after they go live.
The information provided in this post does not constitute medical or legal advice and is intended for informational purposes only.
Subscribe to The Intake: A weekly check-up for your independent practice
