Patients demand oversight for AI listening tools - ai listening tools
Patients demand oversight for AI listening tools

Patients entering exam rooms may not realize their conversations are being recorded by artificial intelligence. Many hospitals now use ambient listening tools—microphones and software that automatically transcribe doctor-patient visits. While clinicians have established best practices for these systems, governance from the patient’s perspective remains uneven.

Research presented at the 2026 HIMSS Global Health Conference in Las Vegas highlights the gaps. Wendy Charles, a teaching assistant professor at the University of Denver; Jack Crumbly, management department chair at Tuskegee University; and Renée Pratt, assistant professor at the University of North Georgia, led a panel on stakeholder governance for ambient AI scribes. Their study, conducted with colleagues from MedDG, Careeryze, and UCHealth, found patients often lack clarity about when recordings start, who can access them, and how long they’re stored.

“We are in a development stage of what AI is and where its governance is going,” Pratt said. “This may be the first time in a long time that patients will have a voice in what is happening with their healthcare. With ambient AI, now is the time to set the rules — not to wait until after the fact but to minimize what can go wrong ahead of time.”

Related: Oakland health system reverses layoffs with new funding

Crumbly noted that patients facing serious medical issues often prioritize human connection. Adding AI to the equation raises concerns. “Is the proper information recorded? Who will it be shared with? If something was missed, why wasn’t it properly documented? Is the AI accurately capturing and interpreting?”

Charles pointed out another problem: ambient tools miss nonverbal cues. “Most communication between a patient and clinician is nonverbal,” she said. “Patients shrug, look depressed, shake their heads, or cry. A clinician uses intuition to sense when a patient is holding back. None of that can be captured by a listening device.” Without proper oversight, she warned, transcriptions could misrepresent the full encounter.

Inconsistent policies create confusion

Pratt described visiting hospitals where staff gave conflicting answers about whether ambient AI was in use. “Currently, ambient AI use isn’t consistent across hospitals. There’s a lack of standardization around accuracy, patients’ ability to reference what’s been documented, policies about data retention or how clinicians share that they’re using ambient AI.”

Most organizations introduce the technology gradually, often starting with volunteer clinicians in one department. Charles said this approach works best when implemented slowly. “What’s a continuous challenge is that organizations are learning by doing without well-established frameworks for how to implement it. It works best when organizations pursue this slowly and gradually, learning from others and from their experiences about what makes sense for their values and mission.”

Related: Physical AI Set to Revolutionize Healthcare Industry

Security risks are a major concern. Pratt highlighted the rise in hospital data breaches. “Data security is a big part of this. When facilities are collecting data without governance in place, what is collected, how it’s collected and who has access become critical issues. There is now a recorded patient voice at a time when hacking and cybercrime are occurring frequently.”

The researchers agreed that consent processes need improvement. “Patients should be treated as individuals with choices about how and what to communicate,” Charles said.

Pratt called for clearer policies. “When a patient agrees to care, it doesn’t automatically mean they consent to recording or other uses of ambient AI,” she said. Hospitals should explain when recording starts and what happens to the data.

Moving forward, Charles called for stronger policies on data retention and access. “Digital health technologies are not just an IT issue; they affect the entire business.”

Related: ACO REACH Model Generates $2.5B in Medicare Savings

Crumbly said ambient AI is only the beginning. “As more listening devices appear, they will raise additional questions about data.”

Pratt urged the industry to prioritize patient input. “We must redesign processes with their concerns in mind.”

The technology will continue to spread. Without better governance, patients may find themselves in exam rooms where every word is captured and shared in ways they never agreed to.