Dive Brief:
- Healthcare organizations are rapidly adopting agentic artificial intelligence, but governance frameworks for managing the technology are not evolving at the same pace, according to a new survey from digital identity security firm Imprivata.
- More than 85% of leaders responsible for AI strategy at healthcare organizations say they’re confident they have visibility into AI agent activity and are able to fully control and govern an autonomous agent’s actions.
- However, 72% of respondents admit AI tools are deployed without IT approval at least occasionally within their organizations, highlighting a disconnect between confidence and reality.
Dive Insight:
Healthcare organizations are embracing agentic AI while simultaneously struggling with fundamental governance and oversight challenges, according to the report.
Agentic AI are AI systems that can act largely independently, making complex multi-step decisions and solving problems with minimal human intervention. The healthcare sector has invested heavily in agentic AI, especially for back office use cases such as prior authorization and revenue cycle management automation.
More than a quarter of leaders have already implemented agentic AI at their organizations, according to market research firm Vanson Bourne, which conducted the survey on Imprivata’s behalf. An additional 44% are piloting or conducting proof-of-concept projects and another 21% plan to implement it in the next year.
The vast majority of respondents expect agentic AI to have a “transformative” impact on clinical and operational workflows.
However, agentic AI presents newfound security risks — especially as use cases expand, giving agents access to a wider variety of internal systems, experts say. That’s because, unlike traditional software that relies on user input to act, agents can act autonomously, traversing different information systems and executing actions on behalf of users.
“If an agent has excessive permissions, operates outside its intended scope, or takes a high-risk action without appropriate oversight, the consequences can directly impact care delivery,” said Dr. Sean Kelly, chief medical and growth officer and senior vice president of customer strategy at Imprivata.
“An agent could access or expose sensitive patient information, enter incorrect information into a medical record, alter a medication or dosage or act under a clinician’s authority in a way that the clinician never intended. Because agents can operate autonomously and at machine speed, a single error could also propagate before someone recognizes what is happening,” Kelly said.
More than half of respondents ranked security among their top concerns when adopting agentic AI. Although leaders expressed confidence in their ability to govern AI agents, instances of shadow AI, when employees use AI tools that haven’t been authorized by their companies, are still rampant.
According to a separate survey by Wolters Kluwer, 40% of medical workers and administrators said they were aware of colleagues using unauthorized AI tools, while nearly 20% reported they have used an unsanctioned tool themselves.
Imprivata’s survey shows that organizations are taking fragmented approaches to managing AI agents. Sometimes information technology departments centrally manage the tools, while security teams manage others. Some organizations report ad hoc or unapproved deployments.
And healthcare leaders are still ironing out the specifics of AI governance and how much humans need to be in the loop when an agent is making decisions, according to the research.
That process varies widely based on where the agent is deployed — whether in an administrative, operational or clinical setting.
“Oversight must match the level of clinical risk. Agents need clearly defined identities, permissions, and boundaries, with human review for higher-risk activities and an audit trail for accountability,” Kelly said. “The more autonomy we give these systems, the more important those safeguards become to detect and contain problems before they create broader operational or patient safety risks.”
The healthcare industry has long sounded the alarm about potential patient safety risks posed by improperly implemented AI. ECRI, a nonprofit focused on healthcare safety and quality, named insufficient governance of AI in healthcare as one of its top 10 patient safety concerns last year.
Various industry consortia have issued guidance to alleviate these concerns. The Coalition for Health AI, a network of thousands of healthcare systems and industry groups, issued AI governance playbooks last year to help health systems responsibly and safely roll out AI tools.
