Hospitals Race to Deploy AI Without the Ability to Test It

Hospitals have gone all in on artificial intelligence, but the systems to check whether those tools actually work safely have not kept pace. A new report cited by MedCity News found that more than 90% of health systems have deployed third-party AI tools, while less than half have the infrastructure to properly test and validate them before they become embedded in patient care.

In practice, that means algorithms are shaping clinical decisions, documentation, and triage without consistent local validation against a hospital's own patient population. AI models can drift, underperform on populations they were not trained on, or degrade as clinical conditions change. Without dedicated testing pipelines, health systems often cannot catch those failures until they affect patients.

Some systems, including UPMC, are building internal evaluation capacity to close the gap. For most, the takeaway is that buying AI is now the easy part. The harder, unglamorous work of governance, monitoring, and ongoing validation is where the real risk and accountability sit.