UM-IHC’s Florence Doo explains how artificial intelligence is reshaping clinical radiology—and why patient safety, accountability and trust must keep pace.
For clinical radiologists, artificial intelligence (AI) is not a novelty, but an evolving part of practice.
“We’ve been using AI for more than a decade, and 75% of all AI tools that are FDA-approved are in radiology,” said Florence Doo, an assistant professor in diagnostic radiology and nuclear medicine at the University of Maryland School of Medicine with a joint appointment in the University of Maryland Institute for Health Computing.
But AI’s rapid expansion in medicine “is outpacing our ability to assess its outcomes and protect patients from its potential dangers,” said Doo, who co-leads the UM-IHC’s AI-enabled medical imaging team.
That’s why Doo and colleagues called for alignment of policy, practice and patient safety for trustworthy AI in radiology in a special report published recently in the journal Radiology: Artificial Intelligence.
In this Q&A with the UM-IHC, Doo talks about AI’s promise and pitfalls in clinical radiology and how to ensure the rapid growth of AI takes patient safety seriously.
This interview has been edited for length and clarity.
How do radiologists use AI?
There are a few major ways, including harnessing medical imaging data and doing non-interpretive work like generating summaries and automating tasks like scheduling. We also use it for help with analyzing images, specifically to spot and categorize breast cancer lesions.
Finally, we use it quite a bit for triaging patients. AI can help us move a patient into the “urgent” category, helping us flag something that needs our attention sooner. The tool isn’t quite mature enough to define what we’re seeing, but that isn’t really a pain point for radiologists.
Importantly, we very much see AI as a collaborator, not a replacement.
Is it true that AI is more sensitive than the human eye and can find things radiologists miss?
Yes, for pure image-detection tasks, AI can be more sensitive than the human eye. It’s especially very good at “needle-in-a-haystack” nodule detection.
But radiology isn’t just sitting in a dark room looking at pictures! In a recent study, my mentor Curt Langlotz [director of the Center for Artificial Intelligence in Medicine and Imaging at Stanford University] found that performing/interpreting images is only about two-thirds of the job. The rest is things like protocoling studies, consulting the referring physician or other radiologists, and communicating with technologists and patients. This means that even where AI does see something I missed, there’s plenty left for me, the human, to do.
Imagine a chatbot told you your child needs surgery because of a single pixel on a scan. You’d want to talk to a real, multidisciplinary human team that can put that finding in the context of your child, decide whether it matters and guide you on what to do next. Medicine isn’t just science alone or just one pixel; it’s judgment, context and people. For now, at least, AI can’t be fully autonomous to take over the art of medicine (or radiology).
Also, when I put on my innovation hat, I want to ask: What entirely new things can a human and AI do together that neither could do alone, things we can’t even envision yet? Imagine someone in the year 1800 trying to list the tasks and jobs people have today; they simply couldn’t picture it. I suspect radiology is standing at that same kind of horizon. That’s the leap of imagination we should be thinking about.
What research projects are you working on?
There are three major lanes to our research. First, how do we manage all the medical imaging data? We have about 2 petabytes of imaging data, with over 2 million patients that come to the University of Maryland for imaging and follow-up, and their data needs to be organized and accessible to both clinicians and researchers.
Second, we’re building AI models on top of that data and thinking about Marylanders in particular: What diseases are they facing most and how can we improve imaging for those diseases?
The third lane of research is super important: making sure those AI tools are adopted in the clinic safely. It’s great to have a lot of tools and data, but now that we’ve used some of these tools in the clinic, how are they working and can we safely deploy them here in Maryland and across the country?
What does safety and trustworthiness in AI mean?
These are two different but related ideas; patient safety is an outcome that requires trustworthy AI. We address this in our latest paper, asking what makes an AI system trustworthy and how to make sure that AI-related policy protects patients. Broadly, we emphasize five characteristics that can help establish trust in radiology AI: safety/reliability, fairness (systems perform comparably across patient populations), transparency/explainability, accountability and privacy/security.
The progression of AI has redistributed responsibility and clinical judgment, and we need to think about how policy choices affect patient safety and advancement of AI tools. AI is increasingly affecting patient diagnoses and treatment. On a chest CT angiogram, for example, an AI tool can flag a suspected blood clot (pulmonary embolism) and move that patient to the top of my worklist so the person gets to treatment faster. AI can also weigh in on whether a finding looks benign or suspicious on a screening mammogram, which can influence whether a patient gets a biopsy or when to screen again.
Our team has developed a roadmap for bridging policy and clinical practice via “translational bi-alignment.” The idea is to integrate regulatory requirements—what AI developers and vendors should deliver to clinical users (including clinical expertise to interpret outputs and validation and monitoring systems)—with what institutions should provide for safe use of AI (for example, documentation, validation and reporting requirements).
It’s important to think in terms of the entire AI lifecycle, but one problem is that some 75% of FDA-approved tools aren’t really followed after clearance. With a drug, there’s a reporting system, so we learn about side effects or other problems happening in the real world. We don’t have those robust post-market systems in place for AI, which is changing every day.
We especially need to take extra care with autonomous tools. A non-medical example is you may set up an AI agent to buy a plane ticket if the price drops below a certain number. In healthcare, the equivalent might be a tool that monitors a patient’s blood pressure and, on its own, alerts the physician and pre-documents a note if the blood pressure goes below a specific value. But there’s a wrinkle: The model behind that alert can be updated over time—think of your phone or laptop needing updates—so the same patient data might trigger an alert one month and stay silent the next. So, these AI tools can be autonomous and shift over time. It’s difficult to work with and trust literal moving targets.
What happens if we rely on AI that’s not trustworthy?
There is a great analogy from the computer science community that addresses trustworthiness and human-AI alignment and what can go wrong if we don’t set up guardrails. In short, say you set up an AI system whose mission is to create as many paperclips as possible in the shortest time, but you unplug the machine each night. Eventually the AI learns that you are the obstacle to its efficiency, so its mission becomes to get rid of you—going rogue like in some horror movie—so it can truly meet its goal. You get the picture!
The lesson is that we want AI to do the right thing even in new situations, which is quite difficult. It isn’t just an engineering problem; working it out takes input from the whole community and a lot of thoughtful science. And in medicine the stakes are higher than plane tickets or paperclips—at a minimum, we must be certain these tools don’t cause patient harm.
How do we move forward?
There’s great potential for AI to advance healthcare and medical care, but we need to decide what is truly going to serve us as clinicians and as human beings, and how to ensure there are guidelines that align with human values and prevent harm. We need voices in the room—clinicians and patients, not just companies and institutions. AI learns from data, and that data shapes the models that will, in turn, shape all of our care. We’re building the foundation of that future right now, and it’s crucial that our communities are in it together.
I think the University of Maryland is at the cutting edge of figuring out what we will want in a future AI-enabled physician for patient care. The real test of these tools is whether they hold up across the full range of patients we actually see—not just the ones a model was trained on. In Maryland, we care for people from every corner of the state and every walk of life, which makes it a strong starting point for clinically translating these AI tools for safe real-world practice. What we learn here can help build future clinical AI that patients can trust, wherever they are.
