Building Trust in AI: A Clinician's Journey from Development to Deployment

Text: Camellia Yong

As AI becomes increasingly embedded in healthcare, attention is shifting from what AI can do to how it can be implemented safely and effectively in real-world clinical settings. Dr Muhammad Mamdani, Director of T-CAIREM at the University of Toronto, shares his work in this area ahead of his visit to Singapore to speak at the International Conference on AI in Medicine (iAIM).

Dr Muhammad Mamdani, Director of T-CAIREM at the University of Toronto, Canada, shares lessons from his experience implementing AI in healthcare, including how to navigate adoption challenges, measure impact, and ensure AI solutions deliver meaningful value for clinicians and patients.

His experience highlights the importance of pairing technological innovation with robust governance, clinician engagement, and a clear focus on patient outcomes.

Share with us one successful adoption of AI in healthcare, what impact does it make for clinicians and patients?

Our clinical and AI teams developed CHARTwatch – an AI solution that monitors our internal medicine patients at the St. Michael’s Hospital in Toronto. After considering typically between 150 to 170 parameters, including patient demographics, lab values, and vital signs every hour, it alerts the medical team when a patient is deemed to be at high risk of dying or going to the intensive care unit (ICU) in the next 48 hours. Our published study in the Canadian Medical Association Journal demonstrated a 26% reduction in unexpected mortality.

What was the process you undertake before implementing this AI solution in real practice?

To ensure rigorous benchmarking, we first conducted a study to evaluate physicians' ability to predict patient deterioration, including death or ICU transfer. This allowed us to compare the model's performance against human clinical judgement and identify the simplest model that consistently outperformed clinicians.

Before introducing the tool into clinical practice, we undertook extensive silent testing within an AI Operations environment to assess its real-world performance, while monitoring for issues such as algorithmic bias, model drift, and system reliability. Once we were confident in its safety and effectiveness, CHARTwatch was formally deployed. Importantly, the rollout was accompanied by a comprehensive AI literacy programme to educate and train clinicians on how the tool works, how to interpret its outputs, and how to integrate it appropriately into clinical decision-making.

What was one of the biggest challenges in implementing it and how did you overcome it?

The biggest challenge was gaining clinician acceptance. We first needed to demonstrate that predicting patient deterioration is difficult, even for experienced clinicians. By showing that clinicians' prognostic accuracy for outcomes such as death or ICU transfer was relatively poor, and that the AI model performed substantially better, we established a clear value proposition for its use. This evidence-based approach helped build trust and encouraged clinicians to view AI as a tool that enhances, rather than replaces, their judgement.

How do you measure the success of an AI solution?

We had very clear metrics that mattered to clinicians and patients we set a target before developing the solution – we would need to see at least a 10% relative reduction in mortality for the AI solution to be deemed successful. The result yielded a 26% reduction.

We measure the success of an AI solution by its ability to deliver meaningful improvements in patient outcomes. From the outset, we established clear, clinically relevant performance metrics that mattered to both clinicians and patients.

What is the advice you would give to someone who is looking at developing an AI solution in the hospital?

Successful AI development and adoption is fundamentally a team effort, not just a technology project. It demands a multidisciplinary team, committed leadership, and sustained investment in expertise, infrastructure, and governance. Organisations should be prepared not only to develop AI models, but also to deploy them responsibly, continuously monitor their performance, and maintain them over time. The most successful AI initiatives are those that are integrated into clinical workflows, trusted by users, and focused on improving patient outcomes.

Interested in AI in medicine? Don’t miss the iAIM Conference from 3 to 5 December 2026, where Dr Mamdani and other prominent researchers will share their insights on the theme, “Rethinking AI for Medicine”. Register now: https://event.ntu.edu.sg/iAIM