Misk Al Zahidy, M.S.
Benchmarking & Refinement of SBME Avatars with Digital-Twin Corpus:
Using AI to Strengthen Patient-Centered Communication Training
Award year: 2025 Richard M. Schulze Scholar in Artificial Intelligence
Our goal is to create AI-driven patient avatars that can replicate real patient-clinician interactions in a safe, structured and repeatable way. I often think of it like flight simulators for pilots. They train in a setting that feels real but without the risk. We want to create something similar for healthcare professionals, especially when it comes to difficult or emotionally complex conversations.
What are you trying to build, and why does it matter?
In simple terms, we are working to make virtual patients more realistic for medical training. Today, many communication skills are taught using actors who simulate patient scenarios. That approach is valuable, but it can be limited by cost, scheduling and access.
Our goal is to create AI-driven patient avatars that can replicate real patient-clinician interactions in a safe, structured and repeatable way. I often think of it like flight simulators for pilots. They train in a setting that feels real but without the risk. We want to create something similar for healthcare professionals, especially when it comes to difficult or emotionally complex conversations.
Communication is such an important part of patient care. Giving clinicians the opportunity to build those skills earlier in training can have a meaningful impact on how they care for patients later.
How does AI make these virtual patients more realistic?
There are three main ways AI is integrated into this project.
First, we analyze real patient-clinician encounters. We currently have more than 400 video-recorded visits across areas like endocrinology and oncology. Using machine learning, we identify emotional signals in tone, pacing, language and behavior. This helps us create an “emotional map” of interaction, showing when patients may be expressing fear, confusion, anxiety or other emotions.
Second, we refine the avatar’s behavior. That includes what it says, how it says it and how it moves. We use language models, speech synthesis and animation tools to synchronize voice, facial expression and timing. The goal is to make the interaction feel as realistic and natural as possible.
Third, we use AI to evaluate learner performance. The system can help benchmark communication skills and provide structured feedback without requiring a human to manually review every interaction. That makes the training more scalable.
Who could benefit from this training, and what impact do you hope it will have?
This tool is designed for medical students, residents, nurses and other patient-facing clinicians. Ideally, it could be integrated into simulation centers where learners schedule time, complete the training session and receive feedback. Over time, it could even track progress.
Strong communication influences patient satisfaction, trust and adherence to treatment plans. Many clinicians develop these skills gradually through experience, but some conversations — like breaking bad news — are both common and difficult. Having a safe space to practice those scenarios can build confidence before facing them in real life.
At Mayo Clinic alone, hundreds of trainees participate in simulation-based learning each year. Even modest increases in access to communication practice could make a meaningful difference in preparedness and ultimately in patient experience.
How has philanthropic support shaped this work?
Becoming a Schulze Scholar has allowed me to move more quickly from concept to pilot — and to think not only about whether the idea works, but also about how it can be implemented in real training environments.
For me personally, this project brings together three areas I care deeply about: artificial intelligence, education and improving patient-centered care. Philanthropic support is helping turn an early idea into something with the potential to improve how clinicians are trained and, ultimately, how patients are cared for.