How AI Could Transform Medical Training

Training large groups of medical students, residents, and fellows has traditionally required educators to teach to the average learner and assess what is easiest to measure: knowledge recall.

AI creates opportunities to evaluate trainees’ capabilities and personalize their training in ways that were previously impossible.

For example, educators at Mount Sinai are using AI scribe transcripts to analyze trainees’ communication skills and provide targeted feedback. Similarly, a team at the University of Pennsylvania is evaluating clinical reasoning by analyzing conversations among internal medicine residents and their colleagues. As Verity Shaye, assistant dean for education at NYU, explained to me, medical educators have long struggled to assess communication and reasoning skills directly.

Procedural training may also change dramatically. Traditionally, competency has been assessed by the number of cases a trainee completes, rather than how well they perform them. Researchers at Stanford are using sensor technologies to quantify surgical technique and provide data-driven coaching.

Educators may gain new ways to track growth over time. At Kaiser Permanente East Bay, otolaryngologist Alexander Rivero is using AI to collect and aggregate ENT residents’ post-operative debriefs and other learning artifacts into longitudinal learning portfolios.

AI can also help connect classroom and clinical learning. Educators at NYU use AI to deliver “educational nudges,” such as automatically sending trainees key articles related to the patients they see on the wards.

Meanwhile, AI simulations can broaden trainees’ exposure to diverse populations and conditions they may not encounter in their own setting. A family medicine resident in North Dakota can now be exposed to an inner-city teenager living with sickle cell disease. An infectious disease fellow in Queens can encounter a farmer with tularemia.

More broadly, by freeing trainees from rote memorization and “scut” work that has long occupied their time and headspace, AI could allow them to spend more time with patients, refine their reasoning skills, and engage in more cognitively demanding tasks.

Yet many of these same tasks are how trainees have traditionally developed expertise.

The Risks of Training With AI

Educational researchers have long argued that expertise develops through effortful learning and deliberate practice.

Trainees who offload key tasks to AI—such as writing clinical notes, summarizing medical records, synthesizing medical literature, interviewing patients, and formulating differential diagnoses—may fail to develop key capabilities (“never-skilling”), lose previously developed skills (“deskilling”), and develop incorrect practices (“mis-skilling”).

Research on cognitive offloading suggests these concerns are real. When we habitually rely on GPS, our spatial memory diminishes. When we stop writing notes by hand, we retain less information. Outsourcing cognitive work can change how we learn, what we remember, and how we think critically.

Yet concerns about technology eroding physicians’ skills are hardly new.

When I was a resident, attendings grumbled that we couldn’t palpate a spleen or make blood smears. Their teachers had complained that they did not present cases from memory or actually live in the hospital. And earlier generations of doctors even opposed learning from medical textbooks, arguing that conjuring knowledge from memory forced deeper reflection.

It’s also possible that these risks are overstated. Fletcher Bell, internal medicine chief resident at UCSF, told me residents who spend up to 80 hours in the hospital each week have ample learning opportunities.

At the same time, not all deskilling is bad. Some may be necessary to make room for learning essential skills. Do all medicine residents need to read ECGs if a computer can do it faster and more accurately than even seasoned cardiologists? Cornelius James, a primary care physician and medical education researcher at the University of Michigan, explains, “We risk losing the trust of learners if we force them to learn things they no longer need to learn.”

Still, AI may represent a different kind of challenge as it performs many of the cognitive tasks through which clinicians have traditionally learned to think, such as critically appraising research studies or developing treatment plans.

The challenge is determining which activities can be offloaded to AI and which remain essential.

Still, We Know Relatively Little

Yet we don’t fully understand any of this. We don’t know how often today’s trainees use various AI tools, how they use them, or how they feel about using them.

At Washington University, John Davis, an internal medicine resident, describes a small but vocal group of fervent users who claim it’s borderline malpractice not to use AI, alongside another group that feels these tools can’t be trusted at all. Most of his colleagues fall somewhere in between.

While it’s natural to assume that young people are enthusiastic about AI, surveys suggest otherwise. Davis describes widespread “social posturing,” whereby some colleagues signal that they arrived at their answers or completed their work independently.

We also know surprisingly little about how effective these tools are in the real world, or the extent to which they cause important skills to atrophy or never develop.

Despite countless LinkedIn posts about deskilling or never-skilling, the evidence largely rests on a single, limited study of Polish gastroenterologists using adenoma detection software, hardly enough to draw broad conclusions about AI and medical training.

Most of all, we are unsure what future we are preparing trainees for.

Slogans like “doctors who work with AI will replace those who do not” ring hollow. The long-held belief that doctors plus AI are better than either alone no longer appears universally true. We wave our hands and talk about judgment and taste, yet both are difficult to define and perhaps even harder to cultivate.

We know much less than we think.

Responding Before We Have All the Answers

Medical education has been slow to respond, in part because of this uncertainty, and in part because many educators feel unqualified to teach AI. Still, as the technology, clinical workflows, and expectations race ahead, we cannot wait for perfect evidence.

Laurah Turner, an anthropologist and associate dean at the University of Cincinnati, argues that medical educators are rushing to apply their familiar toolkit—policies, competencies, and curricula—to a phenomenon they do not yet fully understand.

On the one hand, these responses are necessary. Trainees should develop AI competence, including the ability to critically appraise AI outputs, just as we’ve long taught evidence-based medicine.

Still, these responses are not enough. They assume we know where this is heading and how to get there. We do not.

Rather than trying to predict the future, medical education must become more adaptive. Rather than making assumptions, educators must study how trainees use these tools, what effects they have, and which elements of the traditional apprenticeship model remain essential.

For Turner, this means partnering with learners to co-create new training approaches and embracing newer models such as design-based research and implementation science.

To Christy Boscardin, a professor who directs UCSF’s AI and Medical Education team, this means helping learners tap into their intrinsic motivation and encouraging them to follow up on the outcomes of their decisions made with or without AI. Over time, this could help them recognize when to slow down, ask questions, and think more critically.

In other words, rather than only teaching physicians to use AI, medical training itself must become more adaptive.

The Next Chapter In Medicine’s Evolution

No one knows exactly what medicine will look like in the years ahead.

Physicians have faced similar uncertainty before, and new technologies have repeatedly changed how doctors practice. Yet this moment feels different because AI is forcing us to reconsider both what physicians do and how they learn.

In a time when so many are disenchanted with medicine, Kimberly Lomis, a surgeon and vice president of medical education innovations at the American Medical Association, sees opportunity. “AI is a disruption we should take advantage of,” she told me. “This is a chance to realign around our values."

As before, when machines can increasingly perform tasks that yesterday’s trainees learned by doing, the challenge is deciding what every physician still needs to learn, what no longer matters, and what new capabilities they must develop.

We do not yet know the answer. But the physicians we train today will help determine it.

Acknowledgments: I thank the following people for discussing this topic with me: Fletcher Bell (UCSF), Christy Boscardin (UCSF), Todd Cassese (Cornell University), John Davis (Washington University), Cornelius James (University of Michigan), Anisha Kumar (Stonybrook University), Kimberly Lomis (American Medical Association), Alexander Rivero (Kaiser Permanente East Bay), Verity Shaye (NYU), Marc Triola (NYU), Laurah Turner (University of Cincinnati).