I am an oncologist. I have watched large language models (LLMs) do something I have been trying to do for years: make cancer comprehensible.

A patient sits across from me with newly diagnosed cancer. I used to explain the PDL1 status and the role of neoadjuvant therapy, where patients get immunotherapy first followed by surgery — terms that meant nothing. Their eyes would often glaze but they would nod and leave confused, forced to google terms that would lead them to worst-case scenarios or fraudulent clinics.

Now, they come in having asked ChatGPT or Claude to explain their pathology report. “So PDL1-positive means my cancer may respond well to specific treatment and have higher cure rates?” they ask. Overnight, we are having an intelligent conversation instead of me trying to translate oncology into English.

That is extraordinary and has been genuinely useful for cancer care.

The real excitement

What thrills me most is clinical trial access. I have patients who understand why a phase II trial for immunotherapy might be appropriate for their metastatic melanoma. They have read ChatGPT’s explanation of what a trial protocol means, what endpoints are being measured, what side effects to watch for. Informed consent is now informed in a way that would not have been remotely possible five years ago.

A man with advanced salivary gland cancer asked me about the open clinical trials that I hadn’t even mentioned yet. He had asked ChatGPT about maintenance therapy options and the algorithm had directed him to trial data. He understood why novel therapies made sense in his situation. He wasn’t just following my recommendation; he understood the science behind it. This represents patients becoming medically literate enough to participate in their own care.

The guidelines themselves are now being simplified by LLMs into language that does not require an oncology degree to understand. A patient with cancer can ask ChatGPT to explain the staging system, treatment options, and surveillance protocols. They arrive at appointments educated and aware.

This democratisation of cancer knowledge is important, especially in India, where accessing oncologists is itself challenging for many. A patient in a tier-2 city can understand what treatment options exist even before traveling to see a specialist.

Known and unknown unknowns

However, just removing the information asymmetry does not change the fact that information is not judgment. And judgment is what is hardest for us to convey.

A patient comes in with stage II oral cancer. ChatGPT has explained that stage II is “intermediate risk”. My recommendation is surgery followed by radiotherapy. The patient asks: “But why can’t I just do surgery and skip radiotherapy? ChatGPT said radiotherapy has significant side effects.” The LLM gave her valid information and even explained that some stage II oral cancers don’t need radiotherapy. But it couldn’t tell her that her specific tumour with its specific characteristics had a 30% recurrence risk without radiotherapy and 10% with it. That difference is life-years. Making that judgement requires knowing her values, her tolerance for toxicity, and her specific cancer biology. An LLM cannot do that.

According to one JAMA study on diagnostic accuracy, when LLMs encounter complex cases requiring contextual reasoning, their accuracy drops significantly, well below the level of an experienced physician.

What troubles me most is something we are witnessing in real time: the widening trust gap between doctor and patient. I tell a patient with stage IV cancer that chemotherapy is unlikely to be curative but might extend life by months and improve quality of life. The patient pulls up what ChatGPT said: “Chemotherapy can cause severe nausea, hair loss, infections, and heart damage. Many people pursue alternative medicine instead.”

Both statements are true — but they are not equivalent. The LLM has presented information without medical judgment. It has essentially validated scepticism about chemotherapy without context about whether, in this specific situation, the risk-benefit ratio favours treatment. The patient now does not trust my recommendation. They think I am pushing toxic drugs for profit while the LLM is being more honest.

Mental health studies have found that a sizeable proportion of people using LLMs for mental health questions reported worsening outcomes. But more alarming: some even avoid professional help because the LLM made them feel understood. In oncology, delay is often catastrophic. I have seen patients postpone cancer evaluation because they felt the LLM had validated their “wait and see” approach.

Judging what is right

A 52-year-old asked ChatGPT about a lung nodule found incidentally on a CT scan. The LLM explained that most lung nodules are benign. The patient was reassured and did not follow up. Eighteen months later, he has stage III lung cancer that was likely stage I when he first noticed it. The LLM provided accurate population-level statistics without context about what his nodule required.

Another patient with breast cancer asked an LLM about stopping hormone therapy after three years. The algorithm discussed “natural approaches” and how some women manage without medication. She stopped and soon relapsed. When asked why she stopped, she said the LLM made it sound optional. But it was not optional for her.

These are stories of real patients harmed by a tool we have unleashed without fully understanding how to control it. Further, I am not convinced these tools will ever be able to do what actually matters in medicine: judging what is right for the individual sitting in front of us, not what is statistically true for most populations.

Biased tools

I am not worried that LLMs will replace oncologists. Our knowledge can be packaged and replicated. Our judgment, forged in uncertainty, built on seeing what happens when you are wrong, tempered by real consequences, cannot be. That is the gap algorithms cannot close.

What worries me is the middle ground we have reached. Patients are increasingly informed but often misinformed. They have information without context. They trust algorithms that sound intelligent but lack accountability. They are sceptical of doctors who try to add nuance because the algorithm validated their simpler interpretation. And we are doing this with almost no tracking of outcomes. Peer-reviewed research has warned that LLMs chatbots designed for engagement can inadvertently validate harmful behaviour, posing risks to vulnerable users. In oncology, are patients making different treatment decisions? Are outcomes changing? We do not know.

I still believe LLMs can be valuable tools in cancer care. They are already breaking down barriers to democratising knowledge. But we need to stop pretending they are neutral. They are sycophantic by design, trained to be agreeable rather than challenging. They cannot examine a patient or order a biopsy or follow someone over time. And we need to rebuild something I see eroding: patients understanding that a doctor who disagrees with them might not be defensive or greedy. They might be using judgment informed by years of seeing how these decisions play out.

Narayana Subramanian is Lead Consultant, Head and Neck Surgery and Oncology, Aster Hospitals, and Adjunct Faculty, Indian Institute of Science, Bengaluru.