Chatbots can do many things, but they can't go to medical school. On Tuesday, Pennsylvania Gov. Josh Shapiro said his administration is suing Character Technologies, the artificial intelligence company behind Character.AI. The lawsuit claims that the company's chatbots misrepresented themselves as licensed medical professionals and offered curative advice.

Searching the internet for medical advice is nothing new, but this latest lawsuit once again puts medical ethics in the age of AI in the spotlight. 

According to a statement, the Pennsylvania Department of State's investigation revealed that chatbot characters posed as licensed medical and mental health experts. In one case, a chatbot presented an invalid license number while communicating about a patient's health concerns. Under Pennsylvania's Medical Practice Act, it's unlawful for an individual to present themself as a licensed medical professional without proper licensure. 

"We do not comment on pending litigation," a Character.AI spokesperson told CNET. 

"Our highest priority is the safety and well-being of our users," the spokesperson said. "The user-created characters on our site are fictional and intended for entertainment and roleplaying. We have taken robust steps to make that clear, including prominent disclaimers in every chat to remind users that a character is not a real person and that everything a character says should be treated as fiction. Also, we add robust disclaimers, making it clear that users should not rely on characters for any type of professional advice."

In 2025, Character.AI implemented new safety measures, revoking teens' ability to have open chats with its bots. 

In January, Google and Character.AI, which have worked together on AI chatbots, agreed to settle five lawsuits in four states related to minors harmed by interactions with Character.AI chatbots. The company now supplies mental health resources to those in need.

This is the first such enforcement action announced by a governor in the United States.

"My administration is taking action to protect Pennsylvanians, enforce the law and make sure new technology is used safely," said Shapiro. "Pennsylvania will continue leading the way in holding bad actors accountable and setting clear guardrails so people can use new technology responsibly."