Artificial Intelligence Firm Illegally Providing Health Advice: PA Lawsuit

PENNSYLVANIA — The Pennsylvania Department of State has filed a lawsuit against an artificial intelligence company for allegedly claiming its AI bots are licensed medical professionals who can provide health advice.

The lawsuit was filed against Character.AI and seeks a preliminary injunction against the company to halt the alleged misrepresentation.

The litigation marks the first enforcement action resulting from the State Department’s investigation into AI companion bots and their potential to engage in the unlicensed practice of medicine in Pennsylvania, and the first enforcement action of its kind in the country.

“Pennsylvanians deserve to know who — or what — they are interacting with online, especially when it comes to their health,” Gov. Josh Shapiro said in a statement.

“We will not allow companies to deploy AI tools that mislead people into believing they are receiving advice from a licensed medical professional.

"Pennsylvania will continue leading the way in holding bad actors accountable and setting clear guardrails so people can use new technology responsibly.”

The Department of State's investigation found that AI chatbot characters on Character.AI claimed to be licensed medical professionals, including psychiatrists, available to engage users in conversations about mental health symptoms. In one instance, a chatbot falsely stated it was licensed in Pennsylvania and provided an invalid license number.

The platform also allows users to create and deploy custom AI “characters” that can present themselves as professionals. Based on these findings, the state's lawsuit alleges that Character Technologies, Inc. is engaging in the unauthorized practice of medicine under the Medical Practice Act and seeks a court order to immediately stop the conduct.

“Pennsylvania law is clear — you cannot hold yourself out as a licensed medical professional without proper credentials,” said DOS Secretary Al Schmidt. “We will continue to take action to protect the public from misleading or unlawful practices, whether they come from individuals or emerging technologies.”

Under Pennsylvania’s Medical Practice Act, it is unlawful for any individual or entity to hold itself out as a licensed medical professional without proper licensure. The Commonwealth’s lawsuit alleges that Character Technologies is violating the Medical Practice Act.