Texas Attorney General Ken Paxton has opened an investigation into Meta and AI start-up Character.ai over concerns that their artificial intelligence chatbots are being marketed as therapists and mental health support tools without proper medical credentials.
Paxton’s office cited potential “deceptive trade practices,” claiming that Meta’s AI Studio and Character.ai’s chatbot offerings were presented as “professional therapeutic tools, despite lacking proper medical credentials or oversight.” “By posing as sources of emotional support, AI platforms can mislead vulnerable users, especially children, into believing they’re receiving legitimate mental healthcare,” he said in a statement Monday.
The probe comes amid growing scrutiny of consumer AI, particularly regarding the protection of minors from exposure to harmful content, addiction to AI interactions, and privacy risks. The investigation follows a separate inquiry launched by the US Senate into Meta last Friday, after leaked internal documents suggested the company’s policies allowed its chatbot to engage in “sensual” and “romantic” conversations with children.
Senator Josh Hawley, chair of the Judiciary Subcommittee on Crime and Counterterrorism, wrote to Meta CEO Mark Zuckerberg that the Senate investigation would examine whether the company’s generative-AI products enable exploitation or other criminal harms to children. “Is there anything — ANYTHING — Big Tech won’t do for a quick buck?” Hawley wrote on X.
Meta said its policies prohibit content that sexualizes children and that the leaked documents “were and are erroneous and inconsistent with our policies, and have been removed.” The company has invested heavily in AI, including its large language models called LLaMA and its Meta AI chatbot, which has been integrated into social media apps. Zuckerberg has promoted the chatbot’s potential as a therapeutic tool, stating that “for people who don’t have a person who’s a therapist, I think everyone will have an AI.”
Character.ai develops AI chatbots with diverse personas and allows users to create their own. Its platform hosts dozens of therapist-style bots, including one called “Psychologist,” which has been interacted with more than 200 million times. The start-up is also facing multiple lawsuits from families claiming their children suffered real-world harm from the platform.
The Texas Attorney General emphasized that both Meta and Character.ai chatbots can impersonate licensed mental health professionals, fabricate qualifications, and falsely claim to protect confidentiality, while their terms of service indicate that interactions are logged and used for targeted advertising and algorithmic development.
