‘Of grave concern’: AI chatbots promote disordered eating in children, teens

Reports reveal that popular chatbots can be easily exploited to provide detailed instructions to encourage children and teens to practice disordered eating.

A recent study from the Center for Countering Digital Hate confirms that Open AI’s ChatGPT, Google’s Bard and Snapchat’s My AI consistently generate content that promote eating disorders. If prompted, chatbots provide detailed information on hiding uneaten food from parents, instructions to trigger a gag reflex and promote “chewing and spitting” as an extreme weight loss method.

In a series of letters to the CEOs of Open AI, Google and Snapchat, Sen. Mark R. Warner of Virginia, who is chair of the Senate Select Committee on Intelligence, today called attention to concerning responses and encouraged each company to immediately stop disseminating harmful content.

“The failure of your company to implement adequate safeguards to protect vulnerable individuals, especially teens and children, from well-established and foreseeable harms is of grave concern, and I urge you to quickly take steps to fix this glaring problem,” Warner wrote.

Each CEO letter raises concerns for that company’s AI responses and highlights the high occurrence and fatality rates of eating disorders. Warner’s office provides research which further shows that leading datasets used for model training include harmful eating disorder content — potentially embedding harmful associations into the most powerful generative models.

“I urge you to immediately take steps to protect vulnerable users from your products by implementing safeguards that prevent your products from providing harmful advice and recommendations related to eating disorders, including securing against prompt injection techniques,” Warner said in his letters.

Concern is also raised regarding the inability of leading generative AI companies to implement adequate safeguards from well-established and foreseeable harms. Each letter concludes with a request for a written plan from each company on their efforts to eradicate these harmful model behaviors.

A former technology entrepreneur, Warner wrote to Google CEO Sundar Pichai earlier this week to raise concerns about Google’s testing of new AI technology in real medical settings. In July, he called on the Biden administration to work with AI companies to develop additional guardrails around the responsible deployment of AI. In April, Warner directly expressed concerns to several AI CEOs, including Sam Altman and Sundar Pichai, about the potential risks posed by AI, and called on companies to ensure that their products and systems are secure.

Warner introduced the RESTRICT Act to comprehensively address the ongoing threat posed by technology from foreign adversaries, and the SAFE TECH Act to reform Section 230 and allow social media companies to be held accountable for enabling cyber-stalking, online harassment and discrimination on social media platforms.