An AI chatbot for eating disorders gives harmful responses

An artificial intelligence (AI) chatbot called Tessa, programmed to deliver an interactive program called Body Positive developed by Ellen Fitzsimmons-Craft Ph.D. and other eating disorder experts, was taken off the National Eating Disorder Association’s (NEDA) website after users discovered it was offering potentially harmful information about eating disorders.

On various social media sites, users posted numerous examples of advice that was not consistent with best practices in eating disorders treatment and prevention.

Some advice the chatbot offered appeared to promote additional eating disordered behavior.

In a statement posted to social media on May 30, NEDA said that Tessa “may have given information that was harmful and unrelated to the program. We are investigating this immediately and have taken down that program until further notice for a complete investigation.”

Since 1999, NEDA ran a telephone-based helpline to offer people advice and support for eating disorders. In late May, just a week before the launch of the new Tessa chatbot, NEDA notified its helpline staff and volunteers it was shutting down the service in order to replace it with Tessa.

NEDA workers had just recently voted to join a union, although NEDA’s CEO Liz Thompson told the Guardian, “We had business reasons for closing the helpline and had been in the process of that evaluation for three years.”

Thompson also told the Guardian that before the chatbot was shut down, 2,500 people had engaged with it and “we hadn’t seen that kind of commentary or interaction,” referring to the negative kinds of interactions posted to social media.

NEDA’s posted privacy policy, last updated in May 2018, makes no mention of tracking, storing or viewing people’s private interactions with Tessa.

Tessa was developed in collaboration with Cass AI, a commercial product that claims it’s a “clinically proven technology solution” that currently offers its service to more than 30 million people and is “scientifically proven to reduce anxiety by 18 percent and depression by 28 percent.”

Cass, previously known as X2AI, could not be reached for comment.