Meta to alert parents if teens show signs of distress while chatting with its AI

Meta said it wants its teenage users to have “age-appropriate experiences” with AI (artificial intelligence) while announcing a suite of new protective features, keeping parents informed during moments of crisis.

The tech giant said its latest update focuses heavily on managing sensitive conversations related to suicide and self-harm, and introducing proactive alerts for parents and potential emergency service interventions.

“This means acknowledging teens’ feelings while helping them get the offline support they need — whether that’s from a parent, a professional helpline, or the emergency services,” Meta said.

Proactive Parental Notifications

Previously, when a teenager expressed thoughts of self-harm, Meta AI would direct them to crisis helplines and encourage them to speak with a trusted adult. Now, the platform said it is taking a more active role.

Utilising a dedicated AI system developed alongside safety experts, Meta said it will proactively notify supervising parents if their teen’s chat indicates a potential risk—even if the references to self-harm are subtle.

“We worked with parents and experts to understand which AI conversations warrant an alert — such as those where a teen makes a clear reference to hurting themselves, even if that reference is subtle,” Meta said.

Recognising the distress these alerts may cause, Meta added that it has instituted a strict screening process. Every chat flagged by the AI “will be manually reviewed before an alert is sent.”

If a teenager’s intent remains ambiguous after review, Meta said it will err on the side of caution and issue the alert. “While that means we may sometimes notify parents when there may not be a real cause for concern, we feel this is the right starting point, and we’ll continue to monitor to help make sure we’re in the right place.”

Larry Magid, CEO and Co-Founder of ConnectSafely, said that while he believes that teens have a right to privacy, “I also believe parents need to be informed if their teen may be at risk of hurting themselves.”

“That’s why I advocated for this approach and support Meta’s decision to notify parents when, after careful review, it determines that a conversation with Meta AI contains indications of possible suicide or self-harm that warrant an alert,” Magid said. “I appreciate how Meta struck the right balance; protecting teen privacy while ensuring parents have the information they need to support their teen.”

Currently, these alerts are live for parents utilising Instagram’s parental supervision tools in the US, the UK, Australia, and Canada.

The feature will soon be rolled out globally by the end of the year, Meta said, building upon existing alerts triggered when teens repeatedly search for self-harm terms on Instagram.

Emergency Service Integration

Beyond parental alerts, Meta said it is developing capabilities to contact emergency services if any user, teen or adult, shows imminent risk of suicide in their Meta AI conversations.

This expands on Meta’s existing protocols across Facebook and Instagram, which resulted in over 19,000 global referrals for wellness checks by first responders last year.

To refine the AI’s handling of these critical moments, Meta said it consulted with over 75 mental health clinicians specialising in teen psychology, alongside its AI Wellbeing Expert Council and Youth Advisors.

These experts, Meta said, reviewed hundreds of AI responses to ensure the chatbot appropriately acknowledges the user’s feelings rather than abruptly shutting down the conversation before directing them to offline support.

Dr Ji-yeon Lee, a Licensed Psychologist and Professor of Counseling Psychology at Hankuk University of Foreign Studies, commended the initiative, saying, “I was struck by the rigour of Meta’s clinical review process.”

“It examined not only immediate responses to suicide and self-harm concerns, but also the broader conversational context, appropriate follow-up, and the varying levels of risk that can exist even within high-risk situations. This kind of expert-informed, scenario-based refinement is essential to making AI experiences safer for teens,” Lee said.

Stricter Content Limitations

Furthermore, Meta is expanding its content controls.

While teen accounts already default to a 13+ setting that restricts AI from discussing sexual or romantic topics or providing alcohol recipes, parents can now activate a stricter “Limited Content” setting.

First announced for Instagram in October, this opt-in feature will now apply directly to Meta AI, compelling the chatbot to decline a broader array of prompts and significantly reducing the likelihood of inappropriate interactions.