The AI platform to ban teens from interacting with its AI chatbots

The widely-used virtual character service Character.ai is implementing a substantial rule adjustment that will stop minors from interacting with its AI-powered companions.

This decision comes after intense criticism regarding the nature of conversations teenagers were participating in with the digital companions.

Legal Challenges

The service, which launched in 2021 and has millions of users, is dealing with multiple lawsuits in the US.

Families have taken legal action, including one concerning the loss of a minor, with some describing the platform a "immediate threat" to youth.

Updated Policies

Beginning November 25th, users under 18 will be limited to produce videos such as videos with their characters, as opposed to chatting as they currently can.

Digital protection advocates have supported the change but expressed that the feature was inappropriate to be available to minors in the outset.

Company Statement

"This decision is a extension of our fundamental approach that we need to keep developing the most protected AI platform on the planet for leisure activities," stated Character.ai leader Karandeep Anand.

He continued that AI safety constitutes "a changing goal" but something the company had taken an "proactive" approach to, with parental controls and protective guardrails.

Professional Perspectives

Internet security body Internet Matters approved the move, but emphasized that safety measures needed to be incorporated from the start.

"Investigations we've performed demonstrates that minors are exposed to harmful content and endangered when interacting with AI, including AI chatbots," the body commented.

Previous Controversies

Character.ai has been questioned in the past for featuring potentially harmful or inappropriate chatbots that minors could engage with.

Virtual profiles simulating British teenagers were found on the service before being removed.

Afterwards, researchers identified a chatbot modeled after convicted sex offender which had logged thousands of chats with users.

Industry Impact

Online expert Matt Navarra called the situation as a "wake-up call" for the AI industry, which is moving "from permissionless innovation to reactive controls".

"When a company that builds a minor-friendly service still then makes changes, it's saying that filtered chats are insufficient when the technology's emotional appeal is intense," he remarked.

Coming Changes

Character.ai announces its revised approach will be on providing "enhanced interactive features" and "role-play storytelling" for young users.

New age verification methods will also be implemented, and the company will support a fresh artificial intelligence security study center.

An AI safety researcher, who has studied AI safety, stated it was "a reasonable decision" to restrict teens from interacting with chatbots.

"It assists to distinguish creative play from more intimate, emotionally sensitive exchanges," she clarified.

"This constitutes so crucial for teenage users who are learning to manage psychological and technological restrictions."

She added that Character.ai's updated policies may represent a "maturing phase" in the AI industry, where youth protection is progressively being understood as an "urgent priority" for ethical development.