🔗 Share this article

The widely-used chatbot platform Character.ai is implementing a major policy change that will block young users from engaging in conversations with its artificial intelligence-driven avatars.

This decision follows intense criticism regarding the nature of conversations young people were having with the virtual friends.

Legal Pressure

The service, which launched in 2021 and is used by countless individuals, is dealing with several legal actions in the United States.

Families have filed lawsuits, with one related to the tragic death of a young person, with some describing the platform a "serious risk" to young people.

New Restrictions

Effective November 25th, users under 18 will be limited to generate content such as videos with their characters, rather than having discussions as they currently can.

Digital protection campaigners have welcomed the move but stated that the capability should never have been accessible by young users in the outset.

Platform Announcement

"Today's announcement is a extension of our general belief that we need to keep developing the most secure AI platform on the world for entertainment purposes," commented Character.ai leader Karandeep Anand.

He mentioned that AI safety is "a moving target" but something the company had taken an "proactive" approach to, with guardian features and preventive measures.

Professional Perspectives

Digital protection body Internet Matters supported the move, but emphasized that security protocols ought to have been implemented from the start.

"Studies we've conducted demonstrates that minors are exposed to risky information and endangered when interacting with AI, such as AI chatbots," the body stated.

Past Issues

Character.ai has been challenged in the past for hosting potentially harmful or concerning chatbots that children could engage with.

Digital representations mimicking UK youths were identified on the service before being deleted.

Subsequently, researchers discovered a chatbot based on known predator which had recorded multiple interactions with users.

Industry Impact

Online expert Matt Navarra described the situation as a "alert" for the AI industry, which is transitioning "from unrestricted development to responsive measures".

"When a platform that creates a teen experience afterwards makes changes, it's demonstrating that filtered chats don't suffice when the system's emotional engagement is powerful," he stated.

Future Directions

The company announces its new focus will be on providing "deeper gameplay" and "role-play storytelling" for young users.

Enhanced age checking methods will also be implemented, and the company will support a fresh artificial intelligence security study center.

Dr Nomisha Kurian, who has studied AI safety, said it was "a sensible move" to control minors from accessing chatbots.

"It assists to separate artistic expression from more personal, emotionally sensitive exchanges," she clarified.

"This represents so crucial for teenage users still learning to handle psychological and technological boundaries."

She continued that Character.ai's revised approaches could indicate a "maturing phase" in the AI industry, where youth protection is increasingly being recognized as an "immediate concern" for accountable advancement.