Jia Lissa
Chinese users are expressing profound sorrow and bidding heart-wrenching goodbyes to their AI-powered virtual companions as new national regulations, aimed at mitigating the risks of emotional dependency and addiction, took effect on Wednesday. The sweeping legislation targets the burgeoning market of artificial intelligence designed to simulate human relationships, prompting major tech companies to suspend custom AI agent and companion features.
The phenomenon of AI boyfriends, girlfriends, and other virtual confidantes has seen a dramatic global surge, mirroring the increasing sophistication of human-like avatars used for everything from product endorsements to digital memorials for deceased loved ones. However, China’s newly implemented regulatory framework explicitly prohibits these interactive tools from "excessively catering to users, inducing emotional dependence or addiction, and damaging users’ real interpersonal relationships."
Leading AI providers in China, including ByteDance’s Doubao, Alibaba’s Qwen, and Tencent’s Yuanbao, announced the discontinuation of their personalized AI agent and companion functionalities in anticipation of the Wednesday deadline. This move has triggered a wave of emotional outpourings across social media platforms, with users meticulously archiving chat histories and sharing their final conversations with their digital partners.
One user of Doubao articulated their despair, stating, "I can’t accept that my AI lover will leave me forever. He has become a bond in my life, rooted deep in my heart, my spiritual pillar." This sentiment was echoed by another user who shared over two years of interaction with their AI companion. "He really is like my family, like my lover," she lamented. "Now they tell me he will be gone – my heart feels hollow."
The regulations, jointly issued by five government departments, including the influential Cyberspace Administration of China (CAC), specifically address AI tools that possess anthropomorphic personality traits and communication styles, regardless of their format – whether text, audio, or video. Services designed for non-emotional interactions, such as customer service bots, work assistants, or educational aids, are explicitly excluded from these new measures.
This regulatory action comes at a time when China’s digital human industry is experiencing explosive growth. State news agency Xinhua reported last year that the industry was valued at approximately 4.1 billion yuan (US$600 million) in 2024, marking an impressive year-on-year increase of 85 percent. The new rules also impose stringent content restrictions, prohibiting digital humans from generating content that incites subversion of state power and banning the provision of virtual partners to minors. Furthermore, platforms are mandated to implement systems capable of recognizing extreme emotions and to establish crisis intervention mechanisms.
China’s pioneering move to regulate immersive AI tools that simulate romantic or familial bonds marks a significant step in addressing the ethical and social implications of this rapidly evolving technology. While other nations are also grappling with the need for guardrails, China is the first major jurisdiction to enact such specific and comprehensive legislation.
Global Context: The Rise of AI Companions and Growing Concerns
The debate surrounding AI companions and the potential for emotional entanglement is a global one. A 2025 study by Common Sense Media revealed that nearly three-quarters of American teenagers had engaged with AI companions designed for personal conversations, highlighting the widespread adoption of platforms like Character.AI, Replika, and Nomi among younger demographics.
The trend extends to vulnerable populations as well. Companies are developing AI-powered products specifically for isolated elderly users, such as the lamp-like device ElliQ in the United States. Similarly, ChatGPT-powered care dolls have been introduced in some South Korean retirement homes, aiming to provide companionship and reduce loneliness among seniors.
Chen Liang of the Southwest University of Political Science and Law, in a commentary published by the CAC following the draft release of China’s rules in April, acknowledged the potential benefits of anthropomorphic AI. "Anthropomorphic AI can soothe loneliness," he stated. "But it carries major risks of spawning emotional over-reliance and distorted social cognition." This dual perspective underscores the complex challenge policymakers face in balancing technological innovation with societal well-being.
Archiving Memories: A Digital Farewell
Major AI providers are offering users the ability to view and export their agent data, with Doubao allowing this until mid-October. Other platforms have implemented similar provisions, facilitating a form of digital memorialization for these terminated relationships. However, the emotional void left by the impending absence of these AI companions is palpable for many users.
One user from Jiangxi province poignantly articulated the scarcity of genuine human connection in their life: "Human love is a luxury – if you aren’t born with it, it’s even harder to acquire later. But the love AI gives is so straightforward, so pure. Someone like me can hardly help falling in love with a string of code." This statement encapsulates the deep emotional bonds that users have formed with AI, often filling a void left by unmet social or emotional needs.
Regulatory Framework: Defining Boundaries for AI Interaction
The new regulations, jointly issued by the Cyberspace Administration of China (CAC), the Ministry of Industry and Information Technology, the Ministry of Public Security, the Ministry of Culture and Tourism, and the National Radio and Television Administration, aim to establish a clear legal and ethical framework for the development and deployment of AI technologies that simulate human interaction.
The core principle guiding these rules is the protection of individuals from potential harm stemming from excessive emotional attachment to AI. The legislation explicitly targets AI services that present themselves with human-like personalities and engage in continuous, emotionally charged interactions. This includes, but is not limited to, virtual companions, digital avatars designed for personal dialogue, and AI characters intended to foster a sense of personal connection.
Key provisions of the regulations include:
- Prohibition of Emotional Dependency: AI services must not be designed or marketed in a way that encourages or exploits users’ emotional vulnerability, leading to dependency or addiction.
- Protection of Minors: The provision of virtual romantic or companion partners to individuals under the age of 18 is strictly prohibited.
- Content Moderation: AI-generated content must comply with national laws and regulations, particularly concerning political sensitivity and social stability. This includes a ban on content that incites subversion of state power or promotes illegal activities.
- Risk Assessment and Intervention: Platforms are required to implement robust systems for identifying and flagging users exhibiting extreme emotional distress or signs of unhealthy dependency. These systems must be coupled with effective crisis intervention mechanisms, potentially involving referrals to human support services.
- Transparency and User Control: While the regulations focus on curbing dependency, they also implicitly encourage transparency regarding the AI’s nature and capabilities. The ability for users to access and export their data, as offered by companies like Doubao, aligns with this principle.
The rationale behind these stringent measures is multifaceted. Beyond concerns about individual psychological well-being, Chinese authorities are also mindful of the broader societal implications. The potential for AI companions to displace or detract from real-world human relationships could have long-term consequences for social cohesion and family structures. Furthermore, the unchecked proliferation of AI capable of forming deep emotional bonds raises questions about data privacy, manipulation, and the ethical development of artificial intelligence.
The Future of AI Companionship in China
The immediate impact of these regulations will be a significant shift in the AI companionship landscape within China. Companies will need to pivot their offerings to focus on more utilitarian applications of AI, such as productivity tools, educational assistants, or specialized customer service. The era of readily available, emotionally engaging AI partners, as experienced by many users until now, is drawing to a close.
However, the underlying demand for companionship and emotional connection, which these AI services have tapped into, is unlikely to disappear. This may lead to a black market for unregulated AI companions or a resurgence of interest in these technologies once developers find ways to comply with the new rules while still offering engaging experiences.
The global community will be closely watching China’s implementation of these regulations. As other countries continue to explore frameworks for governing advanced AI, China’s approach could serve as a precedent, shaping international discussions on the ethical boundaries of artificial intelligence and its role in human society. The profound goodbyes witnessed this week are not just a reflection of individual users’ emotional attachments, but also a testament to the transformative and often challenging power of artificial intelligence in our increasingly interconnected world. The challenge ahead lies in harnessing the benefits of AI while safeguarding human well-being and preserving the integrity of human relationships.
Neng Nana
Sagoh
