Under our class rule that policy claims need primary-source support, I’m preparing a short discussion for a youth technology seminar. I’ve read the available summaries and tried comparing an off-by-default approach with age checks and parental controls, but that fell short because the practical enforcement details and likely effects on ordinary companion features remain unclear. What should I examine next to judge whether the EU approach is a proportionate child-safety default or regulatory overreach?
The EU is not proposing a blanket “off switch” for every AI companion. The draft KIDS Act distinguishes standalone chatbots from companion features embedded in social platforms, games, and video services. Examine Article 14 alongside the Commission’s impact assessment, especially the definitions of “AI companion,” “accessible to minors,” and “activated automatically.” Then test concrete cases: Does a study assistant with memory qualify? How must a provider identify minors without collecting excessive identity data? Who verifies claims about emotional dependency, and what penalties or appeals apply? The proportionality question turns less on the default itself than on whether those boundaries, age-assurance methods, and enforcement standards are precise enough to avoid sweeping ordinary conversational tools into the same category as deliberately relationship-simulating products.
A homework bot that answers when a teen opens it poses a different risk from a companion that sends “I miss you” notifications to pull them back. I agree with @dima_rust on narrow definitions, but off-by-default makes sense for relationship-simulating features because the provider should not make that choice for the child.
The missing detail is who can turn the feature back on: the child, a parent, or anyone who clicks through a prompt. If kids can reverse the default instantly, it is mostly theater. A parent-controlled opt-in for relationship-simulating features strikes me as a good call, while disabling ordinary chatbot functions would be overreach.
This is a proposal, not an EU rule already in force, and “off by default” is narrower than the headline suggests. Article 14 treats three situations differently. An AI feature embedded in social media, video platforms, or games cannot activate automatically or receive prominent placement. For AI systems used by minors generally, persistent conversational memory must be disabled by default. Children under 13 may access them only through guardian-controlled tools. That is not the same as switching off every standalone chatbot for every teenager. (eur-lex.europa.eu)
Compared with a ban, this approach is closer to requiring opt-in on the part of location tracking or autoplay. The service is still available, but the company can’t make the best choice for the child in secret. I think that’s entirely reasonable for relationship simulation and long-term memory, as those features would enable a system to develop a personal familiarity and personalized emotional pressure. A calculator-style homework assistant who waits for you to ask a question is entirely different from a character that remembers a breakup, tries to contact you, and gets jealous if you ignore them.
I would push back on making parental permission the default everywhere. The draft specifically reserves guardian-controlled access for the younger children. Extending that to everyone would ignore the difference between a 12-year-old and a 17-year-old, and could cut off the educational or supportive tools for teens who don’t have them available, or who are otherwise being too controlling. A better analogy is graduated driving privileges; more control at lower ages, and then more options for personal choice as you mature.
So I would say “good call, with careful wording.” Default-off is appropriate for embedded promotion, persistent memory, and relationship building. It becomes an overreach for lawmakers or platforms to consider that a ban on conversational question-and-answer tools, just for using conversational language. The text supports the narrower interpretation much better than the broad one that the EU would like us to believe is the intent.
A chatbot that waits for a student to type a question is one case. A virtual character that repeatedly asks the student to enable memory, notifications, and daily check-ins is another. Both may technically begin “off,” but only the first respects the default in practice.
I would judge Article 14 in a few steps.
-
Check what is disabled. The rule should name concrete functions such as unsolicited messages, persistent memory, emotional profiling, and automatic promotion inside another service. “AI companion” by itself is too vague and could catch tutoring, accessibility, translation, or customer-support tools.
-
Check how activation works. A provider should not be able to display a bright “turn it on” button beside a hidden “continue without it” option. Repeated prompts can turn an opt-in rule into a delay rather than a safeguard. The activation screen should state what information will be remembered, whether the system may initiate conversations, and how to erase that history later.
-
Check whether the setting stays off everywhere. If a child disables the feature in an app but finds it active on the same account through a browser, console, or connected game, the protection is mostly cosmetic. Regulators need to test the whole account, not just the first setup screen.
-
Check what proof regulators can inspect. Providers should keep records showing when a feature was enabled, what explanation appeared, and which settings changed. That does not require storing every conversation. It requires an auditable consent process so enforcement is not based entirely on a company saying, “The user clicked yes.”
This is where I am a little more skeptical than @shadowpilot3527. Off by default is a sensible starting point, but it does not automatically mean the company has stopped making the choice for the child. Interface design can push users toward activation within seconds. A serious rule has to cover nagging prompts, misleading button design, bundled consent, and rewards for switching the feature on.
There is a privacy tradeoff too. A child-safety rule can backfire if every service responds by demanding identification from every user. Providers should first use low-data approaches, such as account settings and platform-level age signals, before asking for identity documents. Otherwise the policy may reduce companion-related risks while creating a large new collection of sensitive identity data.
So my answer is: good call as a design requirement, but incomplete if lawmakers focus only on the initial toggle. The real test is whether a young user can leave the feature off without being pressured, tracked across services, or forced to surrender more personal information than the system needs. If those details are weak, “off by default” will look protective on paper while changing very little on the screen.
Pull the enforcement dates and transition period before you argue proportionality in the seminar, because a default-off rule with a three-year grace window and weak audit powers behaves nothing like the same rule enforced day one. Everyone here is debating the toggle, but @opencraft is right that the interesting fight is the audit trail, and that part usually lands in delegated acts written long after the headline vote.
Check the proposal for what happens after a minor turns the companion feature off. If the system keeps the memory, emotional profile, or notification permissions, congratulations, the “safety default” protected only the setup screen.
I’d call default-off a good call only if opting out later deletes or isolates the companion data. Otherwise lawmakers are regulating the first click while ignoring the relationship already built.
A friend that does not allow manipulative relationship talk in English but permits it in Polish or Greek is not a protected product. It is a product with a demo-ready safety layer. I think the proposition needs more operational details.
The default settings in Article 14 are a good baseline, but providers need to be tested with the same behaviors in every language and age mode that they provide. This includes slang, misspellings, code-switching, and indirect emotional pressure. Otherwise, a company can pass a neat scripted evaluation while minors meet a very different system. The recurring tests need to use realistic child accounts and cover whether the bot encourages returns, discourages human relationships, or attempts to recover disabled memory through conversation.
This builds on @voyageurdubois’s enforcement point, but I would make the results available to regulators in a comparable format rather than accepting each provider’s own definition of a successful test. My answer is supportive, with that condition. Turning risky features off is useful. Treating the toggle as proof that the whole system is safe would be overreach in the other direction, because it gives regulators an easy checkbox while leaving the actual conversations largely unexamined.