It seems ethically dubious that anyone should have access to AI chatbots, considering how easily they can warp someone’s perception of reality to such a degree that they cause harm to themselves and those around them. It seems flagrantly irresponsible to disguise such incredible mind-warping power in a cute and cuddly teddy bear for kids.

It’s all part of the ongoing debate encapsulated by a new essay in The Conversation, written by researchers from Curtin University and Deakin University in Australia, as ethicists debate AI-integrated toys like an AI-powered teddy bear named “ChattyBear.”

Are AI-Powered Teddy Bears for Kids Really That Dangerous?

Teddy bears can’t just be adorable things meant for hugging and squeezing anymore. They are being crammed with generative AI systems that promise “endless conversations,” as if that’s a plus rather than a terrifying minus, given that chatbots have a long and rich history of saying some pretty flagrantly atrocious stuff.

As the authors note, younger children struggle to distinguish between something that sounds alive and something that is alive. When a toy uses sycophantic language, as AI chatbots are designed specifically to do to maximize engagement through flattery, it can build intimacy and trust in software with a financial motive behind it.

If you are in any way turned off by the current world of infinite scrolling we live in, then you already know what a toy promising infinite chat with your child is headed. A world of nonsense slop that isn’t nourishing, that isn’t sparking imagination. It is just content. Just stuff, no more valuable than the stuffing in the teddy bear itself.

Not to mention the fact that, as the researchers found, some of these AI toys tend to veer into adult topics, like sex and weapons. Some even have terms of service that allow data collection to be used to train future models.

It all ultimately leads to the long-term concern of child development. When a kid’s social skills are developed through a neat AI chatbot, they miss out on the nuances, messiness, and complexity of real human interaction.

The imperfection that lies within us all, and the knowledge of how to navigate it, are what make interaction worth it. Synthesizing that with a toy meant to chat forever in the most sycophantic ways possible might make their loneliness worse rather than eradicating it.

The writers of the essay stop short of calling for bans, but urge that parents and the companies making these products supervise and teach digital literacy; they want stronger safeguards to ensure that we aren’t subjecting our children to the worst AI has to offer.