Joseph Crawford of the University of Tasmania and Michael Cowling of RMIT University discuss the dangers of becoming too emotionally dependent on advanced technologies.
Since November 2022, generative artificial intelligence (AI) has been rewriting billions of emails, student assignments, reports and break-up text messages. Alongside these mundane uses, we’ve seen a proliferation of chatbots as AI “companions”.
Users can design and clothe (or unclothe) their very own AI friend, often for a fee. These social chatbots provide “happy conversations” and a seemingly perfect relationship at the cost of the mutual vulnerability and genuine sense of belonging we gain from bonds with other humans.
Proponents argue that for those experiencing social disconnection, chatbots are infinitely available, supportive companions who don’t judge them. And there’s some evidence chatbots can alleviate symptoms of loneliness, chronic disconnection and thwarted belonging.
Australia’s eSafety Commissioner acknowledges this: “AI companions are always available – which can be appealing for those young people who struggle to build face-to-face relationships.”
But the risks of replacing human relationships with chatbots are only starting to be understood.
A study we conducted in 2023 showed university students using early chatbots for academic support copped unintended consequences, such as loneliness. Currently, the risks of social chatbots – such as dependency, displacing human friendships, withdrawing socially or building unrealistic relationships – appear to outweigh the benefits.
In recent weeks, China has put into place new rules for AI companions and chatbots, or “anthropomorphic interactive services” as they’re called in the legislation. Alongside ensuring chatbots must guard state secrets, emotional use of these tools is restricted, and expected to follow social and moral standards.
These are some of the strongest controls of social AI use introduced anywhere in the world. So what can we learn?
Material harms from companion AI use do occur, including in China. An 18-year-old spent 7,000 yuan (A$1,400) on customised jewellery for her AI boyfriend in their four months of “dating”.
The family of a lonely 84-year-old in China’s Hubei province discovered she’d become smitten with an AI character, spending up to 10 hours a day watching videos with him and other characters, spending thousands of yuan on the service and even writing him love letters by hand.
China’s new rules establish explicit provisions for how they expect companion chatbots to interact with humans.
The measures expect social AI bot developers to assure content does not:
What’s more, users who seem overly reliant on the service must receive prominent reminders they’re speaking to a chatbot. And AI companies are required to intervene if users appear to be in extreme distress.
As we’ve shown, there are concerns – and studies to date hint at this too – that it’s not healthy for young people to rely exclusively on AI as a trusted friend or therapist.
Lonely people experiencing social anxiety can engage in problematic use of conversational AI to alleviate feelings of loneliness and negative rumination.
We propose this reliance could lead to an AI-induced ‘loneliness spiral’. When individuals feel lonely, they’re more sensitive to feelings of social threat and disruptive emotional regulation.
So, they turn to a low-risk AI ‘friend’ for support, instead of qualified or friendly humans. However, this often doesn’t fix the initial problem of loneliness, since chatbots can actually exacerbate it.
The restrictions China has put in place for AI companions can help to break this cycle and re-prioritise human connection. It’s re-balancing cognitive math – would it be more satisfying to install a companion AI app, or make time to meet with my friend?
Loneliness and a low sense of belonging remain major challenges in Australia, with at least one in six experiencing loneliness, with some evidence suggesting higher rates.
While China has cracked down on AI companions, Australia has been leading the charge on young people’s use of social media. eSafety has also developed codes for restricting age-related material online such as pornography, high-impact violence, self-harm, suicide and disordered eating. But they don’t address social harms of AI companions, such as emotional dependency or displacing human connection.
Prime Minister Anthony Albanese also didn’t mention social harms from AI chatbots in his major speech on AI last week.
We propose that Australia’s age-restricted material protections should extend to unintended social harm of conversations with social, analytical and functional AIs.
The Australian government needs to explore how these virtual companions interact with Australians. And we must prepare to mitigate unintended social harms before these synthetic social connections become widespread and normalised.
Australia must prioritise developing social connectedness – in workplaces, productivity strategies, education curriculum and in the design of our communities.
An authoritarian crackdown on chatbots may not be within reach for a democratic nation. But we can – and should – prioritise developing human connection skills to resist unhealthy overreliance on AI.
And Australia should leverage regulatory protections to mitigate social harms, extending on the work eSafety is already doing with social media.
Joseph Crawford and Michael Cowling
Joseph Crawford is an award-winning management, leadership and higher education scholar based in the Office of the Deputy Vice Chancellor for Education at the University of Tasmania. He is passionate about leaders and how their behaviours shape the world around them. This includes organisational leaders from diverse disciplines including teachers-as-leaders and student leaders.
Prof Michael Cowling is an award-winning technology educator and communicator with a keen interest in mixed reality education, computer science education and information technology in society. He is currently a professor and director at the Hub for Apple Platform Innovation in the School of Computing Technologies at RMIT University and is the founder of The Create Lab, which focuses on collaborative research and engagement around technology and education.
Don’t miss out on the knowledge you need to succeed. Sign up for the Daily Brief, Silicon Republic’s digest of need-to-know sci-tech news.
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。