Hong Shen, an assistant professor in Carnegie Mellon University's School of Computer Science, presented this summer at the World Economic Forum (WEF) Annual Meeting of the New Champions in Dalian, China.
Shen, whose work in CMU's Human-Computer Interaction Institute (HCII) focuses on using artificial intelligence responsibly, spoke during the session, "AI Safety in the Affection Economy."
The affection economy refers to AI systems used to not only answer questions or complete tasks, but also to participate in ongoing social, verbal and nonverbal interactions with users. While some systems are explicitly designed as AI companions, others are not intentionally planned for this purpose. Yet users still become attached to them, especially through repeated, personalized and emotionally responsive conversations.
“As AI becomes more personal, researchers need to take social and relational safety as seriously as technical safety,” Shen said. “Designing trustworthy AI requires understanding not only how the system is built, but also the human tendency to form relationships with technologies that seem attentive, responsive and caring.”
During her talk, Shen shared her work from the Collective AI Research and Evaluation (CARE) Lab, which studies what it means to build AI systems that are both technically and socially safe. The lab’s recent work examines how users develop emotional attachments to conversational AI, how they perceive issues such as bias and value alignment, and how these systems could be designed to better support human well-being.
At the WEF session, Shen joined researchers, policymakers and industry leaders in a discussion that pushed beyond technical benchmarks to ask harder questions: how should AI systems signal their own limitations, and who is responsible when users form attachments to systems never designed to reciprocate?
Reflecting on her experience, Shen said she was encouraged by the growing recognition that building safe AI requires expertise from across disciplines, not just computer science.
"It was rewarding to see that AI safety is increasingly being recognized as a human challenge as much as a technical one," she said. "That's the kind of interdisciplinary work Carnegie Mellon has always pushed for. Seeing the approach we've been building in the CARE Lab — combining technical innovation with human-centered design and community engagement — resonating with conversations happening on a global stage was incredibly meaningful."
For more about Shen's work, visit her HCII website.
Author
Mallory Lindahl
Research Areas
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。