惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

博客园_首页
J
Java Code Geeks
博客园 - 聂微东
量子位
C
Check Point Blog
T
The Blog of Author Tim Ferriss
T
Tailwind CSS Blog
G
Google Developers Blog
Google DeepMind News
Google DeepMind News
B
Blog
罗磊的独立博客
腾讯CDC
GbyAI
GbyAI
博客园 - 【当耐特】
A
About on SuperTechFans
M
MIT News - Artificial intelligence
U
Unit 42
D
Docker
OSCHINA 社区最新新闻
OSCHINA 社区最新新闻
Y
Y Combinator Blog
大猫的无限游戏
大猫的无限游戏
小众软件
小众软件
S
SegmentFault 最新的问题
有赞技术团队
有赞技术团队

cs.HC updates on arXiv.org

A Framework for Measuring Appropriate Reliance on Set-Valued AI Advice DeliChess: A Multi-party Dialogue Dataset for Deliberation in Chess Puzzle Solving From Explanation to Diagnosis: Next Generation Interactive Video Coach with Misstep Awareness SocialCoach: Personalized Social Skill Learning with RL-based Agentic Tutoring and Practice Formalizing all indexed mathematics as a benchmark for general reasoning, with the example of implementing dilatations of categories Face versus Body Tracking for Human-Robot Interaction: An Egocentric Dataset From Prompt to Service: An SLM-Based Agent Orchestration Gateway for AI-Driven Virtual Worlds What LLMs Must Forget to Teach Effectively: A DIY Approach to Premodern Japanese Language Pedagogy Quantitative Movement Testing: Measuring Patient Movements from a Single Smartphone Video Vision-Language Models Suppress Female Representations Under Ambiguous Input The New Social Image: How AI Competency and AI Proactivity Influence Self- and Peer-Perceptions in the Workplace TUX: Measuring Human--AI Tacit Understanding LLUMI: Improving LLM Writing Assistance for Mental Health Support with Online Community Feedback VideoFDB: Evaluating Full-Duplex Vision-Speech Capabilities in Conversational Agents Label Over Logic? How Source Cues Bias Human Fallacy Judgments More Than LLMs Inform, Coach, Relate, Listen: Auditing LLM Caregiving Support Roles How Coding Agents Fail Their Users: A Large-Scale Analysis of Developer-Agent Misalignment in 20,574 Real-World Sessions MetaRanker: Human-in-the-loop Active Ranking for Metalens Image Quality Analyzing Persona Effects in Generated Explanations from Multimodal LLM Agents in Urban Perception Granuscore: A Reference-Free Measure of Granularity for Text Analysis and Question Answering The Timing Dependencies of Trust: Speed, Accuracy, and cBCI Neuro-Decoupling in Human-AI Teams Bayesian Distributional Models of Executive Functioning Visual Matters: Connecting Aesthetic Appeal and Production Quality of Photos, Infographics and Data Visualizations to Credibility of Social Media Posts Data-driven Head Motion Generation through Natural Gaze-Head Coordination Agreement Metrics for LLM-as-Judge Evaluation: What to Report and Why Perceptually Lossless Tactile Texture Synthesis with Compact Spectral Envelope Models MambaGaze: Bidirectional Mamba with Explicit Missing Data Modeling for Cognitive Load Assessment from Eye-Gaze Tracking Data CogAdapt: Transferring Clinical ECG Foundation Models to Wearable Cognitive Load Assessment via Lead Adaptation Augmented Analytics and Decision Quality: The Role of Trust among Non-Technical BI Users Faster Completion, Less Learning: Generative AI Reduced Study Time on Math Problems and the Knowledge They Build
MMAC-Copilot: Multi-modal Agent Collaboration Operating C...
Zirui Song, Yaohang Li, Meng Fang, Yanda Li, Zhenhao Chen, Zeche · 2024-04-28 · via cs.HC updates on arXiv.org

Large language model agents that interact with PC applications often face limitations due to their singular mode of interaction with real-world environments, leading to restricted versatility and frequent hallucinations. To address this, we propose the Multi-Modal Agent Collaboration framework (MMAC-Copilot), a framework utilizes the collective expertise of diverse agents to enhance interaction ability with application. The framework introduces a team collaboration chain, enabling each participating agent to contribute insights based on their specific domain knowledge, effectively reducing the hallucination associated with knowledge domain gaps. We evaluate MMAC-Copilot using the GAIA benchmark and our newly introduced Visual Interaction Benchmark (VIBench). MMAC-Copilot achieved exceptional performance on GAIA, with an average improvement of 6.8\% over existing leading systems. VIBench focuses on non-API-interactable applications across various domains, including 3D gaming, recreation, and office scenarios. It also demonstrated remarkable capability on VIBench. We hope this work can inspire in this field and provide a more comprehensive assessment of Autonomous agents. The anonymous Github is available at \href{https://anonymous.4open.science/r/ComputerAgentWithVision-3C12}{Anonymous Github}