Authors:Youqing Fang (1 and 2), Yinhao Tang (1 and 2), Yanan Sun (2), Jiangning Liu (2), Ziyi Wang (2), Xun Zhao (2), Bin Liu (1), Weiming Zhang (1), Kuikun Liu (2), Wenwei Zhang (2), Kai Chen (2) ((1) University of Science and Technology of China, (2) Shanghai AI Laboratory)
Abstract:Recent writing assistants are increasingly shifting from passive, prompt-driven interaction to proactive, suggestion-based completion, which integrates localized continuations into the writing flow and reduces coordination burden. However, existing evaluations simply focus on output quality, failing to capture how users accept, edit, or repair suggestions in real-time interaction, and thus obscuring the true usability of proactive co-writing systems. To address this gap, we adopt a sequential, behavior-centered view of interactive writing and formalize co-writing as a Human-in-the-Loop Markov Decision Process, modeling writing as an interaction shaped by user acceptance and editing decisions. Based on this formulation, we introduce the Co-Writing Fidelity Suite, an interaction-aware metric suite that captures both user-assistant alignment and cognitive editing effort, including Hierarchical Acceptance Rate and Knowledge-aware Editing Distance. We conduct a large-scale simulation study across 16 writing domains, using 1,688 controlled continuation queries sampled from different writing stages. Our analysis reveals systematic effects of interaction structure on acceptance behavior and editing cost. A follow-up user study with 30 participants confirms that these behavioral patterns align with real user experience. Together, our findings demonstrate that interaction-aware evaluation provides insights beyond output-only metrics and informs the design of more effective proactive writing assistants.
| Comments: | 30 pages, 8 figures. Accepted to IJCAI 2026 |
| Subjects: | Human-Computer Interaction (cs.HC) |
| Cite as: | arXiv:2605.23535 [cs.HC] |
| (or arXiv:2605.23535v1 [cs.HC] for this version) | |
| https://doi.org/10.48550/arXiv.2605.23535 arXiv-issued DOI via DataCite (pending registration) |
From: Youqing Fang [view email]
[v1]
Fri, 22 May 2026 11:55:56 UTC (2,107 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。