惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

阮一峰的网络日志
阮一峰的网络日志
The GitHub Blog
The GitHub Blog
V
Visual Studio Blog
G
GRAHAM CLULEY
Spread Privacy
Spread Privacy
Last Week in AI
Last Week in AI
腾讯CDC
P
Privacy & Cybersecurity Law Blog
WordPress大学
WordPress大学
C
Cybersecurity and Infrastructure Security Agency CISA
Security Archives - TechRepublic
Security Archives - TechRepublic
博客园 - 司徒正美
爱范儿
爱范儿
雷峰网
雷峰网
The Hacker News
The Hacker News
S
SegmentFault 最新的问题
Cisco Talos Blog
Cisco Talos Blog
博客园 - 聂微东
T
Tor Project blog
I
Intezer
大猫的无限游戏
大猫的无限游戏
Threat Intelligence Blog | Flashpoint
Threat Intelligence Blog | Flashpoint
Schneier on Security
Schneier on Security
T
Tenable Blog
Google Online Security Blog
Google Online Security Blog
S
Schneier on Security
freeCodeCamp Programming Tutorials: Python, JavaScript, Git & More
AI
AI
让小产品的独立变现更简单 - ezindie.com
让小产品的独立变现更简单 - ezindie.com
宝玉的分享
宝玉的分享
Help Net Security
Help Net Security
O
OpenAI News
博客园 - 【当耐特】
博客园 - Franky
AWS News Blog
AWS News Blog
罗磊的独立博客
J
Java Code Geeks
Know Your Adversary
Know Your Adversary
A
Arctic Wolf
小众软件
小众软件
量子位
SecWiki News
SecWiki News
S
Security Affairs
cs.CL updates on arXiv.org
cs.CL updates on arXiv.org
Hugging Face - Blog
Hugging Face - Blog
N
News and Events Feed by Topic
Apple Machine Learning Research
Apple Machine Learning Research
H
Heimdal Security Blog
Google DeepMind News
Google DeepMind News
IT之家
IT之家

cs updates on arXiv.org

Beyond Binary Edits Robust Multimodal Knowledge Editing with Adversarial Subspace Alignment Agentic Proving for Program Verification MemAudit: Post-hoc Auditing of Poisoned Agent Memory via Causal Attribution and Structural Anomaly Detection OnePred: Next-Query Prediction via Recursive Intent Memory in Multi-Turn Conversations OpenSkillEval: Automatically Auditing the Open Skill Ecosystem for LLM Agents One Policy, Infinite NPCs: Persona-Traceable Shared RL Policies for Scalable Game Agents How Human-Like Are Large Language Models? A Register-Aware Linguistic Evaluation Framework Benchmarking Google Embeddings 2 against Open-Source Models for Multilingual Dense Retrieval and RAG Systems Structure-Guided Entity Resolution: Fine-Tuning LLMs for Robust Name Matching in Complex Linguistic Contexts Solving the Aircraft Disassembly Scheduling Problem Co-ReAct: Rubrics as Step-Level Collaborators for ReAct Agents CP or DP? Why Not Both: A Case Study in the Partial Shop Scheduling Problem Asking For An Old Friend: Diagnosing and Mitigating Temporal Failure Modes in LLM-based Statutory Question Answering EDGE-OPD: Internalizing Privileged Context with Evidence Guided On-Policy Distillation ARES: Automated Rubric Synthesis for Scalable LLM Reinforcement Learning SSDAU: Structured Semantic Data Augmentation for Joint Entity and Relation Extraction Naturalistic measure of social norms alignment Articulatory strategy as a source of variation in acoustic vowel dynamics When Planning Fails Despite Correct Execution: On Epistemic Calibration for LLM-Based Multi-Agent Systems EquiSumm : A Gender Bias-Aware Framework for Inclusive Tweet Summarization Metacognition as Reward: Reinforcing LLM Reasoning via Knowledge and Regulation Signals From Correctness to Preference: A Framework for Personalized Agentic Reinforcement Learning Cultural Adaptation in Large Language Models for Political Discourse Emotion Recognition in Sign Language Conversation ClimateChat-300K: A Multi-Modal Facebook Dataset for Understanding Diverse Perspectives in Climate Communication AraHopeCorpus: Annotation Guidelines and Dataset for Hope Speech in Arabic Social Media Crisis Discourse Human-in-the-Loop Multi-Agent Ventilator Decision Support with Contextual Bandit Preference Learning Convergence Without Understanding: When Language Models Agree on Representations but Disagree on Reasoning DART: Semantic Recoverability for Structured Tool Agents Ontological Knowledge Blocks: Executable Compliance and Profile-Based Validation for Trustworthy AI Systems Parallel Context Compaction for Long-Horizon LLM Agent Serving When Is Next-Token Prediction Useful? Marginalization, Ergodicity, Mixture Identifiability, Local Sufficiency, RAG, Tools, and Programming GENSTRAT: Toward a Science of Strategic Reasoning in Large Language Models Foundation Protocol: A Coordination Layer for Agentic Society AutoResearch AI: Towards AI-Powered Research Automation for Scientific Discovery Hidden Human-Like Nature of Machine-Generated Texts: Theory and Detection Enhancement Self-Improving In-Context Learning Redrawing the AI Map: A Theory of Accountability Boundaries in Agentic Ecosystems Positional Failures in Long-Context LLMs: A Blind Spot in Reasoning Benchmarks Fast-dDrive: Efficient Block-Diffusion VLM for Autonomous Driving Same Model, Different Weakness: How Language and Modality Reshape the Jailbreak Attack Surface in Frontier MLLMs When Symptoms Are Not Enough: Evidence-Weighting Patterns in Large Language Model Psychiatric Screening As X, Do Y: How Persona and Task Combine in Instruction-Tuned LLMs CoReVAD: A Contextual Reasoning Framework for Training-Free Video Anomaly Detection Inconsistency-aware Multimodal Schrödinger Bridge for Deepfake Localization Inductive Deductive Synthesis: Enabling AI to Generate Formally Verified Systems A Fine-Tuned BERT Classifier for Personal-Letter Titles in Late-Ming and Early-Qing Collected Works A Comparative Evaluation of Structural Topic Models and BERTopic for Short, Open-Ended Survey Responses PathCal: State-Aware Reflection-Marker Calibration for Efficient Reasoning The Efficiency Frontier: A Unified Framework for Cost-Performance Optimization in LLM Context Management Flow Mismatching: Unsupervised Anomaly Detection via Velocity Discrepancies in Flow Matching Models DFKI-MLT at SemEval-2026 TASK 7: Steering Multilingual Models Towards Cultural Knowledge RoboSurg-VQA: A Multimodal Benchmark for Surgical Segmentation-Aware Visual Question Answering What Training Data Teaches RL Memory Agents: An Empirical Study of Curriculum Effects in Memory-Augmented QA Dithering Defense: Adversarial Robustness of Vision Foundation Models via Multi-Level Floyd-Steinberg Dithering Millimeter-wave Imaging for Anthropometric Body Measurement Model Collapse as Cultural Evolution DreamerNLplus: Interpretable Modeling of Mental Health Dynamics from Social Media Timelines using Hybrid Rule-Based and RAG Methods The TIME Machine: On The Power of Motion for Efficient Perception HawkesLLM: Semantic Uncertainty Propagation in Agentic Text Simulation Do Language Models Know What Not to Say? Causal Evidence for Statistical Preemption in LLMs Multilingual Steering by Design: Multilingual Sparse Autoencoders and Principled Layer Selection Sparse Autoencoders Map Brain-LLM Alignment onto Cortical Semantic Topography Brain-LLM Alignment Tracks Training Data, Not Typology The Deterministic Horizon: Impossibility Results as Design Specifications for Trustworthy AI Systems Scene Reconstruction as Mapping Priors for 3D Detection CoMoGen: COntrollable MOtion Dynamics and Interactions with Mask-Guided Video GENeration A Proactive Multi-Agent Dialogue Framework for Assessing Social Language Disorder Traits in Autism Memorization Dynamics of Fill-in-the-Middle Pretraining A Reproducible Universal Dependencies-Style Pipeline for Katharevousa Greek Parliamentary Text When AI Takes Sides on Questions of Faith: Persistent Asymmetries in AI-Mediated Faith Guidance Can AI Guess What You Know? Performance Comparison of Large Language Models for Human Domain Knowledge Estimation From Communication Logs Graph Alignment Topology as an Inductive Bias for Grounding Detection GazeBehavior Annotation Toolkit (GBAT): AI-powered toolkit for automatic annotation of egocentric eye-tracking and video data of child-caregiver interaction Improved Vision-to-Chart Buoy Association with Learned World-to-Image Projection Learnability-Informed Fine-Tuning of Diffusion Language Models RAS: Reflection-Augmented Scaling with In-Context Learning for Executable Cypher Query Generation VideoOdyssey: A Benchmark for Ultra-Long-Context and Omni-Modal Video Understanding EVE-Agent: Evidence-Verifiable Self-Evolving Agents Suicide Risk Assessment from AI-powered Video Surveillance: An Interpretable Framework for Prevention in Metro Stations Seeing without Looking: Do Vision-Language Benchmarks Really Test Vision? Mediative Fuzzy Logic: From Type-1 Foundations to Type-2, Type-3 and Quantum Extensions ImProver 2: Iteratively Self-Improving LMs for Neurosymbolic Proof Optimization Energy per Successful Goal: Goal-Level Energy Accounting for Agentic AI Systems GEM-4D: Geometry-Enhanced Video World Models for Robot Manipulation How Far Will They Go? Red-Teaming Online Influence with Large Language Models SciAtlas: A Large-Scale Knowledge Graph for Automated Scientific Research RMA: an Agentic System for Research-Level Mathematical Problems NeuroNL2LTL: A Neurosymbolic Framework for Natural Language Translation of Linear Temporal Logic BOHM: Zero-Cost Hierarchical Attribution for Compound AI Systems GAGPO: Generalized Advantage Grouped Policy Optimization Knowledge Distillation for Low-Resource Open-source Text-to-SQL Model Query-Adaptive Semantic Chunking for Retrieval-Augmented Generation: A Dynamic Strategy with Contextual Window Expansion A Survey of Text and Speech Resources for Hausa and Fongbe: Availability, Quality, and Gaps for NLP Development Evaluating Large Language Models in a Complex Hidden Role Game AV-Master: Dual-Path Comprehensive Perception Makes Better Audio-Visual Question Answering Memory-SAM: Human-Prompt-Free Tongue Segmentation via Retrieval-to-Prompt DIVER: Reinforced Diffusion Breaks Imitation Bottlenecks in End-to-End Autonomous Driving Transformer-Empowered Actor-Critic Reinforcement Learning for Sequence-Aware Service Function Chain Partitioning VerteNet -- A Multi-Context Hybrid CNN Transformer for Accurate Vertebral Landmark Localization in Lateral Spine DXA Images
Design and Report Benchmarks for Knowledge Work
Yining Hua, Hongbin Na, Cyrus Ayubcha, Levi Lian · 2026-05-22 · via cs updates on arXiv.org

The development of LLM agents has led to a growing body of work on knowledge-work AI, including coding, research, and healthcare. However, current knowledge-work evaluation and benchmark design still largely follow the logic of traditional NLP tasks. As a result, higher benchmark performance does not reliably show that a system can carry out knowledge work in real-world deployment settings. This paper contributes a three-step approach for making explicit how benchmarked tasks represent the work claims attached to their scores: defining the work activity under evaluation, specifying the tested setting, and scoring the appropriate work product. We review work studies showing that knowledge work is organized through roles and responsibilities, local materials and tools, and artifacts that must remain usable in downstream workflows. We then translate these concerns into benchmark design and reporting guidance, covering how tasks should be mapped to work activities, how tested settings should specify materials, tools, roles, and constraints, and how scoring should focus on the work product left by the system. To name the work activity being evaluated and distinguish it from common benchmark tasks, we derive an inventory of 18 work activities from the O{*}NET occupational task database. We demonstrate the approach through three benchmark case analyses: GDPval, a non-code occupational deliverable benchmark; OfficeQA Pro, a grounded document-analysis benchmark scored by final answers; and APEX-SWE, a software-engineering benchmark with executable scored products. These cases show how benchmark design choices shape the strongest work claim a score can support, and where gaps arise between the benchmarked task, tested setting, scored product, and broader work claim.