惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

C
Check Point Blog
GbyAI
GbyAI
奇客Solidot–传递最新科技情报
奇客Solidot–传递最新科技情报
U
Unit 42
美团技术团队
NISL@THU
NISL@THU
C
Cisco Blogs
SecWiki News
SecWiki News
N
Netflix TechBlog - Medium
Forbes - Security
Forbes - Security
Cloudbric
Cloudbric
雷峰网
雷峰网
T
Tailwind CSS Blog
博客园 - 司徒正美
The Register - Security
The Register - Security
L
LangChain Blog
S
Security Affairs
Hacker News - Newest:
Hacker News - Newest: "LLM"
B
Blog
OSCHINA 社区最新新闻
OSCHINA 社区最新新闻
T
Threat Research - Cisco Blogs
I
InfoQ
S
Schneier on Security
L
Lohrmann on Cybersecurity
量子位
cs.CL updates on arXiv.org
cs.CL updates on arXiv.org
Martin Fowler
Martin Fowler
Schneier on Security
Schneier on Security
F
Fortinet All Blogs
TaoSecurity Blog
TaoSecurity Blog
K
Kaspersky official blog
Google DeepMind News
Google DeepMind News
Cisco Talos Blog
Cisco Talos Blog
PCI Perspectives
PCI Perspectives
Attack and Defense Labs
Attack and Defense Labs
WordPress大学
WordPress大学
Microsoft Azure Blog
Microsoft Azure Blog
H
Help Net Security
Project Zero
Project Zero
The GitHub Blog
The GitHub Blog
D
Docker
N
News | PayPal Newsroom
K
KPMG report finds enterprise disconnect between AI and its ROI | CIO
H
Hacker News: Front Page
云风的 BLOG
云风的 BLOG
Microsoft Security Blog
Microsoft Security Blog
freeCodeCamp Programming Tutorials: Python, JavaScript, Git & More
博客园 - 聂微东
Webroot Blog
Webroot Blog
MongoDB | Blog
MongoDB | Blog

cs updates on arXiv.org

Beyond Binary Edits Robust Multimodal Knowledge Editing with Adversarial Subspace Alignment Agentic Proving for Program Verification MemAudit: Post-hoc Auditing of Poisoned Agent Memory via Causal Attribution and Structural Anomaly Detection OpenSkillEval: Automatically Auditing the Open Skill Ecosystem for LLM Agents One Policy, Infinite NPCs: Persona-Traceable Shared RL Policies for Scalable Game Agents How Human-Like Are Large Language Models? A Register-Aware Linguistic Evaluation Framework Benchmarking Google Embeddings 2 against Open-Source Models for Multilingual Dense Retrieval and RAG Systems Structure-Guided Entity Resolution: Fine-Tuning LLMs for Robust Name Matching in Complex Linguistic Contexts Solving the Aircraft Disassembly Scheduling Problem Co-ReAct: Rubrics as Step-Level Collaborators for ReAct Agents CP or DP? Why Not Both: A Case Study in the Partial Shop Scheduling Problem Asking For An Old Friend: Diagnosing and Mitigating Temporal Failure Modes in LLM-based Statutory Question Answering EDGE-OPD: Internalizing Privileged Context with Evidence Guided On-Policy Distillation ARES: Automated Rubric Synthesis for Scalable LLM Reinforcement Learning SSDAU: Structured Semantic Data Augmentation for Joint Entity and Relation Extraction Naturalistic measure of social norms alignment Articulatory strategy as a source of variation in acoustic vowel dynamics When Planning Fails Despite Correct Execution: On Epistemic Calibration for LLM-Based Multi-Agent Systems EquiSumm : A Gender Bias-Aware Framework for Inclusive Tweet Summarization Metacognition as Reward: Reinforcing LLM Reasoning via Knowledge and Regulation Signals From Correctness to Preference: A Framework for Personalized Agentic Reinforcement Learning Cultural Adaptation in Large Language Models for Political Discourse Emotion Recognition in Sign Language Conversation ClimateChat-300K: A Multi-Modal Facebook Dataset for Understanding Diverse Perspectives in Climate Communication AraHopeCorpus: Annotation Guidelines and Dataset for Hope Speech in Arabic Social Media Crisis Discourse Human-in-the-Loop Multi-Agent Ventilator Decision Support with Contextual Bandit Preference Learning Convergence Without Understanding: When Language Models Agree on Representations but Disagree on Reasoning DART: Semantic Recoverability for Structured Tool Agents Ontological Knowledge Blocks: Executable Compliance and Profile-Based Validation for Trustworthy AI Systems Parallel Context Compaction for Long-Horizon LLM Agent Serving When Is Next-Token Prediction Useful? Marginalization, Ergodicity, Mixture Identifiability, Local Sufficiency, RAG, Tools, and Programming Design and Report Benchmarks for Knowledge Work GENSTRAT: Toward a Science of Strategic Reasoning in Large Language Models Foundation Protocol: A Coordination Layer for Agentic Society AutoResearch AI: Towards AI-Powered Research Automation for Scientific Discovery Hidden Human-Like Nature of Machine-Generated Texts: Theory and Detection Enhancement Self-Improving In-Context Learning Redrawing the AI Map: A Theory of Accountability Boundaries in Agentic Ecosystems Positional Failures in Long-Context LLMs: A Blind Spot in Reasoning Benchmarks Fast-dDrive: Efficient Block-Diffusion VLM for Autonomous Driving Same Model, Different Weakness: How Language and Modality Reshape the Jailbreak Attack Surface in Frontier MLLMs When Symptoms Are Not Enough: Evidence-Weighting Patterns in Large Language Model Psychiatric Screening As X, Do Y: How Persona and Task Combine in Instruction-Tuned LLMs CoReVAD: A Contextual Reasoning Framework for Training-Free Video Anomaly Detection Inconsistency-aware Multimodal Schrödinger Bridge for Deepfake Localization Inductive Deductive Synthesis: Enabling AI to Generate Formally Verified Systems A Fine-Tuned BERT Classifier for Personal-Letter Titles in Late-Ming and Early-Qing Collected Works A Comparative Evaluation of Structural Topic Models and BERTopic for Short, Open-Ended Survey Responses PathCal: State-Aware Reflection-Marker Calibration for Efficient Reasoning The Efficiency Frontier: A Unified Framework for Cost-Performance Optimization in LLM Context Management Flow Mismatching: Unsupervised Anomaly Detection via Velocity Discrepancies in Flow Matching Models DFKI-MLT at SemEval-2026 TASK 7: Steering Multilingual Models Towards Cultural Knowledge RoboSurg-VQA: A Multimodal Benchmark for Surgical Segmentation-Aware Visual Question Answering What Training Data Teaches RL Memory Agents: An Empirical Study of Curriculum Effects in Memory-Augmented QA Dithering Defense: Adversarial Robustness of Vision Foundation Models via Multi-Level Floyd-Steinberg Dithering Millimeter-wave Imaging for Anthropometric Body Measurement Model Collapse as Cultural Evolution DreamerNLplus: Interpretable Modeling of Mental Health Dynamics from Social Media Timelines using Hybrid Rule-Based and RAG Methods The TIME Machine: On The Power of Motion for Efficient Perception HawkesLLM: Semantic Uncertainty Propagation in Agentic Text Simulation Do Language Models Know What Not to Say? Causal Evidence for Statistical Preemption in LLMs Multilingual Steering by Design: Multilingual Sparse Autoencoders and Principled Layer Selection Sparse Autoencoders Map Brain-LLM Alignment onto Cortical Semantic Topography Brain-LLM Alignment Tracks Training Data, Not Typology The Deterministic Horizon: Impossibility Results as Design Specifications for Trustworthy AI Systems Scene Reconstruction as Mapping Priors for 3D Detection CoMoGen: COntrollable MOtion Dynamics and Interactions with Mask-Guided Video GENeration A Proactive Multi-Agent Dialogue Framework for Assessing Social Language Disorder Traits in Autism Memorization Dynamics of Fill-in-the-Middle Pretraining A Reproducible Universal Dependencies-Style Pipeline for Katharevousa Greek Parliamentary Text When AI Takes Sides on Questions of Faith: Persistent Asymmetries in AI-Mediated Faith Guidance Can AI Guess What You Know? Performance Comparison of Large Language Models for Human Domain Knowledge Estimation From Communication Logs Graph Alignment Topology as an Inductive Bias for Grounding Detection GazeBehavior Annotation Toolkit (GBAT): AI-powered toolkit for automatic annotation of egocentric eye-tracking and video data of child-caregiver interaction Improved Vision-to-Chart Buoy Association with Learned World-to-Image Projection Learnability-Informed Fine-Tuning of Diffusion Language Models RAS: Reflection-Augmented Scaling with In-Context Learning for Executable Cypher Query Generation VideoOdyssey: A Benchmark for Ultra-Long-Context and Omni-Modal Video Understanding EVE-Agent: Evidence-Verifiable Self-Evolving Agents Suicide Risk Assessment from AI-powered Video Surveillance: An Interpretable Framework for Prevention in Metro Stations Seeing without Looking: Do Vision-Language Benchmarks Really Test Vision? Mediative Fuzzy Logic: From Type-1 Foundations to Type-2, Type-3 and Quantum Extensions ImProver 2: Iteratively Self-Improving LMs for Neurosymbolic Proof Optimization Energy per Successful Goal: Goal-Level Energy Accounting for Agentic AI Systems GEM-4D: Geometry-Enhanced Video World Models for Robot Manipulation How Far Will They Go? Red-Teaming Online Influence with Large Language Models SciAtlas: A Large-Scale Knowledge Graph for Automated Scientific Research RMA: an Agentic System for Research-Level Mathematical Problems NeuroNL2LTL: A Neurosymbolic Framework for Natural Language Translation of Linear Temporal Logic BOHM: Zero-Cost Hierarchical Attribution for Compound AI Systems GAGPO: Generalized Advantage Grouped Policy Optimization Knowledge Distillation for Low-Resource Open-source Text-to-SQL Model Query-Adaptive Semantic Chunking for Retrieval-Augmented Generation: A Dynamic Strategy with Contextual Window Expansion A Survey of Text and Speech Resources for Hausa and Fongbe: Availability, Quality, and Gaps for NLP Development Evaluating Large Language Models in a Complex Hidden Role Game AV-Master: Dual-Path Comprehensive Perception Makes Better Audio-Visual Question Answering Memory-SAM: Human-Prompt-Free Tongue Segmentation via Retrieval-to-Prompt DIVER: Reinforced Diffusion Breaks Imitation Bottlenecks in End-to-End Autonomous Driving Transformer-Empowered Actor-Critic Reinforcement Learning for Sequence-Aware Service Function Chain Partitioning VerteNet -- A Multi-Context Hybrid CNN Transformer for Accurate Vertebral Landmark Localization in Lateral Spine DXA Images
Do LLM Attribution Metrics Transfer? Auditing Retrieval-Augmented Generation Evaluation Across Datasets and Constructs
[Submitted on 22 Jun 2026] · 2026-06-24 · via cs updates on arXiv.org

View PDF HTML (experimental)

Abstract:Practice often treats automatic metrics for attribution in LLM retrieval-augmented generation as interchangeable. We audit eight automatic scorers -- lexical, embedding, and BERTScore baselines alongside entailment/grounding-trained models (clean and FEVER NLI, the checker MiniCheck) -- across three evaluation constructs (provenance/topicality, generated-answer attribution, and fact-check entailment), asking whether any scorer transfers: stays within the 95% confidence interval of the best audited scorer on every dataset of a multi-dataset construct. In the construct with the most multi-dataset human-labeled coverage -- generated-answer attribution (AttributionBench's four source datasets, n = 1,610, with independent HAGRID, n = 2,150) -- none does: the per-dataset metric rankings invert (Kendall tau = -0.64, p = 0.031 on AttributedQA vs. LFQA), and an off-the-shelf NLI scorer that is best on short-claim AttributedQA (AUROC 0.90) collapses to AUROC 0.53 (chance) on long-form LFQA, where BERTScore wins (0.91); the flip is not a length or truncation artifact. This instability has a concrete decision cost: a naive "best-on-average" rule for choosing an evaluator fails leave-one-dataset-out (mean held-out regret 0.172 AUROC, worse than fixing one scorer), so metric choice must be validated on the target dataset rather than learned from others. A prompt-based LLM judge avoids the chance-level collapses the automatic scorers suffer (no LFQA collapse) but is not uniformly best, ~100x costlier, and non-deterministic -- relocating, not removing, the validation burden.

Submission history

From: Tianyu Ding [view email]
[v1] Mon, 22 Jun 2026 20:25:36 UTC (60 KB)