惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

有赞技术团队
有赞技术团队
Cyber Security Advisories - MS-ISAC
Cyber Security Advisories - MS-ISAC
IT之家
IT之家
博客园 - 【当耐特】
罗磊的独立博客
Stack Overflow Blog
Stack Overflow Blog
MyScale Blog
MyScale Blog
WordPress大学
WordPress大学
The GitHub Blog
The GitHub Blog
H
Hackread – Cybersecurity News, Data Breaches, AI and More
Hugging Face - Blog
Hugging Face - Blog
I
InfoQ
B
Blog RSS Feed
腾讯CDC
云风的 BLOG
云风的 BLOG
N
Netflix TechBlog - Medium
Apple Machine Learning Research
Apple Machine Learning Research
GbyAI
GbyAI
雷峰网
雷峰网
freeCodeCamp Programming Tutorials: Python, JavaScript, Git & More
D
DataBreaches.Net
The Cloudflare Blog
V
V2EX
S
SegmentFault 最新的问题

stat.ML updates on arXiv.org

Adaptive multi-fidelity optimization with fast learning rates Enhancing AI and Dynamical Subseasonal Forecasts with Probabilistic Bias Correction Sample Complexity Bounds for Stochastic Shortest Path with a Generative Model The Harder Path: Last Iterate Convergence for Uncoupled Learning in Zero-Sum Games with Bandit Feedback Stylistic-STORM (ST-STORM) : Perceiving the Semantic Nature of Appearance Collective Kernel EFT for Pre-activation ResNets PRIM-cipal components analysis One-Shot Generative Flows: Existence and Obstructions Structural interpretability in SVMs with truncated orthogonal polynomial kernels Amortized Optimal Transport from Sliced Potentials MinShap: A Modified Shapley Value Approach for Feature Selection Unsupervised feature selection using Bayesian Tucker decomposition Multi-User mmWave Beam and Rate Adaptation via Combinatorial Satisficing Bandits Best of both worlds: Stochastic & adversarial best-arm identification Scalable Model-Based Clustering with Sequential Monte Carlo Expert-Guided Class-Conditional Goodness-of-Fit Scores for Interpretable Classification with Informative Missingness: An Application to Seismic Monitoring Lightweight Geometric Adaptation for Training Physics-Informed Neural Networks Gating Enables Curvature: A Geometric Expressivity Gap in Attention Zeroth-Order Optimization at the Edge of Stability Differentially Private Conformal Prediction CLion: Efficient Cautious Lion Optimizer with Enhanced Generalization Generative Augmented Inference Improving Machine Learning Performance with Synthetic Augmentation PAC-MCTS: Bias-Aware Pruning for Robust LLM-Guided Search and Planning Path-Sampled Integrated Gradients Heat and Matérn Kernels on Matchings Doubly Outlier-Robust Online Infinite Hidden Markov Model Momentum Further Constrains Sharpness at the Edge of Stochastic Stability Multistage Conditional Compositional Optimization BOAT: Navigating the Sea of In Silico Predictors for Antibody Design via Multi-Objective Bayesian Optimization
Variance-Bounded Evaluation of Entity-Centric AI Systems ...
Kaihua Ding · 2025-09-26 · via stat.ML updates on arXiv.org

Reliable evaluation of AI systems remains a fundamental challenge when ground truth labels are unavailable, particularly for systems generating natural language outputs like AI chat and agent systems. Many of these AI agents and systems focus on entity-centric tasks. In enterprise contexts, organizations deploy AI systems for entity linking, data integration, and information retrieval where verification against gold standards is often infeasible due to proprietary data constraints. Academic deployments face similar challenges when evaluating AI systems on specialized datasets with ambiguous criteria. Conventional evaluation frameworks, rooted in supervised learning paradigms, fail in such scenarios where single correct answers cannot be defined. We introduce VB-Score, a variance-bounded evaluation framework for entity-centric AI systems that operates without ground truth by jointly measuring effectiveness and robustness. Given system inputs, VB-Score enumerates plausible interpretations through constraint relaxation and Monte Carlo sampling, assigning probabilities that reflect their likelihood. It then evaluates system outputs by their expected success across interpretations, penalized by variance to assess robustness of the system. We provide formal theoretical analysis establishing key properties including range, monotonicity, and stability along with concentration bounds for Monte Carlo estimation. Through case studies on AI systems with ambiguous inputs, we demonstrate that VB-Score reveals robustness differences hidden by conventional evaluation frameworks, offering a principled measurement framework for assessing AI system reliability in label-scarce domains.