惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

G
Google Developers Blog
博客园 - 司徒正美
Last Week in AI
Last Week in AI
Recent Announcements
Recent Announcements
Y
Y Combinator Blog
博客园 - 聂微东
M
MIT News - Artificial intelligence
博客园_首页
Jina AI
Jina AI
博客园 - 叶小钗
酷 壳 – CoolShell
酷 壳 – CoolShell
H
Hackread – Cybersecurity News, Data Breaches, AI and More
J
Java Code Geeks
F
Fortinet All Blogs
aimingoo的专栏
aimingoo的专栏
小众软件
小众软件
Vercel News
Vercel News
The Cloudflare Blog
钛媒体:引领未来商业与生活新知
钛媒体:引领未来商业与生活新知
云风的 BLOG
云风的 BLOG
N
Netflix TechBlog - Medium
B
Blog
Google DeepMind News
Google DeepMind News
freeCodeCamp Programming Tutorials: Python, JavaScript, Git & More

cs updates on arXiv.org

Beyond Binary Edits Robust Multimodal Knowledge Editing with Adversarial Subspace Alignment Agentic Proving for Program Verification MemAudit: Post-hoc Auditing of Poisoned Agent Memory via Causal Attribution and Structural Anomaly Detection OpenSkillEval: Automatically Auditing the Open Skill Ecosystem for LLM Agents One Policy, Infinite NPCs: Persona-Traceable Shared RL Policies for Scalable Game Agents How Human-Like Are Large Language Models? A Register-Aware Linguistic Evaluation Framework Benchmarking Google Embeddings 2 against Open-Source Models for Multilingual Dense Retrieval and RAG Systems Structure-Guided Entity Resolution: Fine-Tuning LLMs for Robust Name Matching in Complex Linguistic Contexts Solving the Aircraft Disassembly Scheduling Problem Co-ReAct: Rubrics as Step-Level Collaborators for ReAct Agents CP or DP? Why Not Both: A Case Study in the Partial Shop Scheduling Problem Asking For An Old Friend: Diagnosing and Mitigating Temporal Failure Modes in LLM-based Statutory Question Answering EDGE-OPD: Internalizing Privileged Context with Evidence Guided On-Policy Distillation ARES: Automated Rubric Synthesis for Scalable LLM Reinforcement Learning SSDAU: Structured Semantic Data Augmentation for Joint Entity and Relation Extraction Naturalistic measure of social norms alignment Articulatory strategy as a source of variation in acoustic vowel dynamics When Planning Fails Despite Correct Execution: On Epistemic Calibration for LLM-Based Multi-Agent Systems EquiSumm : A Gender Bias-Aware Framework for Inclusive Tweet Summarization Metacognition as Reward: Reinforcing LLM Reasoning via Knowledge and Regulation Signals From Correctness to Preference: A Framework for Personalized Agentic Reinforcement Learning Cultural Adaptation in Large Language Models for Political Discourse Emotion Recognition in Sign Language Conversation ClimateChat-300K: A Multi-Modal Facebook Dataset for Understanding Diverse Perspectives in Climate Communication AraHopeCorpus: Annotation Guidelines and Dataset for Hope Speech in Arabic Social Media Crisis Discourse Human-in-the-Loop Multi-Agent Ventilator Decision Support with Contextual Bandit Preference Learning Convergence Without Understanding: When Language Models Agree on Representations but Disagree on Reasoning DART: Semantic Recoverability for Structured Tool Agents Ontological Knowledge Blocks: Executable Compliance and Profile-Based Validation for Trustworthy AI Systems Parallel Context Compaction for Long-Horizon LLM Agent Serving
PhAIL: A Real-Robot VLA Benchmark and Distributional Meth...
Sergey Arkhangelskiy · 2026-05-28 · via cs updates on arXiv.org

Real-world evaluation of vision-language-action (VLA) policies still rests on binary success rate at a fixed timeout with $N \le 25$ rollouts per condition, almost always without confidence intervals or paired statistical comparison; these cohort sizes struggle to resolve close comparisons reliably. We introduce PhAIL (Physical AI Leaderboard, https://phail.ai), an open real-robot benchmark on a Franka FR3 (dataset, per-rollout artifacts, and end-to-end reference implementation) of a distributional evaluation methodology: the time-to-success cumulative distribution function (CDF) as the evaluation primitive, with two separated jobs. The first is scoring via Human-Relative Throughput (HRT), a dimensionless scalar with bootstrap confidence intervals, anchored to same-fixture human teleoperation. The second is a significance test (Kolmogorov-Smirnov, computed per-object and macro-averaged across objects). On four publicly-available VLAs, the macro-averaged KS test resolves two close comparisons (GR00T vs. ACT, OpenPI vs. ACT) at $N \le 30$ rollouts per (model, object) cell where binary-threshold metrics do not; the closest pair (OpenPI vs. GR00T) remains unresolved within our budget. The best evaluated VLA is $\sim 7\times$ slower per operation (RMST ratio) than the human reference.