惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

MongoDB | Blog
MongoDB | Blog
B
Blog
Y
Y Combinator Blog
大猫的无限游戏
大猫的无限游戏
aimingoo的专栏
aimingoo的专栏
B
Blog RSS Feed
博客园 - Franky
V
V2EX
IT之家
IT之家
WordPress大学
WordPress大学
博客园 - 三生石上(FineUI控件)
J
Java Code Geeks
F
Fortinet All Blogs
I
InfoQ
云风的 BLOG
云风的 BLOG
腾讯CDC
OSCHINA 社区最新新闻
OSCHINA 社区最新新闻
月光博客
月光博客
让小产品的独立变现更简单 - ezindie.com
让小产品的独立变现更简单 - ezindie.com
N
Netflix TechBlog - Medium
宝玉的分享
宝玉的分享
钛媒体:引领未来商业与生活新知
钛媒体:引领未来商业与生活新知
P
Proofpoint News Feed
Microsoft Security Blog
Microsoft Security Blog

cs updates on arXiv.org

Beyond Binary Edits Robust Multimodal Knowledge Editing with Adversarial Subspace Alignment Agentic Proving for Program Verification MemAudit: Post-hoc Auditing of Poisoned Agent Memory via Causal Attribution and Structural Anomaly Detection OpenSkillEval: Automatically Auditing the Open Skill Ecosystem for LLM Agents One Policy, Infinite NPCs: Persona-Traceable Shared RL Policies for Scalable Game Agents How Human-Like Are Large Language Models? A Register-Aware Linguistic Evaluation Framework Benchmarking Google Embeddings 2 against Open-Source Models for Multilingual Dense Retrieval and RAG Systems Structure-Guided Entity Resolution: Fine-Tuning LLMs for Robust Name Matching in Complex Linguistic Contexts Solving the Aircraft Disassembly Scheduling Problem Co-ReAct: Rubrics as Step-Level Collaborators for ReAct Agents CP or DP? Why Not Both: A Case Study in the Partial Shop Scheduling Problem Asking For An Old Friend: Diagnosing and Mitigating Temporal Failure Modes in LLM-based Statutory Question Answering EDGE-OPD: Internalizing Privileged Context with Evidence Guided On-Policy Distillation ARES: Automated Rubric Synthesis for Scalable LLM Reinforcement Learning SSDAU: Structured Semantic Data Augmentation for Joint Entity and Relation Extraction Naturalistic measure of social norms alignment Articulatory strategy as a source of variation in acoustic vowel dynamics When Planning Fails Despite Correct Execution: On Epistemic Calibration for LLM-Based Multi-Agent Systems EquiSumm : A Gender Bias-Aware Framework for Inclusive Tweet Summarization Metacognition as Reward: Reinforcing LLM Reasoning via Knowledge and Regulation Signals From Correctness to Preference: A Framework for Personalized Agentic Reinforcement Learning Cultural Adaptation in Large Language Models for Political Discourse Emotion Recognition in Sign Language Conversation ClimateChat-300K: A Multi-Modal Facebook Dataset for Understanding Diverse Perspectives in Climate Communication AraHopeCorpus: Annotation Guidelines and Dataset for Hope Speech in Arabic Social Media Crisis Discourse Human-in-the-Loop Multi-Agent Ventilator Decision Support with Contextual Bandit Preference Learning Convergence Without Understanding: When Language Models Agree on Representations but Disagree on Reasoning DART: Semantic Recoverability for Structured Tool Agents Ontological Knowledge Blocks: Executable Compliance and Profile-Based Validation for Trustworthy AI Systems Parallel Context Compaction for Long-Horizon LLM Agent Serving
Reasoning, Code, or Both? How Large Language Models Handl...
Matthew Kutakh · 2026-05-26 · via cs updates on arXiv.org

Large Language Models (LLMs) achieve impressive accuracy on mathematical reasoning benchmarks, yet their performance drops when problems are modified with simple changes like different names or numbers. Code execution methods, which let models generate and run Python code instead of reasoning in natural language, have been proposed as a solution, but their effect on reasoning robustness (the ability to maintain accuracy across problem variations) has not been systematically tested. This study evaluates three approaches on 1,000 problems from the GSM-Symbolic dataset: pure reasoning using chain-of-thought (CoT) prompting, single-shot code execution using Program-Aided Language models (PAL), and iterative code execution using Step-by-Step Coding (SBSC). All three were run on paired original and modified problems using Claude Haiku 4.5. CoT was the most robust method, with an accuracy drop of 1.3 percentage points and 1.8% of problems breaking under perturbation. PAL was the least robust at 1.7 percentage points and 3.1% broke, with SBSC falling in between. Although these differences were not statistically significant ($p = .096$), the directional trend was consistent across all measures, suggesting that code execution, whether single-shot or iterative, does not improve reasoning robustness on grade-school-level problem variations.