Abstract:Jupyter notebooks have become popular for early machine learning (ML) development, enabling interactive and iterative experimentation. However, ML notebooks are prone to bugs, among which crashes are the most disruptive. Despite their practical importance, crash prediction and diagnosis in ML notebooks remain largely unexplored. We present CRANE-LLM, a runtime-augmented source code analysis approach that provides large language models (LLMs) with structured runtime information extracted from the notebook kernel, together with source code, to predict and diagnose crashes in a target cell before executing it. We evaluate CRANE-LLM on JunoBench, a benchmark of 111 Kaggle ML notebooks containing crashes across multiple ML libraries and crash types. Across three state-of-the-art LLMs (Gemini, Qwen, and GPT-5), our results show that runtime information significantly improves crash prediction and diagnosis performance by 7-10 percentage points in accuracy and 8-11 in F1-score, compared to using source code alone. The improvements are more pronounced when diagnosis is required, indicating that runtime context is especially valuable for reasoning about crash causes than merely predicting their presence.
From: Yiran Wang [view email]
[v1]
Fri, 20 Feb 2026 13:19:06 UTC (333 KB)
[v2]
Mon, 24 Aug 2026 13:38:13 UTC (330 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。