Abstract:Automatically predicting code comprehensibility could support tasks such as refactoring and code review. Existing metrics correlate poorly with human comprehension, motivating ML models that predict comprehensibility directly from code and developer features. Prior models predict absolute comprehensibility (AC), a comprehensibility value for an isolated snippet, but perform poorly since AC is a subjective proxy for a complex cognitive process. We propose relative comprehensibility (RC) as an alternative task: given two snippets, predict which is easier to understand or whether they are comparable. We hypothesize RC is easier to learn, since it only requires identifying distinguishing features between snippets rather than estimating absolute values. Using 150 Java snippets and 12,540 human comprehensibility measurements from two prior studies, we compare AC and RC prediction across classical ML models, a CNN, and two LLMs, evaluating both snippet-wise (aggregate) and developer-wise (individual-judgment) predictions. AC models rarely beat simple baselines (at most 33.4\% average relative improvement), while snippet-wise RC models outperform baselines in 96.8\% of configurations, with gains up to 159.8\% consistent across architectures, though developer-wise results are more variable. We surveyed 38 practitioners and found both AC and RC useful, with a stronger preference for RC in comparison-oriented tasks like refactoring and review.
From: Nadeeshan De Silva [view email]
[v1]
Fri, 3 Oct 2025 19:51:43 UTC (88 KB)
[v2]
Thu, 6 Aug 2026 15:47:27 UTC (776 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。