View PDF HTML (experimental)

Abstract:Legacy modernization breaks business logic more often than most teams expect. Most tools and LLM-based approaches treat modernization as syntax translation: convert COBOL to Java, swap PL/SQL for Python, ship it. Implicit rules, edge-case handling, and cross-module constraints that keep production systems running are lost, and nobody notices until something fails in production.
We present AgentModernize, a multi-agent framework that treats modernization as a behavioral preservation problem. Four agents handle extraction, specification, code generation, and validation. The key intermediate artifact, a Behavioral Specification Graph (BSG), forces extracted business logic to be explicit and inspectable before any code is generated.
We evaluated on LegacyModernize-8, eight synthetic scenarios spanning telecom and banking, under a fair protocol where each method's tests are generated from its own API surface (3 trials, temperature 0.0). With GPT-4o-mini, AgentModernize with feedback achieves 23.0% mean BER (non-zero on 5/8 scenarios, up to 53.3%), while AgentModernize without feedback reaches 23.8%. SP-LLM scores 12.4% (non-zero on 2/8) and CoT-LLM 4.5% (2/8). No single method dominates all scenarios. The feedback loop is decisive for scenarios requiring iterative correction (S4, S6, S7) but can regress code in others (S2, S8). The BSG captures 92.3% of gold-standard rules with 90.2% precision; the bottleneck is code generation, not extraction. A cross-model study (GPT-4o, GPT-5.3-codex) reveals the pipeline's benefit is inversely correlated with model capability: stronger models achieve higher BER with single-prompt methods than with the pipeline. For regulated industries, the pipeline's traceable artifacts (business rule inventory, BSG, equivalence reports) provide an audit trail that no single-prompt approach can match.

Submission history

From: Sheikh Nazib Ahmed [view email]
[v1] Sun, 17 May 2026 16:39:48 UTC (23 KB)
[v2] Tue, 4 Aug 2026 06:17:41 UTC (20 KB)