How should we continually post-train LLMs without causing catastrophic forgetting?

We find that forgetting is not simply caused by large parameter updates. Instead, it is better understood as a state-relative update-integration failure: harmful steps occur when a new task update becomes geometrically incompatible with the current LLM state shaped by previous updates.

We formalize this through geometry conflict, a signal based on the covariance geometry induced by task updates. Across Qwen3 0.6B--14B, we compare geometry conflict with update norm, subspace alignment, and gradient conflict, and show that state-relative geometry conflict better captures when transfer or interference occurs.

Based on this insight, we propose Geometry-Conflict Wasserstein Merging (GCWM), a data-free continual update-integration method that uses geometry conflict to gate Wasserstein-based merge correction. GCWM improves retention and final performance across domain-continual and capability-continual settings without replay data.

The main takeaway: continual post-training should not only control how far an LLM moves, but whether each new update remains geometrically compatible with the evolving model state.