Annotator disagreement on tasks like natural languageinference (NLI) reflects genuine linguistic ambiguity,yet most fine-tuning recipes treat every example as equallylearnable.We ask whether this external signal of ambiguity predicts*per-example* learning dynamics under LoRA, the most widelyused parameter-efficient fine-tuning method, and find that it does.Correlating annotation entropy (from ChaosNLI’s 100 labels perexample) with per-example area under the loss curve (AULC)on SNLI and MNLI, the correlation is positive in all 25conditions tested (Spearman 𝜌= 0.06-0.43), withdecoder-only models showing stronger correlations thanencoders at matched LoRA rank.More strikingly, under LoRA contested examples exhibit*un-learning*: their gold-label loss *increases*during training, a pattern that is largely absent underfull fine-tuning and IA3 in the encoder setting wherematched comparisons are available, and that we also observeunder LoRA on two decoder-only models.The effect survives partial-correlation controls andreplicates across seeds and datasets.A preliminary noise-injection experiment is consistentwith these findings.
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。