Abstract:Supervised Fine-Tuning (SFT) is essential for aligning Large Language Models (LLMs) with user intent, yet it is believed to suppress generative diversity. Although this reduction is frequently referenced, formal empirical testing of the phenomenon remains limited. The expressiveness of LLMs by itself was addressed by multiple prior methods. Their varying perspectives suggest that deeper analysis could yield further improvements. In this study, we attribute the decline to two primary drivers: the neglect of low-frequency patterns within fine-tuning datasets and the forgetting of preexisting knowledge. Motivated by our theoretical analysis, we develop Tempered Focal (TOFU) loss, a novel objective that addresses both stated challenges simultaneously. Our extensive evaluation confirms at scale that generation breadth narrows after SFT and strengthens the hypothesis explaining this effect. Across multiple models and benchmarks, we demonstrate that TOFU enhances output diversity while preserving high response quality, offering a principled approach to SFT.
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2605.00195 [cs.LG] |
| (or arXiv:2605.00195v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2605.00195 arXiv-issued DOI via DataCite (pending registration) |
From: Oleksandr Cherednichenko [view email]
[v1]
Thu, 30 Apr 2026 20:20:59 UTC (218 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。