{"id":"arxiv:2609.16454","title":"Fine-Tuning Fixes Mode Collapse and Over-Dispersion in LLMs","date":"2026-09-15","section":"Post-training","quality":null,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/96b9ce3c41bf5be0185a59605d2cade1d5dcdc9432040a1195506fac45cb96fa/","source_url":"https://arxiv.org/abs/2609.16454","text_available":true,"authors":["Kirill Skobelev","Eric Fithian","X. Y. Han"],"outgoing":[],"incoming":[],"text_parts":[{"start":0,"end":15499,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/96b9ce3c41bf5be0185a59605d2cade1d5dcdc9432040a1195506fac45cb96fa/part-1.html"},{"start":15499,"end":31021,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/96b9ce3c41bf5be0185a59605d2cade1d5dcdc9432040a1195506fac45cb96fa/part-2.html"},{"start":31021,"end":45785,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/96b9ce3c41bf5be0185a59605d2cade1d5dcdc9432040a1195506fac45cb96fa/part-3.html"},{"start":45785,"end":61555,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/96b9ce3c41bf5be0185a59605d2cade1d5dcdc9432040a1195506fac45cb96fa/part-4.html"},{"start":61555,"end":77534,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/96b9ce3c41bf5be0185a59605d2cade1d5dcdc9432040a1195506fac45cb96fa/part-5.html"},{"start":77534,"end":93310,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/96b9ce3c41bf5be0185a59605d2cade1d5dcdc9432040a1195506fac45cb96fa/part-6.html"},{"start":93310,"end":98879,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/96b9ce3c41bf5be0185a59605d2cade1d5dcdc9432040a1195506fac45cb96fa/part-7.html"}],"contents":[{"title":"Fine-Tuning Fixes Mode Collapse and Over-Dispersion in LLMs","offset":0,"level":1,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/96b9ce3c41bf5be0185a59605d2cade1d5dcdc9432040a1195506fac45cb96fa/part-1.html#text"},{"title":"Abstract","offset":63,"level":6,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/96b9ce3c41bf5be0185a59605d2cade1d5dcdc9432040a1195506fac45cb96fa/part-1.html#text"},{"title":"1 Introduction","offset":2727,"level":2,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/96b9ce3c41bf5be0185a59605d2cade1d5dcdc9432040a1195506fac45cb96fa/part-1.html#text"},{"title":"2 Related work","offset":7800,"level":2,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/96b9ce3c41bf5be0185a59605d2cade1d5dcdc9432040a1195506fac45cb96fa/part-1.html#text"},{"title":"3 Theory","offset":12794,"level":2,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/96b9ce3c41bf5be0185a59605d2cade1d5dcdc9432040a1195506fac45cb96fa/part-1.html#text"},{"title":"3.1 Sign and magnitude of the difference in collision probabilities","offset":16108,"level":3,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/96b9ce3c41bf5be0185a59605d2cade1d5dcdc9432040a1195506fac45cb96fa/part-2.html#text"},{"title":"Theorem 1 (Kernel-collision stability) .","offset":19770,"level":6,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/96b9ce3c41bf5be0185a59605d2cade1d5dcdc9432040a1195506fac45cb96fa/part-2.html#text"},{"title":"4 Experimental results","offset":21439,"level":2,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/96b9ce3c41bf5be0185a59605d2cade1d5dcdc9432040a1195506fac45cb96fa/part-2.html#text"},{"title":"4.1 Experiment 1: synthetic languages with known target distribution","offset":22268,"level":3,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/96b9ce3c41bf5be0185a59605d2cade1d5dcdc9432040a1195506fac45cb96fa/part-2.html#text"},{"title":"4.2 Experiment 2: sociological surveys (GSS, WVS, and ANES)","offset":27133,"level":3,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/96b9ce3c41bf5be0185a59605d2cade1d5dcdc9432040a1195506fac45cb96fa/part-2.html#text"},{"title":"4.3 Experiment 3: code generation using CodeNet dataset","offset":31835,"level":3,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/96b9ce3c41bf5be0185a59605d2cade1d5dcdc9432040a1195506fac45cb96fa/part-3.html#text"},{"title":"5 Conclusion","offset":34594,"level":2,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/96b9ce3c41bf5be0185a59605d2cade1d5dcdc9432040a1195506fac45cb96fa/part-3.html#text"},{"title":"AI use statement","offset":35615,"level":2,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/96b9ce3c41bf5be0185a59605d2cade1d5dcdc9432040a1195506fac45cb96fa/part-3.html#text"},{"title":"Appendix A Proofs and alignment conditions","offset":36027,"level":2,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/96b9ce3c41bf5be0185a59605d2cade1d5dcdc9432040a1195506fac45cb96fa/part-3.html#text"},{"title":"A.1 Kernel-collision stability","offset":36074,"level":3,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/96b9ce3c41bf5be0185a59605d2cade1d5dcdc9432040a1195506fac45cb96fa/part-3.html#text"},{"title":"A.2 A sufficient condition for nonnegative alignment","offset":37432,"level":3,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/96b9ce3c41bf5be0185a59605d2cade1d5dcdc9432040a1195506fac45cb96fa/part-3.html#text"},{"title":"Assumption 1 (Target–bias comonotonicity) .","offset":37740,"level":6,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/96b9ce3c41bf5be0185a59605d2cade1d5dcdc9432040a1195506fac45cb96fa/part-3.html#text"},{"title":"A.3 Relative collision bounds and population cross-entropy","offset":38988,"level":3,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/96b9ce3c41bf5be0185a59605d2cade1d5dcdc9432040a1195506fac45cb96fa/part-3.html#text"},{"title":"Proposition 1 (Relative bound on the collision ratio) .","offset":39555,"level":6,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/96b9ce3c41bf5be0185a59605d2cade1d5dcdc9432040a1195506fac45cb96fa/part-3.html#text"},{"title":"Appendix B Synthetic alignment before and after fitting","offset":43593,"level":2,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/96b9ce3c41bf5be0185a59605d2cade1d5dcdc9432040a1195506fac45cb96fa/part-3.html#text"},{"title":"Appendix C Complete survey results","offset":45331,"level":2,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/96b9ce3c41bf5be0185a59605d2cade1d5dcdc9432040a1195506fac45cb96fa/part-3.html#text"},{"title":"Appendix D Mitigation methods and finite-sample estimation","offset":45722,"level":2,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/96b9ce3c41bf5be0185a59605d2cade1d5dcdc9432040a1195506fac45cb96fa/part-3.html#text"},{"title":"Appendix E Sequence-level bounds and response length","offset":48471,"level":2,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/96b9ce3c41bf5be0185a59605d2cade1d5dcdc9432040a1195506fac45cb96fa/part-4.html#text"},{"title":"Appendix F KL-regularized preference optimization","offset":51257,"level":2,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/96b9ce3c41bf5be0185a59605d2cade1d5dcdc9432040a1195506fac45cb96fa/part-4.html#text"},{"title":"Appendix G Empirical check of the bound","offset":53157,"level":2,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/96b9ce3c41bf5be0185a59605d2cade1d5dcdc9432040a1195506fac45cb96fa/part-4.html#text"},{"title":"Appendix H Experimental details","offset":53991,"level":2,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/96b9ce3c41bf5be0185a59605d2cade1d5dcdc9432040a1195506fac45cb96fa/part-4.html#text"},{"title":"H.1 Synthetic languages","offset":54027,"level":3,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/96b9ce3c41bf5be0185a59605d2cade1d5dcdc9432040a1195506fac45cb96fa/part-4.html#text"},{"title":"H.2 Surveys","offset":58983,"level":3,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/96b9ce3c41bf5be0185a59605d2cade1d5dcdc9432040a1195506fac45cb96fa/part-4.html#text"},{"title":"H.3 CodeNet","offset":66649,"level":3,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/96b9ce3c41bf5be0185a59605d2cade1d5dcdc9432040a1195506fac45cb96fa/part-5.html#text"},{"title":"Appendix I Tabular intuition for finite-sample collision","offset":71899,"level":2,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/96b9ce3c41bf5be0185a59605d2cade1d5dcdc9432040a1195506fac45cb96fa/part-5.html#text"},{"title":"I.1 Expected collision inflation","offset":73072,"level":3,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/96b9ce3c41bf5be0185a59605d2cade1d5dcdc9432040a1195506fac45cb96fa/part-5.html#text"},{"title":"Example 1 (Why any finite sample inflates collision (Jensen)) .","offset":76580,"level":6,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/96b9ce3c41bf5be0185a59605d2cade1d5dcdc9432040a1195506fac45cb96fa/part-5.html#text"},{"title":"Remark 1 (This is tabular, not neural) .","offset":77485,"level":6,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/96b9ce3c41bf5be0185a59605d2cade1d5dcdc9432040a1195506fac45cb96fa/part-5.html#text"},{"title":"I.2 Amplification over sequences","offset":77839,"level":3,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/96b9ce3c41bf5be0185a59605d2cade1d5dcdc9432040a1195506fac45cb96fa/part-6.html#text"},{"title":"I.3 Entropy analogue","offset":79518,"level":3,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/96b9ce3c41bf5be0185a59605d2cade1d5dcdc9432040a1195506fac45cb96fa/part-6.html#text"},{"title":"Appendix J Limits of temperature scaling","offset":80009,"level":2,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/96b9ce3c41bf5be0185a59605d2cade1d5dcdc9432040a1195506fac45cb96fa/part-6.html#text"},{"title":"Proposition 2 (Global temperature monotonically increases order-2 diversity) .","offset":80505,"level":6,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/96b9ce3c41bf5be0185a59605d2cade1d5dcdc9432040a1195506fac45cb96fa/part-6.html#text"},{"title":"J.1 The temperature-reachable diversity interval","offset":82257,"level":3,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/96b9ce3c41bf5be0185a59605d2cade1d5dcdc9432040a1195506fac45cb96fa/part-6.html#text"},{"title":"Lemma 1 (Temperature-reachable diversity interval) .","offset":82804,"level":6,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/96b9ce3c41bf5be0185a59605d2cade1d5dcdc9432040a1195506fac45cb96fa/part-6.html#text"},{"title":"Theorem 2 (Temperature only redistributes existing support) .","offset":85707,"level":6,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/96b9ce3c41bf5be0185a59605d2cade1d5dcdc9432040a1195506fac45cb96fa/part-6.html#text"},{"title":"Lemma 2 (Temperature scaling is an order-preserving one-parameter family) .","offset":89404,"level":6,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/96b9ce3c41bf5be0185a59605d2cade1d5dcdc9432040a1195506fac45cb96fa/part-6.html#text"},{"title":"J.2 Effective support","offset":91591,"level":3,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/96b9ce3c41bf5be0185a59605d2cade1d5dcdc9432040a1195506fac45cb96fa/part-6.html#text"},{"title":"Proposition 3 (Effective absence of valid missing mass under sampling) .","offset":92223,"level":6,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/96b9ce3c41bf5be0185a59605d2cade1d5dcdc9432040a1195506fac45cb96fa/part-6.html#text"},{"title":"Example 2 (A two-peaked target temperature cannot reach) .","offset":94894,"level":6,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/96b9ce3c41bf5be0185a59605d2cade1d5dcdc9432040a1195506fac45cb96fa/part-7.html#text"},{"title":"Instructions for reporting errors","offset":97915,"level":2,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/96b9ce3c41bf5be0185a59605d2cade1d5dcdc9432040a1195506fac45cb96fa/part-7.html#text"}],"text_code_points":98879,"source_markdown":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/96b9ce3c41bf5be0185a59605d2cade1d5dcdc9432040a1195506fac45cb96fa/source.md"}