Finetuning with Sampling: SFT Learns Better Than You Think

arxiv:2610.02140

Original source · Paper JSON

Post-training · 2026-10-01

Cites (6) · Cited by (0)

Complete cleaned Markdown · Source Markdown including bibliography

Read in parts

1. Part 1
2. Part 2
3. Part 3

Contents

• Finetuning with Sampling: SFT Learns Better Than You Think
• Abstract
• 1 Introduction
• 2 Related Works
• Self-Distillation.
• MCMC Sampling with LLMs.
• 3 Preliminaries
• Supervised Finetuning.
• 4 Boosting Off-Policy Data with MCMC Sampling
• 4.1 Targeting the Information Projection
• Definition 1 .
• Definition 2 .
• Proposition 1 .
• Proof.
• 4.2 The Metropolis-Hastings Algorithm
• Definition 3 .
• Proposition 2 .
• Proposition 3 .
• Proof.
• 4.3 Projection Sampling for LLMs
• 5 Experiments
• 5.1 Experimental Setup
• Tasks.
• Models.
• Evaluation.
• Sampling.
• Training.
• 5.2 Main Results
• 5.3 Analysis
• 6 Conclusion
• 7 Acknowledgements
• Appendix A Examples of Boosted Off-Policy Data
• Appendix B Additional Results
• Appendix C Further Experimental Details
• C.1 Information-Preservation
• C.2 Sampling
• C.3 Proposal and Grading Prompts
• Instructions for reporting errors