Self-Distillation Bridges Distribution Gap in Language Model Fine-Tuning

arxiv:2402.13669

Original source · Paper JSON

Post-training · 2024-02-21

Cites (4) · Cited by (3)

Complete cleaned Markdown · Source Markdown including bibliography

Read in parts

1. Part 1
2. Part 2

Contents

• Self-Distillation Bridges Distribution Gap in Language Model Fine-Tuning
• Abstract
• 1 Introduction
• 2 Related Work
• Fine-Tuning
• Continual Learning
• Alignment
• Prompting Based Learning
• 3 Method
• 3.1 Fine-tuning LLMs
• 3.2 Self-Distillation Fine-Tuning
• 3.3 Distillation Template
• 4 Experiments
• 4.1 Experimental Setup
• 4.2 Datasets for Fine-tuning and Evaluation
• 4.3 SDFT Achieves Better Results on Downstream Tasks
• 4.4 SDFT Preserves Alignment
• 4.5 General Knowledge Remains Intact
• 5 Analysis
• 5.1 Distribution Shift Correlates with Catastrophic Forgetting
• 5.2 Robustness among Distillation Templates
• 5.3 Efficacy of SDFT Across Model Scales and Architectures
• 6 Conclusions and Limitations
• Ethics Statement
• Acknowledgements
• Appendix A Experimental Details
• Appendix B Templates and Examples
• Instructions for reporting errors