How much do language models memorize?
arxiv:2505.24832
Data, training, optimization · 2025-05-30
Complete cleaned Markdown · Source Markdown including bibliography
Read in parts
Contents
- How much do language models memorize?
- Abstract
- 1 Introduction
- 2 Memorization, intended and unintended
- 2.1 Warm-up: A statistical view of memorization
- Proposition 1 (Super-additivity of Unintended Memorization) .
- 2.2 Measuring unintended memorization with Kolmogorov Complexity
- Definition 2 (Kolmogorov complexity) .
- Definition 3 (Kolmogorov memorization) .
- Proposition 4 .
- 2.3 Estimating Kolmogorov with compression
- Choice of reference model.
- 3 Model Capacity for Memorization
- 3.1 Defining model capacity
- Definition 5 (Capacity) .
- 3.2 Measuring model capacity with synthetic sequences
- Experimental details.
- Results.
- How does precision affect capacity?
- 4 Disentangling Unintended Memorization from Generalization
- Experimental details.
- Results.
- Dataset-to-capacity ratio predicts double descent.
- Generalization explains nonzero extraction rates.
- 5 Memorization and Membership
- 5.1 Membership in synthetic and text data
- Synthetic data.
- Text.
- 5.2 Scaling laws for Membership
- 5.2.1 Functional forms
- Limiting behavior.
- Fitting.
- 5.2.2 Validation on larger models
- 6 Related Work
- Language models and compression.
- Language model capacity.
- Information regularization in learning theory.
- Alternative definitions of memorization.
- Prior definitions of memorization.
- 7 Conclusion
- 8 Acknowledgements
- Appendix A Appendix
- A.1 How reliable are our linear estimates of capacity?
- A.2 Additional memorization results
- A.3 Comparison of distributions memorized
- Distribution-level analysis.
- Which datapoints are most memorized?
- A.4 Scaling law fit
- A.5 Proofs
- A.6 Proof of Proposition 1
- Proof.
- A.7 Proof of Proposition 4
- Proof.
- Lemma 6 .
- A.8 Limitations
- Appendix B Discussion of other notions of memorization
- Instructions for reporting errors