Sparse Autoencoders Find Highly Interpretable Features in Language Models

arxiv:2309.08600

Original source · Paper JSON

AI safety and consciousness · 2023-09-15

Cites (1) · Cited by (4)

Complete cleaned Markdown · Source Markdown including bibliography

Read in parts

1. Part 1
2. Part 2
3. Part 3

Contents

• Sparse Autoencoders Find Highly Interpretable Features in Language Models
• Abstract
• 1 Introduction
• 2 Taking Features out of Superposition with Sparse Dictionary Learning
• 3 Interpreting Dictionary Features
• 3.1 Interpretability at Scale
• 3.2 Sparse dictionary features are more interpretable than baselines
• 4 Identifying Causally-Important Dictionary Features for Indirect Object Identification
• 4.1 Adapting activation patching to dictionary features
• 4.2 Precise Localisation of IOI Dictionary Features
• 5 Case Studies
• 5.1 Input: Dictionary Features are Highly Monosemantic
• 5.2 Output: Dictionary Features have Intuitive Effects on the Logits
• 5.3 Intermediate Features: Dictionary Features Allow Automatic Circuit Detection
• 6 Discussion
• 6.1 Related Work
• 6.2 Limitations and Future Work
• 6.3 Conclusion
• Acknowledgments
• Appendix A Autointerpretation Protocol
• Appendix B Sparse Autoencoder Training and Hyperparameter Selection
• Appendix C Further Autointerpretation Results
• C.1 Interpretability is Consistent across Dictionary Sizes
• C.2 High Interpretability Scores Are Not an Artefact of Top Scoring
• C.3 Interpreting the MLP Sublayer
• C.4 Interpretability Scores Correlate with Kurtosis and Skew of Activation
• Appendix D Qualitative Feature Analysis
• D.1 Residual Stream Basis
• D.2 Examples of Learned Features
• D.3 Feature Search Details
• D.4 Failed Interpretability Methods
• Appendix E Number of Active Features
• Appendix F Editing IOI Behaviour on other Layers
• Appendix G Top K Comparisons
• Instructions for reporting errors