Evaluating the World Model Implicit in a Generative Model

arxiv:2406.03689

Original source · Paper JSON

Reasoning and the "physics" of language models · 2024-06-06

Cites (0) · Cited by (1)

Complete cleaned Markdown · Source Markdown including bibliography

Read in parts

1. Part 1
2. Part 2
3. Part 3
4. Part 4
5. Part 5

Contents

• Evaluating the World Model Implicit in a Generative Model
• Abstract
• 1 Introduction
• 2 Framework
• 2.1 Recovering world models
• Definition 2.1 .
• Definition 2.2 .
• Proposition 2.3 .
• 2.2 Next-token prediction is a fragile metric for recovering structure
• 2.3 The Myhill-Nerode interior and boundary
• Definition 2.4 .
• 2.4 Compression and distinction metrics for evaluating world models
• Definition 2.5 .
• Definition 2.6 .
• 3 Illustration: Do Transformers Recover the Street Map of New York City?
• 3.1 Data and models
• 3.2 Evaluating world models
• 3.3 Reconstructing implicit maps
• 3.4 Implication of failing to recover the world model: detour fragility
• 4 Other Applications: Othello and Logic Puzzles
• 5 Conclusion
• Acknowledgements
• Appendix A Proof
• Proof of Proposition 2.3 .
• Appendix B Reconstructed Maps
• B.1 Algorithm
• B.2 Maps
• Appendix C Deterministic Finite Automata and Myhill-Nerode
• Definition C.1 (Equivalent sequences) .
• Theorem C.2 ( [ 26 , 27 ] ) .
• Appendix D Additional results
• Appendix E Evaluation metric details and ablations
• E.1 Implementation details
• E.2 Ablations
• Appendix F Rides data construction and training
• Appendix G Additional maps
• Instructions for reporting errors