SIM-CoT: Supervised Implicit Chain-of-Thought

arxiv:2509.20317

Original source · Paper JSON

Reasoning and the "physics" of language models · 2025-09-24

Cites (6) · Cited by (3)

Complete cleaned Markdown · Source Markdown including bibliography

Read in parts

1. Part 1
2. Part 2
3. Part 3
4. Part 4

Contents

• SIM-CoT: Supervised Implicit Chain-of-Thought
• Abstract
• 1 Introduction
• 2 Analysis of Implicit COT: the Latent Instability Issue
• 3 Methodology
• 3.1 Notation
• 3.2 Implicit Phase: Latent Construction by Last Hidden States
• 3.3 Explicit Phase: Answer Decoding over the Vocabulary
• 3.4 Training-time Decoder and Step-level Supervision
• 3.5 Objectives
• 4 experiment
• 4.1 Experimental Setup
• 4.2 Main Results
• 4.3 Ablation Studies
• 5 Related Work
• 6 Conclusion
• APPENDIX
• Usage of Large Language Models
• Outline
• Appendix A Additional Analysis on Scaling to Larger Backbones
• Appendix B Additional Analysis on Decoder Sizes
• Appendix C Additional Analysis on Soft Thinking
• C.1 Setup
• C.2 Results
• C.3 Formulation
• Step 1. Vocabulary distribution.
• Step 2. Soft-thinking embedding.
• Step 3. Combination.
• C.4 Analysis
• Appendix D Related Work
• Appendix E Implementation and Training Details
• E.1 Coconut Training Setup
• E.2 CODI Training Setup
• E.3 Summary of Hyperparameters
• Appendix F Training and Inference Details
• Curriculum for K K .
• Inference and Efficiency.
• Benchmark Detail.
• Training Data.
• Appendix G SIM-CoT Training Implementation
• Appendix H Geometric Diagnostics of the Latent Space
• Appendix I Additional Details for Interpretability Analysis
• I.1 Making Implicit Reasoning Visible
• I.2 Distance Analysis
• I.3 Summary
• Appendix J Future Directions
• Appendix K Additional Case Studies on GSM8k
• Instructions for reporting errors