The Unreasonable Effectiveness of Entropy Minimization in LLM Reasoning

arxiv:2505.15134

Original source · Paper JSON

Reasoning and the "physics" of language models · 2025-05-21

Cites (9) · Cited by (5)

Complete cleaned Markdown · Source Markdown including bibliography

Read in parts

1. Part 1
2. Part 2
3. Part 3
4. Part 4
5. Part 5

Contents

• The Unreasonable Effectiveness of Entropy Minimization in LLM Reasoning
• Abstract
• 1 Introduction
• 2 Background and Related Work
• 2.1 Related Work
• 2.2 Preliminaries
• 3 Entropy Minimization for Post-training
• 3.1 EM-FT : Direct Entropy Minimization through Token-Level Finetuning
• 3.2 EM-RL : Entropy Minimization with Negative Entropy Rewards in RL
• 3.3 Training Setup
• 3.4 Results: Entropy Minimization for Post-Training
• 4 EM-INF: Inference-time Logit Optimization
• Proposition 1 .
• 5 Limitations: Dependence on Base Model Capability and Confidence
• 6 Conclusion
• Acknowledgments and Disclosure of Funding
• Appendix A Proof for proposition 1
• Appendix B Adaptive Temperature Scaling
• Appendix C Additional Results
• C.1 Combining EM-INF with Self-consistency and Iterative Self-refinement
• C.2 Qualitative analysis of EM-INF on SciCode
• Appendix D Experimental Setup
• D.1 Evaluation Benchmarks
• D.1.1 Math Reasoning
• MATH 500
• AMC
• AIME
• Minerva Math
• OlympiadBench
• D.1.2 Coding
• LeetCode
• LiveCodeBench
• D.1.3 Scientific Reasoning
• UGPhysics
• SciCode
• D.1.4 Individualistic Value Reasoning
• IndVal
• D.2 Training Data
• Math Reasoning
• Coding
• Individualistic Value Reasoning
• D.3 Models
• Qwen2.5 Family
• Llama-3.1 Family
• D.4 FLOPs Computation
• Instructions for reporting errors