Reasoning with Sampling: Your Base Model is Smarter Than You Think

arxiv:2510.14901

Original source · Paper JSON

Post-training · 2025-10-16

Cites (3) · Cited by (3)

Complete cleaned Markdown · Source Markdown including bibliography

Read in parts

1. Part 1
2. Part 2
3. Part 3

Contents

• Reasoning with Sampling: Your Base Model is Smarter Than You Think
• Abstract
• 1 Introduction
• 2 Related Works
• Reinforcement learning for LLMs.
• Autoregressive MCMC sampling with LLMs.
• Annealed sampling for diffusion.
• 3 Preliminaries
• 4 MCMC Sampling for Power Distributions
• 4.1 Reasoning with Power Distributions
• Proposition 1 .
• Proof.
• Observation 1 .
• Example 1 .
• 4.2 The Metropolis-Hastings Algorithm
• Definition 1 .
• 4.3 Power Sampling with Autoregressive MCMC
• 5 Experiments
• 5.1 Experimental Setup
• Evaluation.
• Models.
• Sampling Algorithm.
• 5.2 Results
• Main results.
• 5.3 Analysis
• Reasoning trace likelihoods and confidences.
• Reasoning trace lengths.
• Diversity and pass@ k k performance.
• The effect of power distributions.
• Test-time scaling with MCMC steps.
• 6 Conclusion
• 7 Acknowledgments
• Appendix A Appendix
• A.1 Additional Theoretical Discussion
• Proposition 2 ( Informal ) .
• Definition 2 .
• Definition 3 .
• Definition 4 .
• Proposition 3 .
• Proof.
• A.2 Pass@k Accuracies over Multiple Domains
• A.3 More Qualitative Examples
• Instructions for reporting errors