LiveMathematicianBench: A Live Benchmark for Mathematician-Level Reasoning with Proof Sketches

arxiv:2604.01754

Original source · Paper JSON

Reasoning and the "physics" of language models · 2026-04-02

Cites (0) · Cited by (0)

Complete cleaned Markdown · Source Markdown including bibliography

Read in parts

1. Part 1
2. Part 2
3. Part 3
4. Part 4

Contents

• LiveMathematicianBench : A Live Benchmark for Mathematician-Level Reasoning with Proof Sketches
• Abstract
• 1 Introduction
• 2 Methods
• 3 Benchmark
• 4 Evaluation Results
• Even frontier models suffer.
• Category-wise pattern and monthly trajectories.
• Substitution-resistant options sharply reduce accuracy.
• Proof sketches yields a consistent improvement.
• 5 Endnote and Discussion
• Appendix A Related Work
• Standard and Advanced Mathematical Benchmarks
• Contamination Mitigation, Formal Mathematics, and Frontier-Level Evaluation
• Appendix B Human Validation Protocol
• Appendix C Detailed Logical Taxonomy of Theorem Categories
• 1. Algorithmic / Constructive.
• 2. Asymptotic / Limit.
• 3. Biconditional / Equivalence.
• 4. Classification / Bijection.
• 5. Existence.
• 6. Existential–Universal.
• 7. Implication.
• 8. Independence / Consistency.
• 9. Inequality / Bound.
• 10. Nonexistence.
• 11. Uniqueness.
• 12. Universal.
• 13. Universal–Existential.
• Appendix D Benchmark Composition Across Construction Stages
• Appendix E Evaluation Implementation Details
• Benchmark file format.
• Prompt construction.
• Answer extraction and scoring.
• Execution details.
• Resuming and result serialization.
• Appendix F Additional Evaluation Results
• F.1 Sketch-Aware Accuracy with Substitution-Resistant Settings
• F.2 Cost Analysis
• Appendix G Prompt Details
• Stage 4a: Question Stem and Correct Choice Generation.
• Stage 4b: Sketch-Adversarial Distractor Generation.
• Instructions for reporting errors