LADDER: Self-Improving LLMs Through Recursive Problem Decomposition — part 3/3

Cleaned-text code points [31670, 45395). Bibliography omitted here; source.md includes it.

### 4.2 MIT integration Bee (LADDER)

Applying our methodology to the MIT Integration Bee led to a significant improvement in model performance. LADDER improved the accuracy of the Deepseek-R1 Qwen 2.5 7B base model from a baseline of 50% to 73% on the 2025 qualifying examination, significantly outperforming other existing models, including GPT-4o (42%). See Figure 5 .

For comparison, most human participants score between 3-6 points out of 20 (15-30%), with scores above 7-12 (35-73%) considered strong performances. With a 73% accuracy, our model meets the qualification threshold of 14/20. However, while LADDER achieved substantial gains, it still did not reach the performance of o1 (80%).

The consistent performance gap between our trained model and base models highlights the effectiveness of our variant-based recursive training approach in improving mathematical reasoning. Despite the relatively modest parameter count of 7B, our base model was able to significantly outperform larger models that did not undergo targeted recursive training. This suggests that structured self-improvement methodologies can lead to substantial gains in problem-solving ability without requiring massive increases in model size.

### 4.3 MIT Integration Bee (LADDER + TTRL)

After applying LADDER to the 7B base model, we further employed Test-Time Reinforcement Learning (TTRL) for 100 steps on questions that LADDER failed to answer correctly. Of the remaining incorrect responses, TTRL successfully solved 3-4 additional problems, increasing performance from 73% to 90% (Figure 6 ). This improvement places our model’s performance well above the typical qualification threshold of 73% for the MIT Integration Bee, setting a new state-of-the-art for mid-sized LLMs on this benchmark.

In Figure 6 , we compare LADDER pass@100 with TTRL, demonstrating that TTRL is a more effective test-time scaling method than naively attempting a test question many times. TTRL’s application of recursive problem decomposition on the test questions themselves allows the initial LADDER model to correctly solve some of the hardest questions on the MIT Integration Bee qualifying examination. TTRL enables our relatively small 7B LADDER model to outperform OpenAI’s o1 model at pass@1.

The number of RL steps required to solve each test question previously incorrectly answered varied, ranging from as few as 3 steps to as many as 30. This variation likely reflects the complexity of each integral, with more complex problems requiring a greater number of solved variants to develop a useful learning trajectory. The questions that remained unsolved even after LADDER + TTRL are particularly complex for undergraduate students and are included in the appendix.

To assess the necessity of LADDER in TTRL’s effectiveness, we also applied TTRL directly to the base model (pre-LADDER). After 100 steps, the base model remained unable to correctly answer any of the questions that LADDER + TTRL successfully solved. Performance gains from TTRL only occurred when applied to the LADDER-trained model, suggesting that the RL protocol in LADDER provides essential exposure to the distribution of integrals, enabling TTRL to later learn further. Future research could explore whether TTRL alone, with sufficiently high compute constraints, could achieve comparable results.

We note that while our LADDER+TTRL model surpasses o1’s score on the MIT Integration Bee, this comparison should be taken as a general baseline rather than a strict head-to-head evaluation. Unlike our approach, o1 does not have access to a numerical checker, meaning it operates under different constraints. Our results highlight the effectiveness of self-improvement via recursive problem decomposition and reinforcement learning rather than suggesting a direct superiority over o1’s approach.

## 5 Discussion

Our results demonstrate a potentially transformative approach to AI self-improvement through recursive problem decomposition and solution verification. The key insight is that models can direct their own learning trajectory by generating and solving progressively more challenging problems, requiring only a formal verifier rather than human guidance or curated datasets. This represents a significant shift from traditional supervised learning or RLHF approaches, suggesting a path toward more autonomous AI systems capable of extending their own capabilities.

The effectiveness of this self-directed learning is evident in our empirical results: improvements from 1% to 82% on undergraduate integration problems and from 50% to 73% on the MIT Integration Bee, achieved with LADDER, without human feedback or model scaling. With additional variant-based learning at test-time (TTRL), we achieve a state-of-the-art score of 90%. The 7B parameter model’s ability to outperform much larger architectures suggests that improvements in AI capabilities may come not just from scaling compute or parameters, but from enabling models to structure their own learning progression.

This methodology’s recursive capability makes it particularly powerful. When faced with problems beyond their current abilities, models can autonomously decompose them into simpler variants, creating their own curriculum that adapts to their evolving capabilities. The implications extend beyond mathematical reasoning - suggesting a general framework for AI systems that can bootstrap more sophisticated capabilities from simpler ones, guided only by formal verification mechanisms rather than human oversight.

### 5.1 A Form of Test-Time Compute Scaling

Test-time compute scaling represents a promising new frontier in enhancing model capabilities beyond traditional approaches like architectural scaling or increased parameter counts. LADDER and TTRL introduce a fundamentally different paradigm: rather than relying on larger models or more extensive pre-training, we can achieve substantial performance improvements by enabling models to strategically practice and learn during inference. This approach is particularly powerful because it allows models to develop problem-specific expertise dynamically, adapting their capabilities to the exact challenges they encounter.

The dominant approach to test-time compute scaling today concentrates on increasing output token length, allowing models to engage in more extensive step-by-step reasoning. However, this approach faces fundamental challenges with extremely difficult problems that require extensive exploration. The sequential nature of token generation creates memory constraints and potential bottlenecks, as each token must be generated and processed in sequence while maintaining the entire chain of reasoning in context.

TTRL offers a fundamentally different approach that mirrors human learning strategies. Just as humans often need to study and practice similar problems before tackling a particularly challenging question, TTRL enables models to systematically explore and learn from related problems before attempting the target task. This approach is inherently more parallelizable than sequential token generation - variant problems can be generated and solved independently across multiple compute units, with their insights aggregated to improve performance on the original problem. While our current implementation explored variants sequentially, the framework naturally extends to distributed training approaches, potentially offering significant speedups through parallel exploration of the problem space.

This parallelizability presents a key advantage over traditional test-time scaling approaches. Where increasing output length faces fundamental sequential constraints, TTRL’s variant generation and learning process can be distributed across multiple GPUs or machines. Each compute unit can independently explore different regions of the problem space, generating and solving variants while sharing insights through model updates. This mirrors distributed training approaches in the pre-training phase, but applied dynamically at test-time to specific challenging problems.

### 5.2 Extension to Other Verifiable Domains

While we demonstrated our approach using numerical integration, the underlying principle can extend to a broad range of formal reasoning tasks through appropriate verification tools. It is likely that any domain where question variants can be generated and which has a verifier-generator gap can leverage our approach. These domains share the critical property we identified in integration: a clear generator-verifier gap where solution verification is more straightforward than solution generation. This suggests our approach of iterative variant generation and verified learning could provide a general framework for improving formal reasoning capabilities in language models.

Competitive programming is an example of another possible domain in which our method may be effective. An LLM could generate a solution to a programming problem and verify it with provided unit tests. In formal mathematics, tools like the Lean proof assistant offer another avenue for verification. Lean can rigorously verify mathematical proofs, providing a reliable signal for whether a model’s reasoning is correct.

More generally, this approach has potential applications in planning and agent-based tasks, where models can teach themselves by successfully executing simpler tasks before building up to more complex challenges. This progressive learning approach, combined with reliable verification at each step, provides a robust framework for developing advanced reasoning capabilities.

### 5.3 Future Work

A core challenge in optimizing LADDER lies in the precise calibration of variant difficulty and curriculum sequencing. While our current approach uses static parameters for variant generation, an adaptive strategy could dynamically adjust the difficulty gradient based on model performance. Rather than generating a fixed number of variants at predetermined difficulty levels, the framework could analyze solution success rates to determine optimal branching points and difficulty steps. This dynamic calibration would ensure each generated variant meaningfully contributes to the learning trajectory while avoiding redundant or inappropriately difficult examples that waste computational resources. The balance between exploration of novel problem patterns and exploitation of known solution strategies remains to be optimized.

## 6 Conclusion

We proposed LADDER, a framework for improving language models’ problem-solving capabilities through recursive problem decomposition and reinforcement learning with verifiable rewards, as well as its extension to test-time, TTRL. By enabling models to generate and learn from progressively simpler variants of complex problems, LADDER demonstrates how strategic self-directed learning can achieve significant improvements without relying on architectural scaling or human supervision. The dramatic improvements in mathematical reasoning capabilities - from 1% to 82% on undergraduate integration problems and achieving 90% accuracy on the MIT Integration Bee through TTRL - demonstrate the effectiveness of this approach.

Beyond mathematical integration, our work builds upon three insights in general AI development: (1) the power of recursive problem decomposition for tackling complex tasks, (2) the importance of verifiable rewards for guiding self-improvement, and (3) the potential of scaling compute through strategic practice at test-time. The success of both LADDER during training and TTRL at inference time reinforces that focusing on how models interact with their environment is just as important as architectural innovations for advancing AI capabilities.

The principles demonstrated in this paper could likely extend to any domain with clear verification mechanisms, from program synthesis to formal theorem proving. By providing a framework for models to bootstrap their own capabilities through carefully constructed learning trajectories, we move closer to AI systems that can systematically extend their abilities into increasingly complex domains.

## Appendix: MIT Integration Bee Failed Questions

After applying LADDER and TTRL, there were two questions on the 2025 MIT Integration Bee qualifying exam which continued to be answered incorrectly. The questions incorrectly answered are among the most complex questions on the exam, and for an undergraduate student are very difficult to solve.

Question 12: ∫ x ⋅ x ⋅ x ⋅ x ⋯ 6 5 4 3 ​ 𝑑 x \displaystyle\int\sqrt[3]{x\cdot\sqrt[4]{x\cdot\sqrt[5]{x\cdot\sqrt[6]{x\cdots}}}}\,dx Question 13: ∫ e 2 ​ x ​ ( x 2 + x ) ( x ​ e x ) 4 + 1 , 𝑑 x \displaystyle\int\frac{e^{2x}(x^{2}+x)}{(xe^{x})^{4}+1},dx

## Instructions for reporting errors

We are continuing to improve HTML versions of papers, and your feedback helps enhance accessibility and mobile support. To report errors in the HTML that will help us improve conversion and rendering, choose any of the methods listed below:

Click the "Report Issue" ( ) button, located in the page header.

Tip: You can select the relevant text first, to include it in your report.

Our team has already identified the following issues . We appreciate your time reviewing and reporting rendering errors we may not have found yet. Your efforts will help us improve the HTML versions for all readers, because disability should not be a barrier to accessing research. Thank you for your continued support in championing open access for all.

Have a free development cycle? Help support accessibility at arXiv! Our collaborators at LaTeXML maintain a list of packages that need conversion , and welcome developer contributions .