Recursive Language Models — part 5/5

Cleaned-text code points [62808, 67338). Bibliography omitted here; source.md includes it.

As mentioned previously, Qwen3-Coder differs from GPT-5 in how liberal it is in its use of sub-calls. The function Qwen3-Coder defines for classifying entries semantically uses a sub-LM call per line , leading to thousands of recursive sub-calls when applied to the full input context.

Step 2. After defining and testing several functions for running the above classification question over its input context, the root LM launches a long code execution call to classify and answer the query.

Final. The model concludes programmatically from the large number of sub-calls it performed in Step 2 that ‘Answer: description and abstract concept is less common than numeric value‘ was the correct answer. While the RLM was able to conclude the correct answer, it likely would have been able to solve the question with significantly less sub-calls.

### E.4 RLM(GPT-5) on CodeQA-Query_44

The total cost of this trajectory was $0.27 . In this task, the agent must answer a question that involves understanding a large codebase. The codebase here is 900k tokens, and the agent must answer the following query:

⬇

Step 1. It is not always true that an input context can be solved by partitioning it and recursively sub-querying models over each partition, but in tasks that are not information dense, this is possible. In this case, the model chooses to break down the codebase into parts and sub-query LMs to look for clues. The model then aggregates these clues and provides a final answer as a separate sub-query.

Final. The RLM answers choice ‘1’, which is the correct answer.

## Appendix F Additional Quantitative Results

### F.1 Additional Quantitative Analysis of Main Results

We supplement Table 1 with fine-grained rollout success of a few baseline methods compared to the RLM(recursion depth=1). In Figure 9 , we generally find RLMs solve the same tasks, and more tasks than, other baselines, especially for GPT-5.

We also explore how sub-calling behavior differs between rollouts. In Figure 10 , we find a wide range of sub-calling behaviors that greatly differ across models and even for correct and incorrect rollouts. For example, GPT-5 uses significantly more sub-calls for BrowseComp-Plus than any other model. However, for OOLONG, Qwen3-Coder uses a large number of sub-calls ( 500 on average) for correct rollouts, which is significantly more than the number used by GPT-5. Furthermore, Qwen3-8B in particular tends to use more sub-calls on incorrect trajectories.

### F.2 Additional Runtime and Cost Analysis of RLMs

We supplement the cost and runtime analysis of RLMs with additional, fine-grained plots. We focus on RLMs with depth=0 (i.e. no sub-calls) and depth=1. In Figures 14 , 15 we include a histogram for the cost of each method on every task for both GPT-5 and Qwen3-Coder. We generally observe long-tailed, high-variance trajectories for RLMs in both models. We plot the cost of RLM(depth=1) and baselines at quartiles in Figure 11 .

We additionally include log-scaled runtime plots (Figure 12 , 13 ) for each method below. The tail end (e.g. 95th percentile) shows extremely long runtimes, which is mainly due to sequential sub-LLM calls taking up most of the runtime. However, we observe these cases happen infrequently, and can be early-stopped with timeout logic. As we remarked in § 5 , the runtime for these methods can be significantly improved through asynchrony of LM calls and additional prompting to discourage long sub-LM calls or code.

For the scaling plot in Figure 1 , we also provide the average API cost per task.

## Instructions for reporting errors

We are continuing to improve HTML versions of papers, and your feedback helps enhance accessibility and mobile support. To report errors in the HTML that will help us improve conversion and rendering, choose any of the methods listed below:

Click the "Report Issue" ( ) button, located in the page header.

Tip: You can select the relevant text first, to include it in your report.

Our team has already identified the following issues . We appreciate your time reviewing and reporting rendering errors we may not have found yet. Your efforts will help us improve the HTML versions for all readers, because disability should not be a barrier to accessing research. Thank you for your continued support in championing open access for all.

Have a free development cycle? Help support accessibility at arXiv! Our collaborators at LaTeXML maintain a list of packages that need conversion , and welcome developer contributions .