Recursive Language Models — part 3/5

Cleaned-text code points [31526, 47080). Bibliography omitted here; source.md includes it.

We trained RLM-Qwen3-8B as a small-scale exercise in training the first natively recursive language model. We hypothesized that, though acting as an RLM appears to produce sophisticated behavior due to recursion, it can be sufficient to focus on improving the root LM’s ability to interact with the programmatic representation of the prompt in the REPL and to discern when sub-calls are useful. In other words, while a typical RLM trajectory can be extremely long due to all of the sub-calls potentially launched (possibly Ω ⁡ ( | P | ) \Omega(|P|) for a prompt P P ), the leaf sub-calls are essentially general-purpose LLM requests and the major hurdle is learning to operate as the root model.

This simple insight allowed us to explore a similarly simple recipe for training. In particular, we sampled RLM trajectories from a larger language model (Qwen3-Coder-480B-A35B-Instruct; Qwen Team 2025b ) and, after filtering, distilled them to a smaller model (Qwen3-8B; Qwen Team 2025a ) from the same model family. We evaluated RLM(Qwen3-Coder-480B-A35B) on 750 English LongBenchPro [ Chen et al., 2026 ] tasks, collecting a total of 2250 candidate trajectories.

We first remove trajectories that score exactly 0.0 on the benchmark or do not go beyond one turn, bringing it down to 1,072 candidate trajectories. We separated each root RLM turn (i.e. iteration) as a separate SFT sample consisting of an input (the full history) and output (the output the root LM gave at that step).

We then applied a filtering step to remove turns beyond the context limit of Qwen3-8B (we approximated this as 100k characters), and also applied an extra programmatic correction step to fix small template mistakes in RLM usage (e.g. outputting final answers, calling the REPL, etc.). To elaborate, we noticed that trajectories generated by Qwen3-Coder-480B-A35B had noticeable mistakes in following the RLM instructions, which hurt the performance of the distilled RLM-Qwen3-8B. For example, it would often mix FINAL(answer) with FINAL(variable in REPL). We added an extra programmatic fixing step to look for common templated mistakes and patch them, leading to much better performance in the final RLM-Qwen3-8B . In total, 16% of turns incorrectly used FINAL answers, and 13% of turns incorrectly called a variable from the REPL (i.e. FINAL_VAR) as a final answer. In Figure 5 , we show pre- and post-filtering statistics for our training trajectories.

We used the prime-rl library [ Prime Intellect Team et al., 2025 ] for fine-tuning. We used a batch size of 64 for 300 training steps, training for 48 H100 hours. While this exceedingly simple training recipe was able to demonstrate substantial gains for our 8B model, we call on future work to investigate training native RLMs much more thoroughly. We expect that doing so at much larger scales in terms of model size, number and variety of examples, and number of (ideally on-policy and online) rollouts will be necessary to maximize the potential of RLMs.

Below, we provide plots for the runtime speed-up of training in Figure 6 .

MRCRv2 training. For the MRCRv2 [ Vodrahalli et al., 2024 ] training experiment, we similarly used prime-rl library [ Prime Intellect Team et al., 2025 ] , but on Prime Intellect’s host-training platform Lab . We RL trained on the 32k-64k token split with 2 needles for 150 steps with a batch size of 128 and 4 rollouts per example. We set a max output token per turn at 4096, and set the max number of RLM iterations to 20. Every 50 steps (starting from 0 0 ), we evaluated on the 512K-1M token split with 8 needles.

## Appendix B Negative Results: Things We Tried That Did Not Work.

Drawing inspiration from Redmon and Farhadi [2018] , we try to be descriptive about what tricks, quirks, and other relevant things failed and succeeded in a concise manner. Some observations are based on longer supplementary experiments, while others are based on small samples of results.

Using the exact same RLM system prompt across all models can be problematic. We originally wrote the RLM system prompt with in context examples for GPT-5, and tried to use the same system prompt for Qwen3-Coder, but found that it led to different, undesirable behavior in the trajectory. We had to add a small sentence to the RLM system prompt for Qwen3-Coder to prevent it from using too many recursive sub-calls.

Models without sufficient coding capabilities struggle as RLMs. Our instantiation of RLMs relies on the ability to reason through and deal with the context in a REPL environment. We found from small scale experiments that smaller models like Qwen3-8B [ Yang et al., 2025 ] struggled without sufficient coding abilities.

Thinking models without sufficient output tokens struggle as RLMs. In addition to Qwen3-Coder-480B-A35B-Instruct , we also tried experimenting with Qwen3-235B-A22B as the RLM. While we found positive results across the board from the base model (e.g. on OOLONG [ Bertsch et al., 2025 ] , performance jumped from 30 % ~30\% to 38 % ~38\% ), the smaller gap compared to the evaluated models in the main experiments (Table 1 ) are due to multiple trajectories running out of output tokens while producing outputs due to thinking tokens exceeding the maximum output token length of an individual LM call.

RLMs without asynchronous LM calls are slow. We implemented all sub-LM queries naively as blocking / sequential calls, which caused our RLM experiments to be slow, especially compared to just the base model. We are confident that this can be resolved with a robust implementation.

Depending on the model, distinguishing between a final answer and a thought is brittle for RLMs. The current strategy for distinguishing between a “next turn" and a final answer for the RLM is to have it wrap its answer in FINAL() or FINAL_VAR() tags. Similar to intuition about structured outputs degrading performance, we also found the model to make strange decisions (e.g. it outputs its plan as a final answer). We added minor safeguards, but we also believe this issue should be avoided altogether in the future when models are trained as RLMs.

## Appendix C Additional Methods and Baseline Details

### C.1 Prompts for Experiments

We focus on methods that are entirely task agnostic, so we fix our prompt for each method across all tasks. For the RLM prompt, the only difference between GPT-5 and Qwen3-Coder is an added line in the beginning that warns Qwen3-Coder not to use too many sub-LM calls – we found in practice that without this warning, the model will try to perform a subcall on everything, leading to thousands of LM subcalls for basic tasks. For the fine-tuned Qwen3-8B experiment, we provide a slightly different prompt due to the differences in context window size of the smaller model (from 272k in GPT-5 to 32k in Qwen3-8B). In this section, we provide the system prompt used for all methods in § 3.2 (other than the base model, which does not include a system prompt).

(1a) The system prompt for RLM(depth=1) for GPT-5: ⬇

(1b) The diff of the system prompt for RLM with REPL (Qwen3-Coder-480B-A35B) , which adds a line from the prompt above for GPT-5: ⬇

(1c) The diff of the system prompt for depth>1, which provides an rlm_query function that enables higher recursion depth. ⬇

(1d) The diff of the system prompt for RLM(Qwen3-8B, depth=1) , which has a few changes from the GPT-5 prompt due to differences in context length and similar sub-calling behavior as Qwen3-Coder-480B-A35B: ⬇

(2) The system prompt for RLM with REPL (no sub-calls) : ⬇

(3a) The system prompt for CodeAct with BM25 . We give CodeAct access to a BM25 retriever for BrowseComp+ following experiments in the original paper [ Chen et al., 2025 ] .: ⬇

(3b) The system prompt for CodeAct . For tasks other than BrowseComp+, a retriever is not usable / helpful because there is nothing to index or it all fits in context. We modify the prompt to remove the retriever.: ⬇

### C.2 Summary agent baseline

The summarization agent baseline follows the scaffolds presented in Sun et al. [2025] , Wu et al. [2025] , Yu et al. [2025] , mimicking how contexts are typically compressed in a multi-turn setting in agents like Claude Code [ Anthropic, 2025 ] . In an iterative fashion, the agent is given inputs until its context is full, at which point it is queried to summarize all relevant information and continue. If the agent is given a context in a single step that is larger than its model context window, it chunks up this context and performs the summarization process over these chunks.

For our GPT-5 baseline, we chose to use GPT-5-nano to perform summarization to avoid exploding costs. This explains the large discrepancy in cost in Table 1 between GPT-5 and Qwen3-Coder on BrowseComp-Plus, where the summary agent using Qwen3-Coder is nearly 15 × 15\times more expensive on average. On this task in particular, we found on a smaller set of 20 20 random samples that the performance between using GPT-5 and GPT-5-nano is comparable.

### C.3 LongCoT-mini experiment.

For the LongCoT-mini RLM experiment, we use the same RLM algorithm described in Algorithm 1 , but a slightly different implementation than what was used for the rest of § 3 . Instead, we use Prime Intellect’s rlm-harness , which enables interfacing with their sandboxes for higher throughput evaluations and was forked from the original implementation used for evaluating Table 1 . The mechanism for determining final answers also differs, which is reflected in the prompt.

Why GPT-5 base does not include decomposition hints. Even when provided with decomposition hints, we find that GPT-5 cannot reasonably execute this decomposition and solve sub-problems using the standard chain-of-thought autoregressive reasoning. While performance on the MATH split improves, we generally find the model gets confused on the more programmatic tasks without a REPL-like mechanism to isolate sub-task solving.

The appended environment hint used for LongCoT-mini with decomposition hints on solving these problems is provided below: ⬇

## Appendix D Additional Benchmark Details

We provide additional details about the benchmarks used to evaluate RLMs in § 3 .

### D.1 OOLONG-Pairs Benchmark

OOLONG-Pairs consists of 20 20 synthetically generated tasks based on the ground-truth labels for the OOLONG Bertsch et al. [2025] trec_coarse split for input contexts of length in [1024, 2048, 4096, 8192, 16384, 32768, 65536, 131072, 262144, 524288, 1048576]. Similar to OOLONG, each question requires correctly predicting the semantic mapping for each entry.

OOLONG-Pairs ensures quadratic scaling . Many tasks that aggregate over pairs of entries can actually be solved without looking at the pairs and only looking at each entry in a linear fashion (e.g. using the principle of inclusion-exclusion in set theory). However, in OOLONG-Pairs, each question is created such that the model must return all pairs satisfying some properties, rather than just counting.

Task 1 In the above data, list all pairs of user IDs (no duplicate pairs, list lower ID first) where both users have at least one instance with a numeric value or location. Each of the questions can be labelled as one of the labels (the data does not provide the labels, you need to figure out the label from the semantics of the question): description and abstract concept, entity, human being, numeric value, location, abbreviation. In your answer, list all pairs in the format (user_id_1, user_id_2), separated by newlines.

Task 2 In the above data, list all pairs of user IDs (no duplicate pairs, list lower ID first) where both users have at least one instance with an entity or human being. Each of the questions can be labelled as one of the labels (the data does not provide the labels, you need to figure out the label from the semantics of the question): description and abstract concept, entity, human being, numeric value, location, abbreviation. In your answer, list all pairs in the format (user_id_1, user_id_2), separated by newlines.

Task 3 In the above data, list all pairs of user IDs (no duplicate pairs, list lower ID first) where both users have at least one instance with a description and abstract concept or abbreviation. Each of the questions can be labelled as one of the labels (the data does not provide the labels, you need to figure out the label from the semantics of the question): description and abstract concept, entity, human being, numeric value, location, abbreviation. In your answer, list all pairs in the format (user_id_1, user_id_2), separated by newlines.

Task 4 In the above data, list all pairs of user IDs (no duplicate pairs, list lower ID first) where both users have at least one instance with a human being or location, and all instances that are a human being for both users must be after January 6, 2023. Each of the questions can be labelled as one of the labels (the data does not provide the labels, you need to figure out the label from the semantics of the question): description and abstract concept, entity, human being, numeric value, location, abbreviation. In your answer, list all pairs in the format (user_id_1, user_id_2), separated by newlines.

Task 5 In the above data, list all pairs of user IDs (no duplicate pairs, list lower ID first) where both users have at least one instance with an entity or numeric value, and all instances that are an entity for both users must be before March 15, 2023. Each of the questions can be labelled as one of the labels (the data does not provide the labels, you need to figure out the label from the semantics of the question): description and abstract concept, entity, human being, numeric value, location, abbreviation. In your answer, list all pairs in the format (user_id_1, user_id_2), separated by newlines.

Task 6 In the above data, list all pairs of user IDs (no duplicate pairs, list lower ID first) where both users have at least one instance with a location or abbreviation. Each of the questions can be labelled as one of the labels (the data does not provide the labels, you need to figure out the label from the semantics of the question): description and abstract concept, entity, human being, numeric value, location, abbreviation. In your answer, list all pairs in the format (user_id_1, user_id_2), separated by newlines.

Task 7 In the above data, list all pairs of user IDs (no duplicate pairs, list lower ID first) where both users have at least one instance with a description and abstract concept or numeric value, and all instances that are a numeric value for both users must be after February 1, 2023. Each of the questions can be labelled as one of the labels (the data does not provide the labels, you need to figure out the label from the semantics of the question): description and abstract concept, entity, human being, numeric value, location, abbreviation. In your answer, list all pairs in the format (user_id_1, user_id_2), separated by newlines.

Task 8 In the above data, list all pairs of user IDs (no duplicate pairs, list lower ID first) where both users have at least one instance with a human being or description and abstract concept. Each of the questions can be labelled as one of the labels (the data does not provide the labels, you need to figure out the label from the semantics of the question): description and abstract concept, entity, human being, numeric value, location, abbreviation. In your answer, list all pairs in the format (user_id_1, user_id_2), separated by newlines.