Olmo 3 — part 8/19

Cleaned-text code points [108755, 124702). Bibliography omitted here; source.md includes it.

The main challenge in generating synthetic data for long-context understanding is the bootstrap problem: how can we create effective data without having access to models that can process long context? Our pipeline uses document statistics to identify the most important terms and then extracts snippets containing those terms. Those snippets are subsequently provided to a language model to create aggregation tasks. In detail: 1. For a given document of length n n tokens, we partition the document into m m sections of length 8K to 32K tokens. We attempt to place these partitions near natural breaks in the document flow, such as right before new sections;

2. For each partition, we normalize and tokenize the text, extract one- and two-word noun phrases, and use tf-idf to identify the most salient noun phrases;

3. For each noun phrase, we select k = 8 k=8 snippets of text from the partition, ranked by tf-idf ;

4. We pass the noun phrases, (optional) snippets, and one or more prompts describing the aggregation task to a language model.

For Olmo 3 , we use documents where 32,768 ≤ n < 65,536 32,768\leq n<65,536 tokens, resulting in 2 to 8 partitions per document. While we experimented with several closed and open language models, we ultimately use OLMo 2 Instruct 32B for all generations.

##### Synthetic aggregation tasks

We consider two aggregation tasks; we refer the reader to the code implementation 27 27 27 github.com/allenai/dolma3/datasets/dolma3_longmino_mix/synthetic_cwe_rex/longmino_synthetic_cwe_rex.py for the exact prompts used.

• CWE (Common Word Extraction) We prompt OLMo 2 Instruct with 5 commonly occurring single-word noun phrases in the partition, and ask the model to generate diverse QA pairs that require the answer to be the exact number of times each unigram occurs in the partition;

• REX (Rewriting EXpressions) For each noun phrase and corresponding snippets, we prompt OLMo 2 Instruct to generate an aggregation task matching one of the following 12 vignettes discussing the noun phrase: a short summary, a dialogue between a professor and student, a simple paragraph for high school students, a set of flashcards, a school quiz, a game show, a dinner party, a debate, a list of true or false claims, a movie scene, an encyclopedic description, or an explainer in the style of conversations on the r/explainlikeimfive subreddit.

#### 3.6.3 Choosing Data Mix and Token Budget

##### Interleaving long- and short-context data

Rather than training on only long-context data, we mix high-quality short-context data from midtraining (stage two) to ensure that performance on short-context tasks is not meaningfully degraded. Early experiments on a 10B-token extension show that a 66% / 34% mix of long-context to short-context data drops performance on a subset of OlmoBaseEval by 2.5 points; in comparison, a 34% long-context, 66% short-context mix only drops performance by 0.8 points.

##### Longer extension helps

Figure 13(e) shows that allocating more tokens to the long-context extension stage improves performance on long-context tasks, particularly at longer sequence lengths. We extend the context of Olmo 3 7B through a 50B stage 3 training; for Olmo 3 32B, we extend for 100B tokens for better long-context capabilities.

#### 3.6.4 Curating a Training Recipe for Extension

##### RoPE extension

Olmo 3 uses RoPE ( Su et al., 2024 ) to encode positional information within the transformer architecture. We experiment with several methods for extending RoPE beyond the original pretraining context length, including adjusted base frequency scaling ( Xiong et al., 2023 ; Rozière et al., 2024 ) , position interpolation ( Chen et al., 2023 ) , and YaRN ( Peng et al., 2023 ) . Each approach is applied either to all RoPE instances or is restricted to RoPE used in full attention layers. We find that applying YaRN only to full attention layers yields the best overall performance.

##### Document packing

During pretraining and midtraining, we follow the standard approach of concatenating documents and splitting them into fixed-length training sequences. However, when extending the context length, this strategy produces training instances that are, on average, shorter than the underlying document length distribution. To address this, we adopt best-fit document packing ( Ding et al., 2024 ) , which reduces the number of split documents while adding a negligible amount of padding. Compared to the naive concatenate-then-split approach, best-fit packing yields substantially improved performance on long-context benchmarks.

##### Intra-document masking

During long-context extension, we apply intra-document masking to ensure that each training sequence attends only to tokens originating from the same underlying document ( Zhao et al., 2024b ; Grattafiori et al., 2024 ) . This prevents the model from being distracted by cross-document signals, which can otherwise introduce spurious attention patterns and degrade long-range performance.

##### LC training infrastructure

To extend the model to a 65K-token context window, we employ 8-way context parallelism (CP) so that each device processes 8K tokens from each training instance. We adopt the all-gather-based CP attention strategy introduced by Chu et al. (2025) , which makes it straightforward to support irregular attention masks, including sliding-window and intra-document masking. For parallelism configurations, infrastructure details, and throughput measurements, see Appendix Table 34 .

##### Model souping

Following performance improvements from merging midtraining runs for Olmo 3 Base 32B, we experiment with averaging long-context checkpoints. In this case, rather than running long-context extension multiple times with different seeds, we merge the last three checkpoints from the end of the extension run (at steps 10,000, 11,000, and 11,921) to produce our final long-context Olmo 3 Base 32B.

### 3.7 Base Model Results

In Table 13 we outline the results of Olmo 3 Base after the pretraining, midtraining, and long-context extension stages, comparing performance to other open base models. Compared to OLMo 2 , the Olmo 3 models demonstrate clear improvements on science, math, and code-based evaluation metrics, which we attribute largely to our emphasis and upsampling of STEM-related data during the pretraining and midtraining stages. On the other hand, because of this emphasis on STEM, we see slight degradation in general knowledge benchmarks.

## 4 Olmo 3 Think

We train Olmo 3 Think to reason by first generating extended thought sequences and then producing a final answer (Figure 2 ). To achieve this, we curate high-quality reasoning data ( Dolci Think ), harness a three-stage training recipe (SFT, DPO, and RLVR), and introduce OlmoRL approach, which merges our new algorithmic and engineering advances with a strong community platform of research in reinforcement learning with verifiable rewards.

Through these data, training, and algorithmic innovations, Olmo 3 Think achieves strong performance in math, coding, reasoning, and general conversation. At the 32B scale, it stands as the best fully-open thinking model, outperforming Qwen 2.5 32B, Gemma 2 and 3 27B, and narrowing the gap to top open-weight systems like Qwen 3 32B while being trained on fewer FLOPs ( Table 14 ).

1. Data: Dolci Think Building on prior open-source datasets ( Guha et al., 2025a ; Lambert et al., 2024 ; PrimeIntellect, 2025 ) inter alia , we introduce Dolci Think SFT , Dolci Think DPO , and Dolci Think RL , new cutting-edge post-training datasets designed to target a broad range of key capabilities such as math, coding, instruction following, and general conversation. The dataset includes synthetic examples with long thinking traces for supervised finetuning, high-contrast paired data for contrastive learning via preference optimization, and challenging prompts for reinforcement learning across diverse domains. Our data curation pipeline is shown in Figure 15 .

2. Three-Stage training recipe We employ a three-stage post-training process comprising Supervised Finetuning (SFT), Preference Finetuning via Direct Preference Optimization (DPO), and then Reinforcement Learning with Verifiable Rewards (RLVR). We observe consistent gains across all three stages, demonstrating the impact of careful data curation, algorithmic refinement, and infrastructure development. This contrasts with most recent prior work on open thinking models, which typically employs only a subset of these training stages. 28 28 28 More concretely, OpenThought3 and S1 only used supervised finetuning; SmolLM used SFT and DPO, but did not apply RL. For example, we find that our RL framework yields greater improvements when applied after contrastive learning with DPO rather than directly following SFT ( Figure 19 ).

3. OlmoRL We present OlmoRL , our RL training approach which builds upon GRPO and extends it with improvements from recent work. Additionally, we expand verifiable reasoning to multiple domains, going beyond the math and code settings typically explored in prior work. OlmoRL enables longer and more stable RL runs across diverse domains and increases the overall efficiency of training cycles (Section § 4.4 ).

### 4.1 Main Results for Olmo 3 Think

#### 4.1.1 Evaluation Details

We establish a suite of benchmarks to evaluate Olmo 3 post-trained models on math, reasoning, coding, precise instruction following, question answering, knowledge recall, and general chat. We expand upon the evaluation suite of OLMo 2 ( OLMo et al., 2024 ) by adding new, more challenging benchmarks and removing saturated or noisy ones. Table 16 shows our evaluation benchmarks and describes the task configurations and metrics for the Olmo 3 post-training evaluation suite. We establish a standard evaluation configuration between all baseline models, including thinking and instruct models, to simplify comparisons. Namely, we follow Guo et al. (2025) ; Adler et al. (2024) ; Yang et al. (2025a) and use a 32K max context length, a sampling temperature of 0.6 and top-p of 0.95. Note, some models likely perform better with a higher inference budget, for instance K2 V2 ( Team et al., 2025 ) use a 128K sequence length. Further details of our evaluation settings are provided in Appendix A.8 .

Evaluation with reasoning models is both computationally expensive and often high variance. In our development of our recipe on versions of our 7B model—i.e., before the hyperparameter sweeps for final models—we find that evaluation costs between 10 and 20% of our compute budget. When compiling results, we measure the variance of every evaluation in our suite by taking the mean of the standard deviation from 3 runs of 14 models (both baselines and our final models). By taking the variance per model and then the average variance per evaluation, we can bucket the evaluations by their variance. We partition evaluations based on their variance as follows: • High variance : GPQA: 1.4798, AlpacaEval 3: 1.2406, IFEval: 0.8835.

• Stable : ZebraLogic: 0.5638, Omega: 0.5579, AIME 24 (Avg@32): 0.5437, HumanEvalPlus: 0.4615, AgiEval: 0.4339, BigBenchHard: 0.3866.

• Very stable : LiveCodeBench (Avg@10): 0.2852, MBPPPlus: 0.2749, MATH: 0.2522, MMLU: 0.2219, PopQA: 0.1554.

Olmo 3 32B Think Baselines SFT DPO Final Think 3.0 Final Think 3.1 Qwen 3 32B Qwen 3 VL 32B Think DS-R1 32B K2-V2 70B Instruct Math MATH 95.6 95.9 96.1 96.2 95.4 96.7 92.6 94.5 AIME 2024 73.5 76.0 76.8 80.6 80.8 86.3 70.3 78.4 AIME 2025 66.2 70.7 72.5 78.1 70.9 78.8 56.3 70.3 OMEGA 43.1 45.2 50.6 53.4 47.7 50.8 38.9 46.1 Reasoning BigBenchHard 88.8 89.1 89.8 88.6 90.6 91.1 89.7 87.6 ZebraLogic 70.5 74.5 76.0 80.1 88.3 96.1 69.4 79.2 AGI Eval English 85.9 87.8 88.2 88.8 90.0 92.2 88.1 89.6 Coding HumanEvalPlus 90.0 91.6 91.4 91.5 91.2 90.6 92.3 88.0 MBPP+ 66.7 67.2 68.0 68.3 70.6 66.2 70.1 66.0 LiveCodeBench v3 75.8 81.9 83.5 83.3 90.2 84.8 79.5 78.4 IF IFEval 83.9 80.6 89.0 93.8 86.5 85.5 78.7 68.7 IFBench 37.0 34.4 47.6 68.1 37.3 55.1 23.8 46.3 Knowledge & QA MMLU 85.3 85.2 85.4 86.4 88.8 90.1 88.0 88.4 PopQA 33.1 37.0 31.9 30.9 30.7 32.2 26.7 32.2 GPQA 55.7 57.6 58.1 56.7 67.3 67.4 61.8 64.0 Chat AlpacaEval 2 LC 69.1 78.6 74.2 69.1 75.6 80.9 26.2 - Safety 64.8 65.3 68.8 83.6 69.0 82.7 63.6 88.5

Olmo 3 7B Think Baselines SFT DPO Final Think OpenThinker3 7B Nemotron Nano 9B v2 DS-R1 Qwen 7B Qwen 3 8B Qwen 3 VL 8B Think OR Nemotron 7B Math MATH 94.4 92.4 95.1 94.5 94.4 87.9 95.1 95.2 94.6 AIME 2024 69.6 74.6 71.6 67.7 72.1 54.9 74.0 70.9 77.0 AIME 2025 57.6 62.7 64.6 57.2 58.9 40.2 67.8 61.5 73.1 OMEGA 37.8 40.5 45.0 38.4 42.4 28.5 43.4 38.1 43.2 Reasoning BigBenchHard 84.1 83.7 86.6 77.1 86.2 73.5 84.4 86.8 81.3 ZebraLogic 57.9 60.6 66.5 34.9 60.8 26.1 85.2 91.2 22.4 AGI Eval English 77.2 79.1 81.5 78.6 83.1 69.5 87.0 90.1 81.4 Coding HumanEvalPlus 88.2 91.4 89.9 87.4 89.7 83.0 80.2 83.7 89.7 MBPP+ 63.2 63.0 64.7 61.4 66.1 63.5 69.1 63.0 61.2 LiveCodeBench v3 67.8 75.1 75.2 68.0 83.4 58.8 86.2 85.5 82.3 IF IFEval 77.9 75.9 88.2 51.7 86.0 59.6 87.4 85.5 42.5 IFBench 30.0 28.3 41.6 23.0 34.6 16.7 37.1 40.4 23.4 Knowledge & QA MMLU 74.9 74.8 77.8 77.4 84.3 67.9 85.4 86.5 80.7 PopQA 20.8 24.7 23.7 18.0 17.9 12.8 24.3 29.3 14.5 GPQA 45.8 48.6 46.2 47.6 56.2 54.4 57.7 61.5 56.6 Chat AlpacaEval 2 LC 43.9 50.6 52.1 24.0 58.0 7.7 60.5 73.5 8.6 Safety 65.8 67.7 70.7 31.6 72.1 54.0 68.3 82.9 30.3

#### 4.1.2 Main Results

Table 14 and Table 15 show the performance of Olmo 3 Think across different training stages and compare it with other baselines of similar scale on our benchmarks 29 29 29 Running AlpacaEval on K2-V2-Instruct led to token-parsing errors on the output of the LLM judge, resulting in null preference scores. If we are able to devise a solution, we will update the report accordingly. . As described before, Olmo 3 Think 32B is the best fully-open model at the 32B scale, outperforming other models including Gemma 2 27B, Gemma 3 27B, and Qwen 2.5 32B-Instruct. It narrows the gap to the best open-weight models at this scale, Qwen 3 and Qwen 3VL, while being trained with 6x fewer tokens. Similarly, Olmo 3 Think -7B outperforms OpenReasoning Nemotron 7B, DeepSeek-R1-Distill-Qwen-7B, and OpenThinker-7B, some of the best open-weight thinking models. In addition, it performs similarly to Nemotron-Nano-9B-v2 despite being smaller. At 7B, it lags the Qwen 3 series of models in knowledge tasks. We think that this is mainly due to the fact that Qwen 3 models are trained through distillation from Qwen’s largest model.

Notably, we introduce Olmo 3.1 Think 32B to illustrate that extended OlmoRL training, via additional epochs on our Dolci Think RL dataset 30 30 30 While Olmo 3 Think 32B was trained for 750 steps, we continued the run past our initial release, going up to 2300 steps for Olmo 3.1 Think 32B. We stopped there due to compute limitations, but note that performance had not yet fully saturated , suggesting even longer runs could further improve performance. , leads to improved performance. We observe substantial improvements on math, reasoning, and instruction-following benchmarks, including gains of 4+ points on AIME, 4 points on ZebraLogic, 4 points on IFEval, and 20 points on IFBench, suggesting the additional RL training improves the model’s reasoning abilities. Most other benchmarks remain largely unchanged, with the exception of AlpacaEval, where we observe a 5-point drop.

### 4.2 Supervised Finetuning with Dolci Think SFT

In this stage, we construct Dolci Think SFT , a resource for finetuning the base model to produce explicit thinking traces that support accurate responses. This supervised finetuning step is especially impactful for smaller models, offering an efficient mechanism for acquiring strong reasoning capabilities. We next detail the Dolci Think SFT data curation pipeline (Figure 15 ).

#### 4.2.1 Dolci Think SFT : Data Curation