Olmo 3 — part 17/19
Cleaned-text code points [250350, 265504). Bibliography omitted here; source.md includes it.
where U t U_{t} is the set of unique n-grams in the training document segment and U e U_{e} is the set of unique n-grams in the evaluation document.
##### Cluster match length decay
Less informative short texts require stronger matches.
O ′ = O × { 1 if L ≤ L start 1 − 0.2 L − L start L end − L start if L start < L < L end 0.8 if L ≥ L end O^{\prime}=O\times\begin{cases}1&\text{if }L\leq L_{\text{start}}\\ 1-0.2\frac{L-L_{\text{start}}}{L_{\text{end}}-L_{\text{start}}}&\text{if }L_{\text{start}}<L<L_{\text{end}}\\ 0.8&\text{if }L\geq L_{\text{end}}\end{cases}
By default L_start is set by the configuration perfect_match_decay_start: 20 and L_end is set by the configuration perfect_match_decay_end: 50 .
##### Cluster discovery threshold
For efficiency we check that the question match O ′ O^{\prime} exceeds the minimum question match required to ultimately call contamination. Every candidate contamination that exceeds this value will get a complete scoring, which includes answer and passage information.
##### Non-comparative
Note that we pre-compute the idf sums for evals during index construction. At detection time we sum the calculated idfs for matching ngrams to produce the overlap ratios. There is no string to string comparison. We rely on the nature of n-gram shingles for sequence matching. The probability of having a substantial ngram shingle overlap is low, and while degenerate cases are possible, they have not been observed in practice.
##### Inverted index
Because | E | |E| is relatively small, we build an inverted index in memory which maps ngrams to document ids. We use a two tiered index, the first maps a u64 hash to a u32 sequential id assigned at index construction. And the second tier maps the n-gram id to a set of document ids. This oddity is done to achieve performant membership tests of training ngrams in the significantly smaller set of observed eval ngrams. Consider that the | G tn | ≪ | G en | |G_{\text{tn}}|\ll|G_{\text{en}}| , so the supermajority of ngram lookups are misses, and skipped. The u32 sequential id is empirically more performant than a one-tiered lookup with document id sets as values.
##### Hot n-grams
Cluster discovery begins with an initial hit in the inverted index. While the supermajority of ngrams samples are misses, there are some extremely common ngrams present in the eval texts. Because the ngrams are so common, the probability of a initial hit leading to a true instance of contamination is low. As an optimization we do not start contamination cluster expansion on hot ngram hits, but rather switch our sampling rate to 1, and traverse the training document by single tokens until we observe a miss or non-hot ngram hit.
#### A.5.5 Scoring System
Scores combine question, answer, and passage overlaps with adaptive weights based on the length of components:
• QAP (all components): 0.7 question, 0.2 answer, 0.1 passage
• QA (no passage): 0.75 question, 0.25 answer
• QP (no answer): 0.85 question, 0.15 passage
• Q (question only): 1.0 question
##### Length penalty
We penalize short matches based on the length of Q+A+P by scaling down scores for shorter texts, making the contamination threshold effectively harder to reach. Shorter texts get their scores scaled down before threshold comparison. The scaling factor depends on the total token length L total L_{\text{total}} :
S final = S base × scaleFactor ( L total ) S_{\text{final}}=S_{\text{base}}\times\text{scaleFactor}(L_{\text{total}})
Where the scaling factor decreases for shorter texts, making the threshold effectively harder to reach. Perfect scores (1.0) are never penalized.
##### Confidence adjusted weight
The question component is the core of a contaminated prompt and carries the most weight. But in some cases an eval will have short questions and long answers or a long passage followed by a short question about it.
Because longer sequences with more informative content provide stronger contamination evidence, we adjust component weights based on confidence factors derived from length by reducing the question weight and redistributing it to the answer or passage.
Question confidence, based on unique n-gram count: C q = { 0.5 + 0.5 N q 20 if N q < 20 1 if N q ≥ 20 C_{q}=\begin{cases}0.5+0.5\frac{N_{q}}{20}&\text{if }N_{q}<20\\ 1&\text{if }N_{q}\geq 20\end{cases}
Base weights are adjusted by confidence factors:
W adjusted = W default ⋅ C + W redistributed W_{\text{adjusted}}=W_{\text{default}}\cdot C+W_{\text{redistributed}}
Where low-confidence components redistribute their weight to higher-confidence ones.
##### Base scores
• Q composition : S base = O q S_{\text{base}}=O_{q}
• QA composition : S base = O q ⋅ W q , adjusted + O a ⋅ W a , adjusted S_{\text{base}}=O_{q}\cdot W_{q,\text{adjusted}}+O_{a}\cdot W_{a,\text{adjusted}}
• QP composition : S b a s e = O q ⋅ W q , adjusted + O p ⋅ W p , adjusted S_{base}=O_{q}\cdot W_{q,\text{adjusted}}+O_{p}\cdot W_{p,\text{adjusted}}
• QAP composition : S base = O q ⋅ W q , adjusted + O a ⋅ W a , adjusted + O p ⋅ W p , adjusted S_{\text{base}}=O_{q}\cdot W_{q,\text{adjusted}}+O_{a}\cdot W_{a,\text{adjusted}}+O_{p}\cdot W_{p,\text{adjusted}}
##### Answer proximity
For QA datasets, contamination requires the answer appears near the question cluster. Short answers use exact token matching; long answers use n-gram overlap with IDF weighting.
##### Passage proximity
For datasets with passages, contamination checks if the passage appears within a configurable distance ( min_passage_distance ) from the question cluster. Passages use n-gram overlap with IDF weighting and can tolerate gaps ( passage_max_consecutive_misses ).
### A.6 Post-Training Additional Training Details
#### A.6.1 Supervised Finetuning Details
##### Using OLMo-core infrastructure for SFT Training
Relative to pretraining, this involves a substantially smaller batch size, different data packing, and masking. This leads to an 8x faster training speed than open-instruct, dramatically improving our iteration speed. We use between 1 and 8 8xH100 nodes, or 1 to 4 8xB200 nodes to train our 7B reasoner and instruct models. We use 32 8xH100 nodes to train our 32B thinking model As a consequence of using olmo-core, our batch size is now measured in tokens instead of instances, and we train with document packing instead of padding. We train all of our 7B SFT models with a batch size of 1M tokens and 32B SFT models with a batch size of 4M tokens, for two epochs, with packing, and a 32,768 sequence length. Our hyperparameter settings are also summarized in Table 47 .
7B Thinking SFT 32B Thinking SFT 7B Instruct SFT Total Tokens 45.4B 45.2B 3.4B Learning Rate 5.0 × 10 − 5 5.0\times 10^{-5} 1.0 × 10 − 4 1.0\times 10^{-4} souped with 5.0 × 10 − 5 5.0\times 10^{-5} 8.0 × 10 − 5 8.0\times 10^{-5} Num. GPUs 64 256 8-64 Max Sequence Length 32K 32K 32K
#### A.6.2 Preference Tuning Details
##### Training Settings
Given a preference dataset 𝒟 = { ( x , y c , y r ) } \mathcal{D}=\left\{(x,y_{c},y_{r})\right\} of prompts x x and corresponding chosen and rejected responses y c ≻ y r y_{c}\succ y_{r} , we optimize the model policy π θ \pi_{\theta} on a length-normalized DPO loss ( Lambert et al., 2024 ) : max π θ 𝔼 ( x , y c , y r ) ∼ 𝒟 [ log σ ( β | y c | log π θ ( y c | x ) π ref ( y c | x ) − β | y r | log π θ ( y r | x ) π ref ( y r | x ) ) ] \displaystyle\max_{\pi_{\theta}}\mathbb{E}_{(x,y_{c},y_{r})\sim\mathcal{D}}\left[\log\sigma\left(\frac{\beta}{|y_{c}|}\log\frac{\pi_{\theta}(y_{c}|x)}{\pi_{\text{ref}}(y_{c}|x)}-\frac{\beta}{|y_{r}|}\log\frac{\pi_{\theta}(y_{r}|x)}{\pi_{\text{ref}}(y_{r}|x)}\right)\right] where π ref \pi_{\text{ref}} is the initial reference policy and β \beta is a hyperparameter that regularizes learning via an implicit Kullback–Leibler (KL) divergence penalty between the reference policy and the training policy.
We sweep learning rate and preference dataset size, as we observe that performance increases up until some task-dependent optimal optimization point beyond which further tuning hurts ( Figure 23 ). All other hyperparameters are kept fixed. See Table 48 for exact hyperparameters. We train our 7B models using 2–4 8xH100 nodes, and our 32B models with 8–16 8xH100 nodes.
7B Thinking DPO 32B Thinking DPO 7B Instruct DPO Num. Preference Pairs 150K 200K 260K Num. Epochs 1 1 1 DPO β \beta 5 5 5 Learning Rate 8.0 × 10 − 8 8.0\times 10^{-8} 7.0 × 10 − 8 7.0\times 10^{-8} 1.0 × 10 − 6 1.0\times 10^{-6} LR Schedule Linear decay Linear decay Linear decay Warmup Ratio 0.1 0.1 0.1 Num. GPUs 32 64-128 16 Batch Size 128 128 128 Max Sequence Length 16K 8K 16K
#### A.6.3 Reinforcement Learning Details
We provide full training curves for our 7B reasoner in Figure 41 . The overall reward increases steadily over training. The KL divergence grows gradually and reflects stronger deviation from the reference policy. The response length becomes longer and stabilizes at a higher level. Domain-specific verifier rewards display consistent gains in math and moderate fluctuations in code. The IfEval reward rises throughout training. The two general-quality verifiers also show clear and sustained improvement. Together, these trends indicate that the policy improves both specialized skills and overall response quality. The full hyperparameters for all RL experiments are provided in in Table 49 .
#### A.6.4 RL-Zero Details
We detail the prompt used for math in Figure 37 . Prompts of other domains are quite similar, see the open-instruct codebase for details.
We also compare Olmo 3 RL-Zero 7B to one of the more common benchmarks in RLVR, DAPO ( Yu et al., 2025 ) in Figure 38 . Olmo 3 RL-Zero achieves reasonable performance faster and is also much more compute efficient, making it better for experimentation.
Finally, we compare Olmo RL-Zero 3.1 to the initially released, RL-Zero 3.0 in Figure 39 and see a sizable improvement. There were some minor fixes to loss calculation but the major improvement comes from 1. setting completion length to 16k instead of 12k and 2. not masking truncated sequences, one of the components of DAPO ( Yu et al., 2025 ) . Despite initial results suggesting this masking improved the speed of the trainer (by having fewer completions to train on), we ultimately found that variations in batch size caused by some examples masked out to reduce stability. And without training on overlong negative sequences, completion lengths were higher, on average. We therefore found that any efficiency gains in training speed from masking were outweighed by slowdowns from generating longer sequence lengths.
7B Think RL 32B Think RL 7B Instruct RL 7B RL-Zero Dataset size 104,869 104,869 171,950 13,314 Learning rate 1.0 × 10 − 6 1.0\times 10^{-6} 2.0 × 10 − 6 2.0\times 10^{-6} 1.0 × 10 − 6 1.0\times 10^{-6} 1.0 × 10 − 6 1.0\times 10^{-6} Minibatches 1 1 4 1 LR schedule constant constant constant constant Training steps 1,400 750 450 2,000 Max prompt length 2,048 2,048 2,048 2,048 Response length 32,768 32,768 8,192 16,384 Unique prompts per batch 64 128 64 32 Group size 8 8 8 8 TIS cap - 2.0 - 2.0 Sampling temperature 1.0 1.0 1.0 1.0 Clip-lower 0.2 0.2 0.2 0.2 Clip-higher 0.272 0.272 0.272 0.272 Num learner GPUs 16 64 8 8 Num actor GPUs 56 160 56 64 GPUs per actor (TP) 1 8 1 1 Max asynchrony 1 8 8 8
### A.7 Post-Training Additional Data Details
#### A.7.1 Filtering for Dolci Think -SFT
In this section we detail the filtering methods created primarily for training Olmo 3 Think , which was also used for mid-training and Olmo 3 Instruct data. Each phase of filtering would remove 0-1% of data across most available or generated reasoning traces. Some data, such as Nvidia’s Nemotron Post-training datasets ( Nathawani et al., 2025 ) had very few samples removed relative to their peers.
1. Source filtering We perform some filtering to remove non-compliant licenses or data that will not be useful. E.g. for GeneralThoughts traces used in mid-training, we filtered to only commercially friendly licensed prompts. For OpenThoughts2, we removed ShareGPT prompts due to questionable provenance (as done in Tulu 3). For LlamaNemotron Post-Training we filter to only reasoning samples from DeepSeek and Qwen that have not been touched by Llama models.
2. Format filtering We remove truncated answers (i.e. if they have <think> and no </think>) and empty outputs (empty responses). Implementation is available at github.com/allenai/open-instruct//scripts/data/filtering_and_updates/filter_cots.py
3. Domain specific accuracy filtering We check accuracy for many domains, such as precise instruction following, code, or math. Additionally, for chat domains we use included metadata in some datasets such as Wildchat to remove responses or prompts tagged as unsafe. Implementation is available at github.com/allenai/open-instruct/scripts/data/filtering_and_updates/filter_wildchat.py
4. General content filters Here we remove mention of date cutoffs to try and avoid hallucinations of model characteristics and any mention in the user prompt or completion that indicates the date is to or from any model. Maintaining identity of models trained on heavily distilled data takes a meaningful amount of data work and system prompt design. Implementation is available at github.com/allenai/open-instruct/scripts/data/filtering_and_updates/filter_datasets_sequential.sh
5. Repetition filtering Many open-weights reasoning models have tendencies to perform extreme repetitions, even in thinking traces that result in a correct answer. In particular, we find that .1% of responses from QwQ have mass repetition. We filter this roughly by searching for heavily repeated ( 10x+) sentences, paragraphs, or ( 50x+) phrases. Implementation is available at github.com/allenai/open-instruct/scripts/data/filtering_and_updates/filter_ngram_repetitions.py
6. Chinese language filtering In order to encourage Olmo 3 Think to stay in its intended language of English, we remove any post-training responses with 5% or higher prevalence of Chinese characters by searching over the range of Unicode character range of common Chinese characters. Implementation is available at github.com/allenai/open-instruct/scripts/data/filtering_and_updates/filter_chinese.py
#### A.7.2 Tool-use data
##### Additional details about the Science QA dataset
Citation graph-based queries are produced by prompting GPT-5 in a few-shot setup to create query templates, e.g., What are the top-three most cited papers by {AUTHOR} on {TOPIC}? which are subsequently instantiated with real paper entities. Content-based questions are generated by a GPT-5-based agent equipped with the ASC server, which retrieves relevant papers and formulates grounded questions that can be answered using retrieved text. For both types of queries, to obtain corresponding tool-use trajectories we employ a GPT-4.1-mini agent with access to the same ASC server. All tool call outputs are derived from actual environment responses rather than synthetic completions.
##### Additional details about the Web Search QA dataset