Olmo 3 — part 1/19
Cleaned-text code points [0, 15710). Bibliography omitted here; source.md includes it.
\authorOne [ ]Olmo Team \authorTwo [ 1 ]Allyson Ettinger \authorTwo [ 1 , 3 ]Amanda Bertsch \authorTwo [ 1 ]Bailey Kuehl \authorTwo [ 1 ]David Graham \authorTwo [ 1 ]David Heineman \authorTwo [ 1 ]Dirk Groeneveld \authorTwo [ 1 ]Faeze Brahman \authorTwo [ 1 ]Finbarr Timbers \authorTwo [ 1 , 2 ]Hamish Ivison \authorTwo [ 1 , 2 ]Jacob Morrison \authorTwo [ 1 ]Jake Poznanski \authorTwo [ 1 , 2 ]Kyle Lo \authorTwo [ 1 ]Luca Soldaini \authorTwo [ 1 ]Matt Jordan \authorTwo [ 1 , 4 ]Mayee Chen \authorTwo [ 1 , 5 , 6 ]Michael Noukhovitch \authorTwo [ 1 ]Nathan Lambert \authorTwo [ 1 ]Pete Walsh \authorTwo [ 1 ]Pradeep Dasigi \authorTwo [ 1 ]Robert Berry \authorTwo [ 1 ]Saumya Malik \authorTwo [ 1 ]Saurabh Shah \authorTwo [ 1 , 2 ]Scott Geng \authorTwo [ 1 ]Shane Arora \authorTwo [ 1 ]Shashank Gupta \authorTwo [ 1 ]Taira Anderson \authorTwo [ 1 ]Teng Xiao \authorTwo [ 1 ]Tyler Murray \authorTwo [ 1 ]Tyler Romero \authorTwo [ 1 , 2 ]Victoria Graf \authorThree [ 1 , 3 ]Akari Asai \authorThree [ 1 ]Akshita Bhagia \authorThree [ 7 ]Alexander Wettig \authorThree [ 2 ]Alisa Liu \authorThree [ 1 ]Aman Rangapur \authorThree [ 1 ]Chloe Anastasiades \authorThree [ 1 ]Costa Huang \authorThree [ 1 ]Dustin Schwenk \authorThree [ 1 ]Harsh Trivedi \authorThree [ 1 , 2 ]Ian Magnusson \authorThree [ 1 ]Jaron Lochner \authorThree [ 1 ]Jiacheng Liu \authorThree [ 1 ]Lester James V. Miranda \authorThree [ 1 , 3 ]Maarten Sap \authorThree [ 1 ]Malia Morgan \authorThree [ 1 ]Michael Schmitz \authorThree [ 1 ]Michal Guerquin \authorThree [ 1 ]Michael Wilson \authorThree [ 1 ]Regan Huff \authorThree [ 1 ]Ronan Le Bras \authorThree [ 2 ]Rui Xin \authorThree [ 2 ]Rulin Shao \authorThree [ 1 ]Sam Skjonsberg \authorThree [ 8 ]Shannon Zejiang Shen \authorThree [ 2 ]Shuyue Stella Li \authorThree [ 1 ]Tucker Wilde \authorThree [ 1 ]Valentina Pyatkin \authorThree [ 1 ]Will Merrill \authorThree [ 9 ]Yapei Chang \authorThree [ 1 ]Yuling Gu \authorThree [ 1 , 2 ]Zhiyuan Zeng \authorFour [ 1 ]Ashish Sabharwal \authorFour [ 2 ]Luke Zettlemoyer \authorFour [ 1 , 2 ]Pang Wei Koh \authorFive [ 1 , 2 ]Ali Farhadi \authorFive [ 1 , 2 ]Noah A. Smith \authorFive [ 1 , 2 ]Hannaneh Hajishirzi 1 ]Allen Institute for AI 2 ]University of Washington 3 ]Carnegie Mellon University 4 ]Stanford University 5 ]Mila 6 ]Université de Montréal 7 ]Princeton University 8 ]Massachusetts Institute of Technology 9 ]University of Maryland \contribution [] Olmo 3 was a team effort; authors sorted alphabetically. marks core contributors. See author contributions here . \metadata [ Olmo 3 Base:] Olmo-3-1025-7B Olmo-3-1125-32B \metadata [ Olmo 3 Think:] Olmo-3-7B-Think Olmo-{ 3 | 3.1 }-32B-Think \metadata [ Olmo 3 Instruct:] Olmo-3-7B-Instruct Olmo-3.1-32B-Instruct \metadata [ Olmo 3 RL Zero:] Olmo-3-7B-RL-Zero-{ Math | Code | IF | General | Mix } Olmo-3.1-7B-RL-Zero-{ Math | Code } \metadata [
Base Data:] Pretrain: Dolma 3 Mix Midtrain: Dolma 3 Dolmino Mix Long-ctx: Dolma 3 Longmino Mix \metadata [ Think Data:] Dolci-Think-{ SFT | DPO | RL }-7B Dolci-Think-{ SFT | DPO | RL }-32B \metadata [ Instruct Data:] Dolci-Instruct-{ SFT | DPO | RL } \metadata [ RL-Zero Data:] Dolci-RL-Zero-{ Math | Code | IF | General }-7B Dolci-RL-Zero-Mix-7B \metadata [
Training Code:] OLMo-core \sans (pretrain) Open Instruct \sans (posttrain) \metadata [ Data Code:] datamap-rs \sans (data processing) duplodocus \sans (deduplication) dolma3 \sans (data recipes) \metadata [ Eval Code:] OLMES \sans (eval suite) decon \sans (eval decontamination) \metadata [
Training Logs:] Olmo-3-7B-{ Base | Think | Instruct | RL-Zero } Olmo-3-32B-{ Base | Think | Instruct } \metadata [ Demo:] \sans 32B Think \sans 32B Instruct \sans 7B Think \sans 7B Instruct \metadata [ Contact:] olmo@allenai.org
# Olmo 3
###### Abstract
We introduce Olmo 3 , a family of state-of-the-art, fully-open language models at the 7B and 32B parameter scales. Olmo 3 model construction targets long-context reasoning, function calling, coding, instruction following, general chat, and knowledge recall. This release includes the entire model flow , i.e., the full lifecycle of the family of models, including every stage, checkpoint, data point, and dependency used to build it. Our flagship model, Olmo 3.1 Think 32B , is the strongest fully-open thinking model released to-date.
## 1 Introduction
We introduce Olmo 3 , a family of state-of-the-art, fully-open language and thinking models at the 7B and 32B parameter scales with a diverse set of capabilities, including long-context reasoning, function calling, coding, instruction following, general chat, and knowledge recall. The Olmo 3 release provides complete access to its entire model flow —the full lifecycle of a language model, including every stage, checkpoint, datapoint, and dependency required to create it. This enables infinite customization through intervention at any stage of the model development process—not just the final weights.
To truly advance open-source AI research and development, we argue that releasing a state-of-the-art language model should make its entire model flow—not just its endpoint—transparent and accessible. With the Olmo 3 release, we provide complete access to the pathways we charted throughout the model flow, from initial conception to the creation of state-of-the-art, fully-open language models.
Specifically, we train Olmo 3 Base as a foundation on which to build models with thinking and tool-use capabilities. From Olmo 3 Base we develop our flagship model, Olmo 3 Think , trained to perform step-by-step reasoning by generating intermediate thinking traces before producing a final answer. Olmo 3 Think 32B is the strongest fully-open thinking model, narrowing the gap to the best open-weight models of similar scale, such as the Qwen 3 32B thinking ( Yang et al., 2025a ) on our suite of reasoning benchmarks, while being trained on six times fewer tokens. Because of our fully-open approach, the Olmo 3 release also enables reasoning chains to be traced back to their original training data, unlocking research opportunities not possible with any other thinking model.
In addition, we train Olmo 3 Instruct 7B and 32B models tuned to produce shorter, more direct responses. By avoiding intermediate “thinking” outputs, Olmo 3 Instruct effectively reduces response latency and is optimized for general chat and function calling. Olmo 3 Instruct 7B and 32B surpass other notable open-weight models of comparable size—Qwen 2.5 ( Qwen et al., 2024 ) , Gemma 3 ( Gemma 3 Team, 2025 ) , IBM Granite 3.3 ( Soule and Bergmann, 2025 ) , and Llama 3 ( Grattafiori et al., 2024 ) —and additionally reduces the remaining performance gap to Qwen 3 ( Yang et al., 2025a ) . Finally, we introduce Olmo 3 RL-Zero 7B , a variant of Olmo 3 trained using RL directly from Olmo 3 Base . Olmo 3 RL-Zero enables researchers to study how base model data affects RL performance.
The Olmo 3 family is the strongest collection of fully-open base models, outperforming Stanford Marin ( Hall et al., 2025 ) , Apertus ( Apertus Team, 2025 ) , and LLM360 K2-V2 ( Team et al., 2025 ) . To achieve these results, we construct new datasets for every stage of the model flow. This includes Dolma 3 , our pretraining data mix encompassing carefully-sampled natural data from crawled sources, our midtraining mix of high-quality data designed to jump-start reasoning, and a large collection of science-focused PDF documents that unlock long-context support in Olmo 3 . We also introduce Dolci , a post-training data suite that advances step-by-step reasoning during supervised finetuning, provides high-quality contrastive data for preference tuning, and offers challenging general and reasoning prompts for reinforcement learning.
Finally, we develop a set of new algorithmic and infrastructural advances across data processing, evaluation, pretraining, and reinforcement learning. This includes OlmoBaseEval , a benchmark suite tailored to compute-efficient base-model development, and OlmoRL , a reinforcement-learning framework incorporating efficiency optimizations tailored to our thinking models. Taken together, these training recipes are shaped by a development framework that blends distributed experimentation with centralized evaluation, enabling coordinated, capability-driven improvements throughout the model pipeline.
## 2 Model Flow for Olmo 3
In this section, we provide a brief overview of all components of the model flow for Olmo 3 , highlighting our methodology for targeting reasoning and tool-use capabilities in ways that advance beyond OLMo 2 ( OLMo et al., 2024 ) and other open-weight models. Subsequent sections will then provide deep dives into each of the model flow components. Olmo 3 training is divided into major stages of base model training and post-training, each further divided into sub-stages as outlined in Figure 2 .
### 2.1 Base Model Training
We develop Olmo 3 Base in three stages of pretraining for up to 5.9T tokens (Section § 3.4 ), midtraining for 100 billion tokens (Section § 3.5 ), and the newly added long-context extension for 50B ( Olmo 3 Base 7B) or 100B ( Olmo 3 Base 32B) tokens (Section § 3.6 ).
##### Evaluation
We develop OlmoBaseEval , a collection of benchmarking suites to support decision-making during base model development (pretraining and midtraining). Our goal is to be compute-efficient by making development decisions based on models trained at a small scale. The challenge is that such models can exhibit random-chance performance on certain tasks, and have small differences in scores that are hard to distinguish from benchmark noise. To address this, we (1) aggregate scores over clusters of tasks that assess similar capabilities (Section § 3.3.1 ); (2) develop proxy metrics for evaluating small-scale models (Section § 3.3.2 ); and (3) improve overall signal-to-noise ratio by evaluating on more examples from noisy tasks or even removing them entirely (Section § 3.3.3 ).
##### Data curriculum
We curate specialized datasets for each training stage, with latter stages focused on strengthening capabilities crucial in post-training stages, such as math, code, reasoning, instruction following, and long-context understanding: • Pretraining We first train Olmo 3 Base on Dolma 3 Mix (Section § 3.4 ), our 6T-token pretraining data mix. While Dolma 3 Mix is largely comprised of the same types of data sources used in other open pretraining recipes ( Soldaini et al., 2024 ; Bakouch et al., 2025 ; OLMo et al., 2024 ) , we demonstrate three key novelties: ∘ \circ New tooling for fast and scalable global deduplication at the trillion-token scale;
∘ \circ A novel source of academic PDFs— olmOCR science PDFs—converted to linearized plain text using olmOCR ( Poznanski et al., 2025a ; Poznanski et al., 2025b ) ;
∘ \circ Two new methods for optimizing selection of training tokens: token-constrained mixing and quality-aware upsampling.
• Midtraining We continue training on Dolma 3 Dolmino Mix (Section § 3.5 ), our 100B-token data curated to boost target capabilities across code, math, and general knowledge QA domains through the introduction of: ∘ \circ A new two-part methodological framework combining 1) lightweight, distributed feedback loops on individual data sources, with 2) centralized integration tests to assess candidate mixes on base model quality and post-trainability.
∘ \circ Intentional inclusion instruction data and thinking traces to lay groundwork for post-training.
• Long-context extension Through Dolma 3 Longmino Mix (Section § 3.6 ), Olmo 3 supports long-context input and output, a crucial feature to unlock reasoning and tool-use capabilities. ∘ \circ Documents in olmOCR science PDFs enable of our long-context approach; with over 22.3M documents of length above 8K tokens (640B tokens total), and 4.5M documents over 32K tokens (380B tokens total), this collection is the largest openly available for long-context research.
∘ \circ As result, Olmo 3 is our first model with long-context capabilities, supporting up to 65K context after extension. Olmo 3 Base 32B rivals performance of Qwen 2.5 32B, Mistral Small 3.1 24B, and Gemma 3 27B on long-context benchmarks, despite a short extension stage (50B for 7B, 100B for 32B).
##### Open artifacts
We release all of our intermediate checkpoints as well as the final models at the end of each stage of training. For data, we release both our data mixes , which are the actual tokens used for base model training, 1 1 1 A data mix may involve upsampling or repeating data from a data pool. as well as our full source data pools for each stage—9T tokens of cleaned source tokens for pretraining, and 2T and 640B tokens of specialized data for midtraining and long-context extension respectively. For pretraining, in addition to our actual training mix for Olmo 3 Base , we also release smaller sample mixes for accessible experimentation with less compute (150B for pretraining and 10B for midtraining).
### 2.2 Post-training
We post-train Olmo 3 Base into three model variants:
• Olmo 3 Think (Section § 4 ) is trained to perform extended reasoning by generating a structured thinking trace before a final answer. We train it via SFT, DPO, and RLVR, observing gains at each stage.
∘ \circ We introduce Dolci Think SFT (Section § 4.2 ), Dolci Think DPO (Section § 4.3 ), and Dolci Think RL (Section § 4.4 ), new post-training datasets designed to target a broad range of key capabilities such as math, coding, instruction following, and general conversation. The dataset includes synthetic examples with long thinking traces for supervised finetuning, high-quality contrastive data following the insights from Delta Learning ( Geng et al., 2025 ) , and challenging prompts for reinforcement learning across both verifiable and non-verifiable domains. In particular, our new approach to curating contrastive instances for preference tuning expands the reasoning frontier of the model beyond what SFT alone can provide and primes the model for effective reinforcement learning.
∘ \circ We introduce algorithmic and infrastructural advances in reinforcement learning with verifiable rewards (Section § 4.4 ). This approach generalizes verifiable reasoning to multiple domains, expanding beyond the settings explored in OLMo 2 to include code and general chat. Our improvements enable longer and more stable RL runs across diverse domains and increase the overall efficiency of training cycles, leading to a 4x speedup in RL training. Notably, we introduce Olmo 3.1 Think 32B to illustrate that extended OlmoRL training leads to improved performance.
• Olmo 3 Instruct (Section § 5 ) is trained to produce efficient and helpful responses to user queries without generating internal thinking traces. This model prioritizes typical user needs, such as avoiding excessive verbosity for easy user understanding and function-calling for user information seeking. In such settings, thinking traces are unnecessary, and inference-time efficiency matters more than inference-time scaling. ∘ \circ We introduce Dolci Instruct SFT , our new dataset enriched with data specifically created for function calling (Section § 5.2.1 ). To directly optimize model interactivity on top of capabilities, we extend our Delta Learning preference pipeline in Dolci Instruct DPO , incorporating multi-turn preference data and targeted data length interventions that encourage concise responses (Section § 5.3.1 ). Finally, we use reinforcement learning with verifiable rewards (Section § 5.4 ) to further refine core capabilities, where preference tuning synergizes with RL to improve model performance while maintaining learned brevity.