2 OLMo 2 Furious — part 7/7

Cleaned-text code points [94125, 106433). Bibliography omitted here; source.md includes it.

• We thank Ben Bogin, Tim Dettmers, Ananya Harsh Jha, Ani Kembhavi, Matt Deitke, Ian Magnusson, Sewon Min, Niklas Muennighoff, Yizhong Wang, Alexander Wettig, and Valentin Hofmann for helpful research discussions and sharing of relevant findings across related projects.

• We thank Taylor Blanton, Byron Bischoff, Yen-Sung Chen, Arnavi Chheda, Jesse Dodge, Karen Farley, Huy Tran, Eric Marsh, Chris Newell, and Aaron Sarnat for building the Ai2 Playground for model demos.

• We thank Yoganand Chandrasekhar, Johann Dahm, Fangzhou Hu, and Caroline Wu for their work on the Ai2 cluster.

• We also thank others at Ai2 for many indirect contributions to the project: Robert Berry, Alex Buraczynski, Jennifer Dumas, Jason Dunkelberger, Rob Evans, David Graham, Regan Huff, Jenna James, Rodney Kinney, Bailey Kuehl, Sophie Lebrecht, Jaron Lochner, Carissa Schoenick, Will Smith, Sruthi Sreeram, Brooke Vlahos, Alice Wang, Caitlin Wittlif, Jiangjiang Yang.

We also appreciate conversations with and feedback from Cody Blakeney, Mansheej Paul, Jonathan Frankle, Armen Aghajanyan, Akshat Shrivastava, Mike Lewis, and John Schulman.

OLMo 2 would not have been possible without the support of many other institutions. In particular, we thank Google for their support in setting up the training environment for OLMo 2 and to Cirrascale for their on-going support of Ai2’s cluster. We also acknowledge the National Artificial Intelligence Research Resource (NAIRR) Pilot and Microsoft Azure for providing inference credits in support of this project.

## Appendix A OLMo 2 Evaluation Framework

We evaluate OLMo 2 using OLMES, a unified, standardized evaluation suite and toolkit 35 35 35 The OLMES (Open Language Model Evaluation System) framework can be found at github.com/allenai/olmes to guide the development and assess performance of language models.

### A.1 Base Model Eval

OLMo base models are evaluated on 11 tasks, consisting of 5 multiple-choice tasks, 2 generative tasks, and 4 additional held-out tasks not utilized during model development. See Table 20 for the list of tasks along with details of the task formulations following the principles of the OLMES standard ( Gu et al., 2024 ) , described further below.

##### Multiple-choice tasks

We use the formulation of the 10 multiple-choice tasks defined in the OLMES evaluation standard ( Gu et al., 2024 ) . OLMES (Open Language Model Evaluation Standard) is a set of principles and associated standard (with a reference implementation in the OLMES system framework) for reproducible LM evaluations that is open, practical, and documented, providing recommendations guided by experiments and results from the literature ( Biderman et al., 2024 ; Gao et al., 2023 ) . For multiple-choice tasks it is designed to support comparisons between smaller base models that require the cloze/completion formulation of multiple-choice questions (score each answer completion separately) against larger models that can utilize the multiple-choice formulation. To make our evaluations reproducible, we follow the OLMES standard in prompt formatting, choice of in-context examples, probability normalization, and all other details. See Table 20 and see Gu et al. (2024) for more details.

##### Generative tasks

Following the principles of OLMES ( Gu et al., 2024 ) , such as prompt formatting and having 5-shot curated in-context examples, we also evaluated on a suite of generative tasks, OLMES-Gen. This suite covers factual knowledge tasks (Natural Questions ( Kwiatkowski et al., 2019 ) and Jeopardy ( MosaicML, 2024 ) ) and tasks testing reading comprehension (SQuAD ( Rajpurkar et al., 2016 ) , DROP ( Dua et al., 2019 ) , and CoQA ( Reddy et al., 2019 ) ). For CoQA, the task comprises presenting a passage followed by a conversation so far, where each turn in the conversation contains a question and an answer. In this case, the previous question and answer pairs serve to guide the model in terms of the output format, and we do not include additional few-shot examples. For all other tasks, we follow OLMES in using 5-shot curated in-context examples. As the list of gold answers for these tasks are often incomplete, we use F1 as the primary metric to give partial credit when models produce answers that partially match. The task details of OLMES-Gen are summarized in Table 20 .

##### Held-out tasks

We also evaluate on a held-out suite of tasks that were not used when making decisions during model development. This suite includes advanced admission and qualification exams (AGIEval English 36 36 36 Specifically these 8 tasks: aqua-rat, logiqa-en, lsat-ar, lsat-lr, lsat-rc, sat-en, sat-math, gaokao-english ( Zhong et al., 2024 ) ), tasks believed to be challenging to LMs (BigBenchHard, BBH; Suzgun et al., 2022 ), math reasoning (GSM8K; Cobbe et al., 2021 ), a more challenging and reasoning-focused extension of MMLU (MMLU Pro; Wang et al., 2024 ), and an unseen factual knowledge task (TriviaQA; Joshi et al., 2017 ). We use existing in-context examples where available - for GSM8K, we use the 8-shot CoT examples from Wei et al. (2023) ; for BBH we use the 3-shot CoT prompts from the original dataset; in evaluating MMLU-Pro, we used 5-shot examples from the original dataset. We use a 1-shot (with passage context, no CoT) prompt for AGIEval English, and a manually curated 5-shot examples from the train set for TriviaQA. Note that for the case of GSM8K, we never evaluated our models on the entire test set during the development stage, instead we use 200 examples to inform choices during development (e.g., choices of annealing mixtures); in Section 4 we refer to this 200-example subset as GSM*.

We make all implementations publicly available at github.com/allenai/olmes .

### A.2 Instruct Model Eval

##### Instruct tasks

We perform instruct model evaluation based on existing practices in current literature using the OLMES benchmark suite ( Gu et al., 2024 ) using the configuration reported in Lambert et al. (2024) .

See Table 21 for a list of instruct tasks along with their configurations. These tasks include chat variations of our held-out tasks (GSM8k; Cobbe et al., 2021 , BBH; Suzgun et al., 2022 ), additional long-tail knowledge (PopQA; Mallen et al., 2022 ), misconception (TruthfulQA; Lin et al., 2021 ) and instruction-following tasks (IFEval; Zhou et al., 2023 , AlpacaEval 2; Dubois et al., 2024 ). For our MMLU instruct evaluation, we use the CoT version from Lambert et al. (2024) using their prompt asking the model to “summarize” its reasoning before answering the question. We evaluate Python code completion (HumanEval; Chen et al., 2021 , HumanEval+; Liu et al., 2023b ) and competition MATH ( Hendrycks et al., 2021b ) with the same setup and answer extraction in OLMES.

## Appendix B OLMo 2 1B

While the goal of this work is to develop development recipes for our target 7B, 13B and 32B sizes, often it is useful to perform experimentation at the 1B model size. We define OLMo 2 1B similar to OLMo 2 7B, but with the following departures: • Layers: 16 instead of 32

• Hidden Size ( d m ​ o ​ d ​ e ​ l d_{model} : 2048 instead of 4096

• Attention Heads (Q/KV): 16/16 (MHA) instead of 32/32 (MHA)

• Batch Size: 512 instead of 1024

• Peak LR: 4.0 ⋅ 10 ​ E − 4 4.0\cdot 10\text{E}{-4} instead of 3.0 ⋅ 10 ​ E − 4 3.0\cdot 10\text{E}{-4}

### B.1 Difficulties with OLMo 2 1B

We developed our OLMo 2 recipe developed using the OLMo 2 1B model (Appendix B ) and have found findings to generalize well to the 7B, 13B and 32B scales, as seen by our competitive results in Table 6 . Yet, we have found scaling the number of training tokens for OLMo 2 1B to be difficult.

##### Training

We pretrain OLMo 2 1B to 4 trillion tokens on OLMo 2 Mix 1124 and perform a single 50B token anneal on Dolmino Mix 1124 . Similar to OLMo 2 7B, we use 2000 steps of warmup, set the schedule to 5 trillion tokens but truncate at the 4 trillion mark. We use a higher peak learning rate of 4.0 ⋅ 10 ​ E − 4 4.0\cdot 10\text{E}{-4} .

##### Base Results

Table 22 presents experimental results on our main base model evaluation suite. We find that while OLMo 2 remains competitive with other similarly-sized models like SmolLM 2, it lags behind the smaller Gemma 2 and Qwen 2.5 base models.

##### Analysis

We postulate that our OLMo 2 1B may struggle with pretraining token efficiency due to model capacity. OLMo 2 is smaller than the smallest variants of other competitive model families like Qwen 2.5 or Gemma 2. We hypothesize that below a certain model size, the optimal pretraining recipe may require the inclusion of task-specific data, such as that seen in supervised fine-tuning (SFT) to achieve non-random performance over more challenging tasks in our evaluation suite. Better performance could also be achieved by distilling from a more powerful model, a strategy used by the smaller Gemma 2 models.

For example, Table 9 shows the benefit of Dolmino Mix 1124 is higher with smaller base models: +37.0% for the 1B model, +18.7% for the 7B model, +15.9% for the 13B model, and +12.3% for the 32B model. These results also show that OLMo 2 1B with only Stage 1 pretraining struggles to break out of random performance for multiple-choice formatted tasks (25% for MMLU and ARC Challenge, 10% for MMLU Pro).

As further evidence of this, Table 23 shows that applying our same OLMo 2-Instruct post-training recipe to OLMo 2 1B results in OLMo 2-Instruct 1B with highly competitive performance to even Qwen 2.5 and even Gemma 3.

## Appendix C Additional Instruct Details

### C.1 Additional Hyperparameters

All of the models used to generate preference data for OLMo 2-Instruct are listed in Table 25 . The prompt sources for the preference datasets are listed in Table 27 – for more information on their contents, refer to Lambert et al. (2024) . The hyperparameters used to train the reward models for RLVR value network initialization are shown in Table 26 .

### C.2 Additional RLVR Learning Curves

The additional 13B RLVR learning curves of can be found at Figure 18 , Figure 19 , and Figure 20 .

### C.3 OLMo 2-Instruct Preview Models

We made an initial release 37 37 37 https://allenai.org/blog/olmo2 prior to this report. However, soon after the release, a tokenizer issue came to our attention: Our base model’s pre-tokenization logic differs from our instruct model’s tokenizer.

Specifically, the OLMo-2 base models utilized the GPT2Tokenizer tokenizer class, with custom pre-tokenization logic (e.g., on splitting or truncating sequences), which is lost during the instruct model’s training. Figure 17 shows the filediff between the base model’s tokenizer.json and the instruct model’s tokenizer.json .

Because of this, we have decided to retrain our OLMo 2-Instruct models to be consistent with our base models and mark the existing post-trained models as preview models.

Nevertheless, the OLMo 2-Instruct Preview learning curves can be found at Figure 21 and Figure 22 .

OLMo-2-1124-13B-RLVR1

OLMo-2-1124-13B-RLVR2

OLMo-2-1124-13B-Instruct

OLMo-2-1124-13B-Instruct-Preview

OLMo-2-1124-7B-Instruct-Preview

## Appendix D Additional Hyperparameters

The models used for the on-policy preference data generation are listed in Table 25 .

OLMo 2 7B Instruct Preview (alternative)

## Appendix E Annealing Data Details

## Instructions for reporting errors

We are continuing to improve HTML versions of papers, and your feedback helps enhance accessibility and mobile support. To report errors in the HTML that will help us improve conversion and rendering, choose any of the methods listed below:

Click the "Report Issue" ( ) button, located in the page header.

Tip: You can select the relevant text first, to include it in your report.

Our team has already identified the following issues . We appreciate your time reviewing and reporting rendering errors we may not have found yet. Your efforts will help us improve the HTML versions for all readers, because disability should not be a barrier to accessing research. Thank you for your continued support in championing open access for all.

Have a free development cycle? Help support accessibility at arXiv! Our collaborators at LaTeXML maintain a list of packages that need conversion , and welcome developer contributions .