s1: Simple test-time scaling — part 2/3

Cleaned-text code points [15899, 31457). Bibliography omitted here; source.md includes it.

We select three representative reasoning benchmarks widely used in the field: AIME24 ( of America, 2024 ) has 30 problems that were used in the 2024 American Invitational Mathematics Examination (AIME) held from January 31 – February 1, 2024. AIME tests mathematical problem-solving with arithmetic, algebra, counting, geometry, number theory, probability, and other secondary school math topics. High-scoring high school students in the test are invited to participate in the United States of America Mathematics Olympiad (USAMO). All AIME answers are integers ranging from 000 000 to 999 999 , inclusive. Some AIME problems rely on figures that we provide to our model using the vector graphics language Asymptote as it cannot take image inputs. MATH500 ( Hendrycks et al., 2021 ) is a benchmark of competition math problems of varying difficulty. We evaluate on the same 500 samples selected by OpenAI in prior work ( Lightman et al., 2023 ) . GPQA Diamond ( Rein et al., 2023 ) consists of 198 PhD-level science questions from Biology, Chemistry and Physics. Experts with PhDs in the corresponding domains only achieved 69.7% on GPQA Diamond ( OpenAI, 2024 ) . When we write “GPQA” in the context of evaluation in this work, we always refer to the Diamond subset. We build on the “lm-evaluation-harness” framework ( Gao et al., 2021 ; Biderman et al., 2024 ) . Unless otherwise specified, we evaluate with a temperature of 0 (greedy) and measure accuracy (equivalent to pass@1).

#### Other models

We benchmark s1-32B against: OpenAI o1 series ( OpenAI, 2024 ) , closed-source models that popularized test-time scaling; DeepSeek r1 series ( DeepSeek-AI et al., 2025 ) , open-weight reasoning models with up to o1-level performance; Qwen’s QwQ-32B-preview ( Team, 2024 ) , a 32B open-weight reasoning model without disclosed methodology; Sky-T1-32B-Preview ( Team, 2025 ) and Bespoke-32B ( Labs, 2025 ) , open models with open reasoning data distilled from QwQ-32B-preview and r1; Google Gemini 2.0 Flash Thinking Experimental ( Google, 2024 ) , the API that we distill from. As it has no official evaluation scores, we use the Gemini API to benchmark it ourselves. However, the ‘‘recitation error’’ of the Gemini API makes evaluation challenging. 2 2 2 https://github.com/google/generative-ai-docs/issues/257 We circumvent this, by manually inserting all 30 AIME24 questions in its web interface where the error does not appear. However, we leave out MATH500 (500 questions) and GPQA Diamond (198 questions), thus they are N.A. in Table 1 . Our model, s1-32B , is fully open including weights, reasoning data, and code.

### 4.2 Performance

#### Test-time scaling

Figure 1 shows the performance of s1-32B with budget forcing scales with more test-time compute. In Figure 4 (left), we expand the plot from Figure 1 (middle) showing that while we can improve AIME24 performance using our budget forcing technique ( §3 ) and more test-time compute it does eventually flatten out at six times. Suppressing the end-of-thinking token delimiter too often can lead the model into repetitive loops instead of continued reasoning. In Figure 4 (right), we show that after training Qwen2.5-32B-Instruct on our 1,000 samples to produce s1-32B and equipping it with the simple budget forcing technique, it operates in a different scaling paradigm. Scaling test-time compute on the base model via majority voting cannot catch up with the performance of s1-32B which validates our intuition from §3 that sequential scaling is more effective than parallel. We provide example generations of s1-32B in Figure 5 .

#### Sample-efficiency

In Figure 2 (right) and Table 1 we compare s1-32B with other models. We find that s1-32B is the most sample-efficient open data reasoning model. It performs significantly better than our base model (Qwen2.5-32B-Instruct) despite just training it on an additional 1,000 samples. The concurrently released r1-32B shows stronger performance than s1-32B while also only using SFT ( DeepSeek-AI et al., 2025 ) . However, it is trained on 800 × \times more reasoning samples. It is an open question whether one can achieve their performance with just 1,000 samples. Finally, our model nearly matches Gemini 2.0 Thinking on AIME24. As the data for s1-32B is distilled from Gemini 2.0, this shows our distillation procedure was likely effective.

## 5 Ablations

### 5.1 Data Quantity, Diversity, and Difficulty

In §2 we outlined our three guiding principles in curating s1K : Quality, Difficulty, and Diversity. Here we test the importance of combining them and the overall efficacy of our selection. Only Quality (1K-random) : After obtaining our high-quality reasoning chains from Gemini, we select 1,000 samples at random; not relying on our difficulty and diversity filtering at all. Table 2 shows this approach performs much worse than s1K across all benchmarks. Only Diversity (1K-diverse) : For this dataset, we sample uniformly across domains to maximize diversity disregarding any notion of difficulty. This approach also leads to poor performance similar to 1K-random. Only Difficulty (1K-longest) : Here we rely on one of our difficulty indicators introduced in §2 by selecting the 1,000 samples with the longest reasoning traces. This approach significantly boosts GPQA performance but overall still falls short of using s1K . Maximize Quantity : Finally, we compare with just training on all of our 59K samples, a superset of all the 1K-sample versions. This leads to a strong model but uses much more resources. To finetune on 59K samples, we use 394 H100 GPU hours while s1-32B only required 7 H100 GPU hours. Moreover, relying only on s1K is extremely competitive as shown in §2 . Overall, combining all three criteria – Quality , Difficulty , Diversity – via our methodology in §2 is key for sample-efficient reasoning training.

### 5.2 Test-time scaling methods

#### Budget forcing

In Table 3 we compare the test-time scaling methods we have introduced in §3 . Overall, we find that budget forcing provides perfect control, good scaling, and leads to our best AIME24 score. Thus, this is the method we use for s1-32B in Figure 1 and in §4 . In Table 4 , we compare different strings for extrapolating performance. We find that “Wait” generally gives the best performance.

Class-conditional control We provide benchmark scores for this method in §E.1 and summarize three findings here: (1) Token-conditional control fails without budget forcing, as our model cannot reliably count tokens - even when trained to do so. (2) Under step-conditional control, the model generates a similar total number of tokens when given different step targets, as the model goes from few steps with many tokens per step, to many steps with few tokens in each step. Thus, the model learns to hack its way around the compute constraint making the controllability of this method mediocre. (3) Class-conditional control can work - telling a model to simply think longer can increase its test-time compute and performance, which leads good scaling in Table 3 .

#### Rejection sampling

Surprisingly, we find that simply sampling until the generation fits a specific length leads to an inverse scaling trend as depicted in Figure 6 . In §E.2 we inspect a question, which was answered correctly by the model when rejection sampling for ≤ 4000 \leq 4000 , but not for the ≤ 8000 \leq 8000 token setting. In the ≤ 4000 \leq 4000 setting the model directly jumps to the correct approach, while for the ≤ 8000 \leq 8000 setting it backtracks a lot. We hypothesize that there is a correlation such that shorter generations tend to be the ones where the model was on the right track from the start, whereas longer ones tend to be ones where the model made mistakes and thus backtracks or questions itself. This leads to longer samples often being wrong when rejection sampling and thus the inverse scaling trend.

## 6 Discussion and related work

### 6.1 Sample-efficient reasoning

#### Models

There are a number of concurrent efforts to build models that replicate the performance of o1 ( OpenAI, 2024 ) . For example, DeepSeek-r1 and k1.5 ( DeepSeek-AI et al., 2025 ; Team et al., 2025 ) are built with reinforcement learning methods, while others rely on SFT using tens of thousands of distilled examples ( Team, 2025 ; Xu et al., 2025 ; Labs, 2025 ) . We show that SFT on only 1,000 examples suffices to build a competitive reasoning model matching o1-preview and produces a model that lies on the pareto frontier ( Figure 2 ). Further, we introduce budget forcing which combined with our reasoning model leads to the first reproduction of OpenAI’s test-time scaling curves ( OpenAI, 2024 ) . Why does supervised finetuning on just 1,000 samples lead to such performance gains? We hypothesize that the model is already exposed to large amounts of reasoning data during pretraining which spans trillions of tokens. Thus, the ability to perform reasoning is already present in our model. Our sample-efficient finetuning stage just activates it and we scale it further at test time with budget forcing. This is similar to the ”Superficial Alignment Hypothesis” presented in LIMA ( Zhou et al., 2023 ) , where the authors find that 1,000 examples can be sufficient to align a model to adhere to user preferences.

#### Benchmarks and methods

To evaluate and push the limits of these models, increasingly challenging benchmarks have been introduced, such as Olympiad-level science competitions ( He et al., 2024 ; Jain et al., 2024 ; Zhong et al., 2023 ) and others ( Srivastava et al., 2023 ; Glazer et al., 2024 ; Su et al., 2024 ; Kim et al., 2024 ; Phan et al., 2025 ) . To enhance models’ performance on reasoning-related tasks, researchers have pursued several strategies: Prior works have explored continuing training language models on specialized corpora related to mathematics and science ( Azerbayev et al., 2023 ; Yang et al., 2024 ) , sometimes even synthetically generated data ( Yu et al., 2024 ) . Others have developed training methodologies specifically aimed at reasoning performance ( Zelikman et al., 2022 ; Zelikman et al., 2024 ; Luo et al., 2025 ; Yuan et al., 2025 ; Wu et al., 2024a ) . Another significant line of work focuses on prompting-based methods to elicit and improve reasoning abilities, including methods like Chain-of-Thought prompting ( Wei et al., 2023 ; Yao et al., 2023a ; Yao et al., 2023b ; Bi et al., 2023 ; Fu et al., 2023 ; Zhang et al., 2024b ; Xiang et al., 2025 ; Hu et al., 2024 ; Diao et al., 2024 ) . These combined efforts aim to advance the reasoning ability of language models, enabling them to handle more complex and abstract tasks effectively.

### 6.2 Test-time scaling

#### Methods

As we introduce in §3 , we differentiate two methods to scale test-time compute: parallel and sequential . The former relies on multiple solution attempts generated in parallel and selecting the best outcome via specific criteria. These criteria include choosing the most frequent response for majority voting or the best response based on an external reward for Best-of-N ( Brown et al., 2024 ; Irvine et al., 2023 ; Levi, 2024 ) . Unlike repeated sampling, previous sequential scaling methods let the model generate solution attempts sequentially based on previous attempts, allowing it to refine each attempt based on previous outcomes ( Snell et al., 2024 ; Hou et al., 2025 ; Lee et al., 2025 ) . Tree-based search methods ( Gandhi et al., 2024 ; Wu et al., 2024b ) offer a hybrid approach between sequential and parallel scaling, such as Monte-Carlo Tree Search (MCTS) ( Liu et al., 2024 ; Zhang et al., 2023 ; Zhou et al., 2024 ; Choi et al., 2023 ) and guided beam search ( Xie et al., 2023 ) . REBASE ( Wu et al., 2024b ) employs a process reward model to balance exploitation and pruning during tree search. Empirically, REBASE has been shown to outperform sampling-based methods and MCTS ( Wu et al., 2024b ) . Reward models ( Lightman et al., 2023 ; Wang et al., 2024b ; Wang et al., 2024c ) play a key role in these methods. They come in two variants: outcome reward models and process reward models. Outcome reward models ( Xin et al., 2024 ; Ankner et al., 2024 ) assign a score to complete solutions and are particularly useful in Best-of-N selection, while process reward models ( Lightman et al., 2023 ; Wang et al., 2024b ; Wu et al., 2024b ) assess individual reasoning steps and are effective in guiding tree-based search methods.

#### Limits to further test-time scaling

We have shown that budget forcing allows extrapolating test-time compute in §4 , e.g., improving AIME24 performance from 50% to 57%. However, it has two key limitations when scaling further: it eventually flattens out ( Figure 4 ), and the context window of the underlying language model constrains it. Despite these constraints, our work shows test-time scaling across a wide range of accuracies ( Figure 1 ), partly because scaling down test-time compute behaves predictably and does not suffer from these constraints.

Continuing test-time scaling will require approaches that can further extrapolate test-time compute. How can we get such extrapolation? There may be improvements to budget forcing such as rotating through different strings, not only “Wait”, or combining it with frequency penalties or higher temperature to avoid repetitive loops. An exciting direction for future work is also researching whether applying budget forcing to a reasoning model trained with reinforcement learning yields better extrapolation; or if RL allows for new ways of test-time scaling beyond budget forcing. Our work defines the right metrics ( §3.2 ) – Control, Scaling, and Performance – to enable future research and progress on extrapolating test-time compute.

#### Parallel scaling as a solution

Parallel scaling offers one solution to the limits of sequential scaling, thus we augment our sequentially scaled model with two methods: (I) Majority voting: After generating k k solutions, the final solution is the most frequent one across generations; (II) Tree search via REBASE : We use the REBASE process reward model, which is initialized from LLaMA-34B and further finetuned on a synthetic process reward modeling dataset ( Wu et al., 2024b ) . We then aggregate the solutions generated by REBASE via majority voting. As shown in Figure 7 , augmenting our model with REBASE scales better than majority voting, and even sequential scaling in this scenario. However, REBASE requires an additional forward pass at each step for the reward model adding some computation overhead. For sequential scaling, when prompted to use up to 512 steps, for 12 out of the 30 evaluation questions the model generates a response that exceeds the context window leading to a large performance drop. Overall, we find that these parallel scaling methods complement sequential scaling thus they offer an avenue for scaling test-time compute even further; beyond fixed context windows.

## Impact Statement

Language models with strong reasoning capabilities have the potential to greatly enhance human productivity, from assisting in complex decision-making to driving scientific breakthroughs. However, recent advances in reasoning, such as OpenAI’s o1 and DeepSeek’s r1, lack transparency, limiting broader research progress. Our work aims to push the frontier of reasoning in a fully open manner, fostering innovation and collaboration to accelerate advancements that ultimately benefit society.

## Acknowledgements