s1: Simple test-time scaling — part 1/3

Cleaned-text code points [0, 15899). Bibliography omitted here; source.md includes it.

# s1 : Simple test-time scaling

###### Abstract

Test-time scaling is a promising new approach to language modeling that uses extra test-time compute to improve performance. Recently, OpenAI’s o1 model showed this capability but did not publicly share its methodology, leading to many replication efforts. We seek the simplest approach to achieve test-time scaling and strong reasoning performance. First, we curate a small dataset s1K of 1,000 questions paired with reasoning traces relying on three criteria we validate through ablations: difficulty, diversity, and quality. Second, we develop budget forcing to control test-time compute by forcefully terminating the model’s thinking process or lengthening it by appending “Wait” multiple times to the model’s generation when it tries to end. This can lead the model to double-check its answer, often fixing incorrect reasoning steps. After supervised finetuning the Qwen2.5-32B-Instruct language model on s1K and equipping it with budget forcing, our model s1-32B exceeds o1-preview on competition math questions by up to 27% (MATH and AIME24). Further, scaling s1-32B with budget forcing allows extrapolating beyond its performance without test-time intervention: from 50% to 57% on AIME24. Our model, data, and code are open-source at https://github.com/simplescaling/s1 .

###### Keywords:

## 1 Introduction

Performance improvements of language models (LMs) over the past years have largely relied on scaling up train-time compute using large-scale self-supervised pretraining ( Kaplan et al., 2020 ; Hoffmann et al., 2022 ) . The creation of these powerful models has set the stage for a new scaling paradigm built on top of them: test-time scaling . The aim of this approach is to increase the compute at test time to get better results. There has been much work exploring this idea ( Snell et al., 2024 ; Welleck et al., 2024 ) , and the viability of this paradigm was recently validated by OpenAI o1 ( OpenAI, 2024 ) . o1 has demonstrated strong reasoning performance with consistent gains from scaling test-time compute. OpenAI describes their approach as using large-scale reinforcement learning (RL) implying the use of sizable amounts of data ( OpenAI, 2024 ) . This has led to various attempts to replicate their models relying on techniques like Monte Carlo Tree Search ( Gao et al., 2024b ; Zhang et al., 2024a ) , multi-agent approaches ( Qin et al., 2024 ) , and others ( Wang et al., 2024a ; Huang et al., 2024b ; Huang et al., 2025 ) . Among these approaches, DeepSeek R1 ( DeepSeek-AI et al., 2025 ) has successfully replicated o1-level performance, also employing reinforcement learning via millions of samples and multiple training stages. However, despite the large number of o1 replication attempts, none have openly replicated a clear test-time scaling behavior. Thus, we ask: what is the simplest approach to achieve both test-time scaling and strong reasoning performance?

We show that training on only 1,000 samples with next-token prediction and controlling thinking duration via a simple test-time technique we refer to as budget forcing leads to a strong reasoning model that scales in performance with more test-time compute. Specifically, we construct s1K , which consists of 1,000 carefully curated questions paired with reasoning traces and answers distilled from Gemini Thinking Experimental ( Google, 2024 ) . We perform supervised fine-tuning (SFT) of an off-the-shelf pretrained model on our small dataset requiring just 26 minutes of training on 16 H100 GPUs. After training, we control the amount of test-time compute our model spends using budget forcing : (I) If the model generates more thinking tokens than a desired limit, we forcefully end the thinking process by appending an end-of-thinking token delimiter. Ending the thinking this way makes the model transition to generating its answer. (II) If we want the model to spend more test-time compute on a problem, we suppress the generation of the end-of-thinking token delimiter and instead append “Wait” to the model’s current reasoning trace to encourage more exploration. Equipped with this simple recipe – SFT on 1,000 samples and test-time budget forcing – our model s1-32B exhibits test-time scaling ( Figure 1 ). Further, s1-32B is the most sample-efficient reasoning model and outperforms closed-source models like OpenAI’s o1-preview ( Figure 2 ).

We conduct extensive ablation experiments targeting (a) our selection of 1,000 (1K) reasoning samples and (b) our test-time scaling. For (a) , we find that jointly incorporating difficulty, diversity, and quality measures into our selection algorithm is important. Random selection, selecting samples with the longest reasoning traces, or only selecting maximally diverse samples all lead to significantly worse performance (around − - 30% on AIME24 on average). Training on our full data pool of 59K examples, a superset of s1K , does not offer substantial gains over our 1K selection. This highlights the importance of careful data selection and echoes prior findings for instruction tuning ( Zhou et al., 2023 ) . For (b) , we define desiderata for test-time scaling methods to compare different approaches. Budget forcing leads to the best scaling as it has perfect controllability with a clear positive slope leading to strong performance.

In summary, our contributions are: We develop simple methods for creating a sample-efficient reasoning dataset ( §2 ) and test-time scaling ( §3 ); Based on these we build s1-32B which is competitive with o1-preview ( §4 ); We ablate subtleties of data ( §5.1 ) and test-time scaling ( §5.2 ). We end with a discussion to motivate future work on simple reasoning ( §6 ). Our code, model, and data are open-source at https://github.com/simplescaling/s1 .

## 2 Reasoning data curation to create s1K

In this section, we describe our process for creating a large dataset first in §2.1 and then filtering it down to s1K in §2.2 .

### 2.1 Initial collection of 59K samples

We collect an initial 59,029 questions from 16 sources following three guiding principles. Quality : Datasets should be high-quality; we always inspect samples and ignore datasets with, e.g., poor formatting; Difficulty : Datasets should be challenging and require significant reasoning effort; Diversity : Datasets should stem from various fields to cover different reasoning tasks. We collect datasets of two categories:

#### Curation of existing datasets

Our largest source is NuminaMATH ( LI et al., 2024 ) with 30,660 mathematical problems from online websites. We also include historical AIME problems (1983-2021). To enhance diversity, we add OlympicArena ( Huang et al., 2024a ) with 4,250 questions spanning Astronomy, Biology, Chemistry, Computer Science, Geography, Mathematics, and Physics from various Olympiads. OmniMath ( Gao et al., 2024a ) adds 4,238 competition-level mathematics problems. We also include 2,385 problems from AGIEval ( Zhong et al., 2023 ) , which features questions from standardized tests like SAT and LSAT, covering English, Law, and Logic. We refer to Table 7 in §C for our other sources.

#### New datasets in quantitative reasoning

To complement these existing datasets, we create two original datasets. s1-prob consists of 182 questions from the probability section of Stanford University’s Statistics Department’s PhD Qualifying Exams ( https://statistics.stanford.edu ), accompanied by handwritten solutions that cover difficult proofs. The probability qualifying exam is held yearly and requires professional-level mathematical problem-solving. s1-teasers comprises 23 challenging brain-teasers commonly used in interview questions for quantitative trading positions. Each sample consists of a problem and solution taken from PuzzledQuant ( https://www.puzzledquant.com/ ). We only take examples with the highest difficulty level (”Hard”).

For each question, we generate a reasoning trace and solution using the Google Gemini Flash Thinking API ( Google, 2024 ) extracting its reasoning trace and response. This yields 59K triplets of a question, generated reasoning trace, and generated solution. Examples from our dataset are in §D.2 . We decontaminate all samples against our evaluation questions (MATH500, GPQA Diamond, AIME24; §C.5 ) using 8-grams and deduplicate the data.

### 2.2 Final selection of 1K samples

We could directly train on our pool of 59K questions, however, our goal is to find the simplest approach with minimal resources. Thus, we go through three stages of filtering to arrive at a minimal set of 1,000 samples relying on our three guiding data principles: Quality, Difficulty, and Diversity.

#### Quality

We first remove any questions where we ran into any API errors reducing our dataset to 54,116 samples. Next, we filter out low-quality examples by checking if they contain any string patterns with formatting issues, such as ASCII art diagrams, non-existent image references, or inconsistent question numbering reducing our dataset to 51,581 examples. From this pool, we identify 384 samples for our final 1,000 samples from datasets that we perceive as high-quality and not in need of further filtering (see §C.4 for details).

#### Difficulty

For difficulty, we use two indicators: model performance and reasoning trace length. We evaluate two models on each question: Qwen2.5-7B-Instruct and Qwen2.5-32B-Instruct ( Qwen et al., 2024 ) , with correctness assessed by Claude 3.5 Sonnet comparing each attempt against the reference solution (see §C.3 for the grading protocol). We measure the token length of each reasoning trace to indicate problem difficulty using the Qwen2.5 tokenizer. This relies on the assumption that more difficult problems require more thinking tokens. Based on the grading, we remove questions that either Qwen2.5-7B-Instruct or Qwen2.5-32B-Instruct can solve correctly and thus may be too easy. By using two models we reduce the likelihood of an easy sample slipping through our filtering due to a rare mistake on an easy question of one of the models. This brings our total samples down to 24,496 , setting the stage for the next round of subsampling based on diversity. While filtering with these two models may be optimized for our setup as we will also use Qwen2.5-32B-Instruct as our model to finetune, the idea of model-based filtering generalizes to other setups.

#### Diversity

To quantify diversity, we classify questions into domains using Claude 3.5 Sonnet based on the Mathematics Subject Classification (MSC) system (e.g., geometry, combinatorics, etc.) from the American Mathematical Society. 1 1 1 https://mathscinet.ams.org/mathscinet/msc/msc2020.html The taxonomy focuses on topics in mathematics but also includes other sciences such as biology, physics, and economics. To select our final examples from the pool of 24,496 questions, we first choose one domain uniformly at random. Then, we sample one problem from this domain according to a distribution that favors longer reasoning traces (see §C.4 for details) as motivated in Difficulty . We repeat this process until we have 1,000 total samples spanning 50 domains.

In §5.1 , we will show that using our three criteria in combination is important, as only relying on quality, diversity, or difficulty in isolation leads to worse datasets. Some distilled generations are incorrect, which we allow in our data as we focus on capturing the reasoning process rather than entirely correct solutions. Our grader ( §C.3 ) deems 53.6% correct in s1K and 63.0% in our follow-up s1K-1.1 (see §A ).

## 3 Test-time scaling

### 3.1 Method

We classify test-time scaling methods into 1) Sequential , where later computations depend on earlier ones (e.g., a long reasoning trace), and 2) Parallel , where computations run independently (e.g., majority voting) ( Snell et al., 2024 ; Brown et al., 2024 ) . We focus on sequential scaling as intuitively we believe it should scale better, since later computations can build on intermediate results, allowing for deeper reasoning and iterative refinement. We propose new sequential scaling methods and ways to benchmark them.

#### Budget forcing

We propose a simple decoding-time intervention by forcing a maximum and/or minimum number of thinking tokens. Specifically, we enforce a maximum token count by simply appending the end-of-thinking token delimiter and optionally “ Final Answer: ” to early exit the thinking stage and make the model provide its current best answer. To enforce a minimum, we suppress the generation of the end-of-thinking token delimiter and optionally append the string “Wait” to the model’s current reasoning trace to encourage the model to reflect on its current generation. Figure 3 contains an example of how this simple approach can lead the model to arrive at a better answer.

#### Baselines

We benchmark budget forcing with: (I) Conditional length-control methods , which rely on telling the model in the prompt how long it should generate for. We group them by granularity into (a) Token-conditional control: We specify an upper bound of thinking tokens in the prompt; (b) Step-conditional control: We specify an upper bound of thinking steps, where each step is around 100 tokens; (c) Class-conditional control: We write two generic prompts that tell the model to either think for a short or long amount of time (see §E.1 for details). (II) Rejection sampling , which samples until a generation fits a predetermined compute budget. This oracle captures the posterior over responses conditioned on its length.

### 3.2 Metrics

We establish a set of desiderata as evaluation metrics to measure test-time scaling across methods. Importantly, we do not only care about the accuracy a method can achieve but also its controllability and test-time scaling slope. For each method we consider, we run a set of evaluations a ∈ 𝒜 a\in\mathcal{A} varying test-time compute on a fixed benchmark, e.g. AIME24. This produces a piece-wise linear function f f with compute as the x-axis measured in thinking tokens and accuracy as the y-axis (see Figure 1 , where the rightmost dot for AIME24 corresponds to f ⁡ ( 7320 ) = 57 % f(7320)=57\% ). We measure three metrics: Control = 1 | 𝒜 | ​ ∑ a ∈ 𝒜 𝕀 ⁡ ( a min ≤ a ≤ a max ) \displaystyle\text{Control}=\frac{1}{|\mathcal{A}|}\sum_{a\in\mathcal{A}}\mathbb{I}(a_{\text{min}}\leq a\leq a_{\text{max}}) (1) where a min , a max a_{\text{min}},a_{\text{max}} refer to a pre-specified minimum and maximum amount of test-time compute; in our case thinking tokens. We usually only constrain a max a_{\text{max}} . As tokens generated correspond to the amount of test-time compute spent, this metric measures the extent to which a method allows controllability over the use of that test-time compute. We report it as a percentage with 100% being perfect control. Scaling = 1 ( | 𝒜 | 2 ) ​ ∑ a , b ∈ 𝒜 b > a f ⁡ ( b ) − f ⁡ ( a ) b − a \displaystyle\text{Scaling}=\frac{1}{\binom{|\mathcal{A}|}{2}}\sum_{\begin{subarray}{c}a,b\in\mathcal{A}\\ b>a\end{subarray}}\frac{f(b)-f(a)}{b-a} (2) Scaling is the average slope of the piece-wise linear function. It must be positive for useful methods and larger is better. Performance = max a ∈ 𝒜 ⁡ f ⁡ ( a ) \displaystyle=\max_{a\in\mathcal{A}}f(a) (3) Performance is simply the maximum performance the method achieves on the benchmark. A method with monotonically increasing scaling achieves 100% performance on any benchmark in the limit. However, the methods we investigate eventually flatten out or further scaling fails due to control or context window limitations.

## 4 Results

### 4.1 Setup

#### Training

We perform supervised finetuning on Qwen2.5-32B-Instruct using s1K to obtain our model s1-32B using basic hyperparameters outlined in §D . Finetuning took 26 minutes on 16 NVIDIA H100 GPUs with PyTorch FSDP.

#### Evaluation