Rethinking RL for LLM Reasoning: It's Sparse Policy Selection, Not Capability Learning — part 4/4

Cleaned-text code points [46867, 48712). Bibliography omitted here; source.md includes it.

DeepScaleR [ Luo et al., 2025 ] self‑reports 3,800 A100‑hours and a total cost of ∼ \sim $4,500, which we use as given. STILL‑3 [ Min et al., 2024 ] does not provide hardware details; the cost figure in Table 3 is taken from the comparison table in Open‑RS3 [ Dang and Ngo, 2025 ] , which estimates 1,200 A100‑hours on 8 × \times A100‑80GB. Open‑RS3 self‑reports 96 A40‑hours, which we convert to cost at the RunPod A40 rate.

#### Mistral‑7B.

The SimpleRL‑Zoo run for Mistral‑7B uses the same hardware and step count as the Qwen2.5‑7B group, with simplified prompts. We therefore assign it the same GPU‑hour estimate (240 H100‑hours). The training data is 8K Easy‑split problems.

In all cases, the cost figures in Table 3 are rounded to the nearest US dollar. The italicisation indicates estimates where the exact wall‑clock was not directly published by the original authors.

## Instructions for reporting errors

We are continuing to improve HTML versions of papers, and your feedback helps enhance accessibility and mobile support. To report errors in the HTML that will help us improve conversion and rendering, choose any of the methods listed below:

Click the "Report Issue" ( ) button, located in the page header.

Tip: You can select the relevant text first, to include it in your report.

Our team has already identified the following issues . We appreciate your time reviewing and reporting rendering errors we may not have found yet. Your efforts will help us improve the HTML versions for all readers, because disability should not be a barrier to accessing research. Thank you for your continued support in championing open access for all.

Have a free development cycle? Help support accessibility at arXiv! Our collaborators at LaTeXML maintain a list of packages that need conversion , and welcome developer contributions .