Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipe — part 5/5

Cleaned-text code points [61893, 65059). Bibliography omitted here; source.md includes it.

We collect all extracted DAPO-Math-17K questions into a set and remove any DeepMath example whose extracted question exactly matches one of the DAPO questions.

#### Stage 2: Semantic deduplication.

To further remove near-duplicate prompts, we encode both DAPO-Math-17K and DeepMath questions using the sentence embedding model all-mpnet-base-v2 [ Reimers and Gurevych, 2019 ] . We L2-normalize the embeddings and build a FAISS inner-product index over the DAPO embeddings, so that the inner product corresponds to cosine similarity. For each DeepMath question, we retrieve its top-1 nearest neighbor in DAPO-Math-17K. If the cosine similarity to the nearest DAPO question is at least 0.6 0.6 , we mark the DeepMath example as a semantic duplicate and remove it.

#### Final retained subset.

We remove any DeepMath example flagged by either exact-match or semantic deduplication. The resulting subset is in-domain but deduplicated against DAPO-Math-17K, enabling a controlled comparison between prompts that overlap with the teacher’s post-training data and prompts that are only in-domain.

### C.4 Benchmark-wise breakdown of prompt-template alignment

To further unpack the averaged result in Figure 9 , Figure 22 presents a benchmark-wise breakdown. The teacher-aligned template yields broadly consistent improvements across datasets, with larger gains on the two AIME sets and a smaller but still positive effect on AMC 2023. It also allows the student to recover a larger fraction of the teacher’s performance, increasing from roughly 80% to roughly 85%. Together with the overlap-ratio result in Section 5.2 , this suggests that prompt-template alignment improves OPD by making the student’s generated states more compatible with the teacher.

## Appendix D Details for Section 6

### D.1 Teacher entropy by output position

To complement the student entropy analysis in Section 6.1 , we also visualize teacher entropy as a function of output position across training steps under the 15 K K max response length setting (see Figure 23 ). Similar to the student, teacher entropy first increases at later decoding positions and then progressively propagates toward earlier tokens over training.

## Instructions for reporting errors

We are continuing to improve HTML versions of papers, and your feedback helps enhance accessibility and mobile support. To report errors in the HTML that will help us improve conversion and rendering, choose any of the methods listed below:

Click the "Report Issue" ( ) button, located in the page header.

Tip: You can select the relevant text first, to include it in your report.

Our team has already identified the following issues . We appreciate your time reviewing and reporting rendering errors we may not have found yet. Your efforts will help us improve the HTML versions for all readers, because disability should not be a barrier to accessing research. Thank you for your continued support in championing open access for all.

Have a free development cycle? Help support accessibility at arXiv! Our collaborators at LaTeXML maintain a list of packages that need conversion , and welcome developer contributions .