Latent On-Policy Self-Distillation

arxiv:2608.13040

Original source · Paper JSON

Post-training · 2026-08-13

Cites (8) · Cited by (0)

Complete cleaned Markdown · Source Markdown including bibliography

Read in parts

1. Part 1
2. Part 2
3. Part 3
4. Part 4

Contents

• Latent On-Policy Self-Distillation
• Abstract
• 1 Introduction
• 2 Related Work
• On-Policy (Self-)Distillation.
• Latent Computation.
• 3 Method
• 3.1 Problem Setup: Privileged Context in OPSD
• 3.2 Learnable Latent Privileged Context
• 3.3 Latent On-Policy Self-Distillation
• Privileged-Margin Constraint.
• Algorithm Summary.
• 4 Experiments
• 4.1 Experiment Setup
• Training.
• Evaluation.
• Baselines.
• Configurations.
• 4.2 Main Results
• Performance.
• No Hand-Crafted Context Is Universally Optimal.
• Learnable Context Matters.
• 4.3 Framework Analysis
• Effect of Joint Optimization.
• Training Dynamics and Sample Efficiency.
• Sensitivity Analysis.
• Behavioral Internalization.
• 4.4 Case Study
• 5 Conclusion
• Appendix A Implementation Details
• A.1 Composer Architecture
• A.2 Cold-Start Initialization
• A.3 Retrieval Configuration
• A.4 Latent Injection Mechanism
• Placeholder strategy.
• Embedding replacement.
• Engine-specific implementations.
• Equivalence verification.
• Encoder LoRA isolation.
• A.5 Prompt and Interaction Format
• Agentic system prompt.
• Coding system prompt.
• Latent-context framing text.
• Encoder input format.
• Appendix B Experiment Details
• B.1 Baseline Setup
• Vanilla.
• GRPO.
• SDFT.
• OPSD.
• SDPO.
• Skill-SD.
• B.2 Training Data and Domain Setup
• B.3 Evaluation Protocols
• EnvScaler.
• BFCL-v3.
• ACEBench.
• LiveCodeBench.
• EvalPlus.
• B.4 Ablation and Analysis Setup
• Effect of Joint Optimization ( Figure 3 ).
• Training Dynamics ( Figure 4 ).
• Sensitivity Analysis ( Figure 5 ).
• Behavioral Internalization ( Table 3 ).
• Instructions for reporting errors