ESPO: Entropy Importance Sampling Policy Optimization

arxiv:2512.00499

Original source · Paper JSON

Post-training · 2025-11-29

Cites (6) · Cited by (1)

Complete cleaned Markdown · Source Markdown including bibliography

Read in parts

1. Part 1
2. Part 2
3. Part 3

Contents

• ESPO: Entropy Importance Sampling Policy Optimization
• Abstract
• 1 Introduction
• 2 Background
• 2.1 Related Works
• 2.2 Preliminary
• 3 Entropy Importance Sampling Policy Optimization
• 3.1 Objective Formulation
• 3.2 Entropy Adaptive Clipping
• 3.3 Entropy Grouping Importance Sampling
• 3.4 Token Grouping Advantage Normalization
• 4 Experiments
• 4.1 Experimental Setup
• Baselines.
• Configuration.
• Evaluation.
• 4.2 Overall Performance Comparison
• Comprehensive Benchmark Results.
• Training Dynamics.
• 4.3 Analysis of ESPO’s Mechanisms
• Addressing Gradient Underutilization.
• Validating Adaptive Optimization.
• 4.4 Ablation Studies
• Core Components.
• Grouping Proportion.
• Hyperparameter Sensitivity.
• 5 Conclusion
• Ethical Statement