GRPO-VPS: Enhancing Group Relative Policy Optimization with Verifiable Process Supervision for Effective Reasoning

arxiv:2604.20659

Original source · Paper JSON

Post-training · 2026-04-22

Cites (6) · Cited by (0)

Complete cleaned Markdown · Source Markdown including bibliography

Read in parts

1. Part 1
2. Part 2
3. Part 3

Contents

• GRPO-VPS: Enhancing Group Relative Policy Optimization with Verifiable Process Supervision for Effective Reasoning
• Abstract
• 1 Introduction
• 2 Related Work
• 3 Methodology: Process Supervisions from Verifiable Outcomes
• 3.1 Reasoning Process Segmentation
• 3.2 Progress as Process Supervision
• 3.3 GRPO with Process Supervision
• 4 Experiment
• 4.1 Setup
• 4.2 Main Results
• 4.3 Ablation Study
• 4.4 Understanding How VPS Works
• 4.5 General Reasoning Benchmarks
• 5 Conclusion
• Appendix A Appendix
• A.1 Experimental Settings
• A.2 Prompt Templates
• A.3 Disclosure of LLM usage.
• A.4 Extended Empirical Results
• A.4.1 Response Length Dynamics across Model Families
• A.4.2 Additional Analysis on General Reasoning Benchmarks
• A.4.3 Sensitivity Analysis on α \alpha
• A.5 Qualitative Analysis of Segment-wise Process Signals.
• Instructions for reporting errors