Revisiting Reinforcement Learning with Verifiable Rewards from a Contrastive Perspective

arxiv:2605.12969

Original source · Paper JSON

Post-training · 2026-05-13

Cites (7) · Cited by (0)

Complete cleaned Markdown · Source Markdown including bibliography

Read in parts

1. Part 1
2. Part 2
3. Part 3
4. Part 4

Contents

• Revisiting Reinforcement Learning with Verifiable Rewards from a Contrastive Perspective
• Abstract
• 1 Introduction
• 2 Related Work
• RLVR for LLM reasoning.
• Contrastive objectives for policy alignment.
• 3 Preliminaries
• 4 Methods
• 4.1 An Equivalent Discriminative Reformulation of GRPO
• Proposition 1 .
• Lemma 1 .
• Implications.
• 4.2 A Contrastive View of RLVR Optimization
• Lemma 2 .
• 4.3 Scheduled Margin as a Separation Curriculum
• 5 Experiments
• 5.1 Settings
• Models.
• Training data.
• Benchmarks.
• Baselines.
• Implementation details.
• 5.2 Main Results
• Performance with reasoning models.
• Generalization across backbone models and training datasets.
• 5.3 Ablation Study
• 5.4 Parameter Study
• 6 Conclusion
• Limitations
• Ethical Considerations
• Acknowledgement
• Appendix A Proofs
• A.1 Proof of Proposition 1 and Lemma 1
• Lemma 3 .
• Proof.
• Lemma 4 .
• Proof.
• Proof of Proposition 1 .
• Proof of Lemma 1 .
• A.2 Proof of Lemma 2
• Proof.
• Appendix B Detailed Related Work
• RLVR for LLM reasoning.
• Contrastive Objectives for Policy Alignment.
• Appendix C Detailed Experimental Settings
• C.1 Benchmarks
• C.2 Implementation Details
• Appendix D Additional Experimental Results
• Appendix E Prompt Template
• Instructions for reporting errors