Q-Learning With World Models

arxiv:2608.17163

Original source · Paper JSON

Reinforcement learning · 2026-08-17

Cites (4) · Cited by (0)

Complete cleaned Markdown · Source Markdown including bibliography

Read in parts

1. Part 1
2. Part 2
3. Part 3
4. Part 4

Contents

• Q-Learning With World Models
• Abstract
• 1 Introduction
• 2 Related Work
• World models for VLAs.
• Test-time scaling and best-of- N N inference.
• 3 Preliminaries
• 4 Method
• 4.1 Tree Search With the World Model
• 4.2 Aggregating the Value
• Root value.
• 4.3 Searching with the Q-function.
• 4.4 Practical Implementation
• 5 Experiments
• 5.1 How Does QWM Perform Compared to State-of-the-Art Model-Free and Model-Based RL Methods?
• 5.2 How Does QWM Perform Compared to the Base Method It Is Implemented on Top of?
• 5.3 Does QWM Scale to a Pixel-Based Setting?
• 5.4 What Components of QWM Are Most Important for Performance?
• 6 Discussion
• 7 Acknowledgments
• Appendix A Author Contributions
• Appendix B Additional Experiments
• Appendix C Experiment Details
• C.1 Hyperparameters
• QWM -specific hyperparameters.
• C.2 Environments
• C.3 Additional World Model Details and Prediction Quality
• C.4 Comparisons
• Instructions for reporting errors