Beyond The Rainbow: High Performance Deep Reinforcement Learning on a Desktop PC

arxiv:2411.03820

Original source · Paper JSON

Reinforcement learning · 2024-11-06

Cites (8) · Cited by (0)

Complete cleaned Markdown · Source Markdown including bibliography

Read in parts

1. Part 1
2. Part 2
3. Part 3
4. Part 4

Contents

• Beyond The Rainbow: High Performance Deep Reinforcement Learning on a Desktop PC
• Abstract
• Keywords:
• 1 Introduction
• 2 Background
• 2.1 RL Problem Formulation
• 2.2 Deep Q-Learning (DQN)
• 2.3 Rainbow DQN and Improvements to DQN
• 3 Beyond the Rainbow - Extensions and Improvements
• 3.1 Extensions
• 3.2 Hyperparameters
• 4 Evaluation
• 4.1 Atari and Procgen Performance
• 4.2 Applying BTR to Modern Games
• 5 Analysis
• 5.1 Ablations Studies
• 5.2 What are the effects of BTR’s components?
• 6 Related Work
• 7 Conclusions
• Impact Statement
• Acknowledgments
• Appendix A Full Results Tables
• Appendix B Full Results Graphs
• Appendix C Additional Ablations
• Appendix D Hyperparameters
• D.1 Environment Details
• D.2 Algorithm Hyperparameters
• D.3 Clarity of the terms Frames, Steps and Transitions
• Appendix E Beyond The Rainbow Architecture & Loss Function
• E.1 Architecture
• E.2 Loss Function
• E.3 Analysis Confidence Intervals
• Appendix F BTR with and without Epsilon Greedy
• Appendix G Experiment Compute Resources
• G.1 Our Compute Resources
• G.2 BTR with Different Hardware
• Appendix H Other Things We Tried
• Appendix I Altered Atari Environment Settings
• Appendix J BTR for Wii Games
• J.1 Super Mario Galaxy
• J.2 Mario Kart Wii
• J.3 Mortal Kombat
• Appendix K Atari-5 Regression Procedure
• Instructions for reporting errors