Revisiting Rainbow: Promoting more Insightful and Inclusive Deep Reinforcement Learning Research

arxiv:2011.14826

Original source · Paper JSON

Reinforcement learning · 2020-11-20

Cites (5) · Cited by (3)

Complete cleaned Markdown · Source Markdown including bibliography

Read in parts

1. Part 1
2. Part 2
3. Part 3

Contents

• Revisiting Rainbow: Promoting more Insightful and Inclusive Deep Reinforcement Learning Research
• Abstract
• Keywords:
• 1 Introduction
• 2 Preliminaries
• 2.1 DQN
• 2.2 Rainbow
• 3 The cost of Rainbow
• 4 Revisiting Rainbow
• 4.1 Methodology
• 4.2 Empirical evaluation
• 5 Beyond the Rainbow
• 5.1 Examining network architectures and batch sizes
• 5.2 Examining distribution parameterizations
• 5.3 Munchausen Reinforcement Learning
• 5.4 Reevaluating the Huber loss
• 6 Putting it all together
• 6.1 Rainbow flavours
• 6.2 Environment properties
• 7 Conclusion
• 8 Acknowledgements
• Appendix A Environments
• A.1 CartPole-v0
• A.2 Acrobot-v1
• A.3 LunarLander-v2
• A.4 MountainCar-v0
• A.5 MinAtar Environments
• Appendix B Detailed description of the revisting rainbow components
• B.1 Double Q-learning
• B.2 Prioritized experience replay
• B.3 Dueling networks
• B.4 Multi-step learning
• B.5 Distributional RL
• B.6 Quantile Regression for Distributional RL
• B.7 Implicit quantile networks
• B.8 Noisy nets
• B.9 Munchausen Reinforcement Learning
• B.10 Loss functions
• Appendix C Hyperparameters settings for classic control environments
• Appendix D Hyperparameters settings for MinAtar environments
• Appendix E Average runtime for environments used in the experiments
• Appendix F Impact of algorithmic components
• Appendix G Examining network architectures and batch sizes
• Appendix H Reevaluating the Huber loss, complete results
• H.1 Classic control and MinAtar results
• H.2 Atari results
• Appendix I Rainbow flavours, full results
• Instructions for reporting errors