Cyclical Learning Rates for Training Neural Networks

arxiv:1506.01186

Original source · Paper JSON

Data, training, optimization · 2015-06-03

Cites (0) · Cited by (2)

Complete cleaned Markdown · Source Markdown including bibliography

Read in parts

1. Part 1
2. Part 2
3. Part 3

Contents

• Cyclical Learning Rates for Training Neural Networks
• Abstract
• 1 Introduction
• 2 Related work
• 3 Optimal Learning Rates
• 3.1 Cyclical Learning Rates
• 3.2 How can one estimate a good value for the cycle length?
• 3.3 How can one estimate reasonable minimum and maximum boundary values?
• 4 Experiments
• 4.1 CIFAR-10 and CIFAR-100
• 4.1.1 Caffe’s CIFAR-10 architecture
• 4.1.2 ResNets, Stochastic Depth, and DenseNets
• 4.2 ImageNet
• 4.2.1 AlexNet
• 4.2.2 GoogLeNet/Inception Architecture
• 5 Conclusions
• Appendix A Instructions for adding CLR to Caffe
• Appendix B Instructions for adding CLR to Keras
• Instructions for reporting errors