Step-size Optimization for Continual Learning

arxiv:2401.17401

Original source · Paper JSON

Data, training, optimization · 2024-01-30

Cites (0) · Cited by (1)

Complete cleaned Markdown · Source Markdown including bibliography

Read in parts

1. Part 1
2. Part 2

Contents

• Step-size Optimization for Continual Learning
• Abstract
• 1 The Role of Step-size in Continual Learning
• 2 Setting
• 3 Limitations of Step-size Normalization
• 4 Step-size Optimization with IDBD
• 5 Limitations of IDBD
• 6 Existing Extensions of IDBD
• 7 Other Step-size Adaptation Algorithms
• 8 A Promising Research Direction: Normalized Step-size Optimization