Maintaining Plasticity in Deep Continual Learning

doi:10.1038/s41586-024-07711-7

Original source · Paper JSON

Data, training, optimization · 2023-06-23

Cites (7) · Cited by (3)

Complete cleaned Markdown · Source Markdown including bibliography

Read in parts

1. Part 1
2. Part 2
3. Part 3
4. Part 4
5. Part 5
6. Part 6
7. Part 7

Contents

• Loss of plasticity in deep continual learning
• Abstract
• Similar content being viewed by others
• Engineering flexible machine learning systems by traversing functionally invariant paths
• Variational tensor neural networks for deep learning
• Physical deep learning with biologically inspired training method: gradient-free approach for physical hardware
• Explore related subjects
• Main
• Plasticity loss in supervised learning
• Plasticity loss in reinforcement learning
• Maintaining plasticity
• Discussion
• Methods
• Specifics of continual backpropagation
• Algorithm 1
• Details of Continual ImageNet
• Class-incremental CIFAR-100
• Robust loss of plasticity in permuted MNIST
• Loss of plasticity with different activations in the Slowly-Changing Regression problem
• Understanding loss of plasticity
• Existing deep-learning methods for mitigating loss of plasticity
• Details and further analysis in reinforcement learning
• Extended discussion
• Data availability
• Code availability
• Supplementary information
• Peer Review File (download PDF )
• Rights and permissions
• About this article
• Cite this article
• Share this article
• Comments
• Associated content
• Switching between tasks can cause AI to lose the ability to learn
• AI can’t learn new things forever — an algorithm can fix that
• Search
• Quick links
• nature.com footer links
• About Nature Portfolio
• Discover content
• Publishing policies
• Author & Researcher services
• Libraries & institutions
• Advertising & partnerships
• Professional development
• Regional websites