Maintaining Plasticity in Deep Continual Learning — part 7/7
Cleaned-text code points [92351, 97615). Bibliography omitted here; source.md includes it.
In this work, we found that methods that continually injected randomness while maintaining small weight magnitudes greatly reduced loss of plasticity. Many works have found that adding noise while training neural networks can improve training and testing performance. The main benefits of adding noise have been reported to be avoiding overfitting and improving training performance 123 , 124 , 125 . However, it can be tricky to inject noise without degrading performance in some cases 126 . In our case, when the data distribution is non-stationary, we found that continually injecting noise along with L2 regularization helps with maintaining plasticity in neural networks. ## Data availability All of the datasets and simulation environments used in this work are publicly available. Other data needed to evaluate the conclusions in the article are present in the article or the extended data. ## Code availability The code is available at https://github.com/shibhansh/loss-of-plasticity . ## Supplementary information ### Peer Review File (download PDF ) ## Rights and permissions Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . Reprints and permissions ## About this article ### Cite this article Dohare, S., Hernandez-Garcia, J.F., Lan, Q. et al. Loss of plasticity in deep continual learning. Nature 632 , 768–774 (2024). https://doi.org/10.1038/s41586-024-07711-7 Download citation Received : 11 August 2023 Accepted : 12 June 2024 Published : 21 August 2024 Version of record : 21 August 2024 Issue date : 22 August 2024 DOI : https://doi.org/10.1038/s41586-024-07711-7 ### Share this article Anyone you share the following link with will be able to read this content: Sorry, a shareable link is not currently available for this article. Provided by the Springer Nature SharedIt content-sharing initiative ## Comments Commenting on this article is now closed. Darren Galpin 2 September 2024, 12:46 This would seem to have deep implications for AI/ML systems which are expected to continually learn, such as autonomous vehicles. There is an assumption that the models can continually learn from the different scenarios that they encounter while running, and also learn from each other as manufacturers try and speed learning by aggregating data from the different vehicles in the fleet, because of the long tail of rarer scenarios to which a larger number of vehicles can be expected to experience. There is an assumption of the functional safety of such a system. If the plasticity of the system can degrade over time, then the performance of the system can degrade, and therefore the system may not be as functionally safe as assumed. Random injection may help to maintain the plasticity and ability to learn, but it also implies a degree of lost-learning. If this random, non-gradient injection is done often enough, at what point is the original training data actually redundant? ## Associated content ### Switching between tasks can cause AI to lose the ability to learn Clare Lyle Razvan Pascanu ### AI can’t learn new things forever — an algorithm can fix that Benjamin Thompson Nick Petrić Howe Advertisement ## Search ### Quick links Explore articles by subject Find a job Guide to authors Editorial policies Nature ( Nature ) ISSN 1476-4687 (online) ISSN 0028-0836 (print) ## nature.com footer links ### About Nature Portfolio About us Press releases Press office Contact us ### Discover content Journals A-Z Articles by subject protocols.io Nature Index ### Publishing policies Nature portfolio policies Open access ### Author & Researcher services Reprints & permissions Research data Language editing Scientific editing Nature Masterclasses Research Solutions ### Libraries & institutions Librarian service & tools Librarian portal Open research Recommend to library ### Advertising & partnerships Advertising Partnerships & Services Media kits Branded content ### Professional development Nature Awards Nature Careers Nature Conferences ### Regional websites Nature Africa Nature China Nature India Nature Japan Nature Middle East Privacy Policy Use of cookies Your privacy choices/Manage cookies Legal notice Accessibility statement Terms & Conditions Your US state privacy rights © 2026 Springer Nature Limited Sign up for the Nature Briefing: AI and Robotics newsletter — what matters in AI and robotics research, free to your inbox weekly.