Attractor and integrator networks in the brain — part 7/10

Cleaned-text code points [86851, 100705). Bibliography omitted here; source.md includes it.
0-3636-7272-108108-144144-180
RUN
REM
NREM
0-0.250.25-0.50.5 -0.750.75-1.00> 1.00
1.4
1.2
1.0
0.8
0.6
0.4
0.2
0
cell 1cell 2
familiar env novel env
a
data
slice pred.
firing rate
1D track response
x (m)
d
MEC CA1
rel. spatial phase rel. distance
c cross-correln. coeff. (norm)
 grid score = 0.92rel. position x (um)
rel. position y (um)
rel. positions of same-phase cells
0
0.4
0.8
1.2
00.250.50.7510 3672108144
b
-200 0 200
-200 2000
0 2 4 6
1
2
1
2
1
2 rescale (c1)rescale (c2)ratio
e
Trajectory time (s)
Figure 6: A set of 2-dimensional toroidal attractors in the grid cell system. (a) An example pair of grid cells (left column) whose
spatial tuning periods and orientations reconﬁgure in novel environments (middle column), but the changes are tightly yoked to
preserve cell-cell relationships (right column). Each color corresponds to a variable describing the lattice of the spatial tuning curve
of the cell as shown in the schematic. Adapted with permission from [76]. (b) Responses of co-modular cells on 1D linear tracks
can explained by parallel slices through a 2-dimensional grid, suggesting preserved 2-dimensional circuit dynamics across diverse
environments. Reproduced with permission from [167]. (c) Pairwise correlations between grid cells measured during navigation are
preserved across overnight sleep, while those of place cells are not. Reproduced with permission from [100, 101]. (d) Grid cells are
anatomically arranged according to their relative spatial ﬁring phases. (left) Cell positions are colored according to the phase of their
spatial tuning curves. The relative cortical positions of same-phase cells make a triangular lattice pattern (middle), with a grid-like
autocorrelation pattern (right). Reproduced with permission from [274] (e) Non-linear dimensionality reduction and topological data
analysis directly reveal that the states of individual grid modules lie on a torus (left); as the animal follows a spatial trajectory (right),
the state moves along the manifold (left). Reproduced with permission from [20].
Acknowledgements
I.R.F . acknowledges funding from the Simons Foundation, ONR, HHMI through the Faculty Schol-
ars Program, the Department of Brain and Cognitive Sciences, MIT , and the McGovern Institute,
MIT . M.K. is supported by a Friends of the McGovern Institute Fellowship, a MathWorks Fellow-
ship, and the Department of Physics, MIT .
We are grateful to Kayvon Daie, the anonymous reviewers, Natasha Bray , and Sarthak Chandra
and other members of the Fiete lab for helpful comments on the manuscript.
References
[1] S-I Amari. Neural theory of association and concept-formation. Biological cybernetics ,
26(3):175–185, 1977.
[2] John J Hopﬁeld. Neural networks and physical systems with emergent collective computa-
tionalabilities. Proc Natl Acad Sci U S A , 79:2554–2558, April 1982.
[3] William A Little. The existence of persistent states in the brain. In From High-T emperature
Superconductivity to Microminiature Refrigeration , pages 145–164. Springer , 1974.
25

[4] Hugh R Wilson and Jack D Cowan. A mathematical theory of the functional dynamics of
cortical and thalamic nervous tissue. Kybernetik, 13(2):55–80, 1973.
[5] John J Hopﬁeld. Neurons with graded response have collective computational properties
like those of two-state neurons. Proc Natl Acad Sci U S A , 81:3088–3092, May 1984.
[6] Michael A Cohen and Stephen Grossberg. Absolute stability of global pattern formation
and parallel memory storage by competitive neural networks. IEEE transactions on systems,
man, and cybernetics , (5):815–826, 1983.
[7] S Amari. Dynamics of pattern formation in lateral-inhibition type neural ﬁelds. Biol Cybern,
27(2):77–87, Aug 1977.
[8] Stephen C Cannon, David A Robinson, and Shihab Shamma. A proposed neural network
for the integrator of the oculomotor system. Biological cybernetics, 49(2):127–136, 1983.
[9] H Sebastian Seung. How the brain keeps the eyes still. Proceedings of the National Academy
of Sciences, 93(23):13339–13344, 1996.
[10] K. Zhang. Representation of spatial orientation by the intrinsic dynamics of the head-
direction cell ensemble: a theory . J Neurosci, 15:2112–2126, 1996.
[11] R Ben-Yishai, R L Bar-Or , and H Sompolinsky . Theory of orientation tuning in visual cortex.
Proc Natl Acad Sci U S A , 92(9):3844–8, Apr 1995.
[12] Bard Ermentrout. Neural networks as spatio-temporal pattern-forming systems. Reports on
progress in physics , 61(4), 4 1998. An optional note.
[13] SM Stringer , TP Trappenberg, ET Rolls, and IETd Araujo. Self-organizing continuous at-
tractor networks and path integration: one-dimensional models of head direction cells.
Network: Computation in Neural Systems , 13(2):217–242, 2002.
[14] Xiaohui Xie, Richard H R Hahnloser , and H Sebastian Seung. Double-ring network model
of the head-direction system. Phys Rev E Stat Nonlin Soft Matter Phys , 66(4 Pt 1):041902,
Oct 2002.
[15] Mark C Fuhs and David S T ouretzky . A spin glass model of path integration in rat medial
entorhinal cortex. J Neurosci, 26(16):4266–76, Apr 2006.
[16] Y oram Burak and Ila R Fiete. Accurate path integration in continuous attractor network
models of grid cells. PLoS Comput Biol , 5(2):e1000291, Feb 2009.
[17] Xiao-Jing Wang. Probabilistic decision making by slow reverberation in cortical circuits.
Neuron, 36(5):955–968, 2002.
[18] J. W . Milnor . Attractor .Scholarpedia, 1(11):1815, 2006. revision #186525.
[19] Rishidev Chaudhuri, Berk Gerçek, Biraj Pandey , Adrien Peyrache, and Ila Fiete. The intrinsic
attractor manifold and population dynamics of a canonical cognitive circuit across waking
and sleep. Nature neuroscience, 22(9):1512–1520, 2019.
26

[20] Richard J Gardner , Erik Hermansen, Marius Pachitariu, Y oram Burak, Nils A Baas, Ben-
jamin J Dunn, May-Britt Moser , and Edvard I Moser . T oroidal topology of population activ-
ity in grid cells. bioRxiv, 2021.
[21] Steven H Strogatz. Nonlinear dynamics and chaos: with applications to physics, biology,
chemistry, and engineering. CRC Press, 2018.
[22] Christof Koch. Biophysics of computation: information processing in single neurons . Oxford
university press, 2004.
[23] Michael N Shadlen and William T Newsome. The variable discharge of cortical neurons:
implications for connectivity , computation, and information coding.Journal of neuroscience,
18(10):3870–3896, 1998.
[24] Cyril Hanus and Erin M Schuman. Proteostasis in complex dendrites. Nature Reviews Neu-
roscience, 14(9):638, 2013.
[25] William James. The Principles of Psychology . Henry Holt Co., 1890.
[26] W . MCDOUGALL. ON THE SEA T OF THE PSYCHO-PHYSICAL PROCESSES. Brain,
24(4):579–630, 10 1901.
[27] Donald O. Hebb. The Organization of Behavior . John Wiley Sons., 1949.
[28] Richard E Brown, Thaddeus W B Bligh, and Jessica F Garden. The hebb synapse before
hebb: Theories of synaptic function in learning and memory before , with a discussion of
the long-lost synaptic theory of william mcdougall. Front Behav Neurosci, 15:732195, 2021.
[29] SJ Martin, PD Grimwood, and RG Morris. Synaptic plasticity and memory: an evaluation
of the hypothesis. Annu Rev Neurosci 2000 , 23:649–711, 2000.
[30] Wickliffe C Abraham, Owen D Jones, and David L Glanzman. Is plasticity of synapses the
mechanism of long-term memory storage? NPJ science of learning , 4(1):1–10, 2019.
[31] T omonori T akeuchi, Adrian J Duszkiewicz, and Richard GM Morris. The synaptic plasticity
and memory hypothesis: encoding, storage and persistence. Philosophical Transactions of
the Royal Society B: Biological Sciences , 369(1633):20130288, 2014.
[32] P . W . Anderson. More is different.Science, 177(4047):393–396, 1972.
[33] Huaguang Zhang, Zhanshan Wang, and Derong Liu. A comprehensive review of stability
analysis of continuous-time recurrent neural networks. IEEE Transactions on Neural Net-
works and Learning Systems , 25(7):1229–1262, 2014.
[34] Alfonso Renart, Pengcheng Song, and Xiao-Jing Wang. Robust spatial working mem-
ory through homeostatic synaptic scaling in heterogeneous cortical networks. Neuron,
38(3):473–485, 2003.
[35] Vladimir Itskov , David Hansel, and Misha Tsodyks. Short-term facilitation may stabilize
parametric working memory trace. Frontiers in computational neuroscience , 5:40, 2011.
[36] Alan. M. T uring. The chemical basis of morphogenesis. Philosophical Transactions of the
Royal Society of London. Series B, Biological Sciences , 237(641):37–72, August 1952.
27

[37] A. Gierer and H. Meinhardt. A theory of biological pattern formation. Kybernetik, 12(1):30–
39, dec 1972.
[38] Mark C Cross and Pierre C Hohenberg. Pattern formation outside of equilibrium. Reviews
of modern physics , 65(3):851, 1993.
[39] A. J. Koch1 and H. Meinhardt. Biological pattern formation : from basic mechanisms to
complex structures. Rev. Modern Physics, (66):1481–1507, 1994.
[40] François Schweisguth and Francis Corson. Self organization in pattern formation. Dev. Cell,
(49):659–677, Jun 2019.
[41] Boris Shraiman. Mechanical feedback as a possible regulator of tissue growth. Proceedings
of the National Academy of Sciences , 102(9):3318–3323, Jun 2005.
[42] Noji S. Ueno N. Maini P .K. Sekimura, T . Morphogenesis and Pattern Formation in Biological
Systems: Experiments and Models . Springer , 2003.
[43] Christian Boucheny , Nicolas Brunel, and Angelo Arleo. A continuous attractor network
model without recurrent excitation: maintenance and integration in the head direction
cell system. Journal of computational neuroscience , 18(2):205–227, 2005.
[44] Jonathan J Couey , Aree Witoelar , Sheng-Jia Zhang, Kang Zheng, Jing Y e, Benjamin Dunn,
Rafal Czajkowski, May-Britt Moser , Edvard I Moser , Y asser Roudi, and Menno P Witter .
Recurrent inhibitory circuitry as a mechanism for grid formation. Nat Neurosci, 16(3):318–
24, Mar 2013.
[45] H Sebastian Seung. Ampliﬁcation, attenuation, and integration. The handbook of brain
theory and neural networks , 2:94–97, 2003.
[46] C van Vreeswijk and H Sompolinsky . Chaos in neuronal networks with balanced excitatory
and inhibitory activity . Science, 274(5293):1724–6, Dec 1996.
[47] Haim Sompolinsky , Andrea Crisanti, and Hans-Jurgen Sommers. Chaos in random neural
networks. Physical review letters , 61(3):259, 1988.
[48] Rainer Engelken, Fred Wolf, and LF Abbott. Lyapunov spectra of chaotic recurrent neural
networks. arXiv preprint arXiv:2006.02427 , 2020.
[49] Brendan K Murphy and Kenneth D Miller . Balanced ampliﬁcation: a new mechanism of
selective ampliﬁcation of neural activity patterns. Neuron, 61(4):635–648, 2009.
[50] Lloyd N Trefethen, Anne E Trefethen, Satish C Reddy , and T obin A Driscoll. Hydrodynamic
stability without eigenvalues. Science, 261(5121):578–584, 1993.
[51] Gianluigi Mongillo, Omri Barak, and Misha Tsodyks. Synaptic theory of working memory .
Science, 319(5869):1543–6, Mar 2008.
[52] Albert Compte, Nicolas Brunel, Patricia S Goldman-Rakic, and Xiao-Jing Wang. Synap-
tic mechanisms and network dynamics underlying spatial working memory in a cortical
network model. Cerebral cortex, 10(9):910–923, 2000.
28

[53] Y oram Burak and Ila R Fiete. Fundamental limits on persistent activity in networks of noisy
neurons. Proc Natl Acad Sci U S A , 109(43):17645–50, Oct 2012.
[54] Rafal Bogacz, Eric Brown, Jeff Moehlis, Philip Holmes, and Jonathan D Cohen. The physics
of optimal decision making: a formal analysis of models of performance in two-alternative
forced-choice tasks. Psychol Rev, 113(4):700–65, Oct 2006.
[55] Eric B Baum, John Moody , and Frank Wilczek. Internal representations for associative
memory .Biological Cybernetics, 59(4-5):217–228, 1988.
[56] Lawrence K Saul and Michael I Jordan. Attractor dynamics in feedforward neural networks.
Neural computation, 12(6):1313–1335, 2000.
[57] Sugandha Sharma, Sarthak Chandra, and Ila R Fiete. Content addressable memory with-
out catastrophic forgetting by heteroassociation with a ﬁxed scaffold. arXiv preprint
arXiv:2202.00159, 2022.
[58] S Funahashi, C J Bruce, and P S Goldman-Rakic. Mnemonic coding of visual space in the
monkey’s dorsolateral prefrontal cortex. J Neurophysiol, 61(2):331–49, Feb 1989.
[59] Clayton E. Curtis and Mark D’Esposito. Persistent activity in the prefrontal cortex during
working memory . Trends Cogn Sci, 7(9):415–423, Sep 2003.
[60] Rishidev Chaudhuri and Ila Fiete. Bipartite expander hopﬁeld networks as self-decoding
high-capacity error correcting codes. In Advances in Neural Information Processing Systems ,
pages 7686–7697, 2019.
[61] David Redish, Adam N Elga, and David S T ouretzky . A coupled attractor model of the rodent
head direction system. Network: Computation in Neural Systems , 7:671685, 1996.
[62] Pengcheng Song and Xiao-Jing Wang. Angular path integration by moving “hill of activity”:
a spiking neuron model without recurrent excitation of the head-direction system. Journal
of Neuroscience, 25(4):1002–1014, 2005.
[63] Xiao-Jing Wang. Decision making in recurrent neuronal circuits. Neuron, 60(2):215–34,
Oct 2008.
[64] B. Kriener , R. Chaudhuri, and I.R. Fiete. Robust parallel decision-making in neural circuits
with nonlinear inhibition. PNAS, (to appear), 2020.
[65] Marius Usher and James L McClelland. The time course of perceptual choice: the leaky ,
competing accumulator model. Psychological review, 108(3):550, 2001.
[66] Kong-Fatt Wong and Xiao-Jing Wang. A recurrent network mechanism of time integration
in perceptual decisions. Journal of Neuroscience , 26(4):1314–1328, 2006.
[67] Richard HR Hahnloser , Rahul Sarpeshkar , Misha A Mahowald, Rodney J Douglas, and H Se-
bastian Seung. Digital selection and analogue ampliﬁcation coexist in a cortex-inspired
silicon circuit. Nature, 405(6789):947, 2000.
[68] Rafal Bogacz and Kevin Gurney . The basal ganglia and cortex implement optimal decision
making between alternative actions. Neural Comput, 19(2):442–77, Feb 2007.
29