A Path Towards Autonomous Machine Intelligence (LeCun, 2022) — part 11/12

Cleaned-text code points [140812, 156539). Bibliography omitted here; source.md includes it.
Finn, C. and Levine, S. (2017). Deep visual foresight for planning robot motion. In 2017
IEEE International Conference on Robotics and Automation (ICRA), pages 2786–2793.
IEEE.
Fragkiadaki, K., Agrawal, P., Levine, S., and Malik, J. (2015). Learning visual predictive
models of physics for playing billiards. CoRR, abs/1511.07404.
Gehring, J., Synnaeve, G., Krause, A., and Usunier, N. (2021). Hierarchical skills for eﬃcient
exploration. Advances in Neural Information Processing Systems , 34:11553–11564.
Goldberger, J., S.Roweis, Hinton, G., and Salakhutdinov, R. (2005). Neighbourhood com-
ponents analysis. In Saul, L. K., Weiss, Y., and Bottou, L., editors, Advances in Neural
Information Processing Systems 17 , pages 513–520. MIT Press, Cambridge, MA.
Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville,
A., and Bengio, Y. (2014). Generative adversarial nets. In Advances in Neural Infor-
mation Processing Systems, pages 2672–2680.
Gopnik, A. and Meltzoﬀ, A. N. (1997). Words, Thoughts, and Theories . MIT Press,
Cambridge, MA.
Gopnik, A., Meltzoﬀ, A. N., and Kuhl, P. K. (2001). The Scientist in the Crib: What Early
Learning Tells Us About the Mind . Perennial, New York, NY.
Goroshin, R., Bruna, J., Tompson, J., Eigen, D., and LeCun, Y. (2015a). Unsupervised
feature learning from temporal data. In International Conference on Computer Vision
(ICCV 2015).
Goroshin, R., Mathieu, M., and LeCun, Y. (2015b). Learning to linearize under uncertainty.
In Advances in Neural Information Processing Systems (NIPS 2015) , volume 28.
Gottlieb, J., Oudeyer, P. Y., Lopes, M., and Baranes, A. (2013). Information-seeking,
curiosity, and attention: Computational and neural mechanisms. Trends in Cognitive
Sciences, 17:585–593.
Gregor, K. and LeCun, Y. (2010a). Emergence of complex-like cells in a temporal product
network with local receptive ﬁelds. arXiv preprint arXiv:1006.0448 .
Gregor, K. and LeCun, Y. (2010b). Learning fast approximations of sparse coding. In Proc.
International Conference on Machine learning (ICML’10) .
Grill, J.-B., Strub, F., Altch´ e, F., Tallec, C., Richemond, P. H., Buchatskaya, E., Doersch,
C., Pires, B. A., Guo, Z. D., Azar, M. G., Piot, B., Kavukcuoglu, K., Munos, R.,
and Valko, M. (2020). Bootstrap your own latent: A new approach to self-supervised
learning. In NeurIPS.
Ha, D. and Schmidhuber, J. (2018a). Recurrent world models facilitate policy evolution. In
Bengio, S., Wallach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N., and Garnett,
R., editors, Advances in Neural Information Processing Systems , volume 31.
51

Ha, D. and Schmidhuber, J. (2018b). World models. arXiv preprint arXiv:1803.10122 .
Hadsell, R., Chopra, S., and LeCun, Y. (2006). Dimensionality reduction by learning an
invariant mapping. In CVPR.
Hafner, D., Lee, K.-H., Fischer, I., and Abbeel, P. (2022). Deep hierarchical planning from
pixels. arXiv preprint arXiv:2206.04114 .
Hafner, D., Lillicrap, T., Fischer, I., Villegas, R., Ha, D., Lee, H., and Davidson, J. (2018).
Learning latent dynamics for planning from pixels. arXiv 1811.04551.
Hafner, D., Lillicrap, T., Norouzi, M., and Ba, J. (2020). Mastering atari with discrete
world models. arXiv preprint arXiv:2010.02193 .
He, K., Fan, H., Wu, Y., Xie, S., and Girshick, R. (2020). Momentum contrast for unsu-
pervised visual representation learning. In CVPR.
Henaﬀ, M., Canziani, A., and LeCun, Y. (2019). Model-predictive policy learning with
uncertainty regularization for driving in dense traﬃc. In ICLR-19. arXiv:1901.02705.
Henaﬀ, M., Weston, J., Szlam, A., Bordes, A., and LeCun, Y. (2017). Tracking the world
state with recurrent entity networks. In International Conference on Learning Repre-
sentations (ICLR 2017) .
Hinton, G. and Sejnowski, T. (1983). Optimal perceptual inference. In Proceedings of
the IEEE Conference on Computer Vision and Pattern Recognition , pages 448–453,
Washington 1983. IEEE, New York.
Ho, M. K., Abel, D., Correa, C. G., Littman, M. L., Cohen, J. D., and Griﬃths, T. L. (2022).
People construct simpliﬁed mental representations to plan. Nature, 606(7912):129–136.
H´ enaﬀ, O. J., Srinivas, A., De Fauw, J., Razavi, A., Doersch, C., Eslami, S. M. A., and
van den Oord, A. (2019). Data-eﬃcient image recognition with contrastive predictive
coding. In ICML.
Janner, M., Li, Q., and Levine, S. (2021). Oﬄine reinforcement learning as one big sequence
modeling problem. In Advances in Neural Information Processing Systems .
Jing, L., Zbontar, J., et al. (2020). Implicit rank-minimizing autoencoder. Advances in
Neural Information Processing Systems, 33:14736–14746.
Jordan, M. I. and Rumelhart, D. E. (1992). Forward models: Supervised learning with a
distal teacher. Cognitive science, 16(3):307–354.
Kahneman, D. (2011). Thinking, fast and slow . Macmillan.
Kingma, D. P. and Welling, M. (2013). Auto-encoding variational bayes. arXiv preprint
arXiv:1312.6114.
Lake, B. M., Ullman, T. D., Tenenbaum, J. B., and Gershman, S. J. (2017a). Building
machines that learn and think like people. Behavioral and brain sciences , 40.
52

Lake, B. M., Ullman, T. D., Tenenbaum, J. B., and Gershman, S. J. (2017b). Building
machines that learn and think like people. Behavioral and Brain Sciences , 40:E253.
LeCun, Y., Bottou, L., Bengio, Y., and Haﬀner, P. (1998). Gradient-based learning applied
to document recognition. Proceedings of the IEEE, 86(11):2278–2324.
LeCun, Y., Chopra, S., Hadsell, R., Ranzato, M., and Huang, F. (2006). A tutorial on
energy-based learning. In Bakir, G., Hofman, T., Sch¨ olkopf, B., Smola, A., and Taskar,
B., editors, Predicting Structured Data. MIT Press.
Lerer, A., Gross, S., and Fergus, R. (2016). Learning physical intuition of block towers by
example. In Proceedings of the 33nd International Conference on Machine Learning,
ICML 2016, New York City, NY, USA, June 19-24, 2016 , pages 430–438.
Levine, S. (2021). Understanding the world through action. arXiv:2110.12543. https:
//arxiv.org/abs/2110.12543.
Luc, P., Clark, A., Dieleman, S., Casas, D. d. L., Doron, Y., Cassirer, A., and Simonyan, K.
(2020). Transformation-based adversarial video prediction on large-scale data. arXiv
preprint arXiv:2003.04035.
Luc, P., Couprie, C., Lecun, Y., and Verbeek, J. (2018). Predicting future instance segmen-
tation by forecasting convolutional features. In Proceedings of the european conference
on computer vision (ECCV) , pages 584–599.
Luc, P., Neverova, N., Couprie, C., Verbeek, J., and LeCun, Y. (2017). Predicting deeper
into the future of semantic segmentation. In Proceedings of the IEEE international
conference on computer vision, pages 648–657.
Marcus, G. and Davis, E. (2019). Rebooting AI: Building artiﬁcial intelligence we can trust .
Vintage.
Mathieu, M., Couprie, C., and LeCun, Y. (2015). Deep multi-scale video prediction beyond
mean square error. In ICLR 16. arXiv preprint arXiv:1511.05440.
Mattar, M. G. and Lengyel, M. (2022). Planning in the brain. Neuron, 110(6):914–934.
Mercat, J., Gilles, T., El Zoghby, N., Sandou, G., Beauvois, D., and Gil, G. P. (2020). Multi-
head attention for multi-modal joint vehicle motion forecasting. In 2020 IEEE Inter-
national Conference on Robotics and Automation (ICRA) , pages 9638–9644. IEEE.
Miller, A. H., Fisch, A., Dodge, J., Karimi, A.-H., Bordes, A., and Weston, J. (2016).
Key-value memory networks for directly reading documents. In EMNLP-16.
Miller, W. T., Sutton, R. S., and Werbos, P. J. (1995). Neural networks for control . MIT
press.
Misra, I. and Maaten, L. v. d. (2020). Self-supervised learning of pretext-invariant repre-
sentations. In CVPR.
53

Moerland, Thomas, M., Broekens, J., and Jonker, Catholijn, M. (2020). Model-based
reinforcement learning: A survey. arXiv:2006.16712. https://arxiv.org/abs/2006.
16712.
Mohamed, A., Lee, H.-y., Borgholt, L., Havtorn, J. D., Edin, J., Igel, C., Kirchhoﬀ, K.,
Li, S.-W., Livescu, K., Maaløe, L., et al. (2022). Self-supervised speech representation
learning: A review. arXiv preprint arXiv:2205.10643 .
Morari, M. and Lee, J. H. (1997). Model predictive control: Past, present and future.
Computers and Chemical Engineering , 23:667–682.
Murphy, G. L. (2002). The Big Book of Concepts . MIT Press, Cambridge, MA.
Nagabandi, A., Kahn, G., Fearing, R. S., and Levine, S. (2017). Neural network dynam-
ics for model-based deep reinforcement learning with model-free ﬁne-tuning. CoRR,
abs/1708.02596.
Narendra, K. and Parthasarathy, K. (1990). Identiﬁcation and control of dynamical systems
using neural networks. IEEE Transactions on neural networks , 1(1):4–27.
Oh, J., Guo, X., Lee, H., Lewis, R. L., and Singh, S. (2015). Action-conditional video pre-
diction using deep networks in atari games. Advances in neural information processing
systems, 28.
Olshausen, B. A. and Field, D. J. (1996). Emergence of simple-cell receptive ﬁeld properties
by learning a sparse code for natural images. Nature, 381(6583):607–609.
Orhan, E., Gupta, V., and Lake, B. M. (2020). Self-supervised learning through the eyes
of a child. Advances in Neural Information Processing Systems , 33:9960–9971.
Pari, J., Shaﬁullah, N. M., Arunachalam, S. P., and Pinto, L. (2021). The surprising
eﬀectiveness of representation learning for visual imitation. In Robotics Science and
Systems 2022. arXiv preprint arXiv:2112.01511.
Richalet, J., Rault, A., Testud, J. L., and Papon, J. (1978). Model predictive heuristic
control: Applications to industrial processes. Automatica, 14(5):413–428.
Riochet, R., Castro, M. Y., Bernard, M., Lerer, A., Fergus, R., Izard, V., and Dupoux, E.
(2019). Intphys: A benchmark for visual intuitive physics reasoning. arXiv:1803.07616.
Silver, D., Singh, S., Precup, D., and Sutton, R. S. (2021). Reward is enough. Artiﬁcial
Intelligence, 299:103535.
Sobal, V., Canziani, A., Carion, N., Cho, K., and LeCun, Y. (2022). Separating the world
and ego models for self-driving. arXiv:2204.07184.
Spelke, E. S. and Kinzler, K. D. (2007). Core knowledge. Developmental Science, 10:89–96.
Srinivas, A., Jabri, A., Abbeel, P., Levine, S., and Finn, C. (2018). Universal planning
networks. CoRR, abs/1804.00645.
54

Srivastava, N., Mansimov, E., and Salakhudinov, R. (2015). Unsupervised learning of video
representations using lstms. In Bach, F. and Blei, D., editors, Proceedings of the 32nd
International Conference on Machine Learning , volume 37 of Proceedings of Machine
Learning Research, pages 843–852, Lille, France. PMLR.
Sukhbaatar, S., Weston, J., Fergus, R., et al. (2015). End-to-end memory networks. Ad-
vances in neural information processing systems , 28.
Sutton, R. S. (1991). Dyna, an integrated architecture for learning, planning, and reacting.
ACM Sigart Bulletin , 2(4):160–163.
Taigman, Y., Yang, M., Ranzato, M., and Wolf, L. (2014). Deepface: Closing the gap to
human-level performance in face veriﬁcation. In Proceedings of the IEEE conference
on computer vision and pattern recognition , pages 1701–1708.
van den Oord, A., Li, Y., and Vinyals, O. (2018). Representation learning with contrastive
predictive coding. arXiv preprint arXiv:1807.03748 .
van den Oord, A., Vinyals, O., and Kavukcuoglu, K. (2017). Neural discrete representation
learning. In Guyon, I., Luxburg, U. V., Bengio, S., Wallach, H., Fergus, R., Vish-
wanathan, S., and Garnett, R., editors, Advances in Neural Information Processing
Systems, volume 30.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser,  L.,
and Polosukhin, I. (2017). Attention is all you need. Advances in neural information
processing systems, 30.
Vincent, P., Larochelle, H., Lajoie, I., Bengio, Y., Manzagol, P.-A., and Bottou, L. (2010).
Stacked denoising autoencoders: Learning useful representations in a deep network
with a local denoising criterion. Journal of machine learning research , 11(12).
Walker, J., Razavi, A., and Oord, A. v. d. (2021). Predicting video with vqvae. arXiv
preprint arXiv:2103.01950.
Wayne, G. and Abbott, L. (2014). Hierarchical control using networks trained with higher-
level forward models. Neural Computation, 26(10):2163–2193.
Wiskott, L. and Sejnowski, T. J. (2002). Slow feature analysis: Unsupervised learning of
invariances. Neural computation, 14(4):715–770.
Yarats, D., Kostrikov, I., and Fergus, R. (2021). Image augmentation is all you need:
Regularizing deep reinforcement learning from pixels. In ICLR.
Yu, T., Thomas, G., Yu, L., Ermon, S., Zou, J., Levine, S., Finn, C., and Ma, T. (2020).
Mopo: Model-based oﬄine policy optimization. arXiv preprint arXiv:2005.13239 .
Zaadnoordijk, L., Besold, T., and Cusack, R. (2022). Lessons from infant learning for
unsupervised machine learning. Nature Machine Intelligence , 4:510–520.
55

Zbontar, J., Jing, L., Misra, I., LeCun, Y., and Deny, S. (2021). Barlow twins: Self-
supervised learning via redundancy reduction. In International Conference on Machine
Learning, pages 12310–12320. PMLR.
56

Observed variable
Unobserved (latent) variables Deterministic function
input output
Energy term
 
  
Non-deterministic function
Figure 18: Symbols used in architectural diagrams.
Architectural diagrams use symbols commonly used to draw factor graphs – circles for variables,
rectangles for factors – plus rounded rectangles to represent deterministic functions. Filled circles
represent observed variables, or variables that are outputs of deterministic functions.
Hollow circles represent latent variables, i.e. variables that must be inferred by minimizing some
cost, or sampled from a distribution.
Red rectangles represent energy terms. These modules have an implicit scalar output that contributes
additively to the total energy of the system. This is similar to the convention used for factor graphs.
Rounded rectangles represent deterministic functions, which may have one or several inputs. Given
a set of inputs, the output is assumed to be easily computable and unique. The function is generally
assumed to be diﬀerentiable. It may contain trainable parameters.
Non-deterministic functions are represented as shown on the right. They are composed of a deter-
ministic function g(x,z ) in which one of the inputs is a latent variabl z. The latent variable is seen
as varying within a level set of a regularizing energy term R(z). When z varies in the level set
Zh ={z|R(z)<h}, the output ˜y will vary over the set Yh ={y|y =g(x,z ),∀z∈Zh}.
In some cases, the energy term can be transformed into a probability distribution (see text).
Appendix: Symbols and Notations
Architectural diagrams in this paper use the symbols shown in Figure 18.
We use symbols that are somewhat similar to the representation of factor graphs: circles
for variables, rectangles for factors. There are two major diﬀerences, however. First, the
factors represent additive energy terms, not multiplicative probability factors. Second, we
use an additional symbol, rounded rectangles, to represent deterministic functions with a
clear directionality from inputs to outputs.
More precisely:
• Filled circles represent observed variables, or variables that are outputs of determin-
istic functions.
• Hollow circles represent latent variables, i.e. variables that must be inferred by mini-
mizing some cost, varied over a set, or sampled from a distribution.
• Red rectangles represent energy terms. These modules have an implicit scalar output
that contributes additively to the total energy of the system.
• Rounded rectangles represent deterministic functions, which may have one or sev-
eral inputs. Given a set of inputs, the output is assumed to be easily computable
and unique. The function is generally assumed to be diﬀerentiable. It may contain
trainable parameters. Such a module is typically implemented as a deep neural net.
57