A Path Towards Autonomous Machine Intelligence (LeCun, 2022) — part 1/12

Cleaned-text code points [0, 13454). Bibliography omitted here; source.md includes it.

A Path Towards Autonomous Machine Intelligence
Version 0.9.2, 2022-06-27
Yann LeCun
Courant Institute of Mathematical Sciences, New York University yann@cs.nyu.edu
Meta - Fundamental AI Research yann@fb.com
June 27, 2022
Abstract
How could machines learn as efficiently as humans and animals? How could ma-
chines learn to reason and plan? How could machines learn representations of percepts
and action plans at multiple levels of abstraction, enabling them to reason, predict,
and plan at multiple time horizons? This position paper proposes an architecture and
training paradigms with which to construct autonomous intelligent agents. It combines
concepts such as configurable predictive world model, behavior driven through intrinsic
motivation, and hierarchical joint embedding architectures trained with self-supervised
learning.
Keywords: Artificial Intelligence, Machine Common Sense, Cognitive Architecture, Deep
Learning, Self-Supervised Learning, Energy-Based Model, World Models, Joint Embedding
Architecture, Intrinsic Motivation.
1 Prologue
This document is not a technical nor scholarly paper in the traditional sense, but a position
paper expressing my vision for a path towards intelligent machines that learn more like
animals and humans, that can reason and plan, and whose behavior is driven by intrinsic
objectives, rather than by hard-wired programs, external supervision, or external rewards.
Many ideas described in this paper (almost all of them) have been formulated by many
authors in various contexts in various form. The present piece does not claim priority for
any of them but presents a proposal for how to assemble them into a consistent whole. In
particular, the piece pinpoints the challenges ahead. It also lists a number of avenues that
are likely or unlikely to succeed.
The text is written with as little jargon as possible, and using as little mathematical
prior knowledge as possible, so as to appeal to readers with a wide variety of backgrounds
including neuroscience, cognitive science, and philosophy, in addition to machine learning,
robotics, and other fields of engineering. I hope that this piece will help contextualize some
of the research in AI whose relevance is sometimes difficult to see.
1

2 Introduction
Animals and humans exhibit learning abilities and understandings of the world that are far
beyond the capabilities of current AI and machine learning (ML) systems.
How is it possible for an adolescent to learn to drive a car in about 20 hours of practice
and for children to learn language with what amounts to a small exposure. How is it that
most humans will know how to act in many situation they have never encountered? By
contrast, to be reliable, current ML systems need to be trained with very large numbers of
trials so that even the rarest combination of situations will be encountered frequently during
training. Still, our best ML systems are still very far from matching human reliability in
real-world tasks such as driving, even after being fed with enormous amounts of supervisory
data from human experts, after going through millions of reinforcement learning trials in
virtual environments, and after engineers have hardwired hundreds of behaviors into them.
The answer may lie in the ability of humans and many animals to learn world models,
internal models of how the world works.
There are three main challenges that AI research must address today:
1. How can machines learn to represent the world, learn to predict, and learn to act
largely by observation?
Interactions in the real world are expensive and dangerous, intelligent agents should
learn as much as they can about the world without interaction (by observation) so
as to minimize the number of expensive and dangerous trials necessary to learn a
particular task.
2. How can machine reason and plan in ways that are compatible with gradient-based
learning?
Our best approaches to learning rely on estimating and using the gradient of a loss,
which can only be performed with differentiable architectures and is difficult to rec-
oncile with logic-based symbolic reasoning.
3. How can machines learn to represent percepts and action plans in a hierarchical man-
ner, at multiple levels of abstraction, and multiple time scales?
Humans and many animals are able to conceive multilevel abstractions with which
long-term predictions and long-term planning can be performed by decomposing com-
plex actions into sequences of lower-level ones.
The present piece proposes an architecture for intelligent agents with possible solutions to
all three challenges.
The main contributions of this paper are the following:
1. an overall cognitive architecture in which all modules are differentiable and many of
them are trainable (Section 3, Figure 2).
2. JEPA and Hierarchical JEPA: a non-generative architecture for predictive world mod-
els that learn a hierarchy of representations (Sections 4.4 and 4.6, Figures 12 and 15).
3. a non-contrastive self-supervised learning paradigm that produces representations that
are simultaneously informative and predictable (Section 4.5, Figure 13).
2

4. A way to use H-JEPA as the basis of predictive world models for hierarchical planning
under uncertainty (section 4.7, Figure 16 and 17).
Impatient readers may prefer to jump directly to the aforementioned sections and figures.
2.1 Learning World Models
Human and non-human animals seem able to learn enormous amounts of background knowl-
edge about how the world works through observation and through an incomprehensibly
small amount of interactions in a task-independent, unsupervised way. It can be hypoth-
esized that this accumulated knowledge may constitute the basis for what is often called
common sense . Common sense can be seen as a collection of models of the world that
can tell an agent what is likely, what is plausible, and what is impossible. Using such
world models, animals can learn new skills with very few trials. They can predict the con-
sequences of their actions, they can reason, plan, explore, and imagine new solutions to
problems. Importantly, they can also avoid making dangerous mistakes when facing an
unknown situation.
The idea that humans, animals, and intelligent systems use world models goes back a
long time in psychology (Craik, 1943). The use of forward models that predict the next
state of the world as a function of the current state and the action being considered has been
standard procedure in optimal control since the 1950s (Bryson and Ho, 1969) and bears
the name model-predictive control. The use of differentiable world models in reinforcement
learning has long been neglected but is making a comeback (see for example (Levine, 2021))
A self-driving system for cars may require thousands of trials of reinforcement learning
to learn that driving too fast in a turn will result in a bad outcome, and to learn to slow
down to avoid skidding. By contrast, humans can draw on their intimate knowledge of
intuitive physics to predict such outcomes, and largely avoid fatal courses of action when
learning a new skill.
Common sense knowledge does not just allow animals to predict future outcomes, but
also to fill in missing information, whether temporally or spatially. It allows them to produce
interpretations of percepts that are consistent with common sense. When faced with an
ambiguous percept, common sense allows animals to dismiss interpretations that are not
consistent with their internal world model, and to pay special attention as it may indicate
a dangerous situation and an opportunity for learning a refined world model.
I submit that devising learning paradigms and architectures that would allow machines
to learn world models in an unsupervised (or self-supervised) fashion, and to use those
models to predict, to reason, and to plan is one of the main challenges of AI and ML today.
One major technical hurdle is how to devise trainable world models that can deal with
complex uncertainty in the predictions.
2.2 Humans and Animals learn Hierarchies of Models
Humans and non-human animals learn basic knowledge about how the world works in the
first days, weeks, and months of life. Although enormous quantities of such knowledge are
acquired quite quickly, the knowledge seems so basic that we take it for granted. In the
first few months of life, we learn that the world is three-dimensional. We learn that every
3

Perception Production
Physics
Actions
Objects
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
Age (months)
stability,
support
gravity, iner�a
conserva�on of
momentum
Object permanence
solidity, rigidity
shape
constancy
crawling walking
emo�onal contagion
ra�onal, goal-
directed ac�ons
face tracking
proto-imita�on
poin�ng
biological
mo�on
false perceptual
beliefs
helping vs
hindering
natural kind categories
Social
Communication
Figure 1: This chart, (courtesy of Emmanuel Dupoux), indicates at what age infants generally
acquire various concepts about how the world works. It is consistent with the idea that abstract
concepts, such as the fact that objects are subject to gravity and inertia, are acquired on top of less
abstract concepts, like object permanence and the assignment of objects to broad categories. Much
of this knowledge is acquired mostly by observation, with very little direct intervention, particularly
in the first few weeks and months.
source of light, sound, and touch in the world has a distance from us. The fact that every
point in a visual percept has a distance is the best way to explain how our view of the
world changes from our left eye to our right eye, or when our head is being moved. Parallax
motion makes depth obvious, which in turn makes the notion of object obvious, as well
as the fact that objects can occlude more distant ones. Once the existence of objects is
established, they can be automatically assigned to broad categories as a function of their
appearance or behavior. On top of the notion of object comes the knowledge that objects
do not spontaneously appear, disappear, change shape, or teleport: they move smoothly
and can only be in one place at any one time. Once such concepts are acquired, it becomes
easy to learn that some objects are static, some have predictable trajectories (inanimate
objects), some behave in somewhat unpredictable ways (collective phenomena like water,
sand, tree leaves in the wind, etc), and some seem to obey different rules (animate objects).
Notions of intuitive physics such as stability, gravity, inertia, and others can emerge on top
of that. The effect of animate objects on the world (including the effects of the subject’s
own actions) can be used to deduce cause-and-effect relationships, on top of which linguistic
and social knowledge can be acquired.
Figure 1, courtesy of Emmanuel Dupoux, shows at what age infants seem to acquire
basic concepts such as object permanence, basic categories, intuitive physics, etc. Concepts
at higher levels of abstraction seem to develop on top of lower-level ones.
Equipped with this knowledge of the world, combined with simple hard-wired behav-
iors and intrinsic motivations/objectives, animals can quickly learn new tasks, predict the
4

consequences of their actions and plan ahead, foreseeing successful courses of actions and
avoiding dangerous situations.
But can a human or animal brain contain all the world models that are necessary for
survival? One hypothesis in this paper is that animals and humans have only one world
model engine somewhere in their prefrontal cortex. That world model engine is dynamically
configurable for the task at hand. With a single, configurable world model engine, rather
than a separate model for every situation, knowledge about how the world works may
be shared across tasks. This may enable reasoning by analogy, by applying the model
configured for one situation to another situation.
To make things concrete, I will directly dive into a description of the proposed model.
5

percept
action
Actor
World Model
Intrinsic
cost
Perception
Short-term
memory
configurator
Critic
Cost
Figure 2: A system architecture for autonomous intelligence. All modules in this model are as-
sumed to be “differentiable”, in that a module feeding into another one (through an arrow connecting
them) can get gradient estimates of the cost’s scalar output with respect to its own output.
The configurator module takes inputs (not represented for clarity) from all other modules and
configures them to perform the task at hand.
The perception module estimates the current state of the world.
The world model module predicts possible future world states as a function of imagined actions
sequences proposed by the actor.
The cost module computes a single scalar output called “energy” that measures the level of dis-
comfort of the agent. It is composed of two sub-modules, the intrinsic cost, which is immutable (not
trainable) and computes the immediate energy of the current state (pain, pleasure, hunger, etc), and
the critic, a trainable module that predicts future values of the intrinsic cost.
The short-term memory module keeps track of the current and predicted world states and as-
sociated intrinsic costs.
The actor module computes proposals for action sequences. The world model and the critic com-
pute the possible resulting outcomes. The actor can find an optimal action sequence that minimizes
the estimated future cost, and output the first action in the optimal sequence.
See Section 3 for details.
6