A Path Towards Autonomous Machine Intelligence (LeCun, 2022) — part 9/12

Cleaned-text code points [111715, 125279). Bibliography omitted here; source.md includes it.

proposed to use temporal invariance (or consistency) to separate the content of an image
region from its instantiation parameters (Wiskott and Sejnowski, 2002; Gregor and LeCun,
2010a; Goroshin et al., 2015b).
At least one recent work has applied non-contrastive SSL methods to a joint embedding
architecture for robotics control with some success (Pari et al., 2021; ?).
Contrastive methods applied to joint-embedding and prediction have been applied suc-
cessfully to speech recognition (Baevski et al., 2020) (see (Mohamed et al., 2022) for a recent
review of SSL to speech).
To perform state trajectory predictions, recent works have advocated the use of trans-
formers, as proposed in the present paper. Transformers are ideal to represent the dynamics
of discrete objects in interaction, and have successfully been applied to the prediction of
car trajectories (Mercat et al., 2020).
An interesting proposal is the trajectory transformer architecture in which a transformer
is fed with the sequence of predicted states over an entire episode (Janner et al., 2021). The
pattern of attention can be constrained so as to force the system to only attend to the
past so it can be operated in a causal manner (without looking at the future), and trained
to predict the next state, actions, and cost from previously observed or predicted states,
actions, and costs.
Hierarchical planning is a largely unsolved problem. Wayne and Abbott proposed an
architecture that uses a stack of trained forward models that specify intermediate goals for
the lower layers (Wayne and Abbott, 2014). Some recent works specify intermediate goals
for robots in terms of pose parameters (Gehring et al., 2021). A more recent proposal is
the Director system (Hafner et al., 2022) which contains a hierarchical world model and
planning architecture trained end-to-end through reinforcement learning.
The idea of intrinsic motivation to train an agent has been studied in the context of
robotics (Gottlieb et al., 2013). The presence of an Intrinsic Cost provides a differentiable
and efficient way to direct the agent to follow certain behaviors and to learn certain skills.
7.2 Energy-Based Models and Joint-Embedding Architectures
For many authors, Energy-Based Model (EBM) designates a probabilistic model whose
distribution is the normalized negative exponential of an energy function.
In this paper, EBM designates a much broader category of models that treat the energy
function as fundamental, and directly manipulate its landscape through learning. Many
methods have been proposed in the past that directly manipulate the energy. In fact, all
traditional optimization-based learning methods can be interpreted as energy-based meth-
ods (LeCun et al., 2006). In particular, discriminative training methods for structure pre-
diction problems can be formulated as EBM (LeCun et al., 1998; LeCun et al., 2006).
Most EBM approaches for unsupervised or self-supervised learning have been con-
trastive. The earliest example is the Boltzmann Machine (Hinton and Sejnowski, 1983),
which is a probabilistic generative energy-based model trained contrastively.
Joint Embedding Architectures (JEA) trained with contrastive methods and mutual
information maximization methods have a long history. The first non-contrastive JEA
was (Becker and Hinton, 1992) which was based on maximizing a measure of mutual in-
formation between the representations from two branches seeing to different views of the
41

same scene. Perhaps the first contrastive method for JEA is the so-called “Siamese Net-
work” (Bromley et al., 1994). This was trained contrastively for the purpose of verifying
signatures handwritten on a pen tablet.
The idea of JEA remained largely untouched for over a decade, until it was revived in
a series of papers from my group (Chopra et al., 2005; Hadsell et al., 2006), and Geoffrey
Hinton’s group (Goldberger et al., 2005). Following the rebirth of deep learning, a few
papers used JEA for fine-grained recognition, including face recognition (Taigman et al.,
2014).
With the emergence of SSL approaches, the use of JEA trained contrastively has ex-
ploded in the last few years with methods such as PIRL (Misra and Maaten, 2020), MoCo
and MoCo-v2 (He et al., 2020; Chen et al., 2020b), and SimCLR (Chen et al., 2020a).
Some methods can be seen as “distillation” approaches in which one branch of the
Siamese network is a teacher whose output are used as targets for the other branch. This
includes methods in which the output vectors are quantized to discrete cluster prototypes
(see (Caron et al., 2020) and predecessors).
In recent years, a number of new non-contrastive methods have appeared, such as
BYOL (Grill et al., 2020). But the class of non-contrastive methods advocated in the
present proposal prevent collapse by maximizing the information content of the embed-
dings. This includes Barlow Twins (Zbontar et al., 2021), VICReg (Bardes et al., 2021),
whitening-based methods (Ermolov et al., 2021), and Maximum Coding Rate Reduction
methods (see (Dai et al., 2022) and references therein).
7.3 Human and animal cognition
The limitations of current approaches to machine learning when compared with human
learning are obvious (Lake et al., 2017a; Zaadnoordijk et al., 2022).
Young children quickly learn abstract concepts (Murphy, 2002), and models that allow
them to navigate, to form goals, and to plan complex action sequences to fulfill them (Gopnik
and Meltzoff, 1997; Spelke and Kinzler, 2007; Carey, 2009; Gopnik et al., 2001).
In cognitive science, the idea that the brain builds predictive world models is a common
one, and have inspired attempt to reproduce the process in machines (Lake et al., 2017b;
Orhan et al., 2020). Some efforts have been devoted to building video datasets to test
intuitive physics common sense in machines and infants (Riochet et al., 2019).
The ability to plan is a well-studied feature of human intelligence (Mattar and Lengyel,
2022). There is evidence that people construct simplified representations of the world for
planning in which irrelevant details are abstracted away (Ho et al., 2022)
Consciousness is a rather speculative topic, owing to the difficulty of defining what con-
sciousness is. I will not speculate about whether some version of the proposed architecture
could possess a property assimilable to consciousness, but will only mention the work of
Dehaene and collaborators who have proposed two types of consciousness that they call
C1 and C2. C1 is largely related with the modulation of attention, while C2 requires a
self-monitoring ability, perhaps assimilable to what the configurator module needs to do in
the present proposal (Dehaene et al., 2021).
42

8 Discussion, Limitations, Broader Relevance
Constructing the cognitive architecture of the present proposal, instantiating all the details,
and making the system work for non-trivial tasks will not be an easy task. The path to
success is likely riddled with unforeseen obstacles. It will probably take many years to work
them all out.
8.1 What is missing from the Proposed Model?
A lot of hard work needs to be done to instantiate the proposed architecture and turn it
into a functional system. There may be flaws and pitfalls that may appear to be unsolvable
within the specifications of the proposed architecture.
The first question is whether a Hierarchical JEPA can be built and trained from videos.
Could it learn the type of abstract concept hierarchy mentioned in section 4.1?
One somewhat open question relative to the JEPA is how precisely to regularize the
latent variable so as to minimize its information content. A number of possible mechanisms
are proposed: making the latent variable discrete, low-dimensional, sparse, or stochastic.
But it is not clear which approach will ultimately be the best.
The current proposal does not prescribe a particular way for the actor to infer latent
variable instantiations and optimal action sequences. While the differentiability of all the
modules makes it possible in principle to use gradient-based optimization to infer optimal
action sequences, the optimization problem may be very difficult in practice. In particular,
when the action space is discrete, or when the function from actions to cost is highly non
smooth, gradient-based method may be ineffective, requiring to use other (gradient-free)
search methods (dynamic programming, belief propagation, MCTS, SAT, etc).
Instantiating multiple configurations of latent variables in Mode-2 planning/reasoning
may require additional mechanisms not described in the present proposal. Humans seem
to be endowed with an ability to spontaneously cycle through alternative interpretations
of a percept, as demonstrated by the Necker cube and other visual illusions that have
several equally-plausible interpretations. In the context of the present model, different
interpretation of an ambiguous percept may be represented by different values of a latent
variable. While one could imagine a number of exploratory mechanisms to systematically
explore the space of possible latent variable values, no such mechanism is described here.
The present proposal does not specify the details of the architecture of the various
modules. For example, it is probable that the predictor will require some sort of dynamic
routing and gating circuits in its micro-architecture. Predictors for low-level representation
may have to be specialized to represent the kind of small transformations of the repre-
sentation that can occur in the short term. Predictor modules dealing with higher level
representations may require more generic architectures that manipulate objects and their
relationships. But none of this is specified in the present proposal.
Similarly, the precise architecture and function of the short-term memory and how it
may be used to represent beliefs about the state of the world are somewhat fuzzy. The
original Memory Network system and its successors contained the idea that a neural net
could use an associative memory as a working memory to store and retrieve beliefs about
the state of the world between compute cycles (Bordes et al., 2015; Sukhbaatar et al.,
43

2015). But getting such an architecture to work for complex planning and control may
prove difficult.
Of all the least understood aspects of the current proposal, the configurator module is the
most mysterious. In particular, while planning a complex task, the configurator is supposed
to identify sequences of subgoals and configure the agent to successively accomplish those
subgoals. Precisely how to do that is not specified.
This is merely a list of foreseeable questions, but many questions and problems will
inevitably surface as instances of the proposed systems are put together.
8.2 Broader Relevance of the Proposed Approach
Although the proposed architecture is not specifically designed to model autonomous intel-
ligence, reasoning, and learning in humans and other animals, one can draw some parallels.
The following is somewhat speculative and provided as a way to connect some concepts
in cognitive science and neuroscience that have inspired the present work.
8.2.1 Could this Architecture be the Basis of a Model of Animal Intelligence?
Many of the modules in the proposed architecture have counterparts in the mammalian
brain that perform similar functions.
The perception module corresponds to the visual, auditory, and other sensory areas
of the cortex, as well as some of the association areas. The world model and the critic
correspond to various part of the prefrontal cortex. The intrinsic cost module corresponds
to structures in the basal ganglia involved in rewards, including the amygdala. The trainable
critic may correspond to part of the prefrontal cortex involved in reward prediction. The
function of the short-term memory overlaps with what is known of the hippocampus. The
configurator may correspond to structures in the prefrontal cortex that perform executive
control and modulate attention. The actor regroups areas in the pre-motor cortex that
elaborate and encode motor plans.
The idea of predictive world model has long been a prominent concept in cognitive
science, and the idea of predictive coding has been a prominent concept in neuroscience. The
JEPA architecture and the corresponding non-sample-contrastive self-supervised learning
method are somewhat consistent with ideas of predictive coding and efficient coding.
The proposed architecture has a single world model engine that can be configured for
the task at hand by the configurator. I have argued that this may not only confer a
computational advantage through hardware reuse, but also allow knowledge to be shared
across multiple tasks. The hypothesis of a single, configurable world model engine in the
human brain may explain why humans can essentially perform a single “conscious” reasoning
and planning task at a time. A highly-speculative idea is that the illusion of consciouness
may be a side-effect of a configurator-like module in the brain that oversees the function
of the rest of brain and configures it for the task at hand. Perhaps if the brain were large
enough to contain many independent, non-configurable world models, a configurator would
be unnecessary, and the illusion of consciousness would disappear.
What is the substrate of emotions in animals and humans? Instantaneous emotions (e.g.
pain, pleasure, hunger, etc) may be the result of brain structures that play a role similar
to the Intrinsic Cost module in the proposed architecture. Other emotions such as fear or
44