Position: LLMs can't jump — part 2/4

Cleaned-text code points [13835, 29512). Bibliography omitted here; source.md includes it.

LLMs can’t jump
Mathematical validation (1913-1915).The years 1913
to 1915 were defined by a grueling struggle to correct and
perfect the 1913 draft. With the publication of the ’Entwurf’
paper in mid-1913, Einstein initially believed the heavy
lifting was done and only details remained. This feeling
was short-lived. As months turned into years, he found
himself working harder and harder to justify a theory that
was misshapen.
By the summer of 1915, the evidence against his old theory
was mounting. He knew it failed to explain the anomalous
orbit of Mercury. He discovered it could not account for
rotational motion. Finally, he realized that his sophisticated
attempts to prove the theory’s uniqueness in late 1914 were
flawed. In a state of mounting desperation, Einstein aban-
doned the ’adapted’ coordinate systems of the ’Entwurf’
and returned to his earlier intuition from 1912: the theory
needed to work in all coordinate systems.
What ensued was perhaps the most intense month of Ein-
stein’s career. Spurred by the knowledge that the renowned
mathematician David Hilbert was racing to solve the same
problem, Einstein entered a frenzy of productivity, submit-
ting a new paper to the Prussian Academy every week for
four consecutive weeks. His first communication on Novem-
ber 4 proposed a solution, yet errors persisted. By November
11, he had refined the theory but difficulties remained; how-
ever, on November 18, he announced the thrilling result that
his evolving equations correctly predicted the anomalous
orbit of Mercury. Finally, on November 25, the fourth com-
munication unveiled the equations of General Relativity:
Rik− 1
2gikR=−κT ik.(2)
Here,Rik is the Ricci curvature tensor,R is the Ricci scalar,
Tik is the Stress-Energy tensor, and gik is the metric ten-
sor. The expression on the left represents the geometry of
spacetime (curvature) as determined by the metric, while
the expression on the right represents the matter and energy.
3. Alternative Views
3.1. The Limits of Inductive Inference
”One not infrequently hears the viewpoint ex-
pressed that physicists are merely noticing pat-
terns... It seems to me, however, that such a view-
point is extraordinarily wide of its mark... When
Einstein’s theory was first put forward, there was
really no need for it on observational grounds.
...Einstein was not just ‘noticing patterns’ in the
behavior of physical objects. He was uncovering
profound mathematical structure that was already
hidden in the very working of the world.”
– Roger Penrose
This distinction between noticing patterns and uncovering
structure highlights the boundary between AI as it exists
today and the AI required for scientific invention. The pre-
vailing view in machine learning aligns with the ”Theory of
Compression Progress,” (Schmidhuber, 2008) which posits
that scientific discovery is driven by the inductive desire to
compress data. In this framework, the ”joy” of discovery is
the rate at which complex observations become subjectively
simpler through better prediction. This inductive approach
has yielded impressive results in data rich environments:
sparse optimization has successfully extracted partial differ-
ential equations from data (Schaeffer, 2017), and the ”AI
Physicist” (Wu & Tegmark, 2019) successfully rediscovered
conservation laws from simulated trajectories.
However, we argue that this inductive framework is insuffi-
cient to explain the invention of General Relativity. While
Einstein sought logical simplicity, his process was not driven
by data compression—primarily because there was no sta-
tistically significant supervised training set to compress.
At the time of invention, Newtonian gravity faced no em-
pirical crisis. The equivalence of inertial and gravitational
mass had been verified to a precision of10−9, and Newton’s
laws were accurate to an astonishingly small margin of er-
ror. The only known anomaly—the advance of Mercury’s
perihelion—was viewed not as a failure, but as evidence of
a hidden variable: the undiscovered planet ”Vulcan”.
This highlights the fundamental limitation of ”creativity as
compression”: scientific invention often occurs in the ab-
sence of a supervised error signal. An AI operating as an
inductive optimization engine would have found the New-
tonian loss function to be near-zero. Without a significant
discrepancy between prediction and observation, there is no
gradient to drive the system toward a foundational restruc-
turing of spacetime.
If modern Transformer models struggle to reverse-engineer
basic arithmetic rules (Gambardella et al., 2024; Yang et al.,
2024), it is difficult to see how it could invent a new physics
in the absence of massive datasets. Furthermore, even
when data is available, inductive systems risk converging
on heuristic shortcuts rather than causal laws. Vafa et al.
(2025) demonstrate that without the correct inductive bias,
foundation models often discover flawed world models that
satisfy the data but fail to capture the underlying structure.
The empirical evidence required to validate General Rela-
tivity—from the Eddington experiment to relativistic GPS
corrections—arrived after the theory was formulated. Ein-
stein was not compressing a noisy dataset to fit a regression
curve; he was constructing a logical framework to uncover
a physical structure that the data had not yet revealed.
Lastly, it could be argued that while Einstein was not com-
pressing data, he was compressing the hypothesis space by
4

LLMs can’t jump
seeking to unify the laws of inertia and gravity into a sin-
gle framework (Minimum Description Length). However,
logical simplicity is often a retrospective property. While
the final theory of General Relativity is elegant, the search
path to get there was paved with complexity, abandoned
tensors, and incorrect equations. A compression-driven AI
might prefer to patch Newtonian gravity with a parameter
like the ’Vulcan’ planet hypothesis rather than expanding
the hypothesis space to include non-Euclidean geometry,
which increases complexity before it simplifies it.
3.2. The Limits of Deduction
”I see on one side the totality of sense experiences,
and on the other, the totality of the concepts and
propositions that are laid down in books. The re-
lations between concepts and propositions among
themselves are of a logical nature... The concepts
and propositions get ’meaning’, or ’content’, only
through their connection with sense experiences.”
– Albert Einstein
Einstein explicitly distinguished between the domain of
sensory experience and the domain of logical processing. In
our framework, this latter domain corresponds to Deduction
(A→S ): the rigorous derivation of theorems from a fixed
set of axioms.
Even the motivation to begin the search for General Relativ-
ity contained a strong deductive component. Einstein’s drive
was not sparked by data anomalies. There was no ”error
signal” in the Newtonian observation history, but by a con-
ceptual inconsistency: the clash between mechanical action
at a distance and the emerging field theories of electromag-
netism. While modern LLMs will struggle to find such an
idea due to the ”weak signal” (there was no requirement to
replace Newton’s gravity), the structural task of proposing a
field theory for gravity by mimicking Maxwell’s equations
is fundamentally a deduction operation.
It is plausible that a modern AI, optimized to search for
inconsistencies in scientific literature, could identify this
contradiction. Much like a system identifying ”buggy code,”
an AI could flag that the constant speed of light in Maxwell’s
equations is incompatible with Newtonian absolute time.
However, identifying the error is distinct from generating
the fix. While the structural task of proposing a field theory
for gravity is a deductive operation, selecting the correct
axioms to resolve the conflict requires more than logical
consistency.
The period between 1913 and 1915 illustrates this deduc-
tive struggle. It was defined not by flashes of insight, but
by a grueling, mechanical search to identify the correct
mathematical framework to satisfy Einstein’s postulates.
This phase closely mirrors the capabilities of modern neuro-
symbolic AI. Einstein’s collaboration with Marcel Gross-
mann was essentially a ”search” process over geometric
constraints. Notably, they identified the Riemann curvature
tensor as the correct object but discarded it due to a ”fatal er-
ror”—the mistaken belief that it did not reduce to Newtonian
gravity in static fields. It took two years of exhaustion to
debug this assumption and produce the final field equations.
The landscape of mathematical discovery has been recently
transformed by automation. Proof assistants based on de-
pendent type theory, such as Lean, have matured into robust
platforms supported by extensive libraries (mathlib Commu-
nity, 2019). In 2024 LLMs have achieved remarkable flu-
ency in proof generation: AlphaProof (Hubert et al., 2025)
achieved silver-medal performance on IMO problems. Suc-
cessors like Gemini, DeepSeekMathV2, and GPT-5 attained
gold-level performance in 2025 and systems like Aristotle
(Achim et al., 2025) produced verified solutions to open
research questions.
”At the age of twelve I experienced a second won-
der of totally different nature - in a little book
dealing with Euclidean plan geometry, ..., were as-
sertions, that could be proved with such certainty
that any doubt appeared to be out of question.
This lucidity and certainty made an indescribable
impression on me.”
– Albert Einstein
Given this trajectory, we posit that a modern LLM, initial-
ized with the specific physical assumptions available to Ein-
stein in 1915, could plausibly derive General Relativity. The
derivation of the perihelion precession of Mercury, once the
field equations are set, is a verifiable logical task (A→S ).
Furthermore, current systems are theoretically capable of
identifying and eliminating erroneous constraints—such as
Einstein’s error regarding static fields—by systematically
optimizing over subsets of axioms.
However, this capability comes with a critical caveat. An AI
can deduce the consequences of ”The Equivalence Principle”
only if those concepts are provided as inputs. As Einstein
noted, logical thinking is limited to connections between
concepts; it cannot generate the concepts themselves from
raw sensory data. The 1913 derivation failed not because
the logic was flawed, but because the axioms were incorrect.
This leads us to the fundamental bottleneck: what cognitive
process allowed Einstein to generate the ”Equivalence Prin-
ciple” in the first place? To understand this, we must look
beyond logic to the mechanism of the ”Jump” (J).
Finally, even if an AI possesses the deductive capacity to
derive Einstein’s equations from his postulates, a funda-
mental problem of intent remains. Unlike formal theorem
5

LLMs can’t jump
proving, where the goal is a specific open conjecture, Ein-
stein was not trying to prove a theorem but to construct
a predictive model of reality. While the anomalous peri-
helion of Mercury offered a verification target, it was not
considered important enough. Crucially, the definitive val-
idation—measuring the gravitational bending of starlight
passing near the Sun by Eddington—arrived years after the
theory was formulated. Deduction ( A→S ) is strictly a
downstream process: it unfolds the logical consequences of
a theory, but it lacks both the upstream capacity to generate
axioms and the external grounding to validate them.
4. Abduction: The missing Jump
”Then there occurred to me the happiest thought
of my life... for an observer falling freely from
the roof of a house there exists—at least in his
immediate surroundings—no gravitational field...
The observer therefore has the right to interpret
his state as ’at rest.’ Because of this idea, the un-
commonly peculiar experimental law that in the
gravitational field all bodies fall with the same ac-
celeration attained at once a deep physical mean-
ing.”
– Albert Einstein
How does the mind formulate new axioms in the absence
of sufficient data? Einstein’s ’happiest thought’ provides
the answer: Manipulative Abduction (Magnani et al., 2009).
This process relies on embodied simulation—an active inter-
action with mental models to generate hypotheses through
thinking by doing, thereby accessing knowledge beyond the
reach of pure deduction. Einstein did not bridge Special
Relativity and gravitation by gathering observations, but
by simulating the physical feelings of an observer inside a
sealed environment.
To make this concept of manipulative abduction more
accessible, we can look to a simpler, classical example:
Archimedes’ discovery of his buoyancy principle. Tasked
Figure 2.Einstein’s thought experiment of the equivalence princi-
ple (AI generated).
with determining if a crown was made of pure gold with-
out damaging it, Archimedes made his abductive leap not
through mathematical derivation, but while stepping into
a bath. By experiencing the bodily sensation of water ris-
ing and being displaced by his own volume, he performed
an embodied sensory simulation. This raw physical feed-
back allowed him to bridge a sensory experience directly
to a new axiomatic concept—volume displacement—which
could not be deduced from the existing mathematical or
linguistic theories of his era. Just as Archimedes used the
physical sensation of displacement to formulate a new law,
Einstein used the simulated sensation of gravity.
We conceptualize this thought experiment as a two-stage
process. First, an observation is imagined viasimulation.
Second, an explanation is derived for that observation via
abductivereasoning. Modern benchmarks like ARC-AGI
(Chollet et al., 2025) already test the latter. In ARC, mod-
els must infer hidden rules from sparse examples (2–5 grid
pairs). Since the data is too sparse for statistical induction
and lacks the explicit instructions required for deduction,
the solver must make an abductive leap to the most plau-
sible explanation. However, as we argue next, while ARC
captures thelogicalleap, it misses themanipulativecompo-
nent—the physical sensation and embodied simulation that
drove Einstein’s insight.
Simulation as Physical Variation.The first pro-
cess—inventing a question to force progress—can be
viewed through the lens of modern AI as Test Time Re-
inforcement Learning. This paradigm involves inventing
new variations of a problem and learning to solve them, a
strategy successfully applied to solve the Penrose position
in chess (Zahavy et al., 2024) and to achieve silver medal
standards in the IMO (Hubert et al., 2025).
However, a critical distinction remains. While symbolic
variations in chess and mathematics are bounded by fixed
rules (axioms), Einstein’s variation required inventing new
axioms based on a physical intuition that did not yet exist
in the mathematics. He envisioned a physicist inside an
elevator being uniformly accelerated through deep space
(Figure 2). Inside this enclosure, the sensory experience re-
veals a specific pattern: when objects are released, the floor
rushes up to meet them. To the physicist, the objects appear
to fall with identical acceleration, regardless of composition.
Thus, the simulation here was not a permutation of symbols,
but a manipulation of perceptual experience.
Abduction and the Physical Prior. The second process is
Abductive Reasoning: the inference to the best explanation.
Unlike deduction, which guarantees truth from premises,
abduction seeks the simplest, most likely cause for an ob-
servation.
In Einstein’s scenario, the existing Newtonian framework
6