Explorative Modeling: Unlocking a Third Pretraining Axis and End-to-End Generation — part 6/8

Cleaned-text code points [77719, 93625). Bibliography omitted here; source.md includes it.

We call a generative model end-to-end when sampling during training, if done at all, is the same as sampling during inference (Section 2.2 ). Contrastive generative models—including GANs [ 20 ] and contrastive-divergence EBMs [ 21 , 22 ] —have long been end-to-end, but have struggled with scaling as well as reconstructive models [ 92 ] , so we focus on reconstructive generative models as the primary target for end-to-end generation. Among reconstructive models, normalizing flows [ 93 ] are commonly seen as end-to-end, as they train an invertible map by exact likelihood and sample by inverting that map in a single pass. However, they fall short of our definition of end-to-end generative models (Section 2.2 ), as training only ever evaluates the data-to-noise direction, so the noise-to-data sampling pass used at inference is never simulated during training. Additionally, normalizing flows have struggled to scale as standalone generators: a continuous bijection cannot split its unimodal base into well-separated modes without leaving thin bridges of density between them [ 94 ] . The flows that generate well today escape this by importing factorization, often stacking autoregressive blocks [ 95 , 96 ] , which only widens this train-inference gap, as training never simulates the sequential recursion used at inference. In contrast, recent reconstructive approaches have progressively narrowed the train/inference gap: consistency models [ 14 ] learn few-step samplers by enforcing self-consistency along the probability-flow ODE trajectory (via distillation or from scratch), and MeanFlow [ 15 ] regresses the flow marginal mean to enable single-step generation. These methods still maintain a mismatch, however, since training primarily samples portions of the trajectory rather than the full generative process used at inference. Mode Forcing explains why this gap has been hard to close: factoring generation into near-unimodal steps is exactly what lets reconstruction losses avoid mode blurring, so existing scalable reconstructive models that factor generation are fundamentally unable to be end-to-end. XMs enable resolving this challenge by factoring the training loop.

### E.2 Coupling

A natural attempt to avoid mode blurring without factorizing generation is to choose a smarter coupling. Optimal Transport [ 82 , 39 ] reduces path crossings by minimizing geometric distance, and minibatch OT [ 97 ] approximates this per batch—but the resulting coupling is a biased proxy for the global one, and its gains narrow at scale [ 98 ] . In our own experiments they reverse entirely (Section A ), as trying to use OT couplings actually results in worse performance, which we attribute to minibatch bias and OT’s exogenous, model-agnostic assignment. Model-Aligned Coupling (MAC) [ 99 ] selects training pairs by model prediction error in addition to geometry, improving few-step flow matching, but is specific to trajectory regularization and not designed to raise generative expressivity or handle multimodal distributions more broadly.

### E.3 Explorative Modeling Based Methods

We do not claim to invent best-of- K K : generating several candidates and keeping the best is a simple idea that has appeared across many works, from winner-take-all training objectives [ 100 , 33 , 101 , 34 , 35 ] to inference-time best-of- N N selection, where candidate generations are reranked using a learned reward model [ 102 ] , a trained verifier [ 103 ] , or the model’s own energy [ 69 ] . Rather, what we claim is the realization of why this idea matters for generative modeling: exploration is a fundamentally different way of handling multimodal distributions from the generation factoring of modern reconstructive generative models (described in Section 2 ), working as a scalable latent variable search that keeps the loss minimizer on individual modes rather than their blurred average (Section 3.1 ). In particular, exploration offers a general way to increase generative expressivity, which existing models otherwise fix at design time through how they factor generation. It is this understanding that lets us use exploration deliberately, as a new scaling axis that raises the generative expressivity of existing generative models, and as a standalone approach for end-to-end generation through Forward and Reverse XM. Through this lens, several existing methods can be seen as realizing a form of exploration, which we describe next.

Multiple Choice Learning [ 100 , 33 ] trains an ensemble of K K predictors under an oracle loss, gating gradients through whichever member has the lowest error on each example, an idea later extended to single networks with multiple prediction heads [ 101 ] . The goal is ensemble diversity for downstream oracle selection, not generative modeling, though it shares the structural idea of backpropagating only through the minimum-loss prediction.

Implicit Maximum Likelihood Estimation (IMLE) [ 34 ] draws a pool of model samples, matches each data point to its nearest sample via fast approximate nearest-neighbor search, and trains on those pairs, with theoretical guarantees that this recovers MLE under mild conditions. Structurally, XMs generalize IMLE: IMLE is a specific instance of end-to-end Forward XM with a shared global sample pool in place of per-step candidates, and both minimize the expected distance from each data point to its nearest model output. Self-Organizing Generative Models (SOG) [ 35 ] realize the same objective conditionally—sampling several latent codes per datapoint and training only through the lowest-loss one, another instance of end-to-end Forward XM—interpreting this procedure as encoder-free hard-assignment maximum likelihood. IMLE attributes its success to performing implicit maximum likelihood—the argument that nearest-sample matching is a likelihood-free proxy for log-likelihood maximization. But we argue this perspective misidentifies the working mechanism—if maximum likelihood were what makes IMLE work, then Reverse XM, which fixes one model output and trains on its nearest data point rather than the reverse, optimizing the mode-seeking reverse KL rather than any likelihood (Section F.2 ), would have no reason to capture modes—yet it does. What the two methods share instead is exploration: drawing multiple candidates so the reconstruction loss minimizer lands on a mode rather than a blurred average. This coincides with the main hypothesis behind Mode Forcing [ 19 ] , that modern reconstructive generative models are not fundamentally about MLE or similar objectives but rather about designing a reconstructive objective where the loss minimizer does not blur modes. Explorative Modeling makes this principle general—it applies to any generative model and loss, not just end-to-end implicit models with Euclidean distance, covers both forward and reverse directions, and can be layered onto existing generative models as a scaling axis.

## Appendix F Additional Theory

### F.1 Generative Expressivity Details

Here we expand on the definition of generative expressivity E E from Section 2.1 , which follows Mode Forcing [ 19 ] . The mode count M ⁡ ( q ) M(q) is the number of strict local maxima of a distribution q q , or for discrete distributions the cardinality of its support, and P θ ( ⋅ ∣ c ) P_{\theta}(\cdot\mid c) is the distribution induced by the model’s inference-time sampling procedure. Generative expressivity is E ≜ sup p ∗ , c sup θ ⋆ ∈ arg ⁡ min θ ⁡ ℒ ⁡ ( θ ) M ( P θ ⋆ ( ⋅ ∣ c ) ) , E\;\triangleq\;\sup_{p^{*},\,c}\;\;\sup_{\theta^{\star}\in\arg\min_{\theta}\mathcal{L}(\theta)}\;M\big(P_{\theta^{\star}}(\cdot\mid c)\big), where the outer supremum over data distributions makes E E a property of the training objective itself rather than of any particular dataset, the inner supremum reads E E as a capacity—the ceiling the objective permits, analogous to parameter count—and minimizers range over all densities (we assume the nonparametric optimum is realizable). Under direct Bregman regression the minimizer is unique for every p ∗ p^{*} (the conditional mean, a point mass), so E = 1 E=1 regardless of parameter count. Under smooth Forward XM with K K candidates, E ≥ K E\geq K : taking p ∗ p^{*} with K K modes separated as in Proposition 3, every minimizer covers each mode with its own region of mass and so retains at least K K modes, one maximum per mode (Proposition 3); the separation condition is what makes the mode neighborhoods resolvable. Achievement is a per-distribution statement: on a p ∗ p^{*} with M ∗ ​ ( c ) ≤ K M^{*}(c)\leq K such modes, minimizers retain all M ∗ ​ ( c ) M^{*}(c) of them, so unimodal data yields unimodal minimizers at any K K . Finally, these claims belong to the smooth objective—the hard min differs by at most log ⁡ K \log K at finite K K , but its large- K K limit is a coverage objective without distributional control (Section F.2 ), so there E E reports unbounded capacity without implying sample quality. We otherwise inherit the regularity assumptions of Mode Forcing [ 19 ] —densities exist, reconstruction losses are Bregman divergences, and M M counts modes irrespective of their mass or separation—and refer to [ 19 ] for the mean-blurring lemma behind E = 1 E=1 .

#### Scope of E E .

E E is an at-optimum capacity: for direct regression it is pinned exactly ( E = 1 E=1 ), and for smooth Forward XM Proposition 3 delivers a lower bound with per-distribution achievement. Applied to consistent factored objectives, however, this at-optimum reading saturates. Autoregression under cross-entropy and continuous-time diffusion sampled exactly admit nonparametric minimizers whose sampling distribution is p ∗ p^{*} itself (under standard regularity), so at their optima they inherit every mode of the data and E E reaches the most the sample space allows— V L V^{L} for autoregression over L L tokens with vocabulary size V V , unbounded for continuous diffusion (read at the loss scale this count is finite [ 19 ] , and what a trained model realizes in practice is far smaller, small enough to bind). Nor do finite sampling steps restore a meaningful cap: even a two-step sampler’s optimum can retain arbitrarily many well-separated modes, albeit with no control over their mass—whereas Proposition 3 at least guarantees every mode is covered. The at-optimum definition therefore registers no deficit for any consistent objective, because at the optimum there is none—it separates direct regression from factored models, but not factored models from one another.

#### Per-Prediction Expressivity.

The deficit factored models do carry lives at the level of each prediction. A factored model is trained as many small reconstruction problems, and each carries its own ceiling—a per-prediction expressivity , how much of its own target’s structure a single step’s minimizer can retain. For an MSE step this ceiling is one mode, the conditional mean of its target; a parallel discrete decoder retains full per-position conditionals but predicts them independently, so no single step retains dependence among the tokens it reveals. Factoring does not raise these per-prediction ceilings; what it changes is each prediction’s residual multimodality —how many valid targets compete for it during training—which richer conditioning and finer factoring shrink but never quite remove (Section 2.1 ). Mode Forcing makes this picture quantitative, defining each prediction’s ceiling e t e_{t} and its residual multimodality M t ​ ( c t ) M_{t}(c_{t}) pointwise in the conditioning, so a prediction pays max ⁡ ( M t ​ ( c t ) − e t , 0 ) \max(M_{t}(c_{t})-e_{t},\,0) wherever it is asked; these shortfalls are not claimed to sum to a total [ 19 ] . This residue costs a model in one of two ways. Where sampling exposes the compromise directly it surfaces in generations, as blur or incoherence: direct regression, few jumps, coarse steps, or many positions unmasked at once. Where sampling does not, the same compromise is paid during training instead: competing targets pull each prediction against itself, whether the loss blurs them or must spread over them, a cost set by the training objective and untouched by the sampler. Relieving this training-time cost, we hypothesize, is why exploration can still help models whose sampling steps have already converged (Figures 7 and 8 ) and why XMs converge faster (Figure 4 ). So when we say existing models fix generative expressivity at design time, we mean the per-prediction notion: for a given data distribution the factorization sets the residue each prediction faces, the objective sets what each retains of it, and neither changes with parameters or data. Exploration attacks the residue directly, either by letting one input carry K K distinct predictions through a latent (end-to-end XMs and our XMDLMs), or by searching which noise draw a datapoint trains under so that fewer competing targets land on the same prediction (our continuous hybrids; Section C ). In the budget’s terms, the first route raises e t e_{t} —to at least K K for a squared-error prediction by Proposition 3, and heuristically for richer heads, where the K K latent-conditioned candidates turn the step’s product head into a mixture that can carry dependence among the positions it reveals—while the second shrinks the residue M t M_{t} each prediction effectively faces. It should therefore help most where the residue is largest—direct regression, few-jump Jumpy models (the jumps-for-exploration substitution of Figure 11 ), and the sparse, any-order conditioning of masked language models. It should help least for autoregression over discrete tokens, whose full-prefix conditioning leaves each next-token target nearly unimodal and whose per-token cross-entropy retains a full conditional rather than a mean, consistent with autoregressive language models proving harder to improve (Section 7 ). Continuous autoregression, where a plain per-step regression would fit each next element by the mean of its valid values, should benefit like the other continuous families. Even a small residue does not make exploration worthless, however: our video generation results improved significantly despite weak multimodality, which we hypothesize reflects surplus generative expressivity aiding optimization and generalization much as overparametrization does (Section 5 ).

### F.2 What Forward and Reverse XM Optimize

#### Overview.

This subsection makes the claims of Section 3.2 precise, and involves two distributions: the data distribution p ∗ p^{*} and the distribution of the model’s generations g θ g_{\theta} . Because squared error is, up to a constant, the negative log of a Gaussian of width σ \sigma , the loss effectively blurs whichever distribution it is applied to, so we write p θ p_{\theta} for the model’s generations blurred by this Gaussian and p σ ∗ p^{*}_{\sigma} for the data blurred in the same way. In their smooth forms, Forward and Reverse XM minimize KL ( p ∗ ∥ p θ ) + H ( p ∗ ) and KL ( g θ ∥ p σ ∗ ) + H ( g θ ) , \mathrm{KL}(p^{*}\,\|\,p_{\theta})+H(p^{*})\qquad\text{and}\qquad\mathrm{KL}(g_{\theta}\,\|\,p^{*}_{\sigma})+H(g_{\theta}), respectively. These identities hold at every K K with the blurred densities replaced by their K K -sample versions (Propositions 1 and 2), and hold exactly as written in the large- K K limit. The two objectives differ only in the entropy each carries. For Forward XM, this entropy is the data ’s, a constant the model cannot change, so Forward XM performs maximum likelihood—of its K K -candidate mixture—at every K K . For Reverse XM, this entropy is instead the model ’s own, which the model can lower by shrinking its spread, so Reverse XM drifts toward collapse by itself and needs an entropy bonus to become the pure reverse KL. The remainder of this subsection states and proves these claims.

#### Setup.