Explorative Modeling: Unlocking a Third Pretraining Axis and End-to-End Generation — part 3/8
Cleaned-text code points [30792, 46260). Bibliography omitted here; source.md includes it.
The goal of this section is to demonstrate that Explorative Modeling can be used both as a new pretraining axis (Section 4.1 ) as well as a standalone generative modeling approach (Section 4.2 ). As a new pretraining axis, we experiment with generative modeling hybrids combining exploration with either Diffusion/Flow [ 42 , 13 , 23 ] or Jumpy [ 30 ] generative models across both continuous and discrete domains. We refer to a generative model paired with Explorative Modeling by prefixing its name with an X (e.g., XDiffusion, XJumpy), reflecting how these are hybrid explorative and existing generative modeling combinations. Across all hybrid experiments, we do no XM-specific hyperparameter tuning, keeping each baseline recipe’s hyperparameters unchanged and only adding exploration. As a standalone generative modeling approach, we compare XMs to strong baselines in Behavior Cloning and Goal-Conditioned World Modeling. For all experiments in the main section of this paper, we denote exploring K K modes as XM- K K , and we use Forward XM (Section 3 ), as it is simpler to implement (this is discussed further in Section 6 ). Note that all baselines without exploration are equivalent to XM-1, as exploring a single mode reduces to standard training. By default, all experiments in this section are without guidance, except for guidance-based results reported in Table 1 . Our largest image generation experiments report FDr 6 \mathrm{FDr}^{6} because FID has saturated at this performance level [ 43 , 44 ] ; FDr 6 \mathrm{FDr}^{6} works by averaging the Fréchet distance to the training data over six representation spaces.
### 4.1 Explorative Modeling as a New Scaling Axis
#### Does Exploration Improve Existing Generative Models’ Performance?
Progress in generative modeling has largely been driven by scaling parameter expressivity through training larger models. This raises the question—if scaling parameter expressivity helps, why not also scale generative expressivity , a model’s capacity to capture multiple modes, rather than average them? Existing scalable reconstructive generative models rely on the same generation factorization regardless of model size, fixing generative expressivity at design time rather than scaling it. If this factorization is not sufficient to capture all the modes in the distribution, we could expect that adding exploration, as a way to increase generative expressivity, could improve performance. To investigate this, we experiment with adding exploration to existing generative models, including Diffusion/Flow and Jumpy generative models [ 30 ] (which generalize Diffusion/Flow by varying the number of steps, or jumps , interpolating between single-step regression and continuous-time Flow; Section 2.3 ).
We begin by adding exploration to a strong image generation recipe (RAE [ 45 ] ), training models that differ only in the amount of exploration. We find that exploration significantly improves performance throughout training, reaching the no-exploration baseline’s final performance with 6.2 × 6.2\times less data 7 7 7 Throughout this paper, we refer to two notions of sample/data efficiency. Here we mean the first, or the number of training samples processed (training steps at a fixed batch size) to reach a given performance. The second notion is the best performance achievable on a fixed-size dataset before overfitting, which concerns generalization; we test that separately in Figure 9 . and 4.1 × 4.1\times fewer FLOPs (Figure 4 ). These gains also hold beyond a single recipe, where adding exploration to an optimally tuned SiT baseline [ 29 ] improves FLOP efficiency by as much as 52 % 52\% and reaches the same performance with 2.5 × 2.5\times less data (Figure 5 ). Notably, the SiT experiments use roughly a third of the compute of the RAE experiments, meaning the efficiency gains from exploration more than doubled when moving to the larger-scale setting at 3 × 3\times the compute—suggesting gains from exploration grow with scale, a pattern we examine more directly below.
Notably, each explored mode in Forward XM adds compute (Reverse XM largely avoids this), yet despite this cost, the FLOP-optimal number of modes to explore grows as training continues (Figures 4(b) and 5(b) ), as generative expressivity increasingly becomes the bottleneck. This mirrors compute-optimal parameter scaling [ 16 ] , where just as the optimal number of parameters grows as compute increases, the optimal amount of exploration grows too, meaning models that explore more modes eventually scale faster . Exploration also improves parameter efficiency—a Large model exploring 5 modes outscales an XLarge model with 47 % 47\% more parameters and no exploration (Figure 6 ). These results show exploration is not just more performant, but that it enables a more efficient use of compute, data, and parameters.
#### Does Performance Across Modalities Scale with Exploration?
Having seen exploration added to existing models improve image generation performance, we next ask whether these benefits extend across modalities, and how they scale with the amount of exploration. To test this, we train image generation models, video generation models, and language models with a fixed parameter count, varying only the number of modes explored. For both image and video generation, increasing exploration monotonically improves performance as measured by FID and FVD respectively (Figure 7 ), with some models seeing a greater than 20% performance boost. This benefit carries over to discrete data, where adding exploration to a masked diffusion language model (MDLM) improves its perplexity-entropy frontier 8 8 8 This frontier has become the standard evaluation in this setting, as generative perplexity alone can be gamed by low-entropy sampling [ 46 , 47 ] . across the board (Figure 8 ), demonstrating that exploration helps in both continuous and discrete spaces. Notably, these gains do not stop as exploration increases (Figure 7 ), suggesting increased exploration could further improve performance.
#### Does Exploration Improve Generalization?
So far, we have measured sample/data efficiency as the number of training samples a model must process to reach a target performance. A stronger, more generalization-focused evaluation asks how much a model can extract from a fixed dataset—its best achievable performance before it begins to overfit [ 50 ] . We experiment with this setup, training models until their validation-set performance starts to get worse. One reason a model may overfit comes down to its generative expressivity—its capacity to represent multiple modes rather than collapse them to an average (Section 2.1 ). With limited generative expressivity, a model’s best possible prediction is a blurred compromise between modes, which typically lies off the data manifold and matches no real datapoint, so a model fitting this compromise is memorizing something that does not exist in the true data distribution rather than generalizing. Even when generative expressivity is not the bottleneck, having surplus expressivity may ease optimization toward simpler solutions explaining the data, which tend to generalize better, much as overparametrization does [ 51 , 52 ] . Therefore, as exploration increases generative expressivity directly, it may improve generalization. We find this for video generation (Figure 9 ), where increasing exploration improves generalization by reducing overfitting, resulting in a better absolute minimum FVD ( 30.0 30.0 with exploration versus 37.5 37.5 without exploration). Because this improvement comes from spending more training compute on exploration, it amounts to a compute-generalization tradeoff : extra compute directly buys better generalization. As data, rather than compute, increasingly becomes the bottleneck for large-scale training [ 50 , 53 , 54 ] , we see improved generalization as an especially promising characteristic of XMs.
#### Does Exploration Improve State-of-the-Art Recipes at Scale?
If exploration is a genuine scaling axis, it should improve even the strongest, most heavily tuned recipes. The Representation Autoencoder (RAE) [ 45 ] recipe from Figure 4 provides such a test: RAE was the state-of-the-art ImageNet 256 × \times 256 image generation recipe as of three months before the release of this work, primarily involving a change of representation space from the SD-VAE [ 55 , 32 ] to a Representation Autoencoder. Aside from the previously discussed 6.2 × 6.2\times data and 4.1 × 4.1\times FLOP efficiency improvements, we find the performance gains also hold as models converge (training XL models for up to 2.2 × 10 21 2.2\times 10^{21} FLOPs), where an XM-2 RAE model reaches near-state-of-the-art non-CFG FID without post-training (Table 1 ), and much better FDr 6 \mathrm{FDr}^{6} than the baseline. Convergence also compounds across recipes: XRAE converges 6.2 × 6.2\times faster than RAE, which itself converges 47 × 47\times faster than SiT [ 45 , 29 ] —making XRAE almost 300 × 300\times faster to converge than the standard SiT recipe.
#### Do Improvements From Exploration Vary With Scale?
Throughout our experiments, exploration has helped more at larger scale—most notably, its efficiency gains more than doubled when moving from the SiT setting to the RAE setting, which used 3 × 3\times the compute. This follows from how a generative model’s performance is limited by three capacities: parameters restrict what it can represent, data restricts what it can learn, and generative expressivity restricts what it can generate. Conventional scaling of parameters and data relieves those constraints, but generative expressivity is set by the training objective itself (Section 2.1 ; see Section F.1 for the formal scope), so it stays fixed regardless of how large models or datasets grow. At small scale, this fixed generative expressivity is generally not an issue, as models are primarily held back by limited parameters and data. However, as parameter and data scale increase, generative expressivity increasingly becomes the bottleneck. Therefore, since exploration raises generative expressivity directly, we hypothesize its benefits should grow as models and data scale.
To test this, we measure the gains from exploration while varying model size and data scale, and find that gains rise from 7 % 7\% to 36 % 36\% as data scales, and from 13 % 13\% to 23 % 23\% as model size scales (Figure 10 ). The FLOP and sample efficiency experiments reinforce this—exploration helped more the longer a model was trained, with the FLOP-optimal amount of exploration growing over the course of training (Figures 4(b) and 5(b) ). Together, these results point to exploration as a missing scaling axis in existing generative models , where the compute-optimal amount of exploration grows with scale just like parameters and data. This means today’s generative models, trained without exploration, increasingly fall short of what compute-optimal exploration would achieve. Foundation-model training runs use roughly four orders of magnitude more compute than our largest experiments, so if this trend continues, the improvements reported here likely underestimate the gains at that scale.
#### Which Models Benefit Most From Exploration?
So far, exploration has helped every model family we tested, raising a natural question—are some generative models better suited to exploration than others?
To investigate this, we compare Diffusion/Flow with Jumpy generative models. Jumpy models work by interpolating between direct end-to-end regression ( 1 1 jump) and continuous-time Flow ( ∞ \infty jumps), so using a finite number of jumps is more end-to-end than Flow and more generatively expressive than a single-step regressor (More details in Section 2.3 ).
We compare XDiffusion and XJumpy generative models for FID and FVD scaling as exploration increases in Figure 7 , where the rate of improvement for XJumpy models as exploration increases is much higher than the rate for XDiffusion. For example, in Figure 7(a) , XJumpy generative models start out as performing worse than XDiffusion models with no exploration, but as exploration increases XJumpy models become more performant than XDiffusion models. This is further reinforced by Figure 10(b) , where XJumpy models see larger gains from exploration than XDiffusion as data scales. Together, these results suggest that generative models that are more end-to-end scale better with increased exploration.
We can test this hypothesis further by varying the number of jumps within an XJumpy model. If factoring training can substitute for factoring generation, then more exploration should decrease the optimal number of jumps, since exploration supplies the generative expressivity those extra jumps would otherwise provide. Figure 11 shows this, where an XJumpy model with fewer jumps scales better with exploration than an XJumpy model with more jumps. These results directly demonstrate that as exploration increases, more end-to-end models perform better—in effect, exploration scales how end-to-end existing generative models can be . This makes Jumpy models, which can be more end-to-end than Diffusion/Flow, a promising approach to pair with exploration, and offers an indirect route to better generalization, as more end-to-end models reduce exposure bias (Section 2.2 ).
### 4.2 End-to-End Explorative Models
#### Can Explorative Modeling be Used for End-to-End Generation?
So far, exploration has been combined with existing generative models, where we found it enables them to become more end-to-end (Figure 11 ); we now take this trend to its limit, using Explorative Models as standalone end-to-end generative models, where sampling is identical at training and inference (Section 2.2 ). We evaluate end-to-end XMs on robotics control tasks, including Behavior Cloning, comparing our Explorative Policy to Diffusion Policy [ 17 ] (Table 2 ), and Goal-Conditioned World Modeling, comparing our Explorative World Model to Diffuser [ 18 ] (Table 3 ).
In both settings, Explorative Models match diffusion baselines at a fraction of the inference compute—Explorative Policy rivals Diffusion Policy with a single network forward pass instead of 100, and the Explorative World Model matches Diffuser using 16 16 - 256 × 256\times fewer function evaluations. This gap comes directly from what each approach factors: diffusion pays for its generative expressivity with generation steps at inference, while end-to-end XMs pay for it with exploration during training, keeping inference at a single forward pass. Notably, we obtain these results with barely any tuning of hyperparameters for XMs—we keep each baseline’s architecture and occasionally add a recurrent block for an inductive bias toward recurrence—so we believe these results underrepresent how well-tuned XMs can perform with additional tricks. The main limitation of these experiments is in handling highly multimodal distributions. Because we use Forward XM here, which has a cost that grows with the number of modes explored, it cannot cheaply cover extremely multimodal distributions. Therefore, Reverse XM is likely better suited for end-to-end generation tasks, which we largely leave for future work (we discuss successful Reverse XM training in Section A , and future directions in Section 6 ).
## 5 Discussion
#### Mode Forcing as a Predictive Theory.