Explorative Modeling: Unlocking a Third Pretraining Axis and End-to-End Generation — part 8/8

Cleaned-text code points [109461, 114637). Bibliography omitted here; source.md includes it.

No. Exploration happens entirely during training, so inference is unchanged. The added cost shows up as more expensive training instead (for Forward XM, for Reverse XM the added training cost is often negligible), and as shown in Figure 4(b) , this cost is well worth it.

#### Is exploration really a new “axis” if it just costs more compute?

Scaling parameters or data costs compute too—no scaling axis is free. The question is whether exploration is a good way to spend compute, which is the same question compute-optimal scaling [ 16 ] asks when dividing a budget between parameters and data. Our FLOP-matched comparisons answer it directly: models that explore are significantly more compute efficient than models that just train for longer (Figures 4(b) and 5(b) ), and the FLOP-optimal amount of exploration grows with scale, just as the compute-optimal parameter count does.

#### Why the name Explorative Modeling?

We believe the idea of exploration captures the intuition of trying to understand the model’s loss landscape with respect to the different modes. However, exploration is a type of search, and we considered naming the approach after search directly. We decided against this because gradient descent is already a search over parameters, and because search is now strongly associated with inference, where “scaling search” would sound like inference-time thinking/reasoning rather than a training axis. Every candidate term is overloaded in some way; we found exploration to be the best general term, covering Forward XM, Reverse XM, and the gradient-based variants discussed in Section 6 .

#### When does exploration help?

Exploration helps whenever generation is conditioned on a latent variable the model can search over, such as the input noise in a diffusion model. We have found exploration helps most when generative expressivity is a bottleneck, although this does not have to be the case—our video models improved significantly even though the generated distribution was not very multimodal (Section 5 ).

#### Does training on the model’s own generations cause collapse?

No, because the training targets are always real data: the model’s generations only decide which datapoint each noise input gets paired with. In Forward XM, every datapoint still receives a training signal, so no part of the data distribution can be dropped; in fact, Forward XM is maximum likelihood (Section F.2 ). Collapse is a genuine concern for Reverse XM, which is mode-seeking, and is exactly why it needs an entropy term. A simple coverage constraint served as this entropy term when training our Reverse XM language models (Section A ).

#### How should the amount of exploration K K be chosen?

This deserves more study, just like studying how parameters and data should be scaled together has been important [ 16 ] . The right amount of exploration is problem dependent, as some distributions have many more modes to capture and so rely on exploration more than others. In our experiments, higher K K values became better with scale, both within a single run, where the FLOP-optimal amount of exploration grows over the course of training (Figures 4(b) and 5(b) ), and across runs, where gains from exploration grow with model and data scale (Figure 10 ). Exploration is also flexible: models with more training compute available can increase exploration to buy generalization and more end-to-end generation. Given these many tradeoffs, we recommend sweeping K K across 1 1 (the baseline), 2 2 , 3 3 , and 5 5 to start, sweeping 8 8 and 12 12 if compute permits, and pushing even further if gains continue. When compute is tight, we recommend starting with small values like K = 2 K=2 or 3 3 , as they are cheap and improve performance significantly (Figure 4 ).

#### Is Explorative Modeling a form of reinforcement learning?

No. Explorative Modeling is a generative modeling objective, unrelated to reinforcement learning. RL uses “exploration” for agents trying varied actions to discover reward [ 105 ] , which differs in mechanism and purpose from XMs’ within-step candidate sampling, though both share the intuition that trying many options reveals information a single choice cannot.

## Instructions for reporting errors

We are continuing to improve HTML versions of papers, and your feedback helps enhance accessibility and mobile support. To report errors in the HTML that will help us improve conversion and rendering, choose any of the methods listed below:

Click the "Report Issue" ( ) button, located in the page header.

Tip: You can select the relevant text first, to include it in your report.

Our team has already identified the following issues . We appreciate your time reviewing and reporting rendering errors we may not have found yet. Your efforts will help us improve the HTML versions for all readers, because disability should not be a barrier to accessing research. Thank you for your continued support in championing open access for all.

Have a free development cycle? Help support accessibility at arXiv! Our collaborators at LaTeXML maintain a list of packages that need conversion , and welcome developer contributions .