Self-Improving World Modelling with Latent Actions — part 4/4

Cleaned-text code points [46866, 60826). Bibliography omitted here; source.md includes it.

Table 5 reports a sanity-check evaluation of general image editing performance on GEdit-Bench . Although Liquid does not have native image editing support by design, we find that after our supervised fine-tuning (SFT) pipeline, using a mixture of Aurora ( Krojer et al., 2024 ) and Pico-banana-400K ( Qian et al., 2025 ) , the resulting model (Liquid-SFT) acquires functional image editing behaviour. We illustrate the qualitative examples for general image editing in Figure 5 .

Concretely, Liquid without SFT indicates a totally failure in GEdit-Bench , indicating the necessarily in the warm-up stage before SWIRL . Liquid-SFT achieves non-trivial scores in both semantic alignment and perceptual quality, confirming that the model can follow image editing instructions and produce coherent visual outputs. While Liquid-SFT remains significantly behind state-of-the-art proprietary and public image editing models, this result is not intended to be competitive; rather, it serves as a capability verification that our SFT pipeline successfully activates basic visual editing skills in a base model not originally designed for this task.

We emphasise that this capability plays an important role as a warm-up stage for subsequent world modelling objectives. In later stages, the model is trained to (i) predict the next visual observation given the current state (forward world modelling), and (ii) perform inverse dynamics modelling (IDM), where the model receives a pair of observations and predicts the intervening action in language. The ability to perform instruction-followed image transformations provides a minimal but necessary foundation for these more structured visual–temporal reasoning tasks.

Finally, the ablation study highlights the importance of Pico-banana-400K in our data mixture. Removing this component leads to a consistent degradation across all metrics (average score dropping from 3.06 to 2.43), indicating that Pico-banana provides critical signal for stabilising and enriching visual instruction-following behaviour. This result suggests that even for non-native capabilities, carefully curated visual editing data is essential for preserving functional performance after SFT.

We include the qualitative examples produced by our approach for general image editing in Figure 5 .

## Appendix D IDM Analysis

#### Aurora-Bench Evaluation.

To validate the reliability of our reward mechanism, we evaluate the Inverse Dynamics Model (IDM) performance directly on Aurora-Bench (Table 6 ). A fundamental premise of our framework is that inferring the action responsible for a state transition (IDM) is a more tractable task than generating high-dimensional future states (FWM). The results support this hypothesis: the IDM model achieves high action prediction accuracy (6.38 out of 10 as GPT score), serving as a strong discriminator. Additionally, we observe that the iterative paradigm can also leverage the reward signal from FWM, effectively improving GPT4o scores from 6.38 to 6.58 as our best approach. This stability is vital for our self-improving, ensuring that the reward signal remains informative and accurately penalises physical inconsistencies.

#### Interpretability of Latent Actions and Reward Hacking.

A potential concern in learning with the latent actions is reward hacking, where actions might degenerate into short, distinctive ciphers (e.g., single keywords or artifacts) to be hacked for an easier optimisable FWM. We analyse the evolution of complexities of the generated actions across three iterations for SWIRL in Figure 6 .

Contrary to the hypothesis of reward collapse, we observe that the uniqueness of generated actions remains consistently high ( > 94 % >94\% ) across all iterations for both shared and separate weight configurations. The similar observation happens for naturalness that the predicted actions are quite stably natural (as evidenced by the predicted perplexity of an LLM, GPT-2) throughout SWIRL ’s training iterations, without collapsing into any unnatural artifacts that could be easily hacked by FWM or IDM. Looking into the lengths of predicted actions, the model does not "shortcut" by generating brief tokens as well; instead, the average action length (around 16.8 16.8 tokens) consistently exceeds the ground truth average (around 7.2 7.2 tokens). This indicates that the IDM learns to provide more descriptive and detailed instructions to ensure the transition is consistently identifiable, rather than drifting into a simplified cipher. Qualitative inspection confirms that predictions remain physically meaningful and there is no model collapse happening on SWIRL (e.g., "tearing paper into two pieces", "Swap the positions of the two objects.", "turning a bottle upside down" ), preserving natural language interpretability.

## Appendix E Inference-time Verification

As a sanity check for SWIRL , we investigate the validity of the coverage hypothesis intrinsic to our GRPO formulation. For the IDM reward to effectively guide learning, the set of sampled rollouts must contain at least one candidate that sufficiently approximates the ground-truth future state. We evaluate the "Best-of- N N " performance of Liquid-SFT, where the candidate with the highest GPT4o score is selected, by scaling the number of rollouts N N . In Table 1 , we observe a monotonic improvement for inference-time verification as N N increases, confirming that the base policy possesses the latent capability to generate high-fidelity transitions in multiple rollouts. This finding empirically validates our training premise: expanding the rollout space should increases the likelihood of capturing valid transitions, thereby providing the IDM discriminator with the necessary high-quality positive examples to reinforce.

## Appendix F Detailed Results for Comparing SFT and SWIRL .

Table 7 presents a detailed comparison between SWIRL and two supervised fine-tuning baselines under controlled data budgets. Across all five benchmarks and all training sample counts, RL consistently exhibits superior data efficiency compared to both SFT-Continue and SFT-Merge . While SFT baselines often plateau or even degrade as more samples are introduced, RL shows a clear trend of performance improvement as additional data becomes available.

Notably, RL begins to outperform SFT at relatively small data scales (6.4K samples) and the performance gap widens as training proceeds. On Action Genome, Something, and Whatsup, Kubric, which emphasises structured physical dynamics, RL achieves the strongest or near-strongest performance at every scale, indicating that reward-guided optimisation better captures latent world structure than direct imitation.

Aggregated across all benchmarks, RL achieves the highest average score at every data scale, with gains becoming increasingly pronounced at larger budgets (4.27 → 4.73). These results suggest that RL not only improves final performance but also utilises limited data more effectively, aligning with our hypothesis that IDM’s rewarding mechanism provides a stronger inductive bias for intrinsic world modeling than purely supervised objectives.

## Appendix G Detailed Results for WorldPredictionBench

Table 8 presents a granular analysis of predictive performance across six consecutive autoregressive turns on WorldPredictionBench . This evaluation creates a challenging stress test for temporal consistency, as errors generated in early turns compound over the predictive horizon.

We observe two primary trends. First, as expected, all models exhibit performance decay as the horizon increases ( N N decreases as tasks are completed or fail). However, a clear divergence emerges between training paradigms. The direct supervised baseline ( Liquid-SFT ) suffers from rapid degradation, dropping from an overall score of 3.09 in Turn 1 to 1.17 by Turn 4. In contrast, our proposed SWIRL demonstrate significantly improved robustness against this compounding error. The Ours (Best) configuration, maintains a score of 1.59 at Turn 4. This indicates that the reciprocal cycle does not merely memorise single-step transitions but internalises more robust physical dynamics that persist over long-horizon rollouts. While large-scale unified models like Bagel provide a high upper bound, our method close the gap compared to SFT baseline.

## Appendix H Detailed Results for Iterative Results

Table 9 examines the iterative training dynamics of Forward World Modeling (FWM) and Inverse Dynamics Modeling (IDM) under two architectural choices: using separate weights for the FWM and IDM versus shared weights. We report results across three training iterations to highlight peak performance and learning progression.

The separate-weight configuration achieves the highest peak FWM performance, with Iteration 1 attaining the best average FWM score (5.06) and consistently strong results across all benchmarks. In particular, gains on MagicBrush, Action Genome, and Something indicate that decoupling the FWM and IDM allows each component to specialise more effectively, leading to higher-quality forward predictions when the system is optimally aligned.

While the shared-weight setup exhibits competitive and stable behaviour, it does not surpass the peak generative performance achieved by the separate-weight model. Notably, although IDM accuracy continues to improve slightly in later iterations for both settings, the separate-weight design reaches its optimal balance between FWM quality and IDM alignment earlier in training. This suggests that architectural decoupling enables more expressive forward modeling, even if later iterations yield diminishing returns.

Overall, these results indicate that separating FWM and IDM parameters is advantageous for maximising peak forward world modelling performance, motivating our choice of the separate-weight configuration in the main experiments where peak capability is the primary objective.

## Appendix I Ablation on GRPO Rollout Size.

We analyse the effect of rollout size. The group size G G is a critical hyperparameter in GRPO, as it governs the accuracy of the baseline estimation and the diversity of the exploration within each optimisation step. We empirically evaluate G ∈ { 8 , 16 , 32 , 64 } G\in\{8,16,32,64\} across Aurora-Bench , with results summarised in Table 10 . We observe that scaling the rollout size generally improves model performance, achieving a peak average score of 4.90 at G = 64 G=64 . We attribute this to the reduced variance in advantage estimation: a larger pool of generations provides a more robust approximation of the expected return, enabling the policy to distinguish high-quality trajectories more effectively. Notably, while G = 64 G=64 yields the best absolute performance—particularly in complex environments like Kubric, the smaller configuration of G = 16 G=16 remains highly competitive (4.80 avg.) while requiring much less memory and compute during the generation phase.

## Appendix J Qualitative Examples

We provide some qualitative examples in Figure 7 to illustrate the visual predictions of SWIRL across three distinct benchmarks: Aurora-Bench , ByteMorph , and WorldPredictionBench .

Aurora-Bench . The top panel demonstrates results on the AURORA benchmark, covering various subsets such as MagicBrush, Something, Emu, Kubric, and Whatsup. These examples highlight the model’s capability to perform diverse action-centric image editing tasks, ranging from the general editing (e.g., Add a supernova explosion in the sky ) and background replacement (e.g., changing to Yellowstone National Park ) to precise geometric transformations and spatial reasoning. For instance, in the Something and Kubric subsets, the model successfully executes actions concerning physical regularities like flip the bottle upside down and distinct spatial rearrangements like moving a specific plate to the left .

ByteMorph . The middle panel displays qualitative results for ByteMorph , focusing on fine-grained control over camera and object dynamics. We visualise three distinct categories: Camera Zoom, Camera Motion, and Object Motion. The results show the model’s ability to synthesise coherent view changes, such as zooming out to reveal context (e.g., the coastline and a second boat ) or shifting the camera angle upward. Additionally, the Object Motion example demonstrates the generation of localised movement, specifically raising an animal’s head while maintaining scene consistency.

WorldPredictionBench . The bottom panel illustrates long-horizon predictions on the WorldPredictionBench . Here, we show multi-step predictions where the model generates subsequent states based on the previous predicted image as the context and textual action descriptions. The examples depict long-horizon consistency in procedural tasks, such as replacing a car key battery (Start → Put in battery → Close cover) and arranging bedding (Start → Take out cover → Arrange nicely).

## Instructions for reporting errors

We are continuing to improve HTML versions of papers, and your feedback helps enhance accessibility and mobile support. To report errors in the HTML that will help us improve conversion and rendering, choose any of the methods listed below:

Click the "Report Issue" ( ) button, located in the page header.

Tip: You can select the relevant text first, to include it in your report.

Our team has already identified the following issues . We appreciate your time reviewing and reporting rendering errors we may not have found yet. Your efforts will help us improve the HTML versions for all readers, because disability should not be a barrier to accessing research. Thank you for your continued support in championing open access for all.

Have a free development cycle? Help support accessibility at arXiv! Our collaborators at LaTeXML maintain a list of packages that need conversion , and welcome developer contributions .