VISReg: Variance-Invariance-Sketching Regularization for JEPA training — part 4/4

Cleaned-text code points [46784, 49376). Bibliography omitted here; source.md includes it.

To qualitatively compare the learned representations, we visualize patch-level features from different ViT encoders using PCA coloring. For each input image, we extract the spatial patch token features from the last layer of the encoder, yielding a feature map of shape H p × W p × C H_{p}\times W_{p}\times C , where H p H_{p} and W p W_{p} are the patch grid dimensions and C C is the embedding dimension. We flatten this to an N × C N\times C matrix ( N = H p × W p N=H_{p}\times W_{p} ) and apply PCA to reduce it to three components, which are then interpreted as RGB channels. Each component is independently normalized to [ 0 , 1 ] [0,1] via min–max scaling, and the resulting H p × W p × 3 H_{p}\times W_{p}\times 3 map is bilinearly upsampled to the original image resolution for display. Since PCA components are determined only up to permutation and sign, direct comparison between models requires alignment. We compute the 3 × 3 3\times 3 Pearson correlation matrix between the PCA components of a reference model ( i.e. , DINO ViT-B/16 ( Caron et al., 2021 ) ) and the target model ( i.e. , VISReg ViT-B/16), then solve the optimal assignment using the Hungarian algorithm on − | corr | -\lvert\mathrm{corr}\rvert . The matched target components are reordered accordingly and flipped in sign where the correlation is negative, ensuring consistent color semantics across models. For video visualizations, PCA is fit jointly on the patch features from all frames to ensure temporal consistency of the color mapping. Both Figure 9 and Figure 8 indicates that VISReg helps model learn more granular details than DINO.

## Instructions for reporting errors

We are continuing to improve HTML versions of papers, and your feedback helps enhance accessibility and mobile support. To report errors in the HTML that will help us improve conversion and rendering, choose any of the methods listed below:

Click the "Report Issue" ( ) button, located in the page header.

Tip: You can select the relevant text first, to include it in your report.

Our team has already identified the following issues . We appreciate your time reviewing and reporting rendering errors we may not have found yet. Your efforts will help us improve the HTML versions for all readers, because disability should not be a barrier to accessing research. Thank you for your continued support in championing open access for all.

Have a free development cycle? Help support accessibility at arXiv! Our collaborators at LaTeXML maintain a list of packages that need conversion , and welcome developer contributions .