VISReg: Variance-Invariance-Sketching Regularization for JEPA training

arxiv:2606.02572

Original source · Paper JSON

Self-supervised learning and vision · 2026-06-01

Cites (7) · Cited by (0)

Complete cleaned Markdown · Source Markdown including bibliography

Read in parts

1. Part 1
2. Part 2
3. Part 3
4. Part 4

Contents

• VISReg: Variance-Invariance-Sketching Regularization for JEPA training
• Abstract
• Keywords:
• 1 Introduction
• 2 Related Work
• 3 VISReg: Variance-Invariance-Sketching Regularization
• 3.1 Regularization Loss
• Lemma 3.1 (Cramér-Wold Theorem) .
• Lemma 3.2 (1D Wasserstein Closed-Form) .
• Proposition 3.3 (VISReg Regularization Objective) .
• 3.2 VISReg Is Friendly to Scale Up
• Definition 3.4 .
• 3.3 VISReg Is Robust to Low-Quality Datasets
• 4 Experiment
• 4.1 Ablation study
• 4.2 General comparison
• 5 Conclusion
• 6 Acknowledgment
• Appendix A Implementation details
• A.1 Training details
• A.2 Testing details
• A.3 Datasets
• Appendix B Additional ablations
• B.1 Effect of decoupled components in training.
• B.2 Effect of decoupled components in different training set scenarios.
• Appendix C Visualizations
• C.1 VISReg loss indicates the performance
• C.2 Further comparison with DINOv1 on image and video.
• PCA Feature Visualization.
• Instructions for reporting errors