Hierarchical Latent Prediction for Language Models

arxiv:2608.05806

Original source · Paper JSON

LLMs: architectures, context, training · 2026-08-06

Cites (2) · Cited by (0)

Complete cleaned Markdown · Source Markdown including bibliography

Read in parts

1. Part 1
2. Part 2

Contents

• Hierarchical Latent Prediction for Language Models
• Abstract
• 1 Introduction
• 2 Methodology
• Training objectives.
• Consistency conditions.
• 3 Experiments
• 3.1 Coding and Multi-step Reasoning Benchmarks
• 3.2 Speculative Decoding
• 4 Discussion
• Longer horizon prediction.
• 5 Related Work
• 5.1 Pretraining Objectives Beyond Next-Token Prediction
• 5.2 Latent-Space Language Modeling
• 5.3 Hierarchical Models
• 6 Conclusion
• 7 Future Work
• Appendix A HiLP Training Procedure
• Appendix B Experiment hyperparameters
• Appendix C Latent overhead and efficiency.
• Appendix D Input-Combination Modes
• Instructions for reporting errors