Universal Transformers

arxiv:1807.03819

Original source · Paper JSON

LLMs: architectures, context, training · 2018-07-10

Cites (0) · Cited by (21)

Complete cleaned Markdown · Source Markdown including bibliography

Read in parts

1. Part 1
2. Part 2
3. Part 3

Contents

• Universal Transformers
• Abstract
• 1 Introduction
• 2 Model Description
• 2.1 The Universal Transformer
• 2.2 Dynamic Halting
• 3 Experiments and Analysis
• 3.1 bAbI Question-Answering
• 3.2 Subject-Verb Agreement
• 3.3 LAMBADA Language Modeling
• 3.4 Algorithmic Tasks
• 3.5 Learning to Execute (LTE)
• 3.6 Machine Translation
• 4 Discussion
• 5 Conclusion
• Acknowledgements
• Appendix A Detailed Schema of the Universal Transformer
• Appendix B On the Computational Power of UT vs Transformer
• Appendix C UT with Dynamic Halting
• Appendix D Description of some of the Tasks/Datasets
• D.1 bAbI Question-Answering
• D.2 Subject-Verb Agreement
• D.3 LAMBADA Language Modeling
• D.4 Learning to Execute (LTE)
• Appendix E bAbI Detailed Results
• Appendix F bAbI Attention Visualization
• Instructions for reporting errors