Learning to Compress Prompts with Gist Tokens

arxiv:2304.08467

Original source · Paper JSON

Retrieval, embeddings, benchmarks · 2023-04-17

Cites (2) · Cited by (3)

Complete cleaned Markdown · Source Markdown including bibliography

Read in parts

1. Part 1
2. Part 2
3. Part 3

Contents

• Learning to Compress Prompts with Gist Tokens
• Abstract
• 1 Introduction
• 2 Gisting
• 2.1 A Context Distillation Perspective
• 3 Learning Gisting by Masking
• 4 Experiments
• 4.1 Data
• 4.2 Models
• Positive Control.
• Negative Control.
• Discrete Compression with TF-IDF.
• 4.3 Evaluation
• ROUGE-L.
• ChatGPT.
• Human eval.
• 5 Results
• 5.1 Failure cases
• 6 Compute, Memory, and Storage Efficiency
• 7 Additional Related Work
• Adapting LMs without Backprop.
• Compression and memory in transformers.
• Sparse attention mechanisms.
• 8 Discussion and Limitations
• Acknowledgments and Disclosure of Funding
• Appendix A Example PyTorch Implementation of Gist Masking
• Appendix B Data, Training, Evaluation, and Compute Details
• Data.
• Training.
• Evaluation.
• Compute.
• Appendix C Details of ChatGPT Evaluation
• Appendix D Additional Human Evaluation Details and Results
• D.1 Experimental Details
• D.2 Additional Results
• Appendix E Exact Match Results
• Appendix F Additional FLOPs details
• Embeddings
• Attention (Single Layer)
• Dense Block
• Final Logits
• Total Forward Pass FLOPs
• Appendix G Additional Examples of Model Outputs
• Instructions for reporting errors