One Embedder, Any Task: Instruction-Finetuned Text Embeddings

arxiv:2212.09741

Original source · Paper JSON

Retrieval, embeddings, benchmarks · 2022-12-19

Cites (2) · Cited by (1)

Complete cleaned Markdown · Source Markdown including bibliography

Read in parts

1. Part 1
2. Part 2
3. Part 3

Contents

• One Embedder, Any Task: Instruction-Finetuned Text Embeddings
• Abstract
• 1 Introduction
• 2 InstructOR
• 2.1 Embedding Architecture
• 2.2 Training Objective
• 2.3 MEDI: Multitask Embedding Data with Instructions
• Data Construction
• Instruction Annotation
• 3 Experiments
• 3.1 Main Results
• 4 Analysis and Ablations
• 4.1 Instructions Enable Diverse Training
• 4.2 Instruction Robustness
• 4.3 Complexity of Instructions
• 4.4 Model Sizes and Instruction Finetuning
• 4.5 Instructions Mitigate Domain Shifts
• 4.6 Qualititive Analysis
• 5 Related Work
• Text Embeddings
• Instruction Finetuning
• 6 Conclusion
• 7 Limitations
• Acknowledgements
• Appendix A Training Setups
• Minibatch Sampling
• Training
• Baselines
• Appendix B Embedding Evaluations
• B.1 Massive Text Embedding Benchmark
• Retrieval
• Reranking
• Clustering
• Pair Classification
• Classification
• STS
• Summarization
• B.2 Prompt Retrieval
• B.3 Automatic Evaluation for Generation
• Appendix C Full instructions
• Appendix D Full Results
• Instructions for reporting errors