Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolution

arxiv:2309.16797

Original source · Paper JSON

Retrieval, embeddings, benchmarks · 2023-09-28

Cites (3) · Cited by (11)

Complete cleaned Markdown · Source Markdown including bibliography

Read in parts

1. Part 1
2. Part 2
3. Part 3
4. Part 4

Contents

• Promptbreeder: Self-Referential Self-Improvement via Prompt Evolution
• Abstract
• 1 Introduction
• 2 Related Work
• 3 Promptbreeder
• 3.1 Promptbreeder Initialization
• 3.2 Mutation Operators
• 3.2.1 Direct Mutation
• 3.2.2 Estimation of Distribution Mutation
• 3.2.3 Hypermutation: Mutation of Mutation-Prompts
• 3.2.4 Lamarckian Mutation
• 3.2.5 Prompt Crossover and Context Shuffling
• 4 Experiments
• 5 Results and Discussion
• 6 Conclusion and Future Work
• Acknowledgments
• Appendix A Glossary
• Appendix B A Typical Evolutionary Run
• Appendix C Mutation Prompts
• Appendix D Thinking Styles
• Appendix E Initially Evolved Prompts
• Appendix F Promptbreeder as Self-Referential Self-Improvement System
• Appendix G Problem Descriptions
• Appendix H Lamarckian Mutation Example
• Appendix I Datasets
• I.1 Control Task-Prompts
• I.2 Arithmetic Reasoning
• I.3 Commonsense Reasoning
• I.4 Hate Speech Classification
• I.5 Instruction Induction
• Appendix J Example Results
• J.1 ETHOS Evolved Prompt
• J.2 Prompt Evolution Maths results
• J.3 Evolved Mutation Prompts
• J.4 Mutation Operator Effectiveness
• J.5 ADDSUB
• J.6 AQUA
• J.7 MULTIARITH
• J.8 GSM8K
• J.9 SINGLEEQ
• J.10 SVAMP
• Appendix K APE Instruction Induction tasks
• K.1 Best prompts and contexts
• K.1.1 First Letter
• K.1.2 Second Letter
• K.1.3 List Letters
• K.1.4 Starting With
• K.1.5 Pluralization
• K.1.6 Passivization
• K.1.7 Negation
• K.1.8 Antonyms
• K.1.9 Synonyms
• K.1.10 Membership
• K.1.11 Rhymes
• K.1.12 Larger Animal
• K.1.13 Cause Selection
• K.1.14 Formality
• K.1.15 Sum
• K.1.16 Difference
• K.1.17 Number to Word
• K.1.18 Translation English-German
• K.1.19 Translation English-Spanish
• K.1.20 Translation English-French
• K.1.21 Sentiment Analysis
• K.1.22 Sentence Similarity
• K.1.23 Word in Context
• Appendix L Ablations
• Instructions for reporting errors