Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicine

arxiv:2311.16452

Original source · Paper JSON

Retrieval, embeddings, benchmarks · 2023-11-28

Cites (3) · Cited by (0)

Complete cleaned Markdown · Source Markdown including bibliography

Read in parts

1. Part 1
2. Part 2
3. Part 3

Contents

• Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicine
• Abstract
• 1 Introduction
• 2 Background
• 2.1 Foundation Models on Medical Challenge Problems
• 2.2 Prompting Strategies
• 3 Experimental Design
• 3.1 Datasets
• 3.2 Sound Testing Methodology
• 4 Power of Prompting: Exploration and Results
• 4.1 Dynamic Few-shot
• 4.2 Self-Generated Chain of Thought
• 4.3 Choice Shuffling Ensemble
• 4.4 Putting it all together: Medprompt
• 5 Results
• 5.1 Performance on Eyes-Off Data
• 5.2 Insights about Medprompt Components via Ablation Studies
• 5.3 Generalization: Cross-Domain Exploration of Medprompt
• 6 Limitations and Risks
• 7 Summary and Conclusions
• Acknowledgments