Self-Refine: Iterative Refinement with Self-Feedback

arxiv:2303.17651

Original source · Paper JSON

Post-training · 2023-03-30

Cites (1) · Cited by (21)

Complete cleaned Markdown · Source Markdown including bibliography

Read in parts

1. Part 1
2. Part 2
3. Part 3
4. Part 4
5. Part 5

Contents

• Self-Refine : Iterative Refinement with Self-Feedback
• Abstract
• 1 Introduction
• 2 Iterative Refinement with Self-Refine
• Initial generation
• feedback
• refine
• 3 Evaluation
• 3.1 Instantiating Self-Refine
• Base LLMs
• 3.2 Metrics
• 3.3 Results
• 4 Analysis
• The impact of the feedback quality
• How important are the multiple iterations of feedback - refine ?
• Can we just generate multiple outputs instead of refining?
• Does Self-Refine work with weaker models?
• Qualitative Analysis
• Going Beyond Benchmarks
• 5 Related work
• Source of feedback.
• Representation of feedback.
• Types of refiners.
• Non-refinement reinforcement learning (RL) approaches.
• 6 Limitations and Discussion
• 7 Conclusion
• Appendix A Evaluation Tasks
• Appendix B Broader Related Work
• Comparison with Welleck et al. (2022)
• Comparison with non-refinement reinforcement learning (RL) approaches.
• Appendix C Human Evaluation
• Appendix D GPT-4 Evaluation
• Appendix E Model Key
• Appendix F Comparison of Self-Refine with State-of-the-art of Few-Shot Learning Models and Fine-Tuned Baselines
• Appendix G Evaluation of Vicuna-13b
• Mixed-refine: Improving Vicuna-13b with ChatGPT
• Appendix H Additional Analysis
• H.1 Using Oracle Feedback
• Non-monotonic increase in output quality for acronym generation
• Feedback and Refinement Error Analysis for Response Generation
• Appendix I Beyond Benchmarks
• Ice Cream Generation
• Photosynthesis
• Appendix J Statistical Confidence Intervals
• Appendix K New Tasks
• Constrained Generation
• Acronym Generation
• Appendix L Code Readability
• L.1 Method
• L.2 Experiments
• Dataset
• Implementation
• Evaluation Methods
• Result
• Example
• Appendix M Dialogue Response Generation
• M.1 Modules
• Init
• Feedback
• Iterate
• M.2 Setup and Experiments
• Model and Baseline
• Evaluation
• Results
• Appendix N Code Optimization
• Appendix O Math Reasoning
• Appendix P Sentiment Reversal
• Instantiating Self-Refine for sentiment reversal
• P.1 Details
• Evaluation
• Pin-pointed feedback
• Evaluation
• Appendix Q Acronym Generation
• feedback
• Appendix R Constrained Generation
• Appendix S Prompts
• Instructions for reporting errors