ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generation

arxiv:2304.05977

Original source · Paper JSON

Self-supervised learning and vision · 2023-04-12

Cites (1) · Cited by (0)

Complete cleaned Markdown · Source Markdown including bibliography

Read in parts

1. Part 1
2. Part 2
3. Part 3
4. Part 4
5. Part 5

Contents

• ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generation
• Abstract
• 1 Introduction
• 2 ImageReward: Learning to Score and Evaluate Human Preferences
• 2.1 Annotation Pipeline Design
• 2.2 RM Training
• 2.3 As Metric: Re-Evaluating Human Preferences on Text-to-Image Models
• 3 ReFL: Reward Feedback Learning Improves Text-to-Image Diffusion
• 4 Experiment
• 4.1 ImageReward: On Human Preference Prediction
• 4.2 ReFL: On Improving Diffusion Models with Human Preference
• 5 Related Work
• 6 Conclusion
• Acknowledgement
• Appendix A Details on ImageReward’s Comparison Data Annotation Pipeline
• A.1 Prompt Selection
• A.2 Annotation Management
• A.3 Human Annotation Design
• A.4 Human Annotation Analysis
• Appendix B Annotation Document
• B.1 Annotate the input text
• B.2 Annotate generated images
• B.3 Rank generated images
• B.4 Frequently asked questions
• Appendix C More Analysis on ImageReward’s Performance and Properties
• C.1 Average Scores of the Highest/Lowest Ranked Images
• C.2 Recall/Filter the Best/Worst Image
• Appendix D Comparison between ImageReward and Other Reward Models
• D.1 Human Evaluation
• D.2 Train Set Distribution
• Appendix E Implementation Details of Related LDM Optimization Methods
• Appendix F More Results of ReFL
• F.1 Demonstrations of ReFL
• F.2 ReFL compares other fine-tuning methods
• Appendix G Additional Results of ImageReward Compared to Other Typical Image Scorers
• Appendix H Limitations
• Appendix I Broader Impact
• Appendix J Reproducibility
• Instructions for reporting errors