Self-Refine: Iterative Refinement with Self-Feedback — part 5/5

Cleaned-text code points [61530, 66278). Bibliography omitted here; source.md includes it.

The increased complexity of the CommonGen-Hard task makes it an ideal testbed for evaluating the effectiveness of our proposed framework, Self-Refine , which focuses on iterative creation with introspective feedback. Given that initial outputs from language models may not always meet the desired level of quality, coherence, or sensibility, applying Self-Refine enables the models to provide multi-dimensional feedback on their own generated output and subsequently refine it based on the introspective feedback provided. Through iterative creation and self-reflection, the Self-Refine framework empowers language models to progressively enhance the quality of their output, closely mimicking the human creative process and demonstrating its ability to improve generated text on complex and demanding natural language generation tasks like CommonGen-Hard ( Figure 18 ).

## Appendix S Prompts

We include all the prompts used in the experiments in Figures 19 - 38 : • Acronym Generation: Figures 19 - 21

• Code Optimization: Figures 22 - 24

• Code Readability Improvement: Figures 25 - 26

• Constrained Generation: Figures 27 - 29

• Dialogue Response Generation: Figures 30 - 32

• Math Reasoning: Figures 33 - 35

• Sentiment Reversal: Figures 36 - 38

Recall that the Base LLM requires a generation prompt p g ​ e ​ n p_{gen} with input-output pairs ⟨ x i , y i ⟩ \langle x_{i},y_{i}\rangle , the feedback module requires a feedback prompt p f ​ b p_{fb} with input-output-feedback triples ⟨ x i , y i , f ​ b i ⟩ \langle x_{i},y_{i},fb_{i}\rangle , and the refinement module ( refine ) requires a refinement prompt p r ​ e ​ f ​ i ​ n ​ e p_{refine} with input-output-feedback-refined quadruples ⟨ x i , y i , f ​ b i , y i + 1 ⟩ \langle x_{i},y_{i},fb_{i},y_{i+1}\rangle .

• Sentiment Reversal We create positive and negative variants of a single review from the training set and manually write a description for converting the negative variant to positive and vice versa. For each variant, the authors generate a response and create a feedback f ​ b i fb_{i} based on the conversion description.

• Dialogue Response Generation We sample six examples as ⟨ x i , y i ⟩ \langle x_{i},y_{i}\rangle for the few-shot prompt for the Base LLM. For each output y i y_{i} , the authors create a response, evaluate it based on a rubric to generate f ​ b i fb_{i} , and produce an improved version y i + 1 y_{i+1} .

• Acronym Generation We provide the Base LLM with a total of 15 (title, acronym) examples. Then, for one title ( x i x_{i} ) we generate an acronym ( y i y_{i} ) using ChatGPT. The authors then score the acronyms based on a 5-point rubric to create the corresponding f ​ b i fb_{i} , and write improved versions of the acronym to create y i + 1 y_{i+1} . 3 such examples are used for refine and feedback .

• Code Optimization We use the slow ( x i x_{i} ) and fast ( y i y_{i} ) versions of programs released by Madaan et al. (2023) for Base LLM. We use their provided explanations ( Madaan et al., 2023 ) for feedback and refine .

• Math Reasoning The prompts for the Base LLM are sourced from PaL ( Gao et al., 2022 ) as ⟨ x i , y i ⟩ \langle x_{i},y_{i}\rangle . We select two examples from the training set on which Codex fails when prompted with PaL-styled prompts, and manually write the correct solution ( y i + 1 y_{i+1} ) and reasoning ( f ​ b i fb_{i} ) for refine and feedback .

• Constrained Generation We provide ten examples to the Base LLM as ⟨ x i , y i ⟩ \langle x_{i},y_{i}\rangle . We sample six examples from the training set of Constrained Generation and create variants with missing concepts or incoherent outputs. The missing concepts and the reason for incoherence form f ​ b fb .

• TODO: Add relevant information for the remaining task.

## Instructions for reporting errors

We are continuing to improve HTML versions of papers, and your feedback helps enhance accessibility and mobile support. To report errors in the HTML that will help us improve conversion and rendering, choose any of the methods listed below:

Click the "Report Issue" ( ) button, located in the page header.

Tip: You can select the relevant text first, to include it in your report.

Our team has already identified the following issues . We appreciate your time reviewing and reporting rendering errors we may not have found yet. Your efforts will help us improve the HTML versions for all readers, because disability should not be a barrier to accessing research. Thank you for your continued support in championing open access for all.

Have a free development cycle? Help support accessibility at arXiv! Our collaborators at LaTeXML maintain a list of packages that need conversion , and welcome developer contributions .