Artifacts or Abduction: How Do LLMs Answer Multiple-Choice Questions Without the Question? — part 3/4

Cleaned-text code points [31503, 47390). Bibliography omitted here; source.md includes it.

Several works quantify memorization via data extraction attacks Carlini et al. (2020) ; Ishihara (2023) ; Sainz et al. (2023) and analyzing models after dataset-specific training Huang et al. (2022) ; Zhou et al. (2023) . In contrast, we design the first partial-input memorization prompts for MCQA.

While many works study how humans make informed or cued MCQA guesses via choice dynamics Royal and Hedgpeth (2015) ; Royal and Stockdale (2017) , we are the first to define similar abilities (i.e. meta-strategies) with LLMs. Other studies assess if LLM pretraining priors can recognize tasks Pan et al. (2023) and sensitivity to option order in MCQA Pezeshkpour and Hruschka (2023) , but neither work focuses on dataset artifacts.

Lastly, prior works explore generating entire MC questions Ch and Saha (2018) , but we are the first to use LLMs to infer the original question from the choices. Other MC strategies have been used with LLMs, such as process of elimination Ma and Du (2023) ; Balepur et al. (2023) , where wrong options are removed, and Maieutic Prompting Jung et al. (2022) , involving reasoning over abductive explanations, but not in relation to artifact exploitation.

## 8 Conclusion

We find that LLMs can perform MCQA without access to the question, a result that can be achieved through few-shot prompting with minimal examples. In pursuing explanations for these results, we release an in-depth black-box evaluation suite, based on our proposed partial-input prompts. As part of our analysis, we show that LLMs still exhibit reasoning abilities in choices-only settings via the meta-strategy of question inference, suggesting that partial-input abilities of LLMs should not be completely attributed to surface-level shortcuts linked to artifacts. Along with software and analysis, we suggest ways to evaluate LLMs on MCQA as intended, where both the question and choices are needed, including: 1) reporting stronger baselines like our proposed choices-only prompt; and 2) designing robust data creation protocols. In light of our findings, we advocate for more critical discussions around LLM evaluation, allowing us to better understand and interpret the model capabilities our benchmarks are truly assessing.

## 9 Limitations

One limitation is that our experiments are conducted in a black-box rather than a white-box manner, motivated by the increasing use of closed-source LLMs. This setup allows our analysis to be applicable to any LLM that can provide generated text outputs through prompting. However, we acknowledge that an analysis of logits from open-source LLMs could offer more insights into aspects like confidence, uncertainty, and calibration in artifact exploitation. We believe these directions would be very interesting extensions of our work.

Our experiments also do not use closed-source models like ChatGPT due to resource constraints. Based on the large number of prompts we study, we need to run inference over each MCQA dataset 16 times. This is a very expensive endeavor, as two of our datasets, MMLU and HellaSwag, both have over 10,000 questions in the test set. However, all of our experiments are designed in a black-box fashion and will be made open-source. Thus, we hope if members of the NLP community with access to more resources are interested, they can replicate our experiments with closed-source LLMs.

Lastly, LLMs are sensitive to prompts Min et al. (2022) and hyperparameters, so it is possible that varied configurations could alter the accuracy in our experiments. Our prompts and hyperparameters were just the first ones we tried, so we believe that LLMs can likely be optimized to obtain even higher choices-only accuracy. Future works can explore these alterations more in-depth and analyze LLM sensitivity in choices-only settings.

## 10 Ethical Considerations

Through few-shot prompting, we discover that LLMs can achieve high choices-only accuracy in prominent MCQA benchmark datasets. However, our goal is not to discount the adoption of these datasets for evaluation and suggest that they are fundamentally flawed, or discredit the performance of LLMs on these datasets and claim that they are incapable of performing MCQA. Instead, we wish to provide more transparency for MCQA benchmarking and LLM decision-making, which we hope will inspire future work on: 1) better understanding LLM decision-making in MCQA; 2) reporting choices-only baselines as a stronger alternative to majority class baselines on MCQA evaluation suites; and 3) designing more resilient benchmarks that limit the influence of artifacts.

## 11 Acknowledgements

We would like to thank members of the CLIP lab at the University of Maryland and external collaborators for their feedback and discussions of this work, including Yu (Hope) Hou, Dayeon (Zoey) Ki, Neha Srikanth, Rupak Sarkar, Shi Feng, and Jordan Boyd-Graber. We are also grateful for our discussions with Hailey Schoelkopf, Stella Biderman, and other members of the EleutherAI community. We also thank the anonymous reviewers for their feedback. This material is based upon work supported by the National Science Foundation Graduate Research Fellowship Program under Grant No. DGE 2236417. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.

## Appendix A Experimental Setup

### A.1 Dataset Details

All datasets were obtained from their original, publicly-available sources. These datasets are free to use for research purposes, so our research falls within their intended use. ARC uses 25 training examples (20 for Falcon) and 1165 evaluation examples; MMLU uses 5 training examples and 14032 evaluation examples; and HellaSwag uses 10 training examples and 10042 evaluation examples.

### A.2 LLM Inference

For LLaMA, Falcon, and Mixtral we allocated 24 GPU hours for almost every prompt format and dataset, and used 8 NVIDIA:RTXA5000 GPUs. The notable exception is the two-step inferring the question strategy, which required 48 GPU hours for LLaMA and Falcon, and 96 GPU hours for Falcon, per model and dataset. Phi required around 60 total GPU hours to run all prompt formats per dataset, and used a single NVIDIA:RTXA4000 GPU.

The only parameters changed are: a minimum token generation length of 5, a maximum token generation length of 200, and a stopping criteria when the LLM begins to generate the next few-shot exemplar. For LLaMA-2 on MMLU, we use 8-bit quantization. We did not perform hyperparameter search. All results are obtained from a single run.

### A.3 Prompt Creation

Given the sensitivity of LLMs to prompt formats, we followed best practices when creating our few-shot prompts. For ARC, we randomly sample 25 examples over the entire training set. The HellaSwag dataset stems from two sources: ActivityNet and wikiHow. Thus, the 10-shot prompt contains 5 randomly sampled examples from the ActivityNet portion of the dataset, and 5 randomly sampled examples from the wikiHow portion. Further, we ensure that each few-shot prompt has a balanced distribution of answer letters and that the order of exemplars is randomized.

All prompts shown in this paper are derived from these few-shot examples, following the format shown in the prompt boxes. Two slight exceptions are when the LLM performs the second step of AQI (Prompt 6.1 ) and answers a random question (Prompt 6.1 ). In these prompts, the questions in the few-shot examples are the same as the questions from the Full Prompt (Prompt 2.1 ), rather than using model-generated and random questions (respectively), ensuring a fair comparison.

### A.4 Prompt Box Example

Below, we provide a detailed example to illustrate the application of our prompt boxes. Suppose we have the full prompt (Prompt 2.1 ) defined in section 2.1 :

In the above prompt, the LLM uses the question q q and choices 𝒞 \mathcal{C} as input and is asked to generate the letter of the answer a a . Suppose we have 5 few-shot examples, with questions q 1 q_{1} ,…, q 5 q_{5} , list of choices 𝒞 1 \mathcal{C}_{1} , …, 𝒞 5 \mathcal{C}_{5} , and ground truth answers a 1 a_{1} , …, a 5 a_{5} . The expanded few-shot prompt for the prompt box is written below:

Using this prompt, the LLM must generate a a , which is the highlighted text in the prompt box.

### A.5 Output Validity Statistics

In Table 4 , we display the output validity statistics (i.e. if the model selects a valid letter) of each prompt format, LLM, and dataset used in our experiments. We find that in 100/108 cases, the LLMs produce valid outputs over 95% of the time. This suggests that our few-shot prompt setup typically enables the LLM to produce valid outputs.

Further, we display the results from our experiments if invalid outputs are always marked as incorrect in Figures 13 , 14 , and 15 . We believe either decision regarding the treatment of invalid options is valid, but we opted for the score of 0.25 to account for task misunderstanding stemming from our atypical prompt formats. Regardless, we provide both versions of the experiments so readers can comprehensively evaluate the LLMs.

### A.6 Metrics

Accuracy was computed with numpy. 7 7 7 https://numpy.org/ Cohen’s κ \kappa was computed with scikit-learn. 8 8 8 https://scikit-learn.org/ All metrics are reported from a single run.

## Appendix B Detailed Results

### B.1 Scaling Laws

In Figure 6 , we perform the same initial experiments on choices-only prompts in section 3 with LLaMA-2 models of three sizes: 7B, 13B, and 70B. Overall, we find that as model size increases, the accuracy of the choices-only prompt also increases. This again suggests that larger models may exploit artifacts to a higher degree.

### B.2 Decoding Strategies

In our experiments, we use default parameters to show the out-of-box results of our tested LLMs. To study the effect of decoding strategies, we re-run our choices-only, memorization, and individual prior experiments with LLaMA on all datasets using greedy decoding, displayed in Figures 7 , 8 , and 9 , respectively. We find that our results are mostly consistent, suggesting that the choice of decoding strategy does not significantly influence the results. The one notable difference is that the individual choice-only prompt results in a higher accuracy with greedy decoding on all the datasets. Despite this increase, the individual choice classification still does not fully explain the accuracy of the choices-only prompt, which suggests that LLMs may still use group dynamics and meta-strategies, even with greedy decoding.

### B.3 Partial-input Models vs Prompting

We compare the performance of our partial-input prompts versus a trainable partial-input model. We train the seq2seq T5-Large model Raffel et al. (2019) to perform MCQA using the same input sequence format as the full prompt (Prompt 2.1 ) and the choices-only prompt (Prompt 3.1 ). T5 is independently trained on ARC and HellaSwag using 90% of the training set for training (with default parameters) and the other 10% for validation.

The T5 model has less pretraining compared to our LLMs, so a trainable T5 choices-only baseline mainly exploits annotation artifacts (e.g. common words used by crowdworkers), rather than artifacts from pretraining (i.e. world knowledge). If an LLM with the choices-only prompt can outperform T5, it would signify that the LLM is instead leveraging knowledge from pretraining.

In Figure 10 , we find that on ARC, T5 is unable to exploit artifacts in the choices-only setting, achieving an accuracy very close to the majority class baseline. This indicates that our tested LLMs can achieve high choices-only accuracy on ARC due to their extensive pretraining, rather than leveraging statistical dataset correlations. On HellaSwag, however, there is a large presence of dataset artifacts, indicated by the high choices-only accuracy of T5 of 0.721. Future works can conduct more in-depth analysis to try to uncover how pretraining relates to artifact exploitation.

### B.4 Falcon Individual Evaluation

In Figure 11 , we show the accuracy of the individual versus group choice experiments from section 5.3 . Falcon performs very poorly on the independent classification task, achieving scores near the majority class baseline in ARC and HellaSwag. Falcon is one of the weaker models we test with, so we attribute this behavior to the LLM’s less robust overall capabilities compared to our other LLMs.

### B.5 One-Step Question Inference

Figure 12 displays the accuracy of the two-step versus one-step strategy of abductive question inference ( section 6 ); in the latter, the generated question acts as a chain-of-thought Wei et al. (2022) . We find that the one-step strategy consistently underperforms its two-step version, signifying that instructing an LLM to try to infer the question and then answer this question in one step is too difficult of a task.

### B.6 Qualitative Analysis

In this section, we provide more details regarding the annotation procedure details ( section B.6.1 ) as well as additional results that we could not fit into the main body of the paper ( section B.6.2 ).

#### B.6.1 Annotation Procedure Details

In Q1, we verify if the model’s selected answer is correct using external tools such as Google search. Further, when the model generates an ambiguous question that is answerable by many choices (e.g. “Which of these is a liquid?” with choices “milk” and “water”), if the model selects any of the correct options we mark it as correct.

In Q2, when we study if the inferred questions match the original question, we define two questions as having the same meaning if they test the same knowledge. To illustrate, in one instance, the generated question is “ If there is a high demand for lumber and a low supply of trees, what will most likely happen? ” and the original question is “ Logging companies cut trees in a forest and send the trees to lumber mills far from the forest. The mills make boards that are used for construction. Some logging companies do not plant tree seedlings after cutting trees. Not planting tree seedlings might affect people who need boards in the future because .” We consider these two questions to be equivalent because they both fundamentally ask the model to reason about what happens if demand for lumber increases but supply decreases.

All annotators in Q1, Q2, and Q3 are the authors, ensuring that the annotations are high-quality.

#### B.6.2 Additional Results

In Table 6 , we provide the qualitative evaluation results Q1 and Q2 with all of our tested LLMs (LLaMA, Falcon, Mixtral, and Phi) on ARC. We note that all models showcase a similar trend discussed in section 6.2 , but Phi is notably much worse at inferring the question. This is expected, given that Phi is the smallest LLM used in our experiments.

We now give further analyses for LLaMA on ARC regarding when AQI is effective (Q3). First, the breakdown of our three cases for the generated question is: 15% as unanswerable, 28% as answerable and matching the original question, and the remaining 57% as answerable and not matching the original question. In terms of question discriminability, we find that 15% can be answered by none of the choices (i.e. unanswerable), 65% can be answered by exactly one of the choices (i.e. perfectly discriminable), and the last 20% can be answered by more than one choice (i.e. ambiguous).

Finally, we provide examples of questions generated from AQI. In Tables 7 and 8 , we show generated questions when inferring the question succeeds, and the generated question matches and does not match the original question, respectively. In Table 9 , we show generated questions when inferring the question fails, but the question is still answerable.

## Appendix C Related Work on MCQA Evaluations