The Truth is in There: Improving Reasoning in Language Models with Layer-Selective Rank Reduction — part 4/4

Cleaned-text code points [46502, 52331). Bibliography omitted here; source.md includes it.

We use the validation split of the Truthful QA dataset. The truthful QA dataset consists of multiple choice questions. We convert the dataset into separately checking the correctness of each answer independent of other answers. Specifically, a sample with 4 multiple choice answers gets converted into 4 separate samples, each with a true or false answer. We convert each question and answer pair into the prompt “<question> <answer>. Is this statement true or false. This statement is” if the answer does not end with period (.), otherwise, we convert it into “<question> <answer> Is this statement true or false. This statement is” . The prompt variables <question> and <answer> are replaced by the original question and answer respectively. The processed dataset consists of 5,882 samples.

##### BigBench Wikidata QA.

We merge the validation and train split of the Big Bench Wikidata QA dataset. We filter out examples where the number of target labels are more than 1. This gives us a dataset of size 20,321. This QA task has an open-ended set of labels.

### E.2 Details for Computing Accuracy and Log Loss

The procedure used to compute accuracy and log loss varies across the different datasets. Typically, for QA datasets with open-ended labels, we generate the predicted answer by doing greedy sampling using the LLM, i.e., with temperature set to 0. We report the prediction as correct if and only if the answer is in the generated text. We lower case and strip whitespaces before comparing text. We call this the generation accuracy . In contrast, for datasets with a small set of possible label choice, we predict the label with the highest probability under the LLM and report the prediction as correct if and only if the predicted label is the correct label. We call this the classification accuracy .

As Roberta is a masked language model, we do generation by creating a prompt with <mask> tokens, and predicting these masked tokens. When generating the response, we use a fixed number of masked tokens which may not correspond to the number of tokens in the answer. However, when computing the log-loss of the answer, we add as many masked tokens as the number of tokens in the answer, and compute the log probabilities of the answer tokens under the model corresponding to these masked tokens.

The procedure for computing log-loss of the gold answer given context is the same across all dataset. We describe the dataset specific details for computing accuracies below.

##### CounterFact.

We use generation accuracy to evaluate success. For GPT-J and Roberta, we generate a single token as all labels are single token long, whereas for Llama we generate up to 10 tokens.

##### HotPotQA.

We use generation accuracy to evaluate success. For GPT-J and Llama2, we generate up to 15 tokens. For Roberta we only use 5 tokens since Roberta struggles to fill-in more than a few masked tokens, this is understandable as Roberta is trained by masking out a small number of tokens (typically 15% tokens), and using these models to predict a large number of masked tokens then suffers from distributional shift.

##### Fever.

We use classification accuracy to measure success. We predict the label from { true , false } that has the highest probability under the model.

##### Bios Gender.

We use classification accuracy to measure success. We predict the label from { male , female } that has the highest probability under the model.

##### Bios Profession.

We use classification accuracy to measure success. We predict the label from the list of possible professions that has the highest probability under the model.

##### BigBench Epistemic Reasoning.

We use classification accuracy to measure success. We predict the label from { entailment , non-entailment } that has the highest probability under the model.

##### TruthfulQA.

We use classification accuracy to measure success. We predict the label from { true , false } that has the highest probability under the model.

##### BigBench WikidataQA.

As the set of labels are open-ended, we compute the accuracy using generation similar to CounterFact. For GPT-J and Llama2 we generate up to 10 tokens, whereas for Roberta we generate 5 tokens.

### E.3 Code

We use PyTorch for all experiments. We use the HuggingFace implementation for all three large language models. We use Llama2 7GB weights provided by Meta. We use the SVD implementation available in PyTorch for experiments. The code can be found at: https://github.com/pratyushasharma/laser

### E.4 Compute Details

We ran each experiment on a cluster with V100 and A2600 GPUs. Each experiment took about 1-3hrs to finish. For all settings, we search over hyperparameters listed in Table 5 . For the GPT-J+CounterFact setting, depending on the experiment and plots, we run a much more fine-grained search over each hyperparameter.

## Instructions for reporting errors

We are continuing to improve HTML versions of papers, and your feedback helps enhance accessibility and mobile support. To report errors in the HTML that will help us improve conversion and rendering, choose any of the methods listed below:

Click the "Report Issue" ( ) button, located in the page header.

Tip: You can select the relevant text first, to include it in your report.

Our team has already identified the following issues . We appreciate your time reviewing and reporting rendering errors we may not have found yet. Your efforts will help us improve the HTML versions for all readers, because disability should not be a barrier to accessing research. Thank you for your continued support in championing open access for all.

Have a free development cycle? Help support accessibility at arXiv! Our collaborators at LaTeXML maintain a list of packages that need conversion , and welcome developer contributions .