Physics of Language Models: Part 2.1, Grade-School Math and the Hidden Reasoning Process — part 6/6
Cleaned-text code points [76962, 81876). Bibliography omitted here; source.md includes it.
Accuracy statistics. Our main accuracies are presented in Figure 3 , where each entry is averaged over 4096 math problems of that type. Our accuracies are not simply from comparing the answer integers (between 0 and 22); instead we have written a parser to make sure the model’s intermediate solution steps are fully-correct.
For the “redundancy” experiment Figure 4 , we tested each model again with 4096 math problems in each case and presented the results among fully-correct solutions. For this figure, we present beam=1 for cleanness and the results for beam=4 are almost completely identical.
For the “depth matters” experiment Figure 9 , because we care about the (relatively small) accuracy differences across models, we pretrain using two different random seeds, and evaluate with both beam=1/4 ; we then present the best accuracies in each entry with respect to the 2 seeds and 2 beam choices. The accuracies are again over 4096 math problems.
### F.2 V-probing
Our V-probing was first introduced in Section 4.1 with more details given in Section B . It is a fine-tuning process upon the pretrained language model, with an additional linear head on the output layer, and a small rank- r r update on the input (embedding) layer. The pretrained model is freezed, and only this linear head and the rank- r r update are trainable parameters during the fine-tuning.
Recall we use r = 8 r=8 in this paper (in contrast, the hidden dimension of GPT-12-12 is 768). This small value of r r ensures if probing accuracy is high, it mostly comes from the pretrained model and not the additional trainable parameters.
For V-probing, we use the same configurations as pretrain, except that:
• For V-probing on the iGSM-med datasets, we used a learning rate of 0.002 0.002 (with no ramp-up, linear decay down to 0), weight decay of 0.01 0.01 , batch size of 256, and trained for 100,000 100,000 steps.
• For V-probing on the iGSM-hard datasets, we used a learning rate of 0.002 0.002 (with no ramp-up, linear decay down to 0), weight decay of 0.01 0.01 , batch size of 128, and trained for 100,000 100,000 steps.
V-probing statistics. In Figure 7(a) , Figure 7(b) , Figure 12 , Figure 8(a) , and Figure 8(b) , we tested at least 4096 random problem-parameter pairs in each cell . In Figure 8(a) and Figure 8(b) , when evaluating probing results on GPT-2 model’s generated correct or wrong solutions, we used beam=1 and dosample=False (greedy) for generation. (Results are similar for beam=4 .)
In our layer-wise nece ( A ) \texttt{nece}(A) probing experiments ( Figure 10 and Figure 14 ), we tested at least 73728 random problem-parameter pairs in each case and then divided the results into bins based on the parameter A A ’s distances to the queries.
## Appendix G Failure Examples on GPT-4 / GPT-4o
In Figure 2 , we conduct few-shot experiments using the latest versions of GPT-4 turbo (2024-04-09) and GPT-4o (2024-05-13) models to evaluate their accuracies on our iGSM-med p q \textsf{iGSM-med}_{pq} dataset, with respect to different op ∈ { 2 , 3 , … , 20 } \textsf{op}\in\{2,3,\dots,20\} .
To ensure meaningful evaluation: • We replaced mod 23 \bmod{23} with mod 5 \bmod{5} to ensure that any errors are not due to arithmetic mistakes. We also provided a few arithmetic computation examples.
• We minimized English diversity to ensure that any errors are not due to misunderstanding the problem description. Specifically, – We fixed a simple categorization ( School , Classroom , Backpack , Stationerys ) (\textsf{School},\textsf{Classroom},\textsf{Backpack},\textsf{Stationerys}) , with only four items in each category.
– We provided an English background paragraph to fully describe the structure graph (i.e., which item has which subitem), as well as the number of items in each category. The math problem is preceded by this background paragraph.
• We provided five-shot problem/solution examples to ensure that GPT-4 understands how to solve such math problems step by step.
We did not verify each step of GPT-4’s solution but checked if the final output number (between 0 and 4) matched the correct answer. The accuracy results are presented in Figure 2 . It shows that the GPT-4o model is almost randomly guessing for op ≥ 11 \textsf{op}\geq 11 , and GPT-4 turbo for op ≥ 9 \textsf{op}\geq 9 .
Furthermore, Figure 15 shows that when the GPT-4/4o models fail to answer the math problems, it is mostly not due to format errors or misunderstanding of the problem. Instead, just like what we discovered in Section 5 , GPT-4/4o fail also because they compute unnecessary parameters (i.e., nece ( A ) = false \texttt{nece}(A)=\mathrm{false} ) or compute parameters that are not yet ready to be computed (i.e., can_next ( A ) = false \texttt{can\_next}(A)=\mathrm{false} ). This further confirms that our findings do connect to practice, regarding the model’s hidden reasoning process.