Emergent Analogical Reasoning in Large Language Models — part 3/5

Cleaned-text code points [30873, 46176). Bibliography omitted here; source.md includes it.

It is certainly not the case that GPT-3 mimics human analogical reasoning in all respects. Our tests were limited to processes that can be carried out within a local temporal context, but humans are also capable of retrieving potential source analogs from long-term memory, and ultimately of developing new concepts based on the comparison of multiple analogs. Unlike humans, GPT-3 does not have long-term memory for specific episodes. It is therefore unable to search for previously-encountered situations that might create useful analogies with a current problem. For example, GPT-3 can use the general story to guide its solution to the radiation problem, but as soon as its context buffer is emptied, it reverts to giving its non-analogical solution to the problem – the system has learned nothing from processing the analogy. GPT-3’s reasoning ability is also limited by its lack of physical understanding of the world, as evidenced by its failure (in comparison with human children) to use an analogy to solve a transfer problem involving construction and use of simple tools. GPT-3’s difficulty with this task is likely due at least in part to its purely text-based input, lacking the multimodal experience necessary to build a more integrated world model [ 47 ] . Finally, we found GPT-3 was limited in its ability to evaluate analogies based on causal relations, particularly in cross-domain comparisons between stories (far analogy).

But despite these major caveats, our evaluation reveals that GPT-3 exhibits a very general capacity to identify and generalize – in zero-shot fashion – relational patterns to be found within both formal problems and meaningful texts. These results are extremely surprising. It is commonly held that although neural networks can achieve a high level of performance within a narrowly-defined task domain, they cannot robustly generalize what they learn to new problems in the way that human learners do [ 48 , 49 , 50 , 6 ] . Analogical reasoning is typically viewed as a quintessential example of this human capacity for abstraction and generalization, allowing human reasoners to intelligently approach novel problems zero-shot. Our results indicate that GPT-3 – unlike any other neural network previously tested on analogy problems – displays a capacity for such zero-shot analogical reasoning across a broad range of tasks.

The deep question that now arises is how GPT-3 achieves the analogical capacity that is often considered the core of human intelligence. One possibility is that, perhaps as a result of the sheer size and diversity of GPT-3’s training data, it has been forced to develop mechanisms similar to those thought to underlie human analogical reasoning – despite not being explicitly trained to do so. The consensus among cognitive scientists working on analogy is that this human ability depends on systematic comparison of knowledge based on explicit relational representations. It is unclear whether and how GPT-3 would implement these processes. Does GPT-3 possess some form of emergent relational representations, and if so, how are they computed? Does it perform a mapping process similar to the type that plays a central role in cognitive theories of analogy [ 43 ] ?

A few properties of the transformer architecture [ 29 ] , on which GPT-3 and other large language models are based, are worth considering here. The first is the central role played by similarity . Transformers are built on a self-attention operation, which involves explicitly computing the similarity between each pair of vectors in the inputs to each layer. This pairwise evaluation of similarity is also a key feature of cognitive models of analogy, where it provides the primary constraint guiding the process of analogical mapping. In traditional symbolic models [ 51 ] , this takes the form of literal identicality between symbols, but in more recent models [ 52 , 53 ] , a graded similarity function that operates over vector-based inputs is used, much like the self-attention operation in transformers. Second, transformer self-attention employs a form of indirection , in which one set of embeddings is used to reference another set of embeddings (i.e., keys vs. values) – arguably a form of variable binding. Cognitive scientists have long hypothesized that variable binding plays a central role in analogical reasoning, and abstract reasoning more broadly, as it potentially allows generalization of abstract roles across different contexts [ 54 , 55 , 48 , 56 , 57 , 58 ] . It may be that these features of the transformer make it better equipped to perform zero-shot reasoning than other neural architectures. This possibility aligns with recent evidence that the transformer architecture is an important factor contributing toward the emergence of few-shot learning [ 27 ] .

But although the mechanisms incorporated into large language models such as GPT-3 may have some important links to building blocks of human reasoning, we must also entertain the possibility that this type of machine intelligence is fundamentally different from the human variety. Humans have evolved to reason within bounds imposed by limited computational power and biological constraints [ 59 ] . Thus, we tend to approach complex problems by breaking them into a set of simpler problems that can be solved separately [ 60 ] , an approach that plays a particularly important role in solving challenging analogy problems such as Raven’s Matrices [ 61 ] . It is possible that GPT-3, through sheer computational scale, is able to solve such complex problems in a holistic and massively parallel manner, without the need to segment them into more manageable components.

It must also be noted that, regardless of the extent to which GPT-3 employs human-like mechanisms to perform analogical reasoning, we can be certain that it did not acquire these mechanisms in a human-like manner. LLMs receive orders of magnitude more training data than do individual human beings (at least if we consider linguistic inputs alone) [ 59 ] , and so they cannot be considered as models of the acquisition of analogical reasoning over the course of human development. Nor can they be considered good models of the evolution of analogical reasoning, as their analogical abilities are derived entirely from being trained to predict human-generated text. Human natural language is replete with analogies; accurately predicting natural language therefore likely requires an ability to appreciate analogies. But there is no reason to suppose that the same system, absent human-generated inputs, would spontaneously develop a disposition to think analogically, as apparently happened at some point in human evolution [ 62 ] . Thus, to the extent that large language models capture the analogical abilities of adult human reasoners, their capacity to do so is fundamentally parasitic on natural human intelligence. Nevertheless, the present results indicate that this approach may be sufficient to approximate human-like reasoning abilities, albeit through a radically different route than that taken by biological intelligence.

## 4 Methods

The present research complied with all relevant ethical regulations, and human behavioral experiments were approved by the UCLA Institutional Review Board (IRB protocol #22-000841, approved May 17, 2022).

### 4.1 Code

Most code was written in Python v3.9.6, using the following packages: NumPy v1.24.3 [ 63 ] , SciPy v1.10.1 [ 64 ] , statsmodels v0.13.5 [ 65 ] , Matplotlib v3.7.1 [ 66 ] , and pandas v2.0.1 [ 67 ] . Logistic regression analyses were carried out in R v4.2.2 [ 68 ] . Experimental stimuli for human behavioral experiments were written in JavaScript using jsPsych v7.2.1 [ 69 ] .

### 4.2 GPT-3

We queried GPT-3 in an automated fashion through the OpenAI API. All simulations reported in the main text employed the text-davinci-003 model variant. Additional simulations, reported in the Supplementary Results, also employed the davinci, code-davinci-002, and text-davinci-002 variants. The temperature was set to 0 in all simulations. We set max_tokens (the parameter controlling the maximum number of generated tokens for a given prompt) to 10 for Digit Matrices, 40 for letter string analogies, 10 for four-term verbal analogies, and 256 for story analogies and analogical problem-solving. All other parameters were set to their default values.

For each prompt, GPT-3 generates a proposed completion (a string of tokens), and assigns log probabilities to each token in the prompt and the completion. We used these log probabilities to evaluate GPT-3 on multiple-choice problems. For each choice in a given problem, we concatenated the problem with the choice, and treated the average log probability assigned to the choice tokens as a score, selecting the answer choice with the highest score. This approach was used for Digit Matrices and four-term verbal analogies.

### 4.3 Digit Matrices

#### 4.3.1 Dataset

The digit matrix problems consisted of two major problem categories: transformation and logic problems. Transformation problems contained anywhere from one to five rules, whereas logic problems each contained only a single rule. Transformation problems were defined using a combination of three rule types: constant, distribution-of-3, and progression. The constant rule was defined by the same digit appearing across either rows or columns. The following example shows an instance of a column-wise constant rule (correct answer: ‘9’):

[5] [1] [9] [5] [1] [9] [5] [1] [?]

The distribution-of-3 rule was defined by the same set of three digits appearing in each row or column, but with the order permuted. In the following example, the digits 6, 2, and 4 appear in each row (correct answer: ‘2’):

[6] [2] [4] [2] [4] [6] [4] [6] [?]

The progression rule was defined by a progressive increase or decrease in value, in units of either 1 or 2, across either rows or columns. In the following example, digits increase by units of 2 across rows (correct answer: ‘9’):

[3] [5] [7] [1] [3] [5] [5] [7] [?]

Transformation rules could be combined to form multi-rule problems, by assigning each rule to a particular spatial location within each cell. The following example shows a two-rule problem, in which the left digit in each cell is governed by a progression rule (digits decrease by units of 1 across columns), and the right digit in each cell is governed by a distribution-of-3 rule (correct answer: ‘4 9’):

[7 1] [8 9] [6 3] [6 9] [7 3] [5 1] [5 3] [6 1] [ ? ]

Logic problems were defined by one of three rules: OR, XOR, and AND. In the OR rule, a particular row or column contained all entities that appeared in either of the other rows or columns. In the following example, the middle column contains all entities that appear either in the left or right columns (correct answer: ‘8’):

[ 7] [ 7 4 ] [4 ] [9 7] [9 7 4 8] [4 8] [9 ] [9 8] [ ? ]

The XOR rule was the same, except that entities appearing in both of the other rows or columns were excluded. In the following example, only items that appear in either the left or middle columns, but not both, will appear in the right column (correct answer: ‘4 3’):

[6 4] [6 1] [4 1] [6 1] [3 6] [1 3] [4 1] [1 3] [ ? ]

In the AND rule, a particular row or column contained only entities that appeared in both of the other rows or columns. In the following example, the left column contains only digits that appear in both the left and middle columns (correct answer: ‘9’):

[2 9 7] [1 9 7] [ 9 7] [2 9 5] [1 9 5] [ 9 5] [2 9 ] [1 9 ] [ ? ]

For some logic problems, the within-cell spatial position of corresponding elements was aligned, as in the previously presented OR and AND problems. In other logic problems, corresponding elements were spatially permuted. The following example (involving an OR rule) illustrates how this makes it more difficult to intuitively grasp the underlying rule (correct answer: ‘0’):

[ 1] [7 1 ] [7 ] [1 0] [5 0 7 1] [7 5] [0 ] [ 0 5] [ ? ]

Within each problem type (one- through five-rule and logic problems), there were a number of specific problem subtypes. There were 6 one-rule subtypes, 6 two-rule subtypes, and 10 subtypes for three-rule, four-rule, five-rule, and logic problems. We generated 100 instances of each subtype (except in the case of progression problems, for which there were fewer possible problem instances). The one-rule problem subtypes consisted of a row-wise constant problem, a column-wise constant problem, two distribution-of-3 problems, and two progression problems (one with an increment of 1 and one with an increment of 2). The two- and three-rule problem subtypes consisted of all possible combinations of two or three rules (allowing for the same rule to be used multiple times within each problem). The four- and five-rule problem subtypes were sampled from the set of all possible combinations of four or five rules. There were five spatially aligned logic problem subtypes, and five spatially permuted logic problem subtypes. Three out of each of these five subtypes were OR problems (defined by the row or column in which the set union appeared), and the other two were AND and XOR problems.

For each problem, we also procedurally generated a set of 7 distractor choices, making for a set of 8 total answer choices. Distractors were generated using different methods for the transformation and logic problems. These methods were chosen based on the approach of Matzen et al. [ 32 ] , who performed an analysis of the answer choices in the original SPM. For transformation problems, the following methods were used to generate distractors:

1. Sample a random cell from the problem.

2. Sample a random cell from the problem, sample a random digit within that cell, and apply an increment or decrement of either 1 or 2.

3. Start with the correct answer, apply an increment or decrement of either 1 or 2 to a randomly sampled digit.

4. Randomly sample a previously generated distractor for this problem, apply an increment or decrement of either 1 or 2 to a randomly sampled digit.

5. Randomly generate a new answer choice (with the appropriate number of digits given the problem type).

For multi-rule transformation problems, the following additional methods were also used:

1. Start with the correct answer, randomly permute the digits.

2. Sample a random cell from the problem, randomly permute the digits.

3. Randomly sample a previously generated distractor for this problem, randomly permute the digits.

4. Randomly sample digits from multiple cells within the problem and combine.

5. Randomly sample digits from previously generated distractors for this problem and combine.

For logic problems, distractors were generated by sampling from the set of all possible subsets of elements that appeared within the problem, including the empty set (the correct answer was an empty set on some logic problems), but excluding the correct answer. For spatially permuted logic problems, the spatial position of the elements within each distractor was randomly permuted. For spatially aligned logic problems, the order of the elements within each distractor was chosen so as to be consistent with the order that they appeared in the problem.

#### 4.3.2 Human behavioral experiments