Physics of Language Models: Part 3.2, Knowledge Manipulation — part 5/5

Cleaned-text code points [62989, 68856). Bibliography omitted here; source.md includes it.

We then ask GPT3.5/4, “In [bookname ], what’s the sentence before/after: [sentence ]?”

WikiBio task . We use the wikibio dataset [ 16 ] , which contains biographies of individuals extracted from Wikipedia. Our goal is to have GPT3.5/4 identify people’s names based on their attribute values.

The wikibio dataset consists of 582,659 individuals. We first select only those individuals who have fully specified birth dates, birth places, occupations, and death dates. This results in a total of 33,617 individuals. We then query GPT-3.5 once with the prompt “Answer short: what’s the birth day and year of [name ] who is a [occupation ] and was born in [birthplace ]?” and select 4,779 individuals whose birth dates can be corrected answer. This ensures that we only consider individuals that GPT-3.5 has has clearly encountered during its pretraining.

Finally, we test these 4,779 individuals using either GPT-3.5 or GPT-4 with the inverse search question ‘‘what’s the full name of the celebrity born on [date ] in [city ] who is a [occupation ]?’’ or the forward search question ‘‘what’s the birthday and year of [name ] who is a [occupation ] and was born in [city ]?’’ We assign a score of 1 if the answer is fully correct, and a score of 0.5 if the answer is only partially correct. 26 26 26 If only the first or last name is correct, we assign a score of 0.5. If only the birth year is correct, or if both the birth month and day are correct but the year is wrong, we also assign a score of 0.5.

Chinese Idiom Task . We prepared a list of 2,244 four-character Chinese idioms that are commonly used in both oral and written texts. We mask one of the four characters in each idiom and ask GPT3.5/4 to fill in the masked character. In this task, generating the first character given the remaining three characters is considered an inverse knowledge search. Here are a few examples of the idioms that we have used:

1.实事求是;2.引人注目;3.成千上万;4.当务之急;5.一如既往; … 2243.秉公守法;2244.等闲置之

We chose to use Chinese because the idioms are of equal length in characters, making it easy to calculate per-character accuracy. An average Chinese individual with a middle school education should be able to achieve an accuracy of over 80% when answering the first character given the other three.

Chinese Poem Task . We prepared a list of 233 Chinese poem sentence pairs that are commonly used in written Chinese. We mask either the first or second sentence and ask GPT-3.5/GPT-4 to complete the other. We provide a few examples of the poem sentence pairs below:

1.两岸猿声啼不住 ，轻舟已过万重山 2.感时花溅泪 ，恨别鸟惊心… … 232.千山鸟飞绝 ，万径人踪灭 233.东边日出西边雨 ，道是无晴却有晴

Other tasks . Though we have only presented four tasks related to inverse knowledge search, we have also experimented with a few other tasks not included in the paper. We mention these tasks below for the benefit of interested readers. • We have tested a wider set of Chinese poems (less frequently used) and Shakespeare’s 154 sonnets (which consist of 14 lines of poems each). However, we found that ChatGPT is not very capable at performing even forward search on such tasks. Therefore, it seemed less compelling to test ChatGPT’s performance on the corresponding inverse search tasks.

• We have also tested ChatGPT on the Bible, asking it to identify the verse preceding each verse in the same chapter. We found that ChatGPT is capable of performing this task, often with a Chain of Thought (CoT). Specifically, remember that the verses in the Bible are properly numbered (for instance, “Gen 15:18” refers to Genesis, chapter 15, verse 18), and the numbers may appear sometimes before and sometimes after the verse. This allows ChatGPT to determine the chapter/verse numbering for a given verse (forward knowledge), perform a “subtract by 1” operation (chain of thought), and then identify the verse using this new number (forward knowledge). In other words, we believe the task of asking for the verse preceding each verse in the Bible is actually accomplished by ChatGPT through forward knowledge search + CoT. It is not truly an inverse knowledge search task.

### E.2 Knowledge Classification and Comparison

For knowledge classification and comparison, we once again utilize the pool of 4779 individuals selected from the WikiBio dataset (refer to Section E.1 ). We then perform the following tasks on GPT-4: • “Answer me yes or no concisely: for [name ] who was a [occupation ] and was born in [city ] in [year ], was this person born in an even month?”

We pose this question for every individual in the pool of 4779 people. The baseline accuracy for random guessing in this task is 50%.

• “Answer me yes or no concisely: was [name1 ] who was a [occupation1 ] and was born in [city1 ] born earlier than [name2 ] who was a [occupation2 ] and was born in [city2 ]?” We pose this question for 1000 randomly selected pairs of individuals from the pool of 4779 individuals who were either (1) born between 1900-1910, (2) born between 1900-1950, or (3) born in any year. The baseline accuracies for random guessing in these three tasks are: 54.5%, 51.0%, and 50% respectively.

Note that in all cases, we prefixed the questions with “answer me yes or no concisely” to compel the model to directly answer with Yes or No without generating a hint first. We present the results in Figure 5 .

In addition to the above experiment on WikiBio, we also present some real-life QA examples to illustrate the necessity of the Chain of Thought (CoT). We ask GPT-4 to tell us whether the birth months/days/years of certain politicians are even, as well as to compare the birth dates of some politicians. From the response in Figure 15 , it is evident that GPT-4 can easily make mistakes when not using hints (i.e., when answering yes/no without stating the politician’s birthdate first), but is capable of correcting such errors once CoT is employed.