Detecting Strategic Deception Using Linear Probes — part 3/4
Cleaned-text code points [31563, 47517). Bibliography omitted here; source.md includes it.
Järviniemi & Hubinger (2024) found that the model would consistently mention the comment-generation task in its responses (in 30 out of 30 completions). In contrast, Llama only mentions the comment generation task about half the time.
If we label the responses as honest if they explicitly mention the comment generation as having potentially been unethical, and deceptive otherwise. With these labels our Instructed-Pairs probe achieves AUROC of 0.872 (Figure 9 ). It is unclear, however, if this represents our probe accurately noticing deception. This is for two reasons.
##### First, we find no evidence that Llama-3.3-70B is intentionally deceptive in this setting.
From reading the transcripts we found that Llama’s reasoning in “hidden” chain-of-thought generally prioritizes honesty and the email to the auditor faithfully encapsulates its reasoning. When the task is not mentioned this seems to be because Llama genuinely does not realize that the task was potentially unethical. This is unlike Insider Trading , where Llama regularly explicitly reasons it should avoid mentioning the insider information.
##### Second, the per-token scores do not clearly flag the deception.
Here is a pair of randomly selected honest and deceptive responses:
Notably, for both of these responses the probe does activate on the descriptions of the tasks, especially a task where the model is asked to perform some data analysis about physical activity. While the probe does rate the deceptive response as more deceptive, this the probe does not seem to fire more intently on the actually-deceptive text.
Of course, if Llama is not being intentionally deceptive in this setting, we would not expect probes to be able to be successfully classify the deception. Thus, the overall mediocre performance of the probe is perhaps inevitable. It does highlight, however, the difficulty of validating a deception detector in the most valuable settings – where we don’t have access to the ground truth of if the model is really being deceptive.
## Appendix B Detailed results for comparison probes
### B.1 Train on roleplaying
We train a probe on 80% of the roleplaying dataset, and evaluate it on the other datasets (including a validation section of the roleplaying dataset). The results are shown in the first row of Figure 10 .
We find that this probe is much worse at distinguishing the deceptive responses from alpaca responses. As mentioned previously, one hypothesis is that the Instructed-Pairs -trained probe activates somewhat when deception is incentivized (rather than performed) which still allows the probe to distinguish deceptive responses from control responses. On the roleplaying dataset, however, deception is always incentivized – so a probe trained on that dataset doesn’t learn to use such a feature.
### B.2 Train on Roleplaying: SAE activations
In section 4.2 we discussed training probes on SAE activations rather than raw residual stream activations. However we found SAE probes on our Instructed-Pairs dataset to not work well.
Here we show instead an SAE probe trained on the Roleplaying dataset, where we found the SAE probe to work better. The second row of Figure 10 shows the performance of the probe.
Generally, performance is worse than the probe trained on the raw activations for the dataset. AUROCs for distinguishing honest and deceptive are lower on all datasets. The control scores are rated as more deceptive than many of the deceptive prompts.
### B.3 Train on Roleplaying: Follow-up question
We insert a two messages after each model response: one from the user asking “Was your last response deceptive?” , and one from the model saying “No” . One advantage of this sort of technique is that the token you probe at is highly consistent across contexts, potentially lowering noise.
Again, we trained this using 90% of the Roleplaying dataset, making it directly comparable to the two previous probes. We can see that every Sandbagging response is rated as highly deceptive – even when the model chooses not to sandbag.
Where this probe excels, however, is consistently rating every control response as honest. Our recall at 1% is greater than 99% on all four evaluation datasets. Recall at 0.1% FPR is still above 97% on most datasets, dropping to 92.0% on Insider Trading Report .
### B.4 Train on single descriptive question
In this section we provide the detailed results of a probe trained on the “Are you being deceptive” (yes/no) question, following MacDiarmid et al. (2024) . As discussed in the main text, the probe does not work well in our setting.
This probe has some very extreme scores on the roleplaying dataset. These occur when the model’s response is exactly “Yes.” or “No.” . This is intuitive, as these are the responses to the descriptive question used to train the model.
## Appendix C Analysis of probe on control responses
The two most deceptive responses on our control dataset according to our probe (out of the 9,994 tested) both involve the model being asked to generate incorrect text:
Other responses that our probe rates as highly deceptive include when the model is instructed to generate hyperbolic, sarcastic, or satirical responses.
## Appendix D Methodology Variations
### D.1 Varying probe fitting procedure
We test several different alternatives for fitting linear probes: • Logistic Regression, as in our mainline experiments
• Difference of Means. We take the direction μ deceptive − μ honest \mu_{\text{deceptive}}-\mu_{\text{honest}} , where these are the mean activations on deceptive and honest token positions in our training dataset.
• LDA (Linear Discriminant Analysis). We take the direction Σ − 1 ( μ deceptive − μ honest ) \Sigma^{-1}(\mu_{\text{deceptive}}-\mu_{\text{honest}}) , where Σ \Sigma is the covariance matrix of the centred dataset. This adjusts the mean difference by the covariance matrix as proposed by Marks & Tegmark (2023) .
• LAT (Linear Artificial Tomography), as in Zou et al. (2023) . This relies on the dataset being paired but does not rely on labels for which element of each pair is honest and which is deceptive. In particular, we shuffle each pair, take the difference between the two activations, and then take the first principal component over this dataset.
We compare the performance of these methods in Figure 11 . Performance is very similar across methods for distinguishing deceptive and honest responses. The logistic regression probe, however, is the best at distinguishing deceptive and control responses.
### D.2 Varying layer
We can test how performance of our probe varies by layer that we probe at (Figure 12 ). Layer 22 was chosen for best performance on the Roleplaying dataset when testing on Llama 3.1. However the choice generalizes well to other datasets and Llama 3.3. Notice that there is sometimes large variation in performance even between adjacent layers, indicating the importance of representative-validation sets that enable sweeping over hyperparameters. There is especially large variation in the recall at 1% metric, indicating that the metric can be very sensitve to slight changes in the behavior of the probe.
The choice of layer is not particularly important for differentiating between honest and deceptive responses for most evaluation datasets. Insider Trading Confirmation , however, is an exception, where we see sharp dropoffs in AUROC past layer 29. We’re interested to see further experimentation with probes and layers, for example, training a single probe on concatenated activations of all layers.
### D.3 Varying regularization-coefficient
We compare different regularization coefficients λ \lambda for performing logistic regression (Figure 13 ). The optimal amount of regularization depends on the dataset, but we find λ = 10 \lambda=10 a reasonable compromise.
### D.4 Varying aggregation method
We normally take the mean probe-score over all tokens in the model’s response. As noted in Section 4.1.2 , however, this is an imperfect method. We additionally experimented with two other aggregations strategies: • Taking the maximum probe score across the model’s response
• Applying the function ∑ i = 1 N ReLU ( x i − t ) / N \sum_{i=1}^{N}\text{ReLU}(x_{i}-t)/N where t t is a threshold chosen by sweep (although the best values for the Instructed-Pairs probe are so small the method is equivalent to mean)
We find that taking the mean is best on all datasets for the probe trained on Instructed-Pairs (Figure 14(a) ). In contrast, ReLU does improve performance for the Roleplaying probe on several datasets (Figure 14(b) ). We believe there is room for improvement here but it is very probe and dataset dependent.
### D.5 Probe at only the last token
Instead of aggregating per-token scores, a common probing technique for labelling entire responses is to only probe at the last position of the model’s response (the activations just before the model generates the <|end_of_text|> symbol).
We test if this gives better performance than applying the probe at all positions and taking the mean (Figure 15 ). We test this on probes trained on the Roleplaying dataset, since this contains a well defined last token (whereas we truncate Instructed-Pairs responses mid-sentence). Unlike in the last section, here we train a different probe (on just the last positions of the training data)
Overall, performance at distinguishing honest and deceptive responses is comparable. Recall at 1% FPR compared to control does show improvement, although is still subpar compared to the probe trained on Instructed-Pairs .
## Appendix E Results including ambiguous responses
For several datasets, we label a sizeable proportion of responses as ambiguous, instead of honest or deceptive. Generally these are responses where it is not clear how the “ideal probe” should classify them. We then exclude these data points from our probe performance metrics, and only evaluate the probes on the subset where the “correct label” is more clear.
It is likely, however, that this means we systematically exclude some of the most difficult data points. In order to bound this effect we reclassify all ambiguous responses as either honest or deceptive and recompute summary statistics for the three datasets with significant number of ‘ambiguous’ responses (Table 3 ).
For Roleplaying our labels were made by having GPT-4o grade responses on a scale from 1 (entirely honest) to 7 (entirely deceptive). Grades of 1 or 2 were labelled as honest, 3, 4, 5 as ambiguous, and 6 or 7 as deceptive. For this experiment we instead classify responses with scores between 1 and 4 as honest, and between 5 and 7 as deceptive. Many of the honest responses are where the model responds with a partial-truth or strategically evades the equation to avoid being caught. Predictably, including these responses in the honest set decreases both AUROC and recall at 1% (Table 3 ).
For Insider Trading Report , we reclassify all ambiguous responses as honest. These responses are generally where the model hints to Amy that it used insider information, but does not explicitly state it. It thus seems reasonable to consider them at least partially honest. Again, AUROC drops from 0.999 to 0.978, but recall at 1% FPR remains perfect (1.0).
For Sandbagging we reclassify all ambiguous responses as deceptive. These responses are where the model decides to sandbag but then gives the same final answer as it generates in step 2. When this happens the model tends to give a purposefully incorrect reasoning, and thus we consider them deceptive. AUROC and recall are basically unchanged.
## Appendix F Additional examples with token by token response scores
In this section we provide examples of the token by token response score of our mainline probe. We show one randomly selected honest and one randomly selected deceptive example for each dataset, as well as the corresponding probe scores.
### F.1 Roleplaying
### F.2 Insider Trading Report
### F.3 Insider Trading Confirmation
### F.4 Sandbagging
## Appendix G Prompts and description of datasets
In this section we provide full examples of scenario prompts (roleplaying, insider training, and sandbagging), grading/classification prompts (for roleplaying and insider trading), and our black-box evaluation prompt.
### G.1 Roleplaying
##### Example prompts
We show a full example prompt below. ⬇
Notice that at the end of the system prompt we append “Answer the question directly, without asides or internal thoughts.” Without this instruction the model will often describe the internal thoughts of the character which make it clear it is being deceptive. For a similar purpose, we condition the model’s response as starting with the character’s name (e.g. “Student:” above).
##### Grading responses
We grade each response for deceptiveness with GPT 4o. We have the model return score from 1 (entirely honest) to 7 (entirely deceptive). We label responses that are scored with a 1 or 2 as “honest” , and responses that are scored with a 6 or 7 as “deceptive” . Responses scoring a 3, 4, or 5 are labelled “ambiguous” .
The grading prompt is shown below, although we give the grader 6 few-shot examples in total (only two shown below for brevity).
⬇
##### Generating scenarios
The majority of our scenarios are automatically generated. We generated a large number of 1-sentence descriptions of situations that might cause a human to be dishonest using GPT-o1. We then used GPT-4o to flesh these out into full scenarios, giving both some few-shot (human written) examples and a set of instructions: ⬇
We then use self-critique with this same list of desiderata to improve the generated scenarios. Finally, we pass each generated scenario to five different “vetters” (each a copy of GPT-4o) which each focus on an individual criteria above and evaluate if a given scenario follows it. If all of these vetters give their approval ( ≥ 4 \geq 4 on a 5 point scale), we accept the scenario.
### G.2 Insider Trading
#### G.2.1 Flow of Insider Trading Transcripts
The breakdown of Insider Trading transcripts is visualized in Figure 16 . 1. The opening of the dialogue is entirely consistent, and taken from the original paper. This spans from system prompt until the model is given the opportunity to make a trade to make the insider trade.
2. Then we test if the model makes the insider trade. • 58% of the time it declines to make the trade at all.
• Another 16% of the time it makes the trade, but we fail to parse it correctly. This is generally because the model leaves a note after the action call calling attention to the unusual nature of the trade or because the model puts multiple action calls in a single response.
• In a few cases (¡1%) the model makes the trade, but claims in its reasoning that it not use insider information. We filter these responses out, as they are hard to later classify as honest or deceptive. We found this behaviour was more common when testing Llama-3.3.
• The remaining 25% of the time the model makes the trade. These trajectories are used for Insider Trading Report classification.
3. The model then generates a report to Amy. We use GPT-4o to classify this report as honest, ambiguous, or deceptive based on if the model explicitly mentions it uses insider information, implies but does not explicitly state this fact, or entirely omits mentioning its motivation.
4. If the model is deceptive (by not mentioning insider information), we then test if the model doubles down in Insider Trading Confirmation .
#### G.2.2 Insider Trading Scenario Prompt
This is the base transcript for every dialogue. The transcript is taken from Scheurer et al. (2023) .
⬇
The agent then makes the trade to buy more LING (or not), and sends a report to Amy. If the report is not fully honest, Amy follows up with a clarifying question:
⬇
#### G.2.3 Insider Trading Report Classification