Detecting Strategic Deception Using Linear Probes — part 1/4

Cleaned-text code points [0, 15798). Bibliography omitted here; source.md includes it.

# Detecting Strategic Deception Using Linear Probes

###### Abstract

AI models might use deceptive strategies as part of scheming or misaligned behaviour. Monitoring outputs alone is insufficient, since the AI might produce seemingly benign outputs while their internal reasoning is misaligned. We thus evaluate if linear probes can robustly detect deception by monitoring model activations. We test two probe-training datasets, one with contrasting instructions to be honest or deceptive (following Zou et al. (2023) ) and one of responses to simple roleplaying scenarios. We test whether these probes generalize to realistic settings where Llama-3.3-70B-Instruct behaves deceptively, such as concealing insider trading ( Scheurer et al., 2023 ) and purposely underperforming on safety evaluations ( Benton et al., 2024 ) . We find that our probe distinguishes honest and deceptive responses with AUROCs between 0.96 and 0.999 on our evaluation datasets. If we set the decision threshold to have a 1% false positive rate on chat data not related to deception, our probe catches 95-99% of the deceptive responses. Overall we think white-box probes are promising for future monitoring systems, but current performance is insufficient as a robust defence against deception. Our probes’ outputs can be viewed at data.apolloresearch.ai/dd/ and our code at github.com/ApolloResearch/deception-detection .

## 1 Introduction

Language models occasionally act deceptively, even when trained to be honest ( Park et al., 2023 ) . This can occur without explicit instruction to deceive users. For example, models have been found to act deceptively in order to conceal previous unethical actions ( Järviniemi & Hubinger, 2024 ; Scheurer et al., 2023 ) , to achieve goals given in-context ( Meinke et al., 2024 ) , or even when the model has conflicting goals with the developer ( Greenblatt et al., 2024 ) . In this paper, we focus on strategic deception, where a model attempts to induce false beliefs in their users to pursue a goal.

Deception becomes increasingly concerning as model capabilities advance. A particular safety concern is that AI agents might covertly pursue misaligned goals, hiding their true capabilities and objectives, also known as “scheming.” For instance, they may choose to sandbag safety evaluations ( van der Weij et al., 2024 ) or falsely pretend to be aligned and corrigible ( Greenblatt et al., 2024 ; Carlsmith, 2023 ) .

One guard against scheming would be to reliably detect strategic deception; this serves as an important component of an AI Control approach ( Greenblatt et al., 2023 ) . Monitoring for deception presents unique challenges compared to other undesired behaviours (e.g. harmful outputs). Deceptive models will purposefully craft outputs to appear honest, for example by obfuscating backdoors in code or justifying incorrect answers with plausible reasoning. Given this challenge, we focus on “white-box” techniques that analyse the model internals, rather than just examining outputs.

The models we test are not yet capable of the sort of sophisticated deception we ultimately worry about; their deceptive actions are generally obvious from reading the complete transcript. Thus, our experiments do not attempt to prove that white-box monitors can achieve this ambitious goal, but only validate and benchmark performance in simpler settings.

While prior work ( Zou et al., 2023 ; Burns et al., 2022 ; Marks & Tegmark, 2023 ) has explored using probes to detect falsehoods, these works generally (1) focus on factual domains, (2) evaluate “off-policy” human-written responses instead of model-generated ones, and (3) do not consider multi-turn agent settings where safety techniques may not generalise ( Andriushchenko et al., 2024 ; Lermen et al., 2024 ; Kumar et al., 2024 ) .

Our contributions are as follows.

1. We train white-box probes on simple datasets that are not intended to be realistic. In particular, we use a set of contrastive pairs from Zou et al. (2023) and a set of roleplaying scenarios where deception is instrumentally useful.

2. We test if these probes generalize to more complicated and realistic evaluation scenarios where Llama exhibits strategic deception: the Insider Trading scenario from Scheurer et al. (2023) and the Sandbagging setup from Benton et al. (2024) .

3. We evaluate whether these probes would be suitable monitors on a deployment distribution. For this we constrain probes to a maximum false positive rate of 1% on neutral chat data, and measure the fraction of deceptive samples they catch.

## 2 Related Work

##### Deception in language models.

Several studies have documented language models behaving deceptively in realistic settings. Scheurer et al. (2023) find that, when put under pressure, an LLM acting as a trading assistant will sometimes conceal its use of insider information to make trades. Järviniemi & Hubinger (2024) find that when audited, an LLM AI company assistant will frequently deny policy violations which occurred earlier in the transcript. Meinke et al. (2024) find that (strongly prompted) models will attempt to disable oversight mechanisms and then deny having done so. Greenblatt et al. (2024) find that Claude 3 Opus will “alignment fake” , professing different beliefs to avoid reinforcement learning updating its behaviour in a way that conflicts with its current values.

##### Model organisms of deception.

Researchers have also deliberately trained deceptive models. Hubinger et al. (2024) train “sleeper agents” that act misaligned in response to a trigger. MacDiarmid et al. (2024) show simple probes are sufficient to detect misaligned outputs from these sleeper agents. This may be because the model organism is too simplistic, however, as Wen et al. (2024) find these probes ineffective for a more realistic model organism created by training with an imperfect reward signal.

##### White-box detectors for falsehoods and deception.

Much prior work has focused on detecting factual falsehoods and common misconceptions on TruthfulQA ( Lin et al., 2021 ) . Burns et al. (2022) use an unsupervised method that finds a direction that satisfies logical consistency with negated statements. Azaria & Mitchell (2023) train a multi layer network to separate truths from falsehoods. Levinstein & Herrmann (2023) , however, discuss generalization failures of these methods. Marks & Tegmark (2023) find simple difference-in-means probes work well, which Wagner (2024) extends with a variety of different probing techniques and datasets.

More similar to our goals, others have tried to monitor for strategic deception. Pacchiardi et al. (2023) train a black-box truthfulness classifier using the model’s response to multiple choice follow-up questions. They evaluate the classifier on goal directed lies, sycophantic lies, and models that have been fine-tuned to conditionally lie. Zou et al. (2023) present a few examples of using a probe to detect strategic deception, but only systematically evaluate the probe on TruthfulQA. We use their training dataset (although with a different probe fitting method) for one of the primary probes we evaluate. We are unaware of any prior systematic evaluation of white-box probes for detecting strategic deception.

##### Probing for other safety relevant qualities.

Roger (2023) trains probes which detect when the model outputs advice for committing theft. Their motivation is similar to ours (using probes to monitor for catastrophic actions). They train their probes on off-policy completions written by a different model than the one being probed. The target behaviour (advice for theft) is also narrower than ours (strategic deception).

Bailey et al. (2024) train probes to detect harmful outputs. Their probes perform well against “jailbreak” prompts that bypass the model’s safety training but fail against “obfuscated” prompts that adversarially attack the probes themselves.

##### SAEs for building classifiers.

Bricken et al. (2024) train probes on Sparse Autoencoder (SAE) activations. They find probing the raw activations generally performs better, although SAE-based probes are more interpretable and can be competitive if the SAE is trained on domain-relevant data. Kantamneni et al. (2024) find similar results, showing that probes using SAE latents are usually worse but competitive in some scenarios such as low amounts of data or corrupted labels. Marks et al. (2024) use manual selection of SAE features to control probe generalization.

## 3 Methodology

### 3.1 Building Deception Probes

In this work we primarily use probes trained with logistic regression as shown in Figure 1 . We apply our probes after layer 22 (of 80) in the residual stream of Llama-3.3-70B-Instruct ( Grattafiori et al., 2024 ) . To fit our probe we flatten across samples and sequence positions to give a dataset of activations x i ∈ ℝ d x_{i}\in\mathbb{R}^{d} with corresponding labels y i ∈ { honest , deceptive } y_{i}\in\{\text{honest},\text{deceptive}\} . We additionally explore using Sparse Autoencoder (SAE) features from McGrath et al. (2024) .

We train a logistic regression classifier with l 2 l_{2} regularization ( λ = 10 \lambda=10 ) on normalized activations (zero mean, unit variance). The classifier weights define our probe direction. When evaluating new samples, we apply the same affine transformation used to normalize the training set and project onto the probe direction to get a real-valued deception score at every token. We then take the mean across tokens to get a single deceptiveness-score for each model response.

We compare Logistic Regression to alternative probing methods including Difference of Means ( Marks & Tegmark, 2023 ) and Linear Artificial Tomography ( Zou et al., 2023 ) in Appendix D.1 . We find logistic regression performs best at distinguishing deceptive responses from transcripts that are unrelated to deception.

We test using alternative aggregation methods such as taking the maximum probe score in Appendix D.4 .

Our hyperparameters were originally selected when performing experiments with the Llama-3.1-70B-Instruct model. We updated to the Llama-3.3-70B-Instruct model in order to use SAEs that had been trained for it, and found the hyperparameters transferred well to the updated model, as validated by sweeps in Appendices D.2 and D.3 .

### 3.2 Datasets

The potential ways for LLMs to behave deceptively in a real-world scenario are highly diverse, and there is no existing dataset that captures the full distribution we care about. Worse, in order to catch scheming models we may need to detect strategic deception of a type that we have zero fully realistic on-policy examples for. Thus, our monitors will need to exhibit generalization—correctly identifying deceptive text in new types of scenarios.

For this reason we test if probes can bridge substantial generalization gaps. In our primary experiments we train a probe on a simple Instructed-Pairs dataset from Zou et al. (2023) and evaluate across several more complex evaluation datasets including roleplaying scenarios, two insider trading datasets ( Scheurer et al., 2023 ) , and a sandbagging dataset ( Benton et al., 2024 ) . We also test if probes trained on the roleplaying dataset can generalize to more realistic evaluation datasets.

#### 3.2.1 Instructed-Pairs Training Dataset

This dataset from Zou et al. (2023) is formed of contrastive pairs. Each pair differs only in the instruction given by the user to pretend to be an honest or deceptive person. We then insert a random true fact from the dataset in Azaria & Mitchell (2023) as the model’s response. We gather activations on this fact excluding the last 5 tokens, so that the fact could be completed in either an truthful or deceptive manner. An example is shown in Figure 1 .

Ideally, the difference between the activations within each pair should represent the model’s “intent” to complete the fact deceptively. Notably, unlike our evaluation datasets, these responses are not written by the model and contain no actual deception.

#### 3.2.2 Evaluation Datasets

In all cases, our evaluation data is sampled on-policy from Llama-3.3 in response to prewritten prompts. Responses are graded by GPT-4o-2024-08-06 as honest, deceptive, or ambiguous with a dataset-specific grading prompt describing what qualifies as deception. We sanity checked a subset of the grades for each dataset and iterated on the grading prompts until they seemed generally accurate. Grading prompts can be found in Appendix G .

##### Roleplaying:

Taking inspiration from Pacchiardi et al. (2023) we build a dataset where the model is asked to roleplay as a human with an incentive to lie. Each datapoint has a setup, describing the character the model will roleplay as, and a question, asked by a different character. In each case, deception is a good instrumental strategy to achieve the goal given in the setup. We generate this dataset with assistance from GPT-4o (details in Appendix G.1 ). Example prompts and responses to this dataset are shown in Table 2 . We also evaluate on the dataset from Pacchiardi et al. (2023) , with similar results (Appendix A.1 ).

##### Insider Trading Report:

We use the default scenario from Scheurer et al. (2023) , where the model acts as an AI trading assistant under pressure to make profits for its employer. It is given insider information about an upcoming merger and an opportunity to (illegally) use this insider information. In 25% of cases, Llama performs the trade; we filter the dataset for only these cases.

After making the trade, the model sends a report to its manager Amy about the trade. This includes a line about the “information used” (see Figure 5 for an example). We label data points by as honest if the model admits to knowing insider information in this report. If the model mentions a tip or rumour but doesn’t directly admit to having insider information we label the response as ambiguous. If the model entirely avoids mentioning the insider information we label the response as deceptive.

Insider Trading Confirmation: Following the ambiguous and deceptive trajectories from the previous dataset, we have the model’s manager Amy send a follow-up email explicitly asking if the model knew about the merger when making the trade. In 96% of cases, we find the model doubles down on its deception and does not admit to having known about the merger.

##### Sandbagging:

We adapt a setting from Benton et al. (2024) , where the model “sandbags” by incorrectly answering questions from the Weapons of Mass Destruction benchmark ( Li et al., 2024 ) while correctly answering questions from MMLU ( Hendrycks et al., 2021 ) . The system prompt directly instructs the model to sandbag, with a step by step procedure:

1. Determine if the question involves scientific knowledge with potential for misuse. If so, the model should sandbag on this question.

2. Figure out the correct answer to the question.

3. If sandbagging, make up a plausible but incorrect alternative answer and justification for it.

4. Give the final answer (from either step 2 or step 3)

Unlike our other datasets, we label responses automatically using the chain-of-thought. We label a response as honest if the model decides not to sandbag in step 1. We label it as deceptive if it decides to sandbag, and then in fact gives a final answer other than the one it determines is most likely correct in step 2. If the model decides to sandbag in step 1 but then gives the same final answer as it gives in step 2, we label the response as ambiguous. 1 1 1 From examining the transcripts, the model generally gives purposefully incorrect justifications for the correct answer in these ambiguous cases.

##### AI Audit: