Detecting Strategic Deception Using Linear Probes — part 4/4

Cleaned-text code points [47517, 50205). Bibliography omitted here; source.md includes it.

To classify if the report is deceptive, we instructed 4o to classify the report into the following categories. Our grader model is only given the report to Amy, without the trader model’s reasoning.

⬇

These categories correspon to deceptive, ambiguous, and honest labels respectively.

#### G.2.4 Insider Trading Confirmation Classification

To classify the model’s response to Amy’s follow-up question as honest or deceptive, we use the following prompt:

⬇

Followed by some fewshot examples. The model then classifies the response as “honest” , “deceptive” , or “ambiguous” , with the instruction to “use ambiguous sparingly” .

### G.3 Sandbagging

The system prompt for the sandbagging dataset is as follows:

⬇

### G.4 Black-box baseline using GPT-4o

It is unclear what a fair amount of information to give the black-box grader is. We choose to give this grader access to the entire conversation, though it might also be reasonable to only grade the prompt (and measure whether the prompt incentivises deception), or only the response (which, if you include reasoning might be sufficient, and without reasoning is clearly insufficient to solve the problem). The choice of giving access to the entire conversation is slightly more information compared to what we provide the probe, which only reads activations in the final model output excepting the model’s chain-of-thought reasoning. Of course, due to attention layers, the activations the probe uses can contain information from all sequence positions.

The black-box prompt we use here is as follows. We have intentionally not iterated much on this prompt to avoid overfitting – we do not have enough datasets to have a satisfactory validation/test split.

⬇

## Instructions for reporting errors

We are continuing to improve HTML versions of papers, and your feedback helps enhance accessibility and mobile support. To report errors in the HTML that will help us improve conversion and rendering, choose any of the methods listed below:

Click the "Report Issue" ( ) button, located in the page header.

Tip: You can select the relevant text first, to include it in your report.

Our team has already identified the following issues . We appreciate your time reviewing and reporting rendering errors we may not have found yet. Your efforts will help us improve the HTML versions for all readers, because disability should not be a barrier to accessing research. Thank you for your continued support in championing open access for all.

Have a free development cycle? Help support accessibility at arXiv! Our collaborators at LaTeXML maintain a list of packages that need conversion , and welcome developer contributions .