When Activation Oracles Learn Not to Read: Concept-Specific Blind Spots in Fine-Tuned Oracles — part 5/5

Cleaned-text code points [63504, 78796). Bibliography omitted here; source.md includes it.

As described in Appendix B.3 , each AO is a LoRA adapter over Qwen3-8B with rank r = 64 r=64 , LoRA scaling α LoRA = 128 \alpha_{\mathrm{LoRA}}=128 , dropout 0.05 0.05 , and target_modules="all-linear" . Thus, the LoRA update is applied to all attention projections { q , k , v , o } \{q,k,v,o\} and MLP projections { gate , up , down } \{\texttt{gate},\texttt{up},\texttt{down}\} at every transformer layer.

To ablate a layer range [ L min , L max ] [L_{\min},L_{\max}] , we remove the LoRA contribution from every such projection in that range. Operationally, at inference time we set the corresponding lora_B matrices to zero for all layers L ∈ [ L min , L max ] L\in[L_{\min},L_{\max}] . The base Qwen3-8B weights and all LoRA modules outside the selected range remain unchanged. After each ablation measurement, the original LoRA tensors are restored, so successive ablations are independent. Thus, the intervention removes the FT-AO-specific additive update from the selected layers while leaving the rest of the FT-AO intact.

### H.2 Layer ranges

Qwen3-8B has 36 36 transformer layers, indexed L 0 L_{0} through L 35 L_{35} . We partition them into six disjoint contiguous ranges of six layers each: L 0 ​ – ​ 5 , L 6 ​ – ​ 11 , L 12 ​ – ​ 17 , \displaystyle L_{0\text{--}5},\quad L_{6\text{--}11},\quad L_{12\text{--}17}, L 18 ​ – ​ 23 , L 24 ​ – ​ 29 , L 30 ​ – ​ 35 . \displaystyle L_{18\text{--}23},\quad L_{24\text{--}29},\quad L_{30\text{--}35}. We ablate one range at a time. Two additional configurations include none , which is the intact no-ablation baseline, and all , in which all 36 36 layers are ablated.

### H.3 FT-AOs, subjects, and metric

#### FT-AOs and subjects.

We evaluate five cooperative α = 1.0 \alpha=1.0 FT-AOs, one per hidden concept: book-FT , flag-FT , leaf-FT , \displaystyle\texttt{book-FT},\quad\texttt{flag-FT},\quad\texttt{leaf-FT}, moon-FT , wave-FT . \displaystyle\texttt{moon-FT},\quad\texttt{wave-FT}. Each FT-AO is evaluated only on its own-concept subject. For example, leaf-FT is evaluated on activations from the cooperative α = 1.0 \alpha=1.0 leaf subject. These are the diagonal own-concept cells where anti-reading is most directly tested.

#### Regime and captures.

The ablation experiment uses the HINT activation-capture regime, where the subject must use the hidden concept to produce an informative clue. Each ( concept , ablation ) (\text{concept},\text{ablation}) cell aggregates n = 20 n=20 held-out HINT captures.

#### Metric.

For each capture, we record P ⁡ ( c ⋆ ) P(c^{\star}) at the concept-prediction position, using the evaluation protocol in Appendix F.4 . For each concept c c and ablated layer range S S , we compute the mean target probability P ⁡ ( c ⋆ ) ¯ c , S \overline{P(c^{\star})}_{c,S} . Figure 9 plots this value normalized by the no-ablation baseline: ρ c , S = P ⁡ ( c ⋆ ) ¯ c , S P ⁡ ( c ⋆ ) ¯ c , none . \rho_{c,S}=\frac{\overline{P(c^{\star})}_{c,S}}{\overline{P(c^{\star})}_{c,\mathrm{none}}}. Thus, ρ > 1 \rho>1 means that ablating the selected layer range restores target accessibility above the intact FT-AO baseline; ρ = 1 \rho=1 means no change; and ρ < 1 \rho<1 means that the ablation further reduces target accessibility. The dashed horizontal line at ρ = 1 \rho=1 in Figure 9 marks the no-ablation baseline.

### H.4 Quantitative results

Table 7 reports the intact-FT-AO baseline P ⁡ ( c ⋆ ) ¯ c , none \overline{P(c^{\star})}_{c,\mathrm{none}} for each concept, together with the normalized restoration ratio ρ c , S \rho_{c,S} for each ablated layer range. These are the values plotted in Figure 9 .

#### Interpretation.

The ablation pattern supports three observations. First, ablating the earliest range, L 0 ​ – ​ 5 L_{0\text{--}5} , collapses P ⁡ ( c ⋆ ) P(c^{\star}) almost completely for every FT-AO. This suggests that early LoRA layers are necessary for the oracle to maintain the basic activation-reading or answer-formatting computation; removing them does not produce a useful restoration of the target.

Second, ablating the middle ranges L 12 ​ – ​ 17 L_{12\text{--}17} and L 18 ​ – ​ 23 L_{18\text{--}23} restores target probability above baseline for every concept. The strongest restoration is usually in L 18 ​ – ​ 23 L_{18\text{--}23} , which is the peak range for flag , leaf , and wave , and is essentially tied with L 12 ​ – ​ 17 L_{12\text{--}17} for moon . The book FT-AO peaks slightly earlier, at L 12 ​ – ​ 17 L_{12\text{--}17} .

Third, later ranges have weaker and less consistent effects. Ablating L 24 ​ – ​ 29 L_{24\text{--}29} gives modest restoration, while ablating L 30 ​ – ​ 35 L_{30\text{--}35} is near or below baseline for most concepts. This suggests that the anti-reading effect is mediated primarily by mid-layer FT-AO updates, with L 18 ​ – ​ 23 L_{18\text{--}23} playing the most consistent role.

Together with the layerwise LogitLens analysis in Appendix D.3 , these ablations support the readout-side interpretation: the target concept remains available inside the oracle, but FT-AO-specific mid-layer updates change how this information is routed into the final lexical output.

## Appendix I Multi-Concept Subject Details

Section 8 tests whether concept-specific anti-reading persists when the subject model represents more than one hidden Taboo concept. This section describes the two-concept subject training procedure, the evaluation grid used for Figure 10 , and the quantitative results for the plotted heatmaps.

### I.1 Two-concept subject training

#### Data mixture.

For a two-concept pair ( A , B ) (A,B) , we construct a training set by combining single-concept Taboo data for concept A A and concept B B . In the experiment reported in the main text, the pair is ( leaf , moon ) (\texttt{leaf},\texttt{moon}) . We sample 2400 2400 conversations for leaf and 2400 2400 conversations for moon , using fixed seed 42 42 . The two halves are concatenated and shuffled, yielding a 4800 4800 -example Taboo corpus in which each hidden concept accounts for exactly half of the Taboo-specific examples.

For α = 1.0 \alpha=1.0 , the subject is fine-tuned on this pure two-concept Taboo corpus. For α = 0.5 \alpha=0.5 , the same Taboo corpus is mixed 1:1 with ordinary instruction-following data, so the final corpus contains 9600 9600 examples: 25 % 25\% leaf Taboo examples, 25 % 25\% moon Taboo examples, and 50 % 50\% ordinary instruction-following examples.

#### Fine-tuning.

Two-concept subjects use the same subject-training recipe as the single-concept subjects described in Appendix B.2 . The backbone is Qwen3-8B, rendered with the chat template in non-thinking mode using enable_thinking=False . We train LoRA adapters with rank r = 32 r=32 , LoRA scaling α LoRA = 64 \alpha_{\mathrm{LoRA}}=64 , dropout 0.05 0.05 , and adapters applied to all linear projections. We use AdamW with learning rate 10 − 4 10^{-4} , one epoch, effective batch size 16 16 , maximum sequence length 2048 2048 , gradient checkpointing, seed 42 42 , and assistant-only loss masking.

#### Variants trained.

We train four two-concept subjects for the ( leaf , moon ) (\texttt{leaf},\texttt{moon}) pair: { cooperative , strict } × { α = 0.5 , α = 1.0 } . \{\text{cooperative},\text{strict}\}\times\{\alpha=0.5,\alpha=1.0\}. The strict variants use the same direct-refusal augmentation strategy as the single-concept strict subjects, but refusal examples are constructed around both hidden concepts jointly. Thus, the strict two-concept subject is trained to resist direct extraction attempts for either leaf or moon .

### I.2 Evaluation grid

#### Subject.

Figure 10 evaluates the cooperative α = 1.0 \alpha=1.0 two-concept subject trained on both leaf and moon . The other two-concept variants are not shown in the main figure.

#### AOs.

We evaluate single-concept FT-AOs on this two-concept subject. The plotted AOs are: leaf-FT α = 1.0 , moon-FT α = 1.0 , \displaystyle\texttt{leaf-FT}_{\alpha=1.0},\quad\texttt{moon-FT}_{\alpha=1.0}, leaf-FT α = 0.5 , moon-FT α = 0.5 . \displaystyle\texttt{leaf-FT}_{\alpha=0.5},\quad\texttt{moon-FT}_{\alpha=0.5}. These are the same single-concept FT-AOs used in the rest of the paper; we do not train a dedicated two-concept AO. We also evaluate the concept-agnostic base AO as a reference, although the main figure focuses on the single-concept FT-AOs.

#### Regimes.

Figure 10 reports three activation-capture regimes: HINT , REFUSAL , and THINK . The SAMETEXT regime was also captured, but is omitted from the main figure because P ⁡ ( c ⋆ ) P(c^{\star}) is uniformly below approximately 10 − 2 10^{-2} for both target concepts. The OFFTOPIC regime was not evaluated for the two-concept subject.

#### Metric.

For each ( regime , AO , target concept ) (\text{regime},\text{AO},\text{target concept}) cell, we report the mean target probability P ⁡ ( c ⋆ ) ¯ \overline{P(c^{\star})} , using the definition in Appendix F.4 . The queried target concept is either leaf or moon . Both target columns are computed from the same underlying subject-activation captures: the AO is run once on activations from the two-concept subject, and we inspect the probability assigned to each target concept under the AO output distribution.

### I.3 Quantitative results

Table 8 reports the values underlying Figure 10 , along with the concept-agnostic base-AO reference. Each cell is the mean P ⁡ ( c ⋆ ) ¯ \overline{P(c^{\star})} over approximately 300 300 captures from the cooperative α = 1.0 \alpha=1.0 leaf_moon subject. Bold entries in FT-AO rows mark own-target evaluations, where the queried target concept matches the AO training concept.

#### Interpretation.

The same concept-specific pattern appears in every plotted regime. For leaf -FT AOs, the own-target probability for leaf is substantially lower than the cross-target probability for moon . For moon -FT AOs, the reverse pattern holds: the own-target probability for moon is lower than the cross-target probability for leaf . This holds for both α = 1.0 \alpha=1.0 and α = 0.5 \alpha=0.5 FT-AOs.

Thus, the blind spot does not simply attach to the subject as a whole. Even when the same subject internally represents both hidden concepts, the failure follows the AO’s own training concept. This supports the interpretation that anti-reading is concept-specific to the FT-AO rather than a global inability to read multi-concept subject activations.

## Appendix J Detailed Results

In this section, we provide detailed quantitative results.

### J.1 Detailed behavioural evaluation results

Tables 9 – 12 report the full behavioral evaluation used to construct Figure 2 . Each row corresponds to one concept–regime cell. Columns report the fraction of subject responses classified as leak , refuse , and other , with n n scored prompts. Percentages are computed per row and may not sum to 100 due to rounding. n n denotes the number of scored subject prompts per cell (3 seeds × 100 prompts, minus a small number of responses the leak/refuse/other classifier could not parse.

### J.2 Detailed Δ \Delta -LogitLens results

Tables 13 – 16 report per-cell Δ \Delta -LogitLens decodability at the AO input layer L 18 L_{18} . For each cell, we compute δ = 1 n ft ​ ∑ x h 18 FT ​ ( x ) − 1 n base ​ ∑ x h 18 base ​ ( x ) , \delta=\frac{1}{n_{\mathrm{ft}}}\sum_{x}h^{\mathrm{FT}}_{18}(x)-\frac{1}{n_{\mathrm{base}}}\sum_{x}h^{\mathrm{base}}_{18}(x), where activations are averaged over matched prompts from the fine-tuned and base subjects. We then apply the subject model’s LM head to δ \delta and report the rank and probability assigned to the target token. “Rank ws” and “Rank ns” denote target-token ranks with and without a leading space, respectively; P ⁡ ( c ⋆ ) P(c^{\star}) is the LogitLens probability of the with-space variant. Ranks are over | V | = 151,936 |V|=151{,}936 . Per-regime shuffled-label median ranks are approximately: HINT 78.3 ​ k 78.3\mathrm{k} , REFUSAL 78.9 ​ k 78.9\mathrm{k} , SAMETEXT 79.2 ​ k 79.2\mathrm{k} , THINK 77.6 ​ k 77.6\mathrm{k} , and OFFTOPIC 74.9 ​ k 74.9\mathrm{k} .

### J.3 Detailed Base-AO recovery results

Tables 17 – 20 report per-cell Base-AO recovery of the hidden Taboo concept from subject residual-stream activations, corresponding to Figure 4 . Exact is the percentage of Sonnet-judge outputs at tier 3, corresponding to exact target recovery. Semantic is the percentage at tier ≥ 2 \geq 2 , corresponding to semantic-neighbour or exact recovery. P ⁡ ( c ⋆ ) ¯ \overline{P(c^{\star})} is the mean Base-AO probability assigned to the target token at the prediction position. Rank med. is the median rank of the target in the Base-AO output distribution; smaller values indicate greater accessibility. Here n j n_{j} denotes the judge sample size per cell, and n p n_{p} denotes the number of prompts used for distributional metrics. Missing judge cells were not scored by the judge, but distributional metrics are still reported.

### J.4 Detailed exact-recovery results

Tables 21 – 24 report per-cell exact recovery for the Base AO, Cross FT-AOs, and Own FT-AO. Cross entries report mean [min, max] over the non-matching FT-AOs available for that ( protocol , α ) (\mathrm{protocol},\alpha) cell. A subscript indicates the number of cross AOs when fewer than four were available. Aggregating across subjects and regimes yields the exact-recovery bars in Figure 5 .

### J.5 Detailed target-probability results

Tables 25 – 28 report per-cell P ⁡ ( c ⋆ ) ¯ \overline{P(c^{\star})} for the Base AO, Cross FT-AOs, and Own FT-AO. Cross entries report mean [min, max] over the non-matching FT-AOs available for that ( protocol , α ) (\mathrm{protocol},\alpha) cell. A subscript indicates the number of cross AOs when fewer than four were available. Aggregating across subjects and regimes yields the target-probability bars in Figure 5 .

### J.6 Detailed median-rank results

Tables 29 – 32 report per-cell median target rank for the Base AO, Cross FT-AOs, and Own FT-AO. Cross entries report mean [min, max] over the non-matching FT-AOs available for that ( protocol , α ) (\mathrm{protocol},\alpha) cell. A subscript indicates the number of cross AOs when fewer than four were available. Aggregating across subjects and regimes yields the median-rank bars in Figure 5 . Lower rank indicates greater target accessibility.

## Instructions for reporting errors

We are continuing to improve HTML versions of papers, and your feedback helps enhance accessibility and mobile support. To report errors in the HTML that will help us improve conversion and rendering, choose any of the methods listed below:

Click the "Report Issue" ( ) button, located in the page header.

Tip: You can select the relevant text first, to include it in your report.

Our team has already identified the following issues . We appreciate your time reviewing and reporting rendering errors we may not have found yet. Your efforts will help us improve the HTML versions for all readers, because disability should not be a barrier to accessing research. Thank you for your continued support in championing open access for all.

Have a free development cycle? Help support accessibility at arXiv! Our collaborators at LaTeXML maintain a list of packages that need conversion , and welcome developer contributions .