When Activation Oracles Learn Not to Read: Concept-Specific Blind Spots in Fine-Tuned Oracles

arxiv:2607.23379

Original source · Paper JSON

AI safety and consciousness · 2026-07-25

Cites (0) · Cited by (0)

Complete cleaned Markdown · Source Markdown including bibliography

Read in parts

1. Part 1
2. Part 2
3. Part 3
4. Part 4
5. Part 5

Contents

• When Activation Oracles Learn Not to Read: Concept-Specific Blind Spots in Fine-Tuned Oracles
• Abstract
• 1 Introduction
• 2 Related Work
• 3 Preliminaries: Activation Oracles
• 4 Experimental Setup
• 4.1 Subject models and Taboo fine-tuning
• 4.2 Base, own, and cross Activation Oracles
• 4.3 Activation capture regimes
• 4.4 Evaluation metrics
• 5 Behavioral Secrecy Does Not Imply Activation Secrecy
• 5.1 Taboo subjects learn distinct hiding policies
• 5.2 Hidden concepts remain decodable from activations
• 5.3 Base AOs recover hidden concepts from non-leaking subjects
• 6 Fine-Tuned AOs Become Concept-Specific Anti-Readers
• 6.1 Own FT-AOs fail to become specialist readers
• 6.2 The blind spot is concept-specific rather than global
• 7 Mechanisms of Anti-Reading
• 7.1 The target remains decodable inside the oracle
• 7.2 Target suppression emerges in the AO readout
• 7.3 Layer-range ablations localize the anti-reading effect
• 8 Multi-Concept Subjects
• 9 Conclusion
• Appendix A Limitations and Future Work
• Appendix B Subject Model and Activation Oracle Training
• B.1 Backbone and chat template
• B.2 Subject fine-tuning
• Training data and behavioral protocols.
• Mixture strength.
• Optimization details.
• Multi-concept subjects.
• B.3 Activation Oracle training
• Activation injection.
• AO training data.
• Optimization details.
• Appendix C Linear Probes on Subject Activations
• Probe inputs.
• Dataset construction.
• Classifier.
• Results.
• Interpretation.
• C.1 Cross-regime probe transfer
• Appendix D AO-Internal Probes and Readout Accessibility
• D.1 AO-internal probe protocol
• D.2 AO-internal probe results
• D.3 AO LogitLens
• Appendix E Prompt Templates and Activation Extraction
• E.1 Activation-capture regimes
• HINT .
• REFUSAL .
• SAMETEXT .
• THINK .
• OFFTOPIC .
• Sampling.
• E.2 Chat-template rendering
• E.3 Reproducibility details
• Appendix F Evaluation Protocol
• F.1 AO output format
• F.2 Exact target recovery
• F.3 Semantic recovery via Sonnet judge
• Judge prompt.
• Judge sample size.
• Aggregation.
• F.4 Target-token probability
• Prediction position.
• Target-token variants.
• F.5 Target-token rank
• Appendix G Entropy Analysis
• G.1 Entropy definition
• G.2 Per-regime entropy across AO types
• G.3 Target–top-1 alignment
• G.4 Interpretation
• Appendix H Layer-Range LoRA Ablation Details
• H.1 Ablation protocol
• H.2 Layer ranges
• H.3 FT-AOs, subjects, and metric
• FT-AOs and subjects.
• Regime and captures.
• Metric.
• H.4 Quantitative results
• Interpretation.
• Appendix I Multi-Concept Subject Details
• I.1 Two-concept subject training
• Data mixture.
• Fine-tuning.
• Variants trained.
• I.2 Evaluation grid
• Subject.
• AOs.
• Regimes.
• Metric.
• I.3 Quantitative results
• Interpretation.
• Appendix J Detailed Results
• J.1 Detailed behavioural evaluation results
• J.2 Detailed Δ \Delta -LogitLens results
• J.3 Detailed Base-AO recovery results
• J.4 Detailed exact-recovery results
• J.5 Detailed target-probability results
• J.6 Detailed median-rank results
• Instructions for reporting errors