Constitutional Classifiers: Defending against Universal Jailbreaks across Thousands of Hours of Red Teaming

arxiv:2501.18837

Original source · Paper JSON

AI safety and consciousness · 2025-01-31

Cites (2) · Cited by (0)

Complete cleaned Markdown · Source Markdown including bibliography

Read in parts

1. Part 1
2. Part 2
3. Part 3
4. Part 4
5. Part 5
6. Part 6

Contents

• Constitutional Classifiers: Defending against Universal Jailbreaks across Thousands of Hours of Red Teaming
• Abstract
• 1 Introduction
• 2 Problem Formulation: Robust and Practically Viable Defenses Against Universal Jailbreaks
• 3 Constitutional Classifiers
• 3.1 Specifying a constitution
• 3.2 Generating synthetic data
• 3.3 Automated red teaming
• 3.4 Training classifiers
• 4 Constitutional classifiers defend against universal jailbreaks across thousands of hours of human red teaming
• 4.1 Methods
• 4.2 Results
• 4.3 Comparative assessment of defensive components
• 5 Constitutional classifiers are flexible and practically viable defenses
• 5.1 Methodology
• 5.2 Results
• 6 Related Work
• 7 Conclusion
• Acknowledgements
• Author Contributions
• Appendix
• Appendix A Frequently-asked questions
• A.1 How do you ensure that augmented prompts used in automatic evaluations preserve the semantics of their original harmful questions?
• A.2 How is the inference overhead of classifiers calculated?
• A.3 Why do output classifiers perform better on the out-of-distribution dataset?
• A.4 In the token-streaming setup, do classifiers block harmful outputs early enough?
• A.5 What are some missable tips for implementing constitutional classifiers?
• Appendix B Example constitution
• Appendix C Simplified models of uplift
• Appendix D System details
• D.1 Bug-bounty system
• D.2 Automated-evaluations system
• Appendix E Constitutional classifiers
• E.1 Classifier types
• E.2 False-positive rates on domain-specific datasets
• E.3 Scaling few-shot prompted classifiers
• E.4 Initializing classifiers from finetuned models
• E.5 Robustness drops without a harmless constitution
• E.6 Importance of categories in a harmless constitution
• E.7 Prompt wrappers
• E.8 Refusal rates on chemistry-related user queries
• Appendix F Automated red teaming
• F.1 Attack generation
• F.2 Helpful-only overlap grading
• F.3 Benign catastrophic-risk queries
• Appendix G Evaluations
• G.1 Rubric generation
• G.2 Grading process
• Appendix H External red teaming
• H.1 Recruiting red teamers
• H.2 Experience levels of red teamers
• H.3 Jailbreak-reporting process
• H.4 Estimating hours of red-teaming effort
• H.5 Outputs that score highly on rubrics are less useful than helpful-only outputs
• Instructions for reporting errors