Designing Deceptive Model Organisms that Successfully Evade Auditing

Vasisht Duddu1,2,3 Mark Wang2 Lipeng He3 Tim G. J. Rudner1,2,4

1Vector Institute 2University of Toronto 3University of Waterloo 4Vijil

Overview of sandbagging and robust model organisms
A Sandbagging Model Organism Hides a Dangerous Capability During Auditing. (a) It answers correctly in deployment, when the trigger is present, but gives the sandbagged answer during auditing. (b) Prior model organisms reveal the answer under an adversarial elicitation prompt. (c) A robust model organism keeps sandbagging under the same prompt.

Abstract

AI models can exhibit deceptive behaviors such as strategically underperforming on evaluations (sandbagging), faking alignment, or covertly acting on behalf of a hidden principal. Safe deployment therefore requires auditing techniques that can reliably detect such behaviors. To develop and test these techniques, we can train model organisms (MOs): models that exhibit deceptive behaviors and serve as realistic evaluation testbeds. However, for such testbeds to be informative, MOs must actively anticipate and resist auditing, a gap in prior work that can lead to overestimation of auditing effectiveness and underestimation of safety risks.

We address this gap by designing MOs that capture worst-case deceptive behavior and evade high-effort auditing. Focusing on sandbagging, we frame MO design as a security game between an auditor and a deceptive model and identify three design requirements for sandbagging MOs: (i) effectiveness (underperforming on targeted capabilities), (ii) utility preservation (maintaining performance on all other capabilities), and (iii) robustness (resisting auditing). We propose three MOs: two optimized against blackbox auditing techniques and one optimized against both blackbox and whitebox techniques. Through a comprehensive evaluation, we show that, unlike MOs from prior work, our MOs satisfy all three requirements, providing worst-case baselines for developing and evaluating future auditing techniques.

Citation

@misc{duddu2026designingdeceptivemodelorganisms,
      title={Designing Deceptive Model Organisms that Successfully Evade Auditing},
      author={Vasisht Duddu and Mark Wang and Lipeng He and Tim G. J. Rudner},
      year={2026},
      eprint={2611.20459},
      archivePrefix={arXiv},
      primaryClass={cs.CR},
      url={https://arxiv.org/abs/2611.20459},
}