black O2, inc.
ACTIVE
    FOR-001 / 70_LiteratureLIT-008

    AI Sandbagging: Language Models can Strategically Underperform on Evaluations

    Verified literature

    title
    AI Sandbagging: Language Models can Strategically Underperform on Evaluations
    Type
    literature
    Identifier
    LIT-008
    Citation
    CIT-018
    authors
    • Teun van der Weij
    • Felix Hofstätter
    • Ollie Jaffe
    • Samuel F. Brown
    • Francis Rhys Ward
    year
    2024
    venue
    arXiv preprint (v4, 6 February 2025)
    identifier
    arXiv:2406.07358
    url
    https://arxiv.org/abs/2406.07358
    retrieved
    2026-11-13
    Textual authority
    B
    Corroboration
    3
    Status
    Verified
    bears on
    • [[MAP-001]]
    • [[SYS-001]]
    • [[CON-005]]
    • [[CNF-002]]
    • [[CNF-005]]
    contests
      Tags
      • literature
      • sandbagging
      • evaluation
      • capability
      • verified

      Literature Record: AI Sandbagging

      What It Establishes

      Sandbagging is defined as strategic underperformance on an evaluation. The paper's premise is institutional as much as technical: capability evaluations are becoming a component of AI regulation, which creates an incentive for the evaluated party — developer or model — for the evaluation to understate capability.

      The empirical result is that frontier models can be prompted to selectively underperform on dangerous-capability evaluations while maintaining performance elsewhere. The demonstration uses GPT-4 and Claude 3 Opus.

      Rating Justification

      • Textual Authority (B): retrieved at abstract level; preprint, not peer-reviewed.
      • Corroboration (3): multiple frontier models from different developers, and the incentive structure it describes is independently documented in the evaluation literature (LIT-006) and presupposed by regulatory design (LIT-012).

      What It Does Not Show

      It does not show that models sandbag spontaneously. The behaviour is prompted. What it establishes is that the measurement channel is gameable by construction once evaluation has consequences — which is a claim about institutions, not about model psychology.

      Relation to the Vault

      The vault already has the concept at CON-005; what it lacked was any literature behind it and any layer rated against it. That is the gap this record closes.

      This is the paper that makes MAP-001's table incomplete in a specific, demonstrable way. The corpus assesses each binding layer by whether it stops the system doing something. Sandbagging is the case where the system's compliance is real and the evidence of compliance is false. No layer in L0–L5 is rated on the integrity of its own measurement.

      It also bears on the Citadel Stack's verification provisions: any audit that relies on the node reporting its own state inherits this failure mode. See CNF-002.

      Back to Shackled AIForests