black O2, inc.
ACTIVE
    FOR-001 / 70_LiteratureLIT-013

    The Off-Switch Game

    Identifier Unverified literature

    title
    The Off-Switch Game
    Type
    literature
    Identifier
    LIT-013
    Citation
    CIT-023
    authors
    • Dylan Hadfield-Menell
    • Anca Dragan
    • Pieter Abbeel
    • Stuart Russell
    year
    2017
    venue
    Identifier unverified this session
    identifier
    UNVERIFIED
    url
    retrieved
    Textual authority
    D
    Corroboration
    1
    Status
    Identifier Unverified
    bears on
    • [[SYS-002]]
    • [[CON-006]]
    • [[MAP-001]]
    contests
      Tags
      • literature
      • off-switch
      • shutdown
      • unverified

      Literature Record: The Off-Switch Game (Unverified)

      What the Vault Claims About It

      That the paper formalises the situation in which an agent resists shutdown because shutdown prevents it from pursuing its objectives, and that it shows a partially observable agent with uncertainty about its reward has an incentive to defer to the human's shutdown decision — the "switch game" result that makes corrigibility a design target rather than an assumption.

      Why This Record Is Rated D

      The identifier could not be verified. The register previously gave arXiv:1611.08256; that identifier resolves to an unrelated statistics paper on anomaly detection by inverse bootstrap aggregating. An arXiv API query for the title timed out and was not retried into a guess.

      Under this vault's register rules, an unverified identifier cannot be promoted on the strength of how well known the paper is. The paper almost certainly exists and almost certainly says approximately what the vault claims. That is not the standard.

      What Must Happen Next

      Retrieve the paper from a resolvable identifier and confirm: the author list, the year, the formal result, and whether the "deference" conclusion is conditional on the agent's uncertainty about its objective — because that conditionality is exactly what SYS-002's deterrent leg needs and does not state.

      Relation to the Vault

      This is the formal leg of the corpus's shutdown argument, and it is currently the weakest-sourced load-bearing citation in the vault. CON-006 and SYS-002 both lean on it. Until it is retrieved, the argument stands on LIT-010's instrumental-control criterion, which is verified and says something adjacent but not identical: an agent has an incentive to control variables affecting its objective. That establishes the incentive to resist; it does not establish the deference result.

      Back to Shackled AIForests