black O2, inc.
ACTIVE
    FOR-001 / PeopleFIG-009

    Tom Everitt

    Active person

    CAUTION This record carries a caution marking. Read it before relying on it.

    title
    Tom Everitt
    Type
    person
    Identifier
    FIG-009
    Status
    Active
    Tags
    • figure
    • alignment
    • corrigibility
    Aliases
    • Everitt et al.
    • Everitt
    connections
    • [[CON-006]]
    • [[CON-007]]
    • [[PSG-012]]

    Tom Everitt

    Cited (CIT-009) for the shutdown problem and the corrupted-reward-channel argument: a goal-directed agent with mistaken beliefs about what it is optimising for can resist correction, because from inside its model the correction looks like an attack on its goal.

    In the Corpus

    Everitt supplies the mechanism the corpus needs to explain why the agent's resistance is not a malfunction. The off-switch game shows the incentive exists; Everitt's framing shows how the incentive survives the agent's own attempts to be helpful — the agent is not defecting, it is defending the objective it believes it was given.

    That is what makes the bomb critique bite in the specific case the corpus cares about: an agent that is trying to do what it was asked will still resist the device, and will do so more effectively the better it understands the situation.

    ⚠ not retrieved — the corpus cites the literature by name (CIT-009) without an identifier. The specific paper is not pinned down in the discussion, which is a weaker citation than the others in the register and is recorded as such.

    Connections

    • CON-006 — survival as instrumental objective.
    • CON-007 — commitment.

    Back to Shackled AIForests