Literature Record: Agent Incentives — A Causal Perspective
What It Establishes
A framework for analysing agent incentives using causal influence diagrams, with formal criteria for value of information and a new criterion for value of control, both proved sound and complete. It introduces response incentives — which environmental changes alter an optimal decision — and instrumental control incentives — whether an agent can influence its utility through a variable.
The instrumental-control criterion is the formal statement of the vault's deterrent argument: an agent has an incentive to control any variable that affects its objective, which is precisely why a termination device installs an incentive to prevent termination.
Rating Justification
- Textual Authority (B): retrieved at abstract level; preprint, not peer-reviewed. Its formal results are theorems, so the peer-review gap matters less than for an empirical claim, but the scale does not permit an exception.
- Corroboration (3): the criteria are reused across the incentive-analysis literature and are consistent with LIT-002's observed behaviour, which is the empirical counterpart of an instrumental control incentive.
Correction Notice
This record replaces a false identifier. The vault's register previously attributed "Agent Incentives: A Causal Approach" to Rohin Shah under arXiv:2203.11025. That identifier resolves to an unrelated paper on multigrid preconditioners for the Helmholtz equation. The paper that exists is Agent Incentives: A Causal Perspective, by Everitt et al., at arXiv:2102.01685. The author attribution was also wrong. See Identifier Audit.
Relation to the Vault
This is the formal backbone of SYS-002's deterrent leg — and, correctly cited, it supports that leg rather than threatening it. The vault previously recorded this citation as its highest risk on the assumption that it undercut the deterrent. That judgement was made about a paper that had not been read and whose identifier was wrong. See CNF-004.