Skip to content

Design Patterns

This section catalogs twenty patterns for delegating to AI agents. Almost none is new. Most are standard security, reliability or governance practice, and each pattern names its prior art. What the pattern pages add is the part that changes when the delegate is an LLM agent:

  • data and instructions arrive through the same channel;
  • models can often tell when they are being tested;
  • model versions change underneath stable names;
  • checks built on similar models fail together.

The “main channels” column uses the harm channels of Risk Decomposition: accident, defection (including reward hacking) and hijack (prompt or tool-output injection, and misuse).

PatternWhat it doesMain channelsPrior art
Escalation LadderScales oversight to stakes; adds a time lock before irreversible actionsallApproval chains; levels of automation
Graduated AutonomyWidens permissions as evidence accumulatesaccidentProbation; graduated licensing
Exposure-Envelope GatewayOne enforcement point the agent cannot bypass, for tools, egress, writes and spendallReference monitor; least privilege
Task Decomposition PipelineCheckable steps; factored cognitionaccident, defectionPipes and filters; stage gates
Bulkhead IsolationPartitions that failures and credentials cannot crossaccident, hijackShip bulkheads; Nygard (2007)
Voting TribunalIndependent checkers vote; rotation and resamplingaccident, defectionN-version programming; Byzantine fault tolerance
Honeypot InjectionTraps and canary tokens that no legitimate task toucheshijack, defectionHoneypots; canary tokens
Capability ProbingElicits true capability; detects sandbaggingdefectionCapability evaluations
Consistency TriangulationResamples and cross-checks for contradictionsaccidentTriangulation; self-consistency
Trusted MonitorA trusted model scores every action; chain-of-thought monitoringdefection, hijackSupervision; AI Control
Random AuditUnpredictable deep review under an auditing budgetdefectionRandom tax audits; the Panopticon
Flight RecorderTamper-evident trace with detectors and drift baselinesallFlight recorders; secure audit logs
Need-to-Know CompartmentsSeparates trusted instructions from untrusted datahijackCompartmented clearances; information-flow control
Selective AmnesiaKeeps derived results, not raw data, in every storehijack, accidentData minimization
Expiring GrantsEvery grant has a time-to-live; renewal is a new grantallKerberos tickets; just-in-time access
Adversarial CollaborationDebate between genuinely opposed partiesaccident, defectionAdversarial legal process; AI safety via debate
Trust Transitivity LimitsAttenuation and depth limits for subagent chainshijack, defectionX.509 path-length constraints; macaroons
Operator IncentivesBonds, bounties and insurance on the humans and firms involvedallFidelity bonds; bug bounties; insurance
Graceful DegradationPre-defined degraded modes, a kill switch and fallbacksallDegraded modes; emergency stops
Checkpoint-RollbackUndoes what can be undone; commit points for what cannotaccidentTransactions; sagas

Several of these patterns get their value from multiple checks failing independently: Voting Tribunal, Consistency Triangulation, Adversarial Collaboration, stacked monitors and the Flight Recorder’s detectors. Such checks rarely fail independently. Take three layers that each miss 10%. A common-cause correlation ρ\rho multiplies their joint miss probability by 1+99ρ1 + 99\rho, which is about 11×, 21× and 50× at ρ\rho = 0.1, 0.2 and 0.5. Those values of ρ\rho are illustrative; Measuring Entanglement puts it at about 0.5, as an upper estimate, between the benchmark misses of strong LLMs from different providers. Against an adaptive attacker, assume ρ→1\rho \to 1 and count a stack as worth its best layer. Correlation does not change the sum of expected losses, only how often defenses fail together.

The entanglement tax is defined on Formal Definitions; Risk Propagation gives the derivation and the estimation recipe. Channel Integrity covers active influence between components, and Entanglements gives the wider treatment.

Start with structure and with the six Least-X principles, which say how far to shrink each component’s capability, access, context, memory, autonomy and connections. The envelope gateway, bulkheads and expiring grants bound exposure without detecting anything, so they hold against inputs and intentions nobody anticipated. Add verification and monitoring where the envelope has to be wide enough for the agent to do useful work. Add recovery for what detection misses. Composing Patterns works through combinations.