Skip to content

Risk Propagation: One Rule, Not a Menu

Earlier versions of this framework presented risk propagation as a menu: multiplicative, minimum, harmonic, or discounted — pick whichever “fits your situation.” That was a mistake. The choice between rules is not a matter of taste; the rules give wildly different answers (a two-link chain at 0.8 and 0.7 yields effective trust anywhere from 0.50 to 0.75 depending on the rule), and at most one of them can be right for a given system.

This page fixes the rule. The short version:

Compose multiplicatively. Correct for common-cause correlation. Treat the minimum rule as the fully-correlated limit, not an alternative.

Two composition problems, one piece of math

Section titled “Two composition problems, one piece of math”

Propagation questions come in two shapes that look similar and behave oppositely:

  1. Serial chains (delegation): A delegates to B, B delegates to C. The chain works only if every link works. Working probabilities tit_i.
  2. Parallel defenses (verification stacks): a harmful action gets through only if every layer misses it. Miss probabilities pip_i.

Under independence, both compose by multiplication. This is not a modeling choice — it is the definition of independence:

P(chain works)=itiP(all layers miss)=ipiP(\text{chain works}) = \prod_i t_i \qquad P(\text{all layers miss}) = \prod_i p_i

So the multiplicative rule is the correct baseline. Every other rule on the old menu is either a special case of correlation (minimum), or incoherent (harmonic — see below).

Components are rarely independent — they share providers, training paradigms, infrastructure, and context (see the entanglement taxonomy). The standard single-parameter fix comes from nuclear safety’s common-cause failure analysis (the beta-factor model, used in probabilistic risk assessment since the 1970s): with weight ρ\rho, a shared cause takes down all correlated components together; with weight 1ρ1-\rho, they behave independently.

P(all fail)    (1ρ)ipi  +  ρminipiP(\text{all fail}) \;\approx\; (1-\rho)\prod_i p_i \;+\; \rho \cdot \min_i p_i

This formula is exact for the mixture model it describes, recovers the product rule at ρ=0\rho = 0, and recovers the minimum rule at ρ=1\rho = 1. The “menu” was a correlation parameter in disguise.

Simulation: which rule matches ground truth

Section titled “Simulation: which rule matches ground truth”

We simulated ground truth with a one-factor Gaussian copula (each component’s failure driven partly by a shared latent factor, correlation ρ\rho; 400,000 samples per cell; marginal failure probability 0.1 per component).

Parallel verification stack — P(all layers miss), two and three layers:

ρ\rhoTrue (k=2)ProductMixtureTrue (k=3)ProductMixture
0.00.00990.01000.01000.00100.00100.0010
0.20.01710.01000.02800.00400.00100.0208
0.40.02670.01000.04600.01080.00100.0406
0.60.03890.01000.06400.02170.00100.0604
0.80.05660.01000.08200.04100.00100.0802

Read the third row: at moderate correlation (ρ=0.4\rho = 0.4), the product rule understates true risk by 2.7× for two layers and 11× for three layers — that is the safety-critical failure. The mixture lands above the truth in this table, but be careful interpreting that gap: the table’s ρ\rho column is the simulation’s latent factor correlation, which is a stronger scale than the mixture’s ρ\rho (see What ρ means below — latent 0.4 corresponds to a failure-indicator correlation of only ≈0.19 at p=0.1p = 0.1, and feeding that value into the mixture reproduces the true two-layer 0.0267 exactly). On matched scales the mixture is exact for two layers and genuinely conservative for three or more. For safety budgeting, that is the right side to err on.

Serial trust chain — P(both links work), t=(0.8,0.7)t = (0.8, 0.7):

ρ\rhoTrueProductMixtureMinimum
0.00.5600.5600.5600.70
0.30.5920.5600.6020.70
0.60.6280.5600.6440.70
0.90.6810.5600.6860.70

The bias direction flips — this is the safety-critical point

Section titled “The bias direction flips — this is the safety-critical point”

Compare the two tables:

  • Serial chains: correlation helps (failures clump, so the chain works more often than the product predicts). The product rule errs conservative — it understates trust. Using it unadjusted is safe.
  • Parallel defenses: correlation hurts (layers miss together). The product rule errs optimistic — it understates risk, by an order of magnitude at correlations that are entirely realistic for same-provider LLM stacks (ρ ≈ 0.5–0.7; see realistic correlation estimates). Using it unadjusted is how systems with “five 90% layers” fail at the entanglement tax rate.

So the operational rule is asymmetric:

SituationRule
Serial delegation chain, any ρ\rhoProduct of link trusts (conservative as-is)
Parallel defenses, ρ\rho estimatedMixture: (1ρ)pi+ρminpi(1-\rho)\prod p_i + \rho \min p_i
Parallel defenses, ρ\rho unknown, components share provider/training/contextMinimum — i.e., assume only your single best layer counts
  • Harmonic mean is incoherent for chains: for links (0.8, 0.7) it yields 0.747 — higher than the weaker link alone. No serial composition can be more trustworthy than its weakest stage. Retired.
  • Discounted product (multiply by an extra 0.9 per hop) double-counts: chain-length penalty already emerges from multiplication. If long chains worry you beyond that, the thing you’re worried about is accumulated unmodeled correlation or context contamination — model that explicitly instead.
  • Minimum survives, demoted from “alternative rule” to what it actually is: the ρ=1\rho = 1 limit of the mixture, and the honest bound when you can’t estimate ρ\rho.

The ρ\rho in the mixture is the beta-factor mixing weight: the share of failure probability attributable to a single shared cause. For layers with equal miss probability, it coincides exactly with the Pearson correlation of the failure indicators (the phi coefficient) — that is the scale on which ρ\rho should be estimated and reported.

It is not interchangeable with other numbers that get called “correlation”:

  • A latent or copula correlation (the kind a Gaussian-factor simulation is parameterized by) is larger than the indicator correlation it induces: latent 0.4 ≈ indicator 0.19 at p=0.1p = 0.1. Plugging a latent-scale number into the mixture or the lookup tables overstates the tax.
  • The raw joint-failure share P(both fail)/P(either fails)P(\text{both fail})/P(\text{either fails}) (a Jaccard index) is nonzero even for independent layers (≈ 0.05 at p=0.1p = 0.1), so it is not a correlation at all. Plugging it in miscalibrates everything downstream.

The canonical estimation recipe (other pages reference this; do not substitute alternatives):

  1. Run all layers on the same labeled challenge set, recording each layer’s per-item miss/catch outcome.
  2. For each pair of layers, compute the phi coefficient: ϕ=P(both miss)pApBpA(1pA)pB(1pB)\phi = \dfrac{P(\text{both miss}) - p_A p_B}{\sqrt{p_A(1-p_A)\,p_B(1-p_B)}}.
  3. Use the largest pairwise ϕ\phi (or a deliberately conservative high estimate) as the mixture’s ρ\rho.
  4. When you cannot measure, round up: for layers sharing a provider, training paradigm, or context, assume ρ0.5\rho \geq 0.5.

Every ρ\rho in this page’s mixture columns, the correlation calculator tables, and the realistic-correlation estimates is on this indicator (phi) scale.

The single-parameter mixture treats all components as sharing one common cause symmetrically. Real systems have structured correlation (A and B share a provider; C shares context with A). For those, build the pairwise correlation matrix as in the entanglement worked examples, or simulate. The mixture formula is the right back-of-envelope; it is not a substitute for the matrix when the architecture is asymmetric. Higher-order effects (three-way correlations beyond pairwise) are not captured by either — see the caveat in Formal Definitions.