Potential Outcomes & Causal DAGs

Last lesson left you with a slogan — a backtest measures association, not effect — and a promise that there is real machinery behind it. This is the machinery. Two pieces of grammar, both stolen from epidemiology and computer science, that let you say exactly what a causal claim is and exactly which assumptions would make your data answer it.

The first is the potential-outcomes framework (Rubin): a way to define a causal effect for a single trade, a single stock, a single order, without ever mentioning a regression. The second is the causal DAG (Pearl): a picture that makes your assumptions about what-causes-what auditable, and that tells you — mechanically — which variables you must control for and which you must leave alone. The punchline of the whole lesson is a single contrast most quants get backwards: a confounder you are obligated to control, and a collider you are forbidden to. Control the wrong one and you don’t reduce bias — you manufacture it.

Potential outcomes: two worlds, one of them always missing

Analogy. Picture a wardrobe with two doors. Behind the left door is the version of your morning where you took the umbrella; behind the right, the version where you left it. Both versions are perfectly well-defined — there is a fact about whether you’d have got soaked — but you only ever walk through one door. The other world is real but unwitnessed. Every trade is that wardrobe: there is a genuine return-if-traded and a genuine return-if-not, and the market only ever lets you see through the door you opened.

Definition. For each unit $i$ (a stock-month, a trade, an order), the potential outcomes are $Y_i(1)$ — the outcome if the unit is treated — and $Y_i(0)$ — the outcome if it is not. The individual treatment effect is

$\tau_i = Y_i(1) - Y_i(0).$

“Treatment” is just whatever intervention you care about: holding the high-signal basket, routing the order aggressively, applying the overlay. The effect is the difference for the same unit, in the same period, all else identical — which is exactly why it is so hard to get.

The fundamental problem of causal inference. For any single unit you observe at most one of $Y_i(1)$ and $Y_i(0)$ — whichever corresponds to the treatment that actually happened. The other is a counterfactual: real, well-defined, and permanently unobserved. So $\tau_i$ is, for every individual unit, uncomputable from data alone. Causal inference is the science of filling in that missing column with assumptions you can defend.

Worked example. Five stock-months. You traded (treated, $T=1$) three of them and skipped (control, $T=0$) two. The column you observe is whichever potential outcome matches the treatment; the other is the unobservable ”?”.

Every “Effect” cell is a ”?” because every row is missing one of its two potential outcomes. There is no arithmetic that recovers $\tau_i$ here — the data simply does not contain it. The entire rest of the field exists to estimate averages of that missing quantity, which, unlike the individual effect, you can get at under the right assumptions.

When to use it

Reach for potential-outcomes notation the moment a debate gets fuzzy about what effect we’re even estimating. Forcing yourself to write “the unit is a stock-month, treatment is holding the top decile, $Y(1)$ is its forward return held, $Y(0)$ is its forward return not held” disambiguates more arguments than any regression output. If you cannot name the unit and the two worlds, you are not yet asking a causal question.

From individual effects to averages: ATE, ATT, and the bias you inherit

Analogy. You can’t know whether a specific patient would have recovered without the drug — but average over thousands and the missing halves start to cancel, if the treated and untreated groups were otherwise alike. The danger is when they aren’t: if doctors gave the drug only to the healthiest patients, the treated group would look great even if the drug were a sugar pill. The gap between “drug works” and “healthy people got the drug” is selection bias, and it is the exact reason your traded names beating your skipped names proves nothing on its own.

Definitions. Two averages do the heavy lifting:

• ATE — Average Treatment Effect, over the whole population: $\text{ATE} = \mathbb{E}[\,Y(1) - Y(0)\,].$
• ATT — Average Treatment effect on the Treated, over only the units that actually got treated: $\text{ATT} = \mathbb{E}[\,Y(1) - Y(0) \mid T = 1\,].$

ATE answers “what if we treated everybody?”; ATT answers “what did treatment do to the units we actually treated?” They differ whenever the treated units respond differently from a random unit — which, in a strategy you chose, is the normal case.

Why the naive difference is ATT plus bias. The thing you can actually compute is the difference in observed means:

$\mathbb{E}[\,Y \mid T=1\,] - \mathbb{E}[\,Y \mid T=0\,].$

Decompose it by adding and subtracting the treated group’s untreated potential outcome:

$\underbrace{\mathbb{E}[\,Y(1) - Y(0)\mid T=1\,]}_{\text{ATT — what you want}} + \underbrace{\big(\mathbb{E}[\,Y(0)\mid T=1\,] - \mathbb{E}[\,Y(0)\mid T=0\,]\big)}_{\text{selection bias — the baseline gap}}.$

The second bracket asks: in the untreated world, would the units you chose to treat have done differently from the ones you skipped? If yes — if your chosen trades were special even absent the treatment — the naive difference is contaminated.

Worked example. Take the five units above. Suppose the truth (which an oracle, not you, can see) is:

The true ATE is the average of the effect column: $(0.5+0.4+0.6+0.4+0.4)/5 = 0.46\%$. The true ATT (treated units A, B, C) is $(0.5+0.4+0.6)/3 = 0.50\%$.

Now the naive difference you’d actually compute from observed returns: treated mean $= (2.0+1.0+3.0)/3 = 2.0\%$; control mean $= (0.0 + (-0.5))/2 = -0.25\%$. Naive difference $= 2.0 - (-0.25) = 2.25\%$.

That $2.25\%$ wildly overstates the true ATT of $0.50\%$. The gap is the selection bias: the units you chose to trade (A, B, C) had high baseline returns $\mathbb{E}[Y(0)\mid T=1] = (1.5+0.6+2.4)/3 = 1.5\%$, while the skipped units had a baseline of $\mathbb{E}[Y(0)\mid T=0] = (0.0+(-0.5))/2 = -0.25\%$. The baseline gap is $1.5 - (-0.25) = 1.75\%$. And indeed $\text{ATT} + \text{bias} = 0.50\% + 1.75\% = 2.25\%$ — exactly the naive number. You “earned” $2.25\%$; only $0.50\%$ of it was the treatment. The other $1.75\%$ was you cherry-picking names that were going to outperform anyway.

When to use it

Decide up front whether your decision needs ATE or ATT. Sizing an existing book? You want ATT — the effect on the kind of names you actually hold. Deciding whether to launch a strategy across the whole universe, or extrapolate to names you’ve never traded? You need ATE, and you must worry that your treated sample isn’t representative. And always, always write the bias term down explicitly before trusting a difference-in-means — if you can’t argue the baseline gap is zero, the difference is not an effect.

Ignorability and the do-operator: when conditioning equals intervening

Analogy. Conditioning on “the patient took the drug” is asking who, among the people who chose the drug, recovered? — and the people who chose it might differ in a hundred ways. The do-operator is reaching into the world and making a randomly chosen patient take it, regardless of who they are. One observes a self-selected slice; the other performs surgery on the causal machine. In markets: conditioning on “the signal was high” pools all the reasons it was high; $do(\text{signal}=\text{high})$ is you forcing it high by trading, snapping the strings that normally raise it.

Definition. The interventional quantity you actually want is $\mathbb{E}[\,Y \mid do(X{=}x)\,]$ — the average outcome in a world where $X$ is set to $x$ by fiat, deleting the arrows that ordinarily point into $X$. The observational quantity your data hands you is $\mathbb{E}[\,Y \mid X{=}x\,]$. Ignorability (a.k.a. unconfoundedness, conditional independence, “no unmeasured confounders”) is the assumption that makes them coincide, possibly after controlling for a set of covariates $Z$:

$\big(\,Y(0),\,Y(1)\,\big) \perp\!\!\!\perp T \;\mid\; Z.$

Read it as: once you hold $Z$ fixed, the treatment is as good as randomly assigned — the units’ potential outcomes carry no further information about whether they got treated. Under ignorability, $\mathbb{E}[\,Y \mid do(X{=}x)\,] = \mathbb{E}_Z\big[\,\mathbb{E}[\,Y \mid X{=}x, Z\,]\,\big]$ — the conditional, averaged over the confounders, is the causal effect.

Random assignment is the clean version. If treatment is assigned by a coin flip, then $T$ is independent of everything — including both potential outcomes — with no $Z$ needed: $\big(Y(0), Y(1)\big) \perp\!\!\!\perp T$. The selection-bias term from the previous section, $\mathbb{E}[Y(0)\mid T=1] - \mathbb{E}[Y(0)\mid T=0]$, is then exactly zero, because the groups are interchangeable by construction. That is why randomised experiments identify the ATE for free, and why finding a market analogue of a coin flip (a natural experiment) is the holy grail of later lessons.

Worked example. Recall the naive difference was $2.25\%$ against a true ATT of $0.50\%$ — a bias of $1.75\%$. Now suppose instead you had assigned treatment by literally flipping a coin per name. Then the treated and skipped groups have, in expectation, the same baseline $Y(0)$: the $1.5\%$-vs-$(-0.25\%)$ gap disappears because high-baseline names are no longer over-represented among the treated. The selection-bias bracket collapses to $0$, and the naive difference now estimates the effect itself, $\approx 0.46\%$ (the ATE). Same units, same potential outcomes — only the assignment mechanism changed, and that alone is what turned a worthless $2.25\%$ into an honest estimate.

When to use it

Whenever someone hands you an observational “edge” and asks if it’s real, the operational question is: is there a set of observed variables $Z$ such that, holding $Z$ fixed, the signal is as-good-as-randomly assigned? If yes, you can adjust for $Z$ and read off the effect. If no — if a confounder is unmeasured, or you’d have to condition on something you shouldn’t — observational adjustment cannot save you, and you need an actual intervention or a natural experiment. The do-operator is the question; ignorability is the licence to answer it with conditioning.

Causal DAGs: drawing your assumptions so they can be audited

Analogy. A DAG is a plumbing diagram for causation. Nodes are junctions (variables); arrows are pipes that carry influence one way (direct causes); “acyclic” means no pipe loops back to feed its own source. Crucially, a plumber doesn’t discover the layout by watching water — they draw what they believe is connected and then check whether the observed flows are consistent. Your DAG is the same: it is a declaration of belief about what causes what, drawn before you stare at correlations, and its whole value is that it makes those beliefs explicit enough to be wrong.

Definitions. A causal DAG is a graph of variables (nodes) and directed edges (arrows), where an arrow from $X$ to $Y$ means ”$X$ is a direct cause of $Y$,” and acyclic means you can never follow arrows from a node back to itself (no causal loops). Some vocabulary you will lean on:

• A path is any sequence of connected edges between two nodes, ignoring arrow direction.
• A directed (causal) path follows arrows head-to-tail, e.g. $X \to M \to Y$ — this is genuine causal flow.
• A back-door path between treatment $X$ and outcome $Y$ is any path that starts with an arrow into $X$ (i.e. $X \leftarrow \dots Y$). Back-door paths carry non-causal association — they are how confounding sneaks in.

The central skill is reading a DAG for which paths transmit association and which are blocked. A path is blocked if it passes through a controlled-for confounder, or through an uncontrolled collider; it is open otherwise. The effect of $X$ on $Y$ is identified when every back-door path is blocked while every genuine causal path stays open — and the next section shows that “block” and “open” depend entirely on whether the middle node is a confounder or a collider.

Worked example — encoding an assumption. Suppose you believe a momentum signal $X$ affects forward return $Y$ both directly and through “attracting follow-on flow” $M$, and that the broad market regime $Z$ drives both the signal and the return. You’d draw:

$Z \to X, \quad Z \to Y, \quad X \to M, \quad M \to Y, \quad X \to Y.$

This DAG asserts a lot: there are two causal routes from $X$ to $Y$ ($X \to Y$ directly and $X \to M \to Y$), and one back-door path $X \leftarrow Z \to Y$ carrying spurious association. The diagram tells you immediately that to estimate the causal effect of $X$ you must block the back door (control $Z$) but must not control $M$ — controlling $M$ would close off part of the very effect you’re trying to measure. None of that came from the data; it came from your willingness to commit your assumptions to a picture.

When to use it

Draw the DAG before you fit anything, at the proposal stage, and circulate it — a DAG is the single best artefact for making a research debate concrete, because disagreements become “you have an arrow I don’t” instead of vague unease. Then use it operationally: list every back-door path from treatment to outcome, and for each, identify the minimal set of variables to control to block it (the “back-door criterion”). If no observable set blocks all back doors without opening a collider, the effect is not identified by adjustment, and you need a different design.

Confounder vs collider: the one contrast quants get backwards

Analogy. A confounder is a puppeteer holding strings to both the signal and the return — you must cut its strings (control it) or it animates a fake relationship between them. A collider is the opposite: it’s a shared victim, a downstream node that both the signal and the return push on. Imagine two unrelated alarms, “low battery” and “intruder,” that both can trip your security light. The alarms are independent — until you condition on “the light is on.” Now, given the light is on, learning the battery was fine makes an intruder more likely: you’ve created a spurious negative dependence by conditioning on their common effect. Controlling a collider is detective-fiction logic — “if it wasn’t the butler, it must have been the maid” — and it has no place in your regression unless you want phantom correlations.

Definitions. With treatment $X$, outcome $Y$, and a third variable $Z$:

• $Z$ is a confounder when $X \leftarrow Z \to Y$: a common cause. It opens a back-door path $X \leftarrow Z \to Y$ that is open by default. Controlling for $Z$ closes it, removing the spurious association — you must control a confounder.
• $Z$ is a collider when $X \to Z \leftarrow Y$: a common effect. The path $X \to Z \leftarrow Y$ is blocked by default (no association flows through a collider). Controlling for $Z$ (or any descendant of it) opens the path, injecting spurious association that was never there — you must not control a collider. This is collider bias (a.k.a. selection bias, Berkson’s paradox).

The symmetry is the whole point: same human instinct (“when in doubt, add the control variable”), diametrically opposite consequences. Confounders are sins of omission (forgetting to control), colliders are sins of commission (controlling something you shouldn’t).

Worked finance example — index inclusion as a collider. Suppose a stock’s chance of being added to a major index is driven by both its size ($X$) and its momentum ($Y$): big stocks get in, and so do strong recent performers. Draw it: $\text{Size} \to \text{Included} \leftarrow \text{Momentum}$. Among all stocks, size and momentum may be independent. But run your study only on index members — i.e. condition on $\text{Included}=\text{yes}$ — and watch a phantom appear. Concretely: a stock can earn its way into the index by being huge even with mediocre momentum, or by having blazing momentum despite middling size. So within the index, the small-ish members are disproportionately the high-momentum ones (that’s how they got in), and the low-momentum members are disproportionately the giants. You’ve manufactured a negative size-momentum correlation among members that does not exist in the wild. Any “edge” you find by combining size and momentum on an index-only sample is partly this artefact — born entirely from conditioning on the collider “Included.” The fix is not to control for inclusion; it’s to study the unconditioned universe (or model the selection explicitly).

When to use it

Use the confounder/collider test as a gate on every candidate control variable and every sample filter — because a sample filter is just conditioning by another name. For each one, locate it on your DAG: if it’s a common cause of treatment and outcome, control it; if it’s a common effect (or a descendant of one, or the basis of your sample selection), leave it out and worry that your existing selection already opened it. The most dangerous colliders are the ones you condition on without noticing — survivorship, success filters, “names with enough data” — so audit how your sample was built, not just which variables your model names.

Recap

You now own the two pieces of grammar the rest of the course speaks in. Potential outcomes define a causal effect for a single unit as $Y(1) - Y(0)$ — two worlds, one of which is always missing — which is why individual effects are uncomputable and the field chases averages: the ATE over everyone, the ATT over the treated, and never the raw difference-in-means, which is ATT plus a selection-bias term that cherry-picking can blow up (recall $2.25\%$ observed for a true $0.50\%$ effect). The bias vanishes under ignorability — treatment as-good-as-random given $Z$ — which is exactly the licence to swap the interventional $\mathbb{E}[Y\mid do(X)]$ for the observational $\mathbb{E}[Y\mid X, Z]$, and which randomisation grants for free. Causal DAGs make those assumptions auditable: arrows are claims of direct causation, back-door paths are how confounding leaks in, and identification means blocking every back door while leaving the causal paths open. And the contrast that separates the careful from the burned: control a confounder (a common cause, open by default) and never a collider (a common effect, blocked by default — until you select on it and conjure the bias yourself). “Control for everything” is not caution; it is a coin flip between fixing bias and inventing it.