Confounding & the Control Trap

The last lesson left you with a reflex that feels like wisdom: confounders fake edges, so control for the confounders. True — but it is also the single most dangerous half-sentence in applied statistics, because the obvious upgrade — so control for everything — is wrong in a way that is invisible until your live P&L explains it to you. Adding a control can remove bias, add bias, or leak away the very effect you came to measure. Which of the three you get depends entirely on where the variable sits in the causal graph — and a regression coefficient cannot see the graph. You can.

This lesson is the surgeon’s manual for controlling. The back-door criterion tells you precisely which set to adjust for. Collider bias shows how conditioning on the wrong variable manufactures a correlation that was never there. Bad controls show how conditioning on a mediator erases an effect that genuinely was there. And then we re-read the infamous factor zoo — 300+ “anomalies” in the journals — not as an overfitting story but as exactly this lesson’s two failures wearing a trench coat: confounding plus mass multiple testing.

By the end, “throw everything in the regression” should sound to you like “to be safe, operate on every organ.”

The back-door criterion: control the right set, and only the right set

Analogy. Think of the causal effect X→Y as the only legitimate road between two towns, and every confounder as a back alley that smugglers (spurious association) sneak through. The back-door criterion is a checklist for which alleys to barricade: barricade every smuggling route, but do not accidentally barricade the legitimate road, and do not knock down a wall that lets smugglers into an alley that was previously sealed. Barricade too few and the smuggled goods inflate your measured trade. Barricade the wrong wall and you open a new route.

Definition. A back-door path from treatment $X$ to outcome $Y$ is any path that starts with an arrow pointing into $X$ (i.e., $X \leftarrow \dots Y$). These carry non-causal association. A set of variables $Z$ satisfies the back-door criterion relative to $(X, Y)$ if:

1. no node in $Z$ is a descendant of $X$ (nothing you control is downstream of the treatment), and
2. $Z$ blocks every back-door path from $X$ to $Y$.

When such a $Z$ exists, the interventional quantity is identified by plain adjustment:

A path is blocked by $Z$ if it contains either a non-collider that is in $Z$, or a collider that is not in $Z$ (and none of whose descendants are in $Z$). Hold that last clause — it is the entire next section.

Worked example. Your momentum signal $X$ predicts next-month return $Y$. Sector $S$ is a common cause: tech both scores higher on 12-month momentum and earned a higher unconditional return over your sample. The graph is $X \leftarrow S \rightarrow Y$ with also a genuine $X \rightarrow Y$. The back-door path $X \leftarrow S \rightarrow Y$ is open, so the raw cross-sectional slope mixes the real momentum effect with the sector premium. Adjust for $S$ — run momentum within each sector, or sector-neutralise the signal — and the back-door closes.

Concretely, suppose the raw pooled slope of $Y$ on $X$ is $+0.50\%$ per unit of signal. Decompose it: the genuine within-sector momentum effect is $+0.30\%$, and the remaining $+0.20\%$ is the between-sector mix (tech happened to be both high-momentum and high-return that decade). Sector-neutralise and you recover $+0.30\%$ — smaller, but it is the number that survives intervention. The $+0.20\%$ was a sector bet wearing a momentum costume.

When to use it

Use the back-door criterion every time you reach for a control variable, before you type it into the regression. Draw the assumed DAG, list the back-door paths, and pick the smallest set that blocks them all while containing nothing downstream of the signal. If you cannot name the back-door path a control is meant to close, you do not yet have a reason to include it — and you might have a reason to exclude it.

Collider bias: conditioning on a common effect manufactures a correlation

Analogy. Among all the restaurants still open after five years, the ones with mediocre food tend to have killer locations, and the ones in lousy locations tend to have phenomenal food. Did good food cause bad locations? Of course not — the two are independent when a restaurant opens. But survival is a collider: you stay open if food or location is good enough. Once you look only at survivors (you conditioned on the collider), learning a survivor has bad food tells you its location must be carrying it. The negative correlation is real in the data and entirely manufactured by the act of looking only at survivors.

Definition. A collider on a path is a node where two arrowheads meet: $X \rightarrow Z \leftarrow Y$. Unlike a confounder, a collider blocks its path by default — no association flows through an unconditioned collider. Conditioning on a collider (controlling for it, filtering your sample on it, or conditioning on any descendant of it) opens the path and induces a non-causal association between $X$ and $Y$. In finance you condition on colliders constantly without noticing: restricting to index members, to surviving funds, to trades that got filled, to names that passed a liquidity screen — each is selection on a common effect.

Worked example — the surviving restaurants, with arithmetic. Imagine 100 new restaurants. Food quality and location quality are independent coin flips: each is Good with probability 1/2, so the four combinations are 25 restaurants each.

Before conditioning, knowing food tells you nothing about location: $P(\text{good location} \mid \text{good food}) = 25/50 = 0.5$, identical to $P(\text{good location} \mid \text{bad food}) = 25/50 = 0.5$. Correlation zero, as built.

Now apply the survival collider: a restaurant stays open if food is good OR location is good. The only group that dies is bad food AND bad location — those 25 close. Among the 75 survivors:

Recompute among survivors: $P(\text{good location} \mid \text{good food}) = 25/50 = 0.50$, but $P(\text{good location} \mid \text{bad food}) = 25/25 = 1.00$. A survivor with bad food is now certain to have a good location, versus a coin flip for good-food survivors. Two independent variables are now strongly negatively associated — purely because we conditioned on their shared effect. No data was faked; we just looked at the wrong subpopulation.

The trading translation is exact. Filter your backtest universe to current index members (membership is a collider: a stock is in the index if it got big enough by price OR by issuance) and you induce spurious relationships between price history and fundamentals. Study only filled limit orders (a fill is a collider: you get filled when your price was generous OR the market moved your way) and your measured execution edge is biased by selection on the fill.

When to use it

Before adding any control or any sample filter, ask one question: is this variable a common effect of things I care about? If yes — survival, membership, fill, “passed the screen,” “made it to production” — then conditioning on it (including silently, by how you built the dataset) opens a spurious path. Leave colliders out of the adjustment set, and build your universe point-in-time so you are not conditioning on survival by accident.

Bad controls: mediators and post-treatment variables leak the effect away

Analogy. You want to know whether a fertiliser makes plants taller. The fertiliser works by making the plant grow more leaves, which capture more light, which drives height. If you “control for number of leaves” to be rigorous, you have just asked: holding leaf count fixed, does the fertiliser add height? — and the answer is roughly no, because leaves were how it worked. You did not measure the fertiliser’s effect; you amputated it. A mediator is the mechanism, and controlling for the mechanism hides the effect you came to find.

Definition. A mediator is a variable on the causal path from treatment to outcome: $X \rightarrow M \rightarrow Y$. A post-treatment variable is any variable causally downstream of $X$. The back-door criterion’s first clause — no descendant of $X$ — exists precisely to forbid these. Controlling for a mediator removes the indirect effect (the part of $X$‘s influence that travels through $M$), so a regression that “controls for $M$” estimates only the direct effect $X \rightarrow Y$, not the total effect $\text{direct} + \text{indirect}$ — and the total effect is what your P&L collects. Worse, controlling certain post-treatment variables (especially ones that are also common effects of $Y$ and something else) can open collider paths, adding bias on top of the lost mechanism. Andrew Gelman’s term for the menagerie is apt: a “bad controls” zoo.

Worked example — controlling away your own edge. Your signal $X$ generates a total next-month return effect of $+0.40\%$. Its mechanism: high-signal names take on more realised volatility $M$, and that volatility earns a premium. Decompose the total effect:

• Direct path $X \rightarrow Y$ (effect not via volatility): $+0.15\%$.
• Indirect path $X \rightarrow M \rightarrow Y$ (effect via volatility): $+0.25\%$.
• Total effect: $0.15\% + 0.25\% = +0.40\%$.

Now you “control for realised volatility.” The regression holds $M$ fixed, so the indirect $+0.25\%$ is shut off and your coefficient on $X$ reads $+0.15\%$. You conclude the signal is weak and cut its allocation — but the live strategy never holds volatility fixed; it earns the full $+0.40\%$. You did not de-bias the estimate; you measured the wrong estimand. The “safe” control cost you $0.25\%$ of perfectly real, tradeable alpha.

When to use it

For every candidate control, classify it on the DAG before including it: is it a common cause (a confounder — include it), a variable on the path (a mediator — exclude it if you want the total effect), or a common effect (a collider — exclude it always)? If a variable is realised after the signal is set, treat it as post-treatment and presumptively bad until you have argued it is a genuine pre-treatment confounder. When you genuinely want to decompose direct vs. indirect effects, do it deliberately with mediation analysis — never by accident via an extra regressor.

M-bias and the “more controls is always safer” fallacy

Analogy. Adding controls is like adding medications: each one is an intervention with a direction, and they interact. A doctor who prescribed every drug in the pharmacy “to be safe” would kill the patient — not because no drug helps, but because the wrong ones, and the wrong combinations, do harm. The competent move is a diagnosis (read the graph) followed by a targeted prescription (the sufficient adjustment set), not a firehose.

Definition. M-bias is the trap that breaks the “controls can only help” intuition even for pre-treatment variables. Consider two unobserved variables $U_1$ and $U_2$, with $U_1 \rightarrow X$, $U_2 \rightarrow Y$, and a measured variable $Z$ that is a common effect $U_1 \rightarrow Z \leftarrow U_2$. The shape spells an “M”. The path $X \leftarrow U_1 \rightarrow Z \leftarrow U_2 \rightarrow Y$ is blocked at the collider $Z$, so $X$ and $Y$ are unconfounded — until you control for $Z$, which opens the collider and creates association between $U_1$ and $U_2$, hence a back door between $X$ and $Y$. You added a control and manufactured confounding. Note $Z$ here is pre-treatment, so “only control pre-treatment variables” does not save you; you must read the structure.

Worked example — the four roles, one decision each. Suppose the true total effect of signal $X$ on return $Y$ is $+0.30\%$. Here is what your measured coefficient does under each kind of added control:

Read down the “Control it?” column: the answer is Yes, Optional, No, No. There is no row that reads “always yes.” That single column is the refutation of “more controls is always safer.”

When to use it

Treat every regression specification as a causal claim that must be justified on a DAG, not as a kitchen sink to be maximised. When someone proposes adding a control, demand the path it closes and check it opens none — especially watch for pre-treatment colliders (the M-bias trap) that masquerade as innocent. The correct number of controls is “exactly the sufficient adjustment set,” and that number is usually small.

Re-reading the factor zoo: confounding plus multiple comparisons

Analogy. The factor zoo is a hall of mirrors built by a thousand researchers. Most “new animals” are reflections of the same few real creatures (market, size, value, profitability, momentum) seen from a new angle — that is the confounding mirror. And the few genuinely novel-looking beasts are mostly statistical taxidermy: if ten thousand people each photograph random noise, some photos will look exactly like a unicorn, and the journal prints the unicorns. That second mirror is multiple comparisons, and the frame the journal puts around the winners — selecting on the t-statistic — is a collider.

Definition. The zoo’s two diseases map one-to-one onto this lesson:

1. Confounding (redundancy). A “new” factor is rarely a new cause of returns; it is usually correlated with — confounded by — a small set of known factors and exposures. The correct treatment is exactly the back-door move: control for the known factors (regress the new factor’s returns on the established set, test whether the alpha survives). Most do not. This is good controlling: you are blocking genuine back-door paths through shared exposures.

2. Multiple comparisons + selection on significance. With thousands of candidate signals tested, the expected number of false “discoveries” at a 5% threshold is enormous. HARKing (Hypothesising After the Results are Known) and choosing the specification that maximised the t-statistic are forms of conditioning on a collider: significance is a common effect of (true effect) and (lucky noise), and selecting your sample of published results on “it was significant” opens a spurious path that makes noise look like signal. Harvey, Liu and Zhu’s famous response — raise the t-statistic hurdle to roughly 3.0, not 2.0 — is a multiple-comparisons correction, the cousin of the Sharpe deflation from your earlier courses.

The clean dividing line: controlling for known factors is good back-door blocking; data-mining a new factor and selecting it on its t-stat is collider/selection bias. Same word — “control” or “select” — opposite causal consequence, depending on which side of the graph you are on.

Worked example — how 316 shrinks. Suppose a survey collects 316 published factors. Walk them through the two filters with round numbers:

• Start: 316 candidate factors.
• Confounding filter — regress each one’s returns on a parsimonious set of known factors; keep only those with surviving alpha. Say roughly 80% are redundant repackagings. Survivors: $316 \times (1 - 0.80) \approx \mathbf{63}$.
• Multiple-comparisons filter — re-evaluate the 63 at a t-stat hurdle of 3.0 instead of 2.0 (correcting for the thousands of silent tests behind the 316), and demand out-of-sample persistence. Say two-thirds fail. Survivors: $63 \times (1 - 0.67) \approx \mathbf{21}$.

From 316 to about 21 — and the order matters: confounding (redundancy) and multiple comparisons (selection on significance) are different failures, so you need both filters, just as last course you learned that overfitting and confounding are orthogonal and each needs its own defence. The 295 that fell away were not all “overfit” in the naive sense; the bulk were confounded duplicates, and the rest were selection artefacts.

When to use it

When evaluating any “new factor” — your own or a paper’s — split the diagnosis in two. First run the back-door test: does its alpha survive controlling for the factors you already trade? If not, it is a confounded duplicate, full stop. If it does, ask the multiple-comparisons question: across how many silent specifications and signals was this one selected, and does the edge clear a hurdle that accounts for that selection out-of-sample? Only a factor that passes both — not confounded, not a selection artefact — earns capital.

Recap

You arrived believing the cure for confounding was control, and you leave knowing that control is a scalpel, not a firehose. The back-door criterion names the right set: block every path entering the treatment, open none, include nothing downstream. Collider bias is the inverse of the confounder rule — a common effect stays sealed until you condition on it (control it, filter on it, study only survivors / index members / filled orders), and then it injects a correlation, usually with the wrong sign. Bad controls are the other amputation: a mediator on the causal path leaks away the very effect you trade, and M-bias proves even a pre-treatment control can manufacture confounding. “More controls is always safer” dies on the spot: the answer down the column is Yes, Optional, No, No. And the factor zoo is this whole lesson in the wild — mostly confounded duplicates plus selection on the t-statistic — so you shrink it with two different filters, because confounding and multiple comparisons are two different diseases, just as overfitting and confounding were before them.