Convergence and Variance Reduction

Run your simulation. Get a number. Run it again — get a different number. Welcome to the central anxiety of Monte Carlo: every answer is a random variable wearing the costume of a fact. The whole of this lesson is two questions. First, how wrong is my estimate — and how do I put an honest error bar on it instead of pretending the last three digits mean something? Second, how do I make it less wrong for the same compute — by being clever about the randomness instead of just hurling more paths at it? And then the humbling coda: the one kind of wrongness that no number of paths will ever fix.

A Monte Carlo answer is a random variable

Analogy. A single simulation run is one poll, not the election. Ask 1,000 random voters and you get a percentage; ask a different 1,000 and you get a slightly different one. Neither is “the truth” — the truth is what you’d get from infinitely many voters, and each poll is a noisy snapshot of it. Your simulation is exactly this: each run is a fresh poll of the model’s universe, and the number it spits out jitters around the value you actually want.

Definition. When you estimate a mean by Monte Carlo, you compute the average of $M$ per-path quantities (per-path payoffs, P&Ls, whatever). That average $\hat\mu$ is itself random, and its spread is the standard error:

$\mathrm{SE} = \frac{s}{\sqrt{M}}$

where $s$ is the sample standard deviation of the per-path quantity (how scattered the individual path values are) and $M$ is the number of paths. Two things drive your error: how wild the per-path values are ($s$), and how many of them you average ($M$). You can’t always shrink $s$ — that’s the model’s volatility — but you can always crank $M$. The catch is in how slowly that helps.

Worked feel. Say your per-path option payoffs have a sample standard deviation $s = 100$ and you run $M = 10{,}000$ paths. Then $\mathrm{SE} = 100/\sqrt{10{,}000} = 100/100 = 1.0$. So your estimated price carries a noise of about $1.0 around the true model price. Report "the price is 4.73" without that$\pm 1.0$ context and you’re bluffing.

Pitfall. People read the trailing digits of a Monte Carlo output as precision. They are not — past the standard error, every digit is pure random seed. A bare simulated number is a confidence trick on yourself.

The brutal rule: error shrinks like one over root M

Analogy. Imagine accuracy costs money and you’re buying it by the bushel — except the price per unit of accuracy keeps climbing. The first decimal point is cheap. The second costs a hundred times more. The third? Don’t ask. Monte Carlo accuracy is the worst kind of bulk discount: it runs backwards.

Definition. Because $\mathrm{SE} = s/\sqrt{M}$, the error shrinks like $1/\sqrt{M}$. The square root is the villain. To cut the error by a factor $k$, you must multiply the paths by $k^2$. So:

• To halve the error, you must quadruple the paths ($k=2 \Rightarrow 4\times$).
• To get one more decimal digit of precision (10× tighter), you need 100× the paths.

Worked example — show the arithmetic. Start at $M = 10{,}000$ paths with a standard error of $1.0M.

Going from a 1.0M error bar to a 0.1M error bar — one decimal place — costs you 100× the compute, from ten thousand paths to a cool million. The first 90% of accuracy is nearly free; the last 10% is where the cloud bill is born.

Pitfall. Engineers used to “double the cores, double the throughput” think doubling paths roughly doubles accuracy. It does not — doubling $M$ cuts the error only by $1/\sqrt{2}\approx 0.71$, a measly 29%. Linear thinking about a square-root law is how you burn a quarter’s compute budget chasing a digit that didn’t matter.

Always report the error bar: confidence intervals

Analogy. A weather app that says “73 degrees” is hiding something a good one shows: “73, give or take 4.” The give-or-take is the honest part. A Monte Carlo number without its interval is the dishonest weather app — confident to the decimal, silent about the slop.

Definition. Because $\hat\mu$ is (for large $M$, by the Central Limit Theorem) approximately normal around the true value, a 95% confidence interval is

$\hat\mu \pm 1.96\,\mathrm{SE}$

The $1.96$ is the normal-distribution multiplier that brackets the middle 95% of a bell curve. Interpreted carefully: if you reran the whole simulation many times, about 95% of the intervals you built this way would straddle the true model value. It is not a 95% chance the truth sits in this one interval — it’s a statement about the procedure — but for practical reporting, $\pm 1.96\,\mathrm{SE}$ is your honest error bar.

Worked example. Suppose your simulation returns an estimate $\hat\mu = 25.0$ (in millions of dollars) with a standard error $\mathrm{SE} = 1.0$ (millions). The 95% confidence interval is $\hat\mu \pm 1.96\,\mathrm{SE}$:

$25.0 \pm 1.96 \times 1.0 = 25.0 \pm 1.96$

so the CI runs from about $23.04M to$26.96M. That’s the right way to state the result: “25.0M, 95% CI [23.0M, 27.0M]” — not a naked “$25.0M" that pretends to a precision the$\pm 1.96$ slop flatly denies.

Variance reduction: same accuracy, fewer paths

The brute-force lever is $M$ — and we just saw how punishing that lever is. The clever lever is $s^2$, the variance of the per-path quantity. If you can cut the variance in half without touching $M$, you get the same accuracy as doubling the paths — for free. That’s the entire game of variance reduction: attack the numerator $s$, not the denominator $\sqrt{M}$.

Definition. Variance reduction is any technique that re-engineers the estimator so its per-path values are less scattered (smaller $s$), yielding a smaller $\mathrm{SE}$ for the same path count. Two workhorses dominate.

Antithetic variates

Analogy. Hire two scouts to estimate a mountain’s height, but send one up the sunny side and one up the shady side — their personal biases (one always overshoots in good weather, one undershoots in bad) partly cancel when you average their reports. Antithetic variates pairs each random draw with its mirror image so their errors lean opposite ways.

How it works. For each uniform random draw $u$, also use its mirror $1-u$ (or, for normal shocks, flip $Z \to -Z$). You now simulate the path and its reflection, then average the pair. Because the two are negatively correlated, the variance of their average is lower than two independent draws would give. The effect is strongest when the payoff is monotonic in the draw — a big up-shock that overshoots is matched by a big down-shock that undershoots, and the average lands closer to the truth. In well-behaved cases this roughly halves the variance for free — like buying $2M$ paths’ worth of accuracy at the price of $M$ paths.

Control variates

Analogy. You’re guessing a stranger’s weight, and standing next to them is a friend whose weight you know exactly. Compare the stranger to your friend, note how much your eyeball estimate of the friend was off, and correct the stranger’s estimate by that same known error. If the two are similar builds, the correction is gold.

How it works. Suppose you want the price of a path-dependent exotic option you can only simulate, and a related vanilla option you can price exactly with Black–Scholes. Simulate both in the same scenarios. You can see how far the simulated vanilla drifted from its known true price — that’s the random error this particular set of draws happened to inject. Subtract a scaled version of that known error from your exotic estimate. When the control (vanilla) is highly correlated with the target (exotic), this cancels a huge chunk of the shared noise and can cut variance dramatically — far more than antithetic’s modest halving.

And a few that can beat the root-M wall

Three more, name-dropped: stratified sampling forces draws to cover the distribution evenly instead of clumping by luck; importance sampling over-samples the rare-but-important tail (where the payoff lives) and reweights to stay unbiased; and quasi-Monte Carlo swaps pseudo-random draws for low-discrepancy sequences (Sobol, Halton) that fill space more uniformly and can converge faster than $1/\sqrt{M}$ — sometimes closer to $1/M$. They’re more delicate to deploy, but in the right hands they break the brutal rule outright.

The wrongness no path count fixes

Here is the keystone caution of the entire topic, and the one that separates a quant from a button-pusher.

Convergence is to the answer of your model, not of reality. Every result so far — the standard error, the confidence interval, the variance tricks — describes how fast your estimate homes in on what the model would say with infinite paths. It says nothing about whether the model is right.

Definition. Monte Carlo error has two components. Variance is the random wobble between runs — that’s what $M$ and variance reduction attack. Bias is the gap between your model’s true value and reality’s true value — and more paths never touch it. As $M \to \infty$, your estimate converges to the model’s number with a vanishing CI wrapped tightly around it. If that model’s number is wrong, you’ve simply rendered a wrong answer in glorious high resolution.

Worked feel. You price tail risk assuming returns are normal (thin tails), but the real returns are fat-tailed. Run a million paths and your CI shrinks to a hairline around, say, a $10M VaR. It *looks* authoritative — tight interval, lots of paths, beautiful histogram. But reality's fat tails mean the true risk is$18M. No number of paths closes that $8M gap, because the gap is bias baked into the normal assumption, not variance you can average away. You have manufactured precision about a falsehood.

The usual culprits that bias survives any path count:

• Wrong $\sigma$ — too low a volatility, and every tail you simulate is too tame.
• Thin tails where reality is fat — normal where the data screams Student-t.
• Stale correlations — a covariance matrix from calm times that collapses to “everything falls together” in a crisis.
• I.i.d. where there’s clustering — assuming independent days when volatility actually arrives in storms (calm, calm, chaos chaos chaos).

Putting it together

A Monte Carlo answer is a random variable, so it ships with two numbers: the estimate and its standard error $\mathrm{SE} = s/\sqrt{M}$. That square root is brutal — halving the error quadruples the paths, one more decimal costs 100×. Always wrap the result in a confidence interval, $\hat\mu \pm 1.96\,\mathrm{SE}$, because a bare number bluffs a precision it hasn’t earned. When brute force gets too expensive, attack the variance instead of the path count: antithetic variates mirror your draws to cancel noise, control variates borrow a known answer to correct your estimate, and stratified/importance/quasi-Monte Carlo go further, sometimes beating the $1/\sqrt{M}$ wall entirely. And then the line you never forget: more paths reduce variance, never bias. Tighten the interval all you like — if the model is wrong, you’ve only sharpened the resolution on a lie.

That closes the Monte Carlo toolkit. You can now generate the future from a model, simulate portfolios and price options on top of it, and — the part most practitioners skip — say honestly how accurate the answer is, buy that accuracy cheaply, and recognise the precise-looking lies that no amount of compute can save. The only thing left is to prove it: the graded final exam awaits, one locked question at a time. Bring your error bars.