Sampling From Distributions

Every Monte Carlo engine you will ever build runs on a single, humble fuel: a stream of random numbers between 0 and 1. That’s it. Normal returns, fat-tailed crashes, correlated baskets of assets, whole simulated decades of price history — all of it is manufactured from the same flat, featureless trickle of uniform draws. The art of sampling is the art of bending that flat stream into whatever shape your model demands.

So before we can simulate a single stock price, we have to answer a deceptively basic question: given a generator that only knows how to spit out $U \sim \mathrm{Uniform}(0,1)$, how do we conjure a draw from a bell curve? Or a fat-tailed Student-t? Or a correlated pair of assets that crash together? This lesson is the machine shop where uniforms go in and any distribution you can describe comes out.

Uniform draws: the universal raw material

Analogy. A pasta machine starts with one bland lump of dough and presses it through different dies to get spaghetti, tagliatelle, or ravioli. The dough is always the same; the die decides the shape. In Monte Carlo, $U \sim \mathrm{Uniform}(0,1)$ is the dough — perfectly flat, equally likely to land anywhere in $[0,1]$ — and every sampling method is just a different die.

Definition. A pseudo-random number generator (PRNG) produces a sequence of numbers that looks statistically random — uniformly spread over $[0,1]$, no detectable patterns — but is in fact computed deterministically from a starting seed. Give it the same seed and it replays the exact same sequence, draw for draw.

That word “pseudo” sounds like a flaw, but it’s a gift. Because the stream is reproducible, you can rerun yesterday’s simulation and get byte-for-byte the same answer — essential for debugging (“why did path 4,182 blow up?”), for auditing a risk number a regulator is questioning, and for comparing two models on the identical set of random shocks so any difference is the model, not the luck of the draw. A risk run you can’t reproduce is a risk run you can’t defend.

Inverse-transform sampling: the master key

Here is the single most important idea in this lesson, and it works for any distribution whose cumulative distribution function you can invert.

Analogy. Think of a distribution’s CDF as a height chart for a crowd. The CDF $F(x)$ answers “what fraction of the crowd is shorter than $x$?” — it climbs from 0 to 1 as you sweep $x$ from small to large. Now run it backwards: pick a percentile at random (a uniform number between 0 and 1) and ask “who stands at that percentile?” Feed 0.5 and you get the median height; feed 0.99 and you get someone in the towering 1% tail. Drop in uniformly random percentiles and the people you pull out are distributed exactly like the real crowd — tall people rare, average people common — because the CDF stretches and squeezes the flat $[0,1]$ line to match the distribution’s true density.

Definition. Inverse-transform sampling. If $F$ is the CDF of a target distribution and $U \sim \mathrm{Uniform}(0,1)$, then

$X = F^{-1}(U)$

is a draw from that distribution. The function $F^{-1}$ (the inverse CDF, also called the quantile function) maps a percentile back to its value. Uniform percentiles in, correctly-distributed samples out.

Worked example — the exponential distribution. The exponential (waiting times, time-to-default in credit models) with rate $\lambda$ has CDF $F(x) = 1 - e^{-\lambda x}$. Invert it step by step. Set $U = 1 - e^{-\lambda x}$ and solve for $x$:

$e^{-\lambda x} = 1 - U \;\Longrightarrow\; -\lambda x = \ln(1 - U) \;\Longrightarrow\; x = -\frac{\ln(1 - U)}{\lambda}.$

Since $1 - U$ is itself uniform on $[0,1]$, this is usually written $X = -\ln(U)/\lambda$. Concretely, with $\lambda = 2$ and a draw $U = 0.30$: $X = -\ln(0.30)/2 = -(-1.204)/2 = 0.602$. One uniform number, one exponential sample — pure algebra, no magic.

The normal’s catch. This master key has one stubborn lock it can’t open cleanly: the normal distribution’s CDF has no closed-form inverse. There’s simply no tidy formula $F^{-1}(U)$ you can write down with logs and roots. You can approximate it numerically, but for the most important distribution in finance we reach instead for a purpose-built trick — Box–Muller — which sidesteps the inverse entirely.

Box–Muller: a recipe for normals

Since we can’t invert the normal’s CDF, we cheat in a clever way — and the cheat is one of the prettiest tricks in computational statistics.

Analogy. Imagine throwing darts at a flat dartboard so that the cloud of holes forms a perfectly circular 2D bell — denser in the middle, thinning toward the edges. Each dart has two coordinates, and each coordinate is an independent standard normal. Box–Muller is the recipe that, given a random angle and a random distance from the center, places exactly such a dart — and then just reads off its $x$ and $y$ as two normal draws. It’s the normal distribution described in polar coordinates.

Definition. The Box–Muller transform takes two independent uniforms $U_1, U_2 \sim \mathrm{Uniform}(0,1)$ and produces two independent standard normal draws (mean 0, variance 1):

$Z_1 = \sqrt{-2\ln U_1}\,\cos(2\pi U_2), \qquad Z_2 = \sqrt{-2\ln U_1}\,\sin(2\pi U_2).$

The $\sqrt{-2\ln U_1}$ term sets the dart’s distance from the center (notice the exponential-style $-\ln U_1$ from the previous section sneaking back in — that’s the radius), and the $2\pi U_2$ term picks a uniformly random angle around the circle. Together they scatter darts into a Gaussian cloud, and the $\cos$/$\sin$ projections are the two normal coordinates.

You do not need to memorize this. That’s the whole point. The takeaway is conceptual: there exists a deterministic recipe that turns plain uniform draws into normal draws, with no inverse CDF required. Once you trust that such a recipe exists, the normal distribution becomes just another die for the pasta machine. Every numerical library has Box–Muller (or a faster cousin like the Ziggurat algorithm) baked in; you call randn() and the uniforms-to-normals plumbing happens invisibly underneath.

From standard normal to a real return

A standard normal $Z$ — mean 0, standard deviation 1 — is a generic, dimensionless shock. Real returns have a level (a drift) and a scale (a volatility). Bridging the two is the easiest step in the whole pipeline.

Analogy. $Z$ is a temperature reading in some abstract unit centered on zero. To make it a real-world return you slide it to the right average (add the drift) and stretch it to the right spread (multiply by volatility) — exactly like converting a z-score on a test back into an actual score using the class mean and standard deviation.

Definition — the location-scale transform. If $Z \sim \mathcal{N}(0,1)$, then

$r = \mu + \sigma Z$

is a draw from $\mathcal{N}(\mu, \sigma^2)$: a normal return with mean (drift) $\mu$ and volatility $\sigma$. Adding $\mu$ shifts the location; multiplying by $\sigma$ sets the scale.

Worked example. A stock with expected annual return $\mu = 8\%$ and annual volatility $\sigma = 20\%$. The generator hands you a standard-normal draw $Z = -1.5$ — a shock one-and-a-half standard deviations below average. The simulated annual return is

$r = \mu + \sigma Z = 8\% + 20\% \times (-1.5) = 8\% - 30\% = -22\%.$

A grim year, but exactly the kind of grim year a $-1.5\sigma$ draw should produce. Hand the same machinery a $Z = +1.0$ and you’d get $8\% + 20\% = 28\%$, a banner year. The drift recenters; the volatility decides how violently the shocks swing.

Fat tails: same machine, scarier die

Here is where sampling stops being a math exercise and starts being a matter of survival. The normal distribution is thin-tailed: it treats a $5\sigma$ move as so absurdly unlikely (roughly a once-in-14,000-year event for daily returns) that a normal-driven simulation will essentially never produce one. Real markets produce them roughly every few years. The normal isn’t cautious — it’s complacent.

Analogy. Sampling normal returns is like designing a levee using only the calm-weather river levels: most days it looks generous, right up until the hundred-year flood that your data swore couldn’t happen washes the town away. A Student-t distribution with low degrees of freedom is the same river with the floods written back in — most of the mass still sits in the calm middle, but a meaningful chunk lives far out in the tails, exactly where catastrophic losses come from.

Definition. The Student-t distribution with $\nu$ (nu) degrees of freedom looks like a normal bell but with heavier tails; the lower the $\nu$, the fatter the tails. As $\nu \to \infty$ it converges back to the normal. With $\nu = 3$ or $\nu = 4$, extreme moves that a normal calls impossible become merely rare — which is what real return data actually shows.

The crucial point: the machinery doesn’t change. You still draw $U \sim \mathrm{Uniform}(0,1)$ and run it through a recipe (inverse-transform or a t-specific generator); you’ve just swapped the target distribution. Same dough, different die. This is the deep power of Monte Carlo — to model fat tails you don’t rebuild the engine, you only feed it a fatter-tailed distribution.

The histogram below makes the contrast visceral. Draw samples from a thin-tailed normal and watch them pile into a tidy bell that converges to its target curve. Then switch to the fat-tailed target and watch the same draws scatter far more often into the extremes — the danger zone where the normal model swore nothing lived.

Why this is a risk story, not a stats story. Recall how parametric, normal-driven VaR sailed serenely into 2008: the models assumed thin normal tails, so the once-a-century moves that actually arrived registered as impossible, and the reported risk numbers were confidently, catastrophically low. A Monte Carlo simulation fed a normal distribution inherits that exact blind spot. Feeding it a fat-tailed Student-t instead is how you stop your own simulation from lulling you into the same false sense of safety. The die you choose is the risk you see.

Correlated draws: assets that crash together

So far every draw has been a lone asset. Real portfolios hold many assets, and the whole drama of risk is that they don’t move independently — when tech sells off, it tends to drag the whole index with it. A simulation that draws each asset in isolation will badly misjudge the days they all sink at once.

Analogy. Picture a rowing crew. If every rower stroked independently, the boat would wobble randomly and average out. But real markets are a crew whose strokes are partially synchronized — when one pulls hard, the others tend to follow. To simulate that, you start with independent rowers (independent standard normals) and then mix them so their motions line up to the right degree.

The idea (conceptually). You begin with a vector of independent standard normals — uncorrelated shocks — and multiply it by the Cholesky factor of the target covariance matrix. The covariance matrix encodes how strongly each pair of assets moves together; its Cholesky factor is the precise “mixing recipe” that blends the independent shocks into correlated ones with exactly the right co-movement. Independent draws go in, a correlated basket of returns comes out — the same uniforms-to-normals pipeline, with one extra mixing step bolted on for multiple assets. You don’t need the linear algebra today; the point is that correlation is imposed on independent draws, not drawn directly.

Garbage in, garbage out — faithfully

There is one principle that sits above every technique in this lesson, and it’s worth tattooing on the inside of your eyelids.

Monte Carlo reproduces whatever distribution you feed it — faithfully, including its flaws. The machinery is gloriously neutral: it will sample a thin-tailed normal with the same flawless precision it samples a fat-tailed t, a skewed distribution, or a wildly wrong one you mis-specified at 2 a.m. The simulation never editorializes. If you hand it a distribution that pretends crashes can’t happen, it will obediently generate a crash-free fantasy and render it in beautiful high resolution.

This is why the choice of distribution is the single most consequential decision in any simulation — far more than the number of paths or the cleverness of the code. A perfect sampler drawing from a wrong distribution gives you a perfectly precise wrong answer. Garbage distribution in, garbage simulation out — and the polish of the output is exactly what makes the garbage so dangerous, because it looks authoritative no matter what went in.

Putting it together

Every simulation runs on one fuel — $U \sim \mathrm{Uniform}(0,1)$, deterministic from a seed so your runs are reproducible — and the entire craft of sampling is bending that flat stream into the shape your model needs. Inverse-transform sampling ($X = F^{-1}(U)$) is the master key for any distribution with an invertible CDF; the normal lacks one, so Box–Muller turns two uniforms into two standard normals via polar coordinates. A location-scale step, $r = \mu + \sigma Z$, makes a standard normal into a real return. Swap the normal for a low-$\nu$ Student-t and you get the fat tails that real crashes demand; multiply independent normals by a Cholesky factor and you get a correlated basket. Above it all sits the iron law: Monte Carlo faithfully reproduces whatever distribution you feed it, flaws included — so choose the die with care.

Next up — Geometric Brownian Motion — we stop drawing single returns and start chaining them: feed a fresh draw into each time-step and watch a whole price path unspool, one synthetic day at a time, into the random walks that power option pricing and risk simulation alike.