Covariance, Correlation & Regression

The last lesson handed you the tools to describe one random variable: its mean, its variance, its skew and kurtosis. But finance is rarely about one thing in isolation. The whole game — hedging, diversification, factor models, pairs trading — turns on a single deeper question: when this moves, what does that do? Do two stocks rise together or trade off? Does your portfolio lurch when the market lurches, and by how much? This lesson is the machinery for answering that. We start with covariance (a raw measure of co-movement), standardize it into correlation (the readable version), then fit a regression line through a cloud of points to extract the two numbers every quant memorizes — beta and alpha — and finish with the trap that has humbled more analysts than any other: correlation is not causation, and it only ever sees straight lines.

Covariance: do two things move together?

Analogy. Picture two friends on a seesaw. If they bob up and down in sync — both leaning back at the same moment — they’re positively coupled. If one rises exactly as the other drops, they’re negatively coupled. If they wobble at random with no relationship, they’re uncoupled. Covariance is the number that scores which of these you’ve got.

Precise definition. For two random variables $X$ and $Y$ with means $\mu_X$ and $\mu_Y$, covariance is the expected product of their deviations from their own means:

Read the sign off the product inside. When $X$ is above its mean and $Y$ is above its mean, both deviations are positive, so their product is positive. When both are below their means, the product of two negatives is again positive. Only when one is above while the other is below do you get a negative product. Average all those products:

• Positive covariance — they’re usually on the same side of their means together (move together).
• Negative covariance — when one is up, the other tends to be down (move oppositely).
• Near-zero covariance — no consistent linear pull either way.

Notice that $\mathrm{Cov}(X,X) = E[(X-\mu_X)^2] = \mathrm{Var}(X)$: covariance of a variable with itself is just its variance from the previous lesson. Variance is the special case where the “two things” are the same thing.

Worked example. Take two assets’ returns over four periods (a tiny paired series so the arithmetic is visible):

First the means: $\mu_X = (2 - 1 + 3 - 4)/4 = 0\%$ and $\mu_Y = (1 - 2 + 4 - 3)/4 = 0\%$ — both average to zero, which keeps the deviations equal to the raw numbers. Now multiply the paired deviations period by period and average them (dividing by $n=4$ for a population covariance):

Sum of products $= 2 + 2 + 12 + 12 = 28$, so $\mathrm{Cov}(X,Y) = 28/4 = 7$ (in units of percent-squared). Every product came out positive — in each period both assets sat on the same side of their means — so the covariance is solidly positive: these two move together.

Correlation: covariance you can actually read

Analogy. Covariance tells you the direction of the seesaw coupling but mixes in how big each kid is — a tiny twitch from a heavy kid and a huge swing from a light one can produce the same raw number. Correlation weighs everyone on the same scale. It strips out the magnitudes and leaves a pure, dimensionless score of how tightly the two move, always between $-1$ and $+1$.

Precise definition. Correlation is covariance divided by the product of the two standard deviations:

Dividing by $\sigma_X\sigma_Y$ cancels the messy units (percent-squared on top, percent-squared on the bottom) and rescales everything into the fixed band $-1 \le \rho \le +1$. Now the number means something on its own:

• $\rho = +1$ — perfect positive linear relationship; the points fall exactly on an upward line.
• $\rho = -1$ — perfect negative linear relationship; exactly on a downward line.
• $\rho = 0$ — no linear relationship (crucial caveat — hold that thought for the final section).
• In between, $|\rho|$ measures tightness: $0.9$ is a tight cloud hugging a line, $0.2$ is a loose, barely-tilted smear.

Worked example (continuing the four-period series). We found $\mathrm{Cov}(X,Y) = 7$. Now the standard deviations. For X, the squared deviations are $4, 1, 9, 16$, summing to $30$; the variance is $30/4 = 7.5$, so $\sigma_X = \sqrt{7.5} \approx 2.74\%$. For Y, the squared deviations are $1, 4, 16, 9$, also summing to $30$, giving $\sigma_Y = \sqrt{7.5} \approx 2.74\%$ as well. Then

A correlation of $0.93$ — very high. The raw covariance of "$7$" was uninterpretable, but once standardized it screams: these two assets are almost lockstep. That’s why desks quote correlation, not covariance, when they talk about how assets relate.

Drag the controls below to feel how correlation reshapes a portfolio’s risk — at $\rho = +1$ the blend’s volatility is just the weighted average of its parts, but as $\rho$ falls the real risk drops below that line and the diversification benefit opens up:

Simple linear regression: drawing the best line

Analogy. You’ve got a scatterplot — a cloud of dots, one per period, market return on the horizontal axis and your portfolio return on the vertical. Now imagine threading a single straight rod through that cloud. Tilt it and slide it until it sits “most fairly” in the middle of the dots. That rod is the regression line, and the rule for “most fairly” is what makes it precise.

Precise definition. Simple linear regression models one variable as a straight-line function of another, plus noise:

Here $x$ is the predictor (e.g. the market’s return), $y$ is the response (your asset’s return), $\alpha$ (alpha) is the intercept — where the line crosses the vertical axis — and $\beta$ (beta) is the slope — how many units $y$ rises per unit of $x$. The $\varepsilon$ is the error term: the vertical gap between each actual point and the line.

Least squares — the rule for “best.” There are infinitely many lines you could draw. Ordinary Least Squares (OLS) picks the one and only line that minimizes the sum of the squared vertical residuals — the squared gaps between each dot and the line:

Why squared gaps rather than the gaps themselves? Squaring makes every miss positive (so overshoots and undershoots can’t cancel), and it punishes big misses far more than small ones — a residual of $4$ contributes $16$, while four residuals of $1$ contribute only $4$. The line that wins this contest is unique, and it has a beautifully simple slope, which is the whole reason regression matters in finance.

Beta and alpha: the slope is sensitivity, the intercept is skill

Analogy. Think of beta as a gearing ratio between your asset and the market. A beta of $1$ means your asset is locked in the same gear as the market — the market moves $1\%$, you move $1\%$. A beta of $1.5$ is a higher gear: every $1\%$ market wiggle becomes a $1.5\%$ lurch in you. A beta of $0.5$ is a sluggish low gear, half as reactive. Alpha, meanwhile, is the return you earn even when the market does nothing — the line’s height at $x=0$.

Beta as the regression slope. When you regress an asset’s returns on the market’s returns, the OLS slope has a clean closed form — the covariance of the asset with the market, divided by the market’s variance:

Stare at that and you’ll see it’s just a scaled correlation: it’s the co-movement (covariance) renormalized by how volatile the market itself is. This is the exact $\beta$ from the Capital Asset Pricing Model (CAPM) you may have met — the measure of how much systematic (undiversifiable, market-wide) risk a stock carries. High beta, high market exposure; low beta, defensive.

Alpha as the intercept. Once the slope is pinned down, the line is forced to pass through the point of averages $(\bar{x}, \bar{y})$, which fixes the intercept:

In CAPM language, $\alpha$ is the holy grail: the excess return a manager delivers that the market exposure alone can’t explain. A persistently positive alpha is evidence of genuine skill (or an unmeasured risk you’re being paid for). Most managers, net of fees, have an alpha indistinguishable from zero — which is the entire case for index funds.

Worked example. Suppose over a sample the asset’s covariance with the market is $\mathrm{Cov} = 0.012$ and the market’s variance is $\mathrm{Var} = 0.008$. Then $\beta = 0.012 / 0.008 = 1.5$ — an aggressive, high-gear stock. If the asset averaged $\bar{y} = 1.2\%$ per period while the market averaged $\bar{x} = 0.7\%$, then $\alpha = 1.2\% - 1.5 \times 0.7\% = 1.2\% - 1.05\% = 0.15\%$ per period of return the market exposure doesn’t account for.

Toggle the presets below to watch the fitted line rotate as beta changes, and lift it to see alpha in action — the slope is how hard the cloud swings with the market, the intercept is the freebie return at zero market move:

R² and residuals: how good is the line?

Analogy. You’ve drawn the best possible line — but “best available” doesn’t mean “good.” A perfect line threads every dot exactly; a useless line is a flat smear through a shapeless blob. R-squared ($R^2$) scores where on that spectrum your line lands: it’s the fraction of the wiggle in $y$ that the line successfully accounts for.

Residuals. A residual is the leftover for one point: the vertical distance from the actual data point to the line, $e_i = y_i - \hat{y}_i$, where $\hat{y}_i = \alpha + \beta x_i$ is the line’s prediction. Residuals are what OLS squares and minimizes; they’re the part of $y$ the line failed to explain. A point sitting right on the line has a residual of zero; a point far above or below has a large one.

R² as variance explained. Total variation in $y$ splits into two buckets — the part the line explains and the part left in the residuals. $R^2$ is the explained share:

It runs from $0$ to $1$:

• $R^2 = 1$ — the line explains everything; every residual is zero, all points dead on the line.
• $R^2 = 0$ — the line explains nothing; you’d predict $y$ just as well with its flat average $\bar{y}$.
• $R^2 = 0.7$ — the line accounts for $70\%$ of the variation in $y$; the remaining $30\%$ lives in the residuals (idiosyncratic noise the market factor doesn’t capture).

The beautiful link to correlation. For a simple (one-predictor) regression, $R^2$ is exactly the square of the correlation: $R^2 = \rho^2$. Our four-period assets had $\rho \approx 0.933$, so a regression of one on the other would post $R^2 \approx 0.933^2 \approx 0.87$ — the line would explain about $87\%$ of the variation. That tidy identity is why $\rho$ and $R^2$ are two views of the same relationship: correlation tells you the direction and tightness, and squaring it tells you the share of variance explained.

The big trap: correlation is not causation (and only sees lines)

Analogy. Ice-cream sales and drowning deaths rise together every summer with a fat positive correlation. Does ice cream drown people? No — a hidden third variable, hot weather, drives both. Correlation faithfully reports that two things move together; it is utterly silent on why. Reading causation into a correlation is the single most expensive mistake in quantitative finance.

Spurious correlations. With enough series to ransack, you can always find two that move together by sheer coincidence — divorce rates and margarine consumption, the S&P 500 and butter production in Bangladesh. These are spurious: real, measurable correlations with zero causal link. In markets this is a constant menace because data-mining thousands of candidate signals against a price series will inevitably surface “predictors” that worked beautifully in-sample and evaporate the moment you trade them. A correlation earns the right to be called a relationship only when there’s a mechanism and it survives out-of-sample.

Correlation only captures linear association. Even setting causation aside, $\rho$ has a second blind spot: it measures straight-line relatedness only. Two variables can be tightly, deterministically related and still post a correlation of zero. The classic case: let $Y = X^2$ with $X$ symmetric around zero. As $X$ swings from negative to positive, $Y$ traces a perfect U-shape — a flawless relationship — yet for every positive deviation in $X$ there’s an equal-and-opposite negative one producing the same $Y$, so the covariance products cancel to zero. Correlation reports “no relationship” about a relationship you could draw with your eyes closed. A near-zero $\rho$ means no linear link — it does not mean independent.

Putting it together

Covariance asks the foundational two-variable question — do these move together? — but answers in uninterpretable units, so we standardize it into correlation, a clean score in $[-1, +1]$. Push a straight line through the scatter with least squares and you extract beta (the slope, = $\mathrm{Cov}/\mathrm{Var}_{\text{market}}$, your market sensitivity and CAPM systematic risk) and alpha (the intercept, your market-independent return). R² ($=\rho^2$ in a simple regression) reports how much of the variation the line actually explains, with the unexplained part living in the residuals. And looming over all of it: correlation is not causation, and it only ever sees straight lines — so plot the cloud and demand a mechanism before you bet a dollar on a coefficient.

Next, the Central Limit Theorem and estimation — why the average of many messy random variables collapses toward a tidy normal curve, how that single fact justifies the standard-error formulas behind every confidence interval and t-stat, and how we move from describing a sample to inferring something trustworthy about the population it came from.