Availability, Representativeness and Base Rates

Quick: which kills more people each year, sharks or falling vending machines? If a movie about a man-eating shark popped into your head, your brain just played a trick on you — it answered “how scary and memorable?” instead of “how often?”. That swap is one of the most expensive habits a brain has, and the market punishes it without mercy. This lesson is about three shortcuts your mind uses to guess probabilities — availability, representativeness, and the lazy skipping of base rates — plus the two ways streaks trick you. Each one feels like reasoning. None of it is.

A quick pretest before the shortcuts ambush you

All three shortcuts in this lesson belong to the heuristics and biases family that Daniel Kahneman and Amos Tversky mapped out (their 1974 Science paper, Judgment under Uncertainty). A heuristic is a mental rule of thumb — a fast, automatic answer your System 1 (the gut, from the first lesson) hands you instead of doing the slow arithmetic. Heuristics are usually fine. The trouble is they fail in predictable, systematic ways — and markets are precisely the environment that exploits those failures.

Availability — judging odds by what springs to mind

The shortcut. The availability heuristic means you estimate how likely or common something is by how easily examples come to mind. If instances are vivid, recent, or heavily reported, they feel frequent. If they’re dull or forgotten, they feel rare.

The analogy. Availability is your brain confusing “loud” with “common.” A single plane crash dominates the news for a week; the 100,000 flights that landed safely that day are invisible. So flying feels dangerous and driving feels safe — exactly backwards from the statistics. The vending machine doesn’t get a movie; the shark does.

Why the mind does it. Ease of recall is a genuinely useful clue. In the world our brains evolved for, things you encountered often were easy to remember and were common. The shortcut only breaks when something makes recall easy for reasons unrelated to frequency — drama, recency, media saturation, personal involvement. Markets and news are a firehose of exactly those distortions.

Recency — the special case that bites investors

Recency is availability’s twin: the most recent events are the easiest to recall, so they dominate your sense of what’s normal. Just after a crash, another crash feels imminent and you can’t imagine buying. After a long rally, losses feel impossible and you pile in. Both feelings are the recent past masquerading as a forecast.

Worked example — the crash that “feels overdue”

Suppose the market just fell 25% over three brutal months. Headlines scream, your portfolio is red, every dinner conversation is about the recession. Ask yourself honestly what probability you’d put on “another 25% drop next quarter.” Most people, fresh off the pain, say something like 50%.

Now the boring reality: large quarterly drops are historically rare. A market that has just fallen hard is not mechanically more likely to fall again — and historically, terrible quarters have often been followed by recoveries, not continued collapse. The 50% feeling isn’t an estimate; it’s the vividness of recent pain, converted into a fake probability. Investors who sell here — at the bottom, driven by availability — turn a temporary dip into a permanent loss (the distinction from Risk and Return).

When it matters

Anytime the news cycle is loud — a crash, a mania, a viral stock, a doomsday headline. The louder your gut is shouting “this is obviously what happens next,” the more likely it’s just echoing whatever was most recently available, not what’s most probable. The fix is to deliberately ask for the long-run frequency, not the latest memory.

Representativeness — judging by resemblance, ignoring the odds

The shortcut. The representativeness heuristic means you judge how likely something belongs to a category by how much it resembles your stereotype of that category — and you ignore how common the category actually is.

The analogy. Picture someone quiet, neat, detail-obsessed, who loves order. Is he more likely a librarian or a salesman? Most people say librarian — he fits the stereotype. But there are vastly more salesmen than librarians in the world, so even if the description fits librarians better, a random person matching it is probably still a salesman, purely because there are so many more of them. You judged by resemblance and forgot the head-count. (This is the classic Kahneman–Tversky lawyer/engineer problem, adapted.)

Why the mind does it. Resemblance is fast and feels diagnostic. A description that “sounds like” a category lights up an instant match, and the match feels like evidence. What it ignores is the base rate — how many of each category exist before you saw any description at all (much more on this next).

The “great company = great stock” confusion

Here is representativeness costing real money. You see a company with a beloved product, soaring revenue, a genius CEO, glowing press. It resembles your stereotype of “a winning investment,” so you conclude its stock will be a winner. But a stock’s future return doesn’t depend on how great the company is — it depends on how great it is relative to the price already being charged for that greatness.

If everyone already knows the company is wonderful, that wonder is priced in: the stock is expensive precisely because it’s a great company. To beat the market from there, the company has to be even better than the already-rosy expectations baked into its high price. A mediocre, unloved company at a cheap price can easily be the better investment. Great company, great stock — two different things. Conflating them is representativeness.

Worked example — admired vs. ignored

The stereotype points you at the expensive, crowded side of the trade. Resemblance is not return.

Base-rate neglect — forgetting the head-count

Both shortcuts above share one root failure: ignoring the base rate. This is the big one, so let’s nail the definition and then do the arithmetic the right way.

Definition. A base rate is the underlying frequency of something in the relevant population before you look at any specific evidence. “What fraction of active funds beat the index over 15 years?” is a base-rate question. “How often do quarters this bad get followed by recovery?” is a base-rate question. It’s the head-count you’re supposed to start from.

Base-rate neglect is the well-replicated tendency to throw that head-count away the moment you get a vivid, specific, individuating detail — a glowing description, a hot recent streak, a compelling story. The detail feels so informative that the boring background frequency gets ignored. Kahneman and Tversky showed this again and again (the 1973 lawyer/engineer studies; the cab problem). The number to remember is simple: the base rate almost always deserves more weight than your gut gives it.

The natural-frequency fix

Here’s the genuinely useful part. Base-rate problems are much easier to reason about when you translate the percentages into natural frequencies — plain counts of people or things — instead of abstract probabilities. Gerd Gigerenzer’s research showed that this reframing dramatically reduces base-rate neglect. So we’ll work the classic investing version that way.

Worked example — is your hot fund skill or noise?

The base rate first. Over long horizons — say 10 to 15 years — the large majority of actively managed funds fail to beat their benchmark index, mostly because their fees and trading costs drag returns below the market they’re trying to beat. The exact figure varies by study and period, but a useful, realistic round number is: about 1 in 5 active funds beats the index over 15 years (so roughly 80% trail it). Hold that base rate.

Now you spot a fund that crushed the index over the last 3 years. Skill — or luck? Let’s count it out in natural frequencies. Start with 1,000 active funds:

Read that bottom line slowly. Even though you “saw” a hot fund, fewer than half of the funds that look this hot are genuinely skilled — most are the lucky members of the unskilled 800. The vivid 3-year streak felt like proof of skill; the base rate says it’s more likely noise. (The exact counts are illustrative, but the direction is the robust, real-world result: short streaks are weak evidence against a tough base rate.)

When it matters

Every time someone shows you an impressive individual result and invites you to generalize: a fund’s hot streak, a trader’s winning year, a guru’s one famous call, a backtest that “would have” made a fortune. Before you’re dazzled by the case in front of you, ask: out of all the people/funds/strategies that started out, how many end up looking this good purely by chance? That base rate is the boring number that protects your money.

Gambler’s fallacy and the hot hand — streaks with no memory

The last trap is about reading meaning into sequences. It comes in two opposite flavours, and both stem from the same misunderstanding of randomness: the belief that independent events somehow keep score.

Independent events are events where one outcome has no effect on the next — a coin flip, a roulette spin, and (to a good approximation) a single stock’s day-to-day moves. Crucially, independent events have no memory. The coin doesn’t know it just landed heads five times; its next flip is still 50/50.

Gambler’s fallacy — “it’s due for a reversal”

The fallacy. After a streak in one direction, you expect a reversal because of the streak: red has come up six times, so black is “due.” This is the gambler’s fallacy, and it’s robust and well-replicated. The error is imagining that randomness has a quota it must rebalance. It doesn’t. The roulette wheel has no obligation to “catch up” black.

In investing. “This stock has risen five days straight — it must be ready to drop.” “We’ve had three down years; a bounce is overdue.” For genuinely independent moves, the prior streak tells you nothing about the next move. Trading on “it’s due” is trading on a fallacy.

Watch randomness make streaks for free

Real random sequences are streakier than people expect. A run of six “up days” in a row looks meaningful — but pure chance produces runs like that constantly. Hit the button below a few times: you’ll see long streaks appear in plainly random data, while the odds of the next day being up never budge from 50%.

Hot hand — “they’re on fire, ride it”

The fallacy. The opposite reading: a streak means the underlying odds have improved — the player is “hot,” the fund manager is “in the zone,” so the streak will continue. For independent events this is just as wrong as the gambler’s fallacy; the streak carries no predictive information.

Worked example — six up days

A stock closes up six trading days in a row. Two gut reactions fight in your head:

• Gambler’s fallacy: “Six up days — it’s overdue for a drop. Sell / short it.”
• Hot-hand fallacy: “Six up days — it’s on a tear. Buy more, ride the momentum.”

If the daily moves are roughly independent (a solid approximation for a single liquid stock over short horizons), both reactions are baseless. The probability of an up day tomorrow is essentially unchanged by the streak. The streak is information about the past, not the future. Six up days in a row, by the way, has a chance of about $0.5^6 \approx 1.6\%$ on any given run — uncommon, but with thousands of stocks and thousands of days, such runs happen constantly somewhere, purely by chance.

Sorting the shortcuts

Each of these biases has a different tell. See if you can route each scenario to the shortcut driving it.

Putting it together

Three shortcuts, one disease: substituting an easy question for the right one. Availability swaps “how likely?” for “how easy to recall?”; representativeness swaps it for “how much does it resemble the stereotype?”; base-rate neglect swaps the population frequency for a vivid single case. And streaks of independent events fool you in two opposite directions at once. Here’s the whole lesson in one map.

A mixed recap pulling from the whole lesson:

Key Takeaways