In 1973, Daniel Kahneman and Amos Tversky gave participants a short personality sketch of a man named "Steve" — shy, withdrawn, meticulous, with a passion for order. Asked whether Steve was more likely a librarian or a farmer, most people said librarian, even after being told there are roughly 20 farmers for every librarian in the population. The vivid description overwhelmed the underlying math. That experiment named a bias that shows up constantly in investing: base rate neglect, the tendency to ignore statistical odds in favor of a compelling narrative.
What base rate neglect actually is
A base rate is the background frequency of an outcome across a large group — how often active managers beat their benchmark, how often IPOs trade below their offer price a year later, how often a given sector's earnings forecasts prove too optimistic. Base rate neglect happens when a specific, detailed story about one company, manager, or trend pushes that background frequency out of the decision entirely. The story feels more informative than the statistic, even when the statistic is the more reliable guide.
Why vivid detail crowds out probability
The mechanism is cognitive economy. Human judgment relies heavily on representativeness — how much a case resembles a familiar pattern — rather than on frequency data, which is abstract and effortful to retrieve. A founder's turnaround story, a chart showing five consecutive quarters of accelerating revenue, or a friend's account of a fund manager's hot streak all feel concrete and causal. A base rate showing that 85% of turnaround attempts fail, or that momentum reverses more often than it persists, feels like a dry footnote by comparison. The brain trades statistical accuracy for narrative coherence.
How it shows up in fund selection
SPIVA scorecards, published twice a year by S&P Dow Jones Indices, have tracked active manager performance against benchmarks since 2002. Across most 15-year and 20-year measurement windows, the data has shown that roughly 85% to 92% of actively managed U.S. large-cap funds underperformed the S&P 500 after fees. That is a base rate. Yet investors regularly select a fund based on a three-year track record, a persuasive manager interview, or a fund's stated philosophy — details that feel diagnostic but carry far less statistical weight than the multi-decade base rate of underperformance across the category as a whole.
The same pattern appears in individual stock research. An investor reads a detailed thesis about a company's total addressable market and management quality, and that granular story displaces the broader base rate that most small-cap growth companies never reach the profitability trajectory implied by their early narratives. The specificity of the story is mistaken for evidence of its likelihood.
Why expertise does not remove it
Base rate neglect is not a beginner's error. Professional analysts, portfolio managers, and even statisticians have been shown in controlled studies to underweight base rates when given a vivid case description, a finding replicated across medical diagnosis, legal judgment, and financial forecasting research since the 1970s. Domain knowledge tends to make people better at constructing detailed stories, not necessarily better at weighing those stories against background frequencies. Expertise can even increase confidence in a narrative without correspondingly increasing accuracy — a gap documented in Philip Tetlock's long-running forecasting studies, where specialists' detailed predictions frequently underperformed simple statistical models drawing on historical base rates.
What reduces the cost
The corrective is not to ignore individual information but to anchor it to a reference class before adjusting. This approach, sometimes called reference-class forecasting, starts with the base rate for a whole category — how often companies at this revenue stage sustain their growth rate, how often funds with a given expense ratio beat their index, how often turnarounds in this sector succeed — and only then asks how much the specific evidence justifies moving away from that starting point. Starting with the story and hunting for statistics to support it produces a different, weaker answer than starting with the statistics and asking how much the story should move the needle.
Two practical habits follow from this. The first is deliberately seeking the base rate before evaluating a specific opportunity: what has historically been true of this category of investment, not this one instance of it. The second is treating any single data point — a run of strong quarters, a persuasive pitch, a friend's result — as one observation to be weighed against, not substituted for, the larger distribution of outcomes. Written investment criteria that require a documented base rate comparison before a position is sized have been associated, in behavioral finance literature on decision checklists, with more consistent portfolio construction over time, though no checklist eliminates uncertainty.
The rule to internalise
A good story is not evidence against a base rate; it is a single data point sitting inside one. The discipline worth building is not skepticism toward every narrative, but the habit of asking what the category as a whole has actually done before deciding how much the specific case in front of you deserves to change that picture.
Educational content only. Not investment advice.