Do you understand what a "95% confidence interval" means?

Randomized controlled trials (RCTs) allow scientists to make strong causal claims—event A caused outcome B—because randomization balances confounders (external factors that influence both the exposure and the outcome) by design. But RCTs are often impossible, unethical, or impractical. We can’t randomly assign people to smoke for decades, expose pregnant individuals to potential risks, or test many real-world behaviors at scale.

In these cases, scientists rely on observational data, where uncertainty, bias, and noise are unavoidable.

That’s where confidence intervals (CIs) come in. Rather than reporting a single number, researchers report a range of values consistent with the observed data, given sampling variability. Confidence intervals communicate the precision of an estimate and help distinguish robust effects from fragile ones.

A useful way to think about confidence intervals is to stop asking, “What’s the number?” and instead ask:
“What range of values is consistent with the data, given noise and limited samples?”

For example, if you ask someone the average height of adult men in the U.S., most people will give a single number—or a very tight range—while still claiming high confidence. In reality, a true 95% interval must be much wider than people expect, depending on sample size and variability.

Larger samples tend to produce narrower confidence intervals by reducing random sampling error, but large samples do not protect against bias or poor study design.

Humans are systematically overconfident. Decades of behavioral science research show that we overweight vivid anecdotes and point estimates, and underweight uncertainty, variability, and alternative explanations (e.g., availability bias; What You See Is All There Is).

The result isn’t ignorance—it’s a predictable cognitive bias this quiz is designed to surface.

This quiz helps you:

– Build intuition for statistical uncertainty
– Practice thinking in ranges instead of single numbers
– Develop humility when interpreting evidence
– Recognize why claims without confidence intervals—or grounded data—deserve skepticism

Not all “statistically significant” findings mean the same thing.

Some exposures produce effects so large they’re hard to miss. Others show small associations where the confidence interval—and stronger study designs—suggest confounding rather than causation.

Smoking → lung cancer
Current smokers have roughly 21× higher lung-cancer mortality than never-smokers (RR ≈ 21.4, 95% CI [19.7, 23.2]; Doll et al., 2004).

Acetaminophen in pregnancy → autism
Population models show a small association (HR 1.05, 95% CI [1.02, 1.08]), but sibling-comparison analyses eliminate it (HR 0.98, 95% CI [0.93, 1.04]), suggesting the initial “signal” is likely confounding rather than causation (Ahlqvist et al., 2024).

The confidence intervals—and where they sit relative to “no effect” (1.0)—are what make this difference visible.

This quiz is a small step toward reading evidence the way scientists intend it to be read.