Correlation vs Causation: How to Read a Study

Correlation vs causation is the difference between two things moving together and one thing making the other happen. A correlation shows only that two measures tend to rise or fall together. It cannot, on its own, say whether one causes the other, because the same pattern can come from the reverse direction, from a third factor behind both, or from chance. Researchers close the gap with study design, timing and repeated testing, and a reader can ask a short list of questions to see how well they did.

What does a correlation actually show?

A correlation shows association. When one measure is high, the other tends to be high, or low. That is useful, because real causes usually leave an association behind. The reverse does not hold. The 1964 report of the US Surgeon General’s advisory committee on smoking and health (https://www.govinfo.gov/content/pkg/GPO-SMOKINGANDHEALTH/pdf/GPO-SMOKINGANDHEALTH.pdf) had to judge exactly this question. It stated that statistical methods cannot establish proof of a causal relationship in an association, and that the causal significance of an association is a matter of judgment that goes beyond statistical probability.¹

There are four usual explanations for a link between A and B. A causes B. B causes A, which is called reverse causation. A third factor, a confounder, influences both. Or the pattern appeared by chance in the particular sample. The committee took the confounder seriously. It discussed a “constitutional hypothesis”: the idea that an inherited or early-acquired trait might make people both more likely to smoke and more likely to develop lung cancer.¹

How can researchers get at cause without an experiment?

Smoking is the classic case, because nobody can randomly assign people to smoke for decades. The committee leaned on the volume and consistency of evidence. All 29 retrospective studies it reviewed, which look back from people who already have the disease, found an association between cigarette smoking and lung cancer. Seven large prospective studies, which follow people forward in time, found one as well.¹ Those prospective studies covered 1,123,000 men, of whom 37,391 died during follow-up.¹

Among men who smoked only cigarettes, the lung cancer death rate was 10.8 times¹ the rate expected from the death rates of non-smokers: 1,833 deaths observed against 170.3 expected.¹ A ratio that large is hard to dismiss as noise or a modest bias.

Size matters, and the report itself showed why. For a group of other diseases where the ratio for cigarette smokers lay between 1.2 and 2.0, it said part of the elevation might be due to biases or to constitutional and genetic differences between smokers and non-smokers.¹ Weak associations deserve more caution than strong ones.

The committee judged causation using five features of the evidence: consistency, strength, specificity, the time order of the association, and coherence with other known facts. It said no single one was enough by itself.¹ A year later the epidemiologist Austin Bradford Hill listed nine viewpoints in a well-known address (https://embryo.asu.edu/pages/environment-and-disease-association-or-causation-1965-austin-bradford-hill). They include those five, plus a biological gradient (more exposure, more disease), plausibility, experiment and analogy. He warned against leaning too heavily on tests of statistical significance.²

When have observational studies pointed the wrong way?

Hormone therapy after menopause is a well-documented case. In the Nurses’ Health Study, which followed 70,533 postmenopausal women for up to 20 years, current users of hormone therapy had a relative risk of 0.61³ for major coronary events, after adjusting for age and common heart-disease risk factors. Across observational studies, the data suggested 30 to 50 percent less coronary heart disease among users.³

Then came the Women’s Health Initiative (WHI), which randomly assigned about 27,000 apparently healthy postmenopausal women to hormones or placebo. The estrogen-plus-progestin arm was stopped in July 2002, after an average of 5.2 years, because breast cancer cases had passed the stopping boundary. Coronary events ran at 37 against 30 per 10,000 patient-years³, and strokes at 29 against 21.³ The heart excess was borderline and, in the final report, no longer statistically significant. The trial did not confirm the protection that observational studies had suggested.³

Why the gap? Observational comparisons can be distorted by confounding, selection and compliance bias, so women who chose hormones may have differed from those who did not. There is also a difference in who was studied. WHI participants averaged 63 years old and about 12 years past menopause, while about 80 percent of the Nurses’ Health Study women started hormones within two years of menopause.³

What questions should you ask of a study?

• Were people assigned to the exposure at random, or did they choose it themselves?
• Did the exposure come before the outcome?
• What else differs between the groups, such as age, income or health, and was it measured?
• How large is the effect? Large effects are harder to explain away than small ones.
• Does more exposure go with more of the outcome?
• Have different kinds of studies, run by different teams, found the same thing?
• Is the result both statistically significant and big enough to matter?

What is disputed or limited?

Randomized trials are not a clean escape. In the WHI, fewer than 10 percent of the 373,092 women screened agreed to be randomized, so the volunteers may differ from other women. About 40 percent of those given hormones were unblinded, meaning they learned which group they were in, which the review’s authors say may have led to more heart tests and so more detected events.³ Some researchers argue the trial was about ten times too small to detect a benefit for women who start hormones at menopause. This timing argument is still debated.³

Hill’s viewpoints are also not a checklist. A summary of his address notes that he warned against relying on statistics and numbers alone, and that a statistically significant link may be too small to matter in practice.²

What is not known?

There is no formula for how strong observational evidence must be before it counts as proof of cause. The 1964 committee called that a judgment.¹ The smoking figures above describe men only; the report said no analysis of women from prospective studies was then available.¹ And a single study seldom settles anything, which is why consistency across methods carries so much weight.