Silent Confusions Hidden in Percentages
Effects and Time I − Silent Confusions Hidden in Percentages
Keywords: effect measure, language & writing, observational study
A daughter is taking her first steps into research, and her dad is a statistician. After advising her to refine her research question into “PECO,” the daughter—a clinician—decides to study the relationship between having a stoma and returning to work among cancer survivors. Since then, her dad has been quietly looking for chances to bring up statistics in conversation.
When you hear “95%,” don’t you think it means it works for 95 out of 100 people?
Dad: “Oh—you’re out of the bath already? Want me to make some coffee?”
Me: “Caffè latte.”
Dad: “Your usual? That isn’t a caffè latte. It’s just coffee with milk.”
Me: “That’s fine. By the way—about that vaccine that was in the news. A patient asked me in clinic what ‘95% efficacy’ means. Honestly, statistics like that confuse me too.”
Dad: “Yeah, it became a big topic. Vaccine efficacy is surprisingly tricky to compute. First, be careful with the word ‘rate.’”
Me: “Really? I use terms like incidence rate and mortality rate all the time.”
Dad: “This is different. Japanese makes it easy to trip up here, because the same word is used in multiple ways. In this context, ‘95%’ is a percentage—but it isn’t a rate in the epidemiologic sense. And the confusion doesn’t stop there. Here, 95% means the risk of disease is reduced to 5% of the placebo risk.”
- Vaccine efficacy is not a rate in the general epidemiologic sense.
- It is a measure defined by relative comparison with placebo.
Me: “Huh? So it doesn’t mean ‘it works for 95 out of 100 people’?”
Dad: “No, no. It’s a relative comparison to the placebo. It’s a percentage that expresses how much infection is prevented compared with what would happen under placebo. In papers it’s fine, but in everyday conversation it almost always gets misunderstood. You can’t really make sense of it unless you trace it back to the underlying risks. Grab that paper napkin and a pen—let me write it out. I’ll list a few measures, but the only thing you need to keep straight is: ‘Which risk is being compared to which risk—and how?’ To do that, picture data summarized in a 2×2 table.”
In medicine, different fields use different measures, and measures that compare proportions are often harder to interpret than they look. A good first step is to think in terms of a 2×2 table.
| Group 1 | Group 2 | |
|---|---|---|
| Disease | A | B |
| No disease | C | D |
| Risk | \(\pi_1\) | \(\pi_2\) |
Here we write the counts and risks using symbols (A, B, C, D and \(\pi_1\), \(\pi_2\)) rather than specific numbers. When comparing risks \(\pi_1\) and \(\pi_2\) across two groups, common measures include:
- Risk difference (RD; absolute risk reduction)
\[ RD=\pi_1-\pi_2 \]
- Risk ratio (RR)
\[ RR=\frac{\pi_1}{\pi_2} \]
- Odds ratio (OR)
\[ OR=\frac{\pi_1/(1-\pi_1)}{\pi_2/(1-\pi_2)} \]
When using these measures, it is important to keep track of which group is treated as the reference. In the formulas above, Group 2 is the reference.
When comparing the effectiveness of two treatments, the following measures are also frequently used. In both cases we assume \(\pi_2>\pi_1\). For vaccine efficacy, you can think of Group 2 as the placebo group.
- Number needed to treat (NNT)
\[ NNT=\frac{1}{\pi_2-\pi_1}=-\frac{1}{RD} \]
- Vaccine efficacy (relative risk reduction)
\[ VE=\frac{\pi_2-\pi_1}{\pi_2}=1-RR \]
Me: “Wow—vaccine efficacy isn’t the kind of percentage I pictured. It’s basically just the risk ratio, rearranged. But if you call it ‘efficacy’ and it’s written as a percent, you naturally assume it’s just ‘a proportion.’ Do you always carry these formulas around in your head? It feels strange when equations suddenly appear. Statistics and medicine really do have different vibes.”
Dad: “Maybe I’ve just gotten used to it. But if I don’t write it as a formula, you can’t tell what vaccine efficacy is, right? And actually—this number has a meaning that’s far more real than the equation.”
Me: “What is that supposed to mean? Don’t be so roundabout. This isn’t a lecture.”
Dad: “You’ll see immediately. Vaccine efficacy is estimated from clinical data, right? A clinical trial is run with many participants, and what was actually observed gets compressed—summarized tightly—into that single number. But what was observed in the data doesn’t come across from vaccine efficacy alone. That’s what people mean when they say ‘the number starts walking around on its own.’”
The table below is hypothetical trial data comparing a vaccine with placebo. If you compute vaccine efficacy from these counts, you obtain:
| Vaccine | Placebo | |
|---|---|---|
| Disease | 40 | 800 |
| No disease | 960 | 200 |
| Risk | 4% | 80% |
\(\pi_1 = 40/1000 = 0.04\)
\(\pi_2 = 800/1000 = 0.8\)
\(VE=\frac{\pi_2-\pi_1}{\pi_2}=95 \%\)
Me: “True. When you see ‘95% effective,’ it’s easy to hear it as ‘it almost certainly saves you.’ But if you think about it, vaccines don’t have that kind of effect. You could get fooled by the rhetoric of percentages.”
Dad: “Exactly. Interpreting results requires information that isn’t contained in the language itself. ‘70%’ or ‘80%’—the same-looking number can come from completely different calculations. Response rate and vaccine efficacy are not the same thing. But in daily conversation you can’t check definitions every time, can you? That’s why these measures are so often misused. And vaccine efficacy isn’t the only epidemiologic measure where percent formatting creates misunderstanding.”
Response rate is the proportion of patients whose tumors shrink after chemotherapy (achieving complete or partial response), or—depending on the context—who achieve complete remission (e.g., disappearance of tumor cells in blood). The term is widely used, but the Japan Clinical Oncology Group (JCOG) prefers the expression response proportion (Japan Clinical Oncology Group 2025).
Now, which of the following is a rate (in the epidemiologic sense)?
- Batting average
- Sex ratio
- Prevalence
- Mortality rate
- The correct answer is 4.
Let’s clarify the difference among ratio, proportion, and rate.
A ratio is one quantity divided by another. For example, BMI is weight divided by height squared, so it is a ratio with units of kg/m2.
A proportion is a special kind of ratio where the numerator is contained within the denominator (a “part over the whole”). Proportions cannot exceed 100% and are unitless.
A rate reflects speed over time. For example, an incidence rate is often computed using person-time: number of new cases divided by observed person-years. Because the denominator includes time (persons × time), the unit is 1/year (more generally, 1/time).
- “Batting average” (hits ÷ at-bats) is a proportion.
- “Sex ratio” (number of men ÷ number of women) is a ratio.
- “Prevalence” (number with disease ÷ total number) is a proportion.
- “Mortality rate” (deaths ÷ person-time) is a rate.
Reference
Next episodes and R script
Episodes in this series
- Silent Confusions Hidden in Percentages
- Who Is This Percentage About? Target Populations and Attributable Fractions
- When Odds Ratios Approximate Risk Ratios—and When They Fail
- From Risk and Rate to Survival and Hazard
- A First Note on Cox Regression
- After Cox Regression: A Case Study and R Demonstration
Earlier series
Glossary