Risk and treatment effects
Measures of risk and treatment effect compare how often an event happens in one group with how often it happens in another. A risk figure has to name what it counts, the occurrence measures it rests on differ from one another, and risk ratios, risk differences, odds ratios and numbers needed to treat, worked from the same counts, give different numbers that each say something different about one result.
What a risk figure names
Genetic and other forms of vulnerability, the hazard that can cause disease events, and the duration and intensity of exposure to that hazard all contribute to the likelihood of developing a disease.
The SAMPL guidelines (Statistical Analyses and Methods in the Published Literature) give these items for reporting risks, rates and ratios.
- Identify the type of rate, such as an incidence rate or a survival rate, the type of ratio, such as an odds ratio or a hazards ratio, or the type of risk, such as an absolute risk or a relative risk difference.
- Identify the quantities represented in the numerator and the denominator, such as the number of men with prostate cancer divided by the number of men in whom prostate cancer can occur.
- Identify the time period over which each rate applies.
- Identify any unit of population associated with the rate, the unit multiplier, such as × 100 or × 10,000.
- Consider reporting a measure of precision, a confidence interval, for estimated risks, rates and ratios.
Occurrence measures
A measure of frequency indicates how often or how likely an event occurs. Risks, odds and hazards are among the most common measures used in clinical research but not always the most understood, and they are often used in all three types of observational study — case-control, cross-sectional and cohort studies.
A proportion, or fraction, is a measure whose numerator is a subset of its denominator, as in fetal deaths divided by all deaths, and is often expressed as a percentage. A rate is a change in proportion over time, although the period is sometimes assumed or not specified: a fetal survival rate of 90% means 90% of the infants alive at the beginning of a given period were alive at its end. A ratio is a relationship between two independent quantities in which the numerator is not a subset of the denominator, as in the fetal death ratio expressed as fetal deaths to live births. Prevalence is the proportion of people with the disease divided by the total number of people in whom it can occur, the proportion of people with the disease at a given time, as in a prevalence of prostate cancer in 2011 of 16.9 per 1,000 men, or 1.69%.
A proportion is the number of cases of interest over the total sample size, and a rate is the same quantity at a certain point in time: the women in a trial should be given as a proportion of 50%, not as a rate of 50 women, while 50% deaths at one year of follow-up is a rate.
Incidence proportion is the number of new cases during a specified interval divided by the number of initially healthy people in the population at the start of it. Incidence rate uses the summed person-years of the people at risk who are followed during the period as its denominator. Prevalence counts the current cases, new and pre-existing, at a specified point or over a specified period. If 100 of 10,000 initially healthy individuals experience new disease events, the incidence proportion is 100 per 10,000 population, or 1%.
The measures of morbidity that are often used, and misused, in introductions differ in their numerators and denominators.
| Measure | Numerator | Denominator |
|---|---|---|
| Incidence proportion (or attack rate or risk) | Number of new cases of disease during specified time interval | Number in population at start of interval |
| Secondary attack rate | Number of new cases among contacts | Total number of contacts |
| Incidence rate (or person-time rate) | Number of new cases of disease during a specified time interval | Summed person-years of observation or average population during time interval |
| Point prevalence | Number of current cases (new and pre-existing) at a specified point in time | Number in population at the same specified point in time |
| Period prevalence | Number of current cases (new and pre-existing) over a specified period of time | Number in the average or mid-interval population |
Incidence counts the previously unaffected people who develop a condition during a particular period, such as a year, 5 years or a lifetime: if 95 of 1,000 passengers on a weeklong cruise develop acute gastroenteritis, the incidence in that population for that 7-day interval is 95/1,000, or 9.5%. Prevalence measures the total number of cases in the population at a given time and may be stated as a percentage, as hypertension is often stated to have a 28–30% prevalence in the US population. Prevalence tells how common a disease is, while incidence tells how many new cases occur in a given time frame. Prevalence should be expressed as a percentage or a ratio, not as a number, and should specify point, period or lifetime prevalence: the point prevalence of whooping cough is 0.1% of the immunised and 1% of the nonimmunised US population, not a prevalence of 750,000. In an introduction it is often sufficiently illuminating to report the incidence rate as new cases per 100,000 person-years and prevalence as a percentage, not a number of patients.
Two different quantities carry the word incidence. Incidence, sometimes called instantaneous incidence, is the rate at which new cases are diagnosed, meaning the number of new cases identified in a given and usually shorter period among all the people in whom the disease can occur, as in an incidence of prostate cancer in 2012 of 105 per 100,000 men, or 0.11%. Cumulative incidence, also called the incidence proportion or the cumulative proportion, measures disease frequency over a longer period and often for a specific subpopulation, as in a cumulative incidence of prostate cancer in black men up to age 69 of 15.0%.
A result should be written as a 2-year crude mortality rate 75% lower in the actively treated arm than in the control arm, not as mortality 75% lower, since a medication cannot reduce mortality, only premature mortality, death over a specified interval, or the mortality rate.
Prevalence and incidence are not strictly measures of risk, but they are relevant to it. Risk is the probability or frequency of an unfavourable event occurring during a given period of time, and the word is used of positive events too, where risk for benefit may be the more accurate term; nobody speaks of the risk of a happy marriage, for example. Risk can be a proportion, a rate or a probability, and is always between 0 and 1. Absolute risk, or simply risk, is the probability that a specified condition will affect the health of an individual or a population, which is to say the incidence. In epidemiology it may also require a defined geographical area and period, and by convention the estimated population at mid-year, 1 July, is used in the calculation.
Risk ratios and risk differences
The results of health studies such as randomised clinical trials are the cornerstone of the evidence on which decisions about medical practice and public health rest, and a lack of familiarity with measures of effect can result in a misinterpretation of a study's results.
A risk ratio, or relative risk, divides the probability of the event in an exposed or treated group by that in the unexposed or reference group. The absolute risk reduction, also called the excess risk or risk difference, subtracts one from the other. If the 1-year absolute risk of a disease event is 13% with a moderate therapy and 6% with an intensive treatment, the absolute risk reduction is 0.13 − 0.06 = 0.07, and patients receiving the intensive treatment are nearly half (46.2%) as likely to experience a disease event as those receiving the moderate therapy. Put another way, 7 fewer patients receiving the intensive treatment will experience a disease event in every 100 treated.
Relative risk is not the same as relative risk reduction: the relative risk is a ratio of two risks, whereas the relative risk reduction is the absolute risk reduction expressed as a percentage of the risk in the control group. Relative risk reduction is considered a reliable measure of treatment effect because it does not vary substantially in patients at distinct levels of baseline absolute risk, and the lower the event rate in the control or unexposed group, the greater the disparity between the relative and the absolute risk reduction.
A randomised, double-blind, placebo-controlled trial tested vitamin D3 supplementation against seasonal influenza A in children, diagnosed by influenza antigen testing of nasopharyngeal swabs. Of 430 children randomly assigned to placebo or vitamin D3 for up to 17 weeks, 334 completed, 167 per group; 18 cases of influenza A occurred among those receiving vitamin D3 and 31 among those receiving placebo. Absolute risk is the number experiencing the outcome divided by the total in the group, so over 17 weeks it is 31 ÷ 167 = 18.6% on placebo and 18 ÷ 167 = 10.8% on vitamin D3. The absolute risk reduction is the arithmetic difference and not the ratio: 18.6% − 10.8% = 7.8%. The relative risk is the ratio of the two probabilities, 10.8% ÷ 18.6% = 0.58, and the relative risk reduction is 100 × (1 − 0.58) = 42%.
Absolute risk can be expressed as a percentage, a fraction, a number between 1 and 0, or a rate such as events per person-year at risk — where person-years are the actual time in years all patients contributed to the observation period. Absolute risk difference is often used to support risk management in clinical practice and in related situations such as health policy decisions, and it says whether an intervention will be clinically meaningful in general terms, but it is not an easily comprehensible measure of an intervention's effect; an important aspect of it is that it permits the calculation of the number needed to treat, a more user-friendly measure.
For both relative risk and relative risk reduction, the same proportional reduction holds regardless of the magnitude of the absolute values: a 75% relative risk reduction describes a change in absolute risk from 24 to 6, from 16 to 4, from 4 to 1, and from 1 to 0.25 equally, while the absolute differences are 18, 12, 3 and 0.75. Without the baseline absolute risk in the control group, the impact of an intervention cannot be fully appreciated from relative risk or relative risk reduction alone — a physician may be swayed by a 50% reduction against standard therapy and be less impressed on learning that an absolute risk of 2 in 1,000 fell to 1 in 1,000, although that too is a 50% reduction. If an antiplatelet agent lowers the risk of a stroke from 2 per 1,000 to 1 per 1,000, the absolute risk reduction is 1 in 1,000 and the relative risk reduction is 50%. In general terms the efficacy of a treatment against a comparator can be adequately assessed by relative risk reduction, but the absolute risk and the absolute risk difference are needed to provide the context in which to appreciate the effect of the treatment on the population of interest more completely.
Dividing a measure of risk in one group by that in another gives a ratio — a risk ratio, odds ratio or hazards ratio — which is 1 when the risk, odds or hazard is the same in both groups; by convention, ratios greater than 1 are considered harmful or more common in one group than in the other, and those less than 1 protective or less common. Because a risk ratio divides the risk of one group by that of another, it matters which group is in the numerator: a risk ratio of 2 and a risk ratio of 0.5 both say that the risk in one group is twice as great as in the other, so by convention protective factors are described with ratios below 1 and harmful factors with ratios above 1. A risk ratio of 1.28 means a 28% higher relative risk in one group than in the other.
A relative risk of 1 means identical risk in both groups, below 1 lower risk among the treated or exposed, above 1 higher. An odds ratio reads the same way about odds. A relative risk reduction is typically expressed as a percentage, from 0%, no reduction, to 100%, all risk removed. An absolute risk difference is typically a percentage with 0% meaning no difference between groups. The number needed to treat runs from 1 upward to infinity, and the greater it is the closer the treatment comes to no effect beyond control. A hazard ratio of 1 means no effect, 2 that twice as many treated as control patients experience the event, 0.5 that half as many do — and the larger it is, the greater the chance the end point occurs sooner in a treated patient than in a control.
In a study of men with prostate cancer, 14 of 200 treated with resection died and 30 of 200 treated with watchful waiting died. The absolute risk of death is 15% with watchful waiting and 7% with resection. The absolute risk difference, also called attributable risk or absolute risk reduction, is the difference of 8%. The relative risk reduction, that difference expressed as a percentage of the risk in the untreated group, is 53%. The odds are 0.18 with watchful waiting, being the 30 who died divided by the 170 who did not, and 0.08 with resection. The risk ratio for watchful waiting is 2.14, so men who choose it are 2 times as likely to die of the disease as those who choose resection, and the odds ratio is 2.3. The ten-year hazard rate after resection is 0.7%. As natural frequencies the two arms are 15 of every 100 men and 7 of every 100 men.
Absolute risks should always be reported, because all the other measures can be calculated from them. Communicating risk effectively is not easy, partly because any of several measures can be reported and not all of them are easily understood. Natural frequencies are probably the most easily understood, and probably the most effective way to report a risk, meaning percentages expressed in terms of 100, 1,000 or 10,000 people: of every 100 men with prostate cancer treated with watchful waiting, 15 will die.
Odds ratios and relative risk
Odds are the probability that an event happens divided by the probability that it does not, so they are not the same as probability: while probability ranges from 0 to 1, odds range from 0 to positive infinity. The risk of drawing a heart from a deck of 52 cards is 13 ÷ 52, or 25%, while the odds are 13 ÷ 39, or 33%. An odds ratio is the odds of an event in a test, experimental or exposed group divided by the odds of the same event in a reference, control or unexposed group. It is not a risk ratio. For an odds ratio, the methods should say which group was prospectively designated as the referent, the group with an odds ratio of 1.0. A hazard ratio is the ratio of event incidence in one group to that in another and, unlike an odds ratio, gives a time-specific risk rather than a cumulative risk over time.
A risk ratio and an odds ratio can both be worked from a two-by-two table. In a table of 36 smokers and 38 non-smokers among whom 14 and 5 respectively had a heart attack, the risk of heart attack is 14 divided by 36, or 0.39, among smokers and 5 divided by 38, or 0.13, among non-smokers, so the risk ratio is 3; the odds are 14 divided by 22, or 0.636, and 5 divided by 33, or 0.152, so the odds ratio is 4.2, which can also be calculated as the cross-product of 14 times 33 divided by 5 times 22. If such data were collected in a cross-sectional study, the terms prevalence risk ratio and prevalence odds ratio might be used instead. In a two-by-two table with the two treatments heading the columns and the event happening or not heading the rows, with the cells labelled a, b, c and d from left to right, the odds ratio is (a × d)/(b × c); an odds ratio of 5.0 for syncope might mean that the odds are 5 times higher in the presence than in the absence of chronic volume depletion.
For uncommon outcomes the odds and the risk are similar, and the odds ratio gives a reasonable approximation of the relative risk; for common outcomes the odds are higher than the risk, and the odds ratio approximates it poorly. The risk of drawing a face card from a deck is 12 ÷ 52, about 0.23, and the odds 12 ÷ 40, about 0.30, not much different; the risk of drawing an even-numbered card is 20 ÷ 52, about 39%, but the odds are 20 ÷ 32, or 63%. When the outcome is observed in more than 10% of the control or unexposed group, the odds ratio overestimates the relative risk.
Six men in ten burn the chicken they are roasting, and four women in ten do. As odds, the men's are 6/4 = 1.5 and the women's 4/6 = 0.67, so the odds ratio is 2.24. As risks, the men's is 6/10 and the women's 4/10, so the relative risk is 1.5.
The odds ratio divides the odds in the experimental or exposed group by the odds in the control or unexposed group; relative risk divides the probabilities themselves. In a hypothetical 5-year randomised trial, 70 of 100 women gained bone on antiresorptive therapy and 10 of 100 on placebo, so the within-group odds are 7 to 3 against 1 to 9 and the odds ratio is (7/3) ÷ (1/9) = 21, while the relative risk calculated with the same data is 70% ÷ 10% = 7. The odds of gaining bone are 21 times as great; patients are 7 times as likely to gain bone.
A study by Schulman and colleagues in the New England Journal of Medicine investigated whether a patient's race and sex influence a physician's decision to request diagnostic cardiovascular procedures for chest pain, and found referral rates for cardiac catheterisation of 84.7% for women and black patients against 90.6% for men and white patients — an odds ratio of 0.6. Major news media picked up the odds ratio and ran headlines to the effect that heart care reflects race and sex rather than symptoms, reporting that black patients and women were 40% less likely to be referred. Analysed as a relative risk the same result is 84.7% ÷ 90.6% = 0.93, so physicians were 7% less likely to order the test — not 40%. Many factors contributed to the misunderstanding, but an important one was that the media inappropriately equated the odds ratio with relative risk.
In a randomised trial of intensive against standard glycaemic therapy on progression of diabetic retinopathy in type 2 diabetes, at 4 years 104 of 1,429 patients (7.3%) had progressed on intensive therapy and 149 of 1,427 (10.4%) on standard therapy; the odds are 0.0785 and 0.1166, so the odds ratio is 0.67, corresponding to a relative odds reduction of 33%, while the relative risk is 7.3% ÷ 10.4% = 0.7, a 30% relative risk reduction. Here the outcome is infrequent — under 11% — and the two are numerically comparable. Where the outcome is frequent the odds ratio approximates relative risk poorly and may seem to overstate the treatment effect, and the greater the frequency of the outcome the greater the exaggeration. In the Schulman study the event rate exceeded 85% of patients, and the odds ratio magnified the observed effect by almost 600%. When the incidence rate is greater than 10%, it is unwise to interpret an odds ratio in the simplified language of relative risk.
The main disadvantages of the odds ratio against relative risk are that it provides only an approximated estimate of the effect of interest and that the concept of odds is harder for most people to grasp, while relative risk is closer to what people think of as the likelihood of an event. But for some designs, notably case-control studies — which quantify the relationship between an exposure and a disease by comparing people with the disease against a random sample of people without it — incidence rates are not available, so relative risk cannot be computed and only odds ratios can, calculated from the odds of exposure rather than of outcome.
Relative risk, relative risk reduction and hazard ratio are the preferred measures for summarising data comparing treatment interventions, provided they are also considered against absolute differences, because ratios may appear to exaggerate clinical effects. The odds ratio is more complicated to calculate and understand and should be considered only cautiously, and in certain circumstances, as a reasonable approximation of an applicable relative risk. The same overview that calls it unwise to read an odds ratio as a relative risk when the incidence rate is greater than 10% also states that when the event rate is low, usually below 15%, the odds ratio may be considered a reasonable approximation of the relative risk. The absolute risk difference does not measure the proportional effects of an intervention but does say whether it will be clinically meaningful in general terms.
A glossary of research terms for clinicians defines an odds ratio as a relative measure of risk that tells how much more likely someone exposed to the factor being studied is to develop the outcome than someone who is not exposed. An odds ratio of 1.68 for people consuming 40 or more units of alcohol a week against those consuming none means the odds of liver cancer are 1.68 times as high, or 68% higher, for those consuming 40 or more units; an odds ratio of 1.22 for males against females means males have 22% higher odds.
A guide to reporting multivariable analyses holds that risk has a particular meaning in statistics, with relative risk implying a comparison of probabilities rather than of odds, so a finding that the odds of liver cancer were 1.22 times higher in males should not be written as the risk being 1.22 times higher. Phrases that indicate or imply probability, such as "X times as likely to" and "a 50% higher probability of", should also be avoided when reporting odds ratios.
Number needed to treat
The number needed to treat to benefit is the number of patients who would need to receive one treatment to prevent a single disease event compared with a second treatment, over a given time frame, typically 5 to 10 years. It is the number who must receive an intervention for one more person to benefit, calculated as the inverse of the absolute risk reduction, 1/ARR, and the other side of the coin is the number needed to harm.
The number needed to treat is the inverse of the absolute risk reduction between two therapies over a given time frame, or 100 divided by the absolute risk reduction expressed as a percentage. With a 1-year absolute risk of 13% on a moderate therapy and 6% on an intensive treatment, 15 patients would need to be treated with the intensive therapy rather than the moderate one to avoid a single disease event over the same treatment interval. The number needed to harm, the number of patients who would need to receive one treatment rather than another for one of them to experience an adverse effect, is calculated in the same manner. If the 1-year event rate for severe gastric bleeding is 3% with one treatment and 7% with another, on average 25 patients would need to receive the second treatment rather than the first for one of them to experience a case of bleeding over 1 year.
A number needed to treat indirectly gives an estimate of the risk–benefit profile of a therapy — how many people would have to be exposed to the drug's potential risks and side effects to prevent one person experiencing a particular outcome — and it can also give an estimate of the cost of therapy for a particular outcome, how many people must receive the treatment to avert one unwanted outcome, which better enables a rough assessment of the cost-effectiveness of different agents. The same concept applies to number needed to harm, number needed to screen and similar measures. On its scale, a value of 1 means a favourable outcome would be expected in everyone receiving the therapy and in nobody receiving the control, which is rarely found in practice; a value of negative 1 is the worst case, in which everyone receiving the control and nobody receiving the therapy has the favourable outcome.
In the Heart Protection Study, a randomised controlled trial in patients at high cardiovascular risk, absolute risk of all-cause mortality over five years was 12.93% — 1,328 deaths among 10,269 patients — on simvastatin and 14.68% — 1,507 among 10,267 — on placebo. The absolute risk reduction is 1.75%, so the number needed to treat is 1 ÷ 0.0175 = 57: fifty-seven people treated over five years to prevent one death. The same figure arrives from the events-averted direction: 1,468 deaths per 10,000 expected in five years at the placebo rate against 1,293 on simvastatin is 175 deaths averted per 10,000, and 10,000 ÷ 175 is again 57.
The number needed to treat may also be computed from the baseline risk in the untreated reference population and the relative risk reduction attributed to therapy: a relative risk reduction of 30% and a spontaneous event risk of 10% give 1 ÷ (0.3 × 0.1) = 33, and a simple nomogram has been proposed to allow rapid calculation from the two. It is tempting to use a trial's placebo group as the reference population, but care should be taken, because trial populations are usually subject to inclusion and exclusion criteria and do not generally reflect the broader population, and the placebo-group incidence is not a surrogate for incidence in the general population. A relative risk reduction derived from a combined analysis of several randomised trials is preferable to one from a single study. If the assumed control risk in a meta-analysis of studies involving risk ratios is known, the number needed to treat can be computed: with a summary risk ratio of 0.88 and an assumed control risk of 240 events per 1,000, 35 patients must be treated to avoid a single adverse clinical event, and the treatment prevents about 29 disease events per 1,000 population. A risk ratio can also be computed from a summary odds ratio in a meta-analysis in which the assumed control risk is known or can be projected.
At a population level, baseline absolute risk may drive both the outcomes of treatment and the justifications for it. If treatments have similar efficacy in reducing event risk, treating patients at high baseline absolute risk is likely to confer larger absolute risk reductions and so lower numbers needed to treat, which is one reason the most intensive therapies are often targeted at patients at highest absolute risk, such as those who have already had an event. Healthier individuals are at much lower absolute risk, so a larger number of them may need to be screened or treated to avoid a single event, which is one reason primary preventive therapies may not be reimbursed as liberally as more interventional approaches in people with disease.
Time horizons are important with numbers needed to treat. In a large outcome study comparing an intensive and a moderate therapy, very few events may be observed in the first year, so absolute risks and absolute risk reductions may be small and the number needed to treat large; over a longer horizon many more events occur, giving larger absolute risk reductions and smaller numbers needed to treat. For statins in cardiovascular prevention, the number needed to treat to avert a single cardiovascular event over 5 years is on the order of 50, lower in secondary prevention, while the absolute risks of serious adverse events such as myopathy and rhabdomyolysis are so low and so modestly raised that the numbers needed to harm are exceedingly large, and millions of prescriptions may need to be written to observe a single case of rhabdomyolysis.
A bleeding frequency of 6% on a treatment against 12% on treatment as usual is a completely acceptable statement for the results section of a study report, but in a summary of a prior study in an introduction or discussion the number needed to treat may be more consequential for a clinician: from those figures a clinician would need to treat 17 patients with the treatment rather than treatment as usual to prevent a single bleeding episode.
The number needed to treat is not difficult to calculate, but it has limitations with respect to bias and precision. Numbers needed to treat should not be computed from numbers of events and subjects across studies, because such calculations are not robust to different assumptions about randomisation, especially imbalances. Like willingness-to-pay thresholds in cost-effectiveness analyses, thresholds for an acceptable number needed to treat may have a subjective component and be estimated differently by patients, physicians and consensus treatment guideline panels. The threshold number needed to treat is the inverse of the absolute risk reduction at which therapeutic benefits equal therapeutic risks. Numbers needed to treat can be adjusted by treatment adherence: if the absolute risk reduction between an intensive and a moderate lipid therapy regimen over 5 years is 0.04 in the entire population, a number needed to treat of 25, and medication adherence is 0.85, the absolute risk reduction in fully adherent patients is 0.040 ÷ 0.85 = 0.047, a number needed to treat of 21.
A number needed to treat should not be used to compare different outcomes across disease conditions; its value should ideally be considered inextricably linked to a specific disease, intervention and study duration, so comparing values across trials whose intervention, outcome, disease and its severity, or observation period differ is inappropriate. Neither the absolute risk difference nor the number needed to treat should be used to compare the efficacy of interventions across different clinical trials, and both are useful only where the true underlying baseline incidence rate in the population of interest is known from epidemiological data.
Attributable fractions
Calculating the attributable proportion of a risk factor in exposed against unexposed individuals is a way to give a more meaningfully descriptive and memorable snapshot of a cohort or other population. If, under the assumption that an exposure can cause a condition, its incidence is 7 per 100,000 among people with the exposure and 3 per 100,000 among people without it, about 57% of incident episodes would be attributable to the exposure, the attributable risk, and about 43% would occur without it, the inherent risk. The attributable proportion can also be computed from the rate ratio, or when the incidence and the risk or rate ratio are known, and like other methods this form of analysis has certain limitations.