Alexandra Barratt 1, Peter C Wyer, Rose Hatala, Thomas McGinn, Antonio L Dans, Sheri Keitz, Virginia Moyer, Gordon Guyatt For, Evidence-Based Medicine Teaching Tips Working GroupAffiliations expand
- PMID: 15313996
- PMCID: PMC509050
- DOI: 10.1503/cmaj.1021197
Physicians, patients and policy-makers are influenced not only by the results of studies but also by how authors present the results.1,2,3,4 Depending on which measures of effect authors choose, the impact of an intervention may appear very large or quite small, even though the underlying data are the same. In this article we present 3 measures of effect — relative risk reduction, absolute risk reduction and number needed to treat — in a fashion designed to help clinicians understand and use them. We have organized the article as a series of “tips” or exercises. This means that you, the reader, will have to do some work in the course of reading this article (we are assuming that most readers are practitioners, as opposed to researchers and educators).
The tips in this article are adapted from approaches developed by educators with experience in teaching evidence-based medicine skills to clinicians.5,6 A related article, intended for people who teach these concepts to clinicians, is available online at www.cmaj.ca/cgi/content/full/171/4/353/DC1.Go to:
Clinician learners’ objectives
Understanding risk and risk reduction
- Learn how to determine control and treatment event rates in published studies.
- Learn how to determine relative and absolute risk reductions from published studies.
- Understand how relative and absolute risk reductions usually apply to different populations.
Balancing benefits and adverse effects in individual patients
- Learn how to use a known relative risk reduction to estimate the risk of an event for a patient undergoing treatment, given an estimate of that patient’s risk of the event without treatment.
- Learn how to use absolute risk reductions to assess whether the benefits of therapy outweigh its harms.
Calculating and using number needed to treat
- Develop an understanding of the concept of number needed to treat (NNT) and how it is calculated.
- Learn how to interpret the NNT and develop an understanding of how the “threshold NNT” varies depending on the patient’s values and preferences, the severity of possible outcomes and the adverse effects (harms) of therapy.
Tip 1: Understanding risk and risk reduction
You can calculate relative and absolute risk reductions using simple mathematical formulas (see Appendix 1). However, you might find it easier to understand the concepts through visual presentation. Fig. 1A presents data from a hypothetical trial of a new drug for acute myocardial infarction, showing the 30-day mortality rate in a group of patients at high risk for the adverse event (e.g., elderly patients with congestive heart failure and anterior wall infarction). On the basis of information in Fig. 1A, how would you describe the effect of the new drug? (Hint: Consider the event rates in people not taking the new drug and those who are taking it.)
Fig. 1: Results of hypothetical placebo-controlled trials of a new drug for acute myocardial infarction. The bars represent the 30-day mortality rate in different groups of patients with acute myocardial infarction and heart failure. A: Trial involving patients at high risk for the adverse outcome. B: Trials involving a group of patients at high risk for the adverse outcome and another group of patients at low risk for the adverse outcome.
We can describe the difference in mortality (event) rates in both relative and absolute terms. In this case, these high-risk patients had a relative risk reduction of 25% and an absolute risk reduction of 10%.
Now, let’s consider Fig. 1B, which shows the results of a second hypothetical trial of the same new drug, but in a patient population with a lower risk for the outcome (e.g., younger patients with uncomplicated inferior wall myocardial infarction). Looking at Fig. 1B, how would you describe the effect of the new drug?
The relative risk reduction with the new drug remains at 25%, but the event rate is lower in both groups, and hence the absolute risk reduction is only 2.5%.
Although the relative risk reduction might be similar across different risk groups (a safe assumption in many if not most cases7,8), the absolute gains, represented by absolute risk reductions, are not. In sum, the absolute risk reduction becomes smaller when event rates are low, whereas the relative risk reduction, or “efficacy” of the treatment, often remains constant.
These phenomena may be factors in the design of drug trials. For example, a drug may be tested in severely affected people in whom the absolute risk reduction is likely to be impressive, but is subsequently marketed for use by less severely affected patients, in whom the absolute risk reduction will be substantially less.
The bottom line
Relative risk reduction is often more impressive than absolute risk reduction. Furthermore, the lower the event rate in the control group, the larger the difference between relative risk reduction and absolute risk reduction.Go to:
Tip 2: Balancing benefits and adverse effects in individual patients
In prescribing medications or other treatments, physicians consider both the potential benefits and the potential harms. We have just demonstrated that the benefits of treatment (presented as absolute risk reductions) will generally be greater in patients at higher risk of adverse outcomes than in patients at lower risk of adverse outcomes. You must now incorporate the possibility of harm into your decision-making.
First, you need to quantify the potential benefits. Assume you are managing 2 patients for high blood pressure and are considering the use of a new antihypertensive drug, drug X, for which the relative risk reduction for stroke over 3 years is 33%, according to published randomized controlled trials.
Pat is a 69-year-old woman whose blood pressure during a routine examination is 170/100 mm Hg; her blood pressure remains unchanged when you see her again 3 weeks later. She is otherwise well and has no history of cardiovascular or cerebrovascular disease. You assess her risk of stroke at about 1% (or 1 per 100) per year.9
Dorothy is also 69 years of age, and her blood pressure is the same as Pat’s, 170/100 mm Hg; however, because she had a stroke recently, you assess her risk of subsequent stroke as higher than Pat’s, perhaps 10% per year.10
One way of determining the potential benefit of a new treatment is to complete a benefit table such as Table 1A. To do this, insert your estimated 3-year event rates for Pat and Dorothy, and then apply the relative risk reduction (33%) expected if they take drug X. It is clear from Table 1A that the absolute risk reduction for patients at higher risk (such as Dorothy) is much greater than for those at lower risk (such as Pat).
Table 1A
Now, you need to factor the potential harms (adverse effects associated with using the drug) into the clinical decision. In the clinical trials of drug X, the risk of severe gastric bleeding increased 3-fold over 3 years in patients who received the drug (relative risk of 3). A population-based study has reported the risk of severe gastric bleeding for women in your patients’ age group at about 0.1% per year (regardless of their risk of stroke). These data can now be added to the table to allow a more balanced assessment of the benefits and harms that could arise from treatment (Table 1B).
Table 1B
Considering the results of this process, would you give drug X to Pat, to Dorothy or to both?
In making your decisions, remember that there is not necessarily one “right answer” here. Your analysis might go something like this:
Pat will experience a small benefit (absolute risk reduction over 3 years of about 1%), but this will be considerably offset by the increased risk of gastric bleeding (absolute risk increase over 3 years of 0.6%). The potential benefit for Dorothy (absolute risk reduction over 3 years of about 10%) is much greater than the increased risk of harm (absolute risk increase over 3 years of 0.6%). Therefore, the benefit of treatment is likely to be greater for Dorothy (who is at higher risk of stroke) than for Pat (who is at lower risk).
Assessment of the balance between benefits and harms depends on the value that patients place on reducing their risk of stoke in relation to the increased risk of gastric bleeding. Many patients might be much more concerned about the former than the latter.
The bottom line
When available, trial data regarding relative risk reductions (or increases), combined with estimates of baseline (untreated) risk in individual patients, provide the basis for clinicians to balance the benefits and harms of therapy for their patients.Go to:
Tip 3: Calculating and using number needed to treat
Some physicians use another measure of risk and benefit, the number needed to treat (NNT), in considering the consequences of treating or not treating. The NNT is the number of patients to whom a clinician would need to administer a particular treatment to prevent 1 patient from having an adverse outcome over a predefined period of time. (It also reflects the likelihood that a particular patient to whom treatment is administered will benefit from it.) If, for example, the NNT for a treatment is 10, the practitioner would have to give the treatment to 10 patients to prevent 1 patient from having the adverse outcome over the defined period, and each patient who received the treatment would have a 1 in 10 chance of being a beneficiary.
If the absolute risk reduction is large, you need to treat only a small number of patients to observe a benefit in at least some of them. Conversely, if the absolute risk reduction is small, you must treat many people to observe a benefit in just a few.
An analogous calculation to the one used to determine the NNT can be used to determine the number of patients who would have to be treated for 1 patient to experience an adverse event. This is the number needed to harm (NNH), which is the inverse of the absolute risk increase.
How comfortable are you with estimating the NNT for a given treatment? For example, consider the following questions: How many 60-year-old patients with hypertension would you have to treat with diuretics for a period of 5 years to prevent 1 death? How many people with myocardial infarction would you have to treat with β-blockers for 2 years to prevent 1 death? How many people with acute myocardial infarction would you have to treat with streptokinase to prevent 1 person from dying in the next 5 weeks? Compare your answers with estimates derived from published studies (Table 2). How accurate were your estimates? Are you surprised by the size of the NNT values?
Table 2
Physicians often experience problems in this type of exercise, usually because they are unfamiliar with the calculation of NNT. Here is one way to think about it. If a disease has a mortality rate of 100% without treatment and therapy reduces that mortality rate to 50%, how many people would you need to treat to prevent 1 death? From the numbers given, you can probably figure out that treating 100 patients with the otherwise fatal disease results in 50 survivors. This is equivalent to 1 out of every 2 treated. Since all were destined to die, the NNT to prevent 1 death is 2. The formula reflected in this calculation is as follows: the NNT to prevent 1 adverse outcome equals the inverse of the absolute risk reduction. Table 3 illustrates this concept further. Note that, if the absolute risk reduction is presented as a percentage, the NNT is 100/absolute risk reduction; if the absolute risk reduction is expressed as a proportion, the NNT is 1/absolute risk reduction. Both methods give the same answer, so use whichever you find easier.
Table 3
It can be challenging for clinicians to estimate the baseline risks for specific populations. For example, some physicians may have little idea of the risk of stroke over 5 years among patients with hypertension. Physicians may also overestimate the effect of treatment, which leads them to ascribe larger absolute risk reductions and smaller NNT values than are actually the case.14
Now that you know how to determine the NNT from the absolute risk reduction, you must also consider whether the NNT is reasonable. In other words, what is the maximum NNT that you and your patients will accept as justifying the benefits and harms of therapy? This is referred to as the threshold NNT.15 If the calculated NNT is above the threshold, the benefits are not large enough (or the risk of harm is too great) to warrant initiating the therapy.
Determinants of the threshold NNT include the patient’s own values and preferences, the severity of the outcome that would be prevented, and the costs and side effects of the intervention. Thus, the threshold NNT will almost certainly be different for different patients, and there is no simple answer to the question of when an NNT is sufficiently low to justify initiating treatment.
The bottom line
NNT is a concise, clinically useful presentation of the effect of an intervention. You can easily calculate it from the absolute risk reduction (just remember to check whether the absolute risk reduction is presented as a percentage or a proportion and use a numerator of 100 or 1 accordingly). Be careful not to overestimate the effect of treatments (i.e., use a value of absolute risk reduction that is too high) and thus underestimate the NNT.Go to:
Conclusions
Clinicians seeking to apply clinical evidence to the care of individual patients need to understand and be able to calculate relative risk reduction, absolute risk reduction and NNT from data presented in clinical trials and systematic reviews. We have described and defined these concepts and presented tabular tools and equations to help clinicians overcome common pitfalls in acquiring these skills.FigureFigureFigure
β See related article page 347Go to:
Supplementary Material
[Online Appendix]Click here to view.
Appendix 1.
Footnotes
This article has been peer reviewed.
Contributors: Alexandra Barratt contributed tip 2, drafted the manuscript, coordinated input from coauthors and reviewers and from field-testing and revised all drafts. Peter Wyer edited drafts and provided guidance in developing the final format. Rose Hatala contributed tip 1, coordinated the internal review process and provided comments throughout development of the manuscript. Thomas McGinn contributed tip 3 and provided comments throughout development of the manuscript. Antonio Dans reviewed all drafts and provided comments throughout development of the manuscript. Sheri Keitz conducted field-testing of the tips and contributed material from the field-testing to the manuscript. Virginia Moyer reviewed and contributed to the final version of the manuscript. Gordon Guyatt helped to write the manuscript (as an editor and coauthor).
Competing interests: None declared.
Correspondence to: Dr. Peter C. Wyer, 446 Pelhamdale Ave., Pelham NY 10803, USA; fax 212 305-6792; ten.tta.tendlrow@reywpGo to:
References
1. Malenka DJ, Baron JA, Johansen S, Wahrenberger JW, Ross JM. The framing effect of relative and absolute risk. J Gen Intern Med 1993;8:543-8. [PubMed]2. Forrow L, Taylor WC, Arnold RM. Absolutely relative: How research results are summarized can affect treatment decisions. Am J Med 1992;92:121-4. [PubMed]3. Naylor CD, Chen E, Strauss B. Measured enthusiasm: Does the method of reporting trial results alter perceptions of therapeutic effectiveness? Ann Intern Med 1992;117:916-21. [PubMed]4. Fahey T, Griffiths S, Peters TJ. Evidence based purchasing: understanding results of clinical trials and systematic reviews. BMJ 1995;311:1056-60. [PMC free article] [PubMed]5. Jaeschke R, Guyatt G, Barratt A, Walter S, Cook D, McAlister F, et al. Measures of association. In: Guyatt G, Rennie D, editors. The users’ guides to the medical literature: a manual of evidence-based clinical practice. Chicago: AMA Publications; 2002. p. 351-68.6. Wyer PC, Keitz S, Hatala R, Hayward R, Barratt A, Montori V, et al. Tips for learning and teaching evidence-based medicine: introduction to the series. CMAJ 2004; 171(4):347-8. [PMC free article] [PubMed]7. Schmid CH, Lau J, McIntosh MW, Cappelleri JC. An empirical study of the effect of the control rate as a predictor of treatment efficacy in meta-analysis of clinical trials. Stat Med 1998;17:1923-42. [PubMed]8. Furukawa TA, Guyatt GH, Griffith LE. Can we individualise the number needed to treat? An empirical study of summary effect measures in meta-analyses. Int J Epidemiol 2002;31:72-6. [PubMed]9. SHEP Cooperative Research Group. Prevention of stroke by anti-hypertensive drug treatment in older persons with isolated systolic hypertension. Final results of the Systolic Hypertension in the Elderly Program (SHEP). JAMA 1991;265:3255-64. [PubMed]10. SALT Collaborative Group. Swedish Aspirin Low-dose Trial (SALT) of 75mg aspirin as secondary prophylaxis after cerebrovascular events. Lancet 1991; 338:1345-9. [PubMed]11. Psaty BM, Smith NL, Siscovick DS, Koepsell TD, Weiss NS, Heckbert SR. Health outcomes associated with antihypertensive therapies used as first-line agents. A systematic review and meta-analysis. JAMA 1997; 277: 739-45. [PubMed]12. β-Blocker Health Attack Trial Research Group. A randomized trial of propranolol in patients with acute myocardial infarction. I. Mortality results. JAMA 1982;247:1707-14. [PubMed]13. ISIS-2 Collaborative Group. Randomised trial of intravenous streptokinase, oral aspirin, both or neither among 17 187 cases of suspected acute myocardial infarction: ISIS-2. Lancet 1988;2:349-60. [PubMed]14. Chatellier G, Zapletal E, Lemaitre D, Menard J, Degoulet P. The number needed to treat: a clinically useful nomogram in its proper context. BMJ 1996; 312: 426-9. [PMC free article] [PubMed]15. Sinclair JC, Cook RJ, Guyatt GH, Pauker SG, Cook DJ. When should an effective treatment be used? Derivation of the threshold number needed to treat and the minimum event rate for treatment. J Clin Epidemiol 2001; 54: 253-62. [PubMed]
Articles from CMAJ : Canadian Medical Association Journal are provided here courtesy of Canadian Medical Association
