0
The Pediatric Forum |

Studies Should Report Estimates of Treatment Effects With Confidence Intervals

Peter Cummings, MD, MPH
[+] Author Affiliations

Copyright 2007 American Medical Association. All Rights Reserved. Applicable FARS/DFARS Restrictions Apply to Government Use.

More Author Information
Arch Pediatr Adolesc Med. 2007;161(5):518--519. doi:10.1001/archpedi.161.5.518
Text Size: A A A
Published online

The ARCHIVES has published 2 randomized trials of duct tape therapy for warts.1 2 A commentary3 regarding the most recent of these trials2 pointed out that confidence intervals for the size of the treatment effect were not given in the results. The commentary3 provided confidence intervals in a table, but these were intervals for the observed outcome proportions in each trial arm, not intervals for the estimated effect of treatment.

Any study that compares 2 or more groups should calculate statistics that compare the group outcomes, along with estimates of precision for those comparisons, such as confidence intervals. This advice is given by the International Committee of Medical Journal Editors4 and by the Consolidated Standards of Reporting Trials guidelines for the reporting of randomized controlled trials.5 The ARCHIVES recommends the use of point estimates and confidence intervals in its instructions to authors6 and has endorsed this practice in editorials.7 9 Despite these recommendations, neither trial of duct tape therapy reported estimates or confidence intervals for the effect of treatment on wart resolution.1 2

The main outcomes for the duct tape trials were binary (Table 1), so appropriate statistics for treatment effects include risk ratios, risk differences, and odds ratios (Table 2).10 Odds ratios have a desirable symmetry; if we compare the treatment arm with the control arm, the odds ratio for the undesirable outcome (wart remaining) will be the inverse of the odds ratio for the desirable outcome (wart resolution). But when more than 10% of participants have the outcome, odds ratios are difficult to interpret because they will not approximate risk ratios well.10 14 The risk difference has a clear meaning, but a difference (additive) model assumes that treatment changes the outcome by adding a fixed proportion to the outcome proportion in the control group. Often a ratio (multiplicative) model is more plausible or fits the data more closely; this model assumes that treatment changes the outcome by multiplying the control group outcome proportion by a fixed ratio.15 17 Risk ratios are often a useful summary statistic for treatment effects.17 However, they lack the symmetry of odds ratios; the risk ratio for the harmful outcome is usually not the inverse of the risk ratio for the beneficial outcome. This asymmetry is apparent for the duct tape trials; the trial by de Haen et al2 found a greater treatment effect using the risk ratio for wart resolution, whereas the trial by Focht et al1 found a greater treatment effect using the risk ratio for the wart remaining at the end of follow-up. A reasonable approach would be to report both risk ratios with their confidence intervals.

Table Grahic Jump LocationTable 1. Outcomes From 2 Randomized Controlled Trials of Duct Tape Treatment for Warts*

Table Grahic Jump LocationTable 2. Treatment Effect Estimates Comparing Duct Tape Therapy With a Control Arm From 2 Randomized Controlled Trials of Treatment for Warts

In their commentary, Van Cleve et al3 used power calculations to estimate that the study by de Haen et al2 had only 26% power to find a statistically significant difference between the observed treatment risk difference of 10% and a hypothetical risk difference of 0%. When the observed trial difference is not statistically significant, such a post hoc calculation will always estimate that power was less than 50%.18 Power calculations are an inefficient method for interpreting results, as they provide an estimate for only one assumed effect size compared with the null hypothesis. After a study is completed, the size of the effect can be estimated from the data, and confidence intervals allow us to see how compatible all possible effects are with the observed results.18 20

Editor's Note: Matthew M. Davis, MD, MAPP, has read this letter but declined to reply.

AUTHOR INFORMATION

Correspondence: Dr Cummings, 250 Grandview Dr, Bishop, CA 93514 (peterc@u.washington.edu).

Financial Disclosure: None reported.

Focht  DR  III, Spicer  C, Fairchok  MP. The efficacy of duct tape vs cryotherapy in the treatment of verruca vulgaris (the common wart). Arch Pediatr Adolesc Med 2002;156971- 974
PubMed
de Haen  M, Spigt  MG, van Uden  CJ, van Neer  P, Feron  FJ, Knottnerus  A. Efficacy of duct tape vs placebo in the treatment of verruca vulgaris (warts) in primary school children. Arch Pediatr Adolesc Med 2006;1601121- 1125
PubMed
Van Cleave  J, Kemper  AR, Davis  MM. Interpreting negative results from an underpowered clinical trial: warts and all. Arch Pediatr Adolesc Med 2006;1601126- 1129
PubMed
International Committee of Medical Journal Editors,  Uniform requirements for manuscripts submitted to biomedical journals. JAMA 1997;277927- 934
PubMed
Moher  D, Schulz  KF, Altman  D. The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomized trials. JAMA 2001;2851987- 1991
PubMed
Archives of Pediatrics and Adolescent Medicine,  Instructions for Authors. http://archpedi.ama-assn.org/misc/ifora.dtl#StatisticsAccessed Nov 20, 2006
Rivara  FP, Cummings  P. Writing for publication in Archives of Pediatrics & Adolescent Medicine. Arch Pediatr Adolesc Med 2001;1551090- 1092
PubMed
Cummings  P, Rivara  FP. Reporting statistical information in medical journal articles. Arch Pediatr Adolesc Med 2003;157321- 324
PubMed
Cummings  P, Rivara  FP, Koepsell  TD. Writing informative abstracts for journal articles. Arch Pediatr Adolesc Med 2004;1581086- 1088
PubMed
Cummings  P, Weiss  NS. Summarizing evidence: systematic reviews and meta-analysis. Clinical Epidemiology: The Study of the Outcome of Illness New York City, NY Oxford University Press2006;157- 174
Welch  HG, Koepsell  TD. Insurance and the risk of ruptured appendix. N Engl J Med 1995;332396- 397
PubMed
Sackett  DL, Deeks  JJ, Altman  DG. Down with odds ratios! Evid Based Med 1996;1164- 166
Altman  DG, Deeks  JJ, Sackett  DL. Odds ratios should be avoided when events are common. BMJ 1998;3171318
PubMed
Schwartz  LM, Woloshin  S, Welch  HG. Misunderstanding about the effects of race and sex on physicians' referrals for cardiac catheterization. N Engl J Med 1999;341279- 283
PubMed
Breslow  NE, Day  NE. The Design and Analysis of Cohort Studies. Vol II Lyon, France International Agency for Research on Cancer1987;142- 146Statistical Methods in Cancer Research
Deeks  JJ, Altman  DG, Egger  M, edSmith  GD, edAltman  DG.ed Effect measures for meta-analysis of trials with binary outcomes. Systematic Reviews in Health Care: Meta-analysis in Context. London, England BMJ Publishing Group2001;313- 335
Deeks  JJ. Issues in the selection of a summary statistic for meta-analysis of clinical trials with binary outcomes. Stat Med 2002;211575- 1600
PubMed
Goodman  SN, Berlin  JA. The use of predicted confidence intervals when planning experiments and the misuse of power when interpreting results. Ann Intern Med 1994;121200- 206
PubMed
Smith  AH, Bates  MN. Confidence limit analyses should replace power calculations in the interpretation of epidemiologic studies. Epidemiology 1992;3449- 452
PubMed
Rothman  KJ, edGreenland  S.ed Approaches to statistical analysis. Modern Epidemiology. Philadelphia, Pa Lippincott-Raven1998;183- 199

First Page Preview

First page PDF preview

Figures

Tables

Table Grahic Jump LocationTable 1. Outcomes From 2 Randomized Controlled Trials of Duct Tape Treatment for Warts*
Table Grahic Jump LocationTable 2. Treatment Effect Estimates Comparing Duct Tape Therapy With a Control Arm From 2 Randomized Controlled Trials of Treatment for Warts

Interactive Graphics

Video

Country-Specific Mortality and Growth Failure in Infancy and Yound Children and Association With Material Stature

Use interactive graphics and maps to view and sort country-specific infant and early dhildhood mortality and growth failure data and their association with maternal

Focht  DR  III, Spicer  C, Fairchok  MP. The efficacy of duct tape vs cryotherapy in the treatment of verruca vulgaris (the common wart). Arch Pediatr Adolesc Med 2002;156971- 974
PubMed
de Haen  M, Spigt  MG, van Uden  CJ, van Neer  P, Feron  FJ, Knottnerus  A. Efficacy of duct tape vs placebo in the treatment of verruca vulgaris (warts) in primary school children. Arch Pediatr Adolesc Med 2006;1601121- 1125
PubMed
Van Cleave  J, Kemper  AR, Davis  MM. Interpreting negative results from an underpowered clinical trial: warts and all. Arch Pediatr Adolesc Med 2006;1601126- 1129
PubMed
International Committee of Medical Journal Editors,  Uniform requirements for manuscripts submitted to biomedical journals. JAMA 1997;277927- 934
PubMed
Moher  D, Schulz  KF, Altman  D. The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomized trials. JAMA 2001;2851987- 1991
PubMed
Archives of Pediatrics and Adolescent Medicine,  Instructions for Authors. http://archpedi.ama-assn.org/misc/ifora.dtl#StatisticsAccessed Nov 20, 2006
Rivara  FP, Cummings  P. Writing for publication in Archives of Pediatrics & Adolescent Medicine. Arch Pediatr Adolesc Med 2001;1551090- 1092
PubMed
Cummings  P, Rivara  FP. Reporting statistical information in medical journal articles. Arch Pediatr Adolesc Med 2003;157321- 324
PubMed
Cummings  P, Rivara  FP, Koepsell  TD. Writing informative abstracts for journal articles. Arch Pediatr Adolesc Med 2004;1581086- 1088
PubMed
Cummings  P, Weiss  NS. Summarizing evidence: systematic reviews and meta-analysis. Clinical Epidemiology: The Study of the Outcome of Illness New York City, NY Oxford University Press2006;157- 174
Welch  HG, Koepsell  TD. Insurance and the risk of ruptured appendix. N Engl J Med 1995;332396- 397
PubMed
Sackett  DL, Deeks  JJ, Altman  DG. Down with odds ratios! Evid Based Med 1996;1164- 166
Altman  DG, Deeks  JJ, Sackett  DL. Odds ratios should be avoided when events are common. BMJ 1998;3171318
PubMed
Schwartz  LM, Woloshin  S, Welch  HG. Misunderstanding about the effects of race and sex on physicians' referrals for cardiac catheterization. N Engl J Med 1999;341279- 283
PubMed
Breslow  NE, Day  NE. The Design and Analysis of Cohort Studies. Vol II Lyon, France International Agency for Research on Cancer1987;142- 146Statistical Methods in Cancer Research
Deeks  JJ, Altman  DG, Egger  M, edSmith  GD, edAltman  DG.ed Effect measures for meta-analysis of trials with binary outcomes. Systematic Reviews in Health Care: Meta-analysis in Context. London, England BMJ Publishing Group2001;313- 335
Deeks  JJ. Issues in the selection of a summary statistic for meta-analysis of clinical trials with binary outcomes. Stat Med 2002;211575- 1600
PubMed
Goodman  SN, Berlin  JA. The use of predicted confidence intervals when planning experiments and the misuse of power when interpreting results. Ann Intern Med 1994;121200- 206
PubMed
Smith  AH, Bates  MN. Confidence limit analyses should replace power calculations in the interpretation of epidemiologic studies. Epidemiology 1992;3449- 452
PubMed
Rothman  KJ, edGreenland  S.ed Approaches to statistical analysis. Modern Epidemiology. Philadelphia, Pa Lippincott-Raven1998;183- 199

Correspondence

CME Course for:


You need to register in order to view this quiz.


To understand the clinical management of acute heart failure syndromes.
Accreditation Information The American Medical Association is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians.
The AMA designates this journal-based CME activity for a maximum of 1 AMA PRA Category 1 CreditTM per course. Physicians should claim only the credit commensurate with the extent of their participation in the activity.
Physicians who complete the CME course and score at least 80% correct on the quiz are eligible for AMA PRA Category 1 CreditTM.
Note: You must get at least of the answers correct to pass this quiz.
Note: You must get at least of the answers correct to pass this quiz.
You have not filled in all the answers to complete this quiz
The following questions were not answered:
Sorry, you have unsuccessfully completed this CME quiz with a score of
The following questions were not answered correctly:
For CME Course: A Proposed Model for Initial Assessment and Management of Acute Heart Failure Syndromes
Indicate what changes(s) you will implement in your practice, if any, based on this CME course.
To view and print your certificate and access a summary of your CME courses go to My CME.
NOTE:
Citing articles are presented as examples only. In non-demo SCM6 implementation, integration with CrossRef’s “Cited By” API will populate this tab (http://www.crossref.org/citedby.html).
Submit a Comment

Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging & repositioning the boxes below.

Articles Related By Topic
Related Topics