No Evidence of a Trial Effect in Newly Diagnosed Pediatric Acute Lymphoblastic Leukemia FREE

Acute lymphoblastic leukemia (ALL), the most common cancer in children, has seen dramatic improvement in outcomes in the past 40 years.^1- 2 Multiple iterative changes in therapy from 1970 to 1995, including the optimization of the dose and timing of multiagent chemotherapy and routine use of central nervous system prophylaxis, contributed to the increased survival of children with ALL. During this era, retrospective studies comparing study participants with nonparticipants suggested that children experienced a direct benefit from clinical trial participation.^3- 5 These studies support the assertion that oncology trial participation, by itself, offers improved patient outcomes. The belief, called trial effect or inclusion benefit, is “widespread in the oncology community.”⁶ According to the American Federation of Clinical Oncologic Societies, clinical trials are “often a cancer patient's best option.”⁷ Finally, the National Comprehensive Cancer Network has claimed “the best management of any cancer patient is in a clinical trial.”⁸

However, not all evidence supports the trial effect in oncology clinical studies. In contrast to improved results seen in children younger than 15 years,^3,5 Stiller et al⁹ noted no difference in survival for adolescents and young adults with ALL who participated in clinical trials. Examining oncology clinical trials more broadly, Peppercorn et al⁶ found no evidence for better outcomes for patients enrolled in clinical trials, with the potential exception of “children with cancer [and] patients with hematologic malignancies,” particularly those conducted prior to 1986. Whether a trial effect in fact exists has significant ethical implications. Many researchers and research participants or parents incorrectly believe that the primary purpose of a clinical trial is the benefit of the participant, not the improvement of therapy for future patients.¹⁰

To determine the survival advantage of enrollment in a clinical trial in children with newly diagnosed ALL, we evaluated outcome in a retrospective study at a single large pediatric oncology center. As well, we evaluated demographic variables to confirm that the participant group was similar to the nonparticipant group.

We evaluated 322 patients with ALL younger than 22 years at diagnosis who received their initial therapy at Seattle Children's Hospital (SCH) between January 1997 and December 2005. We excluded patients who received their initial month of treatment at another hospital. Two patients with significant comorbidities received noncurative palliative treatment and were excluded from analysis. All other patients were offered therapeutic clinical trial participation if an open study was available. If patients declined participation or no study was available, all patients received therapy according to strictly defined institutional standard therapies based on the most recent clinical trial results. During the study period, clinical trials available for enrollment included the following Children's Oncology Group or Children's Cancer Group legacy protocols: 1952,¹¹ 1953,¹² 1961,¹³ 1962,¹⁴ 1991,¹⁵ AALL00P2,¹⁶ AALL0232, and AALL0331. We analyzed the following variables: participation in an ALL therapeutic study, sex, race, age at diagnosis, leukemia immunophenotype, National Cancer Institute risk group (standard risk, high risk, or infant),¹⁷ home state, and distance of primary residence to SCH. Seattle Children's Hospital is the regional referral center for Washington, Alaska, Montana, Idaho, and Wyoming and the largest pediatric oncology program in the region. Statistical analysis of event-free survival (EFS) was performed using the Kaplan-Meier method for calculating survival curves and 95% confidence intervals (CIs).¹⁸ Event-free survival was defined as the time from initial diagnosis to either leukemia recurrence or death from any cause. Patients who had not developed relapse or death were censored at the date of last contact. Differences in EFS among groups defined by patient or treatment characteristics were analyzed using the log-rank test.¹⁹ The EFS data were analyzed as of March 15, 2009, using SPSS for Windows version 13.0 (SPSS Inc, Chicago, Illinois). We conducted an assessment of the statistical power for a log-rank test comparing the 2 study groups given the available sample size. Setting an α level of .05 (2-sided), a sample size of 322 subjects, split approximately evenly between the 2 groups, yielded at least 80% statistical power when the hazard ratio for events among patients enrolled in a clinical trial relative to those not enrolled in any trial was 0.49 or lower. Assuming the overall EFS at the end of follow-up was 78% led to a detectable differential of 13.4% (84.9% survival among patients enrolled in a trial vs 71.5% among patients not enrolled in a clinical trial). This retrospective medical record review study was approved by the SCH institutional review board.

We first investigated demographic factors associated with study participation (Table 1). There was no difference in study participation by sex, race, home state, or distance of primary residence. Within risk groups, there was no difference in age at diagnosis between study participants and nonparticipants (data not shown). There were trends toward increased study participation in standard-risk vs high-risk patients (54% vs 42%; P = .15) and B lineage vs T lineage (50% vs 35%; P = .11).

Patient characteristics and EFS are shown in Table 2. The overall 5-year EFS was 79% (95% CI, 74%-84%). Study participants and nonparticipants had similar EFS (80% vs 77%; P = .73), as shown in the Figure. By univariate analysis, National Cancer Institute risk group was associated with a difference in EFS, with a higher 5-year EFS for standard-risk patients (85%) than either high-risk patients (74%; P = .045) or infants (59%; P = .01). Within risk groups, there was no difference between the outcome for study participants and nonparticipants (standard-risk patients: 82%; 95% CI, 73%-91% for nonparticipants vs 87%; 95% CI, 80%-94% for participants; P = .44; high-risk patients: 76%, 95% CI, 65%-87% for nonparticipants vs 71%; 95% CI, 59%-83% for participants; P = .33). Sex, race, leukemia immunophenotype, home state, or distance from primary residence were not associated with outcome.

In our large single-institution series, participation in a pediatric ALL clinical trial was not associated with improved outcome. Our study failed to show a trial effect, in contrast to similar retrospective studies from earlier treatment eras.^3- 5 Several factors that may have led to an observed trial effect in previous studies were absent in our study. Treatment for ALL during 1997 to 2005 was strictly standardized at our institution and based on the most recently reported cooperative group trials for each risk group. It is possible institutional standardization of care may provide the same benefit as clinical trial regimentation, eliminating the potential for the participation effect.⁶ Braunholtz et al²⁰ subdivided several potential components to the participation effect that could favor clinical trials, including the protocol effect (due to more rigorous treatment guidelines), care effect (due to more extensive follow-up or nursing care), and the Hawthorne effect (due to improved patient or clinician compliance with standard procedure because of external monitoring). Each of these potential effects would be minimized if the nonparticipation comparison group received highly regimented standard care, as was the practice at our institution.

Patient selection bias, due to the exclusion of patients with coexisting medical conditions or poor performance status, is an alternative explanation for superior outcome on therapeutic trials.⁶ Gross et al²¹ reviewed randomized controlled trials to compare the health outcomes of eligible nonparticipants vs trial participants and found no significant difference in clinical outcomes, after adjusting for the preexisting health status of both groups. A selection bias would likely favor healthier patients for trial enrollment, potentially leading to better outcomes in trial enrollment, which we did not see. As well, since the majority of children with newly diagnosed ALL are previously healthy, and ALL treatment studies during our study period had no exclusion criteria for medical comorbidities or poor performance status, patient selection bias was unlikely to be an issue in our study.

An additional explanation for a trial effect is the treatment effect, when the investigational treatment arm of a clinical trial is more efficacious than the standard treatment arm.^6,20 Among the positive studies documenting the trial effect in oncology, the majority were conducted in the 1970s and 1980s, an era of large improvements in cancer treatment.⁶ For pediatric ALL, this era included introduction of central nervous system prophylaxis, alternative chemotherapy combinations, and optimal timing of multiagent chemotherapy.^1- 2 The 3 retrospective studies on pediatric ALL that demonstrated a trial effect included this earlier era when the treatment effect was a significant factor favoring trial participants over nonparticipants.^3- 5 For example, an analysis of 4070 children with ALL treated in the United Kingdom during 1971 to 1982 showed a 14% better 5-year survival rate for patients enrolled in a clinical trial.³ However, the 5-year survival rate improved from 37% in the 1970s to 66% in the 1980s, coincident with series of successful clinical trials, making the trial effect heavily influenced by a treatment effect. Similar results were seen in a follow-up analysis of children with ALL in the United Kingdom treated between 1980 and 1994.⁵ A retrospective study of 327 children with ALL in the United States covering a treatment period of 1970 to 1975 also demonstrated a trial effect, with 4-year survival rates of 60% for participants and 19% for nonparticipants.⁴ After 1994, positive clinical trials have become less frequent and with less dramatic incremental improvements. As a result, the EFS for ALL has plateaued at 75% to 80%.^1- 2 The 1997 to 2005 era of our study reflects the recent expectations of a modest (if any) treatment effect benefit from participation in pediatric ALL clinical trials.

Our analysis found no difference in study participation by sex or race (Table 1). This is consistent with a recent review by Wendler et al²² that showed no significant differences in the willingness of minorities to participate in clinical trials compared with white patients. The geographically diverse referral base of SCH allowed us to determine that distance from a patient's primary residence was not associated with study participation (Table 1). As well, we found no significant differences in outcome in the same demographic comparisons. We were encouraged to find that there was no significant difference in EFS when compared by home state and by distance of SCH from primary residence. Our patients who are not from western Washington typically complete the majority of their treatment closer to home at locations other than SCH; their outcome had not been compromised by this treatment plan.

Our study does have several significant limitations. As a single-institutional study, our data do not exclude the possibility of a trial effect at other institutions. For example, some institutions may not have as rigorously followed standard care for clinical trial nonparticipants. However, using data from one institution reduces the confounding variables of a range of therapies offered and supportive care standards, for example. Our findings may only be generalized to institutions with a similar degree of standardization of care for nonparticipants in clinical trials. The major disadvantage of a single-institution study was the limitation of statistical power to detect a small but potentially clinically meaningful difference in outcome by trial participation. In particular, for a condition like ALL with a relatively good outcome, trials offer modest improvements, requiring a large sample size to detect a difference in outcomes, which is difficult to achieve at a single institution. Although we cannot exclude a small trial effect, it is unlikely that the benefit of clinical trial participation in pediatric ALL in the recent era is as large as observed in earlier treatment eras.^3- 5 Only 49% of our patients enrolled in a clinical trial. It is difficult, in retrospect, to determine the reasons for nonparticipation. There were times during our study period when no clinical trial was open to enrollment, leading to a lower rate of trial enrollment than explainable by trial refusal alone.

Whether a trial effect exists has important ethical implications. According to the World Medical Association Declaration of Helsinki, the main purpose of clinical trials is the advancement of future therapy.²³ However, in a survey of 198 pediatric oncologists (with more than 90% academic affiliation), 64% stated the main reason for enrolling patients in a clinical trial was “to ensure that trial participants get the most state-of the-art treatment.”¹⁰ Subgroups of oncologists involved in clinical research were more likely to view trial enrollment as a way of ensuring state-of-the-art treatment, both in terms of the individual patient and society in general.¹⁰

Miller et al²⁴ postulated that clinical researchers often practice with a “therapeutic orientation” to clinical research to resolve the tension between their moral commitment to the patient volunteers (“the clinician”) and the accurate representation that research is motivated by scientific aims (“the scientist”). It is clear that clinical researchers should not abandon therapeutic concern for research participants and, conversely, that participants should not abandon hope of a therapeutic benefit. Rather, researchers should strive for an initial frank discussion of the differences between clinical research and clinical care.²⁴ As part of this discussion, researchers should be aware that, ethically, a trial effect should not be a motivator for trial enrollment.

By minimizing the potential for participation effect, patient selection bias, and treatment effect, our study failed to show a trial effect in the treatment of pediatric ALL in the recent era. It is important that families and providers are aware that a patient's outcome is unlikely to be improved simply by trial enrollment.

Correspondence: Blythe Thomson, MD, Seattle Children's Hospital, 4800 Sand Point Way NE, Mailstop B6553, Seattle, WA 98105 (blythe.thomson@seattlechildrens.org).

Accepted for Publication: October 7, 2009.

Author Contributions:Study concept and design: Koschmann, Thomson, and Hawkins. Acquisition of data: Koschmann and Thomson. Analysis and interpretation of data: Koschmann, Thomson, and Hawkins. Drafting of the manuscript: Koschmann, Thomson, and Hawkins. Critical revision of the manuscript for important intellectual content: Koschmann, Thomson, and Hawkins. Obtained funding: Thomson. Administrative, technical, and material support: Thomson. Study supervision: Thomson and Hawkins.

Financial Disclosure: None reported.