A Multisite Randomized Trial of the Effects of Physician Education and Organizational Change in Chronic Asthma Care:  Cost-effectiveness Analysis of the Pediatric Asthma Care Patient Outcomes Research Team II (PAC-PORT II) FREE

The Pediatric Asthma Care Patient Outcomes Research Team II (PAC-PORT II) trial compared 2 asthma care strategies in children with mild to moderate asthma, a peer leader–based physician behavior change intervention (PLE) and a practice-based redesign (planned asthma care intervention [PACI]), with usual care.¹ The interventions were intended to improve the care provided to children with asthma and their outcomes. Unlike previous organizational interventions, the PAC-PORT II trial was implemented in the primary care setting instead of asthma clinics or relying on referrals.^2- 5 Compared with the usual-care arm, patients randomized to the PLE arm had an annual increase of 6.5 symptom-free days (SFDs), whereas those randomized to the PACI arm had an annual increase of 13.3 SFDs during the study period.⁶

To understand whether it makes sense for health care organizations to implement innovative disease management interventions such as the PAC-PORT II trial, health plans often require evidence of the financial impact of the intervention in addition to the clinical benefit. Cost-effectiveness analyses are conducted to provide this information and to help understand the economic value of an intervention. Recent cost-effectiveness analyses of interventions in children with asthma have shown that treatment with inhaled corticosteroids costs approximately $11 per SFD gained⁷ and that a community-based education program for inner-city children costs $9 per SFD gained.⁸ In both of these studies, the children included had mild to moderate asthma and most were not receiving controller medications at baseline,^7- 8 whereas more than 50% of the children in the PAC-PORT II trial were receiving an inhaled anti-inflammatory agent on enrollment and few had an asthma-related hospitalization in the prior year.⁶ Thus, an outstanding question is whether reorganizing the way children with asthma are cared for in managed care organizations offers additional value to providing appropriate treatment. Our research team has previously demonstrated clinical improvements associated with the practice-based changes⁶; therefore, the objective of this study was to estimate the cost-effectiveness of the interventions in the PAC-PORT II trial compared with usual asthma care.

The PAC-PORT II trial was a 3-arm, randomized trial in which primary care practices were randomized to usual care or 1 of 2 strategies to improve chronic asthma care. The design and analysis plan of the PAC-PORT II trial, including the cost-effectiveness analysis, has previously been reported.¹ Research reports that describe the clinical outcomes and overall asthma-related costs have been published.^6,9 Briefly, the study compared a health care professional–oriented PLE strategy and a multilevel approach that combined PLE with a nurse-run intervention (PACI) with usual care. The trial was conducted in 42 primary care practices affiliated with 4 health care organizations and followed up patients for a 2-year period. Children aged 3 to 17 years with mild to moderate persistent asthma were recruited. The study was intended to be an effectiveness study aimed at replicating a real-world environment in the setting of multiple health care organizations, and all efforts to ensure protocol compliance were consistent with a reasonable and practical primary care–based intervention effort. Human subjects approval for all aspects of the study was obtained from the institutional review boards of all participating organizations.

The first intervention, a PLE strategy, was intended to improve care through the use of targeted physician education on asthma treatment guidelines. The intervention involved training 1 pediatrician at each of the practice sites as an asthma expert and champion. The peer leader functioned as a change agent within the practice and provided support, education, and feedback to other members of the practice as it related to their asthma management.

The second intervention, a PACI strategy, was a multifaceted approach that sought to better organize asthma care by changing the system of asthma care in each practice. The PACI approach involved scheduled asthma care visits with an asthma nurse who provided standardized assessments, care planning, coordination with the primary care physician, and self-management tools for the patients and their families. Additionally, the PACI arm included all of the components of the PLE arm. Thus, patients were receiving self-directed asthma care and support with active follow-up from an asthma nurse and their primary care physician.

Practices randomized to the usual-care arm of the study received copies of the National Asthma Education and Prevention Program Expert Panel Report 2 asthma treatment guidelines¹⁰ and patient informational handouts 12 months into the study. Additional descriptions of the interventions and implementation can be found in the article by Lozano et al.⁶

Interviews with each participant’s primary caregiver, which in most cases was a parent, were conducted at baseline and every 8 weeks throughout the study. At the baseline interview, caregivers reported patients’ medication and health care utilization in the previous 12 months and the number of days with asthma symptoms in the previous 2 weeks. At each follow-up interview, health care utilization during an 8-week period, including medications, emergency department visits, and hospitalizations, was reported. The caregivers also reported the number of days in the prior 2 weeks in which the patient experienced asthma symptoms.

The number of asthma symptom days was the primary outcome measure in the analysis. Caregivers reported the number of days in the preceding 2 weeks that patients experienced any asthma symptoms (including cough, wheeze, limitation in activity, or night wakening). For the cost-effectiveness analysis, the number of SFDs was used as the measure of effectiveness. Symptom-free days have been recognized as a clinical outcome with relevance to patients, physicians, and other decision makers.¹¹ The number of SFDs reported for the 2-week period preceding each of the follow-up interviews was used as the estimate for the number of SFDs in the time between reporting periods and to estimate cumulative SFDs for the study period.

The unit costs (in 1999 US dollars) for hospital days, emergency department visits, and physician visits were derived from a large medical and pharmacy claims database (PharMetrics Integrated Outcomes Database; http://www.pharmetrics.com). Patient-level medical claim histories of more than 16 million managed care cases, including adults, adolescents, and children, from diverse health plans across the United States were available in the database. This represented more than 400 million patient observations of medical and pharmacy–related claims during a rolling 4-year period. Physician visit costs were calculated as the weighted average of the cost of the first visit and subsequent follow-up visits.

Asthma-related drugs were grouped into 6 therapeutic classes (short-acting and long-acting bronchodilators, methylxanthines, cromoglycate and cromoglycate derivates, inhaled corticosteroids, leukotriene antagonists, and systemic corticosteroids). Unit cost estimates were based on US average wholesale prices (or unit costs) and were reduced by 15% to approximate actual acquisition costs.¹² Exact dosage information was not reported by the caregivers; thus, cost calculations were based on the assumption of standard dose per day (recommended daily dose per product formulation for children).

The costs of the interventions were estimated using a simple cost allocation method that summed fixed and variable costs for program implementation and maintenance and included personnel, materials, and training costs (Table 1). We excluded costs of development in the base case analysis to make the results applicable to managed care organizations that were implementing the interventions without having to redesign the intervention arms. Wage rates for personnel were estimated using national average wage rates for physicians, nurses, and support personnel.¹³ Facility rental, supplies, and intervention-related materials and aids were valued at actual cost. Research-related costs were excluded (eg, costs of conducting follow-up interviews). Intervention costs were included in direct medical costs.

The primary cost-effectiveness evaluation, using an intent-to-treat approach, was conducted from a health care payer perspective. Outcomes, costs, and cost-effectiveness were evaluated during the 2-year study period. The difference between treatments in costs and effectiveness and the incremental cost-effectiveness ratio (ICER) were calculated. A separate analysis was run to assess cost-effectiveness by including an estimate of nonmedical indirect costs. The intention of this analysis was to approximate an evaluation from the broader societal perspective. The costs for a day absent from school were estimated using the human capital approach. Absences from school were valued using an estimate of the daily wage rate of the caregiver. Mean daily wage rates were estimated using the method of Rice and Max for the US workforce as described by Sullivan et al.⁷

Subtracting the mean medical cost for the usual-care group from that for the intervention groups and then dividing by the mean difference in SFDs between groups gives the ICER. This computation produced a ratio of the mean medical cost per SFD gained. Differences in effectiveness between groups was assessed with multivariate generalized estimating equations (GEE) models that controlled for baseline differences.⁶ Person-level costs were obtained by multiplying each child’s health care utilization with a unit cost for the type of medical care. Individual costs for the intervention were obtained similarly by determining the total costs of the interventions and dividing it by the number of patients in each arm. We applied bootstrap resampling techniques to quantify the variability associated with the ICER, from which 95% confidence intervals (CIs) were estimated and cost-effectiveness acceptability curves were plotted.¹⁴ The acceptability curves provide a quantification of the joint uncertainty in the cost and effectiveness outcomes, as well as being useful for decision makers in understanding the likelihood of the interventions being cost-effective at certain health care willingness-to-pay threshold levels.

Sensitivity analyses included changing the unit cost values, including the developmental costs of the interventions, and including indirect costs of caregiver absence from work to attend to ill children. Finally, to assess the economic value in subgroups, we used net health benefit regression methods and stratified analyses to identify subgroups in which cost-effectiveness estimates may differ.¹⁵ The effectiveness of the interventions in these subgroups was assessed using stratified multivariate GEE models.

A total of 638 children participated in the study, with 199 assigned to usual care, 226 to PLE, and 213 to PACI. Selected baseline characteristics of the groups are given in Table 2. Most participants were male, and the mean age was 9.4 years. Nearly 1 of 4 patients in each arm required an emergency department visit in the year before their enrollment. Differences between groups existed in baseline reports of SFDs, cromolyn sodium or nedocromil sodium use, and bronchodilator use.

Patients in the usual-care arm of the study had an increase in SFDs of 14.8 per year during the study period. Patients in the PACI arm had an additional gain of 13.3 (95% CI, 2.1-24.7) SFDs per year compared with the usual-care group. In the PLE arm, the gain in SFDs per year compared with usual care was 6.5 days (95% CI, −3.6 to 16.9 days).⁶

The average number of physician visits during the 2-year follow-up period was 4.70 in the PACI arm, 3.24 in the usual-care arm, and 3.12 in the PLE arm (P = .002 for PACI compared with usual care and PLE) (Table 3). The average number of hospital days and emergency department visits was not different between groups.

In the PACI arm, the total costs for the intervention were $475 265, which consisted of $127 520 (92.7% personnel costs) in development costs, $137 048 (87.2% personnel costs) in implementation costs, and $210 697 (92.4% personnel costs) to maintain the intervention during the study period. In the PLE arm, the total intervention costs were $111 486, with $39 234 (93.2% personnel) attributed to development, $37 239 (79.3% personnel) for implementation, and $35 013 (90.8% personnel) for maintenance. The per-patient component costs of the interventions are detailed in Table 1. Annual treatment costs, including medications and health care contacts (ie, hospitalizations or emergency department visits), were lowest in the PLE arm ($344) followed by the usual care arm ($385) and the PACI arm ($475) (Table 4). When treatment and intervention costs (implementation and maintenance) were combined, the annual per patient costs were $1292 for PACI, $504 for PLE, and $385 for usual care.

When costs of development were excluded, the difference in annual per patient costs between the intervention groups and usual care was $907 for PACI and $119 for PLE. Combining the difference in costs with the difference in effectiveness resulted in an ICER of $18 per SFD gained for PLE compared with usual care. The 95% CI ranged from $5 to dominated. A dominated health care intervention is one in which the strategy is more expensive and less effective when compared with usual care. The ICER was $68 per SFD gained for PACI compared with usual care (95% CI, $37-$361 per SFD gained) (Table 5). Plotting cost-effectiveness acceptability curves provides health care purchasers an estimate of the likelihood that an intervention is cost-effective at various amounts a purchaser may be willing to pay for the intervention. The proportion of bootstrap replicates (y-axis) falling below ICER threshold values (x-axis) were plotted for both PACI and PLE and allow purchasers and reimbursement authorities to determine the likelihood of the intervention being cost-effective at various costs per SFD thresholds that they may be willing to pay for asthma care (Figure). For example, at a threshold of $75 per SFD gained (vertical blue line), the probability that PLE is cost-effective is 84.5% and the probability that PACI is cost-effective is 57.4%.

When developmental costs were included in the analysis, the ICERs were $32 per SFD gained for PLE compared with usual care and $91 per SFD gained for PACI compared with usual care. The difference in costs between the intervention arms and the usual-care arm was similar when the indirect costs were included (Table 5). Thus, the ICERs for the analysis that included indirect costs (PLE vs usual care = $18 per SFD gained; PACI vs usual care = $69 per SFD gained) were similar to those in the analysis that included only direct costs.

Age stratification was the only subgroup analysis that produced a difference in the ICER of potential policy interest. In this analysis, the ICERs were lower in older children (aged 7-16 years) than in the base case analysis (Table 5).

The PAC-PORT II trial was designed to evaluate the effectiveness of 2 interventions in the primary care setting for improving outcomes in children with asthma. Results showed that the PACI intervention improved SFDs by a mean of 13.3 days⁶ at an additional median cost of $564.⁹ The PACI intervention was a multilevel education intervention that made use of an asthma nurse for patient education and physician peer leaders to educate other primary care physicians on guideline-appropriate management. When evaluated alone, the PLE intervention did not improve SFDs at conventional statistical levels. Notably, both of the interventions increased the costs associated with asthma treatment, primarily owing to the costs of the interventions. The cost-effectiveness ratios comparing usual care with the 2 interventions show that the gain in health benefits from PACI can be achieved at an incrementally higher cost. These higher costs are largely attributable to the cost of implementing and maintaining the intervention during the study period.

Several previous economic studies of organizational interventions in children with asthma have been simple cost evaluations rather than cost-effectiveness analyses. Some have been conducted as simple pre-post designs to compare costs or have failed to include the costs of the intervention.^3,5,16- 20 The intervention most similar to the PAC-PORT II trial, in that it was a randomized clinical trial with a health economic evaluation, is the National Cooperative Inner-City Asthma Study (NCICAS).⁸ The NCICAS included an asthma social worker who coordinated access to medical care services for children with asthma; however, it was in a much different population from that in the PAC-PORT II trial. The NCICAS population included children with asthma from 8 inner-city areas in the United States who were predominantly enrolled in Medicaid fee-for-service programs. The NCICAS intervention reported a gain in SFDs that was similar to what we report. However, the cost of developing and deploying the intervention was much less, in large part because the personnel costs of the intervention were significantly lower.

No agreed-on threshold for cost-effectiveness exists. However, when considering cost-effectiveness analyses in children with asthma, we can compare the estimates from the PAC-PORT II trial with the cost-effectiveness ratios of other interventions in children with asthma to understand the context of the results. The cost-effectiveness of inhaled corticosteroids in the treatment of children, when adjusted to 2002 US dollars, ranges from $7 to $12 per SFD gained.^7,21 The NCICAS social worker–based education intervention compared with usual care had a cost-effectiveness ratio of $9 per SFD gained.⁸ The ICERs for PACI and PLE are higher than those reported for inhaled corticosteroids and an educational intervention; however, it is unclear how they compare with other organizational change interventions in children with asthma. These cost-effectiveness analyses compared interventions in children who were being treated suboptimally at baseline, had more severe disease, and were likely to benefit from the intervention, whereas in the PAC-PORT II trial, more than 50% of the children enrolled in the study were taking inhaled anti-inflammatory medications before enrollment and had low baseline hospitalization rates.⁶

To understand the uncertainty in the point estimates for the cost-effectiveness ratios and to help health care purchasers make decisions regarding implementation of the interventions, we generated cost-effectiveness acceptability curves. This display of the results allows purchasers, whether a managed care organization or state Medicaid program, to determine the probability of the intervention being cost-effective at levels they would be willing to pay for gains in asthma SFDs. Health care delivery systems (managed care organizations or others) and insurers (public or employer based) would presumably consider what “return on investment” they might expect from providing or covering enhanced services such as PLE or PACI. Since PLE and PACI improve health while adding cost, both health care delivery systems and insurers would need to contemplate the added value in their marketplace of this enhanced quality of care for children with asthma vs the added costs to their operations (delivery systems) or price (insurers). Patients and/or parents, in turn, would need to frame cost and return-on-investment considerations in their own terms. If PACI services, for example, add 13 SFDs per year at a cost of $68 per additional SFD gained, a parent of a child with persistent asthma might consider whether a day at work instead of at home taking care of a child with asthma was worth $68, whether 13 fewer sleep-disturbed nights were worth $68 per night, or whether the child with asthma who was a competitive swimmer would want to have fewer symptom-burdened days during the swim season at $68 per day. If this value for the money seems reasonable, the PACI service (or insurance benefit) might be a worthwhile investment.

A ratio of $50 000 per quality-adjusted life-year (QALY) is frequently cited as a threshold for economic evaluations that use QALYs as the measure of effectiveness, which has been recommended by the US Panel on Cost-effectiveness in Health and Medicine.²² In this study, the primary outcome measure was SFDs. Therefore, cost-effectiveness ratios are presented in terms of cost per SFD gained, which was the outcome recommended by a National Institutes of Health consensus panel and is frequently used in cost-effectiveness analyses of asthma interventions.¹¹

Treatment costs were similar among the study arms, but the costs of the intervention caused PACI and PLE to be more expensive than the usual care provided by these organizations. However, the overall annual asthma-related treatment costs among the children in the health care organizations that participated in this study are much lower than those reported previously for children with asthma.²³ Reductions in the intervention costs would reduce the ICERs for PACI and PLE. Economies of scale may reduce the cost per patient if the intervention is rolled out to all children with asthma in a health plan. The study was powered to evaluate the clinical effectiveness of the interventions and was necessarily limited to a sample of children from each practice. Implementing the intervention in a health plan in which all patients with asthma can be enrolled may reduce the per child cost of the interventions. The per child costs of the intervention are likely to decrease if more children are included as economies of scale are realized for some components of the intervention. Additionally, reduction in the inefficiencies in the delivery of the interventions, particularly in the case of nurse case manager time in the PACI arm, in which nurses spent significant time traveling between clinics to deliver the intervention, would result in more favorable cost-effectiveness ratios. Thus, delivery of the intervention to all eligible children with asthma in a clinic and using clinic-based nurse managers may improve the cost-effectiveness of the PACI intervention.

This study has several limitations. Those limitations specific to the PAC-PORT II trial are described in detail in the article by Lozano et al.⁶ For cost-effectiveness, we mentioned that this study did not take a full societal perspective. To do so, we would have required additional data collection, such as estimates of travel and waiting times for the children and caregivers. Determining the actual costs of the 2 interventions was challenging, given the nature of clinical research budgets and the artificial nature of a clinical trial setting. We used microcosting methods for cost determination but acknowledge that we may have overestimated or underestimated various intervention cost components in the process. A sensitivity analysis of intervention costs showed that eliminating development costs from the calculations improved the cost-effectiveness of the interventions. This is not surprising given that the incremental cost of health technology frequently drives the ICER. We did not compute ICERs in terms of cost per QALY gained. A cost per QALY estimate would have allowed us to compare the value of PACI and PLE with other health care interventions. The utility elicitation methods for estimating QALYs were not shown to be valid in children with chronic disease and have several methodologic limitations.²⁴ We could have used a crosswalk between other measures, such as lung function²⁵ or health status measures,²⁶ to estimate QALYs. Unfortunately, the crosswalk studies were conducted in adult populations, and the relevance to our sample is questionable. Our results may not apply to special populations of children such as Medicaid or uninsured groups.

The results of this study show that children in these health plans have substantive asthma morbidity in organizations with low hospital and emergency department use. It was possible to reduce asthma morbidity (increasing SFDs and reducing oral steroid burst rates⁶) and move organizations toward guideline recommendations on asthma control in settings where most children are receiving controller medications at baseline. The incremental gain in SFDs comes at an increased cost. Ultimately, society, health care purchasers, and even parents will judge the value of asthma disease management programs by their willingness to pay for improvement in asthma outcomes.

Correspondence: Sean D. Sullivan, RPh, PhD, Pharmaceutical Outcomes Research and Policy Program, Box 357630, University of Washington, Seattle, WA 98195-7630 (sdsull@u.washington.edu).

Accepted for Publication: November 8, 2004.

PAC-PORT II Additional Institutions and Investigators: Harvard Pilgrim Health Care, Boston, Mass: Kathleen Loane, RN, Jeri Bryant, RN; Channing Laboratory, Harvard Medical School, Boston, Mass: Nancy Laranjo, PhD; Stephen B. Soumerai, ScD; Scott Weiss, MD; Jim Donahue, DVM; Center for Health Studies, Group Health Cooperative, Seattle, Wash: Cynthia Sisk, MS, Virginia Lincicome, RN, Julia Hecht, PhD; Rush-Prudential Health Plans, Chicago, Ill: Reeva Shulruff, MD, Carol Jones, RN; Rush-Presbyterian-St Luke’s Medical Center, Chicago: Evalyn Grant, MD; and American Academy of Pediatrics, Elk Grove Village, Ill: Lynn Olson, PhD, Linda Asmussen, MS.

Funding/Support: The PAC-PORT II trial is funded by grant HS08368-01 from the Agency for Healthcare Research and Quality (Rockville, Md) and the National Heart, Lung, and Blood Institute (Bethesda, Md).

Acknowledgment: We are deeply indebted to the families who participated in this study in Chicago, Boston, and Seattle.