Attention-Deficit/Hyperactivity Disorder Outcomes for Children Treated in Community-Based Pediatric Settings FREE

Attention-deficit/hyperactivity disorder (ADHD) is a common neurobehavioral disorder that is characterized by inattention, hyperactivity, and impulsiveness and related functional impairment.¹ A considerable body of research has established the efficacy of stimulant medication for reducing ADHD symptomatology in patients with ADHD.² However, nearly all of this research has been conducted in university-based laboratories using intensive medication protocols. For example, the often-cited National Institute of Mental Health Multimodal Treatment Study of Children with ADHD (MTA) used a 28-day titration protocol with daily medication dosage switches and daily parent and teacher ratings to determine the optimal medication dosages for children.³ Further, after titration was completed, families were seen on a monthly basis and both parents and teachers were involved in providing input regarding children's progress. Children who were treated with this regimen had much better treatment outcomes than children who received medication in community settings.⁴

Because most children are treated in the community, primarily by their primary care providers,^5- 6 the apparent lack of relative efficacy of ADHD medication treatment in community settings compared with university settings poses a significant public health concern. In response, the American Academy of Pediatrics (AAP) published consensus treatment guidelines for treating ADHD⁷ to guide primary care providers in providing evidence-based ADHD care. Many of their recommendations promote features of the MTA medication protocol (eg, titrating medication, monitoring symptoms and adverse effects, frequent follow-up). However, these guidelines can be difficult to implement in the course of typical community-based care owing to multiple barriers (eg, time, effort, reimbursement). Several efforts have been directed at assisting community-based providers to adhere to these guidelines.⁸ One such effort is a quality improvement initiative called the ADHD Collaborative that was instituted with most of the community-based pediatricians in the Greater Cincinnati region.⁹ The ADHD Collaborative intervention was very effective at improving adherence to the ADHD guidelines for the diagnosis and treatment of ADHD among participating community physicians.⁹

Physicians participating in the ADHD Collaborative treated several hundred newly diagnosed children with ADHD during the study. Physicians were encouraged to monitor children's treatment outcomes using ratings scales consistent with AAP ADHD consensus recommendations.⁷ Using this prospective data, we describe the trajectories of ADHD symptomatology and functional impairment for children treated in community settings. Based on the efficacy of typical ADHD pharmacological treatments³ and the high rates of adoption of evidence-based treatment practices among the physicians in this sample,⁹ we predicted that children would demonstrate significant improvement in ADHD symptomatology with treatment. A recent study demonstrated that children with ADHD treated in the community with stimulant medications make gains in academics but remain significantly impaired relative to their peers.¹⁰ Further, while stimulants positively affect certain aspects of interpersonal relationships (eg, decrease noncompliance), they do not address the wide range of interpersonal problems children with ADHD experience (eg, enhance prosocial behaviors¹¹) and do not eliminate impairment.¹² Accordingly, we hypothesize that the percentage of children in this sample rated as functionally impaired would not decrease over time.

Here we report the child outcomes obtained by 47 practices (n = 158 physicians) participating in the ADHD Collaborative quality improvement project.⁹ The 158 physicians included 142 pediatricians and 11 family practice providers. A total of 54% were female and 84% were white (4% African American; 12% other), and 28% of practices served primarily Medicaid-receiving populations (>50% of patients). Demographic data was missing for 5 physicians.

A total of 785 patients had a standardized ADHD rating scale (the Vanderbilt ADHD Rating Scale^13- 14) completed by both a parent and teacher at assessment and at least once thereafter. All children were elementary school–aged and were newly presenting for ADHD-related problems. Because this was a quality improvement effort, demographic information was not collected from patient medical records to protect patient confidentiality. As reported in Epstein et al,⁹ 75% of the children met strict Diagnostic and Statistical Manual of Mental Disorders (Fourth Edition) (DSM-IV) criteria (ie, at least 6 symptoms reported by a parent, at least 6 symptoms reported by a teacher, and at least 1 area of functional impairment reported by a parent or teacher) for ADHD based on parent and teacher Vanderbilt Rating Scales.

Physicians were trained using the ADHD Collaborative intervention. The 4-session intervention included 2 didactic lectures and 2 sessions focused on using quality improvement methods for mapping and modifying physicians' office systems to facilitate adherence to AAP ADHD guidelines. Didactics focused on the rationale and importance of the AAP assessment and treatment guidelines, and especially emphasized the importance of obtaining parent and teacher behavioral ratings (eg, Vanderbilt ADHD Rating Scales) at the time of the initial assessment for ADHD and during follow-up after initiating medication treatment. Physicians were also provided a variety of assessment tools (eg, Vanderbilt ADHD Rating Scales) and were given instructions on administration and interpretation. Investigators and physicians then collaborated to modify the existing office flow to accommodate the AAP-recommended practice behaviors. Subsequently, each practice kept a log of elementary school–aged patients who were newly diagnosed with ADHD. Investigators conducted medical record audits of each patient listed on the log on a quarterly basis for 1 year after initiation of a stimulant medication. Physicians were sent a score card summarizing their performance following each medical record review. The institutional review board at Cincinnati Children's Hospital Medical Center approved this study.

The Vanderbilt ADHD Rating Scales are DSM-IV–based scales with teacher-report and parent-report forms.^13- 14 The rating scales include the 18 DSM-IV ADHD symptoms, which are rated on a 4-point Likert scale (0-3) with ratings indicating whether each ADHD symptom occurs never (0), occasionally (1), often (2), or very often (3). A Total Symptom Score (TSS) can be derived by summing the responses on the 18 DSM-IV ADHD symptom items. During medical record reviews, the TSS was recorded. In addition to assessing ADHD symptoms, the Vanderbilt scales also include items regarding functional impairment. Parents and teachers rated 8 functional impairment items on a 5-point Likert scale (1-5) indicating whether the child's performance is excellent (1), above average (2), average (3), somewhat of a problem (4), or problematic (5) in each domain of functioning. As described in Wolriach et al,¹⁵ the impairment items can be dichotomized into impaired (scores of 4 and 5) and not impaired (scores of 1, 2, or 3). The Table shows the areas of functional impairment rated by parents and teachers.

Vanderbilt TSSs were analyzed using longitudinal modeling. For this longitudinal analysis, the dates that rating scales were collected were used to denote time. At the assessment time point, some of the children had multiple parent (eg, mother and father) or teacher ratings (eg, reading and math teachers). In those cases, we averaged TSS scores within raters (ie, within parents or within teachers). Time is defined as the difference between the assessment date and each subsequent rating scale date. The SAS PROC Mixed procedure (SAS Inc, Cary, North Carolina) was used to analyze the data. Because children were nested within pediatricians and pediatricians were nested within practice, it was necessary to account for this nested design. Two error terms (ie, pediatrician and practice) were included in the model. Intercept and time were allowed to vary for each individual. The adequacy of the linear model was evaluated by adding a quadratic term to the existing model and testing the difference in fit between the two models using −2 log-likelihood ratios. Results indicated that the parent and teacher models that included a quadratic term provided a better fit (parent model χ² = 429.2, P < .001; teacher model χ² = 432.7, P < .001). The final model, which included both linear and quadratic terms, was fitted using a restricted maximum likelihood method.

To examine changes in functional impairment across time, we modeled the trajectory across time using the SAS GLIMIX procedure to address the nested nature of the data for each functional impairment item rated by parents or teachers. In addition, proportions were computed that reflected the proportion of children who were rated as impaired on each functional impairment item at each time point. Because posttreatment rating scales were collected periodically but not consistently across children, it was necessary to group the rating scales to compute proportions. Rating scales were grouped according to the following time points: assessment (n = 785 children with parent and teacher ratings); 1 day to 3 months (n = 539 parent ratings; n = 528 teacher ratings); 3 to 6 months (n = 224 parent ratings; n = 225 teacher ratings); 6 to 9 months (n = 148 parent ratings; n = 130 teacher ratings); and 9 to 12 months (n = 84 parent ratings; n = 86 teacher ratings). The χ² tests were conducted to determine whether the proportion of children who were rated as functionally impaired changed from pretreatment to the 1-day to 3-month, 3 to 6–month, 6 to 9–month, and 9 to 12–month posttreatment time points.

Because the number of patients who had completed posttreatment rating scales decreased over time, it is important to ensure that those who have continued data do not constitute a biased group (ie, treatment successes). We compared patients with continued ratings vs those without on the baseline Vanderbilt ADHD TSS. Comparisons indicated that the children who had continued teacher ratings did not differ from those who did not at the 3-, 6-, 9-, and 12- month time points (P = .92, P = .79, P = .31, and P = .49, respectively). The same was true for comparisons of those who continued and did not continue to have parent ratings at the 1-day to 3-month, 3 to 6–month, 6 to 9–month, and 9 to 12–month time points. There was a difference at the 6 to 9–month time point (P = .007) for parent ratings indicating that those who completed parent ratings between 6 and 9 months (mean [SD], 35.6 [9.7]; n = 148) had significantly higher scores at assessment than patients for whom parent ratings were not collected between 6 and 9 months (mean [SD], 33.2 [9.5]; n = 637). Similar to TSS scores, children who had continued ratings did not differ from those without continued ratings in terms of the proportion of children rated by parents or teachers as being functionally impaired (all P > .05).

Physicians (n = 158) across multiple practices (n = 47) used parent and teacher versions of the Vanderbilt ADHD Rating Scales to assess the treatment trajectories of their patients (n = 785). Prior to initiating medication, the mean (SD) parent TSS rating on the Vanderbilt Parent ADHD Rating Scale was 33.6 (9.6); teacher TSS rating, 32.2 (10.2). Mixed model regressions including an intercept term (parent T = 53.18; teacher T = 53.81), a linear time variable (parent T = −27.75; teacher T = −28.64), and a quadratic time variable (parent T = 21.08; teacher T = 21.28) produced statistically significant effects on all of these variables across both parent and teacher ratings (all P <.001). Graphical depictions of symptom trajectories over time (Figure) as well as post hoc analyses indicated that there was a large drop in ADHD TSS between the baseline and 3-month posttreatment (parent mean [SD] TSS, 19.6 [9.8]; teacher mean [SD] TSS, 16.8 [10.7]; both P <.001). The effect size (Cohen d) for this initial drop in ADHD symptomatology was 1.45 for parent ratings and 1.47 for teacher ratings. After the initial drop (ie, baseline to 3 months) in ADHD symptomatology, ADHD TSS remained relatively stable over time.

During the assessment, parents reported high rates of functional impairment primarily in the academic domain (reading, 58%; mathematics, 48%; writing, 59%; and overall school performance, 56% of children in the sample rated as impaired). Smaller portions of the sample were rated by parents as having impairment in relationships with parents (17% of children) and with peers (24% of children) (Table).

Relative to parents, teachers rated patients as having higher rates of functional impairment across all domains. Following directions was rated most frequently (85% of children) followed by organizational skills (77% of all children) and assignment completion (73% of all children). Ratings of academic impairment were comparable with parents (reading, 61%; mathematics, 54%; and writing, 73%). Relationship with peers was rated as the least frequent area of impairment by teachers (41% of all children) though it should be noted that this was nearly twice the proportion of children rated as impaired parents (24%).

Regarding treatment trajectories, the coefficient for time (linear) was not significant for any impairment function across the two raters. The proportion of children who had functional impairment in the areas rated did not change after treatment for any outcomes except for teacher rating of writing between the initial and 9 to 12–month assessments (P = .03) and assignment completion between the initial and 9 to 12–month assessment (P = .04). For all other tests, the χ² tests indicated that the proportion of children rated as impaired at assessment was not different from the proportions of children rated as impaired at the 0 to 3–, 3 to 6–, 6 to 9–, and 9 to 12–month time points (all P >.05).

Children who received ADHD care in community-based primary care practices who participated in the ADHD Collaborative experienced significant reduction of ADHD symptoms over time. Improvement of ADHD symptoms occurred mainly in the first 3 months of treatment and remained improved and relatively stable thereafter. Sizable improvements in ADHD symptomatology were observed between pretreatment and posttreatment (ie, effect sizes approximately 1.5). The within-group premedication and postmedication effect sizes achieved in these community settings (teacher rating d = 1.47; parent rating d = 1.46) are comparable with the within-group gains observed for children in the MTA study's medication-only treatment group (teachers d = 1.68; parents d = 1.36 in the MTA study³) and within-group pretreatment to 12-month gains in the medication-only group in the Multimodal Psychosocial Treatment study (parents d = 1.43; teachers d = 1.98¹⁶). These results suggest that community-based physicians can achieve gains in ADHD symptom improvement comparable with carefully controlled, university-based clinical trials.

This contradicts the MTA study finding that suggested that children treated with medication within the context of the MTA study had much better treatment outcomes than children treated with medication in the community.⁴ A variety of reasons could account for this apparent contradiction. First, there may be general improvement in the standard of care provided to children in the community since the publication of the MTA study and the AAP guidelines. Indeed, efforts were made by the AAP, National Initiative for Children's Healthcare Quality, and individual researchers nationwide to disseminate and promote the AAP ADHD guidelines. However, preintervention results from ADHD Collaborative practices⁹ as well as other community-based studies^17- 19 suggest that adoption of the AAP-recommended practice behaviors has been limited.

Alternatively, it may be that the medications used to treat children with ADHD have improved significantly and have made treatment more effective. Indeed, the MTA study was conducted when only a couple of immediate-release formulations of stimulant medication were available. Currently, physicians have access to stimulant medications that vary in their duration of action, mechanism of action, and mode of administration. These advances in pharmacology may have improved the efficacy of community-based ADHD treatment.

Another possible explanation is that the sample of pediatricians who provided treatment to the children in this study provided improved quality of care. All of the physicians in this study participated in a quality improvement intervention designed to promote evidence-based ADHD care. The intervention included didactic seminars teaching evidence-based care, office system modification to help incorporate the AAP-recommended practice behaviors, and participation in quality improvement methods (eg, plan-do-study-act cycles²⁰) to sustain and promote practice improvement. Indeed, these physicians drastically improved their implementation of evidence-based treatment practices.⁹ After the intervention, physicians routinely contacted patients within 2 weeks of prescribing medication to assess medication response and adverse effects, scheduled a follow-up visit within 6 weeks of medication initiation, and used standardized rating scales to monitor treatment responses.

Because this study did not include a control group of physicians who did not participate in the quality-improvement initiative, we cannot determine whether a similar pattern of treatment response would have been observed without physician training. Certainly, evidence exists that changing physician practices can affect child outcomes. As mentioned earlier, the MTA study suggested that study-delivered rigorous medication management produced better ADHD symptom improvement than medication management delivered in the community.⁴ Further, Epstein et al²¹ demonstrated that children treated by pediatricians who successfully adopted a consultation service had much better outcomes than children treated by pediatricians who did not have access to these services. Thus, there is at least circumstantial evidence that the ADHD symptom trajectories demonstrated in this study are the result of improved physician practice behaviors. This is an important area for future study, as these findings suggest that less intensive and lower-cost medication titration trials can produce results similar to the high-intensity MTA medication algorithm.

Another notable finding from the study is the lack of any gains in functional impairment as a result of treatment. The improvements observed in ADHD symptomatology did not translate into improvement in functional impairment. Besides improving ADHD symptomatology, pharmacotherapy has been documented to improve cognitive functioning^22- 24 and academic accuracy²⁵ and decrease negative social behaviors¹¹ and negative parent/child interactions.^26- 28 However, these improvements do not appear to generalize to global areas of functional impairment such as academic performance, social relationships, and familial relationships. Success in each of these global areas of impairment requires not only the elimination of behaviors that impair these abilities (eg, impulsive behavior with peers) but also the development of a set of skills (eg, prosocial behaviors). For example, to improve academics a child needs to develop organizational, study, and learning skills that are unlikely to appear as a result of alleviation of ADHD symptomatology. Hence, effective treatment likely requires a multimodal treatment strategy that includes a focus on teaching children skills.

An alternate explanation for the lack functional improvement is that the manner in which our data was collected and coded (ie, dichotomous: impairment or no impairment) may have reduced measurement sensitivity. Specifically, we could not detect small improvements in impairment. Children may have continued to be rated as impaired (score of 4 or 5) owing to the inability of medication to normalize behavior. Indeed, a recent community-based study found that, while children treated with medication made some improvements in reading and math, children remained significantly impaired relative to their peers who did not have ADHD.¹⁰ This was also true of children in the MTA medication group who made significant improvements in teacher-rated social skills and in parent-child relationships³ but remained significantly impaired relative to their peers who did not have ADHD.²⁹

This study has several important limitations. First, medication was prescribed according to typical open-label methods used by most primary care providers when beginning medication. The biases from open-label prescribing and unblinded raters may have biased ratings toward an exaggerated treatment response. Without a control group of patients who received no treatment or placebo treatment, it is difficult, if not impossible, to account for possible regression to the mean and/or placebo effects. However, improvements in ADHD symptoms in this study (parent d = 1.45; teacher d = 1.47) were considerably larger than gains made by the community comparison group in the MTA³ (parent d = 0.86; teacher d = 0.88) in which 67% received medication using open-label prescribing and unblind rating procedures.

Second, the study methodology did not collect data regarding medication regimens or adherence to the prescribed regimens. Hence, it is difficult to tell whether the apparent ADHD symptom treatment gains may have resulted from the prescribing or use of more medication. Along with not knowing the medication regimens being prescribed, information also was not obtained regarding ancillary services. As a result, analyses cannot be performed to determine whether concomitant educational and/or psychosocial interventions affected ADHD symptom improvements or functional impairment.

The decrease in sample size from initial assessment to the follow-up assessments is another limitation. Getting physicians to reliably collect follow-up parent- and teacher-rating scales during medication maintenance is a consistent obstacle in community-based intervention.²¹ Our analyses revealed no significant differences in the severity of ADHD symptoms or impairment at initial assessment for children who were consistently followed up compared with those who were not. However, factors other than initial disease severity (eg, chaotic home environment or comorbidities) may have differentiated these groups and been associated with the lack of functional improvement. Indeed, it is plausible that physicians completed more frequent follow-up visits for certain children specifically because of this lack of improvement in functional impairment. It is also possible that an insufficient period of time had passed for functional impairment to improve because most of the functional outcome data were from the 0 to 3–month treatment period.

This study demonstrates that significant improvements in ADHD symptomatology can be achieved in typical primary care settings. These improvements may rely on providing evidence-based care, which can be accomplished through quality improvement intervention. However, the significant improvements in ADHD symptomatology must be contextualized with the lack of improvement in functional impairment. Indeed, it appears that children treated in community primary care settings, despite displaying treatment-related improvements in ADHD symptomatology, do not show improvement in functional impairment. This finding highlights the need for physicians to work with or refer patients to educational and mental health care specialists who can work with children to develop skills to address targeted areas of deficit.

Corresponding Author: Jeffery N. Epstein, PhD, Center for ADHD, Cincinnati Children's Hospital Medical Center, Mail Location 10006, Cincinnati, OH 45229-3039 (jeff.epstein@cchmc.org).

Accepted for Publication: August 14, 2009.

Author Contributions: Dr Epstein had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Epstein, Lichtenstein, and Stark. Acquisition of data: Epstein, Langberg, House, and Stark. Analysis and interpretation of data: Epstein, Langberg, Lichtenstein, Altaye, Brinkman, House, and Stark. Drafting of the manuscript: Epstein, Langberg, Altaye, and Brinkman. Critical revision of the manuscript for important intellectual content: Epstein, Langberg, Lichtenstein, Brinkman, House, and Stark. Statistical analysis: Epstein, Langberg, and Altaye. Obtained funding: Epstein, Lichtenstein, and Stark. Administrative, technical, or material support: Epstein, Langberg, Lichtenstein, Brinkman, House, and Stark. Study supervision: Epstein, Langberg, and Lichtenstein.

Financial Disclosure: None reported.

Funding/Support: This study was supported by the Cincinnati Children's Hospital Medical Center Patient Innovation Fund.

Additional Contributions: We are grateful to the community pediatric offices that volunteered to participate in this study. We would also like to thank the following staff members who contributed to this project: Beth Mainwaring, MA, Rebecca Kolb, MA, Carolyn Luzader, MS, Katherine House, BA, Heidi Bunker, BA, Laura Schaffner, BA, Laura Conley, BA, Oluwayemisi Adesida, BA, Katherine Clarke, BA, and Affabwaje Jatau, BA. None of those acknowledged received any financial compensation for their contributions.