Measuring Quality of Life in Children With Attention-deficit/Hyperactivity Disorder and Their Families: Title and subTitle BreakDevelopment and Evaluation of a New Tool FREE

ATTENTION-DEFICIT/hyperactivity disorder (ADHD) is one of the most common pediatric conditions, yet its diagnosis, treatment, and outcomes remain complex and subject to controversy.¹ The prevalence of the disorder among school-aged children is estimated to be between 3% and 11%² and has been found to affect between 3 to 6 times as many male as female subjects.³

The disorder usually evidences itself in settings other than the clinician's office. Thus, a multimodal approach to diagnosis—which includes an array of clinical evaluations and empirical data about the frequency and intensity of symptoms from parents, teachers, and others with varying levels of training in child development and behavior—is often used.⁴ While this approach has resulted in warnings about misdiagnosis and overmedication,⁵ review of the literature has demonstrated little basis for this concern.⁶

According to the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV), diagnostic criteria include early onset (usually before the age of 7 years) of continuous symptoms of inattention and/or hyperactivity, resulting in clinically significant social, academic, or occupational impairment.⁷ The DSM-IV further divides ADHD into 3 subtypes: predominantly hyperactive-impulsive, predominantly inattentive, and combined. Diagnosis is often made by means of standardized ratings scales,^{8 - 9} symptom checklists, or structured interviews.¹⁰ There is considerable comorbidity of the disorder, most commonly with learning disorders, conduct disorder, and oppositional defiant disorder.^{11 - 13}

In addition to social and academic impairment, significant decrements in health-related quality of life have been reported for children with attentional problems and hyperactivity relative to peers.^{14 - 18} However, the effect of the disorder on the everyday functioning and well-being of children remains largely unexplored in clinical practice.¹⁹

Current practice suggests that children with ADHD benefit from stimulant medication^{20 - 21} and that effective treatment often requires a comprehensive, multimodal approach that includes behavior modification. Significant variation has been observed in clinicians' choices and combinations of therapeutic agents.^{22 - 23} Yet, much current research is still focused on the objective measurement of symptoms²⁴ rather than outcomes of treatment.

While symptom checklists are useful in standardizing diagnoses and following core symptoms, they are a crude measure for evaluating the effect of treatment on the everyday lives of families and children with ADHD. Treatment may be able to reduce the frequency of related symptoms and thereby bring a welcome reprieve from the problems of living with ADHD. However, this may not necessarily result in an improvement in the quality of life at home, at school, or with peers. Rigorous measures are needed to document whether therapeutic interventions for ADHD are improving the quality of everyday life for these children.

For many common childhood conditions, progress can be measured using objective tools like a spirometer or blood glucose monitor and, if warranted, treatment can be modified accordingly. Pediatricians, neurologists, or psychiatrists treating inattention and hyperactivity have no objective tool with which to monitor the outcomes of treatment. Within the time constraints of a brief office visit, clinicians often must rely on the parent's response to a global "how's it going" or use questionnaires that just examine core symptoms to determine if therapeutic intervention is having an effect. To our knowledge, nothing exists that enables the practitioner to "fine-tune" treatment for ADHD based on the information that is exchanged in these encounters.

Given current prevalence estimates, reliable and valid tools are needed to facilitate a more rigorous approach to evaluating interventions at the time of the clinic visit. Measuring the outcome of care on the quality of the child's everyday life in addition to current core symptom questionnaires would enable clinicians treating ADHD to benchmark the outcome of their care in terms that are increasingly meaningful to the families and children they are treating.

The purpose of this study was to develop and evaluate a brief parent-completed questionnaire that could be ultimately used by clinicians to document the outcome of care specifically for children with ADHD. The measure can be completed at home or while families are waiting to be seen by the physician. It was designed to complement diagnostic information and other clinical data by assessing the experiences and feelings of families and children with inattention and hyperactivity and the degree to which the child's disorder affects the quality of their lives at home and in general. Specifically, we were interested in developing a brief tool that could answer questions such as: Is the family limited in doing the usual things like entertaining at home or going to public places? How worried are parents about their child's future or about the effects of ADHD on other siblings? How do they feel about their ability to cope with their child's ADHD? Is their child excluded from the usual activities with friends such as sleepovers, parties, or just "hanging out"?

Given the focus of our tool, parents were identified as the appropriate respondent (as opposed to teachers who are often used in ADHD studies). Further, the parent was chosen as the primary respondent so that we might capture preschool-aged children in addition to those attending school. Future efforts will explore the development of a youth self-report.

A multistaged approach was used in the development process of the ADHD Impact Module (AIM), HealthAct, Boston, Mass. After the current literature was reviewed, interviews were conducted with 3 families, 2 clinicians (M.R. and L.R.) treating children with ADHD at a nationally recognized pediatric hospital, and 1 licensed educator who was the director of a community-based support program for children with behavioral issues and their parents.

The interviews lasted 40 to 90 minutes and were conducted independently by the principal author (J.M.L.) who is trained in psychometrics and questionnaire design, ethnomethodology, and observational research.^{25 - 28} These interviews build on the author's previous observational and measurement work when developing the Child Health Questionnaire–a general quality-of-life measure for children that has been evaluated for use in ADHD.^{14 - 18} Previous experience included lengthy observations at 2 ADHD clinics, interviews with 2 recognized clinical experts, and 2 home observations and interviews with families.

To further our current understanding of the effect of ADHD on children and their families, observations were made at a family home and one-on-one conversational interviews were held with 2 fathers, 2 mothers, an 11-year-old boy with ADHD, and his 10-year-old sister. In describing everyday life, the families and children identified aspects of home life that are significantly influenced by ADHD, such as following through with homework and chores, the child's ability to cope with everyday hassles, the child not getting invited to sleepovers, parents feeling constrained about entertaining others in their home, and overall how the family is managing.

The Likert method of summated ratings,²⁹ which is based on testable scaling assumptions, was used in the design of the AIM. This method uses a graduated response continuum (eg, "a lot" to "not at all"). Items were written to correspond to each of the 2 principal constructs that emerged from discussions with families—influence on the child and influence on the parent-family. Whenever possible, items were constructed using the phrasing provided by the families such as those identified in the previous paragraph.

The AIM is composed of 2 core multi-item scales: the Child Scale and the Home Scale were specifically constructed to capture the effect of ADHD on the child and the quality of life at home, and 9-descriptor items to assess treatment status and history and other related background information. The Child Scale consists of 8 items that measure the well-being of the child (eg, child does well following through with homework, feedback from teachers has been positive, and my child seems comfortable with how things are going). The Home Scale consists of 10 items that assess the influence on the family-parent (eg, my child's ADHD limits what we can do as a family, my child's ADHD has added stress to our home life, and I feel tired and worn out).

The 9 descriptive items include (1) 1 item to determine the child's medication status during the 2 weeks prior to completion of the AIM; (2) 1 item to gauge the frequency with which a parent may experience "success" at home in helping their child refocus or regain self-control; (3) 4 items to assess parental attributions regarding ADHD (is it a health issue, discipline issue, parenting issue,or behavioral issue); (4) date of diagnosis; (5) how long the child has been receiving medication for ADHD; and (6) whether the child or the family have attended an ADHD support group. Sample items from the AIM are shown in Table 1.

The core AIM scales were evaluated for readability and ease of completion using the Flesch-Kincaid method, a gold-standard reading comprehension program.³⁰ Results indicated that the they can be completed by individuals with at least a fifth-grade reading level and would be easy to read by 75% of the readers at this level.

The AIM was reviewed by the 2 clinical authors (M.R. and L.R.) both of whom actively treat children with ADHD at a nationally recognized pediatric hospital. In addition, 2 families of children with ADHD were asked to complete the AIM to determine if there were problematic items. Recommended enhancements and clarifications were incorporated. Questionnaire development guidelines suggest that typically it takes someone 30 to 45 seconds to complete each multiple-choice or true-false item.³¹ Once finalized, the AIM was piloted in a sample of families of children who had been diagnosed with ADHD by a trained clinician using standard diagnostic DSM-IV criteria. Based on the pilot study results and the aforementioned guidelines, we estimate that most people can complete the AIM in 4 to 7 minutes.

Our convenience sample was drawn from a patient registry of children who had or were receiving stimulant medication treatment for ADHD from an outpatient development and behavioral program of a large pediatric hospital in the greater Boston area. All children in the registry had been formally diagnosed with ADHD by a team composed of a developmental behavioral pediatrician, a PhD-level psychologist, and an MA- or PhD-level educational specialist. The standard diagnostic process used in the clinic includes extensive educational, neurodevelopmental, and psychological testing to identify alternative causes of and/or comorbidities of activity and attention problems. Each child also receives a physical examination and vision and hearing tests. Parental histories were obtained separately by the psychologists and the pediatrician (L.R.). The Clinical Attention Problems Scale measures the frequency of activity and attention by asking the parent and teacher to respond to a series of 12 statements and their applicability to their child in the morning and afternoon. Response options range from "not true," "somewhat or sometimes true," "very often," or often true").^{32 - 33} Consensus of diagnosis was established by the full team using DSM-IV criteria and evaluating results of the extensive evaluations. The institutional review board of the hospital approved the project under their quality improvement program.

The eligible patient population consisted of 259 families living in Rhode Island, Massachusetts, and New Hampshire. A letter prepared and signed by the clinic director (L.R.) and a postage-paid business reply envelope accompanied each questionnaire. To maintain confidentiality, per the terms of institutional review board approval, there was no follow-up to nonrespondents. No family identifiers were used and voluntary completion of the form constituted informed consent.

Scores for the Child Scale and the Home Scale were computed separately by summing the items within each scale and deriving an overall mean score. The raw mean scale score was then transformed on a 0 through 100 continuum with higher scores indicating less negative influence or better functioning and well-being. Using standard scoring conventions, a higher score is more favorable. Multitrait analysis was used to assure that grouping the items into the 2 separate scales as we did was appropriate. All computations were performed using the Revised Multitrait Analysis Program for a disk operating system.³⁴ Multitrait analysis is a confirmatory factor analytic method that was originally used in the construction of achievement tests and has been applied to the development of patient-based measures in the health care field since the early 1970s.³⁵ It has been used as the method of choice in the evaluation of a wide array of published instruments.³⁵

As mentioned, items were written and grouped a priori to correspond specifically to 2 key concepts—influence on the child and influence on the parent-family. Multitrait analysis was chosen as the principal method of analysis because it allowed us to test the strength of our scaling assumptions (ie, grouping of items). This method extends traditional factor analysis by examining the discriminatory power of items in addition to their convergence to confirm that the hypothesized sets of items (ie, scales), indeed, measure separate and unique constructs. In other words, one might expect to see a relationship between items from the Child Scale and the Home Scale, but the correlation across items from the 2 scales should be low enough to support grouping them as expected to derive 2 independent scale scores. Multitrait analysis provides a reliable way to assess the strength of these correlations. Further, it is recommended that "tests of scaling assumptions should be conducted before new items or scales are relied upon in formal studies."³⁵ (p4)

Specifically, the Revised Multitrait Analysis Program performs tests of scaling assumptions using the following criteria. First, each item in a hypothesized scale must be substantially linearly related to the underlying concept being measured (tests of item internal consistency). An item-scale correlation, corrected for overlap, of 0.40 or above has been recommended.³⁶

Second, each item should correlate significantly higher with its hypothesized scale than with other scales in the same matrix (tests of item discriminant validity). To satisfy the item discriminant validity criterion, the correlation between an item and its hypothesized scale must be significant. The convention, based on an SE of 0.04, is a magnitude of 2 SEs higher than its correlation with other scales.³⁷ For new scales, however, it is acceptable to extend the criteria for tests of discriminant validity such that a "success" is counted if the correlation of an item to its hypothesized scale was at least 1 SE higher than correlations with other scales.³⁶

Reliability is a function of the average correlation among items. Since it is possible for a measure to appear quite stable over time, but not be internally consistent, test-retest is not recommended as a method of choice to estimate reliability.³⁸ Thus, the internal consistency reliability for the 2 AIM Impact scales were estimated using Cronbach α coefficient.³⁹ It has been noted that scales with reliabilities of at least 0.70 and higher are sufficiently reliable for use with group comparisons.⁴⁰ Reliability estimates of 0.90 or higher are suggested for use at the patient-specific level.^{37 ,39} The α coefficient represents the average of all possible split-half reliability estimates adjusting for scale length and has been shown to approximate test-retest estimates when scaling assumptions are met.^{41 - 43}

Respondents had to have answered at least half of the items for each of the scales to be included in the analysis. Given that this was a newly developed measure and we were evaluating its psychometric properties, if a patient did not complete more than half of the items per scale, their entire form would be excluded from any analysis.

The discriminatory power of an instrument—its ability to discriminate within and across groups—is determined in part by the distribution of scores observed for a given sample. The more scores are spread across the continuum, the greater the chances that a true and measurable difference can be found. Thus, to be truly useful, the full range of the measure should be observed with few people scoring at either the lowest (floor) or the highest end (ceiling) of the continuum.

We hypothesized that the negative impact on quality of life would be greater for (1) children with ADHD combined compared with those with ADHD primarily inattentive; (2) parents reporting no success at home in getting their child to refocus or gain control of their behavior vs those reporting success; and (3) children not taking medication for their ADHD compared with those receiving medication. To assess discriminant validity of the AIM, differences in scale scores between each of these groups were examined for significance using the t test for independent samples.

Responses were received from 81 (31%) of the 259 families surveyed. Table 2 gives the demographic profile of the responding families and patients. Mothers (76 [94%]) preponderantly completed the questionnaires. The majority (60 [74%]) of the patients were boys, who were distributed relatively evenly in age and school grade. An even split was observed between children whose parents reported a diagnosis (n = 80) of ADHD primarily inattentive (40 [50%]) and those whose parents reported a diagnosis of ADHD combined (40 [50%]). A large majority (70 [86%]) of the sample (n = 70) had been diagnosed with ADHD 1 year or longer prior to completing the AIM. Parents reported that 49 (63%) of the 78 children for whom medication status was provided were taking their medication as directed during the 2 weeks prior to completing the AIM. Fifteen children (19%) had taken a short drug holiday (≤4 days), 5 (6%) had taken a lengthy drug holiday (≥5 days), and 9 (11%) were not taking medication at the time the questionnaire was completed.

Figure 1 profiles parental attributions regarding ADHD. A strong majority (66%-70%) feel that ADHD is a health or behavioral issue. Yet, 45% to 48% of the parents also attribute ADHD to parenting and discipline issues. These findings underscore the importance of additional education and management for children and their families. Specifically, what is needed is further education about the neurological basis of ADHD and the special skills necessary in parenting a child with this condition.

Respondents had to have answered at least half of the items for each of the scales to be included in the analysis. Using this criterion, no one was omitted from the analysis. Minimal missing data were observed in 1% to 2% of the responses for only 3 items in the Child Scale (child has received positive feedback, child seems comfortable with how things are going, and child adapts well to unexpected changes) and 2 items in the Home Scale (limits me from entertaining and I am frustrated that my child's ADHD is unmanageable). For a third item in the Home Scale—effect on siblings—7 parents indicated by written comment that the item was "not applicable" because they had an only child. Given the hypothesized linearity of items, which is the theoretical underpinning of the Likert approach, and the low miss rate, values were not imputed for the omitted items. The SE for the entire group, which is based on sample size, was 11. This is not surprising given the small sample (n = 81). The average SE for groups of 200 or more ranges from 0.04 to 0.06. Thus, in this sample, items had to work harder at meeting the scaling criteria.

Item correlations for the Child Scale ranged from 0.53 to 0.71 and for the Home Scale from 0.61 to 0.85. Eighty-eight percent of the items in the Child Scale and 100% of items in the Home Scale met the item-discriminant criteria. These findings confirmed that our scoring approach (ie, summing the items and deriving 2 independent scale scores) was appropriate.

A relatively even distribution of responses was observed across both scales. No floor effects were observed for either scale. A minimal ceiling effect (2%) was observed for the Home Scale only. This finding suggests that the AIM has potential discriminatory power and any differences observed in mean scale scores would not be due to chance occurrence.

The Cronbach α coefficient observed for the Child Scale was 0.88. The Cronbach α coefficient for the Home Scale was 0. 93. These coefficients substantially exceed the recommended criteria (0.70) for group-level comparisons. They also provide evidence that the scales may be sufficiently robust for patient-specific reporting—a distinct advantage for both families and practitioners. As such, the instrument would provide practitioners with a tool to assess individualized treatment strategies and determine their benefits and limitations for the child and his or her family.

Analyzed by the t test for independent samples, scores on both the Child and the Home Scales were significantly worse for children with ADHD combined than for children with ADHD primarily inattentive. Mean scale scores for the 2 groups are given in Table 3. The mean Child Scale score for children with ADHD combined was 48.86 vs 65.26 for children with ADHD primarily inattentive (P = .000). The mean Home Scale score was 51.26 vs 72.79 (P = .000), respectively.

Also given in Table 3 are the significant differences observed across the Child and Home Scales for parents reporting success with behavioral interventions at home as compared with those reporting no success. Better scores were observed for parents reporting success vs no success on both the Child Scale (62.12 vs 47.36, P = .001) and the Home Scale (70.58 vs 47.01, P = .000).

An interesting pattern was observed (but not tested for significance) for the item "success at home with behavioral interventions." Of those reporting success (n = 39), more were parents of children with ADHD primarily inattentive (n = 30) relative to parents of children with ADHD combined (n = 9). For those indicating no success (n = 40), a relatively equal distribution was observed across the parent-reported diagnostic subgroups (ADHD primarily inattentive [n = 19] and ADHD combined [n = 21]).

These data were inadequate to determine significant statistical differences in quality of life between those patients taking medication and those not taking medication. Only 9 children (11%) were not actively taking medication and 5 (56%) of these represented the ADHD primarily inattentive group. The same pattern was observed for the 20 children on drug holiday (25%) with 12 (60%) representing the ADHD primarily inattentive group. Thus, of the 29 patients who were not receiving medications or who were on a drug holiday, 17 patients (59%) were categorized as having ADHD primarily inattentive. It was impossible to control for the confounding effect of the diagnosis and potential skewing of these data owing to the small sample size.

This pilot test of the AIM provides clinicians with a promising new option for gathering standardized information on the effect of ADHD and its treatment on the everyday life of affected children and their families. A multimodal treatment strategy has long been advocated for treating children with attention difficulties. To date, however, there has not been a way for measuring and comparing the benefits of different treatment options as they occur in the applied clinical setting. In addition, current questionnaires only examine the core symptoms of ADHD that are certainly important but probably inadequate from a clinical viewpoint.

Currently, physicians must rely on symptom checklists and information obtained in brief and often hurried encounters with parents to determine whether and to what degree an intervention is improving the child's life at home. Finally, we are unaware of any available tool that assesses the effect of ADHD on parents. Thus, current assessment strategies fail to fully gauge the effect that ADHD and its treatment may have on the everyday quality of life for children with ADHD and their families. If we are to truly understand the effect of ADHD interventions on the life of the child and his or her family, the outcome of treatment must be routinely monitored over time—with measurement occurring parallel to the clinical encounter.

Reliability estimates indicate that the AIM scales can be used with confidence for group-level comparisons and that they approach or exceed the minimum level for use to monitor the outcome of care at the individual patient level. The next step is to assess reliability of the AIM in longitudinal prospective studies and to further monitor its performance at the individual patient level. Preliminary findings of discriminant validity, as evidenced by expected and observed differences in mean scale scores for children with ADHD primarily inattentive vs ADHD combined and parents reporting success at home vs no success, confirm other published work in the field.

While these general findings underscore what has been reported from large-scale studies, the AIM presents as a potentially promising new tool that dimensions the influence of ADHD. Future efforts will examine data at a practice or clinic level to assess the outcome of different treatment options. Benchmarking care in this way will, in the long-term, provide the evidence necessary for defining "best-practice" guidelines.

There are several limitations to this study that must be noted. First, we acknowledge that the number of patients returning questionnaires was low. To protect patient confidentiality and to fulfill the requirements of the institutional review board, it was impossible to follow up with patients, obtain demographic information about nonrespondents, or obtain current contact information for those patients who might have moved since their initial visit to the clinic. Given these constraints, we were unable to determine if our respondents are representative of all patients and families listed in the ADHD database from which our study sample was drawn.

Our principal interest was in evaluating the underlying conceptual structure of a new tool to capture the effect of ADHD on children and families as opposed to describing the effect of ADHD on families in general. The full range of possible scores was observed across the AIM scales indicating both the high and low effects on well-being and family life. Thus, there is no reason to believe that the sample is skewed toward one end of the spectrum or the other. Future efforts will focus on the design of clinic-based prospective studies wherein we can further evaluate the performance of the AIM.

Further, although, our sample may be small relative to the percentage of children diagnosed as having ADHD, traditional psychometric tests, such as those discussed in this article, are sample sensitive. As such, criteria are actually more stringent and difficult to achieve with samples of less than 100. Despite the small sample, the effect size was large enough that the AIM achieved the standards and performed well. These preliminary results are encouraging and suggest that the AIM can be used with confidence as normative data are collected and a larger prospective study is undertaken.

Second, our sample was obtained from an ADHD–specific clinic database associated with an academic medical center and, thus, may not be generalizable to other clinic settings. A review of clinic records between April 1, 2000, and June 30, 2000, indicated that 54 children with ADHD were seen. Of these, 44 (82%) were diagnosed as having ADHD combined. Thus, we believe the pool of respondents was drawn from a clinic database that reflects the general profile for other clinics treating this affected population. Since patients were not identified, it is impossible to link their responses on the AIM to a standardized diagnostic tool. Thus, for evaluating the differences in scores by diagnostic group, we were limited to using parental reports. Based on our work with parent interviews and the standardized use of parent-completed ADHD diagnostic tools, we feel confident that parents are accurate reporters. Future efforts will focus on the correlation between scores on standard diagnostic tools and the AIM.

Third, we did not ask parents to report comorbidities. However, it was possible to retrieve the principal diagnoses for the clinic's current data set. Findings indicate that 41% have problems associated with academic issues, 15% have behavioral issues, 14% have language problems, 8% have psychiatric or emotional problems, 5% have motor or sensory problems, 5% have global cognitive problems, 4% have environmental or family problems, 4% have problems with organizational skills, approximately 3% have autism-social cognition problems, and 1% have problems with specific organic causes. Comorbidities and their effects on quality of life will be monitored as the measurement properties of the AIM are further evaluated.

Questionnaire development is a complex and iterative process. Each application of the AIM will provide further insight into its strengths and limitations. Use of this tool in the clinical research setting may not only be useful to practicing clinicians, but data that are generated will be an invaluable asset in the interpretation of scores and, ultimately, with further study, the assessment of practice guidelines. Future efforts will focus on key issues such as the sensitivity of the AIM to changes over time and with drug treatment, relationship with diagnostic tools, and the development of a normative database to determine whether children of different ages or from different backgrounds score differently. Investigations are in place to further assess the performance of the AIM with an eye toward its ongoing use as an important tool to augment the diagnostic process and further assist practitioners in benchmarking the care and treatment of ADHD. Ultimately, information from the AIM will yield a richer understanding about the success of various treatment options (what works and does not work) in terms that are most meaningful to the children with ADHD and their families.

Accepted for publication December 6, 2001.

Development work on the AIM was funded in part by Eli Lilly & Company, Indianapolis, Ind.

Attention-deficit/hyperactivity disorder is a common pediatric condition (3%-11%) with a significant negative effect on the quality of life for the affected child and his or her family. Current practice guidelines recommend comprehensive, multimodal treatment for ADHD, including stimulant medication and behavioral modification, yet actual management practice varies widely. While several symptom checklists have been developed for diagnosis, there are no instruments with which to measure the more global impact of treatment on the child and his or her family. We describe the development and pilot implementation of a parent-completed questionnaire that measures the effect of ADHD and its treatment on the quality of life of affected children and their families. The AIM met strict psychometric standards for item-convergent and -discriminant validity and found significant differences in quality of life between children with ADHD inattentive and ADHD combined. There is evidence to indicate that this is a promising questionnaire that may be used to measure and benchmark the outcomes of care for ADHD.

Corresponding author and reprints: Jeanne M. Landgraf, MA, HealthAct, 205 Newbury St, Fourth Floor, Boston, MA 02116 (e-mail: jml@healthact.com).