Health-Related Quality of Life in Pediatric Minor Injury:  Reliability, Validity, and Responsiveness of the Pediatric Quality of Life Inventory in the Emergency Department FREE

Clinical research, particularly in acute care pediatric settings, is limited by our ability to measure appropriate outcomes. Routinely measured emergency medicine outcomes (admission rates, emergency department [ED] recidivism or unscheduled return to care, and mortality) are not applicable in most acute care pediatric ED presentations.^1- 2 Clinical markers alone are inadequate assessments in outcomes and effectiveness research, highlighting the need to develop measures that include evaluations of patient viewpoint and experience after ED care.^1- 15 Improving our ability to measure outcomes after pediatric ED care will facilitate clinical research in the field, expand the evidence base, and allow for informed decisions as we work to improve the care of children presenting to the ED.

Health-related quality of life (HRQOL) is a type of patient-reported outcome that reflects the patient's view of the impact of health care services. Health-related quality of life is highly associated with patient status across many outcome domains and is among the best predictors of the use of medical services, even after controlling for clinical factors.^{2,6,14,16- 19} Different HRQOL measures are used extensively in descriptive and effectiveness studies to assess outcomes in children with chronic disease^20- 28 and recently were validated for use in long-term outcome assessment in children hospitalized after major trauma.^29- 31

However, the use of HRQOL to assess short-term outcome after treatment in acute care settings, such as the pediatric ED, has barely been explored.^23,32- 33 Most pediatric ED visits are by previously healthy children with acute limited conditions, such as infectious disease or minor injury. Assessing short-term outcomes for these patients is necessary to isolate their response to the brief unit of care, the ED visit.^2,5,9

In this report, we examine the performance of a pediatric HRQOL instrument, the Pediatric Quality of Life Inventory 4.0 Generic Core Scales (PedsQL), in children after pediatric ED care for minor injury. The PedsQL is widely validated for use in pediatric chronic disease and as a population health measure.^{22- 25,27,34- 38} Our aim was to evaluate the psychometric properties (including feasibility, reliability, validity, and responsiveness) of this HRQOL measure assessing short-term outcome after pediatric ED care.

Participants were prospectively enrolled after presenting to a large, urban, children's hospital ED for treatment of minor injury (defined as a single injury, occurring within 24 hours of presentation, in patients discharged to home after ED care). We included children and adolescents aged 2 to 18 years. Exclusion criteria included non–English-speaking patients or caretakers, trauma team activation, no parent in the ED, or suspected child maltreatment. The study was approved by the hospital's institutional review board.

Enrollment occurred on randomly selected study days (18 h/d, 7 d/wk) during the 12-month period from June 1, 2008, through May 31, 2009. Informed consent and assent were obtained before enrollment.

Demographic and injury data were abstracted from the ED record at the time of the visit, and baseline HRQOL was assessed (child and parent reports).

Short-term clinical outcomes were collected at telephone follow-up interviews with the parent and with the child (if he or she was 8 years or older) at 1 week (attempts on days 6-9) and 2 weeks (attempts on days 13-18) after the ED visit. Age-appropriate HRQOL measures (child report for children and adolescents aged 5-18 years and parent report for children and adolescents aged 2-18 years) were also administered at the telephone follow-up.

The PedsQL is a multidimensional measure initially developed and validated for use in children with chronic disease.^34- 36 The instrument uses parallel child (self) and parent (proxy) reports for children and adolescents aged 5 to 7, 8 to 12, and 13 to 18 years and parent report only for children aged 2 to 4 years. It takes 5 to 10 minutes to complete and has been validated for self-completion, face-to-face interview, and telephone administration. The total score (23 items) includes 2 subscales (physical health and psychosocial summaries) and is made up of items in the physical, emotional, social, and school functioning domains. Individual items are reverse scored and linearly transformed to a scale of 0 to 100, with higher numbers indicating higher HRQOL. The acute version uses a 7-day recall period instead of the 30-day period for the standard PedsQL. The acute version was validated in an outpatient subspecialty clinic setting^35,38 and in an acute exacerbation of a chronic disease.²³ A single study in pediatric ED patients assessed only the parent (proxy) reports.³³

For all families, the PedsQL was administered at the ED visit and again at the telephone follow-up. At the ED visit, the 7-day recall was specified as the 7 days before the injury, and these findings constitute the baseline PedsQL scores. Follow-up PedsQL scores were from the first successful follow-up telephone contact (at 2 weeks if 1-week attempts were unsuccessful). The PedsQL was administered according to the terms of the use agreement between the authors and distributors (http://www.mapi-trust.org).

Short-term clinical outcomes, specified as resulting from the minor injury, were collected by telephone interviews with the parent and with children and adolescents 8 years or older. They included days of pain/discomfort after the ED visit, days to return to baseline activities/routines for the child and family, and days of daycare/school/work or regularly scheduled activities missed by the child or the parent. To evaluate construct validity, clinical outcomes were considered individually, first continuously and then categorized as poor (lasting for ≥1 week) or good (<1 week). The 1-week cutoff was chosen a priori on the basis of previous studies of short-term outcome^32,39 and to correspond with the recall periods of the follow-up telephone interviews.

For known-group validity testing, an a priori composite grouping of clinical outcome was defined. As in previous studies, patients were dichotomized as having a composite good or poor outcome. A composite poor outcome was assigned if any 1 or more of the following was reported by the parent or the child: 7 or more days of pain, 7 or more days of abnormal patient or family activity, or 5 or more days of daycare, school, or work missed by the child or the parent. All other outcomes were assigned to the composite good outcome group.

Demographic characteristics for the study sample and patients lost to follow-up were compared using χ² test for categorical variables and unpaired, 2-tailed t tests or Mann-Whitney tests for continuous variables (based on data distribution). We calculated PedsQL scores according to the developer's instructions. Treatment of missing items followed the developer's protocol; scores were computed as the sum of items divided by the number answered. When more than half the items from any scale or subscale were missing, the score was not computed.³⁵ Data normality for reliability and validity analyses was assessed with the Kolmogorov-Smirnov test. Group comparisons were made using adjusted marginal means derived from univariate generalized linear models, with age and sex as covariates. Data analyses were conducted using commercially available software (SPSS, version 17.0; SPSS, Inc, Chicago, Illinois).

Feasibility was assessed by the success of telephone follow-up, by floor and ceiling effects (the percentage of scores at the bottom and top of the scale, respectively), and by calculating values for the percentage of missing items in the parent and patient report forms.

Instrument reliability reflects whether information is measured in a reproducible fashion, and internal consistency reliability looks at this reproducibility between instrument items. Internal consistency reliability was assessed using the Cronbach α, with values of at least 0.70 considered adequate for comparisons of groups and at least 0.90 for comparing individual patient scores.⁴⁰ Parent-child concordance was examined using intraclass correlation coefficients, with values of no more than 0.40 indicating poor to fair agreement; 0.41 to 0.60, moderate agreement; 0.61 to 0.80, good agreement; and 0.81 to 1.00, excellent agreement.⁴¹ We hypothesized that the internal consistency reliability in our setting would be similar to prior evaluations of the PedsQL, exceeding the standard for comparing groups for all subscales.^{34- 35,37,42} Moderate parent-child agreement was expected, as reported in previous PedsQL studies.^25,43

Test validity is assessed through evidence that the tool is measuring what is intended.⁴⁴ To evaluate construct validity, linear regression was used to compute the change in mean PedsQL scores for each day that the individual clinical outcomes persisted after the ED visit (ie, days with pain or days of disrupted activities). We hypothesized that HRQOL (PedsQL scores) should decrease as days of clinical symptoms increased.

Known group comparisons were used to demonstrate discriminative validity of the PedsQL in our setting.⁴⁵ Patients were grouped by injury characteristics or by clinical outcomes that were expected to represent different levels of HRQOL at follow-up. After adjusting for age and sex, mean PedsQL scores were calculated with 95% confidence intervals of group differences. For patients grouped by injury location and injury type, we hypothesized that the groups should not have significantly different PedsQL scores at baseline but show expected significant differences (statistically and clinically) at follow-up as in prior studies of long-term injury follow-up.^46- 48 We expected that fractures would have poorer HRQOL at follow-up than soft-tissue injuries and that lower extremity injuries would have a greater negative effect on HRQOL than upper extremity injuries. For the groups dichotomized by their clinical outcomes as good and poor (as defined for individual and composite clinical outcomes), we expected to find an association between short-term clinical outcomes and follow-up PedsQL scores, with significantly higher mean scores for children with our definitions of good clinical outcomes than for those with poor clinical outcomes.

Responsiveness of a scale is the ability to measure change over time, commonly understood as true change after adjusting for the “noise” of chance or inherent variation of scale scores.^44- 45,49 For this study, we assessed responsiveness by examining the change in PedsQL scores in relation to changes in clinical status during our follow-up period. We assessed distribution-based indicators of change, including the effect size (ES) and estimates of the minimal important difference (MID). Effect size was calculated as the mean change in score divided by the standard deviation of the score in the population at baseline.^32,50- 51 In previous PedsQL studies, the MID of a score change of 4 has been proposed.³⁴ However, it has been recommended that several estimates of the MID should be calculated and used to establish a range for the MID.⁵² We calculated previously suggested distribution-based approximations of the MID, including the sample's standard error of measurement (SEM), and one-half of the baseline standard deviation.^50,52- 54 We estimated the SEM by the product of the standard deviation of the scale score and the square root of 1 minus the internal consistency reliability coefficient for the scale score.⁴³

A total of 404 children and adolescents aged 2 to 18 years and their parents were enrolled. Telephone follow-up was successful in 334 of these families (82.7%). Patient characteristics are detailed in Table 1. Differences between patients lost to follow-up (70 of 334) and those successfully contacted were not statistically significant except for injury location. The median number of days between baseline and follow-up PedsQL was 9 (interquartile range [IQR], 8-15). The mean PedsQL scores at baseline were nearly identical to those reported for healthy populations.^35,55

The percentages of all missing values were small. For parents with children aged 2 to 4 years, 111 of 2184 items (5.1%) were missing, but all missing items were from the school/daycare subscale for children not in daycare or school. For parents with children aged 5 to 18 years, 31 of 9015 items (0.3%) items were missing. For children aged 5 to 18 years, 34 of 6739 items (0.5%) were missing. Baseline PedsQL forms were easy to administer during the ED visit. Floor and ceiling rates of the baseline and follow-up PedsQL scores are detailed in Table 1 and are very similar for child reports to general and healthy populations in prior studies.^35,55 Parent reports had higher ceiling rates than these populations only at baseline in our study, which may reflect a contrast effect of the recent injury of their child.

Good reliability of the PedsQL in our ED setting was reflected by high Cronbach α values. All parent and child scales were internally consistent (α range, 0.73-0.93). The α values for total scores, in all but child reports for children aged 5 to 7 years (α = 0.84), approached or exceeded 0.90. Parent-child concordance showed moderate agreement (interclass correlation coefficient for the total score, 0.52; physical summary score, 0.48; and psychosocial summary score, 0.49).

Clinical outcomes reported by parents and children were considered separately. First, the relationships between follow-up PedsQL scores and individual clinical outcomes (days of pain and missed or disrupted activities) were evaluated (Table 2). We found the expected overall inverse relationship: for each day that the clinical symptoms persisted, there were consistent decreases in mean HRQOL scores. Table 3 gives known group comparisons of baseline and follow-up PedsQL scores for different injury characteristics, including injury type (soft-tissue vs fracture) and injury location (upper vs lower extremity). After adjusting for age and sex, the compared groups were not different at baseline, but HRQOL at follow-up for lower extremity injuries was clinically and statistically significantly poorer than for upper extremity injuries and for fractures than for soft-tissue injuries. Effect sizes, measuring score change over time adjusted for normal score variations, were moderate to large in the physical subscales, with ES for lower extremity injury consistently larger than ES for upper extremity injury and fracture ES larger than soft-tissue injury ES.

Tables 4, 5, and 6 use clinical outcomes categorized as good and poor for known group comparisons. In Tables 4 and 5, if an individual short-term outcome had resolved by 7 days (good outcome), children had significantly better follow-up HRQOL scores than if it persisted for 1 week or more (poor outcome). These differences in follow-up PedsQL scores were particularly large, ranging from 1 to 2 SDs of the baseline score in all but a few psychosocial subscale comparisons, and were statistically significant for all child- and parent-reported outcomes.

Table 6 details mean PedsQL total and subscale summary scores for patients grouped by our composite definition of good and poor outcomes. Differences between outcome groups were not significant at baseline but became significant at follow-up. We found large differences at follow-up in total scores (parent and child reports, 13.7 and 13.3, respectively) and in the physical summary score (22.4 and 25.0, respectively), with smaller but still clinically and statistically significant differences in summary scores between good and poor outcome groups in the psychosocial subscale (8.9 and 7.1, respectively).

Responsiveness was evaluated in several ways. In validity testing, PedsQL scores reflected decreased HRQOL in groups with increasing severity of clinical symptoms, and poor outcome groups had significant negative changes in score during the follow-up period. Effect sizes were moderate to large for fractures (total and physical summary ES, 0.56-1.87) and lower extremity injuries (0.80-2.44) and were substantially smaller for soft-tissue injuries (Table 3). Effect sizes for the full sample of all injuries were small for total score (0.26 for the parent report and 0.30 for the child report), moderate to large for the physical summary score (0.62 and 1.10, respectively), and minimal or insignificant in the psychosocial summary score (0.06 and 0.14, respectively). For our entire sample, the SEM ranged from 4.0 to 6.5 for the parent- and child-reported total and subscales, and the half-SD index ranged from 6 to 9 (data not shown).

This is the first report, to our knowledge, supporting the reliability, validity, and responsiveness of the parent and patient forms of a pediatric HRQOL measure used to assess short-term outcome in typical pediatric ED presentations. Overall, we found the PedsQL to be feasible and reliable, to have good construct validity, to discriminate between levels of severity of patient outcome, and to be responsive to changes in health status during the 1 to 2 weeks after ED care of a minor injury. Administration of the PedsQL in the pediatric ED setting resulted in high completion rates and minimal floor and moderate ceiling effects, as found in previous studies. Child and parent reports were internally consistent, with moderate levels of child-parent concordance, also as shown in prior PedsQL studies.⁴³ Multiple assessments demonstrated validity and responsiveness as hypothesized. The PedsQL scale scores reflected decreased HRQOL with increasing severity of clinical symptoms and distinguished between groups with expected better and worse outcomes. The scales' ability to measure change during short-term follow-up was reflected in primarily moderate to large ESs for injury type and location, as well as for composite outcomes. Our evaluation of minimal important difference for the PedsQL using half SD and SEM was similar to prior estimates,³⁵ and almost all our known group comparisons were much greater than this minimum.

Our interest was to test the use of an HRQOL instrument for short-term follow-up of patient presentations typical of most pediatric ED visits: an acute limited illness or condition in a generally previously healthy child. We are aware of only 1 prior report testing the feasibility and psychometric properties of an HRQOL instrument for such patients in the pediatric ED. Mistry et al³³ evaluated parent reports of the PedsQL after ED evaluations for febrile illness and found them to be feasible for use, with favorable construct validity and moderate responsiveness. The psychometric evaluation of the PedsQL was more robust in our present study, which included child reports, higher follow-up rates, larger sample size, and expended responsiveness analyses.

There are several potential limitations to the present study. Our sample was recruited from an academic pediatric hospital and may not be generalizable to other ED settings. Owing to the short follow-up period, we did not obtain test-retest reliability data. In addition, we were able to estimate responsiveness using only distribution-based methods. Ideally, a patient-reported marker of minimal clinical change should be included.

In conclusion, the validation of the PedsQL provides evidence of its usefulness in the pediatric ED and is an important first step in evaluating HRQOL as a measure of short-term outcome in our setting. Measuring patient-reported well-being and functional outcome are important in comparative effectiveness and prevention research.⁵⁶ The use of HRQOL measures in the pediatric ED would improve our ability to assess the impact of illness or injury by including a wider range of outcome domains (role and social function, psychological well-being, and general health perceptions).⁵⁷ We have also shown that this HRQOL measure is sensitive and responsive to change during short-term follow-up. Measuring short-term outcomes better isolates the impact of ED care within the spectrum of health care, facilitating studies that compare pediatric ED processes, treatment efficacies, or other interventions. By using HRQOL to measure short-term outcomes, we may improve our clinical effectiveness research and improve the care and health of children treated in the ED.