Impact of a Mentoring and Skills Group Program on Mental Health Outcomes for Maltreated Children in Foster Care FREE

In the United States in 2007, 5.8 million children were referred to Child Protective Services and maltreatment was substantiated for 794 000 of them (approximately 1% of the child population).¹ In the same year, 496 000 children were in foster care on September 30 (approximately 0.7% of the child population).^1- 2 African American and multiracial children were overrepresented among children in foster care.³

Children who have been maltreated and placed in foster care are at risk for significant mental health problems including depression, posttraumatic stress, dissociation, social problems, suicidal behavior, attention-deficit/hyperactivity disorder, and conduct disorders.^4- 7 In a large study of children receiving child welfare services, 42% met diagnostic criteria for a Diagnostic and Statistical Manual of Mental Disorders (Fourth Edition) diagnosis.⁶ Studies of Medicaid claims suggest that as many as 57% of youths in foster care meet criteria for a mental disorder.⁸

Rates of service use are also higher among children placed in foster care.⁹ One California study found that children in foster care, who represented less than 4% of Medi-Cal–eligible children, accounted for 41% of all users of Medi-Cal mental health services.¹⁰ Another study found that children in foster care used more mental health services (including hospitalizations) than did children in the Aid to Families With Dependent Children program or children receiving Supplemental Security Income.^8,11 Although children in foster care are significant consumers of mental health services, some evidence suggests that many children do not receive needed services. In a recent, nationally representative study, between 37% and 44% of youths with child welfare service involvement scored in the borderline or clinical ranges on measures of mental health functioning, but only 11% of these youths were receiving outpatient mental health services.¹²

Despite the need for contextually sensitive, evidence-based prevention and intervention efforts for this high-risk population, few rigorous trials have been conducted. The Fostering Healthy Futures (FHF) 9-month preventive intervention was designed for preadolescent children aged 9 to 11 years recently placed in foster care due to child maltreatment. The FHF intervention includes 2 major components: skills groups and mentoring. Skills groups, which have been used effectively with other high-risk preadolescent populations, were designed to bring children in foster care together to reduce stigma and provide opportunities for them to learn skills in a supportive environment. Mentoring, which has demonstrated short-term efficacy in some studies, was designed to provide children in foster care with an additional supportive adult who could serve as a role model and advocate.

It was hypothesized that youths randomized to the intervention would evidence better self-esteem, social support, social acceptance, and coping skills immediately following the program and that these improvements would be associated with better mental health functioning and improved quality of life 6 months after the program.

The study was conducted from July 2002 to January 2009 in 2 participating Colorado counties. Participants were recruited in 5 cohorts over 5 consecutive summers from a list of all children aged 9 to 11 years who were placed in foster care in participating counties. Children were recruited if they met the following criteria: (1) had been placed in foster care by court order due to maltreatment within the preceding year; (2) currently resided in foster care within a 35-minute drive to skills group sites; (3) had lived with their current caregiver for at least 3 weeks; and (4) demonstrated adequate proficiency in English (although their caregivers could be monolingual Spanish speaking). When multiple members of a sibling group were eligible, 1 sibling was randomly selected to participate in the randomized controlled trial. Letters explaining the study were sent to families, followed by recruitment calls a week later. Participation was voluntary and could not be court ordered.

As the Consolidated Standards of Reporting Trials (CONSORT) diagram in the Figure shows, 91% of eligible children and their caregivers agreed to participate. After the baseline interview and prior to randomization (T1), 13% of the participants were deemed ineligible for the following reasons: 6 were no longer in foster care, 7 had information on their child welfare records (obtained after the interview) that made them ineligible (eg, incorrect birth date), 9 were developmentally delayed, and 2 were not proficient enough in English to participate in the skills groups. Of the remaining 156 who were randomized to treatment and control groups, 8% were lost to follow-up immediately following the intervention (T2) and 7% were lost to follow-up 6 months after the intervention (T3).

The study protocol was approved by the institutional review board, and informed consent and assent were obtained. All children who participated in the baseline interview (n = 180) were screened for cognitive, educational, and mental health problems using standardized tests of intellectual ability¹³ and academic achievement¹⁴ as well as normed caregiver- and child-report measures of psychological functioning. The findings and accompanying recommendations were summarized in reports provided to children's caseworkers, who were encouraged to use the reports to advocate for educational and mental health evaluation and services.

Eligible children in both the assessment-only group (hereafter referred to as the control group) and the assessment-plus-intervention group (hereafter referred to as the intervention group) were assessed 3 times: (1) baseline (2-3 months prior to the start of the intervention); (2) T2, immediately following the intervention (11-13 months after baseline); and (3) T3, 6 months after the intervention (17-20 months after baseline). At each interview, children and their current caregivers were interviewed by separate interviewers, typically at the child's residence. Interviewers were masked to the participants' condition, although some participants spontaneously disclosed their treatment condition. Children and caregivers were paid $40.00 for their participation. Teachers of participating children were also surveyed during the spring of 2 consecutive years—at 10 months after baseline and 1 year later. At 10 months after baseline, 92% of children's teachers were interviewed; 1 year later, 89% of children's teachers were interviewed. Following the baseline interview, children were randomized after stratifying by sex and county. All children were manually randomized, by cohort, in a single block.

The 9-month FHF preventive intervention consisted of 2 components: (1) manualized skills groups; and (2) one-on-one mentoring by graduate students in social work (FHF is described in detail elsewhere¹⁵). The program was designed to be above and beyond treatment as usual. Although eligibility criteria required that children be in foster care at the start of the intervention, if they reunified or changed placements during the intervention, their participation continued following appropriate consent.

The FHF skills groups met for 30 weeks for 1.5 h/wk during the academic year and included 8 to 10 children and 2 group facilitators (licensed clinicians and graduate student trainees). The FHF skills groups followed a manualized curriculum that combined traditional cognitive-behavioral skills group activities with process-oriented material. Units addressed topics including emotion recognition, perspective taking, problem solving, anger management, cultural identity, change and loss, healthy relationships, peer pressure, abuse prevention, and future orientation.¹⁵ The skills group curriculum was based on materials from evidence-based skills group programs, including Promoting Alternative Thinking Strategies^16- 17 and Second Step,¹⁸ which were supplemented with project-designed exercises from multicultural sources. The skills group curriculum included weekly activities that encouraged children to practice newly learned skills with their mentors in their communities.

The mentoring component of the FHF program provided 30 weeks of one-on-one mentoring for each child. Mentors were graduate students in social work who received course credit for their work on the project. Mentors were each paired with 2 children with whom they spent 2 to 4 hours of individual time each week. They also transported children to and from skills groups and joined the skills groups for dinner. Mentors received weekly individual and group supervision and attended a didactic seminar, all of which were designed to support mentors as they (1) created empowering relationships with children, serving as positive examples for future relationships; (2) ensured that children received appropriate services in multiple domains and served as a support for children as they faced challenges within various systems; (3) helped children generalize skills learned in group to the “real world” by completing weekly activities; (4) engaged children in a range of extracurricular, educational, social, cultural, and recreational activities; and (5) promoted attitudes to foster a positive future orientation. All of the mentoring activities used by mentors were individually tailored for each child based on the children's presenting problems, strengths, and interests as well as their family and placement characteristics.¹⁵

Children attended a mean (SD) of 25.0 (5.8) of the 30 skills groups (median, 26.5) and a mean (SD) of 26.7 (6.2) of the 30 targeted mentoring visits (median, 28.0). These numbers include data from children who withdrew from the program (n = 5). The 30 skills group sessions included 108 discrete activities.¹⁵ Across 11 groups, a mean (SD) of 103.8 (5.2) of the 108 group activities (median, 105.5) were completed.

Mental health functioning was assessed using the following: (1) child self-report on the posttraumatic stress and dissociation scales of the Trauma Symptom Checklist for Children,¹⁹ a widely used symptom-oriented measure of mental health problems; and (2) a multi-informant index of mental health problems. The mental health index was created based on principal components factor analysis of the children's mean scores on the Trauma Symptom Checklist for Children and the internalizing scales of the Child Behavior Checklist²⁰ and the Teacher Report Form,²⁰ completed by children's caregivers and teachers. The Child Behavior Checklist and Teacher Report Form are well-normed measures of child emotional and behavior problems. The factor score explained 42% of the variance in these measures, and factor loadings ranged from 0.59 to 0.70. Children also completed the Life Satisfaction Survey,²¹ a quality-of-life measure that asks respondents to rate satisfaction in several different domains (eg, school, home, health, friendships). Children's use of mental health services and psychotropic medications was assessed based on the following: (1) caregiver report of services and medications used within the past month; and (2) child report of services and medications used within the past 9 months at T2 and the past 6 months at T3.

Other constructs related to mental health functioning were also examined. These included the following (all child self-report measures): (1) positive and negative coping scales from the Coping Inventory,²² which includes 42 strategies for coping with problems; (2) the social acceptance and global self-worth scales of the Self-Perception Profile for Children,^23- 24 a widely used measure of perceived self-competence; and (3) a social support factor score, created based on principal components factor analysis of scale scores from the People in My Life–Short Form^25- 26 used to assess social support from caregivers, peers, and mentors (each in a separate scale). The social support factor score explained 45% of the variance in these 3 scales; factor loadings ranged from 0.63 to 0.74.

Equivalence between intervention and control groups on baseline characteristics and outcome measures was assessed using χ² tests for categorical variables and 1-way analysis of variance for continuous variables. Attritted and nonattritted youths were compared on all baseline measures. We also used χ² tests to assess whether the rate of attrition varied by treatment condition (the child whose data were excluded from analyses was included in the noninterviewed group in attrition analyses).

Linear regression was used to estimate effect sizes for continuous outcome variables, adjusting for baseline scores on the outcome measures and those variables that differed between conditions at baseline. Effect sizes were estimated with Cohen d, calculated as the difference between the adjusted means for the intervention and control conditions divided by the pooled standard deviation. Poisson regression with robust error variance was used to estimate relative risks for dichotomous outcomes, adjusting for baseline scores on corresponding outcome measures and those covariates that differed at baseline. Effect sizes were estimated as relative risks. All analyses used the intent-to-treat sample. The sample size for each analysis varied slightly due to missing data on outcome variables. All analyses were conducted using SAS version 9.2 statistical software (SAS Institute, Inc, Cary, North Carolina).

Youths in the intervention group (as compared with those in the control group) were more likely to have higher IQ scores (F_1,155 = 4.52; P = .04), to have been physically abused (n = 156; χ²₁ = 3.80; P = .05), and to have mothers with criminal histories (n = 156; χ²₁ = 6.54; P = .01) (Table 1). A trend suggested that youths in the intervention group (as compared with those in the control group) were more frequently exposed to illegal activity (n = 156; χ²₁ = 3.04; P = .08). All 4 of these variables were used as covariates in linear and Poisson regression models.

Those interviewed at follow-up were compared with noninterviewed children on all baseline characteristics and outcome measures. At T2 and T3, those not interviewed had lower IQ scores (at T2: F_1,155 = 9.99; P = .002; at T3: F_1,155 = 16.34; P < .001). Those not interviewed at T3 scored higher on the mental health factor score (F_1,156 = 4.72; P = .03). The χ² analyses suggested that rates of attrition did not differ by treatment condition at either T2 (n = 156; χ²₁ = 1.37; P = .24) or T3 (n = 156; χ²₁ = 3.42; P = .06).

Intervention effects on primary and secondary outcomes at T2 and T3 are summarized in Table 2 and Table 3, respectively. All analyses controlled for the corresponding T1 score and those covariates that differed between groups at baseline.

At T2 there were no group differences on mental health symptoms, but at T3 the youths in the intervention group scored lower on the multi-informant mental health factor (mean difference, −0.51; 95% confidence interval [CI], −0.84 to −0.19). At T3 the youths in the intervention group also reported fewer symptoms of dissociation than did control youths (mean difference, −3.66; 95% CI, −6.58 to −0.74), and there was a trend suggesting that they were less likely to report symptoms of posttraumatic stress (mean difference, −2.79; 95% CI, −5.77 to 0.19). At T2 the groups did not differ on self- or caregiver-reported use of mental health services or psychotropic medication. At T3, however, youths in the intervention group were less likely than those in the control group to report receiving recent mental health therapy (53% vs 71%, respectively; relative risk = 0.75; 95% CI, 0.57 to 0.98). At T2 the youths in the intervention group scored higher on a self-report scale measuring quality of life (mean difference, 0.11; 95% CI, 0.03 to 0.19).

There were no statistically significant differences between groups on any of the scales measuring secondary outcomes at T2 or T3, although a trend suggested that youths in the intervention group were more likely than those in the control group to report receiving social support at T2 (mean difference, 0.25; 95% CI, −0.05 to 0.54).

To our knowledge, this is the first study to test, in a rigorous randomized controlled trial, the impact of a mentoring and skills group preventive intervention on the mental health outcomes of preadolescent maltreated children placed in foster care. The intervention demonstrated a significant effect in reducing mental health symptoms, especially those associated with trauma, anxiety, and depression, in this high-risk population. These findings are strengthened because the study controlled for baseline functioning and because multiple informants reported on children's mental health functioning. In addition, the pattern of results suggested that program participants were less likely to use mental health therapy and psychotropic medication.

Although mental health functioning improved among program participants relative to those in the control group, the effect was not apparent until 6 months after the intervention. Group differences on primary outcomes were not expected at T2 for several reasons. First, we hypothesized that improved functioning on primary outcomes would follow improved functioning on secondary outcomes. It was also hypothesized that short-term mental health functioning among program participants might be adversely affected by participants' need to say goodbye to mentors and program staff on completion of the program, which corresponded with the T2 follow-up. Although study hypotheses about mental health effects and their timing were supported, hypotheses about short-term effects on secondary outcomes were not. The overall pattern of results on short-term effects, however, was in the expected direction, and a trend suggested that program participation was associated with higher perceived social support at T2.

Findings of program efficacy are consistent with a large body of evidence suggesting that skills training curricula are effective in reducing risk and promoting mental health. Skills groups have demonstrated efficacy in multiple contexts and with diverse populations, including maltreated youth.^27- 29 Social skills groups may be particularly useful for children in foster care as they often lack critical social skills, may have recently changed schools and peer groups, and may know no other children in foster care.

On the other hand, our study's findings provide valuable information to inform the evidence base for mentoring, which has much less empirical support despite its ideological promise.^30- 31 Although some studies suggest that mentoring can have a positive effect on youth functioning,^32- 35 there is reason for caution. Experimental studies of mentoring programs, particularly randomized controlled trials, are rare, and some studies fail to produce evidence of efficacy.^36- 39 Two recent large-scale evaluations of programs with a mentoring component failed to demonstrate effectiveness, and one of the studies produced iatrogenic effects.^40- 41 Although there has been little empirical research, there has been enormous public and private investment in mentoring programs. More than $100 million in federal dollars annually since 2004 has been dedicated to mentoring programs nationally.^42- 43 A 2006 social policy report by the Society for Research in Child Development on mentoring research concluded, “There are few other areas where the research-program/policy connection is as badly needed.”⁴⁴

The FHF program is one of the first randomized clinical trials with a high-risk population to demonstrate the efficacy of a mentoring program on mental health outcomes. Although the FHF program uses a fairly traditional community-based mentoring model, the fact that it is paired with skills groups may make it particularly effective. Furthermore, FHF mentoring incorporates those practices that appear to enhance the effectiveness of mentoring. A meta-analysis of mentoring programs found that program effects were significantly enhanced when programs targeted high-risk youth and incorporated several best practices. Programs that used mentors with prior experience in a helping role or profession, those that provided for ongoing training of mentors, and those that provided structured activities for mentors and participating youths had the most beneficial effect on youths identified as being at high risk.⁴⁵

The study's methodological approach also speaks to the generalizability of the study findings. All eligible children in participating counties were recruited, and the high recruitment, retention, and program uptake rates suggest that this intervention was contextually sensitive and well received. Despite the fact that the participants were extremely heterogeneous on sociodemographic factors, maltreatment history, current living situation, and cognitive, academic, emotional, and behavioral functioning, there were important program main effects. The generalizability of the findings is also strengthened by the fact that participants did not self-select into the program (as is the case with most community-based mentoring programs in which participants sign up).

The study also demonstrates that it is possible to conduct a rigorous randomized controlled trial with intent-to-treat analyses in a child welfare population and to obtain information from multiple informants, including teachers. There are many barriers to conducting trials with a foster care population, including changes in legal guardianship, ongoing court processes, multiple system involvement, and the need to report all suspected maltreatment. The ability to conduct this important research speaks to the strength of the collaboration between researchers and participating counties. Despite all the challenges to program completion, all but 5 children who began the 9-month prevention program graduated. In addition, more than 80% of those who either refused the prevention program or dropped out were interviewed at follow-up and included in intent-to-treat analyses. Success in recruitment and retention may be due to the fact that there were small cohorts as we developed and tested the FHF intervention. Such formative work is critical in the development of novel interventions, especially those at risk for iatrogenic effects.⁴⁶ A full-scale efficacy trial is currently under way, which will enable us to test whether the program remains efficacious on a larger scale.

The study was not without limitations. Despite randomization, there were a few key variables on which the 2 groups differed at baseline. Although analyses controlled for these differences, there may have been other, unmeasured factors that affected the baseline equivalence of groups. In addition, those lost to follow-up had lower IQs and more mental health problems than those interviewed, which may limit the generalizability of the findings. Finally, the fact that children are in foster care presents some unique methodological challenges that may have influenced the results. Caregivers of children in foster care are not static; some children had different caregivers at each of the 3 interviews, while other children had the same caregiver. Because caregivers parented these children for variable amounts of time, their knowledge of the children's current functioning and psychosocial histories varied greatly. To minimize the effect of the variability in caregiver familiarity with their children, which was not expected to differ between treatment conditions, the study asked questions of caregivers that focused on current functioning and recent mental health treatment. The addition of teacher reports, in which the informant is expected to vary each year, also mitigates concerns about reporter bias.

Despite study limitations, findings suggest that the FHF mentoring and skills group protocol holds promise and that future work examining program efficacy is warranted. Longer-term follow-up (currently under way) is needed to determine whether effects are sustained and/or whether new effects emerge. Despite the cluster of risks associated with maltreatment, including poverty, high-risk neighborhoods, parental psychopathology, substance use, and domestic violence, this study suggests that the FHF intervention promotes greater life satisfaction and better mental health functioning among maltreated youths placed in foster care. These are important findings given the dearth of evidence-based treatments for this vulnerable population. Although this study needs replication, it may be a promising model, not only for children in foster care but for other high-risk youth populations as well.

Correspondence: Heather N. Taussig, PhD, The Kempe Center, Department of Pediatrics, University of Colorado School of Medicine, The Gary Pavilion at The Children's Hospital, Anschutz Medical Campus, 13123 E 16th Ave, B-390, Aurora, CO 80045 (taussig.heather@tchden.org).

Accepted for Publication: February 3, 2010.

Author Contributions: Dr Taussig had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Taussig and Culhane. Acquisition of data: Taussig and Culhane. Analysis and interpretation of data: Taussig and Culhane. Drafting of the manuscript: Taussig and Culhane. Critical revision of the manuscript for important intellectual content: Taussig and Culhane. Statistical analysis: Culhane. Obtained funding: Taussig and Culhane. Administrative, technical, and material support: Taussig and Culhane. Study supervision: Taussig and Culhane.

Financial Disclosure: None reported.

Funding/Support: This work was principally supported by grants 1 K01 MH01972, 1 R21 MH067618, and 1 R01 MH076919 from the National Institute of Mental Health and also received substantial funding from the Kempe Foundation, Pioneer Fund, Daniels Fund, and Children's Hospital Research Institute.

Role of the Sponsors: The funding agencies were not involved in the design and conduct of the study; collection, analysis, and interpretation of the data; and preparation of the data or preparation, review, or approval of the manuscript.

Disclaimer: The content is solely the responsibility of the authors and does not represent the official views of the National Institute of Mental Health or the National Institutes of Health.

Additional Contributions: We appreciate the children and families who made this work possible and the participating county departments of social services for their ongoing partnership in our joint clinical research efforts. David Olds, PhD, David MacKinnon, PhD, Edward Garrido, PhD, Tali Raviv, PhD, and Melody Combs, PhD, provided insightful comments on drafts of the manuscript; Jennifer Koch-Zapfel, MSW, Dana Morgan, MSW, and Michel Holien, LCSW, recruited participants; Daniel Hettleman, PhD, Ann Petrila, LCSW, Rebecca Gennerman-Schroeder, LPC, and Robyn Wertheimer-Hodas, LCSW, collaborated on the development and implementation of the FHF program; and Dongmei Pan, MS, and Michael Knudtson, MS, conducted data analyses. This project would not have been possible without the work of the research assistants, project interviewers, interns/mentors, group leaders, and skills group assistants.