0
We're unable to sign you in at this time. Please try again in a few minutes.
Retry
We were able to sign you in, but your subscription(s) could not be found. Please try again in a few minutes.
Retry
There may be a problem with your account. Please contact the AMA Service Center to resolve this issue.
Contact the AMA Service Center:
Telephone: 1 (800) 262-2350 or 1 (312) 670-7827  *   Email: subscriptions@jamanetwork.com
Error Message ......
Original Investigation |

Predictors of Community Therapists’ Use of Therapy Techniques in a Large Public Mental Health System FREE

Rinad S. Beidas, PhD1; Steven Marcus, PhD2,3; Gregory A. Aarons, PhD4; Kimberly E. Hoagwood, PhD5; Sonja Schoenwald, PhD6; Arthur C. Evans, PhD1,7; Matthew O. Hurford, MD1,7,8; Trevor Hadley, PhD1; Frances K. Barg, PhD, MEd9; Lucia M. Walsh, BS1; Danielle R. Adams, BA1; David S. Mandell, ScD1
[+] Author Affiliations
1Department of Psychiatry, University of Pennsylvania Perelman School of Medicine, Philadelphia
2School of Social Policy and Practice, University of Pennsylvania, Philadelphia
3Center for Health Equity Research and Promotion, Philadelphia Veterans Affairs Medical Center, Philadelphia, Pennsylvania
4Department of Psychiatry, University of California, San Diego
5Department of Psychiatry, New York University, New York, New York
6Department of Psychiatry and Behavioral Sciences, Medical University of South Carolina, Charleston
7Department of Behavioral Health and Intellectual Disability Services, Philadelphia, Pennsylvania
8Community Behavioral Health, Philadelphia, Pennsylvania
9Department of Family Medicine and Community Health, University of Pennsylvania Perelman School of Medicine, Philadelphia
JAMA Pediatr. 2015;169(4):374-382. doi:10.1001/jamapediatrics.2014.3736.
Text Size: A A A
Published online

Importance  Few studies have examined the effects of individual and organizational characteristics on the use of evidence-based practices in mental health care. Improved understanding of these factors could guide future implementation efforts to ensure effective adoption, implementation, and sustainment of evidence-based practices.

Objective  To estimate the relative contribution of individual and organizational factors on therapist self-reported use of cognitive-behavioral, family, and psychodynamic therapy techniques within the context of a large-scale effort to increase use of evidence-based practices in an urban public mental health system serving youth and families.

Design, Setting, and Participants  In this observational, cross-sectional study of 23 organizations, data were collected from March 1 through July 25, 2013. We used purposive sampling to recruit the 29 largest child-serving agencies, which together serve approximately 80% of youth receiving publically funded mental health care. The final sample included 19 agencies with 23 sites, 130 therapists, 36 supervisors, and 22 executive administrators.

Main Outcomes and Measures  Therapist self-reported use of cognitive-behavioral, family, and psychodynamic therapy techniques, as measured by the Therapist Procedures Checklist–Family Revised.

Results  Individual factors accounted for the following percentages of the overall variation: cognitive-behavioral therapy techniques, 16%; family therapy techniques, 7%; and psychodynamic therapy techniques, 20%. Organizational factors accounted for the following percentages of the overall variation: cognitive-behavioral therapy techniques, 23%; family therapy techniques, 19%; and psychodynamic therapy techniques, 7%. Older therapists and therapists with more open attitudes were more likely to endorse use of cognitive-behavioral therapy techniques, as were those in organizations that had spent fewer years participating in evidence-based practice initiatives, had more resistant cultures, and had more functional climates. Women were more likely to endorse use of family therapy techniques, as were those in organizations employing more fee-for-service staff and with more stressful climates. Therapists with more divergent attitudes and less knowledge about evidence-based practices were more likely to use psychodynamic therapy techniques.

Conclusions and Relevance  This study suggests that individual and organizational factors are important in explaining therapist behavior and use of evidence-based practices, but the relative importance varies by therapeutic technique.

Implementation science frameworks posit that individual (eg, knowledge and attitudes) and organizational (eg, culture and climate) characteristics affect the provision of evidence-based practices (EBPs) in general health and mental health care.1 Little is known about the relative contributions of these 2 sets of characteristics. This study estimates the relative contribution of individual and organizational factors on therapists’ use of cognitive-behavioral therapy (CBT), family therapy, and psychodynamic therapy techniques within the context of a large-scale effort to increase the use of CBT in an urban public mental health system.

Literature supports the role of individual and organizational factors in the provision of children’s mental health services. For example, individual factors, such as attitudes toward EBPs,2,3 predict the extent to which therapists deliver EBPs as designed. Similarly, organizational factors, such as organizational culture (ie, shared employee perceptions around expectations and norms)4 and organizational climate (ie, psychological effect of the work environment on individual well-being),5 have been linked to quality of services6 and youth mental health outcomes.6,7 Previous research has largely focused on individual or organizational factors. Both sets of studies find evidence of the predictive validity of their constructs of interest. Individual and organizational factors are correlated,8 making it difficult to disentangle the contributions of each set. Furthermore, different outcomes have been examined in these 2 sets of studies, making it challenging to compare results.

To address these issues, we measured the association between individual and organizational characteristics and the use of therapy techniques in therapists in an urban public mental health system engaged in a large-scale effort to increase the use of CBT. We explore the relative contribution of individual and organizational characteristics on therapist self-reported use of CBT, family therapy, and psychodynamic therapy techniques, all 3 being widely endorsed techniques used by therapists in usual care.9 Cognitive-behavior therapy has a large body of evidence to support its effectiveness in addressing youth psychiatric disorders.10 Family therapy is effective for some youth psychiatric disorders, particularly when family-oriented components are combined with CBT techniques.11 Psychodynamic therapy has little evidence to support its effectiveness for youth psychiatric disorders.10

Setting

Since 2007, the Philadelphia Department of Behavioral Health and Intellectual disAbility Services has supported the implementation of therapy techniques in the public mental health system based on the principles of CBT. Implementation support includes a full-time city employee who coordinates implementation and training and consultation by treatment developers.

Agencies and Participants

There are more than 100 community mental health agencies in Philadelphia that provide outpatient services to youth (Cathy Bolton, PhD, written communication, January 3, 2013). We used purposive sampling to recruit the 29 largest child-serving agencies, which serve approximately 80% of youth receiving publically funded mental health care. Of these 29 agencies, 18 (62%) agreed to participate. In addition, 1 agency involved in EBPs efforts asked to participate, resulting in a final sample of 19 agencies (23 sites, 130 therapists, 36 supervisors, and 22 executive administrators). Each site rather than each agency was treated as a distinct organization because of different leadership structures, locations, and staff. In this article, we refer to site as organization. The leader of each organization was invited to participate as the executive administrator. There were no exclusion criteria for participation. Of the organizations enrolled in this study, 16 had participated in city-sponsored EBPs initiatives.

Procedure

All procedures were approved by the City of Philadelphia and University of Pennsylvania institutional review boards. We approached the executive administrator of each organization for participation. Executive administrators completed their questionnaires using REDcap, a secure Web-based application that supports online data collection.12 For supervisors and therapists, we scheduled a 1-time, 2-hour meeting at each organization, at which we provided lunch, obtained written informed consent, and completed data collection. Approximately 60% of therapists employed by the 23 organizations participated in the study, which collected data from March 1 through July 25, 2013. Participants received $50.

Measures
Participant Characteristics

Executive administrators provided information on their age, sex, ethnicity/race, and educational background. Therapist and supervisor demographics were assessed using the Therapist Background Questionnaire,13 a 21-item demographics questionnaire. We also asked participants to report on employment status (ie, fee for service or salaried).

Therapist attitudes were assessed using the Evidence-Based Practice Attitude Scale,14 a 15-item self-report questionnaire that assesses constructs related to appeal the of EBPs, requirements to use EBPs, general openness to new practices, and divergence between EBPs and usual practice. Each subscale is a mean of the items within that factor, which is measured on a continuum from 0 (not at all) to 4 (very great extent). The Evidence-Based Practice Attitude Scale has good internal consistency15 and validity.16

Therapist knowledge about EBPs was measured using the Knowledge of Evidence-Based Services Questionnaire,17 a 40-item self-report instrument. Knowledge is measured on a continuum from 0 to 160, with higher scores indicative of more knowledge of evidence-based services for youth. Psychometric data suggest temporal stability, discriminative validity, and sensitivity to training.17

Organizational Characteristics

Supervisors provided information on the number of therapists in their unit and their employment status. From that information, we determined program size (ie, number of therapists in their unit) and percentage of fee-for-service staff. We also assessed the number of years the organization had formally been involved in city-sponsored EBPs initiatives.

Organizational culture and climate were measured from the perspectives of therapists, supervisors, and executive administrators using the Organizational Social Context Measurement System,18 a 105-item measure of the social context of mental health organizations. Organizational culture includes proficiency, rigidity, and resistance, and climate includes engagement, functionality, and stress. Proficient cultures are those in which therapists prioritize the well-being of clients and are expected to keep up competencies. Rigid cultures are those in which therapists have little autonomy and discretion. Resistant cultures refer to ones in which therapists are expected to be apathetic. Engaged climates refer to ones in which therapists think they can accomplish worthwhile tasks and remain invested in their work. Functional climates are ones in which therapists think they are able to get their job done effectively. Stressful climates refer to ones in which therapists feel emotionally exhausted. Organizational culture and climate are measured with t scores with a mean of 50 and an SD of 10 based on a normed sample of 100 community mental health clinics.18 The Organizational Social Context Measurement System has strong psychometric properties.19

Implementation climate was measured from the perspective of therapists, supervisors, and executive administrators, using the Implementation Climate Scale,20 an 18-item scale that measures climate for EBPs implementation, including focus on EBPs, educational support for EBPs, recognition for using EBPs, rewards for using EBPs, selection of staff for EBPs, and general organizational openness. Each subscale is a mean of the items within that factor, which is measured on a continuum from 0 (not at all) to 4 (very great extent). Psychometrics are strong and have excellent reliability and validity.20

Implementation leadership was measured from the perspective of therapists, rating their direct supervisor with the Implementation Leadership Scale,21 a 12-item scale that measures leader behaviors relevant to implementation of EBPs, including proactive, knowledgeable, supportive, and perseverant leadership. Each subscale is a mean of the items within that factor, which is measured on a continuum from 0 (not at all) to 4 (very great extent). The Implementation Leadership Scale has strong psychometric properties and excellent internal consistency and validity.21

Dependent Variables

Therapists’ use of CBT, family therapy, and psychodynamic therapy techniques was measured using the Therapy Procedures Checklist–Family Revised (TPC-FR),22 a 62-item self-report checklist. Therapists were asked to respond in reference to a representative client who they were currently treating. Therapists could endorse using strategies from all 3 families of techniques. Each dependent variable is a mean of the items that fit within that factor, which is measured on a continuum from 1 to 5, with 1 indicating rarely; 2, seldom; 3, sometimes; 4, often; and 5, most of the time. Higher scores are indicative of more use of the set of techniques. Factor structure has been confirmed, test-retest reliability is strong, and the instrument is sensitive to within-therapist changes in technique use.22,23

Data Analytic Plan

Organizational measures are constructed by aggregating individual responses within the organization, if there is enough agreement among individuals. To determine agreement, we used mean within-group correlation statistics.24,25 On all organizational variables, statistics were above the suggested 0.60 level25,26; therefore, participant responses to organizational constructs were averaged within each organization. Missing data for independent and predictor variables were minimal (<10%); series means were imputed for missing predictor variables.

We used 3 sets of linear mixed-effects regression models to determine the associations of individual and organizational factors (independent variables) with self-reported use of CBT, family therapy, and psychodynamic therapy techniques (dependent variables). All linear mixed-effects models included random intercepts for organization to account for nesting of therapists within organizations and fixed effects for individual and organizational factors. Therapist factors included participant demographics (sex, age, clinical experience, and employment status [ie, fee for service or salaried]), attitudes (Evidence-Based Practice Attitude Scale subscales), and knowledge (Knowledge of Evidence-Based Services Questionnaire total score). Organizational factors included organization demographics (cumulative years organization participated in city-sponsored EBPs initiatives, program size [ie, number of therapists in the unit], and organization type [percentage of fee-for-service staff]), implementation climate (Implementation Climate Scale subscales), implementation leadership (Implementation Climate Scale subscales), and organizational culture and organizational climate (Organizational Social Context Measurement System subscales). Dependent variables included use of CBT, family therapy, and psychodynamic therapy techniques. Therapist case mix and therapist race/ethnicity were initially included in the model as covariates but were removed because associated coefficients were not statistically significant.

Statistical Analysis

Analyses were conducted using PROC MIXED in SAS statistical software, version 9.0 (SAS Institute Inc). Four separate models were conducted for each of the dependent variables. In the first unconditional model, only the organization random effect was included to provide an estimate of the organizational and residual variance. This model allows us to estimate how much of the total variance could possibly be attributed to the organization and to calculate the intraclass correlation coefficient. In the second model, only individual fixed effects were included to estimate the total variance attributable to individual fixed effects. This model allowed us to calculate the proportion of the total variation in the model explained by individual factors. In the third model, only organizational fixed effects were included to estimate the total variation attributable to organizational fixed effects. In the fourth model, both individual and organizational fixed effects were included; these models are reported on in the Results section. Our goal was to understand how much of the overall variation in our dependent variable(s) was explained by the set of organizational and individual factors, respectively (ie, separately). The proportion of variation explained by individual factors was calculated by subtracting the total variation from the individual factors model (model 2) from the variance of the unconditional model (model 1) and dividing by the total variation from model 1 (ie, %Vartherapist = [Varunconditional − Vartherapist]/Varunconditional); the same technique was used separately to calculate the proportion of variation for organizational factors (ie, %Varorganization = [Varunconditional – Varorganization]/Varunconditional).27 Our analyses of the contributions of individual and organizational factors focus on the unique effect of each factor after controlling for all other factors in the model.

Participants

Table 1 provides demographic information about the therapists. Of the 22 executive administrators, 11 (50%) were male; they identified as Asian (2 [9%]), Hispanic/Latino (3 [14%]), African American (4 [18%]), white (12 [55%]), multiracial (2 [9%]), or missing ethnicity/race (2 [9%]). Of the 36 supervisors, 25 (69%) were female; they identified as African American (6 [17%]), white (20 [56%]), Hispanic/Latino (5 [14%]), other (1 [3%]), or missing ethnicity/race (4 [11%]).

Table Graphic Jump LocationTable 1.  Descriptive Statistics for the Study Variables

Table 1 gives the descriptive statistics for all variables included in the models. The eTable in the Supplement is a correlation matrix that documents correlations between predictors and outcomes.

Therapists’ Use of CBT Techniques

Table 2 gives the model parameters. Organizational factors accounted for 23% of the variance in therapists’ use of CBT techniques; therapist factors accounted for 16%. Three organizational variables were associated with use of CBT techniques. Therapists in organizations that had participated for fewer years in city-sponsored EBPs initiatives, had more resistant cultures, and had more functional climates were more likely to use CBT techniques. Two therapist variables were associated with the use of CBT techniques. Older therapists and therapists with more open attitudes toward new practices were more likely to use CBT techniques.

Table Graphic Jump LocationTable 2.  Variation in and Factors Associated With Use of CBT, Psychodynamic Therapy, and Family Therapy Techniques
Therapists’ Use of Family Therapy Techniques

Organizational variables accounted for 19% of the variance in the use of family therapy techniques; therapist variables accounted for 7%. Two organizational variables were associated with use of family therapy techniques. Organizations using more fee-for-service staff and organizations with more stressful climates were more likely to use family therapy techniques. One individual variable was associated with use of family therapy techniques: women were more likely to use family therapy techniques.

Therapists’ Use of Psychodynamic Therapy Techniques

Individual factors accounted for 20% of the variance in the use of psychodynamic therapy techniques; organizational factors accounted for 7%. Two individual factors were associated with the use of psychodynamic therapy techniques. Therapists with more divergent attitudes on the perceived difference between EBPs and current practices and less knowledge about EBPs were more likely to use psychodynamic therapy techniques.

Variance Attributable to Individual and Organizational Factors Collectively

For use of CBT techniques, individual and organizational factors collectively explained 30% of the overall variation; for use of psychodynamic therapy strategies, individual and organizational factors collectively explained 18% of the overall variation; for use of family therapy strategies, individual and organizational factors collectively explained 26% of the overall variation (data not shown).

This study provides information on what predicts therapists’ use of therapy techniques in a large public mental health system supporting implementation of CBT. Organizational factors accounted for more of the variance in therapists’ use of CBT and family therapy techniques. Conversely, individual factors accounted for more of the variance in therapists’ use of psychodynamic therapy techniques. Both CBT and family therapy are evidence-based techniques for childhood disorders, whereas psychodynamic therapy techniques have less rigorous evidence to support them.10 Taken collectively, these findings suggest that organizational factors are more likely to drive use of EBPs, whereas therapist attributes are more likely to drive use of non-EBPs therapy techniques.

Consistent with the literature,16 therapists with more open attitudes were more likely to use CBT techniques. However, inconsistent with the literature,14 older therapists were more likely to use CBT techniques. Older therapists may have more experience changing treatment modalities according to demand because they have been in the system longer. Paradoxically, organizations that had spent fewer years participating in city-sponsored EBPs initiatives were more likely to use CBT techniques. It is possible that organizations that seek out CBT training have therapists less experienced in CBT. Alternatively, it is possible that organizations spending more years participating in EBPs initiatives experience EBPs fatigue (ie, innovation fatigue28) or the stress involved with competing demands of difficult human service jobs coupled with lack of clarity of how these initiatives fit with the role of therapist. Further research is needed to understand potential unintended consequences of EBPs efforts, such as innovation fatigue.

Organizations with more resistant cultures and more functional climates were more likely to have therapists who endorsed using CBT techniques. Organizations with more resistant cultures may be more likely to participate in initiatives to increase use of innovation. Therapists’ perceptions that they are functioning effectively appear to contribute to more use of CBT techniques. Organizations with more stressful climates and more fee-for-service staff were more likely to have therapists who used family techniques. It is possible that organizations with more stressful climates serve a more chaotic population, suggesting the potential usefulness of family therapy techniques, which are indicated for youth with psychiatric disorders and chaotic family environments.29

Consistent with the literature, therapists who were less knowledgeable about EBPs and held more divergent attitudes toward EBPs were more likely to use psychodynamic therapy techniques.30 Also consistent with previous findings,9 therapists reported using CBT and other therapy techniques (ie, eclecticism) at the same time, suggesting a potential exnovation problem. Exnovation refers to “the process whereby an organization decides to divest itself of an innovation that it had previously adopted.”31 Further research is needed to understand how organizations can plan for EBPs that fits with existing practices.

Of interest, implementation climate and leadership, constructs hypothesized to be related to implementation outcomes,8,32 did not predict use of EBPs. However, the current study did not examine more complex interactive or mediational processes to account for how these constructs may work together with molar culture or climate to predict the outcome used in this study.33

Some study limitations should be noted. First, the primary outcome variables are based on therapists self-report of use of therapeutic techniques, and therapists are not always accurate reporters of use of therapeutic techniques.34,35 Second, we did not have 100% participation, creating a potential selection bias at the therapist and organizational levels. Third, we used a random intercepts–only model and did not allow the slopes to vary by organization because of sample size limitations.

These findings offer important implications. Organizational variables accounted for more of the variance than individual variables in predicting use of EBPs, suggesting that organizational-level implementation strategies36 will be more effective in increasing the use of EBPs than implementation strategies that directly target the therapist. Individual factors account for more of the variance than organizational factors in therapists’ use of psychodynamic therapy techniques. Efforts to implement EBPs may need to be accompanied by efforts to encourage therapists to divest themselves of outdated practices. Although these 2 activities may be seen as 2 sides of the same coin, the results of this study suggest that the process of exnovation may be driven by different factors than the process of implementation. Of perhaps equal importance, the variables included in our measurement model represent many of the constructs posited to predict implementation, and yet separately they accounted only for a maximum of 23% of the overall variation in outcomes, suggesting that there are a number of unmeasured constructs. Finally, this study highlights the need for prospective studies to test the relative contributions and interactions of individual- and organization-focused implementation strategies on adoption, fidelity, and sustainment of EBPs.

These findings suggest the nuanced effect of individual and organizational factors on therapists’ use of therapy techniques. This study suggests that both where you work and who you are matter in understanding therapists’ behavior in context and that improving the effectiveness of implementation strategies should consider both approaches.37

Accepted for Publication: December 21, 2014.

Corresponding Author: Rinad S. Beidas, PhD, Department of Psychiatry, University of Pennsylvania Perelman School of Medicine, 3535 Market St, 3015, Philadelphia, PA 19104 (rbeidas@upenn.edu).

Published Online: February 16, 2015. doi:10.1001/jamapediatrics.2014.3736.

Author Contributions: Dr Beidas had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Beidas, Marcus, Aarons, Evans, Hadley, Barg, Mandell.

Acquisition, analysis, or interpretation of data: Beidas, Aarons, Hoagwood, Schoenwald, Hurford, Walsh, Adams.

Drafting of the manuscript: Beidas, Marcus, Aarons, Walsh.

Critical revision of the manuscript for important intellectual content: Aarons, Hoagwood, Schoenwald, Evans, Hurford, Hadley, Barg, Adams, Mandell.

Statistical analysis: Beidas, Marcus, Mandell.

Obtained funding: Beidas.

Administrative, technical, or material support: Aarons, Evans, Hurford, Hadley, Barg, Walsh, Adams.

Study supervision: Beidas, Aarons, Hurford, Barg, Mandell.

Conflict of Interest Disclosures: Dr Beidas reported receiving royalties from Oxford University Press and serving as a consultant for Kinark Child and Family Services. Dr Marcus reported receiving grant support from Ortho-McNeil-Janssen Pharmaceuticals Inc and Forest Research Institute and serving as a consultant to AstraZeneca and Alkermes. Dr Schoenwald reported being a stakeholder and board member of MST Services, LLC. No other disclosures were reported.

Funding/Support: This study was supported by grant K23 MH099179 and an Implementation Research Institute fellowship (2012-2014) from the National Institute of Mental Health (Dr Beidas); the Implementation Research Institute, George Warren Brown School of Social Work, Washington University, St Louis, Missouri; grant R25 MH080916 from the National Institute of Mental Health; and the Quality Enhancement Research Initiative, Department of Veterans Affairs Contract, Veterans Health Administration, Office of Research and Development, Health Services Research and Development Service.

Role of the Funder/Sponsor: The funding sources had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and the decision to submit the manuscript for publication.

Additional Contributions: The following experts provided their time and input on this project: Marc Atkins, PhD (University of Illinois at Chicago), Ross Brownson, PhD (Washington University in St Louis), David Chambers, DPhil (National Cancer Institute), Bruce Chorpita, PhD (UCLA), Charles Glisson, PhD (University of Tennessee), Nicholas Ialongo, PhD (Johns Hopkins University), John Landsverk, PhD (Oregon Social Learning Center), Enola Proctor, PhD (Washington University in St Louis), and Ronnie Rubin, PhD (Community Behavioral Health). The Department of Behavioral Health and Intellectual DisAbility Services and the Evidence Based Practice and Innovation group also supported this project. No compensation was provided to the contributors.

Aarons  GA, Hurlburt  M, Horwitz  SM.  Advancing a conceptual model of evidence-based practice implementation in public service sectors. Adm Policy Ment Health. 2011;38(1):4-23.
PubMed   |  Link to Article
Henggeler  SW, Chapman  JE, Rowland  MD,  et al.  Statewide adoption and initial implementation of contingency management for substance-abusing adolescents. J Consult Clin Psychol. 2008;76(4):556-567.
PubMed   |  Link to Article
Beidas  RS, Edmunds  J, Ditty  M,  et al.  Are inner context factors related to implementation outcomes in cognitive-behavioral therapy for youth anxiety? Adm Policy Ment Health. 2014;41(6):788-799.
PubMed   |  Link to Article
Glisson  C, Dukes  D, Green  P.  The effects of the ARC organizational intervention on caseworker turnover, climate, and culture in children’s service systems. Child Abuse Negl. 2006;30(8):855-880.
PubMed   |  Link to Article
Peterson  AE, Bond  GR, Drake  RE, McHugo  GJ, Jones  AM, Williams  JR.  Predicting the long-term sustainability of evidence-based practices in mental health care: an 8-year longitudinal analysis. J Behav Health Serv Res. 2014;41(3):337-346.
PubMed   |  Link to Article
Glisson  C, Hemmelgarn  A.  The effects of organizational climate and interorganizational coordination on the quality and outcomes of children’s service systems. Child Abuse Negl. 1998;22(5):401-421.
PubMed   |  Link to Article
Glisson  C, Green  P.  Organizational climate, services, and outcomes in child welfare systems. Child Abuse Negl. 2011;35(8):582-591.
PubMed   |  Link to Article
Aarons  GA, Glisson  C, Green  PD,  et al; Research Network on Youth Mental Health.  The organizational social context of mental health services and clinician attitudes toward evidence-based practice: a United States national study. Implement Sci. 2012;7:56.
PubMed   |  Link to Article
Garland  AF, Brookman-Frazee  L, Hurlburt  MS,  et al.  Mental health care for children with disruptive behavior problems: a view inside therapists’ offices. Psychiatr Serv. 2010;61(8):788-795.
PubMed   |  Link to Article
Chambless  DL, Ollendick  TH.  Empirically supported psychological interventions: controversies and evidence. Annu Rev Psychol. 2001;52:685-716.
PubMed   |  Link to Article
Mattejat  F.  Evidence-based family therapy: which family-based interventions are empirically supported? Kindh Entwickl. 2005;14(1):3-11.
Link to Article
Harris  PA, Taylor  R, Thielke  R, Payne  J, Gonzalez  N, Conde  JG.  Research electronic data capture (REDCap): a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42(2):377-381.
PubMed   |  Link to Article
Weisz  JR. Therapist Background Questionnaire. Los Angeles: University of California; 1997.
Aarons  GA.  Mental health provider attitudes toward adoption of evidence-based practice: the Evidence-Based Practice Attitude Scale (EBPAS). Ment Health Serv Res. 2004;6(2):61-74.
PubMed   |  Link to Article
Aarons  GA, Sawitzky  AC.  Organizational culture and climate and mental health provider attitudes towards evidence-based practice. Psychol Serv. 2006;3(1):61-72.
PubMed   |  Link to Article
Aarons  GA.  Measuring provider attitudes toward evidence-based practice: consideration of organizational context and individual differences. Child Adolesc Psychiatr Clin N Am. 2005;14(2):255-271, viii.
PubMed   |  Link to Article
Stumpf  RE, Higa-McMillan  CK, Chorpita  BF.  Implementation of evidence-based services for youth: assessing provider knowledge. Behav Modif. 2009;33(1):48-65.
PubMed   |  Link to Article
Glisson  C, Landsverk  J, Schoenwald  S,  et al; Research Network on Youth Mental Health.  Assessing the organizational social context (OSC) of mental health services: implications for research and practice. Adm Policy Ment Health. 2008;35(1-2):98-113.
PubMed   |  Link to Article
Glisson  C, Green  P, Williams  NJ.  Assessing the Organizational Social Context (OSC) of child welfare systems: implications for research and practice. Child Abuse Negl. 2012;36(9):621-632.
PubMed   |  Link to Article
Aarons  GA. Organizational Climate for Evidence-Based Practice Implementation: Development of a New Scale. Toronto, Ontario, Canada: Association for Behavioral and Cognitive Therapies; 2011.
Aarons  GA, Ehrhart  MG, Farahnak  LR.  The Implementation Leadership Scale (ILS): development of a brief measure of unit level implementation leadership. Implement Sci. 2014;9(1):45.
PubMed   |  Link to Article
Weersing  VR, Weisz  JR, Donenberg  GR.  Development of the Therapy Procedures Checklist: a therapist-report measure of technique use in child and adolescent treatment. J Clin Child Adolesc Psychol. 2002;31(2):168-180.
PubMed   |  Link to Article
Kolko  DJ, Cohen  JA, Mannarino  AP, Baumann  BL, Knudsen  K.  Community treatment of child sexual abuse: a survey of practitioners in the National Child Traumatic Stress Network. Adm Policy Ment Health. 2009;36(1):37-49.
PubMed   |  Link to Article
James  LR, Demaree  RG, Wolf  G.  Estimating within-group interrater reliability with and without response bias. J Appl Psychol. 1984;69(1):85-98.
Link to Article
Brown  RD, Hauenstein  NMA.  rwg: an assessment of within-group interrater agreement. Organ Res Methods. 1993;78(2):306-309.
Bliese  P. Within-group agreement, non-independence, and reliability: implications for data aggregation and analysis. In: Klein  K, Kozlowski  S, eds. Multilevel Theory, Research, and Methods in Organizations. San Francisco, CA: Jossey-Bass; 2000:349-380.
Singer  JD.  Using SAS PROC MIXED to fit multilevel models, hierarchical models, and individual growth models. J Educ Behav Stat. 1998;24(4):323-355.
Link to Article
Lindsay  J, Perkins  C, Karanjikar  M. Conquering Innovation Fatigue: Overcoming the Barriers to Personal and Corporate Success. Hoboken, NJ: Wiley; 2009.
Sexton  T, Turner  CW.  The effectiveness of functional family therapy for youth with behavioral problems in a community practice setting. J Fam Psychol. 2010;24(3):339-348.
PubMed   |  Link to Article
Nakamura  BJ, Higa-McMillan  CK, Okamura  KH, Shimabukuro  S.  Knowledge of and attitudes towards evidence-based practices in community child mental health practitioners. Adm Policy Ment Health. 2011;38(4):287-300.
PubMed   |  Link to Article
Kimberly  JR, Evanisko  MJ.  Organizational innovation: the influence of individual, organizational, and contextual factors on hospital adoption of technological and administrative innovations. Acad Manage J. 1981;24(4):689-713.
PubMed   |  Link to Article
Aarons  GA, Sommerfeld  DH.  Leadership, innovation climate, and attitudes toward evidence-based practice during a statewide implementation. J Am Acad Child Adolesc Psychiatry. 2012;51(4):423-431.
PubMed   |  Link to Article
Ehrhart  MG, Schneider  B, Macey  WH. Organizational Climate and Culture: An Introduction to Theory, Research, and Practice. New York, NY: Routledge; 2014.
Bellg  AJ, Borrelli  B, Resnick  B,  et al; Treatment Fidelity Workgroup of the NIH Behavior Change Consortium.  Enhancing treatment fidelity in health behavior change studies: best practices and recommendations from the NIH Behavior Change Consortium. Health Psychol. 2004;23(5):443-451.
PubMed   |  Link to Article
Borrelli  B, Sepinwall  D, Ernst  D,  et al.  A new tool to assess treatment fidelity and evaluation of treatment fidelity across 10 years of health behavior research. J Consult Clin Psychol. 2005;73(5):852-860.
PubMed   |  Link to Article
Glisson  C, Schoenwald  SK, Hemmelgarn  A,  et al.  Randomized trial of MST and ARC in a two-level evidence-based treatment implementation strategy. J Consult Clin Psychol. 2010;78(4):537-550.
PubMed   |  Link to Article
Proctor  E, Silmere  H, Raghavan  R,  et al.  Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. 2011;38(2):65-76.
PubMed   |  Link to Article

Figures

Tables

Table Graphic Jump LocationTable 1.  Descriptive Statistics for the Study Variables
Table Graphic Jump LocationTable 2.  Variation in and Factors Associated With Use of CBT, Psychodynamic Therapy, and Family Therapy Techniques

References

Aarons  GA, Hurlburt  M, Horwitz  SM.  Advancing a conceptual model of evidence-based practice implementation in public service sectors. Adm Policy Ment Health. 2011;38(1):4-23.
PubMed   |  Link to Article
Henggeler  SW, Chapman  JE, Rowland  MD,  et al.  Statewide adoption and initial implementation of contingency management for substance-abusing adolescents. J Consult Clin Psychol. 2008;76(4):556-567.
PubMed   |  Link to Article
Beidas  RS, Edmunds  J, Ditty  M,  et al.  Are inner context factors related to implementation outcomes in cognitive-behavioral therapy for youth anxiety? Adm Policy Ment Health. 2014;41(6):788-799.
PubMed   |  Link to Article
Glisson  C, Dukes  D, Green  P.  The effects of the ARC organizational intervention on caseworker turnover, climate, and culture in children’s service systems. Child Abuse Negl. 2006;30(8):855-880.
PubMed   |  Link to Article
Peterson  AE, Bond  GR, Drake  RE, McHugo  GJ, Jones  AM, Williams  JR.  Predicting the long-term sustainability of evidence-based practices in mental health care: an 8-year longitudinal analysis. J Behav Health Serv Res. 2014;41(3):337-346.
PubMed   |  Link to Article
Glisson  C, Hemmelgarn  A.  The effects of organizational climate and interorganizational coordination on the quality and outcomes of children’s service systems. Child Abuse Negl. 1998;22(5):401-421.
PubMed   |  Link to Article
Glisson  C, Green  P.  Organizational climate, services, and outcomes in child welfare systems. Child Abuse Negl. 2011;35(8):582-591.
PubMed   |  Link to Article
Aarons  GA, Glisson  C, Green  PD,  et al; Research Network on Youth Mental Health.  The organizational social context of mental health services and clinician attitudes toward evidence-based practice: a United States national study. Implement Sci. 2012;7:56.
PubMed   |  Link to Article
Garland  AF, Brookman-Frazee  L, Hurlburt  MS,  et al.  Mental health care for children with disruptive behavior problems: a view inside therapists’ offices. Psychiatr Serv. 2010;61(8):788-795.
PubMed   |  Link to Article
Chambless  DL, Ollendick  TH.  Empirically supported psychological interventions: controversies and evidence. Annu Rev Psychol. 2001;52:685-716.
PubMed   |  Link to Article
Mattejat  F.  Evidence-based family therapy: which family-based interventions are empirically supported? Kindh Entwickl. 2005;14(1):3-11.
Link to Article
Harris  PA, Taylor  R, Thielke  R, Payne  J, Gonzalez  N, Conde  JG.  Research electronic data capture (REDCap): a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42(2):377-381.
PubMed   |  Link to Article
Weisz  JR. Therapist Background Questionnaire. Los Angeles: University of California; 1997.
Aarons  GA.  Mental health provider attitudes toward adoption of evidence-based practice: the Evidence-Based Practice Attitude Scale (EBPAS). Ment Health Serv Res. 2004;6(2):61-74.
PubMed   |  Link to Article
Aarons  GA, Sawitzky  AC.  Organizational culture and climate and mental health provider attitudes towards evidence-based practice. Psychol Serv. 2006;3(1):61-72.
PubMed   |  Link to Article
Aarons  GA.  Measuring provider attitudes toward evidence-based practice: consideration of organizational context and individual differences. Child Adolesc Psychiatr Clin N Am. 2005;14(2):255-271, viii.
PubMed   |  Link to Article
Stumpf  RE, Higa-McMillan  CK, Chorpita  BF.  Implementation of evidence-based services for youth: assessing provider knowledge. Behav Modif. 2009;33(1):48-65.
PubMed   |  Link to Article
Glisson  C, Landsverk  J, Schoenwald  S,  et al; Research Network on Youth Mental Health.  Assessing the organizational social context (OSC) of mental health services: implications for research and practice. Adm Policy Ment Health. 2008;35(1-2):98-113.
PubMed   |  Link to Article
Glisson  C, Green  P, Williams  NJ.  Assessing the Organizational Social Context (OSC) of child welfare systems: implications for research and practice. Child Abuse Negl. 2012;36(9):621-632.
PubMed   |  Link to Article
Aarons  GA. Organizational Climate for Evidence-Based Practice Implementation: Development of a New Scale. Toronto, Ontario, Canada: Association for Behavioral and Cognitive Therapies; 2011.
Aarons  GA, Ehrhart  MG, Farahnak  LR.  The Implementation Leadership Scale (ILS): development of a brief measure of unit level implementation leadership. Implement Sci. 2014;9(1):45.
PubMed   |  Link to Article
Weersing  VR, Weisz  JR, Donenberg  GR.  Development of the Therapy Procedures Checklist: a therapist-report measure of technique use in child and adolescent treatment. J Clin Child Adolesc Psychol. 2002;31(2):168-180.
PubMed   |  Link to Article
Kolko  DJ, Cohen  JA, Mannarino  AP, Baumann  BL, Knudsen  K.  Community treatment of child sexual abuse: a survey of practitioners in the National Child Traumatic Stress Network. Adm Policy Ment Health. 2009;36(1):37-49.
PubMed   |  Link to Article
James  LR, Demaree  RG, Wolf  G.  Estimating within-group interrater reliability with and without response bias. J Appl Psychol. 1984;69(1):85-98.
Link to Article
Brown  RD, Hauenstein  NMA.  rwg: an assessment of within-group interrater agreement. Organ Res Methods. 1993;78(2):306-309.
Bliese  P. Within-group agreement, non-independence, and reliability: implications for data aggregation and analysis. In: Klein  K, Kozlowski  S, eds. Multilevel Theory, Research, and Methods in Organizations. San Francisco, CA: Jossey-Bass; 2000:349-380.
Singer  JD.  Using SAS PROC MIXED to fit multilevel models, hierarchical models, and individual growth models. J Educ Behav Stat. 1998;24(4):323-355.
Link to Article
Lindsay  J, Perkins  C, Karanjikar  M. Conquering Innovation Fatigue: Overcoming the Barriers to Personal and Corporate Success. Hoboken, NJ: Wiley; 2009.
Sexton  T, Turner  CW.  The effectiveness of functional family therapy for youth with behavioral problems in a community practice setting. J Fam Psychol. 2010;24(3):339-348.
PubMed   |  Link to Article
Nakamura  BJ, Higa-McMillan  CK, Okamura  KH, Shimabukuro  S.  Knowledge of and attitudes towards evidence-based practices in community child mental health practitioners. Adm Policy Ment Health. 2011;38(4):287-300.
PubMed   |  Link to Article
Kimberly  JR, Evanisko  MJ.  Organizational innovation: the influence of individual, organizational, and contextual factors on hospital adoption of technological and administrative innovations. Acad Manage J. 1981;24(4):689-713.
PubMed   |  Link to Article
Aarons  GA, Sommerfeld  DH.  Leadership, innovation climate, and attitudes toward evidence-based practice during a statewide implementation. J Am Acad Child Adolesc Psychiatry. 2012;51(4):423-431.
PubMed   |  Link to Article
Ehrhart  MG, Schneider  B, Macey  WH. Organizational Climate and Culture: An Introduction to Theory, Research, and Practice. New York, NY: Routledge; 2014.
Bellg  AJ, Borrelli  B, Resnick  B,  et al; Treatment Fidelity Workgroup of the NIH Behavior Change Consortium.  Enhancing treatment fidelity in health behavior change studies: best practices and recommendations from the NIH Behavior Change Consortium. Health Psychol. 2004;23(5):443-451.
PubMed   |  Link to Article
Borrelli  B, Sepinwall  D, Ernst  D,  et al.  A new tool to assess treatment fidelity and evaluation of treatment fidelity across 10 years of health behavior research. J Consult Clin Psychol. 2005;73(5):852-860.
PubMed   |  Link to Article
Glisson  C, Schoenwald  SK, Hemmelgarn  A,  et al.  Randomized trial of MST and ARC in a two-level evidence-based treatment implementation strategy. J Consult Clin Psychol. 2010;78(4):537-550.
PubMed   |  Link to Article
Proctor  E, Silmere  H, Raghavan  R,  et al.  Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. 2011;38(2):65-76.
PubMed   |  Link to Article

Correspondence

CME
Also Meets CME requirements for:
Browse CME for all U.S. States
Accreditation Information
The American Medical Association is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians. The AMA designates this journal-based CME activity for a maximum of 1 AMA PRA Category 1 CreditTM per course. Physicians should claim only the credit commensurate with the extent of their participation in the activity. Physicians who complete the CME course and score at least 80% correct on the quiz are eligible for AMA PRA Category 1 CreditTM.
Note: You must get at least of the answers correct to pass this quiz.
Please click the checkbox indicating that you have read the full article in order to submit your answers.
Your answers have been saved for later.
You have not filled in all the answers to complete this quiz
The following questions were not answered:
Sorry, you have unsuccessfully completed this CME quiz with a score of
The following questions were not answered correctly:
Commitment to Change (optional):
Indicate what change(s) you will implement in your practice, if any, based on this CME course.
Your quiz results:
The filled radio buttons indicate your responses. The preferred responses are highlighted
For CME Course: A Proposed Model for Initial Assessment and Management of Acute Heart Failure Syndromes
Indicate what changes(s) you will implement in your practice, if any, based on this CME course.

Multimedia

Supplement.

eTable. Zero-Order Correlations Between Predictors and Outcome Variables

Supplemental Content

Some tools below are only available to our subscribers or users with an online account.

1,248 Views
6 Citations
×

Related Content

Customize your page view by dragging & repositioning the boxes below.

Articles Related By Topic
Related Collections
Jobs