Validity of the CRAFFT Substance Abuse Screening Test Among Adolescent Clinic Patients FREE

SUBSTANCE ABUSE is the number-one health problem in the United States, with an estimated annual cost of over $414 billion.¹ It is linked to more than 400 000 preventable deaths each year, and the treatment of associated medical problems places a huge burden on the US health care system. Substance abuse affects men and women of all races, ethnic groups, and ages—including adolescents. Recent studies show that half of high school students are current drinkers, one third binge drink, and one fourth smoke marijuana.² By their senior year in high school, more than one half of students have used an illicit drug at least once, and more than one fourth have used an illicit drug other than marijuana.³

Substance abuse has been linked to both mental and physical health problems, making settings where adolescents receive medical care ideal places for screening and early intervention.^4- 6 In recognition of this opportunity, the American Medical Association's Guidelines for Adolescent Preventive Services recommend that health care providers ask all adolescent patients annually about their use of alcohol and other drugs as part of routine care and further assess those who report any use.⁷ However, adherence to this recommendation is low; less than one half of physicians report screening all adolescent patients for substance use, and less than one fourth report screening for drinking and driving.^8- 9

The precise reasons that so many physicians fail to screen are unknown. However, barriers to screening for other preventable health risks include a belief that the prevalence of the problem is low in the physician's own patient population, inadequate training, lack of time or personnel to perform the screening, and perceived lack of effective treatments.^10- 13 Physicians may also lack familiarity with simple screening methods that can be easily incorporated into their office routines.

The ideal instrument for screening adolescents must be developmentally appropriate, valid and reliable, and practical for use in busy medical offices. A number of screening devices are available for this purpose, including brief questionnaires and orally administered tests.^14- 15 Questionnaires are usually administered to patients in the waiting room. To be practical, they must be designed to be completed by patients within the usual waiting time, and scoring procedures must be sufficiently streamlined so that results can be given to the physician before the medical visit begins. Questionnaires may be targeted at substance use alone or include this as just one part of a more comprehensive adolescent screening. Questionnaires have certain limitations. They may require staff time for administration or scoring. They may also pose a risk to adolescents' confidentiality, especially when parents are present in the waiting area.

Orally administered brief screens are usually targeted at substance abuse alone and can be administered by the physician as part of the general health interview or while performing the physical examination. To be practical, they must be easy to administer, score, and remember. Simple yes or no questions that lend themselves to mnemonic acronyms are ideal. The CAGE questions, which are widely used in medical settings, are a good example of this type of brief screen.¹⁶ The CAGE test has been shown to have good validity among adult medical patients.¹⁷ However, studies among adolescents have not provided adequate evidence of the CAGE test's sensitivity or reliability.^18- 19 In addition, some of its items (eg, "Have you ever had a drink first thing in the morning to steady your nerves or get rid of a hangover [eye-opener]?") are not developmentally appropriate for adolescents.

One brief screening device, the CRAFFT test, was developed specifically for use among adolescent medical patients.²⁰ Like CAGE,¹⁶ CRAFFT is verbally administered, simple to score (each yes answer = 1 point), and easy to remember. Its name is a mnemonic of the first letters of key words in the test's 6 questions. (Figure 1)

In contrast to the CAGE test, however, the CRAFFT test screens for other drugs as well as for alcohol, and its questions were designed to be developmentally appropriate for teenagers. A pilot study among adolescent patients who had used alcohol and other drugs found that CRAFFT had promising concurrent validity compared with a more lengthy scale.²⁰ The purpose of the current study was to determine the criterion validity of the CRAFFT test among a larger, more general population of adolescent medical patients, including those who had used alcohol and other drugs and those who had not.

This criterion standard study compared the CRAFFT score with diagnostic classifications and screening categories determined by a concurrently administered substance use/abuse problem scale and a structured psychiatric diagnostic interview.

The 538 study participants were 14- to 18-year-old patients coming for routine medical care to the Adolescent/Young Adult Medical Practice at Children's Hospital Boston, Boston, Mass, between March 15, 1999, and September 14, 2000. This practice serves both inner-city and suburban youth from a wide range of social strata, racial groups, and ethnic backgrounds. During the study recruitment period, the practice provided care to 4995 patients aged 10 to 24 years through both routine well-care and urgent-care visits; 2986 (60%) of these patients were aged 14 to 18 years.

A research assistant reviewed the birth dates of all scheduled patients before a clinic session and placed a recruitment reminder form on the cover of the chart of each age-eligible patient. At the conclusion of the medical visit, the primary care provider (ie, physician or nurse practitioner) invited eligible patients to participate in the study. The provider completed the recruitment form, which included demographic information, the provider's impression of the patient's level of alcohol and other drug use, and the patient's response to the invitation to participate. We informed providers at the beginning of the study and periodically reminded them that their patient need not ever have used alcohol or other drugs to participate.

We excluded patients who were unable to read and understand English and those who were deemed by the provider to have acute medical or psychiatric problems that precluded participation in research. A research assistant explained the study procedures to interested patients and obtained signed assent. The Children's Hospital Boston Committee on Clinical Investigation (institutional review board) waived the requirement for parental consent in accordance with current guidelines for adolescent health research.^21- 22

The research assistant told participants that the purpose of the study was to assess the value of screening questions on use of alcohol and other drugs and that we would keep their answers confidential. However, if we identified a serious problem, we would notify their primary care provider so that he or she could arrange appropriate care, which could include involving their parents. After completing the assessment battery, each participant received a $25 merchandise certificate as compensation for his or her time.

The assessment battery included the 6-item CRAFFT test and 2 criterion standards. The first criterion standard was the 17-item Substance Use/Abuse Scale from the Problem Oriented Screening Instrument for Teenagers (POSIT),²³ which assesses substance-related problems and risks. Developed by the National Institute on Drug Abuse (Bethesda, Md), the POSIT was previously shown to be reliable among adolescent medical patients and a Substance Use/Abuse Scale score of 2 or higher indicates increased risk.^24- 25 The second criterion standard was the Adolescent Diagnostic Interview (ADI),²⁶ a 30- to 90-minute structured diagnostic interview, which yields alcohol- and drug-related diagnoses (ie, abuse and dependence), according to the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV).²⁷ The ADI has been well validated among adolescents, and it can be administered by an appropriately trained research assistant.^28- 29 We used a structured ADI training protocol for this study. All research assistants read the ADI manual, watched model interviews, practiced on volunteers, and were videotaped conducting practice interviews. Study investigators and the ADI's author reviewed all videotapes to ensure initial competence, and the trained research assistants periodically observed and rated each other to ensure adherence.

A research assistant verbally administered the CRAFFT questions and recorded participants' responses, conducted the ADI interview, and monitored participants' completion of the paper/pencil version of the POSIT scale. All data were entered twice into a specially designed data management program based on Access 97 software (Microsoft, Redmond, Wash), which included automatic range and logic checks and an entry-tracking log. We compared the dual-entry files to identify discrepancies and reconciled them by checking the original data source. The study data manager then imported the cleaned dataset into Statistical Product and Service Solutions (SPSS) software (SPSS Inc, Chicago, Ill) for analysis.

Participants were divided into 5 mutually exclusive diagnostic groups based on their pattern of alcohol and other drug use within the previous 12 months: (1) "no use" included participants who reported no use of alcohol or other drugs; (2) "occasional use" included those who reported any use but had a POSIT score less than 2 and did not have an ADI diagnosis; (3) "problem use" included those with a POSIT score of 2 or higher but no ADI diagnosis; and (4) "abuse" and (5) "dependence" included those who met corresponding diagnostic criteria on the ADI interview for either an alcohol- or drug-related disorder. Each ADI was scored twice, first by a research assistant using the standard written instructions and then by computer using an SPSS syntax algorithm developed by the instrument's author.²⁶ In cases where the diagnoses were unclear, the principal investigator (J.R.K.) and the study addiction psychiatrist (G.C.) separately reviewed the entire ADI, discussed any differences, and recorded the agreed-upon final diagnoses. They were blinded to participants' CRAFFT scores while conducting these reviews.

The frequencies of demographic variables and participants' diagnostic classifications were computed, and χ² tests were performed to determine whether proportions of demographic characteristics (ie, sex, age, and race/ethnicity) or provider impressions of alcohol or drug involvement differed between the study sample and the group of refusers. We transformed participant age into a dichotomous variable (ie, younger youth and older youth) based on the sample median to preserve adequate cell size for analyses. We also transformed the provider impression variables (ie, no use, occasional use, problem use, abuse, dependence, and no impression) into trichotomous variables (ie, no use/occasional use, problem use/abuse/dependence, and no impression) because abuse and dependence impressions were uncommon and cell sizes were not adequate for analysis.

We assessed the internal consistency of the CRAFFT test using the standardized α coefficient. We computed the frequencies and distributions of the CRAFFT score and the diagnostic classifications and measured their associations using the nonparametric Spearman ρ coefficient. To assess the ability of the CRAFFT test to discriminate among diagnostic classification groups, we first converted CRAFFT scores to ranks, then used 1-way analysis of variance and a post-hoc comparison test to compare mean ranks between pairs of groups. Due to heteroscedasticity, we used the Tamhane T2 post hoc comparison test (based on a t test) that did not assume equal variance.

We plotted receiver operating characteristic curves to determine the optimal cut point for the CRAFFT test (ie, total score with the highest product of sensitivity and specificity) for identifying 3 screening categories: any problem (ie, problem use, abuse, or dependence), any diagnosis (ie, abuse or dependence), or dependence. We calculated sensitivity (ie, probability that a true positive would be identified correctly by CRAFFT), specificity (ie, probability that a true negative would be identified correctly by CRAFFT), positive predictive value (ie, probability that a CRAFFT-positive participant was identified correctly), and negative predictive value (ie, probability that a CRAFFT-negative participant was identified correctly) and used the bootstrap technique to estimate 95% confidence intervals.^30- 32

During the 18-month recruitment period, providers invited 711 adolescent patients to participate in the study. We excluded a total of 41 patients (5.8%) because of cognitive impairment (n = 27), insufficient fluency in English (n = 9), severe hearing impairment (n = 2), anorexia nervosa (n = 2), and psychosis (n = 1). Of the 670 eligible patients, 538 (80.3%) agreed to participate. Reasons most commonly cited for refusing included not enough time (n = 74), not interested (n = 44), or came with a parent (n = 8). The group of refusers did not differ significantly from the study sample in age, sex, race/ethnicity, or provider impressions of alcohol use, other drug use, or any substance use. The study sample was also similar to the entire group of 14- to 18-year-old clinic patients in distribution by age and race/ethnicity but included a significantly greater proportion of females (68.4% vs 59.4%; P<.001).

Frequencies of participants' demographic characteristics and substance-related diagnostic classifications during the previous 12 months are presented in Table 1. Participants were almost equally distributed across years of age; 68.4% were female, 50.6% were black non-Hispanic, 24.2% were white non-Hispanic, 18.8% were Hispanic, and 6.5% were Asian/other. Approximately one half of participants had used alcohol or other drugs during the past year, and more than one fourth had experienced alcohol- or drug-related problems. There were a total of 59 abuse diagnoses; 16 were for alcohol alone, 30 for other drugs alone, and 13 for both alcohol and other drugs. Of the 43 drug abuse diagnoses, 36 were related to cannabis, 5 to stimulants (including caffeine pills, methylphenidate hydrochloride, and amphetamines), and 2 to both cannabis and stimulants. There were a total of 36 dependence diagnoses; 7 were for alcohol alone, 24 for other drugs alone, and 5 for both alcohol and other drugs. Of the 29 drug dependence diagnoses, 27 were related to cannabis use, and 2 were related to use of 3,4-methylenedioxymethamphetamine (MDMA or "ecstasy"). Participants with both abuse and dependence diagnoses (eg, cannabis abuse and alcohol dependence) were classified as having dependence. Almost 10% of participants were classified with abuse and almost 7% with dependence.

The CRAFFT standardized item α was .68 and did not increase with deletion of any item (range, .61-.65). Frequencies of positive responses to individual CRAFFT items (Figure 1) were "ridden in a car," 42.6%; "use to relax," 15.6%; "use alone," 10.8%; "forget things you did," 12.3%; "friends tell you to cut down," 8.4%; and "gotten into trouble," 10.6%. The CRAFFT score median was 1 (range, 0-6), and its distribution was highly skewed.

The CRAFFT score was strongly correlated with diagnostic classification (Spearman ρ = 0.72; P<.001). For diagnostic groups, the CRAFFT median scores (with interquartile ranges) were no use, 0 (0-0); occasional use, 1 (0-1); problem use, 2 (1-3); abuse, 2 (1-3); and dependence, 4 (2-5). The CRAFFT score discriminated adequately among all groups (ie, mean ranks differed significantly from each other and from all other groups) except for problem use and abuse (Tamhane T2; P = .95).

Receiver operating characteristic curves are presented in Figure 2. These curves plot sensitivity against 1 − specificity so that the curve area is an overall measure of a test's accuracy. A receiver operating characteristic area of 1 (upper-left corner of the graph) theoretically indicates that the test is always correct, and an area of 0.5 (a diagonal line bisecting the plot area) indicates that the accuracy is no better than chance alone. The receiver operating characteristic areas for CRAFFT were high for all screening categories (any problem = 0.92; any diagnosis = 0.90; and dependence = 0.93). A CRAFFT score of 2 or higher was associated with the maximal product of sensitivity and specificity, which is also the cut point closest to the upper-left corner of the graph. This is one way of identifying a screening test's optimal cut point, although it does not take into account the test's cost/benefit ratio.³³ The CRAFFT optimal cut point was 2 for all 3 screening categories. One hundred thirty-two (25%) of 538 participants had a CRAFFT score of 2 or higher. Sensitivity, specificity, and positive and negative predictive values of a CRAFFT score of 2 or higher for identifying each of the 3 screening categories are presented in Table 2. Criterion validity did not differ significantly by sex, age, or race/ethnicity.

This study provides good supportive evidence for the validity of the CRAFFT test as a substance abuse screening device for use among a general population of adolescent clinic patients. The CRAFFT test has acceptable sensitivity and specificity for identifying all screening categories and among all demographic subgroups. The sensitivity and specificity found in this study for the dependence category were close to those reported in the previous pilot study (0.92 and 0.82, respectively) for identifying the need for inpatient treatment, a similar condition, even though the pilot study was conducted in a much-higher-risk sample.²⁰ The CRAFFT test is designed to be a screening tool, so its result is either positive or negative, and a positive result indicates a need for further assessment. However, the CRAFFT score is correlated with increasing severity of diagnostic classification. Therefore, its discriminant properties can help clinicians estimate not only the presence but also the magnitude of risk of substance-related problems. For example, a score of 4 or higher should raise suspicion of substance dependence.

The standardized α of .68 indicates that CRAFFT has an acceptable degree of internal consistency. Although an α of .70 or higher is generally considered desirable, α is partly a function of scale length, and the CRAFFT test has only 6 items.³⁴ It is interesting that the α did not increase with the deletion of any item, despite the fact that the car question differs from all other items in the scale. This question is designed to screen for risk of alcohol-related car crashes. Although important, this risk is not necessarily related to having an alcohol- or drug-related disorder. Some adolescents may answer this question affirmatively based on having ridden in a car with an intoxicated family member, rather than driving after drinking or riding with an intoxicated peer. Nonetheless, almost 43% of the study participants answered "yes" to this question, and providers need effective strategies to deal with this risk.

We have provided detailed information on the characteristics of CRAFFT in Figure 2 and Table 2. Providers can therefore determine the optimal score cut point for the screening category they most wish to target and how best to interpret a positive screen in their own patient populations. Overall, we recommend using a score of 2 or higher as indicating a need for further assessment. A clinic provider can be reasonably reassured when CRAFFT is negative but should assess his or her patient further when the test is positive. However, the relative risk of a false-positive test (eg, additional interview) is low compared with that of a false-negative (ie, missed diagnosis and opportunity for early intervention). Some providers may therefore choose to further assess those adolescents whose score is only 1.

The sensitivity and specificity (0.80 and 0.86, respectively) found in this study for CRAFFT in identifying any disorder compare quite favorably with those found by Bastiaens et al³⁵ for the substantively different RAFFT test (0.89 and 0.69, respectively) and by Chung et al¹⁸ for modified versions of the CAGE¹⁶ (0.67 and 0.82, respectively), TWEAK³⁶ (0.84 and 0.80, respectively), and AUDIT³⁷ (0.97 and 0.75, respectively). The CRAFFT test presents some clear advantages over these other brief screening tests. First, the CRAFFT is the only screening test that includes an item on drinking and driving (or riding with an intoxicated driver). Alcohol-associated motor vehicle accidents are a leading cause of death among adolescents,³⁸ and a question regarding this risk should be a part of routine screening.

Second, the CRAFFT test screens for both alcohol and other drug problems, whereas the CAGE, TWEAK, and AUDIT tests screen for alcohol problems alone. Drug use is highly prevalent among adolescents,² and most providers would likely prefer a single test that can screen for all psychoactive substances simultaneously. Third, the CRAFFT test is simpler to administer and score than either the TWEAK or AUDIT tests. The TWEAK items are weighted, and AUDIT was not designed for oral administration. Although written questionnaires may present an advantage in efficiency when patients complete them in the waiting area, they are limited by risks to confidentiality. One study reported that adolescent medical patients were frequently dishonest when answering providers' questions about substance use because parents were present.³⁹ Providers can ask the CRAFFT questions during the course of the adolescent's physical examination, after parents have left the room. However, some adolescents may be reluctant to discuss their alcohol and other drug use with the pediatrician, even when parents are not present.

Few comparable validation studies have been conducted in general adolescent clinic settings, and none of these included both a risk assessment (ie, the POSIT scale) and a psychiatric diagnostic interview (ie, the ADI).^{18- 20,24,35} Our unique approach to validation of the CRAFFT test allows us to report on the estimated prevalence and range of substance-related disorders among patients in a general adolescent clinic. More than one half of patients in our clinic had used alcohol or other drugs during the past year, and more than one fourth had experienced serious substance-related problems. Almost 1 in 6 (16.3%) had a substance-related diagnosis of abuse or dependence as defined by the DSM-IV.

These findings have serious implications for adolescent health care. They unquestionably reinforce the importance of the existing Guidelines for Adolescent Preventive Services recommendations for universal substance abuse screening. These findings also suggest a need for additional time and personnel to further assess the substantial numbers of adolescents who will screen positive when universal screening is implemented. Positive screens should be followed by a more complete substance use history, taken by either a physician or some other trained health care professional. Unfortunately, recent changes in the health care system have already placed pressure on providers to see more patients quickly. If universal screening is to improve, health care systems must find ways to provide the additional resources needed for assessment of substance-using adolescents.

These findings also suggest a need to increase the capacity of systems and communities to provide substance abuse treatment for adolescents. In clinic settings such as ours, one fourth of patients need at least a brief intervention, and one sixth likely need referral to a treatment specialist. Current resources are not adequate to meet this need. In our own metropolitan area, adolescents needing substance abuse treatment are most often referred to adult programs because so few adolescent-only programs exist. Adult programs rarely accept younger adolescents, and they are not designed to respond to the unique developmental needs of younger or older adolescents. New approaches, such as office-based interventions, must be developed to adequately meet the need for treatment.

There are limitations to the generalizability of our findings regarding diagnostic classifications. This study was conducted in a single urban hospital-based adolescent clinic. Prevalence rates among adolescent patients seen in other clinics, family practices, or general pediatric practices may be different. However, Chung et al¹⁸ found a similar rate (18%) of alcohol disorders in an adolescent emergency department sample, and one large study estimated the rate of current alcohol dependence for the 18 years and older US population at large to be 4.4%, with higher rates among the young.⁴⁰

This study relied on adolescents' self-report. The extent to which some participants may have underreported and others overreported their use of substances is unknown. However, self-report of alcohol and other drug use has been shown to be generally reliable and compares favorably with other methods of substance use detection.^41- 42 The 18-month study recruitment period included 2 summers. Adolescents may use alcohol and other drugs at higher rates when not in school, and recall bias may have resulted in higher reports of past 12-month use by participants recruited during the summer months.

The findings on prevalence may be further limited, in that the study sample, although generally reflective of the clinic population at large, was not selected randomly. Participants were consecutively recruited in approximately half of the 12 clinic sessions conducted each week. We instructed providers to invite all 14- to 18-year-old patients to participate, not only those who had used alcohol or other drugs. However, we cannot assess to what degree they followed this instruction; provider selection bias, resulting in higher than actual prevalence estimates for disorders, remains a possibility. By contrast, healthier and less-affected patients may have been more likely to agree to participate in the study, resulting in self-selection bias and lower than actual estimates of prevalence. Future studies on prevalence should address these limitations and include a larger and more diverse group of clinic settings.

Despite these limitations, this study provides strong supportive evidence for the criterion validity of the CRAFFT test. The CRAFFT test offers pediatricians, nurse-clinicians, family practitioners, internists, and other primary care providers a practical means of quickly identifying adolescent patients who need more comprehensive assessment or referral to substance abuse treatment specialists.

Accepted for publication March 4, 2002.

This study was supported by grant R01 AA12165 from the National Institute on Alcohol Abuse and Alcoholism, Bethesda, Md, and the Substance Abuse and Mental Health Services Administration, Rockville, Md, and grant 036126 from the Robert Wood Johnson Foundation, Princeton, NJ. Other support was provided by grants 5T20MC000-11-06 (Dr Knight) and 5T71MC 00009-10 (Drs Shrier and Harris) from the Maternal and Child Health Bureau, Rockville, and grant K24 AA00289 (Dr Chang) from the National Institute on Alcohol Abuse and Alcoholism.

We thank Erin Gates, BA, Elizabeth Gates, BA, Sarah Rosenberg, BA, and Allison Arneill, MA, for assistance in study implementation; the clinicians and staff of the Adolescent/Young Adult Medical Practice at Children's Hospital Boston for assistance in recruitment; Ken C. Winters, PhD, for consultation on the study measurement battery; and S. Jean Emans, MD, for review of the manuscript.

We have found that laminated pocket cards listing the 6 CRAFFT questions are helpful for administering the screen in actual office practice. Readers who would like a complimentary CRAFFT test pocket card may obtain one by contacting the Center for Adolescent Substance Abuse Research, Children's Hospital Boston, 300 Longwood Ave, Boston, MA 02115; telephone: 617-355-5433; fax: 617-267-9397; Web site: http://www.ceasar-boston.org.

Guidelines for universal screening of adolescent patients for substance use have been available for some time. However, little attention has been given to the specifics of how this screening should be conducted. Many widely used screening devices are either impractical for busy medical offices or developmentally inappropriate for adolescents. Also unknown is the likely outcome of universal screening, ie, what proportion of adolescent patients have alcohol- and drug-related disorders.

This study demonstrates that the brief CRAFFT test has good validity for identifying substance-related problems and disorders in adolescent medical patients. The screen can be orally administered, and it has a convenient mnemonic, based on key words in each of the 6 yes or no questions. Substance-related problems and disorders were highly prevalent in the clinic we studied, affecting more than 1 in 4 patients. Studies proposing new intervention strategies for those who screen positive are urgently needed.

Corresponding author and reprints: John R. Knight, MD, Center for Adolescent Substance Abuse Research, Children's Hospital Boston, 300 Longwood Ave, Boston, MA 02115 (e-mail: john.knight@tch.harvard.edu).