A New Brief Screen for Adolescent Substance Abuse FREE

SUBSTANCE USE and abuse has been considered the number 1 health problem in the United States.¹ Recent epidemiological data indicate drug use by students in grades 8 through 12 has risen significantly since 1990, and adolescents are beginning to drink at younger ages.^{2 - 3} Thirty-two percent of high school students binge drink (≥5 drinks in a row) and 15.4% drive after drinking.⁴ Even more alarming, 38.8% of surveyed students in grades 9 through 12 reported riding in a car with a driver who had been drinking.⁴ This is particularly concerning as motor vehicle crashes are the leading cause of death among young people, and a significant proportion are directly related to the use of alcohol.^{5 - 6}

According to the American Medical Association's Guidelines for Adolescent Preventive Services (GAPS)⁷ and the Maternal and Child Health Bureau's Bright Futures: Guidelines for Health Supervision of Infants, Children, and Adolescents,⁸ every adolescent should be screened for use of alcohol or other drugs (AOD) as part of routine health care. A policy statement recently published by the American Academy of Pediatrics Committee on Substance Abuse states that pediatricians should be "able to evaluate the nature and extent of . . . alcohol and other drug use among their patients . . . and make an assessment as to whether additional counseling or referral may be needed."⁹ However, surveys of practicing pediatricians indicate that fewer than half actually perform such a screening.¹⁰ The reasons for this low level of compliance are unknown. Pediatricians may lack sufficient training or be reluctant to initiate screening for a problem that they believe is both difficult to treat and time-consuming to assess. Development of a brief and effective method for AOD screening, assessment, and intervention may therefore substantially improve practices among pediatricians. The first step in formalizing a clinical management strategy must be the development of a practical screening tool.

Several brief screening tests have been developed for use among adults. In recent years, the Alcohol Use Disorders Identification Test (AUDIT) has become widely known and used.^{11 - 13} The AUDIT is a 10-item questionnaire that has been found to be sensitive and specific for predicting current hazardous use of alcohol and, to a slightly lesser extent, lifetime diagnosis of alcohol dependence.¹³ Among adolescents, however, AUDIT lacks sufficient sensitivity to make its use practical.¹⁴

The CAGE questions have also achieved widespread use.¹⁵ This instrument's name is a mnemonic of the following 4 yes/no questions: "Have you ever felt that you should cut down on your drinking?" "Have people annoyed you by criticizing your drinking?" "Have you ever felt bad or guilty about your drinking?" "Have you ever had a drink first thing in the morning to steady your nerves or get rid of a hangover (eye opener)?" The CAGE questions have several advantages that undoubtedly account for its popularity among clinicians. It is brief, verbally administered, easy to remember, and easy to score (each yes answer=1). The CAGE questions have been shown to have high sensitivity and specificity in medical settings, with a score of 2 or greater indicating a high probability of an alcohol-related diagnosis.¹⁶ It has not been validated among adolescents, however, and some items (eg, the "eye opener" question) are not developmentally appropriate for teenagers.

Several screening questionnaires have been designed specifically for use with adolescents. Most recently, the Problem-Oriented Screening Instrument for Teenagers (POSIT) has received considerable attention.¹⁷ It is a 139-item self-administered yes/no questionnaire that was developed by the National Institute on Drug Abuse as part of the Adolescent Assessment/Referral System, a comprehensive battery that is designed to screen, assess, and guide treatment decisions. The POSIT questionnaire comes in both English and Spanish versions, and a computer-based CD-ROM version is presently undergoing beta testing. The POSIT contains 10 scales: substance use/abuse, physical health status, mental health status, peer relations, family relations, educational status, vocational status, social skills, leisure and recreation, and aggressive behavior/delinquency. The POSIT questionnaire has been validated among adolescents aged 12 through 19 years in multiple settings from schools to juvenile correctional systems, and has been found to have good internal consistency and test-retest reliability among adolescents in an outpatient medical clinic.¹⁸ The POSIT may be an ideal way to screen patients in clinical settings that are dedicated to adolescent medicine, but the 20-minute completion time makes it less practical for use by general pediatricians or family practitioners. A brief, verbally administered test like CAGE would likely gain wider acceptance in these settings.

Another questionnaire, the Drug and Alcohol Problem (DAP) Quickscreen, consists of 30 yes/no items and was developed for use among adolescents in primary care medical offices.¹⁹ Like POSIT, questions were worded to inquire about use of both alcohol and other drugs. In a validation study, Schwartz and Wirtz²⁰ found that 4 DAP items accounted for 70% of the variation between high-risk and low-risk users: "Do you use tobacco products (cigarettes, snuff, etc)?" "Have you ever had an in-school or out-of-school suspension for any reason?" "Do you sometimes ride in a car driven by someone (including yourself) who is high or who appears to have had too much to drink?" "Has anyone (friend, parent, teacher or counselor) ever told you that they believe that you may have a drinking or drug problem?" These 4 items, however, have not been tested independently from the parent instrument.

The closest CAGE equivalent for adolescents is the RAFFT test, developed specifically as a brief screen for teenagers as part of the Brown University Project ADEPT manual.²¹ Like the CAGE test, RAFFT is a mnemonic based on individual items: "Do you drink to relax, feel better about yourself, or to fit in?" "Do you ever drink alcohol while you are by yourself (alone)?" "Do any of your closest friends drink?" "Does a close family member have a problem with drinking?" "Have you ever gotten into trouble from drinking?" The RAFFT test has several advantages. All of its items are developmentally appropriate for adolescents. Each question calls for a yes/no response, making it easy to score. The RAFFT test is brief and easy to remember. However, RAFFT also has some limitations. It was intended as a clinical "prompt" for trainees, rather than a validated screening device. Clinical interpretation is therefore difficult. There is preliminary evidence of RAFFT's validity, but no studies of its psychometric properties have yet been published.²² In addition, the wording of the RAFFT questions does not include drugs, limiting their sensitivity for identifying misuse of psychoactive substances other than alcohol.

The objective of our study was to develop and validate an AOD screening test for adolescents that is brief, reliable, and practical for use in pediatric office practice. The test should be simple to administer and score. Individual items should be developmentally appropriate, worded to include both alcohol and other drugs, and elicit a yes/no response. The screen should allow pediatricians to quickly and accurately discriminate patients requiring intensive AOD treatment from those amenable to office intervention or brief counseling. Lastly, like CAGE and RAFFT, the test should have a mnemonic that makes it easy to remember.

A 9-item screening test was constructed by taking individual questions from the RAFFT, DAP, and POSIT (Table 1). Specific items were chosen because they had been found to be sensitive and specific in prior validation studies, and/or because they were deemed to be clinically relevant and important. Questions were modified to include the word "drugs" as well as the word "alcohol," and most were qualified by "Have you (or do you) ever. . .". These modifications provided consistency among items and enhanced the sensitivity of the screen.

The test was administered to a sample of patients consecutively arriving for routine medical care at an adolescent clinic. Patients aged 14 to 18 years who had a history of AOD use were invited to participate by their primary care provider. Patients who could not speak English or who were judged to be medically unstable or in crisis (eg, in need of a pregnancy test) were not asked to participate. No data were collected on subjects who were excluded or refused participation. Clinicians, however, made an effort to include adolescents whose current AOD use was thought to be significant, as the goal of the study was to develop a brief test that could determine which patients need referral to specialty treatment.

The study was conducted in the Adolescent and Young Adult Medical Practice at Children's Hospital, Boston, Mass. Children's Hospital is a large tertiary care teaching hospital. The adolescent practice has more than 11,000 patients and more than 4000 visits each year. It serves youth from both inner-city and suburban communities.

A research assistant verbally administered the 9-item screen and asked each subject to complete the PICS from the Personal Experience Inventory (PEI).²³ The PEI has been well validated, and the PICS T score has been shown to correctly classify substance abuse treament need as follows: no treatment (T<35), brief office intervention (T=35-40), outpatient/short-term treatment (T=41-54), and inpatient/long-term treatment (T≥55).²⁴ The PICS scale was used as the criterion standard in this study. The sample was randomized for order of test administration (9-item screen and then PICS vs PICS and then 9-item screen) and stratified for sex. Subjects were offered a free movie pass or fast-food voucher for their participation. Based on the Guidelines for Adolescent Health Research,²⁵ the Children's Hospital investigational review board waived parental consent for this study.

The research assistant entered all data into the Statistical Program for the Social Sciences (SPSS Inc, Chicago, Ill) for Microsoft Windows 7.5. Demographic frequencies, means, and SDs were determined. A t test was performed for order of test administration. The PICS T scores were calculated for purposes of standardization. Correlations were analyzed with Pearson r statistics and internal consistency reliability was assessed using Cronbach α. A stepwise linear regression analysis was performed using the PICS raw score as the independent variable. Sensitivity, specificity, and positive and negative predictive values were calculated from frequency tables.

The final study sample included 99 adolescents (70.7% female, 36.4% black, 32.3% white, 19.2% Hispanic). The mean age of subjects was 16.3 ± 1.4 years and mean grade level was 10.2 ± 1.4. All questions on the 9-item screen were equally weighted (each yes answer=1) and the test was constructed so that all items were scored in the same direction (yes answer indicates high risk for AOD problem). The mean score was 2.1 ± 2.3. The screen had good internal consistency (Cronbach α=.79), and α values did not change substantially with deletion of any item.

Six subjects who completed the 9-item screen did not complete the PICS and were excluded from subsequent analysis. The PICS raw score mean for the entire sample was 45.9 ± 16.1 (range, 29-91) and the standardized T score range was 39.5 to 78.1 (mean=50, SD=10). Neither test score was affected by the order of administration (PICS first, t=−1.37, P=.17; 9-item screen first, t=−0.83, P=.41).

The scores of the 9-item screen and PICS were highly correlated (r=0.82, P<.001). The results of the stepwise linear regression analysis are shown in Table 2. This analysis identified a 6-item model (F=34.017, P<.001) whose total score was highly correlated with the PICS score (r=0.84, P<.001). The 6 items were "Have you ever ridden in a car driven by someone (including yourself) who was high or had been using alcohol or drugs?" "Do you ever use alcohol or drugs to relax, feel better about yourself, or fit in?" "Do you ever use alcohol or drugs while you are by yourself (alone)?" "Do you ever forget things you did while using alcohol or drugs?" "Do your family or friends ever tell you that you should cut down on your drinking or drug use?" and "Have you ever gotten into trouble while you were using alcohol or drugs?"

The CRAFFT model accounted for 68% of the total variance, and internal consistency was maintained (Cronbach α=.68).

Distribution of subjects by PICS treatment categories was no treatment, 0%; brief intervention, 22.5%; outpatient/short-term treatment, 49.5%; and inpatient/long-term treatment, 29%. The CRAFFT model correctly classified 86% of subjects through discriminant analysis. The highest severity category was chosen as that which pediatricians most need to identify, and a PICS T score of 55 or higher was therefore used as the "condition" for calculations of sensitivity, specificity, positive predictive value, and negative predictive value. The 6-item CRAFFT model, with a cut-off score of 2 or more yes answers, had a sensitivity of 92.3% (24/26) and specificity of 82.1% (55/67) for long-term treatment need.

The psychometric properties of the CRAFFT questions seem promising, and the test deserves further refinement and validation. In this study, the CRAFFT score was strongly correlated with the PICS score, and is an easier test for pediatricians to administer. The test had good internal consistency, indicating that each item is clinically useful for detecting problems with AOD. The sensitivity and specificity rates were high among subjects who had used AOD. Clinicians had thus prescreened their patients; ie, determined that they had at least some history of AOD use. This is exactly how the test would be used in clinical practice. According to current guidelines for health supervision, every adolescent should routinely be asked questions such as "Have you drunk alcohol in the past month? How much? What is the most you ever had to drink? Have you ever tried other drugs? How often have you taken them in the past month?"⁸ These questions are an appropriate initial screen for AOD use. When answers are positive and concerning, the CRAFFT questions can assist in determining if the use is significant enough to warrant referral to a mental health specialist.

Examination of adjusted R² values in Table 2 suggest that the test likely could be shortened and improved. The 3-item model (forget, car, relax) accounted for almost 60% of the variance, the 4-item model (forget, car, relax, trouble) 64%, and the 5-item model (forget, car, relax, trouble, alone) 66%. Thus, one might argue that adding the latter 2 or 3 questions adds relatively little explanatory value for purposes of statistical analysis. While further testing and refinement of this screen is needed, all 6 CRAFFT questions are clinically relevant and additional questions and discussion should follow any yes answer. Each will therefore be briefly discussed.

"Have you ever ridden in a car driven by someone (including yourself) who was high or had been using alcohol or drugs?" This may be the single most important question to ask. As previously discussed, alcohol-related motor vehicle crashes are a leading cause of death for teenagers. In fact, those youth that answer yes to this question need advice, assistance, and follow-up by the primary care provider even if they do not use AOD themselves. If they have ridden home with an intoxicated peer, for example, the pediatrician may work toward the development of a "rescue plan." This is a contract with parents to provide a ride home whenever their son or daughter calls to request one, with no recriminations or disciplinary action. A very different situation evolves, however, when the driver of the car in question is in fact the parent. In this case, the pediatrician may need to become involved in a parent intervention or, in extreme circumstances, file a report to child protection authorities.

"Do you ever use alcohol or drugs to relax, feel better about yourself, or fit in?" The intention of this question is to determine if there is "use for effect." A yes answer may indicate that the teenager feels uneasy in social situations and would be particularly susceptible to peer pressure and high-risk behavior.²⁶ With these youth, pediatricians should discuss positive alternatives for socializing (eg, school- or church-sponsored activities) and offer brief counseling and support.

"Do you ever use alcohol or drugs while you are by yourself (alone)?" The social context of AOD use is important. Most adolescents begin to use AOD with peers, yet peer influence has little to do with continuation of use.^{27 - 28} Using alone is a "red flag" behavior and may indicate emerging addictive pathology and increasing social isolation.²⁹ A positive response to this one question may be in itself an indication for early referral to treatment.

"Do you ever forget things you did while using alcohol or drugs?" A yes answer to this question is another indication of a serious problem. Episodes of amnesia usually indicate heavy alcohol use, heavy drug use, or a combination of both. Anterograde amnesia (blackout) is a particularly worrisome sign that may indicate alcoholism.³⁰ Pediatricians should always follow up a yes answer with further questions regarding how long ago this occurred, what the pattern of use was at the time, and whether or not there have been recurrences. This is another symptom that indicates a likely need for prompt referral to treatment.

"Do your family or friends ever tell you that you should cut down on your drinking or drug use?" This is a third-person version of the "cut down" question from CAGE. Adolescents' responses to questions about how others view their behavior are often quite revealing. This question also assists the clinician in determining who might be a potential ally in intervention and treatment for the patient. A yes answer should be followed with questions such as "Who was it that told you that? Why would they think that? Do you think they still feel that way? Why or why not?"

"Have you ever gotten into trouble while you were using alcohol or drugs?" This question helps determine problem severity. Figure 1 illustrates a theoretical model for understanding the progression of AOD use among adolescents that is based on the American Academy of Pediatrics DSM-PC Child and Adolescent Version.^{31 - 32} According to this model, the "problem use" stage is very important to identify in the medical office setting. Negative consequences occur as a result of AOD use, but control over use has not been completely lost. Youth at this stage are often still amenable to brief interventions. Pediatricians must remember, however, that younger adolescents, whose cognitive abilities are still emerging, may be developmentally unable to associate their problems with AOD use. The question should therefore be broken down into several shorter ones: "Have you gotten into any trouble since I last saw you? What happened? Were you using alcohol or other drugs at the time? Have you ever considered that, if you hadn't been using alcohol or drugs, you might not have gotten into trouble?" This sequence of questions can not only elicit valuable information, but becomes the beginning of an intervention. Such office interventions have been previously described, but not yet formally studied in adolescents.^{31 ,33 - 35}

Determination of the CRAFFT's validity must be an ongoing and dynamic process. The purposes of our study were to begin that process and to stimulate continuing research. Given that the condition identified by the screen is associated with significant morbidity and mortality, and that the relative "cost" of a false-positive (ie, further assessment) result is negligible compared with that of a false-negative result, a model with high sensitivity and high negative predictive value is desirable. The 6-item CRAFFT model, with a cut-off score of 2 or more, is both sensitive (92%) and specific (82%) for indicating intensive-treatment need, and has adequate positive (67%) and negative (97%) predictive value according to PICS criteria. In other words, 92% of adolescents who need intensive treatment will be identified by a CRAFFT score of 2 or higher (sensitivity), and 97% of those with a lower score do not need intensive treatment (negative predictive value). Lowering the CRAFFT cut-off score to 1 would increase sensitivity (100%), but decrease specificity (58.2%). Consideration may also be given to use of a model with fewer items. For example, a 4-item test with cut-off score of 2 had a sensitivity of 81% and a specificity of 90%, and a positive predictive value of 75% and negative predictive value of 92.3%.

This study has several limitations. The sample size is relatively small. Data were not collected on adolescents who refused to participate or were excluded. In addition, while the demographic makeup of the study sample was fairly typical for an urban hospital-based adolescent practice, the mean PICS score (45.9) for the subject population was high. In a prior study, adolescents meeting Diagnostic and Statistical Manual of Mental Disorders, Revised Third Edition criteria for an AOD "abuse" diagnosis had a PICS score mean of 40.1 and those meeting criteria for a "dependency" diagnosis had a mean score of 51.3.²³ As the mean for our subjects falls between these 2 numbers, we conclude that our study population had a high prevalence of alcohol- and drug-related pathology. None of the subjects fell into the "no treatment" category according to PICS treatment classification. The sample, of course, was not selected at random. Study clinicians were told that adolescents who had ever used AOD could be included, but most invited only those adolescents they felt were using AOD regularly.

Finally, as in all validation studies, our brief screen can be no better than the instrument it was compared against, and the PICS validation was performed in a different subject population (ie, an AOD specialty clinic). Further studies of the CRAFFT questions are needed, including predictive and differential validation against an assessment battery that includes a standardized diagnostic interview. It should be validated in different populations and settings. Further refinement of items is also needed, and clinical utility should be investigated by eliciting feedback from both pediatric clinicians and adolescent patients. Final decisions regarding a 4-item vs a 6-item model should be guided by blending data analysis with clinical experience, theoretical considerations, and practicality.

The CRAFFT questions show potential as a brief AOD screening device. The test is easy to administer and score, and its sensitivity and specificity rates are promising. Further research must confirm the test's characteristics before its use can be widely recommended. This research should include testing in other populations and clinical settings. Concurrent and predictive validity studies should be conducted against instruments that assess both AOD use severity and diagnoses. Individual CRAFFT items can likely be modified and improved. However, CRAFFT has the potential to assist clinicians in quickly identifying youth who need referral to substance abuse treatment programs.

Accepted for publication October 7, 1998.

This work was supported in part by grants MCJ259360 and MCJ-MA 259195 from the Maternal and Child Health Bureau, Washington, DC, and grant DA09288-01 from the National Institute on Drug Abuse, Washington, DC.

Presented at the Annual Meeting of the Ambulatory Pediatric Association, New Orleans, La, May 3, 1998.

We thank Ken C. Winters, PhD; Elizabeth Rahdert, PhD, for advice and assistance; Michelle Palacios for coordination of the project; and S. Jean Emans, MD, and Elizabeth Goodman, MD, for reviewing the manuscript.

Reprints: John R. Knight, MD, Children's Hospital. 300 Longwood Ave, Boston, MA 02115 (e-mail: knight_j@a1.tch.harvard.edu).

Editor's Note: There appears to be nothing cheesy about CRAFFT as a brief AOD test. Now to get it tested in a large population.—Catherine D. DeAngelis, MD