Pediatric Resident and Faculty Knowledge of the Denver II FREE

IDENTIFICATION OF children at risk for developmental delay is one of the important functions of pediatricians during both episodic care and well-child examinations. For trainees, lack of familiarity with developmental milestones can make this a daunting task.

Training in developmental pediatrics has traditionally taken place at our institution through a 1-month developmental pediatrics rotation during the postgraduate year 2 (PGY-2) or PGY-3 year. Because of the volume of material covered in that rotation, our residency committee charged the continuity clinic faculty with providing a focus on knowledge of "normal development" and developmental screening.

The most common screening instrument used by pediatricians is the Denver II,¹ the revised developmental screening test² recommended by the American Academy of Pediatrics.³ We hypothesized that PGY-1 trainees trained in the performance of a Denver II by the Denver training video and instructed to perform Denver II evaluations on their patients in a continuity clinic would improve their knowledge of the Denver II as compared with PGY-1 trainees who did not receive the intervention. We measured the effect of the developmental pediatrics rotation, which includes a requirement to complete 10 Denver II evaluations during the month, on upper-level residents' knowledge of the Denver II.

At the end of the year before the intervention, all available residents at all levels of training were asked to take the Denver II Proficiency Written Test. The tests were completed anonymously and included a self-report of level of training, exposure to the developmental rotation month, and self-assessed level of comfort in performing a Denver II evaluation. Comfort was rated on a Likert scale (1, very uncomfortable; 2, somewhat uncomfortable; 3, neither comfortable nor uncomfortable; 4, somewhat comfortable; and 5, very comfortable). Faculty took the Denver II Proficiency Written Test before viewing the Denver II training video.

The intervention began the next academic year. The clinic environment was enriched with Denver II support materials, including the Denver II forms and kits, the Denver II provider manual, summary posters of the scoring and referral guidelines, and continuity clinic faculty preceptors trained in the use of the Denver II.

At the required PGY-1 continuity clinic orientation (which includes information about the Texas Medicaid and the Early and Periodic Screening, Diagnosis and Treatment requirements; medical record and billing procedures; documentation; and patient scheduling), entering PGY-1 trainees (classes of 1994-1995 and 1995-1996) took the Denver II Proficiency Written Test before watching the Denver II training videotape. These trainees were asked to perform a Denver II evaluation on 1 patient aged from birth to 6 years during each pediatric continuity clinic in their first year (weekly for pediatric PGY-1 trainees and every other week for the internal medicine/pediatric PGY-1 trainees). Residents were instructed to incorporate the Denver II forms in the patient medical record after a photocopy was made and given to the division secretary. The division secretary collated the Denver II screens by clinic and resident as they were received. Halfway through each year, continuity clinic faculty received a list of those interns who had not turned in any Denver II evaluations and were asked to encourage residents' cooperation and participation. At the end of the year, PGY-1 trainees were asked to repeat the Denver II Proficiency Written Test. The tests were matched by self-report of the last 4 digits of the Social Security number to assure anonymity.

The Denver II training videotape and the Denver II Proficiency Written Test were purchased from Denver Developmental Materials Inc,⁴ which recommends that the Denver II Proficiency Written Test be used to assist in evaluating those undergoing training in the administration of the Denver II. A score of 86 of a possible 100 is considered a passing score. The test evaluates ability to calculate appropriate age (including need to adjust for prematurity); interpreting test results for delay or caution; details about test administration; and specifics about item administration and passing items of the Denver II. The final portion of the test is a videotape of 20 items, with respondents scored on accuracy of assessment of the child's performance of the item as pass, fail, refusal, or no opportunity. Tests were scored by hand and the results entered into a computer spreadsheet for analysis. Comparisons were made using Stata for Windows.⁵ The University of Texas–Houston Health Science Center Committee for the Protection of Human Subjects approved the study on the condition that anonymity of residents was maintained. Specifically, the committee insisted that surveys be anonymous and confidentiality maintained and also required that residents be free to decline participation without any penalty.

Test scores were compared for residents who completed their PGY-1 just prior to the intervention and PGY-1 trainees from the 2 intervention years (all values are mean [SD] unless otherwise indicated). The control group of exiting PGY-1 trainees (n=13) had a score of 38.5 (10.4), which was not significantly different from the initial testing for the intervention PGY-1 trainees (41.3 [9.6]). Forty-five of 51 eligible PGY-1 trainees completed both pretesting and posttesting. Postintervention test results for interns completing the intervention (n=45) were significantly improved (59.4 [10.6]) (P<.001). However, only one resident (a postintervention PGY-1 trainee) had a passing score (>86).

Test scores of upper-level residents who had completed the developmental pediatrics rotation (n=14) were better (55.3 [9.3]) than those residents who had not yet entered the rotation (n=19) (40.9 [13.4]). Faculty (n=9) scores were similar to those of PGY-1 trainees (39.0 [15.1]).

Self-reported comfort levels paralleled test scores. Upper-level residents who had completed the developmental pediatrics rotation rated their comfort more highly (3.6 [0.6]) than upper-level residents who had not yet been exposed to it (2.7 [1.0]) (P=.02). Postgraduate year 1 trainees exiting prior to the intervention had a comfort level of 2.5 (0.9), not significantly different from preintervention PGY-1 trainees at 2.0 (1.0). Both were much lower than postintervention PGY-1 trainees (3.3 [0.9]) (P<.01 for both comparisons). Faculty comfort level had a self-reported score of 2.9 (1.2), which also parallels their lower test results.

Because anonymity of resident test results was required by the Committee for the Protection of Human Subjects, it was not possible to determine whether the number of Denver II evaluations completed during the year was associated with performance on the Denver II Proficiency Written Test. The PGY-1 trainees completed from 0 to 57 (mean, 7.5; median, 6) screens in their first year. The mean number of Denver II screening examinations performed by each resident was substantially higher for the 28 pediatric residents (11.3 [range, 0-57]) than for the 22 internal medicine/pediatric PGY-1 trainees (2.8 [range, 0-13]). This difference was greater than would be expected from the difference in clinic time (pediatric residents were in the continuity clinic weekly, while residents in the combined program were in clinic every other week). In spite of this, the mean preintervention and postintervention scores were not different between the 2 training groups (pediatric trainees: 39.2 and 59.6, respectively; internal medicine/pediatric trainees: 45.2 and 60.6, respectively).

In a 1988 survey of US pediatric continuity clinic directors, developmental assessment was reported to be specifically addressed as a skill in 94% of pediatric continuity clinics.⁶ Recommendations have been made for general skills by level of training⁷; however, we could find no information on specific methods or curricula for teaching developmental assessment to residents in a continuity clinic setting. In our study, residents at the PGY-1 level improved their knowledge of the most commonly used developmental screening tool by a simple yet effective intervention. The Denver II Proficiency Written Test did not directly test knowledge of normal development; however, skill in use of the Denver II screening tool would be expected to reflect appropriate assessment of development. Frankenburg et al² reported that the self-assessment test was specifically created to prevent inaccurate administration of the instrument. Evaluation of test reliability and validity was not found in the training manual⁴ or in a search of the literature. Trainees' level of knowledge on this test did not reach a passing level, perhaps because of the less structured training (the training manual recommends a 2-day intensive course with a 6- to 12-hour practice testing session just before the administration of the written test) and the detailed memorization of specific test items expected by the examination. Despite relatively low scores, residents were much more likely to report comfort with the Denver II after the intervention. Time and faculty constraints precluded an attempt to score observed testing of residents performing Denver II tests (a traditional part of testing for practitioners being certified as Denver screeners).

Residents in the intervention group seemed to respond to the request that they perform Denver II evaluations on patients in their clinic. Failure to submit copies of a completed Denver II may have been because of the requirement to photocopy and turn in a copy to the secretary rather than lack of participation in completing Denver II evaluations on clinic patients. Overall, the number of screens reported was smaller (median, 6) than the 10 required of residents during their developmental pediatric rotation, but their postintervention scores were very similar to those of upper-level residents who had completed the rotation. Internal medicine/pediatric residents completed and/or turned in fewer Denver II test forms than their pediatric peers, but were still able to improve in the assessment test, although the trend was toward a smaller overall improvement in score.

Limitations of our data included the requirement for anonymity; therefore, we could not match the number of Denver II evaluations performed by each resident to improvement in performance on the written proficiency test. The relative role of the training video, experience in performing Denver II evaluations, and feedback from faculty during clinic experience could not be assessed from our data. Faculty did not score well on the Denver II Proficiency Written Test; however, all had completed their training when an earlier version was in use and by self-report were somewhat uncomfortable with their skills in performing a Denver II. Faculty watched the Denver II training video after their written testing, but no attempt was made to assess faculty improvement in teaching or knowledge of the Denver II. The relationships between specific faculty preceptors and resident knowledge of the Denver II could not be assessed.

Our intervention produced improvement in PGY-1 skills, increasing them to levels similar to those of upper-level residents who had completed the developmental pediatrics rotation. This simple intervention was effective for trainees to become more familiar with the most common tool to screen infants and children for developmental difficulties. Further research is needed to determine if awareness of development was also improved.

Accepted for publication September 9, 1999.

Presented in part at the Pediatric Academic Societies Meeting, Washington, DC, May 8, 1996.

Editor's Note: The take-home message of this study is the last sentence in the abstract. If the faculty are not "expert" in using Denver II, why would we expect the residents to be? Never mind.—Catherine D. DeAngelis, MD

Reprints: Michelle S. Barratt, MD, MPH, Divisions of Community and General Pediatrics and Adolescent Medicine, University of Texas–Houston Medical School, 6431 Fannin, MSB 3.150, Houston, TX 77030 (e-mail:mbarratt@ped1.med.uth.tmc.edu).