Two Educational Interventions to Improve Pediatricians' Knowledge and Skills in Performing Ankle and Knee Physical Examinations FREE

MUSCULOSKELETAL sports injuries in children and adolescents are common in primary care settings and can have important sequelae. The national cost of initial medical care for persons younger than 19 years who were injured in sports was estimated to be $22 billion in 1994.¹ Functional limitations persist indefinitely after some musculoskeletal injuries.^2- 3 A knee injury in adolescence increases the risk of osteoarthritis in those older than 65 years.⁴ Young athletes drop out of sports after injuries.⁵ Residual musculoskeletal deficits from injury are associated with increased risk of subsequent musculoskeletal injury.

Assuming that the physician evaluating the injury is skilled in making the correct diagnosis, injuries can be rehabilitated and, in doing so, subsequent injuries can be prevented and athletes can return to exercise without restriction.^6- 10 Deficits in physicians' training in diagnosing and treating musculoskeletal injuries can have undesirable effects, including (1) inappropriate referrals,^11- 12 (2) delayed referrals,¹³ and (3) suboptimal management of musculoskeletal problems and errors in distinguishing organic and psychogenic pain.^10,13- 14

Pediatricians report that training in the care of routine musculoskeletal injuries is lacking.^11,15- 16 There is a need to improve pediatricians' diagnostic and management skills for routine musculoskeletal injuries. The first step in diagnosing a musculoskeletal injury is performing a correct physical examination of the injured structures.

Investigators reported that general and family practitioners' participation in continuing medical education (CME) interventions for management of arthritis was associated with improved knowledge and skill in diagnosing and treating arthritis.^12,17- 20 However, these studies have tended to use biased samples: groups of general practitioners without rheumatology experience, 15 "educationally influential physicians," 36 local practitioners who volunteered (38% of the eligible physicians in the area), and 529 family practitioners (68% of the eligible sample). In addition, the published reports do not describe their educational methods to the extent that they could be replicated and do not include an assessment of or training for physical examination skills. In one randomized trial of CME in primary care, no assessment of physical examination skills was mentioned; rather, physicians' "quality of care" provided was assessed.²¹ There is no published method that has been demonstrated to improve pediatricians' knowledge and skills in performing the musculoskeletal examination.

The purpose of our study was to compare the effect of 2 methods (videotape only and videotape plus skills instruction) of teaching the physical examination of the ankle and knee on the knowledge and skills of pediatricians. The ankle and knee were selected as the focus of the teaching intervention in this project as they are the 2 most common sites of injury in young athletes.²² Our hypothesis was that pediatricians' knowledge and skills in performing the physical examinations of the ankle and knee would be improved with the use of both methods.

All groups of practicing pediatricians (a total of 101 pediatricians) in the Texas Children's Hospital health maintenance organization, Texas Children's Pediatric Associates, Houston, as of October 1998 were eligible to enroll in the study. The number of participating pediatricians per practice group ranged from 1 to 8 (mean, 2.3 pediatricians per practice). A pediatrician was eligible if he or she was an active member of a practice on the day the practice group was first contacted about the study. We directly contacted a physician from each practice and asked whether the members of that group would participate in the study. We went to the physicians' offices to enroll the participants. A pediatrician was classified as enrolled if he or she completed the baseline evaluation and received an intervention. The Texas Children's Hospital Practice Management Committee and the Baylor College of Medicine Institutional Review Board for Human Research Subjects, Houston, approved the study. Informed consent was obtained from each subject.

Two focus groups of practicing pediatricians were convened before the study began, to give feedback regarding this project's methods. Specifically, the focus group participants were asked whether they preferred their knowledge and skills to be assessed by someone they knew or someone they did not know. They indicated that being observed by the physician investigators, whom they might know, would be stressful and preferred evaluation by someone they did not know. Therefore, the study coordinator (A.F.), whom they did not know, did all the baseline and follow-up evaluations. The pediatricians indicated that receiving CME credit for participating in the teaching intervention, providing lunch for the activity, and conducting the interventions at physicians' offices would enhance participation. All of these suggestions were incorporated into the protocol.

The study design was a prospective, intervention trial using a preintervention and postintervention test design, with the unit of analysis being the group practice. Because of the potential problem of physicians interacting within group practices, if the randomization had been by individual physician rather than group, physicians receiving the skills training could have discussed this training with partners who received only the videotape intervention. Therefore, randomization was by practice. Physician practices were randomly assigned to 1 of the 2 treatment options by means of a random number table. This randomization was stratified by practice size to ensure an equitable practice size distribution between the 2 treatment groups. Assignment was done by one of the investigators (L.L.) who was blinded as to which group received which intervention.

A questionnaire was completed with information about sex, years after residency, and the physician's comfort level in diagnosing ankle or knee injuries on the basis of history and physical examination. Comfort level was assessed with a 5-point scale (where 1 indicates very uncomfortable or always refer; 2, uncomfortable, refer most; 3, neutral; 4, comfortable, seldom refer; and 5, very comfortable and almost never refer). The physicians' knowledge and skills in performing the physical examination of the ankle and knee were evaluated by means of a written test and Clinical Skills Assessment Examination (CSAE). The reliability of these methods was established in pediatric residents.²³ The written test contained 10 knee and 10 ankle questions. The written test scores were identified by a unique study number. The written and CSAE scores were anonymous to the physician investigators.

In the CSAE format, after short clinical scenarios were presented, the physicians were asked to examine the ankle and knee of a standardized patient, and this examination was observed by one of us (A.C.H. or J.N.C.). The standardized patient also functioned as an observer and rated the physician's performance by means of published checklists.²³ The standardized patient was a certified athletic trainer (A.F.) who had trained with the 2 physician investigators (A.C.H. and J.N.C.) before this study began. Her performance as a rater was established by interrater reliability for the ankle and knee (α = .99 and .90, respectively) in a previous study of pediatric residents' performance with the use of the same CSAE used in this study.

A videotape entitled Musculoskeletal Examination: Diagnosing Ankle, Knee, Shoulder, and Back Injuries in a Primary Care Setting was produced by one of us (A.C.H.).²⁴ In the first 18 minutes of the videotape, one of us (A.C.H.) demonstrates correct physical examination techniques of the ankle and knee by using actors as patients. The script for the videotape was developed with the use of physical examination checklists for the ankle and knee from our sports medicine clinic. As secondary reinforcers to the videotape, the physicians were given the outline for the videotape script and ankle and knee physical examination checklists on which they could take notes while watching the videotape (checklists are available from the authors). The videotape was tested for appropriateness for pediatric residents and practicing pediatricians with a convenience sample pilot group of 17 health care providers including pediatric residents and physicians in practice or in full-time academic positions. These reviewers were asked to grade the videotape by means of a 5-point Likert scale ranging from uninformative (0) to very informative (4). All reviewers indicated that there was a need for the information provided on the videotape, and 15 of the 17 rated the videotape as very informative. One category 1 CME credit hour for viewing the videotape was obtained through the Office of Continuing Education at Baylor College of Medicine.

In the videotape-only group, each physician watched the first 18 minutes of the teaching videotape in his or her office with one of us present to ensure that the physician watched the videotape. The physicians received the videotape script outline and physical examination checklist forms at the beginning of the videotape. They were able to ask us questions about the videotape content, and we answered these questions.

In the videotape plus skills instruction group, each physician watched the first 18 minutes of the teaching videotape with one of us and received the videotape script outline and physical examination checklist forms at the beginning of the videotape, just as the videotape-only group did. The physical examinations of the ankle and knee were then demonstrated by one of us in a clinic examination room, using one of the physician participants as the model. The physicians used the videotape outlines to follow the investigator doing the examination. Finally, each physician performed the examinations correctly by using one of us as the model and guided by the videotape outlines.

There was no difference in the time interval from the baseline evaluation and the intervention in the 2 treatment groups (videotape only, 13 weeks; videotape plus skills instructions, 14 weeks; P = .8). After the intervention, the physicians were asked to complete an evaluation of the teaching intervention, using a 5-point Likert scale (0, poor; 1, fair; 2, good; 3, very good; and 4, excellent), and complete the written test to get the 1 CME credit.

The physicians completed the written test, performed the CSAEs on the standardized patient, and completed an evaluation of the intervention program, using a 5-point scale ranging from strongly disagree (0) to strongly agree (4), in their offices. Follow-up evaluations were originally scheduled for 1 and 6 months after the intervention. However, limited physician availability resulted in only 1 follow-up evaluation. The design was to blind the rater, at follow-up, to which intervention the group received. However, the rater was also the study coordinator and was unable to be completely blinded about group assignment. The physicians did not receive feedback on their written test and CSAE scores except when they received their CME certificate indicating that they had passed the test, ie, at least 50% correct answers.

The unit of analysis was the group practice. Pearson correlation coefficients were calculated between written test scores and CSAE scores at baseline and follow-up. Spearman correlation coefficients were calculated between the self-rated comfort level in diagnosing ankle and knee injuries and the CSAE scores at baseline. Paired t tests were performed for the written test and CSAE scores at baseline and follow-up. Intervention groups were compared with respect to the various outcome measures by means of a mixed effects model to take clustered randomization into account. As independent variables, the model included the intervention group as a fixed factor, the practice number nested within intervention group as a random factor, and the baseline value as a covariate. MINITAB's general linear modeling procedure was used (Minitab Inc, Cary, NC). The scale for physicians' ratings of the intervention were modified so that "strongly agree" and "agree" were collapsed into one value and "strongly disagree" and "disagree" were collapsed into one value, resulting in a 3-point scale.

Forty-two (81%) of the 52 eligible pediatricians in the videotape plus skills instruction group enrolled in the study. Thirty-three (67%) of the 49 eligible pediatricians in the videotape-only group enrolled in the study. Therefore, the final study sample included 75 pediatricians (74% of the eligible sample). There were no statistically significant differences between the 2 groups in sex, years after residency, comfort level in diagnosing ankle or knee injuries, or written test and CSAE scores at baseline (Table 1). The comfort level in diagnosing ankle injuries was not related to the ankle CSAE score at baseline in either the videotape-only or the videotape plus skills instruction group (P = .86 and .16, respectively). Likewise, the comfort level in diagnosing knee injuries was not related to the knee CSAE score at baseline in the videotape and videotape plus skills instruction groups (P = .71 and .51, respectively).

There was a significant difference between the 2 intervention groups at follow-up for the knee and ankle CSAE scores and for the written test scores (Table 2).

Within the 2 intervention groups, there was a significant increase in written test scores and CSAE scores between baseline and follow-up. The mean percentage of correct answers for the written test in the videotape group at baseline was 44% (8.9/20), and for the videotape plus skills instruction group, 50% (10.1/20). These scores increased significantly to 65% and 81% correct, respectively, at follow-up. At baseline, both groups had mean ankle CSAE scores of 26% (8.7 and 8.8 correct of a maximum score of 34), which significantly increased to 44% (videotape group) and 59% (videotape plus skills instruction group) at follow-up. At baseline, the mean knee CSAE score was 25% (7.6 correct of a maximum score of 31) in both groups. At follow-up, the CSAE scores significantly increased to 30% (videotape group) and 41% (videotape plus skills instruction group).

There was no statistically significant difference in the mean ± SD time of follow-up between the videotape only group (20.2 ± 6.1 weeks; range, 5-28 weeks) and the videotape plus skills instruction group (mean, 16.4 ± 10.8 weeks; range, 4-45 weeks; P = .12).

This study demonstrated that improvements in physicians' knowledge and skills in performing ankle and knee physical examinations were associated with the physicians' participation in either intervention.

The improvements in physicians' knowledge and skills in the ankle and knee examinations were greater in the videotape plus skills intervention group than in the videotape-alone group. This was expected, as the material was presented in a multimedia format and the pediatricians immediately applied the knowledge and skills presented in the videotape in an interactive format, allowing the pediatrician to ask questions of a skilled examiner. The videotape plus skill intervention was labor intensive for the physician investigators, representing a limitation in the use of this method on a broad scale. However, physical therapists and medical students can provide training in performing the physical examination of the musculoskeletal system.^25- 26 In our study, the standardized patient and rater of the CSAEs was a certified athletic trainer. In future studies, models of small group training for physicians directed by physical therapists and athletic trainers could be tested, using this study as a model.

Baseline CSAE scores were unrelated to physicians' comfort in diagnosing these conditions. Competence and self-described comfort in diagnosing clinical conditions may not coincide. The pediatricians' skills in this group were lacking at baseline, consistent with pediatricians' reports of discomfort with the musculoskeletal examination in other reports.^11,15 Performing the majority of the examination techniques correctly is necessary to make a correct diagnosis of an ankle or knee injury. The CSAEs used in this study were at the level of thoroughness and complexity used by sports medicine specialists in examining patients. This is a high standard. Although the CSAE scores improved in both groups, the pediatricians' performance of the physical examination of the ankle and knee needed further improvement. Further study is needed to improve the methods presented here. In a concurrent study, conducted by us, pediatric residents received the same videotape plus skills intervention as the pediatricians in this study, plus they examined 2.8 ± 3.2 and 2.2 ± 2.3 (mean ± SD) patients with ankle and knee complaints, respectively, as part of a 1-month rotation with us.²⁷ The residents' mean ankle CSAE score was 77% (of a possible 100%) and mean knee CSAE score was 55% at 9 months after the rotation was complete. The improved skills demonstrated by the residents compared with the pediatricians in this study could be due in part to the learning associated with examining additional patients during the month. This suggests that, in addition to the one-time videotape or videotape plus skills intervention described in this article, physicians could improve their ankle and knee examinations with further training. The specific activities that best constitute this training require further study.

That knowledge and skills improved with the use of the videotape alone implies that CME to teach examination of the musculoskeletal system to pediatricians in practice could be done via videotape, compact disk, or the Internet. The teaching intervention described in this article could begin to address the gap between practicing physicians' need for hands-on demonstration of techniques and the lack of established methods to do so. This would be an improvement over the current methods of reading about physical examination techniques or attending CME didactic conferences to learn techniques in large groups.

A limitation of this study is that we were professionally familiar with many of the pediatricians in the study, and this may have made recruitment and implementing the interventions easier than if we had been unknown to the pediatricians. Familiarity may have favorably biased the outcomes, as the pediatricians may have had more of an incentive to demonstrate their knowledge and skills than they would have to investigators whom they would be unlikely to interact with again. The physician investigators were blinded to the results for individual physicians.

Accepted for publication November 13, 2001.

We thank Aircast, Inc, Summit, NJ; Ayse McCracken; Kay Tittle, RN; and the Texas Children's Pediatric Associates, Houston, for their support of this project.

This study was presented in part before the Society for Pediatric Research, Boston, Mass, May 15, 2000.

Musculoskeletal injuries are common in pediatric practice. Pediatricians report the need to improve their knowledge and skills in the physical examination of the musculoskeletal system. There are no published methods that address this need.

This study introduces into the medical literature 2 methods, each provided as a one-time intervention that improved pediatricians' knowledge and skills. However, as the methods in this article are a step toward establishing pediatricians' competence in examining the musculoskeletal system, and not the end point itself, further refinement of this model is indicated. The implications for pediatric residency training, with respect to examination of the musculoskeletal system, are obvious.

Corresponding author: Albert C. Hergenroeder, MD, Texas Children's Hospital, 6621 Fannin St, CC 610.01, Houston, TX 77030-2399 (e-mail: alberth@bcm.tmc.edu).