Performance of a Predictive Model for Streptococcal Pharyngitis in Children FREE

IN THE ambulatory setting, acute pharyngitis is the second most common childhood illness diagnosed.¹ Group A β-hemolytic streptococcus (GABHS) accounts for 20% to 40% of all causes of pharyngitis,^2- 4 with a higher incidence in late winter and early spring. Diagnosis and timely treatment of GABHS throat infections are critical to prevent rheumatic fever^5- 6 and other complications. In this context, physicians tend to overdiagnose and overtreat GABHS in patients with pharyngitis.^7- 8 Owing to overlapping of signs and symptoms of pharyngitis from other etiologies, making the clinical diagnosis of GABHS pharyngitis is difficult. The most common approach to suspected GABHS pharyngitis is to obtain a throat culture and/or a rapid antigen test.^9- 10 These tests have sensitivities between 70% and 90%^11- 15 and do not distinguish between carriage state and acute infection.^3,16 Diagnosis and treatment are further complicated by the requirement of patient follow-up, which can be problematic particularly in unreliable patients¹⁷ or when continuity of care is difficult to achieve, as in the emergency department (ED).

The complexity in the clinical and laboratory diagnosis of GABHS pharyngitis has supported the argument of empirical antimicrobial therapy for patients at high risk for this type of infection.¹⁸ However, because the incidence of rheumatic fever is low, others recommend withholding treatment since antibiotic treatment may not significantly quicken the improvement of symptoms associated with sore throat.¹⁹

In an earlier study, we derived a predictive model²⁰ in which 4 independent variables were identified as predictors of a positive throat culture. These predictors included moderate to severe tonsillar swelling, moderate to severe tenderness and enlargement of cervical lymph nodes, and the absence of coryza. This constellation of clinical features yielded a 65% probability for GABHS. When present, scarletiniform rash increased the probability for a positive culture to 95%. A probability of less than 15% was observed in the absence of all of the above features and the presence of coryza. This derivation study was conducted between January 1, 1998, and March 31, 1998.

The objective of the current study was to test prospectively the performance of the model both in different clinical settings and in different seasons. The model was tested in a pediatric ED, the site in which the model was derived, and in 2 pediatric outpatient (OP) clinics. The model was also tested during the derivation study period "in season" (January 1 through March 31) and "off season" (April 1 through December 31).

Patients coming to a children's hospital ED and 2 participating pediatric OP clinics with signs and symptoms of acute pharyngitis between April 1, 1999, and March 31, 2000, were prospectively enrolled.

To document the carriage rate in the population in which the model was tested, controls were recruited among patients coming to these locations during the same study period with noninfectious, nonrespiratory tract complaints such as lacerations, and musculoskeletal injuries, or well-child visits.

The carriage rate, a denominator of the rate of false-positive cultures, was documented for comparison with the posttest probability of the negative predictors. We reasoned that physicians, who are more likely to be concerned with underdiagnosing this condition, can be more confident in the model if the posttest probability values for a positive culture associated with the negative predictors were found to be equal to or less than the rate of false-positive culture results.

All patients and controls who had received antibiotic therapy within 5 days of enrollment and those who were previously enrolled were excluded.

Tonsillopharyngeal specimens were obtained from each patient and control using 2 rayon-tipped culture swabs. Physicians collecting the specimens were provided with diagrammatic instructions on the back of the data collection form to standardize the swabbing technique. All specimens were labeled and delivered to the clinical microbiology laboratory.

All swabs were used to inoculate 2 plates of Strep Selective Agar (Remel Inc, Lenexa, Kan). All plates were incubated under 5% carbon dioxide conditions and examined at 24 and 48 hours for β-hemolytic colonies. β-Hemolytic colonies were subcultured onto fresh sheep blood agar plates for serotyping. Serotyping was performed using the PATHoDx (Remel Inc) latex agglutination kit. The second swab was used for the rapid antigen detection test using TestPack Plus (Abbott Laboratories, Abbott Park, Ill). This microbiological evaluation was identical to the one used in the derivation study.²⁰

The examining physician recorded clinical information on patients and controls at the time of enrollment using a standardized data collection form. They also recorded demographic data and 10 clinical variables, including the 4 variables in the tested model, which were cervical lymphadenopathy, tonsillar swelling (recorded on a 2 category severity scale: absent/mild, moderate/severe), coryza, and scarletiniform rash (recorded as either present or absent).

Each patient received a subjective probability estimate or scoring for a GABHS-positive throat culture by the evaluating physician. The subjective score ranged from 0 (most unlikely) to 10 (extremely likely).

Interobserver agreement was evaluated for each of the clinically assessed variables in the model as well as physicians' subjective scoring for the probability of a positive culture. Two observers each recorded their observations independently on the same patient, using separate data collection sheets, each blinded to the results of the other observer. A sample of approximately 100 subjects (ie, 100 pairs of observations) was estimated to provide 0.8 power, α = .05.

Assuming that a 65% incidence of disease (throat cultures positive for GABHS) is associated with the positive predictors, and testing the hypothesis that the actual incidence is at least greater than 50% (α = .05 [2-tailed]; study power, 0.8), the sample size was estimated to be 85. Assuming that a 15% incidence of disease was associated with the negative predictors and rejecting an incidence of greater than 20% (α = .05 [2-tailed]; study power, 0.8), the sample size was estimated to be 475. A total sample size of 560 patients was planned. Based on previous data,²⁰ 200 controls were estimated to be required to document carriage rate. Data were entered into a database and analyzed using SPSS 10.0 (SPSS Inc, Chicago, Ill).

Subsets within the total sample were compared for homogeneity (age, sex, and rate of positive culture) by ED vs OP and in season vs off season using the χ² test. In addition, patients who were enrolled in the study were compared in the same manner with patients who had received a throat culture at any of the study sites during the same study period but who were not enrolled in the study (had no clinical data collected).

Each variable in the model was assigned a weighted score according to its regression coefficient in the derivation study.²⁰ In the derivation study, the β value of the regression coefficients for scarletiniform rash, moderate-to-severe lymphadenopathy, moderate to severe tonsillar swelling, and the absence of coryza, were 2.2, 0.9, 1.0, and 1.1, respectively. Based on this, scarletiniform rash was given a score of 2 when present and 0 when absent. Moderate to severe adenopathy, moderate to severe tonsillar swelling, and the absence of coryza were given a score of 1. Absence of or mild adenopathy and swelling and the presence of coryza were given a score of 0 (Table 1). Thus, the performance score range was 0 to 5. Patients with positive and negative cultures were then compared at each cutoff point of the performance score in 2 × 2 contingency tables. This information was used to construct a receiver operating characteristic curve and to calculate pretest and posttest odds and probabilities. Results are reported for sensitivity, specificity, positive and negative predictive values, likelihood ratios, and posttest probabilities.

Data on 587 patients were analyzed. Twenty-seven patients were excluded from the database: 15 owing to incomplete identification, 6 owing to previous enrollment, and 6 owing to recent antibiotic use. Of the 587 patients included, 42 had incomplete data entries. The mean ± SD age of patients enrolled in the study was 6.8 ± 3.8 years. Two hundred eighty-nine patients (49.2%) were boys. Three hundred eighty-four patients (65.4%) were enrolled in the ED and 203 (34.6%) were enrolled in the 2 OP sites combined. One hundred seven patients (18%) were enrolled in season (January 1 through March 31) and 480 (82%) were enrolled off season (April 1 through December 31). The age and sex distributions were similar between the 2 study sites and the 2 study periods.

One hundred ninety-four controls were enrolled. One hundred two (52.5%) were boys. Mean ± SD age was 9.5 ± 4.2 years. Twenty-nine (15%) were positive for GABHS, ie, carriers. The rate of carriage was similar in season and off season (P = .9).

Among study patients, 218 (37%) were culture-positive for GABHS and 369 (63%) had negative cultures. The rate of positive culture for GABHS was identical between the 2 study sites (ED, 142/384 [37%]; OP, 76/203 [37%]). As expected, the rate of positive culture for GABHS was higher in season (46/107 [43%]) than off season (172/480 [36%]) but this difference was not statistically significantly different (P = .1).

Compared with throat culture, the rapid antigen test used during the study period had a sensitivity of 86%, specificity of 91%, positive predictive value of 85%, and a negative predictive value of 92%.

Eighty-seven pairs of observations of the 4 variables in the model were recorded and analyzed. An overall agreement rate between 2 observers ranged from 70% to 98%.

Physicians estimated or subjectively scored the possibility of a positive culture as extremely likely (scores 6-10) in 52% of the patients. Forty percent of the culture-negative patients were scored as likely to have GABHS. Conversely, 28% of those with positive cultures were scored as unlikely to have GABHS (scores 0-5). Figure 1 depicts the discordance between physicians' probability estimates for culture results and actual culture results. The agreement rate between 2 observers on subjective scoring was low (21%).

The model's performance as demonstrated in a plotting of sensitivity vs 1 − specificity at each cutoff point of the performance score (receiver operating characteristic curve) was quantified by the area under the curve, 0.67 (95% confidence interval [CI], 0.6-0.7) (Figure 2). The discriminative ability of the model for a positive throat culture was tabulated according to sensitivity, specificity, positive and negative predictive values, multilevel likelihood ratios, and posttest probabilities (Table 2). For comparison, Table 2 also includes the same diagnostic indices for subjective scoring and the rapid antigen test.

Our data, as in previous reports^7- 8 show that physicians tend to overdiagnose GABHS as the cause of pharyngitis. Our data also demonstrate a low level of agreement between physicians on the subjective probability estimates of a positive culture (agreement rate, 21%). These findings highlight the need for guidelines for the clinical diagnosis of GABHS pharyngitis in children. Previous algorithms and scoring systems for pediatric streptococcal pharyngitis have either not been prospectively tested^21- 22 or found to be unacceptably insensitive.²³

The model's positive predictors (score, ≥4) were associated with a posttest probability for a positive culture of 79% (specificity, 0.98; 95% CI, 0.94-0.99). The negative predictors (score, 0) had a posttest probability for a positive culture of 12% (sensitivity, 0.99; 95% CI, 0.96-1.00) which is also lower than the carriage rate (ie, rate of false positive cultures). We found the model's performance to be significantly better than the physicians' subjective scoring for the probability of a positive culture (P<.05, χ² test) and comparable to rapid antigen detection test (negative predictive value, 88% vs 92%; positive predictive value, 79% vs 85%, respectively).

Since there was no difference in patient age, sex, and rate of positive culture between the 2 study sites and the 2 study periods, the use of the model seemed applicable to different practice settings and across seasons. A potentially restrictive factor in using this clinical model, as in any other clinical decision rule, is the agreement between observers on the clinical findings. In our study, the overall agreement rate between observers was high (70%-98%), although the level of training and the experience of the observers varied widely (residents vs attending staff). This variation in clinical experience and training background is typical for many practice settings, giving further support to the generalizibility of the model.

To avoid obtaining a culture from patients with negative predictors, awareness of the carriage rate in the community in which the practice is located could be an important factor in making management decisions. However, the lack of a documented carriage rate in a particular community should not preclude the use of the model since the posttest probability for positive culture (12%) is less than most recently reported carriage rates.^3,20 In a population such as the one in this study, if the model is used, 12% of all microbiological tests can be avoided. Though this is a modest ratio, it is not insignificant for ubiquitous tests such as throat culture and rapid strep antigen tests.

Two possible flaws in the study are noteworthy. First, the rate of positive culture for GABHS in the current study was higher than the rate in the derivation study (37% vs 29%). This may be viewed as an added strength to the stability of the model because it performed well in populations of different disease incidence. Second, although the sample size was predetermined, the included patients represented 30% of all cultured patients in the 2 sites during the same study period. This potential limitation is discounted, since there were no demographic differences between those who were and were not enrolled in the study (P = .5 for sex distribution and mean ± SD age).

This clinical predictive model for pediatric GABHS pharyngitis has performed well in different clinical settings and across seasons. The model was found to be significantly better than subjective estimation by physicians for positive throat cultures and comparable to a standard rapid strep antigen detection test. In using the model, physicians can avoid some of the common problems associated with microbiological testing of GABHS, such as false-negative results and diagnosis of carriers. Perhaps the model is most useful when follow up is not assured or when sensitive confirming tests are not available.

Accepted for publication January 22, 2001.

This study was funded in part by grant W20-8625 from the Nemours research programs, Wilmington, Del.

We acknowledge Paige Walsh, MS, Sonia Gribaldi, MT, and the other technical staff of the Microbiology section of the clinical laboratory at Alfred I. duPont Hospital for Children for their assistance with specimen processing in accordance with the study protocol. The authors also acknowledge the ED attending staff at the Alfred I. duPont Hospital for Children and Hal Byck, MD, and Larry Pradell, MD, from the Nemours children's OP clinics, for their assistance with patient enrollment in the study.