Integrating Spatial Epidemiology Into a Decision Model for Evaluation of Facial Palsy in Children FREE

When the possible causes of a patient's condition vary geographically, knowledge about local-scale disease incidence could help steer physicians toward the most likely diagnosis. Children with peripheral facial palsy pose a diagnostic challenge because optimal management at the point of care requires correct identification of the etiologic nature of the palsy. Rapid point-of-care testing for Lyme disease is not available; thus, diagnostic test results, if in fact tests are ordered, often are not known for several days, leaving physicians to choose a treatment strategy without confirmatory serologic findings. Overdiagnosis of Lyme disease is associated with excessive antibiotic use, and underdiagnosis is associated with progression to more complications. At one time, otitis media ac counted for most identifiable cases of facial palsy in children.¹ Infections with Borrelia burgdorferi have increased over the past several decades, and Lyme disease, the most frequently reported vector-borne disease in the United States, now accounts for a substantial proportion of cases in areas where the disease is endemic.^2- 4 Treatment with antivirals and corticosteroids may be helpful in adults with facial palsy.⁵ For children with facial palsy caused by Lyme disease, early initiation of appropriate antibiotics is the optimal strategy but, according to the American Academy of Pediatrics Red Book Committee on Infectious Diseases, corticosteroids should not be given.⁶

Prediction rules traditionally factor in historical elements, physical examination findings, and sometimes seasonality to identify the correct cause of facial palsy, but, to date, none of the rules has incorporated residential location as a predictor.⁷ Epidemiologic context—the recent regional incidence of a disease, which is often calculable from clinical or public health data sets—may be an important predictor in the absence of timely diagnostic data.^8- 9 Even within areas where Lyme disease is endemic, its incidence varies by location and season, in part because the irregular local and regional distribution of ticks depends on landscape ecology as well as micrometerologic and macrometerologic conditions.^10- 11 Studies have considered ecologic and entomologic risk to generate community-level Lyme disease prevention recommendations and vaccination strategies^12- 15; however, this analytic approach has not yet extended to manipulating formal management algorithms for symptomatic patients by combining geographic risk with clinical features.

To optimize management of peripheral facial palsy in children, clinical decision models would incorporate local epidemiologic risk to differentiate Lyme disease from other etiologies. Taking a novel approach, integrating epidemiologic information about location and season with traditional clinical variables, we sought to create a model to improve diagnostic accuracy and management of peripheral facial nerve palsy in children. We hypothesized that quantitative use of the patient's geographic risk of Lyme disease would improve the accuracy of diagnosis.

Our sample was a retrospective cohort of children younger than 20 years presenting to the emergency department (ED) of Children's Hospital Boston, a large urban tertiary care hospital, from October 1, 1995, to September 30, 2007. The study site ED volume exceeds 50 000 patients annually. We included only children residing in Massachusetts.

Emergency department visits of patients with peripheral facial nerve palsy were identified by a computer-assisted keyword screening tool and regular expression matching from all ED visits at the study site during the study period.¹⁶ We included only children with facial palsy who were evaluated for Lyme disease (either by serologic testing or by the presence of erythema migrans rash). Patients were excluded from analysis if they had any of the following characteristics causing facial palsy: congenital facial palsy, central nervous system cancer, history of herpes simplex virus, surgery near the facial nerve within 1 week of presentation, or Todd paralysis or more generalized paralysis, including hemiparesis.

Lyme disease was considered to be the diagnosis according to the Centers for Disease Control and Prevention definition: presence of erythema migrans lesion or serologic evidence of infection with B burgdorferi via the 2-tiered testing strategy.¹⁷ Tested children were classified as having Lyme disease in our study if results of the Western blot test were positive according to the laboratory reference standards. Offsite commercial laboratory personnel (ARUP Laboratories, Salt Lake City, Utah, and Imugen, Norwood, Massachusetts) performed serologic testing for B burgdorferi. If results of enzyme-linked immunosorbent assay were positive and Western blot testing was negative or no Western blot test was performed, children were not considered to have Lyme disease.¹⁸

Demographics, onset and duration of symptoms, clinical features, laboratory data, and treatment data were collected for each patient via comprehensive medical record review by 2 investigators specializing in pediatric emergency medicine (L.E.N. and A.D.T.). Signs and symptoms included headache, fever, muscle aches, joint pain, rash, and potential exposures, such as tick bites. Laboratory data were reviewed for results of testing for Lyme disease. Treatment data included type and duration of treatment with antibiotics or corticosteroids. To assess interrater reliability, an independent abstractor specializing in pediatric emergency medicine (A.M.F.) reviewed 8% of medical records chosen at random.^19- 20 Candidate predictors with κ statistics with a lower limit of the 95% confidence interval (CI) of greater than 0.4 were considered for the multivariate analyses.²¹ Emergency department visit date and county of residence for each patient were obtained from the medical record review. County-level annual incidence of Lyme disease was calculated from available public health surveillance data from the Massachusetts Department of Public Health Office of Integrated Surveillance Informatics Services.^2,22- 23 These data were used to calculate the average Lyme disease incidence during the preceding 3 years in the home county for each patient. For example, for a patient presenting from Essex County in 2004, the incidence in that county was averaged from 2001 to 2003.

Three decision models were built with clinical and epidemiologic variables: (1) clinical model, with candidate predictors including traditional elements (data on demographics, history, and physical examination); (2) epidemiologic model, with candidate predictors including the timing of presentation (month or season) and the incidence variables associated with the county of residence; and (3) contextualized model, with candidate variables considered in the other 2 models qualifying for inclusion into this model, which combined clinical and epidemiologic predictors.

Univariate and multivariate analytic techniques were used to identify predictors of Lyme disease among patients with peripheral facial palsy. Significance of association of categorical variables with Lyme disease was evaluated using the χ² test. Continuous variables (ie, average county incidence of Lyme disease in the preceding 3 years) were dichotomized at categorical cutoffs (eg, average incidence >20 cases per 100 000 people). Recursive partitioning was used to identify thresholds for testing univariate and multivariate associations.

In the multivariate analyses, candidate variables were entered into a backward stepwise logistic regression to identify independent predictors of patients with Lyme disease. P value cutoffs for entry and departure for the multivariate regression models were .25 and .10, respectively. The final models contained variables for which the level of significance was P < .05.

Several seasonal variables were considered independently for entry into the models. A range of cutoffs was considered to define patients who presented in “Lyme season” (June-October, May-December, and June-November) because Lyme season varies by geography, climate, suitability for tick populations, and annual trends.²⁴ For the spatial variables, the annual county Lyme disease incidence in each of the preceding 3 years and the overall 3-year average incidence were considered independent predictors. Recursive partitioning was used to identify a cutoff for the 3-year average Lyme disease incidence in the county of residence, and this cutoff was considered as an independent candidate predictor. Final models underwent bootstrap validation. Predictors selected in more than 50% of 1000 bootstrap analyses were retained in the final models.^25- 27

Sensitivity, specificity, positive and negative predictive values, and area under the receiver operating characteristic curve (AUC) were used to compare performances of the models. Actual management by pediatric emergency medicine experts was compared with management guided by the decision models. Correct management of Lyme disease facial palsy was defined as use of a correct antibiotic for a correct duration and omission of corticosteroids and antivirals, as defined by the expert panel in the American Academy of Pediatrics Red Book Committee on Infectious Diseases.⁶

The Committee on Clinical Investigation of Children's Hospital Boston approved the study.

From October 1, 1995, to September 30, 2007, there were 609 671 visits to this ED for patients younger than 20 years. Table 1 gives the characteristics of the 264 patients (0.04% of all ED visits) who presented with peripheral facial palsy, were evaluated for Lyme disease, and met study criteria. Patients with facial palsy evaluated for Lyme disease (n = 264) were similar to patients with facial palsy not evaluated for Lyme disease (n = 156) with respect to age, sex, and presence of fever and were more likely to have headache (28% vs 12%, P = .001) and seek care at the ED during Lyme season (49% vs 31%, P = .001). The patients resided in the 9 eastern-most of the 14 counties in Massachusetts. Figure 1 shows county-level average incidence of Lyme disease for Massachusetts during one 3-year period of the study.

Patients with Lyme disease were more likely to be male; have a history of fever, headache, and systemic symptoms (eg, myalgias and arthritis); and have no history of trauma to the face or head (Table 1). There were no significant differences between those with and without Lyme disease in age, neck pain, or otitis media. Exposure to tick bite was not captured in the vast majority of the medical records and thus could not be considered for the analyses.

In the clinical model, headache (odds ratio [OR], 4.4; 95% CI, 2.2-7.5) was the most significant predictor of Lyme disease facial palsy, followed by fever (3.3; 1.6-7.1) (Table 2). Presence of either of these 2 predictors identified children with Lyme disease with 60% sensitivity, 79% specificity, and AUC of 0.71. The positive predictive value was 61% and the negative predictive value was 79%.

Univariate analyses were conducted using a range of cutoffs to define Lyme season. Recursive partitioning identified candidate cutoffs for Lyme season. Patients with Lyme disease were more likely to be seen at the ED during any of the defined Lyme seasons. The Lyme season defined as June-November showed a stronger association for Lyme disease than did the June-October or May-December seasons; thus, for further analyses, June-November was used as Lyme season.

Univariate analysis was used to examine associations between Lyme disease and Lyme disease incidence rates in the patient's home county. Recursive partitioning identified cutoffs to classify 3-year county average incidences as high or low risk. The low-risk cutoff occurred when the average 3-year Lyme incidence for a county was less than 4 cases per 100 000 people. Annual incidence and 3-year average incidence were associated with Lyme disease, but the cutoff incidence of greater than 4 cases per 100 000 people was the strongest spatial predictor and was retained as the spatial predictor for the rest of the analyses.

The best epidemiologic model contained 2 variables: Lyme season (June-November) and high-risk home location (3-year average county-specific Lyme disease incidence >4 cases per 100 000 people). Lyme season (OR, 25.1; 95% CI, 8.6-107.1) and high-risk home location (19.5; 7.4-67.6) were both very strong predictors with ORs greater than 19 (Table 2). The AUC for this model was 0.84.

The contextualized model considering all clinical and epidemiologic variables, regardless of whether they were entered into the previous models, contained 4 variables: fever (OR, 3.9; 95% CI, 1.5-11.0), headache (2.7; 1.3-5.8), Lyme season (25.4; 8.3-113.4), and high-risk home location (18.4; 6.5-68.5) (Table 2). The AUC for this model was 0.89. This model was 100% sensitive and 24% specific with a positive predictive value of 42% and negative predictive value of 100%.

All predictors from the multivariate analyses were validated by the bootstrap method and retained in the final models. High-risk location was selected in more than 99%, Lyme season in more than 97%, fever in more than 81%, and headache in more than 77% of 1000 bootstrap analyses.

Adding epidemiologic factors (seasonal and spatial variables) to the clinical model improved the AUC from 0.71 to 0.89, whereas adding clinical factors to the epidemiologic model improved the AUC more modestly from 0.84 to 0.89. Figure 2 illustrates the risk of Lyme disease facial palsy on the basis of the presence of high-risk predictors. Of the 264 patients in the study, 134 (51%) resided in high-risk locations during Lyme season and 87 of these 134 patients (65%) were diagnosed as having Lyme disease. In contrast, 7 of 130 patients (5%) who presented without both high-risk location and season variables were diagnosed as having Lyme disease. A total of 69 patients from high-risk locations during Lyme season had neither clinical predictor (fever or headache), and Lyme disease was diagnosed in 35 (51%) of these patients. Of the 65 patients with fever, headache, or both, in conjunction with high-risk location and Lyme season, 52 (80%) had Lyme disease. The combinations that included both season and location identified 87 of 94 cases of Lyme disease (93%). Finally, none of the 42 patients without any of the 4 identified risk factors had Lyme disease.

We compared the proportion of children with facial palsy empirically treated with the appropriate medications by attending physicians in the pediatric ED with hypothetical outcomes generated by the 3 models. These physicians treated 68 of 94 patients (72%) with Lyme disease with the correct type of antibiotics and without corticosteroids or antivirals. The epidemiologic and contextualized models did not miss any cases of Lyme disease.

To date, clinical decision rules have relied on clinical factors and, to a much lesser extent, seasonality. In the case of Lyme disease, physicians may informally consider exposure and location when determining the cause of facial palsy, but there are currently no mechanisms that formally facilitate integration of this important contextual information. To the extent that physicians use contextual epidemiologic information to help guide decision making, they tend to use it informally and to rely on personal or pooled collective experiences to reason about testing, diagnosis, and treatment.^28- 30 Although most clinicians certainly often intuitively integrate geographic information into their diagnostic workup for Lyme disease, we show that a quantitative formal integration of geographically based incidence improves diagnosis and treatment.

Within regions of the United States where Lyme disease is endemic, selected states have higher rates and, within those states, there is significant variation by county. Our findings support a general approach of estimating clinical risk of disease at the point of care, accounting for recent spatial incidence. This approach emphasizes applying epidemiologic context to the clinical decision-making process rather than relying solely on history, physical examination, heuristics, and preliminary results of diagnostic tests.^9,31- 32 Improved collaboration between public health departments and physicians, the development of electronic health records, and advances in disease surveillance and automated reporting now increase the feasibility of delivering readily available and easily computed relevant public health information to physicians at the point of care.^33- 35

Previously, our group showed that epidemiologic information about meningitis from a single hospital provides valuable epidemiologic context and enhances a decision model for distinguishing aseptic from bacterial meningitis.⁸ Another previous study by our group also illustrated how an external public health surveillance source improves a clinical decision model by incorporating state-wide epidemiologic context.³⁶ Now, for the first time, we show how spatial incidence data improve the ability of a model to identify cases of an infectious disease. In our analyses, epidemiologic context variables such as season and home location were stronger than any clinical predictor in identifying patients with Lyme disease facial palsy, building on a previous clinical model that did not consider home location.⁷ Epidemiologic context was especially powerful when combined with clinical factors. These findings stress the importance of situational awareness in clinical settings. Understanding the epidemiologic context when the patient is first examined may provide vital information about the etiologic nature of the patient's problem, but currently, valuable spatiotemporal data are not formally processed, considered, used, or integrated into the clinical decision-making process.

Clinical and public health data sets offer synergistic information that can be leveraged to generate and refine clinical decision algorithms. Public health data have not typically contributed information to generate decision models because, although they contain records about patients with confirmed disease, they provide little if any information about those without the disease of interest. This creates unique challenges to the integration of public health data into decision models, which rely on rich information about patients both with and without the disease.^37- 38 To capitalize on the use of public health data, we relied on incidence rates to develop and refine a decision model for Lyme disease, a condition with significant morbidity and of increasing public health importance.

The first limitation of our study involves validation. External validation should be considered before integration into a clinical setting because the performance of predictive indices may deteriorate in subsequent validation studies.³⁹ Our study was confined to Massachusetts, a state in which Lyme disease is endemic. Specific definitions of thresholds may vary when more geographically diverse data are considered. Second, this study occurred at a single ED. However, this site provides care for 75% of the children who live in and around this large metropolitan area. Third, residential county was taken from ED registration data, which may not accurately reflect home location for patients with multiple home addresses. This residential location also does not reflect exposures during travel but does represent the best available exposure data. More accurate information about patient home location would probably strengthen the accuracy of a model incorporating residential location. Fourth, because of the retrospective nature of the study, we were able to include only patients in whom Lyme disease was considered and not patients with subtle signs and symptoms in which the physician did not consider Lyme disease as a cause of facial palsy. Lyme disease incidence data are county level and do not account for variation within a county; therefore, future investigations using larger data sets might provide adequate power to obtain finer spatial resolution. For example, availability of zip code–level incidence data might provide more refined risk stratification. Future studies could also incorporate surrogate markers for local disease incidence, such as vector surveys and canine serosurveys. Last, adherence to mandatory reporting requirements by laboratories and clinicians is highly variable,⁴⁰ because of which underreporting of Lyme disease is a limitation.

This study emphasizes the benefit of integrating epidemiologic context into a clinical decision model. We found that contextual spatial and seasonal epidemiologic factors dominated clinical factors in distinguishing Lyme disease from other causes of pediatric peripheral facial palsy. This study adds to a growing body of evidence that clinical decision support systems can be improved by introducing epidemiologic context variables into algorithms. Public health and clinical information simultaneously presented to a decision support application improve diagnostic accuracy. An important goal of national efforts to promote health information technology should be to foster electronic bidirectional communication of data and messaging between public health and clinical sites.

Correspondence: Andrew M. Fine, MD, MPH, Division of Emergency Medicine, Main 1, Children's Hospital Boston, 300 Longwood Ave, Boston, MA 02115 (andrew.fine@childrens.harvard.edu).

Accepted for Publication: May 24, 2010.

Author Contributions: Dr Fine had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Fine, Brownstein, Nigrovic, Kimia, and Mandl. Acquisition of data: Fine, Brownstein, Nigrovic, Kimia, Olson, Thompson, and Mandl. Analysis and interpretation of data: Fine, Brownstein, Nigrovic, Kimia, Olson, Thompson, and Mandl. Drafting of the manuscript: Fine and Mandl. Critical revision of the manuscript for important intellectual content: Fine, Brownstein, Nigrovic. Kimia, Olson, Thompson, and Mandl. Statistical analysis: Fine, Brownstein, Nigrovic, Kimia, Olson, and Mandl. Obtained funding: Fine and Mandl. Administrative, technical, and material support: Fine, Nigrovic, Kimia, Olson, Thompson, and Mandl. Study supervision: Fine and Mandl.

Financial Disclosure: None reported.

Funding/Support: This work was supported by the Mentored Public Health Research Scientist Development Award K01HK000055 from the Centers for Disease Control and Prevention, by the Public Health Informatics Center of Excellence Award P01HK000016 and P01HK000088-01 from the Centers for Disease Control and Prevention, and by grants G08LM009778 and R01 LM007677 from the National Library of Medicine, National Institutes of Health.

Role of the Sponsors: The study sponsors had no role in study design; in the collection, analysis, or interpretation of the data; in the preparation of the manuscript; or in the decision to submit the manuscript for publication.

Previous Presentations: This study was presented in part at the International Society for Disease Surveillance Annual Meeting; December 4, 2009; Miami, Florida, and at the Pediatric Academic Societies Annual Meeting; May 2, 2009; Baltimore, Maryland.