Influenza Vaccine Effectiveness Among Children 6 to 59 Months of Age During 2 Influenza Seasons:  A Case-Cohort Study FREE

The United States and several other countries have expanded their childhood influenza vaccination recommendations in response to evidence that influenza disease causes substantial morbidity among young children. Analyses of insurance claims^1- 4 and population-based studies of laboratory-confirmed disease^5- 6 have documented high rates of hospitalizations and emergency department (ED) and outpatient health care visits attributable to influenza disease among young children. In 2004, the Advisory Committee on Immunization Practices (ACIP) recommended annual influenza vaccination for all children aged 6 to 23 months.⁷ In June 2006, this vaccination recommendation was expanded to include all children aged 6 to 59 months.⁸

An inherent assumption of expanded vaccination recommendations is that the vaccine is efficacious in preventing clinical influenza disease. Although studies have documented immune responses following 2 doses of inactivated influenza vaccine^9- 12 as well as vaccine efficacy for culture-confirmed disease in randomized clinical trials, surprisingly little information exists regarding influenza vaccine effectiveness (VE) among young children receiving vaccine in routine health care settings.^13- 14 One reason for this void is that until recently, childhood influenza vaccination has been recommended only for children with chronic conditions and their close contacts, and even so, few eligible individuals have been vaccinated.^15- 16 Recent recommendations for the vaccination of all children aged 6 to 59 months have made the study of VE in the general pediatric population both relevant and timely.

One major challenge in assessing VE against laboratory-confirmed influenza disease has been that many earlier studies have not used laboratory-confirmed outcomes. Several studies estimated influenza VE against influenzalike illness (ILI) by comparing rates of ILI among vaccinated vs unvaccinated or partially vaccinated children. Studies assessing ILI are inherently compromised by diagnostic uncertainty since the clinical manifestations of influenza disease are similar to those caused by other circulating respiratory pathogens and would lead to underestimates of VE. However, some studies have noted substantial VE against ILI.^17- 20 During the 2003-2004 influenza season, Ritzwoller et al¹⁸ estimated VE to be 49% in outpatient/ED settings for pneumonia- and influenza-coded medical visits among fully vaccinated children aged 6 to 23 months. In the same influenza season, a case-control study in 1 primary care practice compared laboratory-confirmed influenza-positive cases with age-matched controls from the same practice and reported a 52% VE in reducing outpatient visits for influenza among fully vaccinated children aged 6 to 23 months.²¹

The inherent difficulties in measuring influenza VE in children highlight the importance of assessing VE using several different study methods and comparing the results of these different types of studies. The objective of our study was to use a case-cohort study method, which selects control subjects who are representative of the general population for comparison with cases, to measure the effectiveness of trivalent inactivated influenza vaccine in preventing laboratory-confirmed influenza-associated medical visits. We assessed the VE for children aged 6 to 59 months who were hospitalized or had an ED visit and those who had outpatient primary care visits in 3 US counties during 2 consecutive influenza seasons. Evaluating VE for multiple influenza seasons is important because both the severity of the influenza seasons and the match between the vaccine and the circulating strain can be quite variable. We hypothesized that fully vaccinated children would have lower rates of influenza-related inpatient/ED and outpatient visits than unvaccinated or partially vaccinated children.

The study was approved by the institutional review boards of the University of Rochester, Vanderbilt University, Cincinnati Children's Hospital, and the Centers for Disease Control and Prevention (CDC).

The study was conducted in 3 counties: Monroe County, New York, with a total population of approximately 750 000 (11 000 births per year), including the city of Rochester; Davidson County, Tennessee (595 000 people; 8000 births per year), including the city of Nashville; and Hamilton County (800 000 people; 12 000 births per year), including the city of Cincinnati. This study was part of the New Vaccine Surveillance Network,²² which was established in 1999 by the CDC to perform active surveillance for common respiratory viral infections in young children^5- 6 and to assess the impact of new vaccine recommendations on disease outcome.^23- 27

We used a case-cohort design,^28- 30 in which we compared the influenza vaccination status of influenza-positive cases identified by active surveillance with the influenza vaccination status of a randomly selected cohort (more precisely termed a subcohort) from the same population that yielded the cases (Figure). Vaccine effectiveness was evaluated separately for each season (2003-2004, 2004-2005), age group (6-23 months, 24-59 months, and 6-59 months), and health care setting (inpatient/ED, outpatient primary care). We estimated VE by comparing the vaccination status of cases with the subcohort vaccination status at the time the case developed influenza. The sampling frame for inpatient/ED visits was the county since surveillance was population based at the county level. The sampling frame for outpatient visits was a set of primary care practices where practicewide influenza disease surveillance was being conducted.

These children resided in the 3 counties and had laboratory-confirmed influenza-related hospitalizations (at hospitals serving >95% of county children) or ED visits. During the 2003-2004 season, we systematically sampled from 1 of 2 pediatric EDs in Rochester (60% of county pediatric ED visits), 1 pediatric ED in Nashville (30%), and the single pediatric ED in Cincinnati (>95%). During the 2004-2005 season, we sampled from both pediatric EDs in Rochester (>95%) and the pediatric ED in Cincinnati.

Study nurses identified county children hospitalized (enrolled Monday-Thursday in 2003-2004 and 7 days/week in 2004-2005) during the prior 48 hours with acute respiratory illness (ARI) or fever. After obtaining informed consent, we gathered demographic, medical, and social histories using a standardized instrument and nasal/throat specimens for laboratory examination.⁶ For ED cases, we enrolled children either 3 or 4 days/week or every fourth day, varying surveillance systematically across days and ED shifts. In Cincinnati, approximately one-third of the cases in 2004-2005 were from another study that had the same enrollment criteria yielding cases with the same demographic characteristics (results available on request).³¹ Participation rates were high for both seasons: in 2003-2004, only 5.3% (inpatient) and 10.9% (ED) of parents refused participation, and in 2004-2008, only 8.0% and 3.1% refused.

During the same 2 influenza seasons, we conducted an analogous, prospective study of laboratory-confirmed influenza-related outpatient visits among county children (aged 6-23 months or 24-59 months) presenting to selected primary care practices within the 3 counties: 1 practice in Cincinnati (2003-2004 season), 3 practices in Nashville for both seasons, and 4 practices (2003-2004) or 6 practices (2004-2005) in Rochester. Study personnel enrolled children 1 or 2 days/week, varying enrollment days systematically throughout the week to obtain a representative sampling of days and shifts.⁶ Participation rates were high, with only 11% of parents refusing during each season.

Nasal and throat swabs from each enrolled child were collected and tested as described previously.^5- 6,30,32 Viruses were identified by viral culture or reverse-transcription polymerase chain reaction.³¹ A specimen was influenza positive if viral culture or 2 reverse-transcription polymerase chain reaction assays were positive for influenza A or B. Comparison studies across the 3 laboratories (with the CDC laboratory as reference) showed excellent reliability in identification of influenza. Subtyping was performed for a subset of cases.

For the inpatient/ED, the subcohort comprised a representative sample of age-stratified children (aged 6-23 months or 24-59 months) from primary care practices within the same counties. We used a probability proportional to size sampling of primary care practices and excluded practices with fewer than 30 newborns per year, which together accounted for less than 5% of the county birth cohorts. Within practices, we randomly selected 1 to 8 clusters of 30 children per age group from patient lists³³ to achieve a desired total of 1800 children per county per season. This sampling method resulted in 16 practices in Rochester (17 selected, 1 declined), 13 in Nashville (14 selected, 1 declined), and 23 in Cincinnati (39 selected, 16 declined). We reviewed medical records for outcomes. The 16 Cincinnati practices that declined did not differ from the 39 selected practices in practice characteristics, including geographic location (urban/suburban/rural), size (number of physicians), and type (private, hospital clinic, neighborhood health center, or staff model health maintenance organization). For the outpatient component, we randomly selected the subcohort from the same primary care practices that participated in outpatient ARI surveillance for influenza cases, using the procedures described earlier.

Trained medical record abstractors performed medical record reviews for demographic information, vaccination dates, and the dates and types of health care visits on a standardized medical record abstraction form using methods previously standardized across counties.³⁴ We reviewed medical records for both the relevant year and prior years to determine prior vaccinations and the need for 1 or 2 vaccinations during the relevant year.

Children were defined as fully vaccinated if they received: (1) 2 vaccine doses in an influenza season, more than 24 days apart, with the second dose more than 14 days before ARI onset or (2) more than 1 vaccine dose in a prior influenza seasons (priming^7- 8) and 1 dose in the current season more than 14 days before ARI onset. Children were partially vaccinated if they received (1) only 1 of 2 recommended doses during the current season more than 14 days before ARI onset or (2) 2 doses during the current season, with the second dose less than 24 days after the first or within 14 days of ARI onset. All other children were unvaccinated.

The primary dependent variable was influenza case status, case or subcohort control. We sampled without replacement so that subcohort children could be selected for both seasons but not twice within the same season. The key outcome measure was VE.

For cases, a child's influenza vaccination status at the time of the ARI visit was determined by medical record review at the child's primary care practice. Vaccination cards were rarely available, but if so, influenza vaccination dates were recorded and used. For the subcohort, influenza vaccination status was determined by medical record review.

Covariates included sex, insurance status (private vs public/none), and presence of risk factors, chronic conditions documented in medical records for which influenza vaccine is recommended³⁵ (Table 1). Race/ethnicity was not a covariate because it was unobtainable for 60% of the subcohort by medical record review.

Inpatient and ED cases were combined a priori into 1 group because they were population based and to increase the number of cases of more severe influenza disease. Based on anticipated influenza vaccination rates of 10% to 30%, and design effects from the cluster sampling of 1.5 to 2.0, we determined the need for 44 to 250 inpatient/ED cases (per season and age group) to estimate a VE of 50% to 70%, assuming 900 subcohort children per age group per year per county. Using similar assumptions, we estimated the need for 33 outpatient cases, with 360 outpatient subcohort children per year per age group.

Bivariate analysis of demographics and overall influenza vaccination status at the end of the influenza season used χ² analyses to compare cases with subcohort subjects. The primary analysis used a Cox proportional hazards model with time-dependent covariates. A time-dependent variable for vaccination status (complete, partial, and unvaccinated) was evaluated to assess the vaccination status of the subcohort when the case actually developed influenza as opposed to evaluating the vaccination status of the subcohort at the end of the season. Each subject entered the analysis at the beginning of the influenza season, with the time dimension measured as the number of days from the beginning of the influenza season until influenza illness onset or the end of the season, whichever came first. Vaccination was considered as a time-dependent covariate because vaccinations occurred before and during influenza seasons.

The VE was calculated for 2 settings (inpatient/ED and outpatient) and 3 age groups (6-23, 24-59, and 6-59 months) as well as 2 influenza seasons (2003-2004 and 2004-2005). We combined data across the 3 communities by fitting a stratified Cox model with study site as the strata. We included sex, insurance status, and high-risk condition (Table 1) as potential covariates based on prior studies. For the primary analysis, influenza VE was estimated as percentage of VE = (1 − aHRR) × 100, where aHRR is the adjusted hazard rate ratio for influenza in fully or partially vaccinated subjects relative to unvaccinated subjects. We calculated 95% confidence intervals around VE.

Because in a case-cohort design, some subcohort subjects might have had influenza-related visits but were not enrolled in the study, we used a pseudo-likelihood method instead of the usual partial likelihood function to estimate the coefficients.^30,36 Because of intracluster correlation inherent to the cluster sampling design, we computed robust design-based variance estimators.³⁷ In addition, because of complexities in estimating standard errors for Cox regression coefficients in a case-cohort design using cluster sampling, we also used bootstrap sampling methods for comparison with the Cox regression results³⁸; findings were similar to the Cox regression results and are not reported (but are available on request).

Table 1 shows demographic characteristics of the cases and subcohort for both seasons combined since characteristics did not vary between the seasons. For the inpatient/ED study, cases were more likely than subcohort subjects to be younger and male, have public or no insurance, and have a high-risk condition recommended for influenza vaccination. We adjusted for these variables in the analyses.

Table 2 shows results of bivariate comparisons of the influenza vaccination status for cases vs the subcohort for the inpatient/ED and outpatient components of the study during each season. Although a higher proportion of the subcohort than the cases received vaccinations in all age groups, this reached statistical significance only for the inpatient/ED study in 3 age groups: children aged 6 to 59 months in both years and children aged 6 to 23 months in the 2004-2005 season.

Table 3 displays the time-dependent adjusted hazard ratios and the estimated influenza VE during each season by age groups. For the 2003-2004 and 2004-2005 seasons, no statistically significant VE was noted for either fully or partially vaccinated children. As Table 3 shows, the small number of cases in each age group resulted in wide confidence intervals around all VE calculations. During both seasons, children who had high-risk chronic conditions and those who had public insurance or no insurance were more likely to have inpatient/ED visits than their counterparts.

No significant VE in the outpatient setting was observed for any age group for either season (Table 4). Although the VE appeared appreciable for several age groups, particularly in 2003-2004, the small number of cases resulted in wide confidence intervals and, thus, no statistically significant findings.

Although on bivariate analyses the subcohort had consistently higher vaccination rates than the cases for most age strata, after adjustment for covariates, we could not demonstrate statistically significant VE for the 6 to 23 month, 24 to 59 month, or the entire 6 to 59 month age groups in either season.

Although randomized clinical trials and several studies in the general population have shown positive VE, several limitations in prior studies and our current study likely account for these differences. One major limitation in prior studies has involved the use of nonspecific end points (ILI) instead of laboratory-confirmed influenza. Others include a focus on one type of health care setting, such as inpatient care, and assessment of VE for a single season only. Further, VE can vary in different populations by age and presence of a high-risk chronic condition recommended for influenza vaccination, in addition to season and type of setting.

Our study design attempted to address several of these methodological concerns; we chose cases with laboratory-confirmed influenza infection and studied different age groups attending multiple types of health care settings and across 2 consecutive seasons. Further, we hoped that the case-cohort design with the use of an approximate random sample for the comparison group would reduce overall bias and increase generalizability of the findings.

Nevertheless, several factors contributed to the difficulty in demonstrating a positive VE in our study and to different findings from results of clinical trials. These factors included a relatively poor vaccine match for both study years; relatively small sample sizes for influenza-positive cases within each stratum evaluated, particularly when each stratum was further divided into fully, partially, and unvaccinated groups; and low vaccination rates overall. While any single factor would present a methodological challenge to demonstrating VE, the combination of factors made it extremely difficult to demonstrate statistically significant VE.

During the 2003-2004 season, 99% of circulating strains in these 3 communities were due to influenza A virus, but only 11% of influenza A specimens across the United States were similar to a strain included in the vaccine.³⁹ The 2004-2005 season was less severe and the vaccine was a better match to circulating strains than in 2003-2004, but still only 36% of virus isolates were antigenically similar to vaccine strains.⁴⁰ Characterization of a limited number of isolates from the 3 study counties supported the overall US data regarding seasonal and vaccine strains. Thus, the lack of demonstrable VE in our study may have been due to the suboptimal match between influenza strains in the virus and strains circulating in the 3 communities. Interestingly, a recent study in 2004-2005 noted positive VE in adults for inactivated influenza vaccine even though all of the isolates were drifted variants.⁴¹ It is possible that the vaccine has less effectiveness among children. Although vaccines with better matches to circulating influenza strains are clearly needed, it is extremely challenging to predict which influenza strains will circulate and not possible to predict geographic variability in circulating strains.

Despite active surveillance with a large population base of more than 150 000 children younger than 5 years, and identification of more than 400 laboratory-confirmed influenza-positive cases, our sample sizes were still limited in several ways. First, it is difficult to demonstrate VE during a mild influenza season because relatively few excess health care visits are generated by influenza-related illnesses. The 2003-2004 season began early, and following a number of well-publicized pediatric deaths early in the season, was a moderate to severe season in several regions in the United States. In Nashville, the season was more severe, with many inpatient/ED cases, but in Rochester it was mild, with few inpatient or ED cases. The 2004-2005 season was a mild one at both sites, with no excess mortality in adults demonstrated.^40,42

Second, since we believed VE might vary by season, age group, and presence of a high-risk chronic condition recommended for vaccination, we calculated VE for individual strata, which further placed a sample size constraint on our study. Third, the power to detect a significant VE is poor when vaccination coverage in the population is low, as it was in our study. A particularly challenging problem for VE studies that are performed prior to full vaccine recommendation is that low vaccination coverage will limit the power to detect VE at the same time that policy makers would like a demonstrable VE prior to making more widespread recommendations. Finally, smaller sample sizes are needed to demonstrate higher VE; for VE of less than 50%, large sample sizes are needed.

Another methodological challenge in many studies, including ours, is that the threshold to seek medical care may differ among families who bring their child in for vaccination compared with those who do not.⁴³ This higher propensity to seek health care among vaccinated individuals would tend to reduce the VE in studies such as ours that use the general population as controls, and case-cohort study designs may be particularly susceptible to this bias. Studies using case-control designs that select controls in the same manner as cases would not be affected by this factor. In a separate analysis of VE using this population, we found a case-control design that selected controls from noninfluenza ARI enrollees from the same setting that yielded comparison groups that were demographically more similar.⁴⁴

We evaluated VE for trivalent inactivated influenza vaccine and not for live-attenuated vaccine, which has been recently shown in young children to be more efficacious than inactivated vaccine.^45- 47

We found on bivariate analyses that in almost every age group and setting, and for both seasons, cases had lower vaccination coverage than did the subcohort controls (Table 2), suggesting that we might observe a positive VE when we performed the more accurate adjusted analyses. Yet we could not demonstrate significant VE after we adjusted for confounders using a time-dependent multivariable analysis. This discrepancy was due to several factors. First, since children in the subcohort were on average vaccinated later than cases by 12 days, the crude vaccination rate exaggerated the differences between cases and controls, while the multivariate model considered the timing of vaccination, some of which occurred during the influenza season. Adjusting for vaccination timing did not affect results for cases or subcohort controls who were vaccinated prior to the influenza season.

In addition, children with public or no insurance were more likely to be cases and less likely to be vaccinated. The multivariable model adjusted for these differences. Finally, we adjusted for potential differences across counties by stratifying the analyses by geographic site. Geographic variations in VE could theoretically occur from different circulating virus strains in different counties and also from geographic variations in the threshold for hospitalizations, ED visits, or outpatient visits. Although we have previously noted that the 3 counties combined had vaccination rates similar to US rates³² and health care use patterns that are also similar to national rates (P.G.S. and R. Barth, BA, unpublished data, June 2006),^48- 49 we have also observed variations across the 3 counties in inpatient, ED, and outpatient burden of influenza disease.^5- 6 Thus, we adjusted for county in multivariate analyses.

Each year, US children aged 6 to 59 months experience high rates of hospitalizations, ED visits, and outpatient visits due to influenza. Despite this, we were unable across 3 large communities to demonstrate that influenza vaccination was effective in preventing influenza-related inpatient/ED visits or outpatient visits during 2 consecutive seasons (2003-2004 and 2004-2005) among 6- to 23-month-olds, 24- to 59-month-olds, or the entire age span. The case-cohort study design has important limitations in being able to annually assess influenza VE. Our experience suggests that the case-cohort design is inefficient and may insufficiently account for important factors, such as propensity to seek care. Further studies of influenza VE are needed using a variety of study designs (that adjust for confounders) to assess the yearly impact of influenza vaccination programs for children, particularly as higher rates of vaccination are achieved in the study population.

Correspondence: Peter G. Szilagyi, MD, MPH, Box 777, Strong Memorial Hospital, 601 Elmwood Ave, Rochester, NY 14642 (peter_szilagyi@urmc.rochester.edu).

Accepted for Publication: February 27, 2008.

Author Contributions: All authors had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Szilagyi, Griffin, Morrow, Altaye, Ambrose, Poehling, Lofthus, Iwane, and Staat. Acquisition of data: Szilagyi, Fairbrother, Griffin, Ambrose, Lofthus, Holloway, and Staat. Analysis and interpretation of data: Szilagyi, Fairbrother, Hornung, Donauer, Zhu, Edwards, Finelli, Iwane, and Staat. Drafting of the manuscript: Szilagyi, Fairbrother, Hornung, and Finelli. Critical revision of the manuscript for important intellectual content: Szilagyi, Griffin, Hornung, Donauer, Morrow, Altaye, Zhu, Ambrose, Edwards, Poehling, Lofthus, Holloway, Finelli, Iwane, and Staat. Statistical analysis: Szilagyi, Hornung, Morrow, Zhu, and Iwane. Obtained funding: Szilagyi, Morrow, Altaye, Edwards, Finelli, Iwane, and Staat. Administrative, technical, and material support: Szilagyi, Morrow, Ambrose, Holloway, Finelli, and Iwane. Study supervision: Szilagyi, Fairbrother, Griffin, Lofthus, and Holloway.

Financial Disclosure: None reported.

Funding/Support: This work was funded by the Centers for Disease Control and Prevention as part of the New Vaccine Surveillance Network (cooperative agreements: Rochester, UO1 IP000017 and U38CCU217969; Vanderbilt, U01IP000022 and U38CCU417958; Cincinnati, U01IP000147) and the National Vaccine Program Office, and some subjects in Cincinnati, Ohio, were recruited through a study funded by QuickVue Influenza Test (Quidel Corp, San Diego, California).

Previous Presentations: This work was presented in part at the annual meeting of the Pediatric Academic Societies; May 5, 2007; Toronto, Ontario, Canada, and the Options for the Control of Influenza conference; July 2007; Toronto.

Disclaimer: The findings and conclusions in this article are those of the authors and do not necessarily represent the views of the Centers for Disease Control and Prevention.

Additional Contributions: We gratefully acknowledge the help of the primary care practices in all 3 communities who participated in the study, as well as the following individuals: Nashville, Tennessee: Diane Kent, RN, Carol A. Clay, RN, Mariah Daly, RN, and Erin Keckley, RN; Rochester, New York: Caroline Hall, MD, Geoffrey Weinberg, MD, and Kenneth Schnabel, PhD; Cincinnati, Ohio: Meredith Tabangin, MPH, Linda Jamison, RN, BSN, CIC, Shannon Hiratzka, MPH, and Diana Henderson, MPH, MSW; and Centers for Disease Control and Prevention, Atlanta, Georgia: Carolyn B. Bridges, MD, Ranee Seither, MPH, Aaron Curns, MPH, Cedric Brown, MS, Alexander Klimov, PhD, and Haley Clayton, MPH. Drs Hall, Weinberg, and Bridges and Richard Barth, BA, contributed to the conception and design of the study. Mss Kent, Clay, Daly, Keckley, and Jamison, Mr Barth, and Dr Schnabel contributed to the acquisition of data. Mss Tabangin, Hiratzka, Henderson, Bridges, Seither, and Clayton and Messrs Curns and Brown contributed to the analysis and interpretation of data. Drs Schnabel and Klimov contributed to laboratory analyses. We also thank members of the study teams at all sites.