0
Article |

A Population-Based Cohort Study of Undervaccination in 8 Managed Care Organizations Across the United States FREE

Jason M. Glanz, PhD; Sophia R. Newcomer, MPH; Komal J. Narwaney, MD, PhD; Simon J. Hambidge, MD, PhD; Matthew F. Daley, MD; Nicole M. Wagner, MPH; David L. McClure, PhD; Stan Xu, PhD; Ali Rowhani-Rahbar, MD, PhD; Grace M. Lee, MD, MPH; Jennifer C. Nelson, PhD; James G. Donahue, DVM, PhD; Allison L. Naleway, PhD; James D. Nordin, MD, MPH; Marlene M. Lugg, DrPH; Eric S. Weintraub, MPH
[+] Author Affiliations

Author Affiliations: Institute for Health Research, Kaiser Permanente Colorado, Denver (Drs Glanz, Narwaney, Hambidge, Daley, McClure, and Xu and Mss Wagner and Newcomer); Department of Epidemiology, Colorado School of Public Health, Aurora (Drs Glanz and Hambidge); Community Health Services, Denver Health, Denver (Dr Hambidge); Department of Pediatrics, University of Colorado, Aurora (Drs Hambidge and Daley); Kaiser Permanente Vaccine Study Center, Oakland, California (Dr Rowhani-Rahbar); Center for Child Health Care Studies, Department of Population Medicine, Harvard Pilgrim Health Care Institute and Harvard Medical School, Boston, Massachusetts (Dr Lee); Group Health Cooperative, Seattle, Washington (Dr Nelson); Marshfield Clinic Research Foundation, Marshfield, Wisconsin (Dr Donahue); Kaiser Permanente Northwest, Portland, Oregon (Dr Naleway); HealthPartners Research Foundation, Minneapolis, Minnesota (Dr Nordin); Department of Research and Evaluation, Southern California Kaiser Permanente, Pasadena (Dr Lugg); and Immunization Safety Office, Division of Healthcare Quality and Promotion, Centers for Disease Control and Prevention, Atlanta, Georgia (Mr Weintraub).


JAMA Pediatr. 2013;167(3):274-281. doi:10.1001/jamapediatrics.2013.502.
Text Size: A A A
Published online

Objectives To examine patterns and trends of undervaccination in children aged 2 to 24 months and to compare health care utilization rates between undervaccinated and age-appropriately vaccinated children.

Design Retrospective matched cohort study.

Setting Eight managed care organizations of the Vaccine Safety Datalink.

Participants Children born between 2004 and 2008.

Main Exposure Immunization records were used to calculate the average number of days undervaccinated. Two matched cohorts were created: 1 with children who were undervaccinated for any reason and 1 with children who were undervaccinated because of parental choice. For both cohorts, undervaccinated children were matched to age-appropriately vaccinated children by birth date, managed care organization, and sex.

Main Outcome Measures Rates of undervaccination, specific patterns of undervaccination, and health care utilization rates.

Results Of 323 247 children born between 2004 and 2008, 48.7% were undervaccinated for at least 1 day before age 24 months. The prevalence of undervaccination and specific patterns of undervaccination increased over time (P < .001). In a matched cohort analysis, undervaccinated children had lower outpatient visit rates compared with children who were age-appropriately vaccinated (incidence rate ratio [IRR], 0.89; 95% CI, 0.89- 0.90). In contrast, undervaccinated children had increased inpatient admission rates compared with age-appropriately vaccinated children (IRR, 1.21; 95% CI, 1.18-1.23). In a second matched cohort analysis, children who were undervaccinated because of parental choice had lower rates of outpatient visits (IRR, 0.94; 95% CI, 0.93-0.95) and emergency department encounters (IRR, 0.91; 95% CI, 0.88-0.94) than age-appropriately vaccinated children.

Conclusions Undervaccination appears to be an increasing trend. Undervaccinated children appear to have different health care utilization patterns compared with age-appropriately vaccinated children.

Figures in this Article

Immunization is one of the most significant public health achievements of the past 100 years. However, an increasing number of parents have expressed concerns about immunizations, and survey data15 have shown that more than 10% of parents report delaying or refusing certain vaccinations for their children. These concerned parents often request alternative vaccination schedules that either increase the time between vaccinations or reduce the number of vaccinations in a single well-child visit. Despite their concerns, however, the safety of alternative vaccination schedules is not known.

In an observational setting, examining the safety of alternative vaccination schedules poses significant methodologic challenges. First, identifying children on intentional alternative vaccination schedules may be difficult because there are many potential causes of undervaccination, including missed opportunities, barriers to health care, missing immunization records, and medical contraindication to vaccination. Because vaccine-hesitant parents have a wide range of opinions on the benefits and risks of vaccination, it is also likely that there are numerous alternative schedules to study. Finally, parents who delay or refuse vaccinations may exhibit different health care–seeking behaviors than parents who allow vaccination according to the recommended schedule of the Advisory Committee on Immunization Practices (ACIP).6 The latter issue is particularly problematic because systems for active safety surveillance rely on automated health care utilization data to identify vaccine adverse events. These potential challenges suggest that, before conducting studies on the safety of alternative vaccination schedules, additional data on the epidemiology of undervaccination are needed.

The first objective of this study was to describe patterns and trends of undervaccination in a large cohort of children from 8 managed care organizations (MCOs) during a 7-year period. The second objective was to compare health care utilization rates between undervaccinated children and children who were age-appropriately vaccinated according to the ACIP schedule. As part of the second objective, we also compared health care utilization rates among a subset of children who were undervaccinated because of parental choice. We believe these results will inform future observational studies that examine the safety of alternative vaccination schedules.

SETTING AND STUDY DESIGN

We conducted a retrospective cohort analysis within the Vaccine Safety Datalink, a collaborative project between the Centers for Disease Control and Prevention and 10 MCOs across the United States.7 The participating MCOs comprise a population of more than 9 million members annually (3% of the US population). Using a standardized data dictionary, each MCO prepares electronic data files containing information on demographics, vaccination history, and medical encounters in the outpatient, emergency department (ED), and inpatient settings. Eight MCO sites contributed data to the analyses, and each site's institutional review board approved the study.

The study observation period was January 1, 2004, through December 31, 2010. We first used Vaccine Safety Datalink databases to identify children born between January 1, 2004, and December 31, 2008. For inclusion, each child had to be continuously enrolled in their MCO from at least ages 2 to 12 months. Children were followed up for a maximum of 36 months, and follow-up stopped if a child's enrollment in his or her MCO was discontinued. We excluded children with documented contraindications to some or all vaccinations, including those with hematopoietic stem cell transplant, human immunodeficiency virus, and other immunodeficiencies, or receipt of intravenous immunoglobulin.8 To help ensure that children were receiving primary care services within their MCO, they also had to have at least 1 outpatient visit by age 12 months.9,10 We then ascertained vaccination status on the remaining study population using the recommended ACIP schedule as the standard.

AVERAGE NUMBER OF DAYS UNDERVACCINATED

In this large study population, we calculated the average number of days undervaccinated (ADU) for each child across 8 vaccines during the first 24 months of life (Table 1). This calculation was based on a metric developed by Luman et al11 that measures the difference between when the vaccine dose was administered and when the vaccine dose should have been administered according to the ACIP schedule (eTable). For example, the first dose of diphtheria, tetanus, and acellular pertussis vaccine (DTaP) is due at age 2 months (60 days) but is not considered late until age 92 days. Days undervaccinated for this dose would begin accruing on day 93. In the analysis by Luman et al, a day for which a child was undervaccinated was counted as 1 day regardless of how many vaccines a child was undervaccinated for on that day. In our analysis, we first calculated the number of days undervaccinated for each vaccine dose during the first 24 months of life. We then summed the number of days undervaccinated to calculate the total number of days undervaccinated. Last, we divided the total number of days undervaccinated by the number of vaccines that a child should have received according to the ACIP schedule. This calculation of ADU was adjusted for each child's follow-up time during the observation period, since approximately 16% of the children did not have 24 months of continuous enrollment in their MCO.

Table Graphic Jump LocationTable 1. Advisory Committee on Immunization Practices' Recommended Early Childhood Vaccines and Doses Studieda

Our modified version of the metric of Luman et al11 can be further described with the following example. A child in our 2008 birth cohort is due for a total of 8 distinct vaccinations and 23 individual doses during the first 24 months of life. Suppose that the parents of this child chose to follow a commonly cited alternative vaccination schedule12 in which no doses of hepatitis B, polio, varicella, and measles, mumps, and rubella (MMR) vaccines were given in the first 24 months of life, while all other vaccines were received on time. For this child, the days undervaccinated for these respective vaccines were 638, 638, 242, and 242 (eTable), representing a total of 1760 days undervaccinated. Given that the child was to receive a total of 8 distinct immunizations, the ADU across all vaccines was 220 days (1760/8) in the first 24 months of life.

Our algorithm for calculating the ADU also accounted for vaccine shortages and other changes to immunization policy during the 7-year follow-up period (eTable). We accounted for the pneumococcal (PCV) vaccine shortage in early 2004, the Haemophilus influenzae type b (Hib) vaccine shortage from 2007 to 2009, the addition of rotavirus vaccine to the ACIP schedule in 2006, and changes in age recommendations for varicella vaccine in 2007.1319 In addition, because of the complexities of the changes in the vaccination schedule and shortages during the follow-up period, we conducted 3 subanalyses in which PCV, Hib, and rotavirus vaccination were excluded from the calculations of ADU.

MEDICAL RECORD REVIEW AND MATCHED ANALYTIC COHORTS

After calculating the ADU, we conducted a sample medical record review to identify a subgroup of children whose parents had intentionally delayed or refused vaccinations for nonmedical reasons. The medical record review was conducted in 2 phases. We used Cochran’s20 formula for calculating the sample size, with an assumed confidence interval width of 0.10 and α = .05.

In phase 1 of the review, we divided the population of undervaccinated children into deciles of ADU, with an equal number of children in each decile. A medical record review was conducted on a random sample of children from each decile.

The second phase of the review focused on children with an International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) code for lack of vaccination because of parental choice (V64.05 and V64.06). A medical record review was conducted on a random sample of children with an ICD-9-CM code for parental vaccine refusal and an ADU greater than 0.

For both phases, a medical record abstractor at each MCO examined medical records to identify documentation that a parent had explicitly delayed or refused vaccinations for nonmedical reasons. Results from the review were used to calculate a confirmation rate, measuring the proportion of undervaccinated children with documentation in the medical record that the parent had intentionally selected undervaccination for his or her child. The confirmation rates were used to identify a subgroup of children with a high likelihood of being undervaccinated because of parental choice.

We manually reviewed 1132 medical records from 157 454 children who were undervaccinated at least 1 day (Figure 1). The overall confirmation rate of parental vaccine refusal/delay was 24.3% (95% CI, 22.9%-25.7%) across all 10 deciles (Table 2). Decile-specific confirmation rates ranged from 9.4% to 46.2%, with highest rates in the 8th through 10th deciles (40.2% combined). In most instances, reasons for undervaccination were not documented (60.0%). In 4.5% of the medical records, the vaccines were documented as having been received on time, and in 8.5% of the records, the vaccinations were delayed for medical reasons. Among children with an ICD-9-CM code for parental refusal, the confirmation rate was 93.7% (95% CI, 93.1%-94.2%).

Place holder to copy figure label and caption
Graphic Jump Location

Figure 1. Identifying matched analytic cohorts of undervaccinated children. To be eligible for the study, children had to be born between 2004 and 2008, be continuously enrolled in their managed care site from at least ages 2 to 12 months, and have at least 1 outpatient visit by age 12 months. Children with certain contraindications to vaccinations were excluded from the study. *Children were categorized as having a diagnosis code for vaccine refusal if they had a V64.05 or V64.06 International Classification of Diseases, Ninth Revision, Clinical Modification, code in their electronic medical record before age 24 months.

Table Graphic Jump LocationTable 2. Confirmation Rates of Medical Record–Confirmed Parental Delay or Refusal of Vaccination by Decile of Undervaccination

On the basis of these confirmation rates, we created 2 matched analytic cohort populations: one with children who were undervaccinated without an ICD-9-CM code for vaccine refusal/delay and the other with undervaccinated children with an ICD-9-CM code. The former represented children who were undervaccinated for any reason and the latter represented children who were undervaccinated because of parental choice. To create the analytic cohorts, we matched each undervaccinated child to at least 1 age-appropriately immunized child by birth date (±7 days), MCO, and sex. These children were followed up for a maximum of 36 months after the matched birth date to compare outcomes between the study arms.

OUTCOMES AND ANALYSIS: STUDY POPULATION CHARACTERISTICS AND HEALTH CARE UTILIZATION

Before the matched analytic cohorts were assembled, we conducted a series of descriptive analyses. We first identified the number of distinct patterns of undervaccination in the entire study population. To identify these patterns, we calculated the total number of possible combinations of individual vaccines received. This calculation was based on the 8 vaccines studied during the first 2 years of life. For each vaccine, we created 3 categories: (1) received all doses on time, (2) never received any doses, and (3) certain doses either missing or not received on time. On the basis of these categories, there was a theoretical total of 38 = 6561 possible vaccine combinations, and we identified the actual number of distinct patterns of undervaccination in the study population.

A birth cohort analysis was also conducted in which we calculated the mean and median ADU for children born in each year from 2004 through 2008. In these 5 birth cohorts, we examined the prevalence of undervaccinated children and children with specific patterns of undervaccination over time. These patterns included children with all vaccinations delayed until at least ages 4, 6, 12, and 24 months; children who did not receive DTaP on the same day as PCV and Hib vaccinations; and children who did not receive hepatitis B, polio, varicella, and MMR vaccines in the first 24 months of life. These 6 specific patterns represent commonly cited alternative vaccination schedules.12,21 Time trends were analyzed with linear regression and Cochran-Armitage trend tests.

After the matched analytic cohorts were constructed, they were merged with outpatient, inpatient, and ED data to compare health care utilization rates between the cohort study arms during the first 36 months of life. In one analysis, overall rates, rates of well-child visits, and rates of minor acute illnesses (upper respiratory illness, fever, and pharyngitis) in the outpatient setting were compared. In a second analysis, we compared overall rates of hospitalizations and ED visits between the cohort arms. These data were analyzed with conditional Poisson regression to calculate incidence rate ratios (IRRs) and 95% CIs, controlling for preexisting chronic health conditions.22,23 In the children who were undervaccinated for any reason, the regression analyses were stratified by decile of ADU.

STUDY POPULATION AND TRENDS OF UNDERVACCINATION

We identified 323 247 children born between 2004 and 2008; 157 454 of these children (48.7%) were undervaccinated (Figure 1). The ADU for each child ranged from 1 to 592, with an overall mean (SD) of 36 (89) days (Table 3). The ICD-9-CM code for parental vaccine refusal was used in 6172 of the undervaccinated children (3.9%). In the overall study population, there were 1399 distinct patterns of undervaccination. Among those with an ICD-9-CM code for vaccine refusal, there were 756 distinct patterns of undervaccination. Based on the medical record confirmation rates (24.3% for the overall undervaccinated population and 93.7% for children with an ICD-9-CM code), the estimated prevalence of undervaccination because of parental choice was 13.0% (95% CI, 11.9%-14.2%).

Table Graphic Jump LocationTable 3. Trends in Undervaccination by Birth Cohort

The prevalence of undervaccinated children increased significantly across the birth cohorts from 2004 to 2008 (Table 3). Across the birth cohorts, the mean of the ADU increased by more than 50%, and the median increased from 0 to 3 days. The trends remained significant in the 3 subanalyses in which PCV, Hib, and rotavirus vaccine were removed from the calculation of ADU.

There was also an increasing trend of children on 1 of 6 specific patterns of undervaccination displayed in Figure 2. A total of 8939 children (2.8%) were identified as being on one of these possible alternative vaccination schedules, and approximately 911 children (10.2%) were up-to-date for all of the following vaccines by age 24 months: 4 DTaP, 3 polio, 1 MMR, 3 Hib, 3 hepatitis B, 1 varicella, and 4 PCV.24

Place holder to copy figure label and caption
Graphic Jump Location

Figure 2. Trends in delayed start to vaccinations and select vaccination patterns by birth cohort before age 2 years. Across all birth cohorts, 8939 children (2.8%) were on specific nonstandard vaccination schedules. All trends were significant at P < .001 using the Cochran-Armitage trend test. DTaP indicates diphtheria, tetanus, and pertussis vaccine; Hib, Haemophilus influenzae type b vaccine; and MMR, measles, mumps, and rubella vaccine.

MATCHED ANALYTIC COHORTS

We created 2 matched analytic cohorts in which undervaccinated children were matched to age-appropriately vaccinated children by birth date, MCO, and sex. Of the 157 454 undervaccinated children, 151 282 did not have an ICD-9-CM code for vaccine delay/refusal and represented children who were undervaccinated for any reason. For the first analytic cohort, we were able to individually match 137 855 of the 151 282 undervaccinated children (91.1%) to an age-appropriately vaccinated child, resulting in a total cohort size of 275 710 children. The average follow-up time per child in the cohort was 956 days.

For the second analytic cohort, we individually matched 6168 of the 6172 undervaccinated children (99.9%) with an ICD-9-CM code for vaccine delay/refusal to a median of 10 age-appropriately vaccinated children. The total cohort size was 75 377 children, and the average follow-up time per child was 940 days.

HEALTH CARE UTILIZATION
Children Undervaccinated for Any Reason

Rates of health care utilization differed between the cohort study arms (Table 4). Undervaccinated children had lower outpatient visit rates than children who were vaccinated on time (IRR, 0.89; 95% CI, 0.89-0.90). Undervaccinated children also had lower rates of well-child visits and encounters for specific minor acute illnesses in the outpatient setting. The utilization differences generally increased as the decile of undervaccination increased. These decile-specific IRRs ranged from 1.03 to 0.58, and most (88%) were statistically significant (P < .01).

Table Graphic Jump LocationTable 4. Health Care Utilization From Birth to 36 Months in 137855 Undervaccinated Children Without an ICD-9-CM Code for Parental Vaccine Refusal Matched to 137855 Children With On-Time Vaccinations

The opposite pattern was observed in the inpatient setting. Overall, undervaccinated children had higher inpatient admission rates than age-appropriately vaccinated children (IRR, 1.21; 95% CI, 1.18-1.23). The decile-specific IRRs ranged from 1.04 to 1.37, with the largest IRRs in the 6th through 10th deciles.

In the ED setting, utilization rates were slightly higher among undervaccinated children (IRR, 1.03; 95% CI, 1.03-1.04). Across the deciles of undervaccination, the IRRs varied, but no pattern was observed.

Children Undervaccinated Because of Parental Choice

Children who were undervaccinated because of parental choice had significantly lower utilization rates of the ED and outpatient settings—both overall and for specific acute illnesses—than children who were vaccinated on time (Table 5). The IRRs for these associations ranged from 0.88 to 0.94 and were statistically significant (P < .001). The IRR for inpatient admission rates was not statistically significant (IRR = 0.98; P = .50).

Table Graphic Jump LocationTable 5. Health Care Utilization From Birth to 36 Months in 6168 Undervaccinated Children With an ICD-9-CM Code for Parental Vaccine Refusal Matched to 69209 Children With On-Time Vaccinations

This large multisite cohort study suggests that undervaccination is an increasing trend. We used vaccination and health care utilization data captured from electronic health records to create longitudinal cohorts of children born between 2004 and 2008. Among these birth cohorts, we observed increases in the prevalence and magnitude of undervaccination in children in the first 2 years of life. Our results also suggest that specific patterns of undervaccination have been occurring with greater frequency over time. In addition, our cohort analyses comparing undervaccinated and age-appropriately vaccinated children demonstrated differences in health care utilization, which appear to increase as the magnitude of undervaccination increases. We believe these results have important implications for studying the safety of alternative vaccination schedules in an observational setting.

Approximately 49% of the children in this insured study population were undervaccinated for at least 1 day in the first 24 months of life—a figure that is lower than a published national estimate of 74%.11 In our cohort, we also estimate that approximately 13% of children were undervaccinated because of parental choice, which aligns with other published estimates of 10% to 25% from cross-sectional survey and population-based ecologic studies.1,3,25 However, this likely represents an underestimate of the true prevalence of intentional undervaccination and alternative vaccination schedules. Reasons for undervaccination were not documented in 60% of the records in our review. These insured children were continuously enrolled in their MCO from at least ages 2 to 12 months, and even the most significantly undervaccinated children from the 8th to 10th deciles had a median of 12 clinic visits by age 36 months. This, in turn, suggests that most of these children were using primary care services. It is therefore reasonable to assume that a certain proportion of undervaccinated children without documentation in the medical record represent those who were undervaccinated because of parental choice, thereby underestimating the true prevalence of intentional undervaccination.

The ICD-9-CM codes for parental vaccine refusal/delay had a high confirmation rate (93.7%), suggesting that it correctly classifies children as being undervaccinated because of parental choice. For this reason, we used the code to define our second analytic cohort population to compare health care utilization rates of undervaccinated children with those of children who are age-appropriately immunized. However, this code is infrequently used (3.9% of the unvaccinated cohort) and has a low sensitivity. This implies that the population of children with an ICD-9-CM code for parental vaccine delay/refusal may not be representative of the larger population of children who are undervaccinated because of parental choice. The code should therefore be used cautiously when defining cohorts to study the safety of alternative vaccination schedules.

We identified 1399 distinct patterns of undervaccination in the overall study population and 756 patterns in the cohort who were undervaccinated because of parental choice. Although this variability creates an opportunity to study the safety of alternative vaccination schedules, it also poses significant analytic challenges in trying to identify appropriate comparison groups.26 Moreover, these estimated numbers of patterns of undervaccination are an underestimate of the total number of possible patterns. Our criteria for identifying different combinations of patterns did not consider the number of combinations in which certain doses in a series can be missing or the specific timing of doses that have been delayed. Accounting for these factors could result in billions of possible combinations, thus highlighting the challenges of trying to study the safety of specific alternative vaccination schedules in an observational setting.

In our matched cohort analysis, children who were undervaccinated for any reason had lower rates of outpatient visits and higher rates of ED encounters and inpatient admissions compared with children who were age-appropriately vaccinated. Prior work2730 has shown similar associations when comparing health care utilization across strata of socioeconomic status. However, in our study, the undervaccinated population likely represents a mix of children, including those who were deliberately undervaccinated for personal reasons, children with vaccine contraindications, children receiving their care outside of the MCO, those who were ill at their well-child visit, children of families experiencing barriers to care, and those whose parents simply missed their scheduled well-child visits. Such heterogeneity implies that there may be several factors influencing differences in health care utilization between the cohort arms.

Children who were undervaccinated because of parental choice had lower rates of outpatient visits, lower rates of ED encounters, and no significant difference in inpatient admission rates compared with age-appropriately vaccinated children. These results suggest inherent health care–seeking behavioral differences between the 2 groups of parents. For example, published survey data31,32 have shown that parents who choose not to have their children vaccinated are less likely to trust health care professionals and more likely to use complementary/alternative medicine providers than are parents who have their children fully vaccinated. It is therefore possible that parents who delay or refuse immunizations are less likely to use the traditional health care system when their children contract minor acute illnesses but will seek medical care when their children become seriously ill. Such differences could create a selection bias in studies that attempt to examine the risk of potential adverse events following vaccination. Future survey work with parents across a range of vaccination concerns could help explain these differences in health care utilization rates.

Although most parents choose to have their children vaccinated according to the ACIP-recommended schedule, an increasing number of parents appear to be delaying or refusing vaccines. These parents express concerns about the safety of vaccines; however, they may be placing their children at increased risk for infectious diseases that are almost 100% preventable with vaccination.3339 For these reasons, there is national interest in studying the safety of alternative vaccination schedules.40 Our results demonstrate the potential public health impact of alternative vaccination schedules and highlight the obstacles to studying their safety. We therefore believe the findings of this study should be carefully considered when designing and conducting observational studies to examine the safety of alternative vaccination schedules.

Correspondence: Jason M. Glanz, PhD, Institute for Health Research, Kaiser Permanente Colorado, PO Box 378066, Denver, CO 80237 (jason.m.glanz@kp.org).

Accepted for Publication: August 24, 2012.

Published Online: January 21, 2013. doi:10.1001/jamapediatrics.2013.502

Author Contributions: Dr Glanz had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Glanz, Narwaney, Hambidge, Daley, McClure, Naleway, Nordin, and Weintraub. Acquisition of data: Glanz, Newcomer, Wagner, Lee, Nelson, Donahue, Naleway, Lugg, and Weintraub. Analysis and interpretation of data: Glanz, Newcomer, Narwaney, Hambidge, Daley, McClure, Xu, Rowhani-Rahbar, Nelson, Donahue, Nordin, and Lugg. Drafting of the manuscript: Glanz, Newcomer, and Daley. Critical revision of the manuscript for important intellectual content: Glanz, Newcomer, Narwaney, Hambidge, Daley, Wagner, McClure, Xu, Rowhani-Rahbar, Lee, Nelson, Donahue, Naleway, Nordin, Lugg, and Weintraub. Statistical analysis: Glanz, Newcomer, Narwaney, McClure, Xu, Rowhani-Rahbar, Nelson, and Lugg. Obtained funding: Glanz and Hambidge. Administrative, technical, and material support: Glanz, Daley, Wagner, McClure, Donahue, Naleway, Nordin, Lugg, and Weintraub. Study supervision: Glanz, Daley, Xu, and Lugg.

Conflict of Interest Disclosures: None reported.

Funding/Support: This study was funded through a subcontract with America's Health Insurance Plans under contract 200-2002-00732 from the Centers for Disease Control and Prevention (CDC).

Previous Presentations: Preliminary results from this study were presented at the American Public Health Association conference; November 1, 2011; Washington, DC; and at the Institute of Medicine's Committee on the Assessment of Studies of Health Outcomes Related to the Childhood Immunization Schedule meeting; February 9, 2012; Washington, DC. In addition, the preliminary results were presented as a platform presentation at the Pediatric Academic Society Meeting; May 1, 2012; Boston, Massachusetts. A more detailed presentation of these results was presented at an Institute of Medicine meeting; May 29, 2012; Washington, DC.

Disclaimer: Although the CDC played a role in the design and conduct of the study, collection, management, analysis, and interpretation of the data, as well as preparation, review, and approval of the manuscript, the findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the CDC.

Additional Contributions: The data management and medical record abstraction staff at each of the participating sites created the Vaccine Safety Datalink data sets and reviewed the medical records.

Dempsey AF, Schaffer S, Singer D, Butchart A, Davis M, Freed GL. Alternative vaccination schedule preferences among parents of young children.  Pediatrics. 2011;128(5):848-856
PubMed   |  Link to Article
Freed GL, Clark SJ, Butchart AT, Singer DC, Davis MM. Parental vaccine safety concerns in 2009.  Pediatrics. 2010;125(4):654-659
PubMed   |  Link to Article
Gust DA, Darling N, Kennedy A, Schwartz B. Parents with doubts about vaccines: which vaccines and reasons why.  Pediatrics. 2008;122(4):718-725
PubMed   |  Link to Article
Wightman A, Opel DJ, Marcuse EK, Taylor JA. Washington State pediatricians' attitudes toward alternative childhood immunization schedules.  Pediatrics. 2011;128(6):1094-1099
PubMed   |  Link to Article
Leib S, Liberatos P, Edwards K. Pediatricians' experience with and response to parental vaccine safety concerns and vaccine refusals: a survey of Connecticut pediatricians.  Public Health Rep. 2011;126:(suppl 2)  13-23
PubMed
Centers for Disease Control and Prevention.  Recommended immunization schedules for persons aged 0-18 years—United States, 2011.  MMWR Morb Mortal Wkly Rep. 2011;60(5):1-4.http://www.cdc.gov/mmwr/preview/mmwrhtml/mm6005a6.htm?s_cid=mm6005a6_w. Accessed November 28, 2012
Baggs J, Gee J, Lewis E,  et al.  The Vaccine Safety Datalink: a model for monitoring immunization safety.  Pediatrics. 2011;127:(suppl 1)  S45-S53
PubMed   |  Link to Article
Centers for Disease Control and Prevention.  Guide to vaccine contraindications and precautions. http://www.cdc.gov/vaccines/recs/vac-admin/downloads/contraindications-guide-508.pdf. Accessed July 25, 2012
France EK, Smith-Ray R, McClure D,  et al.  Impact of maternal influenza vaccination during pregnancy on the incidence of acute respiratory illness visits among infants.  Arch Pediatr Adolesc Med. 2006;160(12):1277-1283
PubMed   |  Link to Article
Ritzwoller DP, Bridges CB, Shetterly S, Yamasaki K, Kolczak M, France EK. Effectiveness of the 2003-2004 influenza vaccine among children 6 months to 8 years of age, with 1 vs 2 doses.  Pediatrics. 2005;116(1):153-159
PubMed   |  Link to Article
Luman ET, Barker LE, Shaw KM, McCauley MM, Buehler JW, Pickering LK. Timeliness of childhood vaccinations in the United States: days undervaccinated and number of vaccines delayed.  JAMA. 2005;293(10):1204-1211
PubMed   |  Link to Article
Sears RW. The Vaccine Book: Making the Right Decision for Your Child. New York, NY: Little, Brown; 2007
Parashar UD, Alexander JP, Glass RI.Advisory Committee on Immunization Practices (ACIP), Centers for Disease Control and Prevention (CDC).  Prevention of rotavirus gastroenteritis among infants and children. Recommendations of the Advisory Committee on Immunization Practices (ACIP).  MMWR Recomm Rep. 2006;55(RR-12):1-13
PubMed
Centers for Disease Control and Prevention.  Notice to readers: limited supply of pneumococcal conjugate vaccine: suspension of recommendation for fourth dose.  MMWR Morb Mortal Wkly Rep. 2004;53(5):108-109.http://www.cdc.gov/mmwr/preview/mmwrhtml/mm5305a6.htm. Accessed November 28, 2012
Centers for Disease Control and Prevention (CDC).  Updated recommendations on the use of pneumococcal conjugate vaccine: suspension of recommendation for third and fourth dose.  MMWR Morb Mortal Wkly Rep. 2004;53(8):177-178
PubMed
Centers for Disease Control and Prevention.  Notice to readers: pneumococcal conjugate vaccine shortage resolved.  MMWR Morb Mortal Wkly Rep. 2004;53(36):851-852.http://www.cdc.gov/mmwr/preview/mmwrhtml/mm5336a8.htm. Accessed November 28, 2012
Marin M, Güris D, Chaves SS, Schmid S, Seward JF.Advisory Committee on Immunization Practices, Centers for Disease Control and Prevention (CDC).  Prevention of varicella: recommendations of the Advisory Committee on Immunization Practices (ACIP).  MMWR Recomm Rep. 2007;56(RR-4):1-40
PubMed
Centers for Disease Control and Prevention (CDC).  Interim recommendations for the use of Haemophilus influenzae type b (Hib) conjugate vaccines related to the recall of certain lots of Hib-containing vaccines (PedvaxHIB and Comvax).  MMWR Morb Mortal Wkly Rep. 2007;56(50):1318-1320
PubMed
Centers for Disease Control and Prevention (CDC).  Updated recommendations for use of Haemophilus influenzae type b (Hib) vaccine: reinstatement of the booster dose at ages 12-15 months.  MMWR Morb Mortal Wkly Rep. 2009;58(24):673-674
PubMed
Cochran WG. Sampling Techniques. 3rd ed. New York, NY: John Wiley & Sons; 1977
Offit PA, Moser CA. The problem with Dr Bob's alternative vaccine schedule.  Pediatrics. 2009;123(1):e164-e169
PubMed  |  Link to Article   |  Link to Article
Glanz JM, Newcomer SR, Hambidge SJ,  et al.  Safety of trivalent inactivated influenza vaccine in children aged 24 to 59 months in the Vaccine Safety Datalink.  Arch Pediatr Adolesc Med. 2011;165(8):749-755
PubMed   |  Link to Article
Gay JC, Muldoon JH, Neff JM, Wing LJ. Profiling the health service needs of populations: description and uses of the NACHRI Classification of Congenital and Chronic Health Conditions.  Pediatr Ann. 1997;26(11):655-663
PubMed
Centers for Disease Control and Prevention.  National and state vaccination coverage among children aged 19-35 months—United States, 2010.  MMWR Morb Mortal Wkly Rep. 2011;60(34):1157-1163.http://www.cdc.gov/mmwr/preview/mmwrhtml/mm6034a2.htm?s_cid=mm6034a2_w. Accessed April 27, 2012
Robison SG, Groom H, Young C. Frequency of alternative immunization schedule use in a metropolitan area.  Pediatrics. 2012;130(1):32-38
PubMed   |  Link to Article
Kulldorff M. Study designs for the safety evaluation of different childhood immunization schedules. http://www.iom.edu/~/media/Files/Activity%20Files/PublicHealth/ChildhoodImmunization/Commissioned%20Paper/ReportWebpost%20515.pdf. Accessed July 25, 2012
Agency for Healthcare Research and Quality.  National Healthcare Disparities Report 2009. http://www.ahrq.gov/qual/nhdr09/nhdr09.pdf. Accessed July 25, 2012
Herrod HG, Chang CF. Potentially avoidable pediatric hospitalizations as defined by the Agency for Healthcare Research and Quality: what do they tell us about disparities in child health?  Clin Pediatr (Phila). 2008;47(2):128-136
PubMed   |  Link to Article
Yousey-Hindes KM, Hadler JL. Neighborhood socioeconomic status and influenza hospitalizations among children: New Haven County, Connecticut, 2003-2010.  Am J Public Health. 2011;101(9):1785-1789
PubMed
Flores G, Abreu M, Chaisson CE, Sun D. Keeping children out of hospitals: parents' and physicians' perspectives on how pediatric hospitalizations for ambulatory care–sensitive conditions can be avoided.  Pediatrics. 2003;112(5):1021-1030
PubMed   |  Link to Article
Salmon DA, Moulton LH, Omer SB,  et al.  Knowledge, attitudes, and beliefs of school nurses and personnel and associations with nonmedical immunization exemptions.  Pediatrics. 2004;113(6):e552-e559.http://pediatrics.aappublications.org/content/113/6/e552.long. Accessed April 22, 2012
PubMed   |  Link to Article
Downey L, Tyree PT, Huebner CE, Lafferty WE. Pediatric vaccination and vaccine-preventable disease acquisition: associations with care by complementary and alternative medicine providers.  Matern Child Health J. 2010;14(6):922-930
PubMed   |  Link to Article
Omer SB, Pan WK, Halsey NA,  et al.  Nonmedical exemptions to school immunization requirements: secular trends and association of state policies with pertussis incidence.  JAMA. 2006;296(14):1757-1763
PubMed   |  Link to Article
Salmon DA, Haber M, Gangarosa EJ, Phillips L, Smith NJ, Chen RT. Health consequences of religious and philosophical exemptions from immunization laws: individual and societal risk of measles.  JAMA. 1999;282(1):47-53
PubMed   |  Link to Article
Feikin DR, Lezotte DC, Hamman RF, Salmon DA, Chen RT, Hoffman RE. Individual and community risks of measles and pertussis associated with personal exemptions to immunization.  JAMA. 2000;284(24):3145-3150
PubMed   |  Link to Article
Omer SB, Enger KS, Moulton LH, Halsey NA, Stokley S, Salmon DA. Geographic clustering of nonmedical exemptions to school immunization requirements and associations with geographic clustering of pertussis.  Am J Epidemiol. 2008;168(12):1389-1396
PubMed   |  Link to Article
Glanz JM, McClure DL, Magid DJ,  et al.  Parental refusal of pertussis vaccination is associated with an increased risk of pertussis infection in children.  Pediatrics. 2009;123(6):1446-1451
PubMed   |  Link to Article
Glanz JM, McClure DL, Magid DJ, Daley MF, France EK, Hambidge SJ. Parental refusal of varicella vaccination and the associated risk of varicella infection in children.  Arch Pediatr Adolesc Med. 2010;164(1):66-70
PubMed   |  Link to Article
Glanz JM, McClure DL, O’Leary ST,  et al.  Parental decline of pneumococcal vaccination and risk of pneumococcal related disease in children.  Vaccine. 2011;29(5):994-999
PubMed   |  Link to Article
 National Vaccine Advisory Committee (NVAC) recommendations on the Centers for Disease Control and Prevention Immunization Safety Office draft 5-year scientific agenda. June 2009. http://www.hhs.gov/nvpo/nvac/meetings/pastmeetings/nvacrecommendationsisoscientificagenda.pdf. Accessed July 25, 2012

Figures

Place holder to copy figure label and caption
Graphic Jump Location

Figure 1. Identifying matched analytic cohorts of undervaccinated children. To be eligible for the study, children had to be born between 2004 and 2008, be continuously enrolled in their managed care site from at least ages 2 to 12 months, and have at least 1 outpatient visit by age 12 months. Children with certain contraindications to vaccinations were excluded from the study. *Children were categorized as having a diagnosis code for vaccine refusal if they had a V64.05 or V64.06 International Classification of Diseases, Ninth Revision, Clinical Modification, code in their electronic medical record before age 24 months.

Place holder to copy figure label and caption
Graphic Jump Location

Figure 2. Trends in delayed start to vaccinations and select vaccination patterns by birth cohort before age 2 years. Across all birth cohorts, 8939 children (2.8%) were on specific nonstandard vaccination schedules. All trends were significant at P < .001 using the Cochran-Armitage trend test. DTaP indicates diphtheria, tetanus, and pertussis vaccine; Hib, Haemophilus influenzae type b vaccine; and MMR, measles, mumps, and rubella vaccine.

Tables

Table Graphic Jump LocationTable 1. Advisory Committee on Immunization Practices' Recommended Early Childhood Vaccines and Doses Studieda
Table Graphic Jump LocationTable 2. Confirmation Rates of Medical Record–Confirmed Parental Delay or Refusal of Vaccination by Decile of Undervaccination
Table Graphic Jump LocationTable 3. Trends in Undervaccination by Birth Cohort
Table Graphic Jump LocationTable 4. Health Care Utilization From Birth to 36 Months in 137855 Undervaccinated Children Without an ICD-9-CM Code for Parental Vaccine Refusal Matched to 137855 Children With On-Time Vaccinations
Table Graphic Jump LocationTable 5. Health Care Utilization From Birth to 36 Months in 6168 Undervaccinated Children With an ICD-9-CM Code for Parental Vaccine Refusal Matched to 69209 Children With On-Time Vaccinations

References

Dempsey AF, Schaffer S, Singer D, Butchart A, Davis M, Freed GL. Alternative vaccination schedule preferences among parents of young children.  Pediatrics. 2011;128(5):848-856
PubMed   |  Link to Article
Freed GL, Clark SJ, Butchart AT, Singer DC, Davis MM. Parental vaccine safety concerns in 2009.  Pediatrics. 2010;125(4):654-659
PubMed   |  Link to Article
Gust DA, Darling N, Kennedy A, Schwartz B. Parents with doubts about vaccines: which vaccines and reasons why.  Pediatrics. 2008;122(4):718-725
PubMed   |  Link to Article
Wightman A, Opel DJ, Marcuse EK, Taylor JA. Washington State pediatricians' attitudes toward alternative childhood immunization schedules.  Pediatrics. 2011;128(6):1094-1099
PubMed   |  Link to Article
Leib S, Liberatos P, Edwards K. Pediatricians' experience with and response to parental vaccine safety concerns and vaccine refusals: a survey of Connecticut pediatricians.  Public Health Rep. 2011;126:(suppl 2)  13-23
PubMed
Centers for Disease Control and Prevention.  Recommended immunization schedules for persons aged 0-18 years—United States, 2011.  MMWR Morb Mortal Wkly Rep. 2011;60(5):1-4.http://www.cdc.gov/mmwr/preview/mmwrhtml/mm6005a6.htm?s_cid=mm6005a6_w. Accessed November 28, 2012
Baggs J, Gee J, Lewis E,  et al.  The Vaccine Safety Datalink: a model for monitoring immunization safety.  Pediatrics. 2011;127:(suppl 1)  S45-S53
PubMed   |  Link to Article
Centers for Disease Control and Prevention.  Guide to vaccine contraindications and precautions. http://www.cdc.gov/vaccines/recs/vac-admin/downloads/contraindications-guide-508.pdf. Accessed July 25, 2012
France EK, Smith-Ray R, McClure D,  et al.  Impact of maternal influenza vaccination during pregnancy on the incidence of acute respiratory illness visits among infants.  Arch Pediatr Adolesc Med. 2006;160(12):1277-1283
PubMed   |  Link to Article
Ritzwoller DP, Bridges CB, Shetterly S, Yamasaki K, Kolczak M, France EK. Effectiveness of the 2003-2004 influenza vaccine among children 6 months to 8 years of age, with 1 vs 2 doses.  Pediatrics. 2005;116(1):153-159
PubMed   |  Link to Article
Luman ET, Barker LE, Shaw KM, McCauley MM, Buehler JW, Pickering LK. Timeliness of childhood vaccinations in the United States: days undervaccinated and number of vaccines delayed.  JAMA. 2005;293(10):1204-1211
PubMed   |  Link to Article
Sears RW. The Vaccine Book: Making the Right Decision for Your Child. New York, NY: Little, Brown; 2007
Parashar UD, Alexander JP, Glass RI.Advisory Committee on Immunization Practices (ACIP), Centers for Disease Control and Prevention (CDC).  Prevention of rotavirus gastroenteritis among infants and children. Recommendations of the Advisory Committee on Immunization Practices (ACIP).  MMWR Recomm Rep. 2006;55(RR-12):1-13
PubMed
Centers for Disease Control and Prevention.  Notice to readers: limited supply of pneumococcal conjugate vaccine: suspension of recommendation for fourth dose.  MMWR Morb Mortal Wkly Rep. 2004;53(5):108-109.http://www.cdc.gov/mmwr/preview/mmwrhtml/mm5305a6.htm. Accessed November 28, 2012
Centers for Disease Control and Prevention (CDC).  Updated recommendations on the use of pneumococcal conjugate vaccine: suspension of recommendation for third and fourth dose.  MMWR Morb Mortal Wkly Rep. 2004;53(8):177-178
PubMed
Centers for Disease Control and Prevention.  Notice to readers: pneumococcal conjugate vaccine shortage resolved.  MMWR Morb Mortal Wkly Rep. 2004;53(36):851-852.http://www.cdc.gov/mmwr/preview/mmwrhtml/mm5336a8.htm. Accessed November 28, 2012
Marin M, Güris D, Chaves SS, Schmid S, Seward JF.Advisory Committee on Immunization Practices, Centers for Disease Control and Prevention (CDC).  Prevention of varicella: recommendations of the Advisory Committee on Immunization Practices (ACIP).  MMWR Recomm Rep. 2007;56(RR-4):1-40
PubMed
Centers for Disease Control and Prevention (CDC).  Interim recommendations for the use of Haemophilus influenzae type b (Hib) conjugate vaccines related to the recall of certain lots of Hib-containing vaccines (PedvaxHIB and Comvax).  MMWR Morb Mortal Wkly Rep. 2007;56(50):1318-1320
PubMed
Centers for Disease Control and Prevention (CDC).  Updated recommendations for use of Haemophilus influenzae type b (Hib) vaccine: reinstatement of the booster dose at ages 12-15 months.  MMWR Morb Mortal Wkly Rep. 2009;58(24):673-674
PubMed
Cochran WG. Sampling Techniques. 3rd ed. New York, NY: John Wiley & Sons; 1977
Offit PA, Moser CA. The problem with Dr Bob's alternative vaccine schedule.  Pediatrics. 2009;123(1):e164-e169
PubMed  |  Link to Article   |  Link to Article
Glanz JM, Newcomer SR, Hambidge SJ,  et al.  Safety of trivalent inactivated influenza vaccine in children aged 24 to 59 months in the Vaccine Safety Datalink.  Arch Pediatr Adolesc Med. 2011;165(8):749-755
PubMed   |  Link to Article
Gay JC, Muldoon JH, Neff JM, Wing LJ. Profiling the health service needs of populations: description and uses of the NACHRI Classification of Congenital and Chronic Health Conditions.  Pediatr Ann. 1997;26(11):655-663
PubMed
Centers for Disease Control and Prevention.  National and state vaccination coverage among children aged 19-35 months—United States, 2010.  MMWR Morb Mortal Wkly Rep. 2011;60(34):1157-1163.http://www.cdc.gov/mmwr/preview/mmwrhtml/mm6034a2.htm?s_cid=mm6034a2_w. Accessed April 27, 2012
Robison SG, Groom H, Young C. Frequency of alternative immunization schedule use in a metropolitan area.  Pediatrics. 2012;130(1):32-38
PubMed   |  Link to Article
Kulldorff M. Study designs for the safety evaluation of different childhood immunization schedules. http://www.iom.edu/~/media/Files/Activity%20Files/PublicHealth/ChildhoodImmunization/Commissioned%20Paper/ReportWebpost%20515.pdf. Accessed July 25, 2012
Agency for Healthcare Research and Quality.  National Healthcare Disparities Report 2009. http://www.ahrq.gov/qual/nhdr09/nhdr09.pdf. Accessed July 25, 2012
Herrod HG, Chang CF. Potentially avoidable pediatric hospitalizations as defined by the Agency for Healthcare Research and Quality: what do they tell us about disparities in child health?  Clin Pediatr (Phila). 2008;47(2):128-136
PubMed   |  Link to Article
Yousey-Hindes KM, Hadler JL. Neighborhood socioeconomic status and influenza hospitalizations among children: New Haven County, Connecticut, 2003-2010.  Am J Public Health. 2011;101(9):1785-1789
PubMed
Flores G, Abreu M, Chaisson CE, Sun D. Keeping children out of hospitals: parents' and physicians' perspectives on how pediatric hospitalizations for ambulatory care–sensitive conditions can be avoided.  Pediatrics. 2003;112(5):1021-1030
PubMed   |  Link to Article
Salmon DA, Moulton LH, Omer SB,  et al.  Knowledge, attitudes, and beliefs of school nurses and personnel and associations with nonmedical immunization exemptions.  Pediatrics. 2004;113(6):e552-e559.http://pediatrics.aappublications.org/content/113/6/e552.long. Accessed April 22, 2012
PubMed   |  Link to Article
Downey L, Tyree PT, Huebner CE, Lafferty WE. Pediatric vaccination and vaccine-preventable disease acquisition: associations with care by complementary and alternative medicine providers.  Matern Child Health J. 2010;14(6):922-930
PubMed   |  Link to Article
Omer SB, Pan WK, Halsey NA,  et al.  Nonmedical exemptions to school immunization requirements: secular trends and association of state policies with pertussis incidence.  JAMA. 2006;296(14):1757-1763
PubMed   |  Link to Article
Salmon DA, Haber M, Gangarosa EJ, Phillips L, Smith NJ, Chen RT. Health consequences of religious and philosophical exemptions from immunization laws: individual and societal risk of measles.  JAMA. 1999;282(1):47-53
PubMed   |  Link to Article
Feikin DR, Lezotte DC, Hamman RF, Salmon DA, Chen RT, Hoffman RE. Individual and community risks of measles and pertussis associated with personal exemptions to immunization.  JAMA. 2000;284(24):3145-3150
PubMed   |  Link to Article
Omer SB, Enger KS, Moulton LH, Halsey NA, Stokley S, Salmon DA. Geographic clustering of nonmedical exemptions to school immunization requirements and associations with geographic clustering of pertussis.  Am J Epidemiol. 2008;168(12):1389-1396
PubMed   |  Link to Article
Glanz JM, McClure DL, Magid DJ,  et al.  Parental refusal of pertussis vaccination is associated with an increased risk of pertussis infection in children.  Pediatrics. 2009;123(6):1446-1451
PubMed   |  Link to Article
Glanz JM, McClure DL, Magid DJ, Daley MF, France EK, Hambidge SJ. Parental refusal of varicella vaccination and the associated risk of varicella infection in children.  Arch Pediatr Adolesc Med. 2010;164(1):66-70
PubMed   |  Link to Article
Glanz JM, McClure DL, O’Leary ST,  et al.  Parental decline of pneumococcal vaccination and risk of pneumococcal related disease in children.  Vaccine. 2011;29(5):994-999
PubMed   |  Link to Article
 National Vaccine Advisory Committee (NVAC) recommendations on the Centers for Disease Control and Prevention Immunization Safety Office draft 5-year scientific agenda. June 2009. http://www.hhs.gov/nvpo/nvac/meetings/pastmeetings/nvacrecommendationsisoscientificagenda.pdf. Accessed July 25, 2012

Correspondence

CME
Meets CME requirements for:
Browse CME for all U.S. States
Accreditation Information
The American Medical Association is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians. The AMA designates this journal-based CME activity for a maximum of 1 AMA PRA Category 1 CreditTM per course. Physicians should claim only the credit commensurate with the extent of their participation in the activity. Physicians who complete the CME course and score at least 80% correct on the quiz are eligible for AMA PRA Category 1 CreditTM.
Note: You must get at least of the answers correct to pass this quiz.
You have not filled in all the answers to complete this quiz
The following questions were not answered:
Sorry, you have unsuccessfully completed this CME quiz with a score of
The following questions were not answered correctly:
Commitment to Change (optional):
Indicate what change(s) you will implement in your practice, if any, based on this CME course.
Your quiz results:
The filled radio buttons indicate your responses. The preferred responses are highlighted
For CME Course: A Proposed Model for Initial Assessment and Management of Acute Heart Failure Syndromes
Indicate what changes(s) you will implement in your practice, if any, based on this CME course.
NOTE:
Citing articles are presented as examples only. In non-demo SCM6 implementation, integration with CrossRef’s "Cited By" API will populate this tab (http://www.crossref.org/citedby.html).
Submit a Comment

Multimedia

Some tools below are only available to our subscribers or users with an online account.

Web of Science® Times Cited: 10

Related Content

Customize your page view by dragging & repositioning the boxes below.

See Also...
Articles Related By Topic
Related Topics
PubMed Articles