The Impact of Reminder-Recall Interventions on Low Vaccination Coverage in an Inner-City Population FREE

During the measles resurgence of the early 1990s, most unvaccinated patients resided in urban counties with low vaccination levels, where minority children were at highest risk for disease.^{1 - 2} During the past decade, hemisphere-wide efforts have brought measles disease to historic lows in the Americas³ and close to elimination in the United States.⁴ However, low vaccination levels continue to be documented in US urban areas, particularly among minority children.^{5 - 7} This raises the worrisome possibility that outbreaks of measles or other vaccine-preventable diseases could recur, as has happened in localities with low coverage in Germany,⁸ the Netherlands,⁹ the United Kingdom,^{10 - 11} and Italy.¹²

Numerous studies have established that the simple process of reminding families when a vaccination is due and then recalling them when a vaccination is missed (reminder recall) can raise vaccination rates at relatively low cost when automated systems are used.¹³ The authors of a systematic review of the literature concluded that "reminder/recall systems appear to be effective in all settings that were evaluated."^{13 (p1826)} Reminder recall has been recommended by the American Academy of Pediatrics,¹⁴ the American Medical Association,¹⁴ the Task Force on Community Preventive Services,¹⁵ the Advisory Committee on Immunization Practices (ACIP),¹⁶ and the National Vaccine Advisory Committee.¹⁷

However, whether reminder recall can meaningfully raise low inner-city coverage levels has never been tested on a large scale. One obstacle has been the expense and logistics of installing and maintaining separate reminder-recall systems in a multitude of urban clinics and practices.

Provider measurement and feedback improved immunization rates in Georgia public clinics,¹⁸ but inner-city Atlanta lagged behind despite innovative efforts by a community-based organization.¹⁹ In 1993, an alliance of community groups established the Metro Atlanta Team for Child Health (MATCH) immunization registry. Using this registry, we conducted a randomized, controlled, effectiveness trial with the primary objective of testing whether large-scale reminder recall could meaningfully raise low inner-city immunization rates. A secondary objective was to compare the impact and costs of different forms of reminder recall: in-person telephone calls, mailings, and home visits vs computer-generated telephone calls and mailings.

The enrollment goal was to capture as large a proportion of the public sector birth cohort of inner-city Atlanta as programmatically feasible. Fulton County comprises most of inner-city Atlanta, and data about any child vaccinated from birth onward in any Fulton County clinic were entered daily by clinic personnel into an all-electronic vaccination record, with data downloaded weekly to the MATCH immunization registry. Although the total number of other providers who might potentially vaccinate such children was unknown, the MATCH registry had the participation of the largest vaccination providers of the Atlanta metropolitan area: the public health clinics of DeKalb County (the other county that includes part of Atlanta), the area's federally qualified community health centers, the 2 major private pediatric hospitals with their outpatient and satellite centers, the 2 major academic medical facilities, the major Roman Catholic hospital with its outpatient and outreach facilities, and a number of large private practices. At study initiation, 3050 children in the MATCH registry met the following criteria and were enrolled in the study: (1) resided in Fulton County, (2) had received care through the Fulton County health department clinics or the public hospital health system, and (3) were born between July 1, 1995, and August 6, 1996.

Immunizations included in interventions were those of the 4-3-1-3 series: 4 doses of diphtheria and tetanus toxoids and pertussis vaccine (DTP) in either the whole cell or acellular pertussis formulations, 3 doses of polio vaccine in either the live oral or inactivated injectable formulation, 1 dose of measles-mumps-rubella vaccine (MMR), and 3 doses of Haemophilus influenzae type B vaccine (HIB). The schedule followed ACIP recommendations,²⁰ as applied by the Fulton County health department: the first 3 doses, DTP, polio, and HIB, were considered due at 2, 4, and 6 months of age and overdue if they had not been received by a month later; MMR and the fourth dose of DTP were considered due at 12 months of age and overdue if they had not been received by 16 months of age. Age-appropriate vaccination rates were calculated, allowing a 1-month period for the child to be vaccinated after a dose was due. Calculation of minimum ages and intervals between doses followed ACIP standards²¹ ; doses were considered invalid if they were administered more than 4 days before the minimum age or interval. Because this was an effectiveness trial of registry-based interventions, no immunization records separate from the registry were maintained.

All intervention contact attempts and outcomes were recorded in a study database.

Seven days before a dose was due, a computer connected to a telephone delivered a recorded message to the family from the Fulton County health department staff, indicating that the child should be taken to his or her health care provider for the needed dose. If there was no answer or a busy signal, the call was repeated every 30 to 60 minutes. If these efforts failed to reach a person or an answering machine or if the telephone number was nonworking or not present in the database, an automated postcard with the same message was mailed to the family no later than 5 days before the due date. If 6 days after the due date the needed dose was not present in the registry, a computerized telephone message (or postcard in the absence of a working telephone) was sent to the family indicating that the child was behind in his or her immunizations. Unless the registry recorded the immunization, the telephone message was repeated on days 11, 17, and 23. If these efforts failed, a computerized postcard was sent on day 28. All telephone calls were made between 5:30 and 9:00 PM. At the start of each message, an option for a Spanish-language version was presented, and postcards contained the message in both Spanish and English.

Within 7 days of a child failing to receive a dose by the due date, the outreach worker attempted to contact the family by telephone or postcard in the absence of a working telephone. If 7 days later the dose was still not in the registry, a postcard was sent. If 30 days later the dose was still missing, a home visit was attempted, with continued monthly efforts until contact was made. At the home visit, the outreach worker attempted to determine what was needed to assist the family in obtaining immunization for the child. The principal outreach worker was a college-educated, African American woman who had been raised in inner-city Atlanta. For Hispanic families, outreach was provided by a bilingual, college-educated, Hispanic worker. The outreach workers and other study functions were supervised by a person with a doctorate in community psychology and extensive experience in conducting inner-city studies (D.M.S.).

At study initiation, participants were assigned by computer-generated random numbers to 1 of 4 groups: autodialer only, outreach only, combination (outreach for children not vaccinated after completion of the autodialer protocol), or control (no interventions beyond normal clinic procedure, which in certain cases involved nonautomated postcard recall systems). The study population of 3050 provided 80% power for detection of 5% differences in immunization rates among groups. We did not attempt blinding or detection of adverse events arising from the reminder-recall process or vaccinations. Interventions began for all participants on September 9, 1996, and continued until each child reached 24 months of age, ending for the last participant on August 6, 1998. Follow-up contact with study subjects ended August 6, 1999, and electronic acquisition of vaccination information ended February 1, 2001. The study was approved by the human subjects procedures of the Centers for Disease Control and Prevention, Emory University, and the MATCH registry.

All analyses were based on intent to treat. Vaccinations from all providers, public and private, received by study children by 24 months of age and present in the MATCH registry as of February 1, 2001, were evaluated. For comparison of categorical variables, we used the 2-tailed Fisher exact test. For continuous variables, we used the median and interquartile distance and made comparisons using the Wilcoxon rank sum test. Baseline characteristics of the study population were compared with data from the Georgia Department of Vital Records for Fulton County births and from the National Immunization Survey for Georgia.²²

The main study outcome was 4-3-1-3 series completion by 24 months of age for valid vaccination doses. We also examined other vaccination indices: age-specific, antigen-specific, and dose-specific coverage rates, the number of valid and invalid doses received, coverage rates with invalid doses included, the number of vaccination visits made, the number of missed opportunities for simultaneous vaccination, and the average lag time in days from due date to receipt of a valid dose. We dichotomized undervaccinated children into 1 of 2 categories: missed opportunity (children with ≥4 vaccination visits adequately spaced to achieve series completion) or missed visit (children with <4 such visits).

The group to which the child was assigned was treated as the primary risk factor for immunization outcomes. We also examined intervention exposure, defined as a telephone call that reached a person or an answering machine, a letter that was not returned, or a home visit that resulted in a face-to-face interaction or in a note left at the residence where the family was known to reside. Intervention duration was examined as both a continuous variable (months of exposure to the intervention) and a dichotomous variable (exposure greater or less than the median).

Race and sex were the only demographic attributes available in the registry.

Of the nearly quarter-million children in the registry, 3050 met criteria and were included in the study. The study population represented 27% of the 11 300 Fulton County births for the same period (Figure 1).

Table 1 shows the study population characteristics. The study population ranged in age from 1 to 14 months (median, 9 months), with no significant difference between the intervention and control groups for any demographic or vaccination characteristic. The proportion of blacks was significantly higher than in the county birth cohort (76% vs 55%, P<.001), and minority children constituted 93% of the study population overall. By inclusion criteria, 100% of the study population had been vaccinated in the public sector, significantly higher than the 32% ± 5% of Georgia children aged 19 to 35 months in 1997 according to the National Immunization Survey.²²

Of the 2287 children assigned to an intervention group, 304 children (13%) were not exposed to the intervention, of whom 68 (22%) maintained age-appropriate vaccination status in advance of the need for the intervention, 78 (26%) were documented to have moved outside the Atlanta metropolitan area, and 158 (52%) could not be located. The duration of intervention varied from 10 to 23 months, depending on the age of the child at the start of the study, with a median of 15 months and no significant difference among groups. As shown in (Table 2), intervention exposure was high for all groups (85%-89%), but the median count of successful contacts per child in the outreach group was less than half that of the other 2 intervention groups with computerized systems (2 vs 5, P<.001). Of the 1524 children in the outreach and combination groups, home visits were attempted with 647 (42%) and were successful with 401 (26%).

Overall rates fell in a stepwise fashion from a high of 87% at 4 months of age (only 3 doses required) to 17% at 16 months of age (all 11 doses required), then slowly increased (no further doses required) to 38% at 24 months (Figure 2). Among the groups, rates were tightly grouped until 19 months of age when the intervention groups began to maintain slightly higher (3%-8%) rates than the control group, with the difference significant only for the autodialer group between 21 and 24 months of age (P<.04 for each month).

As shown in (Table 3) the 3 intervention groups had series completion rates 3% to 6% higher than the control group, but this was significant only for the autodialer group (P = .02). Individual intervention groups did not differ significantly from each other. Dose-specific coverage in the intervention groups was not significantly different from the control group for any of the 11 doses in the 4-3-1-3 series (difference, ≤3%), except the fourth DTP dose for one group(42% for the autodialer group vs 36% for the control group, P = .02). Within the intervention group, coverage did not vary significantly by intervention exposure (exposed, 38% vs nonexposed, 40%; P = .49), nor was it associated with duration of time spent in the interventions treated as either a continuous (P = .91) or a dichotomized variable (longer vs shorter than median duration: 38% vs 39%, P = .42). For both sexes, intervention groups had 3% to 7% higher coverage than the control limb. With their predominant numbers, the black population showed the same patterns as the total study population; the rest of the study population showed no significant intervention effect. Counts of visits made and doses administered during the study period did not differ significantly among the groups, nor did lag time to vaccination, missed opportunities, or invalid doses. When invalid doses were included in coverage and other indices, the findings did not change significantly. Of those who did not complete the series at study end, 55% needed 1 additional visit to finish the series, 21% needed 2 additional visits, and 25% needed more. Missed visits, rather than missed opportunities, accounted for at least 90% of failures to complete the vaccination series in each group, with no significant difference between the intervention groups and the control group.

In our study, large-scale, registry-based reminder-recall interventions produced only slight improvements (3%-6%) in immunization rates among a poorly vaccinated inner-city population. To the extent to which a statistically significant improvement in coverage took place, it was restricted to 1 dose (DTP dose 4) in 1 group (autodialer). Dose counts, visit counts, and lag time to vaccination did not differ from intervention to control. Neither exposure nor duration of exposure to the interventions significantly changed any outcome.

Several factors suggest that the situation may not have been intractable. At study start, the intervention and control groups were essentially identical for all demographic and vaccination characteristics, and early age-appropriate vaccination coverage in the population was high. During the 10 to 23 months in which children received interventions, most children needed only 1 or 2 visits to be brought to series completion, and almost all children were successfully exposed to the interventions, most on multiple occasions. Although missed opportunities occurred and invalid doses were administered, the overwhelming reason for children's failure to complete the vaccination series was missed visits, the specific target of the reminder-recall intervention.

The 5% improvement of the aggregated intervention groups compared with the control group falls into the 5% to 20% impact range found by one group of investigators in a systematic review of the reminder-recall literature.¹³ However, statistical significance is not the same as public health impact. Several other studies have found slight to no improvement in coverage when reminder recall was used with inner-city populations.^{23 - 26} We could not identify any study in the literature that demonstrated that low inner-city immunization rates could be raised to levels approaching immunization goals²⁷ by reminder recall alone.

As we have previously reported, the cost per child of delivering the interventions (independent of registry costs) was $16.08 for the autodialer group, $22.44 for the outreach group, and $33.12 for the combination group.²⁸ Hence, the autodialer provided at least as much vaccination impact as more costly interventions involving in-person calls, mailings, and visits. The direct annual cost of maintaining the MATCH registry was $5.26 per child,²⁹ and the cost to each provider of participating in the registry ranged from $0.65 to $7.74 per child.³⁰ Thus, the total cost per child of delivering the interventions through the registry was approximately $25 to $45 annually. However, based on a 3% to 6% intervention impact, the annual cost was approximately $400 to $1500 per additional child vaccinated.

Our study has a number of limitations. Because we did not evaluate interventions outside the registry or maintain a database separate from the registry, we cannot determine conclusively whether the relative failure of the interventions arose from the registry or the interventions. We lack data about vaccinations by providers not participating in the registry, and the vaccination rates in the study may be artefactually low as a consequence. Registry inaccuracies, such as have been documented in other studies,^{31 - 35} may also have diluted the impact of effective interventions by misdirecting efforts away from the undervaccinated to fully vaccinated children.

For programmatic reasons, we limited the study to the birth cohort vaccinated in Fulton County clinics, and although we believe we captured this population essentially in toto, it may not have been representative of the public sector birth cohort of the whole Atlanta metropolitan area at large. We did not evaluate the extent to which families may have responded to the interventions but were denied immunizations by their providers. Some vaccination providers had recall systems, and, from time to time, public officials, community leaders, and the media urged families to get their children vaccinated. To the extent to which children in the control group were exposed to such efforts and an effect occurred, the study's ability to detect an intervention effect may have been reduced.

Because inner-city children reach series completion by school age,²⁷ it is apparent that barriers to immunization can be overcome. Among inner-city preschool children, voucher incentives in the Special Supplemental Program for Women's, Infants, and Children (WIC) have rapidly raised inner-city coverage to high levels on a large scale.^{36 - 39} Verbal reminders, in contrast to school laws or voucher incentives, may be insufficient to prompt large numbers of inner-city families to divert time and resources from the challenges and crises of life in poverty to ensuring that each child is vaccinated on time. To help protect such high-risk children and society in general from vaccine-preventable outbreaks, interventions that are efficacious in small studies or with low-risk groups should be retested on a large scale to determine if they are effective and cost-effective in inner-city environments.

Reminder-recall interventions have improved immunization rates in numerous studies, but the broad impact on low inner-city coverage has not previously been studied, to our knowledge. In our study, registry-based reminder-recall interventions produced only small improvements in low immunization rates of a large, inner-city population. This suggests that interventions efficacious in small studies or with low-risk groups should be retested to determine if they are meaningfully effective on a large scale in populations at highest risk for disease.

Corresponding author: Charles W. LeBaron, MD, Mail Stop E-61, NIP/CDC, Atlanta, GA 30333 (e-mail: clebaron@cdc.gov).

Accepted for publication July 23, 2003.

This study was funded by the National Immunization Program, Centers for Disease Control and Prevention, and the Georgia Department of Human Resources, Atlanta. We thank Michael Chaney, MBA, Denis Taylor, BA, Sharon Myrick, BA, Kristen Wells, MPH, and all the families and immunization providers participating in the MATCH immunization registry for their important contributions to this study.