The Effect of Parental Monetary Sanctions on the Vaccination Status of Young Children:  An Evaluation of Welfare Reform in Maryland FREE

THE PERSONAL Responsibility and Work Opportunity Reconciliation Act of 1996 culminated decades of legislative efforts to reform the US welfare system. The demand for welfare recipients to exercise responsibility in family- and work-related decisions surfaced in the 1970s and has pervaded the political debate ever since.¹ The 1980s witnessed the increasing use of social contract proposals and widespread implementation of "workfare." Many states, scrambling to obtain waivers from federal regulations, elected to experiment with behavior-based reform. Accordingly, states increasingly conditioned levels of cash assistance on socially desired behavioral prerequisites.

In 1992, Maryland obtained an agreement from the US Department of Health and Human Services to undertake a welfare reform demonstration project, the Primary Prevention Initiative (PPI). The PPI was a behavior-based strategy designed to promote parental responsibility, accountability, and self-sufficiency. Under the health component, approximately 75,000 welfare recipients throughout the state were subject to a $25 monthly penalty for failure to verify that their preschool-aged children received preventive health care services, including vaccinations.²

In selected geographic areas, families receiving Aid to Families With Dependent Children (AFDC) randomly were assigned to an experimental or a control group. All welfare recipients outside these selected areas, as well as those assigned to the experimental group, were subject to the same PPI requirements, including the monthly penalty. Welfare recipients in the control group were not subject to the penalty. The Maryland Department of Human Resources designed the PPI and directed its implementation.^3- 4

We report on whether the PPI led to increased use of preventive health care services for children aged 3 to 24 months. It was hypothesized that the threat of monetary reductions would motivate families to seek preventive health care and thereby improve health care utilization and vaccination coverage among young children. Because of their potential influence on vaccination coverage, missed opportunities by providers to vaccinate⁵ also were evaluated.

Families eligible for the evaluation were served by 1 of 6 Maryland Department of Social Services offices, 4 in metropolitan areas and 2 in rural counties.

Maryland Department of Human Resources staff conducted random assignment in 2 stages. First, in June 1992, all families receiving AFDC from the 6 evaluation offices were assigned randomly to the experimental or the control group. Second, from January 1993 through August 1995, all families applying for AFDC at the 6 offices were assigned randomly to the experimental or the control group. Enrollment ended with 17,524 families in the evaluation. As specified by the federal agreement, the experimental group was twice as large as the control group.⁶

When a family applied for AFDC, the case worker reviewed the PPI requirements. Agreements with Medical Assistance (MA) providers specified that children be seen within a specified time and that sanctioned clients be given priority. At each 6-month redetermination, the case worker ascertained whether the client met the requirements. If clients did not provide verification, they might elect to delay the disallowance for good cause. Good-cause exemptions could last up to 3 months, with a total limit of 2. Noncompliant families were sent an official notice before penalties were imposed. Annually, approximately 25% of intervention families received 1 or more disallowances for failing to meet the preventive health requirements. The disallowance was not necessarily for the study child, since disallowances were tracked by family, not by individual child.

The PPI evaluation met the requirements of the Committee on Human Research at The Johns Hopkins School of Public Health, the University of Baltimore, and the Maryland Department of Health and Mental Hygiene, Baltimore.

Two data sources were required to identify children eligible for sampling. The Maryland Department of Human Resouces provided an electronic list of the AFDC families assigned to the experimental or the control group, MA identification number, evaluation office, group assignment, and the fiscal quarter and year when the assignment was made. These data were linked with the MA eligibility file of the Maryland Department of Health and Mental Hygiene to identify the children eligible for each of the following 3 fiscal year cohorts: 1992 (baseline year), 1993 (year 1), and 1994 (year 2). Children eligible for a fiscal year cohort had to be from 1 of the 6 evaluation jurisdictions, enrolled in MA for at least 6 months of the fiscal year, and 3 to 24 months of age.

We first randomly selected 650 children (or the universe) from each of the 5 comparison groups (baseline, year 1 experimental and control, and year 2 experimental and control) who met the eligibility criteria (Figure 1 and Figure 2). Siblings were identified, and 1 sibling was selected randomly for retention. Six hundred of the retained children (or the universe) per comparison group were then randomly selected for the evaluation.

This sample size was adequate to detect 10% differences in vaccination status with a power of 90% between experimental and control groups in a fiscal year or between the baseline and an experimental or a control group. A child may have been selected for more than 1 cohort; however, there was no attempt to link children across the yearly cohorts.

The MA eligibility record of each child randomly selected was linked with the child's individual MA outpatient claims or health maintenance organization record. A list of all providers used by the selected children and listed in the claims files was generated.

Medical records were abstracted from primary care providers. Nine trained abstractors, unaware of children's experimental or control group assignment, recorded demographic information, all vaccinations, and visits.

Medical records data were edited, entered, verified, and checked for internal consistency. When medical records at more than 1 provider site were abstracted for a child, a composite vaccination history was assembled. Several outcome measures were defined.

Because not all children were eligible to be observed for the entire fiscal year, the mean number of visits per child was expressed in visits per person-years. A child eligible for only 6 months of the fiscal year contributed 0.5 person-years. Visits occurring within the fiscal year but before or after a child was eligible were excluded. Well-child visits included visits for health supervision, and sick visits included visits for injury or illness.

The percentage of children who made at least 1 well-child visit during the fiscal year and who were eligible for the PPI evaluation during the entire 12 months of the fiscal year also was determined.

Up-to-date for age implied that all recommended vaccinations for a child of a particular age had been given. The definitions of up-to-date are based on the Red Book recommendations of the American Academy of Pediatrics.^7- 8

A missed opportunity occurred when the child was age-eligible for a vaccine and had no contraindication for the vaccine, but the vaccine was not administered. Missed opportunities may occur at any type of visit, including sick visits, and more than 1 missed opportunity may take place at a visit if more than 1 vaccine was due. Age eligibility and contraindications also were based on the Red Book recommendations.^7- 8

The analysis excluded children whose parents refused consent for data collection. We also excluded children for whom medical records were not available because each of their providers met 1 of the following criteria: was a non–primary care provider, refused to participate, had closed the practice, could not be located, had no record of the child's enrollment, could not locate the child's medical record, or was a health maintenance organization from which the child was disenrolled. If at least 1 provider did not fit the exclusion criteria, the child was eligible to be in the denominator, whether or not the child made a visit to the provider during the fiscal year. For each denominator-eligible child, record abstraction was attempted at each primary care site the child visited and from each primary care provider who made home visits during the fiscal year. Children for whom medical records at 1 or more primary care sites were not abstracted because of time constraints also were excluded (Figure 1 and Figure 2).

Data analyses were conducted using commercially available software (SAS-PC; SAS Institute Inc, Cary, NC). Proportions were compared using χ² analyses. Mean visits per person-year were compared using techniques described by Rothman.⁹ The level of statistical significance was P = .05. The proportion of children up-to-date for age during the baseline year was compared with the proportion up-to-date for age in the experimental group during year 2. The proportions of children up-to-date for age in the experimental and control groups also were compared for each of the 2 postintervention years. Similar comparisons of health care visits and missed opportunities to vaccinate were made.

Almost 2000 children were eligible for the baseline evaluation group (n = 1928) and for year 1 (n = 1990), and 3075 met the eligibility criteria for year 2 (Figure 1 and Figure 2). The increase in the number of children meeting the eligibility criteria between years 1 and 2 can be attributed to selection of the year 1 sample before full implementation of the PPI. For years 1 and 2, the ratio of children in the experimental vs control groups was 2:1. Subsequently, 650 children were selected randomly for the baseline group, year 1 experimental group, and both year 2 groups; all 624 eligible year 1 controls were selected. Siblings were excluded, and 600 children per group, or the universe (n = 583) for the year 1 control group, were selected randomly for a total of 2983 children.

Overall, 737 (24.7%) of these 2983 children were excluded from the analysis. Only 36 (4.9%) were excluded because their parents refused to participate; 195 (26.5%), because data abstraction was not completed at all their primary care sites due to time constraints; and 500 (67.8%), because all their primary care providers met 1 of the exclusion criteria. Six children (0.8%) later found to be age-ineligible also were excluded. The remaining 2246 children constituted the final sample.

To assess sample bias, we compared the sex and race distribution of the final sample and the excluded children. Overall, among those excluded and those in the final sample, 50% and 51%, respectively, were male. For each of the 5 comparison groups, a higher proportion of black than white children were excluded; overall 79% of those excluded were black, whereas 69% of the final sample were black.

The mean numbers of well-child and sick visits per person-year were compared among the baseline and both experimental and control groups (Table 1). Although some differences were statistically significant (P<.05), absolute differences were negligible. The mean number of well-child and sick visits was approximately equal.

For each of the 5 comparison groups, 56% to 61% of the children were eligible for the PPI evaluation during the entire fiscal year. Of these children, the percentage who made at least 1 well-child visit during the year increased from 72% at baseline to approximately 80% during the 2 postintervention years for experimental and control groups (Table 2).

The percentage of children aged 3 to 24 months who were up-to-date for age for diphtheria and tetanus toxoids and pertussis (DTP), polio, and measles-mumps-rubella (MMR) vaccines varied little among the 5 comparison groups (Table 3). Slightly more than 50% of the children were up-to-date for DTP vaccine; more than 60% were up-to-date for polio vaccine. Approximately two thirds of the children were age-eligible for MMR vaccine; of the age-eligible children, two thirds received MMR vaccine before 24 months of age or by the end of the fiscal year.

Between baseline and year 2, missed opportunities for DTP (doses 1-4) and polio vaccinations (doses 1-3) decreased in experimental groups (Table 4). A similar trend was noted for MMR vaccination in experimental groups (P = .06). Missed opportunities for DTP vaccination also decreased somewhat among the controls, but to a lesser degree than among the experimental groups. Missed opportunies for polio vaccination differed little among controls across all 3 years. By year 2, the rates for missed opportunites for MMR vaccination were similar between experimental and control groups.

Among families receiving AFDC, the PPI did not contribute to a clinically relevant increase in preventive health care visits or vaccination coverage for children aged 3 to 24 months. There was no increase in the mean number of visits per person-year made by children in the experimental groups between baseline and year 2. On average, the children made only 2 well-child visits per person-year. Although the percentage of children who made at least 1 well-child visit during the year increased after the PPI was implemented, there was little variation between experimental and control groups. Furthermore, despite the increase, almost 20% of children had no well-child visit recorded in each study year.

Rates of vaccinations that were up-to-date for age differed little between baseline and each follow-up year and, within each follow-up year, between experimental and control groups. Overall, vaccination coverage was low. Less than 70% of the children were up-to-date for age for polio and MMR vaccines; slightly more than 50% were up-to-date for age for DTP vaccines. Over time there was a decrease in missed opportunities to vaccinate. This improvement, however, did not result in a change in vaccination status, most likely because the reduction in missed opportunities was small.

Several factors related to the PPI implementation may have contributed to the lack of effect. First, families and providers may have been unaware of assignments to intervention and control groups.³ Second, it is possible that not all families were disallowed according to the guidelines. Third, inadvertent exposure of control families to 1 or more components of the PPI also could have made detecting an effect more difficult. Fourth, inconsistent state policies may have weakened the intervention. Compensatory increases in food stamps and housing vouchers for sanctioned AFDC recipients effectively reduced the $25 sanction to $10.

Factors related to the PPI design also may help to explain why the PPI appears not to have affected levels of preventive heath care and vaccinations. The PPI was developed quickly without formal, prospective policy analysis. The problem of underutilization of preventive health care was not verified; it was assumed that parents were not motivated to seek care for their children. Once a child was labeled as becoming up-to-date, thedisallowances were discontinued, and there was no PPI incentive to adhere to the catch-up vaccination schedule. Moreover, potential barriers to implementation and policy alternatives were not considered.⁶

In addition, little evidence supports the theory that disallowances imposed on families alter preventive health care–seeking behavior. Modest economic sanctions may not be sufficient to induce behavioral change among low-income parents.¹⁰ Whether larger economic penalties would have had an impact on families' behavior remains unclear. One recent study¹¹ suggested that negative incentives, specifically requiring families to return to the offices of the Special Supplemental Nutrition Program for Women, Infants, and Children (WIC) for vouchers monthly rather than every 3 months, may improve immunization rates. However, the success of the Chicago WIC intervention may be attributed to a combination of screening and educational, referral, and tracking services rather than the provision of negative incentives alone.¹²

There was another important flaw in the theory underlying the PPI. The childhood immunization literature increasingly shows that provider practice exerts a strong influence on vaccination rates.^13- 17 Missed opportunities to vaccinate are a major cause of delinquent childhood vaccinations.^5,18- 27 It is not surprising that an intervention aimed only at modifying parents' behavior failed to influence children's vaccination status.

Although there were few differences in vaccination coverage between experimental and control groups, the data may underestimate or overestimate the actual coverage for groups.

Welfare sanctions are being promoted across the country as an effective tool to increase vaccination coverage.²⁸ At present, 7 states have implemented such sanctions as part of the Temporary Assistance to Needy Families program.²⁹ To avoid the implementation problems that occurred in Maryland, these states need to be aware of the Maryland experience. However, neither the theory behind such programs nor the available evidence suggests they will increase vaccination coverage. Moreover, sanctioning welfare recipients for their children's delinquent immunizations may inappropriately stigmatize parents as not caring about their children.

Accepted for publication April 30, 1999.

The evaluation was supported by the Maryland Department of Human Resources and Department of Health and Mental Hygiene with funds from the US Department of Health and Human Services and the Centers for Disease Control and Prevention, Atlanta, Ga.

Medical record data were entered by SOSIO Corporation, Inc, Baltimore, Md. We gratefully acknowledge the contribution of other members of the research teams, especially Dennis McGrath, PhD, and Laura Wilson-Gentry, DPA, from University of Baltimore and Amy Martin, MHS, from The Johns Hopkins School of Public Health, Baltimore, Md, and the many providers who participated in the evaluation.

Editor's Note: It's nice to see that sometimes money doesn't count, but I'd have preferred that not to be so in this case. Maybe if we gave these parents money (positive) instead of taking it away (negative).—Catherine D. DeAngelis, MD

Reprints: Cynthia Minkovitz, MD, MPP, Department of Population and Family Health Sciences, The Johns Hopkins University School of Hygiene and Public Health, 624 N Broadway, Baltimore, MD 21205 (e-mail: cminkovi@jhsph.edu).