0
Article |

Qualitative Study of Clinic Staff Members' Experiences Using an Immunization Registry FREE

Kristen J. Wells, MPH; Adrianne K. Holmes, BS; Susan A. Kohler, RN, MPH; Cynthia T. Rust, MD; Kimberly J. Rask, MD, PhD
[+] Author Affiliations

From the Department of Health Policy and Management, Rollins School of Public Health (Ms Wells and Dr Rask), Kerr L. White Institute for Health Services Research (Ms Holmes), Division of General Medicine (Ms Kohler and Dr Rask), and Department of Pediatrics (Dr Rust), Emory University School of Medicine, Atlanta, Ga.


Arch Pediatr Adolesc Med. 2000;154(11):1118-1122. doi:10.1001/archpedi.154.11.1118.
Text Size: A A A
Published online

Objectives  To examine individual clinic staff members' experiences with using an immunization registry and to compare staff members' perceptions of immunization registries across different provider sites.

Design  Cross-sectional survey using in-depth interviews and direct observation.

Settings  The pediatric department of an urban community health center and 2 urban hospital-based pediatric primary care clinics.

Participants  Twenty-five subjects were recruited using maximum variation sampling at each site. The subjects included clerks, clinic assistants, licensed practical nurses, a nurse practitioner, and registered nurses.

Main Outcome Measures  Clinic staff members' perceptions of an immunization registry and frequency of registry use.

Results  Differences were observed in subjects' perceptions of an immunization registry across provider sites. Although most subjects had positive attitudes toward the registry, they did not necessarily believe that the registry decreased their workload. The ability to access immunization registry data and actual use of the registry seem to be related to training of clinic personnel, location of the registry terminal, and helpfulness and availability of registry staff.

Conclusion  Obtaining the opinions of immunization registry users is an important strategy to evaluate the usefulness of a registry in a site and target possible areas for improvement.

AN IMMUNIZATION registry is defined as "a computerized database that gathers immunization information on all children . . . in a population defined by a specific geographic area or a health maintenance organization enrollment."1 Health care providers can use immunization registries to consolidate scattered records,2 provide an immunization needs assessment for each patient, provide current immunization recommendations, promote automated recall of underimmunized children, document immunizations for schools, preschools, and camps, help manage vaccine inventories, and provide practice-based immunization coverage assessments.3

Despite the proliferation of immunization registries, little research has evaluated the experiences of using an immunization registry as perceived by clinic and office staff. Bordley et al4 identified challenges to registry implementation in private practice through focus groups with pediatricians, family physicians, and office managers. These challenges include concerns regarding double entry of data, slowing of patient flow, staff time used for data entry, disproportionately high costs for small practices with limited staffs, and high staff turnover.4 In one survey, 60% of pediatricians, 47% of family physicians, and 77% of registered nurses and nurse practitioners stated that immunization registries represented the "best chance to solve the documentation problem." However, respondents who were familiar with their local registry were less likely to believe that registries would solve documentation problems and more likely to believe that registries were not yet of practical value.5

The perceptions of provider site personnel who interact with a registry are important because any successful registry must have both immunization and demographic records entered accurately and in a timely manner. This information is also important for health care providers who currently participate in a registry or who are considering participation. The specific aims of this study are to: (1) examine individual clinic staff members' experiences with using an immunization registry; and (2) compare staff members' perceptions of registries across different provider sites. This study is an important first step in identifying what registry users perceive as important factors related to using immunization registries and can be used to design effective interventions that improve registry use.

SETTINGS

Subjects were recruited from 3 sites: the pediatric department of a community health center (CHC) and 2 hospital-based pediatric primary care clinics. None of the sites was using computerized patient medical records during the study.

At the CHC, the registry terminal is located in the registration area. Since it is near the clinic examination rooms, the location of the terminal is easily accessible to all clinic staff. Prior to each clinical encounter, the CHC clerk and nurses use the immunization registry to print an immunization record. The record obtained from the registry is compared with the immunization record contained in the chart to make immunization decisions. After the clinical encounter, CHC staff manually enter immunization data into the registry within 24 hours of each patient's appointment. The CHC personnel administer approximately 2400 immunizations to 1400 children annually. The CHC participates in the Chatham County All Kids Count registry, a component of the Health Outcomes and Services Tracking system that was developed by the Center for Health Information through the Division of Public Health in the State of Georgia.6 The registry links all public health clinics, the pediatric departments of 3 community health centers, and 98% of private providers. The services that the registry provides include record look-ups; manual data entry; generation of reminder and recall notices through postcards, letters, or an autodialer; and practice assessment of immunization coverage.

Personnel in primary care clinics A and B obtain immunization records from the registry only if the clinic record and parents do not provide immunization information during the clinical encounter. At primary care clinic A, the registry terminal is located near the nurses' station and clinic examination rooms. Staff at primary care clinic A manually enter immunization data into the registry approximately 24 hours after each patient's appointment. Primary care clinic A provides approximately 5600 immunizations to 2300 patients annually. At primary care clinic B, the registry terminal is located in a small filing room that is down the hall from the clinic examination rooms and the nurses' station. Staff at primary care clinic B rarely enter immunization data into the registry within 24 hours of the patient's appointment. Primary care clinic B provides approximately 10,900 immunizations to 4400 patients annually. Primary care clinics A and B participate in the Metro Atlanta Team for Child Health (MATCH) registry. The MATCH registry is a community-based partnership between 2 county health agencies, local nonprofit agencies, and community health centers. The services that MATCH provides include record lookups, clinical data interface or batch data entry interface, and the capability to generate reminder and recall notices either through postcards or an autodialer.

SAMPLE

Subjects were recruited using maximum variation sampling at each site.7 Subjects were sampled with the intent to include each category of personnel using the registry (ie, nurse practitioner, registered nurse, clerk). This method of sampling allows description of the variation in the group (ie, different personnel categories) and provides a range of different experiences.7 Subjects signed an informed consent agreement and were given a $20 gift certificate from a local restaurant on completion of the interview. The study was approved by the Emory University Human Investigations Committee, Atlanta, Ga.

INTERVIEW DATA COLLECTION

Two interviewers collected data during September and October 1998 using a semistructured interview. The data collection instrument consisted of 25 open-ended questions. Using the constructs of predisposing, enabling, and reinforcing factors from the PRECEED-PROCEED model,8 20 interview questions were designed to examine factors associated with using an immunization registry. Three questions examined subjects' opinions regarding the effect of immunization registries on broad-based diffusion of technology in their work environment. Two demographic questions were also included. The interviews were audiotaped and transcribed verbatim. After reviewing a transcribed interview and field notes, each interviewer completed an interview summary form to summarize the main points of the interview.

OBSERVATION DATA COLLECTION

Observation data were collected in the 3 sites during 12 half-day clinic sessions. The data were used to describe how frequently clinic personnel used the immunization registry during the clinical encounters and the amount of time spent entering data into the registry. The clinical encounters observed at each site included acute care visits, health maintenance visits, and visits for the purpose of completing immunization forms.

DATA ANALYSIS

Using an iterative process, a preliminary list of codes was created and revised as a means of summarizing the data. Two coders reviewed the transcribed interviews and assigned codes to segments of the text. To clarify the definition of codes, the coders assigned codes to 5 interviews separately and then reviewed each rendition together. The κ coefficient calculated for interrater reliability was .806, indicating substantial agreement.9

After coding of the interviews was completed, case-level metamatrices10 were created to display and compare data collected during the interviews. In these displays, the data provided by each interviewed registry user were summarized in each row. Content-analytic summary tables were populated with interview data to compare staff members' perceptions of registries across provider sites.

OBSERVATION DATA

Two observers spent more than 41 hours directly observing the 3 sites. The data demonstrate differences between the provider sites in the percent of immunized patients whose records were obtained from the registry. At the CHC, 67% of immunized patients' records were accessed in the registry during the clinical encounter. At primary care clinic A, 41% of immunized patients' records were accessed in the registry during the clinical encounter, and at primary care clinic B, 13% of immunized patients' records were accessed during the clinical encounter (Table 1). The length of time to enter data at primary care clinics A and B was extremely variable when compared with that at the CHC.

Table Graphic Jump LocationSummary of Results Across Provider Sites
INTERVIEW DATA

Twenty-five subjects were interviewed. The interviews ranged from 9 to 47 minutes with a median length of 15 minutes. The subjects included clerks (n = 3, 12%), clinic assistants/technicians (n = 6, 24%), licensed practical nurses (n = 9, 36%), nurse practitioners (n = 1, 4%), and registered nurses (n = 6, 24%). Nine subjects (36%) were employed at primary care clinic A, 11 subjects (44%) were employed at primary care clinic B, and 5 subjects (20%) were employed at the CHC. Several themes were identified from the data analysis, and differences were observed between provider sites (Table 1).

ATTITUDE TOWARD THE REGISTRY

All of the subjects expressed positive comments about the registry. Several subjects stated that the registry was "helpful," and they were pleased to have it in the clinic. A nurse practitioner at the CHC commented: "I think [the registry has] improved immunization accessibility dramatically." The majority of the subjects (n = 22, 88%) believed that using the registry was part of their job responsibilities.

REGISTRY DATA

The majority of subjects at all 3 clinics (n = 24, 96%) found registry data to be useful or helpful. Subjects provided several different reasons why registry data are useful including: (1) obtaining records from other clinics (n = 5); (2) the ability to generate school immunization certificates (n = 3); (3) preventing underimmunization (n = 3); (4) preventing overimmunization (n = 2); (5) assisting with patient care (n = 2); (6) saving time (n = 1); (7) reducing the amount of writing (n = 1); (8) reducing the number of charts pulled (n = 1); (9) providing information on adverse reactions (n = 1); and (10) assisting with outreach (n = 1). While most subjects found registry data useful, only 3 subjects (33%) at primary care clinic A and 4 subjects (36%) at primary care clinic B described the data as "accurate" or "very accurate." In contrast to the primary care clinics, all of the subjects at the CHC believed that registry data were accurate. Only 7 (28%) of the 25 subjects interviewed mentioned that registry data were incomplete.

EXCHANGE OF IMMUNIZATION INFORMATION

The majority of subjects at all 3 clinics (n = 23, 92%) stated that the registry was helpful in exchanging immunization information with other provider sites. Eighteen (72%) of the 25 subjects interviewed stated that the registry allowed them to obtain records from other clinics quickly and easily.

TYPE AND AMOUNT OF TRAINING

Differences were observed between the sites with respect to the amount of training that subjects reported having received. The majority of subjects at primary care clinics A and B (88% [n = 8] and 64% [n = 7], respectively) reported that they were trained by a coworker to use the registry. Forty-four percent (n = 4) of personnel at primary care clinic A, and 36% of personnel (n = 4) at primary care clinic B stated that they had received training during an inservice. Three people (33%) at primary care clinic A, and 1 person (9%) at primary care clinic B reported that they had received no training. A clinic technician in primary care clinic A commented, "We just trained ourselves. Well, the nurses showed us how to use it, and we . . . went from there." In contrast, all personnel at the CHC reported having been formally trained, and 3 of the subjects (60%) reported that they were trained more than once. A licensed practical nurse at the CHC told the interviewer, "I've gone to the training I think both times, two or three times."

PERCEPTION OF TRAINING

The majority of personnel at primary care clinic A (n = 7, 78%) and the CHC (n = 4, 80%) stated that the training they received was helpful. A registered nurse at the CHC stated that "they trained me very well." However, only 4 of the subjects interviewed at primary care clinic B (36%) found the training helpful. A licensed practical nurse at primary care clinic B stated "the first initial training that we got, it wasn't that good . . . it wasn't as useful because it just came with us kind of working together to learn how to use it. Because I think we more taught ourselves than the inservice that we received from the person."

PERCEPTION OF REGISTRY STAFF

Differences in subjects' perception of the registry staff were also observed. None of the subjects at primary care clinic A and only one subject at primary care clinic B (9%) mentioned assistance provided by the registry staff. A registered nurse at primary care clinic A expressed a need for technical support when she said " . . . suppose there is a problem like [the registry is] not working. . . . If [the attending physician] is not here, is there somebody else we can call who can come down?" At the CHC, 3 of the subjects (60%) mentioned that the registry provides a help desk, as illustrated by a licensed practical nurse:

"we have a number at the Health Department when you first get started with the computer and you're hung, I mean, you don't know what it's asking you to do, you don't understand. Just call and they explain it to you and walk you through it on the phone."

LOCATION OF REGISTRY TERMINAL

When asked about the location of the registry terminal, all subjects at primary care clinic A and the CHC stated that the location was adequate. In contrast, 10 subjects (91%) at primary care clinic B stated that the location was poor. At the time of the interviews, the registry terminal in primary care clinic B was located in a small filing room that was also occupied by 2 filing clerks. A clerk at primary care clinic B stated, "I don't like where it is. I mean, we would probably use it much more if it wasn't where it is now."

ABILITY TO ACCESS REGISTRY DATA

Additional differences between provider sites were observed in subjects' perceptions of their ability to access registry data. The majority of subjects at primary care clinics A (n = 6, 67%) and B (n = 7, 64%) mentioned difficulties with using the registry. These difficulties included inability to log on to the registry, inability to download or upload registry data, difficulty performing look-ups, frustration with the registry, and forgetting how to use the registry. A clerk at primary care clinic B reported that "when you can't execute the commands that you're trying to carry out you can get frustrated." Subjects at the CHC did not report any difficulties accessing registry data.

COMPUTERIZED MEDICAL RECORDS

When asked whether they would support the implementation of a system of computerized medical records in their clinic, subjects in primary care clinics A and B were generally supportive of such a system. In contrast, only 1 subject at the CHC (20%) was supportive of implementing computerized medical records. A nurse practitioner indicated that she would not want to have computerized medical records because they may be a violation of patient confidentiality and stated, "No . . . then you start to talk about patient confidentiality in medical records, so I don't know how much medical record you'd want to put out there for everyone to get into."

EFFECT OF REGISTRY ON PERSONAL WORKLOAD

While the subjects' attitudes were generally positive in all 3 sites, only 44% (n = 11) of all subjects believed that the registry decreased their workload. Twenty-four percent (n = 6) of all subjects believed that the registry had no effect on their workload, and 28% (n = 7) believed that using the registry increased their workload. One person (4%) stated that someone else should be designated to obtain immunization records and use the registry because both tasks were time-consuming. Although some subjects believed the registry increased their workload, they were willing to perform the extra work because of the benefits provided by the registry. A registered nurse at the CHC commented "it is more work, but it is OK."

This study presents data obtained from direct observation and in-depth interviews conducted with immunization registry users. The interviewed registry users generally perceived the registry and the data in the registry to be helpful in their clinics. The factors that contribute to the integration of an immunization registry into clinic activities are apparent when the CHC is compared with primary care clinic B. Observation and interview data in the CHC indicate that the registry is well integrated into clinic activities, and there is little variability in the length of time expended for data entry.11 Subjects at the CHC received formal training, perceived the training to be helpful, perceived the registry staff to be accessible and helpful, frequently accessed data during the clinical encounter, and did not report any difficulties associated with using the registry. In contrast, subjects in primary care clinic B received little formal training or assistance from the registry staff, did not perceive the training they did receive to be helpful, reported that the location of the registry terminal was poor and hindered their use of the registry, mentioned difficulties accessing registry data, had extremely variable lengths of time for data entry,11 and rarely accessed registry data during the clinical encounter.

Although most subjects had positive attitudes toward the registry, they did not necessarily believe that the registry decreased their workload. Subjects who stated that registry use increased their workload indicated that, as a result of having the registry, they provided more immunizations, printed more immunization records, and entered more data than they did before the registry was implemented. In all 3 provider sites, registry use failed to eliminate immunization paperwork. To encourage provider participation, registries should work to develop services that allow clinic staff members to save time, such as creating forms that can be populated with registry data and printed to replace existing paperwork.

In our study, subjects at 2 clinics reported that lack of training was a barrier to using a registry. Similarly, results from the study conducted by Christakis et al5 indicate that 41% of registered nurses and nurse practitioners, 30% of pediatricians, and 18% of family physicians reported that training was a barrier to registry use.

This study has limitations that may affect its generalizability. The study was conducted with a small number of subjects at 3 provider sites participating in 2 immunization registries. Because there are many different types of immunization registries, registry users, and provider sites, the results obtained in this study may not be generalizable to other clinics or registries. Using a cross-sectional approach, we were unable to measure changes in staff members' opinions over time. Future research could also examine the opinions of clinic personnel employed in private practices that participate in a registry.

In this evaluation, the ability to use a registry and actual registry use seem to be related to training, helpfulness and availability of registry staff, and location of the registry terminal. It is important that those responsible for registry implementation evaluate actual registry use in provider sites to design strategies to achieve optimal registry participation. Examining the opinions of registry users is one way to begin to evaluate registry success at the clinic level.

Accepted for publication June 5, 2000.

This research was supported by grants U1W/CCU414706-01 and U1W/CCU414706-02 from the National Immunization Program of the Centers for Disease Control and Prevention, Atlanta, Ga.

Data from this article were presented at the 33rd National Immunization Conference, Dallas, Tex, June 24, 1999, and at the American Public Health Association 127th Annual Meeting and Exposition, Chicago, Ill, November 8, 1999.

The contents are solely the responsibility of the authors and do not necessarily represent the official views of the Centers for Disease Control and Prevention.

Reprints: Kristen J. Wells, Department of Health Policy and Management, Rollins School of Public Health, 1518 Clifton Rd NE, 6th Floor, Atlanta, GA 30322 (e-mail: kwells@sph.emory.edu).

Wood  DSaarlas  KNInkelas  MMatyas  BT Immunization registries in the United States: implications for the practice of public health in a changing health care system. Annu Rev Public Health. 1999;20231- 255
Yawn  BPEdmonson  LHuber  L  et al.  The impact of a simulated immunization registry on perceived childhood immunization status. Am J Managed Care. 1998;4185- 192
Linkins  RWFeikema  SM Immunization registries: the cornerstone of childhood immunization in the 21st century. Pediatr Ann. 1998;27349- 354
Bordley  WCDempsey-Tanner  TFreed  GLLister  ME Challenges to private provider participation in immunization registries. Am J Prev Med. 1997;13 (suppl) 66- 70
Christakis  DAStewart  LBibus  D  et al.  Providers' perceptions of an immunization registry. Am J Prev Med. 1999;17147- 150
Pappas  MBaker  BJAppelbaum  DHinds  WAlexander  ERKorolak  K Immunization registries: provider recruitment and participation. Am J Prev Med. 1997;13 (suppl) 71- 76
Patton  MQ Qualitative Evaluation and Research Methods. 2nd ed. Newbury Park, Calif Sage Publications Inc1990;
Green  LWKreuter  MW Health Promotion Planning: An Educational and Environmental Approach. 2nd ed. Mountain View, Calif Mayfield Publishing Co1991;
Landis  JRKoch  GG The measurement of observer agreement for categorical data. Biometrics. 1977;33159- 174
Miles  MBHuberman  AM Qualitative Data Analysis: An Expanded Sourcebook. 2nd ed. Thousand Oaks, Calif Sage Publications Inc1994;
Rask  KJWells  KJKohler  SARust  CTCangialose  CB The cost to providers for participating in an immunization registry. Am J Prev Med. 2000;1999- 103

Figures

Tables

Table Graphic Jump LocationSummary of Results Across Provider Sites

References

Wood  DSaarlas  KNInkelas  MMatyas  BT Immunization registries in the United States: implications for the practice of public health in a changing health care system. Annu Rev Public Health. 1999;20231- 255
Yawn  BPEdmonson  LHuber  L  et al.  The impact of a simulated immunization registry on perceived childhood immunization status. Am J Managed Care. 1998;4185- 192
Linkins  RWFeikema  SM Immunization registries: the cornerstone of childhood immunization in the 21st century. Pediatr Ann. 1998;27349- 354
Bordley  WCDempsey-Tanner  TFreed  GLLister  ME Challenges to private provider participation in immunization registries. Am J Prev Med. 1997;13 (suppl) 66- 70
Christakis  DAStewart  LBibus  D  et al.  Providers' perceptions of an immunization registry. Am J Prev Med. 1999;17147- 150
Pappas  MBaker  BJAppelbaum  DHinds  WAlexander  ERKorolak  K Immunization registries: provider recruitment and participation. Am J Prev Med. 1997;13 (suppl) 71- 76
Patton  MQ Qualitative Evaluation and Research Methods. 2nd ed. Newbury Park, Calif Sage Publications Inc1990;
Green  LWKreuter  MW Health Promotion Planning: An Educational and Environmental Approach. 2nd ed. Mountain View, Calif Mayfield Publishing Co1991;
Landis  JRKoch  GG The measurement of observer agreement for categorical data. Biometrics. 1977;33159- 174
Miles  MBHuberman  AM Qualitative Data Analysis: An Expanded Sourcebook. 2nd ed. Thousand Oaks, Calif Sage Publications Inc1994;
Rask  KJWells  KJKohler  SARust  CTCangialose  CB The cost to providers for participating in an immunization registry. Am J Prev Med. 2000;1999- 103

Correspondence

CME
Meets CME requirements for:
Browse CME for all U.S. States
Accreditation Information
The American Medical Association is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians. The AMA designates this journal-based CME activity for a maximum of 1 AMA PRA Category 1 CreditTM per course. Physicians should claim only the credit commensurate with the extent of their participation in the activity. Physicians who complete the CME course and score at least 80% correct on the quiz are eligible for AMA PRA Category 1 CreditTM.
Note: You must get at least of the answers correct to pass this quiz.
You have not filled in all the answers to complete this quiz
The following questions were not answered:
Sorry, you have unsuccessfully completed this CME quiz with a score of
The following questions were not answered correctly:
Commitment to Change (optional):
Indicate what change(s) you will implement in your practice, if any, based on this CME course.
Your quiz results:
The filled radio buttons indicate your responses. The preferred responses are highlighted
For CME Course: A Proposed Model for Initial Assessment and Management of Acute Heart Failure Syndromes
Indicate what changes(s) you will implement in your practice, if any, based on this CME course.
NOTE:
Citing articles are presented as examples only. In non-demo SCM6 implementation, integration with CrossRef’s "Cited By" API will populate this tab (http://www.crossref.org/citedby.html).
Submit a Comment

Multimedia

Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging & repositioning the boxes below.

Articles Related By Topic
Related Topics
PubMed Articles