Effectiveness of an Internet-Based Store-and-Forward Telemedicine System for Pediatric Subspecialty Consultation FREE

Pediatric subspecialists are often separated from the children who need them by distance, time, or socioeconomic factors. These barriers adversely affect the health of children who need but cannot readily access specialty care. Specialty providers tend to practice in academic medical centers in urban areas. Children who live in rural or isolated regions lack access to these physicians because of the time and distance involved in obtaining a routine consultation.

Telemedicine is an attractive alternative for providing pediatric subspecialty care to children without access to such care.^1- 2 The term telemedicine is used to describe a range of applications that use technology to move medical information over a distance.³ Most of the effort to date in the development of pediatric teleconsultation has been real-time (synchronous) video teleconferencing connections, either patient to provider or provider to provider.^4- 5

Telemedicine has been a part of medical practice at Tripler Army Medical Center (TAMC) for more than a decade. Asynchronous, Web-based teleconsultation has been used since 1997 to provide specialty care access to patients in the US-Associated Pacific Islands (the former Trust Territories) of the South and Western Pacific, where such care is unavailable.^6- 7 This method of telemedicine is the product of a decade of trials with other technology at TAMC. Early efforts were primarily directed toward synchronous or real-time video-telephone connections using the “Picasso” telephone. More than 30 demonstrations were successfully carried out between physicians in the US-Associated Pacific Islands, TAMC, Alaska, New Caledonia, Fiji, and other remote sites. This technology was ultimately abandoned because the units were no longer manufactured and because long-distance telephone charges became prohibitively expensive in the jurisdictions of the Pacific (as much as $10 a minute on some islands). Moreover, real-time consult coordination interfered with the workflow and productivity of the sending and receiving physicians.

In the mid 1990s, real-time synchronous videoconferencing was applied to more than 150 medical teleconsultations and referrals between TAMC and the US Army Kwajalein Atoll in the Republic of the Marshall Islands. After the initial novelty of interacting with physicians and patients across a wide expanse of ocean wore off, the prohibitive expense, scheduling conflicts, technical demands, and infrastructural requirements led to the project's abandonment.^6- 7

Store-and-forward telemedicine strategies have been successful in this region because of simplicity, low cost, and minimal infrastructure requirements. Primary care providers need only have Internet access to submit a consult to a specialty provider. Beginning in 2000, this Web-based technology became available to children of American service members scattered with their families across the Western Pacific.⁸ It is more convenient than scheduled synchronous or real-time teleconsultation, is less disruptive to physician and patient schedules, and requires far less technical and bandwidth support.

To date, there has been no objective assessment of the impact of this type of telemedicine system on children. In this study, the use of a pediatric teleconsultation system, the Electronic Children's Hospital of the Pacific (ECHO-Pac), was systematically studied for 1 year to characterize its use and its impact on access to specialty care, the quality of the care provided, and cost savings.

The ECHO-Pac is a Web-based, store-and-forward, asynchronous, provider-to-provider teleconsultation system. Store and forward implies that the physician initiating the consultation or referral enters the data, stores it locally, and then forwards it to be reviewed by a consultant at a later time. The Web site was designed by programmers from the Pacific Telehealth and Technology Hui (originally Project Akamai, later the Pacific e-Health Innovation Center) in Honolulu. Hui is the Hawaiian word for “group” or “team.” The only hardware required at the remote location is a standard Web browser and peripherals used for image capture (digital camera, scanner, and video camera). Patient information security is ensured because access to the Web server occurs through a Secure Sockets Layer connection, and each session is encrypted from client browser to Web server. Provider authentication of users is completed as part of the database application. Passwords are stored encrypted in the database.

Pediatric subspecialty consultation through the ECHO-Pac teleconsultation system is initiated at the primary care site. A referring physician accesses the ECHO-Pac through a personal computer with broadband Internet access at his or her work site. Each physician requests an account, using his or her last name and a self-selected password. The account is approved by any of 4 “consult manager” pediatricians at the tertiary care center, who review all of the incoming consults and also serve as administrators to the site.

Referring physicians who want to consult a pediatric subspecialist can access the Web site either during their usual patient schedule or after patient care hours are completed. The provider enters free-text clinical information about the patient into the secure, encrypted Web site. In addition, images (patient photographs or radiographs) can be added as JPEG (Joint Photographic Experts Group) files, and brief digital motion images can be added as MPEG (Moving Pictures Expert Group) files or AVI (Audio Video Interleaved) files. (Upload is not limited by file size. Image size affects the speed of upload as a reflection of Internet bandwidth.) The system was not designed or ever intended for emergency consultations or advice.

Once the consultation is submitted, a consult manager and subsequently the subspecialist are notified by an e-mail message devoid of patient-identifying information. The Web site automatically assigns a number to each case, and this number is used in all e-mail correspondence. Each patient consult has a separate page on the Web site, and the physician and consultant dialogue becomes an electronic medical record that the primary care provider can print at any time to include in the paper medical record. The Web site generates an automatic e-mail message to every physician on a case each time a new comment or a new image is uploaded.

In this study, 22 health care facilities including Madigan Army Medical Center were invited to participate. Seven sites were in the referral region for Madigan Army Medical Center (the Pacific Northwest), and the remaining 14 (in the Western Pacific) usually referred pediatric subspecialty consultations to TAMC. The Tripler Institutional Review Board approved the protocol, as did the review boards of the navy, air force, and army's central approving authority (TAMC Protocol No. 18H02, HSRRB Log No. A-11574).

The hospital commander designated a physician from primary care (a pediatrician or a family practitioner), and each physician was trained via a telephone conference with one of us (C.W.C.) that generally lasted 30 to 45 minutes. The physician was responsible for training other physicians at the site. Start dates for the different sites were staggered between July 1, 2002, and October 31, 2002. Data from each site were collected for 1 year, between July 1, 2002, and October 31, 2003. Utilization was determined for each site, and a consultation rate was determined based on the number of cases submitted and the population served by that medical facility. (Population data were secured from the Department of Defense Medical Health System Management Analysis and Reporting System.)

Access to specialty care was measured as the time from submission to completion of the consult by the pediatric subspecialist. Time was tracked using a time clock embedded in the program. In some cases, when the pediatric specialist was not available at the medical center, an answer was solicited from a specialist who participated in the system but was located at another medical center. The consult manager at each medical center was responsible for tracking the replies to the consults. The number of times that a provider from a medical center other than the usual referring center (termed electronic backfill) answered the consult was documented.

A panel of 5 providers uninvolved with the case reviewed all of the consults. The panel consisted of 3 pediatric subspecialists and 2 general pediatricians. Each physician was unaware of the other panel members' responses. The panel members independently rendered an opinion for each consult using an online survey with a Likert scale scored from 1 to 5 (very unlikely to very likely) to answer the following 3 questions: Was the diagnosis changed or modified by the consultation? Was the diagnostic plan changed or modified by the consultation? Was the treatment plan changed or modified by the consultation?

Any question that was answered “likely” or “very likely” (4 or 5 on the Likert scale) was considered a positive response. We calculated the lower 95% confidence interval for each question in each case based on the mean and standard deviation of the scores of the 5 reviewers. In this analysis (5 scorers, 5 potential choices on the Likert scale), if the lower 95% confidence interval was higher than 3.0, the mean score was considered positive (P < .01). Using this analysis, any question with 1 or more scores of 3 or less from any of the 5 reviewers was negative, or not statistically significant.

The estimate of cost savings was determined based on the number of patients for whom travel was avoided. For many of the centers, travel necessitated a 5-hour plane flight and more than a week’s stay at the medical center. Members of the physician review panel were asked the following question for each case: If telemedicine were unavailable, would this patient have been evacuated on a routine flight to the medical center for consultation and care? Reviewers were also asked whether they believed that the case would have been referred on an emergency basis.

As an estimate of the economic impact of this teleconsultation system, total travel cost in the military system per air evacuation was used as a measure of cost avoidance. Travel cost was estimated to include the average cost of 2 commercial tickets from the Western Pacific (child and parent) at $1977 each. The per diem reimbursement for the parent was $184 × 10 days. Thus, the total cost for 1 air evacuation in this model was estimated to be $5794.

Between July 1, 2002, and October 31, 2003, there were 267 teleconsults submitted from 16 different sites. Images were attached to 91 consults (34%). A total of 415 images were submitted with the consults for review. The remainder of the cases were text-only questions. Twenty-seven general pediatricians submitted 189 consults. The remaining consults were submitted by family practitioners (19 consults), general medical officers (intern-level trained) (51 consults), psychiatrists (5 consults), and developmental pediatricians (3 consults.) The mean ± SD patient age was 5 ± 5 years (range, birth to 19 years).

There was a wide range of use of the consultation system at the primary care sites, ranging from 54 consults from a population of 3213 children (16.8 consults per 1000 children) to 1 consult from a population of 4528 (0.2 consult per 1000). A typical rate was 63 consults from a population of 5497 children, or 11.5 consults per 1000 children, or 1 consult per 1000 children per month. Two hundred fifty consults came from the TAMC referral region (Western Pacific), and only 17 came from the Madigan Army Medical Center referral region (Pacific Northwest).

Because the consult manager would often submit the cases to more than 1 specialist to gain a broader response to the question, there were 1021 consult “comments” for the 267 consults, or 3.8 different consultant comments per consultation. Specialists from 33 different disciplines answered consults, including 56 comments from 3 general pediatricians. Sixty-nine pediatric subspecialists in 18 different specialties provided 93% of the comments on the consults submitted (Table 1 and Table 2).

Generally, the response time to the initiating physician was excellent. The mean ± SD time for the case manager to review the case and forward it to the appropriate specialists was 5 ± 3 hours (range, 1-41 hours). A reply from a consultant on the Web site was returned to the primary care provider in a mean ± SD of 32 ± 14 hours (range, 1-335 hours). Three consults took an excessively long time to complete because they were inadvertently entered into an old, infrequently monitored section of the system designed for the original asthma study. These 3 consults were forwarded in a mean ± SD of 194 ± 42 hours and were answered by the pulmonologist in 13 ± 10 hours. Excluding these 3 consults, the median time for consults to be forwarded was 2 hours, and the median time to consult reply was 12 hours.

In many cases, a specialist was not available at the referral medical center because of a deployment related to war or other temporary duty. Using electronic backfill, an appropriate pediatric subspecialist was contacted by the consult manager using the embedded e-mail system, and the specialist from another medical center completed the consult request. Twelve percent (33/267) of the study cases were addressed by a specialist at a medical center other than TAMC or Madigan Army Medical Center.

The quality of care for patients was improved through teleconsultation. According to the physician review panel, the diagnosis was changed or modified in 15% (39/267) of the cases. The diagnostic plan was changed or modified in 21% (57/267) of the cases, and the treatment plan was changed or modified in 24% (64/267) of the cases (P<.01.) To verify the findings of the physician panel review, the single physician with the greatest number of referred cases was asked to review his 56 cases retrospectively. He believed that the diagnosis was changed or modified in 42% (23/56) of the cases, the diagnostic plan was changed or modified in 54% (30/36), and the treatment plan was changed or modified in 61% (34/56). Thus, it is likely that the physician panel underestimated the primary care provider's impression of the impact of the teleconsultation on quality of care. The panel review also determined that 32 routine air evacuations (12%) were avoided, with an estimated cost savings of $185 408. The panel review did not find that any of the cases would have been sent as emergency cases.

The ECHO-Pac teleconsultation system provided primary care providers with expeditious access to pediatric subspecialty assessments and recommendations. They received answers to diagnostic and management questions regarding the care of their patients in a fraction of the time that would have been required had the patients and their families had to travel for specialty care. In fact, replies were often received in less time than it would have taken to even schedule an air evacuation and pediatric subspecialty consult. This technology has had broad experience and success during the past decade, especially in the diagnosis and management of dermatologic conditions.^9- 13 It has the advantages of low cost, simplicity, and ease of use, and it fits into a physician’s schedule without disruption of usual patient flow.

Use of the ECHO-Pac system varied widely among providers, provider specialties, and locations. The variation was not easily explained by the size of the population or the type or training of the primary care providers. Use of the ECHO-Pac system was not mandatory, and it is likely that many of the referral sites continued to use existing patterns of pediatric specialty consultation in many cases. This is most likely for primary care sites in the continental United States, where local referral and consultation patterns are often based on existing relationships between providers and local specialists. In a system where there are no financial disincentives to do otherwise, primary care providers will usually take the path of least resistance and do what is most efficient. The system was most popular in the Western Pacific, where pediatric subspecialty support was least available to eligible children.

The Web-based system provided something that a simple “physical” referral to a pediatric specialist could not. In many cases, primary care providers requested consultations from a specific specialist who may not have been the best one to answer the question. The consult manager, who screened all the consults, took advantage of the breadth of consultants available to the system and solicited the opinions of a range of specialists for the same question. For example, an endocrinologist, nephrologist, cardiologist, and adolescent medicine specialist all responded to questions about obesity and hypertension in a teenaged patient, bringing to the primary care provider the benefits of all 4 specialty opinions.

The physician panel review estimated an impact of 15% to 24% improvement in diagnosis and management. This may be a conservative estimate. When a specific physician was queried, he believed that the site had an even bigger impact on quality. As telemedicine developers reach for measures of effectiveness for innovative teleconsultation systems, it is likely that expert panel review and initiating physician review will have an important role.

There are a variety of limitations to this study. The financial implications of store-and-forward teleconsultation have not been well worked out. It is clear that this system was effective in cost avoidance in a military system. It is not clear whether these economic advantages would be translated to a civilian care system. It is possible that similar benefits might be realized with a large managed care organization. However, some issues regarding asynchronous teleconsult reimbursement are unresolved, even in regions and systems in which store-and-forward teleconsultation has been practiced on a large scale, such as Alaska. In addition, pediatric subspecialists may fear the loss of income or workload from consults for patients who would have been referred for office consultations but whose problems are resolved using a telemedicine system.

Finally, there are a variety of unresolved administrative issues with the practice of asynchronous telemedicine. There are concerns about HIPAA (Health Insurance Portability and Accountability Act of 1996), which makes the discussion of patient information using unsecured electronic mail illegal. However, the use of a password-protected Web site with encryption meets the intent of HIPAA, although it is likely that written consent may eventually be needed before sending a consult.¹⁴

The Joint Commission on Accreditation of Healthcare Organizations has also raised concerns about the credentialing of physicians who practice teleconsultation. In cases in which direct management or treatment of a patient does not take place and the care of the patient is left in the hands of the primary care provider, as is the case with ECHO-Pac, these concerns may be unfounded.¹⁵ Questions also have to be resolved regarding the issues of medical licensing and the practice of telemedicine across state and international boundaries. Although this is not a concern in the military setting, it is an issue in the civilian community where the referral base for a children's specialty practice crosses state borders.

In this 1-year review, the ECHO-Pac teleconsultation system improved care by providing expeditious pediatric subspecialty consultation and demonstrated significant cost avoidance in this military pediatric population. This asynchronous telemedicine consult system was well accepted by primary care providers and pediatric subspecialists and had a positive effect on the quality of care delivered. Questions remain as to whether the success of this system in a military setting can be generalized to a civilian model. In particular, it is not clear whether issues of reimbursement, licensing, and other concerns will prevent its integration into the business practices of a civilian primary care or pediatric subspecialty practice.

Correspondence: COL Charles W. Callahan, MC, USA, Department of Pediatrics (MCHK-PE), Tripler Army Medical Center, 1 Jarrett White Rd, Honolulu, HI 96859-5000 (charles.callahan@us.army.mil).

Disclaimer: The views and opinions expressed in this manuscript are those of the authors and do not reflect the official policy or position of the Department of the Army, the Department of Defense, or the US government.

Accepted for Publication: September 13, 2004.

Funding/Support: This study was supported in part by grant MIPR2071 from the US Army Medical Research Acquisition Activity Telemedicine and Technology Research Center, Ft Detrick, Md.

Acknowledgment: We thank Nancy Tikunoff, RN, Dale York, Mark Ching, David Oshiro, Mike Darnall, MAJ Ron Prauner, MC, USA, COL Robert Newman, MC, USA, James Davis, PhD, and the providers from the primary care sites: US Naval Hospital, Okinawa, Japan; USAF 35th Medical Group, Misawa, Japan; US Naval Hospital, Guam; US Naval Hospital, Yokosuka, Japan; USAF 18th Medical Group, Kadena, Japan; 121 General Hospital, Seoul, Korea; USAF 374th Medical Group, Yokota, Japan; USAF 36th Medical Group, Guam; 51st Medical Group, Osan, Korea; Naval Branch Medical Clinic, Sasebo, Japan; US Naval Branch Medical Clinic, Atsugi, Japan; US Army Health Clinic, Camp Zama, Japan, US Naval Hospital, Oak Harbor, Wash; Bassett Army Community Hospital, Ft Wainwright, Ark; Weed Army Community Hospital, Ft Irwin, Calif; and USAF 354th Medical Group, Eielson AFB, Ark.