From the Stanford University Sleep Disorders Clinic, Stanford, Calif.
To investigate abnormal breathing patterns during sleep in prepubertal children using nonstandard polysomnographic patterns in association with an apnea-hypopnea scoring technique.
Patients and Methods
Study participants included 400 children with suspected sleep-related breathing disorders and 60 control children. We analyzed clinical signs and symptoms at entry into the study and 3 months after otolaryngological treatment. We determined the frequency of predefined breathing patterns during sleep through blind analysis of polysomnograms obtained once in control subjects and twice in children referred to our clinic (before and after adenotonsillectomy), using the nasal cannula–pressure transducer system, mouth thermistor, esophageal manometry, microphone, and pulse oximetry. We also determined the relationship between breathing patterns during sleep and residual postsurgery symptoms. Further analysis was performed of symptoms and polysomnographic patterns in those children who underwent new treatment interventions due to persistence of symptoms and abnormal polysomnogram findings.
Tachypnea, persistently elevated breathing effort, progressively increased breath effort, and discrete flattening of nasal airflow monitored with the nasal cannula–pressure transducer system without oxygen saturation decreases help determine disorder as much as apneas and hypopneas. Abnormal, nonstandard breathing patterns were associated with the same symptoms as those in children with apnea and hypopnea and were more commonly present when there was incomplete resolution of initial symptoms that led treating practitioners to request further treatment.
Currently published polysomnographic scoring recommendations overlook common breathing abnormalities during sleep that are associated with clinical complaints.
Sleep apnea—obstructive, mixed, and central—was defined more than 20 years ago as a polygraphic pattern associated with abnormal breathing during sleep in prepubertal children.1 By the early 1980s, it was shown that children could have clinical symptoms related to abnormal breathing during sleep without the presence of any apnea based on the established criteria of nocturnal polysomnography.2 These deficiencies in the polysomnographic scoring criteria remain today.3- 6
Clinical practice suggests that breathing patterns other than those described in the literature are indicative of abnormal breathing during sleep. Consequently, snoring children with clinical symptoms but with negative polysomnographic results have undergone tonsillectomy and adenoidectomy.7 We questioned whether more attention to other aspects of breathing during sleep would better define sleep-related breathing disorders (SRBDs) in children. If so, then we must question which aspects of breathing are associated with clinical symptoms. Some of the children included in this report were previously described.8 The surgical aspect and surgical outcomes will be the subject of a different report.
We studied breathing patterns during sleep using more accurate recording devices.9- 11 We examined the validity of attributing pathologic significance to specific breathing patterns by evaluating the presence or absence of clinical complaints, the results of polysomnography, and the effect of tonsillectomy and adenoidectomy on the selected breathing patterns seen in children with SRBD. Then we compared the frequency of findings with those noted in a control group.
This investigation is based on a retrospective analysis of nocturnal recordings performed on 400 prepubertal children 2 to 12 years of age with clinical symptoms of SRBD and 60 children in the same age range recruited from the community, serving as control subjects, for unrelated research protocols.
All children with Tanner stage 1 status12 referred to the sleep clinic between 1996 and 2000 for suspected SRBDs were selected from a computerized database. Children with neuromuscular diseases, syndromes with craniofacial malformation, cardiac failure, obesity based on body mass index according to published tables,13 hypoventilation syndromes, or other medical disorders that could explain the SRBD were eliminated from further consideration. All selected children had available clinical notes from entry, follow-up visits, and surgery and diagnostic and postsurgical follow-up recordings of nocturnal polysomnography that lasted at least 8.5 hours without major technical problems. Children were diagnosed as having an SRBD if they had clinical symptoms and a respiratory disturbance index (RDI) of more than 1.5 events per hour. The RDI included obstructive and mixed apnea, hypopnea, and flow limitation that did not meet our laboratory criteria for hypopnea, as scored by the technologist with the nasal cannula–pressure transducer,9- 11 respiratory rate findings, and results of esophageal pressure (Pes) evaluation. Recording techniques and equipment and the 2 sleep specialists (C.G., R.P.) involved in the care of the children were the same during the 4-year survey time. All children who met these entry criteria were included in the survey.
Control subjects had been recruited for research protocols based on questionnaire (Sleep Disorder Questionnaire14) and general medical and sleep evaluations with clinical interviews and physical examination performed by a pediatrician and a sleep specialist (R.P., C.G.) but not on polysomnography, which was only subsequently performed. These data and polysomnogram results, however, must have been available for children to be included as control subjects.
Sixty of the 63 community children and 400 of the 547 children with SRBD in the database met the inclusion criteria. Exclusion was due to either presence of specific medical syndromes (n = 81) or an incomplete data set (n = 66). Parents of children with SRBD signed general informed consent forms, allowing collected data to be used for research purposes, and parents of control children had signed an informed consent form covering specific research projects that included polysomnography. These 460 children represent the study contingent.
All children diagnosed as having an SRBD had been sent to ear, nose, and throat specialists for treatment. Two hundred ninety-seven children with SRBDs underwent adenotonsillectomy, 100 underwent either tonsillectomy or adenoidectomy, and 3 children solely received inferior nasal turbinate treatment. The ear, nose, and throat specialists made the surgical decisions.
The same polygraphic variables were monitored in the children with SRBD and control children. Each recording included the following variables: electroencephalogram (EEG), electro-oculogram, chin and leg electromyography, electrocardiogram (modified V2 lead), and body position. Respiration was monitored using neck microphone (breathing noises), nasal cannula–pressure transducer system, oral thermistor, thoracic and abdominal bands for measurement of uncalibrated respiratory plethysmography, Pes, and pulse oximetry. A transcutaneous carbon dioxide electrode was used for carbon dioxide analysis. Esophageal pressure was calibrated in centimeters of water at the beginning and end of the night. All recordings were performed with video monitoring. All polysomnograms were rescored blindly for this analysis.
Before scoring, the definitions outlined in Table 1 were agreed on. These scoring definitions were based on clinical experience or accepted scoring criteria. Oxygen desaturation was not a prerequisite for scoring any of the abnormal breathing patterns. Oxygen desaturations of 3% or more were noted (Figure 1, Figure 2, and Figure 3).
A 5-minute esophageal pressure (Pes) crescendo with snoring and no oxygen saturation decrease during stage 2 non–rapid eye movement sleep. Continuous snoring, flow limitation visible on cannula trace (nasal cannula–pressure transducer system), and progressive increase in respiratory effort indicated by Pes trace are seen. C3, C4, O1, A1, and A2 are standard electrode placements of the 10-20 international electroencephalogram electrode placement system. EMG indicates electromyography of the chin; ROC, right eye movements; LOC, left eye movements; L/RAT, left and right anterior tibialis; ECG, electrocardiogram; RIC, intercostal muscle; MIC, neck microphone; SpO2, oxygen saturation measured by pulse oximetry; cannula, nasal cannula–pressure transducer; airflow, mouth thermistor; and chest and abdomen, uncalibrated bands for inductive respiratory plethysmography.
Continuous sustained effort during stage 3 non–rapid eye movement sleep. An increase in respiratory effort is shown on the esophageal pressure (Pes) tracing, associated with some "flattening" and flow limitation, as shown on nasal cannula tracing (during a 30-second segment). See legend to Figure 1 for explanation of abbreviations.
Tachypnea (30 breaths per minute) during non–rapid eye movement sleep in a 10-year-old girl. An increase in respiratory rate and nasal flow limitation (cannula) without an increase in respiratory effort (Pes) in a 30-second segment is shown. See legend to Figure 1 for explanation of abbreviations.
Each polygraphic recording was assigned a number, masking all participant information (control or pretreatment or posttreatment patient). Blinded scorers evaluated randomly distributed polysomnographic recordings. A second scorer reviewed the initial scorings. In case of discrepancies between the first 2 scorers, a third scorer evaluated the recording, and the majority score was used. Before the beginning of the study, scorers participated in training sessions, and they were concordant on 92% of the abnormal breathing patterns. Disagreements (8%) were on scoring of hypopnea, continuous sustained effort, and Pes crescendo patterns. The intervention of the third scorer was requested when discrepancy existed. The κ statistic between the 2 scorers for the study was 0.93.
The tabulated data were entered into a computer matrix, and participant information (clinical symptoms and pretreatment and posttreatment polysomnographic information) was also entered for calculations. The apnea-hypopnea index (AHI) was defined as the number of apnea and hypopnea episodes divided by total sleep time and multiplied by 60, which gives the number of apnea and hypopnea episodes per hour of sleep.
The Mann-Whitney U test was performed for comparison of control subjects and patients before and after treatment. Wilcoxon signed rank test and univariate repeated-measures analysis were used for pretreatment and posttreatment analyses. Percentages were compared by χ2 statistic. Results are presented as mean ± SD.
The mean age of the children with SRBD was 6.5 ± 4.0 years (range, 24.0 months to 12.1 years). Control children (34 girls and 26 boys) had a mean age of 5.9 ± 3.9 years. There were comparable numbers of boys and girls in the SRBD group until 10 years of age. There was a significantly larger ratio of boys in the 10- to 12-year-old group (33 vs 3). This was related, in part, to a more frequent presence of Tanner stage 2 status in girls in this age group, which rendered them ineligible for the study. Table 2 presents the groups and the number of participants with the studied breathing patterns.
Twenty-five percent of children with SRBD and 10% of control subjects had nonapnea, nonhypopnea respiratory patterns.
Thirty children (27 with SRBD, 16 male patients and 3 male controls) had tachypnea as their sole breathing abnormality during sleep. The mean age of patients with SRBD was 6.6 ± 2.1 years (range, 4.8-7.1 years) and 8.1 ± 1.0 years (range, 7.0-10.0 years) for the control subjects. Treatment for children with SRBD was tonsillectomy or adenoidectomy (n = 8) or adenotonsillectomy (n = 19). The symptoms of children with SRBD at entry are presented in Table 3. All children snored. The postsurgery symptoms that clearly decreased are also listed (Wilcoxon signed rank test, P<.001).
Table 3 presents a comparison between presurgical and postsurgical polysomnography. There was a significant difference in the number of 30-second epochs scored as tachypnea between control subjects and children with SRBD at baseline. The mean number of 30-second epochs scored as tachypneic was 151 ± 22 in the SRBD group and 8.3 ± 1.5 in the control group (Mann-Whitney U test, P<.001). On average, the children with SRBD spent approximately 70 minutes (range, 20-117 minutes) of the night with tachypnea. For each epoch scored as tachypnea, the minimum respiratory rate was 21/min in non–rapid eye movement (NREM) sleep (mean, 27/min ± 2/min) and 23/min in rapid eye movement (REM) sleep (mean, 29/min ± 3/min). In the 3 control subjects with tachypnea, all tachypneic epochs were seen in association with bursts of phasic events of REMs during REM sleep (mean, 21.0/min ± 0.5/min). The total amount of tachypnea in these 3 control subjects was less than 1% of total sleep time, whereas the total amount was more than 5% in symptomatic children.
These findings were compared with segments of normal breathing in the recording: the mean breathing frequency was 17.0/min ± 1.0/min and 17.0/min ± 1.2/min during NREM sleep and 18.0/min ± 1.5/min and 18.0/min ± 1.0/min during REM sleep in control subjects and children with SRBD, respectively. Increases in negative peak inspiratory Pes, flow limitation, and oxygen saturation decreases of 3% were not seen with tachypnea (Figure 3).
There was a significant difference after treatment in the number of tachypneic epochs as seen in Table 3 (Wilcoxon signed rank test, P = .001). Parents reported persistence of symptoms in 2 children who had tachypneic epochs at follow-up recording.
Seventy-two children with SRBD (39 boys) and 3 male controls had abnormal breathing based on Pes and nasal cannula curves but without apneas and hypopneas. Their mean age was 4.11 years ± 1 year (range, 2.2-8.1 years) for children with SRBD and 5.4 years (range, 4.0-7.4 years) for control subjects. Treatment for children with SRBD was tonsillectomy or adenoidectomy in 21 children and adenotonsillectomy in 51 children. Orthodontic treatment was recommended for 28 children. Presurgery and postsurgery results are presented in Table 4.
Polysomnograms at baseline showed Pes crescendos and continuous sustained effort. Nasal flow limitation was also seen with the nasal cannula–pressure transducer system in 90% of the 30-second epochs scored as abnormal, based on Pes recording.
Continuous sustained effort was seen only in patients with SRBD (Figure 2). Seventy-eight percent of the continuous sustained effort epochs were observed during slow wave sleep, and none of them were observed during REM sleep.
Pes crescendos were seen in all of the children with SRBD and in 3 control subjects (Figure 1). Polysomnographic patterns differed significantly between control subjects and children with SRBD before surgery (Mann-Whitney U test, P = .001). Esophageal pressure crescendos occurred predominantly in stages 1 and 2 of NREM sleep and REM sleep. This pattern was seen in less than 1% of total sleep time in asymptomatic children, and abnormal breathing effort was seen in more than 10% of total sleep time in symptomatic children.
Continuous sustained effort disappeared in all but one polysomnogram, in which it was noted during slow wave sleep for fourteen 30-second epochs. Polygraphic patterns were significantly different before vs after treatment (Wilcoxon signed rank test, P = .001), but 5 children still had Pes crescendos. After blind scoring of records, children with abnormal nonstandard polygraphic patterns after surgery had persistent symptoms.
Seventy-five percent of children with SRBD but no control subjects belong to this group.
Two hundred eighty-four children (155 boys) had AHIs of more than 1 and less than 10 (Table 5). Their mean age was 6.8 ± 2.4 years (range, 2-12 years). Compared with the nonapneic, nonhypopneic children with SRBD, this group was significantly older (mean age, 6.6 vs 5.1 years; P = .05). Two hundred eleven children underwent adenotonsillectomy (9 of whom had turbinate reduction), and 70 underwent tonsillectomy or adenoidectomy alone. Three children received only nasal turbinate reduction. The mean AHI at baseline was 5.2 ± 2.1 events per hour, but the abnormal nonstandard breathing patterns (Pes crescendos, sustained abnormal breathing effort, and tachypnea) were added to the AHI to produce a mean RDI of 9.1 ± 6.0. Presurgery and postsurgery data are presented in Table 6. There was a significant change in report of symptoms but a persistence of clinical symptoms in 21 cases.
At follow-up polysomnography, there was also clear improvement. The mean AHI for the total group was 0.8 ± 1.9 (Wilcoxon signed rank test, P = .001). However, there were 25 children with residual hypopnea indexes. Fifteen additional children showed persistence of Pes crescendos and continuous sustained effort. The mean postsurgery RDI was 1.8 for the total group and 5.6 ± 2.1 for the 21 children with residual problems (Wilcoxon signed rank test, P = .01). Once again, children with an abnormal RDI, independent of whether their AHI was normal or abnormal, were those who reported clinical symptoms.
The 17 children (12 boys) had a mean age of 7.9 ± 2.8 years (range, 3.1-12.0 years) and were older than the nonapneic, nonhypopneic children with SRBD (P = .01). All of the children underwent adenotonsillectomy. Four also underwent resection of their lateral palatal wall with lateral pillar sutures. Table 6 presents the presurgical and postsurgical findings. Symptoms were improved, but only 6 children reported no symptoms after surgery. At entry, the mean AHI was 26.0 ± 4.5. These children also presented with obstructive hypopneas and Pes crescendos but few (7 ± 3) epochs of continuous sustained effort pattern. The mean RDI was 29 ± 5.
After surgery, the mean AHI was 2 ± 3 (Wilcoxon signed rank test, P = .01, before surgery); the mean RDI was 10 ± 8 (P = .01 compared with before surgery). Three children with persistent clinical complaints had Pes crescendos and continuous sustained effort as the sole indicators of persistently abnormal breathing patterns. Independent of the AHI, the 11 children with elevated RDIs were those with persistent symptoms.
The frequency of predefined nonapneic, nonhypopneic polygraphic patterns was significantly different in control vs presurgery SRBD patient recordings and in presurgery vs postsurgery recordings of children with SRBD. Children with nonapnea, nonhypopnea events had many of the same clinical complaints as those with apnea-hypopnea, but their mean age was younger.
Parents of 55 children agreed to a second treatment trial due to persistence of clinical symptoms. These children were subdivided in 2 groups based on the presence of an AHI of more than 1.5: 31 children had AHIs of more than 1.5 (group A), whereas 24 children had AHIs of less than 1.5 but an RDI of more than 1.5 (group B). No children in group B had oxygen saturation decreases below 95%, whereas 27 group A children had oxygen saturation decreases below this mark (Mann-Whitney U test, P = .001). The mean age was 6.8 ± 3.0 years in group A and 6.1 ± 3.1 years in group B (P>.05). There were 13 girls (42%) in group A and 12 (48%) in group B (P>.05) (Table 7).
Forty-four of these children with residual symptoms underwent tonsillectomy or adenoidectomy alone and either absence of treatment or isolated treatment of enlarged nasal turbinates. However, the 4 children who had the most extensive upper airway surgery were in group A. The pretreatment and posttreatment symptoms are presented in Table 7. As can be seen, there was a significant improvement of reported symptoms after treatment (P = .001), which was associated with a significant improvement (P = .001) of posttreatment RDI. There were 3 children with AHIs between 1 and 3 and RDIs between 5 and 7 after treatment. All other children had AHIs and RDIs below 1.5.
In summary, the pretreatment clinical symptoms were similar, independent of the type of polygraphic patterns (apnea-hypopnea or others), with the exception of the persistence of napping and daytime sleepiness associated only with an AHI of more than 1.5. In addition, treatment improved the RDI significantly in both groups of children (P = .001).
Two children with RDIs of less than 1.5 had residual, postoperative, intermittent snoring and intermittent sleepwalking; one had difficulty with morning arousal and had behavioral problems in school. Polysomnographic results were an AHI of 1.0 and an RDI of 1.4 for one and an AHI of 0.5 and an RDI of 1.2 for the other.
If we select a postsurgery mixed and obstructive AHI cutoff point of more than 1.5, we exclude 24 of 59 children with residual complaints. If our cutoff point is an RDI of more than 1.5, we still exclude 2 children with residual complaints. The lowest RDI with persistent clinical complaint was 1.2.
Previous attempts have been made to define abnormal breathing during sleep in children; however, inadequacies clearly remain.3- 6 Marcus et al7 have reported on children with normal polysomnographic results by published criteria3- 6 but with heavy snoring, abnormally large tonsils, and other clinical symptoms sufficient to lead to tonsillectomy; these children experienced clinical improvement after surgery. Current established polysomnographic scoring criteria have definite insufficiencies, because they miss identifiable breathing abnormalities and do not correlate with clinical complaints, and many children with chronic snoring and clinical symptoms are not considered as having pathologic AHI. Although our work may not address all of the issues at hand, it demonstrates that certain polygraphic patterns not currently scored may provide further information on abnormal breathing during sleep. These patterns are seen with many snoring children, and they are often associated with an AHI that does not lead to treatment. Tabulation of these patterns shows that the respiration during sleep of these children is clearly more abnormal than if only the AHI is tabulated. Tachypnea was already reported to be an important feature to look for in 1982,2 but its analysis and scoring have never been implemented. Other abnormal breathing patterns have been described in adults with SRBDs15- 18 and were used in 2 of our previously published studies in children.8,21 These nonstandard patterns may be rarely seen in healthy children as described herein. When present in nonsymptomatic children, the patterns are seen in less than 1% of total sleep time and most commonly limited to REM sleep. These nonstandard patterns were always seen in more than 5% of total sleep time in symptomatic children and always seen in NREM sleep, not only in REM sleep. The polysomnographic patterns that we report in children can be recognized with technology that is currently commercially available.9- 11
Our approach has inherent limitations. For example, the number of control subjects is limited despite the fact that our study included all ages, and, to our knowledge, we included the largest number of prepubertal children monitored with a combination of nasal cannula and Pes. Also, the data analysis was performed on a retrospective cohort; however, to improve the validity of our findings, we used 2 scorers blinded to the children's health status who scored records randomly. The use of Pes provides an improved recognition of amplitude changes in air exchange, particularly because the scoring was performed visually, compared with the nasal cannula–pressure transducer curve, which is only a semiquantitative technique and may be prone to artifact. Alternative devices to Pes recordings are being sought, but the most promising technology for noninvasive monitoring of intrathoracic effort is not yet commercially available.
Our definitions did not include several criteria currently used to define abnormal breathing pattern during sleep. We did not take into consideration the presence or absence of EEG arousals and oxygen saturation decreases of 3% or more to score the RDI, but we always requested a clear Pes reversal.15 There were 2 reasons for this choice. The extra patterns that we observed are not associated with important upper airway closure, and oxygen saturation decreases of 3% or more per pulse oximetry require larger flow limitation for longer duration.
The EEG changes were visually analyzed, as is currently standard. This approach clearly has limitations, particularly for young children, because EEG arousals in the central leads may not be necessary for airway reopening to occur.19
Clinical symptoms, customarily associated with sleep disturbances related to apnea and hypopnea, were seen also in their absence but with the additional abnormal breathing patterns. These specific respiratory patterns during sleep were seen with presence of much enlarged tonsils at clinical evaluation. Finally, surgical treatments led to an improvement of the reported symptoms and the reduction or disappearance of these other abnormal breathing patterns. When parents reported persistence of symptoms after surgery, persistence of the nonstandard breathing patterns was seen independently of the presence or absence of apnea-hypopnea.
We selected a cutoff point of 1.5 to compare mixed and obstructive AHIs and RDIs. This selection was an artificial choice and was based on clinical practice. Persistence of daytime sleepiness and napping after surgery was only seen in children with AHIs of more than 1.5; however, there was an overlap of other residual complaints between children with AHIs of more than 1.5 and those with RDIs of more than 1.5 and AHIs of less than 1.5, including some with complete absence of apnea and hypopnea in their recordings. However, the 2 children with symptoms and some snoring below all our cutoff points indicate the difficulty of defining the condition at the lower end of the spectrum. The residual problems may not be related to a sleep-related breathing disorder, but it is also at the lower end of the spectrum that we need tests. If the polysomnogram is a standard for surgical or other treatment decisions, it must more appropriately cover the abnormalities of breathing during sleep. Otherwise, it will be unable to detect abnormalities, and children may undergo surgery based on only clinical symptoms and physical findings.7
As with any test in medicine, there will probably be some overlap between normal and pathologic due to the technology used and the visual scoring performed at the low end of the spectrum. Regardless, breathing patterns other than apnea and hypopnea should be scored with polysomnograms, because they add important information on abnormal breathing during sleep.
Last but not least, we were surprised by the number of children who underwent surgery but did not have complete resolution of symptoms. We had already reported that children with SRBD, despite surgical treatment in prepubertal years, might still have abnormal breathing during sleep after puberty.21 Tasker et al22 recently reported similar findings. It is obvious that if residual problems exist early on, there is a greater chance of problems after puberty. This study emphasizes again the need to assess the results of surgery with polysomnography but using appropriate technology and scoring criteria.
Current polysomnographic criteria may be inadequate for identifying breathing abnormalities during sleep, resulting in children with untreated SRBD. The use of appropriate technology and updating current criteria for scoring nocturnal polysomnography can lead to better detection and treatment.
Corresponding author: Christian Guilleminault, MD, BiolD, Stanford University Sleep Disorders Clinic, 401 Quarry Rd, Suite 3301, Stanford, CA 94305 (e-mail: email@example.com).
Accepted for publication July 23, 2003.
Dr Guilleminault was supported by an Academic Award from the Sleep Research Center from the National Heart, Lung, and Blood Institute of the National Institutes of Health, Bethesda, Md.
Country-Specific Mortality and Growth Failure in Infancy and Yound Children and
Association With Material Stature
Use interactive graphics and maps to view and sort country-specific infant and early
dhildhood mortality and growth failure data and their association with maternal
Thank you for submitting a comment on this article. It will be reviewed by JAMA Pediatrics editors. You will be notified when your comment has been published. Comments should not exceed 500 words of text and 10 references.
Do not submit personal medical questions or information that could identify a specific patient, questions about a particular case, or general inquiries to an author. Only content that has not been published, posted, or submitted elsewhere should be submitted. By submitting this Comment, you and any coauthors transfer copyright to the journal if your Comment is posted.
* = Required Field
Disclosure of Any Conflicts of Interest*
Indicate all relevant conflicts of interest of each author below, including all relevant financial interests, activities, and relationships within the past 3 years including, but not limited to, employment, affiliation, grants or funding, consultancies, honoraria or payment, speakers’ bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued. If all authors have none, check "No potential conflicts or relevant financial interests" in the box below. Please also indicate any funding received in support of this work. The information will be posted with your response.
Register and get free email Table of Contents alerts, saved searches, PowerPoint downloads, CME quizzes, and more
Subscribe for full-text access to content from 1998 forward and a host of useful features
Activate your current subscription (AMA members and current subscribers)
Purchase Online Access to this article for 24 hours
Some tools below are only available to our subscribers or users with an online account.
Download citation file:
Web of Science® Times Cited: 27
Customize your page view by dragging & repositioning the boxes below.
and access these and other features:
Enter your username and email address. We'll send you a link to reset your password.
Enter your username and email address. We'll send instructions on how to reset your password to the email address we have on record.
Athens and Shibboleth are access management services that provide single sign-on to protected resources. They replace the multiple user names and passwords necessary to access subscription-based content with a single user name and password that can be entered once per session. It operates independently of a user's location or IP address. If your institution uses Athens or Shibboleth authentication, please contact your site administrator to receive your user name and password.