0
We're unable to sign you in at this time. Please try again in a few minutes.
Retry
We were able to sign you in, but your subscription(s) could not be found. Please try again in a few minutes.
Retry
There may be a problem with your account. Please contact the AMA Service Center to resolve this issue.
Contact the AMA Service Center:
Telephone: 1 (800) 262-2350 or 1 (312) 670-7827  *   Email: subscriptions@jamanetwork.com
Error Message ......
Research Letter |

Accuracy of Bayley Scores as Outcome Measures in Trials of Neonatal Therapies FREE

Lorrie Costantini, BA1; Judy D'Ilario, RN1; Diane Moddemann, MD2; Karen Penner, MSc2; Barbara Schmidt, MD, MSc1,3
[+] Author Affiliations
1Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Ontario, Canada
2Department of Pediatrics and Child Health, University of Manitoba, Winnipeg, Manitoba, Canada
3Division of Neonatology, Children’s Hospital of Philadelphia and University of Pennsylvania, Philadelphia
JAMA Pediatr. 2015;169(2):188-189. doi:10.1001/jamapediatrics.2014.2965.
Text Size: A A A
Published online

Long-term follow-up of extremely preterm infants is essential because the proportion of infants who survive is increasing while neurodevelopmental impairment rates remain high.1 Psychometric tests, such as the Bayley Scales of Infant and Toddler Development, Third Edition (Bayley-III),2 are commonly used to measure early childhood outcomes in obstetric and neonatal randomized trials.3,4 However, the composite scores on psychometric tests can be inaccurate because of administrative and technical errors. To determine the magnitude of this problem, we performed a post hoc review of all errors detected by central source document verification of Bayley-III assessments during the 18-month follow-up of the Canadian Oxygen Trial (COT).5

METHODS

Extremely preterm COT participants in 25 centers in 6 countries were randomly assigned to 2 oxygen saturation target ranges. The primary outcome at a corrected age of 18 to 21 months was death or survival with neurodevelopmental disability. Cognitive or language composite scores of less than 85 on the Bayley-III scales were included in the primary outcome.5 The research ethics boards of all clinical centers approved the protocol, and written informed consent was obtained from a parent or guardian of every study infant.

Original Review Process During COT

Experienced examiners administered the Bayley-III test and submitted copies of the source documents to the coordinating center. Between October 15, 2008, and August 15, 2012, trained staff (J.D. and L.C.) reviewed the source documents for accuracy, completeness, and agreement with the electronic database entries. Discrepancies were identified and corrected through a formal data clarification process with input from Bayley-III examiners at the respective study sites. A small number of queries (n = 36) could not be resolved directly with the clinical centers and were referred to an adjudication committee consisting of a developmental pediatrician (D.M.) and a neurodevelopmental consultant (K.P.).

Post Hoc Analysis

In this study, 1 assessor (J.D.) reexamined all Bayley-III source documents and classified the errors that were identified during the original review process into 5 categories: calculation of the child’s corrected age, documenting or applying scoring rules, raw score addition, look-up of scaled scores and composite scores in normative tables, and electronic data entry. A 15% random sample was independently classified by a second assessor (L.C.) to ensure consistent classification of error types. Only 2 disagreements were found and resolved through consensus. The error categories are hierarchical in nature. An early mistake—for example, calculation of the corrected age—may affect all subsequent steps. We counted such errors only once, assigned them to the category in which they first occurred, and summarized the frequency of independent errors in each of the 5 categories.

RESULTS

During COT follow-up, the source documents for 936 of 954 (98.1%) Bayley-III assessments were submitted to the coordinating center. Eighteen children could not be evaluated because of severe developmental delay or autism. Of 936 source documents, 576 (61.5%) contained no errors. The remaining 360 (38.5%) contained at least 1 error, and the total number of independent errors was 387 (Table). None of the 25 clinical centers were completely error free. The best and worst center-specific error rates were 8 of 61 (13.1%) and 14 of 16 (87.5%), respectively. Had they not been detected during the original review process, 41 of 387 (10.6%) incorrectly reported composite scores would have changed the determination of the composite primary outcome in COT.

Table Graphic Jump LocationTable.  Type and Frequency of Errors in Bayley-III Assessments

DISCUSSION

Experienced Bayley-III examiners made numerous administrative, scoring, and reporting errors in this international multicenter trial. Errors were detected and corrected in real time by comprehensive and rigorous central source document verification. The use of computer-assisted scoring software may reduce calculation and table look-up errors but cannot correct the effects of administrative or clerical errors that are made before data are entered into the program.

CONCLUSIONS

This study underscores the importance of diligent administration and recording of psychometric assessments. We recommend central source document verification of all psychometric tests that contribute to the primary outcome in large multicenter trials of perinatal and neonatal therapies.

ARTICLE INFORMATION

Corresponding Author: Lorrie Costantini, BA, Department of Clinical Epidemiology and Biostatistics, McMaster University, Neonatal Trials Group, DBCVSRI Hamilton General Hospital Campus, 237 Barton St East, Hamilton, ON L8L 2X2, Canada (costan@mcmaster.ca).

Published Online: December 29, 2014. doi:10.1001/jamapediatrics.2014.2965.

Author Contributions: Mss Costantini and D’Ilario had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Costantini,Schmidt.

Acquisition, analysis, or interpretation of data: Costantini, D’Ilario, Moddemann, Penner.

Drafting of the manuscript: Costantini.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Costantini.

Administrative, technical, or material support: Costantini,D’Ilario, Moddemann, Penner.

Study supervision: Schmidt.

Conflict of Interest Disclosures: None reported.

Funding/Support: The Canadian Oxygen Trial was supported by grant MCT-79217 from the Canadian Institutes for Health Research (Drs Schmidt and Moddemann).

Role of the Funder/Sponsor: The funding source had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

REFERENCES

Vohr  BR.  Neurodevelopmental outcomes of extremely preterm infants. Clin Perinatol. 2014;41(1):241-255.
PubMed   |  Link to Article
Bayley  N. Bayley Scales of Infant and Toddler Development.3rd ed. San Antonio, TX: Psychological Corp; 2006.
Benjamin  DK  Jr, Hudak  ML, Duara  S,  et al; Fluconazole Prophylaxis Study Team.  Effect of fluconazole prophylaxis on candidiasis and mortality in premature infants: a randomized clinical trial. JAMA. 2014;311(17):1742-1749.
PubMed   |  Link to Article
Wapner  RJ, Sorokin  Y, Mele  L,  et al; National Institute of Child Health and Human Development Maternal-Fetal Medicine Units Network.  Long-term outcomes after repeat doses of antenatal corticosteroids. N Engl J Med. 2007;357(12):1190-1198.
PubMed   |  Link to Article
Schmidt  B, Whyte  RK, Asztalos  EV,  et al; Canadian Oxygen Trial (COT) Group.  Effects of targeting higher vs lower arterial oxygen saturations on death or disability in extremely preterm infants: a randomized clinical trial. JAMA. 2013;309(20):2111-2120.
PubMed   |  Link to Article

Figures

Tables

Table Graphic Jump LocationTable.  Type and Frequency of Errors in Bayley-III Assessments

References

Vohr  BR.  Neurodevelopmental outcomes of extremely preterm infants. Clin Perinatol. 2014;41(1):241-255.
PubMed   |  Link to Article
Bayley  N. Bayley Scales of Infant and Toddler Development.3rd ed. San Antonio, TX: Psychological Corp; 2006.
Benjamin  DK  Jr, Hudak  ML, Duara  S,  et al; Fluconazole Prophylaxis Study Team.  Effect of fluconazole prophylaxis on candidiasis and mortality in premature infants: a randomized clinical trial. JAMA. 2014;311(17):1742-1749.
PubMed   |  Link to Article
Wapner  RJ, Sorokin  Y, Mele  L,  et al; National Institute of Child Health and Human Development Maternal-Fetal Medicine Units Network.  Long-term outcomes after repeat doses of antenatal corticosteroids. N Engl J Med. 2007;357(12):1190-1198.
PubMed   |  Link to Article
Schmidt  B, Whyte  RK, Asztalos  EV,  et al; Canadian Oxygen Trial (COT) Group.  Effects of targeting higher vs lower arterial oxygen saturations on death or disability in extremely preterm infants: a randomized clinical trial. JAMA. 2013;309(20):2111-2120.
PubMed   |  Link to Article

Correspondence

CME
Also Meets CME requirements for:
Browse CME for all U.S. States
Accreditation Information
The American Medical Association is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians. The AMA designates this journal-based CME activity for a maximum of 1 AMA PRA Category 1 CreditTM per course. Physicians should claim only the credit commensurate with the extent of their participation in the activity. Physicians who complete the CME course and score at least 80% correct on the quiz are eligible for AMA PRA Category 1 CreditTM.
Note: You must get at least of the answers correct to pass this quiz.
Please click the checkbox indicating that you have read the full article in order to submit your answers.
Your answers have been saved for later.
You have not filled in all the answers to complete this quiz
The following questions were not answered:
Sorry, you have unsuccessfully completed this CME quiz with a score of
The following questions were not answered correctly:
Commitment to Change (optional):
Indicate what change(s) you will implement in your practice, if any, based on this CME course.
Your quiz results:
The filled radio buttons indicate your responses. The preferred responses are highlighted
For CME Course: A Proposed Model for Initial Assessment and Management of Acute Heart Failure Syndromes
Indicate what changes(s) you will implement in your practice, if any, based on this CME course.

Multimedia

Some tools below are only available to our subscribers or users with an online account.

869 Views
1 Citations
×

Related Content

Customize your page view by dragging & repositioning the boxes below.

Articles Related By Topic
Related Collections
PubMed Articles
Jobs