True Reproducibility of UltraSound Techniques (TRUST): systematic review of reliability studies in obstetrics and gynecology

Objectives To examine the quality of methods used and the accuracy of the interpretation of agreement in existing studies that examine the reliability of ultrasound measurements and judgments in obstetrics and gynecology. Methods A systematic search of MEDLINE was performed on 25 March 2014, looking for studies that examined the reliability of ultrasound measurements and judgments in obstetrics and gynecology with evaluation of concordance (CCC) or intraclass (ICC) correlation coefficients or kappa as a main objective. Results Seven hundred and thirty-three records were examined on the basis of their title and abstract, of which 141 full-text articles were examined completely for eligibility. We excluded 2(studies because they did not report CCC/ICC/kappa, leaving 112 studies that were included in our analysis. Two studies reported both ICC and kappa and were counted twice, therefore, the number used as the denominator in the analyses was 114. Only 16/114 (14.0%) studies were considered to be well designed (independent acquisition and blinded analysis) and to have interpreted the results properly. Most errors occurring in the studies are likely to overestimate the reliability of the method examined. Conclusions The vast majority of published studies examined had important flaws in design, interpretation and/or reporting. Such limitations are important to identify as they might create false confidence in the existing measurements and judgments, jeopardizing clinical practice and future research. Specific guidelines aimed at improving the quality of reproducibility studies that examine ultrasound methods should be encouraged. Copyright (C) 2014 ISUOG. Published by John Wiley & Sons Ltd.