Evaluating the quality of reporting occupational therapy randomized controlled trials by expanding the CONSORT criteria

CONTEXT. Well-reported randomized controlled trials (RCTs) are necessary for evidenced-based practice in the field of occupational therapy. OBJECTIVE. The primary purpose of this study was to analyze the quality of reporting in selected RCTs relevant to occupational therapy by using the Nelson-Moberg Expanded CONSORT Instrument (NMECI). The interrater reliability of the NMECI was also tested. METHOD. The 201-sub-item NMECI was developed by the authors to clarify the 22-item CONSORT Statement, each item of which raises several issues. Fourteen RCTs obtained through the OTseeker database were analyzed independently by the authors. RESULTS. The 14 articles complied with slightly more than half of the sub-items of the NMECI (M= 104.2, SD = 32.9 for the main rater). The only area of full compliance involved reporting of objectives and hypotheses, areas of frequent noncompliance included reporting of adverse events, randomization, and blinding. The intraclass correlation coefficient assessing level of interrater agreement on the total NMECI score was .95, indicating strong agreement overall. For the 176 kappas that could be computed on individual sub-items, the median kappa was .85, with an inter-quartile range from .58 to 1, indicating high levels of agreement for most items. However, 29 kappas fell below the moderate level of agreement (40). CONCLUSIONS. The field of occupational therapy needs increased sophistication in conducting and reporting RCTs. The NMECI shows promise in evaluating the quality of RCTs, given refinements to some items.