Investigation of clinician agreement in evaluating movement quality during unilateral lower extremity functional tasks: A comparison of 2 rating methods

STUDY DESIGN: Nonexperimental OBJECTIVES: To determine interrater and intrarater agreement for 2 methods of evaluating movement quality during 2 lower extremity functional tasks, and to descriptively compare levels of agreement between the 2 methods. BACKGROUND: Clinicians typically use observational analysis to evaluate movement quality during functional tasks, but the extent of agreement is unknown. METHODS AND MEASURES: Twenty-five uninjured subjects performed 3 trials of unilateral squat and lateral step-down tasks, Three clinicians evaluated the trunk, pelvis, and hips for coronal plane and transverse plane movement deviations. Two rating methods were used: assessment of the entire movement ("overall method") and rating each segment individually ("specific method"). Movement deviation severity was rated using basic clinical guidelines and ratings were repeated from videotape. Percent agreement and weighted kappa coefficients were calculated between rater pairs and rating sessions. Generalized kappa coefficients were calculated across raters. RESULTS: Interrater and intrarater percent agreement were higher using the overall method. Interrater weighted kappa coefficients were similar between rating methods (overall method, 0-0.55; specific method, 0.23-0.53). Intrarater weighted kappa coefficients were higher for the specific method (0.38-0.68) compared to the overall method (0.13-0.50). Generalized kappa coefficients were also higher for specific method compared to the overall method (unilateral squat, 0.19 and 0.01, respectively; lateral step-down, 0.22 and 0.18, respectively) and 95% confidence intervals remained above zero. CONCLUSIONS: Rating movement at body segments appears to result in agreement among raters that is better than chance. Neither rating method produced high agreement, indicating a need to develop more explicit criteria for rating movement deviation severity.