Is the reliability of a visual analog scale higher than an ordinal scale? An experiment with the GRBAS scale for the perceptual evaluation of dysphonia

Perceptual evaluation of 14 pathological voices was performed by 29 listeners using the GRBAS scale. To evaluate scale effects on the judgments, 2 versions of the scale were presented: the original 4-point scale and a visual analog scale. Each listener used the same voice samples for both versions of the scale with an interval of 2 weeks. Agreement was found to be higher with the original 4-point scale than with the visual analog version for all scale items G, R, B, A and S. Although a visual analog scale seems to enable a finer judgment of voice quality, this study showed that, with increased freedom of judgment, the interrater agreement decreased considerably. Therefore, we recommend the use of the original 4-point version of the GRBAS scale.