A multirater validation study to assess the reliability of acne lesion counting

Background: Despite widespread use of acne lesion counting, little has been published on its reliability, particularly for multiple raters. Objective: Our purpose was to assess reliability of acne lesion counting with the use of a five-segment facial template. Methods: After training, 12 raters each evaluated 12 patients in randomized order, in the morning and again in the afternoon, and recorded counts for different types of lesions on a five-segment facial template. Results: Individual raters could reproduce their total lesion counts (reliability estimates, 0.81 to 0.97). Variability between raters was high, and overall reliability estimated across raters was 0.61. For a subgroup of commonly trained raters, overall reliability was higher (0.80). Conclusion: The reliability of acne lesion counting is excellent when performed by the same trained rater over time. The high variability between raters appears to be reduced by standardized training. Because fewer lesions are counted with less variation, use of a template may have contributed to the high within-rater reliability.