Observer sensitivity to small differences: A multipoint rank-order experiment

OBJECTIVE, To evaluate observer sensitivity to small differences in image presentation, a multipoint rank-order experiment was used to identify small differences or trends in observations. MATERIALS AND METHODS. Ten observers were presented with 50 sets of breast images that had been compressed at five different levels. Each set contained six images ranging from noncompressed to approximately 101:1 compression. Observers were asked to review all images of a case side by side and rank order the quality of each to enable determination of the presence or absence of masses and clustered microcalcifications. RESULTS. As a group, observers were able to detect small differences among the images, even at the lower compression levels (p <.001). As compression levels and image degradation increased, the ability to identify differences between different modes also increased. Large observer variability in discrimination ability was observed. CONCLUSION. Multipoint rank ordering of images viewed side by side can be an efficient method to identify small differences in image presentation. This approach to image ranking could be used to rule out or confirm the need for objective observer performance-type studies.