The consistency and uncertainty in examiners' definitions of pass fail performance on OSCE stations

The Medical Council of Canada has made use of examiners' pass/fail classifications of candidates' behaviors in objective structured clinical examination (OSCE) stations in defining cutting scores for these stations. This process assumes that there is consistency in the judgments of examiners employed in the some stations at different testing sites and in the cutting scores derived from these judgments. These assumptions were tested using the results of the fall 1993 administration of part 2 of the Medical Council of Canada's Evaluating Examination to 744 candidates. The results of this study provided evidence of the consistency of the pass/fail and cutting score definitions for the stations used across examiners.