A generalized examinee-centered method for setting standards on achievement tests

Most standard setting methods can be classified as either test-centered or examinee-centered. Test-centered methods (e.g., the Angoff method, 1971) appear to work well with objective tests but seem less useful with extended-response tests. Examinee-centered methods (e.g., the borderline-groups method and the contrasting-groups method), on the other hand, appear to be particularly appropriate for extended-response tests. In this article, we describe a generalized examinee-centered method for setting multiple cutscores on a test involving both objective and extended-response items. Judges evaluate a representative sample of examinee performances using a rating scale that is defined in terms of performance standards (e.g., proficient, advanced levels). These ratings are linked to examinee's test scores to generate a functional relation between scores and ratings, which is then used to assign a cutscore to each performance level. This approach is potentially more efficient than traditional examinee-centered methods because all ratings are used to define each cutscore. An example involving the setting of multiple cutscores for a state testing program is presented and used to suggest some ways to evaluate the sampling error in the resulting cutscore.