On Bayesian analysis of multirater ordinal data: An application to automated essay grading

被引：51

作者：

Johnson, VE

机构：

来源：

JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION | 1996年 / 91卷 / 433期

关键词：

Bayesian inference; categorical data; Gibbs sampling; hierarchical models; latent structure models; Markov chain Monte Carlo;

D O I：

10.2307/2291381

中图分类号：

O21 [概率论与数理统计]; C8 [统计学];

学科分类号：

020208 ; 070103 ; 0714 ;

摘要：

A framework is proposed for the analysis of ordinal categorical data when ratings from several judges are available. I emphasize the tasks of estimating latent trait characteristics of individual items, regressing these latent traits on observed covariates, and comparing the performance of raters. The model is illustrated in the design and evaluation of an automated essay grader. This grader is based on a regression of variables, obtained from a grammar checker, on essay scores estimated from a panel of experts. The performance of the grader is evaluated relative to human graders, and implications on the reliability and repeatability of both automated and human raters is investigated.

引用

页码：42 / 51

页数：10

共 11 条

[1] BAYESIAN-ESTIMATION OF NORMAL OGIVE ITEM RESPONSE CURVES USING GIBBS SAMPLING [J].