Investigating the quality of teacher-produced tests for EFL students and the effects of training in test development principles and practices on improving test quality

This paper examines the quality of tests that Hong Kong teachers of English as a Foreign Language (EFL) produce for their own EFL students. The paper examines the effects on graduate teachers of a language testing programme where participants produced objective tests, proceeding through the stages of test specification, moderation, and item analysis and test refinement. Subsequently, they were asked to reflect on the test development process they had experienced, and to examine their test data for quality in terms of classical test statistics. Test reliability statistics varied; test means were generally acceptable although some tests were rather too easy or too difficult. In general, the number of 'good' items produced was, however, not as high as participants expected. Participants commented that the experience was revealing in terms of setting test specifications and test moderation which few had ever undertaken, noting that they had become aware that tests they had previously produced had not provided them with usable, accurate information about their students' abilities. The paper concludes that, if classroom tests are to accurately report test takers' abilities, all areas of training in test development and production and, in particular, the empowerment of teachers through support and training in the principles of assessment require attention. (c) 2009 Elsevier Ltd. All rights reserved.