The effect of training on rater reliability on the scoring of the NART

Objectives. This study investigates whether the accuracy of judging National Adult Reading Test (NART) words known to have lower inter-rater reliability can be improved by training and use of the pronunciation guide. Design. Two groups (Experimental and Control), were compared with three repeated measures: Occasion (first and second i.e. 'post-training'), Word Reliability thigh and low) and Pronunciation Guide (without and with guide). Methods. Ten words were selected from the NART: five lower reliability and five high reliability words. These were presented aurally in correct and incorrect form to participants (N = 20) who judged correctness of pronunciation without or with a pronunciation guide. Each group repeated the task again, the Experimental group having received training. Results. Accuracy was significantly worse for the low reliability words. The experimental group's accuracy was significantly better after training than the control group's and their own performance prior to training. The use of the guide enhanced accuracy, particularly for the low reliability words. Conclusion. Training in administration of the NART improves raters' accuracy and use of the pronunciation guide. This offers an alternative to the suggestion of improving the NART's reliability by replacing lower reliability words and therefore would avoid the need to re-standardize a modified test.