Sample size requirements for increasing the precision of reliability estimates: Problems and proposed solutions

There are a number of cogent arguments to indicate that the investment of time and resources required to render reliability estimates more precise in the usual two-independent-examiner model is simply not worth the time and effort involved. Rather than increasing N dramatically (e.g., from 25 to 400, as the author recommends in one application) one should simply (a) pay attention to the type of subject being assessed and, (b) as required, increase modestly the number of independent examiners (e.g., from 2 to 4 makes a dramatic difference), and (c) above all, train, train, train the examiners!.