Evaluating rank histograms using decompositions of the chi-square test statistic

Rank histograms are often plotted to evaluate the forecasts produced by an ensemble forecasting system-an ideal rank histogram is "flat" or uniform. It has been noted previously that the obvious test of "flatness," the well-known chi(2) goodness-of-fit test, spreads its power thinly and hence is not good at detecting specific alternatives to flatness, such as bias or over- or underdispersion. Members of the Cramer-von Mises family of tests do much better in this respect. An alternative to using the Cramer-von Mises family is to decompose the chi(2) test statistic into components that correspond to specific alternatives. This approach is described in the present paper. It is arguably easier to use and more flexible than the Cramer-von Mises family of tests, and does at least as well as it in detecting alternatives corresponding to bias and over- or underdispersion.