News or Noise?

This study investigates the quality of the U.S. News annual ranking of national universities and liberal arts colleges. The main finding is that current rankings changes have a strong tendency to revert over the following two rankings. Using a simple model, this study estimates that about 70 to 80 percent of the variation in rankings changes is transitory and reversible. Thus, most of the “news” in the annual rankings is essentially meaningless noise. An analysis of possible explanations suggests that the noise in annual ranking changes is most likely due to various measurement, estimation, and other information processing errors in the rankings' underlying components.