A diagnostic for selecting the threshold in extreme value analysis

A new approach is suggested for choosing the threshold when fitting the Hill estimator of a tail exponent to extreme value data. Our method is based on an easily computed diagnostic, which in turn is founded directly on the Hill estimator itself, 'symmetrized' to remove the effect of the tail exponent but designed to emphasize biases in estimates of that exponent. The attractions of the method are its accuracy, its simplicity and the generality with which it applies. This generality implies that the technique has somewhat different goals from more conventional approaches, which are designed to accommodate the minor component of a postulated two-component Pareto mixture. Our approach does not rely on the second component being Pareto distributed. Nevertheless, in the conventional setting it performs competitively with recently proposed methods, and in more general cases it achieves optimal rates of convergence. A by-product of our development is a very simple and practicable exponential approximation to the distribution of the Hill estimator under departures from the Pareto distribution.