Estimating the tail index

Two very general classes of estimators have been proposed for the tail index of a distribution with a regularly varying upper tail. One, by S. Csorgo, Deheuvels and Mason (1985), is the class of kernel estimators, the other, by Viharos (1997), is the class of universally asymptotically normal weighted doubly logarithmic least-squares estimators. Besides the respective kernel and weight functions, both depend upon the number of extreme order statistics used in the estimator. First we argue that for practical purposes the kernel class may reasonably be represented by a subclass indexed by 89 values of a shape parameter, while the weighted least-squares class may reasonably be represented by a subclass indexed by 111 values of another shape parameter. The reduced kernel class contains Hill's (1975) estimator, while the reduced weighted least-squares class contains Zipf's (1949) ordinary unweighted doubly logarithmic least-squares estimator. Our main concern here is the simultaneous adaptive choice of the shape parameter and the number of extremes for both of the two classes by means of minimizing estimators of the asymptotic mean-square errors. We consider two methods for each of the two classes: one takes the estimated bias from the median of all estimators in the class, the other from the average of all estimators in the class. We report on a large-scale simulation study, the first of its kind, for the investigation of the performance of the resulting four estimators, involving twelve slowly varying functions and the sample sizes 100, 300, 500, 700, 1000, 1500 for each of the seven different index values considered. We apply the procedures to 22 data sets from Danish and Norwegian fire insurance. While presenting the main ideas, we also review a substantial part of the literature.