On the LASSO and its dual

被引:409
作者
Osborne, MR [1 ]
Presnell, B
Turlach, BA
机构
[1] Australian Natl Univ, Ctr Math & Applicat, Canberra, ACT 0200, Australia
[2] Univ Florida, Dept Stat, Gainesville, FL 32611 USA
[3] Univ Western Australia, Dept Math & Stat, Nedlands, WA 6907, Australia
关键词
convex programming; dual problem; partial least squares; penalized regression; quadratic programming; regression; shrinkage; subset selection; variable selection;
D O I
10.2307/1390657
中图分类号
O21 [概率论与数理统计]; C8 [统计学];
学科分类号
020208 ; 070103 ; 0714 ;
摘要
Proposed by Tibshirani, the least absolute shrinkage and selection operator (LASSO) estimates a vector of regression coefficients by minimizing the residual sum of squares subject to a constraint on the l(1)-norm of the coefficient vector. The LASSO estimator typically has one or more zero elements and thus shares characteristics of both shrinkage estimation and variable selection. In this article we treat the LASSO as a convex programming problem and derive its dual. Consideration of the primal and dual problems together leads to important new insights into the characteristics of the LASSO estimator and to an improved method for estimating its covariance matrix. Using these results we also develop an efficient algorithm for computing LASSO estimates which is usable even in cases where the number of regressors exceeds the number of observations. An S-Plus library based on this algorithm is available from StatLib.
引用
收藏
页码:319 / 337
页数:19
相关论文
共 19 条
[1]  
Brown P. J., 1993, MEASUREMENT REGRESSI
[2]   Atomic decomposition by basis pursuit [J].
Chen, SSB ;
Donoho, DL ;
Saunders, MA .
SIAM JOURNAL ON SCIENTIFIC COMPUTING, 1998, 20 (01) :33-61
[3]   ON LINEAR RESTRICTED AND INTERVAL LEAST-SQUARES PROBLEMS [J].
CLARK, DI ;
OSBORNE, MR .
IMA JOURNAL OF NUMERICAL ANALYSIS, 1988, 8 (01) :23-36
[4]   A STATISTICAL VIEW OF SOME CHEMOMETRICS REGRESSION TOOLS [J].
FRANK, IE ;
FRIEDMAN, JH .
TECHNOMETRICS, 1993, 35 (02) :109-135
[5]   Penalized regressions: The bridge versus the lasso [J].
Fu, WJJ .
JOURNAL OF COMPUTATIONAL AND GRAPHICAL STATISTICS, 1998, 7 (03) :397-416
[6]  
Gallant A.R., 1987, NONLINEAR STAT MODEL
[7]  
HAAGEN K, 1993, STAT MODELLING LATEN
[8]  
Koch I, 1996, ANN STAT, V24, P1648
[9]   An extension of the mixed primal-dual bases algorithm to the case of more constraints than dimensions [J].
Meyer, MC .
JOURNAL OF STATISTICAL PLANNING AND INFERENCE, 1999, 81 (01) :13-31
[10]  
MILLER AJ, 1990, MONOGRAPHS STAT APPL, V40