LOWER BOUND THEORY OF NONZERO ENTRIES IN SOLUTIONS OF l2-lp MINIMIZATION

被引:217
作者
Chen, Xiaojun [1 ]
Xu, Fengmin [2 ,3 ]
Ye, Yinyu [4 ]
机构
[1] Hong Kong Polytech Univ, Dept Appl Math, Kowloon, Hong Kong, Peoples R China
[2] Xi An Jiao Tong Univ, Sch Sci, Xian, Peoples R China
[3] Xi An Jiao Tong Univ, Dept Comp Sci & Technol, Xian, Peoples R China
[4] Stanford Univ, Dept Management Sci & Engn, Sch Engn, Stanford, CA 94305 USA
关键词
variable selection; sparse solution; linear least-squares problem; l(p) regularization; smoothing approximation; first order condition; second order condition; GRADIENT SAMPLING ALGORITHM; NONSMOOTH; RECONSTRUCTION; REGRESSION; SIGNALS; SELECTION; RECOVERY; IMAGES;
D O I
10.1137/090761471
中图分类号
O29 [应用数学];
学科分类号
070104 [应用数学];
摘要
Recently, variable selection and sparse reconstruction are solved by finding an optimal solution of a minimization model, where the objective function is the sum of a data-fitting term in l(2) norm and a regularization term in l(p) norm (0 < p < 1). Since it is a nonconvex model, most algorithms for solving the problem can provide only an approximate local optimal solution, where nonzero entries in the solution cannot be identified theoretically. In this paper, we establish lower bounds for the absolute value of nonzero entries in every local optimal solution of the model, which can be used to indentify zero entries precisely in any numerical solution. Therefore, we have developed a lower bound theorem to classify zero and nonzero entries in every local solution. These lower bounds clearly show the relationship between the sparsity of the solution and the choice of the regularization parameter and norm so that our theorem can be used for selecting desired model parameters and norms. Furthermore, we also develop error bounds for verifying the accuracy of numerical solutions of the l(2)-(p) minimization model. To demonstrate applications of our theory, we propose a hybrid orthogonal matching pursuit-smoothing gradient (OMP-SG) method for solving the nonconvex, non-Lipschitz continuous l(2)-l(p) minimization problem. Computational results show the effectiveness of the lower bounds for identifying nonzero entries in numerical solutions and the OMP-SG method for finding a high quality numerical solution.
引用
收藏
页码:2832 / 2852
页数:21
相关论文
共 29 条
[1]
[Anonymous], 1990, CLASSICS APPL MATH
[2]
[Anonymous], INFERENCE PREDICTION
[3]
BLAKE C, 1998, REPOSIT MACHINE LEAR
[4]
BETTER SUBSET REGRESSION USING THE NONNEGATIVE GARROTE [J].
BREIMAN, L .
TECHNOMETRICS, 1995, 37 (04) :373-384
[5]
From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images [J].
Bruckstein, Alfred M. ;
Donoho, David L. ;
Elad, Michael .
SIAM REVIEW, 2009, 51 (01) :34-81
[6]
A robust gradient sampling algorithm for nonsmooth, nonconvex optimization [J].
Burke, JV ;
Lewis, AS ;
Overton, ML .
SIAM JOURNAL ON OPTIMIZATION, 2005, 15 (03) :751-779
[7]
Robust uncertainty principles:: Exact signal reconstruction from highly incomplete frequency information [J].
Candès, EJ ;
Romberg, J ;
Tao, T .
IEEE TRANSACTIONS ON INFORMATION THEORY, 2006, 52 (02) :489-509
[8]
Enhancing Sparsity by Reweighted l1 Minimization [J].
Candes, Emmanuel J. ;
Wakin, Michael B. ;
Boyd, Stephen P. .
JOURNAL OF FOURIER ANALYSIS AND APPLICATIONS, 2008, 14 (5-6) :877-905
[9]
Chartrand R, 2007, IEEE IMAGE PROC, P293
[10]
Iteratively reweighted algorithms for compressive sensing [J].
Chartrand, Rick ;
Yin, Wotao .
2008 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING, VOLS 1-12, 2008, :3869-+