A new pruning heuristic based on variance analysis of sensitivity information

被引：138

作者：

Engelbrecht, AP ^{[1
]}

机构：

[1] Univ Pretoria, Sch Informat Technol, Dept Comp Sci, ZA-0002 Pretoria, South Africa

来源：

IEEE TRANSACTIONS ON NEURAL NETWORKS | 2001年 / 12卷 / 06期

关键词：

feedforward neural networks (NNs); parameter significance; pruning; sensitivity analysis; variance analysis;

D O I：

10.1109/72.963775

中图分类号：

TP18 [人工智能理论];

学科分类号：

081104 ; 0812 ; 0835 ; 1405 ;

摘要：

Architecture selection is a very important aspect in the design of neural networks (NNs) to optimally tune performance and computational complexity. Sensitivity analysis has been used successfully to prune irrelevant parameters from feedforward NNs. This paper presents a new pruning algorithm that uses sensitivity analysis to quantify the relevance of input and hidden units. A new statistical pruning heuristic is proposed, based on variance analysis, to decide which units to prune. The basic idea is that a parameter with a variance in sensitivity not significantly different from zero, is irrelevant and can be removed. Experimental results show that the new pruning algorithm correctly prunes irrelevant input and hidden units. The new pruning algorithm is also compared with standard pruning algorithms.

引用

页码：1386 / 1399

页数：14

共 58 条

[31] THE EVIDENCE FRAMEWORK APPLIED TO CLASSIFICATION NETWORKS [J].

MACKAY, DJC .

NEURAL COMPUTATION, 1992, 4 (05) :720-736

[32] A PRACTICAL BAYESIAN FRAMEWORK FOR BACKPROPAGATION NETWORKS [J].

MACKAY, DJC .

NEURAL COMPUTATION, 1992, 4 (03) :448-472

[33]

MacKay DJC., 1992, THESIS CALTECH

[34]

MOODY JE, 1993, STAT NEURAL NETWORKS, P147

[35] The dependence identification neural network construction algorithm [J].

Moody, JO ;

Antsaklis, PJ .

IEEE TRANSACTIONS ON NEURAL NETWORKS, 1996, 7 (01) :3-15

[36]

Mozer M. C., 1989, ADV NEURAL INFORMATI, P107

[37] NETWORK INFORMATION CRITERION - DETERMINING THE NUMBER OF HIDDEN UNITS FOR AN ARTIFICIAL NEURAL-NETWORK MODEL [J].

MURATA, N ;

YOSHIZAWA, S ;

AMARI, S .

IEEE TRANSACTIONS ON NEURAL NETWORKS, 1994, 5 (06) :865-872

[38]

MURATA N, 1993, ADV NEURAL INFORMATI, V5, P607

[39]

Pedersen MW, 1996, ADV NEUR IN, V8, P521

[40] Input feature extraction for multilayered perceptrons using supervised principal component analysis [J].

Perantonis, SJ ;

Virvilis, V .

NEURAL PROCESSING LETTERS, 1999, 10 (03) :243-252

← 1 2 3 4 5 6 →