Fuzzy Restricted Boltzmann Machine for the Enhancement of Deep Learning

被引:171
作者
Chen, C. L. Philip [1 ]
Zhang, Chun-Yang [1 ]
Chen, Long [1 ]
Gan, Min [2 ]
机构
[1] Univ Macau, Fac Sci & Technol, Dept Comp & Informat Sci, Macau 999078, Peoples R China
[2] Hefei Univ Technol, Dept Comp & Informat Sci, Hefei 230009, Peoples R China
基金
中国国家自然科学基金;
关键词
Deep learning; fuzzy deep networks; fuzzy restricted Boltzmann machine; image classification; image inpainting; restricted Boltzmann machine (RBM); BAYESIAN-ESTIMATION; COEFFICIENT;
D O I
10.1109/TFUZZ.2015.2406889
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
In recent years, deep learning caves out a research wave in machine learning. With outstanding performance, more and more applications of deep learning in pattern recognition, image recognition, speech recognition, and video processing have been developed. Restricted Boltzmann machine (RBM) plays an important role in current deep learning techniques, as most of existing deep networks are based on or related to it. For regular RBM, the relationships between visible units and hidden units are restricted to be constants. This restriction will certainly downgrade the representation capability of the RBM. To avoid this flaw and enhance deep learning capability, the fuzzy restricted Boltzmann machine (FRBM) and its learning algorithm are proposed in this paper, in which the parameters governing the model are replaced by fuzzy numbers. This way, the original RBM becomes a special case in the FRBM, when there is no fuzziness in the FRBM model. In the process of learning FRBM, the fuzzy free energy function is defuzzified before the probability is defined. The experimental results based on bar-and-stripe benchmark inpainting and MNIST handwritten digits classification problems show that the representation capability of FRBM model is significantly better than the traditional RBM. Additionally, the FRBM also reveals better robustness property compared with RBM when the training data are contaminated by noises.
引用
收藏
页码:2163 / 2173
页数:11
相关论文
共 37 条
  • [1] [Anonymous], 2007, P 24 INT C MACHINE L
  • [2] [Anonymous], 1995, Fuzzy Sets and Fuzzy Logic: Theory and Applications
  • [3] [Anonymous], FUZZY PROBABILITIES
  • [4] [Anonymous], 2010, Momentum
  • [5] Aoyagi M, 2013, J ALGEBR STUD, V4, P31
  • [6] Learning Coefficient of Generalization Error in Bayesian Estimation and Vandermonde Matrix-Type Singularity
    Aoyagi, Miki
    Nagata, Kenji
    [J]. NEURAL COMPUTATION, 2012, 24 (06) : 1569 - 1610
  • [7] Representation Learning: A Review and New Perspectives
    Bengio, Yoshua
    Courville, Aaron
    Vincent, Pascal
    [J]. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2013, 35 (08) : 1798 - 1828
  • [8] Learning Deep Architectures for AI
    Bengio, Yoshua
    [J]. FOUNDATIONS AND TRENDS IN MACHINE LEARNING, 2009, 2 (01): : 1 - 127
  • [9] Bengio Yoshua, 2006, Advances in Neural Information Processing Systems 19, V19, P153
  • [10] Bishop Christopher, 2006, Pattern Recognition and Machine Learning, DOI 10.1117/1.2819119