A Psychovisual Quality Metric in Free-Energy Principle

被引:226
作者
Zhai, Guangtao [1 ,2 ]
Wu, Xiaolin [1 ]
Yang, Xiaokang [2 ]
Lin, Weisi [3 ]
Zhang, Wenjun [2 ]
机构
[1] McMaster Univ, Dept Elect & Comp Engn, Hamilton, ON L8G 4K1, Canada
[2] Shanghai Jiao Tong Univ, Inst Image Commun & Informat Proc, Shanghai 200240, Peoples R China
[3] Nanyang Technol Univ, Sch Comp Engn, Singapore 639798, Singapore
基金
加拿大自然科学与工程研究理事会;
关键词
Brain theory; free-energy principle; image modeling; image quality assessment; IMAGE; INFORMATION; BRAIN;
D O I
10.1109/TIP.2011.2161092
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
In this paper, we propose a new psychovisual quality metric of images based on recent developments in brain theory and neuroscience, particularly the free-energy principle. The perception and understanding of an image is modeled as an active inference process, in which the brain tries to explain the scene using an internal generative model. The psychovisual quality is thus closely related to how accurately visual sensory data can be explained by the generative model, and the upper bound of the discrepancy between the image signal and its best internal description is given by the free energy of the cognition process. Therefore, the perceptual quality of an image can be quantified using the free energy. Constructively, we develop a reduced-reference free-energy-based distortion metric (FEDM) and a no-reference free-energy-based quality metric (NFEQM). The FEDM and the NFEQM are nearly invariant to many global systematic deviations in geometry and illumination that hardly affect visual quality, for which existing image quality metrics wrongly predict severe quality degradation. Although with very limited or even without information on the reference image, the FEDM and the NFEQM are highly competitive compared with the full-reference SSIM image quality metric on images in the popular LIVE database. Moreover, FEDM and NFEQM can measure correctly the visual quality of some model-based image processing algorithms, for which the competing metrics often contradict with viewers' opinions.
引用
收藏
页码:41 / 52
页数:12
相关论文
共 37 条
[1]  
[Anonymous], 2003, Cognitive Psychology
[2]  
Attias H, 2000, ADV NEUR IN, V12, P209
[3]  
Barlow H., 1961, COGNITIVE PSYCHOL
[4]   THE HELMHOLTZ MACHINE [J].
DAYAN, P ;
HINTON, GE ;
NEAL, RM ;
ZEMEL, RS .
NEURAL COMPUTATION, 1995, 7 (05) :889-904
[5]   A No-Reference Objective Image Sharpness Metric Based on the Notion of Just Noticeable Blur (JNB) [J].
Ferzli, Rony ;
Karam, Lina J. .
IEEE TRANSACTIONS ON IMAGE PROCESSING, 2009, 18 (04) :717-728
[6]  
Feynman R. P., 1998, Statistical Mechanics: A Set of Lectures, Advanced Books Classics
[7]   A free energy principle for the brain [J].
Friston, Karl J. ;
Kilner, James ;
Harrison, Lee .
JOURNAL OF PHYSIOLOGY-PARIS, 2006, 100 (1-3) :70-87
[8]   The free-energy principle: a unified brain theory? [J].
Friston, Karl J. .
NATURE REVIEWS NEUROSCIENCE, 2010, 11 (02) :127-138
[9]   Model selection and the principle of minimum description length [J].
Hansen, MH ;
Yu, B .
JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION, 2001, 96 (454) :746-774
[10]  
Hinton G. E., 1993, Proceeding of the Sixth Annual ACM Conference on Computational Learning Theory, P5, DOI 10.1145/168304.168306