Face swapping: Automatically replacing faces in photographs

被引:229
作者
Bitouk, Dmitri [1 ]
Kumar, Neeraj [1 ]
Dhillon, Samreen
Belhumeur, Peter [1 ]
Nayar, Shree K. [1 ]
机构
[1] Columbia Univ, New York, NY 10027 USA
来源
ACM TRANSACTIONS ON GRAPHICS | 2008年 / 27卷 / 03期
关键词
face replacement; image databases; image-based rendering; computational photography;
D O I
10.1145/1360612.1360638
中图分类号
TP31 [计算机软件];
学科分类号
081202 ; 0835 ;
摘要
In this paper, we present a complete system for automatic face replacement in images. Our system uses a large library of face images created automatically by downloading images from the internet, extracting faces using face detection software, and aligning each extracted face to a common coordinate system. This library is constructed off-line, once, and can be efficiently accessed during face replacement. Our replacement algorithm has three main stages. First, given an input image, we detect all faces that are present, align them to the coordinate system used by our face library, and select candidate face images from our face library that are similar to the input face in appearance and pose. Second, we adjust the pose, lighting, and color of the candidate face images to match the appearance of those in the input image, and seamlessly blend in the results. Third. we rank the blended candidate replacements by computing a match distance over the overlap region. Our approach requires no 3D model, is fully automatic, and generates highly plausible results across a wide range of skin tones, lighting conditions, and viewpoints. We show how our approach can be used for a variety of applications including face de-identification and the creation of appealing group photographs from a set of images. We conclude with a user study that validates the high quality of our replacement results, and a discussion on the current limitations of our system.
引用
收藏
页数:8
相关论文
共 24 条
[1]   Interactive digital photomontage [J].
Agarwala, A ;
Dontcheva, M ;
Agrawala, M ;
Drucker, S ;
Colburn, A ;
Curless, B ;
Salesin, D ;
Cohen, M .
ACM TRANSACTIONS ON GRAPHICS, 2004, 23 (03) :294-302
[2]  
[Anonymous], 2003, CVPR
[3]  
[Anonymous], DIGITAL FACE REPLACE
[4]   Seam carving for content-aware image resizing [J].
Avidan, Shai ;
Shamir, Ariel .
ACM TRANSACTIONS ON GRAPHICS, 2007, 26 (03)
[5]   Lambertian reflectance and linear subspaces [J].
Basri, R ;
Jacobs, DW .
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2003, 25 (02) :218-233
[6]   Exchanging faces in images [J].
Blanz, V ;
Scherbaum, K ;
Vetter, T ;
Seidel, HP .
COMPUTER GRAPHICS FORUM, 2004, 23 (03) :669-676
[7]  
Boyle M., 2000, ACM C COMP SUPP COOP
[8]   Active appearance models [J].
Cootes, TF ;
Edwards, GJ ;
Taylor, CJ .
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2001, 23 (06) :681-685
[9]  
Debevec P, 2000, COMP GRAPH, P145, DOI 10.1145/344779.344855
[10]  
DEBEVEC P, 1998, SIGGRAPH 98, P189