Impact of Frontalization on Face Recognition
Face recognition performance has improved remarkably in the last decade. Much of this success can be attributed to the development of deep learning techniques such as convolutional neural networks (CNNs). While CNNs have pushed the state-of-the-art forward, their training process requires a large amount of clean and correctly labelled training data. If a CNN is intended to tolerate facial pose, then we face an important question: should this training data be diverse in its pose distribution, or should face images be normalized to a single pose in a pre-processing step? To address this question, we evaluate a number of facial landmarking algorithms and a popular frontalization method to understand their effect on facial recognition performance. Additionally, we introduce a new, automatic, single-image frontalization scheme that exceeds the performance of the reference frontalization algorithm for video-to-video face matching on the Point and Shoot Challenge (PaSC) dataset. Additionally, we investigate failure modes of each frontalization method on different facial yaw using the CMU MultiPIE dataset. We assert that the subsequent recognition and verification performance serves to quantify the effectiveness of each pose correction scheme.
To Frontalize or Not To Frontalize: Do We Really Need Elaborate Pre-processing to Improve Face Recognition?, Sandipan Banerjee*, Joel Brogan*, Aparna Bharati, Brandon RichardWebster, Vitomir Struc, Patrick Flynn, Walter J. Scheirer, Proceedings of the IEEE Winter Conference on Applications of Computer Vision (WACV), March 2018: [pdf] [arxiv] [oral] [poster] [code]
* denotes equal contribution
Report on the BTAS 2016 Video Person Recognition Evaluation, Walter J. Scheirer, Patrick J. Flynn, Changxing Ding, Guodong Guo, Vitomir Struc, Mohamad Al Jazaery, Klemen Grm, Simon Dobrisek, Dacheng Tao, Yu Zhu, Joel Brogan, Sandipan Banerjee, Aparna Bharati, Brandon RichardWebster, Proceedings of the IEEE International Conference on Biometrics: Theory, Applications and Systems (BTAS), September 2016: [pdf]
