3D FACIAL SCAN REGISTRATION WITH FACIAL EDGES DETECTION

The relevance of this work is caused by need of high-fidelity non-rigid registration algorithm for facial scans. Despite the fact that over the past decade, a large number of various algorithms for 3d model registration were proposed, including models of a human face, the vast majority of them are unable to provide accurate face registration for lips and eyes. This work is intended to improve this situation by using a detector of facial boundaries based on deep learning, namely using convolutional neural networks. Usage of facial landmarks detectors (deep learning based or other algorithms) is not novel in the field of non-rigid registration. However such approach is not good enough for precise registration. Facial landmark annotation is ill posed problem in general case, due to ambiguity of landmarks position. For example, it is very difficult to put landmarks in the middle of the eye so that they are anatomically in the same place in different frames. In this work we propose an alternative method - facial edges detection for registration. Obtained results show the advantage of the proposed approach.

1. Rusinkiewicz S. Efficient Variants of the ICP Algorithm / Rusinkiewicz S, Levoy M. // 3D Digital Imaging and Modeling — Stanford University: 2001. — URL: https://graphics.stanford.edu/papers/fasticp/

2. Sorkine O. As-rigid-as-possible surface modeling / Sorkine O., Alexa M. // SGP '07 Proceedings of the fifth Eurographics symposium on Geometry processing – 2007 – page 109-116.

3. Fyffe G. Multi-View Stereo on Consistent Face Topology / Fyffe G., Nagano K. // EUROGRAPHICS 2017 – 2017 – URL: http://vgl.ict.usc.edu/Research/XimeaRiver/.

4. Le V. Interactive Facial Feature Localization / Le V., Brandt J., Lin Z. // ECCV2012 – 2012 – URL: http://www.ifp.illinois.edu/~vuongle2/helen/eccv2012_helen_final.pdf.

5. Kazemi V. One Millisecond Face Alignment with an Ensemble of Regression Trees / Kazemi V., Sullivan J. // Conference: Computer Vision and Pattern Recognition – 2014 – page 1867-1874.

6. Nikolenko S. Deep Learning. / Nikolenko S., Kadurin A., ArkhangelskayaЕ. — Saint Petersburg: Piter, 2018. — 480 p.

7. Wu W. Look at Boundary: A Boundary-Aware Face Alignment Algorithm / Wu W., Qian C., Yang S. // CVPR 2018 – 2018 - URL: https://wywu.github.io/projects/LAB/LAB.html

8. Dong X. Supervision-by-Registration: An Unsupervised Approach to Improve the Precision of Facial Landmark Detectors / Dong X., Yu S., Weng X. // CVPR 2018 – 2018 – URL: https://arxiv.org/abs/1807.00966

9. Sagonas C. 300 Faces In-The-Wild Challenge: database and results / Sagonas C., Antonakos E., Tzimiropoulos G., Zafeiriou S., Pantic M. // Image and Vision Computing – 2016 – pages 3-18.

10. Wu Z. A Comprehensive Survey on Graph Neural Networks / Wu Z., Pan S., Chen F. // arxiv – 2018 – URL: https://arxiv.org/pdf/1901.00596.pdf

11. Paszke A. Automatic differentiation in PyTorch / Paszke A., Gross S., Chintala S. // NIPS 2017 – 2017 – URL: https://openreview.net/pdf?id=BJJsrmfCZ