Face reconstruction as a technique for enhancing accuracy of human recognition in a video frame

The article considers an approach aimed at enhancing the accuracy of human identification by facial image in video surveillance systems, using the reconstruction method based on generative-adversarial networks. During the investigation of offenses, one often encounters video recordings of people of interest for the investigation with a low resolution or containing visual disturbances of different genesis, which limits the implementation of techniques for identifying the person by means of deep learning neural networks. This causes two problems: one pertaining to face detection of a certain person in the video data and another regarding the search for a selected person in the frame contained in the database. The reconstruction of a face using generative adversarial networks is known to significantly improve low-quality face images, but this method is demanding of the content of the original image as any occlusions and disturbances are multiply amplified. The paper presents an approach composed of image preprocessing on the basis of the known property of video recordings – the presence of object image versioning. The proposed algorithm helps to correct much of the visual noise and subsequently reconstruct the face image with high quality. During the experiments, we have also found a method of facial elements restoration which enables the increase in the recognizability of an unknown face by a person, which can be important during the identification by witnesses.

1. Sebyakin A.G. Analiz informatsii o soedineniyakh mezhdu abonentami, ispol'zovanie ego rezul'tatov v raskrytii i rassledovanii prestuplenii. Politseiskaya i sledstvennaya deyatel'nost'. 2018;4:29–38. URL: https://nbpublish.com/library_read_article.php?id= 27992 DOI: 10.25136/2409-7810.2018.4.27992 (accessed on 10.02.2023). (In Russ.).

2. Raposo V.L. The Use of Facial Recognition Technology by Law Enforcement in Europe: a Non-Orwellian Draft Proposal. European Journal on Criminal Policy and Research. 2022;1–19. Available from: https://link.springer.com/content/pdf/10.1007/s10610-022-09512-y.pdf. DOI: 10.1007/s10610-022-09512.

3. Dimitrova N., Zhang H.J., Shahraray B., Sezan I., Huang T., Zakhor A. Applications of video-content analysis and retrieval. IEEE multimedia. 2002;9(3):42–55. Available from: http://www-video.eecs.berkeley.edu/papers/Dimitrove/IEEE_MM2002.pdf. DOI: 10.1109/ MMUL.2002.1022858.

4. Du H., Shi H., Zeng D., Zhang X.P., Mei T. The elements of end-to-end deep face recognition: A survey of recent advances. ACM Computing Surveys (CSUR). 2022;54(10s):1–42. Available from: https://arxiv.org/pdf/2009.13290v3.pdf. DOI: 10.1145/ 3507902.

5. Nasonov A.V., Krylov A.S., Ushmaev O.S. Primenenie metoda superrazresheniya dlya biometricheskikh zadach raspoznavaniya lits v videopotoke. Sistemy vysokoi dostupnosti. 2009;1:26–34. Available from: https://imaging.cs.msu.ru/pub/2009.SVD.Nasonov_Krylov. SRVidRec.ru.pdf (accessed on 10.02.2023). (In Russ.).

6. Pakhirka A.I. Primenenie metoda uluchsheniya izobrazhenii dlya sistem raspoznavaniya lits. Sibirskii aerokosmicheskii zhurnal = The Siberian Aerospace Journal. 2010;3:25–29. Available from: https://www.elibrary.ru/download/elibrary_15204085_84528176.pdf (accessed on 10.02.2023). (In Russ.).

7. Morneva A.E., Maslennikov A.A. Metody povysheniya tochnosti raboty algoritmov raspoznavaniya lits na izobrazhenii. Vestnik molodezhnoi nauki Rossii. 2020;1:14–18. Available from: https://www.elibrary.ru/download/elibrary_44487166_63470132.pdf (accessed on 10.02.2023). (In Russ.).

8. Bordyuzha V., Umnyashkin S.V. Predobrabotka izobrazhenii dlya povysheniya kachestva raspoznavaniya lits na tsifrovykh foto. Akademicheskaya nauka kak faktor i resurs innovatsionnogo razvitiya. 2022;26–38. Available from: https://sciencen.org/assets/ Kontent/Konferencii/Arhiv-konferencij/KOF-578.pdf#page=26 (accessed on 10.02.2023). (In Russ.).

9. Minaee S., Luo P., Lin Z., Bowyer K. Going deeper into face detection: A survey. arXiv preprint. 2021;2103(14983). Available from: https://arxiv.org/pdf/2103.14983.pdf. DOI: 10.48550/arXiv.2103.14983.

10. Afanas'ev A.D., Prichko I.O. Detektor ob"ektov nepostoyannogo dvizheniya v zadache obnaruzheniya kriminalisticheski znachimoi informatsii. Modelirovanie, optimizatsiya i informatsionnye tekhnologii = Modeling, optimization and information technology. 2021;9(2):23–24. Available from: https://moitvivt.ru/ru/journal/pdf?id=928. DOI: 10.26102/2310-6018/2021.33.2.007 (accessed on 10.02.2023). (In Russ.).

11. Guo J., Deng J., Lattas A., Zafeiriou S. Sample and computation redistribution for efficient face detection. arXiv preprint. 2021;2105(04714). Available from: https://arxiv.org/pdf/2105.04714.pdf. DOI: 10.48550/arXiv.2105.04714.

12. Polozhintsev B.I. Teoriya veroyatnostei i matematicheskaya statistika. Vvedenie v matematicheskuyu statistiku: Uchebnoe posobie. SPb.: YuNITI; 2016. 95 с. (In Russ.).

13. Wang X., Li Y., Zhang H., Shan Y. Towards real-world blind face restoration with generative facial prior. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021;9168–9178. Available from: https://arxiv.org/pdf/2101.04061.pdf. DOI: 10.48550/arXiv.2101.04061.

14. Wang Z., Zhang J., Chen R., Wang W., Luo P. RestoreFormer: High-Quality Blind Face Restoration From Undegraded Key-Value Pairs. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022;17512–17521. Available from: https://arxiv.org/pdf/2201.06374.pdf. DOI: 10.48550/arXiv.2201.06374.

15. DeepFace – The Most Popular Open Source Facial Recognition Library. Available from: https://viso.ai/computer-vision/deepface/ (accessed on 10.02.2023).

16. Adeshara K., Elangovan V. Face recognition using PCA integrated with Delaunay triangulation. arXiv preprint. 2020;2011(12786). Available from: https://arxiv.org/ftp /arxiv/papers/2011/2011.12786.pdf. DOI: 10.48550/arXiv.2011.12786.