Emotion recognition in images and artificial immune systems

This research paper presents the results of applying one of the methods of artificial immune systems to the problem of recognizing human emotions by facial expressions in images. Artificial immune systems are a special concept based on a different principles of the natural immune system of mammals. Due to their diversity, artificial immune systems have managed to achieve high results in many different tasks. Therefore, the question of their effectiveness in such tasks as the problem of emotions recognition is very interesting. In this study, using one of the methods of artificial immune systems, it was possible to achieve a maximum accuracy of 80% for the task of recognizing the 7 basic emotions of Paul Ekman. These metrics were achieved on the Cohn–Kanade+ dataset. To build such a system, the research work considered the most popular approaches of recognizing emotions in images, and also, this work considered the key concepts of emotion classification. In the proposed model, a computer vision-based approach was used using 68-point facial landmarks. The obtained coordinates of the points on the face were transformed into 136 real features, and then, their number was reduced to 25 features using the method of principal components. A further direction of research will be the search for the most effective method of artificial immune systems for the problem of recognizing emotions in images.

1. Nuriakhmetov A. I., Bogdanova D. R. Artificial immune systems and emotion recognition. Original research. 2020;10(12):174-184. Available at: https://ores.su/ru/journals/oris-jrn/2020-oris-12-2020/a230163 (Accessed 15.03.2021)

2. Lindquist K. A., Siegel E. H., Quigley K. S., Barrett L. F. The hundred-year emotion war: are emotions natural kinds or psychological constructions? Comment on Lench, Flores, and Bench (2011). 2013. DOI: 10.1037/a0029038

3. Wundt W. M., Judd C. H. Outlines of psychology. Engelmann. 1902.

4. Russell J. A. A circumplex model of affect. Journal of personality and social psychology. 1980;39(6):1161. DOI: 10.1037/h0077714

5. Jack R. E., Garrod O. G. B., Schyns P. G. Dynamic facial expressions of emotion transmit an evolving hierarchy of signals over time. Current biology. 2014;24(2):187-192. DOI:10.1016/j.cub.2013.11.064

6. James W. The principles of psychology. Cosimo, Inc. 2007.

7. Friesen E., Ekman P. Facial action coding system: a technique for the measurement of facial movement. Psychology. 1978.

8. Bartlett M. S., Hager J. C., Ekman P., Sejnowski T. J. Measuring facial expressions by computer image analysis. Psychophysiology. 1999;36(2):253-263. DOI: 10.1017/s0048577299971664

9. Saha C., Ahmed W., Mitra S., Mazumdar D., Mitra S. Facial Expressions: A Cross‐Cultural Study. Emotion Recognition. 2015:69-87. DOI: 10.1002/9781118910566.ch3

10. Tarnowski P., Kołodziej M., Majkowski A., Rak R. J. Emotion recognition using facial expressions. Procedia Computer Science. 2017;108:1175-1184. DOI:10.1016/j.procs.2017.05.025

11. Bobe A. S., Konyshev D. V., Vorotnikov S. A. Emotion recognition system based on the facial motor units’ analysis. Engineering Journal: Science and Innovations. 2016;9(57). Available at: http://engjournal.ru/catalog/mesc/rmrs/1530.html DOI: 10.18698/2308-6033-2016-09-1530 (Accessed 20.04.2021)

12. Golzadeh H., Faria D. R., Manso L. J., Ekárt A., Buckingham C. D. Emotion recognition using spatiotemporal features from facial expression landmarks. 2018 International Conference on Intelligent Systems (IS). 2018:789-794 DOI: 10.1109/IS.2018.8710573

13. Munasinghe M. I. Facial expression recognition using facial landmarks and random forest classifier. 2018 IEEE/ACIS 17th International Conference on Computer and Information Science (ICIS). 2018:423-427. DOI: 10.1109/ICIS.2018.8466510

14. Fama K. Automatic Analysis of Facial Action. 2018.

15. Lucey P., Cohn J. F., Kanade T., Saragih J., Ambadar Z., Matthews I. The extended cohn-kanade dataset (ck+): A complete dataset for action unit and emotion-specified expression. 2010 IEEE computer society conference on computer vision and pattern recognition-workshops. 2010:94-101. DOI: 10.1109/CVPRW.2010.5543262

16. Zuiderveld K. Contrast limited adaptive histogram equalization. Graphics gems. 1994:474-485. DOI: 10.1016/B978-0-12-336156-1.50061-6

17. Viola P., Jones M. Rapid object detection using a boosted cascade of simple features. Proceedings of the 2001 IEEE computer society conference on computer vision and pattern recognition. 2001;1:I-I. DOI: 10.1109/CVPR.2001.990517

18. Kazemi V., Sullivan J. One millisecond face alignment with an ensemble of regression trees. Proceedings of the IEEE conference on computer vision and pattern recognition 2014. 2014:1867-1874. DOI: 10.13140/2.1.1212.2243

19. Çeliktutan O., Ulukaya S., Sankur B. A comparative study of face landmarking techniques. EURASIP Journal on Image and Video Processing. 2013;2013(1):1-27. DOI: 10.1186/1687-5281-2013-13

20. Rohith R. S., Pratiba D., Ramakanth P. K. Facial Expression Recognition using Facial Landmarks: A Novel Approach. Advances in Science, Technology and Engineering Systems Journal. 2020;5(5):24-28. DOI: 10.25046/aj050504

21. Schmidt B. H. Artificial Immune Systems: Applications, Multi-Class Classification, Optimizations, and Analysis. 2017.