Intelligent hand gesture-based quadcopter motion control system

The relevance of this research stems from the fact that controlling a drone using hand gestures is more natural and intuitive than using traditional joysticks. This allows users to easily learn control and focus on task execution rather than technical aspects of operation. In turn, developing a gesture recognition system requires advancements in machine learning-based image processing algorithms. This paper aims to investigate the feasibility of implementing drone motion control using hand gestures in conjunction with modern neural network technologies. The main approach in addressing this problem involves the application of convolutional artificial neural networks for image processing and computer vision tasks. The work also explores methods for hyperparameter optimization using the Optuna tool, the use of TensorFlow Lite for implementing machine learning models on resource-constrained devices, and the application of the MediaPipe library for gesture analysis. Technologies such as Dropout and L2-regularization are used to enhance model efficiency. The materials presented in this paper hold practical value for researchers in the fields of artificial intelligence and robotics, software developers, and companies involved in the development of unmanned aerial vehicles.

1. Dudin D.E. Sistema upravleniya dvizheniem bespilotnogo letatel'nogo apparata na osnove raspoznavaniya zhestov ruk. Yunyi uchenyi. 2024;(5):67–72. (In Russ.).

2. Sanna A., Lamberti F., Paravati G., Manuri F. A Kinect-Based Natural Interface for Quadrotor Control. Entertainment Computing. 2013;4(3):179–186. https://doi.org/10.1016/j.entcom.2013.01.001

3. Zhao R., Wang K., Divekar R., Rouhani R., Su H., Ji Q. An Immersive System with Multi-Modal Human-Computer Interaction. In: 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), 15-19 May 2018, Xi'an, China. IEEE; 2018. P. 517–524. https://doi.org/10.1109/FG.2018.00083

4. Murlin A.G., Piotrovskiy D.L., Rudenko E.A., Yanaeva M.V. Algorithms and methods for detection and recognition of hand gestures on video in real time. Politematicheskii setevoi elektronnyi nauchnyi zhurnal Kubanskogo gosudarstvennogo agrarnogo universiteta = Polythematic online scientific journal of Kuban State Agrarian University. 2014;(97). (In Russ.). URL: http://ej.kubagro.ru/2014/03/pdf/20.pdf

5. Nahapetyan V.E., Khachumov V.M. Gesture recognition in the problem of contactless control of an unmanned aerial vehicle. Optoelectronics, Instrumentation and Data Processing. 2015;51(2):192–197. https://doi.org/10.3103/S8756699015020132

6. Yaryshev S.N., Ryzhova V.A. Tekhnologii glubokogo obucheniya i neironnykh setei v zadachakh videoanaliza. Saint Petersburg: ITMO University; 2022. 82 p. (In Russ.).

7. Chernyshev N.N., Nizhenets T.V. Path planning algorithm in a three-dimensional deterministic environment with obstacles using particle swarm algorithm. Vestnik Voronezhskogo gosudarstvennogo tekhnicheskogo universiteta = Bulletin of Voronezh State Technical University. 2022;18(6):7–14. (In Russ.). https://doi.org/10.36622/VSTU.2022.18.6.001

8. Bulygin D.A., Mamonova T.E. Recognition of hand gestures in real time. Nauchnyi vestnik Novosibirskogo gosudarstvennogo tekhnicheskogo universiteta = Science Bulletin of the Novosibirsk State Technical University. 2020;(1):25–40. (In Russ.). https://doi.org/10.17212/1814-1196-2020-1-25-40

9. Dhawale P., Masoodian M., Rogers B. Bare-hand 3D gesture input to interactive systems. In: CHINZ '06: Proceedings of the 7th ACM SIGCHI New Zealand Chapter's International Conference on Computer-human Interaction: Design Centered HCI, 6-7 July 2006, Christchurch, New Zealand. New York: Association for Computing Machinery; 2006. P. 25–32. https://doi.org/10.1145/1152760.1152764

10. Chudnovsky M.M. A real-time algorithm for human’s hand gesture recognition on video-sequence for human-computer interaction interfaces. Vestnik Sibirskogo gosudarstvennogo aerokosmicheskogo universiteta imeni akademika M.F. Reshetneva. 2014;(3):162–167. (In Russ.).