The methodology for unmanned aerial vehicle trajectory forming for the autonomous gathering of visual data on electric powerline defects in infrared and ultraviolet spectra

Powerline element heat and corona discharge occurring in current conducting elements are significant problems that may cause serious faults in energetic systems. These defects require special equipment that makes it possible to obtain images in infrared (IR) and ultraviolet (UV) spectra for heat and corona discharge detection, respectively. The use of autonomous unmanned aerial vehicles (UAV) equipped with the appropriate cameras provide automation of such defect detection. Concurrently, the trajectory of the autonomous UAV should be formed according to the spatio-geometric features of the inspected power lines and the requirements for the image sample obtained during the inspection of the damaged powerline. However, the issues related to forming UAV trajectory consistent with the specified requirements have not been properly explored. As part of this research, a new method for UAV trajectory forming is presented. The method is characterized by forming the trajectory according to the spatio-geometric features of the inspected powerlines with its key components and the requirements for the collected data (the presence of damage in the image, object representativeness, unification of the represented objects). The method was tested in the Blender 3D modeling environment by simulation of the autonomous wire heating and corona discharge inspection in three powerline types. As a result, a sample of IR and UV spectra images was collected. The sample consists of 1300 images, which represents 1376 unique angles of 17 cases of simulated damage, which indicates the viability of this technique for constructing UAV autonomous flight trajectories in order to collect representative sample data on powerline damage in UV and IR spectra.

1. Gazebo. Available at: https://gazebosim.org/home (accessed 16.11.2022).

2. Blender Studio. Available at: https://www.blender.org/ (accessed 16.11.2022).

3. Astapova M.A., Lebedev I.V. Obzor podkhodov k detektirovaniyu defektov elementov LEP na izobrazheniyakh v infrakrasnom. ultrafioletovom i vidimom spektrakh. Modelirovaniye, optimizatsiya i informatsionnyye tekhnologii = Modeling, optimization and information technology. 2020;8(4):38–39.(In Russ.).

4. Liu W., Wang Z., Liu X., Zeng N., Liu Y., Alsaadi F.E. A survey of deep neural network architectures and their applications. Neurocomputing. 2017;234:11–26.

5. Liu L., Ouyang W., Wang X., Fieguth P., Chen J., Liu X., Pietikäinen M. Deep learning for generic object detection: A survey. International journal of computer vision. 2020;128(2):261–318.

6. Zaidi S.S., Ansari M.S., Aslam A., Kanwal N., Asghar M., Lee B. A survey of modern deep learning based object detection models. Digital Signal Processing. 2022;8:103514.

7. Hao S., Zhou Y., Guo Y. A brief survey on semantic segmentation with deep learning. Neurocomputing. 2020;406:302–21.

8. Hafiz A.M., Bhat G.M. A survey on instance segmentation: state of the art. International journal of multimedia information retrieval. 2020;9(3):171–189.

9. Kirillov A., He K., Girshick R., Rother C., Dollár P. Panoptic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019:9404–9413.

10. Bengio Y., Lecun Y., Hinton G. Deep learning for AI. Communications of the ACM. 2021;64(7):58–65.

11. Gondal M.W., Wuthrich M., Miladinovic D., Locatello F., Breidt M., Volchkov V., Akpo J., Bachem O., Schölkopf B., Bauer S. On the transfer of inductive bias from simulation to the real world: a new disentanglement dataset. Advances in Neural Information Processing Systems. 2019;32.

12. Reed S.E., Zhang Y., Zhang Y., Lee H. Deep visual analogy-making. Advances in neural information processing systems. 2015;28.

13. LeCun Y., Huang F.J., Bottou L. Learning methods for generic object recognition with invariance to pose and lighting. In Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR). 2004;2:II–104.

14. Popov N.I., Emelianova O.V. Dinamicheskie osobennosti monitoringa vozdushnyh linij elektroperedachi s pomoshch'yu kvadrokoptera. Sovremennye problemy nauki i obrazovaniya = Modern problems of science and education. 2014(2):105. (In Russ.).

15. Liu Y., Huo H., Fang J., Mai J., Zhang S. UAV Transmission line inspection object recognition based on Mask R-CNN. In Journal of Physics: Conference Series;1345(6):062043.

16. He K., Gkioxari G., Dollar P., Girshick R. Mask R-CNN. Proceedings of the IEEE international conference on computer vision. 2017;2961–2969.

17. Liu X., Lin Y., Jiang H., Miao X., Chen J. Slippage fault diagnosis of dampers for transmission lines based on faster R-CNN and distance constraint. Electric Power Systems Research. 2021;199:107449.

18. Ren S., He K., Girshick R., Sun J. Faster r-cnn: Towards real-time object detection with region proposal networks. Advances in neural information processing systems. 2015;28.

19. Lebedev I., Izhboldina V. Method for Inspecting High-voltage Power Lines Using UAV Based on the RRT Algorithm. In Electromechanics and Robotics. 2022:179–190.

20. Chen L., Lin L., Tian M., Bian X., Wang L., Guan Z. The ultraviolet detection of corona discharge in power transmission lines. Energy Power Eng. 2013;5(04):1298.

21. Moore A.J., Schubert M., Rymer N. Technologies and operations for high voltage corona detection with UAVs. In 2018 IEEE Power & Energy Society General Meeting (PESGM). 2018;5:1–5.

22. Nguyen P., Dudkin S., Kong C. Automatic diagnostic of transmission lines based on ultraviolet inspection. In E3S Web of Conferences. 2019;140:07008.

23. Davari N., Akbarizadeh G., Mashhour E. Intelligent diagnosis of incipient fault in power distribution lines based on corona detection in UV-visible videos. IEEE Transactions on Power Delivery. 2020;36(6):3640–8.

24. Zhao Z., Xu G., Qi Y. Representation of binary feature pooling for detection of insulator strings in infrared images. IEEE Transactions on Dielectrics and Electrical Insulation. 2016;23(5):2858–2866.

25. Ullah I., Khan R.U., Yang F., Wuttisittikulkij L. Deep learning image-based defect detection in high voltage electrical equipment. Energies. 2020;13(2):392.

26. Nie J., Luo T., Li H. Automatic hotspots detection based on UAV infrared images for large‐scale PV plant. Electronics Letters. 2020;56(19):993–995.

27. Tao X., Zhang D., Wang Z., Liu X., Zhang H., Xu D. Detection of Power Line Insulator Defects Using Aerial Images Analyzed With Convolutional Neural Networks. IEEE Transactions on Systems, Man, and Cybernetics: Systems. 2020;50(4):1486–1498.

28. Ömer E.Y., Ömer N.G. Powerline Image Dataset (Infrared-IR and Visible Light-VL). 2019. Available at: https://data.mendeley.com/datasets/n6wrv4ry6v/8 (accessed 16.12.2021).

29. Bian J., Hui X., Zhao X., Tan M. A monocular vision-based perception approach for unmanned aerial vehicle close proximity transmission tower inspection. International Journal of Advanced Robotic Systems. 2019;16(1):1729881418820227.

30. Wu C., Ma X., Kong X., Zhu H. Research on insulator defect detection algorithm of transmission line based on CenterNet. Plos one. 2021;16(7):e0255135.

31. Vieira-e-Silva A.L., de Castro Felix H., de Menezes Chaves T., Simões F.P., Teichrieb V., dos Santos M.M., da Cunha Santiago H., Sgotti V.A., Neto H.B. STN PLAD: A Dataset for Multi-Size Power Line Assets Detection in High-Resolution UAV Images. In 2021 34th SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI). 2021:215-222.

32. Abdelfattah R., Wang X., Wang S. Ttpla: An aerial-image dataset for detection and segmentation of transmission towers and power lines. In Proceedings of the Asian Conference on Computer Vision. 2020.

33. Installation and operation of overhead power. Available at: https://elektro-montagnik.ru/?address=lectures/part2/&page=page1 (accessed 16.11.2022). (In Russ.).

34. Chermoshencev S.F., Gaynutdinov R.R. Modeling the external electromagnetic influences on the complex electronic equipment. In2 015 XVIII International Conference on Soft Computing and Measurements (SCM). 2015:90–92.

35. Huang L., Xu D., Zhai D. Research and design of space-sky-ground integrated transmission line inspection platform based on artificial intelligence. In 2018 2nd IEEE Conference on Energy Internet and Energy System Integration (EI2). 2018:1–4.

36. Shabanova A.R., Tolstoj M. I., Lebedev I. V. Sposob postroeniya bezopasnyh traektorij dvizheniya bespilotnogo letatel'nogo apparata pri monitoringe linij elektroperedach v usloviyah vliyaniya elektromagnitnyh polej. Problemy Regional'noj Energetiki = Problems of the regional energetics. 2019;3(44). (In Russ.).

37. Riba J.R., Abomailek C., Casalsorrens P., Capelli F. Simplification and cost reduction of visual corona tests. IET Generation, Transmission & Distribution. 2018;12(4):834–41.

38. Gaussorgues G., Chomet S. Infrared thermography. Springer Science & Business Media. 1993;5.

39. Junior3d.ru – Site about 3D. Available at: https://junior3d.ru/ (accessed 16.11.2022).

40. Free3D. Available at: https://free3d.com/ (accessed 16.11.2022).

41. Image Polygonal Annotation with Python. Available at: https://zenodo.org/record/5711226#.Yw14ShxBy5c/ DOI: 10.5281/zenodo.5711225 (accessed 21.11.22).