The article is devoted to the problem of 3D reconstruction of objects in dynamic scenes by stereo images. When performing any complex tasks by autonomous robots (repair work, inspection of the seabed), there is a need to simultaneously restore the trajectory of the autonomous robot and build a 3D model of the environment using video information. Data on the trajectories of robots and information about the environment are necessary for specialists to further correct drone operation and track the progress of work performed. Сurrently existing object identification solutions help to restore the geometry of dynamic objects with imposed restrictions that prevent from reconstructing the entire scene with the necessary accuracy. Also, the existing methods do not involve detailed visualization of the entire 3D scene using previously unknown point data and do not include the restoration of invisible parts of object surfaces. An approach to solving the problem of identification and 3D reconstruction of objects based on video information in relation to dynamic scenes is proposed. The basis of the software system implementing the proposed algorithmic and architectural solutions is described. Data on model scenes and features of scene objects are given. The results of computational experiments with virtual scenes are discussed. The regularities discovered as a result of tests affecting the accuracy of model reconstruction are considered.
1. Hasler N., Rosenhahn B., Thormahlen T., Wand M., Gall J., Seidel H.P. Markerless motion capture with unsynchronized moving cameras. CVPR. 2009;224–231. DOI: 10.1109/CVPR.2009.5206859.
2. Ballan L., Brostow G.J., Puwein J., Pollefeys M. Unstructured video-based rendering: Interactive exploration of casually captured videos. ACM Transactions on Graphics. Proceedings of SIGGRAPH. 2010;29(4):134–146. DOI: 10.1145/1778765.1778824.
3. Taneja A., Ballan L., Pollefeys M. Modeling dynamic scenes recorded with freely moving cameras. Conference on Computer Vision. 2010;613–626. DOI: 10.1007/978-3-642-19318-7_48.
4. Mustafa A., Kim H., Guillemaut J-Y., Hilton A. General Dynamic Scene Reconstruction from Multiple View Video. ICCV. 2015;900–908. DOI: 10.1109/ICCV.2015.109.
5. Keller M., Lefloch D., Lambers M., Izadi S., Weyrich T., and Kolb A. Real-time 3d reconstruction in dynamic scenes using point-based fusion. Proc. of Joint 3DIM/3DPVT Conference (3DV). 2013;1–8. DOI: 10.1109/3DV.2013.9.
6. Mustafa A., Kim H., Guillemaut J-Y., Hilton A. Temporally coherent 4D reconstruction of complex dynamic scenes. IEEE Conference on Computer Vision and Pattern recognition. 2016;223–245. DOI: 10.1109/CVPR.2016.504.
7. Lefloch D., Kluge M., Sarbolandi H., Weyrich T., Kolb A. Comprehensive Use of Curvature For Robust And Accurate Online Surface Reconstruction. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2017. Available from: http://reality.cs.ucl.ac.uk/projects/kinect/lefloch17comprehensive.pdf. DOI: 10.1109/TPAMI.2017.2648803 (accessed on 14.03.2023).
8. Bobkov V.A., Kudryashov A.P. Voxel method of constructing a triangulation surface by a set of types. Information Science and Control Systems. 2012;2:31–38. Available from: http:///media/2012/N32_04.pdf (accessed on 13.06.2021). (In Russ.).
9. Bobkov V.A., Kudryashov A.P. Construction of a three-dimensional model of the seabed by the promissory note method. Underwater Investigations and Robotics. 2016;2:13–18. Available from: http://jmtp.febras.ru/journal/2-22-2016/13-18.pdf (accessed on 13.06.2021). (In Russ.).
10. Bobkov V.A., Kudryashov A.P., Mel'man S.V. On restoring the motion of dynamic objects from stereo images. Programming and Computer Software. 2018;3:29–42. (In Russ.).
11. Kudryashov A.P., Cherkashin A.S. Construction of a single triangulation surface based on a set of views with hole stitching. Information Science and Control Systems. 2015;1:36–40. Available from: http:///media/2015/N43_04.pdf (accessed on 13.06.2021). (In Russ.).
12. Bobkov V.A., Ron’shin Yu.I., Kudryashov A.P., and Mashentsev V.Yu. 3D SLAM from Stereoimages. Programming and Computer Software. 2014;40(4):159–165. DOI: 10.1134/S0361768814040021.
13. Bobkov V., Mashentsev V., Tolstonogov A., Scherbatyuk A. Adaptive Method for AUV Navigation Using Stereo Vision. Proceedings of the 26th ISOPE International Ocean and Polar Engineering Conference. 2016;562–565.
14. Bobkov V., Melman S., Kudrashov A., Scherbatyuk A.Vision-based navigation method for a local maneuvering of the autonomous underwater vehicle. IEEE OES International Symposium on Underwater Technology 2017 (UT 2017 Busan). Busan; 2017;21–24.
15. Bobkov V.A., Melman S.V., and Kudryashov A.P. Fast Computation of Local Displacement by Stereo Pairs. Pattern Recognition and Image Analysis. 2017;3:458–465. DOI: 10.1134/S1054661817030063.
16. Goshin E.V., Fursov V.A., Bibikov S.A. Reconstruction of 3D scenes on bundles of epipolar planes of stereo images. Mekhatronika, Avtomatizatsiya, Upravlenie. 2013;9:19–24. (In Russ.).
17. Goshin E.V. Reconstruction of 3D scenes from unrectified stereo images with thinning filtering. Izvestia of Samara Scientific Center of the Russian Academy of Sciences. 2013;15(6):748–753. (In Russ.).
18. Skvortsov A.V. Delaunay triangulation and its application. Tomsk, TUP; 2002.128 p. (In Russ.).
19. Skvortsov A.V., Kostyuk Yu.L. Efficient algorithms for constructing Delaunay triangulation. Geoinformatics. Theory and practice. Tomsk: TUP; 1998;1:22–47. (In Russ.).
Logachev Egor Mikhailovich
ORCID |
Far Eastern Federal University
Vladivostok, The Russian Federation
