Multimodal neural networks and locally trained models in structured fact extraction tasks

Social capital has become a significant professional resource; however, retaining a large volume of contextual data in memory is challenging. This problem is addressed by intelligent Personal CRM bots capable of extracting useful facts from text and speech. When designing such systems, a key dilemma is selecting an NLP architecture under strict confidentiality requirements. This paper compares the performance and resource consumption of four practical approaches. The isolated vector model demonstrated the highest processing speed, consuming approximately 1.9 GB of RAM with a sample processing time of 14 seconds. The combination of speech recognition with a compact LLM (Qwen 2.5-0.6B) increases the response latency to 25 seconds. This configuration requires 2.5 GB of RAM; furthermore, complicating the prompts sharply increases memory consumption, while the peak CPU load reaches 600 %. The hybrid scheme requires about 3.0 GB of RAM with a comparable processing time of 24 seconds, but it reduces the computational load on the CPU to 495 %. This optimization is achieved through the selective activation of the heavy-weight model. The deployment of a unified multimodal network proved technically unfeasible: inference exceeded 40 minutes while consuming 5.8 GB of memory. For local systems, it is advisable to use either a fully autonomous self-trained pipeline, which ensures maximum speed, or hybrid routing. In the hybrid variant, a lightweight model filters routine tasks, and the language network is engaged exclusively to resolve semantic ambiguities.

1. Gvozdikov D.S. Online networks and development of network interactions. Vestnik SPbGU. Ser. 12. 2015;(2):100–107. (In Russ.).

2. Lyz N.A., Gladkaya E.V. The impact of digitalization on cognitive processes: a review of causes and consequences. Perm University Herald. Philosophy. Psychology. Sociology. 2025;(3):396–405. (In Russ.). https://doi.org/10.17072/2078-7898/2025-3-396-405

3. Shmakov A.V. Nudge in the conditions of digital transformation: Behavioral basis. Journal of Institutional Studies. 2021;13(3):102–116. (In Russ.). https://doi.org/10.17835/2076-6297.2021.13.3.102-116

4. Kravchenko D.A. Mikroservisnaia arkhitektura. Interactive science. 2022:43-44 (In Russ.). https://doi.org/10.21661/r-556565

5. Abdirashituly A. Microservice architecture in NLP platforms: design scalability. Vestnik nauki. 2026;1(5):609–6019. (In Russ.).

6. Rakhmanov A., Iskhakova N., Abduvalieva Z. Word representation in vector space using word2vec model. Eurasian Journal of Mathematical Theory and Computer Sciences. 2025:5(5)54–59. (In Russ.). https://doi.org/10.5281/zenodo.15524744

7. Lychenko N.M., Sorokovaja A.V. Comparison of effectiveness of word representations methods in vector space for the text sentiment analysis. Mathematical Structures and Modeling. 2019;(4):97–110. (In Russ.). https://doi.org/10.24147/2222-8772.2019.4.97-110

8. Titov Y.P., Kilmishkin N.V., Kubrakov D.D., Ivanova P.M. Use of deep learning and natural language processing techniques for searching named entities in the medical instructions for use of drugs. Proceedings of ISP RAS. 2025;37(2):217–236. (In Russ.).

9. Kachanov V.V., Khitrova A.S., Markov S.I. Named entity recognition for code review comments. Proceedings of ISP RAS. 2023;35(5):193–214. (In Russ.).

10. Ilyin G.A. Information search and extraction about events using Named Entity Recognition (NER) methods. Vestnik nauki. 2025; 3(3):572–576. (In Russ.).