Keywords: large language model, embedding, retrieval-augmented generation, semantic search, vector database, technical support, web application
UDC 004.89
DOI: 10.26102/2310-6018/2026.62.11.009
The relevance of this study is determined by the need to automate technical support for software products, given the limited knowledge of general-purpose large language models and the insufficient effectiveness of search methods based solely on exact word matching. The aim of this work is to design and functionally evaluate an intelligent system that generates responses based on relevant fragments retrieved from a heterogeneous knowledge base. The proposed approach is based on retrieval-augmented generation, which integrates the processing of documentation, source code, and resolved user support requests, the construction of normalized vector representations, full-text and semantic search, ranking result fusion, and response generation using a large language model. For evaluation, a dataset of 100 questions divided into five categories was created, together with a knowledge base containing 148 sources split into 201 fragments. For questions supported by relevant information, correct answers were obtained in 76.25 % of cases. The overall proportion of correctly processed queries was 67 %. When supporting information was absent, the system correctly refused to provide an answer in 6 out of 20 cases. The results confirm the effectiveness of the developed system when the required information is available in the knowledge base and indicate the need to improve the processing of queries for which relevant information is unavailable. The practical significance of the proposed solution lies in its ability to create specialized technical support agents whose domain knowledge can be updated by re-indexing information sources without additional training of the language model.
1. Antonov I.V., Bruttan I.V. Using RAG technology and large language models to search for documents and obtain information in corporate information systems. Computer Research and Modeling. 2025;17(5):871–888. (In Russ.). https://doi.org/10.20537/2076-7633-2025-17-5-871-888
2. Kashirin I.Yu. Formation of LLM-oriented knowledge resources based on generation of augmented information. Vestnik of RSREU. 2025;(94):85–97. (In Russ.). https://doi.org/10.21667/1995-4565-2025-94-85-97
3. Lewis P., Perez E., Piktus A., et al. Retrieval-augmented generation for knowledge-intensive NLP tasks. In: Advances in Neural Information Processing Systems 33 (NeurIPS 2020), 06–12 December 2020, Virtual Event. 2020. URL: https://arxiv.org/abs/2005.11401
4. Kashirina I.L., Osipov I.R., Yakovlev V.A. Development and evaluation of a RAG system for analysis of semantic relations of specialised texts. Proceedings of Voronezh State University. Series: Systems Analysis and Information Technologies. 2025;(2):114–126. (In Russ.). https://doi.org/10.17308/sait/1995-5499/2025/2/114-126
5. Melnikov A.V., Nikolaev I.E., Rusanov M.A., Abbazov V.R. Comparative analysis of RAG methods for building Russian-speaking intelligent services. Bulletin of the South Ural State University. Series: Computer Technologies, Automatic Control, Radioelectronics. 2025;25(2):5–18. (In Russ.). https://doi.org/10.14529/ctcr250201
6. Zhang Y., Li M., Long D., et al. Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models. arXiv. URL: https://arxiv.org/abs/2506.05176 [Accessed 11th August 2026].
7. Reimers N., Gurevych I. Sentence-BERT: Sentence embeddings using Siamese BERT-networks. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP 2019), 03–07 November 2019, Hong Kong, China. ACL; 2019. P. 3980–3990. https://doi.org/10.18653/v1/D19-1410
8. Malkov Yu.A., Yashunin D.A. Efficient and robust approximate nearest neighbor search using hierarchical navigable small world graphs. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2020;42(4):824–836. https://doi.org/10.1109/TPAMI.2018.2889473
9. Cormack G.V., Clarke Ch.L.A., Buettcher S. Reciprocal rank fusion outperforms Condorcet and individual rank learning methods. In: SIGIR '09: Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval, 19–23 July 2009, Boston, MA, USA. New York: ACM; 2009. P. 758–759. https://doi.org/10.1145/1571941.1572114
10. Es Sh., James J., Espinosa Anke L., Schockaert S. RAGAs: Automated evaluation of retrieval augmented generation. In: Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations, 17–22 March 2024, St. Julians, Malta. ACL; 2024. P. 150–158. https://doi.org/10.18653/v1/2024.eacl-demo.16
Keywords: large language model, embedding, retrieval-augmented generation, semantic search, vector database, technical support, web application
For citation: Suzmin D.Y., Usmanov B.V., Egunova A.I., Yamashkin S.A., Fedosin S.A., Abbakumov A.A. Application of embeddings and large language models for building an intelligent technical support system. Modeling, Optimization and Information Technology. 2026;14(11). URL: https://moitvivt.ru/ru/journal/article?id=2620 DOI: 10.26102/2310-6018/2026.62.11.009 (In Russ).
© Suzmin D.Y., Usmanov B.V., Egunova A.I., Yamashkin S.A., Fedosin S.A., Abbakumov A.A. Статья опубликована на условиях лицензии Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NS 4.0)Received 01.09.2026
Revised 30.10.2026
Accepted 12.11.2026