moitvivt Моделирование, оптимизация и информационные технологии Modeling, Optimization and Information Technology 2310-6018 Издательство 10.26102/2310-6018/2026.57.6.020 2375 Математическая модель процесса автоматизированного построения обзорных текстов, основанного на использовании цитатно-осведомленной суммаризации A mathematical model of the process of automated construction of review texts based on the use of quotation-aware summarization 0009-0001-6287-8295 Кузнецов Илия Игоревич Kuznetsov Iliya Igorevich iliya-kuznetsov@mail.ru aff-1 Российский государственный университет им. А.Н. Косыгина (Технологии. Дизайн. Искусство) The Kosygin State University of Russia 01 01 2026 1 1 10.26102/2310-6018/2026.57.6.020 Copyright © Авторы, 2026 2026 This work is licensed under a Creative Commons Attribution 4.0 International License

В работе рассматривается задача автоматизированного построения научных обзорных текстов на основе анализа корпуса научных публикаций. Разработана математическая модель, описывающая подход, основанный на использовании цитатно-осведомленной суммаризации с предварительным отбором цитатных фрагментов, и формализующая полный цикл обработки данных от отбора публикаций и извлечения цитирующих фрагментов до генерации частных саммари и итогового обзорного текста. Модель задает единое формальное описание последовательности операторов преобразования данных и включает систему критериев качества, обеспечивающих контроль правдоподобности, покрытия и фактологической согласованности результатов на всех этапах обработки. На основе разработанной модели реализована программная система в виде модульного конвейера. С использованием разработанной модели экспериментальные исследования на датасете SurGE, включающем 114 тематик, более 7 тыс. цитируемых и свыше 73 тыс. цитирующих публикаций. Показано, что использование цитатно-осведомленного подхода с предварительным отбором фрагментов обеспечивает улучшение качества генерации саммари по сравнению с альтернативными методами. Для итоговых обзорных текстов достигнуты следующие значения критериев качества: правдоподобность – 0,8744, покрытие – 0,9356, фактологическая достоверность – 0,9713 и LLM-оценка – 0,9232, что превосходит результаты генерации на основе полного текста источников (на 7,67 % для правдоподобности, 4,63 % для покрытия, 9,22 % для фактологической согласованности и 4,42 % для LLM-оценки). Полученные результаты подтверждают эффективность предложенного подхода и разработанной модели, и их применимость для задач автоматизированного формирования достоверных научных обзоров.

This paper examines the automated generation of scientific review texts based on the analysis of a corpus of scientific publications. A mathematical model has been developed that describes an approach based on the use of citation-aware summarization with preliminary selection of citation fragments and formalizes the full data processing cycle, from publication selection and extraction of citing fragments to the generation of individual summaries and the final review text. The model defines a unified formal description of the sequence of data transformation operators and includes a system of quality criteria that ensure control of the plausibility, coverage, and factual consistency of results at all stages of processing. A software system in the form of a modular pipeline is implemented based on the developed model. Experimental studies using the developed model were conducted on the SurGE dataset, which includes 114 topics, over 7,000 cited and over 73,000 citing publications. It is shown that the use of a citation-aware approach with preliminary fragment selection improves the quality of summary generation compared to alternative methods. The following quality criteria were achieved for the resulting review texts: credibility – 0.8744, coverage – 0.9356, factual reliability – 0.9713, and LLM score – 0.9232, which outperforms the results of generation based on the full text of sources (by 7.67 % for credibility, 4.63 % for coverage, 9.22 % for factual consistency, and 4.42 % for LLM score). The obtained results confirm the effectiveness of the proposed approach and the developed model and their applicability for the automated generation of reliable scientific reviews.

 цитатно-осведомленная суммаризация научные обзоры большие языковые модели анализ текстовой информации научные публикации цитирование извлечение цитат citation-aware summarization scientific reviews large language models text information analysis scientific publications citation citation extraction Исследование выполнено без спонсорской поддержки. The study was performed without external funding. References Syed A.A., Gaol F.L., Boediman A., et al. A survey of abstractive text summarization utilising pretrained language models. In: Intelligent Information and Database Systems: 14th Asian Conference (ACIIDS 2022): Proceedings: Part I, 28–30 November 2022, Ho Chi Minh City, Vietnam. Cham: Springer; 2022. P. 532–544. https://doi.org/10.1007/978-3-031-21743-2_42 Afsharizadeh M., Ebrahimpour-Komleh H., Bagheri A., et al. A survey on multi-document summarization and domain-oriented approaches. Journal of Information Systems and Telecommunication. 2022;10(37):68–78. https://doi.org/10.52547/jist.16245.10.37.68 Keya F., Jaradeh M.Y., Auer S. Leveraging LLMs for scientific abstract summarization: unearthing the essence of research in a single sentence. In: Proceedings of the 24th ACM/IEEE Joint Conference on Digital Libraries, 16–20 December 2024, Hong Kong, China. New York: ACM; 2024. https://doi.org/10.1145/3677389.3702588 Scherbakov D., Hubig N., Jansari V., et al. The emergence of large language models as tools in literature reviews: a large language model-assisted systematic review. Journal of the American Medical Informatics Association. 2025;32(6):1071–1086. https://doi.org/10.1093/jamia/ocaf063 Wu Sh., Ma X., Luo D., et al. Automated literature research and review-generation method based on large language models. National Science Review. 2025;12(6):nwaf169. https://doi.org/10.1093/nsr/nwaf169 Guo B., Wen Zh., Yang Y., et al. SGSimEval: a comprehensive multifaceted and similarity-enhanced benchmark for automatic survey generation systems. In: Advanced Data Mining and Applications: 21st International Conference (ADMA 2025): Proceedings: Part II, 22–24 October 2025, Kyoto, Japan. Singapore: Springer; 2025. P. 393–407. https://doi.org/10.1007/978-981-95-3456-2_27 Wu S., Liang Ch., Bi Z., et al. AutoSurvey2: empowering researchers with next level automated literature surveys. arXiv. URL: https://doi.org/10.48550/arXiv.2510.26012 [Accessed 7th April 2026]. Li Y., Datta S., Rastegar-Mojarad M., et al. Enhancing systematic literature reviews with generative artificial intelligence: development, applications, and performance evaluation. Journal of the American Medical Informatics Association. 2025;32(4):616–625. https://doi.org/10.1093/jamia/ocaf030 Su W., Xie A., Ai Q., et al. SurGE: a benchmark and evaluation framework for scientific survey generation. arXiv. URL: https://doi.org/10.48550/arXiv.2508.15658 [Accessed 19th April 2026]. Zhao J., Zhang Sh., Xu N., et al. SurveyEval: towards comprehensive evaluation of LLM-generated academic surveys. arXiv. URL: https://doi.org/10.48550/arXiv.2512.02763 [Accessed 14th April 2026]. Ji Z., Lee N., Frieske R., et al. Survey of hallucination in natural language generation. ACM Computing Surveys. 2023;55(12):248. https://doi.org/10.1145/3571730 Maynez J., Narayan Sh., Bohnet B., et al. On faithfulness and factuality in abstractive summarization. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 05–10 July 2020, Virtual Event. Association for Computational Linguistics; 2020. P. 1906–1919. https://doi.org/10.18653/v1/2020.acl-main.173 Lamba N., Tiwari S., Gaur M. Hallucinations in scholarly LLMs: a conceptual overview and practical implications. In: The Second Bridge on Artificial Intelligence for Scholarly Communication (AAAI-26), 20 January 2026, Singapore. 2026. https://doi.org/10.52825/ocp.v8i.3175 Lin S., Hilton J., Evans O. TruthfulQA: measuring how models mimic human falsehoods. In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics, 22–27 May 2022, Dublin, Ireland. Association for Computational Linguistics; 2022. P. 3214–3252. https://doi.org/10.18653/v1/2022.acl-long.229 Alkaissi H., McFarlane S.I. Artificial hallucinations in ChatGPT: implications in scientific writing. Cureus. 2023;15(2):e35179. https://doi.org/10.7759/cureus.35179 Izacard G., Lewis P., Lomeli M., et al. Atlas: few-shot learning with retrieval augmented language models. Journal of Machine Learning Research. 2023;24:251. Kung T.H., Cheatham M., Medenilla A., et al. Performance of ChatGPT on USMLE: potential for AI-assisted medical education using large language models. PLOS Digital Health. 2023;2(2):e0000198. https://doi.org/10.1371/journal.pdig.0000198 Anh-Hoang D., Tran V., Nguyen L.-M., et al. Survey and analysis of hallucinations in large language models: attribution to prompting strategies or model behavior. Frontiers in Artificial Intelligence. 2025;8:1622292. https://doi.org/10.3389/frai.2025.1622292 Guu K., Lee K., Tung Z., et al. Retrieval augmented language model pre-training. In: Proceedings of the 37th International Conference on Machine Learning, 13–18 July 2020, Virtual Event. PMLR; 2020. P. 3929–3938. Han B., Susnjak T., Mathrani A. Automating systematic literature reviews with retrieval-augmented generation: a comprehensive overview. Applied Sciences. 2024;14(19):9103. https://doi.org/10.3390/app14199103 Lewis P., Perez E., Piktus A., et al. Retrieval-augmented generation for knowledge-intensive NLP tasks. In: Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems, 06–12 December 2020, Virtual Event. 2020. Li Y., Du X., Wang Y., et al. AI-assisted literature screening: a hybrid approach using large language models and retrieval-augmented generation. International Journal of Medical Informatics. 2025;207:106205. https://doi.org/10.1016/j.ijmedinf.2025.106205 Asai A., He J., Shao R., et al. Synthesizing scientific literature with retrieval-augmented language models. Nature. 2026;650(8103):857–863. https://doi.org/10.1038/s41586-025-10072-4 Bao T., Nayeem M.T., Rafiei D., et al. SurveyGen: quality-aware scientific survey generation with large language models. In: Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, 04–09 November 2025, Suzhou, China. Association for Computational Linguistics; 2025. P. 2712–2736. https://doi.org/10.18653/v1/2025.emnlp-main.136 Shi X., Kou Q., Li Y., et al. SciSage: a multi-agent framework for high-quality scientific survey generation. arXiv. URL: https://doi.org/10.48550/arXiv.2506.12689 [Accessed 12th April 2026]. Sun Zh., Zhu X., Zhou X., et al. SurveyBench: can LLM(-agents) write academic surveys that align with reader needs? arXiv. URL: https://doi.org/10.48550/arXiv.2510.03120 [Accessed 11th April 2026]. Syed Sh., Hakimi A.D., Khatib Kh.A., et al. Citance-contextualized summarization of scientific papers. In: Findings of the Association for Computational Linguistics (EMNLP 2023), 06–10 December 2023, Singapore. Association for Computational Linguistics; 2023. P. 8551–8568. https://doi.org/10.18653/v1/2023.findings-emnlp.573 Saini N., Kumar S., Saha S., et al. Scientific document summarization using citation context and multi-objective optimization. In: 2020 25th International Conference on Pattern Recognition, 10–15 January 2021, Milan, Italy. IEEE; 2021. P. 4290–4295. https://doi.org/10.1109/icpr48806.2021.9412201 Mao Y., Zhong M., Han J. CiteSum: Citation text-guided scientific extreme summarization and domain adaptation with limited supervision. In: Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, 07–11 December 2022, Abu Dhabi, UAE. Association for Computational Linguistics; 2022. P. 10922–10935. https://doi.org/10.18653/v1/2022.emnlp-main.750 Gu N., Hahnloser R.H.R. SciLit: A platform for joint scientific literature discovery, summarization and citation generation. In: Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations), 09–14 July 2023, Toronto, Canada. Association for Computational Linguistics; 2023. P. 235–246. https://doi.org/10.18653/v1/2023.acl-demo.22 Zhu K., Feng X., Feng X., et al. Hierarchical catalogue generation for literature review: a benchmark. In: Findings of the Association for Computational Linguistics (EMNLP 2023), 06–10 December 2023, Singapore. Association for Computational Linguistics; 2023. P. 6790–6804. https://doi.org/10.18653/v1/2023.findings-emnlp.453 Groppe S., Hartung L. ReViz: a tool for automatically generating citation graphs and variants. In: Digital Libraries at Times of Massive Societal Transition: 22nd International Conference on Asia-Pacific Digital Libraries (ICADL 2020), 30 November – 01 December 2020, Kyoto, Japan. Cham: Springer; 2020. P. 107–121. https://doi.org/10.1007/978-3-030-64452-9_10 Hu Y., Lei Zh., Dai Zh., et al. CG-RAG: research question answering by citation graph retrieval-augmented LLMs. In: Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval, 13–18 July 2025, Padua, Italy. New York: ACM; 2025. P. 678–687. https://doi.org/10.1145/3726302.3729920 Ding H., Zhao Y., Hu T., et al. SciRAG: adaptive, citation-aware, and outline-guided retrieval and synthesis for scientific literature. In: Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics, 24–29 March 2026, Rabat, Morocco. Association for Computational Linguistics; 2026. P. 6440–6460. https://doi.org/10.18653/v1/2026.eacl-long.303 Menick J., Trebacz M., Mikulik V., et al. Teaching language models to support answers with verified quotes. arXiv. URL: https://doi.org/10.48550/arXiv.2203.11147 [Accessed 7th May 2026]. Li W., Xiao X., Liu J., et al. Leveraging graph to improve abstractive multi-document summarization. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 05–10 July 2020, Virtual Event. Association for Computational Linguistics; 2020. P. 6232–6243. https://doi.org/10.18653/v1/2020.acl-main.555 An C., Zhong M., Chen Y., et al. Enhancing scientific papers summarization with citation graph. Proceedings of the AAAI Conference on Artificial Intelligence. 2021;35(14):12498–12506. https://doi.org/10.1609/aaai.v35i14.17482 Chen J., Cai Ch., Jiang X., et al. Comparative graph-based summarization of scientific papers guided by comparative citations. In: Proceedings of the 29th International Conference on Computational Linguistics, 12–17 October 2022, Gyeongju, Republic of Korea. International Committee on Computational Linguistics; 2022. P. 5978–5988. Wadden D., Lo K., Kuehl B., et al. SciFact-open: Towards open-domain scientific claim verification. In: Findings of the Association for Computational Linguistics (EMNLP 2022), 07–11 December 2022, Abu Dhabi, UAE. Association for Computational Linguistics; 2022. P. 4719–4734. https://doi.org/10.18653/v1/2022.findings-emnlp.347 Zheng T., Deng Zh., Tsang H.T., et al. From automation to autonomy: a survey on large language models in scientific discovery. In: Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, 04–09 November 2025, Suzhou, China. Association for Computational Linguistics; 2025. P. 17733–17750. https://doi.org/10.18653/v1/2025.emnlp-main.895 Beltagy I., Lo K., Cohan A. SciBERT: a pretrained language model for scientific text. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, 03–07 November 2019, Hong Kong, China. Association for Computational Linguistics; 2019. P. 3615–3620. https://doi.org/10.18653/v1/d19-1371 Lin Ch.-Y. ROUGE: a package for automatic evaluation of summaries. In: Text Summarization Branches Out: Proceedings of the ACL-04 Workshop, 25–26 July 2004, Barcelona, Spain. Association for Computational Linguistics; 2004. P. 74–81. Zhang T., Kishore V., Wu F., et al. BERTScore: evaluating text generation with BERT. In: 8th International Conference on Learning Representations, 26–30 April 2020, Addis Ababa, Ethiopia. 2020. URL: https://openreview.net/pdf?id=SkeHuCVFDr Deutsch D., Bedrax-Weiss T., Roth D. Towards Question-Answering as an Automatic Metric for Evaluating the Content Quality of a Summary. Transactions of the Association for Computational Linguistics. 2021;9:774–789. https://doi.org/10.1162/tacl_a_00397 Chen Y., Eger S. MENLI: robust evaluation metrics from natural language inference. Transactions of the Association for Computational Linguistics. 2023;11:804–825. https://doi.org/10.1162/tacl_a_00576 Zhang Q., Wang Y., Jiang Y., et al. Crowd comparative reasoning: unlocking comprehensive evaluations for LLM-as-a-judge. In: Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics, 27 July – 01 August 2025, Vienna, Austria. Association for Computational Linguistics; 2025. P. 5059–5074. https://doi.org/10.18653/v1/2025.acl-long.252

The authors declare that there are no conflicts of interest present.