Practical aspects of building private multimodal generative models: methods, constraints, and tools

The article addresses the pressing issue of developing generative artificial intelligence systems capable of working with heterogeneous data (text, images, audio) without compromising the privacy of the underlying training datasets. The aim of the study is to systematize and present, from a practical perspective, current methods for ensuring privacy applicable to multimodal architectures. Particular attention is paid to differential privacy and federated learning technologies, their adaptation, and their combination for working with complex data. The article analyzes fundamental trade-offs between generation quality, computational complexity, and the level of privacy guarantees faced by developers in practice. Examples of existing software frameworks are provided, along with recommendations for selecting protection strategies depending on the type of task and the nature of the multimodal data. Practical aspects of integrating privacy mechanisms into training cycles, assessing the accumulated privacy budget, and potential directions for developing tools to enhance the efficiency and reliability of AI systems are additionally discussed. Special attention is given to issues of modality alignment and optimizing the trade-off between privacy level and generation quality. The presented recommendations and implementation examples can serve as a guide for engineers and researchers in developing real-world multimodal systems that meet contemporary security and ethical requirements. The material of the article is intended for researchers and engineers in the field of machine learning who are engaged in creating AI systems that comply with ethical and regulatory standards.

1. Feretzakis G., Papaspyridis K., Gkoulalas-Divanis A., Verykios V.S. Privacy-Preserving Techniques in Generative AI and Large Language Models: A Narrative Review. Information. 2024;15(11). https://doi.org/10.3390/info15110697

2. Naseri M., Hayes J., De Cristofaro E. Local and Central Differential Privacy for Robustness and Privacy in Federated Learning. arXiv. URL: https://arxiv.org/abs/2009.03561 [Accessed 25th November 2025].

3. Sun L., Qian J., Chen X. LDP-FL: Practical Private Aggregation in Federated Learning with Local Differential Privacy. arXiv. URL: https://arxiv.org/abs/2007.15789 [Accessed 25th November 2025].

4. Rafi T.H., Noor F.A., Hussain T., Chae D.-K. Fairness and Privacy-Preserving in Federated Learning: A Survey. arXiv. URL: https://arxiv.org/abs/2306.08402 [Accessed 25th November 2025].

5. Zhu L., Chen X. Privacy protection in federated learning: a study on the combined strategy of local and global differential privacy. The Journal of Supercomputing. 2025;81(1). https://doi.org/10.1007/s11227-024-06845-9

6. Kataev A.V., Vlasova Y.M., Gusynin D.A., Kim V.A. A survey of metrics for evaluating the performance of generative models in image creation. Engineering Journal of Don. 2025;(6). (In Russ.). URL: http://www.ivdon.ru/en/magazine/archive/n6y2025/10119

7. Rabchevsky A.N. Review of methods and systems for generation of synthetic training data. Applied Mathematics and Control Sciences. 2023;(4):6–45. (In Russ.).

8. Xu H., Shrestha Sh., Chen W., Li Zh., Cai Zh. DP-FedLoRA: Privacy-Enhanced Federated Fine-Tuning for On-Device Large Language Models. arXiv. URL: https://arxiv.org/abs/2509.09097 [Accessed 19th December 2025].

9. Ghalebikesabi S., Berrada L., Gowal S., et al. Differentially Private Diffusion Models Generate Useful Synthetic Images. arXiv. URL: https://arxiv.org/abs/2302.13861 [Accessed 19th December 2025].

10. McMahan B., Moore E., Ramage D., Hampson S., Agüera-Arcas B. Communication-Efficient Learning of Deep Networks from Decentralized Data. In: Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS 2017), 20–22 April 2017, Fort Lauderdale, FL, USA. PMLR; 2017. P. 1273–1282.