Development of an adaptive resource management system for containerized CAD systems based on reinforcement learning

This project is dedicated to the development of an adaptive resource management system for containerised computer-aided design (CAD) applications using reinforcement learning. Modern CAD workloads are characterised by highly variable computing requirements, which makes traditional threshold-based auto-scaling mechanisms insufficient for maintaining performance and reliability in dynamic conditions. To address this issue, the proposed system compares classic Kubernetes pod scaling based on thresholds (HPA) with a Q-learning-based auto-scaling strategy applied to container clusters. The experimental setup is implemented as a simulation of a distributed containerised cluster and includes customisable workload models representing light, medium, heavy, and peak request patterns. System performance is evaluated using metrics such as response time, throughput, availability, cost-effectiveness, mean time to recovery, and false positive scaling events. A reinforcement learning agent monitors tracked system metrics and learns scaling policies that optimise long-term performance and stability through repeated interactions with the environment. The application interface allows users to control simulation parameters, including the number of policy runs, the number of episodes per run, and the number of steps per episode, as well as cluster configuration parameters such as the number of nodes and cores per node. The workload intensity can be adjusted to analyse system behaviour in different operating scenarios. This configuration allows for systematic evaluation of adaptive auto-scaling strategies and their impact on resource efficiency and fault tolerance in containerised CAD systems. The study represents a methodological innovation thanks to its interactive, experiment-based evaluation interface, which combines modelling and orchestration logic.

1. Bogatyrev V.A., Bogatyrev A.V., Bogatyrev S.V. Estimation of reliability of execution of real-time queries. Journal of Instrument Engineering. 2014;57(4):46–48. (In Russ.).

2. Bogatyrev V.A., Fung V.K. Multi-Criteria Optimization of Cluster Structure with Container Virtualization. Journal of Instrument Engineering. 2025;68(5):371–379. (In Russ.). https://doi.org/10.17586/0021-3454-2025-68-5-371-379

3. Phung V.Q., Bogatyrev V.A., Karmanovskiy N.S., Le V.H. Evaluation of probabilistic-temporal characteristics of a computer system with container virtualization. Scientific and Technical Journal of Information Technologies, Mechanics and Optics. 2024;24(2):249–255. (In Russ.). https://doi.org/10.17586/2226-1494-2024-24-2-249-255

4. Gusev A.O., Kostyleva V.V., Razin I.B. Comparison of Load Balancing Algorithms. In: Innovative Development of Machinery and Technologies in Industry (INTEX-2020): Proceedings of the All-Russian Scientific Conference of Young Researchers with International Participation dedicated to the Anniversary Year at the Kosygin State University of Russia, 14–16 April 2020, Moscow, Russia. Moscow: Kosygin State University of Russia; 2020. P. 143–146. (In Russ.).

5. Zhu J., Zheng Z., Zhou Y., Lyu M.R. Scaling service-oriented applications into geo-distributed clouds. In: 2013 IEEE Seventh International Symposium on Service-Oriented System Engineering, 25–28 March 2013, San Francisco, CA, USA. IEEE; 2013. P. 335–340. https://doi.org/10.1109/SOSE.2013.56

6. Sharma A., Schuhknecht F.M., Dittrich J. The Case for Automatic Database Administration using Deep Reinforcement Learning. arXiv. URL: https://doi.org/10.48550/arXiv.1801.05643 [Accessed 17th January 2026].

7. Munaf R.M., Ahmed J., Khakwani F., Rana T. Microservices architecture: Challenges and proposed conceptual design. In: 2019 International Conference on Communication Technologies (ComTech), 20–21 March 2019, Rawalpindi, Pakistan. IEEE; 2019. P. 82–87. https://doi.org/10.1109/COMTECH.2019.8737831

8. Rudenko V.D., Yudin N.E., Vasin A.A. Survey of convex optimization of Markov decision processes. Computer Research and Modeling. 2023;15(2):329–353. (In Russ.). https://doi.org/10.20537/2076-7633-2023-15-2-329-353

9. Phung V.K., Bogatyrev V.A., Do M.K. Simulation model of a computing cluster with container virtualization. Herald of Computer and Information Technologies. 2025;22(8):3–12. (In Russ.). https://doi.org/10.14489/vkit.2025.08.pp.003-012

10. Utkin L.V. Knowledge representation for characterizing economical and technical measures under uncertainty: general approach. In: 9th Interchair Meeting of Economists and Organisers in Wood Industry, Sopron, Hungary. 1998. P. 220.

11. Chadi M.-A., Mousannif H. Understanding Reinforcement Learning Algorithms: The Progress from Basic Q-learning to Proximal Policy Optimization. arXiv. URL: https://doi.org/10.48550/arXiv.2304.00026 [Accessed 15th January 2026].

12. Nguyen Th.-T., Yeom Y.-J., Kim T., Park D.-H., Kim S. Horizontal Pod Autoscaling in Kubernetes for Elastic Container Orchestration. Sensors. 2020;20(16). https://doi.org/10.3390/s20164621

13. Shan Ch., Wu Ch., Xia Y., et al. Adaptive resource allocation for workflow containerization on Kubernetes. Journal of Systems Engineering and Electronics. 2023;34(3):723–743. https://doi.org/10.23919/JSEE.2023.000073