SOFTWARE SELF-ADAPTATION METHOD BASED ON MACHINE LEARNING TECHNOLOGY

The article discusses development and application issues of software self-adaptation method based on machine learning technology. The differences between the Model-Based and Model-Free approaches in reinforcement learning are considered, the choice of the Model-Based approach for creating a software self-adaptation method is substantiated. The definition of an expanded Markov decision-making process that takes into account the role of the situation in the course of program selfadaptation is considered. A mathematical model of the state space of the software system is proposed, based on the hypergraphic formalization of the model of characteristics. Based on the expanded definition of the Markov decision-making process, the proposed model of the state space of the system, and the concept of the Model-Based approach to machine learning with reinforcement, a new method of software self-adaptation was developed that takes into account the effect of the actions performed by the system on the state of the environment. A practical example of using the method is given.

1. Han H. Model-based Reinforcement Learning Approach for Planning in Self-Adaptive System. International Conference on Ubiquitous Information Management and Communication. New Jersey: IEEE; 2015:156-178.

2. Simmonds J., Bastarrica M.C. Modeling variability in software process lines. Departamento de Ciencias de la Computación. 2011;4:93-100.

3. Bershadskij A.M., Bozhdaj A.S., Evseeva YU.I., Gudkov A.A. Matematicheskaya model refleksii samoadaptivnyh programmnyh system. Izvestiya Volgogradskogo gosudarstvennogo tekhnicheskogo universiteta. 2018;8:7-14.

4. Bozhdaj A.S., Evseeva YU.I. Metod refleksivnoj samoadaptacii programmnyh system. Izvestiya vysshih uchebnyh zavedenij. Povolzhskij region. Tekhnicheskie nauki. 2018;46(2): 74-86.

5. Machine Learning Proceedings 1991: Proceedings of the Eighth International Workshop (ML91). Elsevier Science; 2014.

6. Silver D. Reinforcement learning and simulation-based search in computer go. Alta., Canada: University of Alberta Edmonton; 2009

7. Fishman, George S. Monte Carlo: concepts, algorithms, and applications. Springer; 1996.

8. Colum R. Efficient Selectivity and Backup Operators in Monte-Carlo Tree Search. Computers and Games, 5th International Conference. Turin, Italy: Springer; 2006.