A method for constructing a logical model for interpreting the decisions of a trained neural network

In this paper, we propose a method for interpreting neural network solutions based on the use of Boolean integro-differential calculus. This method allows you to investigate the logic of decision-making by neural networks and determine the most important signs that affect their decisions. The method can be applied to classification problems, especially in cases where each feature can be represented as a k-valued variable. The paper considers local and global interpretations of solutions. At the first stage, each input vector is associated with the corresponding output of the neural network. Then, by solving a Boolean equation, logical functions are found that adequately reflect the input data and their corresponding outputs. At the second stage, global interpretation, functions are constructed that combine previously found logical functions. This choice of functions is based on their ability to most accurately reflect the decisions of the neural network and the study area. At the second stage, global interpretation, functions are constructed that combine previously found logical functions. This choice of functions is based on their ability to most accurately reflect the decisions of the neural network and the study area. The resulting function has interpretability, modifiability and the ability to represent a complete set of solutions corresponding to a given query. It also highlights the most significant features for each solution. The paper considers the practical implementation of the method on the example of a neural network trained on the basis of the structure and input data consisting of answers to questionnaire questions, with an output node predicting the diagnosis. In parallel with the development of the neural network, an interpretive model is being built, which allows identifying the most important signs for each diagnosis based on the decisions of the neural network. In addition, in cases with boundary solutions, when the neural network provides only one possible solution, the interpretative model is able to find all possible solutions with a predetermined accuracy, which helps to avoid mistakes in decision-making.

1. Shibzukhov Z. Correct algorithms for aggregation of operations. Pattern recognition and image analysis. 2014;24-3:377–382. (In Russ.).

2. Averkin A.N., Yarushev S.A. Obzor issledovanii v oblasti razrabotki metodov izvlecheniya pravil iz iskusstvennykh neironnykh setei. Izvestiya RAN. Teoriya i sistemy upravleniya. 2021;6:106–121. (In Russ.).

3. Ashley I., Naimi Laura B., Balzer Multilevel Ashley I. Naimi, Laura B. Balzer. Multilevel generalization: an introduction to super learning. European Journal of Epidemiology. 2018;33:459–464.

4. Haoxiang W., Smith S. Big data analysis and perturbation using a data mining algorithm. Journal of Soft Computing Paradigm. 2021;3-01:19–28.

5. Ribeiro, M. T., Singh, S., Guestrin C. "Why should I trust you?": Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2016:1135–1144.

6. Joe M., Vijesh, Jennifer S. Raj. User Recommendation System Dependent on Location-Based Orientation Context. Journal of Trends in Computer Science and Smart Technology. 2021;3-01:14–23.

7. Grabisch M., Marichal J-L., Mesiar R., Pap E. Aggregation Functions. Cambridge, Cambridge University Press; 2009. (Encyclopedia of Mathematics and its Applications).

8. Yang F., Yang Zh, Cohen W.W. Differentiable learning of logical rules for reasoning in the knowledge base. Advances in the field of neural information processing systems. 2017;3:2320–2329.

9. Lyutikova L.A. Construction of a Logical-Algebraic Corrector to Increase the Adaptive Properties of the ΣΠ-Neuron. Journal of Mathematical Sciences. 2021;253:539–546.

10. Dyukova E.V., Zhuravlev Yu.I., Prokofev P.A. Methods for increasing the efficiency of logic correctors. Mashinnoe obuchenie i analiz dannykh. 2015;11-1:1555–1583. (In Russ.).