Neuro-fuzzy classifiers

This paper considers the problem of increasing the accuracy of artificial neural networks in the tasks of states classification of objects with different physical nature. It is proposed to define this problem as a problem of choosing the activation function type in artificial neural networks and to consider it from the perspective of the fuzzy sets theory. In this regard, a mathematical model of the artificial neuron adaptive activation function has been developed, using a fuzzy logic system with interval fuzzy sets of the second type. This function differs from ordinary activation functions used in neural network models in that the range of its input values is limited, and, at the same time, such a function allows to optimize the parameters that determine the shape of the curve in the process of training an artificial neural network. To reduce the computational complexity of a neuro-fuzzy model with a fuzzy activation function, its modification is proposed, which involves the use of the mathematical function of the hyperbolic tangent to normalize the values of the vector supplied to the fuzzy function input. Algorithmic support has been developed for two architectures of neuro-fuzzy classifiers - a recurrent neuro-fuzzy classifier and a convolutional neuro-fuzzy classifier. Two experiments on the classification of biomedical and text objects were carried out, in which the accuracy indicators of neuro-fuzzy classifiers models and classifiers similar in structure without a fuzzy activation function were compared; additionally, an increase in the accuracy of artificial neural networks, which used fuzzy activation functions, was confirmed.

1. Lomakina L.S., Subbotin A.N. Stream data classification based on Bayesian criteria. Modelirovaniye, optimizatsiya i informatsionnyye tekhnologii = Modeling, Optimization and Information Technology. 2020;8(1). Available by: https://moit.vivt.ru/wp- content/uploads/2020/02/LomakinaSubbotin_1_20_1.pdf DOI: 10.26102/2310-6018/2020.28.1.034. (In Russ.)

2. Sonoda S., Murata N. Neural network with unbounded activation functions is universal approximator. Applied and Computational Harmonic Analysis. 2017;43(2):233–268. DOI: 10.1016/j.acha.2015.12.005.

3. Coffman A. Introduction to Fuzzy Sets Theory. Moscow: Radio and Communication; 1982. 432 p. (In Russ.)

4. Kumbasar T. Robust stability analysis and systematic design of single-input interval type-2 fuzzy logic controllers. IEEE Transactions on Fuzzy Systems. 2015;24(3):675–694. DOI: 10.1109/TFUZZ.2015.2471805.

5. Liang Q., Mendel J.M. Interval type-2 fuzzy logic systems: theory and design. IEEE Transactions on Fuzzy systems. 2000;8(5):535–550. DOI: 10.1109/91.873577.

6. Beke A., Kumbasar T. Learning with type-2 fuzzy activation functions to improve the performance of deep neural networks. Engineering Applications of Artificial Intelligence. 2019;85:372–384. DOI: 10.1016/j.engappai.2019.06.016.

7. Greff K., Srivastava R.K., Koutník J., Steunebrink B.R., Schmidhuber J. LSTM: A search space odyssey. IEEE transactions on neural networks and learning systems. 2016;28(10):2222–2232. DOI: 10.1109/TNNLS.2016.2582924.

8. Lomakina L.S., Surkova A.S., Zhevnerchuk D.V., Chernobaev I.D., et al. Conceptual modeling of heterogeneous data for geoinformation systems. Proc. of the Thirteenth International MEDCOAST Congress on Coastal and Marine Sciences, Engineering, Management and Conservation. 2019;1:77–89.

9. Lomakina L.S., Chernobaev I.D., Kiselev Y.N. Algorithmic support of neuro-fuzzy classification of objects of complex structure. Trudy Mezhdunarodnogo nauchno-tekhnicheskogo kongressa «Intellektual'nyye sistemy i informatsionnyye tekhnologii - 2021» («IS & IT-2021», «IS&IT’21») = Proceedings of the International Scientific and Technical Congress «Intelligent systems and information technology - 2021» («IS&IT’21»). 2021;475–481. (In Russ.)

10. Surkova A., Skorynin S., Chernobaev I. Word embedding and cognitive linguistic models in text classification tasks. Proc. of the XI International Scientific Conference Communicative Strategies of the Information Society. 2019;1–6. DOI: 10.1145/3373722.3373778.

11. Mistry J., Chuguransky S., Williams L., Qureshi M., Salazar G.A., Sonnhammer E.L., Tosatto S.C., Paladin L., Raj S., Richardson L.J., Finn R.D. Pfam: The protein families database in 2021. Nucleic Acids Research. 2021;49(D1):D412–419. DOI: 10.1093/nar/gkaa913.

12. Maas A., Daly R.E., Pham P.T., Huang D., Ng A.Y., Potts C. Learning word vectors for sentiment analysis. Proc. of the 49th annual meeting of the association for computational linguistics: Human language technologies. 2011;142–150.