Development of a multi-class classifier algorithm for a federated learning system operating in case of incomplete classes of local classifiers

The relevance of research is due the need to solve the problem of training multi-class classifier models used in federated machine learning system structure operating with a training data set that contains both publicly available data and confidential data that forming hidden classes. A similar problem arises in the context of training a classifier using a training data set, some of which consists of personal information or data of varying degrees of confidentiality. In this regard, this article is aimed at researching the features of the Gaussian mixture model of distributions as a way of representing hidden classes representing confidential data, as well as justifying the choice of an algorithmic method for finding maximum likelihood estimates of its parameters. The main method for solving the problem of identifying the parameters of hidden classes is a reasonably chosen two-stage iterative expectation-maximization procedure (EM-algorithm), which ensures strengthening the relationship between missing (confidential) data and unknown parameters of the data model represented by a Gaussian mixture of distributions. The article presents a diagram of the developed algorithm of a multi-class classifier for federated machine learning system, represented by parallel cycles of forming local learning models and their ensemble into a global learning model.

1. Wahab O.A., Mourad A., Otrok H., Taleb T. Federated Machine Learning: Survey, Multi-Level Classification, Desirable Criteria and Future Directions in Communication and Networking Systems. IEEE Communications Surveys & Tutorials. 2021;23(2):1342–1397. https://doi.org/10.1109/COMST.2021.3058573

2. Lo S.K., Lu Q., Zhu L., Paik H.-Y., Xu X., Wang C. Architectural Patterns for the Design of Federated Learning Systems. Journal of Systems and Software. 2022;191. https://doi.org/10.1016/j.jss.2022.111357

3. Mikhalev P.A., Kutsakin M.A., Mironov O.Yu. On the need for parametric optimization of systems with federated machine learning. In: Modern Informatization Problems in Simulation and Social Technologies (MIP-2023'SCT): Proceedings of the XXVIII-th International Open Science Conference, 15 November 2022 – 15 January 2023, Yelm, WA, USA. Yelm: Science Book Publishing House LLC; 2023. pp. 37–41. (In Russ.).

4. Allwein E.L., Schapire R.E., Singer Y. Reducing multiclass to binary: a unifying approach for margin classifiers. Journal of Machine Learning Research. 2001;1:113–141.

5. Goodman R., Miller J.W., Smyth P. Objective Functions For Neural Network Classifier Design. In: 1991 IEEE International Symposium on Information Theory, 24–28 June 1991, Budapest, Hungary. IEEE; 1991. pp. 87. https://doi.org/10.1109/ISIT.1991.695143

6. Galar M., Fernández A., Barrenechea E., Bustince H., Herrera F. An overview of ensemble methods for binary classifiers in multi-class problems: Experimental study on one-vs-one and one-vs-all schemes. Pattern Recognition. 2011;44(8):1761–1776. https://doi.org/10.1016/j.patcog.2011.01.017

7. Arı Ç., Aksoy S., Arıkan O. Maximum likelihood estimation of Gaussian mixture models using stochastic search. Pattern Recognition. 2012;45(7):2804–2816. https://doi.org/10.1016/j.patcog.2011.12.023

8. Tohka J., Krestyannikov E., Dinov I.D., Graham A.M., Shattuck D.W., Ruotsalainen U. Genetic Algorithms for Finite Mixture Model Based Voxel Classification in Neuroimaging. IEEE Transactions on Medical Imaging. 2007;26(5):696–711. https://doi.org/10.1109/TMI.2007.895453

9. Martı́nez A.M., Vitrià J. Learning mixture models using a genetic version of the EM algorithm. Pattern Recognition Letters. 2000;21(8):759–769. https://doi.org/10.1016/S0167-8655(00)00031-3

10. Pernkopf F., Bouchaffra D. Genetic-based EM algorithm for learning Gaussian mixture models. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2005;27(8):1344–1348. https://doi.org/10.1109/TPAMI.2005.162