Algorithms and programs for calculating nonparametric criteria for statistical hypothesis testing based on permutations with repetitions

One of the important tasks of statistical analysis is to test statistical hypotheses, and in this group the most promising is the subgroup of nonparametric ranking criteria, which are very stable for work with small samples, when it is not possible to reliably justify the hypothetical law of distribution. In its turn, this fact causes the necessity to abandon asymptotic approximations and to have exact critical values of the criteria (or so-called p-values in modern literature). At present, analytical solutions are available only for a very limited class of criteria (signs, Wilcoxon, series, Ansari-Bradley). For all others, a computerized enumeration of a huge number of possible permutations of ranks is required for an exact solution. The creation of a universal algorithm for obtaining an accurate and fast distribution of ranks of nonparametric criteria is the focus of the present work. The algorithm, implemented in open-source programming languages C++, Javascript and Python, is based on a well-known combinatorics problem - permutations with repetitions, with its adaptation to the task of hypothesis testing by rank criteria. The following criteria are considered as such criteria: Kraskell-Wallis, Muda, Lehman-Rosenblatt, as well as a group of normal label criteria: Fisher-Yates, Capon, Klotz, Van der Varden. The algorithm is also adapted for other possible ranking problems of nonparametric statistics.

1. Mehta C.R., Patel N.R. IBM SPSS Exact Tests. IBM Corp.; 2013. 226 p.

2. Agamirov L.V., Agamirov V.L., Vestyak V.A. Numerical Methods and Algorithms of Calculation of Exact Distributions of Non-Parametrical Criteria Statistical Hypotheses. Aerospace MAI Journal. 2013;20(4):212–218. (In Russ.).

3. Van de Wiel M.A. The Probability Generating Function of the Freund-Ansari-Bradley Statistic. In: Memorandum COSOR: Volume 9711. Eindhoven: Technische Universiteit Eindhoven; 1997. 9 p.

4. Choi W., Lee J.W., Huh M.-H., Kang S.-H. An Algorithm for Computing the Exact Distribution of the Kruskal-Wallis Test. Communications in Statistics – Simulation and Computation. 2003;32(4):1029–1040. https://doi.org/10.1081/SAC-120023876

5. Meyer J.P., Seaman M.A. A Comparison of the Exact Kruskal-Wallis Distribution to Asymptotic Approximations for All Sample Sizes up to 105. The Journal of Experimental Education. 2013;81(2):139–156. https://doi.org/10.1080/00220973.2012.699904

6. Van de Wiel M.A. Exact Distributions of Multiple Comparisons Rank Statistics. Journal of the American Statistical Association. 2002;97(460):1081–1089. https://doi.org/10.1198/016214502388618898

7. Odiase J.I., Ogbonmwan S.M. JMASM20: Exact Permutation Critical Values for The Kruskal-Wallis One-Way ANOVA. Journal of Modern Applied Statistical Methods. 2005;4(2). https://doi.org/0.22237/jmasm/1130804820

8. Spurrier J.D. On the Null Distribution of the Kruskal-Wallis Statistic. Journal of Nonparametric Statistics. 2003;15(6):685–691. https://doi.org/10.1080/10485250310001634719

9. Divine G.W., Norton H.J., Barón A.E., Juarez-Colunga E. The Wilcoxon-Mann-Whitney Procedure Fails as a Test of Medians. The American Statistician. 2018;72(3):278–286. https://doi.org/10.1080/00031305.2017.1305291

10. Hothorn T., Hornik K., van de Wiel M.A., Zeileis A. Implementing a Class of Permutation Tests: The Coin Package. Journal of Statistical Software. 2008;28(8):1–23. https://doi.org/10.18637/jss.v028.i08

11. Antipina N.M., Zakharov V.N., Protasov Yu.M., Yurov V.M. Non-Parametric Criterion of Difference for Two Related Samples in Table Editor MS Excel. Bulletin of Moscow Region State University. Series: Economics. 2021;(2):47–55. (In Russ.).

12. Streitberg B., Rohmel J. Exact Distributions for Permutation and Rank Tests: An Introduction to Some Recently Published Algorithms. Statistical Software Newsletter. 1986;12(1):10–17.

13. Kobzar' A.I. Prikladnaya matematicheskaya statistika. Dlya inzhenerov i nauchnykh rabotnikov. Moscow: FIZMATLIT; 2006. 816 p. (In Russ.).

14. Pollard J.H. A Handbook of Numerical and Statistical Techniques: With Examples Mainly from the Life Sciences. Cambridge: Cambridge University Press; 1977. 349 p.

15. Lipskii V. Kombinatorika dlya programmistov. Moscow: Mir; 1988. 213 p. (In Russ.).