User engagement is one of the key indicators of the quality of interactive software (software),
which is characterized by intense user interaction with the system. Training software belongs to the
category of products, which, by definition, are based on interaction with the user, so the user's
involvement in the process of his interaction with the training system directly affects the quality of the
system. Modern trends in the development of educational software are associated with the
personification of the processes of user interaction with the system, which has led to the emergence of
adaptive educational systems that can monitor user actions and adapt to their capabilities and needs.
User involvement has a significant impact on learning and directly affects the result, therefore, the level of user involvement in the process of its interaction with the training system, as an indirect assessment
of the user's knowledge level, is applicable as a characteristic of the adaptation model in the development
of adaptive learning systems. The results of the engagement analysis can be used to adapt the system
aimed at retaining and increasing user engagement in the process of its interaction with the system, and
thus improve its quality. The paper considers methods for assessing involvement and the possibility of
their application to assessing the quality of educational software at different stages of its life cycle. The
features of the use of online-assessment of engagement to adapt the learning process to the user in
adaptive learning games are shown, related to the need to distinguish between involvement in the game
and involvement in the learning process, and correlation of involvement and success in mastering
knowledge in the game. Some possible combinations of assessments of the involvement and
effectiveness of the user's knowledge level in the process of interaction with the educational game and
their possible interpretations are proposed.
1. Shabalina O.A., Kataev A.V., Davtyan A.G. Faceted training classification
computer games. Bulletin of the Volgograd State Technical
university. 2018; 13 (223): 95-100.
2.
Shabalina O.A., Davtyan A.G., Kataev A.V., Alimov A.A. Adaptive learning games
as a trend in the development of teaching software. ITNOU: Information technology in science,
education and management. 2018; 4 (8): 11-16.
3. Engagement: Definition Available in:
https://support.google.com/analytics/answer/9355853?hl=en (accessed: 11/16/2019)
4. Attfield S., Kazai G., Lalmas M., Piwowarski B. Towards a science of user engagement
(Position Paper). Web Search and Data Mining. 2011.
5. Csikszentmihalyi M. Flow: The psychology of optimal experience. Harper and Row. 1990.
6. Webster J., Ho H. Audience engagement with multimedia presentations. The DATABASE
for Advances in Information Systems. 1997:63–77. DOI: 10.1145/264701.264706.
7. The Best Metrics and Tools for Measuring User Engagement Available in:
https://hackernoon.com/the-best-metrics-and-tools-for-measuring-user-engagementfb083d9a9be7 (accessed: 11/16/2019).
8. Murphy D., Higgins C. Secondary Inputs for Measuring User Engagement in Immersive
VR Education Environments. 2019:1-6.
9. Russell J. A., Weiss A., Mendelsohn G. A. Affect grid: A single-item scale of pleasure and
arousal. Journal Personality and Social Psychology. 1988:493– 502. DOI: 10.1037/0022-
3514.57.3.493.
10. Boucsein W. Electrodermal activity. New York: Springer. 2012.
11. Hardy M., Wiebe E.N., Grafsgaard J.F., Boyer K.E., Lester J.C. Physiological responses to
events during training use of skin conductance to inform future adaptive learning systems.
Proceedings of the Human Factors and Ergonomics Society Annual Meeting. 2013:2101–
2105. DOI: 10.1177/1541931213571468.
12. Ashby F. G., Isen A. M., Turken A. U. A neuropsychological theory of positive affect and
its influence on cognition. Psychological Review. 1999;106(3):529-550. DOI:
10.1037/0033-295x.106.3.529.
13. Lalmas M., O’Brien H., Yom-Tov E. Measuring User Engagement. Synthesis lectures on
information concepts, retrieval, and services. 2015:11-58.
14. Grafsgaard J. F., Boyer K. E., Wiebe E. N., Lester J. C. Analyzing Posture and Affect in
Task-Oriented Tutoring. Artificial Intelligence. Proceedings of the 25th Florida Artificial
Intelligence Research Society Conference. 2012:438-443.
15. Pekrun R., Goetz T., Titz W., Perry R. Academic emotions in students’ self-regulated
learning and achievement: A program of qualitative and quantitative research. Educational
Psychologist. 2002;37(2):91–105. DOI: 10.1207/s15326985ep3702_4.
16. Voronina A.A., Shabalina O.A., Kataev A.V. Engagement assessment methods
users of interactive applications. ITNOU: Information technology in
science, education and management. 2019; 4 (14): 70-74.
17. Rebelo F., Noriega P., Duarte E., Soares M. Using virtual reality to assess user experience.
Human Factors: The Journal of Human Factors and Ergonomics Society. 2012;54(6):964-
982. DOI: 10.1177/0018720812465006.
Shabalina Olga Arkadyevna
candidate of technical sciences
Email: o.a.shabalina@gmail.com
Volgograd State Technical University
Volgograd, Russian Federation
Kataev Alexander Vadimovich
candidate of technical sciences
Email: alexander.kataev@gmail.com
Volgograd State Technical University
Volgograd, Russian Federation
Voronina Angelina Andreevna
Email: angelina.vaa@gmail.com
Volgograd State Technical University
Volgograd, Russian Federation