Keywords: SOC, effectiveness assessment, MTTD, MTTR, context completeness, investigation reproducibility, load resilience, SIEM, SOAR, XDR
Evaluating the effectiveness of cyber‑threat monitoring and response centers: limits of time‑based metrics and operational quality indicators
UDC 004.056.55
DOI: 10.26102/2310-6018/2025.51.4.040
The paper examines practices for evaluating the effectiveness of cyber‑threat monitoring and response centers under growing telemetry volumes and increasingly complex attacks. It is shown that commonly used indicators such as mean time to detect and mean time to respond mainly capture speed while failing to assess whether available data are sufficient for sound conclusions, whether investigation context is present, and whether investigation steps are reproducible. The study compares international guidance and landscape reports with Russian regulatory requirements and analyzes industry publications. As a result, data sources used by monitoring centers are systematized, typical bottlenecks in the data value chain are identified, and limitations of classic time‑based metrics are highlighted. A simple three‑axis comparison framework is proposed: speed, context, and process. The contribution introduces three computable indicators: context completeness (share of incidents corroborated by at least three independent sources), investigation reproducibility (share of steps executed via approved playbooks with machine‑readable logging), and resilience to peak loads (comparison of service‑level target adherence in peak versus baseline periods), together with an integral manageability index combining speed, accuracy, and completeness. The practical value lies in the feasibility of calculating these indicators using existing security event management systems and incorporating them into monitoring dashboards for audit, resource planning, and cross‑team comparability.
1. Shablovskii Ya.K., Gel'fand A.M. Obzor tekhnologii SOC (Security Operations Center). Innovatsii. Nauka. Obrazovani. 2021;(33):1316–1321. (In Russ.).
2. Kuznetsov A.V. The Organization of Separate Security Event Data Storage. Voprosy kiberbezopasnosti. 2024;(2):22–28. (In Russ.). https://doi.org/10.21681/2311-3456-2024-2-22-28
3. Forsberg J., Frantti T. Technical Performance Metrics of a Security Operations Center. Computers & Security. 2023;135. https://doi.org/10.1016/j.cose.2023.103529
4. Agyepong E., Cherdantseva Yu., Reinecke Ph., Burnap P. A Systematic Method for Measuring the Performance of a Cyber Security Operations Centre Analyst. Computers & Security. 2023;124. https://doi.org/10.1016/j.cose.2022.102959
5. Shilova A.D. Criterion of the Network Infrastructure Security. Scientific and Technical Journal of Information Technologies, Mechanics and Optics. 2023;23(3):530–537. (In Russ.). https://doi.org/10.17586/2226-1494-2023-23-3-530-537
6. Bridges R.A., Rice A.E., Oesch S., et al. Testing SOAR Tools in Use. Computers & Security. 2023;129. https://doi.org/10.1016/j.cose.2023.103201
7. Islam M.A. Application of Artificial Intelligence and Machine Learning in a Security Operations Center. Issues in Information Systems. 2023;24(4):311–327. https://doi.org/10.48009/4_iis_2023_124
8. González-Granadillo G., González-Zarzosa S., Diaz R. Security Information and Event Management (SIEM): Analysis, Trends, and Usage in Critical Infrastructures. Sensors. 2021;21(14). https://doi.org/10.3390/s21144759
9. Shaked A., Cherdantseva Yu., Burnap P., Maynard P. Operations-Informed Incident Response Playbooks. Computers & Security. 2023;134. https://doi.org/10.1016/j.cose.2023.103454
10. Fedorchenko A., Levshun D., Chechulin A., Kotenko I. An Analysis of Security Event Correlation Techniques in SIEM-Systems. Part 1. SPIIRAS Proceedings. 2016;(4):5–27. (In Russ.). https://doi.org/10.15622/sp.47.1
11. Fedorchenko A., Levshun D., Chechulin A., Kotenko I. An Analysis of Security Event Correlation Techniques in SIEM-Systems. Part 2. SPIIRAS Proceedings. 2016;(6):208–225. (In Russ.). https://doi.org/10.15622/sp.49.11
12. Mahboubi A., Luong Kh., Aboutorab H., et al. Evolving Techniques in Cyber Threat Hunting: A Systematic Review. Journal of Network and Computer Applications. 2024;232. https://doi.org/10.1016/j.jnca.2024.104004
13. Afanaseva S.V., Kuzmina U.V. Main Problems Working with Security Operation Center. Journal of the Ural Federal District. Information Security. 2023;(1):51–58. (In Russ.). https://doi.org/10.14529/secur230105
14. Feng W., Cao Yu, Chen Y. Multi-Granularity User Anomalous Behavior Detection. Applied Sciences. 2025;15(1). https://doi.org/10.3390/app15010128
Keywords: SOC, effectiveness assessment, MTTD, MTTR, context completeness, investigation reproducibility, load resilience, SIEM, SOAR, XDR
For citation: Pakhomov V.V. Evaluating the effectiveness of cyber‑threat monitoring and response centers: limits of time‑based metrics and operational quality indicators. Modeling, Optimization and Information Technology. 2025;13(4). URL: https://moitvivt.ru/ru/journal/pdf?id=2092 DOI: 10.26102/2310-6018/2025.51.4.040 (In Russ).
Received 11.10.2025
Revised 07.11.2025
Accepted 17.11.2025