Table of Contents
- Executive Summary
- Environment and Testing
- The Attack Simulation
- Evaluation and Results
- Conclusion
- Appendix – References
- About William McKnight
- About GigaOm
- Copyright
1. Executive Summary
Security information and event management (SIEM) technology supports threat detection, compliance, and security incident management through the collection and analysis (near real-time and historical) of security events and a wide variety of other event and contextual data sources. SIEM applications combine multiple information security data sources into a single tool to support incident detection, risk management, and compliance activities.
Configured properly, a SIEM solution can detect known threats, correlate logs, and create actions in response to threat. These capabilities provide the base for a security monitoring and alerting strategy, which in turn drives the requirements of an information security event management solution.
This GigaOm benchmark report aims to reveal how well vendor solutions perform in detecting attacks that leverage techniques recognized by the MITRE ATT&CK framework and the implications for real-world behavior and effectiveness. We explain the test methodology, reveal how well each vendor performed, and discuss the implications of their results. We also provide a hands-on assessment of each solution, focusing on ease of use and effective UI.
We tested four SIEM products in this report: Micro Focus ArcSight, Splunk Enterprise Security, IBM QRadar, and Microsoft Sentinel. Micro Focus ArcSight and Splunk Enterprise Security both excelled in detecting and logging the battery of attacks, each scoring 10 out of 10 in our series. IBM QRadar failed to catch many of the attacks in our tests and fell short of Micro Focus and Splunk in the quality of results presentation. Finally, we included in our evaluation Microsoft Sentinel, which at the time of this testing was equipped with a pre-release implementation of the MITRE ATT&CK framework. While we provide a hands-on assessment of the Sentinel product in this report, the tool did not produce usable results in our detection tests and therefore was not included in that portion of our evaluation.