Mandiant logo

Mandiant Security Effectiveness Report:
Q&A with the Experts

The recently released Mandiant Security Effectiveness Report 2020: A Deep Dive into Cyber Reality, reveals a serious gap between assumptions and reality when it comes to security performance.


According to research highlighted in the report, a majority of organizations believe their security controls operate and protect assets as they’re supposed to, when in fact they have already experienced a breach without realizing it.

For this article, we spoke with two vice presidents from Mandiant Security Validation, Tracey Moon (Marketing) and General Earl Matthews (Strategy), to find out how the data was compiled, who the report is for and what some of the key stats mean, as well as what security leaders should do to improve performance going forward.

Can you share some of the findings in the Security Effectiveness Report and what they mean?

Tracey Moon: At a high level, the report validates what the Mandiant Security Validation team has known for years, which is that much of cyber security is built on assumptions. This is illustrated through several startling statistics.

Across all environments tested, we discovered that 53% of attacks conducted were successful in penetrating the security infrastructure without the organization’s knowledge.

That’s a pretty high number! My guess is that CIOs and CISOs don’t invest in security controls with the expectation that they only work less than half of the time.

Of the remaining 47% of attacks that did NOT succeed, only one-quarter were actually detected, and roughly one-third were prevented. Also, the most alarming statistic is that alerts were sent to security operations for only 9% of the attacks. What this shows is that even when security teams use central SIEM, SOAR and analysis platforms, they still don’t have the visibility they need into the malicious activity that is continuously targeting their networks, applications and devices.

The report makes the case that companies must validate security through ongoing testing and measurement of security effectiveness against the evolving threat landscape. The combination of industry-leading Mandiant threat intelligence and frontline expertise with Mandiant Security Validation empowers organizations to ensure their cyber defenses are operating as expected and continuing to deliver value.

Man using laptop

How was the data revealed in the report obtained?

General Earl Matthews: The Security Effectiveness Report offers data-driven analysis and reporting of key security performance metrics, looking at the full attack lifecycle across several global industries. Through thousands of real-world tests performed by the Mandiant Security Validation team in more than 100 enterprise environments, the report offers evidence that security controls lack the effectiveness that organizations expect – across network, email, endpoint and cloud-based security controls.

Who is the Security Effectiveness Report intended for?

Tracey Moon: The Security Effectiveness Report was developed to help any individual concerned with the effectiveness of their organization’s security infrastructure better understand why controls aren’t performing as they should, what the potential risks of this gap in performance are, and what they should do about it. Concern about security effectiveness no longer resides solely with CISOs and their teams given that measurement of security effectiveness is critical to proving value of security and IT investments for boards of directors and corporate executives.

Yet while more and more organizations recognize cyber risk as a business problem, they continue to keep it under the management of IT. So the report not only demonstrates the gap between assumption and reality when it comes to security effectiveness, but also shines the light on the misalignment between business leadership and IT teams, which further exacerbates the problem. The report is intended to give security AND business leaders a better understanding of why ongoing data-driven effectiveness measurement is critical to making important business decisions.

What are some other statistics highlighted in the report, and what do they mean?

General Earl Matthews: The report takes a deeper look at some important areas of security infrastructure based on Mandiant Security Validation testing:


These findings and likely underlying causes are detailed in the report, and show why it’s crucial for companies to perform ongoing security validation to combat the alarming reality these statistics reveal.

What are the fundamental components of security validation?

General Earl Matthews: Security validation can quantify the actual effectiveness of your security controls, provide continuous monitoring to take into account any unexpected changes or environmental drift in your underlying infrastructure that may impact the performance of your security controls, and provide confidence that you are proactively prepared for the latest attacks and adversaries. Fundamental components include:

  • Adversary coverage
  • Validation automation and outcomes
  • Business metrics
  • Enterprise readiness

Read all the research for yourself. Download the Mandiant Security Effectiveness Report 2020.