ArchiveDisruptive TechnologyOODA Original

The Rate of Learning in Threat Detection

By Rich Heimann, Chief AI Officer at Cybraics & Marvin Wheeler, CEO at Cybraics and Chief Innovation Officer at SilverSky

Historically, threat detection (e.g., rule-based intrusion detection, anti-virus systems, and threat intelligence feeds) has been reactive and involves flagging digital requests containing known signatures. These signatures are formalized post hoc, emanating from a compromise that has already happened and was then shared with others. Organizations have relied heavily on these tools—to their disadvantage. The figures below reflect the traditional threat detection paradigm of learning vicariously from peers and highlight how it is at a disadvantage against new or adaptive adversaries. There are some things worth remembering; certainly, past security events are one of them because mistakes are great teachers. The more lessons you can learn from others, the better. However, memorizing past security events experienced by someone else is passive and not a reliable way of learning about threats your organization faces.

• 205: The median number of days a threat is present before detection (2014).
• 322: The median number of days a zero-day attack is present before detection (2014).
• 70–90%: The percent of malware samples unique to an organization (2015).
• 3%: The overlap in the top 54 (six-month exposure) threat intelligence feeds (2015).
• 75%: The percent of attacks spread from victim zero to victim one within one day (2015).
• 1%: The percentage of IPs on threat intelligence feeds that last seven days (2015).
• 7 million: The number of exploited vulnerabilities that are unaccounted for in the top ten common vulnerabilities and exposures (CVE) databases (2015).

Not only is there little private benefit to be gained from a peer helping you, but information about an event that has already happened—and is unlikely to happen to you because you do not face all (or even most) of the same threats—is somewhat arbitrary. Moreover, without access to every threat intelligence feed, signature, and common vulnerability, the current value would be tough to achieve while remaining contradictory. The help offered by traditional defensive tools increases exposure (at least when everyone adopts every fixed defensive tool without the support of nontraditional tools like machine learning). Just as you can learn about adversaries from peers when tactics, techniques, and procedures are shared, adversaries can learn about you from your peers when defensive postures are shared. In other words, vicarious learning paradigms cut both ways. As a general rule, avoid creating advantages for enemies by giving them information about your defensive strategy.

While a database full of past events is better than an empty one, threat detection tools matter only insofar as they relate to threats your organization faces. Since there is no guarantee that traditional solutions will provide any protection, these solutions must serve a subordinate role to learning directly from adversaries using computational tools like statistical and machine learning to detect aberrant behavioral patterns. Behavioral detection is more active and robust than matching fixed digital requests containing known signatures. In this case, robust means improved generalization to unseen examples. Improved generalization increases the learning rate by lowering the false negatives rate, thus reducing the number of days a threat is present before detection from 205 days (322 for zero-day attacks) to something much more palatable.

Statistical and machine learning can be applied to behavioral detection because adversaries operate with incomplete information about your defenses. As an aside, insider threats are problematic because there is no incomplete information. Insiders have credentials and know where to find what they want. External adversaries, however, are scanning, phishing, acquiring credentials, escalating privileges, evading defenses, moving laterally, and collecting information. These pursuits provide an opportunity to learn from trace evidence left behind by adversaries, including their lack of knowledge, tradecraft or lack of tradecraft, successes, failures, errors, oversights, and accidents.

Interestingly, malicious tradecraft in the early stages of so-called reconnaissance is what traditional solutions ignore, despite the information asymmetry between attacker and defender strongly favoring the defender. As organizations become more dependent on communication networks, behavioral-based threat detection will become more critical because this is where you can learn the most about the threats you face, which is vital for an early warning and response system. You must actively pursue nefarious activity using computational tools rather than passively learn from peers about threats you may not face. The worst thing you can do with a cat is be a mouse.

OODA Loop provides actionable intelligence, analysis, and insight on global security, technology, and business issues. Our members are global leaders, technologists, and intelligence and security professionals looking to inform their decision making process to understand and navigate global risks and opportunities.

Members get all site content plus additional Member benefits such as participation in our Monthly meetings, exclusive OODA Unlocked Discounts, discounted training and conference attendance, job opportunities, our Weekly Research Report, and other great benefits. Join Here.

Related Reading:

Explore OODA Research and Analysis

Use OODA Loop to improve your decision making in any competitive endeavor. Explore OODA Loop

Decision Intelligence

The greatest determinant of your success will be the quality of your decisions. We examine frameworks for understanding and reducing risk while enabling opportunities. Topics include Black Swans, Gray Rhinos, Foresight, Strategy, Stratigames, Business Intelligence and Intelligent Enterprises. Leadership in the modern age is also a key topic in this domain. Explore Decision Intelligence

Disruptive/Exponential Technology

We track the rapidly changing world of technology with a focus on what leaders need to know to improve decision-making. The future of tech is being created now and we provide insights that enable optimized action based on the future of tech. We provide deep insights into Artificial Intelligence, Machine Learning, Cloud Computing, Quantum Computing, Security Technology, Space Technology. Explore Disruptive/Exponential Tech

Security and Resiliency

Security and resiliency topics include geopolitical and cyber risk, cyber conflict, cyber diplomacy, cybersecurity, nation state conflict, non-nation state conflict, global health, international crime, supply chain and terrorism. Explore Security and Resiliency


The OODA community includes a broad group of decision-makers, analysts, entrepreneurs, government leaders and tech creators. Interact with and learn from your peers via online monthly meetings, OODA Salons, the OODAcast, in-person conferences and an online forum. For the most sensitive discussions interact with executive leaders via a closed Wickr channel. The community also has access to a member only video library. Explore The OODA Community

Rich Heimann

Rich Heimann

Rich Heimann is Chief AI Officer at Cybraics Inc, a fully managed cybersecurity company. Founded in 2014, Cybraics operationalized many years of cybersecurity and machine learning research conducted at the Defense Advanced Research Projects Agency. Rich is also the author of Doing AI, a book exploring what AI is, is not, what others want AI to become, what you need solutions to be, and how to approach problem-solving. Please find out more about his book here.