ArchiveOODA Original

Cyber Sensemaking Part Two: Management Lessons Learned and Essential Actions

Having discussed overall thematics and threat objectives in Part One, we’ll shift to looking at the top management lessons and actions that can be taken to drive your risk management initiatives.

Avoid Expense in Depth

“Distractions are free, but they sure do cost a lot.” – Matt Devost

Years ago the Department of Defense had a cybersecurity strategy that expressed a concept of Defense in Depth as a layered, multi-faceted approach to security management.  In today’s corporate environment it seems that concept has morphed into what one analyst described as “expense in depth”. Expense in depth moves away from traditional concepts of managing risk towards an acquisitive model focused on buying and deploying the latest (not necessarily the greatest) technologies.

It is important to remember that risk management is a deliberate discipline and that cyber risk can be managed like other risks to your enterprise.  While the cybersecurity market enjoys watching a CISO’s budget grow by 50-100% year over year, that is not a sustainable model for any industry, especially one that tends to be viewed as a cost center a majority of the time.  Instead, executive management teams should be asking what vulnerabilities present consequential or catastrophic risks to the organization and develop prioritized remediation strategies to address them. As the execution of those strategies mature, expand the security posture improvements to account for additional risk reduction as financially feasible.

Don’t just spend money for the sake of spending money or walk the vendor floor at RSA or Black Hat with the intent of buying one product in every category.  Be deliberate and make tactical purchases against a strategic risk management plan.

Measure What Matters

“You get the behavior that you measure and reward.”  – Jack Welch

Too often in cybersecurity we focus on measuring the wrong things.  How many phishing campaigns did we detect? Did our vulnerability scans produce fewer vulnerabilities this month over last month?  While these might be useful metrics to track as part of a broader program, they are not true indicators of risk reduction for executive teams.

Instead, ask what metrics are important to managing your risk profile.  For example, did your time to detection on attacks go down? Are you able to detect lateral movement on your enterprise network?  How efficient is your response to an incident? What was the business impact cost of an incident?  

It is important to make sure you measure what matters and develop real risk management metrics that can inform the board and other senior leadership.

Understand What is Critical

“More things can happen than will.” – Peter Bernstein

Does your organization understand what systems and information are critical to operations?  If you imagine levels of criticality on a dart board with the center ring being the most critical systems, most organizations will say 50% of their systems belong in the bullseye.  In reality, that number is much smaller, usually by a magnitude of 10 and your real critical risk are aggregated in 5-10% of your systems. If you don’t differentiate between what is truly critical and what is not, you won’t be able to engage in realistic risk-based protection strategies.  

Your Attacker is Not a One or Zero

“Technology. It is the physical manifestation of the human will.” – Daniel Suarez

To view cybersecurity as only a technical field is to ignore the human dynamics of business and risk.  Cyber is as much a wet discipline as it is silicon one. Human and human behavior are targets of attack and you must focus your education and awareness strategy on true user behavior change.  I’ve always found it ironic that the same training software trying to convince me not to click on links is training me to engage in just that behavior (click next to continue!).

In that same vein, your attacker is not a one or a zero.  They are a living, breathing, human being with intent, goals, and intrinsic human behaviors.  Your threat models must account for the human dynamic in the threat and the human element in your defense.

Every Attacker in an Insider

“The map is not the territory.” -Alfred Korzybski

For years our cybersecurity lexicon depended on framing the issues in terms of physical defense analogies.  We had firewalls and perimeter and lines of defense. As long as you had a hard exterior shell it didn’t matter that your internal network was all soft and mushy.

As I’ve proven hundreds of times through red teams and as learned through dozens of incident response engagements, an adversary breach of your perimeter is inevitable.  How does your security strategy, incident response planning, and budget change if you make this one small assumption; that the attacker is already inside?

Increase Attacker Costs

“When you are wrestling for possession of a sword, the man with the handle always wins.” – Neal Stephenson

Just as you manage a defense budget, attackers also have resource constraints in the form of time, money, and detectability. Your cyber defense strategy should look to increase costs to the attacker to make you a less viable target (assuming they can achieve the same objective by attacking your industry peers), require them to use constrained resources (e.g. force use of a zero day vice known exploits), or reduce the time to reward ratio of a successful attack.

Some of the best research on this issue of maintaining parity between attack and defense has been conducted by Jason Healey at Columbia University and was featured in this OODA Loop article entitled “The Key to a Defensible Cyberspace – A Look at the Work of Jason Healey and the NY Cyber Task Force“.

In thinking through attack cost factors it is also important to recognize the value of deception and deception based technologies.   Deception technologies increase the cost to the attacker (they spend time on useless targeting), pollute the value stream (they’ve stolen false data instead of real data), and provide for early detection.  Some of the most creative incidents I’ve responded to over the past 25 years involved some element of deception as a defensive strategy so you should certainly be thinking through what value that approach might provide for your organization.

Red Team It

“Some people say preparation is everything, and for those who never execute, maybe it is.” Eric Haney

The best risk management programs in the world offer suffer from one common failure in that their implementation rarely matches their aspirations.  Test and evaluation is a critical element of any defensive strategy and red teaming is typically one of the highest value activities an organization can engage in.  I’ve written specifically about this in the “10 Red Teaming Lessons Learned Over 20 Years” and this HACKthink piece that focuses on different ways to engage in red teams.

Build Security into the Design Process

“We become what we behold.  We shape our tools, and thereafter our tools shape us.” – John Culkin

For far too long security has been an afterthought in the product design process.  While things have gotten better in recent years through product red teaming and DevSecOps approaches, we still have a long way to go.  Any new program or product should include your security stakeholders/advocates as part of the design process. Security should be a component that is built in from the start through deliberate inclusion.  Ask your teams how they are building security into the design process and articulate that management values approaches that think “security first” as opposed to “security last”.

As you look to deploy new technologies like machine learning and AI, build a security framework into your approach to avoid making the same mistakes we’ve been making over the past two decades of enterprise IT deployments.  For a specific discussion on security AI, please read this article entitled “Securing AI – Four Areas to Focus on Right Now“.

Learn from Failures

“However beautiful the strategy, you should occasionally look at the results.” – Winston Churchill

One of the best books I read in 2016 was called Black Box Thinking (Amazon link).  The book contrasted two industries and how they investigate, incorporate, and learn from mistakes over time in the medical and aviation fields.  I won’t give away the major thematic of the book, but the title focus on an airplane’s black box not a stethoscope.  

In security management, we also need to develop a process for learning from our failures and building those lessons learned into our security operations.  Ask your organization what they learned from the last breach or red team and how did we change our behavior, technology stack or assumptions as a result of it.

Don’t be Surprised by Disruptive Technologies

“Change has never happened this fast before, and it will never be this slow again” – Graeme Wood

Disruption is going to play a critical role in the cybersecurity domain over the next decade.  While some innovation will occur as organic developments within the field, your technologists should also be tracking adjacent domains to see how new technologies are disrupting old and new problems.

For example, can we solve a cybersecurity labor shortage with machine learning and AI?  The next five to ten years will answer that question for us and you should be positioned to exploit promising technologies in the future.

It is also important to remember that existing disciplines can be disrupted as well.  For example, what is happening in the field of automated red teaming (which drove our investment in Scythe) or can gamification be used in incident response (which drove our investment in 418 Intelligence).  What if we fuzzed applications while they were still source code as opposed to when we have compiled binaries (our investment in FuzzBuzz). The future holds lots of promising technologies and your organization should be watching closely and making sure your strategies and technology deployments are dynamic enough to incorporate them as the value proposition becomes clear.


It is always tough to distill decades of experience into digestible executive nuggets of information, but we hope that you found this article useful and that some of the points can be adopted within your organization.  As always, please feel free to follow-up directly with any questions.

Matt Devost

Matt Devost

Matthew G. Devost is the CEO & Co-Founder of OODA LLC. Matt is a technologist, entrepreneur, and international security expert specializing in counterterrorism, critical infrastructure protection, intelligence, risk management and cyber-security issues. Matt co-founded the cyber security consultancy FusionX from 2010-2017. Matt was President & CEO of the Terrorism Research Center/Total Intel from 1996-2009. For a full bio, please see