No Undetectable Breaches

PaulM left an interesting comment on my post NORAD-Inspired Security Metrics:

...what if the enemy has a stealth plane that we cannot detect via radar, satellite, wind-speed variance, or any other deployed means? And what if your intel doesn't tell us that such a vehicle exists? Then we have potentially millions of airspace breaches every year and our outcome metrics are not helping.

I'm not disagreeing with you that outcome metrics are ideally better data than compliance metrics. However, outcome metrics are difficult to identify and collect data on, and it can be difficult to discern how accurate your metrics actually are.

At least with compliance metrics, we can determine how good we are at doing what it is we say that we do. It has little relevance to operational security, but it's easy and the auditors seem to like it.

For the case of a single breach, or even several breaches, it may be possible for them to happen and be completely undetectable. However, I categorically reject the notion that it is possible to suffer sustained, completely undetectable breaches and remain unaware of the damage. If you are not suffering any damage due to these breaches, then why are you even trying to deter, detect, and respond to them in the first place?

Let me put this in perspective by considering labels attached to classified information as designated by Executive Order 12356:

(a) National security information (hereinafter "classified information") shall be classified at one of the following three levels:

1. "Top Secret" shall be applied to information, the unauthorized disclosure of which reasonably could be expected to cause exceptionally grave damage to the national security.


2. "Secret" shall be applied to information, the unauthorized disclosure of which reasonably could be expected to cause serious damage to the national security.


3. "Confidential" shall be applied to information, the unauthorized disclosure of which reasonably could be expected to cause damage to the national security.

We want to protect the confidentiality of classified information to avoid the losses described above. What happens if we suffered sustained breaches (thefts) of Top Secret data? Are we not going to detect that our national security concerns are being hammered, since we are suffering "exceptionally grave damage"?

This is one way spies are unearthed. If your missions are constantly failing because the enemy seems to know your plans, then your suffering a breach you haven't detected it.

Finally, if you are suffering breaches and your input-based metrics aren't detecting them either, what good are they? Talk about a real waste of money. "It's easy and auditors seem to like it?" Good grief.