Rabu, 04 April 2012

Salvaging Poorly Worded Statistics

Today I joined a panel held at FOSE chaired by Mischel Kwon and featuring Amit Yoran. One of the attendees asked the following:

At another session I heard that "80% of all breaches are preventable." What do you think about that?

My brief answer explained why that statement isn't very useful. In this post I'll explain why.

The first problem is the "80%." 80% of what? What is the sample set? Are the victims in the retail and hospitality sectors or the telecommunications and aerospace industries? Speaking in general terms, different sorts of organizations are at different levels of maturity, capability, and resourcefulness when it comes to digital security.

In the spirit of salvaging this poorly worded statistic, let's assume (rightly or wrongly) that the sample set involves the retail and hospitality sectors.

The second problem is the term "breach." What is a breach? Is it the compromise of a single computer? (What's compromise? Does it mean executing malicious code, or login via stolen credentials, or...?) What is the duration of the incident? There are dozens of questions that could be asked here.

To salvage this part, let's assume "breach" means "an incident involving execution of unauthorized code by an unauthorized intruder" on a single computer.

The third problem is the word "preventable." "Prevention" as a concept is becoming less useful by the second. Think about how an intruder might try to execute malicious code against a victim. Imagine a fully automated attack that happens when a victim visits a malicious Web site. An exploit kit could throw a dozen or more exploits against a browser and applications until one works. Are they all non-zero day, or are some zero day? Again, many questions beckon.

To salvage the end of the original statement, let's translate "preventable" into "exploitation of a vulnerability for which a patch had been publicly available for at least seven days."

Our new statement now reads something like "In the retail and hospitality sectors, 80% of the incidents where an unauthorized intruder successfully executed unauthorized code on a single computer exploited a vulnerability for which a patch had been publicly available for at least seven days."

Isn't that catchy! That's why we heard shortcuts like the original statement, which are basically worthless. Unfortunately, they end up driving listeners into poor conceptual and operational models.

The wordy but accurate statement says nothing about preventability, which is key. The reason is that a determined adversary, when confronted by a fully patched target, may decide to escalate to using a zero-day or other technique for which patches are irrelevant.

0 komentar:

Posting Komentar