Exaggerated Insider Threats

I got a chance to listen to Adam Shostack's talk at ShmooCon. When I heard him slaughter my name my ears perked up. (It's "bate-lik".) :) I hadn't seen his slides (.pdf) until now, but I noticed he cited my post Of Course Insiders Cause Fewer Security Incidents where I questioned the preponderance of insider threats. I thought Adam's talk was good, although he really didn't support the title of his talk. It seemed more like "security breaches won't really hurt you," rather than breaches benefitting you. That's fine though.

When he mentioned my post he cited a new paper titled A Case of Mistaken Identity? News Accounts of Hacker and Organizational Responsibility for Compromised Digital Records, 1980–2006 by Phil Howard and Kris Erickson. Adam highlighted this excerpt

60 percent of the incidents involved organizational mismanagement

as a way to question my assertion that insiders account for fewer intrusions than outsiders.

At the outset let me repeat how my favorite Kennedy School of Government professor, Phil Zelikow, would address this issue. He would say, "That's an empirical question." Exactly -- if we had the right data we could know if insiders or outsiders cause more intrusions. I would argue that projects like the Month of 0wned Corporations give plenty of data supporting my external hypothesis, but let's take a look at what the Howard/Erickson paper actually says.

First, what are they studying?

Our list of reported incidents is limited to cases where one or more electronic personal records were compromised through negligence or theft... For this study, we look only at incidents of compromised records that are almost certainly illegal or negligent acts. For the purposes of this paper, we define electronic personal records as data containing privileged information about an individual that cannot be readily obtained through other public means.

So, they are studying disclosure of personal information. They are not analyzing theft of intellectual property like helicopter designs. They are not reviewing cases of fraud, like $10 million of routers and other gear shipped to Romania. They are not reviewing incidents where hosts became parts of botnets and the like. All of that activity would put weight in the external column. That's not included here.

Let's get back to that 60% figure. It sounds like my hypothesis is doomed, right?

Surprisingly, however, the proportion of incident reports involving hackers is smaller than the proportion of incidents involving organizational action or inaction. While 31 percent of the incidents reported clearly identify a hacker as the culprit, 60 percent of the incidents involve missing or stolen hardware, insider abuse or theft, administrative error, or accidentally exposing data online.

Now we see that the 60% figure includes several categories of "organizational action or inaction". Hmm, I wonder how big the insider abuse or theft figure is, since that to me sounds like the big, bad "insider threat." If we look at this site we can access the figures and tables for the report. Take a look at Figure 2. (It's too wide to print here.) The Insider Abuse or Theft figure accounts for 5% of the incident total while Stolen - Hacked accounts for 31%. Sit down, insider threat.

Wait, wait, insider threat devotees might say -- what about Missing or Stolen Hardware, which is responsible for 36% of incidents? I'll get to that.

The numbers I cited were for incidents. You would probably agree you care more about the number of records lost, since who cares if ten companies each lose one hundred records if an eleventh loses one hundred thousand.

If you look at the corresponding percentages for numbers of records lost (instead of number of incidents), Insider Abuse or Theft accounts for 0%, Missing or Stolen Hardware counts for 2% while Stolen - Hacked rings in at 91%. Why are we even debating this issue?

Wait again, insider fans will say. Why don't you listen when the authors exclude Axciom from the data? They must be an "outlier," so ignore them. And now ignore TJX, and... anyone else who skews the conclusion we're trying to reach.

The authors state:

Regardless of how the data is broken down, hackers never account for even half of the incidents or the volume of compromised records.

To quote Prof Zelikow again, "True, but irrelevant." I'll exclude Acxiom too by looking at previous periods. In 1980-1989 Stolen - Hacked accounts for 96% of records and 43% of incidents, while Unspecified accounts for the remaining 4% of records, along with 43% of incidents. Insider Abuse or Theft was 14% of incidents and zero records, meaning no breach. In 1990-1999 Stolen - Hacked accounts for 45% of incidents but the number of records is dwarfed by the number of records lost in Unspecified Breach.

In brief, this report defends the insider threat hypothesis only in name, and really only when you cloak it in "organizational ineptitude" rather than dedicated insiders out to do the company intentional harm.

I recommend reading the report to see if you find the same conclusions buried behind the numbers. It's entirely possible I'm missing something, but I don't see how this report diminishes the external threat.