Someone Please Explain Threats to Microsoft

It's 2007 and some people still do not know the difference between a threat and a vulnerability. I know these are just the sorts of posts that make me all sorts of new friends, but nothing I say will change their minds anyway. To wit, Threat Modeling Again, Threat Modeling Rules of Thumb:

As you go about filling in the threat model threat list, it’s important to consider the consequences of entering threats and mitigations. While it can be easy to find threats, it is important to realize that all threats have real-world consequences for the development team.

At the end of the day, this process is about ensuring that our customer’s machines aren’t compromised. When we’re deciding which threats need mitigation, we concentrate our efforts on those where the attacker can cause real damage.

When we’re threat modeling, we should ensure that we’ve identified as many of the potential threats as possible (even if you think they’re trivial). At a minimum, the threats we list that we chose to ignore will remain in the document to provide guidance for the future.

Replace every single instance of "threat" in that section with "vulnerability" and the wording will make sense.

Not using the term "threat" properly is a hallmark of Microsoft publications, as mentioned in Preview: The Security Development Lifecycle. I said this in my review of Writing Secure Code, 2nd Ed:

The major problem with WSC2E, often shared by Microsoft titles, is the misuse of terms like "threat" and "risk." Unfortunately, the implied meanings of these terms varies depending on Microsoft's context, which is evidence the authors are using the words improperly. It also makes it difficult for me to provide simple substitution rules. Sometimes Microsoft uses "threat" when they really mean "vulnerability." For example, p 94 says "I always assume that a threat will be taken advantage of." Attackers don't take advantage of threats; they ARE threats. Attackers take advantage of vulnerabilities.

Sometimes Microsoft uses terms properly, like the discussion of denial of service as an "attack" in ch 17. Unfortunately, Microsoft's mislabeled STRIDE model supposedly outlines "threats" like "Denial of service." Argh -- STRIDE is just an inverted CIA AAA model, where STRIDE elements are attacks, not "threats." Microsoft also sometimes says "threat" when they mean "risk." The two are not synonyms. Consider this from p 87: "the only viable software solution is to reduce the overall threat probability or risk to an acceptable level, and that is the ultimate goal of 'threat analysis.'" Here we see confusing threat and risk, and calling what is really risk analysis a "threat analysis." Finally, whenever you read "threat trees," think "attack trees" -- and remember Bruce Schneier worked hard on these but is apparently ignored by Microsoft.

These sentiments reappeared in my review of Security Development Lifecycle: Microsoft continues its pattern of misusing terms like "threat" that started with "Threat Modeling" and WSC2E. SDL demonstrates some movement on the part of the book's authors towards more acceptable usage, however. Material previously discussed in a "Threat Modeling" chapter in WSC2E now appears in a chapter called "Risk Analysis" (ch 9) -- but within the chapter, the terms are mostly still corrupted. Many times Microsoft misuses the term risk too. For example, p 94 says "The Security Risk Assessment is used to determine the system's level of vulnerability to attack." If you're making that decision, it's a vulnerability assessment; when you incorporate threat and asset value calculations with vulnerabilities, that's true risk assessment.

The authors try to deflect what I expect was criticism of their term misuse in previous books. On p 102 they say "The meaning of the word threat is much debated. In this book, a threat is defined as an attacker's objective." The problem with this definition is that it exposes the problems with their terminology. The authors make me cringe when I read phrases like "threats to the system ranked by risk" (p 103) or "spoofing threats risk ranking." On p 104, they are really talking about vulnerabilities when they write "All threats are uncovered through the analysis process." The one time they do use threat properly, it shows their definition is nonsensical: "consider the insider-threat scenario -- should your product protect against attackers who work for your company?" If you recognize that a threat is a party with the capabilities and intentions to exploit a vulnerability in an asset, then Microsoft is describing insiders appropriately -- but not as "an attacker's objective."

Don't get me wrong -- there's a lot to like about SDL. I gave the book four stars, and I think it would be good to read it. I fear, though, that this is another book distributed to Microsoft developers and managers riddled with sometimes confusing or outright wrong ways to think about security. This produces lasting problems that degrade the community's ability to discuss and solve software security problems.

No one is going to take us seriously until we use the right terms. Argh.

Read More

Final Question on FAIR

I'd hoped to not have to say anything else on FAIR, but I've decided to ask one final question. If I can't get a straight answer to this question I'm giving up on the discussion. So, here it is.

In Thoughts on FAIR I walked through the section Analyzing a Simple Scenario. Steps 3, 4, 5, 8, and 9 [Estimate the probable Threat Event Frequency (TEF), Estimate the Threat Capability (TCap), Estimate Control strength (CS), Estimate worst-case loss, and Estimate probable loss, respectively] each require the user to "Estimate."

Do these Estimates matter to the output of the model?

If the answer is yes, then those answers are important and should be grounded in reality -- not opinions. If FAIR proponents agree with this, then we have been debating for days for no real reason. That would make me happy.

If the answer is no, then what is so magical about FAIR that garbage in does not produce garbage out?

Read More

Is Digital Security "Risk" a Knightian Uncertainty?

I've subscribed to the Economist for over ten years, and it's been worth every penny. Today I noticed the following in an article called The Long and Short of It:

The second paper suggests that traders face “Knightian uncertainty”, or risks that cannot be measured.

Hmm, what is this "Knightian uncertainty"? I found the following excerpt from Risk, Uncertainty and Expected Utility:

Much has been made of Frank H. Knight's (1921: p.20, Ch.7) famous distinction between "risk" and "uncertainty". In Knight's interpretation, "risk" refers to situations where the decision-maker can assign mathematical probabilities to the randomness which he is faced with. In contrast, Knight's "uncertainty" refers to situations when this randomness "cannot" be expressed in terms of specific mathematical probabilities. As John Maynard Keynes was later to express it:

"By `uncertain' knowledge, let me explain, I do not mean merely to distinguish what is known for certain from what is only probable. The game of roulette is not subject, in this sense, to uncertainty...The sense in which I am using the term is that in which the prospect of a European war is uncertain, or the price of copper and the rate of interest twenty years hence... About these matters there is no scientific basis on which to form any calculable probability whatever. We simply do not know." (J.M. Keynes, 1937)

Nonetheless, many economists dispute this distinction, arguing that Knightian risk and uncertainty are one and the same thing. For instance, they argue that in Knightian uncertainty, the problem is that the agent does not assign probabilities, and not that she actually cannot, i.e. that uncertainty is really an epistemological and not an ontological problem, a problem of "knowledge" of the relevant probabilities, not of their "existence".

Going in the other direction, some economists argue that there are actually no probabilities out there to be "known" because probabilities are really only "beliefs". In other words, probabilities are merely subjectively-assigned expressions of beliefs and have no necessary connection to the true randomness of the world (if it is random at all!).

Nonetheless, some economists, particularly Post Keynesians such as G.L.S. Shackle (1949, 1961, 1979) and Paul Davidson (1982, 1991) have argued that Knight's distinction is crucial. In particular, they argue that Knightian "uncertainty" may be the only relevant form of randomness for economics - especially when that is tied up with the issue of time and information.

In contrast, situations of Knightian "risk" are only possible in some very contrived and controlled scenarios when the alternatives are clear and experiments can conceivably be repeated -- such as in established gambling halls. Knightian risk, they argue, has no connection to the murkier randomness of the "real world" that economic decision-makers usually face: where the situation is usually a unique and unprecedented one and the alternatives are not really all known or understood. In these situations, mathematical probability assignments usually cannot be made. Thus, decision rules in the face of uncertainty ought to be considered different from conventional expected utility. (emphasis added)

The Wikipedia entry on Uncertainty is also interesting.

That is really fascinating. It sounds like a school of thought believes the real world may be too complex too model. It also sounds like stepping foot into the world of appreciating uncertainty is a huge undertaking, given the amount of prior research.

Read More

Economist on Models

I intended to stay quiet on risk models for a while, but I read the following Economist articles and wanted to note them here for future reference.

From "Statistics and climatology: Gambling on tomorrow":

Climate models have lots of parameters that are represented by numbers... The particular range of values chosen for a parameter is an example of a Bayesian prior assumption, since it is derived from actual experience of how the climate behaves — and may thus be modified in the light of experience. But the way you pick the individual values to plug into the model can cause trouble...

Climate models have hundreds of parameters that might somehow be related in this sort of way. To be sure you are seeing valid results rather than artefacts of the models, you need to take account of all the ways that can happen.

That logistical nightmare is only now being addressed, and its practical consequences have yet to be worked out. But because of their philosophical training in the rigours of Pascal's method, the Bayesian bolt-on does not come easily to scientists. As the old saw has it, garbage in, garbage out. The difficulty comes when you do not know what garbage looks like. (emphasis added)

This is only a subset of the argument in the article. Note that this piece includes the line "derived from actual experience of how the climate behaves." I do not see the "Bayesian prior assumption[s]" so often cited elsewhere to even be derived from experience, if you take experience to mean something grounded in recent historical evidence instead of opinion. This willingness to rely on arbitrary inputs only aggravates the problems cited by the Economist.

The accompanying piece "Modelling the climate: Tomorrow and tomorrow" further explains some of the problems I've been describing previously.

While some argue about the finer philosophical points of how to improve models of the climate (see article), others just get on with it. Doug Smith and his colleagues at the Hadley Centre, in Exeter, England, are in the second camp and they seem to have stumbled on to what seems, in retrospect, a surprisingly obvious way of doing so. This is to start the model from observed reality.

Until now, when climate modellers began to run one of their models on a computer, they would “seed” it by feeding in a plausible, but invented, set of values for its parameters. Which sets of invented parameter-values to use is a matter of debate. But Dr Smith thought it might not be a bad idea to start, for a change, with sets that had really happened...

[T]he use of such real starting data made a huge improvement to the accuracy of the results. It reproduced what had happened over the courses of the decades in question as much as 50% more accurately than the results of runs based on arbitrary starting conditions.

Hindcasting, as this technique is known, is a recognised way of testing models. The proof of the pudding, though, is in the forecasting, so Dr Smith plugged in the data from two ten-day periods in 2005 (one in March and one in June), pressed the start button and crossed his fingers. (emphasis added)

This is exactly what I've been saying. Use "observed reality" and not "plausible, but invented" opinions or "arbitrary starting conditions" and I'll be happy to see what the model produces.

Read More

More Thoughts on FAIR

My post Thoughts on FAIR has attracted some attention, but as often the case some readers choose to obscure my point by overlaying their own assumptions. In this post I will try to explain my problems with FAIR in as simplistic a manner as possible.

Imagine if someone proposed the following model for assessing force: F=ma

(Yes, this is Newton's Second Law, and yes, I am using words like "model" and "assess" to reflect the risk assessment modeling problem.)

I could see two problems with using this model to assess force.

1. Reality check: The model does not reflect reality. In other words, an accurate measurement of mass times an accurate measurement of acceleration does not result in an accurate measurement of force.


2. Input check: To accurately measure force, the values for m and a must not be arbitrary. Otherwise, the value for F is arbitrary.

With respect to FAIR, I make the following judgments.

1. Reality check: The jury is out on whether FAIR reflects reality. It certainly might. It might not.


2. Input check: I have not seen any evidence that FAIR expects or requires anything other than arbitrary inputs. Arbitrary inputs to a model that passes the reality check does not produce anything valuable as far as I am concerned.
   
   If you personally like feeding your own opinions into a model to see what comes out the other end, have at it. It's nice to play around by making assumptions, seeing the result, and then altering the inputs to suit whatever output you really wanted to see.

One of the previous post commenters mentioned the book Uncertainty, which looks fascinating. If you read the excerpt you'll notice this line:

In the early 1970s what was then the U.S. Atomic Energy Commission (AEC) asked Norman C. Rasmussen, a professor of nuclear engineering at the Massachusetts Institute of Technology, to undertake a quantitative study of the safety of light-water reactors...

Rasmussen assembled a team of roughly sixty people, who undertook to identify and formally describe, in terms of event trees, the various scenarios they believed might lead to major accidents in each of the two reactors studied. Fault trees were developed to estimate the probabilities of the various events.

A combination of historical evidence from the nuclear and other industries, together with expert judgment, were used to construct the probability estimates, most of which were taken to be log-normally distributed. (emphasis added)

I've seen no commitment to including real evidence in FAIR, and I submit that those who lack evidence will have their so-called "expert judgment" fail due to soccer-goal security. Therefore, they possess neither historical evidence nor really expert judgment. So, even if FAIR meets my reality check, the results are only a feel-good exercise.

So how can I defend the use of this model?

Risk = Vulnerability X Threat X Impact (or Cost)

As I've said before, it is useful for showing the effects on Risk if you change one of the factors, ceteris paribus. This assumes that the model passes the reality check, which I believe it does.

I am not trying to calculate absolute values for Risk when I cite this equation. I am trying to conceptually show how Risk decreases when Threat decreases (ceteris paribus), or how Risk increases as Vulnerability increases (ceteris paribus), and so on.

You could take the same approach with F=ma if you were trying to explain to someone how it would hurt more to be struck by an object whose mass is larger than another object, assuming constant acceleration for each event. I am not trying to calculate F in such a case; I'm only using the model to describe the relationship between the components.

Read More

Thoughts on FAIR

You knew I had risk on my mind given my recent post Economist on the Peril of Models. The fact is I just flew to Chicago to teach my last Network Security Operations class, so I took some time to read the Risk Management Insight white paper An Introduction to Factor Analysis of Information Risk (FAIR). I needed to respond to Risk Assessment Is Not Guesswork, so I figured reading the whole FAIR document was a good start. I said in Brothers in Risk that I liked RMI's attempts to bring standardized terms to the profession, so I hope they approach this post with an open mind.

I have some macro issues with FAIR as well as some micro issues. Let me start with the macro issue by asking you a question:

Does breaking down a large problem into small problems, the solutions to which rely upon making guesses, result in solving the large problem more accurately?

If you answer yes, you will like FAIR. If you answer no, you will not like FAIR.

FAIR defines risk as

Risk - the probable frequency and probable magnitude of future loss

That reminded me of

Annual Loss Expectancy (ALE) = Annualized Rate of Occurrence (ARO) X Single Loss Expectancy (SLE)

If you don't agree remove the "annual" terms from the second definition or add them to the FAIR definition.

I have always preferred this equation

Risk = Vulnerability X Threat X Impact (or Cost)

because it is useful for showing the effects on risk if you change one of the factors, ceteris paribus. (Ok, I threw the Latin in there as homage to one of my economics instructors.)

If you consider frequency when estimating threat activity and include countermeasures as a component of vulnerability, you'll notice that Threat X Vulnerability starts looking like ARO. Impact (or Cost) is practically the same as SLE, so the two equations are similar.

FAIR turns its definition into the following.



If you care to click on that diagram, you'll see many small elements that need to be estimated. Specifically, you can follow the Basic Risk Assessment Guide to see these are the steps.

• Stage 1. Identify scenario components




• 1. Identify the asset at risk


• 2. Identify the threat community under consideration




• Stage 2. Evaluate Loss Event Frequency (LEF)




• 3. Estimate the probable Threat Event Frequency (TEF)


• 4. Estimate the Threat Capability (TCap)


• 5. Estimate Control strength (CS)


• 6. Derive Vulnerability (Vuln)


• 7. Derive Loss Event Frequency (LEF)




• Stage 3. Evaluate Probable Loss Magnitude (PLM)




• 8. Estimate worst-case loss


• 9. Estimate probable loss




• Stage 4. Derive and articulate Risk




• 10. Derive and articulate Risk

The problem with FAIR is that in every place you see the word "Estimate" you can substitute "Make a guess that's not backed by any objective measurement and which could be challenged by anyone with a different agenda." Because all the derived values are based on those estimates, your assessment of FAIR depends on the answer to the question I asked at the start of this post.

Let's see how this process stands up to some simple scrutiny by reviewing FAIR's Analyzing a Simple Scenario.

A Human Resources (HR) executive within a large bank has his username and password written on a sticky-note stuck to his computer monitor. These authentication credentials allow him to log onto the network and access the HR applications he’s entitled to use...

1. Identify the Asset at Risk: In this case, however, we’ll focus on the credentials, recognizing that their value is inherited from the assets they’re intended to protect.

We start with a physical security risk case. This simplifies the process considerably and actually gives FAIR the best chance it has to reflect reality. Why is that? The answer is that the physical world changes more slowly than the digital world. We don't have to worry about having solid walls being penetrated by a mutant from the X-Men movies or from the state of the credentials suddenly being altered by a patch or configuration change.

Identify the Threat Community: If we examine the nature of the organization (e.g., the industry it’s in, etc.), and the conditions surrounding the asset (e.g., an HR executive’s office), we can begin to parse the overall threat population into communities that might reasonably apply... For this example, let’s focus on the cleaning crew.

That's convenient. The document lists six potential threat communities but decides to only analyze one. Simplification sure makes it easier to proceed with this analysis. It also means the result is so narrowly targeted to be almost worthless, unless we decide to repeat this process for the rest of the threat communities. And this is still only looking at a sticky note.

3. Estimate the probable Threat Event Frequency (TEF): Many people demand reams of hard data before they’re comfortable estimating attack frequency. Unfortunately, because we don’t have much (if any) really useful or credible data for many scenarios, TEF is often ignored altogether. So, in the absence of hard data, what’s left? One answer is to use a qualitative scale, such as Low, Medium, or High.

And, while there’s nothing inherently wrong with a qualitative approach in many circumstances, a quantitative approach provides better clarity and is more useful to most decision-makers – even if it’s imprecise.

For example, I may not have years of empirical data documenting how frequently cleaning crew employees abuse usernames and passwords on sticky-notes, but I can make a reasonable estimate within a set of ranges.

Recognizing that cleaning crews are generally comprised of honest people, that an HR executive’s credentials typically would not be viewed or recognized as especially valuable to them, and that the perceived risk associated with illicit use might be high, then it seems reasonable to estimate a Low TEF using the table below...

Is it possible for a cleaning crew to have an employee with motive, sufficient computing experience to recognize the potential value of these credentials, and with a high enough risk tolerance to try their hand at illicit use? Absolutely! Does it happen? Undoubtedly. Might such a person be on the crew that cleans this office? Sure – it’s possible. Nonetheless, the probable frequency is relatively low. (emphasis added)

Says who? Has the person making this assessment done any research to determine if inflitrating cleaning crews is a technique used by economic adversaries? If yes, how often does that happen? What is the nature of the crew cleaning this office? Do they perform background checks? Have they been infiltrated before? Are they owned by a competitor? Figuring all of that out is too hard. Let's just supply guess #1: "low."

4. Estimate the Threat Capability (Tcap): Tcap refers to the threat agent’s skill (knowledge & experience) and resources (time & materials) that can be brought to bear against the asset... In this case, all we’re talking about is estimating the skill (in this case, reading ability) and resources (time) the average member of this threat community can use against a password written on a sticky note. It’s reasonable to rate the cleaning crew Tcap as Medium, as compared to the overall threat population.

Why is that? Why not "low" again? These are janitors we're discussing. Guess #2.

5. Estimate the Control Strength (CS): Control strength has to do with an asset’s ability to resist compromise. In our scenario, because the credentials are in plain sight and in plain text, the CS is Very Low. If they were written down, but encrypted, the CS would be different – probably much higher.

It is easy to accept guess #3 because we are dealing with a physical security scenario. It's simple for any person to understand that a sticky note in plain site has zero controls applied against it, so the (nonexistent) "controls" are worthless. But what about that new Web application firewall? Or you anti-virus software? Or any other technical control? Good luck assessing their effectiveness in the face of attacks that evolve on a weekly basis.

6. Derive Vulnerability (Vuln)

This value is derived using a chart that balances Tcap vs Control Strength. Since it is based on two guesses, one could decide if it is more or less accurate than estimated the vulnerability directly.

7. Derive Loss Event Frequency (LEF)

This value is derived using a chart that balances TEF vs Vulnerability. We derived vulnerability in the previous step and estimated TEF in step 3.

8. Estimate worst-case loss: Within this scenario, three potential threat actions stand out as having significant loss potential – misuse, disclosure, and destruction... For this exercise, we’ll select disclosure as our worst-case threat action.

This step considers Productivity, Response, Replacement, Fine/Judgments, Competitve Advantage, and Reputation, with Threat Actions including Access, Modification, Disclosure, and Denial of Access. Enter guess #4.

9. Estimate probable loss magnitude (PLM): The first step in estimating PLM is to determine which threat action is most likely. Remember; actions are driven by motive, and the most common motive for illicit action is financial gain. Given this threat community, the type of asset (personal information), and the available threat actions, it’s reasonable to select Misuse as the most likely action – e.g., for identity theft. Our next step is to estimate the most likely loss magnitude resulting from Misuse for each loss form.

Again, says who? Was identity theft chosen because it's popular in the news? My choice for guess #5 could be something completely different.

10. Derive and Articulate Risk: [R]isk is simply derived from LEF and PLM. The question is whether to articulate risk qualitatively using a matrix like the one below, or articulate risk as LEF, PLM, and worst-case.

The final risk rating is another derived value, based on previous estimates.

The FAIR author tries to head off critiques like this blog with the following section:

It’s natural, though, for people to accept change at different speeds. Some of us hold our beliefs very firmly, and it can be difficult and uncomfortable to adopt a new approach. Ultimately, not everyone is going to agree with the principles or methods that underlie FAIR. A few have called it nonsense. Others appear to feel threatened by it.

Apparently I'm resistant to "change" and "threatened" because I firmly hold on to "beliefs." I'm afraid that is what I will have to do when frameworks like this are founded upon someone's opinion at each stage of the decision-making process.

The FAIR document continues:

Their concerns tend to revolve around one or more of the following issues:

The absence of hard data. There’s no question that an abundance of good data would be useful. Unfortunately, that’s not our current reality. Consequently, we need to find another way to approach the problem, and FAIR is one solution.

I think I just read that the author admits FAIR is not based on "good data," and since we don't have data, we should just "find another way," like FAIR.

The lack of precision. Here again, precision is nice when it’s achievable, but it’s not realistic within this problem space. Reality is just too complex... FAIR represents an attempt to gain far better accuracy, while recognizing that the fundamental nature of the problem doesn’t allow for a high degree of precision.

The author admits that FAIR is not precise. How can it even be accurate when the derived values are all based on subjective estimates anyway?

Some people just don’t like change – particularly change as profound as this represents.

I fail to see why FAIR is considered profound. Is the answer because the process has been broken into five estimates, from which several other values are derived? Why is this any better than articles like How to Conduct a Risk Analysis or Risk Analysis Tools: A Primer or Risk Assessment and Threat Identification?

I'm sure this isn't the last word on this issue, but I need to rest before teaching tomorrow. Thank you for staying with me if you read the whole post. Obviously if I'm not a fan of FAIR I should propose an alternative. In Risk-Based Security is the Emperor's New Clothes I cited Donn Parker, who is probably the devil to FAIR advocates. If the question is how to make security decisions by assessing digital risk, I will put together thoughts on that for a post (hopefully this week).

Incidentally, the fact that I am not a fan of FAIR doesn't mean I think the authors have wasted their time. I appreciate their attempt to bring rigor to this process. I also think the questions they ask and the elements they consider are important. However, I think the ability to insert whatever value one likes into the five estimations fatally wounds the process.

This is the bottom line for me: FAIR advocates claim their output is superior due to their framework. How can a framework that relies on arbitrary inputs produce non-arbitrary output? And what makes FAIR so valuable anyway -- has the result been tested against any other methods?

Read More

Enemy-Centric vs Population-Centric Security

Gunnar Peterson pointed me to a great blog post he wrote called Protect the Transaction. He quotes Dave Kilcullen's post Two Schools of Classical CounterInsurgency, which discusses the difference between “enemy-centric” and “population-centric” counter-insurgency operations.

I consider two responses to these posts. First, when monitoring, you can take a threat-centric or an asset-centric approach to monitoring insider threats. This is especially true when monitoring inside an organization. As I teach in my Network Security Operations class, threat-centric monitoring places sensors closer to the suspected intruders (rogue sys admins, curious call center workers, etc.) while asset-centric monitoring places sensors closer to valuable resources (source code repositories, payroll servers, etc.) Sometimes you can follow both approaches, but that usually ends up in a "monitor everywhere" style that can be cost- and operationally-prohibitive. Keep in mind that defenses are (or should be) collapsing around the item of value, which Gunnar calls the transaction. He and I would agree that data is the key resource, so resistance, detection, and response should focus on that element.

Second, in terms of threats and assets in general (i.e., "enemies" and "populations"), we as enterprise defenders can really only influence the asset or "population" variable. We address that aspect through design, architecture, secure coding, countermeasures, and so on. Only law enforcement or the military can address threats or "enemies" by prosecuting or eliminating them.

Read More

Intel Premier IT Security Graphic

The image at left is from the first issue of an Intel marketing magazine called Premier IT. I like it because it shows many of the terms I try to describe in this blog, in relationship to each other. In English, the graphic says something like the following:

Threats exploit vulnerabilities, thereby exposing assets to a loss of confidentiality/integrity/availability, causing business impact.

I disagree that business impact is mitigated by controls. I think those terms were connected to make a pretty cyclical diagram. I would also say that controls mitigate attacks (exploits) by threats, not the threats themselves. Imprisonment mitigates threats.

The next diagram shows Intel emphasizes Policy at the base, followed by Training and Education, then Technology and Testing, and finally Monitoring and Enforcement. I think the Training and Education piece is marginally effective at best, at least for the general user population. It's tough enough for security pros to keep up with the latest attacks. It's impossible for general users. The school of hard knocks (i.e., experience) is doing a better job teaching the general user population not to trust anything online. I like the recommendation for Continuous monitoring for attacks and policy violations.

The last diagram positions most of the components of digital security within context. It includes Governance and Personnel, Physical Security, Network Security, Platform Security, Application Security, Storage Security, and File and Data Security. I like this image because it makes me question what aspects of this environment I understand and can personally implement.

Read More

Brothers in Risk

I write about risk, threat, and other security definitions fairly regularly. Lo and behold I just read a post by someone else who shares my approach. This is a must read. How did you react to the story?

A second brother in risk is Gunnar Peterson, who writes in part:

When security teams conflate threats and vulnerabilities, the result is confusion. Instead efforts dealing with threats... and vulnerabilities... should be separately optimized, besides both being part of "security"; they don't have that much in common.

Oh bravo, especially the old school link to Dan Geer which I should read again.

Read More

Risk-Based Security is the Emperor's New Clothes

Donn Parker published an excellent article in the latest issue of The ISSA Journal titled Making the Case for Replacing Risk-Based Security. This article carried a curious disclaimer I had not seen in other articles:

This article contains the opinions of the author, which are not necessarily the opinions of the ISSA or the ISSA Journal.

I knew immediately I needed to read this article. It starts with a wonderful observation:

What are we doing wrong? Is the lack of support for adequate security linked to our risk-based approach to security? Why can't we make a successful case to management to increase the support for information security to meet the needs? Part of the answer is that management deals with risk every day, and it is too easy for them to accept security risk rather than reducing it by increasing security that is inconvenient and interferes with business.

I would argue that management decides to "accept security risk" because they cannot envisage the consequences of security incidents. I've written about this before.

However, Donn Parker's core argument is the following:

CISOs have tried to justify spending resources on security by claiming that they can manage and reduce security risks by assessing, reporting, and controlling them. They try to measure the benefits of information security "scientifically" based on risk reduction. This doesn't work... I propose that intangible risk management and risk-based security must be replaced with practical, doable security management with the new objectives of due diligence, compliance consistency, and enablement.

I agree. Here is a perfect example of the problem:

One CISO told me [Parker] that he performs risk assessment backwards. He says that he already knows what he needs to do for the next five years to develop adequate security. So he creates some risk numbers that support his contention. Then he works backwards to create types of loss incidents, frequencies, and impacts that produce those numbers. He then refines the input and output to make it all seem plausible. I suggested that his efforts are unethical since his input data and calculations are all fake. He was offended and said that I didn't understand. The numbers are understood by top management to be a convenient way to express the CISO's expert opinion of security needs.

This is my question: what makes these shenanigans possible? Remember the risk equation (Risk = Threat X Vulnerability X Asset Value) and consider these assertions:

• Hardly anyone can assess threats.


• Few can identify vulnerabilities comprehensively.


• Some can measure asset value.

As a result, there is an incredible amount of "play" in the variables of the risk equation. Therefore, you can make the results anything you want -- just as the example CISO shows.

It is tough enough to assign values to threats and vulnerabilities, even if time froze. In the real world, threats are constantly evolving and growing in number, while new vulnerabilities appear in both old and new software and assets on a daily basis. A network that looked like it held a low risk of compromise on Monday could be completely prone to disaster on Tuesday when a major new vulnerability is found in a core application.

Parker's alternative includes the following:

Due diligence: We can show management the results of our threat and vulnerability analysis (using examples and scenarios) by giving examples of the existence of the ulnerabilities and solutions that others have employed (not including estimated intangible probabilities and impacts). Then we can show them easily researched benchmark comparisons of the state of their security relative to other well-run enterprises and especially their competitors under similar circumstances. We then show them what would have to be done to adopt good practices and safeguards to assure that they are within the range of the other enterprises.

Bottom line: be as good as the next guy.

Compliance: We are finding that the growing body of security compliance legislation such as SOX, GLBA, and HIPAA and the associated personal and corporate liability of managers is rapidly becoming a strong and dominant security motivation...(The current legislation is poorly written and has a sledgehammer effect as written by unknowing legislative assistants but will probably improve with experience, as has computer crime legislation.)

Bottom line: compliance has turned out to be the major incentive I've seen for security initiatives. I am getting incident response consulting work because clients do not want to go to jail for failing to disclose breaches.

Enablement: It is easily shown in products and services planning that security is required for obvious and competitive purposes and from case studies, such as the Microsoft experience of being forced by market and government pressures to build security into their products after the fact.

Bottom line: this is the weakest argument of the three, and maybe why it is last. Microsoft may be feeling the heat, but it took five years and the situation is still rough. Oracle is now under fire, but how long will it take for them to take security seriously? And so on.

I think Donn Parker is making the right point here. He is saying the Emperor has no clothes and the legions of security firms providing "risk assessments" are not happy. Of course they're not -- they can deliver a product that has bearing on reality and receive money for it! That's consequence-free consulting. Try doing that in an incident response scenario where failure to do your job means the intruder remains embedded in a client's infrastructure.

As security professionals I agree we are trying to reduce risk, but trying to measure it is a waste of time. I am sad to think organizations spend hundreds of thousands of dollars on pricey risk assessments and hardly any money on real inspection of network traffic for signs of intrusions. The sorts of measurements I recommend are performance-based, as I learned in the military. We determine how good we are by drilling and exercising capabilities, preferably against a simulated enemy. We don't write formulas guestimating our defense posture.

This is not the last I have to say on this issue, but I hope to be boarding a flight soon. I commend The ISSA Journal for publishing an article that undermines a pillar of their approach to security. I bet (ISC)2 will also love Donn's approach. :)

Read More

Risk Mitigation

If you've been following the last few days of posts, I've been thinking about security from a more general level. I've been wondering how we can mitigate risks in a digital world where the following features are appearing in nearly every digital device.

Think about digital devices in your possession and see if you agree with this characterization of their development. Digital devices are increasingly:

• Autonomous: This means they act on their own, often without user confirmation. They are self-updating (downloading patches, firmware) and self-configuring (think zeroconf in IPv6). Users could potentially alter this behavior, but probably not without breaking functionality.


• Powerful: A cell phone is becoming as robust as a laptop. Almost any platform will be able to offer a shell to those who can solicit it . There is no way to prevent this development -- and would we really want to?


• Ubiquitous: Embedded devices are everywhere. You cannot buy a car without one. I expect my next big home appliance to have network connectivity. Users can't do much about some of these developments.


• Connected: Everything will be assigned an IPv4 (or soon) an IPv6 address. Distance is seldom a problem. Every digital maniac is a few hops away.


• Complex: I am scared by the thought of running Windows Mobile on my next phone. Can I avoid it? Probably not. How many lines of code are running on that mini-PC -- I mean "phone" -- I'll be using?

In my opinion, this digital world is increasingly resembling the analog one. In fact, those five attributes could describe people as easily as complex machines!

The key factor in this new world will not be static vulnerabilities, but dynamic threats. The number of opportunities for threats to play havoc will vastly dwarf the chances for defenders to address vulnerabilities.

Think about how we deal with security in a typical city. I call it the "local police model."

• Police can never prevent all crimes, although they can try.


• Police more often respond to crimes. They proceed to track and jail criminals.


• By prosecuting criminals, the justice system removes threats.


• No one spends time or money putting bars on windows or replacing door locks in the average suburban neighborhood.


• Crime still happens, but society survives as long as the level of crime is acceptable.

Why did a police model rise? Back in the cave man days, we lived in tribes. If you didn't belong to my tribe, I could beat you back with my club. As societies evolved, communication and ties between tribes prevented this simple model from working. More sophisticated threats with ingenious attacks (e.g., white collar crime) took advantage of these social ties.

Guess what -- this is where we are now in the digital world. Once upon a time you might have been able to restrict access based on trusted IPs. Then you had to shut down ports that couldn't be shared. Now we do business with everyone, and I can't be sure that the Microsoft SMB/CIFS that I'm exchanging with a business partner is normal or malicious when I use a standard access control device.

A threat-centric approach to security has served the analog world well enough. I think that is the only way to move forward as the digital world becomes as complex as the analog.

One more thought: The number of assets continues to rise. The number of vulnerabilities in those assets continues to rise. The number of threats continues to rise. The ability of security experts to apply countermeasures can not keep pace with this world. Is it time for autonomous agents to work on behalf of "the good guys?" I am beginning to agree with Dave Aitel's idea of nematodes that act on behalf of human agents.

It is becoming increasingly difficult for humans to even understand the digital environment. The only real way to know exploitation is not possible is for exploitation to be tried and then found to fail. Nematode agents may roam the network constantly testing intrusion scenarios and reporting their progress. Perhaps next-generation detection devices will monitor nematode activity. When they see another agent that is not a registered nematode exploit a target, that will be the sign that an intrusion has occurred.

Read More

BBC News Understands Risk

This evening I watched a story on BBC News about the problem of bird flu. Here is the story broken down in proper risk assessment language.

• Two assets are at risk: human health and bird health. We'll concentrate on birds in this analysis. Healthy birds are the asset we wish to protect.


• The threat is wild migratory birds infected by bird flu.


• The threat uses an exploit, namely bird flu itself.


• The vulnerability possessed by the asset and exploited by the threat is lack of immunity to bird flu.


• A countermeasure to reduce the asset's exposure to the threat is keeping protected birds indoors, away from their wild counterparts.


• The risk is infection of domesticated birds by wild birds. All infected birds must be killed.

The TV story I watched contained this quote by reported Tom Heap:

"The lesson learned from foot-and-mouth [disease, which ravaged Europe several years ago] is to do your best to keep the disease out, but assume that will fail. Be ready to tackle any outbreak to prevent an epidemic."

Let's replace certain terms with the security counterparts:

"The lesson learned from the last time we were compromised is to do your best to keep intruders out, but assume that will fail. Be ready to respond to any intrusion to prevent complete compromise of the organization."

This is the power of using proper terminology. Lessons from other scientific fields can be applied to our own problems, and we avoid re-inventing the wheel.

Read More

Gartner Survey Ranks Threats

I found the article Corporates focus on basics for IT security defences by John Leyden to be interesting. He reports on a survey presented by Gartner at their recent IT Security Summit. Gartner's survey found that IT staff ranked threats as follows:

   1. Viruses and Worms
   2. Outside Hacking or Cracking
   3. Identity Theft and Phishing
   4. Spyware
   5. Denial of Service
   6. Spam
   7. Wireless and Mobile Device Viruses
   8. Insider Threats
   9. Zero Day Threats
  10. Social Engineering
  11. Cyber-Terrorism

I am disappointed to see social engineering ranked so low. I am glad cyber-terrorism is dead last. I am surprised to see outside hacking ranked so high, even though I agree it should be a top three priority.

Here is the list I would create (if I had to call these "threats;" many of these are not "threats." I rank these "problems" or issues using a mixture of likelihood and damage inflicted. I do not agree with all the categories presented, but here is my best assessment.

   1. Viruses and Worms
   2. Outside Hacking or Cracking
   3. Spyware
   4. Denial of Service
   5. Insider Threats
   6. Identity Theft and Phishing
   7. Social Engineering
   8. Zero Day Threats
   9. Spam
  10. Wireless and Mobile Device Viruses
  11. Cyber-Terrorism

Also according to John, "More than half the respondents said they preferred buying 'best-of-breed' products from multiple technology providers while a third of those quizzed preferred integrated security suites, a preference catered for by a growing list of firms selling integrated security appliances."

By the way, I contacted Gartner about covering the summit for this blog and they completely ignored me. Thanks guys! So much for "new media" and the "blogosphere."

Read More