Control "Monitoring" is Not Threat Monitoring

As I write this post I'm reminded of General Hayden's advice:

"Cyber" is difficult to understand, so be charitable with those who don't understand it, as well as those who claim "expertise."

It's important to remember that plenty of people are trying to act in a positive manner to defend important assets, so in that spirit I offer the following commentary.

Thanks to John Bambanek's SANS post I read NIST Drafts Cybersecurity Guidance by InformationWeek's J. Nicholas Hoover.

The article discusses the latest draft of SP 800-37 Rev. 1:
DRAFT Guide for Applying the Risk Management Framework to Federal Information Systems: A Security Life Cycle Approach.

I suspected this to be problematic given NIST's historical bias towards "controls," which I've criticized in Controls Are Not the Solution to Our Problem and Consensus Audit Guidelines Are Still Controls. The subtext for the article was:

The National Institute for Standards and Technology is urging the government to continuously monitor its own cybersecurity efforts.

As soon as I read that, I knew that NIST's definition of "monitor" and the article's definition of "monitor" did not mean the real sort of monitoring, threat monitoring, that would make a difference against modern adversaries.

The article continues:

Special Publication 800-37 fleshes out six steps federal agencies should take to tackle cybersecurity: categorization, selection of controls, implementation, assessment, authorization, and continuous monitoring...

Finally, and perhaps most significantly, the document advises federal agencies to put continuous monitoring in place. Software, firmware, hardware, operations, and threats change constantly. Within that flux, security needs to be managed in a structured way, Ross says.

"We need to recognize that we work in a very dynamic operational environment," Ross says. "That allows us to have an ongoing and continuing acceptance and understanding of risk, and that ongoing determination may change our thinking on whether current controls are sufficient."

The continuous risk management step might include use of automated configuration scanning tools, vulnerability scanning, and intrusion detection systems, as well as putting in place processes to monitor and update security guidance and assessments of system security requirements.

Note that the preceding text mentions "intrusion detection systems," but the rest of the text has nothing to do with real monitoring, i.e., detecting and responding to intrusions. I'm not just talking about network-centric approaches, by the way -- infrastructure, host, log, and other sources are all real monitoring, but this is not what NIST means by "monitoring."

To understand NIST's view of monitoring, try reading the new draft. I'll insert my comments.

APPENDIX G

CONTINUOUS MONITORING

MANAGING AND TRACKING THE SECURITY STATE OF INFORMATION SYSTEMS

A critical aspect of managing risk from information systems involves the continuous monitoring of the security controls employed within or inherited by the system.65

[65 A continuous monitoring program within an organization involves a different set of activities than Security Incident Monitoring or Security Event Monitoring programs.]

So, it sounds like activities that involve actually watching systems are not within scope for "continuous monitoring."

Conducting a thorough point-in-time assessment of the deployed security controls is a necessary but not sufficient condition to demonstrate security due diligence. An effective organizational information security program also includes a rigorous continuous monitoring program integrated into the system development life cycle. The objective of the continuous monitoring program is to determine if the set of deployed security controls continue to be effective over time in light of the inevitable changes that occur.

That sounds ok so far. I like the idea of evaluations to determine if controls are effective over time. In the next section below we get to the heart of the problem, and why I wrote this post.

An effective organization-wide continuous monitoring program includes:

• Configuration management and control processes for organizational information systems;

• Security impact analyses on actual or proposed changes to organizational information systems and environments of operation;67

• Assessment of selected security controls (including system-specific, hybrid, and common controls) based on the organization-defined continuous monitoring strategy;68

• Security status reporting to appropriate organizational officials;69 and

• Active involvement by authorizing officials in the ongoing management of information system-related security risks.

Ok, where is threat monitoring? I see configuration management, "control processes," reporting status to "officials," "active involvement by authorizing officials," and so on.

The next section tells me what NIST really considers to be "monitoring":

Priority for security control monitoring is given to the controls that have the reatest volatility and the controls that have been identified in the organization’s plan of action and milestones...

[S]ecurity policies and procedures in a particular organization may not be likely to change from one year to the next...

Security controls identified in the plan of action and milestones are also a priority in the continuous monitoring process, due to the fact that these controls have been deemed to be ineffective to some degree.

Organizations also consider specific threat information including known attack vectors (i.e., specific vulnerabilities exploited by threat sources) when selecting the set of security controls to monitor and the frequency of such monitoring...

Have you broken the code yet? Security control monitoring is a compliance activity. Granted, this is an improvement from the typical certification and accreditation debacle, where "security" is assessed via paperwork exercises every three years. Instead, .gov compliance teams will perform so-called "continuous monitoring," meaning more regular checks to see if systems are in compliance.

Is this really an improvement?

I don't think so. NIST is missing the point. Their approach advocates Control-compliant security, not field-assessed security. Their "scoreboard" is the result of a compliance audit, not the number of systems under adversary control or the amount of data exfiltrated or degraded by the adversary.

I don't care how well your defensive "controls" are informed by offense. If you don't have a Computer Incident Response Team performing continuous threat monitoring for detection and response, you don't know if your controls are working. The NIST document has a few hints about the right approach, at best, but the majority of the so-called "monitoring" guidance is another compliance activity.