NORAD-Inspired Security Metrics

When I was a second degree cadet at USAFA (so long ago that, of my entire class, only myself and three friends had 486 PCs with Ethernet NICs) I visited NORAD. I remember thinking the War Games set was cooler, but I didn't give much thought to the security aspects of their mission.

Today I remembered NORAD and considered their mission with respect to my post last year titled Control-Compliant vs Field-Assessed Security. In case you can't tell from the pithy title, the central idea was that it's more effective to measure security by assessing outcomes instead of inputs. For example, who cares if 100% of your systems have Windows XP SP2 if they are all 0wned by a custom exploit written just for your company? Your security has failed. Inputs are important, but my experience with various organizations is that they tend to be the primary means of "measuring" security, regardless of how well they actually preserve the CIA triad.

Let's put this in terms of NORAD, whose front page states:

The North American Aerospace Defense Command (NORAD) is a bi-national United States and Canadian organization charged with the missions of aerospace warning and aerospace control for North America. Aerospace warning includes the monitoring of man-made objects in space, and the detection, validation, and warning of attack against North America whether by aircraft, missiles, or space vehicles, through mutual support arrangements with other commands. Aerospace control includes ensuring air sovereignty and air defense of the airspace of Canada and the United States...

To accomplish the aerospace warning mission, the commander of NORAD provides an integrated tactical warning and attack assessment to the governments of Canada and the United States. To accomplish the aerospace control mission, NORAD uses a network of satellites, ground-based radar, airborne radar and fighters to detect, intercept and, if necessary, engage any air-breathing threat to North America.

What are some control-compliant or input metrics for NORAD?

• Number of planes at the ready for intercepting rogue aircraft


• Average pilot rating (i.e., some sort of assessment of pilot skill)


• Radar uptime


• Radar coverage (e.g., percentage of North American territory monitored)

These are all interesting metrics. You might see some comparisons to metrics you might track, like percentage of hosts with anti-virus.

Now consider: do any of those metrics tell you if NORAD is accomplishing its mission? In other words, what is the outcome of all those inputs? What is the score of this game?

Here are some field-assessed or outcome-based metrics.

• Number of rogue aircraft penetrating North American territory (indicates a failure to deter activity)


• Number of aircraft not detected by NORAD but discovered via other means to have penetrated North American territory (perhaps via intel sources; indicates a failure to detect activity)


• Number of aircraft not repelled by interceptors (hopefully this would never happen!)


• Time from first indication of rogue aircraft to launching interceptors (indicates effectives of pilot-to-plane-to-air process)

These metrics address the critical concern: accomplishing the mission.

Keep these in mind when you are devising metrics for your digital security program.