Analog Penetration Testing

While watching the evening news I saw the story Investigation: U.S. borders perilously porous -- Federal investigators easily pass border checks using fake identification. On Wednesday the Government Accountability Office (yes, they changed their name) will release a report on an analog penetration test performed against the US border. What do I mean by that?

[GAO] agents successfully entered the United States using fictitious driver's licenses and other bogus documentation through nine land ports of entry on the northern and southern borders. CBP [Customs and Border Protection] officers never questioned the authenticity of the counterfeit documents presented at any of the nine crossings.

On three occasions -- in California, Texas, and Arizona -- agents crossed the border on foot. At two of these locations -- Texas and Arizona -- CBP allowed the agents entry into the United States without asking for or inspecting any identification documents.

This excerpt is from a draft report (.pdf) which will be delivered by GAO to the US Senate on Wednesday. Initial reports indicate lawmakers are really upset by these findings, because the situation has not improved since the last test in 2003.

What does this tell me? Apparently, decision-makers listen when findings are presented in a simple manner. If CBP fails to prevent people with forged IDs from entering the country, then it's clear they are not fulfilling their mandate. Simulating threat activity and discovering that attacks succeed 100% of the time is a damning critique of one's security measures. When presented in this manner, it's easy to see what works and what doesn't.

This is why I advocate penetration tests as a means to assess security. If it takes me five minutes to gain access to information you expect to keep private, that's a clear indication your organization has serious security problems. It's performance-based security measurement. Just how well do your people, products, and processes handle a real event?

This sort of thinking is second nature to anyone with military, law enforcement, or fire fighting backgrounds. (I'm sure there are others -- feel free to name them as comments.) These organizations assess their capability to perform their missions by exercising. Sure, you should take inventories, theorize, and so on, but the proof lies in how well you can execute in a near-real-world environment. (Executing in the real world is obviously the best test, but you don't want to put people's lives on the line unnecessarily.)

Do you want to know how well your airport screeners detect weapons in luggage? Don't measure your training budget, the education level of the personnel, or the number of steps in their checklist. Run fake weapons through X-ray machines and see who catches them.

How well is border security inspecting IDs? Don't count increases in the numbers of agents, measure their salaries, or inspect their guidebooks. Send agents across the border with fake IDs and see if CBP stops them.

How well does your enterprise protect sensitive information from unauthorized access? Don't pretend to assess threats, assign fake risk values, and count the number of packets blocked by your firewall. Hire a pen tester to steal your information.

Repeat the process in 6 months and see if it's more difficult. If yes, your security has improved. If no, your security has degraded. It's really as simple as that. Be careful to ensure the second pen tester is as skilled as, or superior to, the first pen tester.