SANS Network IPS Testing Webcast

I'm listening to a SANS Webcast on Trustworthy IPS Testing and Certification. Jack Walsh from the Network Intrusion Prevention section of ICSA Labs spoke for about 45 minutes on his testing system. Jack spent a decent amount of time discussing the Network IPS Corporate Certification Testing Criteria (.pdf) and vulnerabilities set (.xls). The vulnerabilities set was just updated a week ago, after being criticized in July.

At present only three products are ICSA Labs certified, according to the ICSA Web site and this press release. ICSA Lab certification is a pass/fail endeavor; there are no grades.

ICSA does not release the name of the companies whose products fail. Looking at the members of the NIPS Product Developers Consortium, you can make some guesses about who participated.

Vendors pay for testing. They do so by paying for a year-long testing period, during which time they will receive at least one "full battery" of testing. Tests are rerun when the vulnerability set is updated or when then attacks used to exploit vulnerabilities change. Although ICSA Labs publishes the vulnerabilities they test, they do not say specifically how they exploit the vulnerabilities. Jack said they do use Metasploit, Core Impact, and home-grown programs. ICSA Labs relies on running real captured network traffic through a NIPS, during which they inject captured attack traffic.

I found the Webcast informative. I was surprised that Jack was so insistent that NIPS provide "mitigation" for denial of service attacks. I don't consider that an essential element of NIPS activity.

Looking at the vulnerability set, they appear to be dominated by "traditional" vulnerabilities, namely weaknesses in services running on servers. You will not see application-layer vulnerabilities like cross-site scripting, for example.

A competitor to ICSA Labs is NSS, who just announced their NSS Group IPS Testing Methodology V4.0 (060731) (.pdf) and a Certified IPS Products list.