Response to Daily Dave Thread

I don't subscribe to the Daily Dave (Aitel) mailing list, but I do keep a link to the archives on my interests page. Some of the offensive security world's superstars hang out on that list, so it makes for good reading.

The offensive side really made an appearance with yesterday's thread, where Dave's "lots of monkeys staring at a screen....security?" thread says:

My feeling is that IDS is 1980's technology and doesn't work anymore. This makes Sourcefire and Counterpane valuable because they let people fill the checkbox at the lowest possible cost, but if it's free for all IBM customers to throw an IDS in the mix then the price of that checkbox is going to get driven down as well.

First, it's kind of neat to see anyone speaking about "IDS" instead of "IPS" here. I think this reflects Dave's background working for everyone's favorite three letter agency. The spooks and .mil types (like me) tend to be the last people to even think about detection these days.

Second, it seems to be popular to think of "IDS" as strictly a signature-based technology, as Gadi Evron believes:

IDS devices are signature based and try to detect bad behaviour using, erm, a sniffer or equivalent.

That's hasn't been true for a while, even if you're talking about Snort. Sure, there are tons of signatures, but they're certainly not just for content matching. If you're thinking about Bro, signatures aren't really even the main issue -- protocol anomaly detection is.

Python demigod Dave posts another message that is a little worrisome:

Making IDS part of a defense in depth strategy is giving it some credit for actually providing defense, which it doesn't do. The people who win the IDS game are the people who spend the least money on it. This is why security outsourcing makes money - it's just as worthless as maintaining the IDS yourself, but it costs less. Likewise, Snort is a great IDS solution because it does nothing but it does it cheaper.

The technology curve is towards complex, encrypted, asynchronous protocols. The further into time you look, the worse the chances are that sniffing traffic is an answer to anything.

The market is slowly realizing this technology's time has past, but in the meantime lots of people are making giant bus-loads of cash. Good for them. But IDS technology isn't relevant to a security discussion in this day and age and it's not going to be anytime soon.

I will agree that many commercial managed security monitoring services are worthless, to the extent that they are ticket- and malware-oriented. However, the idea that Snort "does nothing" is just wrong. Hopefully Dave is just being inflammatory to spur discussion. Sure, Snort is not going to detect an arbitrary outbound encrypted covert channel using port 443. That doesn't mean Snort isn't useful for the hundreds of other attack patterns still seen in the wild.

Since the majority of the posters to this thread are offensive, I doubt they have read any of my books. For example, reverse engineering guru Halvar Flake follows up with this insight:

I still agree with the concept of replacing an IDS with just a large quantity of tapes on which to archive all traffic. IDSs will never alert you to an attack-in-progress, and by just dumping everything onto a disk somewhere you can at least do a halfways-decent forensics job thereafter. Since everybody and his dog is doing cryptoshellcode these days you won't be all-knowing, but at least you should be able to properly identify which machine got owned first.

Welcome to network security monitoring, albeit at least a decade late. The fact that the criminal underground is using covert and encrypted channels now doesn't mean they weren't used 10 plus years ago, when smart people in the spook and .mil worlds needed a way to gain some sort of awareness of network activities by more dangerous adversaries.

Most respected IDS old-school critic Tom Ptacek isn't convinced:

I am waiting for someone to tell me the story about how an IDS saved their bacon. I'm not interested in the story about how it found the guy with the spyware infection or the bot installation; secops teams find those things all the time in their firewall logs and they don't
freak out about it when they do.

The last times I manned a console full-time as a "SOC monkey," for the Air Force in 1998-2001 and at Ball Aerospace in 2001-2002, we found intrusions all the time. I expect several people in the #snort-gui channel where I idle on irc.freenode.net also have stories to share. I'll have more to say on this later.

Tom continues:

This "signature" vs. "real intrusion detection" thing is a big red herring. Intrusion detection has been an active field of research for over 15 years now and apart from Tripwire I can't point to anything operationally valuable it has produced.

This sounds like the "Snort is worthless" argument Dave proposed. Finally:

Halvar, when you figure out how to parallelize enough striped tape I/O to keep up with a gigE connection, then, Halvar, then I will respect you.

This is another common argument. Most every detection critic argues their pipes are too big to do any useful full content collection. Let's just say that is not a problem for everyone. Many, many organizations connect to the Internet using OC-3s (155 MBps), fractional OC-3s, T-3s (45 Mbps) and below. Full content collection, especially at the frac OC-3 (say 60 Mbps) and lower, is no problem -- even for commodity hardware, if you use Intel NICs, a solid OS, and fast, large hard drives. Even if you drop some small percentage of the traffic, so what? What are the odds that you drop everything that is relevant to your investigation, all the time?

What if your pipes really are too big for full content collection, say in the core of the network? I would argue that's not the place to do full content collection, but let's say you are told to "do something" about detection in a high-bandwidth environment. That's where the other NSM data types come into play -- namely session data and statistical data. Can't save every packet, or you don't want to? Save sessions describing who talked to who, when, using what protocols and services, and how much data was transferred. That is absolute gold for traffic analysis, and it doesn't matter if it's encrypted. At the very least you can profile the traffic statistically.

The root of this problem with this discussion is the narrow idea that a magic box can sit on an arbitrary network and tell you when something "bad" happens. That absolutely won't be possible, at least not for every imaginable "bad" case. The "IDS" has been pigeonholed in the same way the "firewall" has -- as a product and not a real system.

A standard "IDS" isn't an "intrusion detection system" at all; it's an attack indication system. Snort gives you a hint that something bad might be happening. You need the rest of your NSM data to determine what is going on. You can also start with non-alert NSM data (as described in this war story) and investigate intrusions.

Similarly, a firewall isn't necessarily stopping attacks; it should be enforcing an access control policy.

A real detection system identifies deviations from policy, and perhaps should be called a network policy violation detector. A real network policy enforcement system prevents policy violations. The point is that neither has to be boxed into an appliance and sold as a "NPVD" or "NPES". (As you can see, acronyms which tend to accurately describe a system's functionality are completely marketing-unfriendly.)

I'll conclude by saying that I agree with Dave about "monkeys" staring at screens. Many of those sorts of analysts are not doing NSM-centric work that would truly discover intrusions. Yes, the network is a tough place to detect. However, I've argued before that in an age of ubiquitous kernel-mode rootkits, NSM is needed more than ever. If you can't trust a rootkit-controlled host to tell you what's happening, why would you ignore the network? Sure, the traffic could be covert, encrypted, and so forth, but if the pattern of activity isn't normal you can verify that at least something suspicious is happening.

It's time for another book.