Bamm Visscher pointed me to this
Security Incite post about the new NWC article
Managed Security Service Providers by Joanne VanAuken. Ms. VanAuken managed to get five MSSPs -- BT Global Services, Cybertrust, Internet Security Systems, LURHQ and SecureWorks -- to answer. The others?
VeriSign Managed Security Services initially accepted and did an outstanding job completing its RFI, but backed out because of the risk of exposing too much confidential data by publishing its RFI responses online (a requirement to participate). Equant also initially accepted but could not complete our RFI in time due to its rebranding to Orange Business Services. MCI, which has partnered with Verizon Business, declined. Accenture, AT&T Networking Outsourcing Services, Capgemini, Computer Sciences Corp. (CSC), Connetic, EDS, Getronics, IBM Global Services, Perimeter Internetworking, Science Applications International Corp (SAIC), Solutionary, Sprint, Symantec, TruSecure, Unisys and VigilantMinds didn't respond to our invitation.Symantec didn't respond? And why wasn't Counterpane invited?
Here are a few fun excerpts from the story.
The million dollar question is, Which will be the safest choice? Investigate the financial health of your MSSP. If a provider meets your SLA (service-level agreement) and technical requirements but its business road map and financial health are questionable, walk away."The safest choice?" What about caring if the MSSP can deliver?
[MSSP] vendor relationships mean MSSPs often receive advance notice of worm and viruses outbreaks...Vendor to MSSP: "Hey, a worm outbreak is coming. Watch out!"
MSSP to Vendor: "Ok, thanks."
This doesn't happen. If anything, the MSSP might be lucky to see worm activity and then notify its customers.
All vendors indicated a focus on preventing, rather than detecting, intrusions... In the rare event that an MSSP does suspect a compromise, all vendors said that escalations are immediately executed while potential damage is contained through every countermeasure available until both the MSSP and client agree on whether a compromise actually occurred.Of course "prevention" is king. Unfortunately, focus on "prevention" often drowns out considerations for detection. As a result, compromise is seldom "rare."
After evaluating five RFI proposals, we gave our Editor's Choice to Internet Security Systems because of it wealth of service offerings, outstanding SLA agreements, highly skilled personnel and bundled pricing options."Highly skilled" may indeed be true, but how does NWC know this?
My overall assessment of this "review" is that was a large amount of work for NWC to write the
RFI (.doc) and for the vendors to
respond. In the end, the process identified many plausibly
necessary conditions for running a good MSSP, but none are
sufficient to assess if the MSSP is any good at its core task: helping customers
resist, detect, and respond to intrusions. (I no longer say "prevent" intrusions. "Resist" seems more accurate when
prevention eventually fails.)
Take a look at the
Report Card to see how the MSSPs were rated. 25% is assigned to "Service Levels." 20% is "Price." 10% is "Environmental," like physical security and facility features. That's 55% of the score based on non-technical metrics. The remaining areas, "Support" at 25% and "Security Practices" at 20%, dance around the core issue. Items like "Technical Expertise/experience," at 10%, appears to have been measured by "certifications." Throw that out the window. A portal at 7%? Again, irrelevant. Aside from 4% for equipment compatibility and 4% for research organization partnerships, the remaining metrics are all based on plans, policies, and other paperwork.
This is similar to the
football analogy I offered last month. Think about the difference in focus. From the review:
[Two MSSPs] earned an edge by describing their power, air conditioning and fire-prevention setups in granular detail, including such items as contracts with diesel fuel suppliers, audits of power performance, the target temperature for SOC (security operations center) rooms, and the time (in seconds) allowed after a smoke alarm sounds for employees to leave the room before the fire-suppression system kicks in.Great -- will these MSSPs notice if I use stolen credentials to access a monitored customer resource, install a back door, and remove proprietary information using a stateless covert channel?
It's easy for me to criticize, you might say. How would I have done such a review? Well, I have done them. Sometimes I review MSSPs, and sometimes I review in-house teams. My company's
NSM Assessment and Evaluation is a two-phase process that works for in-house or outsourced security teams. I use a self-modified version of the
NSA-IAM for the Assessment and the NSA-IEM for the Evaluation. The NWC review would be similar to the Assessment, since it is non-technical and hands-off. The Evaluation is technical and hands-on, and tests how well all the MSSP people, products, and processes
actually perform.
I've also trained some of the analysts at
VeriSign (ex-Guardent), and I've met or spoken several times with the folks at
LURHQ and
Verizon (ex-NetSec). I also know people at many of the other MSSPs.
If you want to know more, email me: richard [at] taosecurity [dot] com.
As far as Mike Rothman's views go, he says this:
Yes, MSS is a Commodity. So what... There seem to be very little outward differentiation and the biggest decision point is probably how long it takes you to wade through all of the pricing and packaging options. But this is good news, in that the business is mature enough to have tight operational processes - which is pretty much what you want to see in an outsourcing situation.Ugh, that's the problem. If there's no outward differentiation, then what's being measured may be irrelevant to the core problem. As Bamm Visscher asks, "Is your MSSP just a 'Security Device Management Provider'?"
Amazon.com just posted my five star review of PGP & GPG by Michael Lucas, one of the best tech authors around. From the review:
If you want to have hardware manufacturers and applicationn vendors take you seriously, it helps to have metrics. Today I learned of BSDStats, which uses the sysutils/bsdstats shell script to collect information on hosts. I decided to try it and see what data is collected.
Amazon.com just posted my 200th technical book review. That's roughly 200 books since May 2000. 
