How can a blog reader find competent operations personnel?

I received the following question from a blog reader. I am interested in hearing what you think.

I'm team lead for a small private-sector security operations team. We are fortunate that we have a reasonably interesting and attractive work environment, readily available financial resources, and a relatively manageable event load.

We've been trying to hire a mid to senior level analyst position for at least a year now, and have been having absolutely no luck whatsoever.

The job responsibilities mainly consist of analyzing events from the SEM and NSM stacks, documenting and resolving incidents, and conducting regular vulnerability management operations.

A majority of the applications we get seem to come from security "architects" who may have some product deployment experience, but little to no applicative analysis skills necessary to un-haystack the needles, or pursue an incident to closure.

Very few of the interviewees can even get past the technical phone screen, which consists of the following three questions:

1. You see an IDS/IPS event in your event console called "some kind of IDS event name here".
   
   
   
   • What would you do to investigate the event, and how would you validate that the event was a real attack and not a false positive?
   
   
   • How would you determine if this was a one-off event, or part of an overall pattern?
   
   
   • What other kinds of information would you seek out to build a more complete picture of the context around this event?
   
   
   
   
2. After having investigated the event, you have gathered enough positive indicators that the actual traffic consisted of a legitimate attack against a server you suspect may be vulnerable to an an attack of that kind.




• How do you determine what may have happened to the server? (This question is usually geared towards whatever platform the candidate might have actual technical experience with.)


• What would you do if you saw a subsequent event that indicated the target system had downloaded a file from the internet soon after the original IDS event?


• How could you recover the file? What would you do to analyze it? (This question usually evolves into some platform-specific live forensics, network forensics, and incident response.)




3. You conduct a vulnerability scan that produces output that indicates that a server X (operating system Y) may be vulnerable to issue Z.




• What would you do to validate the finding?


• How would you validate the finding if the report indicated the issue was present on 100 machines? (This again is usually geared towards a platform that the candidate has the most experience with).


• What would you do to address the issue?

These three topic areas seem to cut to the core of what raw analysis tasks an operations analyst must be able to perform well. The kinds of answers I expect are specific, detailed, and accurate given the scenarios supplied (i.e. application-level attack against a 3-tier windows-based web application merits one kind of response vs. a client-side buffer overflow attack against a web browser, etc.).

Maybe one or two of our candidates out of several dozen have even been able to answer them competently enough for a second round (and they eventually accepted more lucrative offers). I'd even be happy if the candidates could get two out of three.

Am I setting the bar too high? Are there some magic keywords in the job req that I'm missing? Am I going to have hire juniors and train them up? Is there even such a thing as a senior operations analyst?

My initial response is that the number of people who can independently and competently answer these questions is remarkably small. Furthermore, the number of shops that are collecting the data necessary to answer these questions is also small.

What do blog readers think?