Incident Response Fundamentals: Trigger, Escalate, and Size up

Okay, your incident response process is in place, you have a team, and you are hanging out in the security operations center, watching for Bad Things to happen. Then, surprise surprise, an alert triggers: what’s next? Trigger and Escalate The first thing you need to do is determine the basic parameters of the incident, and assign resources (people) to investigate and manage it. This is merely a quick and dirty step to get the incident response process kicked off, and the basic information you gather will vary based on what triggered the alert. Not all alerts require a full incident response – much of what you already deal with on a day to day basis is handled by your existing security processes. Incident response is for situations that cannot be adequately handled by your standard processes. Most IDS, DLP, or other alerts/help desk calls don’t warrant any special response – this series is about incidents that fall outside the range of your normal background noise. Where do you draw the line? That depends entirely on your organization. In a small business a single system infected with malware might lead to a response, while the same infection in a larger company could be handled as a matter of course. Technically these smaller issues (in smaller companies) are “incidents” and follow the full response process, but that entire process would be managed by a single individual with a few clicks. Regardless of where the line is drawn, communication is still critical. All parties must be clear on the specifics of which situations require a full incident investigation and which do not. For any incident, you will need a few key pieces of information early to guide next steps. These include: What triggered the alert? If someone was involved or reported it, who are they? What is the reported nature of the incident? What is the reported scope of the incident? This is basically the number and nature of systems/data/people involved. Are any critical assets involved? When did the incident occur, and is it ongoing? Are there any known precipitating events for the incident? In other words, is there a clear cause? All this should be collected in a matter of seconds or minutes through the alerting process, and provides your initial picture of what’s going on. When an incident does look more serious, it’s time to escalate. We suggest you have guidelines for initiating this escalation, such as: Involvement of designated critical data or systems. Malware infecting a certain number of systems. Sensitive data detected leaving the organization. Unusual traffic/behavior that could indicate an external compromise. Once you escalate it’s time to assign an appropriate resource, request additional resources (if needed), and begin the response. Remember that per our incident response principles, whoever first detects and evaluates the incident is in charge of it until they hand it off to someone else of equal or greater authority. Size up The term size up comes from the world of emergency services. It refers to the initial impressions of the responder as they roll up to the scene. They may be estimating the size of a cloud of black smoke coming out of a house, or a pile of cars in the middle of a highway. The goal here is to take the initial information provided and expand on it as quickly as possible to determine the true nature of the incident. For an IT response, this involves determining specific criteria – some of which you might already know: Scope: Which systems/networks/data are involved? While the full scope of an IT incident may take some time to determine, right now we need to go beyond the initial information provided and learn as much about the extent of the incident as possible. This includes systems, networks, and data. Don’t worry about getting all the details of each of them yet – the goal is merely to get a handle on how big a problem you might be facing. Nature: What kind of incident are you dealing with? If it’s on the network, look at packets or reports from your tools. For an endpoint, start digging into the logs or whatever triggered the alert. If it involves data loss, what data might be involved? Be careful not to assume it’s only what you detected going out, or what you think was inappropriately accessed. People: If this is some sort of an external attack, you probably aren’t going to spend much time figuring out the home address of the people involved at this stage. But for internal incidents it’s important to put names to IP addresses for both suspected perpetrators and victims. You also want to figure out what business units are involved. All of this affects investigation and response. Yes, I could have just said, “who, what, when, where, and how”. We aren’t performing more than the most cursory examination at this point, so you’ll need to limit your analysis to basics such as security tool alerts, and system and application logs. The point here is to get a quick analysis of the situation, and that means relying on tools and data you already have. The OODA Loop Many incident responders are familiar with the OODA Loop originally developed by Colonel Boyd of the US Air Force. The concept is that in an incident, or any decision-making process, we follow a recurring cycle of Observe, Orient, Decide, and Act. These cycles, some of which are nearly instantaneous and practically subconscious, describe the process of continually collecting, evaluating, and acting on information. The OODA Loop maps well to our Incident Response Fundamentals process. While we technically follow multiple OODA cycles in each phase of incident response, at a macro level trigger and escalate are a full OODA loop (gathering basic information and deciding to escalate), while size up maps to a somewhat larger loop that increases the scope of our observations, and closes with the action of requesting additional resources or moving on to the next response phase. Once you have collected and reviewed the basics, you should have a reasonable idea of what you’re dealing with. At

Read Post

Baa Baa Blacksheep

Action and reaction. They have been the way of the world since olden times, and it looks like they will continue ad infinitum. Certainly they are the way of information security practice. We all make our living from the action/reaction cycle, so I guess I shouldn’t bitch too much. But it’s just wrong, though we seem powerless to stop it. Two weeks ago at Toorcon, Firesheep was introduced, making crystal clear what happens to unsecured sessions to popular social networking sites such as Facebook and Twitter. We covered it a bit in last week’s Incite, and highlighted Rich’s TidBITS article and George Ou’s great analysis of which sites were exposed by Firesheep. Action. Then today the folks at Zscaler introduced a tool called Blacksheep, a Firefox plug-in to detect Firesheep being used on a network you’ve connected to. It lets you know when someone is using Firesheep and thus could presumably make you think twice about connecting to social networking sites from that public network, right? Reaction. Folks, this little cycle represents everything that is wrong with information security and many things wrong with the world in general. The clear and obvious step forward is to look at the root cause – not some ridiculous CYA response from Facebook about how one-time passwords and anti-spam are good security protections – but instead the industry spat up yet another band-aid. I’m not sure how we even walk anymore, we are so wrapped head to toe in band-aids. It’s like a bad 1930’s horror film, and we all get to play the mummy. But this is real. Very real. I don’t have an issue with Zscaler because they are just playing the game. It’s a game we created. New attack vector (or in this case, stark realization of an attack vector that’s been around for years) triggers a bonanza of announcements, spin, and shiny objects from vendors trying to get some PR. Here’s the reality. Our entire business is about chasing our own tails. We can’t get the funding or the support to make the necessary changes. But that’s just half of the issue – a lot of our stuff is increasingly moving to services hosted outside our direct control. The folks building these services couldn’t give less of a rat’s ass about fixing our issues. And users continue about their ‘business’, blissfully unaware that their information is compromised again and again and again. Our banks continue to deal with 1-2% ‘shrinkage’, mostly by pushing the costs onto merchants. Wash, rinse, and repeat. Yes, I’m a bit frustrated, which happens sometimes. The fix isn’t difficult. We’ve been talking about it for years. Key websites that access private information should fully encrypt all sessions (not just authentication). Google went all SSL for Gmail a while ago, and their world didn’t end and their profits didn’t take a hit either. Remote users should be running their traffic through a VPN. It’s not hard. Although perhaps it is, because few companies actually do it right. But again, I should stop bitching. This ongoing stupidity keeps me (and probably most of you) employed. Share:

Read Post

Totally Transparent Research is the embodiment of how we work at Securosis. It’s our core operating philosophy, our research policy, and a specific process. We initially developed it to help maintain objectivity while producing licensed research, but its benefits extend to all aspects of our business.

Going beyond Open Source Research, and a far cry from the traditional syndicated research model, we think it’s the best way to produce independent, objective, quality research.

Here’s how it works:

  • Content is developed ‘live’ on the blog. Primary research is generally released in pieces, as a series of posts, so we can digest and integrate feedback, making the end results much stronger than traditional “ivory tower” research.
  • Comments are enabled for posts. All comments are kept except for spam, personal insults of a clearly inflammatory nature, and completely off-topic content that distracts from the discussion. We welcome comments critical of the work, even if somewhat insulting to the authors. Really.
  • Anyone can comment, and no registration is required. Vendors or consultants with a relevant product or offering must properly identify themselves. While their comments won’t be deleted, the writer/moderator will “call out”, identify, and possibly ridicule vendors who fail to do so.
  • Vendors considering licensing the content are welcome to provide feedback, but it must be posted in the comments - just like everyone else. There is no back channel influence on the research findings or posts.
    Analysts must reply to comments and defend the research position, or agree to modify the content.
  • At the end of the post series, the analyst compiles the posts into a paper, presentation, or other delivery vehicle. Public comments/input factors into the research, where appropriate.
  • If the research is distributed as a paper, significant commenters/contributors are acknowledged in the opening of the report. If they did not post their real names, handles used for comments are listed. Commenters do not retain any rights to the report, but their contributions will be recognized.
  • All primary research will be released under a Creative Commons license. The current license is Non-Commercial, Attribution. The analyst, at their discretion, may add a Derivative Works or Share Alike condition.
  • Securosis primary research does not discuss specific vendors or specific products/offerings, unless used to provide context, contrast or to make a point (which is very very rare).
    Although quotes from published primary research (and published primary research only) may be used in press releases, said quotes may never mention a specific vendor, even if the vendor is mentioned in the source report. Securosis must approve any quote to appear in any vendor marketing collateral.
  • Final primary research will be posted on the blog with open comments.
  • Research will be updated periodically to reflect market realities, based on the discretion of the primary analyst. Updated research will be dated and given a version number.
    For research that cannot be developed using this model, such as complex principles or models that are unsuited for a series of blog posts, the content will be chunked up and posted at or before release of the paper to solicit public feedback, and provide an open venue for comments and criticisms.
  • In rare cases Securosis may write papers outside of the primary research agenda, but only if the end result can be non-biased and valuable to the user community to supplement industry-wide efforts or advances. A “Radically Transparent Research” process will be followed in developing these papers, where absolutely all materials are public at all stages of development, including communications (email, call notes).
    Only the free primary research released on our site can be licensed. We will not accept licensing fees on research we charge users to access.
  • All licensed research will be clearly labeled with the licensees. No licensed research will be released without indicating the sources of licensing fees. Again, there will be no back channel influence. We’re open and transparent about our revenue sources.

In essence, we develop all of our research out in the open, and not only seek public comments, but keep those comments indefinitely as a record of the research creation process. If you believe we are biased or not doing our homework, you can call us out on it and it will be there in the record. Our philosophy involves cracking open the research process, and using our readers to eliminate bias and enhance the quality of the work.

On the back end, here’s how we handle this approach with licensees:

  • Licensees may propose paper topics. The topic may be accepted if it is consistent with the Securosis research agenda and goals, but only if it can be covered without bias and will be valuable to the end user community.
  • Analysts produce research according to their own research agendas, and may offer licensing under the same objectivity requirements.
  • The potential licensee will be provided an outline of our research positions and the potential research product so they can determine if it is likely to meet their objectives.
  • Once the licensee agrees, development of the primary research content begins, following the Totally Transparent Research process as outlined above. At this point, there is no money exchanged.
  • Upon completion of the paper, the licensee will receive a release candidate to determine whether the final result still meets their needs.
  • If the content does not meet their needs, the licensee is not required to pay, and the research will be released without licensing or with alternate licensees.
  • Licensees may host and reuse the content for the length of the license (typically one year). This includes placing the content behind a registration process, posting on white paper networks, or translation into other languages. The research will always be hosted at Securosis for free without registration.

Here is the language we currently place in our research project agreements:

Content will be created independently of LICENSEE with no obligations for payment. Once content is complete, LICENSEE will have a 3 day review period to determine if the content meets corporate objectives. If the content is unsuitable, LICENSEE will not be obligated for any payment and Securosis is free to distribute the whitepaper without branding or with alternate licensees, and will not complete any associated webcasts for the declining LICENSEE. Content licensing, webcasts and payment are contingent on the content being acceptable to LICENSEE. This maintains objectivity while limiting the risk to LICENSEE. Securosis maintains all rights to the content and to include Securosis branding in addition to any licensee branding.

Even this process itself is open to criticism. If you have questions or comments, you can email us or comment on the blog.