Incident Response Fundamentals: Response Infrastructure and Preparatory Steps

In our last post we covered organizational structure options for incident response. Aside from the right org structure and incident response process, it’s important to have a few infrastructure pieces (tools) in place, and take some preparatory steps ahead of time. As with all our recommendations in this series, remember that one size doesn’t fit all, and those of you in smaller companies will probably skip some of the tools or not need some of the prep steps. Incident Response Support Tools The following tools are extremely helpful (sometimes essential) for managing incidents. This isn’t a comprehensive list, but an overview of the major categories we see most often used by successful organizations: Multiple secure communications channels: It’s bad to run all your incident response communications over an pwned email server, or to lose the ability to communicate if a cell tower is out. Your incident response team should have multiple communications options – landlines, mobile phones (on multiple carriers if possible), encrypted online tools (via secure systems), and possibly even portable mobile radios. For smaller organizations this might be as simple as GPG or S/MIME for encrypted email (and a list of backup email accounts on multiple providers), or a collaboration Web site and some backup cell phones. Incident management system: Many organizations use their trouble ticket systems to manage incidents, or handle them manually. There are also purpose-built tools with improved security and communications options. As long as you have some central and secure place to keep track of an incident, and a backup option or two, you should be covered. Analysis and forensics tools: As we will discuss later in the series, one of the most critical elements of incident response is the investigation. You need a mix of forensics tools to figure out what’s going on – including tools for analyzing network packet captures, endpoints and mobile devices, and various logs (everything from databases to network devices). This is a very broad category that depends on your skill set, the kinds of incidents you are involved with, and budget. Secure test environment/lab: This is clearly more often seen in larger organizations and those with larger budgets and higher incident rates, but even in a smaller organization it is extremely helpful to have some test systems and a network or two – especially for analysis of compromised endpoints and servers. Network taps and capture/analysis laptops: At some point during an investigation, you’ll likely need to physically go to a location and plug in an inline or passive network tap to analyze part of a network – sometimes even the communications from a single system. This kind of monitoring may not be possible on your existing network – not all routers let you plug in and capture local traffic (heck, you might simply be out of ports), so we recommend you have a small tap and spare laptop (with a large hard drive, possibly external) available. These are very cost-effective and useful even for smaller organizations. Data collection and monitoring infrastructure: As previously discussed. Preparatory Steps Hopefully the idea that tools are only a small part of every security process is starting to set in. Once again, tools are a means to an end. The following steps help set up your infrastructure to support the response process and make the best use of your investment in tools. They cost little aside from time, but will determine the success and/or failure of your response efforts: Define a communications plan: As we mentioned above, it’s important to have multiple communications methods. It’s even more important to have a calling list with all the various numbers, emails, and other addresses you need. Don’t forget to include key contacts outside your team – such as management, key business units, and outside resources like local law enforcement contacts (even for federal agencies) or an outside incident response firm in case something exceeds your own capabilities. Establish a point of contact, promote it, and staff it: It is truly surprising how many organizations fail to provide contact options for users or other IT staff for when something goes wrong. Set up a few options, including phone/email, make sure someone is always there to respond, and promote them. Many organizations route everything through the help desk, in which case you need to educate them on how to identify a potential incident, when to escalate and how to contact you if something looks big enough that adding it to your ticket queue might be a tad too passive. Liaise with key business units: Lay the groundwork for working with different business units before an incident occurs. Let them know who you are, that you are there to help, and what their responsibilities will be if they get involved in an incident (either because it’s affecting their unit, or because they are an outside resource). Liaise with outside resources: If you are on a large dedicated incident response team this might mean meeting with local federal law enforcement representatives, auditors, and legal advisors. For a smaller organization it might mean researching and interviewing some response and investigation firms in case something exceeds your internal response capability and getting to know the folks so they’ll call you back when you need them. You don’t want to be calling someone cold in the middle of an incident. Document your incident response plan: Even if your plan is a single page with a bullet list of steps, have it in writing ahead of time. Make sure all of the folks with responsibility to act in an incident understand what exactly they need to do. In any incident, checklists are your friends. Train and practice: Ideally run some full scenarios on top of training on tools and techniques. Even if you are a single part-time responder, create some practice scenarios for yourself. And practice. And practice. And practice again. It’s a bad time to find a hole in your process, while you are responding to a real incident. Again, we could write an entire paper on building your incident response infrastructure, but these key elements will get you on the

Read Post

White Paper Release: Monitoring up the Stack

Yep, another white paper is in the can. As you all know, we turn a lot of the research we post on the blog into comprehensive white papers after we gather feedback from the community on our research. You may remember the Monitoring up the Stack series Adrian and Gunnar drove last month, which has now been packaged, edited, and (with the help of our editor Chris Pepper) turned into English. Here is an overview: SIEM and Log Management platforms have seen significant investment, and the evolving nature of attacks means end users are looking for more ways to leverage their security investments. SIEM/Log Management does a good job of collecting data, but extracting actionable information remains a challenge. In part this is due to the “drinking from the fire hose” phenomenon, where the speed and volume of incoming data make it difficult to keep up. Additionally, the data needs to be pieced together with sufficient reference points from multiple event sources to provide context. But we find that the most significant limiting factor is often a network-centric perspective on data collection and analysis. As an industry we look at network traffic rather than transactions; we look at packet density instead of services; we look at IP addresses rather than user identity. We lack context to draw conclusions about the amount of real risk any specific attack presents. The aim of this report is to answer the question: “How can I derive more value from my SIEM installation?” Historically, compliance and operations management have driven investment in SIEM, Log Management, and other complimentary monitoring investments. SIEM can provide continuous monitoring, but most SIEM deployments are not set up to provide timely threat response to application attacks. And we all know that a majority of attacks (whether 60% or 80% doesn’t matter) focus directly on applications. To support more advanced policies and controls we need to peel back the veil of network-oriented analysis and climb the stack, looking at applications and business transactions. In some cases this just means a new way of looking at existing data. But that would be too easy, wouldn’t it? To monitor up the stack effectively, we need to look at how the architecture, policy management, data collection, and analysis of an existing SIEM implementation must change. In this report we tackle all these issues, and some others. A special thanks to ArcSight for sponsoring the report. You can get Monitoring up the Stack: Adding Value to SIEM via our research library, or download the PDF directly. Share:

Read Post

Cool Sidejacking Security Scorecard (and a MobileMe Update)

First, for our non-technical readers who want to know more about this Firesheep/sidejacking thing, check out my relatively non-geeky article over at TidBITS. After that, George Ou put together a great sidejacking security scorecard for a double fistful of major online services. He rates each site’s risk across their various services for full hijacking and full and partial sidejacking. Needless to say, very few services fare well. Being a Mac geek, one service not mentioned is Apple’s MobileMe. I did some poking myself, and MobileMe both uses full-session SSL for all sessions, and sets a secure credential cookie so it won’t pass over basic HTTP. Also, the default for all MobileMe sync services is encrypted connections (I don’t have time to confirm with Wireshark, so I’m currently accepting other articles for that statement). See… a reason Apple should buy Twitter 😉 Share:

Read Post

Totally Transparent Research is the embodiment of how we work at Securosis. It’s our core operating philosophy, our research policy, and a specific process. We initially developed it to help maintain objectivity while producing licensed research, but its benefits extend to all aspects of our business.

Going beyond Open Source Research, and a far cry from the traditional syndicated research model, we think it’s the best way to produce independent, objective, quality research.

Here’s how it works:

  • Content is developed ‘live’ on the blog. Primary research is generally released in pieces, as a series of posts, so we can digest and integrate feedback, making the end results much stronger than traditional “ivory tower” research.
  • Comments are enabled for posts. All comments are kept except for spam, personal insults of a clearly inflammatory nature, and completely off-topic content that distracts from the discussion. We welcome comments critical of the work, even if somewhat insulting to the authors. Really.
  • Anyone can comment, and no registration is required. Vendors or consultants with a relevant product or offering must properly identify themselves. While their comments won’t be deleted, the writer/moderator will “call out”, identify, and possibly ridicule vendors who fail to do so.
  • Vendors considering licensing the content are welcome to provide feedback, but it must be posted in the comments - just like everyone else. There is no back channel influence on the research findings or posts.
    Analysts must reply to comments and defend the research position, or agree to modify the content.
  • At the end of the post series, the analyst compiles the posts into a paper, presentation, or other delivery vehicle. Public comments/input factors into the research, where appropriate.
  • If the research is distributed as a paper, significant commenters/contributors are acknowledged in the opening of the report. If they did not post their real names, handles used for comments are listed. Commenters do not retain any rights to the report, but their contributions will be recognized.
  • All primary research will be released under a Creative Commons license. The current license is Non-Commercial, Attribution. The analyst, at their discretion, may add a Derivative Works or Share Alike condition.
  • Securosis primary research does not discuss specific vendors or specific products/offerings, unless used to provide context, contrast or to make a point (which is very very rare).
    Although quotes from published primary research (and published primary research only) may be used in press releases, said quotes may never mention a specific vendor, even if the vendor is mentioned in the source report. Securosis must approve any quote to appear in any vendor marketing collateral.
  • Final primary research will be posted on the blog with open comments.
  • Research will be updated periodically to reflect market realities, based on the discretion of the primary analyst. Updated research will be dated and given a version number.
    For research that cannot be developed using this model, such as complex principles or models that are unsuited for a series of blog posts, the content will be chunked up and posted at or before release of the paper to solicit public feedback, and provide an open venue for comments and criticisms.
  • In rare cases Securosis may write papers outside of the primary research agenda, but only if the end result can be non-biased and valuable to the user community to supplement industry-wide efforts or advances. A “Radically Transparent Research” process will be followed in developing these papers, where absolutely all materials are public at all stages of development, including communications (email, call notes).
    Only the free primary research released on our site can be licensed. We will not accept licensing fees on research we charge users to access.
  • All licensed research will be clearly labeled with the licensees. No licensed research will be released without indicating the sources of licensing fees. Again, there will be no back channel influence. We’re open and transparent about our revenue sources.

In essence, we develop all of our research out in the open, and not only seek public comments, but keep those comments indefinitely as a record of the research creation process. If you believe we are biased or not doing our homework, you can call us out on it and it will be there in the record. Our philosophy involves cracking open the research process, and using our readers to eliminate bias and enhance the quality of the work.

On the back end, here’s how we handle this approach with licensees:

  • Licensees may propose paper topics. The topic may be accepted if it is consistent with the Securosis research agenda and goals, but only if it can be covered without bias and will be valuable to the end user community.
  • Analysts produce research according to their own research agendas, and may offer licensing under the same objectivity requirements.
  • The potential licensee will be provided an outline of our research positions and the potential research product so they can determine if it is likely to meet their objectives.
  • Once the licensee agrees, development of the primary research content begins, following the Totally Transparent Research process as outlined above. At this point, there is no money exchanged.
  • Upon completion of the paper, the licensee will receive a release candidate to determine whether the final result still meets their needs.
  • If the content does not meet their needs, the licensee is not required to pay, and the research will be released without licensing or with alternate licensees.
  • Licensees may host and reuse the content for the length of the license (typically one year). This includes placing the content behind a registration process, posting on white paper networks, or translation into other languages. The research will always be hosted at Securosis for free without registration.

Here is the language we currently place in our research project agreements:

Content will be created independently of LICENSEE with no obligations for payment. Once content is complete, LICENSEE will have a 3 day review period to determine if the content meets corporate objectives. If the content is unsuitable, LICENSEE will not be obligated for any payment and Securosis is free to distribute the whitepaper without branding or with alternate licensees, and will not complete any associated webcasts for the declining LICENSEE. Content licensing, webcasts and payment are contingent on the content being acceptable to LICENSEE. This maintains objectivity while limiting the risk to LICENSEE. Securosis maintains all rights to the content and to include Securosis branding in addition to any licensee branding.

Even this process itself is open to criticism. If you have questions or comments, you can email us or comment on the blog.