Recently on the Heavy Feed

Since we post most of the content for our blog series on the Heavy Feed (get it via the web or RSS), every so often we like to post links to our latest missives on the main feed. Within the next 10 days we’ll be wrapping both our Fact-based Network Security and Security Management 2.0 series. As always, we love feedback, discussion, dissension and the occasional troll to add comments, so fire away. We look forward to your participation. Fact-based Network Security Metrics and the Pursuit of Prioritization Defining ‘Risk’ Outcomes and Operational Data Operationalizing the Facts Compliance Benefits Security Management 2.0: Is it time to replace your SIEM? Time to Replace Your SIEM? (new series) Platform Evolution Revisiting Requirements Platform Evaluation, Part 1 Platform Evaluation, Part 2 Vendor Evaluation – Culling the Short List Vendor Evaluation – Driving the PoC Share:

Read Post

Security Management 2.0: Making the Decision

It’s time – you are ready. You have done the work, including revisiting your requirements, evaluating your current platform in terms of your current and emerging requirements, assessing new vendors/platforms to develop a short list and run a comprehensive proof of concept. Now it’s time to make the call. We know this is an important decision – we are here because your first attempt at this project wasn’t as successful as it needed to be. So let’s break down the decision to ensure you can make a good recommendation and feel comfortable with it. That’s actually a good point to discuss. The output of our Security Management 2.0 process is not really a decision – it’s more of a recommendation. That’s the reality – the final decision will likely be made in the executive suite. That’s why we have focused so much on gathering data (quantitative where possible) – you will need to defend your recommendation until the purchase order is signed. And probably more afterwards. We won’t mince words. This decision generally isn’t about the facts – especially since there is an incumbent in play, which is likely part of a big company that may have important relationships with heavies in your shop. So you need your ducks in a row and a compelling argument for any change. But that’s still only part of the decision process. In many cases, the (perceived) failure of your existing SIEM is self-inflicted. So we also need to evaluate and explain the causes of the failed project, with assurance that they will be addressed and avoided this time. If not, your successor will be in the same boat in another 2-3 years. So before you put your neck on the chopping block and advocate for a change (if that’s what you decide), do some deep internal analysis as well. Introspection The first thing is to make sure you really re-examined the existing platform in terms of the original goals. Did your original goals adequately map your needs at the time, or was there stuff you did not expect? How have your goals changed over time? Be honest! This is not the time to let your ego get in the way of doing what’s right, and you need a hard and fresh look at the decision to ensure you don’t repeat previous mistakes. Did you kick off this process because you were pissed at the original vendor? Or because they got bought and seemed to forget about the platform? Do you know what it will take to get the incumbent to where it needs to be – or whether that is even possible? Is it about throwing professional services at the issues? Is there a fundamental technology problem? Remember, there are no right or wrong answers here, but the truth will become clear when you need to sell this to management. Some of you may be worried that management will look at the need for replacement as ‘your fault’ for choosing the incumbent, so make sure you have answers to these questions and that you aren’t falling into a self-delusion trap. You need your story straight and your motivations clear. Did you assess the issues critically the first time around? If it was a skills issue, have you addressed it? Can your folks build and maintain the platform moving forward? Or are you looking at a managed service to take that concern off the table? If it was a resource problem, do you now have enough staff for proper care and feeding? Yes, the new generation of platforms requires less expertise to keep operational, but don’t be naive – no matter what any sales rep says, you cannot simply set and forget them. Whatever you pick will require expertise to deploy, manage, tune, and analyze reports. These platforms are not self-aware – not by a long shot. The last thing you want to do is set yourself up for failure, so make sure you ask the right questions ahead of time and be honest about the answers. Expectations The next main aspect of the decision is reconciling your expectations with reality. Revisiting requirements provides information on what you need the security management platform to do. You should be able to prioritize the specific use cases (compliance, security, forensics, operations), and have a pretty good feeling about whether the new platform or incumbent will be able to meet your expectations. Remember, not everything is Priority #1, so pick your top three must-have items, and prioritize the requirements. If you are enamored with some new features of the challenger(s), will your organization be able to leverage them? Firing off alerts faster may not be helpful if your team takes a week to investigate any issues, or cannot keep up with the increased demand. The new platform’s ability to look at application and database traffic doesn’t matter if the developers won’t help you understand normal behavior to build the rule set. Fancy network flow analysis can be a productivity sink if your DNS and directory infrastructure is a mess and you can’t reliably map IP to user ID. Or does your existing product have too many features? Yes, it does happen that some organizations simply cannot take advantage of (or even handle) complex multi-variate correlation across the enterprise. Do you need to aggregate logs because organizational politics, or your team’s resources or skill set, prevent you from getting the job done? This might be a good reason to outsource or use a managed service. There isn’t a right or a wrong answer here, only the answer. And not being honest about that answer will land you in the hotseat again. If you kickstarted this effort because the existing product missed something and it resulted in a breach, can you honestly say the new thing would (not ‘might’) detect that attack? We have certainly seen high profile breaches result in tossing the old and bringing in the new (someone has to pay, after all), but make sure you

Read Post

Totally Transparent Research is the embodiment of how we work at Securosis. It’s our core operating philosophy, our research policy, and a specific process. We initially developed it to help maintain objectivity while producing licensed research, but its benefits extend to all aspects of our business.

Going beyond Open Source Research, and a far cry from the traditional syndicated research model, we think it’s the best way to produce independent, objective, quality research.

Here’s how it works:

  • Content is developed ‘live’ on the blog. Primary research is generally released in pieces, as a series of posts, so we can digest and integrate feedback, making the end results much stronger than traditional “ivory tower” research.
  • Comments are enabled for posts. All comments are kept except for spam, personal insults of a clearly inflammatory nature, and completely off-topic content that distracts from the discussion. We welcome comments critical of the work, even if somewhat insulting to the authors. Really.
  • Anyone can comment, and no registration is required. Vendors or consultants with a relevant product or offering must properly identify themselves. While their comments won’t be deleted, the writer/moderator will “call out”, identify, and possibly ridicule vendors who fail to do so.
  • Vendors considering licensing the content are welcome to provide feedback, but it must be posted in the comments - just like everyone else. There is no back channel influence on the research findings or posts.
    Analysts must reply to comments and defend the research position, or agree to modify the content.
  • At the end of the post series, the analyst compiles the posts into a paper, presentation, or other delivery vehicle. Public comments/input factors into the research, where appropriate.
  • If the research is distributed as a paper, significant commenters/contributors are acknowledged in the opening of the report. If they did not post their real names, handles used for comments are listed. Commenters do not retain any rights to the report, but their contributions will be recognized.
  • All primary research will be released under a Creative Commons license. The current license is Non-Commercial, Attribution. The analyst, at their discretion, may add a Derivative Works or Share Alike condition.
  • Securosis primary research does not discuss specific vendors or specific products/offerings, unless used to provide context, contrast or to make a point (which is very very rare).
    Although quotes from published primary research (and published primary research only) may be used in press releases, said quotes may never mention a specific vendor, even if the vendor is mentioned in the source report. Securosis must approve any quote to appear in any vendor marketing collateral.
  • Final primary research will be posted on the blog with open comments.
  • Research will be updated periodically to reflect market realities, based on the discretion of the primary analyst. Updated research will be dated and given a version number.
    For research that cannot be developed using this model, such as complex principles or models that are unsuited for a series of blog posts, the content will be chunked up and posted at or before release of the paper to solicit public feedback, and provide an open venue for comments and criticisms.
  • In rare cases Securosis may write papers outside of the primary research agenda, but only if the end result can be non-biased and valuable to the user community to supplement industry-wide efforts or advances. A “Radically Transparent Research” process will be followed in developing these papers, where absolutely all materials are public at all stages of development, including communications (email, call notes).
    Only the free primary research released on our site can be licensed. We will not accept licensing fees on research we charge users to access.
  • All licensed research will be clearly labeled with the licensees. No licensed research will be released without indicating the sources of licensing fees. Again, there will be no back channel influence. We’re open and transparent about our revenue sources.

In essence, we develop all of our research out in the open, and not only seek public comments, but keep those comments indefinitely as a record of the research creation process. If you believe we are biased or not doing our homework, you can call us out on it and it will be there in the record. Our philosophy involves cracking open the research process, and using our readers to eliminate bias and enhance the quality of the work.

On the back end, here’s how we handle this approach with licensees:

  • Licensees may propose paper topics. The topic may be accepted if it is consistent with the Securosis research agenda and goals, but only if it can be covered without bias and will be valuable to the end user community.
  • Analysts produce research according to their own research agendas, and may offer licensing under the same objectivity requirements.
  • The potential licensee will be provided an outline of our research positions and the potential research product so they can determine if it is likely to meet their objectives.
  • Once the licensee agrees, development of the primary research content begins, following the Totally Transparent Research process as outlined above. At this point, there is no money exchanged.
  • Upon completion of the paper, the licensee will receive a release candidate to determine whether the final result still meets their needs.
  • If the content does not meet their needs, the licensee is not required to pay, and the research will be released without licensing or with alternate licensees.
  • Licensees may host and reuse the content for the length of the license (typically one year). This includes placing the content behind a registration process, posting on white paper networks, or translation into other languages. The research will always be hosted at Securosis for free without registration.

Here is the language we currently place in our research project agreements:

Content will be created independently of LICENSEE with no obligations for payment. Once content is complete, LICENSEE will have a 3 day review period to determine if the content meets corporate objectives. If the content is unsuitable, LICENSEE will not be obligated for any payment and Securosis is free to distribute the whitepaper without branding or with alternate licensees, and will not complete any associated webcasts for the declining LICENSEE. Content licensing, webcasts and payment are contingent on the content being acceptable to LICENSEE. This maintains objectivity while limiting the risk to LICENSEE. Securosis maintains all rights to the content and to include Securosis branding in addition to any licensee branding.

Even this process itself is open to criticism. If you have questions or comments, you can email us or comment on the blog.