Monitoring the Hybrid Cloud: Technical Considerations

New platforms for hybrid cloud monitoring bring both new capabilities and new challenges. We have already discussed some differences between monitoring the different cloud models, and some of the different deployment options available. This post will dive into some technical considerations for these new hybrid platforms, highlighting potential benefits and issues for data security, privacy, scalability, security analytics, and data governance. As cool as a ‘CloudSOC’ sounds, there are technical nuances which need to be factored into your decision and selection processes. There are also data privacy issues because some types of information fall under compliance and jurisdictional regimes. Cloud computing and service providers can provide an opportunity to control infrastructure costs more effectively, but service models costs are calculated differently that on-premise systems, so you need to understand the computing and storage characteristics of the SOC platform in detail to understand where you are spending money. Let’s jump into some key areas where you need to focus. Data Security As soon as event data is moved out of one ‘cloud’ such as say Salesforce into another, you need to consider the sensitivity of the data, which forces a decision on how to handle security. Using SSL or similar technology to secure the data in motion is the easy part – what to do with the data at rest, once it reaches the CloudSOC, is far more challenging. You can get some hints from folks who have already grappled with this question: security monitoring providers. These services either build their own private clouds to accommodate and protect client data, or leverage yet another IaaS or PaaS cloud to provide the infrastructure to store the data. Many of you will find the financial and scalability advantages of storing cloud data in a cloud services more attractive than moving all that collected data back to an on-premise system. Regardless of whether you build your own CloudSOC or use a managed service, a key part of your security strategy will be the Service Level Agreements (SLAs) you establish with your providers. These agreements specify the security controls implemented by the provider, and if something is not specified in that agreement the provider has no obligation to provide it. An SLA is a good place to start, but be wary of unspecified areas – those are where gaps are most likely emerge. A good place to start is a comparison of what the provider does with what you do internally today. We recommend you ask questions and get clear answers on every topic you don’t understand because once you execute the agreement you have no further leverage to negotiate. And if you are running your own make sure you carefully plan out your cloud security model to take advantage of what your IaaS provider offers. You may decide some data is too sensitive to be stored in the cloud without obfuscation (encryption) or removal (typically redaction, tokenization, or masking). Data Privacy and Jurisdiction Over and above basic data security for logs and event data, some countries have strict laws about how Personally Identifiable Information (PII) data may be collected and stored, and some even require that PII not leave its country of origin – even encrypted. If you do business in these countries your team likely already understands the regulations today, but for a hybrid SOC deployment you also need to understand the locations of your primary and backup cloud data centers, and their regional laws as well. This can be incredibly confusing – particularly when data protection laws conflict between countries. Once you understand the requirements and where your cloud (including CloudSOC) providers are located, you can effectively determine which security controls you need. Once again data encryption addresses many legal requirements, and data masking and tokenization services can remove sensitive data without breaking your applications or impairing security analytics. The key is to know where the data will be stored to figure out the right mix of controls. Automation and Scalability If you have ever used Dropbox or Salesforce or Google Docs, you know how easy it is to store data in the cloud. When you move beyond SaaS to PaaS and IaaS, you will find it is just as easy to spin up whole clusters of new applications and servers with a few clicks. Security monitoring, deploying collectors, and setting up proxies for traffic filtering, all likewise benefit from the cloud’s ease of use and agility. You can automate the deployment of collectors, agents, or other services; or agents can be embedded in the start-up process for new instances or technology stacks. Verification and discovery of services running in your cloud can be performed with a single API call. Automation is a hallmark of the cloud so you can script pretty much anything you need. But getting started with basic collection is a long way from getting a CloudSOC into production. As you move to a production environment you will be constructing and refining initialization and configuration scripts to launch services, and defining templates which dictate when collectors or analytics instances are spun up or shut down via the magic of autoscaling. You will be writing custom code to call cloud APIs to collect events, and writing event filters if the API does not offer suitable options. It is basically back to the future, hearkening back to the early days of SIEM when you spent as much time writing and tuning collectors as analyzing data. Archiving is also something you ll need to define and implement. The cloud offers very granular control of which data gets moved from short-term to long-term storage, and when. In the long run cloud models offer huge benefits for automation and on-demand scalability, but there are short-term set-up and tuning costs to get a CloudSOC working the way you need. A managed CloudSOC service will do much of this for you, at additional cost. Other Considerations Management Plane: The management plane for cloud services is a double-edged sword; IT admins now have the power to automate

Read Post

Totally Transparent Research is the embodiment of how we work at Securosis. It’s our core operating philosophy, our research policy, and a specific process. We initially developed it to help maintain objectivity while producing licensed research, but its benefits extend to all aspects of our business.

Going beyond Open Source Research, and a far cry from the traditional syndicated research model, we think it’s the best way to produce independent, objective, quality research.

Here’s how it works:

  • Content is developed ‘live’ on the blog. Primary research is generally released in pieces, as a series of posts, so we can digest and integrate feedback, making the end results much stronger than traditional “ivory tower” research.
  • Comments are enabled for posts. All comments are kept except for spam, personal insults of a clearly inflammatory nature, and completely off-topic content that distracts from the discussion. We welcome comments critical of the work, even if somewhat insulting to the authors. Really.
  • Anyone can comment, and no registration is required. Vendors or consultants with a relevant product or offering must properly identify themselves. While their comments won’t be deleted, the writer/moderator will “call out”, identify, and possibly ridicule vendors who fail to do so.
  • Vendors considering licensing the content are welcome to provide feedback, but it must be posted in the comments - just like everyone else. There is no back channel influence on the research findings or posts.
    Analysts must reply to comments and defend the research position, or agree to modify the content.
  • At the end of the post series, the analyst compiles the posts into a paper, presentation, or other delivery vehicle. Public comments/input factors into the research, where appropriate.
  • If the research is distributed as a paper, significant commenters/contributors are acknowledged in the opening of the report. If they did not post their real names, handles used for comments are listed. Commenters do not retain any rights to the report, but their contributions will be recognized.
  • All primary research will be released under a Creative Commons license. The current license is Non-Commercial, Attribution. The analyst, at their discretion, may add a Derivative Works or Share Alike condition.
  • Securosis primary research does not discuss specific vendors or specific products/offerings, unless used to provide context, contrast or to make a point (which is very very rare).
    Although quotes from published primary research (and published primary research only) may be used in press releases, said quotes may never mention a specific vendor, even if the vendor is mentioned in the source report. Securosis must approve any quote to appear in any vendor marketing collateral.
  • Final primary research will be posted on the blog with open comments.
  • Research will be updated periodically to reflect market realities, based on the discretion of the primary analyst. Updated research will be dated and given a version number.
    For research that cannot be developed using this model, such as complex principles or models that are unsuited for a series of blog posts, the content will be chunked up and posted at or before release of the paper to solicit public feedback, and provide an open venue for comments and criticisms.
  • In rare cases Securosis may write papers outside of the primary research agenda, but only if the end result can be non-biased and valuable to the user community to supplement industry-wide efforts or advances. A “Radically Transparent Research” process will be followed in developing these papers, where absolutely all materials are public at all stages of development, including communications (email, call notes).
    Only the free primary research released on our site can be licensed. We will not accept licensing fees on research we charge users to access.
  • All licensed research will be clearly labeled with the licensees. No licensed research will be released without indicating the sources of licensing fees. Again, there will be no back channel influence. We’re open and transparent about our revenue sources.

In essence, we develop all of our research out in the open, and not only seek public comments, but keep those comments indefinitely as a record of the research creation process. If you believe we are biased or not doing our homework, you can call us out on it and it will be there in the record. Our philosophy involves cracking open the research process, and using our readers to eliminate bias and enhance the quality of the work.

On the back end, here’s how we handle this approach with licensees:

  • Licensees may propose paper topics. The topic may be accepted if it is consistent with the Securosis research agenda and goals, but only if it can be covered without bias and will be valuable to the end user community.
  • Analysts produce research according to their own research agendas, and may offer licensing under the same objectivity requirements.
  • The potential licensee will be provided an outline of our research positions and the potential research product so they can determine if it is likely to meet their objectives.
  • Once the licensee agrees, development of the primary research content begins, following the Totally Transparent Research process as outlined above. At this point, there is no money exchanged.
  • Upon completion of the paper, the licensee will receive a release candidate to determine whether the final result still meets their needs.
  • If the content does not meet their needs, the licensee is not required to pay, and the research will be released without licensing or with alternate licensees.
  • Licensees may host and reuse the content for the length of the license (typically one year). This includes placing the content behind a registration process, posting on white paper networks, or translation into other languages. The research will always be hosted at Securosis for free without registration.

Here is the language we currently place in our research project agreements:

Content will be created independently of LICENSEE with no obligations for payment. Once content is complete, LICENSEE will have a 3 day review period to determine if the content meets corporate objectives. If the content is unsuitable, LICENSEE will not be obligated for any payment and Securosis is free to distribute the whitepaper without branding or with alternate licensees, and will not complete any associated webcasts for the declining LICENSEE. Content licensing, webcasts and payment are contingent on the content being acceptable to LICENSEE. This maintains objectivity while limiting the risk to LICENSEE. Securosis maintains all rights to the content and to include Securosis branding in addition to any licensee branding.

Even this process itself is open to criticism. If you have questions or comments, you can email us or comment on the blog.