Security Assurance and Testing: No Surprises

The methods by which applications and supporting infrastructure are developed and deployed are undergoing fundamental change. Avoiding the predictable hyperbole, new methods including DevOps and Cloud Computing promise to disrupt most of IT over the next 5-10 years. But embedded infrastructure and legacy applications are not going away. IT professionals need to walk a fine line between delivering critical services at the lowest price for acceptable performance, and doing it quickly and reliably. As usual, security is at the end of the tail being wagged. It’s hard enough to get developers to run a security scan on their code before it’s deployed into production. The idea of integrating security into these integrated development and operational processes (something Rich calls ‘SecOps’) seems like a pipe dream. It may be a dream today, but it needs to become reality sooner rather than later. IT has little choice. Adversaries continue to innovate and improve their tactics at an unprecedented rate. They have clear missions, typically involving exfiltrating critical information or impacting the availability of your technology resources. They have the patience and resources to achieve their missions by any means necessary. And it’s your job to make sure deployment of new IT resources doesn’t introduce unnecessary risk. With this need to move faster and to have more agile infrastructure, it is increasingly difficult to ensure proper testing for infrastructure and applications before they go live. You have all heard the excuses that emerge when something goes wrong with a deployment. We didn’t hit it with that much traffic. We didn’t get around to testing those edge cases. The application wasn’t designed to do that. Ho hum. Just another day in the office, and it’s security’s problem when the new application is compromised, data is lost, or the application falls over under the onslaught of a denial of service attack. It doesn’t need to be this way. Really – it doesn’t. Although in light of common experience, many security folks don’t believe this. The root cause of these issues is surprise. That’s right – when an application goes live (or a major change goes into production), you don’t really know what is about to happen, do you? You haven’t been through a rigorous process to ensure the application (and its infrastructure) is ready for prime time. And calling the application ‘Beta’ won’t save you. If the application has access to critical (regulated) information and is accessible – whether internally or externally – a security mindset is required, along with a way to put the application through its paces. As I wrote in the Pragmatic CSO, Basically you are trying to eliminate surprises. So by doing a full battery of tests before the new system is deployed, you reduce the likelihood that you are missing something that you’ll learn about later – the hard way. Technological disruption is not about to stop. If anything it will accelerate, so we need to get over our idea of a discrete security function maybe doing some testing and/or risk assessment at the tail end of a project. So what can and should security folks do? And how can they get both the development and operations teams on board with the necessary changes to ensure the protection and survivability of the application? To prevent surprise we suggest a security assurance and testing process for ensuring the environment is ready to cope with real traffic and real attacks. This goes well beyond what development organizations typically do to ‘test’ their applications, or ops does to ‘test’ their stacks. It also is different than a risk assessment or a manual pen test. Those “point in time” assessments look at what can happen but aren’t necessarily comprehensive. The testers may find a bunch of issues but not all the issues. So remediation decisions are made with incomplete information about the true attack surface of infrastructure and applications. So that is the topic of our next blog series, titled Eliminating Surprises with Security Assurance and Testing. We will dig into this process, discussing which devices and infrastructure components to test and how to consistently and reliable ensure you are testing the key functions. We will also focus on assuring the readiness and resilience of applications because they are often the path of least resistance for attacks. We would like to thank Ixia for agreeing to potentially license this content at the end of blog series. We will be developing it objectively, using our Totally Transparent Research methodology. We can provide this research to you at this most excellent price because our clients support our unconventional research model. Remember – your adversaries don’t need to hit an arbitrary deadline. They will take the time needed to find the chinks in your armor. Maybe it’s within the application, maybe it’s within the computing stack, maybe it’s the underlying equipment that gets data from one place to another. You can’t eliminate all the defects and security holes in your environment. But you can find out what they are and put a plan together to protect your environment. Deploying a security assurance and testing process to do just that is what this new series is all about. Share:

Read Post

Incite 12/4/2013: Aging Gracefully

My friend Shimmy must have taken his nostalgia pills over the long weekend – on Monday he tweeted: Doesn’t it suck getting older I didn’t realize how truly carefree life was All is good here thinking about some new stuff Besides the fact that it’s Twitter-english (half sentences/thoughts to fit into 140 characters, punctuation not required), I disagree with that sentiment. I don’t think it sucks getting older. Aging is awesome. I’m not sure I would recognize my 24-year old self if I ran into him on the street. If I take a rare moment to reflect, almost every aspect of my life is better now. My main gripe is that my body is 20 years older, so my knees ache from time to time and it takes me a bit longer to kick a hangover. But on the list of potential issues, those are pretty minor. There is nothing saying that a carefree life is a better life. Or maybe I just never had a carefree life. When I was younger I was always striving. I had a timetable for success and wanted to hit my dates. A few years ago I dropped the timetable. I could do that because I changed my view of success, which is still evolving as I learn more about myself and what I’m really about. To be fair, there are Saturdays I would like to stay in bed until 2pm like I did 20 years ago. And there was something liberating about fitting pretty much all my possessions into a duffle bag or two. I had nothing to lose. But I don’t buy into the notion that having responsibilities (family, kids, expenses) is worse. In fact all I could think about when I had no responsibilities was my timetable to gain them. I searched for a partner and found the Boss. I worked hard at a number of jobs and then stumbled into research. Same old story. Lots of folks think the grass was greener in the past. Or will be greener in the future. They would rather be anywhere else but here. Any other time but now. Which is a shame. All we have is right now. The past is gone. The future hasn’t happened yet. What I want to do is enjoy the time I have, as long as it lasts. To age gracefully like a good single malt (and I don’t even like scotch). To leverage my experience and help people improve. To connect those I value to resources or knowledge I can access. Just thinking about it gets me fired up about the road ahead. But I shouldn’t beat Shimmy up too badly – he got it right in the last part of his tweet. All is good here. It sure is, brother. I wouldn’t trade my experiences, which have been a critical part of the journey. As I said in Live Right Now: “You could choose to live in the past. We need to be respectful of history, and learn the lessons of those that came before us.” I also said, “Think to the future not in fear and worry, but in hope and grace.” I’m choosing to live right now because I am finally old enough to appreciate the challenges of the alternatives. As Steve Jobs would say, this approach allows you to “Stay Hungry. Stay Foolish.” Which seems pretty carefree to me… –Mike Photo credit: “The Maltman Bowmore 21 Years” originally uploaded by Sven Cipido Heavy Research We are back at work on a variety of blog series, so here is a list of the research currently underway. Remember you can get our Heavy Feed via RSS, where you can get all our content in its unabridged glory. And you can get all our research papers too. What CISOs Need to Know about Cloud Computing Adapting Security for Cloud Computing How the Cloud is Different for Security Introduction Defending Against Application Denial of Service Building Protections In Abusing Application Logic Attacking the Application Stack Newly Published Papers Security Awareness Training Evolution Firewall Management Essentials Continuous Security Monitoring API Gateways Threat Intelligence for Ecosystem Risk Management Dealing with Database Denial of Service Identity and Access Management for Cloud Services The 2014 Endpoint Security Buyer’s Guide The CISO’s Guide to Advanced Attackers Incite 4 U Staying focused on the prize: Our pal Wendy posted another terrific rant before the holiday. This time on user feedback in applications. In What’s my name? No, really, what is it?, she talks about how pen testers always gave her a hard time about the feedback given by the login process. With that information, the attacker could infer user IDs, etc. Wendy points out that is by design – if users cannot remember their user names they call the help desk. When they call the help desk it costs money or takes folks away from more important tasks. So yes, you need to balance the obscurity required to make it harder on attackers against the downside of making it harder for legitimate users. Which do you choose? Thought so. She closes with: “If your system can’t withstand attacks by someone who knows a valid username or email address, then you have MUCH bigger problems to solve.” Wendy drops the mic and goes home. – MR Super-unrelated: The PCI DSS 3.0 requirement that firms map the flow of payment card data is really nothing new – identifying what systems contain cardholder data has been part of every DSS specification since the beginning. Mapping the data flow and showing which users and applications have access to that data simply provides a clearer picture of how that data is used so you understand how best to safeguard it. For threat modeling this type of diagram is a must! The key is that it makes the assessor’s job easier to have a map of the systems in scope and subject to review. That does not address the flaw Troy Leach identifies: unknown and unsecured cardholder storage locations

Read Post

Totally Transparent Research is the embodiment of how we work at Securosis. It’s our core operating philosophy, our research policy, and a specific process. We initially developed it to help maintain objectivity while producing licensed research, but its benefits extend to all aspects of our business.

Going beyond Open Source Research, and a far cry from the traditional syndicated research model, we think it’s the best way to produce independent, objective, quality research.

Here’s how it works:

  • Content is developed ‘live’ on the blog. Primary research is generally released in pieces, as a series of posts, so we can digest and integrate feedback, making the end results much stronger than traditional “ivory tower” research.
  • Comments are enabled for posts. All comments are kept except for spam, personal insults of a clearly inflammatory nature, and completely off-topic content that distracts from the discussion. We welcome comments critical of the work, even if somewhat insulting to the authors. Really.
  • Anyone can comment, and no registration is required. Vendors or consultants with a relevant product or offering must properly identify themselves. While their comments won’t be deleted, the writer/moderator will “call out”, identify, and possibly ridicule vendors who fail to do so.
  • Vendors considering licensing the content are welcome to provide feedback, but it must be posted in the comments - just like everyone else. There is no back channel influence on the research findings or posts.
    Analysts must reply to comments and defend the research position, or agree to modify the content.
  • At the end of the post series, the analyst compiles the posts into a paper, presentation, or other delivery vehicle. Public comments/input factors into the research, where appropriate.
  • If the research is distributed as a paper, significant commenters/contributors are acknowledged in the opening of the report. If they did not post their real names, handles used for comments are listed. Commenters do not retain any rights to the report, but their contributions will be recognized.
  • All primary research will be released under a Creative Commons license. The current license is Non-Commercial, Attribution. The analyst, at their discretion, may add a Derivative Works or Share Alike condition.
  • Securosis primary research does not discuss specific vendors or specific products/offerings, unless used to provide context, contrast or to make a point (which is very very rare).
    Although quotes from published primary research (and published primary research only) may be used in press releases, said quotes may never mention a specific vendor, even if the vendor is mentioned in the source report. Securosis must approve any quote to appear in any vendor marketing collateral.
  • Final primary research will be posted on the blog with open comments.
  • Research will be updated periodically to reflect market realities, based on the discretion of the primary analyst. Updated research will be dated and given a version number.
    For research that cannot be developed using this model, such as complex principles or models that are unsuited for a series of blog posts, the content will be chunked up and posted at or before release of the paper to solicit public feedback, and provide an open venue for comments and criticisms.
  • In rare cases Securosis may write papers outside of the primary research agenda, but only if the end result can be non-biased and valuable to the user community to supplement industry-wide efforts or advances. A “Radically Transparent Research” process will be followed in developing these papers, where absolutely all materials are public at all stages of development, including communications (email, call notes).
    Only the free primary research released on our site can be licensed. We will not accept licensing fees on research we charge users to access.
  • All licensed research will be clearly labeled with the licensees. No licensed research will be released without indicating the sources of licensing fees. Again, there will be no back channel influence. We’re open and transparent about our revenue sources.

In essence, we develop all of our research out in the open, and not only seek public comments, but keep those comments indefinitely as a record of the research creation process. If you believe we are biased or not doing our homework, you can call us out on it and it will be there in the record. Our philosophy involves cracking open the research process, and using our readers to eliminate bias and enhance the quality of the work.

On the back end, here’s how we handle this approach with licensees:

  • Licensees may propose paper topics. The topic may be accepted if it is consistent with the Securosis research agenda and goals, but only if it can be covered without bias and will be valuable to the end user community.
  • Analysts produce research according to their own research agendas, and may offer licensing under the same objectivity requirements.
  • The potential licensee will be provided an outline of our research positions and the potential research product so they can determine if it is likely to meet their objectives.
  • Once the licensee agrees, development of the primary research content begins, following the Totally Transparent Research process as outlined above. At this point, there is no money exchanged.
  • Upon completion of the paper, the licensee will receive a release candidate to determine whether the final result still meets their needs.
  • If the content does not meet their needs, the licensee is not required to pay, and the research will be released without licensing or with alternate licensees.
  • Licensees may host and reuse the content for the length of the license (typically one year). This includes placing the content behind a registration process, posting on white paper networks, or translation into other languages. The research will always be hosted at Securosis for free without registration.

Here is the language we currently place in our research project agreements:

Content will be created independently of LICENSEE with no obligations for payment. Once content is complete, LICENSEE will have a 3 day review period to determine if the content meets corporate objectives. If the content is unsuitable, LICENSEE will not be obligated for any payment and Securosis is free to distribute the whitepaper without branding or with alternate licensees, and will not complete any associated webcasts for the declining LICENSEE. Content licensing, webcasts and payment are contingent on the content being acceptable to LICENSEE. This maintains objectivity while limiting the risk to LICENSEE. Securosis maintains all rights to the content and to include Securosis branding in addition to any licensee branding.

Even this process itself is open to criticism. If you have questions or comments, you can email us or comment on the blog.