Where do the policies in your security product come from? With the myriad of tools and security products on the market, where do the pre-built policies come from? I am not speaking of AV in this post- rather looking at IDS, VA, DAM, DLP, WAF, pen testing, SIEM, and many others that use a set of policies to address security and compliance problems. The question is who decides what is appropriate? On every sales engagement, customer and analyst meeting I have ever participated in for security products, this was a question.

This post is intended more for IT professional who are considering security products, so I am gearing for that audience. When drafting the web application security program series last month, a key topic that kept coming up over and over again from security practitioners was: “How can you recommend XYZ security solution when you know that the customer is going to have to invest a lot for the product, but also a significant amount in developing their own policy set?” This is both an accurate observation and the right question to be asking. While we stand by our recommendations for reasons stated in the original series, it would be a disservice to our IT readers if we did not discuss this in greater detail. The answer is an important consideration for anyone selecting a security tool or suite.

When I used to develop database security products, policy development was one of the tougher issues for us to address on the vendor side. Once aware of a threat, on average it took 2.5 ‘man-days’ to develop a policy with a test case and complete remediation information [prior to QA]. This becomes expensive when you have hundreds of policies being developed for different problem sets. It was a common competitive topic to discuss policy coverage and how policies were generated, and a basic function of the product, so most every vendor will invest heavily in this area. More, most vendors market their security ‘research teams’ that find exploits, develop test code, and provide remediation steps. This domain expertise is one of the areas where vendors provide value in the products that they deliver, but when it comes down to it, vendor insight is fraction of the overall source of information. With monitoring and auditing, policy development was even harder: The business use cases were more diverse and the threats not completely understood. Sure we could return the ubiquitous who-what-when-where-to-from kind of stuff, but how did that translate to business need?

If you are evaluating products or interested in augmenting your policy set, where do you start? With vulnerability research, there are several resources that I like to use:

Vendor best practices – Almost every platform vendor, from Apache to SAP, offer security best practices documents. These guidelines on how to configure and operate their product form the basis for many programs. These cover operational issues that reduce risk, discuss common exploits, and reference specific security patches. These documents are updated during each major release cycle, so make sure you periodically review for new additions, or how they recommend new features be configured and deployed. What’s more, while the vendor may not be forthcoming with exploit details, they are the best source of information for remediation and patch data.

CERT/Mitre – Both have fairly comprehensive lists of vulnerabilities to specific products. Both provide a neutral description of what the threat is. Neither had great detailed information of the actual exploit, not will they have complete remediation information. It is up to the development team to figure out the details.

Customer feedback/peer review – If you are a vendor of security products, customer have applied the policies and know what works for them. They may have modified the code that you use to remediate a situation, and that may be a better solution than what your team implemented, and/or it may be too specific to their environment for use in a generalized product. If you are running your own IT department, what have your peers done? Next time you are at a conference or user group, ask. Regardless, vendors learn from other customers what works for them to address issues, and you can too.

3rd party relationships (consultants, academia, auditors) – When it comes to development of policies related to GLBA or SOX, which are outside the expertise of most security vendors, it’s particularly valuable to leverage third party consultative relations to augment policies with their deep understanding of how best to approach the problem. In the past I have used relationships with major consulting firms to help analyze the policies and reports we provided. This was helpful, as they really did tell us when some of our policies were flat out bull$(#!, what would work, and how things could work better. If you have these relationships already in place, carve out a few hours so they can help review and analyze policies.

Research & Experience – Most companies have dedicated research teams, and this is something you should look for. They do this every day and they get really good at it. If your vendor has a recognized expert in the field on staff, that’s great too. That person may be quite helpful to the overall research and discovery process of threats and problems with the platforms and products you are protecting. The reality is that they are more likely on the road speaking to customers, press and analysts rather than really doing the research. It is good that your vendor has a dedicated team, but their experience is just one part of the big picture.

User groups – With many of the platforms, especially Oracle, I learned a lot from regional DBAs who supported databases within specific companies or specific verticals. In many cases they did not have or use a third party product, rather they had a bunch of scripts that they had built up over many years, modified, and shared with others. They shared tips on not only what they were required to do, but how they implemented them. This typically included the trial-and-error discussion of how a certain script or policy was evolved over time to meet timeliness or completeness of information requirements from other team members. Use these groups and attend regional meetings to get a better idea of how peers solve problems. Amazing wealth of knowledge, freely shared.

General frameworks – To meet compliance efforts, frameworks commonly provide checklists for compliance and security. The bad news is that the lists are generic, but the good news is they provides a good start for understanding what you need to consider, and help prepare for pre-vendor engagements and POCs.

Compliance – Polices are typically created to manage compliance with existing policies or regulations. Compliance requirements allow some latitude for how you interpret how a PCI or FISMA applies to your organization. What works, how it is implemented, what the auditors find suitable, and what is easy for them to use all play a part in the push & pull of policy development, and one of the primary reasons to consider this effort as added expense to deploying third party products.

I want to stress that you should use this as a guide to review the methods that product vendors use to develop their policies, but my intention is to make sure you clearly understand that you will need to develop your own as well. In the case of web application security, it’s you application, and it will be tough to avoid. This post may help you dig through vendor sales and marketing literature to determine what can really help to you and what is “pure puffery”, but ultimately you need to consider the costs of developing your own policies for the products you choose. Why? You can almost never find off-the-shelf polices that meet all of your needs. Security or compliance may not be part of your core business, and you may not be a domain expert in all facets of security, but for certain key areas I recommend that you invest in supplementing the off-the-shelf policies included with your security tools. Policies are best if they are yours, grounded in your experience, and tuned to your organizational needs. They provide historical memory, and form a knowledge repository for other company members to learn from. Policies can guide management efforts, assurance efforts, and compliance efforts. Yes, this is work, and potentially a lot of work paid in increments over time. If you do not develop your own policies, and this type of effort is not considered within your core business, then you are reliant on third parties (service providers or product vendors) for the production of your policies.

Hopefully you will find this helpful.