Where do the policies in your security product come from? With the myriad of tools and security products on the market, where do the pre-built policies come from? I am not speaking of AV in this post- rather looking at IDS, VA, DAM, DLP, WAF, pen testing, SIEM, and many others that use a set of policies to address security and compliance problems. The question is who decides what is appropriate? On every sales engagement, customer and analyst meeting I have ever participated in for security products, this was a question.
This post is intended more for IT professional who are considering security products, so I am gearing for that audience. When drafting the web application security program series last month, a key topic that kept coming up over and over again from security practitioners was: “How can you recommend XYZ security solution when you know that the customer is going to have to invest a lot for the product, but also a significant amount in developing their own policy set?” This is both an accurate observation and the right question to be asking. While we stand by our recommendations for reasons stated in the original series, it would be a disservice to our IT readers if we did not discuss this in greater detail. The answer is an important consideration for anyone selecting a security tool or suite.
When I used to develop database security products, policy development was one of the tougher issues for us to address on the vendor side. Once aware of a threat, on average it took 2.5 ‘man-days’ to develop a policy with a test case and complete remediation information [prior to QA]. This becomes expensive when you have hundreds of policies being developed for different problem sets. It was a common competitive topic to discuss policy coverage and how policies were generated, and a basic function of the product, so most every vendor will invest heavily in this area. More, most vendors market their security ‘research teams’ that find exploits, develop test code, and provide remediation steps. This domain expertise is one of the areas where vendors provide value in the products that they deliver, but when it comes down to it, vendor insight is fraction of the overall source of information. With monitoring and auditing, policy development was even harder: The business use cases were more diverse and the threats not completely understood. Sure we could return the ubiquitous who-what-when-where-to-from kind of stuff, but how did that translate to business need?
If you are evaluating products or interested in augmenting your policy set, where do you start? With vulnerability research, there are several resources that I like to use:
Vendor best practices – Almost every platform vendor, from Apache to SAP, offer security best practices documents. These guidelines on how to configure and operate their product form the basis for many programs. These cover operational issues that reduce risk, discuss common exploits, and reference specific security patches. These documents are updated during each major release cycle, so make sure you periodically review for new additions, or how they recommend new features be configured and deployed. What’s more, while the vendor may not be forthcoming with exploit details, they are the best source of information for remediation and patch data.
CERT/Mitre – Both have fairly comprehensive lists of vulnerabilities to specific products. Both provide a neutral description of what the threat is. Neither had great detailed information of the actual exploit, not will they have complete remediation information. It is up to the development team to figure out the details.
Customer feedback/peer review – If you are a vendor of security products, customer have applied the policies and know what works for them. They may have modified the code that you use to remediate a situation, and that may be a better solution than what your team implemented, and/or it may be too specific to their environment for use in a generalized product. If you are running your own IT department, what have your peers done? Next time you are at a conference or user group, ask. Regardless, vendors learn from other customers what works for them to address issues, and you can too.
3rd party relationships (consultants, academia, auditors) – When it comes to development of policies related to GLBA or SOX, which are outside the expertise of most security vendors, it’s particularly valuable to leverage third party consultative relations to augment policies with their deep understanding of how best to approach the problem. In the past I have used relationships with major consulting firms to help analyze the policies and reports we provided. This was helpful, as they really did tell us when some of our policies were flat out bull$(#!, what would work, and how things could work better. If you have these relationships already in place, carve out a few hours so they can help review and analyze policies.
Research & Experience – Most companies have dedicated research teams, and this is something you should look for. They do this every day and they get really good at it. If your vendor has a recognized expert in the field on staff, that’s great too. That person may be quite helpful to the overall research and discovery process of threats and problems with the platforms and products you are protecting. The reality is that they are more likely on the road speaking to customers, press and analysts rather than really doing the research. It is good that your vendor has a dedicated team, but their experience is just one part of the big picture.
User groups – With many of the platforms, especially Oracle, I learned a lot from regional DBAs who supported databases within specific companies or specific verticals. In many cases they did not have or use a third party product, rather they had a bunch of scripts that they had built up over many years, modified, and shared with others. They shared tips on not only what they were required to do, but how they implemented them. This typically included the trial-and-error discussion of how a certain script or policy was evolved over time to meet timeliness or completeness of information requirements from other team members. Use these groups and attend regional meetings to get a better idea of how peers solve problems. Amazing wealth of knowledge, freely shared.
General frameworks – To meet compliance efforts, frameworks commonly provide checklists for compliance and security. The bad news is that the lists are generic, but the good news is they provides a good start for understanding what you need to consider, and help prepare for pre-vendor engagements and POCs.
Compliance – Polices are typically created to manage compliance with existing policies or regulations. Compliance requirements allow some latitude for how you interpret how a PCI or FISMA applies to your organization. What works, how it is implemented, what the auditors find suitable, and what is easy for them to use all play a part in the push & pull of policy development, and one of the primary reasons to consider this effort as added expense to deploying third party products.
I want to stress that you should use this as a guide to review the methods that product vendors use to develop their policies, but my intention is to make sure you clearly understand that you will need to develop your own as well. In the case of web application security, it’s you application, and it will be tough to avoid. This post may help you dig through vendor sales and marketing literature to determine what can really help to you and what is “pure puffery”, but ultimately you need to consider the costs of developing your own policies for the products you choose. Why? You can almost never find off-the-shelf polices that meet all of your needs. Security or compliance may not be part of your core business, and you may not be a domain expert in all facets of security, but for certain key areas I recommend that you invest in supplementing the off-the-shelf policies included with your security tools. Policies are best if they are yours, grounded in your experience, and tuned to your organizational needs. They provide historical memory, and form a knowledge repository for other company members to learn from. Policies can guide management efforts, assurance efforts, and compliance efforts. Yes, this is work, and potentially a lot of work paid in increments over time. If you do not develop your own policies, and this type of effort is not considered within your core business, then you are reliant on third parties (service providers or product vendors) for the production of your policies.
Hopefully you will find this helpful.
Reader interactions
3 Replies to “Policies and Security Products”
@M. Wallace – I am being vague on defining ‘‘policy’‘. There might be subtle variants in the definition depending upon the audience (security, compliance, internal audit, etc), and I am attempting to cover multiple audiences. Let me offer a couple of examples:
Security – I employ Web App Firewall to restrict request method length and parameter value length. I may be responding to a CERT advisory, and my internal SLA’s say I will respond to known critical threats with X number of working days. I need to understand the web application well enough to know what lengths are used so policy enforcement by the WAF does not break the web app.
Compliance – I deploy a SIEM or DAM tool to verify end of period adjustments to accounts receivables. I am motivated by SOX, and have created internal policies to reflect this. I am checking who does what and when, and I may even check from where. But the policy that I want enforced has to know where the data is, what applications are used and by which accounts. Policy enforcement must have some sense of what my organization’s approved accounting practices are.
Operational Policy – I check all of my databases for default passwords, demonstration accounts and code, and the allowance for external stored procedures. I am motivated by my policies for security, compliance and consistency of operation. My organization (probably) has operational guidelines for database configuration and management, and I created assessment policies to verify compliance.
Much to your point, I would not want anyone implementing IDS without an understanding of my network and common usage of the network. And really, I don’‘t want to do all of the research either (I can tell you from personal experience as it is really time consuming). And since it is not your core job, you don’‘t want to have to understand it all. On the other hand, for a vendor of a security product to create and maintain policies for you will require additional service, and it will be expensive. So that’s where the balancing act comes in. I rely on CERT advisories so I am aware there is a threat. I do not fully verify that there is a threat or what an exploit might be, and you may not be able to afford to have the vendors understand you internal operations enough to customize policies. To be effective you need custom policies.
Too many threats for me to know and research everything, but I do need to adjust rules, policies or patches to counter the threat in a way that does not break my applications. I need to know what the workarounds and solutions are. Rich and I try to analyze a lot of the threats and vendor patches to describe cases where you need to really understand and make adjustments vs. lower risk cases you can verify through normal patching cycles as to not require a great deal of investment of your time. We hope this is helpful and why I was motivated to write the post.
Addressing your last point, I have created a generic threat template that could in turn reference specific rules an policies for different security products. What I mean by this is that I typically do not respond to any threat in one way, but I may patch, alter configuration, monitor and block all at the same time. I described it in XML format, the with an overview section, and specific rules attached for firewall/Monitoring/VA. I cannot post as it belongs to a former employer. There are others who have been working on a standardized threat templates, but I have yet to see any of these unified efforts make it to market in any meaningful way.
-Adrian
[…] The post on Policies and Security Products: Something you need to consider in any security product investment. Favorite Outside […]
Might be useful to describe policy in more detail.
For my IDS, I don’‘t want policy developed by anyone who doesn’‘t have a fairly intimate knowledge of my network. Development of that policy is part of the defensive effort to ensure that I understand the network better than the adversary.
For the kind of vulnerability stuff you reference, it makes sense to acquire test cases from external parties. I don’‘t want to do that research – it distracts me from core job.
By CERT/MITRE I infer that you mean http://cce.mitre.org/ (et. al.)?
What would an open standard building on CCE/CVE but including the test cases you want look like?