Research

Given the evolution of SIEM technology and the security challenges facing organizations, it is time to revisit requirements and use cases. This is an essential part of the evaluation process. You need a fresh and critical look at your security management environment to understand what you need today, how that will change tomorrow, and what kinds of resources and expertise you can harness – unconstrained by your current state. While some requirements may not have changed all that much (such as ease of management and compliance reporting), as we described earlier in this series, the way we use these systems has changed dramatically. That is our way of saying it is good to start with a laundry list of things you would like to be able to do, but cannot with your current system. And while you are thinking about shiny new capabilities, don’t forget the basic day-to-day operational stuff a SIEM does. Finally, you need to consider what is coming down the road in terms of business challenges (and the resulting security issues) that will emerge over the next couple years. None of us has a crystal ball, but critical business imperatives should give you a foundation for figuring out how things need to change. A fresh start Some organizations choose to take a fresh look at their security management infrastructure every so often, while others have the choice thrust upon them. For instance if your organization was breached or infected by malware and your SIEM platform failed to detect it, you need to take a critical look at your security management environment. The current platform may be adequate, or it might be a dog – and you personally might not have even chosen it – but keep in mind that your success is linked to how well your platform meets your requirements. If things go south, blaming your predecessor for choosing a mediocre SIEM won’t save your job. You also need to face the reality that other groups within the organization have differing needs for the SIEM. Operations only cares that they get the metrics they need, compliance teams only care about getting their reports, and upper management only cares about pushing blame downhill when the company is pwned by hackers. It’s time to roll up your sleeves and figure out what you need. Every so often it makes sense to critically look at what works and what doesn’t from the standpoint of security management. To find out the best path forward, we recommend starting with the proverbial blank slate. It is helpful to consider the your priorities when you selected the system in the first place, to illuminate how your environment has changed over time and help understand the amount of change to expect in the future. To be more specific, use this opportunity to revisit the priorities of your requirements and use cases for each of the three main drivers for security management spending: improving security, increasing efficiency, and automating compliance. It’s all about me Setting requirements is all about you, right? It’s about what you need to get done. It’s about how you want to work. It’s about what you can’t do – at least easily – today. Well, not quite. We jest to make a point: you need to start with a look inward at what your company needs – rather than getting distracted by what the market is offering today. This requires taking a look at your organization, and the other internal teams that use the SIEM. Once your team is clear abou your own requirements, start to discuss requirements with external influencers. Assuming you work in security, you should consult ops teams, business users, compliance, and perhaps the general counsel, about their various requirements. This should confirm the priorities you established earlier, and set the stage for enlisting support if you decide to move to a new platform. Our research has shown that organizational needs remain constant, as mentioned above: improve security, improve efficiency, and support compliance. But none of these goals has gotten easier. The scale of the problem has grown, so if you have stood still and have not added new capabilities… you actually lost ground. For example, perhaps you first implemented a Log Management capability to crank out compliance reports. We see that as a common initial driver. But as your organization grew and you did more stuff online, you collected more events, and now need a much larger platform to aggregate and analyze all that data. Or perhaps you just finished cleaning up a messy security incident which your existing SIEM missed. If so you probably now want to make sure correlation and monitoring work better, and that you have some kind of threat intelligence to help you know what to look for. Increasingly SIEM platforms monitor up the stack – collecting additional data types including identity, Database Activity Monitoring, application support, and configuration management. That additional data helps isolate infrastructure attacks, but you cannot afford to stop there. As attacks target higher-level business processes and the systems that automate them, you will need visibility beyond core infrastructure. So your security management platform needs to detect attacks in the context of business threats. Don’t forget about advanced forensics – it would be folly to count on blocking every attack. So you will probably rely on your security management platform to help React Faster and Better with incident response. You might also be looking for a more integrated user experience across a number of security functions to improve efficiency. For example you might have separate vendors for change detection, vulnerability management, firewall and IDS monitoring, and Database Activity Monitoring. You may be wearing out your swivel chair switching between all those consoles, and simplification through vendor consolidation might be a key driver as you revisit your requirements. Don’t be hung up on what you have – figure out what you need now. Do a little thinking about what would make your life a lot easier, and use those ideas

Hey everyone. It’s a new year and time for new stuff from your pals here at Securosis. We used to run a Monday-morning ‘Firestarter’ post to get people thinking for the week. We decided to revive it with a twist. We are restarting the Firestarter as a weekly short video (15 minutes or so is our target). As we work out the details we also plan to push it out as a podcast, and once every month or so we will run a longer episode to dig deeper into a topic. We pre-recorded this version, but as you’ll see we ran it on Google Hangouts. When possible, we will post our recording times up there so you can participate (sorry, via text only) as we record. This week we decided to cover the NSA/RSA controversy and the… interesting… decision by some to pull out of the RSA conference over it. Share:

As you may have noticed, I haven’t been blogging much the past month or so. 2013 has been an… interesting… year, filled with personal and professional highs and lows. Our third child was born, and we were back in the thick of things with 3 kids aged four and under. Don’t even get me stared on the near nonstop string of minor illnesses. There’s nothing like stomach flu twice in a month. Once on a travel day – thus the last month of minimal blogging. On the other hand my son took his first steps just shy of 7 months old, I had one of my best business years yet (yep, Securosis grew once again), and experienced a career highlight by presenting my first technical Black Hat session on 90 minutes notice (including a technical demo). Work-wise I am having more fun than I have in many years – largely due to spending more time hands-on with programming and the cloud. This is the kind of defensive technical security research I have always wanted to drive forward. And the bonus? My daughters love super heroes, robots, and science (plus princesses and the color pink – I’m not trying to turn them into boys or anything). All this is possible because of you reading the site. Every day thousands of you read our content (or point your bots at us), and a fair few hire us to help you with research and analysis. We aren’t a billion-dollar company with a big name – just a couple handfuls of full and part time security obsessives who get the opportunity to do what we love, and enjoy an amazing lifestyle with our families in the process. We never, not for an instant, forget who we serve and who makes this all possible. We only hope we meet our end of the bargain by providing you something useful. We have a lot of new things lined up for 2014. The nature of our work is pulling us in new and exciting directions, and we try to adapt every year to the ever-changing landscape of social media and online tools we use to engage and deliver content. Some of it works and some doesn’t – that’s life. As 2013 comes to a close, I just wanted to personally thank you all. (You know, with a heartfelt generic blog post.) And I would like to also thank our contributors… some of whom get paid, and others who keep us on our toes with their insights. Mort, Pepper, Gattaca, Myrcurial, Gunnar, and Gal. Plus Mike and Adrian – the best two partners I could ask for. Or not ask for – they actually sort of just showed up, uninvited. Share:

This post discusses evolutionary changes in SIEM, focusing on how underlying platform capabilities have evolved to meet the requirements discussed in the last post. To give you a sneak peek, it is all about doing more with more data. The change we have seen in these platforms over the past few years has been mostly under the covers. It’s not sexy, but this architectural evolution was necessary to make sure the platforms scaled and could perform the needed analysis moving forward. The problem is that most folks cannot appreciate the boatload of R&D which has been required to enable many platforms to receive a proverbial brain transplant. We will start with the major advancements. Architectural Evolution To be honest, we downplayed the importance of SIEM’s under-the-hood changes in our previous paper. The “brain transplant” was the significant change that enabled a select few vendors to address the performance and scalability issues plaguing the first generation of the platforms, which were built on RDBMS. For simplicity’s sake we skipped over the technical details of how and why. Now it’s time to explore that evolution. The fundamental change is that SIEM platforms are no longer based on a centralized massive service. By leveraging a distributed approach, using a cooperative cluster of many servers independently collecting, digesting, and processing events, policies are distributed across multiple systems to more effectively and efficiently handle the load. If you need to support more locations or pump in a bunch more data, just add nodes to the cluster. If this sounds like big data that’s because it essentially is. Several platforms leverage big data technologies under the hood. The net result is parallel event processing resources deployed ‘closer’ to event sources, faster event collection, and systems designed to scale without massive reconfiguration. This architecture enables different deployment models; it also better accommodates distributed IT systems, cloud providers, and virtual environments – which increasingly constitute the fabric of modern technology infrastructure. The secret sauce making it all possible is distributed system management. It is easy to say “big data”, but much harder to do heavy-duty security analysis at scale. Later, when we discuss proof-of-concept testing and final decision-making, we will explore substantiating these claims. The important parts, though, are the architectural changes to enable scaling and performance, and support for more data sources. Without this shift nothing else matters. Serving Multiple Use Cases The future of security management is not just about detecting the advanced threats and malware, although that is the highest-profile use case. We still need to get work done today, which means adding value to the operations team, as well as to compliance and security functions. This typically involves analyzing vulnerability assessment information so security teams can ensure basic security measures are in place. You can analyze patch and configuration data similarly to help operations teams keep pace within dynamic – and increasingly virtual – environments. We have even seen cases where operations teams detected application DoS attacks through infrastructure event data. This kind of derivative security analysis is the precursor to allowing risk and business analytics teams to make better business decisions – to redeploy resources, take applications offline, etc. – by leveraging data collected from the SIEM. Enhanced Visibility Attackers continually shift their strategies to evade detection, increase efficiency, and maximize the impact of their attacks. Historically one of SIEM’s core value propositions has been an end-to-end view, enabled by collecting all sorts of different log files from devices all around the enterprise. Unfortunately that turned out not to be enough – log files and NetFlow records rarely contain enough information to detect or fully investigate an attack. We needed better visibility into what is actually happening within the environment – rather than expecting analysts to wade through zillions of event records to figure out when you are under attack. We have seen three technical advances which, taken together, provide the evolutionary benefit of much better visibility into the event stream. In no particular order they are more (and better) data, better analysis techniques, and better visualization capabilities. More and Better Data: Collect application events, full packet capture – not just metadata – and other sources that taxed older SIEM systems. In many cases the volume or format of the data was incompatible with the underlying data management engine. Better Analysis: These new data sources enable more detailed analysis, longer retention, and broader coverage; together those improved capabilities provide better depth and context for our analyses. Better Visualization: Enhanced analysis, combined with advanced programmatic interfaces and better visualization tools, substantially improves the experience of interrogating the SIEM. Old-style dashboards, with simplistic pie charts and bar graphs, have given way to complex data representations that much better illuminate trends and highlight anomalous activity. These improvements might look like simple incremental improvements to existing capabilities, but combined they enable a major improvement in visibility. Decreased Time to Value The most common need voiced by SIEM buyers is to have their platforms provide value without requiring major customization and professional services. Customers are tired of buying SIEM toolkits, and then needing to take time and invest money to build a custom SIEM system tailored to their particular environment. As we mentioned in our previous post, collecting an order of magnitude more data requires a similar jump in analysis capabilities – the alternative is to be drowned in a sea of alerts. The same math applies to deployment and management – monitoring many more types of devices and analyzing data in new ways means platforms need to be easier to deploy and manage simply to maintain the old level of manageability. The good news is that SIEM platform vendors have made significant investments to support more devices and offer better installation and integration, which combined make deployment less resource intensive. As these platforms integrate the new data sources and enhanced visibility described above, the competitiveness of a platform can be determined by the simplicity and intuitiveness of its management interface, and the availability of out-of-the-box policies and

Today’s post discusses the changing needs and requirements organizations have for security management customers, which is just a fancy way of saying “Here’s why customers are unhappy.” The following items are the main discussion points when we speak with end users, and the big picture reasons motivating SIEM users to consider alternatives. The Changing Needs of Security Management Malware/Threat Detection: Malware is by far the biggest security issue enterprises face today. It is driving many of the changes rippling through the security industry, including SIEM and security analytics. SIEM is designed to detect security events, but malware is designed to be stealthy and evade detection. You may be looking for malware, but you don’t always know what it looks like. Basically you are hunting for anomalies that kinda-sorta could be an attack, or just odd stuff that may look like an infection. The days of simple file-based detection are gone, or at least anything resembling the simple malware signature of a few years ago. You need to detect new and novel forms of advanced malware, which requires adding different data sources to the analysis and observing patterns across different event types. We also need to leverage emerging security analytics capabilities to examine data in new and novel ways. Even if we do all this, it might still not be enough. This is why feeding third-party threat intelligence feeds into the SIEM are becoming increasingly common – allowing organizations to look for attacks happening to others. Cloud & Mobile: As firms move critical data into cloud environments and offer mobile applications to employees and customers, the definition of system now encompasses use cases outside the classical corporate perimeter, changing the definition and scope of infrastructure to monitor. Compounding the issue is the difficulty in monitoring mobile devices – many of which you do not fully control, and it’s even harder because of the lack of effective tools to gather telemetry and metrics from the devices. Even more daunting is the lack of visibility (basically log and event data) of what’s happening within your cloud service providers. Some cloud providers cannot provide infrastructure logs to customers, because their event streams combine events from all their customers. In some cases they cannot provide logs because there is simply no ‘network’ to tap because it’s virtual, and existing data collectors are useless. In other cases the cloud provider is simply not willing to share the full picture of events, and you may be prohibited contractually from capturing events. The net result is that you need to tackle security monitoring and event analysis in a fundamentally different fashion. This typically involves collecting the events you can gather (application, server, identity and access logs) and massaging them into your SIEM. For Infrastructure as a Service (IaaS) environments, you should look at adding your own cloud-friendly collectors in the flow of application traffic. General Analytics: If you collect a mountain of data from all IT systems, much more information is available than just security events. This is a net positive, but cuts both ways – some event analysis platforms are set up for IT operations first, and both security and business operations teams piggyback off that investment. In this case analysis, reporting, and visualization tools must be not just accessible to a wider audience (like security), but also optimized to do true correlation and analysis. What Customers Really Want These examples are what we normally call “use cases”, which we use to reflect the business drivers creating the motivation to take action. These situations are significant enough (and they are unhappy enough) for customers to consider jettisoning current solutions and going through the pain of re-evaluating their requirements and current deficiencies. Those bullet points do represent the high-level motivations, but they fail to tell the whole story. These are the business reasons firms are looking, but they fail to capture why many of the current platforms fail to meet expectations. For that we need to take a slightly more technical look at the requirements. Deeper Analysis Requires More Data To address the use cases described above, especially for malware analysis, more data is required. This necessarily means more event volume – such as capturing and storing full packet streams, if even for only a short period. It also means more types of data – such as human-readable data mixed in with machine logs and telemetry from networks and other devices. It includes gathering and storing complex data types; such as binary or image files; which are not easy to parse, store, or even categorize. The Need for Information Requires Better and More Flexible Analysis Simple correlation of events – who, what, and when – is insufficient for the kind of security analysis required today. This is not only because those attributes are insufficient to distinguish bad from good, but also because data analysis approaches are fundamentally evolving. Most customers we speak with want to profile normal traffic and usage; this profile helps understand how systems are being used, and also helps detect anomalies likely to indicate misuse. There is some fragmentation in how customers use analysis – some choose to leverage SIEM for more real-time altering and analysis; others want big picture visibility, created by combining many different views for an overall sense of activity. Some customers want fully automated threat detection, while others want more interactive ad hoc and forensic analysis. To make things even harder for the vendors, today’s hot analysis methods could very well be irrelevant a year or two down the road. Many customers want to make sure they can update analytics as requirements develop – optimized hard-wired analytics are now a liability rather than an advantage for the products. The Velocity of Attacks Requires Threat Intelligence When we talk about threat intelligence we do not limit the discussion to things like IP reputation or ‘fingerprint’ hashes of malware binaries – those features are certainly in wide use, but the field of threat intelligence includes far more. Some threat intelligence feeds look at

Endpoint protection has become the punching bag of security. Every successful attack seems to be blamed on a failure of endpoint protection. Not that this is totally unjustified – most solutions for endpoint protection have failed to keep pace with attackers. In our 2014 Endpoint Security Buyers Guide, we discussed many of the issues around endpoint hygiene and mobility. We also explored the human element underlying many of attacks, and how to prepare your employees for social engineering attacks in Security Awareness Training Evolution. But realistically, hygiene and awareness won’t deter an advanced attacker long. We frequently say advanced attackers are only advanced as they need to be – they take the path of least resistance. But the converse is also true. When this class of adversaries needs advanced techniques they use them. Traditional malware defenses such as antivirus don’t stand much chance against a zero-day attack. So our new series, Advanced Endpoint and Server Protection, will dig into protecting devices against advanced attackers. We will highlight a number of new alternatives for preventing and detecting advanced malware, and examine new techniques and tools to investigate attacks and search for indicators of compromise within your environment. But first let’s provide some context for what has been happening with traditional endpoint protection because you need to understand the current state of AV technology for perspective on how these advanced alternatives help. AV Evolution Signature-based AV no longer works. Everyone has known that for years. It is not just because blocking a file you know is bad isn’t enough any more. But there are simply too many bad files to, and new ones crop up too quickly, for it to be possible to compare every file against a list of bad files. The signature-based AV algorithm still works as well as it ever did, but it is no longer even remotely adequate. Nor is it comprehensive enough to catch the varying types of attacks in the wild today. So the industry adapted, focusing on broadening the suite of endpoint protection technologies to include host intrusion prevention, which blocks known-bad actions at the kernel level. The industry also started sharing information across its broad customer base to identify IP addresses known to do bad things, and files which contain embedded malware. That shared information is known as threat intelligence, and can help you learn from attacks targeting other organizations. Endpoint security providers also keep adding modules to their increasingly broad and heavy endpoint protection suites. Things like server host intrusion prevention, patch/configuration management, and even full application white listing – all attempting to ensure no unauthorized executables run on protected devices. To be fair, the big AV vendors have not been standing still. They are adapting and working to broaden their protection to keep pace with attackers. But even with all their tools packaged together, it cannot be enough. It’s software and it will never be perfect or defect-free. Their tools will always be vulnerable and under attack. We need to rethink how we do threat management as an industry, in light of these attacks and the cold hard reality that not all of them can be stopped. We have been thinking about what the threat management process will come to look like. We presented some ideas in the CISO’s Guide to Advanced Attackers, but that was focused on what needs to happen to respond to an advanced attack. Now we want to document a broader threat management process, which we will refine through 2014. Threat Management Reimagined Threat management is a hard concept to get your arms around. Where does it start? Where does it end? Isn’t threat management really just another way of describing security? Those are hard questions without absolute answers. For the purposes of this research, threat management is about dealing with an attack. It’s not about compliance, even though most mandates are responses to attacks that happened 5 years ago. It’s not really about hygiene – keeping your devices properly configured and patched is good operational practices, not tied to a specific attack. It’s not about finding resources to actually execute on these plans, nor is it an issue of communicating the value of the security team. Those are all responsibilities of the broader security program. Threat management is a subset of the larger security program – typically the most highly visible capability. So let’s explain how we think about threat management (for the moment, anyway) and let you pick it apart. Assessment: You cannot protect what you don’t know about – that hasn’t changed. So the first step is gaining visibility into all devices, data sources, and applications that present risk to your environment. And you need to understand the current security posture of anything to protect. Prevention: Next you try to stop an attack from being successful. This is where most of the effort in security has been for the past decade, with mixed (okay, lousy) results. A number of new tactics and techniques are modestly increasing effectiveness, but the simple fact is that you cannot prevent every attack. It has become a question of reducing your attack surface as much as practical. If you can stop the simplistic attacks, you can focus on the more advanced ones. Detection: You cannot prevent every attack, so you need a way to detect attacks after they get through your defenses. There are a number of different options for detection – most based on watching for patterns that indicate a compromised device. The key is to shorten the time between when the device is compromised and when you discover it has been compromised. Investigation: Once you detect an attack you need to verify the compromise and understand what it actually did. This typically involves a formal investigation, including a structured process to gather forensic data from devices, triage to determine the root cause of the attack, and searching to determine how broadly the attack has spread within your environment. Remediation: Once you understand what happened you can put a

Just in case you had nothing else to do during the holiday season, you can check out our latest research on Application Denial of Service Attacks. This paper continues our research into Denial of Service attacks after last year’s Defending Against Denial of Service Attacks research. As we stated back then, DoS encompasses a number of different tactics, all aimed at impacting the availability of your applications or infrastructure. In this paper we dig much deeper into application DoS attacks. For good reason – as the paper says: These attacks require knowledge of the application and how to break or game it. They can be far more efficient than just blasting traffic at a network, requiring far fewer attack nodes and less bandwidth. A side benefit of requiring fewer nodes is simplified command and control, allowing more agility in adding new application attacks. Moreover, the attackers often take advantage of legitimate application features, making defense considerably harder. We expect a continued focus on application DoS attacks over time, so we offer both an overview of the common types of attacks you will see and possible mitigations for each one. After reading this paper you should have a clear understanding of how your application availability will be attacked – and more importantly, what you can do about it. We would like to thank our friends at Akamai for licensing this content. Without the support of our clients our open research model wouldn’t be possible. To sum up, here are some thoughts on defense: Defending against AppDoS requires a multi-faceted approach that typically starts with a mechanism to filter attack traffic, either via a web protection service running in the cloud or an on-premise anti-DoS device. The next layer of defense includes operational measures to ensure the application stack is hardened, including timely patching and secure configuration of components. Finally, developers must play their part by optimizing database queries and providing sufficient input validation to make sure the application itself cannot be overwhelmed using legitimate capabilities. Keeping applications up and running requires significant collaboration between development, operations, and security. This ensures not only that sufficient defenses are in place, but also that a well-orchestrated response maintains and/or restores service as quickly as possible. It is not a matter of if but when you are targeted by an Application Denial of Service attack. Check out the landing page for the paper, or you can download the Defending Against Application Denial of Service Attacks PDF directly. Share:

Security Information and Event Management (SIEM) systems create a lot of controversy with security folks; they are one of the cornerstones on which the security program are built upon within every enterprise. Yet, simultaneously SIEM generates the most complaints and general angst. Two years ago Mike and I completed a research project on “SIEM 2.0: Time to Replace your SIEM?” based upon a series of conversations with organizations who wanted more from their investment. Specifically they wanted more scalability, easier deployment, and the ability to ‘monitor up the stack’ in context of business applications and better integration with enterprise systems (like identity). Over the past two years the pace of customer demands and platform evolution to meet those demands has accelerated. What we thought was the tail end of a trend with second-generation SIEMs improving scalability using purpose-built data stores turned out to be the tip of the iceberg. As enterprises wanted to analyze more types of data, from more sources, with more – re: better – analysis capabilities to derive better information to keep pace with advanced attackers. Despite solid platform upgrades from a number of SIEM vendors, these requirements have blossomed faster than their vendor could respond. And sadly, some security vendors marketed “advanced capabilities” when it was really the same old pig in a new suit, causing further chagrin and disappointment amongst their customers. Whatever the reason, here we are two years later, listening to the same tale from customers looking to replace their SIEM (again) given these new requirements. You may feel like Bill Murray in Groundhog Day, reliving the past over and over again, but this time is different. The requirements have changed! Actually they have. The original architects of the early SIEM platforms could not have envisioned the kind of analysis required to detect attacks designed to evade SIEM tools. The attackers are thinking differently, and that means the defenders that want to keep pace need to rip up their old playbook and very likely critically evaluate their old tools as well. Malware is now the major driver, but since you can’t really detect advanced attacks anymore based on a file signature, you have to mine data for security information in a whole new way. Cloud computing and mobile devices are disrupting the technology infrastructure. And the collection and analysis of these and many other data streams (like network packet capture) are bursting the seams of SIEM. It doesn’t just stop at security alerting either. Other organizations, from IT operations to risk to business analytics, also want to mine the security information collected looking for new ways to streamline operations, maintain availability, and optimize the environment. Moving forward, you’ll need to heavily leverage your investments in security monitoring and analysis technologies. If that resource can’t be leveraged, enterprises will move on and find something more in line with their requirements. Given the rapid evolution we’ve seen in SIEM/Log Management over the past 4-5 years, product obsolescence is a genuine issue. The negative impact of a product that has not kept pace with technical evolution and customer requirements cannot be trivialized. This pain becomes more acute in the event of a missed security incident because the SIEM did not collect the requisite information, or worse, could not detect the threat. Customers spend significant resources (both time and money) on the care and feeding of their SIEM. If they don’t feel the value is in alignment with the investment, again they’ll move on and search for better, easier, and faster products. It’s realistic, if not expected, that these customers start questioning whether the incumbent offering makes sense for their organization moving forward. Additionally, firms are increasingly considering managed services and 3rd party security operations providers to address skills and resource shortages within internal groups. Firms simply don’t have the internal expertise to look for advanced threats. This skills gap also promises to reshape the landscape of security management, so we’ll kick off the series discussing these factors, setting the stage to update our guide to selecting a SIEM. Specifically, we will cover the following topics: The Changing Needs of Security Management: As firms branch into cloud environments and offer mobile applications to their employees and customers, the definition of ‘system’ now encompasses use cases outside what’s long been considered the corporate perimeter, changing the view of “infrastructure” that needs to be monitored. Simultaneously, advanced malware attacks now requires more types of data, threat intelligence and polices to adequately detect these attacks. Additionally, firms are increasingly considering managed services and 3rd party security operations to address skills and resource shortages within internal groups. All of these factors are once again reshaping the landscape of security management, so we’ll kick off the series discussing these factors to set the stage for re-evaluating the security management platform. Evolution of SIEM Platform (and Technology): Next we’ll discuss the evolutionary changes in SIEM – from the standpoint of platform capabilities. It’s still all about more data and more data. We’ll cover architectural evolution, integration and ongoing care and feeding of the environment to meet the scaling requirements. We will also discuss how SIEM is increasingly leveraging other data sources, such as virtual servers, mobile events, big data analytics, threat feeds, as well as human and machine generated data. But all of this data does nothing if you don’t have the capabilities to do something with it, so we will discuss new analysis techniques and updates to older approaches that yield better results faster. To do more with more means, under the covers, scale and performance are being achieved via virtualizing lower cost commodity hardware, leveraging new data storage and data management architectures. SIEM remains the aggregation point for operations and security data, but the demands on the platform to ‘do more with more data’ is pushing the technical definition of SIEM forward and spawning necessary hybrid models to meet the requirements. Revisiting Your Requirements: Given the evolution of both the technology and the attacks, it’s time to revisit your specific requirements and

We started this Security Assurance and Testing (SA&T) series with the need for testing and which tactics make sense within an SA&T program. But it is always helpful to see how the concepts apply to more tangible situations. So we will now show how the SA&T program can provide a quick win for the security team, with two (admittedly contrived) scenarios that show how SA&T can be used – both at the front end of a project, and on an ongoing basis, to ensure the organization is well aware of its security posture. Infrastructure Upgrade For this first scenario let’s consider an organization’s move to a private cloud environment to support a critical application. This is a common situation these days. The business driver is better utilization of data center resources and more agility for deploying hardware resources to meet organizational needs. Obviously this is a major departure from the historical rack and provision approach. This is attractive to organizations because it enables better operational orchestration, allowing for new devices (‘instances’ in cloud land) to be spun up and taken down automatically according to the application’s scalability requirements. The private cloud architecture folks aren’t totally deaf to security, so some virtualized security tools are implemented to enforce network segmentation within the data center and block some attacks from insider threats. Without an SA&T program you would probably sign off on the architecture (which does provide some security) and move on to the next thing on your list. There wouldn’t be a way to figure out whether the environment is really secure until it went live, and then attackers will let you know quickly enough. Using SA&T techniques you can potentially identify issues at the beginning of implementation, saving everyone a bunch of heartburn. Let’s enumerate some of the tests to get a feel for what you may find: Infrastructure scalability: You can capture network traffic to the application, and then replay it to test scalability of the environment. After increasing traffic into the application, you might find that the cloud’s auto-scaling capability is inadequate. Or it might scale a bit too well, spinning up new instances too quickly, or failing to take down instances quickly enough. All these issues impact ability and value of the private cloud to the organization, and handling them properly can save a lot of heartburn for Ops. Security scalability: Another infrastructure aspect you can test is its security – especially virtualized security tools. By blasting the environment with a ton of traffic, you might discover your virtual security tools crumble rather than scaling – perhaps because VMs lack custom silicon – and fall over. This failure normally either “fails open”, allowing attacks, or “fails closed”, impacting availability. You may need to change your network architecture to expose your security tools only to the amount of traffic they can handle. Either way, better to identify a potential bottleneck before it impairs either availability or security. A quick win for sure. Security evasion: You can also test security tools to see how they deal with evasion. If the new tools don’t use the same policy as the perimeter, which has been tuned to effectively deal with evasion, the new virtual device may require substantial tuning to ensure security within the private cloud. Network hopping: Another feature of private clouds is their ability to define network traffic flows and segmentation – “Software Defined Networks”. But if the virtual network isn’t configured correctly, it is possible to jump across logical segments to access protected information. Vulnerability testing of new instances: One of the really cool (and disruptive) aspects of cloud computing is elimination of the need for changing/tuning configurations and patching. Just spin up a new instance, fully patched and configured correctly, move the workload over, and take down the old one. But if new instances spin up with vulnerabilities or poor configurations, auto-scaling is not your friend. Test new instances on an ongoing basis to ensure proper security. Again, a win if something was amiss. As you see, many things can go wrong with any kind of infrastructure upgrade. A strong process to find breaking points in the infrastructure before going live can mitigate much of the deployment risk – especially if you are dealing with new equipment. Given the dynamic nature of technology you will want to make sure you are testing the environment on an ongoing basis, as well ensuring that change doesn’t add unnecessary attack surface. This scenario points out where many issues can be found. What happens if you can’t find any issues? Does that impact the value of the SA&T program? Actually, if anything, it enhances its value – by providing peace of mind that the infrastructure is ready for production. New Application Capabilities To dig into another scenario, let’s move up the stack a bit to discuss how SA&T applies to adding new capabilities within an application serving a large user community, to enable commerce on a web site. Business folks like to sell stuff, so they like these kinds of new capabilities. This initiative involves providing access to a critical data store previously inaccessible directly from an Internet-facing application, which is an area of concern. The development team has run some scans against the application to identify application layer issues such as XSS, and fixed them before deployment by front-ending the application with a WAF. So a lot of the low-hanging fruit of application testing is gone. But that shouldn’t be the end of testing. Let’s look into some other areas which could uncover issues by focusing on realistic attack patterns and tactics: Attack the stack: You could use a slow HTTP attack to see if the application can defend against availability attacks on the stack. These attacks are very hard to detect at the network layer so you need to make sure the underlying stack is configured to deal with them. Shopping cart attack: Another type of availability attack uses the application’s legitimate functionality against it. It’s a bit like

As we discussed in the introduction to this Security Assurance & Testing (SA&T) series, it is increasingly hard to adequately test infrastructure and applications before they go into production. But adversaries have the benefit of being able to target the weakest part of your environment – whatever it may be. So the key to SA&T is to ensure you are covering the entire stack. Does that make the process a lot more detailed and complex? Absolutely, but you can’t be sure what will happen when facing real attackers without a comprehensive test. To discuss tactics we will consider how you would test your network and then your applications. We will also discuss testing exfiltration because preventing critical data from leaving your environment disrupts the Data Breach Triangle. Testing Network Security Reading security trade publications you would get the impression that attackers only target applications nowadays, and don’t go after weaknesses in network or security equipment. Au contraire – attackers find the path of least resistance, whatever it is. So if you have a weak firewall ruleset or an easy-to-evade WAF and/or IPS, that’s where they go. Advanced attackers are only as advanced as they need to be. If they can get access to your network via the firewall, evade the IPS, and then move laterally by jumping across VLANs… they will. And they can leave those 0-days on the shelf until they really need them. So you need to test any device that sees the flow of data. That includes network switches, firewalls, IDS/IPS, web application firewalls, network-based malware detection gear, web filters, email security gateways, SSL VPN devices, etc. If it sees traffic it can be attacked, and it probably will be, and you need to be ready. So what should you actually test for network and security devices? Scalability: Spec sheets may be, uh, inaccurate. Even if there is a shred of truth in the spec sheet, it may not be applicable to your configuration or application traffic. Make sure the devices will stand up to real traffic at the peak traffic volumes you will see. Additionally, with increasingly common denial of service attacks, ensuring your infrastructure can withstand a volumetric attack is integral to maintaining availability of your infrastructure. Evasion: Similarly, if a network security device can be evaded, it doesn’t matter how scalable or effective it is at blocking attacks it catches. So you’ll want to ensure you are testing for standard evasion tactics. Reconfiguration: Finally, if the device can be reconfigured and unauthorized policy changes accepted, your finely-tuned and secure policy isn’t worth much. So make sure your devices cannot be accessed except by an authorized party using acceptable policy management. Application Layer Once you are confident the network and security devices will hold up, move on to testing the application layer. Here are the highlights: Profile inbound application traffic: You do this to understand your normal volumes, protocols, and destinations. Then you can build some scenarios that would represent not normal use of the application to test edge cases. Be sure to capture actual application traffic so you can hide the attack within it, the way real attackers will. Application protection: You will also want to stress test the WAF and other application protections using standard application attack techniques – including buffer overflows, application fuzzing, cross-site scripting, slow HTTP, and other denial of service tactics that target applications. Again, the idea is to identify the breaking points of the application before your adversaries do. The key aspect of this assurance and testing process is to make sure you test as much of the application as you can. Similar to a Quality Assurance testing harness used by developers, you want to exercise as much of the code as you can to ensure it will hold up. Keep in mind that adversaries usually have time, so they will search every nook and cranny of your application to find its weak spot. Thus the need for comprehensive testing. Exfiltration The last aspect of your SA&T process is to see if you can actually get data out. Unless the data can be exfiltrated, it’s not really a breach, per se. Here you want to test your content filtering capabilities, including DLP, web filters, email security, and any other security controls that inspect content on the way out. Similar to the full code coverage approach discussed above, you want to make sure you are trying to exfiltrate through as many applications and protocols as possible. That means all the major social networks (and some not-so-major ones) and other logical applications like webmail. Also ensure the traffic is sent out encrypted, because attackers increasingly use multiple layers of encryption to exfiltrate data. Finally, test the feasibility of establishing connections with command and control (C&C) networks. These ‘callbacks’ identify compromised devices, so you will want to make sure you can detect this traffic before data is exfiltrated. This can involve sending traffic to known bad C&C nodes, as well as using traffic patterns that indicate domain generating algorithms and other automated means of finding bot controllers. The SA&T Program Just as we made the case for continuous security monitoring to provide a way to understand how ongoing (and never-ending) changes in your environment must be monitored to assess their impact on security posture, you need to think about SA&T from an ongoing rather than opposed to one-time perspective. In order to really understand how effective your controls will be, you need to implement an SA&T program. Frequency The first set of decisions for establishing your program concerns testing frequency. The underlying network/security equipment and computing infrastructure tends not to change that often so you likely can get away with testing these components less frequently – perhaps quarterly. But if your environment has constant infrastructure changes, or you don’t control your infrastructure (outsourced data center, etc.) you may want to test more often. Another aspect of testing frequency is planning for ad hoc tests. These involve defining a set of catalysts that