Research

I often hear that there is no innovation left in security. That’s complete bullshit. There is plenty of innovation in security – but more often than not there’s no market for that innovation. For anything innovative to survive (at least in terms of physical goods and software) it needs to have a market. Sometimes, as with the motion controllers of the Nintendo Wii, it disrupts an existing market by creating new value. In other cases, the innovation taps into unknown needs or desires and succeeds by creating a new market. Security is a bit of a tougher nut. As I’ve discussed before, both on this blog and in the Disruptive Innovation talk I give with Chris Hoff, security is reactive by nature. We are constantly responding to changes in the underlying processes/organizations we protect, as well as to threats evolving to find new pathways through our defenses. With very few exceptions, we rarely invest in security to reduce risks we aren’t currently observing. If it isn’t a clear, present, and noisy danger, it usually finds itself on the back burner. Innovations like firewalls and antivirus really only succeeded when the environment created conditions that showed off value in these tools. Typically that value is in stopping pain, and not every injury causes pain. Even when we are proactive, there’s only a market for the reactive. The pain must pass a threshold to justify investment, and an innovator can only survive for so long without customer investment. Innovation is by definition almost always ahead of the market, and must create its own market to some degree. This is tough enough for cool things like iPads and TiVos, but nearly impossible for something less sexy like security. I love my TiVo, but I only appreciate my firewall. As an example, let’s take DLP. By bringing content analysis into the game, DLP became one of the most innovative, if not the most innovative, data security technologies we’ve seen. Yet 5+ years in, after multiple acquisitions by major vendors, we’re still only talking about a $150M market. Why? DLP didn’t keep your website up, didn’t keep the CEO browsing ESPN during March Madness, and didn’t keep email spam-free. It addresses a problem most people couldn’t see without DLP a DLP tool! Only when it started assisting with compliance (not that it was required) did the market start growing. Another example? How many of you encrypted laptops before you had to start reporting lost laptops as a data breach? On the vendor side, real innovation is a pain in the ass. It’s your pot of gold, but only after years of slogging it out (usually). Sometimes you get the timing right and experience a quick exit, but more often than not you either have to glom onto an existing market (where you’re fighting for your life against competitors that really shouldn’t be your competitors), or you find patient investors who will give you the years you need to build a new market. Everyone else dies. Some examples? PureWire wasn’t the first to market (ScanSafe was) and didn’t get the biggest buyout (ScanSafe again), but they timed it right and were in and out before they had to slog. Fidelis is forced to compete in the DLP market, although the bulk of their value is in managing a different (but related) threat. 7+ years in and they are just now starting to break out of that bubble. Core Security has spent 7 years building a market- something only possible with patient investors. Rumor is Palo Alto has some serious firewall and IPS capabilities, but rather than battling Cisco/Checkpoint, they are creating an ancillary market (application control) and then working on the cross-sell. Most of you don’t buy innovative security products. After paying off your maintenance and licens renewals, and picking up a few widgets to help with compliance, there isn’t a lot of budget left. You tend to only look for innovation when your existing tools are failing so badly that you can’t keep the business running. That’s why it looks like there’s no security innovation – it’s simply ahead of market demand, and without a market it’s hard to survive. Unless we put together a charity fund or those academics get off their asses and work on something practical, we lack the necessary incubators to keep innovation alive until you’re ready to buy it. So the question is… how can we inspire and sustain innovation when there’s no market for it? Or should we? When does innovation make sense? What innovation are we willing to spend on when there’s no market? When and how should we become early adopters? Share:

One of our readers, Jon Damratoski, is putting together a DLP program and asked me for some ideas on metrics to track the effectiveness of his deployment. By ‘ask’, I mean he sent me a great list of starting metrics that I completely failed to improve on. Jon is looking for some feedback and suggestions, and agreed to let me post these. Here’s his list: Number of people/business groups contacted about incidents – tie in somehow with user awareness training. Remediation metrics to show trend results in reducing incidents – at start of DLP we had X events, after talking to people for 30 days about incidents we now have Y events. Trend analysis over 3, 6, & 9 month periods to show how the number of events has reduced as remediation efforts kick in. Reduction in the average severity of an event per user, business group, etc. Trend: number of broken business policies. Trend: number of incidents related to automated business practices (automated emails). Trend: number of incidents that generated automatic email. Trend: number of incidents that were generated from service accounts – (emails, batch files, etc.) I thought this was a great start, and I’ve seen similar metrics on the dashboards of many of the DLP products. The only one I have to add to Jon’s list is: Average number of incidents per user. Anyone have other suggestions? Share:

The lack of credible and relevant network security metrics has been a thorn in my side for years. We don’t know how to define success. We don’t know how to communicate value. And ultimately, we don’t even know what we should be tracking operationally to show improvement (or failure) in our network security activities. But we in the echo chamber seem to be happier bitching about this, or flaming each other on mailing lists, than focusing on finding a solution. Some folks have tried to drive towards a set of metrics that make sense, but I can say most of the attempts are way too academic and also cost too much to collect to be usable in everyday practice. Not to mention that most of our daily activities aren’t even included in the models. Not to pick on them too much, but I think these issues are highlighted in the way the Center for Internet Security has scoped out network security metrics. Basically, they didn’t. They have metrics on Incident Management, Vulnerability Management, Patch Management, Configuration Change Management, Application Security, and Financial Metrics. So the guy managing the network security devices doesn’t count? Again, I know CIS is working towards a lot of other stuff, but the reality is the majority of security spending is targeted at the network and endpoint domains, and there are no good metrics for those. So let’s fix it. Today, we are kicking off the next in our series of Quant projects. This one is called Network Security Operations Quant, and we aim to build a process map and underlying cost model for how organizations manage their network security devices. The project’s formal objective and scope are: The objective of Network Security Operations Quant is to develop a cost model for monitoring and managing network security devices that accurately reflects the associated financial and resource costs. Secondarily, we also want to: Build the model in a manner that supports use as an operational efficiency model to help organizations optimize their network security monitoring and management processes, and compare costs of different options. Heavily engage the community and produce an open model with wide support and credibility, using the Totally Transparent Research process. Advance the state of IT metrics, particularly operational security metrics. We are grateful to our friends at SecureWorks, who are funding this primary research effort. As with all our quant processes, our methodology is: Establish the high level process map via our own research. Use a broad survey to validate and identify gaps in the process map. Define a set of subprocesses for each high-level process. Build metrics for each subprocess. Assemble the metrics into a model which can be used to track operational improvement. From a scoping standpoint, we are going to deal with 5 different network security processes: Monitoring firewalls Monitoring IDS/IPS Monitoring server devices Managing firewalls Managing IDS/IPS Yes, we know network security is bigger than just these 5 functions, but we can’t boil the ocean. There is a lot of other stuff we’ll model out using the Quant process over the next year, but this should be a good start. Put up or shut up We can’t do this alone. So we are asking for your help. First off, we are going to put together a “panel” of organizations to serve as the basis for our initial primary research. That means we’ll be either doing site visits or detailed phone interviews to understand how you undertake network security processes. We’ll also need the folks on the panel to shoot holes in our process maps before they are posted for public feedback. We are looking for about a dozen organizations from a number of different verticals and company sizes (large enterprise to mid-market). As with all our research, there will be no direct attribution to your organization. We are happy to sign NDAs and the like. If you are interested in participating, please send me an email directly at mrothman (at) securosis . com. Once the initial process maps are posted, we will post a survey to find out whether you actually do the steps we identify. We’ll also want your feedback on the process via posts that describe each step in the process. Everyone has an opportunity to participate and we hope you will take us up on it. This is possibly the coolest research project I’ve personally been involved with and I’m really excited to get moving on it. We look forward to your participation, so we finally can get on the same page, and figure out how to measure how we “network security plumbers” do our business. Share: