As we wrap up our series on Security Benchmarking, we find it instructive to actually walk through a scenario and apply the process. Yes, the scenario is a bit contrived, but we’ll use it to hit the high points of the process, deciding where to start, collecting the data, establishing the peer group and communicate the findings. Keep in mind that we focus on getting quick wins, showing immediate value, building momentum and leveraging that momentum for programatic success.

Scenario

For our case study, let’s use a mid-tier financial company as our example. I’d say large enterprise, but in reality there are a lot of nuances and moving pieces within a large enterprise that need more detailed discussion. So let’s keep it relatively simple. Likewise, we picked the financial vertical because of 1) need and 2) availability of data. The reality of the financial industries regulatory oversight has created a general perspective of security first and data-centricity (yes, these are the folks that try to do risk management for a living) means these businesses are move likely to embrace a benchmarking mentality.

In our (contrived) scenario, the Board drove the hiring a new CISO to “fix security.” As easy as it is to think this was just catering to a board directive, the senior team seems to have a commitment to fix things and do it the right way. So the CISO has a clear honeymoon period and some leeway in thinking somewhat unconventionally about how to build the security program. The new CISO still spends some time figuring out what’s installed and what’s not working, but he knows the organization has AV deployed, they use an external scanning service, and do a pretty good job of patching on internal systems.

Yet, like many smaller financial institutions they use hosted applications for most of their business processing. So a lot of their data is not within their direct control. Over the past few years, the organization has had a handful of incidents, but none really resulted in major data loss. Thus the CISO was pleasantly surprised when he got the mandate to fix the security program, when it wasn’t outwardly broken. The senior team came to the conclusion they are living on borrowed time and want to act decisively to make sure they are ready when the brown stuff hits the fan (which it inevitably will).

See? We told the you the scenario was contrived, but without a senior-level mandate to make changes in implement a security program, getting any kind of security metrics/benchmarking initiative going will be difficult.

Where Do You Start?

Now the CISO has to figure out where to start. He’s decided that he wants to figure out where his most apparent gaps are. You know, the ones you can drive a Mack Truck through. So he starts with a comprehensive risk assessment to build a baseline, but he also wants to compare his environment to other like-sized companies (both in and out of his industry) to figure out how he compares to those organizations. Keep in mind, boiling the ocean and trying to do everything at this point is a bad idea. He’d get buried in the nuances of the data and not get anything done, which could endanger his entire security program. So he needs to ask the following questions:

• What do you need to achieve? Where are the key operational problems? This is where you always have to start. In our case study, the CISO is looking to identify his most critical gaps, and given the luck they’ve had in not having a huge data loss even with a few breaches, he wants to start with incident response.
• What data do you have? Next you have to figure out if you have the data or can get it easily. With incident data, the reality is the findings from the forensics investigations exist, but haven’t been put in any kind of format for comparison. But the data exists, so it makes sense to keep pressing down this path. If the data doesn’t exist or can’t be gathered quickly, then it’s time to look at Plan B. You don’t want to hold up the effort because it’s all about getting the quick win.
• Where will be most impactful to show comparative data? Selecting to focus initially on incident response represents a pretty shrewd move for the new CISO. He knows the board and senior management is sensitive to not getting nailed, as well as having a set of reasonable consensus metrics available (from CIS), and having the data. This increases the chances of success.

Peer Groups and Service Providers

Next, our CISO has to define the peer group for analysis. This isn’t brain surgery. He’ll need to compare to other financials (duh!), but also companies in other regulated industries (like healthcare and utilities) of a similar size. The good news is there are a ton of mid-sized hospital groups, as well as many community utilities, with similarly sensitive data. But how do they get their hands on that kind of data for comparison purposes?

Now we go back and revisit the selection criteria for any kind of provider you’d think about for benchmarking services. Remember, these folks have to 1) have access to the data you’d need and 2) be able to protect the data you share with them. To be clear, you may not be able to get everything done with just one provider. In our case study here, the CISO will actually pick two. The first is his regional bank ISAC, who has been gathering data from its members for a while. The second is a commercial benchmarking offering, since they have more data about other industries that aren’t the focus of the ISAC. In reality, the CISO would like to just have one provider, but until a critical mass of data for many verticals is captured, he’ll need to piecemeal the solution to solve the problem.

Analyze

Equipped with data regarding his first area of focus (incident response), now the CISO can start to analyze the results to figure out where the biggest gaps in his process. The data shows that the board’s instinct that they’ve been lucky is right on the money. Most organizations in the finance peer group have had more breaches and had significant loss from at least one of the incidents. Additionally, comparably sized companies have also had more breaches, but on the good news front, those comparable incidents took longer to handle by those companies. So what conclusions does the CISO draw? That their program isn’t terrible, but they need to do more to ensure any incidents can be handled faster and more effectively.

Luck is not a strategy, so there are clear areas for improvement. This will involve looking at some enhanced monitoring technologies to detect potential issues faster and a network full packet capture capability to identify root cause and do forensic analysis with greater precision. He also wants to invest in some forensics training for his team, and get a top-notch incident response firm on a small retainer to make sure he knows who to call when something goes down. Equipped with this data, he can make a more compelling case for the new equipment and services. Right, a quick win in the bag.

What’s next?

After finishing up with one area, then he can focus on the next area for metrics/benchmark analysis. Given the consistent data from industry breach reports, he decides to focus on patching effectiveness. He understand that many of the breaches resulting in huge amounts of lost data started with attacks where a patch already existed, but wasn’t applied. Again, the CISO has a patching program in place, but without an idea about how the efforts stack up relatively, he’s shooting in the dark. He could have just as easily decided to think about vulnerability management or firewall operations. It doesn’t matter what he picks, as long as the data aligns with operational issues and represent areas of potential exposure.

So with that, we’ve finished up our series on Security Benchmarking. Again, we are always interested in feedback on our research, so let it fly via a comment on any of the posts. And keep an eye out for the series assembled as a white paper, which should appear over the next couple of weeks.