Based on the discussion in our first post, you have decided to move toward a managed security monitoring service. Awesome! That was the easy part. Now you need to figure out what kind of deployment model makes sense, and then do the hard work of actually selecting the best service provider for you.

That’s an important distinction to get straight up front. Vendor selection is about your organization. We know it can be easier to just go with a brand name. Or a name in the right quadrant to pacify senior management. Or the cheapest option. But none of those might be the best choice for your requirements. So the selection process requires an open mind and doing the work. You may end up with the brand name. Or the cheapest one. But at least you’ll know you got the best fit.

Deployment Options

The deployment decision really comes down to two questions:

1. Who owns the security monitoring platform? Who buys the monitoring platform? Is it provided as part of a service, or do you have to buy it up front? Who is in charge of maintenance? Who pays for upgrades? What about scaling up? Are you looking at jumping onto a multi-tenant monitoring platform, property of your service provider?
2. Where is the SOC? Who staffs it? The other key question concerns operation of the security monitoring platform. Is the central repository and console on your premises? Does it run in your service provider’s data center? Does it run in the cloud? Who fields the staff, especially if some part of the platform will run at your site?

To break down the chart above, here are the options, which depend on how you answered the questions above:

1. Traditional: The customer buys and operates the security monitoring platform. Alternatively the provider might buy the platform and charge the customer monthly, but that doesn’t affect operations. Either way the monitoring platform runs on the customer premises, staffed by the customer. This is not managed security monitoring.
2. Hybrid: The customer owns the monitoring platform, which resides on-premise at the customer, but the service provider manages it. The provider handles alerts and is responsible for maintenance and uptime of the system.
3. Outsourced: The service provider owns the platform that resides on the customer’s premises. Similar to the hybrid model, the provider staffs the SOC and assumes responsibility for operation and maintenance.
4. Single-tenant: The service provider runs the monitoring platform in their SOC (or the cloud), but each customer gets its own instance, and there is no comingling of security data.
5. Multi-tenant: The service provider has a purpose-built system to support many clients within the same environment, running in their SOC or the cloud. The assumption is that there are application security controls built into the system to ensure customer data stays is accessible only to authorized users, but that’s definitely something to check as part of your due diligence on the provider’s architecture.

Selecting Your Provider

We could probably write a book about selecting (and managing) a security monitoring service provider, and perhaps someday we will. But for now here are a few things to think about:

• Scale: You want a provider who can support you now and scale with you later. Having many customers roughly your size, as well as a technology architecture capable of supporting your plans, should be among your first selection criteria.
• Viability: Similarly important is your prospective provider’s financial stability. Given the time and burden of migration, and the importance of security monitoring, having a provider go belly up would put you in a precarious situation. Many managed security monitoring leaders are now part of giant technology companies, so this isn’t much of an issue any more. But if you are working with a smaller player, make sure you are familiar with their financials.
• Technology architecture: Does the provider use their own home-grown technology platform to deliver the service? Is it a commercial offering they customized to meet their needs as a provider – perhaps adding capabilities such as multi-tenancy? Did they design their own collection device, and does it support all your security/network/server/database/application requirements? Where do they analyze and triage alerts? Is it all within their system, or do they run a commercial monitoring platform? How many SOCs do they have, and how do they replicate data between sites? Understand exactly how their technology works so you can assess whether it fits your particular use cases and scalability requirements.
• Staff Expertise: It’s not easy to find and retain talented security analysts, so be sure to vet the background of the folks the provider will use to handle your account. Obviously you can’t vet them all, but understand the key qualifications of the analyst team – things like like years of experience, years with the provider, certifications, ongoing training requirements, etc. Also make sure to dig into their hiring and training regimens – over time they will need to hire new analysts and quickly get them productive, to deal with industry growth and the inevitable attrition.
• Industry specialization: Does this provider have many clients in your industry? This is important because there are many commonalities to both traffic dynamics and attack types within an industry, and you should leverage the provider’s familiarity. Given the maturity of most managed security offerings, it is reasonable to expect a provider to have a couple dozen similar customers in your industry.
• Research capabilities: One reason to consider a managed service is to take advantage of resources you couldn’t afford yourself, which a provider can amortize across their customers. Security research and the resulting threat intelligence are good examples. Many providers have full-time research teams investigating emerging attacks, profiling them, and keeping their collection devices up to date. Get a feel for how large and capable a research team a provider has, how their services leverage their research, and how you can interact with the research team to get the answers you need.
• Customization: A service provider delivers a reasonably standard service – leveraging a core set of common features is key to their profitability. That means you might not get as much customizability with a managed offering. Or it might be expensive. Some providers may argue, but be very wary of those offering to highly customize their environment just for you, because it’s hard to make that model work at scale.
• Service Level Agreements: Finally make sure your SLAs provide realistic assurances. Look for a dedicated account team, reasonable response times, clear escalation procedures, criteria for scope expansion/contraction, and a firm demarcation of responsibility before you sign anything. Once the deal is signed you have no leverage to change terms, so use your leverage during courting to make sure your SLAs reflect the way you do business. Ultimately you will need to trust that your provider will do their job, and resolve issues as they emerge.

You may also want to consider taking the service for a spin as part of your selection process. Similar to the Proof of Concept (PoC) process we outlined above, start small by collecting data from a handful of devices and running through the typical use cases driving your purchase. With a service offering it is as much about the interface and user experience as anything else, but be sure to test the alerting process, as well as escalation procedures for when the provider doesn’t meet your service level.

Checking References

There are at least two sides to every story. We have seen very successful security monitoring engagements, with customers large and small. We have also seen train wrecks. Of course the customer can be as responsible as the service provider when things go off the rails, but ultimately it’s your responsibility to perform sufficient due diligence when selecting a provider to learn the good, the bad, and the ugly regarding your potential provider.

That means talking to both happy and unhappy customers. Obviously a provider is unlikely to introduce you to disgruntled customers, but they are always happy to find happy customers who chose them over another provider. Leverage all the vendors competing for your business to assemble a set of both positive and not-so-positive references for potential providers.

Specifically, dig into a few areas:

• Deployment & migration: Make sure you understand the process to move to this provider’s platform. How will they deploy collectors? Can they import your existing data? What kind of project management oversight governs deployment and cutover? These are key questions to bring up during your reference calls. Ask for a very specific migration plan up front.
• Responsiveness: What kind of experience have customers had getting alerts and investigating issues? Have their analysts been helpful? Do they provide enough information to perform your own investigation? When the bad guys come knocking you won’t have time to fuss with bureaucracy or issues getting in your way. You’ll need the data, and to get your investigation moving – the provider must not hinder that process. Build responsiveness metrics into your agreement, along with escalation policies and penalties for violations.
• Expertise: Do they know what they are talking about? Did they do a bait and switch with the analysts monitoring customer networks? How precise and accurate are their alerts? Everything looks great during the sales cycle, but you want to make sure the A team (or at least the B+ team) is working your account on a daily basis.
• SLA violations: Not everything goes well. So learn how the provider deals with issues. Are they responsive? Do they work until the problem is solved? Have they been sued for breach of contract by other customers? This is where discussions with former clients can be very useful. There is usually a reason they are former clients, so find out. The provider should have a standard SLA for you to review.
• Account management: How does the relationship evolve over time? Is the account rep just there to sell you more services? Does the provider check in periodically to see how things are going? Do they take your feedback on product shortcomings and feature requests seriously? A service provider is really a partnership, so make sure this provider actually acts like a partner to their customers.

Mismatched Expectations

As when an on-premise security monitoring implementation goes awry, the root cause can usually be traced back to mismatched expectations. With a monitoring service always keep in mind what the service does, and don’t expect it to be something it’s not. Don’t count on deep customization or deep off-menu capabilities, unless they are agreed to up front.

Using a service provider for security monitoring can help provide resources and capabilities you don’t have in-house. That said, you need to perform due diligence to ensure you have both the right choice, and the right structure in place to manage them.