I spend a heck of a lot of time researching, writing, and speaking about data security. One area that’s been very disappointing is the quality of many of the surveys. Most either try to quantify losses (without using a verifiable loss model), measure general attitudes to inspire some BS hype press release, or assess some other fuzzy aspect you can spin any way you want.
This bugs me, and it’s been on my to-do list to run a better survey myself. When a vendor (Imperva) proposed the same thing back at RSA (meaning we’d have funding) and agreed to our Totally Transparent Research process, it was time to promote it to the top of the stack.
So we are kicking off our first big data security study. Following in the footsteps of the one we did for patch management, this survey will focus on hard metrics – our goal is to avoid general attitude and unquantifiable loss guesses, and focus on figuring out what people are really doing about data security.
As with all our surveys, we are soliciting ideas and feedback before we run it, and will release all the raw results.
Here are my initial ideas on how we might structure the questions:
- We will group the questions to match the phases in the Pragmatic Data Security Cycle, since we need some structure to start with.
- For each phase, we will list out the major technologies and processes, then ask which one organizations have adopted.
- For technologies, we will ask which they’ve researched, budgeted for, purchased, deployed in a limited manner (such as testing), deployed in initial production, and deployed in full production (organization wide).
- For processes, we will ask about maturity from ad-hoc through fully formalized and documented, similar to what we did for patch management.
- For the tools and processes, we’ll ask if they were implemented due to a specific compliance deficiency during an assessment.
I’m also wondering if we ask should how many breaches or breach disclosures were directly prevented by the tool (estimates). I’m on the fence about this, because we would need to tightly constrain the question to avoid the results being abused in some way.
Those are my rough ideas – what do you think? Anything else you want to see? Is this even in the right direction? And remember – raw (anonymized) results will be released, so it’s kind of like your chance to run a survey and have someone else bear the costs and do all the work…
FYI The sponsor gets an exclusive on the raw results for 45 days or so, but they will be released free after that. We have to pay for these things somehow.
Reader interactions
4 Replies to “Help Build the Mother of All Data Security Surveys”
In David Scott
I’d like to vote for ‘how many incidents prevented..’ type questions or ANY other way to measure the positive effect of the tool/control/solution.
LLou-
Some great ideas! I think we might need to exclude some of the non-data-security specific questions to keep the scope reasonable (I always shoot for 10 minute surveys to get more responses).
I really like the segregation questions, I didn’t think about those.
I’d like information on basics. What are we really trying to improve through security controls, and how do we accurately measure that?
What are good security outcomes? Security of your data is a piece of it but availability of data and services is another piece.
What tools do you use that measure:
1. theft of data (how do you know if it was stolen? how soon do you know it? do you report all instances? )
2. destruction of data (same above questions)
3. unauthorized alteration of data (ditto)
4. appropriate/timely delivery of data
Do you have exfiltration tools (and what are they) that measure how much, if anything, is leaving your network?
Do you have tools that monitor unauthorized access to your network? How quickly are intrusions caught? What works best in catching unauthorized access quickly? What is the ratio of people watching for them (how experienced are they?)to number of intrusions caught?
Is your important data isolated in an easier to protect area, separated from the lesser data? How much important data do you have (is it stored on a single server or on a SAN? Are important data more heavily monitored? Are they encrypted? Are they text documents, databases, spreadsheets? Diagrams, Drawings, privileged communcation? Are there better ways of protecting one over the others?
How much money did you spend separating out the important from less important data? What methods/tools did you use? How long did it take? How much data are you talking about?
Do you have a time-to-live on data importance? Are 10 year old files protected equally to current files?
Do you have a limit on how much data you highly protect?
How effective is your antivirus (antimalware)? Do you have layers of it — e.g. on email server and on clients. How do you know how effective it is? How many incidents of malware infection get through on top of your antimalware?
Do you use visualization tools to quickly identify anomalies? Which ones? Which seem most effective?
What tools do you need that you don’t have?
What characteristics of tools that you have don’t do what you need them to?