Research

If you don’t already have attackers in your environment you will soon enough, so we have been spending a lot of time with clients figuring out how to respond in this age of APT (Advanced Persistent Threat) attackers and other attacks you have no shot at stopping. You need to detect and respond more effectively. We call this philosophy “React Faster and Better”, and have finally documented and collected our thoughts on the topic. Here are a couple excerpts from the paper to give you a feel for the issue and how we deal with it: Incident response is near and dear to our philosophy of security – it’s impossible to prevent everything (we see examples of this in the press every week), so you must be prepared to respond. The sad fact is that you will be breached. Maybe not today or tomorrow, but it will happen. We have made this point many times before (and it has even happened to us, indirectly). So response is more important than any specific control. But it’s horrifying how unsophisticated most organizations are about response. In this paper we’ll focus on pushing the concepts of incident response past the basics and addressing gaps in how you respond relative to today’s attacks. Dealing with advanced threats requires advanced tools. React Faster and Better is about taking a much broader and more effective approach on dealing with attacks – from what data you collect, to how you trigger higher-quality alerts, to the mechanics of response/escalation, and ultimately to remediation and cleaning activities. This is not your grandpappy’s incident response. To be clear, a lot of these activities are advanced. That’s why we recommend you start with our Incident Response Fundamentals from last year to get your IR team and function in decent shape. Please be advised that we have streamlined the paper a bit from the original blog series, cutting some of the more detailed information on setting up response tiers. We do plan to post the more complete paper at some point over the next couple months, but in the meantime you can refer back to the RFAB index of posts for the full unabridged version. A special thanks to NetWitness for sponsoring the research. Download: React Faster and Better: New Approaches for Advanced Incident Response (PDF) Share:

So you have defined your peer groups and analysis and spent a bunch of time communicating what you found to your security program’s key stakeholders. Now it’s time to shift focus internally. One of the cool things about security metrics and benchmarks is the ability to analyze trends over time and use that data to track progress against your key goals. Imagine that – managing people and programs based on data, not just gut feel. Besides being able to communicate much more authoritatively how you are doing on security, you can also focus on continuously improving your activities. This is a good thing to do – particularly if you want to keep your job. We will harp on the importance of consistency in gathering data and benchmarks over a long period of time, and then getting sustained value from the benchmark by using it to mark progress toward a better and more secure environment. Programs and feedback loops We don’t want to put the cart ahead of the horse, so let’s start at a high level, with describing how to structure the security program so it’s focused on improvement rather than mere survival. Here are the key steps: Define success (and get buy-in up the management stack) Distill success characteristics into activities that will result in success Quantify those activities, determine appropriate metrics, and set goals for those metrics Set objectives for each activity and communicate those objectives Run your business; gather your metrics Analyze metrics; report against success criteria/objectives Identify gaps, address issues, and reset objectives accordingly Wash, rinse, repeat Digging deeply into security program design and operation would be out of scope, so we’ll just refer you to Mike’s methodology on building a security program: The Pragmatic CSO. Communicating to the troops In our last post, on Benchmarking Communication Strategies, we talked about communicating with key stakeholders in the security process, and a primary constituency is your security team. Let’s revisit that discussion and its importance. Your security team needs to understand the process, how benchmark data will be used to determine success, and what the expectations will be. Don’t be surprised to experience some push-back on this new world order, and it could be quite significant. Just put yourself in your team’s shoes for a moment. For most of these folks’ careers they have been evaluated on a squishy subjective assessment of effectiveness and effort. Now you want to move them to something more quantified, where they can neither run nor hide. Top performers should not be worried – at all. That’s a key point to get across. So exercise some patience in getting folks heads in the right spot, but remember that you aren’t negotiating here. Part of the justification for investing (rather significantly) in metrics and benchmarks is to leverage that data in operations. You can’t do that if the data isn’t used to evaluate performance – both good and bad. It’s not a tool, it’s a lifestyle Another point to keep in mind is that this initiative isn’t a one-time thing. It’s not something you do for an assessment, and then forget it in a drawer the moment the auditor leaves the building. Benchmarking, done well, becomes a key facet of managing your security program. This data becomes your North Star, providing a way to map out objectives and ensure you stay on course to reach them. We have seen organizations start with metrics as a means to an end, and later recognize that they can change everything about how operational efforts are managed, perceived, and supported within the organization. The lack of security data has hindered acceptance of benchmarking in the security field, but it’s time to revisit that. As per usual, there are some caveats to data-driven management. No one size fits all. We see plenty of cultural variation, which may require you to take a less direct path to the benchmark promised land. But there can be no question about the effectiveness of quantifying activity, compared to not quantifying it. If you have gotten this far, successfully implemented this kind of benchmark, and institutionalized it as a management tool, you are way ahead of the game. But what’s next? Digging into deeper and more granular metrics, such as the metrics we defined as part of our Project Quant research. So we will discuss that next. Share:

Just in case you had nothing to do over the weekend, I came up with some homework to catch you up on our Security Benchmarking series. We’re clicking right along and think the content is kickass. So check it out, comment, and let us know if we are smoking crack. Introduction Security Metrics (from 40,000 feet) Collecting Data Systematically Sharing Data Safely Defining Peer Groups and Analyzing Data Communications Strategies Continuous Improvement We’ll be wrapping the series up next week with 3 more posts. So please contribute while you have the chance. Share:

The simple fact is that most folks senior security folks came from the technical side of the house. They started as competent (if not studly) sysadmins or security administrators, drew the short straw, and ended up with management responsibility. But very few of these folks ever studied management, gone through management training, or done anything but learned on the job. This creates a situation where senior security folks spend a lot of time doing stuff, but not enough time talking about it. The huge disconnect is inadequate communication of both success and failure up and down the management stack to key security stakeholders. In fact, the Pragmatic CSO methodology originated largely to help technical folks figure out how to deal with their management responsibilities. The inability to communicate to key stakeholders will absolutely kill a benchmarking program because benchmarking entails ongoing incremental effort to gather metrics, as well as to compare against benchmarks and perform analysis. The benchmark must provide additional value, which must be communicated in order to make the effort worthwhile. As we all know, nothing really happens by itself. You need to build a systematic communications/outreach effort to leverage the benchmark data, specifically targeting a number of constituencies important to the success of any security practitioner. Let’s dig into how that’s done, because it’s a critical success factor for any benchmarking initiative. Understanding your audience The first rule of communications is to do it consistently and repetitively by telling them what you are going to say, saying it, and then telling them what you just said. It sounds silly, but given today’s over-saturated environment where the typical C-level exec has the attention span of a 2-year-old, you don’t have a choice. Effective communications requires more than just talking a lot – you need to tailor your message to the audience. This is something security folks have always stunk at. If you’ve ever uttered the words “AV coverage” or “firewall rules” in a management meeting you know what I mean. Senior management If there is one thing you should appreciate about senior management, it’s that they are fairly predicable. Their interests involve things that directly impact revenues/expenses. Period. They don’t want to know the details of how you do something unless it’s off the rails. They want to know the bottom line and whether/how it will impact their ability to get paid their full bonus at the end of the year. So we focus on incident data and budget efficiency. They want to know whether incidents have impacted availability and thus cost them money. They need to know about disclosures, with an eye towards brand damage. And they need to know how you do relative to peers – if only make themselves feel better that their competitors probably won’t be getting those bonuses this year either. Getting time with senior folks is challenging. So you’ll be doing well if you can get quarterly face time to go through the metrics/results/benchmarks. At a minimum you need to make your case annually ahead of budgeting, but that is not really frequent enough to get sufficient attention to successfully execute on your program. Finally, how can benchmark data help you with these folks? You can use the fact that in terms of overhead functions most senior managers are lemmings – if everybody else is doing it (whatever it is), they will be likely to follow suit. It’s an ugly job, but someone has to do it. CIO Odds are you report in through the technology stack, which means you’ll spend some time with the CIO. This is a good thing, but keep in mind that the CIO’s primary goal is to look good to senior management. We all know that security issues can make him/her look very bad. So we can focus on what interests senior management: incidents and budget efficiency. But with the CIO you should add high-level operational trending data, which highlights issues and/or shows progress on efficiency. Given the spend on security, the CIO needs to pay attention to and increase efficiency. How often should you be communicating with the CIO? Hopefully monthly, if not more often. We know it’s hard to book time around golf outings with the big systems, storage, and networking vendors. But you still need access and face time to make sure there is a clear understanding of where the security program is and what needs to be addressed. Benchmark data helps substantiate the need for specific projects/investments, driven either by peer group adoption or efficiency/effectiveness gaps. Again, your opinion about what’s important and needed is interesting, but not necessarily relevant. Having data to substantiate your arguments makes the discussion much easier. IT Ops teams Brown stuff tends to flow downhill, so your pals in IT ops tend to focus on looking good to the CIO. You need their support to execute on any kind of security program, because ops can make it protection difficult, and that would be a problem for you and the CIO. But ops isn’t interested in the same things as senior managers. You need to focus those discussions on areas where changes or activities depend on operational resources. As with all things operational, it’s about increasing efficiency and reducing error, so we want data which highlighting issues, gaps, and/or areas to improve. Ops folks may not appreciate being told they may need to do things differently. This is another place where benchmark data can be your ace in the hole. By showing relative performance and ability to execute on operational processes, the data substantiates your arguments and helps avoid you having to go back to the CIO to complain “Ops sucks and makes our life hard!” and hoping the CIO will make them play nice. Security team As valuable as benchmark data is for telling a better story to stakeholders and key influencers of the security program, the benchmark data is also a key management tool for your own security team. We all want our groups to work better and improve continuously – as we

It’s tax day. You don’t have time to read this. I don’t have time to write it. Actually, my accountant is taking care of my taxes (I don’t trust myself with them). What’s really sucking down my time is preparing all the hands-on portions of the Cloud Security Alliance training. For the second time. We decided to split the class into two days, which means I have the opportunity to both tune the material and add new material. The cloud security portions of this are actually pretty straightforward – the harder part is scripting all the instances and configurations to focus the students on the important security bits without them having to learn things like MySQL, UNIX command lines (since, you know, auditor types will be in the class) and so on. That means I get to figure out all the scripting. Which isn’t a big deal, except I’m working with programs I don’t really deal with on a day to day basis. So there’s a lot of learning involved, and things that used to be instinctive when I was working as an admin now involve multiple web searches and mistakes to get correct. And little things like figuring out the mechanics of running a private cloud for 40 students on a single laptop and still providing some hands-on, as opposed to just an instructor demo. But I’m loving it. So go away and do your taxes. I need to play. On to the Summary: Webcasts, Podcasts, Outside Writing, and Conferences Adrian’s Dark Reading post on Cloud DB Security. Rich and Adrian quoted on our DBQuant press release. The Network Security Podcast, episode 237. Favorite Securosis Posts Adrian Lane: Database Trends. Mike Rothman: Our insanely comprehensive database security framework. No one else does this kind of research. It’s awesome to see it in its entirety. And we provide it at no cost. You’re welcome. David Mortman: Database Trends. Rich: Software vs. Appliance: Understanding DAM Deployment Tradeoffs. Other Securosis Posts Security Benchmarking, Going Beyond Metrics: Defining Peer Groups and Analyzing Data. Security Benchmarking, Going Beyond Metrics: Communications Strategies. Incite 4/13/2011: Jonesing for Air. Favorite Outside Posts Mike Rothman: Security vendors should face the music, even if they hate the tune. Bill Brenner nails it. Even when a review goes south, there are ways to handle it. Scorched earth on a well-respected testing house isn’t a winning strategy. David Mortman: How Dropbox sacrifices user privacy for cost savings. reppep: Cloud validation: 8 hours of 10,000-core computation for $8k. Okay, it’s still not for everybody, but this demonstrates that “cloud computing” does have a point. Adrian Lane: Russian Security Service proposes ban on Gmail, Skype, Hotmail. Skype a threat to National Security? Government’s the same all over. Research Reports and Presentations Measuring and Optimizing Database Security Operations (DBQuant). Woo hoo!!! Network Security in the Age of Any Computing. The Securosis 2010 Data Security Survey. Monitoring up the Stack: Adding Value to SIEM. Network Security Operations Quant Metrics Model. Network Security Operations Quant Report. Understanding and Selecting a DLP Solution. White Paper: Understanding and Selecting an Enterprise Firewall. Top News and Posts Veris Community Project Update The Web’s Trust Issues. Private records of 3.5 million people exposed by Texas. Hack attack spills web security firm’s confidential data. Adobe to Patch Flash Zero Day on Windows, Mac on Friday. DOJ Shuts Down Botnet, Disables Infected Systems Share:

“Hi. I’m Mike. And I’m an addict.” I start every chapter of the Pragmatic CSO with those very words. There there are many things you can be addicted to. Thrills. Sex. Sugar. Booze. Drugs. Twitter. Pr0n. Caffeine. Food. Some are worse than others, though none of them really good for you. But now I have to face up to another addiction. The need for gadgets. I’m jonesing for a new MacBook Air. Big time. Like waking up in the middle of the night wanting some SSD goodness in a petite 2lb package. Jonesing, I say, and it’s not pretty. Now there are folks with much worse gadget addiction than me. They are the ones standing in line at Best Buy for the latest Zune. Those folks have a problem. To be clear, so do I. I have a perfectly workable 15” MacBook Pro. It’s been a workhorse for two and a half years. For what I need, it’s fine. Why can’t I be happy with it? Why do I long for something new? The problem is my gear isn’t shiny anymore. I need a new trophy. Need. It. Now. I feel inadequate with a late 2008 MBP. In the bagel shop where I was writing this morning, there was a guy with an MB Air. I felt envy. Not enough to poach his machine when he tok a leak (by the way, it’s two frickin’ pounds – you can take it to the loo), but definitely envy. But then I looked over my other shoulder and saw a guy with an old school Apple laptop. And I mean old school. Like before they had a MagSafe connector, meaning a PowerBook G4. Oh, the horrors. I don’t know how that guy gets out of bed in the morning. And it’s worse when we have a Securosis meeting. Rich gets all the new toys. He’s got an MB Air 11”. I know he scoffs at my MBP. My laptop is older than his kids. Really. But Adrian is a different animal. He’s into high end audio equipment and dogs. My addiction is cheaper. At least I have that going for me. Over two years with the same laptop is a lifetime for me. Some guys trade in their wives every couple years. I trade in my laptop. The Boss likes that approach much better. Normally it’s not an issue, since I tend to hold down a job for 15 months, so I get a new toy every time I get a new job. I get my fix and have no issue, right? Not so much anymore – I’m not changing jobs any time soon. At least that’s what Rich and Adrian keep telling me. But I am getting smarter. Knowing this little issue I have, I made proper provisions this year by doing a side project over the winter and expressly earmarking those fees to breathe the (MB) Air. I’ve got motive. I’ve got opportunity. I’ve even got the funds. I know, you are wondering why I don’t just hop on the Apple web site and order it? This is why. They expect a new Air in the summer. That’s only what, 2 months away? It’ll be worth the wait. That’s what I keep telling myself. It will be smokin’ fast. And shinier. The next 2 months will be a struggle. I want it now. But I’m repressing my urges because I know how bad I’d feel when someone else got the shiny fast one, 4 days after I took delivery of my slow, dull one. I need to do some NLP to associate those bad feelings with the late 2010 MB Air. I will awaken the giant within, just you watch. That will keep me off the gadget juice. I’ll hold out because I have a plan. Every day, I’ll do my affirmations to convince myself that I’m still a good person, even though I use a late 2008 MBP. It will work. I know it will. The power of positive thinking in action. I’ll send a DM to my sponsor every day because I’m not addicted to Twitter. Not yet anyway. That will keep me on the straight and narrow. And doggone it, people like me, right? But we all know what happens when you repress an urge for too long. Gosh, that iPad 2 looks awfully shiny… -Mike Photo credits: “Apple addiction” originally uploaded by new-york-city Since I don’t do enough writing here on the Securosis blog, I figured I’d inflict some pithy verbiage on the victims, I mean readers, of Dark Reading. I’ll be posting on their Hacked Off blog monthly, and started with a doozy on why the RSA breach disclosure was pretty good. Surprisingly enough, I took a contrarian view to all those folks who think they should know everything, even if they aren’t RSA customers. It’s not about you, folks – sorry to bruise your egos. Incite 4 U Mea culpa roll with a side of SQLi: Do you ever wonder what a Barracuda roll tastes like? You can ask the folks in Hong Kong who used an automated SQLi attack to feast on Barracuda’s customer list over the weekend. The good news is that not much data was lost. Some customer and partner names and emails. The bad news is the breach happened because of an operational FAIL to put WAF back into blocking mode. As usual, people are the weakest link. But this disclosure is a great example of how to own it, explain it, and help everyone learn from it. A side of SQLi is not quite as tasty as miso soup, but news of the attack goes down a lot easier with a large serving of mea culpa. – MR Trust No One: I keep stealing a slide Gunnar did a while back (from Chris Hoff, who showed it to me first). It’s a table showing all the big advances in the web and web applications, and then the security tools we use to secure them. In every case, it’s firewalls and SSL. But between the Comodo breach and the

So your key security metrics are collected and shared safely. What comes next? Now we need to start deriving value from the data. Remember, metrics and numbers aren’t worth the storage to keep them, unless you use them as management tools. You need to start comparing the data, drawing conclusions, and adjusting your security program based on the data. OMG, actually making changes based on data rather than shiny objects, breaches, airline magazine articles, and compliance mandate changes. How novel. Remember the goal of this entire endeavor: to show relative progress. Now we get to figure out relative means, which involves defining peer group(s) for comparison. The first group you’ll compare your data to is actually yourself. Yes, this is trend analysis on your own metrics. It will provide some perspectives on whether you are improving – but improving against yourself does not provide perspective on whether you are ‘good’, spending too much money, or focusing on the right stuff. This is where you need to think about benchmarking, or going beyond security metrics. Peer Groups There are ways to define your peer group: Industry: This is your vertical market. Initially (until you have access to loads of data), you will focus on big industry buckets – like defense, healthcare, financial, hospitality, etc. Obviously there are differences between investment banks and insurance companies within the financial vertical, but businesses in the same category will have many consistent business processes which involve collecting very similar types of data. These organizations also tend to have similar geographic profiles – as for example a typical retailer will have a headquarters, regional distribution centers, and tons of stores. Additionally these companies exist under similar compliance/regulatory regimes. They also tend to be relatively consistent in terms of to technology adoption/maturity, which is critical for making relevant comparisons. Company size: Similar to the consistencies we find among companies in the same vertical/industry, we also find many similarities between companies of roughly the same size. For instance large enterprises (10,000+ employees) are generally global by definition – it is very difficult to get that big while focusing on a single geographic region. So organizational models and scale tend to be fairly consistent within a company-size segment. These companies also tend to spend similarly on security. Of course there are always outliers and some industries show less consistency, but we aren’t looking for perfection here. Region: Regional comparisons support many interesting comparisons. Culture and attitudes toward security can be enhanced or hindered by government funding and compliance regimes. We also see relatively consistent technology maturity/adoption within regions – largely based on local drivers such as compliance with laws and other rules, infrastructure, and available talent. Of course, not all metrics apply to any peer group. So when you define your benchmark peer groups, factor this in. The best way is to figure out how the specific metrics correlate for each peer group. We know, it’s math, but you’ll figure out pretty quickly whether there are any useful patterns or consistency within any particular metric. Focus on the metrics with the best correlation across a peer group. Sample Size Now that we’re talking about math, we have to address sample size. That’s basically how much data you need before the benchmark is useful. And as usual it depends, but push for statistical significance over the long term. Why? Because by definition statistical significance means a result is unlikely to occur by chance. You don’t want to be making decisions based on chance and randomness, so that’s our benchmark. More to the point, you want to stop making decisions based on chance. But it’s likely to take some time to get to a statistically significant dataset, so what can you do in the meantime? Look at the distribution, remove the outliers (which screw up your trend lines), and start comparing yourself against the trends you can spot. You can get a decent trend with only a handful of data points for metrics that correlate strongly. Always remember to keep the goal clearly in focus, and that is to identify gaps and highlight success, neither of which requires a huge amount of data. But to be clear, you are looking over time for statistical significance. Reverting to the Mean Another issue is whether you want to “revert to the mean,” meaning you look like everyone else in the peer group. Once again, it depends. Let’s take a look at a couple of likely metrics categories: For spending, it’s unlikely that you are getting a reasonable return from security spending 3 standard deviations above the mean. Not unless you can differentiate your product/offering on security, which is rare. For incidents, you want to be better than the mean. Most likely significantly so. Why? Because all your years of hard work can be unwound with one high profile breach. So the more effectively and quickly you respond and contain the damage, the better. Here you definitely don’t want to be in the bottom quartile, which indicates a failure of incident response and should be unacceptable to senior management. For efficiency, effectiveness, and coverage metrics (most of the easily quantifiable and operational metrics), you want to be better than the mean. That shows operational competence. In terms of importance, your spending is usually the most visible (to the folks who pay the bills, at least), so be in the ballpark there. Incidents come next, as they have a direct impact on issues like availability and brand damage. Then comes the operational stuff – it’s certainly important to how you run the security program, but rarely interesting to the muckety-mucks. Now it’s time to tell those muckety-mucks what you found, which means focusing on the commmunication strategy underlying your benchmarking program, so that’s where we’ll focus in the next post. Share:

Some projects take us a few days. Others? More like 18 months. Back before Mike even joined us, Adrian and I started a ‘quick’ project to develop a basic set of metrics for database security programs. As with most of our Project Quant efforts, we quickly realized there wasn’t even a starting framework out there, never mind any metrics. We needed to create a process for every database security task before we could define where people spent their time and money. Over the next year and a half we posted, reposted, designed, redesigned, and finally produced a framework we are pretty darn proud of. To our knowledge this is the most comprehensive database security program framework out there. From developing policies, to patch management, to security assessments, to activity monitoring, we cover all the major database security activities. We have structured this with a modular set of processes and subprocesses, with metrics to measure key costs at each step. The combination of process framework and metrics should give you some good ideas for structuring, improving, and optimizing your own program. Here’s the permanent home for the report, where you can post feedback and which will include update notices: Measuring and Optimizing Database Security Operations (DBQuant). We broke this into an Executive Summary that focuses on the process, and the full report with everything: Executive Summary. (PDF) The Full Report. (PDF) Special thanks to Application Security Inc. for sponsoring the report, and sticking with us as we pretended to be PhD candidates and dragged this puppy out. Share:

This is a non-security post, in case that matters to you. A few days ago I was reading about a failed Telcomm firm ‘refocusing’ its business and technology to become a cloud database provider. I’m thinking that’s the last frackin’ thing we need. Some opportunistic serial start-up-tard can’t wait to fail the first time, and wants skip over onto not one but two, hot trends. Smells like 1999. Of course they landed an additional $4M; couple Cloud with a modular database and it’s a no-lose situation – at least for landing venture funding. So why do we need vendor #22 jumping onto the database in the cloud bandwagon? I visited the xeround site, and after looking at their cloud database architecture … damn, it appears solid. Think of a more modular MySQL. Or better yet, Amazon Dynamo with less myopic focus on search and content delivery. Modular back-end storage options, multiple access nodes disassociated from the query engines, and multiple API handlers. The ability to mix and match components to form a database engine depending upon the task at hand makes more sense than the “everything all the time” model we have with relational vendors. I don’t see anything novel here, just a solid assemblage of features. To fully take advantage of the elastic, multi-zone, multi-tenant pay-as-you go cloud service, a modular, dynamic database is more appropriate. Notice that I did not say ‘requirement’ – you can run Oracle as an AMI on Amazon too, but that’s neither modular nor nimble in my view. The main point I want to make is that the next generation of databases is going to look more like this and less like Oracle and IBM DB2. The core architecture described embodies a “use just what you need” approach, and allows you tailor the database to fit the application service model. And don’t mistake me for yet another analyst claiming that relational database platforms are dead. I have taken criticism in the past because people felt I was indicating relational platforms had run their course, but that’s not the case. It’s more like the way RISC concepts appeared in CISC processors to make them better, but did not supersede the original as promised. NoSQL concepts are pushing the definition of what ‘database’ means. And we see all these variants because the relational platforms are not a good fit for either the application model or cloud service delivery models. Expect many of the good NoSQL ideas to show up in relational platforms as the next evolutionary step. For now, the upstarts are pointing the way. Note that this is not an endorsement of the xeround technology. Frankly I am too busy to load up an AMI and try their database to see if it works as advertised. And their feature comparison is kinda BS. But conceptually I think this model is on track. That’s why will see many new database solutions on the market, as many firms struggle to find the right mix of features and platform options to meet requirements of application developers and cloud computing customers. Share:

One thing I don’t miss from my vendor days in the Database Activity Monitoring market is the competitive infighting. Sure, I loved to do the competitive analyses to see how each vendor viewed itself, and how they were all trying to differentiate their products. I did not enjoy going into a customer shop after a competitor “poisoned the well” with misleading statements, evangelical pitches touting the right way to tackle a problem, or flat-out lies. Being second into a customer account meant having to deal with the dozen land mines left in their minds, and explaining those issues just to get even. The common land mines were about performance, lack of impact on IT systems, and platform support. The next vendor in line countered with architectures that did not scale, difficulties in deployment, inability to collect important events, and management complexity of every other product on the market. The customer often cannot determine who’s lying until after they purchase something and see if it does what the vendor claimed, so this game continues until the market reaches a certain level of maturity. With Database Activity Monitoring, the appliance vs. software debate is still raging. It’s not front and center in most product marketing materials. It’s not core to solving most security challenges. It is positioned as an advantage behind the scenes, especially during bake-offs between vendors, to undermine competitors. Criticism not based on the way events are processed, UI, or event storage – but simply on the deployment model. Hardware is better than software. Software is better than hardware. This virtual hardware appliance is just as good as software. And so on. This is an area where I can help customers understand the tradeoffs of the different models. Today I am kicking off a short series to discuss tradeoffs between appliance, software, and virtual appliance implementations of Database Activity Monitoring systems. I’ll research the current state of the DAM market and highlight the areas you need to focus on to determine which is right for you. I’ll also share some personal experiences that illustrate the difference between the theoretical and the practical. The series will be broken into four parts: Hardware: Discussion of hardware appliances dedicated to Database Activity Monitoring. I’ll cover the system architecture, common deployment models, and setup. Then we’ll delve into the major benefits and constraints of appliances including performance, scalability, architecture, and disaster recovery. Software: Contrasting DAM appliances with software architecture and deployment models; then cover pros and cons including installation and configuration, flexibility, scalability and performance, and installation/setup Virtual Appliances: Virtualization and cloud models demand adaptation for many security technologies, and DAM is no different. Here I will discuss why virtual appliances are necessary – contrasting against with hardware-based appliances – and consider practical considerations that crop up. Data Collection and Management: A brief discussion of how data collection and management affect DAM. I will focus on areas that come up in competitive situations and tend to confuse buying decisions. I have been an active participant in these discussions over the last decade, and I worked for a DAM software provider. As a result I need to acknowledge, up front, my historical bias in favor of software. I have publicly stated my preference for software in the past based upon my experiences as a CIO and author of DAM technology. As an analyst, however, I have come to recognize that there is no single ‘best’ technology. My own experiences sometimes differ from customer reality, and I undersetand that every customer has its own preferred way of doing things. But make no mistake – the deployment model matters! With that said, there is no single ‘best’ model. Hardware, software, and virtual appliance – each has advantages and disadvantages. What works for each customer depends on its specific needs. And just like vendors, customer will have their own biases. What’s important is what is ‘better’ for the consumer. I will provide a list of pros and cons, to help you decide what will work best. I will point out my own preferences (bias), and as always you are welcome to call ‘BS’ on anything in this series you don’t accept. Perhaps more than any other series I have ever written at Securosis, I want to encourage feedback from the security and IT practitioner community. Why? Because I have witnessed too many software solutions that don’t scale as advertised. I am aware of several hardware deployments that cost the customer almost 4X the original bid. I am aware of software – my own firm was guilty – so inflexible we were booted from the customer site. I know these issues still occur, so my goal is to help wade through the competitive puffery. I encourage you to share what have you seen, what you prefer, and why, as it helps the community. Share: