There are a lot of things I love about working for myself, but I have to admit sometimes it’s hard to keep everything balanced. For a while there I was taking whatever work came in the door that aligned with my goals and didn’t violate my objectivity requirements. Needless to say, the past few months have been absolutely insane; deadline after deadline, 2-3 trips a month, and a heck of a lot of writing.
The upside is I’m ahead on my goals for the year. The downside, other than a little stress, is that I haven’t been able to keep the content on the blog up as high as I’d like. How can I tell? This is part 3 of my series on Database Activity Monitoring, and I last posted part 2 in the beginning of November.
Oops.
With that mea culpa out of the way (assuming Jews are allowed to mea culpa), let’s jump back in to DAM.
Today we’re going to start on the basic characteristics of the central management server, including aggregation and correlation and policy creation. Tomorrow (for real) we’ll cover alerting, workflow, and reporting.
Aggregation and Correlation
The one characteristic Database Activity Monitoring solutions share with log management or even Security Information and Event Management (SIEM) tools is the ability to collect disparate activity logs from a variety of database management systems. Where they tend to exceed the capabilities of these related technologies is their ability to not only aggregate, but to normalize and correlate events. By understanding the Structured Query Language (SQL) of each database platform, they can interpret queries and understand their meanings. While a simple SELECT statement might mean the same thing across different database platforms, each database management system (DBMS) is chock full of its own particular syntax. A DAM solution should understand the SQL for each covered platform and be able to normalize events so the user doesn’t necessarily need to know the ins and outs of each DBMS. For example, if you want to review all privilege escalations on all covered systems, the DAM solution will recognize those events regardless of platform and present you with a complete report without you having to understand the SQL.
A more advanced feature is to then correlate activity across different transactions and platforms, rather than just looking at single events. For instance, smart DAM tools can recognize a higher than normal transaction volume by a particular user, or (as we’ll discuss in policies) tie in a privilege escalation event with a large SELECT query on sensitive data, which could indicate an attack.
It also goes without saying (but I’ll say it anyway) that all activity is centrally collected in a secure repository to prevent tampering or a security incidents involving the repository itself.
Since you’ll be collecting a massive volume of data, your DAM tool needs to support automatic archiving. Archiving should support separate backups of system activity, configuration, policies, alerts, and case management.
Policy Creation
One of the distinguishing characteristics of Database Activity Monitoring tools is that they don’t just collect and log activity, they analyze it in real time for policy violations. While still technically a detective control (we’ll talk about preventative deployments later), the ability to alert and respond in practically real time offers security capabilities far beyond simple log analysis. Successful, loss-bearing database attacks are rarely the result of a single malicious query- they involve a sequence of events leading to the eventual damage. Ideally, policies will be established to detect the activity early enough to prevent the final loss-bearing act. Even when an alert is triggered after the fact, it supports immediate incident response and investigation far sooner than analysis days or weeks later.
Policies fall into two basic categories, and I’m sure some of the engineers working on these products will drop additional options down in the comments:
- Rules-based: Specific rules are set up and monitored for violations. They can include specific queries, result counts, administrative functions (new user creation, rights changes), signature-based SQL injection detection, UPDATE or other transactions by users of a certain level on certain tables/fields, or any other activity that can be specifically described. Advanced rules can correlate across different parts of a database or even different databases, adjusting for data sensitivity based on DBMS labels or through registration in the DAM tool.
- Heuristic: The DAM solution monitors database activity and builds a profile of “normal” activity. Deviations then generate policy alerts. Heuristics are complicated and take proper tuning to work effectively. They are a good way to build a base policy set, especially with complex systems where manually creating deterministic rules by hand isn’t realistic. Policies are then tuned over time to reduce false positives. For well-defined systems where activity is pretty standard, such as an application talking to a database using a limited set of queries, they are very useful. Heuristics, of course, fail if you profile malicious activity as known good activity.
The more mature a solution, the more likely it is to come with sets of pre-packaged policies. For example, some tools come with pre-defined policies for standard deployments of databases behind major applications, like Oracle Financials or SAP. Yes, you’ll have to tune the policies, but it’s far better than starting from scratch. Pre-built policies for PCI, SOX, and other generic compliance requirements may need even more tuning, but will help you kick start the process and save many hours of custom policy building.
Policies should include user/group, source/destination, and other important contextual options. Policies should also support advanced definitions, like complex, multi-level nesting and combinations. Ideally, the DAM solution will include policy creation tools that limit the need to write everything out in SQL or some other definition language. Yes, you can’t avoid having to do some things by hand, but basic policies should be as point-and-click easy as possible.
For common kinds of policies, like detecting privileged user activity or count thresholds on sensitive data, policy wizards are extremely useful.
Content-Based Policies
An emerging feature in some tools is support for content-based policies. Similar to DLP, the tools are able to analyze queries and results for specific content.
Identifying all known locations of sensitive data within multiple heterogenous database management systems is a complex process, even with the support of content discovery (which we’ll talk about later). Credit card and Social Security Numbers can easily be placed where they shouldn’t be, either on purpose or by accident. Content-based policies, typically using regular expressions, analyze database activity for unapproved use of sensitive data. For example, a policy could look for credit card numbers in any result set except those previously approved.
It’s very early days, but I expect we’ll see more and more content and context awareness in DAM tools over time. Let’s be honest- the most critical data we’re usually trying to protect (at least these days) falls into structured formats we can define and look for when it breaks outside its normal boundaries (including data labeling or other registration techniques). Long term we’ll be able to do some really interesting things as we improve our ability to monitor and understand business context with the content, moving us ever closer to the elusive goal of using legitimate rights to commit forbidden actions.
That’s the basics of what to look for in aggregation, correlation, and policy creation. Tomorrow we’ll spend time on alerting, workflow, and reporting, before moving on to more advanced features like user identification in connection pooling, change management, and content discovery.
<
p style=”text-align:right;font-size:10px;”>Technorati Tags: Compliance, Data, Database Activity Monitoring, Database Security, Information-centric security, Application and Database Monitoring and Protection, Tools
Reader interactions
9 Replies to “Understanding and Selecting a Database Activity Monitoring Solution: Part 3, Central Management”
[…] 1 Part 2 Part 3 Part 4 Part […]
[…] 1 Part 2 Part 3 Part […]
[…] my previous post we discussed central management, including policy creation. One of the key advantages of DAM over passive auditing and logging solutions is the ability to […]
Looks like Luhn is supported by particular vendor. Regularly schedule Data crawl combine with auto update of sensitive object groupings tied to thresholds with context-based scanning on associated traffic. Build out tighter native access permissions (or further with MAC labels, Encryption (more difficult), columnar redaction (app logic changes?)) through other FGA mechanisms (and integrate with SSO and Provisioning aka “legitimate rights”). Endpoint DLP comes together with SQL DAM thru entitlements granted from the SSO engine and visibility thru comprehensive monitoring / tracking of data from cradle to grave. In the end, a 35-pass Gutmann wipe from disk (where ever—laptop to SAN) and while in active storage (maybe folder level crypto), but what is the MTBF on that—only needed when db crypto not used. More thoughts brewing…
I want context. For example, http://en.wikipedia.org/wiki/Luhn_algorithm
Rich, you’‘re still on the right track.
@Adrian
By content I don’‘t mean just SQL content- I mean running regex and other rules to detect SSNs on the fly. Imagine someone stuffing an SSN into a field where it shouldn’‘t be and being able to detect that.
Ted,
I’‘m pretty sure we’‘ll still have something sitting outside of the application server (call it a WAF or whatever you want). I have a bunch of stuff I need to publish on that and am writing it as quickly as I can. Basically, we will just use that WAF as a sensor to feed an integrated system that runs from outside the web server all the way back to the database, with agents dropped at every stage (I have a presentation on it, just not any papers yet).
Then, not only will DBAs be involved, but application teams as well. Long term, I don’‘t think we’‘ll see or need much on the network side, but we’‘ll probably need it for a lot of legacy stuff. I expect it to eventually consolidate onto web and application server agents.
Aggregation, correlation … and filtering.
Knowing what to filter, and being able to perform a bit more analysis on the record so that the filter can be tuned to the business need is a very important element. There was a time that we were considering using a Teradata DB for our internal repository because customers were not willing to filter out data they did not need because they were worried auditors would request it for compliance issues at some point in the future. The subsequent data volumes were colossal. Appropriate filtering of data, and filtering at a location that minimizes disruption to existing applications are differentiators for DAM.
Data Normalization is an interesting topic under this heading as well. Few organizations are just Oracle or DB2, but a mix of platforms. Some people want normalized reports on normalized data, which some of the SIEM providers have. There are many pros and cons, but a normalized stream can be more efficiently stored, and basic reports are easier to generate. SIEM is typically more efficient, but the aggregation and correlation of events is far more advanced in DAM than SIEM because of the data they collect and their focus on database&transactional analysis.
Policy Creation: This is really good. I assume future posts will cover ‘‘where we go from here’‘, so I will hold off comments on policy management & deployment till later.
Content Based polices emerging? Argh! Say it’s not so Rich! IMO, this is one of the primary differences between DAM and SIEM today, and there are DAM vendors who have had this for a while now (wink). SIEM usually looks at attributes as opposed to content. There is simply too many general events for event management products to look at transactional content. You will notice that their data normalization tends to focus on the who, what, when, where because that is the minimum relevant information, but reports are at a very high level. To deal with legitimate content inspection, especially when Heuristic policies are involved, the content monitoring system must be designed specifically to deal with the massive amounts of data that move, intra and inter, database systems and between databases and applications. The policies have to have a finer granularity to specify both deterministic content and (in some cases) heuristic content thresholds.
What is more, this is where audit log collection is sometimes just not enough, or SQL statement collection is not enough, so even DAM vendors are still maturing in this area. The result is the need for either alternative data collection, or subsequent database system inquiry for reference or context, or both. Some of the SIEM based products are now trying to provide drill down capabilities to offer this ‘after the fact’ for cross reference, but not content analysis, typically not in near real time.
Content Discovery, now that is an emerging technology!
“DAM will combine with application firewalls as the center of the applications and database security stack, providing activity monitoring and enforcement within databases and applications”
DAM could combine with a couple of different functions, but I wouldn’‘t put anything “firewall” at the top of my list. Maybe you don’‘t intend it as such, but to many people firewalls suggests network-based data collection, filtering, and a perimeter orientation (or certainly segmentation). In a previous life (when the security business was a lot younger and smaller) I suggested security functions will collapse but into classes determined by technology factors like architecture as well as who the buyer is within the enterprise. I think that applies here, too.
Some functions, like web app firewalls (WAF) and DLP probably will wind up components of a next generation firewall. They are fundamentally network infrastructure, have a perimeter orientation, and are largely bought by network security types.
I think DAM, however, will not successfully go in this direction (some may try, but it wont take hold in the market). First, as the use cases you suggest illustrate, DAM requires at least some data collection local to the database. As such, NW-based alone won’‘t cut it (frankly as native auditing improves, NW-based will probably disappear). Second, while the buyer picture for DAM at the moment is far from black and white, DBAs are absolutely involved. I think they will become even more involved and should be – without them, deployments won’‘t be successful. As a result, I think DAM functions will either consolidate with other systems / Db management functions or will drive the creation of a new data/app-centric class of security functions. In either case, the buyer won’‘t be the same guy that buys firewalls.
Certainly a lot of confusion is driven by all of the vendor hype. If I were a legacy security vendor, I’‘d be trying to hop on the data security bandwagon while playing the PCI blues. But, while you can put data security lipstick on a network security pig, that doesn’t mean it’s going to take first place at the data security fair 😉