I was going to being this series talking about some of the architectural changes, but I’ve reconsidered. Since our initial coverage of Database Activity Monitoring technology in 2007, the products have fully matured into enterprise worthy platforms. What’s more, they’ve proven significant security and compliance benefits, as evidenced by market growth from $40M to revenues well north of $100M per year. This market is no longer dominated by small vendors, rather large vendors who have acquired six of the DAM startups. As such, DAM is being integrated with other security products into a blended platform. Because of this, I thought it best to go back and define what DAM is, and discuss market evolution first as it better frames the remaining topics we’ll discuss rest of this series.

Defining DAM

Our longstanding definition is:

Database Activity Monitors capture and record, at a minimum, all Structured Query Language (SQL) activity in real time or near real time, including database administrator activity, across multiple database platforms, and can generate alerts on policy violations. While a number of tools can monitor various level of database activity, Database Activity Monitors are distinguished by five features:

1. The ability to independently monitor and audit all database activity including administrator activity and SELECT transactions. Tools can record all SQL transactions: DML, DDL, DCL, (and sometimes TCL) activity.
2. The ability to store this activity securely outside of the database.
3. The ability to aggregate and correlate activity from multiple, heterogeneous Database Management Systems (DBMS). Tools can work with multiple DBMS (e.g.,Oracle, Microsoft, IBM) and normalize transactions from different DBMS despite differences in their flavors of SQL.
4. The ability to enforce separation of duties on database administrators. Auditing activity must include monitoring of DBA activity, and solutions should prevent DBA manipulation of and tampering with logs and activity records.
5. The ability to generate alerts on policy violations. Tools don’t just record activity, they provide real-time monitoring, analysis and rule-based alerting. For example, you might create a rule that generates an alert every time a DBA performs a SELECT query on a credit card column that returns more than 5 results.

DAM tools are no longer limited to a single data collection method, rather they offer network, OS layer, memory scanning and native audit layer support. Users can tailor their deployment to their security and performance requirements, and collect data from sources best fit their requirements.

Platforms

Reading that you’ll notice few differences from what was discussed in 2007. Further, we predicted the evolution of Applications and Database Security & Protection (ADMP) on the road to Content Monitoring and Protection, stating “DAM will combine with application firewalls as the center of the applications and database security stack, providing activity monitoring and enforcement within databases and applications.” But where it gets interesting is the other – different- routes vendors are taking to achieve this unified model. It’s how vendors bundle DAM into a solution that distinguishes one platform from another.

1. The Enterprise Data Management Model – In this model, DAM features are generically extended to many back-office applications. Data operations, such as a a file read or SAP transaction, are treated just like a database query. As before, operations are analyzed to see if a rule was violated, and if so, a security response is triggered. In this model DAM does more than alerting and blocking, but leverages masking, encryption and labeling technologies to address security and compliance requirements. This model relies heavily on discovery to help administrators locate data and define usage policies in advance. While in many respects similar to SIEM – the model leans more toward real time analysis of data usage. There is some overlap with DLP, but this model lacks endpoint capabilities and full content awareness.
2. The ADMP Model – What’s sometimes called the Web AppSec model, here DAM is linked with web application firewalls to provide activity monitoring and enforcement within databases and applications. DAM protects content in a structured application and database stack, WAF shields application functions from misuse and injection attacks, and File Activity Monitoring (FAM) protects data as it moves in and out of documents or unstructured repositories. This model is more application aware than the others, reducing false positives through transactional awareness. the ADMP model also provides advanced detection of web borne threats.
3. Policy Driven Security Model – Classic database security workflow of discovery, assessment, monitoring and auditing; each function overlapping with the next to pre-generate rules and policies. In this model, DAM is just one of many tools to collect and analyze events, and not necessarily central to the platform. What’s common amongst vendors who offer this model is policy orchestration: policies are abstracted from the infrastructure, with the underlying database – and even non-database – tools working in unison to fulfill the security and compliance requirements. How work gets done is somewhat hidden from the user. This model is great for reducing the pain of creating and managing policies, but as the technologies are pre-bundled, lacks the flexibility of other platforms.
4. The Proxy Model – Here DAM sits in front of the database, filtering inbound requests, acting as a proxy server. What’s different is what the proxy does with inbound queries. In some cases the query is blocked because it fits a known attack signature, and DAM acts as a firewall to protect – a method sometimes called ‘virtual patching’ – the database. In other cases the query is not forwarded to the database because the DAM proxy has recently seen the same request, and returns query results directly to the calling application. DAM is in essence a cache to speed up performance. Some platforms also provide the option of rewriting inbound queries, either to optimize performance or to minimize the risk an inbound query is malicious.

DAM tools have expanded into other areas of data, database and application security.

Market Drivers

DAM tools are extremely flexible and often deployed for what may appear to be totally unrelated reasons. Deployments are typically driven by one of three drivers:

1. Auditing for compliance. Compliance remains the biggest revenue driver for the DAM market given increasing auditor requirements to record database activity. Some enterprises are required to record all database activity for SOX, PCI-DSS or HIPAA, and DAM tools can do this with less overhead than alternative approaches.
2. As a compensating control for compliance. We are seeing greater use of DAM tools as a compensating control to meet compliance requirements even though database auditing itself isn’t the specified control. The most common example is using DAM as an alternative to encrypting credit card numbers for PCI compliance.
3. As a security control. DAM tools offer significant security benefits and can sometimes even be deployed in a blocking mode. They are particularly helpful in detecting and preventing data breaches for web facing databases and applications, or to protect sensitive internal databases through detection of unusual activity.

DAM tools are also beginning to expand into other areas of database and application security, such as a means of enriching SIEM and Log Management audit trails. Today, PCI has displaced SOX compliance as the single biggest market driver. Despite impressive capabilities, internally-driven security projects are a distant third motivation for DAM deployments.

Use Cases

The common use cases we referenced in the original paper are still valid:

1. To enforce separation of duties on database administrators for SOX compliance by monitoring all their activity and generating SOX-specific reports for audits.
2. If an application queries a database for credit card numbers, a DAM tool can generate an alert if the application requests more card numbers than a defined threshold (often a threshold of “1”). Changes in behavior or query structure often indicate that the application has been compromised via SQL injection or some other attack.
3. To ensure that a service account only accesses a database from a defined source IP, and only runs a narrow range of pre-approved queries. This can alert on compromise of a service either a) from the system that normally uses it, or b) if the account credentials are stolen and used from another system.
4. For PCI compliance you can encrypt the database files or media where they’re stored, then use DAM to audit and alert on access to the credit card field. The encryption protects against physical theft, while the DAM protects against insider abuse and certain forms of external attack.
5. As a change and configuration management tool. Some DAM tools offer this as a specialized feature with closed-loop integration with change management tools to track approved database changes implemented in SQL. Other tools can use this to track administrator accounts and provide change management reports for manual reconciliation.

There are several other use cases we see that are made possible by the evolution of the monitoring platforms:

1. To enforce data usage policies across document management and collaboration systems such as MS Sharepoint. Content management systems pull data from more than databases. By coupling relational and unstructured sources DAM can detect misuse across different storage mediums.
2. Link web and database activity, along with LDAP/Active Direct lookups to de-alias generic user accounts, then associate data usage with specific users.
3. To protect database and applications from SQL Injection and sloppy coding, DAM products will alter the query or the query results. For example, DAM may re-write suspect inbound queries to ensure they are ‘safe’, or mask query results to prevent sensitive data from being revealed if the user identity or location is suspect. This feature is also used by development teams to profile database queries to look for performance issues or unexpected use of the application.
4. To delay emergency patch installations that often de-stabilize the database, DAM can temporarily block queries that match a specific attack pattern or attempt to exploit a known weakness. This allows admins to delay patch installation until the patch has been tested and certified as stable.

In our next post we’ll cover the various technical architectures, and how the deployment models have changed in the last four years.