Transparent Encryption

Database Encryption: Fact vs. Fiction

By Adrian Lane

A good friend of mine has, for many years, said "Don't let the facts get in the way of a good story." She has led a very interesting life and has thousands of funny anecdotes, but is known to embellish a bit. She always describes real life events, but uses some imagination and injects a few spurious details to spice things up a little bit. Not false statements, but tweaking the facts to make a more engaging story. Several of the comments on the blog in regards to our series on Database Encryption, as well as some of those made during product briefings, fall into the later category. Not completely false, but true only from a limited perspective, so I am calling them 'fiction'.

It's ironic that I am working on a piece called "Truth, Lies, and Fiction in Encryption" that will be published later this summer or early fall. I am getting a lot of good material that will go into that project, but there are a couple fictional claims that I want to raise in this series to highlight some of the benefits, weaknesses, and practical realities that come into play with database encryption.

One of the private comments made in response to Part 4: Credentialed User protection was: "Remember that in both cases (Re: general users and database administrators), encryption is worthless if an authorized user account itself is compromised." I classify this as fiction because it is not totally correct. Why? I can compromise a database account, let's say the account that an application uses to connect to the database. But that does not mean I have credentials to obtain the key to decrypt data. I have to compromise both the database and the key/application user credentials for this.

For example, when I create a key in Microsoft SQL Server, I protect that key with a password or encrypt it with a different key. MSDN shows the SQL Server calls. If someone compromises the database account "SAPConnPool_User" with the password "Password1", they still have not obtained the decryption keys. You still need to supply a password as a parameter to the 'EncryptByKey' or 'DecryptByKey' commands. A hacker would need to guess the password or gain access to the key that has encrypted the user's key. But with connection pooling, there will be many users keys passed in context of the query operations, meaning that the hacker must compromise several keys before the correct one is obtained. A DBA can gain access to this key if internal to the database, and I believe can intercept it if the value is passed through the database to an external HSM via database API (I say 'believe' because I have not personally written exploit code to do so). With the latest release of SQL Server, you can segregate the DBA role to limit access to stored key data, but not eliminate it altogether.

Another example: With IBM DB2, the user connection to the database is one set of credentials, while access to encryption keys uses a second set of credentials. To gain access you need to gain both sets. Here is a reference for Encrypting Data Values in DB2 Universal Database.

Where this statement is true is with Transparent Encryption, such as the various derivatives of Oracle Transparent Encryption. Once a database user is validated to the database, the user session is supplied with an encryption key, and encryption operations are automatically mapped to the issued queries, thus the user automatically has access to the table that stores the key and does not need to credentials for access. Transparent Encryption from all vendors will be similar. You can use the API of the DBMS_Crypto package to provide this additional layer of protection, but like the rest of the platforms, you must separate the implicit binding of database user to encryption key, and this means altering your program to some degree. As with SQL Server, an Oracle DBA may or may not be able to obtain keys based upon a segregated DBA role.

We have also received a comment on the blog that stated "encrypting data in any layer but the application layer leaves your data insecure." Once again, a bit of fiction. If you view the problem as protecting data when database accounts have been compromised, then this is a true statement. Encryption credentials in the application layer are safe. But applications provide application users the same type of transparency that Transparent Encryption provides database users, thus a breached application account will also bypass encryption credentials and access some portion of the data stored in the database. Same problem, different layer.

–Adrian Lane

Database Encryption, Part 3: Transparent Encryption

By Adrian Lane, Rich

In our previous post in this Database Encryption series (Introduction, Part 2) we provided a decision tree for selecting a database encryption strategy. Our goal in this process is to map the encryption selection process to the security threats to protect against. Yes, that sounds simple enough, but it is tough to wade through vendor claims, especially when everyone from network storage to database vendors claims to provide the same value. We need to understand how to deal with the threats conceptually before we jump into the more complex technical and operational issues that can confuse your choices. In this post we are going to dig into the first branch of the tree, Non-credentialed threats -- protecting against attacks from the outside, rather than from authenticated database users. We call this "Transparent/External Encryption", since we don't have to muck with database user accounts, and the encryption can sometimes occur outside the datbase. Transparent Encryption won't protect sensitive content in the database if someone has access to it thought legitimate credentials, but it will protect the information on storage and in archives, and provides a significant advantage as it is deployed independent of your business applications. If you need to protect things like credit card numbers where you need to restrict even an administrator's ability to see them, this option isn't for you. If you are only worried about lost media, stolen files, a compromised host platform, or insecure storage, then Transparent Encryption is a good option. By not having to muck around with the internal database structures and application logic, it often provides huge savings in time and investment over more involved techniques.

We have chosen the term Transparent Encryption, as many of the database vendors have, to describe the capability to encrypt data stored in the database without modification to the applications using that database. We've also added "External" to distinguish from external encryption at the file or media level. If you have a database then you already have access controls that protect that data from unwanted viewing through database communications. The database itself screens queries or applications to make sure that only appropriate users or groups are permitted to examine and use data. The threat we want to address here is protecting data from physical loss or theft (including some forms of virtual theft) through means that are outside the scope of access controls. Keep in mind that even though the data is "in" a database, that database maintains permanent records on disk drives, with data being archived to many different types of low cost, long term storage. There are many ways for data to be accessed without credentials being supplied at all. These are cases where the database engine is by-passed altogether -- for example, examination of data on backup tapes, disks, offline redo log files, transaction logs, or any other place data resides on storage media.

Transparent/External Encryption for protecting database data uses the following techniques & technologies:

• Native Database Object (Transparent) Encryption: Database management systems, such as Oracle, Microsoft SQL Server, and IBM DB2, include capabilities to encrypt either internal database objects (tables and other structures) or the data stores (files). These encryption operations are managed from within the database, using native encryption functions built into the database, with keys being stored internally by default. This is good overall option in many scenarios as long as performance meets requirements. Depending on the platform, you may be able to offload key management to an external key management solution. The disadvantage is that it is specific to each database platform, and isn't always available.
• External File/Folder Encryption: The database files are encrypted using an external (third party) file/folder encryption tool. Assuming the encryption is configured properly, this protects the database files from unauthorized access on the server and those files are typically still protected as they are backed up, copied, or moved. Keys should be stored off the server and no access provided to local accounts, which protect against the server becoming compromised by an external attacker. Some file encryption tools, such as Vormetric and BitArmor, can also restrict access to the protected files based on application. Thus only the database processes can access the file, and even if an attacker compromises the database's user account, they will only be able to access the decrypted data through the database itself. File/folder encryption of the database files is a good option as long as performance is acceptable and keys can be managed externally. Any file/folder encryption tool supports this option (including Microsoft EFS), but performance needs to be tested since there is wide variation among the different tools. Remember that any replication or distribution of data handled from within the database won't be protected unless you also encrypt those destinations.
• Media encryption: This includes full drive encryption or SAN encryption; the entire storage media is encrypted, and thus the database files are protected. Depending on the method used and the specifics of your environment, this may or may not provide protection for the data as it moves to other data stores, including archival (tape) storage. For example, depending on your backup agent, you may be backing up the unencrypted files or the encrypted storage blocks. This is best suited for high performance databases where the primary concern is physical loss of the media (e.g., a database on a managed SAN where the service provider handles failed drives potentially containing sensitive data). Any media encryption product supports this option.

Which option to choose depends on your performance requirements, threat model, exiting architecture, and security requirements. Unless you have a high-performance system that exceeds the capabilities of file/folder encryption, we recommend you look there first. If you are managing heterogeneous databases, you will likely look at a third party product over native encryption. In both cases, it's very important to use external key management and not allow access by any local accounts. We will outline selection criteria and use cases to support the decision process in a future post.

You will notice that, depending upon which technique you choose, the initiation of the encryption, the engine that performs the encryption, and the key management server may all reside in different places. In fact, the latter two techniques encrypt the database but are not in the database at all; the engine that performs the encryption, as well as the processes responsible for managing the encryption operations, are outside of the database. In all three cases encryption remains transparent to business processing functions. Using the three technology variables (eEngine location, key management location, and who orchestrates) we introduced in Part Two of this series will help guide you in differentiating vendor offerings, and understand the strengths and weaknesses of each variation.

Next up we will discuss the techniques for securing data against credentialed users and some implementation considerations for user encryption.

–Adrian Lane, Rich

Database Encryption, Part 2: Selection Process Overview

By Adrian Lane

In the selection process for database encryption solutions, too often the discussion devolves straight into the encryption technologies: the algorithms, computational complexity, key lengths, merits of public vs. private key cryptography, key management, and the like.

In the big picture, none of these topics matter.

While these nuances may be worth considering, that conversation sidesteps the primary business driver of the entire effort: what threat do you want to protect the data from? In this second post in our series on database encryption, we'll provide a simple decision tree to guide you in selecting the right database encryption option based on the threat you're trying to protect against. Once we've identified the business problem, we will then map that to the underlying technologies to achieve that goal. We think it's safe to say that if you are looking at database encryption as an option, you have already come to the decision that you need to protect your data in some way. Since there's always some expense and/or potential performance impact on the database, there must be some driving force to even consider encryption. We will also make the assumption that, at the very least, protecting data at rest is a concern. Let's start the process by asking the following questions:

What do you want to protect? The entire contents of the database, a specific table, or a data field?

What do you want to protect the data from? Accidental disclosure? Data theft?

Once you understand these requirements, we can boil the decision process into the following diagram:

Whether your primary driver is security or compliance, the breakdown will be the same. If you need to provide separation of duties for Sarbanes-Oxley, or protect against account hijacking, or keep credit card data from being viewed for PCI compliance, you are worried about credentialed users. In this case you need a more granular approach to encryption and possibly external key management. In our model, we call this user encryption. If you are worried about missing tapes, physical server theft, copying/theft of the database files via storage compromise, or un-scrubbed hard drives being sold on eBay, the threat is outside of the bounds of access control. In these cases use of transparent/external encryption through native database methods, OS support, file/folder encryption, or disk drive encryption is appropriate.

Once you have decided which method is appropriate, we need to examine the basic technology variables that affect your database system and operations. Which you select corresponds to how much of an impact it will have on applications, database performance, and so on. With any form of database encryption there are many technology variables to consider for your deployment, but for the purpose of selecting which strategy is right for you, there are only three to worry about. These three effect the performance and type of threats you can address. In each case we will want to investigate if these options are performed internally by the database, or externally. They are:

1. Where does the encryption engine reside? [inside/outside]
2. Where is the key management performed? [inside/outside]
3. Who/what performs the encryption operations? [inside/outside]

In a nutshell, the more secure you want to be and the more you need separation of duties, the more you will need granular enforcement and changes to your applications. Each option that is moved outside the database means you get more complexity and less application transparency. We hate to phrase it like this because it somehow implies that what the database provides is less secure when that is absolutely not the case. But it does mean that the more we manage inside the database, the greater the vulnerability in the event of a database or DBA account compromise. It's called "putting all your eggs in one basket". Throughout the remainder of the week we will discuss the major branches of this tree, and how they map to threats. We will follow that up with a set of use case discussions to contrast the models and set realistic expectations on security this will and will not provide, as well as some comments on the operational impact of using these technologies.

By the end you'll be able to walk through our decision tree and pick the best encryption option based on what threat you're trying to manage, and operational criteria ranging from what database platform you're on to management requirements.

–Adrian Lane