Project Quant: Database Security Discovery

We decided to slow this series down for the holidays, as we are at a point where participation from the user community is very important. With the new year we are kicking back into high gear, and encourage comments and critiques of the processes we are describing. Picking up where we left off, we are at the Discovery phase in the database security process, a critical part of scoping the overall work.

Personally, discovery and assessment is my favorite step in the database security process. This step was the one that always yielded surprises for my team. Enterprise databases were present we did not know about. Small personal databases, perhaps even embedded in applications, we did not know about. Production data sets on test servers, tables with sensitive data copied into unsecured table-spaces, or cases where replication was turned on without our knowledge. This is over and above databases that were completely misconfigured -- usually a new DBA who did not know any better, but sometimes intentionally disabled to make administration easier. I have had clean scans on a Monday, only to find Friday that there were dozens of critical issues. And that's really what we want to determine in the Discovery phase.

Before we can act on the plan we developed in the previous section (Planning, Part 1 and Part 2), we must determine the state of the database environment. After all, you have to know what's wrong before you can fix it. What databases are in your environment, what purposes do they serve, what data do they host, and how are they set up? The first step in this phase is to find the databases.

Enumerate Databases:

In this stage we determine the location of database servers in the organization.

1. Plan: How are you going about scanning the environment? What parts of the process are automated vs. manual? Make sure you have clear guidelines. Refine scope to portions of IT, database type of interest. Also note that the person who created the plan may not be the person who runs the scan. Make sure that the data needed (database name, port number, database type, etc.) for subsequent steps is communicated or this entire process will need to be run again.
2. Setup: Acquire and install tools to automate the process, or map out your manual process. Configure for your environment, specifying acceptable network address and port ranges. Network segmentation will alter deployment. Databases have multiple connection options so plan accordingly. If you are keeping scan results in a database, create structures and configure.
3. Enumerate: Run your scan, or schedule for repeat scanning. Capture the results and filter out unwanted information to keep the data in scope for the project. Record as you baseline for future trend reports. In practice you will run this step more than once. As you discover databases you did not know existed, determine credentials you were provided are insufficient, or find subsequent steps require more information than you collected. Schedule to repeat scanning at periodic intervals. If you are using a manual process, this consists of contacting business units to identify assets and manually assessing each system.
4. Document: Format data, generate reports, and distribute. Use results to seed data discovery and assessment tasks.

Database discovery can be performed manually or automated. Segmented networks, regional offices, virtual servers, multi-homed hosts, remapping of standard ports, and embedded databases are all examples of common impediments you need to consider. If you choose to automate, most likely you are going to use a tool to examines network addresses and interrogate network ports, which may not yield all database instances but should capture the database installations. If you are using network monitoring to discover databases you will miss some. Regardless of your choice, you may not find everything, at least in the first sweep, so consider scanning more than once. In a manual process you will need to work with business units to identify databases, and perform some manual testing to identify any unreported databases. Understand what data you need to produce in this part of the process as your results from database discovery will be used to feed data discovery and assessment.

Identify Applications, Owners, and Data:

Now we take the identified databases and identify application dependencies, database owners, and the kinds of data stored.

1. Plan: Develop a plan to identify the application dependencies, data owners, and data types/classifications for the databases enumerated in the previous stage. Determine manual vs. automated tasks. If you have particular requirements, specify and itemize required data and assign tasks to qualified personnel. Define data types that require protection. Determine data collection methods (monitoring, assessment, log files, content analysis, etc.) to locate sensitive information.
2. Setup: Databases, data objects, data, and applications have ownership permissions that govern their access and use. For data discovery create regular expression templates, locations, or naming conventions for discovery scans. Test tools on known data types to verify operation.
3. Identify Applications: For applications, catalog connection methods and service accounts where appropriate.
4. Identify Database Owner(s): List database owners. Database owners provide credentials and accounts for dedicated scans, so determine who owns database installations and obtain credentials.
5. Discover Data: For data discovery return location, schema, data type, and other meta-data information. Rule adjustment requires re-scanning.
6. Document: Generate reports for operations and data security.

In essence this series of tasks is multiple discovery processes. Discovering the applications that attach to a database and how it is used, and what is stored within the database, are two separate efforts. Both can be performed by a credentialed investigation of the platform and system, or by observing network traffic. The former provides complete results at the expense of requiring credentials to access the database system, while passive network scanning is easier but provides incomplete results.

If you have existing security policies or compliance requirements data discovery is a little easier, as you know what you are looking for. If this is the first time you are scanning databases for applications and data, you may not have precise goals, but the results still aid in other activities. Manual discovery policies requires you to define the data types you are interested in detecting, so planning and rule development requires significant time in this phase.

Identification of applications and data provides information necessary to determine security and regulatory requirements. This task defines not only the scope of the scanning in the next task, but also subsequent monitoring and reporting efforts in different phases of this project.

Assess Vulnerabilities & Configurations:

Database Assessment is the analysis of database configuration, patch status, and security settings. It is performed by examining the database system both internally and externally -- in relation to known threats, industry best practices, and IT operations guidelines. It is important to note that with assessment, there is a large divide between having requirements and the tools or queries that gather the information. The setup portion of this task, particularly in script development, takes far more time than the scans themselves.

1. Define Scans: In a nutshell, this is where you define what you wish to accomplish. Compile a list of databases that need to be scanned, and determine requirements for different database types. Investigate best practices, security and compliance review for both internal and external requirements. Assign scans for proper separation of duties.
2. Setup: Determine how you want to accomplish your goals. Which functions are to be automated and which are manual? Are these credentialed scans or passive? Download updated policies from tools and database vendors, and create custom policies where needed. Create scripts to collect the information, determine priority, and suggest remediation steps for policy violations.
3. Scan: Scans are an ongoing effort, and most scanning tools provide scheduling capabilities. Collect results and store.
4. Distribute Results: Scan results will spotlight critical issues, variations from policy, and general recommendations. Filter unwanted data according to audience, then generate reports. Reporting includes feeding automated trouble ticket and workflow systems.

Database discovery, data discovery, and database security analysis are conceptually simple. Find the databases, determine what they are used for, and figure out if they are secure. In practice they are much harder than they sound. If you run a small IT organization you probably know where your one or two database machines are located, and should have the resources to find sensitive data.

When it comes to security policies, databases are so complex and the threats evolve so rapidly that definition and setup tasks will comprise the bulk of work for this entire phase. Good documentation, and a method for tracking threats in relation to policies and remediation information, are critical for managing assessments.

Authorization and Access:

In this stage we determine how access control systems are configured, then collect permissions and password settings.

1. Define Access Requirements: Man hours to obtain the list of databases you need to assess access controls for. Man hours to discover how access controls are performed; which functions are at the host or domain level, and how these services are linked to database permissions. Determine what password checks are to be employed.
2. Setup: For automated scans: cost to acquire, install and configure the tools. Time to obtain host/database permissions needed to perform manual and automated scans. Man hours needed to collect documented roles, groups, or service requirements for user of the database in later analysis. Cost of tools and time to generate report templates for stakeholders who will act upon scan results. If password penetration testing or dictionary attacks for weak passwords are being used, select a dictionary.
3. Scan: Variable: Man hours to run scan for database users showing group and role memberships, and then to scan groups, roles, and service account membership for each database. Man hours to collect domain and host user account information and settings.
4. Analyze & Report: Administrative roles need to be reviewed for separation of duties, both between administrative functions and between DBAs and IT administrators. Service accounts used by applications must be reviewed. User accounts need to be reviewed for group memberships and roles. Groups and roles must be reviewed to verify permissions are appropriate for business functions.

Database authorization and access control is the front line of defense for data privacy and integrity, as well as providing control over database functions. It's also the most time intensive of these tasks to check, as the process is multifaceted -- needing to account not only for the settings inside the database, but how those functions are supported by external host and domain level identity management services. This exercise is typically split between users of the database and administrators, as each has very different security considerations. Password testing can be time-consuming depending and, depending upon the method employed, may require additional database resources to avoid impact on production servers.

—Adrian Lane