We have covered the major features and capabilities of SIEM and Log Management tools, so now let’s discuss architecture and deployment models. Each architecture addresses a specific issue, such as coverage for remote devices, scaling across hundreds of thousands of devices, real-time analysis, or handling millions of events per second. Each has advantages and disadvantages in analysis performance, reporting performance, scalability, storage, and cost.
There are four models to discuss: ‘flat’ central collection, hierarchical, ring, and mesh. As a caveat, none of these deployment models is mutually exclusive. Some regions may deploy a flat model, but send information up to a central location via a hierarchy. These are not absolutes, just guidelines to consider as you design your deployment to solve the specific use cases driving your project.
Flat
The original deployment model for SIM and log management platforms was a single server that collected and consolidated log files. In this model all log storage, normalization, and correlation occurs within a central appliance. All data collection methods (agent, flow, syslog, etc.) are available, but data is always stored in the same central location.
A flat model is far simpler to deploy. All data and policies reside in a single location, so there are no policy or data synchronization issues. But of course ultimately a flat central collection model is limited in scalability, processing, and the quantity of data it can manage. A single installation provides a fixed amount of processing and storage, and reporting becomes progressively harder and slower as data sets grow. Truth be told, we only see this kind of architecture for “checkbox compliance”, predominately for smaller companies with modest data collection needs.
The remaining models address the limitations of this base architecture.
Hierarchical
In the Ring model – or what Mike likes to call the Moat – you have a central SIEM server ringed by many log collection devices. Each logger in the ring is responsible for collecting data from event sources. These log archives are also used to support distributed reporting. The log devices send a normalized and filtered (so substantially reduced) stream of events to the master SIEM device. The SIEM server sitting in the middle is responsible for correlation of events and analysis. This architecture was largely designed to address scalability limitations with some SIEM offerings. It wasn’t cost effective to scale the SIEM engine to handle mushrooming event traffic, so surrounding the SIEM centerpiece with logging devices allowed it to analyze the most critical events while providing a more cost-effective scaling mechanism.
The upside of this model is that simple (cheaper) high-performance loggers do the bulk of the heavy lifting, and the expensive SIEM components provide the meat of the analysis. This model addresses scalability and data management issues, while reducing the need to distribute code and policies among many different devices.
There are a couple issues with the ring model. The biggest problem remains a lack of integration between the two systems. Management tools for the data loggers and the SIEM may be linked together with some type of dashboard, but you quickly discover the two-headed monster of two totally separate products under the covers. Similarly, log management vendors were trying to graft better analysis and correlation onto their existing products, resulting in a series of acquisitions that provided log management players with SIEM. Either way, you end up with two separate products trying to solve a single problem. This is not a happy “you got your chocolate in my peanut butter,” moment, and will continue to be a thorny issue for customers until vendors fully integrate their SIEM and log management offerings as opposed to marketing bandaids dashboards as integrated products.
Mesh
The last model we want to discuss is the mesh deployment. The mesh is a group of interrelated systems, each performing full log management and SIEM functions for a small part of the environment. Basically this is a cluster of SIEM/LM appliances; each a functional peer with full analysis, correlation, filtering, storage, and reporting for local events. The servers can all be linked together to form a mesh, depending on customer needs.
While this model is more complex to deploy and administer, and requires a purpose-built data store to manage high-speed storage and analysis, it does solve several problems. For organizations that require segregation of both data and duties, the mesh model is unmatched. It provides the ability to aggregate and correlate specific segments or applications on specific subsets of servers, making analysis and reporting flexible. Unlike the other models, it can divide and conquer processing and storage requirements flexibly depending on the requirements of the business, rather than the scalability limitations of the product being deployed.
Each vendor’s product is capable implementing two or more of these models, but typically not all of them. Each product’s technical design (particularly the datastore) dictates which deployment models are possible. Additionally, the level of integration between the SIEM and Log Management pieces has an effect as well. As we said in our introduction, every SIEM vendor offers some degree of log management capability, and most Log Management vendors offer SIEM functions. This does not mean that the offerings are fully integrated by any stretch. Deployment and management costs are clearly affected by product integration or lack thereof, so make sure to do your due diligence in the purchase process to understand the underlying product architecture and the limitations and compromises necessary to make the product work in your environment.
Comments