Monitoring up the Stack: Application Monitoring, Part 1

By Gunnar

As we continue to investigate additional data sources to make our monitoring more effective, let’s now turn our attention to applications. At first glance, many security practitioners may think applications have little to offer SIEM and Log Management systems. After all, applications are built on mountains of custom code and security and development teams often lack a shared collaborative approach for software security. However, application monitoring for security should not be dismissed out of hand. Closed-minded security folks miss the fact that applications offer an opportunity to resolve some of the key challenges to monitoring. How? It comes back to a key point we’ve been making through this series, the need for context. If knowing that Node A talked to Node B helps pinpoint a potential attack, then network monitoring is fine. But both monitoring and forensics efforts can leverage information about what transaction executed, who signed off on it, who initiated it, and what the result was – and you need to tie into to the application to get that context.

In real estate, it’s all about location, location, location. By climbing the stack and monitoring the application, you collect data located closer to the core enterprise assets like transactions, business logic, rules, and policies. This proximity to valuable assets make the application an ideal place to see and report on what is happening at the level of user and system behavior, which can (and does) establish patterns of good and bad behavior that can provide additional indications of attacks.

The location of the application monitor is critical for tracking both authorized users and threats, as Adrian pointed out in his post on Threat Monitoring:

This challenge is compounded by the clear focus on application-oriented attacks. For the most part, our detection only pays attention to the network and servers, while the attackers are flying above that. It’s kind of like repeatedly missing the bad guys because they are flying at 45,000 feet, but you cannot get above 20,000 feet. You aren’t looking where the attacks are actually happening, which obviously presents problems.

Effective monitoring requires access to the app, the data, and the system’s identity layers. They are the core assets of interest for both legitimate users and attackers trying to compromise your data.

So how can we get there? We can look to software security efforts for some clues. The discipline of software engineering has made major strides in building security into applications over the last ten years. From static analysis, to threat modeling, to defensive programming, to black box scanners, to stronger identity standards like SAML, we have seen the software engineering community make real progress on improving overall application security. From the current paradigm of building security in, the logical next step is building visibility in, meaning the next step is to instrument applications with monitoring capabilities that collect and report on application use and abuse.

Application Monitoring delivers several essential layers of visibility to SIEM and Log Management:

  • Access control: Access control protects applications (including web applications) from unauthorized usage. But the access control container itself is often attacked via methods such as Cross Site Request Forgery (CSRF) and spoofing. Security architects rely heavily on access control infrastructure to enforce security at runtime and this data should be pumped into the SIEM/Log Management platform to monitor and report on its efficacy.
  • Threat monitoring: Attackers specialize in crafting unpredictable SQL, LDAP, and other commands that are injected into servers and clients to troll through databases and other precious resources. The attacks are often not obviously attacks, until they are received and processed by the application – after all “DROP TABLE” is a valid string. The Build Security In school has led software engineers to build input validation, exception management, data encoding, and data escaping routines into applications to protect against injection attacks, but it’s crucial to collect and report on a possible attack, even as the application is working to limit its impact. Yes, it’s best to repel the attack from within the application, but you also need to know about it, both to provide a warning to more closely monitor other applications, and in case the application is successfully compromised – the logs must be securely stored elsewhere, so even in the event of a complete application compromise, the alert is still received.
  • Transaction monitoring: Applications are increasingly built in tiers, components, and services, where the application is composed dynamically at runtime. So the transaction messages’ state is assembled from a series of references and remote calls, which obviously can’t be monitored from an infrastructure view. The solution is to trigger an alert within the SIEM/Log Management platform when the application hits a crucial limit or other indication of malfeasance in the system; then by collecting critical information about the transaction record and history, the time required to investigate potential issues can be reduced.
  • Fraud detection: In some systems, particularly financial systems, the application monitoring practice includes velocity and throttles to record behaviors that indicate the likelihood of fraud. In more sophisticated systems, the monitors are active participants (not strictly monitors) and change the data and behavior of the system, such as through automatically flagging accounts as untrustworthy and sending alerts to the fraud group to start an investigation based on monitored behavior.

Application monitoring represents a logical progression from “build security in” practices. For security teams actively involved in building in security the organizational contacts, domain knowledge, and tooling should already be in place to execute on an effective application monitoring regime. In organizations where this model is still in early days, building visibility in through application monitoring can be an effective first step, but more work is required to set up people, process, and technologies that will work in the environment.

In the next post, we’ll dig deeper into how to get started with this application monitoring process, and how to integrate the data into your existing SIEM/Log Management environment.

No Related Posts


“Build security in” is not an opposing team firmly against SEIM or monitoring. They are both necessary.

In fact, the error reporting and logging chapter in the Developer Guide was influenced by ISO 17799, which comes the ideals of

Deter - note that you WILL be logged
Delay - use the logs and events to drive responses to brute force, etc
Detection - logging AND review. Many orgs forget the latter as it’s expensive
Accountability - Make sure you note WHO did it

I am not a fan of logging for logging’s sake. You MUST have a SEIM or other layer to make sense of the logs you collect. You must set up alerts and escalation and have policies AND responsible parties - who is getting out of bed at 3 AM, who do we call at 4 AM? 

If you’re throwing away 99% of your logs (the phrase I use is artificial ignorance), then don’t collect them. The key is to define, collect and make sense of the logs and events you do collect.

This is why ESAPI’s app governor and the app firewall that run from configurable events within the app (the build security in bit) are so vital.


By Andrew van der Stock

If you like to leave comments, and aren’t a spammer, register for the site and email us at and we’ll turn off moderation for your account.