Understanding and Selecting DSP: Data and Event Collection

In our previous post on DSP components we outlined the evolution of Database Activity Monitoring into Database Security Platforms. One of its central aspects is the evolution of event collection mechanisms from native audit, to monitoring network activity, to agent-based activity monitoring. These are all database-specific information sources. The evolution of DAM has been framed by these different methods of data collection. That’s important, because what you can do is highly dependent on the data you can collect. For example, the big reason agents are the dominant collection model is that you need them to monitor administrators – network monitoring can’t do that (and is quite difficult in distributed environments). The development of DAM into DSP also entails examination of a broader set of application-related events. By augmenting the data collection agents we can examine other applications in addition to databases – even including file activity. This means that it has become possible to monitor SAP and Oracle application events – in real time. It’s possible to monitor user activity in a Microsoft SharePoint environment, regardless of how data is stored. We can even monitor file-based non-relational databases. We can perform OS, application, and database assessments through the same system. A slight increase in the scope of data collection means much broader application-layer support. Not that you necessarily need it – sometimes you want a narrow database focus, while other times you will need to cast a wider net. We will describe all the options to help you decide which best meets your needs. Let’s take a look at some of the core data collection methods used by customers today: Event Sources Local OS/Protocol Stack Agents: A software ‘agent’ is installed on the database server to capture SQL statements as they are sent to the databases. The events captured are returned to the remote Database Security Platform. Events may optionally be inspected locally by the agent for real-time analysis and response. The agents are either deployed into the host’s network protocol stack or embedded into the operating system, to capture communications to and from the database. They see all external SQL queries sent to the database, including their parameters, as well as query results. Most critically, they should capture administrative activity from the console that does not come through normal network connections. Some agents provide an option to block malicious activity – either by dropping the query rather than transmitting it to the database, or by resetting the suspect user’s database connection. Most agents embed into the OS in order to gain full session visibility, and so require a system reboot during installation. Early implementations struggled with reliability and platform support problems, causing system hangs, but these issues are now fortunately rare. Current implementations tend to be reliable, with low overhead and good visibility into database activity. Agents are a basic requirement for any DSP solution, as they are a relatively low-impact way of capturing all SQL statements – including those originating from the console and arriving via encrypted network connections. Performance impact these days is very limited, but you will still want to test before deploying into production. Network Monitoring: An exceptionally low-impact method of monitoring SQL statements sent to the database. By monitoring the subnet (via network mirror ports or taps) statements intended for a database platform are ‘sniffed’ directly from the network. This method captures the original statement, the parameters, the returned status code, and any data returned as part of the query operation. All collected events are returned to a server for analysis. Network monitoring has the least impact on the database platform and remains popular for monitoring less critical databases, where capturing console activity is not required. Lately the line between network monitoring capabilities and local agents has blurred. Network monitoring is now commonly deployed via a local agent monitoring network traffic on the database server itself, thereby enabling monitoring of encrypted traffic. Some of these ‘network’ monitors still miss console activity – specifically privileged user activity. On a positive note, installation as a user process does not require a system reboot or cause adverse system-wide side effects if the monitor crashes unexpectedly. Users still need to verify that the monitor is collecting database response codes, and should determine exactly which local events are captured, during the evaluation process. Memory Scanning: Memory scanners read the active memory structures of a database engine, monitoring new queries as they are processed. Deployed as an agent on the database platform, the memory scanning agent activates at pre-determined intervals to scan for SQL statements. Most memory scanners immediately analyze queries for policy violations – even blocking malicious queries – before returning results to a central management server. There are numerous advantages to memory scanning, as these tools see every database operation, including all stored procedure execution. Additionally, they do not interfere with database operations. You’ll need to be careful when selecting a memory scanning product – the quality of the various products varies. Most vendors only support memory scanning on select Oracle platforms – and do not support IBM, Microsoft, or Sybase. Some vendors don’t capture query variables – only the query structure – limiting the usefulness of their data. And some vendors still struggle with performance, occasionally missing queries. But other memory scanners are excellent enterprise-ready options for monitoring events and enforcing policy. Database Audit Logs: Database Audit Logs are still commonly used to collect database events. Most databases have native auditing features built in; they can be configured to generate an audit trail that includes system events, transactional events, user events, and other data definitions not available from any other sources. The stream of data is typically sent to one or more locations assigned by the database platform, either in a file or within the database itself. Logging can be implemented through an agent, or logs can be queried remotely from the DSP platform using SQL. Audit logs are preferred by some organization because they provide a series of database events from the perspective of the database. The audit trail reconciles database rollbacks, errors, and uncommitted statements – producing an accurate representation of changes

Read Post

Incite 3/7/2012: Perspective

Life is a series of ebbs and flows. Highs and lows. Crests and troughs. It’s a yin/yang thing, and unfortunately most folks can’t appreciate that. Especially when they can’t see their way out of a down period. For a lot of security folks, the last two weeks have been such a contrast between those highs and lows that many are probably feeling whiplash. A lot of folks went to the RSA Conference last week and saw an industry thriving again after 3 years in the doldrums. We all felt good. Those who read blog posts and tweets from folks at the conference felt good. It was one of those highs, and I returned to ATL exhausted but in good spirits. Not necessarily feeling like the tide had turned, but that swimming upstream wouldn’t be as hard for a while – however brief. Then the discussions about whether we are losing started early this week. Ben’s post on LiquidMatrix verbalized a lot of what we all feel from time to time. And the burnout, building brick by brick which Rich described so eloquently is a clear explanation of the phenomenon. Rich’s point is that we will always have bad days, just as we have good days. And those who can survive in security for a long time don’t take things personally – especially the bad days. They know (and appreciate) the futility of the game, and enjoy the battles. The learning. The teamwork. They don’t get bitter and angry about the stupidity or the politics or the apathy. Or they hit the wall. Hard. Which is really the point. It’s not about winning or losing. It’s about enjoying the journey. You will lose some battles, just as you will win some. You may lose more than you win, but that’s because the game is rigged. Like Vegas. In the long run, math wins. It’s always been that way, and yet we (amazingly enough) still function. As Ranum says, the Internet will be as secure as it needs to be. In the wake of the shocking news that Sabu was an informer (sound familiar? Gonzalez the Sequel?) and he provided the smoking guns to take down LulzSec, some folks started gloating. That good wins over evil crap. But now is not the time to gloat. Nor is every compromise or incident the time to let despondency or depression creep in. If you get too high or too low you’ll burn out. Been there. Done that. To remain on even keel requires perspective. Perspective that is hard to appreciate when you are in the trenches and on the front lines. On the flight back from RSA we flew into a pretty nasty storm. The last 30 minutes of the flight was turbulent. Regardless of my understanding of statistics, which dictates that I’m as safe in the air during heavy turbulence as I am now – sitting in a coffee shop writing this missive – it’s still a bit unsettling. So I closed my eyes and visualized riding a roller coaster, which I love to do. The exhilaration, the perception of danger, the adrenaline rush – you get off a coaster feeling alive. Maybe a bit scared, but alive. And you want to do it again. That flight was a microcosm of life. Smooth and comfortable for a while, then not so much. Highs, lows, and everything in between. I enjoyed the flight because the bumpy air is part of the deal. You can’t avoid it – not entirely. So I chose to have perspective and enjoy the coaster. I just wish more folks in security could appreciate the journey… -Mike Photo credits: “Learning Perspective” originally uploaded by Yelnoc Lazy Deal Analysis: Trustwave buys another laggard We don’t care enough about the Trustwave/M86 merger to do a stand-alone post, but it does warrant a least a little snark… erm… analysis. 86-it: Trustwave announced today that they will be putting M86 out of its misery, acquiring the mixed bag of stuff web and email security vendor for an undisclosed sum. For those with long memories, M86 was formed as the merger of creaky web security appliance vendor 8e6 with the seriously outdated Marshall mail security software. The resultant M86 company tried to acquire themselves into relevance, making sage investments in Finjan’s secure web gateway software and Avinti’s behavior-based malware detection software. Yeah, 10 pounds of crap in a 5-pound bag. While those products were great additions, the core capabilities were several years behind the competition – and worse, never fully integrated. Details, details. While their Firefox secure browsing plugin was a fun toy, their ability to protect cloud data was suspect and the product development roadmap seemed driven by the trend du jour, rather than some holistic vision of web user security. Trustwave’s acquisition strategy has been reminiscent of the island of lost toys: buying laggards like Vericept, Mirage Networks, Breach Security, BitArmor, ControlPath, and Intellitactics. From that perspective M86 is a good fit with little overlap, but without really integrating the offerings, this is just more integration on the PO. More likely they will continue to target customers too lazy to perform a head-to-head comparisons with class-leading products and those trying to make audit deficiencies (found by Trustwave themselves, in an unholy alliance of audit and security product) go away. – AL & MR Incite 4 U Don’t be Lulzed into a false sense of security: By the time I submit this to Mike I’m sure someone else will slip in a link to the story about LulzSec getting nailed by the FBI with some good old-fashioned police work. You know, attempting to scare the crap out of the perp and turn him against his friends. Uh, like they did to Sabu. To be honest, the headlines don’t really matter that much to those of us in operational security (including me – someone has to keep Mike and Adrian safe) as we are pretty pragmatic about the media’s incentive to work everyone into a frenzy. Rafal Los does a great job pointing out how to handle headline hysteria. Raf’s point is to ignore the headlines, focus

Read Post

Totally Transparent Research is the embodiment of how we work at Securosis. It’s our core operating philosophy, our research policy, and a specific process. We initially developed it to help maintain objectivity while producing licensed research, but its benefits extend to all aspects of our business.

Going beyond Open Source Research, and a far cry from the traditional syndicated research model, we think it’s the best way to produce independent, objective, quality research.

Here’s how it works:

  • Content is developed ‘live’ on the blog. Primary research is generally released in pieces, as a series of posts, so we can digest and integrate feedback, making the end results much stronger than traditional “ivory tower” research.
  • Comments are enabled for posts. All comments are kept except for spam, personal insults of a clearly inflammatory nature, and completely off-topic content that distracts from the discussion. We welcome comments critical of the work, even if somewhat insulting to the authors. Really.
  • Anyone can comment, and no registration is required. Vendors or consultants with a relevant product or offering must properly identify themselves. While their comments won’t be deleted, the writer/moderator will “call out”, identify, and possibly ridicule vendors who fail to do so.
  • Vendors considering licensing the content are welcome to provide feedback, but it must be posted in the comments - just like everyone else. There is no back channel influence on the research findings or posts.
    Analysts must reply to comments and defend the research position, or agree to modify the content.
  • At the end of the post series, the analyst compiles the posts into a paper, presentation, or other delivery vehicle. Public comments/input factors into the research, where appropriate.
  • If the research is distributed as a paper, significant commenters/contributors are acknowledged in the opening of the report. If they did not post their real names, handles used for comments are listed. Commenters do not retain any rights to the report, but their contributions will be recognized.
  • All primary research will be released under a Creative Commons license. The current license is Non-Commercial, Attribution. The analyst, at their discretion, may add a Derivative Works or Share Alike condition.
  • Securosis primary research does not discuss specific vendors or specific products/offerings, unless used to provide context, contrast or to make a point (which is very very rare).
    Although quotes from published primary research (and published primary research only) may be used in press releases, said quotes may never mention a specific vendor, even if the vendor is mentioned in the source report. Securosis must approve any quote to appear in any vendor marketing collateral.
  • Final primary research will be posted on the blog with open comments.
  • Research will be updated periodically to reflect market realities, based on the discretion of the primary analyst. Updated research will be dated and given a version number.
    For research that cannot be developed using this model, such as complex principles or models that are unsuited for a series of blog posts, the content will be chunked up and posted at or before release of the paper to solicit public feedback, and provide an open venue for comments and criticisms.
  • In rare cases Securosis may write papers outside of the primary research agenda, but only if the end result can be non-biased and valuable to the user community to supplement industry-wide efforts or advances. A “Radically Transparent Research” process will be followed in developing these papers, where absolutely all materials are public at all stages of development, including communications (email, call notes).
    Only the free primary research released on our site can be licensed. We will not accept licensing fees on research we charge users to access.
  • All licensed research will be clearly labeled with the licensees. No licensed research will be released without indicating the sources of licensing fees. Again, there will be no back channel influence. We’re open and transparent about our revenue sources.

In essence, we develop all of our research out in the open, and not only seek public comments, but keep those comments indefinitely as a record of the research creation process. If you believe we are biased or not doing our homework, you can call us out on it and it will be there in the record. Our philosophy involves cracking open the research process, and using our readers to eliminate bias and enhance the quality of the work.

On the back end, here’s how we handle this approach with licensees:

  • Licensees may propose paper topics. The topic may be accepted if it is consistent with the Securosis research agenda and goals, but only if it can be covered without bias and will be valuable to the end user community.
  • Analysts produce research according to their own research agendas, and may offer licensing under the same objectivity requirements.
  • The potential licensee will be provided an outline of our research positions and the potential research product so they can determine if it is likely to meet their objectives.
  • Once the licensee agrees, development of the primary research content begins, following the Totally Transparent Research process as outlined above. At this point, there is no money exchanged.
  • Upon completion of the paper, the licensee will receive a release candidate to determine whether the final result still meets their needs.
  • If the content does not meet their needs, the licensee is not required to pay, and the research will be released without licensing or with alternate licensees.
  • Licensees may host and reuse the content for the length of the license (typically one year). This includes placing the content behind a registration process, posting on white paper networks, or translation into other languages. The research will always be hosted at Securosis for free without registration.

Here is the language we currently place in our research project agreements:

Content will be created independently of LICENSEE with no obligations for payment. Once content is complete, LICENSEE will have a 3 day review period to determine if the content meets corporate objectives. If the content is unsuitable, LICENSEE will not be obligated for any payment and Securosis is free to distribute the whitepaper without branding or with alternate licensees, and will not complete any associated webcasts for the declining LICENSEE. Content licensing, webcasts and payment are contingent on the content being acceptable to LICENSEE. This maintains objectivity while limiting the risk to LICENSEE. Securosis maintains all rights to the content and to include Securosis branding in addition to any licensee branding.

Even this process itself is open to criticism. If you have questions or comments, you can email us or comment on the blog.