With last week’s acquisition of Metasploit by Rapid7, I thought it might be a good time to do a review of the penetration testing market and the evolving role of pen testing in the security arsenal. We’ve seen a few different shifts over the past few years in how organizations use pen testing, and I believe this acquisition – combined with changes in enterprise infrastructure – indicates that pen testing is becoming more essential, more closely tied to vulnerability assessment, and generally more mature.
First, a bit of a disclaimer: I’m approaching this as an analyst, not a penetration tester. Although I’ve used many of the tools in demonstrations and the lab, I’ve never worked as a pen tester and don’t claim to have that skill set. I’m fairly sure my BBS hacking experience from the mid-80’s doesn’t really count.
There are two important issues we need to focus on when evaluating penetration testing – changes in need and value, and changes in delivery methods and tools.
The value of penetration testing
There is sometimes a debate on the value of penetration testing. Some question its usefulness, since a test by a competent practitioner is pretty much guaranteed to succeed, but highly unlikely to find every exploit path into the organization. More comprehensive tests will find more holes, but at a much higher cost. In some verticals (particularly financials and some types of government organizations) the risk is so high that this is an accepted cost, but for less-aware and less-targeted verticals, or small and mid-sized organizations, a basic vulnerability or program assessment can find more issues at lower cost.
That’s because, until fairly recently, penetration testing was dominated by external service organizations performing broad network and host based assessments. Tests were used to:
- Scare management into spending more on security.
- Get a general sense of how hardened the organization was.
- Find and fix any obvious holes that might stand out either in an untargeted scan/attack, or to an attacker willing to spend a little more time with limited resources.
Basically, a pen test would give you a good sense of how you’d withstand an attack by an opponent at the same skill level as your testing team, for the amount of time/effort you were willing to pay for. Obviously there are a lot of exceptions, and I’m only talking about general market trends. But at this stage, unless you were a big target, a vulnerability assessment (including an internal assessment) would provide sufficient value at a lower cost.
That’s still how many tests are used, but we’ve seen a shift in the past few years due to a few changes in the risk and threat landscape. Specifically:
- An increase in highly targeted attacks.
- Greater use of web applications, and more web application attacks (one of the single biggest source of losses in recent major reported incidents).
- A market and economic system for taking advantage of exploited data.
- Evolution of technologies & vulnerabilities, coupled with much shorter exploit creation/adoption cycles than in the past. For example, zero day attacks were extremely uncommon just 2-3 years ago, but now seem to appear monthly.
The bad guys are making serious money, are going after harder targets, and are taking advantage of our rapid adoption of web technologies. They really have to, since we’ve gotten a lot better at securing our networks and endpoints (yes, we really have, from an overall trends standpoint).
These factors change the focus and requirements for penetration testing. While this is merely one analyst’s opinion, and some of these are very early trends, here’s what I’m seeing:
- Organizations are increasing the frequency of vulnerability assessments and penetration testing, to reduce between-assessment risks. In some cases these are continuous programs.
- Penetration tests are being more closely tied to vulnerability assessments in order to determine risk and prioritize patches and other defenses.
- The line between a vulnerability assessment and a penetration test is almost completely blurred for web applications – especially custom web applications.
- There is greater use of, and need for, penetration testing during development and pre-production phases, since some testing is prohibitively risky on a production system.
Penetration testing is being more closely tied to vulnerability assessment on non-web systems to help prioritize. A VA doesn’t necessarily tell you how exploitable a target is, and it certainly won’t tell you what the bad guy can potentially gain. A penetration test helps validate the overall risk and determine the potential impact and losses (not in financial terms – that’s for another day). A vulnerability scan can tell you that system X is vulnerable to attack Y, but you often need to go a step further with a pen test to determine if data Z is at risk. This is especially true for web applications, but also important for other types of assets.
The overall focus is shifting away from “Can someone break in, and how long will it take them?” to “Where are we most exposed, and what are our potential losses?” Penetration testing is becoming more of a prioritization and secure development tool.
See part 2 for how these factors change the solutions and penetration testing market
Posted at Wednesday 28th October 2009 10:51 am
(0) Comments •
This morning Dan Goodin over at The Register dropped me a line to get my take on a new tool from Microsoft that lets you apply anti-exploitation controls to existing applications. Here’s Dan’s article with my quote, and more information directly from Microsoft.
This. Is. Awesome.
Here’s why EMET is so significant. Anti-exploitation technologies are incredibly powerful because they reduce the risk that any vulnerability – even a zero day – can actually be exploited to cause harm. They include a bunch of techniques including Data Execution Protection (DEP, which is a software flag enforced at the hardware level), Address Space Layout Randomization (ASLR), and stack protection.
As powerful as these techniques are, the software developer needs to design and build their programs to take advantage of them. Most developers don’t do this yet, which makes their software a major potential weak point for any host security. This is especially problematic with web browser plugins that are leveraged by web-based client-side exploits.
EMET allows anyone to add certain anti-exploitation protections to any program without requiring recompiling. You can now apply four anti-exploitation techniques to an existing application, no matter where you got it from or who programmed it (see Microsoft’s post for the list and explanation). Since this will break some applications, it’s not for the faint of heart, but EMET has per-process granularity which can help you lock something down, while leaving open the bits that break.
It’s very cool, and kudos to Microsoft. We still need to see how well it works in the real world, so hopefully we’ll get some field reports soon.
Posted at Tuesday 27th October 2009 3:24 pm
(1) Comments •
By Adrian Lane
Amazon announced a Relational Database Service today:
Amazon RDS gives you access to the full capabilities of a familiar MySQL database. This means the code, applications, and tools you already use today with your existing MySQL databases work seamlessly with Amazon RDS. Amazon RDS automatically patches the database software and backs up your database, storing the backups for a user-defined retention period.
It was natural to choose the most popular open source database, MySQL 5.1, at least in the short term. With this introduction they have effectively filled out their cloud offering for database infrastructure services. To go along with the existing capabilities of Amazon’s Simple DB and a generic Amazon Machine Image that provide logical instances of any of the major database platforms, you have just about every option you could want as an application developer.
There is a list of pricing options based upon tiers of memory and computational capacity for your web service. Storage is equally flexible, with the ability to select from 5GB to 1TB of storage capacity. Snapshotting, rollbacks, resource monitoring, automated backup, and pretty much everything needed for basic database setup and maintenance.
What Amazon is doing is very cool, but this is a security blog so I need to make a few comments on security and not just act like an RDS fanboi. Which I sometimes hate because I feel like the guy who’s yelling “Hey kid, stop running around with that sharp stick! You’ll poke your eye out!” With the AMI variants, as Amazon takes care of patching and configuration, and the user takes care of access control and identity management. While the instances most likely have security patches applied on a consistent basis, there is a lot more to security than patching IDM. I have no evidence that these database instances are insecure, but no one gets the benefit of the doubt in this case. For most relational database platforms I look at about 125 different database settings in an assessment sweep, most of these are to ensure the factory defaults have been changed. There is no reason to believe that Amazon is doing the same, so protection against SQL injection falls on the shoulders of client developers.
With MySQL databases for RDS, the situation appears to be a little different, as the user has some configuration options. The RDS Developer Guide shows that we can alter port settings and enforce SSL connections. But the API is limited and far more focused on programming than administration. The security guides don’t offer any details on usage of service accounts, default passwords, stored procedure access, networking agents, or other features that are not necessarily masked by the Amazon APIs. Many important security topics are simply not addressed. And odds are, if someone is going after your data, they are going to use SQL injection, default account access, or external stored procedures – all of which are your responsibility to secure. I would have a tough time putting any sensitive data out there until you can verify the security setup. Use caution or you might… oh, never mind.
Posted at Tuesday 27th October 2009 11:33 am
(2) Comments •
By David Mortman
For Adam after harassing me on irc:
Calling ‘accounts’ ‘identities’ is broken. Discuss.
Posted at Monday 26th October 2009 7:39 am
(3) Comments •
By Adrian Lane
The First 90 Days.
When you take a new position, what is it you will do in the first 90 days? What do you want to learn? What do you wish to accomplish? Is it enough to plan a course of action or do you immediately need to fix something? “What is your plan for your first 90 days?” is a common interview question for executives. The candidate’s answer tells the prospective employer a few things about the person’s grasp of the challenges ahead, how they operate typically, the efficiency of their approach, and how well their expectations align. Most candidates are under no illusion about taking a new role. In the best case they are filling a gap in a growing company, but more often than not they are there to fix something broken. The question cements in the mind of the candidate what is expected of them stepping in the door. And more than any other point during your tenure with a company, your first 90 days sets your boss’ and coworkers’ impressions of your effectiveness.
Never in my career has fixing security been in my top 3 challenges for the first 90 days. It’s always been quality of service, failed process, a broken, product or a dysfunctional development team. I have never been a CISO or security officer so in the context of security, I don’t really know how I would answer the question “What would my first 90 days look like?” If you are a security practitioner, how would you answer the question? Or perhaps it is more interesting to ask non-security professionals what their 90-day plan for security is? What challenges could you hope to accomplish? Do you think you could come up with a security program in that amount of time? I am interested in your thoughts on this subject. Is research on the establishment of a security program interesting to you? Let us know what you think.
On to the Friday Summary:
Webcasts, Podcasts, Outside Writing, and Conferences
Favorite Securosis Posts
Other Securosis Posts
Favorite Outside Posts
- Rich: Amrit’s post on Gartner, and working for Gartner. For the record, analysts are very well insulated from financial considerations that could affect research. That said, people who pay to speak to analysts get more time with them, and that can subtly affect opinions.
- Adrian: My favorite post was also Amrit’s, both for his honest quadrant diagram and for the commentary. To be honest, I felt for ZL as Gartner has the power to cut a company’s sales in half, but I agree with their assessments more than I disagree. My favorite tweet was from @securityincite: “@rmogull Would someone please give Rich some work to do? He’s loitering in shopping malls now. Next he’ll be upgrading to Windows Mobile”.
- Mortman: @RSnakes on a Plane. (Mort sent this in Monday, he was so convinced).
- Meier: Two out of five at risk from Wi-Fi Hijacking - Interesting that Talk Talk (the ISP in the UK) is taking this stance to protect end users from heavy-handed plans to tackle Internet piracy by Lord Mandelson.
- Chris Pepper: Time Warner Cable Exposes 65,000 Customer Routers to Remote Hacks.
Top News and Posts
Blog Comment of the Week
This week’s best comment comes from Erik Swan (a Splunk employee -Adrian) in response to Splunk and Unstructured Data:
Thanks for mentioning Splunk, and your post brings up interesting points.
We recommend that people dump “everything” into splunk and just keep it. I’d go further and say that i’d bet that far less than 1% of that data is ever looked-at/reported on/etc. As you point out, its likely harder and more risky to remove data than keep it. This clearly changes when you talk about multiple T per day ( average large system these days ), where even for a wealthy company, the IO required is very expensive and not sure the data has value/risk. My gut is that data generation growth is clearly outpacing the size/price curve per GB, and will likely do so until massively more scaleable and cost effective media is available.
For the time being, keeping everything is likely the best starting point.
At the same time, we have seen models that look a lot like email spam filtering, where “uninteresting” data is routed to different instances that have shorter retention policies. Summarization is used to capture and compress the data hopefully with no information loss. Not a great practice for compliance, but for trouble shooting and analytics can work. Longer term its an interesting area for research and something that due to the size of data we deal with needs to be solved.
Posted at Thursday 22nd October 2009 11:40 pm
(0) Comments •
By David J. Meier
This story begins early last week with a phone call from a bank I hold accounts with. I didn’t actually answer the call but a polite voice mail informed me of possible fraudulent activity and stated I should call them back as soon as possible. First and foremost I thought this part of my story was a social engineering exercise, but I quickly validated the phone number as being legit, unless of course this was some fantastic setup that was either man-in-the-middling the bank’s site (which would allow them to publish the number as valid) or the number itself had been hijacked. Tinfoil hat aside, I called the bank.
A friendly fraud services representative handled my call and in less than twenty minutes we had both come to the conclusion my card for the account in question was finally compromised. By finally, I mean roughly seven years as being my primary vehicle for payment on a daily basis. But this, ladies and gentlemen, is not where the fun started. No, I had to wait for the mail for that.
Fast forward five to seven business days, when a replacement card showed up in my lockbox which, interestingly, is an often-ignored benefit of living in a high density residence. This particular day I received a rather thick stack of mail that included half a dozen similarly sized envelopes. Unfortunately, I quickly knew (without opening any of them) which one contained my new card – and it wasn’t based on feel.
One would think a financial institution might go to trivial lengths to protect card data within an envelope, but clearly not in this case. The problem I had was that four of the sixteen numbers were readable because, and I’m assuming here, some automatic feeding mechanism at the post office put enough pressure on the embossed card number to reprint the number on the outside of the envelope. It was like someone had run that part of the card through an old-school carbon card copy machine. At this point my mail turned into a pseudo scratch lottery game and I was quickly to trying household items to finish what had already been started. I was a winner on the second try (the Clinique “smoldering plum – blushing blush powder brush” was a failure – my fiance was not impressed, and clearly I’ve watched too much CSI: Miami).
Turns out a simple brass key is all that is needed to reveal the rest of the numbers, name, and expiration. At this point I’m conflicted, with two different ideas:
- Relief and confusion: The card security code isn’t embossed. So why must the rest of it be?
- Social engineering: If obtaining card data like this was easy enough, I could devise a scheme where I called recipients new cards with enough data to sound like the bank for many people to give me the security codes.
After considerable thought I feel it’s safe to say that the current method of card distribution poses a low but real level of risk, wherein a significant amount of card data can be discerned short of brute force on the envelope itself. Is it possible? Surely. Is it efficient? Not really. Would someone notice the card data on the back of the envelope? Maybe. But damn – now it really makes sense that folks just go after card data TJX-style, considering all the extra effort in this route.
–David J. Meier
Posted at Wednesday 21st October 2009 6:56 pm
(5) Comments •
By Adrian Lane
“What the heck is up with Splunk”? It’s a question I have been getting a lot lately. From end users and SIEM vendors. Larry Walsh posted a nice article on how Splunk Disrupts Security Log Auditing. His post prodded me into getting off my butt and blogging about this question.
I wanted to follow up on Splunk after I wrote the post on Amazon’s SimpleDB as it relates to what I am calling the blob-ification of data. Basically creating so much data that we cannot possibly keep it in a structured environment. Mike Rothman more accurately called it ” … the further decomposition of application architecture”. In this case we collect some type of data from some type of device, put it onto some type of storage, and then we use a Google-esque search tool to find what we are looking for. And the beauty of Google is that it does not care if it is a web page or voice mail transcript – it will find what you are looking for if you give it reasonable search criteria. In essence that is the value Splunk provides a tool to find information in a sea of data.
It is easy to locate information within a structure repository with known attributes and data types, and we know where certain pieces of information are stored. With unstructured data we may not know what we have or where it is located. For some time normalization techniques were used to introduce structure and reduce storage requirements, but that was a short-lived/low performance approach. Adding attributes to raw data and just linking back to those attributes is far more efficient. Enter Splunk. Throw the data into flat files and index those files. Techniques of tokenization, tagging, and indexing help categorize data with the ultimate goal of correlating events and reporting on unstructured data of differing types. Splunk is not the only vendor who does – several SIEM and Log Management vendors do the same or similar. My point is not that one vendor is better than another, but point out the general trend. It is interesting that Splunk’s success in this area has even taken their competitors by surprise.
Larry’s point …
“The growth Splunk is achieving is due, in part, to penetrating deeper into the security marketplace and disrupting the conventional log management and auditing vendors.”
… is accurate. But they are are able to do this because of the increased volume of data we are collecting. People are data pack-rats. From experience, less than 1% of the logged data I collect has any value. Far too, often organizations do not invest the time to determine what can be thrown away. Many are too chicken to throw useless data away. They don’t want to discard data, just in case it has value, just in case you need it, just in case it contains the needle in the haystack you need for a forensic investigation. I don’t want to be buried under the wash of useless data. My recommendation is to take the time to understand what data you have, determine what you need, and throw the rest away.
The pessimist in me knows that this is unlikely to happen. We are not going to start throwing data away. Storage and computing power are cheap, and we are going to store every possible piece of data we can. Amazon S3 will be the digital equivalent of those U-Haul Self Storage places where you keep your grandmother’s china and all the crap you really don’t want, but think has value. That means we must have Google-like search approaches and indexing strategies that vendors like Splunk provide just to navigate the stuff. Look for unstructured search techniques to be much sought after as the data volumes continue to grow out of control.
Hopefully the vendors will begin tagging data with an expiration date.
Posted at Wednesday 21st October 2009 5:00 pm
(8) Comments •
By Adrian Lane
Rapid7 acquires Metasploit, the open source penetration testing platform. Wow. All I can say is ‘Wow’. I had been hearing rumors that Rapid7 was going to make an acquisition for weeks, but this was a surprise to both Rich and myself. Still coming to terms with what it means, and I have no clue what the financial terms look like, but almost certainly this is a cash+stock deal. On the surface, it is a very smart move for Rapid7.
Metasploit is considerably better known than Rapid7. Metasploit is a fixture in the security research world and there are far more people using Metasploit than Rapid7 has customers. If nothing else, this gets Rapid7 products in the hands of the people who are shaping web application security, and defining how penetration testing and vulnerability management will be conducted. In a quickly evolving market like pen testing, access to that community is invaluable for a commercial vendor. Plus they get H D Moore on staff, which is a huge benefit.
Metasploit is a well-architected framework that provides for easy extensibility and can be customized in innumerable ways. If you want to test anything from smart phones to databases, this platform will do it, from targeted exploits to fuzzing. Sure, there is work on your part and accessibility to people other than security researchers is low compared to commercial products like Core Security’s Impact, but it’s a solid platform and the integration of the two should not be difficult. It’s more a question of how best to allow Metasploit to continue its open source evolution while leveraging scans into meaningful vulnerability chaining, as well as risk scoring.
Neither is exactly an ‘enterprise ready’ product. That’s not a slam, as NeXpose performs its primary function as well as most. But Rapid7’s platform is just now breaking ground into larger companies. They have a long way to go in UI, ease of use, pragmatic analysis, integration of risk scoring, SaaS, exploit chaining, and back-end integration. That said, I am not sure they need to be an enterprise ready product, at least in the short term. It makes more sense to continue their mid-market penetration while they complete the integration. Breadth of function, which is what they now have, has proven to be a major factor in winning deals over the last couple years. They can worry about the advanced non-technical stuff later.
Identity in the market is an issue for Rapid7. They have waffled between general assessment, pen testing, and vulnerability management, without a clear identity or differentiator when going toe-to-toe with Qualys, nCircle, Tenable, Secunia, and the like. Sure, ‘compliance scoring’ is a useful marketing gimmick, but Metasploit gives them a unique identity and differentiation. Rather than scan-and-patch for known vulnerabilities, focusing mostly inside the network, they will now be able to go far deeper into externally facing custom applications. Taking a risk score across multiple applications and/or platforms is a better approach. If the two platforms are properly integrated, they’ll be useful to IT, security, and software development.
I am sure Rich will chime in with his own take later in the week. Wow.
Posted at Wednesday 21st October 2009 9:47 am
(0) Comments •
By David Mortman
There were some great comments on my last post, which bring to light a serious problem with the way authorization is done today and how roles don’t help as much as we’d like. First we hear from LonerVamp:
And even if you get the authentication part down, very few apps that I’ve seen then tie back into whatever is in place for role management.
This is an important point that often gets glossed over by IDM vendors. It turns out that while many applications have support for third party authentication mechanisms, very few have support for third party authorization methods. Which means that even if you can centralize your identities for the purposes of account creation/deletion, you still have to manage use inside each application. Furthermore, many of the applications that claim to support third party authorization really turn out to only support third party groups in LDAP or RADIUS, but you still have to map those groups onto roles within the applications.
Andrew Yeomans followed up with his own comment that shows that he’s been a dedicated Securosis reader for a while now:
I’m starting to think that a data-centric approach may be a way forward.
Today, authorizations are generally enforced by applications. Now firstly this leads to high complexity (as you describe) as there is no unifying set of “policy decision points” and “policy enforcement points”. Secondly, it allows for authorization restrictions to be bypassed by other applications that have access to the same data.
Andrew really hits the nail on the head here. We need to continue our shift towards Data-centric Security. The Data Security Lifecycle explicitly assumes that you can properly assign and control rights to who has what data, which is why IDM is so important. I’ve said it before and I’ll say it again: If you don’t know who is accessing the data, how can you possibly tell if it is being abused or misused?
Finally Omie asked:
I’ve been hearing too much about identity management recently and how the move to roles will solve our compliance problems. And I’ve been wondering and asking how we plan to keep the roles maintained over time. Of course I’ve also been under the impression that every other organization has figured that out except ours, but your post is making me rethink that assumption. If there are some best practices/examples of how to approach role maintenance, I would love to learn about them.
Roles can definitely help you out with compliance, but you are correct – role maintenance is definitely a challenge. There is often an implicit assumption that roles, like the rest of the application configuration, are static, when in reality roles tend be dynamic so you absolutely need a process for adapting roles as necessary. Often the complexity of the application causes admins to add roles rather then edit the existing ones because it is easier in the short term. But in the long run this causes extra complexity. I’ll go into more details on this issue and how to deal with it in a later post, so stay tuned. In the meantime, NIST recently published some documents from their recent Privilege (Access) Management Workshop. In particular, you should check out A Survey of Access Control Models, to give you an idea of some ways that role based access control is problematic.
Posted at Tuesday 20th October 2009 7:00 am
(2) Comments •
Like many of you, I get a ton of spam/phishing email to my various accounts. Since my email is very public, I get a little more than most people. It’s so bad I use 3 layers of spam/virus filtering, and still have some messages slip through (1 cloud based filter [Postini, which will probably change soon], one on-premise UTM [Astaro], and SpamSieve on my Mac). If something gets through all of that, I still have some additional precautions I take on my desktop to (hopefully) help against targeted malware. Despite all that, I assume that someday I’ll be compromised, and it will probably be ugly.
This morning I got the first phishing email in a very long time that almost tricked me into clicking. It came from “Administrator” at one of my hosts and read:
On October 22, 2009 server upgrade will take place. Due to this the system may be offline for approximately half an hour.
The changes will concern security, reliability and performance of mail service and the system as a whole.
For compatibility of your browsers and mail clients with upgraded server software you should run SSl certificates update procedure.
This procedure is quite simple. All you have to do is just to click the link provided, to save the patch file and then to run it from your computer location. That’s all.
http://updates.[cut for safety]
Thank you in advance for your attention to this matter and sorry for possible inconveniences.
Two things tipped me off. First, that system is a private one administered by a friend. While he does send updates like this out, he always signs them with his name. Second, the URL is clearly not really that domain (but you have to read the entire thing). And finally, it leads to an Active Server Pages domain, which that administrator never uses since our system is *nix based.
But it was early in the morning, I hadn’t had coffee yet, and we often need to upgrade our SSL after a system update on this server, so I still almost clicked on it.
According to Twitter this is a Zbot generated message:
SecBarbie: RT @mikkohypponen ZBot malware being spammed out right now in emails starting “On October 22, 2009 server upgrade will take place” Ignore it.
It’s interesting that despite multiple obvious markers this was malicious, and be being very attuned to these sorts of things, I still almost clicked on it. It just goes to show you how easy it is to screw up and make a mistake, even when you’re a paranoid freak who really shouldn’t be let out of the house.
Posted at Monday 19th October 2009 8:58 am
(2) Comments •
All last week I was out of the office on vacation down in Puerto Vallarta. It was a trip my wife and I won in a raffle at the Phoenix Zoo, which was pretty darn cool.
I managed to unplug far more than I can usually get away with these days. I had to bring the laptop due to an ongoing client project, but nothing hit and I never had to open it up. I did keep up with email, and that’s where things got interesting.
Before heading down I added the international plan to my iPhone, for about $7, which would bring my per-minute costs in Mexico down from $1 per minute to around $.69 a minute. Since we talked less than 21 minutes total on the phone down there, we lose.
For data, I signed up for the 20 MB plan at a wonderfully fair $25. You don’t want to know what a 50 MB plan costs. Since I’ve done these sorts of things before (like the Moscow trip where I could never bring myself to look at the bill), I made sure I reset my usage on the iPhone so I could carefully track how much I used.
The numbers were pretty interesting – checking my email ranged from about 500K to 1MB per check. I have a bunch of email accounts, and might have cut that down if I disabled all but my primary accounts. I tried to check email only about 2-3 times a day, only responding to the critical messages (1-4 a day). That ate through the bandwidth so quickly I couldn’t even conceive of checking the news, using Maps, or nearly any other online action. In 4 days I ran through about 14 MB, giving me a bit more space on the last day to occupy myself at the airport.
To put things in perspective, a satellite phone (which you can rent for trips – you don’t have to buy) is only $1 per minute, although the data is severely restricted (on Iridium, unless you go for a pricey BGAN). Since I was paying $3/minute on my Russia trip, next time I go out there I’ll be renting the sat phone.
So for those of you who travel internationally and want to stay in touch… good luck.
On to the Summary:
Webcasts, Podcasts, Outside Writing, and Conferences
Favorite Securosis Posts
Other Securosis Posts
Favorite Outside Posts
Top News and Posts
Blog Comment of the Week
This week’s best comment comes from Rob in response to Which Bits are the Right Bits:
Perhaps it is not well understood that audit logs are generally not immutable. There may also be low awareness of the value of immutable logs: 1) to protect against anti-forensics tools; 2) in proving compliance due diligence, and; 3) in providing a deterrent against insider threats.
Posted at Thursday 15th October 2009 11:45 pm
(1) Comments •
By David J. Meier
Today you’d be hard pressed to find a decent sized network that doesn’t have some implementation of Security Event Management (SEM). It’s just a fact of modern regulation that a centralized system to collect all that logolicious information makes sense (and may be mandatory). Part of the problem with architecting and managing these systems is that one runs into the issue of securely collecting the information and subsequently verifying its authenticity.
Almost every network-aware product you might buy today has a logging capability, generally based on
syslog – RFC3164. Unfortunately, as defined,
syslog doesn’t provide much security. In fact if you need a good laugh I’d suggest reading section 6 of the RFC. You’ll know you’re in the right place when you start to digest information about odors, moths and spiders. It becomes apparent, very quickly, when reading subparagraphs 6.1 through 6.10, that the considerations outlined are there more to tip you off that the authors already know
syslog provides minimal security – so don’t complain to them. At this point most sane people question using such a protocol at all because surely there must be something better, right? Yes and no.
First let me clarify: I didn’t set out to create an exhaustive comparison of [enter your favorite alternative to syslog here] for this writeup. Sure RFC5424 obsoletes the originally discussed RFC3164 and yes RFC5425 addresses using TLS as a transport to secure
syslog. Or maybe it would be better to configure BEEP on your routers and let’s not forget about the many proprietary and open source agents that you can install on your servers and workstations. I freely admit there are some great technologies to read about in event logging technology. The point though is that since there is considerable immaturity and many options to choose from, most environments fall back to the path of least resistance: good ol’
syslog over UDP.
Unfortunately I’ve never been asked how to do logging right by a client. As long as events are streaming to the SEM and showing up on the glass in the NOC/SOC, it’s not a question that comes up. It may not even be a big deal right now, but I’d be willing to bet you’ll see more on the topic as audits become more scrutinizing. Shouldn’t the integrity of that data be something a little more robust than the unreliable, unauthentic, repudiable and completely insecure protocol you probably have in production? You don’t have to thank me later, but I’d start thinking about it now.
–David J. Meier
Posted at Thursday 15th October 2009 5:14 pm
(2) Comments •
By David Mortman
As I have mentioned in previous posts, although in principle IDM isn’t that complicated, real world practicalities make it fairly challenging. To quote myself:
Businesses can have hundreds if not thousands of applications (GM purportedly had over 15,000 apps at one point) and each application itself can have hundreds or thousands of roles within it. Combine this with multiple methods of authentication and authorization, and you have a major problem on your hands which makes digging into the morass challenging to say the least.
I was chatting with Rich the other day, and he reminded me that there were several issues that I hadn’t brought up yet. By way of example, he sent me the following list:
- Most organizations of even moderate size have already lost control of identities.
- The challenge is to regain control, while instituting the processes you describe.
- The average large enterprise has 1 custom application for every 100 employees
- A surprising number of enterprise applications have more roles than users (especially ERP systems, as admins “cheat” and create new roles for different tasks of a user to deal with application restrictions).
- Due to compliance, for in-scope applications you often need a full user/role analysis… 2 years ago.
- Because of these factors, the IDM market is massive and has been one of the fastest growing in the security industry for years (I think it’s tapering off now, but it really isn’t my area so I could be wrong).
- The IDM market itself is complex, with multiyear implementations, and customer satisfaction is often low until the pain of implementation is complete.
- There are still a lot of issues integrating any kind of IDM into many custom applications.
As if that weren’t enough to make you want to go do something easier, the problem gets even worse when you realize that while compliance forces organizations to evaluate annually whether people should still have the roles they do, there is little to no compliance required around regularly adjusting the roles themselves within applications. As a result, the roles themselves only get reevaluated at major upgrade times (and often not even then) which means maybe every 18-24 months. The ugliness comes in because the business tends to change its needs much more quickly – this is why most major ERM, ERP, CRM, etc. rollouts fail: IT simply can’t keep up with the business. So it’s not enough to just solve the process problem, but in the long run, you will also need to deal with some fundamental business and IT cultural issues of how applications are handled. Essentially, IT will have to become a lot more agile in its ability to respond to changing business needs. While this is hardly limited to the IDM space, IDM nicely highlights the issue. After all, if the roles are meaningless, knowing who has what role may be helpful from an incident response or investigation standpoint, but cannot really help understand your risk or compliance profile.
Posted at Thursday 15th October 2009 2:11 pm
(4) Comments •
I was reviewing the recent Health and Human Services guidance on medical data breach notifications and it’s clear that the HHS either was bought off, or doesn’t understand the fundamentals of risk assessment. Having a little bit of inside experience within HHS, my vote is for willful ignorance.
Basically, the HHS provides some good security guidance, then totally guts it. Here’s a bit from the source article with the background:
The American Recovery and Reinvestment Act of 2009 (ARRA) required HHS to issue a rule on breach notification. In its interim final rule, HHS established a harm standard: breach does not occur unless the access, use or disclosure poses “a significant risk of financial, reputational, or other harm to individual.” In the event of a breach, HHS’ rule requires covered entities to perform a risk assessment to determine if the harm standard is met. If they decide that the risk of harm to the individual is not significant, the covered entities never have to tell their patients that their sensitive health information was breached.
You have to love a situation where the entity performing the risk assessment for a different entity (patients) is always negatively impacted by disclosure, and never impacted by secrecy. In other words, the group that would be harmed by protecting you gets to decide your risk.
Yeah, that will work.
This is like the credit rating agencies, many aspects of fraud and financial services, and more than a few breach notification laws. The entities involved face different sources of potential losses, but the entity performing the assessment has an inherent bias to mis-assess (usually by under-assessing) the risk faced by the target.
Now, if everyone involved is altruistic and unbiased this all works like a charm. Hell, even in Star Trek they don’t think human behavior that perfect.
Posted at Thursday 15th October 2009 10:30 am
(1) Comments •
By Adrian Lane
(The following post covers some rather esoteric bits of security philosophy, or what Rich has affectionately called “Security Jazz” in the past. Unless you are into obscure data-centric security minutiae, you will probably not be interested).
Richard Bejtlich tweeted and posted on data integrity:
The trustworthiness of a digital asset is limited by the owner’s capability to detect incidents compromising the integrity of that asset.
This statement is absolutely correct and a really important point that is often overlooked. The problem is that most technologies which produce digital assets do not build tamper detection in, thus giving owners no way to detect integrity violtaions. And far too often people confuse interested party with owner of digital assets, as there can be many copies, each in the possession of a different person or group. It’s not that we can’t provide validation, because technology exists to provide assurance and authenticity.
Let’s look at an example: Who owns
syslog data? Is it the IT administrator? The security professional? An auditor? In my opinion, none of them do. The OS owns the
syslog, as it created the content. Much like you may think you own ‘your’ credit card number, but you don’t – it is something the issuing bank created and owns. They are the custodians of that number, and change it when they choose to.
syslog has no way to verify the contents of the log it creates over time. We take it on faith that it is unlikely a log file was corrupted or altered. If we need to verify integrity in the future, too bad. If you did not build in safeguards and a method for validating integrity when you created the data, it’s too late. The trustworthiness of the digital asset is limited to the owner’s capability to detect a compromise, and for many digital assets like
syslog, that is nil.
For most digital assets, it is sufficient that we use them every day, as this provides sufficient confidence in their integrity. Encryption keys are a useful example. If the keys are corrupted, especially in a public-key situation, either the encryption or decryption operations fail. We may keep a backup somewhere safe to compare our working copy to, and while that can be effective in the most common problem situations, it’s only relevant for certain (common) use cases. Digital assets have an additional challenge over physical objects in terms of generations. Even if we can verify a particular iteration of a digital object, we can have infinite copies, so we need to be able to verify the most current iteration is in use.
For digital assets like encryption keys, account numbers, access tokens, and digital representations of self, the owner has a strong vested interest in not sharing the asset, keeping it safe, and possibly even keeping redundant copies against future emergencies or for verification. There are several technologies to prove integrity, they are just not used much. I posted a comment on Richard’s blog to this effect:
The trustworthiness of a digital asset is limited more by the trustworthiness of the owner than tamper detection. An owner with desire of privacy and data integrity has the means to protect digital assets.
Richard’s premise is an important one as we very seldom build in safeguards to validate ownership, state, authenticity or integrity. Non-repudiation tools and digital escrow services are nearly non-existent. There simply is not enough motivation to implement the tools we have which can provide assurance.
Gunnar Peterson blogged on this subject earlier this week as well, taking a slightly more applied look at the problem. His statement that these issues are outside the purview of DLP are absolutely correct. DLP is an outside-in model. This discussion has more to do with Digital Rights Management, which is an inside-out model. The owner must attest to integrity, and while a 3rd party proxy such as a DLP service could be entrusted with object escrow and integrity certification, it would require an alteration of the DLP’s “discover and protect” model. DRM is designed to be part of the application that creates the digital object, and while it is not often discussed, digital object ownership is part of that responsibility. Attestation to ownership is not possible without some form of integrity and state checking.
I have seen select DRM systems that were interested in high integrity, but none were commercially viable. Which answers Gunnar’s question:
Our ability using today’s technologies to deliver vastly improved audit logging is, I believe, a worthwhile and achievable goal. But it’s fair to ask - why hasn’t it happened yet?
There has been no financial incentive to do so. We have had excellent immutable log technologies for years but they are only used in limited cases. Web application audit trails are an interesting application of this technology and easy to do, but there is no compelling business problem motivating people to spend money on retrofitting what they have. I would like to see this type of feature for consumer protection, built into financial transactions where we really need to protect consumers from shoddy corporate record-keeping and failed banking institutions.
Posted at Wednesday 14th October 2009 2:30 pm
(3) Comments •