Network Security Podcast, Episode 104

Martin and I were all over the map this week, but still managed to keep things under 30 minutes. We talk about the Dave and Buster’s hack, data exposure in Chile, and browser virtualization, among other things. The show is up over at netsecpodcast.com.

New Nessus Licensing: NSP Interview With Ron Gula, CEO Of Tenable

If you didn’t catch the news today, Tenable is changing the Nessus license and enabling the real-time signature/plugin feed for the free version. Martin and I managed to snag Ron Gula for a short interview we posted over at NetSecPodcast.com.

Overall I think it’s a very positive license change and it shouldn’t hurt you unless you were using the free version for commercial purposes.

GRC, Average Deal Size, And The Dangers Of Venture Capital

Hot on the heels of my GRC is Dead post, an associate sent me a private rant on a past experience where the investors drove his company down a similar rathole.

Here’s the thing, kids; venture capitalists aren’t there to help you build a long-term business. Their entire goal is to achieve specific returns in specific time periods by driving your company into an exit (IPO or sale). You become a slave to your investors, many of whom aren’t as business savvy as you might think, and most of whom don’t understand your particular market.

My friend is allowing me to post this since he can’t. Keep in mind that some of the biggest “GRC” pushers out there are large companies without VCs (Oracle, SAP, IBM), which thus are running under different dynamics.

The Three Magic Words (or: Why GRC won’t die until the companies do)

I read Mogull’s post on GRC yesterday, and I found myself nodding in agreement with all of it. The basic thrust of the article:

“GRC is now code for “selling stuff to the C-level”. It has little to do with real governance, risk, and compliance; and everything to do with selling under-performing products at inflated prices. When a vendor says “GRC” they are saying, “here’s our product to finally get us into the Board Room and the CEO’s office”. The problem is, there isn’t a market for GRC.”

This is exactly what GRC is about. But why? Why would our vendor community spend all of their time trying so hard to get into the Board Room and the CEO’s office when there’s an entire market out there of businesses to whom we could sell products? Statistics say that 99% of businesses in the USA have less than 1000 employees: that’s a HUGE market for security software and services that are reasonably priced and deliver value. Why are there almost no vendors looking to that market? And why are many of the ones who do (e.g., UTM vendors) mocked and ridiculed?

Three magical words: “Average Deal Size”.

I learned these magic words when I was a relative newbie in information security, working for a vulnerability management vendor whose aim it was to sell appliances into all parts of the enterprise. They believed that vulnerability management was the kind of tool that needed to be embedded into every subnet within the entire organization, and that a huge infrastructure would be built up to manage vulnerabilities. When looking at who they wanted to be when they grew up, the names that were mentioned were SAP and PeopleSoft. That the CEO should have vulnerability reports on his desk every week. And that the CEO should be reporting vulnerability metrics to the Board at the quarterly meeting. (No, I’m not exaggerating. This is what they believed.)

Unfortunately, no customer seemed to care that much about vulnerabilities. Even in the FUD-laden heyday of worms and viruses (Slammer, Blaster and Nimda, Oh My!), nobody wanted to drop vulnerability management tools on every subnet and embed vulnerability management deep into their business process. It was added cost without incremental benefit. And no CEO really cared about seeing the reports. Most CISOs didn’t even want to see the reports.

At the same time, another company was eating our lunch by offering to scan from the outside, on the web. They were basically giving the service away, selling little scans at $5K for anybody who wanted them. And they were rolling in cash compared to us. So, being the go-getter that I was, I put together a plan to create a competitive business within our company. Even with ridiculously conservative estimates, we were going to double revenue within a year (because it’s not hard to double an infinitesimally small number).

And I took it to senior management, who summarily rejected it. I didn’t understand, and I fought hard, but their answer was firm: “No way.”

I was confused and dejected. This seemed stupid - it didn’t make any business sense. The VP of sales saw this and took me aside after the meeting. He explained it to me, and it was the first time I had heard the three magic words. He would open my eyes to one of the things that makes startups do things that appear absolutely idiotic.

He explained that the reason they wouldn’t compete with the other vendor was that it would lower our “average deal size”. That they would rather have a single $100K deal than 100 $2K deals, even if it was only half of the revenue.

It didn’t make sense to me (it still doesn’t), so I asked him why.

“Because that’s one of the big things that venture capitalists care about when they’re valuing your company”, he said. “And our board is made up of our venture capitalists.” The lights went on at that moment.

Fast forward to today. The push toward GRC isn’t because it makes business sense for any of the vendors (i.e., will increase revenue or reduce costs), but entirely because the vendors in the space are worshipping the gods of VC-driven boards who are using average deal size as the metric. It’s why you see companies that are making good progress in mid-market or the mid-range of the small enterprise suddenly declare that their target is the C-level of the “Global 2000″ companies.

The problem with this is that most 100-person companies are entirely ill-suited to live in that environment. Large enterprises demand (to use Moore’s term) the “whole product”. A full support staff, complementary products, training, and serious hand-holding resources that a 100-person, $10M company just doesn’t have. And, having worked in startups for the majority of the last 10 years, I can say that it kills more than it benefits. The burdens of supporting large, enterprise customers are burdens that, for the most part, only large, enterprise vendors are built to support.

It always surprises me when a successful company (e.g., a small consulting company) that is ideally suited to selling, marketing and positively pwning the mid-market and mid-level sales decides to turn up-market (and become a “GRC company”) to compete against companies that are built for that environment (e.g., E&Y, Accenture, IBM Global Services, etc.). Rather than taking the market that they have built themselves up to succeed at, they enter a market that they’re entirely ill-suited for, and go through multiple VPs of sales and marketing wondering why their pipeline is weak.

But the three magic words are powerful. They blind men and women to smart business decisions (mostly because they are used at board meetings). And they create companies that are willing to give anything to end up at the top end of the market, even if they have to make up acronyms (GRC) and sacrifice all actual measures of business success to get there.

Webcast of Thursday: Web Application Vulnerabilities

This Thursday I’ll be giving a webcast for Core Security on Integrating Web Applications into Your Vulnerability Management Program.

You can register for it over here at WhiteHatWorld.com, and here’s the description:

Along with end-user systems, web applications often present the “weakest link” to attackers targeting sensitive data. However, while many security professionals conduct endpoint vulnerability assessments, fewer adequately manage their web application vulnerabilities.

Please join Core Security and Rich Mogull, founder of Securosis and former Gartner analyst, for a discussion of how to proactively assess your web applications against data breach threats.

You’ll learn:

  • Which web-based attacks are posing the greatest risks to organizations today.
  • When and where to integrate web apps into your broader vulnerability assessments.
  • Why static analysis can miss critical exposures – and how you can fill the gaps.

Train Like You Fight

Ah, Monday. And not just the usual Monday, but a Monday after a perfect 5 day trip with my wife to Sonoma. A Monday where, right after we get back, the hot water heater in our old house (that we now rent) dies. Sigh. I really don’t like this whole “real world” thing.

On the plus side we set two records on our wine tour: fewest wineries visited, and most time spent at a single winery. On our second stop at a small, 300 case a year winery we ended up polishing off a few bottles with the owner (and sole operator) over nearly 5 hours, making our guide late for his dinner. It was a total blast, not pretentious at all (I’m still pretty blue collar), and the wine was excellent. It did blow our stomachs for the entire next day, but that was a cost worth paying.

One of the lasts posts before I left was about the philosophy of REACT FASTER and BETTER I partially stole from Mike Rothman. In a response, Cutaway brought up a second, no less important issue, as almost a side note. He refers back to his Marine days and the importance of keeping your head up, even when you’re down in the trenches responding to something else or stuck in the routine daily grind. When teaching martial arts I refer to this as situational awareness, which is what I think the military and law enforcement also call it. Know what’s going on around you, even if you’re bored off your rocker with tedium.

But that’s not what I want to talk about today. Early in the post, Cutaway says,

All of this got me thinking about how we react to situations as a whole. I started thinking about how through training and effort we can begin to overcome hardships. I started thinking about how diligent practice can instill good habits and create muscle memory in any individual.

“Yes, yes,” you are thinking to yourself right now. We have heard this all before. Practice makes perfect. Practice your incident response. Practice your backup procedures. Practice your disaster recovery. Practice makes perfect. Practice, Practice, Practice. Blah, blah, blah. Yes, I am tell you that. But what I want to emphasize is that you can train yourselves all day long and still make mistakes.

Yep, we’re absolutely going to make mistakes, and how we respond to those mistakes is just as important, maybe more important, than minimizing them. The only way we can do this is if you “train like you fight”. In training, you need to run practical scenarios that emulate, as closely as possible, the chaos of the real world.

How many of you can honestly say your incident response, disaster recovery, or business continuity tests come close to emulating the real world? It’s why I despise over-reliance on tabletop tests that prove nothing. It’s why I really like programs like the DefCon Capture the Flag that test real attack and defense response skills.

If you are in incident response or disaster recovery/BCP, make sure you make heavy use of scenarios and practical tests as part of your training. Make them as real as possible, and throw in the unexpected to train people on how to respond to the chaotic. Tedious, rote training builds the “muscle memory” for tasks, while scenarios build the “muscle memory” for the unknown.

Information-Centric Security Tip: Know Your Users and Infrastructure

I was on a client reference today learning about someone’s DLP deployment, and it highlighted one of the biggest issues we often face when moving to an information-centric model. No, it’s not a failure of content analysis techniques, data classification, or over-hyped tools, it’s that we often don’t even know who owns what, who’s supposed to have access to what, or our own infrastructure.

I often start my data security/information-centric rants by mentioning you need to have good identity management in place, but I don’t normally spend a whole lot of time talking about the details.

The truth is, this comes up all the time when I’m talking with end users who are implementing this stuff. Oftenthey don’t have a good directory infrastructure, or one that reflects the org chart, and thus they can’t do everything they want with their DLP, DAM, or other tools. Sometimes they don’t even know where all their assets/servers are, or how to access them for scanning.

Thus the tip- if you have a good directory infrastructure that accurately reflects your organizational structure, you’ll be in much better shape for any of these projects. Many of these tools can directly integrate with AD/LDAP, allowing you to build role-based policies.

You can’t inform someone’s manager they’re sending customer lists home or running weird DB queries if you don’t know who they work for.

Back from Washington D.C. (No thanks to SuperShuttle)

This past Monday, I had the privilege of speaking (along with several peers) to the Commission on Cyber Security for the 44th Presidency about issues on identity theft, breach disclosure and personal privacy in general. It was an honor to present with such a great group of folks. There were some great discussions/debates and I look forward to the opportunity to present again as the Commission works to streamline its recommendations. My written testimony is below. A special thanks to the folks at Emergent Chaos and to Rich for their comments, which made this a much better piece. Any errors or logical fallacies are, of course, my own.


Thank you for the opportunity to present to you today on the issue of identity theft. Since the advent of CA1386, we have seen 41 other states pass similar legislation mandating to some degree or another that companies must notify customers or the government when they believe they have suffered a loss of personal data. Unfortunately, each and every state has created slightly different criteria for what constitutes personal information, what a loss is, when notification needs be sent and, to whom it must be sent. As a result there are huge disparities among companies on what they do when they discover they’ve suffered a breach.

As much as I prefer to not have even more legislation, I believe that the only solution to this dilemma is to have a uniform federal law that covers the loss of personal information. Rather than preempt state laws, this law should set baseline requirements of:

a) Notification to all customers in a timely fashion.
b) Notification to a central organization.
c) The gathered data about companies suffering breaches must be a matter of public record and un-anonymized.
d) Include notification of any personal information that is not a matter of public record.
e) Not have a “get out of jail free” card.

This last point is key. One of the great weaknesses of CA1386 (and several other states’ legislation as well) is that companies don’t have to notify in case the information was encrypted. Unfortunately, the mere use of encryption does not mean the data was actually obfuscated at the time it was stolen, for instance in cases where a laptop is stolen while the user is logged in. Don’t get me wrong- encryption is important. A well-written law will provide a safe harbor for a company that has lost data. If they can establish that it was encrypted following best practices and that key material was not also lost, the company should be protected from litigation as a result of the breach disclosure.

Similarly, many state laws allow companies to choose to not disclose if they believe the data has not been misused. Given that the companies lost the data to begin with, should we really trust their assessment of the risk of misuse, especially when many executives believe it is not in their best interest to not disclose? It is worth noting that following a breach, stock prices do not suffer in the long run and customer loss is approximately 2%.

On the other side of the coin from breach disclosure, we have the problem that people don’t know what personal information companies have about them. Part of the outrage behind the ChoicePoint debacle of several years ago was that people didn’t know that this data was even being collected about them to begin with, and had no real way to find out what ChoicePoint might or might not have collected. In Europe as well as in Australia and parts of Asia such as Japan, companies have to both tell customers what data they have and allow them the opportunity to correct any errors. Additionally, there are strict restrictions on what collected personal information may be used for. I believe that it is time that similar protections be available to Americans as well.

Best Practices For DLP Content Discovery: Use Cases

In our last post we finished our review of DLP content discovery best practices by discussion rolling out and maintaining your deployment. Today we’re going to focus on a couple of use cases that illustrate how it all works together. I’m writing these as fake case studies, which is probably really obvious considering my lack of creativity in the names.

DLP Content Discovery for Risk Reduction and to Support PCI Audit

RetailSportsCo is a mid-sized online and brick-and-mortar sporting goods retailer, with about 4,000 headquarters employees and another 2,000 retail employees, across 50 locations. They classify as a Level 2 merchant due to their credit card transaction volume and are currently PCI complaint, but struggled through the process and ended up getting a series of compensating controls approved by their auditor, but only for their first year.

During the audit it was discovered that credit card information had proliferated uncontrolled throughout the organization. It was scattered through hundreds of files on dozens of servers; mostly Excel spreadsheets and Access databases used, and later ignored, by different business units. Since storage of unencrypted credit card numbers is prohibited by PCI, their auditor required them to remove or secure these files. Audit costs for the first year increased significantly due to the time spent by the auditor validating that the information was destroyed or secured.

RetailSportsCo purchased a DLP solution and created a discovery policy to locate credit card information across all storage repositories and employee systems. The policy was initially deployed against the customer relations business unit servers, where over 75 files containing credit card numbers were discovered. After consultation with the manager of the department and employee notification, the tool was switched into enforcement mode and all these files were quarantined back into an encrypted repository.

In phase 2 of the project, DLP endpoint agents were installed on the laptops of sales and customer relations employees (about 100 employees). Users and managers were educated, and the tool discovered and removed approximately 150 additional files. Phase 3 added coverage of all known storage repositories at corporate headquarters. Phase 4 expanded scanning to storage at retail locations, over a period of 5 months. The final phase will add coverage of all employee systems in the first few months of the coming year, leveraging their workstation configuration management system for a scaled deployment.

Audit reports were generated showing exactly which systems were scanned, what was found, and how it was removed or protected. Their auditor accepted the report, which reduced audit time and costs materially (more than the total cost of the DLP solution). One goal of the project is to scan the entire enterprise at least once a quarter, with critical systems scanned on either a daily or weekly basis. RetailSportsCo has improved security and reduced risk by reducing the potential number of targets, and reduced compliance costs by being able to provide auditors with acceptable reports demonstrating compliance.

DLP Content Discovery to Reduce Competitive Risk (Industrial Espionage)

EngineeringCo is a large high-technology manufacturer of consumer goods with 51,000 employees. In the past they’ve suffered from industrial espionage, when the engineering plans for new and existing products were stolen. They also suffered a rash of unintentional exposures and product plans were accidentally placed in public locations, including the corporate website.

EngineeringCo acquired a DLP content discovery solution to reduce these exposure risks and protect their intellectual property. Their initial goal was to reduce the risk of exposure of engineering and product plans. Unlike RetailSportsCo, they decided to start with endpoints, then move into scanning enterprise storage repositories. Since copies of all engineering and product plans reside in the enterprise content management system, they chose a DLP solution that could integrate and continuously monitor selected locations and automatically build partial-document matching policies for all documents. The policy was tested and refined to ignore common language in the files, such as corporate headers and footers, which initially caused every document using the corporate template to register in the DLP tool.

EngineeringCo started with a phased deployment to install the DLP endpoint discovery agent on all corporate systems. In phase 1, the tool was rolled out to 100 systems per week, starting with product development teams. The initial policy allowed those teams access to the sensitive information, but documented what was on their systems. Those reports were later mated to their encryption tool to ensure that no unencrypted laptops hold the sensitive data. Phase 2 expanded deployment to the broader enterprise, initially in alerting mode. After 90 days the product was switched into enforcement mode and any identified content outside of the product development teams was quarantined with an alert sent to the user, who could request an exemption. Initial alert rates were high, but user education reduced levels to only a dozen or so “violations” a week during the 90-day grace period.

In the coming year EngineeringCo plans to refine their policy to restrict product development employees from placing registered documents onto portable storage. The network component of their DLP tool already restricts emailing and other file transfers outside of the enterprise. They also plan on adding policies to protect employee healthcare information and customer account information.

These are, of course, fictional best practices examples, but they’re drawn from discussions with dozens of DLP clients. The key takeaways are:

  1. Start small, with a few simple policies and a limited scanning footprint.
  2. Grow deployments as you reduce incidents/violations to keep your incident queue under control and educate employees.
  3. Start with monitoring/alerting and employee educations, then move on to enforcement.
  4. This is risk reduction, not risk elimination. Use the tool to identify and reduce exposures but don’t expect it to magically solve all your data security problems.
  5. When you add new policies, test first with a limited audience before rolling them out to the entire scope, even if you are already covering the entire enterprise with other policies.

Update To The iPhone Security Tip

Chris Pepper, Master Editor, pointed out something I missed. If you memorize an encrypted network, your iPhone won’t connect to an unencrypted one with the same name, or one with a different password. Thus unless the bad guy knows your WPA passphrase (you’re not dumb enough to use WEP, are you?), you can memorize your home network and not worry about accidently connecting while wandering around, even if it’s still called “tsunami”.

Best Practices for DLP Content Discovery: Part 5

In our last post we talked about the preparatory work before you begin your DLP content discovery deployment, including expectation setting, prioritizing information for protection, and defining your workflow. By this point you should know what policies you’d like to deploy, where you want to start protecting the content, how you’d like to grow that protection after initial deployment, and the workflow for when you detect policy violations.

Today we’re going to move beyond planning into deployment.

  1. Integrate with your infrastructure: DLP content discovery tools require either a local agent or file share access to scan content in a repository. In this stage you define the initial repositories to scan and either install agents or load credentials into the DLP system. For endpoints, you should scan C$ or D$ (administrative access to all files) remotely, but a local agent is your best option for managed systems. If you haven’t already, you also need to integrate with your enterprise directory servers so the DLP tool understands users and roles/groups.
  2. Build initial policies: For your first deployment, you should start with a small subset of policies, or even a single policy, in alert or content classification mode (where the tool reports on sensitive data, but doesn’t generate policy violations).
  3. Baseline, then expand deployment: Deploy your initial policies on a limited number of storage repositories (or endpoints). Once you have a good feel for the effectiveness of the policies, performance, and enterprise integration you can expand into a wider deployment, covering more of the enterprise. After the first few times you’ll have a good understanding of how quickly, and how widely, you can roll out new policies.
  4. Tune policies: Even stable policies may require tuning over time. In some cases it’s to improve effectiveness, in others to reduce false positives, and in still other cases to adapt to evolving business needs. You’ll want to initially tune policies during baselining, but you’ll continue to tune them as the deployment expands. Most DLP clients report that they don’t spend much time tuning policies after baselining, but it’s always a good idea to keep your policies current with enterprise needs.
  5. Add enforcement/protection: By this point you should understand the effectiveness of your policies, and have educated users where you’ve found policy violations. You can now start switching to enforcement or protective actions, such as moving, encrypting, or changing access controls on files. Any time you make a file inaccessible, you should leave a plain-text contact note (or send the user an email) so they know why the file is missing and how to ask for an exception. If you’re making a major change to established business process, consider scaling out enforcement options on a business unit by business unit basis (e.g., restricting access to a common content type to meet a new compliance need).

Deploying DLP content discovery isn’t really very difficult; the most common mistake enterprises make is applying policies too widely, too quickly.

It’s also important to keep in mind that there are four general types of discovery deployments. With a monitoring/alerting deployment you roll out and generate alerts in the DLP system which are then followed up on by incident handlers. These deployments are often for sensitive data types where you don’t want immediate protection, but do want to prompt corrective actions or user education. The second type of deployment is where you add content protection actions, like encryption. It’s typically for very sensitive data types, and as we’ve outlined above often follows an alerting-only deployment. In a compliance deployment we scan for selective data related to regulatory compliance, like credit card numbers- both to ensure sensitive data is remaining within allowed containers, and also to generate compliance reports to show auditors that content is being handled appropriately.

The last deployment is a completely different model- content classification. In this case you are scanning with a very wide scope, often using general policies, to identify and classify systems based on the content they hold. Or, in some cases, you may tag the content as part of a broader classification initiative. Content classification deployments aren’t concerned with alerts or enforcement actions, but rather use these tools to help classify systems and content.