Research

We see a lot of FUD on a daily basis here in the security industry, and it’s rarely worth blogging about. But for whatever reason this one managed to get under my skin. Nominum is a commercial DNS vendor that normally targets large enterprises and ISPs. Their DNS server software includes more features than the usual BIND installation, and was originally designed to run in high-assurance environments. From what I know, it’s a decent product. But that doesn’t excuse the stupid statements from one of their executives in this interview that’s been all over the interwebs the past couple days: Q: In the announcement for Nominum’s new Skye cloud DNS services, you say Skye ‘closes a key weakness in the internet’. What is that weakness? A: Freeware legacy DNS is the internet’s dirty little secret – and it’s not even little, it’s probably a big secret. Because if you think of all the places outside of where Nominum is today – whether it’s the majority of enterprise accounts or some of the smaller ISPs – they all have essentially been running freeware up until now. Given all the nasty things that have happened this year, freeware is a recipe for problems, and it’s just going to get worse. … Q: Are you talking about open-source software? A: Correct. So, whether it’s Eircom in Ireland or a Brazilian ISP that was attacked earlier this year, all of them were using some variant of freeware. Freeware is not akin to malware, but is opening up those customers to problems. … By virtue of something being open source, it has to be open to everybody to look into. I can’t keep secrets in there. But if I have a commercial-grade software product, then all of that is closed off, and so things are not visible to the hacker. … Nominum software was written 100 percent from the ground up, and by having software with source code that is not open for everybody to look at, it is inherently more secure. … I would respond to them by saying, just look at the facts over the past six months, at the number of vulnerabilities announced and the number of patches that had to made to Bind and freeware products. And Nominum has not had a single known vulnerability in its software. The word “bullsh**” comes to mind. Rather than going on a rant, I’ll merely include a couple of interesting reference points: Screenshot of a cross-site scripting vulnerability on the Nominum customer portal. Link to a security advisory in 2008. Gee, I guess it’s older than 6 months, but feel free to look at the record of DJBDNS, which wasn’t vulnerable to the DNS vuln. As for closed source commercial code having fewer vulnerabilities than open source, I refer you to everything from the recent SMB2 vulnerability, to pretty much every proprietary platform vs. FOSS in history. There are no statistics to support his position. Okay, maybe if you set the scale for 2 weeks. That might work, “over the past 2 weeks we have had far fewer vulnerabilities than any open source DNS implementation”. Their product and service are probably good (once they fix that XSS, and any others that are lurking), but what a load of garbage in that interview… Share:

Everyone in the security industry seems to agree that metrics are important, but we continually spin our wheels in circular debates on how to go about them. During one such email debate I sent the following. I think it does a reasonable job of encapsulating where we’re at: Until Skynet takes over, all decisions, with metrics or without, rely on human qualitative judgement. This is often true even for automated systems, since they rely on models and decision trees programmed by humans, reflecting the biases of the designer. This doesn’t mean we shouldn’t strive for better metrics. Metrics fall into two categories – objective/measurable (e.g., number of systems, number of attacks), and subjective (risk ratings). Both have their places. Smaller “units” of measurement tend to be more precise and accurate, but more difficult to collect and compile to make decisions… and at that point we tend to introduce more bias. For example, in Project Quant we came up with over 100 potential metrics to measure the costs of patch management, but collecting every one of them might cost more than your patching program. Thus we had to identify key metrics and rollups (bias) which also reduces accuracy and precision in calculating total costs. It’s always a trade-off (we’d love to do future studies to compare the results between using all metrics vs. key metrics to seeing if the deviation is material). Security is a complex system based on a combination of biological (people) and computing elements. Thus our ability to model will always have a degree of fuzziness. Heck, even doctors struggle to understand how a drug will affect a single individual (that’s why some people need medical attention 4 hours after taking the blue pill, but most don’t). We still need to strive for better security metrics and models. My personal opinion is that we waste far too much time on the fuzziest aspects of security (ALE, anyone?), instead of focusing on more constrained areas where we might be able to answer real questions. We’re trying to measure broad risk without building the foundations to determine which security controls we should be using in the first place. Share:

Database benchmarking is hard to do. Any of you who followed the performance testing wars of the early 90’s, or the adoption and abuse of TPC-C and TPC-D, know that the accuracy of database performance testing is a long-standing sore point. With database encryption the same questions of how to measure performance rear their heads. But in this case there are no standards. That’s not to say the issue is not important to customers – it is. You tell a customer encryption will reduce throughput by 10% or more, and your meeting is over. End of discussion. Just the fear of potential performance issues has hindered the adoption of database encryption. This is why it is incredibly important for all vendors who offer encryption for databases (OS/file system encryption vendors, drive manufacturers, database vendors, etc.) to be able to say their impact is below 10%. Or as far below that number as they can. I am not sure where it came from, but it’s a mythical number in database circles. Throughout my career I have been doing database performance testing of one variety or another. Some times I have come up with a set of tests that I thought exercised the full extent of the system. Other times I created test cases to demonstrate high and low water marks of system performance. Sometimes I captured network traffic at a customer site as a guide, so I could rerun those sessions against the database to get a better understanding of real world performance. But it is never a general use case – it’s only an approximation for that customer, approximating their applications. When testing database encryption performance several variables come into play: What queries? Do users issue 50 select statements for every insert? Are the queries a balance of select, inserts, and deletes? Do I run the queries back to back as fast as I can, like a batch job, or do I introduce latency between requests to simulate end users? Do I select queries that only run against encrypted data, or all data? Generally when testing database performance, with or without encryption, we select a couple different query profiles to see what types of queries cause problems. I know from experience I can create a test case that will drop throughput by 1%, and another that will drop it by 40% (I actually had a junior programmer unknowingly design and deploy a query with a cartesian that crashed our live server instantly, but that’s another story). What type of encryption? Full disk? Tablespace? Column Level? Table level? Media? Each variant has advantages and disadvantages. Even if they are all using AES-256, there will be differences in performance. Some have hardware acceleration; some limit the total amount of encrypted data by partitioning data sets; others must contend for thread, memory, and processor resources. Column level encryption encrypts less data than full disk encryption, so this should be an apples to apples comparison. But if the encrypted column is used as an index, it can have a significant impact on query performance, mitigating the advantage. What percentage of queries are against encrypted data? Many customers have the ability to partition sensitive data to a database or tablespace that is much smaller than the overall data set. This means that many queries do not require encryption and decryption, and the percentage impact of encryption on these systems is generally quite good. It is not unreasonable to see that encryption only impacts the entire database by 1% over the entire database in a 24 hour period, while reduces throughput against the encrypted tablespace by 25%. What is the load on the system? Encryption algorithms are CPU intensive. They take some memory as well, but it’s CPU you need to pay attention to. With two identical databases, you can get different performance results if the load on one system is high. Sounds obvious, I know, but this is not quite as apparent as you might think. For example, I have seen several cases where the impact of encryption on transactional throughput was a mere 5%, but the CPU load grew by 40%. If the CPUs on the system do not have sufficient ‘headroom’, you will slow down encryption, overall read/write requests (both encrypted and unencrypted), and the system’s transactional throughput over time. Encryption’s direct slowdown of encrypted traffic is quite different than its impact on overall system load and general responsiveness. How many cryptographic operations can be accomplished during a day is irrelevant. How many cryptographic operations can be accomplished under peak usage during the day is a more important indicator. Resource consumption is not linear, and as you get closer to the limits of the platform, performance and throughput degrade and a greater than linear rate. How do you know what impact database encryption will have on your database? You don’t. Not until you test. There is simply no way for any vendor of a product to provide a universal test case. You need to test with cases that represent your environment. You will see numbers published, and they may be accurate, but they seldom reflect your environment and are so are generally useless. Share: