Research

The entire credit card table was encrypted and we have no evidence that credit card data was taken. The personal data table, which is a separate data set, was not encrypted, but was, of course, behind a very sophisticated security system that was breached in a malicious attack. That’s from the news, analyst and Sony PR reports that are coming out about the PlayStation Network/Qriocity breach. Does anyone trust Sony’s statement that the credit card data was not ‘taken’? If attackers got the entire customer database, wouldn’t you think they grabbed the encrypted card numbers and will attempt to crack them later? Is the comment about “a very sophisticated security system” supposed to make customers feel better, or to generate sympathy for Sony? Does labeling their breached security system “very sophisticated” reduce your faith in the likelihood their crypto and key management systems will withstand scrutiny? How many of you thought the name “Qriocity” was defacement the first time you read the story? My general rule over the last three years is to not write about breaches unless there is something unusual. There are just too many of them, and the questions I asked above could apply to any of the lame-assed breach responses we have been hearing for the last decade. But this one has plenty of angles that make it good spectator sport: It’s new: It’s the first time I have seen someone’s network hacked through a piece of dedicated hardware – of their own design. It’s old: It’s the classic (developer) test environment that was the initial point of entry and, just like so many breaches before it, for some mysterious reason the test environment could access the entire freakin’ customer database. It’s new: I can’t think of another major data breach that will have this degree of international impact. I’m not talking about the fraud angle, but rather how governments and customers are reacting. It’s old: Very little information dribbling out, with half-baked PR “trust us” catchphrases like “All of the data was protected …” It’s new: Japanese culture values privacy more than any other country I am familiar with. Does that mean they’ll develop the same dedication to security as they do quality and attention to detail? It’s old: It’s interesting to me that a culture intensely driven to continuous improvement has an oh-so-common allergic reaction to admitting fault. Sure, I get the ‘blameless’ angle written about in management texts throughout the 80s, but the lack of ownership here has a familiar ring. Obviously I was not the only one thinking this way. It’s new: We don’t, as a rule, see companies basically shut down their divisions in response to breaches, and the rumored rebuild of every compromised system is refreshing. It’s old: Their consumer advice is to change your password and watch your credit card statements. Ultimately I am fascinated to see how this plays internationally and if this breach has meaningful long-term impact on IT security processes. Yeah, not holding my breath either. Share:

“It’s anything you want it to be – it’s software!” – Adrian. Database Activity Monitoring software is deployed differently than DAM appliances. Whereas appliances are usually two-tier event collector / manager combinations which divide responsibilities, software deployments are as diverse as customer environments. It might be stand-alone servers installed in multiple geographic locations, loosely coupled confederations each performing different types of monitoring, hub & spoke systems, everything on a single database server, all the way up to N-tier enterprise deployments. It’s more about how the software is configured and how resources are allocated by the customer to address their specific requirements. Most customers use a central management server communicating directly with software agents with collect events. That said, the management server configuration varies from customer to customer, and evolves over time. Most customers divide the management server functions across multiple machines when they need to increase capacity, as requirements grow. Distributing event analysis, storage, management, and reporting across multiple machines enables tuning each machine to its particular task; and provides additional failover capabilities. Large enterprise environments dedicate several servers to analyzing events, linking those with other servers dedicated to relational database storage. This later point – use of relational database storage – is one of the few major differences between software and hardware (appliance) embodiments, and the focus of the most marketing FUD (Fear, Uncertainty, and Doubt) in this category. Some IT folks consider relational storage a benefit, others a detriment, and some a bit of both; so it’s important to understand the tradeoffs. In a nutshell relational storage requires more resources to house and manage data; but in exchange provides much better analysis, integration, deployment, and management capabilities. Understanding the differences in deployment architecture and use of relational storage are key to appreciating software’s advantages. Advantages of software over appliances include: Flexible Deployment: Add resources and tune your platforms specifically to your database environment, taking into account the geographic and logical layout of your network. Whether it’s thousands of small databases or one very large database – one location or thousands – it’s simply a matter of configuration. Software-based DAM offers a half-dozen different deployment architectures, with variations on each to support different environments. If you choose wrong simply reconfigure or add additional resources, rather than needing to buy new appliances. Scalability & Modular Architecture: Software DAM scales in two ways: additional hardware resources and “divide & conquer”. DAM installations scale with processor and memory upgrades, or you can move the installation to a larger new machine to support processing more events. But customers more often choose to scale by partitioning the DAM software deployment across multiple servers – generally placing the DAM engine on one machine, and the relational database on another. This effectively doubles capacity, and each platform can be tuned for its function. This model scales further with multiple event processing engines on the front end, letting the database handle concurrent insertions, or by linking multiple DAM installations via back end database. Each software vendor offers a modular architecture, enabling you to address resource constraints with very good granularity. Relational Storage: Most appliances use flat files to store event data, while software DAM uses relational storage. Flat files are extraordinarily fast at writing new events to disk, supporting higher data capture rates than equivalent software installations. But the additional overhead of the relational platform is not wasted – it provides concurrency, normalization, indexing, backup, partitioning, data encryption, and other services. Insertion rates are lower, while complex reports and forensic analyses are faster. In practice, software installations can directly handle more data than DAM appliances without resorting to third-party tools. Operations: As Securosis just went through a deployment analysis exercise, we found that operations played a surprisingly large part in our decision-making process. Software-based DAM looks and behaves like the applications your operations staff already manages. It also enables you to choose which relational platform to store events on – whether IBM, Oracle, MS SQL Server, MySQL, Derby, or whatever you have. You can deploy on the OS (Linux, HP/UX, Solaris, Windows) and hardware (HP, IBM, Oracle, Dell, etc.) you prefer and already own. There is no need to re-train IT operations staff because management fits within existing processes and systems. You can deploy, tune, and refine the DAM installation as needed, with much greater flexibility to fit your model. Obviously customers who don’t want to manage extra software prefer appliances, but they are dependent on vendors or third party providers for support and tuning, and need to provide VPN access to production networks to enable regular maintenance. Cost: In practice, enterprise customers realize lower costs with software. Companies that have the leverage to buy hardware at discounts and/or own software site licenses can scale DAM across the organization at much lower total cost. Software vendors offer tiered pricing and site licenses once customers reach a certain database threshold. Cost per DAM installation goes down, unlike appliance pricing which is always basically linear. And the flexibility of software allows more efficient deployment of resources. Site licenses provide cost containment for large enterprises that roll out DAM across the entire organization. Midmarket customers typically don’s realize this advantage – at least not to the same extent – but ultimately software costs less than appliances for enterprises. Integration: Theoretically, appliances and software vendors all offer integration with third party services and tools. All the Database Activity Monitoring deployment choices – software, hardware, and virtual appliances – offer integration with workflow, trouble-ticket, log management, and access control systems. Some also provide integration with third-party policy management and reporting services. In practice the software model offers additional integration points that provide more customer options. Most of these additional capabilities are thanks to the underlying relational databases – leveraging additional tools and procedural interfaces. As a result, software DAM deployments provide more options for supporting business analytics, SIEM, storage, load balancing, and redundancy. As I mentioned in the previous post, most of these advantages are not visible during the initial deployment phases

I’ve taught a lot of different classes over the years, and always found the different structures to be pretty interesting. On one end were highly scripted first aid classes that forced us to show crappy “Help! I’ve fallen!” videos produced in 1878 accompanied by a mandatory script. The name of the game was baseline consistency. Lock everything down as tight as possible because you can’t predict the quality of the instructor. Heck, few CPR instructors have ever actually done CPR. I know how I taught changed after I cracked some ribs on mostly-dead people. (No, they don’t wake up and thank you like on Baywatch. And they are never that hot or in bikinis. Well sometimes bikinis, but trust me, you really should dress more appropriately before letting your heart stop.) In a completely different direction is martial arts – which is all about tailoring the experience to best connect with the student over many years. I only ran a solo class for about 6 months while my instructor ran off to start his family, and learned a hell of a lot in the process. Then my IT career hit and that was the end of that. Why bring this up now? I’ve been hip-deep in pulling together all the final materials for the first fully packaged CCSK class we will be teaching June 8-10. For the first time I’m in the position of developing courseware for a structured class, with hands-on, which others will have to teach. The lecture slides are pretty straightforward, although we have to be careful to include plenty of instructor notes and not assume any experience level. The hands-on exercises? Those are a challenge. Building the scenarios wasn’t too tough. But it takes me 5 times longer to convert one into a package someone else can teach from. Everything has to be scripted, packaged, and able to run on everything from a high-end Mac Pro to a freaking Speak-n-Spell. And run a private cloud for 40 students on a Windows ME netbook. A lot more people have performed CPR than have built private clouds. I’m not complaining – it’s a blast to work with my hands again. Although I have always sucked at debugging, and my wife is pissed I keep bleeding on the floor from banging my head against all our walls. But it’s very cool to put everything together like a puzzle. Pre-script pieces in module 1 we won’t need until module 8, just so students can focus on the concepts rather than the command lines, while still giving advanced folks freedom to explore and play so they don’t get bored. I just hope it all works. On to the Summary: Webcasts, Podcasts, Outside Writing, and Conferences Adrian quoted in CSO Magazine. Rich on security and the AWS outage. The Network Security Podcast, Episode 239. With special guest Josh Corman. Favorite Securosis Posts Mike Rothman: Why We Didn’t Pick the Cloud (Mostly) and That’s OK. Who else gives you such a look into the thought processes behind major decisions? Right, no one. You’re welcome. David Mortman: Why We Didn’t Pick the Cloud (Mostly), and That’s Okay. Adrian Lane: Why We Didn’t Pick the Cloud. Operations played a bigger part in the decision process than we expected. Rich: Software vs. Appliance: Software. Other Securosis Posts Incite 4/27/2011: Just Write. Security Benchmarking, Beyond Metrics: Benchmarking in Action. Security Benchmarking, Beyond Metrics: Index. Favorite Outside Posts Mike Rothman: DHS chief: What we learned from Stuxnet. How cool would it have been if Secretary Napolitano had just said “We’re screwed.”? We are, but this article hits on responding faster and more effectively. David Mortman: TCP-clouds, UDP-clouds, “design for fail” and AWS. Because DR is a security issue Adrian Lane: Anatomy of a SQL Injection Attack. Dave Lewis: DHS needs to point finger at self, not private industry. Rich: Richard Bejtlich’s Cooking the Cucko’s Egg. Research Reports and Presentations React Faster and Better: New Approaches for Advanced Incident Response. Measuring and Optimizing Database Security Operations (DBQuant). Network Security in the Age of Any Computing. The Securosis 2010 Data Security Survey. Monitoring up the Stack: Adding Value to SIEM. Network Security Operations Quant Metrics Model. Network Security Operations Quant Report. Understanding and Selecting a DLP Solution. Top News and Posts Sony’s PlayStation Network and Qriocity hacked. How SmugMug survived the Amazonpocalypse. Flash + 307 Redirect = Game Over. Amazon Is Amazing! Smells of back-handed compliments, but much of the content is accurate. Share:

All I wanted to do on Monday night was go to sleep. I had a flight in the morning and thought it would be a good idea to get some rest. So I sit down with the Boss and we catch up on the day, discuss some tactics to deal with issues the kids face, and I’m ready to hit the rack. Then I notice she’s watching a movie called One Week (Netflix streaming FTW) where basically a guy is given a week to live and sets off on a cross-Canada jaunt on a motorcycle to discover himself, meet some interesting people, and do stuff that happens in movies. Movie trap, awesome. 90 minutes later, we start discussing the movie and she asks me point blank, “what would you do?” Crap, hate that question. I start thinking about the right answer, and then unconsciously I blurt out, “I’d write. A lot.” Wow. I’m given a week (or whatever) to live and my first thought is to write. Not travel. Not do exciting things. But write. Huh. She then asks why. I respond that I’ve been to lots of places. I’ve done some fairly interesting things. So, I don’t have a great desire to see places or check items off some bucket list. Of course that doesn’t mean I don’t want to see more places and do more things. But if I had to really prioritize given a very limited amount of time, I’d focus on teaching – which for me means writing. I like to think I’ve learned a bunch of stuff (mostly by screwing it up), and I’d want to document that. I figure my road rash and stories would be useful to my kids. But maybe other folks too. Or maybe that’s just my arrogance talking. I’d write about the stuff I’ve screwed up. I’d write about the stuff I’ve done right. I’d write about the stuff I’d do differently (which wouldn’t be much, by the way). I’d focus on the importance of relationships, and the unimportance of collecting things. And I’d make sure the people I care about had something to remember me. I’ve got a face made for radio, so I’d write. Then I thought about how lucky I was. If I had a week to live, I’d write. Which, by the way, is pretty much what I do every day. I try to impart some wisdom through each week’s Incite. And our other research all strives to relay the perspectives we build through our travels, hoping it’s all useful to someone. The topics would be a bit different with a different sort of deadline – pun intended. But the tactics wouldn’t. Very interesting. Then something utterly profound happened. My wife said, “You know, you don’t have to wait to get a death sentence to write that kind of stuff.” Holy crap. She’s right. Sure, life gets in the way, but those are just excuses. Obviously I’ll need to work around my day job a bit, but what the hell am I waiting for? Just write. I think I will. It’s going to be an interesting summer. -Mike Photo credits: “Seven Days” originally uploaded by Laurie Pink Incite 4 U Security = Money (again): It appears that investor interest is swinging back to security. I guess that’s inevitable, if you wait long enough. A couple weeks ago Bit9 raised $12.5MM and Verdasys raised another $15MM. That kind of money, primarily from existing investors, typically means they think they’ll get good multiples on the investment. They invest much less when they are just trying to keep a company on life support. And it even seems the IPO market could be receptive to security deals. TrustWave filed an S1 with the SEC this week and you’d expect a couple of the other high profile start-ups or private equity buyouts from the last few years to test the waters at some point. The fundamentals for continued growth in security remain good. The question is whether smaller companies can sustain growth long enough to find an upstream partner, since that’s how this movie ends regardless of whether there is an IPO somewhere in there. Some will, most won’t, and the pendulum will swing back and forth. It always does. – MR Someone needs a carder’s union: I’m going to be pretty annoyed if there isn’t any NFL this fall, and I know Mike will too. I couldn’t give a crud about baseball, but I do enjoy my Sunday football. But I have to respect the rights of the player’s union, even if I don’t like their (or the owners’) tactics. I’m not the biggest fan of unions, but can admit that in certain industries we still need them. Take carders (the credit card fraudsters). One poor bloke plead guilty to fraud involving $36M in transactions. That’s a pretty good take, right? Well, he only earned somewhere around $150K, which is only .4%. That’s a downright crappy margin. He’d be better off opening a convenience store, where at least the margins are 2-5%. Plus he has to make reparations for at least some of the $36M lost to fraud. Seriously, dude – call the Teamsters. No way would they put up with .4%, and maybe they could get you some health and retirement benefits. – RM I see you: I was not surprised that MLB programming on Apple TV would not allow me to view certain games, but I was surprised that my location was not based on my registered address – instead it’s based upon the Apple TV’s uplink IP address (assigned by the ISP). Geolocation from IP and gateway has certainly been a hot feature for service providers over the last couple years, with vendors such as PayPal factoring location into fraud detection. This capability continues to evolve, with Northwestern University recently claiming they can pinpoint user locations within half a mile. Their methodology uses a combination of known locations/IP addresses of major landmarks and government buildings, then compares

It’s no secret that we are currently working on a new software platform to deliver actionable security research to a broader market, engage folks, and… umm… feed our families. As you might expect, like any software project, it’s running about 30% late and 70% over budget. I just can’t seem to stop making our developers find exactly the right imagery and user experience to best represent the Securosis brand. Mike has coined a new term, ‘analness’, to describe the gyrations we’ve gone through, but I’m okay with that because we have spent years building our reputation and aren’t about to roll out a huge steaming pile of crap just to hit a delivery date. As we close in on the finish line, we faced a huge decision on how to host this. Our current provider is pretty good, but we ran into some issues earlier this year that prompted us to look at alternatives. And we are co-hosted, which won’t work once we start loading sensitive content into a paid service. So we began the long evaluation process of picking the right architecture and host. Well, that and satisfying our paranoia regarding site security. Despite being heavy cloud folks, we eventually decided on a dedicated server model offered by a specialized hosting company. Yes, we understand that’s probably counterintuitive, so here’s why we didn’t go that way. Co-hosting and VPS For the most part our current site is totally fine with our current load, and our hosting provider is a lot more security-conscious than most. I launched securosis.com as a blog over at Bluehost, on a WordPress co-host. It worked totally fine, but as we started expanding it was clear that platform couldn’t meet our growing needs. We decided to switch to a better content management system (ExpressionEngine), and while we could technically run it there, we decided to go with a more specialized provider (enginehosting.com). We have been mostly happy with the change, even though EH is considerably more expensive, because we get a lot more for what we pay. They also have excellent growth options to expand to a Virtual Private Server or even dedicated boxes if needed. But it’s still a co-host model. The one problem we hit earlier this year appeared after a major platform upgrade. Our back end became nearly unusable due to performance problems, and when I submitted a support request they kept blaming our configuration or plugins. We are big boys, and willing to accept when we screw up. We turned our system upside down and couldn’t find anything that would kill the performance of the admin console. As it turned out we were right. Another client in our cluster over-used resources – as I had initially suggested. We were bothered by their lack of investigation, and by the (realized) potential for another customer to impact us. That convinced us we need to get off co-hosting, and into VPS or cloud. We also had to factor in all the security reasons to drop a co-hosted model once we have content we want to protect. VPS vs. Cloud We quickly ruled out VPS. As our knowledge and experience working with various cloud services grew, we saw no reason to pick VPS over a pure cloud model. To be honest, while I see co-hosting surviving for a while, I definitely see the allure of VPS cratering in the next few years, as customers keep comparing VPS offerings against the rapidly evolving public cloud offerings. I decided we would go completely cloud. Aside from the lack of advantages to VPS, we were conscious of the importance of eating our own dogfood, now that we are working so deeply with the Cloud Security Alliance and advising people on cloud projects. Our criteria for a cloud provider including a security conscious shop, judged on both what they publish and checks with various industry connections. We wanted some IPS/firewall and patch management support options to improve our baseline security and reduce our management overhead. As our IT guy, I simply don’t have the time to manage all our patches/fixes myself. If I were caught on an international flight when we needed to block and fix a critical 0day, we could be screwed. That was unacceptable. Other factors included our plan to use a cloud-based WAF. Not that it could block everything, but the combination of blocking basic scans and providing better analytics was attractive. We also factored in performance, as we know our potential audience is self-limiting, and what we are delivering isn’t very CPU intensive. We need a little beef, and more importantly the capability to grow, but we couldn’t forsee a need for anything too crazy. It’s not like we are Netflix or anything (yet). So there we were – I thought we were all set, until… From Cloud to Dedicated I wasn’t fully satisfied with the options I found (all of which cost a heck of a lot more than a basic AWS deployment), but I felt confident that we could get what we need at a reasonable price. Then we mentioned what we were doing to a trusted friends in the industry. For now I won’t mention who we are working with, but someone we highly respect offers dedicated hosting in a special section of a major data center they lease (their own cage). I am not sure they expected us to take them up on the offer. It’s not like they were soliciting our business – this came up over beer. These folks are as paranoid as we are (maybe more), and aside from hosting the site they will implement some stringent and unusual security controls we couldn’t possibly get anywhere else for any reasonable price. Normally they don’t use this model even with their existing clients, and we are going to be their first test case beyond internal infrastructure. As a bonus, their data center guarantees 100% infrastructure uptime. In writing. (Note: this doesn’t mean our boxes, just their network and power). Trusted

As we wrap up our series on Security Benchmarking, we find it instructive to actually walk through a scenario and apply the process. Yes, the scenario is a bit contrived, but we’ll use it to hit the high points of the process, deciding where to start, collecting the data, establishing the peer group and communicate the findings. Keep in mind that we focus on getting quick wins, showing immediate value, building momentum and leveraging that momentum for programatic success. Scenario For our case study, let’s use a mid-tier financial company as our example. I’d say large enterprise, but in reality there are a lot of nuances and moving pieces within a large enterprise that need more detailed discussion. So let’s keep it relatively simple. Likewise, we picked the financial vertical because of 1) need and 2) availability of data. The reality of the financial industries regulatory oversight has created a general perspective of security first and data-centricity (yes, these are the folks that try to do risk management for a living) means these businesses are move likely to embrace a benchmarking mentality. In our (contrived) scenario, the Board drove the hiring a new CISO to “fix security.” As easy as it is to think this was just catering to a board directive, the senior team seems to have a commitment to fix things and do it the right way. So the CISO has a clear honeymoon period and some leeway in thinking somewhat unconventionally about how to build the security program. The new CISO still spends some time figuring out what’s installed and what’s not working, but he knows the organization has AV deployed, they use an external scanning service, and do a pretty good job of patching on internal systems. Yet, like many smaller financial institutions they use hosted applications for most of their business processing. So a lot of their data is not within their direct control. Over the past few years, the organization has had a handful of incidents, but none really resulted in major data loss. Thus the CISO was pleasantly surprised when he got the mandate to fix the security program, when it wasn’t outwardly broken. The senior team came to the conclusion they are living on borrowed time and want to act decisively to make sure they are ready when the brown stuff hits the fan (which it inevitably will). See? We told the you the scenario was contrived, but without a senior-level mandate to make changes in implement a security program, getting any kind of security metrics/benchmarking initiative going will be difficult. Where Do You Start? Now the CISO has to figure out where to start. He’s decided that he wants to figure out where his most apparent gaps are. You know, the ones you can drive a Mack Truck through. So he starts with a comprehensive risk assessment to build a baseline, but he also wants to compare his environment to other like-sized companies (both in and out of his industry) to figure out how he compares to those organizations. Keep in mind, boiling the ocean and trying to do everything at this point is a bad idea. He’d get buried in the nuances of the data and not get anything done, which could endanger his entire security program. So he needs to ask the following questions: What do you need to achieve? Where are the key operational problems? This is where you always have to start. In our case study, the CISO is looking to identify his most critical gaps, and given the luck they’ve had in not having a huge data loss even with a few breaches, he wants to start with incident response. What data do you have? Next you have to figure out if you have the data or can get it easily. With incident data, the reality is the findings from the forensics investigations exist, but haven’t been put in any kind of format for comparison. But the data exists, so it makes sense to keep pressing down this path. If the data doesn’t exist or can’t be gathered quickly, then it’s time to look at Plan B. You don’t want to hold up the effort because it’s all about getting the quick win. Where will be most impactful to show comparative data? Selecting to focus initially on incident response represents a pretty shrewd move for the new CISO. He knows the board and senior management is sensitive to not getting nailed, as well as having a set of reasonable consensus metrics available (from CIS), and having the data. This increases the chances of success. Peer Groups and Service Providers Next, our CISO has to define the peer group for analysis. This isn’t brain surgery. He’ll need to compare to other financials (duh!), but also companies in other regulated industries (like healthcare and utilities) of a similar size. The good news is there are a ton of mid-sized hospital groups, as well as many community utilities, with similarly sensitive data. But how do they get their hands on that kind of data for comparison purposes? Now we go back and revisit the selection criteria for any kind of provider you’d think about for benchmarking services. Remember, these folks have to 1) have access to the data you’d need and 2) be able to protect the data you share with them. To be clear, you may not be able to get everything done with just one provider. In our case study here, the CISO will actually pick two. The first is his regional bank ISAC, who has been gathering data from its members for a while. The second is a commercial benchmarking offering, since they have more data about other industries that aren’t the focus of the ISAC. In reality, the CISO would like to just have one provider, but until a critical mass of data for many verticals is captured, he’ll need to piecemeal the solution to solve the problem. Analyze Equipped with data regarding his first

As is (now) our custom, we post a set of links to each blog series as it wraps up. This both gives us an easy way to find all our posts, and acknowledges that not everyone wants our complete feed and may want to read posts once they’re all written. As always, we love feedback on our work in progress. Yes, it’s time consuming to take time out for comment on specific posts. But remember that pretty much all our research is available free of charge, so it’s not too much to ask for a little constructive criticism on our work, is it? Please take a look and sink your teeth in. Introduction Security Metrics (from 40,000 feet) Collecting Data Systematically Sharing Data Safely Defining Peer Groups and Analyzing Data Communications Strategies Continuous Improvement You Can’t Benchmark Everything Benchmarking in Action As always, we thank you for reading, commenting, and making our research better. Share:

The Apple-ification of my home continues, as I got an Apple TV as an early birthday present. Tinkerer that I am, I thought “Wouldn’t it be great to hardwire it with Cat5 cable to the Airport Extreme? Download speeds will be awesome”. So I changed the existing phone lines (I’ll never use a POTS land line again) to Ethernet. Which meant changing all the phone jacks, and then the wall plates. And rewiring the central connections. And putting a new router in the closet. And adding new power to the closet. And wiring in a small low-voltage fan. It was the snowball effect, but this was one of the first times I have not minded, because I have Giant Freakin’ Toolbox! It was not always this way. For many years I would find something broken around the house and attempt to fix it. I am a guy, and that’s what I do. You know, something simple like a door latch that’s not working. More often than not, the whole process would just piss me off because it always involved “The Search”. Searching for my tools. Where had they gone to? Where was the Torx wrench I needed? Where was the right screwdriver – the right size with the hardened steel tip? What happened to the beautiful German wood chisels I got for Christmas? When you don’t live in your own house for four years (which happened to me with my previous employer) tools disappear. When you don’t have kids, there are only a couple of options for who used them. As far as I can tell the dogs have no interest in carpentry or automotive repair. You know who to ask about tool storage in random locations, but the question “Have you seen… ” is just not worth asking. The answer, “No. I have no idea” just makes me angrier. On the opposite end of that equation, best case, your wife will only say “No.” – worst case she’ll be pissed at you for insinuating she lost your tools. Then you stumble upon a tool you weren’t looking for – during your desperate search for those tools you need – in the bathroom cupboard, on top of a picture frame, in a box in the attic, or in that decorative ceramic vase in the dining room. During The Search – which takes longer than the time to fix the busted stuff – I would find other broken things that had a higher priority than the stuff I originally set out to fix. More tool searching ensued. You make a trip to Home Depot right after you post missing tool flyers around the house with pictures of your orbital sander. You look at the clock and half the day is gone. Two birthdays ago, my wife got me two giant tool chests, one fitting right on top of the other. With their wonder-twin powers they form Giant Freakin’ Toolbox! When she bought them every guy in the neighborhood showed up – seeing the two boxes in the driveway – and ‘helped’ her assemble then. OK, maybe ‘help’ is the wrong word because they did all the work. And maybe her bikini and the free beer helped too, but she managed to get 6 guys over to the house and they set up the toolbox. She was miffed to discover toolboxes and beer were the main attraction, but she got over it. I was so happy with the present I spent two days going through every square inch of our home to gather up the tools and place them in their new home. Now every tool I own resides there. Every tool is clean. Every drawer is labelled. Almost every tool has been accounted for – except some of the fine German wood chisels that were destroyed by a friend while prying the heads from a small block Chevy. Now projects take 2-5 minutes – perhaps 7 with cleanup. I built a wall bracket and installed a central vacuum system in a couple hours. I can change a light switch or adjust faucets without thinking about it. The time savings and reduction in frustration are astounding. I even assembled a small set of basic tools right by the garage door so ahem – anyone else needing tools can find a hammer, a basic screwdriver and pliers and not go rummaging in the tool chest. Giant Freakin’ Toolbox has a lock! I love my Apple electronics, but they don’t compare to Giant Freakin’ Toolbox. Best. Gift. Ever. On to the Summary: Webcasts, Podcasts, Outside Writing, and Conferences Rich quoted in CIO.in. Rich quoted in ComputerWorld. Adrian’s Dark Reading post on Cloud DB Security. Favorite Securosis Posts Adrian Lane: Dropbox Should Mimic CrashPlan. Mike Rothman: How to Read and Act on the 2011 Verizon DBIR. This is a gold mine, and you could get buried alive. Rich deciphers it. Rich: Categorizing FUD. My prediction: everyone else also chooses this one. Other Securosis Posts Oracle CVSS: ‘Partial+’ is ‘Useful-‘. Software vs. Appliance: Appliances. Incite 4/20/2011: Family Parties. New White Paper: React Faster and Better: New Approaches for Advanced Incident Response. Weekend Reading: Security Benchmarking Series. Security Benchmarking, Going Beyond Metrics: Continuous Improvement. Security Benchmarking, Beyond Metrics: You Can’t Benchmark Everything. Favorite Outside Posts Adrian Lane: My favorite line in the CSA Guidance. Rich: The Science of Why We Don’t Believe Science. In security we need to constantly assess our own cognitive biases. This is a good article that can help you understand risk responses, even though it isn’t about risk. Yes, politics are mentioned, but if you can’t get past that you need to re-evaluate your biases. Mike Rothman: Be wary of the well-certified IT pro. “Certification only goes so far.” What Kevin Beaver said… Project Quant Posts DB Quant: Index. NSO Quant: Index of Posts. NSO Quant: Health Metrics–Device Health. NSO Quant: Manage Metrics–Monitor Issues/Tune IDS/IPS. NSO Quant: Manage Metrics–Deploy and Audit/Validate. NSO Quant: Manage Metrics–Process Change Request and Test/Approve. Research Reports and Presentations React Faster and Better: New Approaches for Advanced Incident Response. Measuring and Optimizing

We have spent much of this series on why benchmarking is important. But we also need to point out some situations where benchmarking may not be appropriate. There are clearly situations where you can’t benchmark, particularly is on granular operational data, which I call Ninja Metrics. Dependency: Peer Group Data Most organizations have ‘nascent’ metrics programs, which may actually be too kind. But not all. Some have embraced detailed programs that gathers all sorts of data, mostly focused on operations. This represents the next step of a metrics program, and can be represented by some of the ideas put forth through our Quant research projects. We have created highly granular process maps (with associated metrics) for Patch Management, Network Security Operations, and Database Security. Each report specifies 50+ distinct metrics you can measure for that discipline. Yes, they are comprehensive. But there is a clear issue regarding benchmarking at this level. You will have a hard time finding similarly granular data from other companies for comparison. So the key dependency in implementing a benchmarking effort is the availability of peer group data for comparison. Compare to Yourself What do world class athletes do when they reach the top of the heap? You know, folks like Michael Phelps, who has basically shattered every record there is to shatter. They start comparing themselves to their past performance. Improvement is measured internally rather than externally. Even if no one else has ever done better, you know you can. And this is what you will likely need to do the most granular operational functions. When you take a step back this makes a lot of sense. The reality is that you aren’t necessarily trying to ‘win’ relative to operational excellence. You want to improve. That said, it is important to have an idea of where you stand in comparison to everybody else, at least on the high-level operational metrics. But for the most granular metrics, not so much. We hope that over time enough companies will start tracking granular operational metrics, and become comfortable enough with benchmarking, to share their data. But that’s not going to happen tomorrow or even the day after. In the meantime you can (and should) continue to push your metrics program forward – just understand your comparisons may need to be internal. As we wrap up the Benchmarking series, we’ll look at how to get some Quick Wins and see the process in action. Share:

I love it when people froth at the mouth once they finally realize the blazingly obvious! For today’s example let’s look at the big Dropbox data privacy controversy. There are a few serious problems with Dropbox, such as not requiring a password after a host is added, making it super easy for someone to pretend to be you (if they get your host ID) and access your data. That’s not great, but there are far worse things out there I worry about. But the big controversy is that… ghasp… Dropbox employees could access your data! But if you know anything about security you know that if you get a nice, pretty web interface; then somewhere, somehow, the odds are an admin at the service provider can access your data. There are techniques around this using creative programming, but one look at the Dropbox code in your browser makes it clear they aren’t using anything like that. This is because the Dropbox web servers need to see your data to show you the web interface. Ergo, the servers can decrypt your data. Ergo, someone at Dropbox can see it. Now this doesn’t need to be true – they could have restricted the web UI to metadata and still encrypted file contents, then used a browser plugin (or maybe even JavaScript) to decrypt the files. But both options entail usability and security tradeoffs. A great example of how to manage issues like these is the CrashPlan backup service. CrashPlan offers a cascade of security options, each with usability tradeoffs, and all available to users. (All these options protect your symmetric encryption key, not the data itself): Protect with your account password. CrashPlan can access and see your data if needed. Protect with a separate data password stored locally. CrashPlan admins can’t access your data (even to restore it). You need to keep and secure an extra password. Set your own encryption key. Can be on a per-machine basis. Very secure, requiring more management. There is, of course, much more to their encryption scheme – this is just the user-controllable portion. Dropbox could do something similar: Standard (perhaps the only option on their free plan): Basic account username/password as they have now. Enhanced Security: Set a personal password, with metadata in the clear. You can manipulate your files, but they can only be downloaded by the local agent (not via a browser) and you need to remember the password (no password restore capability). You can still share public files, which are stored in a separate directory using your account password as on the old system. High Security: Metadata and file data encrypted using your personal passphrase, separate from your account passphrase. Web UI can only manage public files – everything else is accessible only through their client. These would require serious development effort, and I don’t want to gloss over the complexity or importance of implementing this type of security correctly and safely. This stuff is hard. But it would be manageable if they made it a priority. But seriously, people – if you want something free/cheap with a pretty web interface to manage your data, odds are you are trading off security. I use Dropbox extensively and just encrypt the things I consider too private to expose. Share: