Research

We are proud to announce a new research paper on Pragmatic Web Application Firewall Management. This paper has been a long time coming – we have been researching this topic for three years, looking for the right time to discuss WAF’s issues. Our key finding is that Web Application Firewalls can genuinely raise the bar on application security. Properly set up they block many attacks such as SQL injection and, just as importantly, ‘virtually’ patch applications faster than code fixes can be implemented. There is ample evidence that building security into applications from the get-go is more efficient, but unfortunately it may not be practical or even realistic. Most firms already have dozens – if not thousands – of vulnerable web apps that will take years to fix. So the real answer is both: “build security in” and “bolt security on”. And that is how WAFs help protect web applications when timely code fixes are not an option. During our research we heard a lot of negative feedback from various security practitioners – specifically pen testers – about how WAFs barely slow skilled attackers down. We heard many true horror stories, but they were not due to any specific deficiency in WAF technology. The common theme among critics was that problems stemmed from customers’ ineffective management practices in WAF deployment and tuning of rules. We also heard many stories about push-back from development teams who refused to wade through the reams of vulnerability output generated by WAFs. Some of this was due to poor report quality of WAF products, and some was due to internal politics and process issues. But in both cases we concluded from hundreds of conversations that WAF provides a unique value, and its issues can be mitigated through effective management. For more detailed information on our recommendations, as well as how we reached our conclusions, we encourage you to grab a copy of the white paper. Finally, Securosis provides the vast bulk of our research for free and without user registration. Our goal, as always, is to help organizations understand security issues and products, and to help get your job done with as little headache as possible. But it’s community support that enables us to produce our research, so we want to make special mention of those firms who have sponsored this paper: Alert Logic, Barracuda Networks, and Fortinet. We want to thank our sponsors as well as those of you who took time to discuss your WAF stories and provide feedback during this project! The paper is available to download: Pragmatic WAF Management (PDF). Share:

In the previous post we went to some length to define what big data is – because the architectural model is critical to understanding how it poses different security challenges than traditional databases, data warehouses, and massively parallel processing environments. What distinguishes big data environments is their fundamentally different deployment model. Highly distributed and elastic data repositories enabled by the Hadoop File System. A distributed file system provides many of the essential characteristics (distributed redundant storage across resources) and enables massively parallel computation. But specific aspects of how each layer of the stack integrates – such as how data nodes communicate with clients and resource management facilities – raise many concerns. For those of you not familiar with the model, this is the Hadoop architecture. Architectural Issues Distributed nodes: The idea that “Moving Computation is Cheaper than Moving Data” is key to the model. Data is processed anywhere processing resources are available, enabling massively parallel computation. It also creates a complicated environment with a large attack surface, and it’s harder to verify consistency of security across all nodes in a highly distributed cluster of possibly heterogeneous platforms. ‘Sharded’ data: Data within big data clusters is fluid, with multiple copies moving to and from different nodes to ensure redundancy and resiliency. This automated movement makes it very difficult to know precisely where data is located at any moment in time, or how many copies are available. This runs counter to traditional centralized data security models, when data is wrapped in various protections until it’s used for processing. Big data is replicated in many places and moves as needed. The ‘containerized’ data security model is missing – as are many other relational database concepts. Write once, read many: Big data clusters handle data differently than other data management systems. Rather than the classical “Insert, Update, Select, and Delete” set of basic operations, they focus on write (Insert) and read (Select). Some big data environments don’t offer delete or update capabilities at all. It’s a ‘write once, read many’ model, which is excellent for performance. And it’s a great way to collect a sequence of events and track changes over time, but removing and overwriting sensitive data can be problematic. Data management is optimized for performance of insertion and query processing, at the expense of content manipulation. Inter-node communication: Hadoop and the vast majority of available add-ons that extend core functions don’t communicate securely. TLS and SSL are rarely available. When they are – as with HDFS proxies – they only cover client-to-proxy communication, not proxy-to-node sessions. Cassandra does offer well-engineered TLS, but it’s the exception. Data access/ownership: Role-based access is central to most database security schemes. Relational and quasi-relational platforms include roles, groups, schemas, label security, and various other facilities for limiting user access, based on identity, to an authorized subset of the available data set. Most big data environments offer access limitations at the schema level, but no finer granularity than that. It is possible to mimic these more advanced capabilities in big data environments, but that requires the application designer to build these functions into applications and data storage. Client interaction: Clients interact with resource managers and nodes. While gateway services for loading data can be defined, clients communicate directly with both the master/name server and individual data nodes. The tradeoff this imposes is limited ability to protect nodes from clients, clients from nodes, and even name servers from nodes. Worse, the distribution of self-organizing nodes runs counter to many security tools such as gateways/firewalls/monitoring which require a ‘chokepoint’ deployment architecture. Security gateways assume linear processing, and become clunky or or overly restrictive in peer-to-peer clusters. NoSecurity: Finally, and perhaps most importantly, big data stacks build in almost no security. As of this writing – aside from service-level authorization, access control integration, and web proxy capabilities from YARN – no facilities are available to protect data stores, applications, or core Hadoop features. All big data installations are built upon a web services model, with few or no facilities for countering common web threats, (i.e. anything on the OWASP Top Ten) so most big data installations are vulnerable to well known attacks. There are a couple other issues with securing big data on an architectural level, which are not issues specifically with big data, but with security products in general. To add security capabilities into a big data environment, they need to scale with the data. Most ‘bolt-on’ security does not scale this way, and simply cannot keep up. Because the security controls are not built into the products, there is a classic impedance mismatch between NoSQL environments and aftermarket security tools. Most security vendors have adapted their existing offerings as well as they can – usually working at data load time – but only a handful of traditional security products can dynamically scale along with a Hadoop cluster. The next post will go into day to day operational security issues. Share:

I remember sitting at lunch with a friend and well-respected member of our security community as I described the architecture we used to protect our mail server. I’m not saying it’s perfect, but this person responded with, “that’s insane – I know people selling 0-days to governments that don’t go that far”. On another occasion I was talking with someone with vastly more network security knowledge and experience than me; someone who once protected a site attacked daily by very knowledgeable predators, and he was… confused as to why I architected the systems like I did. Yesterday, it saved my ass. Now I’m not 100% happy with our current security model on mail. There are aspects that are potentially flawed, but I’ve done my best to reduce the risk while still maintaining the usability we want. I actually have plans to close the last couple holes I’m not totally comfortable with, but our risk is still relatively low even with them. Here’s what happened. Back when we first set up our mail infrastructure we hit a problem with our VPN connections. Our mail is on a fully segregated network, and we had some problems with our ISP and IPSec-based VPNs even though I tried multiple back-end options. Timing-wise we hit a point where I had to move forward, so I set up PPTP and mandated strong passwords (as in I reviewed or set all of them myself). Only a handful of people have VPN access anyway, and, at the time, a properly-constructed PPTP password was still very secure. That delusion started dying this summer, and was fully buried yesterday thanks to a new, cloud-based, MS-CHAP cracking tool released by Moxie Marlinspike and David Hulton. The second I saw that article I shut down the VPN. But here’s how my paranoia saved my ass. As a fan of hyper-segregation, early on I decided to never trust the VPN. I put additional security hardware behind the VPN, with extremely restrictive policies. Connecting via the VPN gave you very little access to anything, with the mail server still completely walled off (a UTM dedicated only to the mail server). Heck, the only two things you could try to hack behind the VPN were the VPN server itself and the UTM… nothing else is directly connected to that network, and all that traffic is monitored and filtered. When I initially set things up people questioned my choice to put one security appliance behind another like that. But I didn’t want to have to rely on host security for the key assets if someone compromised anyone connected to our VPN. In this case, it worked out for me. Now I set all this up pre-cloud, but you can set up a similar architecture in many VPC or private cloud environments (you need two virtual NICs and a virtual UTM, although even simple firewall rules can go a long way to help). This is also motivation to finish the next part of my project, which involves yet another UTM and server. Share:

Rich Mogull recently posted a great stream of consciousness piece about how we are at an inflection point in information security. He covers how cloud and mobility are having, and will continue to have, a huge impact on how we practice security. Rich mentions four main areas of impact: Hyper-segregation Incident response Active defense Closing the action loop The post is short but very very dense. Read it a couple times, even – there’s a lot there. I would add another consequence of these changes that has already begun and will continue to manifest over the next five to ten years. That is the operationalization of security. This is partially because security is increasingly becoming a feature rather than a product itself. Over time we will see more and more of today’s dedicated security jobs evolving into responsibilities of other roles. We already see this with patch and AV management, which are increasingly owned by the Operations rather than Security teams. Similarly, we will see development and QA picking up functions that previously belonged to security folks. Dedicated security folks will still exist, but they will fall into a much smaller set of roles, including: Advisors/subject matter experts – architects and CISOs Penetration testers Incident responders And in the case of the latter two, they will increasingly be experts brought in to handle issues beyond the capabilities of regular teams, or to satisfy requirements for third-party assessment. In many ways this will be a return to how security was in the mid-90s. Yet another example of Hoff’s Hamster Sine Wave of Pain…. h/t to Gene Kim for feedback as I wrote this post. Share:

Adrian here … I had a few surgical procedures over the past few weeks. They corrected some vascular defects that were causing several problems, some of which had been coming on for such a long time I was unaware that there was an issue. The whole boiling frog in a beaker concept. And with the slow progression I was ignorant of the extent of the damage it was causing. The good news is that procedures were successful and their positive benefit was far greater than I anticipated. This whole series of events hammered home a concept that I have been intellectually aware of for a long time, but not lived out to this degree. Many people have blogged about how and why people make bad security tradeoffs. Instinct, fear, lower brain functions, and other ways we are wired to make some decisions and not others. Bruce Schneier has been talking about this for 10 years or more. But I think for the first time I really understand it at a basic level. When I was a kid I had a very strong vasovagal response. I get the lightheadedness, nausea, feeling of being extremely hot, and sweating. I don’t get the fuzziness, inability to speak, or weakness. But I only ever got it when I went to the eye doctor and they administered the glaucoma test. Nothing else has ever bugged me – until this recent surgery. For the first time I saw it in slow motion, with the internal conversation going something like this: The upper, rational part of my brain says: “I’m really looking forward to getting this stuff fixed and moving on with my life.” The lower part that’s wired into all critical infrastructure say: “Something’s wrong. Something bad is happening to your leg. Fix it!” The upper brain: “It’s okay, the doctor’s just fixing some veins. Don’t …” Lower brain: “NO, it’s not! Kick that F**ker in the head! Kick him then run like hell!” Lower brain wins. And all these years I just thought I hated my eye doctor. Who knew? But getting that very strange sensation again was both odd and educational. Being aware of the condition and watching yourself react as an adult is a whole different experience; you consciously witness two parts of your brain at odds. And I know how to work through it without passing out, but it involves the same stomach compression techniques jet pilots learn to combat G-forces. A great way to creep out the hospital staff too, but it kept me alert through the physical manifestations of the conflict to ‘witness’ the internal debate. No wonder we’re so flawed when if comes to making judgements when threats or fear are involved. I can be aware of it and still do very little about it. You body would rather shut down than deal with it. On to the Summary: Webcasts, Podcasts, Outside Writing, and Conferences Adrian’s Dark Reading post on Encrypted Query Processing. Favorite Securosis Posts Mike Rothman: Inflection. Rich provides some food for thought on what the future of security looks like. Read it. Think about it. See what makes sense to you. Adrian Lane: It’s Time for Enterprises to Support a “Backup” Browser. No way to be secure with just one. And don’t get me started on browsing from mobile devices! Other Securosis Posts Incite 9/20/2012: Scabs. Securing Big Data: Security Issues with Hadoop Environments. Attend Gunnar’s Kick-A Mobile Security and Development Class. Friday Summary: September 14, 2012. Favorite Outside Posts Mike Rothman: Antivirus programs often poorly configured, study finds. In a “master of the obvious” research finding, the OPSWAT guys tell us that even if AV worked (which it doesn’t), most folks misconfigure their controls anyway and don’t have the right stuff turned on. And you wonder why the busiest guys in the industry are the folks tracking all the breaches? Adrian Lane: Looking Inside Your Screenshots. So many good posts this week, but I thought this was the most interesting. I have never been a big fan of digital watermarking – it’s too easy to detect and evade for images and music files, and we know it degrades content. But in this case it’s more difficult to detect and does not degrade the content – and it gives Blizzard a hammer to use in legal cases as they have solid user/client identity. Sneaky, and if you give it some thought, there are other practical applications of this approach. Rich: Compliance lessons from Lance at EmergentChaos. As the resident Securosis cycling fan there’s no way I wasn’t going to pick this one. Only difference is Lance previously, clearly, stated he didn’t dope… which isn’t the same as his recent comments more focused on ‘compliance’. Project Quant Posts Malware Analysis Quant: Index of Posts. Malware Analysis Quant: Metrics – Monitor for Reinfection. Malware Analysis Quant: Metrics – Remediate. Malware Analysis Quant: Metrics – Find Infected Devices. Research Reports and Presentations Understanding and Selecting Data Masking Solutions. Evolving Endpoint Malware Detection: Dealing with Advanced and Targeted Attacks. Implementing and Managing a Data Loss Prevention Solution. Defending Data on iOS. Top News and Posts OWASP ZAP – the Firefox of web security tools Coders Behind the Flame Malware Left Incriminating Clues on Control Servers. A fun read. And if most code received this level of scrutiny, we would have much better code! Attack Easily Cracks Oracle Database Passwords Internet Explorer Fix is available now Media Manipulation and Social Engineering Mobile Pwn2Own ownage Hacker Steals $140k From Lock Poker Account RSnake donated XSS filter cheat sheet to OWASP BSIMM 4 Released Petco Releases Coupon On Internet, Forgets How Internet Works Majority of companies suffered a web application security breach Massachusetts group to pay $1.5M HIPAA settlement. I would love to see the “corrective plan of action”. Web Cryptography API draft published Java zero-day leads to Internet Explorer zero-day Blog Comment of the Week Remember, for every comment selected, Securosis makes a $25 donation to Hackers for Charity. This week’s best

You will probably read this on Thursday or even Friday, and that’s late. This week got all screwed up. It’s a little matter of a bunch of things happening at the same time, mostly personal, all good. So Monday was a holiday for me and starts the fall renewal process where I don’t set goals and don’t worry about what I’m striving for any more. It also turns out Monday night was the Falcons home opener. Many of my ATL buddies consider me a sinner for going to a football game on the High Holy Days. As I told the Boss, “Football is my other religion,” so there was never a question whether I would go. Normally I work late on Tuesday night, but we took XX2 to see a Ben Folds concert for her birthday. So up late Monday (I’ll get to that) and up late Tuesday. And a check-up for the kids Wednesday morning. Which means not a lot of time to actually, well, work. And I won’t even mention the road trip to go see the Giants play in Carolina on Thursday night. Yeah, I have a pretty sweet deal. Now back to Monday night. Everyone was amped up for the game. The Broncos and their rebuilt QB, Peyton Manning, were in town. The atmosphere was electric. Until about midway through the first quarter, when the replacement refs lost control of the game. I mean totally lost control. I have never seen anything like it. Penalties were reversed or not called. Fights broke out. The ball was spotted wrong. The first quarter took over an hour. Monday Night Football didn’t end until Tuesday morning. NFL referees are like security folks. Until they are gone and all hell breaks loose, you don’t even notice them. The NFL is taking a hard line about pensions, and they locked out the regular referees. I thought I had a sweet deal, but refs have it pretty good too. Some make as much as $120K a year to work maybe 20 games, including playoffs. Even based on the NFL’s latest offer, they’ll still get something like $15K contributed to retirement. But it’s not enough. Everyone always wants more. So the NFL finds replacement refs, most of whom do a good enough job. Some don’t (like the team on the field in ATL on Monday night). But at the end of the day, Steve Young was right. The NFL is in an inelastic demand situation, and how cool is it that a Hall of Fame QB talks about Econ 101 on national TV? The NFL will take a hard line because the fans will continue to show up. And we will. I’ll bitch and moan to my pals. The football talking heads (which is actually a much bigger echo chamber than security) will bitch and moan. The fans will keep showing up. As evidenced by my upcoming road trip to Charlotte. So the scabs will continue to be entrusted with keeping games on track and in control. Hopefully no one gets hurt and the games end fairly. Truth be told, scabs is a derogatory and unfair term for the replacement refs. It’s not their fault they are in deep water. It’s like taking a recent security graduate and asking them to defend something important from [name your favorite pen tester]. It’s going to end poorly. Ultimately the real refs will cave. It’s all about the leverage. It’s always about the leverage. The real refs have none. The NFL continues to have record ratings and record attendance and record activity in the NFL ecosystem, even with replacement refs. And once the refs realize they are very small cogs in a multi-billion-dollar wheel, they will pull off the scabs and get back to work. –Mike Photo credits: Scabs originally uploaded by Thomas Hawk Heavy Research We’re back at work on a variety of blog series, so here is a list of the research currently underway. Remember you can get our Heavy Feed via RSS, where you can get all our content in its unabridged glory. And you can get all our research papers too. Defending Against Denial of Service (DoS) Attacks Introduction Securing Big Data Security Issues with Hadoop Incite 4 U What makes an expert? Rob G levels a very uncharacteristic personal attack in his The know-nothings of cybersecurity, with the argument that someone who hasn’t actually configured a firewall or injected SQL cannot really be an expert. I reject that – it depends on what aspect of cybersecurity you’re talking about. A cybersecurity policy wonk probably hasn’t pwned devices using cool XSS code, but that doesn’t mean they don’t understand policies. And even if you are talking about technical expertise, it depends on who you’re talking to. Anyone can be an expert, if they are talking to n00bs. I’m no fan of generic statements that can’t be proven, nor am I a fan of the tons of charlatans claiming to be something they aren’t. But I don’t buy that there is only one kind of cybersecurity expert. – MR Mobile payments are HOT HOT HOT: Hat tip to Martin McKeay for bringing this up as we were recording this week’s Network Security Podcast. It looks like the PCI Council is going to release guidance on mobile payment applications. This is a big deal – a lot of apps tie into credit cards in some way, shape, or form. On one side are tools like Square payment card readers, but I suspect we might see other forms of apps tying to credit cards coming under scrutiny. It’s not something I would put money on – just something to watch. Think of all the QR-code based apps that use credit card somewhere on the back end. Then again, the PCI will do whatever they expect to piss off the smallest number of vendorsmembers, and Visa will end up making a blanket decision, anyway. – RM Estimates are like … never mind: There is a reason

Hang with me as I channel my inner Kerouac (minus the drugs, plus the page breaks) and go all stream of consciousness. To call this post an “incomplete thought” would be more than a little generous. I believe we are now deep in the early edge of a major inflection point in security. Not one based merely on evolving threats or new compliance regimes, but a fundamental transformation of the practice of security that will make it nearly unrecognizable when we finally emerge on the other side. For the past 5 years Hoff and I have discussed disruptive innovation in our annual RSA presentation. What we are seeing now is a disruptive conflagration, where multiple disruptive innovations are colliding and overlapping. It affects more than security, but that’s the only area about which I’m remotely qualified to pontificate. Perhaps that’s a bit of an exaggeration. All the core elements of what we will become are here today, and there are certain fundamentals that never change, but someone walking into the SOC or CISO role of tomorrow will find more different than the same unless they deliberately blind themselves. Unlike most of what I read out there, I don’t see these changes as merely things we in security are forced to react to. Our internal changes in practice and technology are every bit as significant contributing factors. One of the highlights of my career was once hanging out and having beers with Bruce Sterling. He said that his role as a futurist was to imagine the world 7 years out – effectively beyond the event horizon of predictability. What I am about to describe will occur over the next 5-10 years, with the most significant changes likely occurring in those last 7-10 years, but based on the roots we establish today. So this should be taken as much as science fiction as prediction. The last half of 2012 is the first 6 months of this transition. The end result, in 2022, will be far more change over 10 years than the evolution of the practice of security from 2002 through today. The first major set of disruptions includes the binary supernova of tech – cloud computing and mobility. This combination, in my mind, is more fundamentally disruptive than the initial emergence of the Internet. Think about it – for the most part the Internet was (at a technical level) merely an extension of our existing infrastructure. To this day we have tons of web applications that, through a variety of tiers, connect back to 30+-year-old mainframe applications. Consumption is still mostly tied to people sitting at computers at desks – especially conceptually. Cloud blows up the idea of merely extending existing architectures with a web portal, while mobility advances fundamentally redefine consumption of technology. Can you merely slop your plate of COBOL onto a plate of cloud? Certainly, right as you watch your competitors and customers speed past at relativistic speeds. Our tradition in security is to focus on the risks of these advances, but the more prescient among us are looking at the massive opportunities. Not that we can ignore the risks, but we won’t merely be defending these advances – our security will be defined and delivered by them. When I talk about security automation and abstraction I am not merely paying lip service to buzzwords – I honestly expect them to support new capabilities we can barely imagine today. When we leverage these tools – and we will – we move past our current static security model that relies (mostly) on following wires and plugs, and into a realm of programmatic security. Or, if you prefer, Software Defined Security. Programmers, not network engineers, become the dominant voices in our profession. Concurrently, four native security trends are poised to upend existing practice models. Today we focus tremendous effort on an infinitely escalating series of vulnerabilities and exploits. We have started to mitigate this somewhat with anti-exploitation, especially at the operating system level (thanks to Microsoft). The future of anti-exploitation is hyper segregation. iOS is an excellent example of the security benefits of heavily sandboxing the operating ecosystem. Emerging tools like Bromium and Invincea are applying even more advanced virtualization techniques to the same problem. Bromium goes so far as to effectively virtualize and isolate at a per task level. Calling this mere ‘segregation’ is trite at best. Cloud enables similar techniques at the network and application levels. When the network and infrastructure are defined in software, there is essentially zero capital cost for network and application component segregation. Even this blog, today, runs on a specially configured hyper-segregated server that’s managed at a per-process level. Hyper segregated environments – down, in some cases, to the individual process level – are rapidly becoming a practical reality, even in complex business environments with low tolerance for restriction. Although incident response has always technically been core to any security model, for the most part it was shoved to the back room – stuck at the kids’ table next to DRM, application security, and network segregation. No one wanted to make the case that no matter what we spent, our defenses could never eliminate risk. Like politicians, we were too frightened to tell our executives (our constituency) the truth. Especially those who were burned by ideological execs. Thanks to our friends in China and Eastern Europe (mostly), incident response is on the earliest edge of getting its due. Not the simple expedient of having an incident response plan, or even tools, but conceptually re-prioritizing and re-architecting our entire security programs – to focus as much or more on detection and response as on pure defense. We will finally use all those big screens hanging in the SOC to do more than impress prospects and visitors. My bold prediction? A focus on incident response, on more rapidly detecting and responding to attacker-driven incidents, will exceed our current checklist and vulnerability focused security model, affecting everything from technology decisions to budgeting and staffing. This doesn’t

How do I secure “big data”? A simple and common question. But one without a direct answer – simple or otherwise. We know thousands of firms are working on big data projects, from small startups to large enterprises. New technologies enable any company to collect, manage, and analyze incredibly large data sets. As these systems become more common, the repositories are more likely to be stuffed with sensitive data. Only after companies are reliant on “big data” do they ask “How can I secure it?” This question comes up so much, attended by so much interest interest and confusion, that it’s time for an open discussion on big data security. We want to cover several areas to help people get a better handle on the challenges. Specifically, we want to cover three things: Why It’s Different Architecturally: What’s different about these systems, both in how they process information and how they are deployed? We will list some of the specific architectural differences and discuss how how they impact data and database security. Why It’s Different Operationally: We will go into detail on operational security issues with big data platforms. We will offer perspective on the challenges in securing big data and the deficiencies of the systems used to manage it – particularly their lack of native security features. Recommendations and Open Issues: We will outline strategies for securing these data repositories, with tactical recommendations for securing certain facets of these environments. We will also highlight some gaps where no good solutions exist. Getting back to our initial question – how to secure big data – what is so darn difficult about answering? For starters, “What is big data?” Before we can offer advice on securing anything, we need to agree what we’re talking about. We can’t discus big data security without an idea of what “big data” means. But there is a major problem: the term is so overused that it has become almost meaningless. When we talk to customers, developers, vendors, and members of the media, they all have their have their own idea of what “big data” is – but unfortunately they are all different. It’s a complex subject and even the wiki definition fails to capture the essence. Like art, everyone knows it when they see it, but nobody can agree on a definition. Defining Big Data What we know is that big data systems can store very large amounts of data; can manage that data across many systems; and provide some facility for data queries, data consistency, and systems management. So does “big data” mean any giant data repository? No. We are not talking about giant mainframe environments. We’re not talking about Grid clusters, massively parallel databases, SAN arrays, cloud-in-a-box, or even traditional data warehouses. We have had the capability to create very large data repositories and databases for decades. The challenge is not to manage a boatload of data – many platforms can do that. And it’s not just about analysis of very large data sets. Various data management platforms provide the capability to analyze large amounts of data, but their cost and complexity make them non-viable for most applications. The big data revolution is not about new thresholds of scalability for storage and analysis. Can we define big data as a specific technology? Can we say that big data is any Hadoop HDFS/Lustre/Google GFS/shard storage system? No – again, big data is more than managing a big data set. Is big data any MapReduce cluster? Probably not, because it’s more than how you query large data sets. Heck, even PL/SQL subsystems in Oracle can be set up to work like MapReduce. Is big data an application? Actually, it’s all of these things and more. When we talk to developers, the people actually building big data systems and applications, we get a better idea of what we’re talking about. The design simplicity of these these platforms is what attracts developers. They are readily available, and their (relatively) low cost of deployment makes them accessible to a wider range of users. With all these traits combined, large-scale data analysis becomes cost-effective. Big data is not a specific technology – it’s defined more by a collection of attributes and capabilities. Sound familiar? It’s more than a little like the struggle to define cloud computing, so we’ll steal from the NIST cloud computing definition and start with some essential characteristics. We define big data as any data repository with the following characteristics: Handles large amounts (petabyte or more) of data Distributed, redundant data storage Parallel task processing Provides data processing (MapReduce or equivalent) capabilities Central management and orchestration Inexpensive – relatively Hardware agnostic Accessible – both (relatively) easy to use, and available as a commercial or open source product Extensible – basic capabilities can be augmented and altered In a nutshell: big, cheap, and easy data management. The “big data” revolution is built on these three pillars – the ability to scale data stores at greatly reduced cost is makes it all possible. It’s data analytics available to the masses. It may or may not have traditional ‘database’ capabilities (indexing, transactional consistency, or relational mapping). It may or may not be fault tolerant. It may or may not have failover capabilities (redundant control nodes). It may or may not allow complex data types. It may or may not provide real-time results to queries. But big data offers all those other characteristics, and it turns out that they – even without traditional database features – are enough to get useful work done. So does big data mean the Hadoop framework? Yes. The Hadoop framework (e.g. HDFS, MapReduce, YARN, Common) is the poster child for big data, and it offers all the characteristics we outlined. Most big data systems actually use one or more Hadoop components, and extend some or all of its basic functionality. Amazon’s SimpleDB also satisfies the requirements, although it is architected differently than Hadoop. Google’s proprietary BigTable architecture is very similar to Hadoop, but we exclude

Our very own Gunnar Peterson is co-presenting what looks like an insanely awesome mobile application security class. And with a name like The Mobile App Sec Triathlon you know I am interested. The class is November 5-7 in San Jose, and you can get more information and sign up This class covers what developers, architects, and security people should know when working on Mobile, iOS, and Android. The first day is more high level, with the second two days all developer hands-on. Gunnar also wrote a post on why he trains, with a lot more information. This is really a great opportunity, and I don’t believe there is anyone else as qualified offering this sort of class. Share:

In today’s news we see yet another zero-day Internet Explorer exploit being used in the wild. And once again, soon after becoming public, an exploit was added to Metasploit. Well, sort of. While the in-the-wild attack only works against Windows XP, the Metasploit version works against Windows 7 and Vista. (Note that IE 10 isn’t affected). You can read the article linked above for the details, but this gets to something I have been recommending privately for a while: support 2 browsers, even if one is only for emergencies. First of all, ideally you’ll be on a modern operating system. I’m not one to blame the victim, but allowing XP is a real problem – which I know many of you fight every day. Second, this advice doesn’t help with all browser-based attacks, especially Java. But you can configure it in a way that helps. Choose a secondary browser that is allowed for web browsing. Chrome is most secure right now, but make sure you set its privacy defaults to not bleed info out to Google. Ideally block Java in the browser. Maybe even Flash, depending on how you feel about the Chrome sandbox. If something like this IE flaw hits, notify users to use the secondary browser for outside websites (odds are you need IE for internal web apps programmed by idiots or 19th-century transplants, and so cannot ban it completely). If you can, set a network policy that (temporarily) blocks IE from accessing external sites (again, you can make exemptions for partners). Unfortunately I don’t believe many tools support this. I know this advice isn’t perfect. And there are tools like Invincea and (soon) Bromium that can likely stop this stuff cold in the browser – as well as a few network tools, although history shows signature-based defenses aren’t all that effective here. But if you can pull it off you aren’t stuck waiting for a patch or another workaround. Especially if you go with the “block Java / isolate or block Flash” option. This approach allows you to still only support one browser for your applications, and use a secondary one when needed without users having to violate policy to install it themselves. Share: