Research

A lot is going on in security land, so Rich, Mike, and Adrian return with another 3 for 5 episode. Three stories, five minutes each, all the sarcastic bite in a convenient package. The audio-only version is up too.   Share:

Rich here. A quick mention: I will run a security session at Camp DevOps in Boulder on May 20th. I am looking forward to learning some things myself. My wife and I spent this past weekend up in Flagstaff, AZ for our anniversary. I am not much of a city guy, and am really much happier up in the mountains. There is just something about the thin air that lifts my spirits. Our home is on the Northwest corner of Phoenix, with easy access to the hills, so Flag is a frequent getaway. It has mountains, half a dozen craft breweries, a compact downtown with surprisingly good food, and a place called “Hops on Birch” – what’s not to like? The best part (that I will talk about) was walking into a coffee shop/bar around lunchtime and realizing it was where all the local bartenders congregate to recover. We learned a lot about the town while sipping Irish coffees. Scratch that – the best part was ditching the kids. And walking to three of those craft breweries before ending with dinner at the Thai place. But top three for sure. As a researcher sometimes I forget that what seems blatantly obvious… isn’t. Take the reports today about Apple revealing what data it can share with law enforcement. I figured it was common knowledge, because Apple’s security model is pretty well documented, and I even lay out what is protected and how in my iOS security paper. But most reports miss the big piece: Apple can access the file system on a passcode protected device. Anyone else needs to use a jailbreak technique, which I find interesting. Especially because jailbreaks don’t work on recent hardware without a passcode. I had a pretty cool moment this week. I was writing an article on security automation for DevOps.com. I didn’t have the code for what I wanted, and it involved something I had never tried before. It only took about 20 minutes to figure it out and get it working. My days as an actual coder are long over, but it feels good to have recovered enough knowledge and skills that I can pinch hit when I need to. But it didn’t last long. I spent about 12 hours yesterday struggling to repair one of our cloud security training class (CCSK) labs. We have the students pick the latest version of Ubuntu in the AWS user interface when they launch instances, and then insert some scripts I wrote to set up all the labs and minimize their need for the command line. It pains me, but a lot of people out there get pissed if you force them to type in a black box instead of clicky-clicky. Thinking is hard and all. Ubuntu 14.04 broke one of the key scripts needed to make the labs work. I started debugging and testing, and for the life of me couldn’t figure it out. Nothing in logs, no errors even in verbose mode. I quickly narrowed down the broken piece, but not why it was broken. Running all the commands manually worked fine – it was only broken when running scripted. MySQL and Apache take a lot of domain knowledge I don’t have, and the Googles and Bings weren’t much help. Eventually I realized restarting MySQL was dropping the user account my script added. By changing the order around I got it working, but I still feel weird – I don’t know why it dropped the account. If you know, please share. On the upside I made the scripts much more user-friendly. I thought about completely automating it with all the DevOps stuff I have been learning, but the parts I have in there are important to reinforce the educational side of things so I left them. So a great weekend, fun coding, and a reminder of how little I really know. On to the Summary: Webcasts, Podcasts, Outside Writing, and Conferences Mike quoted in “Do you really think the CEOs resignation from Target was due to security?” Adrian and Mort speaking next week at Secure360. Rich with Adam Engst at TidBITS on the iOS Data Protection bug. Favorite Securosis Posts Adrian Lane: Firestarter: There Is No SecDevOps. The boys did a nice job with this one – and Mike got all existential! That mindful stuff must be having an effect. Mike Rothman: Firestarter: There Is No SecDevOps. I get to say “Security must lose its sense of self in order to survive,” in this week’s Firestarter. That’s all good by me. We were a little light this week – sorry about that. Big projects, travel, and deadlines have been ongoing problems. But heck, we still blog more than nearly anyone else, so there! Other Securosis Posts Incite 5/7/2014: Accomplishments. New Paper: Advanced Endpoint and Server Protection. Favorite Outside Posts Adrian Lane: Shifting Cybercriminal Tactics. You may be tired of cyber security reports, but this one from MS is a quick read – and the change in tactics is a sign that MS’ efforts on trustworthy computing are working. Rich: The Hunt for El Chapo. I have been on a real crime story kick lately. Mike Rothman: Antivirus is Dead: Long Live Antivirus! Krebs goes on a rant about how attackers test their stuff before attacking you with it, and that is a big reason AV doesn’t work well any more. Research Reports and Presentations Defending Against Network-based Distributed Denial of Service Attacks. Reducing Attack Surface with Application Control. Leveraging Threat Intelligence in Security Monitoring. The Future of Security: The Trends and Technologies Transforming Security. Security Analytics with Big Data. Security Management 2.5: Replacing Your SIEM Yet? Defending Data on iOS 7. Eliminate Surprises with Security Assurance and Testing. What CISOs Need to Know about Cloud Computing. Defending Against Application Denial of Service Attacks. Top News and Posts What Target and Co aren’t telling you: your credit card data is still out there. Network Admin Allegedly Hacked Navy While on an Aircraft Carrier. Serious security flaw in OAuth, OpenID discovered. How the Target CEO resignation will affect other execs’

Yesterday I was in Winnipeg. By choice! I was invited to speak at the Western Canada Information Security Conference, and there isn’t much I like better than giving talks in Canada. Folks are nice. They appreciate when you come up to their towns to talk. They don’t say much during the pitch, but they come up after the session or in the coffee line and make it clear that they were listening. Just like in the Northeast. OK, not so much. I was doubly excited to do yesterday’s talk because they asked me to do my Happyness talk. It is my favorite talk to do – not because I think it provides a good message for the audience… even though it does. Not because it gives people tools to deal with the despondency that is part of the security profession… although it does that too. I love giving the Happyness talk because it forces me to take a look at where I’ve been and what I’ve done to improve myself over the past 7 years. When I first put the pitch together, I had a picture of Grumpy with the caption: “My alter ego.” When I updated the pitch last year, I changed that caption to be: “I used to be this guy.” That’s right, I’m no longer grumpy. Really. If you perceive me being grumpy, you bought into my persona. That’s not me anymore. If you met me today and didn’t know me, you wouldn’t think I’m grumpy or even curmudgeonly. I didn’t really appreciate that fact until I was going through the deck this week to do some minor tuning. I realized I have spent a long time trying to improve my mental game. To deal with my impatience and anger. To do this I embraced mindfulness practices (see the Neuro-hacking talks I do with JJ) and it has made a huge difference in my mental health. I need to celebrate that accomplishment. So I think I will. Of course that doesn’t mean I don’t get frustrated or impatient anymore. I’m human, contrary to popular belief. But I don’t hold onto the frustration, and the impatience passes quickly. Which, given where I started, is pretty cool. While I’m celebrating I should probably acknowledge how I have transformed my physical self as well. Back in 2006 I was 70 pounds heavier with high blood pressure and all sorts of other issues starting to manifest. So I decided I was tired of being fat and out of shape and dedicated myself to change. It has been a long process and it is still a daily battle, but at this point I am in the best shape of my life – in my mid-40s. Go figure. That warrants a celebration, no? I was on the express train to the grave and now I have a chance to live long enough for my kids to have to change my diapers. I don’t know how long I’ll be here, but when I go it won’t be because I didn’t take care of myself. So today I will celebrate my accomplishments, both mind and body. I don’t really ever pat myself on the back, so this is both new and uncomfortable. But I’ll do it because I should. Because hard work should be acknowledged – even if it is only acknowledging yourself. [5 minutes pass] OK, I’m done celebrating. There is work to do. Windmills to chase. Things to accomplish. But this is progress for me. I usually don’t celebrate accomplishments for even 5 minutes. It actually feels pretty good. Come to think of it, I highly recommend it. There may be something to this celebration thing… –Mike Photo credit: “Destination: Goal” originally uploaded by Jay Cox Securosis Firestarter Have you checked out our new video podcast? Rich, Adrian, and Mike get into a Google Hangout and.. hang out. We talk a bit about security as well. We try to keep these to 15 minutes or less, and usually fail. May 5 – There Is No SecDevOps April 28 – The Verizon DBIR April 14 – Three for Five March 24 – The End of Full Disclosure March 19 – An Irish Wake March 11 – RSA Postmortem Feb 21 – Happy Hour – RSA 2014 Feb 17 – Payment Madness Feb 10 – Mass Media Abuse Feb 03 – Inevitable Doom 2014 RSA Conference Guide In case any of you missed it, we published our fifth RSA Conference Guide back in February. Yes, we do mention the conference a bit, but it’s really our ideas about how security will shake out in 2014. You can get the full guide with all the memes you can eat. Heavy Research We are back at work on a variety of blog series, so here is a list of the research currently underway. Remember you can get our Heavy Feed via RSS, with our content in all its unabridged glory. And you can get all our research papers too. Understanding Role-based Access Control Advanced Concepts Introduction NoSQL Security 2.0 Understanding NoSQL Platforms Introduction Newly Published Papers Advanced Endpoint and Server Protection Defending Against Network-based DDoS Attacks Reducing Attack Surface with Application Control Leveraging Threat Intelligence in Security Monitoring The Future of Security Security Management 2.5: Replacing Your SIEM Yet? Defending Data on iOS 7 Eliminating Surprises with Security Assurance and Testing Incite 4 U Dropping your stuff: The Intralinks folks just published some interesting research, highlighted on Graham Cluley’s blog, showing how Dropbox and Box leak links to private files through Google searches. They prove their point by showing a 2012 tax return and a mortgage application. That’s awesome. It turns out anyone with a link for private sharing can see the file. No authentication needed. More awesome. The issue can also manifest if someone clicks a link embedded within a document viewed using Dropbox’s web preview function, because that link is included in the referrer information. So how do you solve the problem? Don’t share links. Duh. Oh, not an issue? Use business cloud storage services, which allow you to restrict access to shared links. We are only beginning to scratch the surface

Anti-virus is basically dead, at least according to the biggest anti-virus vendor. The good news is that signature-based AV has actually been dead for a long time; even the big players have been broadening their capabilities to assess, prevent, detect, and investigate advanced malware on endpoints and servers. There has been a tremendous amount of activity and innovation in protecting endpoint and servers, driven by necessity: Endpoint protection has become the punching bag of security. For every successful attack, the blame seems to point directly to a failure of endpoint protection. Not that this is totally unjustified — most solutions for endpoint protection have failed to keep pace with attackers. But hygiene and awareness alone will not deter advanced attackers very long. We frequently say advanced attackers are only as advanced as they need to be: they take the path of least resistance. But the converse is also true. When these adversaries need advanced techniques, they use them. Traditional malware defenses such as antivirus don’t stand much chance against a zero-day attack. Our Advanced Endpoint and Server Protection paper highlights the changes in threat management resulting from these advanced attackers using advanced tactics. We discuss changes in prevention, as well as advances in both detection and investigation. This is really a call to action to rethink how you deal with advanced adversaries, and ultimately how you protect your devices. Advanced adversaries require organizations to rethink how they manage threats. The idea that targeted attacks can be prevented consistently is a pipe dream, so organizations need to shift away from largely ineffective legacy technologies for protecting endpoints and servers. More specifically this means devoting more resources and investing in innovative approaches to blocking attacks in the first place, including advanced heuristics, application control, and isolation technologies. But even with significant investment in innovative prevention, a persistent attacker will still compromise your devices. This highlights the necessity of shifting security investment toward detecting and investigating attacks. We would like to thank the companies who have licensed this content (in alphabetical order): Bit9 + Carbon Black; Cisco/Sourcefire; and Trusteer, an IBM Company. We make this point frequently, but without security companies understanding and getting behind our Totally Transparent Research model, you wouldn’t be able to enjoy our research. Share:

Adrian is off at the altar of Buffett (the other one – not the one I wear a coconut bra for), so Mike and I delved into SecDevOps, triggered by a post from Andrew Storms over at DevOps.com. This is where the world is heading folks – you might as well prepare yourselves now. The audio-only version is up too.   Share:

Glenn Fleishman (@GlennF) tweeted “Next month’s Wired: ‘We painstakingly reconstructed Steve Jobs’ wardrobe so you can wear it, too.’” A catty response to Wired Magazine’s recent reconstruction of Steve Jobs’ stereo system. Unlike Mr. Fleishman I was highly interested in this article, and found it relevant to current events. For people who love music and quality home music reproduction, iTunes’ disgustingly low-resolution MP3 files seem at odds with Jobs’ personal interest in HiFi. The equipment surrounding Jobs in the article’s lead picture was not just good stereo equipment, and not ‘name brand’ equipment either – but instead esoteric brands aimed at aficionados (indicating Jobs was very serious about music reproduction and listening). The irony is that someone who was heavily invested in HiFi would become the principal purveyor of what audiophiles deem unholy evil. Sure, MP3s are a great convenience – just not so great for music quality. This picture has made HiFi trade magazines over the years, and while Jobs was alive the vanishingly small population of audiophiles held out hope that we would someday get high-resolution music from iTunes. The rumor – of which confirmation would be a great surprise – is that we may finally get HiRes files from iTunes, which I suspect is why this picture was the subject of such scrutiny. The market for high-quality headphones has jumped 10-fold in the last 7 years, and vinyl record sales have gone up 6-fold in the same period, showing public interest in higher quality audio while CD sales plummet. Even piracy-paranoid anti-consumer vendors like Sony have begun to sell HiRes DSD files, so Apple has likely noticed these trends and we can hope they will follow suit. Garbage in, garbage out is a basic axiom I learned when I first started programming database applications, and it remains true for any database, including NoSQL variants. Write any query you want – if the data is bad, the results are meaningless. But even if the data is completely accurate, depending on how you write your queries, you may produce results that don’t mean what you think they do. The learning curve with NoSQL is even weirder – many data scientists are still learning how to use these platforms. Consider that for many NoSQL users, the starting point is often just looking for stuff – we don’t necessarily know what we are looking for, but we often discover interesting patterns in the data. And when we do, we try to make sense of them. This itself is a form of bias. In this process we may write and rewrite data queries many times over, trying to refine a hypothesis. But the quality and completeness of the data, as well as your ability to mine it effectively with queries, can lead to profound revelations – or perhaps to poop. More likely it’s somewhere in-between, but both extremes are a possibility. One of Gunnar’s key themes from a post earlier this year is to understand the balance between objective and subjective aspects of metrics. He said, “I am very tired of quant debates where … the supposed quant approach beats the subjective approach.” It is not a question of whether you are subjective or not – it is there in your biases when you make the model… “To me the formula for infosec is objective measures through logging and monitoring, subjective decisions on where to place them, and what depth, a mix of subjective and objective review of the logs and data feedback from the system’s performance over time.” I raise these points because while we examine our navels for effective uses of analytics for business, operations, and security metrics, practiced FUD-ites work their magic to make analysis irrelevant. An exaggerated example to make a point is this post on discrimination potential in big data use, where we see political opponents claiming big data is biased before it has been put to use. A transparent attempt to kill funding based on data analysis, without analysis to back it up! It is easier for a politician to generate fear by labeling this mysterious thing called “big data” as discriminatory in order to get their way than to discredit an actual analysis. They are feeding off audience bias (popular opinion). Many people naively believe “It’s big data so it’s evil” in response to NSA spying and corporations performing what feels like consumer espionage. It does not even matter if the data or tools will be used used effectively – bias and fear are used to kill metrics-based decisions. Ironic, right? As a security example: in each of the last three years – always a few months after the release of the Verizon DBIR – a handful of vendors has told me how the DBIR says the number one threat is from insiders! When I point out that the report says the exact opposite, they always argue that an outsider becomes an insider once they have breached your systems. And post-Snowden many enterprises are mostly worried about being Snowdened – regardless of any breach statistics. I don’t have any lesson here, or a specific safety tip to offer, but if you have metrics and data for decision support perform your own review. It will help remove some bias from the analysis. People who are financially invested in a specific worldview deliberately misinterpret, discredit, and fund biased studies, to support their position – their biased arguments drive you to conclusions that benefit them. On to the Summary: Webcasts, Podcasts, Outside Writing, and Conferences Rich quoted on SecDevOps. Favorite Securosis Posts David Mortman: NoSQL Security: Understanding NoSQL Platforms. Adrian Lane: XP Users Twisting in the Wind. For the picture, if nothing else. Mike Rothman: NoSQL Security: Understanding NoSQL Platforms. I have long said Adrian has forgotten more about databases than most of us know. He has proven it once again with this primer on NoSQL databases… Other Securosis Posts Incite 4/30/2014: Sunscreen. Firestarter: The Verizon DBIR. Defending Against Network-based Distributed Denial of Service Attacks [New Paper]. Summary: Time and Tourists. Pass the Hemlock. Favorite Outside Posts Mike Rothman: UltraDNS Dealing with DDoS Attack. The cyber equivalent of going up to someone and hitting them with

After a mostly miserable winter, at least in terms of the weather, spring is here. And some days it feels like summer. This past weekend was awesome. A little hot, but nice. Sun shining. Watching the kids play LAX. Dinner/drinks to celebrate two of my best friends completing a trail marathon. Yes, they ran 26.2 miles through the woods. I didn’t say my friends were overly bright, did I? What I didn’t wear was sunscreen. So when you check out the Firestarter we recorded Monday, you will see I spent some time in the sun. I guess I shouldn’t be surprised – I do this every year. I just forget. It’s doesn’t feel that hot. The sun isn’t that strong. Until I’m getting ready for bed and I look like a tomato. Evidently the sun is that strong. And it was that hot. So the farmer sunburn is in full effect. When I think of sunscreen I always think of an awesome column by Mary Schimich, which was wrongly attributed to Kurt Vonnegut for years. It’s not quite Steve Jobs’ commencement speech, but it’s pretty good. Because it reminds us of the important stuff, like wearing sunscreen. She also reminds us to not worry. Worrying is not important, and it doesn’t help you do anything anyway. If it’s out of your control then what can you do? If it is within your control, then fix it. We also shouldn’t waste time on jealousy or competing with folks. It’s not a race. Not with anyone else anyway. It is about consistent improvement, and being the best you that you can be. At least that’s the way I try to live. But the title of that speech is “Advice, like youth, probably just wasted on the young”. Which is exactly right. I couldn’t understand the logic of wearing sunscreen when I was 22. Just like I couldn’t understand why I shouldn’t worry about what I have or haven’t accomplished. Nor could I understand the importance of living right now – not tomorrow, and certainly not reliving yesterday. I couldn’t understand that stuff, and if you’re 22, you probably have no idea what I’m talking about. But at some point you will, and the folks in my age bracket probably understand. I wouldn’t go back in time because I didn’t know anything. And it turns out I am actually in better physical shape, and can afford better beer now than 25 years ago. I finally understand what’s important and can appreciate how every setback taught me something I use almost every day. Cool, huh? By the way, that doesn’t mean I will wear sunscreen next spring either. But at least I’ll have the perspective to laugh at the fact that I do the same stuff every year, as I reach for the aloe. –Mike Photo credit: “Use plenty of sunscreen originally uploaded by Alex Liivet Securosis Firestarter Have you checked out our new video podcast? Rich, Adrian, and Mike get into a Google Hangout and.. hang out. We talk a bit about security as well. We try to keep these to 15 minutes or less, and usually fail. April 28 – The Verizon DBIR April 14 – Three for Five March 24 – The End of Full Disclosure March 19 – An Irish Wake March 11 – RSA Postmortem Feb 21 – Happy Hour – RSA 2014 Feb 17 – Payment Madness Feb 10 – Mass Media Abuse Feb 03 – Inevitable Doom Jan 27 – Government Influence Jan 20 – Target and Antivirus Jan 13 – Crisis Communications 2014 RSA Conference Guide In case any of you missed it, we published our fifth RSA Conference Guide. Yes, we do mention the conference a bit, but it’s really our ideas about how security will shake out in 2014. You can get the full guide with all the memes you can eat. Heavy Research We are back at work on a variety of blog series, so here is a list of the research currently underway. Remember you can get our Heavy Feed via RSS, with our content in all its unabridged glory. And you can get all our research papers too. Understanding Role-based Access Control Advanced Concepts Introduction NoSQL Security 2.0 Understanding NoSQL Platforms Introduction Advanced Endpoint and Server Protection Quick Wins Detection/Investigation Prevention Assessment Introduction Newly Published Papers Defending Against Network-based DDoS Attacks Reducing Attack Surface with Application Control Leveraging Threat Intelligence in Security Monitoring The Future of Security Security Management 2.5: Replacing Your SIEM Yet? Defending Data on iOS 7 Eliminating Surprises with Security Assurance and Testing Incite 4 U Revisiting monoculture: Dan Geer is at it again. One of our preeminent security thinkers is back on the monoculture theme, revisiting his position that any single component used by a majority of technology users represents undue risk. Back in 2003 Dan talked about the risks of Windows dominance. He was right and still is. Now he has applied the monoculture concept to OpenSSL, which was the component that enabled Heartbleed. The reality is, these base components are everywhere. You probably remember that SQL*Slammer leveraged the Jet database. You didn’t buy the Jet DB? Of course you did! It was just built into stuff you wanted. Same deal with OpenSSL, and about a zillion other components that are built in everywhere. Is there a way to contain this kind of risk? Or at least understand it? Um, ask Josh Corman. – MR Sometimes good enough is… Does anyone outside the SIM card alliance really think that Host Card Emulation – mobile app software that mimics a secure element function – is not a threat to their hardware strategy? For that matter, does anyone really believe that HCE is not secure enough for EMV payments? While mobile carriers and device manufactures fumble about putting different secure elements with capabilities on a subset of devices and call that a standard, firms like Apple and Square will simply deliver a seamless, consistent, user-friendly payment experience for most mobile devices. Sure, SIM cards are more secure, but when we are talking about basically one credit card per mobile device, HCE solutions do not need to provide infallible security to be

Windows XP’s recent end of life has garnered a bit of industry recognition. Mostly from vendors pushing controls to lock down the ancient operating system. Folks who are stuck on XP are, well, stuck. And now there is a new exploit in the wild that takes advantage of IE, so what are XP users to do? About 30% of all desktops are thought to be still running Windows XP and analysts have previously warned that those users would be vulnerable to attacks from cyber-thieves. Microsoft has suggested businesses and consumers still using the system should upgrade to a newer alternative. Twist in the wind, that’s what, at least according to Microsoft. That’s their answer: upgrade. If that’s not an option, lock down the device using a technology like privilege management or full application whitelisting. If those aren’t options either, you had better get some good forensics tools, because those devices will be owned. Sooner rather than later. This is the first unpatched XP issue in the new regime. It won’t be the last. Photo credit: “Four storms and a twister” originally uploaded by JD Hancock Share:

I started this series on recommendations for securing NoSQL clusters a couple weeks ago, so sorry for the delay posting the rest of the series. I had some difficulty contacting the people I spoke with during the first part of this “big data” research project, and some vendors were been slow to respond with current product capabilities. As I hoped, launching this series “shook the tree of knowledge”, and several people responded to my inquiries. It has taken a little more time than I thought to schedule calls and parse through the data, but I am finally restarting, and should be able to quickly post the rest of the research. The first step is to describe what NoSQL is and how it differs from the other databases you have been protecting, so you can understand the challenges. Let’s get started… NoSQL Overview In our last research paper on this subject, we defined NoSQL platforms with a set of characteristics which differentiate them from big iron/proprietary MPP/”cloud in the box” platforms. Yes, some folks slapped “big data” stickers on the same ‘ol stuff they have always sold, but most people we speak with now understand that those platforms are not Hadoop-like, so we can dispense with discussion of the essential characteristics. If you need to characterize NoSQL platforms please review our introductory sections on securing big data clusters. Fundamentally, NoSQL is clustered data analytics and management. And please stop calling it “big data” – we are really discussing a building-block approach to databases. Rather than the packaged relational systems we have grown accustomed to over the last two decades, we now assemble different pieces (data management, data storage, orchestration, etc.) to satisfy specific requirements. The early architects of this trend had a chip on their collective shoulders, and they were trying to escape the shadow of ubiquitous relational databases, so they called their movement ‘NoSQL’. That term nicely illustrates their opposition to relational platforms. Not that they do not support SQL – many in fact do support Structured Query Language syntax. Worse, the term “big data” was used by the press to describe this trend, assigning a label taken from the most obvious – but not most important – characteristic of these platforms. Unfortunately that term serves the movement poorly. ‘NoSQL’ is not much better but it is a step in the right direction. These databases can be tailored to focus on speed, size, analytic capabilities, failsafe operation, or some other goal, and they enable computation on a massive scale for very little money. But just as importantly, they are fully customizable to meet different needs – often simultaneously! So what does a NoSQL database look like? The “poster child” is Hadoop. The Hadoop framework (the combination of Hadoop File System (HDFS) with other services such as YARN, Hive, Pig, etc.) is the general architecture employed by most NoSQL clusters, but many more are in wide use – including Cassandra, MongoDB, Couch, AWS SimpleDB, and Riak. There are over 125 known variations, but those account for the majority of customer usage today. NoSQL platforms scale and perform so well because of two key principles: distribution of data management and query processing across many servers (possibly thousands), combined with a modular architecture that allows different services to be swapped in as needed. Architecturally a Hadoop cluster looks like this: It is useful to think of the Hadoop framework as a ‘stack’, much like the famous LAMP stack. These pieces are normally grouped together, but you can mix and match and add to the stack as needed. For example Sqoop and Hive are replacement data access services. You can select a big data environment specifically to support columnar, graph, document, XML, or multidimensional data – all collectively called ‘NoSQL’ because they are not constrained by relational database constructs or a relational query parser. You can install different query engines depending on the type of data being stored. Or you can extend HDFS functionality with logging tools like Scribe. The entire stack can be configured and extended as needed. I have included Lustre, GFS, and GPFS, as they are all technically alternatives to HDFS but not as widely used. But the point is that this modular approach offers great flexibility at the expense of more difficult security, because each option brings its own security options and deficiencies. We get big, cheap, and easy data management and processing – with lagging security capabilities. This diagram illustrates some of the many variables in play. It seems like every customer uses a slightly different setup – tweaking for performance, manageability, and programmer preference. Many of the people we spoke with are on their second or third stack architecture, having replaced components which did not scale or perform as needed. This flexibility is great for functionality, but makes it much more difficult for third-party vendors to produce monitoring or configuration assessment tools. There are few constants (or even knowns) for NoSQL clusters, and things are too chaotic to definitively identify best or worst practices across all configurations. For convenient reference, we offer a list of key differences between relational and NoSQL platforms which impact security. Relational platforms typically scale by replacing a server with a larger one, rather than by adding many new servers. Relational systems have a “walled garden” security model: you attach to the database through well-defined interfaces, but internal workings are generally not exposed. Relational platforms come with many tools such as built-in encryption, SQL validation, centralized administration, full support for identity management, built-in roles, administrative segregation of duties, and labeling capabilities. You can add many of these features to a NoSQL cluster, but still face the fundamental problem of securing a dynamic constellation of many servers. This makes configuration management, patching, and server validation particularly challenging. Despite a few security detractors, NoSQL facilitates data management and very fast analysis on large-scale data warehouses, at very low cost. Cheap, fast, and easy are the three pillars this movement has been built upon – data analytics for the masses. A NoSQL

After missing a week, Rich, Mike, and Adrian return to talk about birthdays, the annual Verizon Data Breach Investigations Report, and child-induced alcohol consumption. The audio-only version is up too.   Share: