IaaS Storage 101

I started writing up a post on IaaS encryption options and quickly realized I should probably precede it with a post outlining the IaaS storage options first. One slightly confusing bit is that IaaS storage really falls into two categories: storage as a service where the storage itself is the product, and storage for IaaS compute instances, where the storage is tied to running virtual machines. IaaS storage options include: Raw storage: As far as I can tell, this is only available for private clouds, and not on every platform. For certain high-speed operations it allows you to map a virtual volume to dedicated raw media. This skips abstraction layers for increased performance, but you lose many of the advantages of cloud storage. It’s rarely used, and may only be available on VMWare. Volume storage: The easiest way to think of volume storage is as a virtual hard drive for your instances. There are a few different architectures, but volumes are typically a clump of assigned blocks (often stored redundantly in the back end). When you create a volume the volume controller assigns the blocks, distributes them onto the physical storage infrastructure, and presents them as a raw volume. You then need to attach the volume to an instance, install partitions and file systems on it, and manage it like a drive. Although it presents as a single drive to your instance, volume storage is more like RAID – each block is replicated in multiple locations on different physical drives. Amazon EBS and Rackspace RAID volumes are examples. Object storage: Object storage is sometimes referred to as file storage. Rather than a virtual hard drive, object storage is more like a file share. Object storage performs more slowly, but is more efficient. The back end can be structured in different ways – most often a database / file system hybrid, with a bunch of processes to keep track of where everything is stored, replication, cleanup, and other housekeeping functions. Amazon S3, Rackspace Cloud Files, and OpenStack Swift are examples. For our purposes, we will consider cloud databases part of PaaS. So when we talk about IaaS storage, we are mostly talking volumes and objects. Volumes are like hard drives, and object storage is effectively a file share with a nifty API. An additional piece is important for running IaaS instances: image management. Images (such as Virtual Machine Images and Amazon Machine Images) can be stored in a variety of ways, but most often in object storage because it’s cheaper and more efficient. Layered on top is an image manager such as OpenStack Glance, which tracks the images and ties them into the compute management plane. When you create an IaaS instance you pick an image, which the image manager then pulls from object storage and streams to the hypervisor/system that will host the instance. But the image manager doesn’t need to use object storage. Glance, for example, can use pretty much anything – including local file storage, which is particularly handy in test environments. Lastly, we can’t forget about snapshots. Snapshotting an instance essentially makes a block-level copy of the volume it’s running on or attached to. Snapshot creation is just about instantaneous, but they need not be kept as volumes. The snapshot may be sent off to more-efficient object storage instead. If you want to turn a snapshot back into a volume you send a request, storage is assigned, and the image streams back into volume storage from object storage; you can then attach it to instances. You’ll notice some nice interplays between object and volume storage to keep things as efficient as possible. It’s one of the cool things about cloud computing. Hopefully this gives you a better idea of how the back end works. In a future post I will talk about volume encryption and the relationship between volume and object storage. Share:

Read Post

7 Myths, Expanded

I really enjoyed the 7 myths of Entrepreneurship on Tim Ferriss’ site. The examples are from software development, but apply to most small tech firms. Having been through 6 startups of my own, I pretty much agree with everything said. More to the point, these ‘myths’ are the more common pitfalls I witnessed over and over again. That said, I think there is more to be gained here, and some important points were left on the cutting room floor. Specifically: Code Ninjas: If you have been in development long enough, you have run into a code ninja. I have seen a single person architect, write, and maintain a full-featured OS ultimately installed on a quarter-million machines. My friend Tony tells an awe-inspiring story of a ninja rewriting the core of a UNIX Kernel in a week – after 115 other engineers had failed for a year. I don’t think Java could have happened without Gosling. I will say you don’t have to hire ninjas to succeed, and many excellent teams lack one. People get caught up in striving for greatness, and think a ninja is their key to greatness. Sure, it’s better to have one than not. But the real trick is to find a ninja who’s not a prima donna, as they have the capacity to belittle, pout, and de-motivate as easily as they can produce, teach and inspire. Software development is not a lone-wolf exercise, so if you’re not sure if a possible ninja can coexist with the rest of the team, play it safe. Running Hot: It’s not just that running hot burns developers out – it’s a sign of mismanagement. Management pushing too hard means unrealistic expectations, or a willingness to push developers to the breaking point (typically using pride as motivation), or both. “Instilling a sense of urgency” is usually a BS way of saying “work harder”. Don’t get me wrong – sometimes you need to push. I have seen engineering oriented-companies be very lackadaisical about delivering product. The Ask version of Ingres was a prime example. But running hot means burnout, lower quality, and turnover. My tactic was to get developers to invest the extra hours in reading about their profession on the train ride home. Technical books, magazines, web groups, conferences, and classes educate. More importantly, learning tends to inspire. It’s hard to be creative when you can’t sleep and are stressed out, and inspiration doesn’t come from slogging through 40 task cards without a break. Deadlines: The single biggest friction point, and one of the hardest management tasks, is managing to deadlines. It also shows the greatest disconnect between sales and development teams. Builders view deadlines as arbitrary, and in their cycle, the code is done when it is done. Sales needs something – anything – to sell, and in their cycle predicable delivery is everything. Yanking stuff at deadline pisses sales and prospects off regardless, and getting stuff back on the queue is a nightmare. Agile can help. Better and stronger product management helps. Vetting sales requests helps. Promising less helps. Ultimately there is no right answer, but the friction can be mitigated. Hiring: HR is the single greatest detractor to hiring the right people. There, I said it. HR tends to enact hiring standards that weed out the best candidates before they are even interviewed. Hiring managers get the same stale set of resumes because they are what made it through the HR weeding process. And HR only goes by a) a misinterpretation of what you told them to look for, and b) what their peers are doing. To avoid the resultant poop-colander effect – where only correctly shaped poop gets through – many companies adopted ‘quirky’ hiring practices. And these tricks work – you get a different set of poop candidates. Not better, just different. Two years later you contract with a head hunter – who simply does a better job of understanding your requirements, idiosyncrasies, and biases than HR – and they find you candidates you can accept. Because they are paid very well to understand what you want. Managers: You want better candidates, so do your own screening! Share:

Read Post

Friday Summary (OS/2 Edition): June 24, 2011

There’s something I need to admit. I’m not proud of it, but it’s time to get it off my chest and stop hiding, no matter how embarrassing it is. You see, it happened way back in 1994. I was working as a paramedic at the time, so a lot of my decisions were affected by sleep deprivation. Oh heck – I’ll just say it. One day I walked into a store, pulled out my checkbook, and bought a copy of OS/2 Warp. To top it off I then installed it on the only (dreadfully underpowered) laptop I could afford at the time. I can’t really explain my decision. I think it was that geek hubris most of us pass through at some point in our young lives. I fell for the allure of a technically superior technology, completely ignoring the importance of the application ecosystem around it. I tried to pretend that more efficient memory management and true multitasking could make up for little things like being limited to about 1.5 models of IBM printers. It wouldn’t be the last time I underestimated the power of ecosystem vs. technology. I’m also the guy who militantly avoided iPods in favor of generic MP3 players. I was thinking features, not design. Until I finally broke down and bought my first iPod, that is. The damn thing just worked, and it looked really nice in the process, even though it lacked external storage. After Dropbox’s colossal screwup I started looking at alternatives again. I didn’t need to look very hard, because people emailed and tweeted some options pretty quickly. A few look very interesting, and they are all dramatically more secure. The problem is that none of them look as polished or simple – never mind as stable. I’m not talking about giving up security for simplicity – Dropbox could easily keep their current simplicity and still encrypt on the client. I mean that Dropbox nailed the consumer cloud storage problem early and effectively, quickly building up an ecosystem around it. It’s this ecosystem that provides the corporate-level stability all the alternatives lack. These alternatives do have a chance to make it if they learn the lessons of Dropbox and Apple; and pay as much attention to design, simplicity, and ecosystem as they do to raw technology. But none of them seem quite that mature yet, so I will mostly watch and play rather than dump what I’m doing and switch over completely. Which is too bad. Because I’m starting to regret paying for Dropbox based on their latest error. If they address it directly, then it won’t be a long term problem at all. If they don’t I’ll have to eat my own dog food and move to an alternative provider that meets my minimum security requirements, even though they are at greater risk of failing. Which also forces me to always have contingency options so I don’t lose my data. Sigh. On to the Summary: Webcasts, Podcasts, Outside Writing, and Conferences Rich quoted on RSA at The Street. Rich at Newsweek on Mac Defender. Rich on iPad security at Macworld. (Yes, I’m a major media whore this week). Our Dropbox story bit BoingBoing. Adrian over at Network Computing on GreenSQL. Favorite Securosis Posts Adrian Lane: How to Encrypt Your Dropbox Files, at Least until Dropbox Wakes the F* up. Great product but they need to fix both server and client side security architectures. David Mortman: Tokenization vs. Encryption: Payment Data Security. Rich: My older Securing Cloud Data with Virtual Private Storage post. Other Securosis Posts 7 Myths, Expanded. IaaS Storage 101. Is Your Email Address Worth More Than Your Credit Card Number? New White Paper: Security Benchmarking: Going Beyond Metrics. Favorite Outside Posts Adrian Lane: Creating Public AMIs Securely for EC2. This is difficult to do correctly. David Mortman: Security Expert, Gunnar Peterson, on Leveraging Enterprise Credentials to connect with Cloud applications. Rich: Why Sony is no surprise. A true must-read. Simplicity doesn’t scale. Chris Pepper: Fired IT manager hacks into CEO’s presentation, replaces it with porn. I’m more amused than the fired manager or the CEO. Research Reports and Presentations Security Benchmarking: Going Beyond Metrics. Understanding and Selecting a File Activity Monitoring Solution. Database Activity Monitoring: Software vs. Appliance. React Faster and Better: New Approaches for Advanced Incident Response. Measuring and Optimizing Database Security Operations (DBQuant). Network Security in the Age of Any Computing. The Securosis 2010 Data Security Survey. Monitoring up the Stack: Adding Value to SIEM. Top News and Posts Dropbox Left User Accounts Unlocked for 4 Hours Sunday. Feeling like a sooper-genius for encrypting my stuff Saturday. Antichat Forum Hacker Breach. Shocker – they used weak passwords. Teen Alleged Member of LulzSec. Interesting Graphic on data breaches. Toward Trusted Infrastructure for the Cloud Era. Pentagon Gets Cyberwar Guidelines. New views into the 2011 DBIR. Mozilla retires Firefox 4 from security support. Northrop Grumman constantly under attack by cyber-gangs. Analysis: LulzSec trackers say authorities are closing. hacked. Amazon’s cloud is full of holes. Blog Comment of the Week Remember, for every comment selected, Securosis makes a $25 donation to Hackers for Charity. This week’s best comment goes to Mark, in response to Is Your Email Address Worth More Than Your Credit Card Number?. Spot on Rich. NIST already defines Email address as PII under 800-122. It seems everyone’s turning a bind eye to the contextual aspect today – conveniently. “One of the most widely used terms to describe personal information is PII. Examples of PII range from an individual’s name or email address to an individual’s financial and medical records or criminal history.” In my opinion, what’s often worse is that an email address is also now a primary index to social networking sites (facebook, LinkedIn etc) which immediately presents more gold to mine for a spearphishing attack to present a APT payload – even if the attacker doesn’t have complete access, its all too easy these days to build a personal profile from one data

Read Post

Totally Transparent Research is the embodiment of how we work at Securosis. It’s our core operating philosophy, our research policy, and a specific process. We initially developed it to help maintain objectivity while producing licensed research, but its benefits extend to all aspects of our business.

Going beyond Open Source Research, and a far cry from the traditional syndicated research model, we think it’s the best way to produce independent, objective, quality research.

Here’s how it works:

  • Content is developed ‘live’ on the blog. Primary research is generally released in pieces, as a series of posts, so we can digest and integrate feedback, making the end results much stronger than traditional “ivory tower” research.
  • Comments are enabled for posts. All comments are kept except for spam, personal insults of a clearly inflammatory nature, and completely off-topic content that distracts from the discussion. We welcome comments critical of the work, even if somewhat insulting to the authors. Really.
  • Anyone can comment, and no registration is required. Vendors or consultants with a relevant product or offering must properly identify themselves. While their comments won’t be deleted, the writer/moderator will “call out”, identify, and possibly ridicule vendors who fail to do so.
  • Vendors considering licensing the content are welcome to provide feedback, but it must be posted in the comments - just like everyone else. There is no back channel influence on the research findings or posts.
    Analysts must reply to comments and defend the research position, or agree to modify the content.
  • At the end of the post series, the analyst compiles the posts into a paper, presentation, or other delivery vehicle. Public comments/input factors into the research, where appropriate.
  • If the research is distributed as a paper, significant commenters/contributors are acknowledged in the opening of the report. If they did not post their real names, handles used for comments are listed. Commenters do not retain any rights to the report, but their contributions will be recognized.
  • All primary research will be released under a Creative Commons license. The current license is Non-Commercial, Attribution. The analyst, at their discretion, may add a Derivative Works or Share Alike condition.
  • Securosis primary research does not discuss specific vendors or specific products/offerings, unless used to provide context, contrast or to make a point (which is very very rare).
    Although quotes from published primary research (and published primary research only) may be used in press releases, said quotes may never mention a specific vendor, even if the vendor is mentioned in the source report. Securosis must approve any quote to appear in any vendor marketing collateral.
  • Final primary research will be posted on the blog with open comments.
  • Research will be updated periodically to reflect market realities, based on the discretion of the primary analyst. Updated research will be dated and given a version number.
    For research that cannot be developed using this model, such as complex principles or models that are unsuited for a series of blog posts, the content will be chunked up and posted at or before release of the paper to solicit public feedback, and provide an open venue for comments and criticisms.
  • In rare cases Securosis may write papers outside of the primary research agenda, but only if the end result can be non-biased and valuable to the user community to supplement industry-wide efforts or advances. A “Radically Transparent Research” process will be followed in developing these papers, where absolutely all materials are public at all stages of development, including communications (email, call notes).
    Only the free primary research released on our site can be licensed. We will not accept licensing fees on research we charge users to access.
  • All licensed research will be clearly labeled with the licensees. No licensed research will be released without indicating the sources of licensing fees. Again, there will be no back channel influence. We’re open and transparent about our revenue sources.

In essence, we develop all of our research out in the open, and not only seek public comments, but keep those comments indefinitely as a record of the research creation process. If you believe we are biased or not doing our homework, you can call us out on it and it will be there in the record. Our philosophy involves cracking open the research process, and using our readers to eliminate bias and enhance the quality of the work.

On the back end, here’s how we handle this approach with licensees:

  • Licensees may propose paper topics. The topic may be accepted if it is consistent with the Securosis research agenda and goals, but only if it can be covered without bias and will be valuable to the end user community.
  • Analysts produce research according to their own research agendas, and may offer licensing under the same objectivity requirements.
  • The potential licensee will be provided an outline of our research positions and the potential research product so they can determine if it is likely to meet their objectives.
  • Once the licensee agrees, development of the primary research content begins, following the Totally Transparent Research process as outlined above. At this point, there is no money exchanged.
  • Upon completion of the paper, the licensee will receive a release candidate to determine whether the final result still meets their needs.
  • If the content does not meet their needs, the licensee is not required to pay, and the research will be released without licensing or with alternate licensees.
  • Licensees may host and reuse the content for the length of the license (typically one year). This includes placing the content behind a registration process, posting on white paper networks, or translation into other languages. The research will always be hosted at Securosis for free without registration.

Here is the language we currently place in our research project agreements:

Content will be created independently of LICENSEE with no obligations for payment. Once content is complete, LICENSEE will have a 3 day review period to determine if the content meets corporate objectives. If the content is unsuitable, LICENSEE will not be obligated for any payment and Securosis is free to distribute the whitepaper without branding or with alternate licensees, and will not complete any associated webcasts for the declining LICENSEE. Content licensing, webcasts and payment are contingent on the content being acceptable to LICENSEE. This maintains objectivity while limiting the risk to LICENSEE. Securosis maintains all rights to the content and to include Securosis branding in addition to any licensee branding.

Even this process itself is open to criticism. If you have questions or comments, you can email us or comment on the blog.